EmbodiedAct

Core runtime and MATLAB communication layer for embodied scientific agents.

Overview

EmbodiedAct exposes the runtime structure behind an embodied scientific agent for scientific computing:

• A four-module controller loop for planning, code generation, runtime perception, and reflection.
• A platform abstraction layer for MATLAB, Octave, Python, and notebook-style execution targets.
• A MATLAB WebSocket server for code execution, model editing, and simulation control.

This public release is scoped to reusable runtime components. Benchmark datasets, evaluation pipelines, and paper-specific analysis scripts are intentionally excluded.

Architecture

EmbodiedAct follows a Brain-Nerve-Body split between the agent controller, the platform adapters, and the execution environment.

Module	Responsibility
M_plan	Decompose scientific intent into executable sub-tasks
M_code	Translate plans into platform-specific code
M_perc	Observe execution feedback from stdout, stderr, figures, and model state
M_ref	Decide whether to accept, repair locally, or re-plan globally

Repository Layout

EmbodiedAct/
├── src/embodiedact/
│   ├── agent/                 # Agent loop and answer extraction
│   ├── platforms/             # MATLAB / Octave / Python / Jupyter adapters
│   ├── runtime/               # Session and event data models
│   ├── core/                  # Shared chat message primitives
│   ├── llm/                   # Minimal async LLM client
│   ├── config/                # TOML loading helpers
│   └── run.py                 # CLI for adapter discovery and smoke checks
├── matlab/
│   ├── enhanced_websocket_matlab_server_v2_complete.m
│   ├── start_websocket_matlab_server.m
│   ├── stop_websocket_matlab_server.m
│   └── diagnose_websocket_port.m
├── configs/
│   └── agent.example.toml
├── assets/readme/
│   ├── logo.png
│   ├── paradigm_comparison.png
│   └── architecture.png
└── tests/

Source Map

Path	Purpose
src/embodiedact/agent/runner.py	Four-module controller and dual-loop execution logic
src/embodiedact/platforms/base.py	Stable adapter interface and capability schema
src/embodiedact/platforms/registry.py	Adapter registration and lookup
src/embodiedact/platforms/matlab_code.py	Local MATLAB code execution backend
src/embodiedact/runtime/events.py	Session, artifact, and streamed event types
src/embodiedact/core/messages.py	Minimal message abstraction for LLM calls
matlab/enhanced_websocket_matlab_server_v2_complete.m	MATLAB WebSocket server implementation

Quick Start

python -m venv .venv
source .venv/bin/activate
pip install -e '.[dev]'

python -m embodiedact.run platform list
python -m embodiedact.run platform smoke --platform matlab_code

To enable local MATLAB execution through the Python adapter:

export EMBODIEDACT_ENABLE_LOCAL_MATLAB=1
export EMBODIEDACT_MATLAB_CMD=/path/to/matlab
python -m embodiedact.run platform smoke --platform matlab_code

MATLAB WebSocket Server

The MATLAB communication layer is centered on matlab/enhanced_websocket_matlab_server_v2_complete.m, with small wrapper commands for start, stop, and port diagnosis.

Files

File	Purpose
matlab/enhanced_websocket_matlab_server_v2_complete.m	Main WebSocket server implementation
matlab/start_websocket_matlab_server.m	Convenience entry point for startup
matlab/stop_websocket_matlab_server.m	Stop the timer, close the socket, and clear globals
matlab/diagnose_websocket_port.m	Inspect timer state and test port availability

Quick Use

addpath(genpath('/absolute/path/to/EmbodiedAct/matlab'));
start_websocket_matlab_server

% later
stop_websocket_matlab_server
status = diagnose_websocket_port;

diagnose_websocket_port returns a struct with state_present, timer_present, timer_running, port_available, and port_error.

Runtime Summary

Item	Behavior
Listen port	9001
Connection model	Accepts WebSocket clients and creates per-client sessions
Request scope	Model editing, simulation control, and code execution
Output stream	Acknowledgement, progress, and completion messages
Workspace safety	Restores critical server state if user code clears the workspace

Client Workflow

1. Connect a WebSocket client to ws://127.0.0.1:9001.
2. Send session_init to register the client and declare capabilities.
3. Send operation_request messages for model edits, simulation control, or code execution.
4. Consume acknowledgement, progress, and completion messages until the operation finishes.

Protocol Summary

Message types

Type	Purpose
session_init	Register a client session
operation_request	Execute a model or code operation
state_verification	Query current model state
heartbeat	Keep the session alive
command	Execute server-side command handling
simulink_action	Trigger Simulink-specific actions
ping	Basic liveness check

Operation types

Operation	Purpose
new_system	Create a model
add_block	Add a Simulink block
add_component	Add a component with parameters
add_line	Connect blocks
set_param	Update block or model parameters
get_model_info	Read current model information
run_simulation	Execute a simulation
execute_code	Run MATLAB code or a MATLAB script

execute_code modes

Mode	Required fields	Behavior
selection	code	Execute inline MATLAB code
script	file_path	Run a script file and optionally write file_content first

Minimal client message examples

Session initialization:

{
  "id": "msg-1",
  "type": "session_init",
  "payload": {
    "client_type": "python",
    "version": "1.0",
    "capabilities": ["operation_tracking", "progress_monitoring"]
  }
}

Inline code execution:

{
  "id": "msg-2",
  "type": "operation_request",
  "operation_id": "op-1",
  "payload": {
    "operation_type": "execute_code",
    "parameters": {
      "mode": "selection",
      "code": "result = 2 + 2; disp(['Result: ', num2str(result)]);",
      "capture_output": true
    }
  }
}

File-based execution:

{
  "id": "msg-3",
  "type": "operation_request",
  "operation_id": "op-2",
  "payload": {
    "operation_type": "execute_code",
    "parameters": {
      "mode": "script",
      "file_path": "/tmp/task.m",
      "file_content": "result = 21 * 2; disp(['Result: ', num2str(result)]);",
      "capture_output": true
    }
  }
}

For each operation, the server emits an acknowledgement, a start event, one or more progress updates, and a completion message. execute_code can also stream structured execution output back to the client.