A local control room for developing, simulating, and validating Microduck.
AI coding ready: Codex · Claude Code · Cursor Agent · GitHub Copilot CLI
AI startup · Quick start · Monitor · Architecture · Troubleshooting
Microduck Studio connects the existing
microduck runtime and
microduck_rl simulation/training project in
one browser-based development workflow. It presents status and safe control; it does not duplicate
robot safety, policy inference, simulator physics, or training logic.
Any local AI coding agent with repository and terminal access can start the complete stack; no tool-specific integration is required. Supported examples include Codex, Claude Code, Cursor Agent, and GitHub Copilot CLI.
Open the microduck-dev workspace in a local session and use this prompt:
Start all Microduck Studio services. First read AGENTS.md and microduck-studio/README.md. Check the
three sibling repositories, Docker, uv, and ports 7801 and 8090. Then run
./scripts/dev-stack.sh from microduck-studio. Wait for "contract probe passed" before reporting the
Studio URL and the status of Studio, robotd, tofd, and MuJoCo. Do not start a training job, and do not
switch or modify the working branches of the sibling repositories.
Use a local agent session with access to Docker and the desktop. A cloud or background agent can run in a different machine and cannot display the native MuJoCo Viewer on the development host.
- Docker with Docker Compose
gitandcurl- Python 3 and
uv
The individual projects can run directly on the host and do not inherently require Docker. The
dev-stack.shlauncher does require Docker because it isolates therobotdand Studio Web build and runtime environments. On macOS, Docker Desktop is one option; any compatible Docker runtime is suitable.
Studio is not standalone: microduck supplies robotd and its policies, while microduck_rl
supplies the MuJoCo body and Viewer. All three repositories must be sibling directories under one
workspace:
microduck-dev/ # workspace only; not a Git repository
├── microduck/ # robotd, robotctl, policies, and runtime protocol
├── microduck_rl/ # MuJoCo body, Viewer, environments, and training
└── microduck-studio/ # Web UI and development-stack orchestration
For a fresh workspace, clone the repositories used by this development setup:
mkdir -p ~/microduck-dev
cd ~/microduck-dev
git clone https://github.com/microai-lab/microduck.git
git clone https://github.com/microai-lab/microduck_rl.git
git clone https://github.com/microai-lab/microduck-studio.gitDownload links: microduck, microduck_rl, and microduck-studio. The first two are development forks of the official runtime and official RL repository.
Existing repositories do not need to be cloned again; their directory layout only needs to match the structure above.
The regular microduck main branch provides the remote RobotIo implementation that connects
robotd --sim to the MuJoCo body service at TCP 127.0.0.1:7801 by default. Keep the sibling
repository on an up-to-date main; the launcher archives that commit into isolated Studio state
without switching or modifying its working tree. Set MICRODUCK_SIM_REF to a release tag when a
reproducible runtime version is required.
Policies are no longer stored in the runtime source tree. On first launch, Studio runs the selected
runtime's scripts/seed-policies.sh and stores the official set under
.studio-runtime/dev-stack/policies; subsequent launches reuse it. While upgrading an existing
Studio checkout, a cached policy set from the old embedded layout is imported if the Hub is
temporarily unavailable; this does not change which runtime source is built.
The native macOS Viewer runs from the microduck_rl virtual environment:
cd ~/microduck-dev/microduck_rl
uv synccd ~/microduck-dev/microduck-studio
./scripts/dev-stack.shOpen http://127.0.0.1:8090. The default native mode starts MuJoCo in the macOS background for
smooth native OpenGL offscreen rendering without opening the desktop Viewer. A simulation-enabled
robotd, tofd, and Studio run in Docker. The launcher then verifies the selected runtime revision,
policy mount, robot/ToF/head-IMU subscriptions, movement, explicit stop, and disconnect stop.
Do not treat a reachable page alone as ready; wait for:
contract probe passed: runtime source, policies, subscription, movement ... m, stop, disconnect
The launcher never switches a sibling repository's working branch. It extracts the selected runtime revision into isolated Studio state and reports that exact ref and commit in the UI.
| Area | Included |
|---|---|
| Web control | Phone-friendly movement, enable/stop, sit/stand, roulade, and kicks |
| Live visibility | Selectable MuJoCo scenes, world/head-camera views, synchronized IMU/ToF/frame telemetry, repository/job status, and model discovery |
| Safe orchestration | Docker Compose lifecycle, a cross-repository contract probe, and allowlisted RL smoke tests |
Motion uses one persistent robotd connection. Releasing a control, hiding the page, disconnecting,
or shutting Studio down sends robot.stop; robotd remains the final safety and motor authority.
Run these commands from microduck-studio:
| Goal | Command |
|---|---|
| Start or cleanly restart everything and verify control | ./scripts/dev-stack.sh |
| Run native MuJoCo in the background (default) | ./scripts/dev-stack.sh |
| Start with a specific RL scene | ./scripts/dev-stack.sh --scene scene_vslam.xml |
| Explicitly open the desktop MuJoCo Viewer | ./scripts/dev-stack.sh --viewer |
| Start without the simulated movement probe | ./scripts/dev-stack.sh --skip-control-probe |
| Stop background MuJoCo 10 s after the last page closes | ./scripts/dev-stack.sh --stop-on-browser-close |
| Start the fully containerized CPU renderer | ./scripts/dev-stack.sh --sim-mode docker --gpu none |
| Use Linux DRI/EGL GPU passthrough | ./scripts/dev-stack.sh --sim-mode docker --gpu dri |
| Use Linux NVIDIA/EGL GPU passthrough | ./scripts/dev-stack.sh --sim-mode docker --gpu nvidia |
Check Studio, robotd, tofd, and MuJoCo together |
./scripts/dev-stack.sh status |
| Open the live terminal monitor | ./scripts/dev-stack.sh monitor |
| Stop only this development stack | ./scripts/dev-stack.sh stop |
The robotd and MuJoCo status cards show Start or Restart according to connection state. These
buttons use a restricted host manager created by the launcher; it accepts only fixed service
operations. Restarting MuJoCo waits for its port and then restarts robotd as well. If the Web
service itself is down, use ./scripts/dev-stack.sh because the buttons are not reachable.
The scene selector uses only scene*.xml files discovered in microduck_rl; applying a scene
updates the validated launcher configuration and safely restarts MuJoCo followed by robotd.
Only --viewer opens the desktop window. Closing that window stops the simulator; use the Start
button on its status card to restore it. The default background mode uses the same native
authoritative world and offscreen renderer.
Add --stop-on-browser-close to stop only MuJoCo after the final scene WebSocket remains
disconnected for 10 seconds. A page refresh or reconnect inside that grace period cancels the
shutdown. Without this option, background MuJoCo continues until dev-stack.sh stop. --headless
remains available as an explicit compatibility spelling of the default behavior.
Use the default command for a local macOS development machine. It is the only mode intended for a smooth local demo without a Linux GPU host:
| Situation | Command | Renderer and expected result |
|---|---|---|
| macOS local development (recommended) | ./scripts/dev-stack.sh |
Native macOS offscreen OpenGL; the authoritative MuJoCo world runs in the background and the desktop Viewer stays closed. |
| Inspect the native Viewer | ./scripts/dev-stack.sh --viewer |
Opens the desktop Viewer against the same simulation service; closing it stops MuJoCo. |
| Docker Desktop on macOS or CPU-only CI | ./scripts/dev-stack.sh --sim-mode docker --gpu none |
Dockerized OSMesa software renderer. Physics and scene state remain authoritative, but the video stream can be slow. |
| Linux host with an integrated/AMD/Intel GPU | ./scripts/dev-stack.sh --sim-mode docker --gpu dri |
Maps /dev/dri into the MuJoCo container and uses EGL. |
| Linux host with NVIDIA Container Toolkit | ./scripts/dev-stack.sh --sim-mode docker --gpu nvidia |
Requests gpus: all and uses EGL. |
dri is Linux Direct Rendering Infrastructure: access to the host GPU device files rather than a
separate GPU API. It is not available through Docker Desktop on macOS. GPU choices are explicit
command-line arguments, so a shell history shows exactly which hardware path was used. --gpu is
valid only with --sim-mode docker; --viewer and --stop-on-browser-close are native-mode
options, and the latter requires background/headless operation.
The page has three live surfaces with deliberately different sources of truth:
| Surface | Source | What you can do |
|---|---|---|
| MuJoCo scene | Cached JPEG/PNG world frames plus the simulated 640×360 UYVY head-camera stream from duck-body |
Switch between World and Head camera, select an allowlisted RL scene, or drag/zoom/reset the authoritative world camera. |
| ROBOTD TELEMETRY | The robotd monitor protocol plus read-only tofd subscriptions |
Inspect policy, commands, trunk/head IMUs, dynamic sensor poses, 8×8 ToF depth/status, odometry, joint targets/errors, robot thumbnail, and loop rate. Shared monotonic timestamps expose sensor age without introducing another control loop. |
| Control and service cards | robotd JSON-RPC plus the launcher-installed, fixed-operation service manager |
Send motion intents, enable/stop skills, and Start/Restart robotd or MuJoCo when the launcher is running. |
The selector shows only the translated scene name. Click the Scene control immediately before
the selector to copy the selected scene's .xml filename; Apply then restarts MuJoCo with the
selected allowlisted scene.
On desktop, the live FPS indicator, World/Head camera switch, Scene control, selector, and Apply button stay in one compact row, with Apply directly beside the selector. A long label may be clipped in the closed native selector, but its complete name remains available in the opened menu. Chinese mode displays only Chinese control labels; English mode displays only English labels.
| File | Chinese name | Purpose |
|---|---|---|
scene.xml |
标准场景 | Default flat floor with the curated ground-contact collision model. |
scene_allcollisions.xml |
全身碰撞 | Enables collision geometry for every robot part to inspect self-collision and complex contacts. |
scene_apartment.xml |
公寓场景 | Six-room furnished apartment for indoor navigation and localization experiments. |
scene_backlash.xml |
关节回差 | Standard ground-contact model with simulated servo gear play for sim-to-real checks. |
scene_ball.xml |
足球场景 | Adds a ball for kick-policy and ball-interaction testing. |
scene_rollers.xml |
滚轮场景 | Uses passive foot rollers for glide, roller-balance, and roller-policy tests. |
scene_vslam.xml |
视觉定位 | Large feature-rich room with asymmetric landmarks for VSLAM drift and loop-closure tests. |
scene_walk.xml |
行走模型 | Walking-specific robot model on a flat floor for locomotion-policy inspection. |
scene_walk_backlash.xml |
行走回差 | Walking model with simulated servo backlash for stricter sim-to-real validation. |
Use the language switch in the page header to choose Chinese or English. It changes UI labels only; the underlying service, policy, and unit values do not change. Frame profiles are also selected in the scene card:
| Profile | Maximum frame size | Encoding | When to use it |
|---|---|---|---|
| Smooth | 960×540 | JPEG, quality 82 | Lowest bandwidth and latency. |
| Clear (default) | 1920×1080 | JPEG, quality 95 | Normal desktop use. |
| Lossless | 1920×1080 | PNG | Still inspection; it may reduce frame rate considerably. |
The browser asks for the scene's CSS size multiplied by the display pixel ratio, then the selected profile applies its cap. This avoids a blurry low-resolution stream on high-density displays. Scene content and simulation time are identical to the MuJoCo authority; pixel-for-pixel equality between different operating systems, GL drivers, or GPU vendors is not promised.
The recommended command opens the live visual monitor directly in the current terminal:
./scripts/dev-stack.sh monitorThe direct Compose equivalent is:
docker compose run --rm --no-deps --build robotctl monitorBoth launch a disposable robotctl tool container attached to the existing runtime socket. They do
not enter a Docker shell or the running robotd container. The wrapper is preferred because it
first verifies that robotd is running.
| Key | Action |
|---|---|
q, Esc, or Ctrl-C |
Exit |
[ / ] or Left / Right |
Rotate the 3D robot view |
d |
Show or hide the 3D robot view |
The 3D view needs a terminal at least 110 columns wide. Status and joint tables still work in a narrower terminal.
The workspace contains three independent Git repositories with one-way ownership boundaries:
Browser
│ HTTP / JSON API
▼
Microduck Studio
│ JSON-RPC / NDJSON over a Unix socket
▼
robotd --sim ◀──────── policy.onnx + manifest ─────── microduck_rl export
│
│ TCP / NDJSON :7801
▼
duck-body / MuJoCo ────────────────────────────────── microduck_rl
- microduck owns
robotd, hardware I/O, safety, policy loading, and motor authority. - microduck_rl owns MuJoCo models, environments, rewards, training, and ONNX export.
- microduck-studio owns the browser experience, status aggregation, and allowlisted local orchestration.
Neither runtime nor training depends on Studio. Studio consumes their public protocols and tools.
For the browser scene, duck-body snapshots its authoritative MjData under the world lock, then
renders and JPEG-encodes outside that lock. Studio only long-polls the cached frame and proxies it
to the browser. Dynamic objects, contacts, and every other scene element therefore come from the
same world as the physics rather than from a reconstructed pose mirror.
Drag the browser scene to orbit the authoritative MuJoCo camera, use the mouse wheel or trackpad
to zoom, and double-click to restore the default view.
robotd --sim is a startup mode, not a separate runtime environment. It runs the complete
robotd while replacing physical motor and sensor I/O with a remote simulation adapter:
- Joint and sensor state comes from the
microduck_rlduck-bodyservice. - Control targets are sent back to MuJoCo over TCP.
- The control loop, policy inference, safety checks, and JSON-RPC interface remain unchanged.
- Studio and
robotctltherefore use the same control interface for simulation and hardware.
The simulation backend does not start the Viewer, run training, or bypass robotd safety logic.
Docker provides process and dependency isolation for the one-command workflow; it is separate from
the --sim mode itself.
All project-owned container definitions live here and are separated by component:
docker/
├── microduck/ # robotd + robotctl runtime image
├── microduck-rl/ # headless authoritative MuJoCo renderer image
└── studio/ # Studio web and frame proxy image
compose.yaml # base services and CPU renderer
compose.gpu-dri.yaml # Linux /dev/dri + EGL override
compose.gpu-nvidia.yaml # Linux NVIDIA + EGL override
The no-argument default uses the native macOS authoritative renderer in the background without a
Viewer window; pass --viewer to open it explicitly. Docker/GPU selection is intentionally
command-line-only so the active hardware
mode remains visible in shell history. The --sim-mode docker mode also moves duck-body into
Compose. Docker Desktop on macOS has no Linux
GPU passthrough, so its OSMesa renderer is scene-faithful but slow; use the native mode for a smooth
demo. On a Linux host, explicitly passing --gpu dri maps /dev/dri, while --gpu nvidia
requests gpus: all; both use
the same EGL render protocol and browser UI.
The browser defaults to the Clear profile and requests its actual CSS size multiplied by the
display pixel ratio. Smooth caps the stream at 960x540 with JPEG quality 82, Clear caps
it at 1920x1080 with JPEG quality 95, and Lossless caps it at 1920x1080 with PNG. The
startup fallback is 1280x720, 24 FPS, JPEG quality 95. Override that fallback with
MICRODUCK_RENDER_WIDTH, MICRODUCK_RENDER_HEIGHT, MICRODUCK_RENDER_FPS, and
MICRODUCK_RENDER_QUALITY. MICRODUCK_MUJOCO_GL can explicitly select a MuJoCo GL backend.
docker compose ps
docker compose logs -f studio robotd tofd
docker compose run --rm --no-deps robotctl healthThe final command runs a temporary tool container; it does not open a Docker shell.
| Symptom | Check and recovery |
|---|---|
| The scene stays on “Waiting for MuJoCo frames” | Run ./scripts/dev-stack.sh status, then inspect .studio-runtime/dev-stack/mujoco.log. Use the MuJoCo card's Start/Restart action when Studio is online, or run ./scripts/dev-stack.sh again if the page is unavailable. |
| Controls do not move the simulated robot | Confirm all three status cards are connected, click Enable RL, then run the default launcher without --skip-control-probe. The launcher reports contract probe passed only after the cross-repository contracts pass. |
| The Docker scene is choppy | On macOS, use the default native mode. OSMesa inside Docker is CPU rendering. On a supported Linux host, use the explicit dri or nvidia GPU mode. Reduce the profile to Smooth before reducing the authoritative physics rate. |
| A desktop MuJoCo window appeared unexpectedly | The Viewer opens only when --viewer was supplied. Stop the stack and restart with ./scripts/dev-stack.sh; that is the default background mode. |
| Start/Restart buttons are unavailable | They are intentionally available only after dev-stack.sh has installed its local, allowlisted service manager. They cannot start the Studio Web service itself; restart the stack from a terminal for that case. |
Run only MuJoCo Viewer
Stop the full stack first if it owns port 7801, then start the simulator body from the sibling RL repository:
./scripts/dev-stack.sh stop
cd ../microduck_rl
uv run mjpython -m mjlab_microduck.sim.body_server --keyframe HOME --port 7801 --renderClose the Viewer or press Ctrl-C to stop it. This mode exposes the simulator TCP endpoint but
does not start robotd or Studio.
Run only Studio
uv sync --extra dev
cp .env.example .env
uv run microduck-studioThis starts only the Web service. The default robot socket is /run/robotd.sock; the Compose
stack instead shares that socket through its named runtime volume. Port 8090 avoids microduck
services that commonly use port 8080.
- Run
./scripts/dev-stack.sh status; Studio,robotd, and MuJoCo must all be online/healthy. - Confirm the
robotdand simulator cards are connected, then click Enable RL. - Check
docker compose logs -f studio robotdand.studio-runtime/dev-stack/mujoco.logfor disconnects or policy refusals. - Run
./scripts/dev-stack.shagain. It stops processes owned by the prior launch and repeats the cross-repository contract probe before reporting ready.
Training jobs are disabled by default. When running Studio directly on the host, enable them with:
MICRODUCK_STUDIO_ENABLE_JOBS=true uv run microduck-studioStudio exposes only the documented 64-environment, 5-iteration smoke test. It constructs an
allowlisted uv run train ... argument list with shell=False; arbitrary shell commands and
long training runs are intentionally unavailable.
uv sync --extra dev
uv run pytest
uv run ruff check .
uv run ruff format --check .Studio v0.1 has no authentication. Bind it to 127.0.0.1; non-loopback binding should be limited
to trusted LANs. Studio sends intents only; robotd remains the sole safety and motor authority.