Skip to content

feat(jmex): add AgileMaster full-body plugin and G1 teleop example - #1250

Open
jmex-sw wants to merge 2 commits into
NVIDIA:mainfrom
jmex-sw:jmex-plugin-full-body
Open

jmex-sw wants to merge 2 commits into
NVIDIA:mainfrom
jmex-sw:jmex-plugin-full-body

Conversation

@jmex-sw

@jmex-sw jmex-sw commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Description

Adds support for the j-mex AgileMaster motion capture suit: a device plugin that publishes the
operator's body pose as FullBodyPose under a new body.jmex vendor, the core registration that
lets Isaac Capture read it, and an Isaac Lab example that drives a Unitree G1 from it. None of this
is upstream yet; the plugin, the example and their tests are all new.

AgileMaster has three parts: the Mocap Suit the operator wears; MOXI Player, which solves the suit
onto a skeleton and streams it on a numbered channel; and the MOXI Receiver SDK, which the plugin
links against.

The plugin (src/plugins/jmex/) receives one Player channel and fills the 24 BodyJoint slots
from the standard human skeleton loaded in Player, one slot per joint, in OpenXR axes. It pushes one
FullBodyPose per frame, on the jmex_full_body collection by default. A slot with no joint, or
with no usable value in a frame, goes out with is_valid false; nothing is inferred from
neighboring joints. The raw device clock carries the time Player stamped on the frame
(--device-clock=local uses the pickup time instead). If the CloudXR runtime restarts underneath
it, the plugin re-establishes its session and carries on. An argument it cannot use, such as a
misspelt flag or a channel outside 1–255, stops it with exit code 1 and names the argument. It runs
on Linux and needs MOXI Receiver SDK 0.8.0 or newer. Without the SDK, configure prints
Skipping jmex plugin build: MOXIReceiverSDK 0.8.0 or newer not found and builds only the plugin's
SDK-free core library, which the tests link; the rest of the tree builds as before.

Where the conversion happens. Player streams its own skeleton. The plugin maps it onto the 24
slots and into OpenXR axes (body_joint_mapping, openxr_convention); Player does not emit
FullBodyPose. There are two reasons for this. The change of axes was measured in this tree and is
pinned by the plugin's tests, next to the code that consumes the slots. Done in Player, those tests
would live where this repository's CI never runs them. And Player does not have to produce Isaac
Capture's internal schema, so a change to FullBodyPose stays in this tree instead of waiting on a
Player release.

Registration in core. Outside the new directories, every change is an addition; no existing line
changes.

Where What
src/core/live_trackers/cpp/live_full_body_tracker_jmex_impl.{hpp,cpp} New: the live tracker for body.jmex. It reads the plugin's collection and takes one vendor parameter, collection_id
src/core/live_trackers/cpp/CMakeLists.txt Those two files in the library's source list
live_deviceio_factory.{hpp,cpp} A factory method and its try_create_* function; one row in k_tracker_dispatch, after the existing full-body vendors, so FullBodyTracker's default vendor is unchanged; one branch in validate_vendor_selections
tests/cpp/core/live_trackers/test_vendor_validation.cpp Five SECTIONs for body.jmex, beside the existing vendors'
src/plugins/CMakeLists.txt add_subdirectory(jmex) in the ungated list; the plugin skips itself off Linux, and without the SDK builds only its core library
tests/cpp/plugins/CMakeLists.txt add_subdirectory(jmex) for the plugin's tests, when its core library is built: on Linux, with or without the SDK
tests/python/examples/CMakeLists.txt add_subdirectory(jmex) for the example's tests
cmake/ClangFormat.cmake An SDK unpacked into the plugin directory, under any versioned name, is excluded from the format check
docs/source/_data/devices.yaml A row for the Ecosystem page

The example (examples/jmex/, package isaaccapture_examples.jmex) has two parts:

  • jmex_g1_tasks registers Isaac-PickPlace-Locomanipulation-G1-Jmex-Abs-v0, which inherits Isaac
    Lab's LocomanipulationG1EnvCfg (Isaac Lab 3.0.0 EA or 3.0.0 beta2) and takes its action from the
    operator; nothing in Isaac Lab is modified. It is upper-body teleop. Each arm is rebuilt segment
    by segment onto the G1's own shoulder, so the operator's arm angles carry over at the robot's arm lengths, and the elbows and
    shoulders get IK targets from that same chain. The wrist's orientation is the operator's hand,
    carried through a forearm frame. The operator's torso turns the robot's shoulders about its waist,
    so leaning, bending sideways and twisting come through; their heading does not, so the operator
    can turn to face a screen without moving the robot. Only a frame that carries all 24 slots drives
    the robot.
  • record_jmex_full_body records the same stream to a standard full-body MCAP
    (core.FullBodyPoseRecord on the full_body channel), which the shared replay_full_body script
    plays back unmodified.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

Testing

The tests in this PR need no SDK, so CI builds and runs them. ctest -L jmex selects 24: the
plugin's 23 Catch2 cases (argument parsing, slot mapping, change of axes, device clock choice, one
frame becoming one FullBodyPose, session recovery after a failed push), which link the plugin's
SDK-free core library; and the example's Python test, whose 91 pytest cases cover its action source, including
the geometry of the rebuilt arm and the orientation carried through the forearm frame. The Python
test is registered when BUILD_EXAMPLES and BUILD_VIZ are on, as in the ubuntu and Windows jobs.
The body.jmex sections of the vendor validation test run under ctest -R 'vendor validation'.

Local gate, Ubuntu x86_64. We ran the same Python and build-type matrix as the x64 entries of
build-ubuntu (Python 3.11, 3.12 and 3.13, each in Debug and Release) without the SDK, each from an
empty tree, twice: once with CI's toolchain (GCC 11.4, CMake 3.31.6, uv 0.12.13, clang-format 14.0.0)
and once with GCC 13.3. Local builds left BUILD_OVXR_CAMERA_QUADS and BUILD_PLUGIN_OAK_CAMERA
disabled; CI enables both. In all twelve:

  • configure skipped the plugin with the line above;
  • the build compiled 820 objects, and its clang-format check passed over 424 files;
  • the 24 jmex tests and the 3 vendor validation cases passed;
  • the 10 of this change's 12 .cpp files that build without the SDK compiled without warnings (the
    other two are covered below);
  • of the 588 tests in the full ctest, 576 passed, 9 skipped and 3 failed. The three are upstream
    GPU tests (VkContext with an explicit device index, and two camera_viz tests) that fail on this
    workstation's GPU setup and are unrelated to this change; CI's runners have no GPU and skip them.

The two plugin sources that compile only with the SDK were built against SDK 0.8.0 with GCC 13 and
syntax-checked with GCC 11, without warnings. With CI's toolchain we also repeated the Release
install with auditwheel repair, the pip-editable-install job (its import check, and its
wheel-content tests: 10 passed) and the configure of test-viz-sanitizers.
SKIP=check-copyright-year pre-commit run --all-files, make current-docs (-W),
scripts/check_doc_refs.py --all, ruff, and clang-format 14 on the changed files all pass. Windows
and arm64 were not built locally.

Since this PR comes from a fork, its workflows wait for a maintainer's approval. The same commit has
already been through the Build Ubuntu workflow on our fork, where all twelve build-ubuntu entries
(x64 and arm64) passed, along with DCO and pre-commit:
https://github.com/jmex-sw/IsaacTeleop/actions/runs/41576012036

With the device. These runs predate the merge with main at 1.8; the merge changed none of the
plugin's or the example's code. On 2026-10-30, on the tree of the previous head 8dcc95c53, driven
live by an operator wearing the upper-body part of the suit, with MOXI Player on the same machine
(Isaac Lab 3.0.0 beta2 on Isaac Sim 6.0.1, XR running, retargeting in sync mode), the G1's waist
followed lean, side bend and twist with a correlation of 0.97–0.99, 90–270 ms behind. The operator
went past the waist's limits in lean and twist, where the waist stops following. An earlier run in
the full suit, on 2026-09-23 with pipelined retargeting, gave 0.99 at the same latencies, and turning
on the spot through ±90° moved the G1's yaw less than 12°. Replayed headless from a recording (rerun
on 2026-10-30), the waist followed all three with a correlation of 1.00, one or two physics steps
behind. On 2026-10-30, on Isaac Lab 3.0.0 EA (Isaac Sim 6.1.0), the G1 followed a replayed recording
with Kit's window open and the reference overlay drawn.

On this tree, with isaaccapture 1.8 in Isaac Lab's Python, a recorded Player stream replayed over
loopback drove the task headless on both Isaac Lab versions: each of 900 steps was computed from a
full body pose, at about 12 steps/s on beta2 and 11 on EA, and the README's teleop steps ran on both
without Kit's window.

Running it yourself. Isaac Lab needs this repository's isaaccapture installed in its Python:
no released package carries the body.jmex vendor yet, and Isaac Lab pins the package under its
previous name. The example README has the steps. The SDK and MOXI Player are not distributed with
this PR. A maintainer validation package can be provided through an agreed private channel on
request.

Checklist

  • I have read and understood the contribution guidelines
  • I have run the linter and formatter with SKIP=check-copyright-year pre-commit run --all-files
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix/feature works (or explained why not)
  • I have signed off all my commits (git commit -s) per the DCO

@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview is not auto-deployed for fork PRs.

A maintainer with write access to NVIDIA/IsaacCapture can deploy a preview by
commenting /preview-docs on this PR. Once deployed, the preview
will live at:

https://nvidia.github.io/IsaacCapture/preview/pr-1250/

@coderabbitai

coderabbitai Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Walkthrough

This change adds a j-mex plugin that receives AgileMaster full-body poses and publishes them through OpenXR. It adds body.jmex live-tracker support and a G1 teleoperation example that converts received poses into robot action targets. It also adds an MCAP recording example, build and test configuration, and documentation for the plugin and examples.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant PluginMain
  participant ReceiverSession
  participant FullBodyPublisher
  participant OpenXRTransport
  PluginMain->>ReceiverSession: Poll for a new frame
  ReceiverSession-->>PluginMain: Return mapped full-body frame
  PluginMain->>FullBodyPublisher: Publish frame
  FullBodyPublisher->>OpenXRTransport: Push serialized pose
Loading
Merge Risk: Minimal · up to a91f3

This change adds an optional j-mex full-body plugin, a body.jmex tracker, and examples. No concrete defects are established, and builds without the SDK skip the plugin, so the change appears ready to merge.

Pre-merge checks: Passed 4
Check name Status Explanation
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Title check Passed The title clearly and concisely describes the main changes: adding the AgileMaster full-body plugin and the G1 teleoperation example.
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Tip

🎯 Customize your risk assessments

Guide how CodeRabbit assesses merge risk and security architecture risk. Highlight sensitive changes and explain your security boundaries using Merge Risk Instructions and Security Risk Instructions in your review settings. Security architecture risk assessments require the Advanced plan or above.

✅ Automatic approvals, with your rules

Let CodeRabbit approve pull requests once required threads are resolved, pre-merge checks pass, and your risk limits are met. Enable Automatic approvals in review settings, with overrides for teams or file paths.

CodeRabbit Verify

  • Run CodeRabbit Verify to test the changes in this pull request.
ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: NVIDIA/IsaacCapture/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Enterprise
  • Run ID: b150d31f-05b8-4673-9d90-291312365719
📥 Commits

Reviewing files that changed from the base of the PR and between 2fe1bbe and a91f3af.

📒 Files selected for processing (41)
  • cmake/ClangFormat.cmake
  • docs/source/_data/devices.yaml
  • examples/jmex/.gitignore
  • examples/jmex/README.md
  • examples/jmex/pyproject.toml
  • examples/jmex/python/isaaccapture_examples/jmex/__init__.py
  • examples/jmex/python/isaaccapture_examples/jmex/jmex_g1_retargeting.py
  • examples/jmex/python/isaaccapture_examples/jmex/jmex_g1_tasks.py
  • examples/jmex/python/isaaccapture_examples/jmex/record_jmex_full_body.py
  • src/core/live_trackers/cpp/CMakeLists.txt
  • src/core/live_trackers/cpp/inc/live_trackers/live_deviceio_factory.hpp
  • src/core/live_trackers/cpp/live_deviceio_factory.cpp
  • src/core/live_trackers/cpp/live_full_body_tracker_jmex_impl.cpp
  • src/core/live_trackers/cpp/live_full_body_tracker_jmex_impl.hpp
  • src/plugins/CMakeLists.txt
  • src/plugins/jmex/.gitignore
  • src/plugins/jmex/CMakeLists.txt
  • src/plugins/jmex/README.md
  • src/plugins/jmex/app/CMakeLists.txt
  • src/plugins/jmex/app/main.cpp
  • src/plugins/jmex/core/CMakeLists.txt
  • src/plugins/jmex/core/body_joint_mapping.cpp
  • src/plugins/jmex/core/device_clock.cpp
  • src/plugins/jmex/core/full_body_publisher.cpp
  • src/plugins/jmex/core/inc/jmex/body_joint_mapping.hpp
  • src/plugins/jmex/core/inc/jmex/device_clock.hpp
  • src/plugins/jmex/core/inc/jmex/full_body_publisher.hpp
  • src/plugins/jmex/core/inc/jmex/receiver_session.hpp
  • src/plugins/jmex/core/openxr_convention.cpp
  • src/plugins/jmex/core/openxr_convention.h
  • src/plugins/jmex/core/receiver_session.cpp
  • src/plugins/jmex/plugin.yaml
  • tests/cpp/core/live_trackers/test_vendor_validation.cpp
  • tests/cpp/plugins/CMakeLists.txt
  • tests/cpp/plugins/jmex/CMakeLists.txt
  • tests/cpp/plugins/jmex/test_jmex_plugin_core.cpp
  • tests/python/examples/CMakeLists.txt
  • tests/python/examples/jmex/CMakeLists.txt
  • tests/python/examples/jmex/conftest.py
  • tests/python/examples/jmex/pyproject.toml
  • tests/python/examples/jmex/test_jmex_g1_action_source.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

@ivany-nv ivany-nv self-assigned this Oct 13, 2026
@ivany-nv
ivany-nv self-requested a review October 13, 2026 18:16
@ivany-nv

Copy link
Copy Markdown
Contributor

/preview-docs

@github-actions

Copy link
Copy Markdown
Contributor

✅ Preview deployed: https://NVIDIA.github.io/IsaacCapture/preview/pr-1250/

@ivany-nv

Copy link
Copy Markdown
Contributor

@aristarkhovNV Can you please help review the core change part.

@ivany-nv ivany-nv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary
This adds a MOXI Receiver plugin that publishes AgileMaster poses as FullBodyPose, a body.jmex live tracker that reads them (same shape as the Noitom and Xsens readers), and an Isaac Lab G1 example. CI is green on every job. Built locally without the SDK, the jmex tests pass, and the plugin's own publisher, run against a CloudXR runtime, round-trips through body.jmex into a standard full-body MCAP with every joint value intact. Read independently, the axis change, slot mapping, quaternion order, timestamps and MCAP wiring all hold up. One suggestion, nothing blocking. The src/core/live_trackers reader and factory registration are with @aristarkhovNV; this review covers the plugin, example, tests and docs.

Legend: 🚫 BLOCKER = merge-blocking | 💡 SUGGESTION = meaningful improvement | 🧹 NIT = precise small correction

Finding
🚫 none
💡 src/plugins/jmex/app/main.cpp:126-129 - An unrecognized argument is only logged as a warning. So --channel 2 (space instead of =) or a typo like --chanel=2 leaves the plugin on channel 1, waiting for Player forever. That is the silent mismatch the strict --channel= parsing at L92-L93 exists to prevent. The Xsens plugin rejects unknown options; doing the same here makes the plugin fail fast with exit code 1.
🧹 none

@ivany-nv ivany-nv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary
A second pass, this time against this PR merged onto current main, which is 55 commits past your base. It merges cleanly and builds. ctest -L jmex, the vendor-validation cases, clang_format, pre-commit on the changed files and make current-docs all pass. The body.jmex reader is safe to add: the default vendor is unchanged, there is no schema or recording change, and it compiles against main's newer SchemaTracker. Before approval there are three things: two example fixes (the first two 💡 below) and a rebase onto main so CI reruns against it. The rest are questions and small items.

Legend: 🚫 BLOCKER = merge-blocking | 💡 SUGGESTION = meaningful improvement | ❓ QUESTION = needs your answer | 🧹 NIT = precise small correction

Finding
🚫 none
💡 jmex_g1_tasks.py:30-32 - Isaac Lab 3.0.0 EA, the release main's README names as current, moved this config to isaaclab_tasks.contrib.locomanip_pick_place.locomanipulation_g1_env_cfg (isaac-sim/IsaacLab#5891). The manager_based path no longer exists there, so --external_callback ...register_tasks fails with ModuleNotFoundError. Please import the new path with a fallback to the beta2 one, and name the tested Isaac Lab version in README prerequisite 4.
💡 jmex_g1_tasks.py:711-725 - Returning [] tells teleop_se3_agent.py to reject none of the leftover arguments: list_intersection(remaining_args, []) is empty, and with None it is all of them. So --cloudxr-env none (hyphen) is silently ignored and Isaac Lab starts its own auto-webrtc runtime. That is the failure README L97-99 warns about, now with no error. The callback parses no arguments of its own, so return None is right.
💡 main.cpp:59-75 - This extends my earlier comment. parse_port checks that the whole string is an integer but not its range. So --tcp-port=70000, --channel=0 or --channel=300 go straight to the SDK, and udp_start_port + 1 + channel (L151) can overflow. The noitom and xsens plugins reject out-of-range ports, and receiver_session.hpp:27-28 already states the valid channels.
❓ app/CMakeLists.txt:8-14 - Is Moxi::Receiver static or shared? If it is shared, the installed jmex_plugin has no RPATH to find it. The plugins that link a vendor shared library set INSTALL_RPATH; sharpa_avatar points it at an external SDK. readelf -d install/plugins/jmex/jmex_plugin on your machine answers this.
❓ receiver_session.cpp:136-153 - The joint table is re-read only after the channel drops. Can Player load a different model, as the README's troubleshooting tells operators to do, without the pairing dropping? If it can, slots keep the old joint indices and go out valid. Re-checking MxRGetBoneJointCount on each new sequence id would cover it.
❓ jmex_g1_tasks.py:769-777 - retargeting_execution is unset, so Isaac Lab runs the pipeline in pipelined mode, and the overlay's debug-draw calls run on the retarget worker thread. Which mode was the live run in? Setting mode="sync", or drawing from process_actions, settles it either way.
🧹 .gitignore:5 / ClangFormat.cmake:43 - Use MOXIReceiverSDK*/ and MOXIReceiverSDK[^/]*, so an archive unpacked under its versioned name is not picked up by git add -A or the format check.
🧹 full_body_publisher.hpp:42-48 - body.jmex accepts max_flatbuffer_size, but the plugin always creates a 16 KiB collection, so a smaller value makes the reader throw. Either drop the parameter or add --max-flatbuffer-size= as noitom and xsens do.
🧹 device_clock.cpp:52-60 - A Player timestamp of 2^63 or more turns negative, push_buffer throws, and publish() treats every such frame as a lost session. Falling back to the sample time there keeps push failures transport-only.
🧹 Naming, after the rebase: main now calls the library and the repository "Isaac Capture"; Isaac Teleop stays the teleop component. This affects the plugin README L8-9, the example README L11/29/31/49 and record_jmex_full_body.py:4. The devices.yaml row's URL should be NVIDIA/IsaacCapture. The example README's wheel steps (L22-43) can link the build-from-source page and keep only the Isaac-Lab-specific parts.
🧹 src/plugins/jmex/CMakeLists.txt - The README says Linux x86_64 only. A CMAKE_SYSTEM_NAME check with a "Linux only" skip message, as the xsens and noitom plugins have, reports the right reason on other platforms.

Optional, not needed for approval:

  • Build the SDK-free sources as an always-built core library, with the receiver in the SDK-gated part. The test could then link it, and those files would get the project warning set, as gamepad and spacemouse do.
  • Split JmexG1ActionSource into FullBodySource plus a retargeter, to match the IDeviceIOSource "pure converter" contract.

Please rebase onto main, or we can update the branch, so CI reruns against it before approval.

Adds `jmex`, a plugin for the j-mex AgileMaster motion capture suit, and an example, in two parts,
that consumes what it publishes.

The plugin reads one MOXI Player channel through the MOXI Receiver SDK (0.8.0 or newer, obtained
from j-mex) and runs on Linux only. It fills the 24 `FullBodyPose` slots from the standard human
skeleton Player has loaded, one slot per joint, already in OpenXR axes, and pushes them under the
`body.jmex` vendor on every frame. A slot with no joint, or with no usable value in a frame, goes
out invalid; nothing is inferred from its neighbors. The raw device clock carries the time Player
stamped on the frame (`--device-clock=local` uses the pickup time instead), and if the CloudXR
runtime restarts underneath the plugin, the plugin reconnects rather than ending the run. An
argument it cannot use stops it with exit code 1, naming the argument.

Everything but the receiver is a library that builds without the SDK, so on Linux its unit tests
run in every build that has tests enabled; without the SDK only the plugin executable is skipped.

Isaac Capture learns `body.jmex` as a full-body vendor: a live tracker that reads the plugin's
collection, with a `collection_id` parameter, its entry in the DeviceIO factory, the vendor
validation cases, and a row in `devices.yaml`. An SDK unpacked into the plugin directory, under any
versioned name, is kept out of git and the clang-format check.

`isaaccapture_examples.jmex.jmex_g1_tasks` registers a task that inherits Isaac Lab's
`LocomanipulationG1EnvCfg`, on Isaac Lab 3.0.0 EA or 3.0.0 beta2, and takes its action from the
operator; nothing in Isaac Lab is modified. It is upper-body teleop. Each arm is rebuilt segment by
segment onto the robot's own shoulder, so the operator's arm angles survive while the lengths stay
the robot's, and the elbows and shoulders get IK targets of their own from that same chain. The
wrist's orientation is the operator's hand, carried across through a forearm frame. The operator's
torso turns the robot's shoulders about its waist, so leaning, bending sideways and twisting come
through; their heading does not, so the operator can turn to face a screen without moving the
robot. Only a frame that carries all 24 slots drives the robot, and slots that stay missing are
named on the console.

`isaaccapture_examples.jmex.record_jmex_full_body` writes the same stream to a standard full-body
MCAP.

Tests cover the plugin's argument parsing, slot mapping, change of axes, device clock choice and
session rebuild after a failed push, the vendor's parameters, and the example's action source,
including the geometry of the rebuilt arm and the orientation carried through the forearm frame.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: j-mex <jmex.dev@jmex.com.tw>
@jmex-sw
jmex-sw force-pushed the jmex-plugin-full-body branch from a91f3af to 8dcc95c Compare October 30, 2026 09:26
@jmex-sw

jmex-sw commented Oct 30, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the second pass. I rebased onto main at 49a218122 and force-pushed: the branch is
still one commit, now 8dcc95c53, and everything below is in it.

Finding Done
💡 Isaac Lab 3.0.0 EA path jmex_g1_tasks.py imports the isaaclab_tasks.contrib.locomanip_pick_place config and falls back to the beta2 path only when that module is missing. README prerequisite 4 and "Verified on" name v3.0.0-EA (Isaac Sim 6.1.0) and v3.0.0-beta2.patch1 (Isaac Sim 6.0.1). Running on EA also showed that the overlay enabled its debug-draw extension through isaacsim.core.experimental, which EA's app does not load; it now goes through Kit's extension manager. The README no longer uses --headless, and notes that on EA Ctrl-C leaves Isaac Lab running, as it does for EA's own tasks.
💡 register_tasks() returning [] Returns None.
💡 Argument ranges Parsing moved into the core library as parse_plugin_args, with unit tests. A channel outside 1–255 (the SDK keeps the channel in one byte; Player uses 1–10 live and 255 for replay), a port outside 1–65535, or a channel data port past 65535 stops the plugin with exit code 1, naming the argument. So does an unknown or positional argument, which covers your first-round comment too.
❓ Static or shared Static: the SDK ships only libmoxi_receiver.a. readelf -d on the installed jmex_plugin shows no RPATH or RUNPATH, and its only NEEDED entry besides the C and C++ runtime is libjsoncpp.so.25, which the SDK does not use and most of the in-tree plugins link as well.
❓ Model change without a disconnect Player always drops the pairing before it loads another model, and the plugin re-reads the joint table on the next pairing. Checked with Player on channel 255: standard human skeleton, 65 joints, 24 of 24 slots; loading a robot model logged MOXI Player disconnected, then paired, 50 joints with 4 of 24 slots and the missing ones named; loading the skeleton again logged the disconnect, then paired, 65 joints, 24 of 24.
❓ Retargeting mode The live run was pipelined, Isaac Lab's default. The task now sets RetargetingExecutionConfig(mode="sync"). Headless, the step rate is unchanged (about 11 steps/s on EA, 12 on beta2), and the overlay was checked in Kit on EA. Remeasured live on beta2 in sync: the waist follows at the same 90–270 ms, with correlation 0.97–0.99, as the PR description says.
🧹 Versioned SDK directory MOXIReceiverSDK*/ in .gitignore, MOXIReceiverSDK[^/]* in the format check's exclusions.
🧹 max_flatbuffer_size Dropped: body.jmex takes only collection_id, and both ends use 16 KiB. This is the only change under src/core since the first review.
🧹 Player time past int64 Falls back to the sample time, with a test at the boundary.
🧹 Naming Isaac Capture where the library or the repository is meant, the devices.yaml URL, and the example's wheel step now links Build from source and keeps only what differs for Isaac Lab.
🧹 Linux only CMAKE_SYSTEM_NAME check with a "Linux only" skip message; the plugin's tests moved into the Linux-only block.
Optional Core library Done. Everything but the receiver is jmex_plugin_core, built with or without the SDK and with the project warning set, and the tests link it. The receiver is compiled into jmex_plugin in app/, the only part that needs the SDK.
Optional Splitting JmexG1ActionSource Not in this PR. I'd like to do it as a follow-up.

The PR description has the updated local gate (both toolchains, six entries each) and the device
checks. The same commit passed the Build Ubuntu workflow on our fork: https://github.com/jmex-sw/IsaacTeleop/actions/runs/41028280279. Could you approve
the workflow runs here when you have a moment?

@ivany-nv

Copy link
Copy Markdown
Contributor

Heads-up: #1394 just merged. It moves only-include from [tool.hatch.build.targets.wheel] to [tool.hatch.build], so the sdist gets the same file list as the wheel. Without that, uv build packs the whole example directory into the sdist: media, recordings, a local uv.lock, build trees. Wheels and editable installs are unchanged.

Could you switch examples/jmex/pyproject.toml to the layout in Adding an example?

[tool.hatch.build]
only-include = ["python/isaaccapture_examples/jmex"]

[tool.hatch.build.targets.wheel]
sources = ["python"]

Nothing checks where only-include lives, so rebasing won't flag the old layout.

main conflicted in tests/cpp/plugins/CMakeLists.txt and now asks
examples to set only-include for the sdist too. On top of the merge:

- tests/cpp/plugins/CMakeLists.txt registers the plugin's tests under
  if(TARGET jmex_plugin_core) at the end of the file instead of in the
  Linux-only block, so no existing line of the file changes. The core
  library exists on Linux, with or without the SDK.
- examples/jmex/pyproject.toml sets only-include in [tool.hatch.build],
  so the sdist gets the same file list as the wheel.
- The example's recording and replay steps and the plugin's runtime
  step start from a venv set up as in Running an example.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: j-mex <jmex.dev@jmex.com.tw>
@jmex-sw

jmex-sw commented Nov 3, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the heads-up. I merged main at 834ecad34 into the branch, so the PR head is now
3f3bb7be6, one merge commit on top of 8dcc95c53; nothing was force-pushed.

  • examples/jmex/pyproject.toml sets only-include in [tool.hatch.build], as in Adding an example.
    Built from a copy with media, a stray recording, a uv.lock and a build tree beside the package,
    the sdist holds only the package, pyproject.toml, .gitignore and PKG-INFO, and the wheel built
    from the unpacked sdist has the same four modules as before.
  • main conflicted in tests/cpp/plugins/CMakeLists.txt, next to the Linux-only note this PR had
    edited. The plugin's tests now register under if(TARGET jmex_plugin_core) at the end of that
    file, so the PR no longer changes any existing line there. The core library is defined on Linux,
    with or without the SDK, as before.
  • The example's recording and replay steps and the plugin README's runtime step now start from a venv
    set up as in Running an example.

The PR description has the local gate rerun on this tree. The same commit passed the Build Ubuntu
workflow on our fork: https://github.com/jmex-sw/IsaacTeleop/actions/runs/41576012036. Could you
approve the workflow runs for the updated head 3f3bb7be6?

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

✅ Preview deployed: https://NVIDIA.github.io/IsaacCapture/preview/pr-1250/

Built from commit 3f3bb7b. It may take a minute for the updated preview to appear.

@ivany-nv

ivany-nv commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

Review in progress.

@ivany-nv ivany-nv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary
Thanks. Every item from the last round is fixed. Merged onto main (75d567089), the jmex code builds without warnings, all jmex tests (23 Catch2, 91 pytest) pass, and upstream CI is green. What is left is inline: 1 🚫, 3 💡, 1 ❓, 8 🧹.

Legend: 🚫 BLOCKER = merge-blocking | 💡 SUGGESTION = meaningful improvement | ❓ QUESTION = needs your answer | 🧹 NIT = precise small correction

The CTS review has passed.

add_subdirectory(rebot_devarm_leader)
add_subdirectory(manus)
add_subdirectory(haptikos)
add_subdirectory(jmex)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 Please gate the plugin behind BUILD_PLUGIN_JMEX, default OFF. This line is unconditional, so every default configure, Build Ubuntu included, builds jmex_plugin_core and runs its tests. Every other partner plugin option defaults to OFF (CMakeLists.txt:70-73); whether a partner plugin runs in CI is for the CI owners to decide.

  1. Add the option after CMakeLists.txt:73.
  2. Move this line into if(BUILD_PLUGIN_JMEX) with the gated plugins below.
  3. Gate tests/python/examples/CMakeLists.txt:6 the same way; the C++ tests follow via if(TARGET jmex_plugin_core).
  4. Add -DBUILD_PLUGIN_JMEX=ON to the build commands in both READMEs and the install hints in jmex_g1_tasks.py and record_jmex_full_body.py, as noitom's README does.
  5. Optional: warn, naming the option, when it is ON but no SDK is found, as sharpa_avatar does.


# Without this the agile lower-body policy crouches over the first steps, and the
# IK then cannot reach the targets it is being given.
self.scene.robot.spawn.articulation_props.fix_root_link = True

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Isaac Lab v3.0.0, released 2026-10-20 after EA, changed G1_29DOF_CFG.spawn.articulation_props to a list, [PhysxArticulationCfg(...), NewtonArticulationCfg(...)], and put fix_root_link on the spawner (isaaclab_assets/robots/unitree.py:426-434). Its own fixed-base G1 task sets self.robot.spawn.fix_root_link = True. On v3.0.0 the import succeeds, and then __post_init__ raises AttributeError: 'list' object has no attribute 'fix_root_link'. Please set it on the config when articulation_props is a single legacy config, and on spawn.fix_root_link otherwise. Add v3.0.0 to "Verified on" once you have run it.

Comment on lines +8 to +10
args:
- "--channel=1"
- "--collection-id=jmex_full_body"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 When Isaac Capture launches the plugin with these args, the receiver binds TCP on every interface and advertises to 255.255.255.255 (receiver_config.hpp:25-34). Any MOXI Player on the subnet can pair first. Nothing authenticates the pairing, and the poses it delivers can drive a robot. The xsens plugin's plugin.yaml pins its receiver to 127.0.0.1 for this reason and explains why in a comment.

If Player can pair over loopback, please add --bind-ip=127.0.0.1 and --broadcast-ip=127.0.0.1 here with a short comment. Then say in the README what a Player on another machine needs: both flags set to the NIC facing it, plus a firewall rule. If Player cannot pair over loopback, document the exposure in the README instead.

Comment on lines +90 to +93
else if (starts_with(arg, "--collection-id="))
{
args.collection_id = arg.substr(std::string("--collection-id=").size());
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 An empty --collection-id= is accepted, for example --collection-id=$ID with ID unset. The plugin then publishes on a collection no body.jmex consumer can select, because the reader rejects an empty id (live_full_body_tracker_jmex_impl.cpp:44-47). The README says a value the plugin cannot use stops it. Rejecting the empty id here, with a test, keeps that promise and matches what the xsens plugin does.

Comment on lines +41 to +45
//! Base of the per-channel data ports -- channel N receives on this + 1 + N. Only worth moving
//! to make those ports predictable, for a firewall rule or to tell two receivers apart; a
//! collision already relocates itself and announces where it landed. It does not move where
//! advertisements are sent, which Player fixes at 10100.
int udp_start_port = 10100;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ Does the SDK accept motion datagrams on the channel's UDP data port only from the paired Player? If it doesn't, a firewall rule is the only protection, and the README note on the network default (see the plugin.yaml comment) should say so.

| `--tcp-port=` | `7000` | A second plugin on this machine; with the default, the second one to start finds the port taken |
| `--udp-start-port=` | `10100` | A firewall rule needs predictable data ports. Channel N receives on this + 1 + N |

One plugin receives one channel. A second plugin on this machine needs only its own `--tcp-port=`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 A second plugin also needs its own --channel= and --collection-id=, as the table above says, not only --tcp-port=.

Comment thread examples/jmex/README.md
1. **The plugin, installed.** `cmake --install build` — a plugin that has only been built is not
discoverable. The [plugin README](../../src/plugins/jmex/README.md) lists the SDK it needs and
the build prerequisites.
2. **MOXI Player**, with the **standard human skeleton** loaded, on a known channel. The plugin's

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 "MOXI Player" here, but the plugin README says only MOXI Player for Robot works.

Comment on lines +297 to +299
requirements:
- MOXI Receiver SDK 0.8.0 or newer (on request from j-mex)
- MOXI Player with the standard human skeleton loaded, on this machine or the same subnet

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Same naming point as in the example README: only MOXI Player for Robot works. These requirements could also say Linux x86_64, as the Wuji row does (L268).

Comment on lines +60 to +63
logger->info("receiver on tcp {}:{}, advertising to {}, channel data from port {}",
args.receiver.bind_ip.empty() ? "every interface" : args.receiver.bind_ip, args.receiver.tcp_port,
args.receiver.broadcast_ip.empty() ? "255.255.255.255" : args.receiver.broadcast_ip,
args.receiver.udp_start_port + 1 + channel);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 This repeats the data-port formula from plugin_args.cpp:111. A ReceiverConfig::data_port() would keep the log line and the check in step. The SDK can relocate the port, so this line should call it the expected port.

Comment on lines +103 to +106
else
{
throw std::runtime_error("unknown argument '" + arg + "': every option is --name=value");
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Optional: --help lands here as an unknown argument, with exit code 1. Accepting it and printing the flags would help operators.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants