feat(soma): add body and hand pose transport

docs/source/device/soma.rst

@@ -0,0 +1,244 @@
+.. SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+.. SPDX-License-Identifier: Apache-2.0
+
+SOMA Pose Transport
+===================
+
+Isaac Teleop can transport SOMA body and hand data through FlatBuffers and
+``SchemaPusher``. Producers publish either joint rotations or evaluated joint
+poses. Consumer-side sources normalize both representations to global joint
+poses for retargeting or visualization.
+
+The schemas and examples are defined and verified against the SOMA-X v0.3.1
+pose interface. See the `SOMA-X repository <https://github.com/NVlabs/SOMA-X>`_
+for the upstream model and assets.
+
+.. contents:: On this page
+   :local:
+   :depth: 2
+
+Data flow
+---------
+
+Body and hand data use separate schemas and collections. Applications can
+select their body and hand sources independently.
+
+.. code-block:: text
+
+   vendor body SDK -> SOMA body FlatBuffer -> SchemaPusher -> SOMA body tracker
+   vendor hand SDK -> SOMA hand FlatBuffer -> SchemaPusher -> SOMA hand tracker
+
+The body schema contains SOMA's complete body control set, including its hand
+controls. The standalone hand schemas provide independent left and right hand
+streams. Isaac Teleop does not automatically fuse standalone hands with the
+hand controls in a body payload.
+
+Choose a representation
+-----------------------
+
+Use the representation closest to the producer's native data.
+
+.. list-table::
+   :header-rows: 1
+   :widths: 20 30 50
+
+   * - Representation
+     - FlatBuffer types
+     - Behavior
+   * - ``joint-rotations``
+     - ``SomaBodyJointRotations`` and ``SomaHandJointRotations``
+     - Publishes SOMA joint rotations and the global Hips or Wrist translation.
+       The consumer evaluates forward kinematics against a prepared SOMA
+       identity.
+   * - ``joint-poses``
+     - ``SomaBodyJointPoses`` and ``SomaHandJointPoses``
+     - Publishes global position and orientation for each provided joint. The
+       consumer maps these poses directly and does not run per-frame forward
+       kinematics.
+
+``joint-rotations`` most directly matches the SOMA-X pose-control interface.
+``joint-poses`` is an Isaac Teleop transport profile for producers that already
+have evaluated global joint transforms.
+
+Both representations use positions in meters and unit XYZW quaternions. They
+use a right-handed reference frame with positive Y up and positive Z forward.
+The publisher must convert its native skeleton and reference frame before
+serialization. See :ref:`tracker-reference` for the complete field and
+validity contracts.
+
+Consumers normalize finite quaternions whose norm is greater than ``1e-12``. A
+zero, degenerate, or non-finite quaternion, position, or global translation is
+unavailable. Quaternion signs are not canonicalized, so ``q`` and ``-q`` remain
+equivalent.
+
+Rotation consumers resolve the SOMA v0.3.1 reference through SOMA-X and pass it
+explicitly during forward kinematics. The prepared identity and subject scale
+remain caller-selected calibration and are not constrained by this check.
+
+Body and hand joint values are sorted, unique keyed vectors. Every entry carries
+an explicit ``SomaBodyJoint`` or ``SomaHandJoint`` identifier. A missing joint is
+unavailable for that frame; producers do not serialize placeholder values for
+joints they cannot provide. Consumers expand the keyed values into SOMA-X's
+canonical dense order when invoking its body or hand helpers.
+
+Build the examples
+------------------
+
+Start with a configured Isaac Teleop checkout. See
+:doc:`../getting_started/build_from_source/index` for the complete build
+prerequisites. Install the live viewer and both deterministic publishers:
+
+.. code-block:: bash
+
+   uv pip install -e ./examples/deviceio_live_view
+   uv pip install -e ./examples/soma_body_publisher
+   uv pip install -e ./examples/soma_hand_publisher
+
+   .venv/bin/cmake -S . -B build
+   .venv/bin/cmake --build build \
+     --target isaacteleop_python \
+     -j4
+
+The example packages install ``py-soma-x==0.3.1``.
+The publishers use the Python-accessible ``TensorPushTracker`` and its native
+``SchemaPusher`` transport; no separate sender executable is needed. Demo samples
+use the session's monotonic clock for both timestamps.
+
+Run the body example
+--------------------
+
+Start the viewer first. It launches the CloudXR runtime and prints the viewer
+URL. Open that URL before starting the publisher. On a remote host, replace
+``localhost`` in the URL with the host name or IP address.
+
+.. code-block:: bash
+
+   uv run --no-sync python -m isaaccapture_examples.deviceio_live_view \
+     --accept-eula \
+     --body-schema soma
+
+In another terminal, load the runtime environment and start the body
+publisher:
+
+.. code-block:: bash
+
+   source ~/.cloudxr/run/cloudxr.env
+   uv run --no-sync python -m isaaccapture_examples.soma_body_publisher \
+     --loop
+
+These commands use ``joint-rotations``. To publish evaluated joint poses, add
+``--body-representation joint-poses`` to the publisher and
+``--soma-body-representation joint-poses`` to the viewer.
+
+Run the hand example
+--------------------
+
+Start the viewer with SOMA hands:
+
+.. code-block:: bash
+
+   uv run --no-sync python -m isaaccapture_examples.deviceio_live_view \
+     --accept-eula \
+     --hand-schema soma
+
+In another terminal, load the runtime environment and start the hand
+publisher:
+
+.. code-block:: bash
+
+   source ~/.cloudxr/run/cloudxr.env
+   uv run --no-sync python -m isaaccapture_examples.soma_hand_publisher \
+     --loop
+
+These commands use ``joint-rotations``. To publish evaluated joint poses, add
+``--hand-representation joint-poses`` to the publisher and
+``--soma-hand-representation joint-poses`` to the viewer.
+
+Collection IDs and schema identifiers
+-------------------------------------
+
+A collection ID pairs a producer with a consumer and is not encoded in the
+FlatBuffer payload. The tensor identifier selects the representation carried by
+that collection. The demo publishers use these fixed collection IDs; another
+producer can choose different IDs as long as its consumers use the same values.
+
+.. list-table::
+   :header-rows: 1
+   :widths: 30 30 40
+
+   * - Data
+     - Collection ID
+     - Viewer override
+   * - Body
+     - ``soma_body_demo``
+     - ``--soma-body-collection-id``
+   * - Left hand
+     - ``soma_hand_left_demo``
+     - ``--soma-left-hand-collection-id``
+   * - Right hand
+     - ``soma_hand_right_demo``
+     - ``--soma-right-hand-collection-id``
+
+The representation selected by the publisher and consumer must match. The
+transport identifiers are ``soma_body_joint_rotations``,
+``soma_body_joint_poses``, ``soma_hand_joint_rotations``, and
+``soma_hand_joint_poses``.
+
+Vendor integration
+------------------
+
+A vendor integration converts its native skeleton to one of the checked-in
+SOMA contracts and publishes it through ``SchemaPusher``. Use these schemas as
+the serialization source:
+
+- :code-file:`src/core/schema/fbs/soma_common.fbs`
+- :code-file:`src/core/schema/fbs/soma_body_common.fbs`
+- :code-file:`src/core/schema/fbs/soma_body_joint_rotations.fbs`
+- :code-file:`src/core/schema/fbs/soma_body_joint_poses.fbs`
+- :code-file:`src/core/schema/fbs/soma_hand_joint_rotations.fbs`
+- :code-file:`src/core/schema/fbs/soma_hand_joint_poses.fbs`
+
+The FlatBuffer transport and generated trackers do not depend on SOMA-X or
+PyTorch. A rotation consumer uses SOMA-X and its resolved neutral identity to
+evaluate forward kinematics. A joint-pose consumer does not need SOMA-X for
+per-frame evaluation.
+
+Current limitations
+-------------------
+
+- The examples evaluate rotations against a neutral identity. They do not
+  perform subject calibration.
+- Body and standalone hand streams are not fused automatically.
+- A producer and consumer must select the same representation.
+- The live viewer selects sources at launch. It does not switch schemas while
+  running.
+
+Troubleshooting
+---------------
+
+No module named ``isaaccapture_examples``
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Install the corresponding example with ``uv pip install -e`` and launch it
+with ``uv run --no-sync`` from the repository root.
+
+The publisher cannot connect
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Start the viewer, open its client, and then source
+``~/.cloudxr/run/cloudxr.env`` in the publisher terminal. The environment file
+identifies the active CloudXR OpenXR runtime.
+
+The viewer receives no skeleton
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Confirm that the publisher and viewer use the same representation and
+collection names.
+
+See also
+--------
+
+- :code-file:`examples/soma_body_publisher/README.md`
+- :code-file:`examples/soma_hand_publisher/README.md`
+- :code-file:`examples/deviceio_live_view/README.md`
+- :doc:`trackers`

docs/source/device/trackers.rst

@@ -24,6 +24,10 @@ reading it from OpenXR tensor collections via the
 - :code-file:`Generic3AxisPedalTracker <src/core/deviceio_trackers/trackers.toml>` -- foot pedal axis values (generated)
 - :code-file:`JointStateTracker <src/core/deviceio_trackers/trackers.toml>` -- named joint-space device state (leader arms, exoskeletons, gloves, ...) (generated)
 - :code-file:`Se3Tracker <src/core/deviceio_trackers/trackers.toml>` -- generic SE3 (6-DoF) pose sources (tracker pucks, mocap rigid bodies, logical trackers) (generated)
+- :code-file:`SomaBodyJointRotationsTracker <src/core/deviceio_trackers/trackers.toml>` -- SOMA body joint rotations with 77 controls (generated)
+- :code-file:`SomaBodyJointPosesTracker <src/core/deviceio_trackers/trackers.toml>` -- evaluated SOMA body poses for 77 joints (generated)
+- :code-file:`SomaHandJointRotationsTracker <src/core/deviceio_trackers/trackers.toml>` -- SOMA hand joint rotations with 25 controls (generated)
+- :code-file:`SomaHandJointPosesTracker <src/core/deviceio_trackers/trackers.toml>` -- evaluated SOMA hand poses for 25 joints (generated)
 
 All trackers follow the same lifecycle:
 
@@ -269,6 +273,109 @@ reads the PICO ``XR_BD_body_tracking`` extension directly.
    ``FullBodyTrackerPico`` remains available as a deprecated alias for
    ``FullBodyTracker`` so existing scripts run unchanged.
 
+SOMA body trackers
+~~~~~~~~~~~~~~~~~~~~~
+
+Reads the SOMA body joint-rotation transport schema from an OpenXR tensor collection. The schema is
+defined and verified against SOMA-X v0.3.1 and its v0.3 pose interface. A vendor adapter converts
+its native skeleton to :code:`SomaBodyJointRotations`, serializes the checked-in schema, and
+publishes it through ``SchemaPusher``. The tracker transports and records the fixed payload; it does
+not perform vendor-specific skeleton or reference-pose conversion.
+
+The payload contains the 77 user-facing joint rotations for SOMA's 78-joint public skeleton. It omits
+the virtual Root, which is always identity and is added internally by ``SOMALayer``. Hips is the
+global rotation; the remaining rotations use SOMA-X v0.3.1 with ``reference_pose=None`` and
+``absolute_pose=False``. Producers must convert historical, custom, or vendor-native references
+before publishing. If the native skeleton has a separate Root, the producer composes it into the
+global Hips rotation and translation. Rotations are unit XYZW quaternions; ``q`` and ``-q`` represent
+the same rotation, and the transport requires no quaternion-sign convention. ``global_translation``
+is applied to Hips and is measured in meters. The reference frame is right-handed with +Y up and +Z
+forward. Each joint rotation and the global translation have independent validity.
+
+A SOMA-X consumer can convert the quaternion array with
+`soma.geometry.transforms.quaternion_xyzw_to_matrix <https://nvlabs.github.io/SOMA-X/stable/api/geometry.html>`__
+and pass the resulting matrices to ``SOMALayer.pose(..., pose2rot=False)``.
+
+When a producer already has evaluated global positions and orientations, it can
+publish ``SomaBodyJointPoses`` through ``SomaBodyJointPosesTracker`` instead.
+That profile carries a global pose and independent validity for each of the same
+77 public joints. Positions use meters and orientations use unit XYZW
+quaternions in the same reference-frame convention. It requires no downstream
+FK.
+
+The integration defines the reference frame and keeps it stable for the collection. To align this
+pose with another tracker, such as HMD full-body tracking, the producer transforms its native
+tracking frame into the same physical reference frame before publishing.
+
+For a retargeting graph, ``SomaBodySource(name="body", collection_id="vendor.soma", layer=layer)``
+registers the rotation tracker by default. Pass ``representation="joint-poses"``
+to select the evaluated-pose tracker; that profile does not require ``layer``.
+The source evaluates rotations once per graph step against a caller-supplied
+SOMA identity or directly maps evaluated poses, then emits ``SomaBodyInput``
+with global poses for all 77 joints.
+An inactive tracker produces an absent output. The raw tracker and transport do
+not require SOMA-X or PyTorch. The live-view example supplies those evaluation
+dependencies, renders the SOMA public hierarchy, and offers launch-time
+body-schema selection. It does not map SOMA into ``FullBodyPose``.
+
+- Schemas: :code-file:`src/core/schema/fbs/soma_body_common.fbs`, :code-file:`src/core/schema/fbs/soma_body_joint_rotations.fbs`, :code-file:`src/core/schema/fbs/soma_body_joint_poses.fbs`
+- Manifest: :code-file:`src/core/deviceio_trackers/trackers.toml` (``soma_body_joint_rotations`` and ``soma_body_joint_poses``)
+- C++ header: ``#include <deviceio_trackers/soma_body_joint_rotations_tracker.hpp>``
+- Python import: ``from isaaccapture.deviceio_trackers import SomaBodyJointRotationsTracker``
+- Evaluated-pose C++ header: ``#include <deviceio_trackers/soma_body_joint_poses_tracker.hpp>``
+- Evaluated-pose Python import: ``from isaaccapture.deviceio_trackers import SomaBodyJointPosesTracker``
+- Graph source: :code-file:`src/python/isaaccapture/retargeting_engine/deviceio_source_nodes/soma_body_source.py`
+- Live-view example: :code-file:`examples/deviceio_live_view/README.md`
+- Demo publisher: :code-file:`examples/soma_body_publisher/README.md`
+- Record channels: ``soma_body_joint_rotations``, ``soma_body_joint_rotations_tracked`` | MCAP schema: ``core.SomaBodyJointRotationsRecord``
+- Evaluated-pose record channels: ``soma_body_joint_poses``, ``soma_body_joint_poses_tracked`` | MCAP schema: ``core.SomaBodyJointPosesRecord``
+- Tests:
+
+  - :code-file:`tests/cpp/core/schema/test_soma_body_joint_rotations.cpp`
+  - :code-file:`tests/cpp/core/schema/test_soma_body_joint_poses.cpp`
+  - :code-file:`tests/python/core/schema/test_soma_body_joint_rotations.py`
+  - :code-file:`tests/python/core/retargeting_engine/test_soma_body_source.py`
+
+SomaHandJointRotationsTracker and SomaHandJointPosesTracker
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Reads either of the two SOMA hand transport profiles defined and verified against SOMA-X v0.3.1.
+``SomaHandJointRotations`` contains 25 unit XYZW quaternions in ``SomaHandJoint`` order. Wrist
+is the global rotation; the remaining rotations are joint-local relative to the fixed v0.3.1
+reference contract. ``global_translation`` is applied to Wrist and uses the same units and
+reference-frame convention as ``SomaBodyJointRotations``.
+
+When a producer has already evaluated forward kinematics, it can instead publish
+``SomaHandJointPoses``. That profile carries a global position, orientation, and validity for each
+of the same 25 joints and requires no downstream FK. Create one tracker per hand collection; each
+payload identifies its side through ``handedness``. As with the body contract, a vendor adapter
+converts its device-native hand data before calling ``SchemaPusher``.
+
+For a retargeting graph, ``SomaHandSource`` selects the rotation tracker by default and emits an
+evaluated ``SomaHandInput``. Pass ``representation="joint-poses"`` to select the evaluated-pose
+tracker. A rotation source requires a prepared SOMA hand layer; an evaluated-pose source does not.
+The DeviceIO live-view example selects body and hand inputs independently and creates one source for
+each hand.
+
+- Schemas: :code-file:`src/core/schema/fbs/soma_common.fbs`, :code-file:`src/core/schema/fbs/soma_hand_joint_rotations.fbs`, :code-file:`src/core/schema/fbs/soma_hand_joint_poses.fbs`
+- Manifest: :code-file:`src/core/deviceio_trackers/trackers.toml` (``soma_hand_joint_rotations`` and ``soma_hand_joint_poses``)
+- Rotation C++ header: ``#include <deviceio_trackers/soma_hand_joint_rotations_tracker.hpp>``
+- Rotation Python import: ``from isaaccapture.deviceio_trackers import SomaHandJointRotationsTracker``
+- Evaluated-pose C++ header: ``#include <deviceio_trackers/soma_hand_joint_poses_tracker.hpp>``
+- Evaluated-pose Python import: ``from isaaccapture.deviceio_trackers import SomaHandJointPosesTracker``
+- Graph source: :code-file:`src/python/isaaccapture/retargeting_engine/deviceio_source_nodes/soma_hand_source.py`
+- Live-view example: :code-file:`examples/deviceio_live_view/README.md`
+- Demo publisher: :code-file:`examples/soma_hand_publisher/README.md`
+- Rotation record channels: ``soma_hand_joint_rotations``, ``soma_hand_joint_rotations_tracked`` | MCAP schema: ``core.SomaHandJointRotationsRecord``
+- Evaluated-pose record channels: ``soma_hand_joint_poses``, ``soma_hand_joint_poses_tracked`` | MCAP schema: ``core.SomaHandJointPosesRecord``
+- Tests:
+
+  - :code-file:`tests/cpp/core/schema/test_soma_hand_joint_rotations.cpp`
+  - :code-file:`tests/cpp/core/schema/test_soma_hand_joint_poses.cpp`
+  - :code-file:`tests/python/core/schema/test_soma_hand_joint_rotations.py`
+  - :code-file:`tests/python/core/schema/test_soma_hand_joint_poses.py`
+  - :code-file:`tests/python/core/retargeting_engine/test_soma_hand_source.py`
+
 FrameMetadataTrackerOak
 ~~~~~~~~~~~~~~~~~~~~~~~
 

docs/source/index.rst

@@ -62,6 +62,7 @@ Table of Contents
 
    device/trackers
    device/add_device
+   device/soma
    device/joint_space
    device/body_tracking
    device/haptic_feedback

docs/source/references/generated_trackers.rst

@@ -89,6 +89,18 @@ Generated today
    * - ``se3_tracker``
      - ``Se3Tracker``
      - Overrides ``class``; the default would give ``Se3TrackerTracker``
+   * - ``soma_body_joint_rotations``
+     - ``SomaBodyJointRotationsTracker``
+     - Keyed SOMA body rotations for up to 77 controls of the 78-joint skeleton; defined against SOMA-X v0.3.1
+   * - ``soma_body_joint_poses``
+     - ``SomaBodyJointPosesTracker``
+     - Keyed evaluated SOMA positions and orientations for up to 77 public joints
+   * - ``soma_hand_joint_rotations``
+     - ``SomaHandJointRotationsTracker``
+     - Keyed SOMA hand rotations for up to 25 controls; defined against SOMA-X v0.3.1
+   * - ``soma_hand_joint_poses``
+     - ``SomaHandJointPosesTracker``
+     - Keyed evaluated SOMA positions and orientations for up to 25 joints
    * - ``oglo_tactile``
      - ``OgloTactileTracker``
      - MCAP channels are ``oglo``/``oglo_tracked``, so ``channel`` is overridden