Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 1 addition & 0 deletions .cursorrules
1 change: 1 addition & 0 deletions .github/copilot-instructions.md
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ repos:
rev: v2.4.1
hooks:
- id: codespell
args: ["--write-changes", "-L", "atleast,inout,ether"] # Provide a comma-separated list of misspelled words that codespell should ignore (for example: '-L', 'word1,word2,word3').
args: ["--write-changes", "-L", "atleast,inout,ether,retuned"] # Provide a comma-separated list of misspelled words that codespell should ignore (for example: '-L', 'word1,word2,word3').
exclude: \.(svg|pyc|stl|dae|lock)$

- repo: https://github.com/pre-commit/mirrors-clang-format
Expand Down
150 changes: 149 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# AI Code Assistant Instructions for MoveIt Pro Example Workspace

> This file is the canonical source of repository-wide instructions for every AI assistant. The `CLAUDE.md` beside it, plus `.cursorrules` and `.github/copilot-instructions.md`, are symlinks to it (the same convention as the moveit_pro repository). Edit `AGENTS.md`; never replace a symlink with a copy.

## MuJoCo Scene Files

### Keyframe qpos must match model DOF count
Expand All @@ -15,7 +17,36 @@ Each joint type contributes to qpos:
- **hinge/slide**: 1 value each
- **ball**: 4 values (quaternion)

After adding or removing bodies with joints, **remove the keyframe** and let MuJoCo use body `pos=` attributes for initial positions.
After adding or removing bodies with joints, either widen the keyframe's `qpos` to match or
**remove the keyframe** and let MuJoCo use body `pos=` attributes for initial positions. A
keyframe that has to stay (one a `ResetMujocoKeyframe` Objective or a `mujoco_keyframe`
hardware param targets) is worth guarding with a parse-the-MJCF length check - see
`test_keyframe_qpos_matches_model_dof_count` in
`src/lunar_sim/test/test_husky_mujoco_geometry.py`.

### `<include>` inside a `<body>` discards the included file's own wrapping element

When an included file's root element is itself a `<body>` (e.g. one file per wheel, mirroring `hangar_sim/description/*_wheel_link.xml`), MuJoCo's compiler splices in only that root element's **children** — the wrapping `<body>`'s own `name`/`pos` are silently discarded, not nested. Two wheel bodies written this way collapse into one, and their `<inertial>` tags collide: `Schema violation: unique element 'inertial' found N times`.

The real body (with its real `name`/`pos`) must be declared inline at the include site; the included file's own root element is just a throwaway wrapper to satisfy "one root element" for valid standalone XML:

```xml
<!-- in the parent file -->
<body name="front_left_wheel_link" pos="0.256 0.2829 0.02913">
<include file="front_left_wheel_link.xml" />
</body>
```

```xml
<!-- front_left_wheel_link.xml -->
<body>
<inertial .../>
<joint name="front_left_wheel_joint" .../>
<geom .../>
</body>
```

Verified against MuJoCo 3.6.0 (`picknikciuser/moveit-pro:main-jazzy-amd64-cuda13.2-cudnn9`'s bundled Python bindings) with a minimal `mj_name2id`/body-count check; see `src/lunar_sim/description/husky_a300.xml` for a real usage.

### Velocity actuators: `armature/kv` time-constant must stay below the timestep

Expand All @@ -27,6 +58,123 @@ The two coupled numbers live in different files: the actuator `kv` is in the `<v

Rule of thumb when changing a sim `timestep`: for every velocity actuator, check `armature/kv < timestep`. The symptom of violation is a joint that ignores commands (pinned), not one that oscillates.

### Vendored A300 visual meshes: converting DAE to OBJ for MuJoCo

MuJoCo has no COLLADA (`.dae`) loader, but `clearpath_platform_description`'s A300 visual
meshes (`chassis.dae`, `livery.dae`, `status_lights.dae`, `attachments/bumper.dae`) are only
shipped as DAE - unlike the wheel/collision meshes, which are already STL. The vendored files
are kept byte-identical to upstream (never edit them); convert to OBJ and commit the result into
the consuming package's own `description/assets/` instead (cross-package MuJoCo mesh paths
don't survive a colcon install split, so the vendored STL meshes get copied in the same way).

`trimesh` is already present in the `picknikciuser/moveit-pro` runtime image, but its DAE loader
needs `pycollada`, which is not. Installing it into a `--rm` container only touches that
container's throwaway layer, not the image itself:

```
docker run --rm --mount type=bind,src=$PWD,dst=/work --entrypoint bash picknikciuser/moveit-pro:<tag> -c \
"pip install --break-system-packages --no-cache-dir pycollada && python3 -c '
import trimesh
mesh = trimesh.load(\"/work/chassis.dae\", force=\"scene\").dump(concatenate=True)
mesh.export(\"/work/chassis.obj\", include_texture=False)
'"
```

`include_texture=False` skips emitting a companion `.mtl` - apply color via a plain MuJoCo
`<material>` on the geom instead of trying to carry over per-face COLLADA materials. The
converted mesh keeps the DAE's own local-frame vertices, so it drops onto its parent body with
whatever `pos`/`quat` the URDF's visual `<origin>` implies - work that out from the xacro rather
than eyeballing it against a render; see `src/lunar_sim/description/husky_a300.xml`'s bumper
comment for a worked example (a mount-frame offset and a visual-origin counter-offset that cancel
to a plain identity transform).

### `mode="targetbody"` only rotates a camera, it doesn't move it

A `<camera mode="targetbody" target="...">` continuously re-aims to face the target body, but its
`pos` is still a fixed point in its parent body's frame - world-fixed only when the camera is a
direct child of `<worldbody>` (as in this repo's scene cameras); one nested under a moving body
would still be dragged along by that body, just no longer re-oriented independently. Either way,
for a scene camera meant to frame a mobile base throughout an
objective (not just at its starting pose), remember the viewing angle toward the rest of the
scene - the ground plane, in particular - changes as the base drives away from that fixed point,
which can reveal rendering artifacts (e.g. directional-light shadow-map aliasing past the shadow
frustum's edge, see `src/lunar_sim/description/husky_scene.xml`) that were not visible from the
starting pose. Verify renders at more than one point along the objective, not just at rest.

### `texrepeat` with `texuniform="true"` is repeats-per-metre, not repeats-over-the-whole-geom

With `texuniform="true"` on a `<material>`, `texrepeat="R R"` means the texture repeats R times
per metre of world space, not R times across the whole geom - the opposite of the more intuitive
"total tiles across this surface" reading. A 20x20 m ground plane with `texrepeat="8 8"` therefore
tiles every `1/8 = 0.125 m`, not every `20/8 = 2.5 m`; the smaller tile shows as an obvious
repeating grid at any zoom that puts more than a couple tiles in frame. To get a target tile size
of `S` metres, use `texrepeat="${1/S} ${1/S}"` (e.g. `0.3 0.3` for a ~3.3 m tile), independent of
the geom's own size.

### `<texture file="...">` doesn't load JPEG

MuJoCo 3.6's built-in texture loader for `<texture type="2d" file="...">` accepts PNG, KTX, or its
own custom binary format - a `.jpg`/`.jpeg` file fails the model load with `Non-PNG texture,
assuming custom binary file format, unexpected file size`, not a clearer "unsupported format"
error.
Photoreal ground/wall textures are often distributed as JPEG (e.g. ambientCG, Poly Haven, or a NASA
mission-photo scan); convert to PNG before wiring into a scene - see
`src/lunar_sim/description/assets/lunar_regolith_untiled.png` and the texture's
provenance note in `src/lunar_sim/README.md`.

Separately: this repo's root `.gitattributes` LFS-tracks every `*.jpg`/`*.png`/`*.jpeg` with no
per-file exceptions (confirmed against every existing image asset in the repo, down to 19 KB
thumbnails) - a new texture/image asset should go through the normal `git add` + LFS flow like any
other, not a one-off `.gitattributes` carve-out. `*.stl`/`*.STL` are LFS-tracked the same way.

### `<compiler meshdir="...">` also resolves `<hfield file="...">`, unlike `<texture file="...">`

A `<compiler meshdir="assets">` attribute applies to both `<mesh>` **and** `<hfield>` file paths,
but not to `<texture>` (textures use the separate, here-unset `texturedir`, which defaults to the
model file's own directory). So with `meshdir="assets"`, a mesh reference like
`file="chassis.obj"` is correct un-prefixed, and an hfield PNG must be written the same
un-prefixed way (`file="lunar_hfield.png"`, not `file="assets/lunar_hfield.png"`) even though a
`<texture>` in the very same file needs the full `file="assets/regolith.png"` path. Getting this
wrong fails the model load with `Error opening file 'assets/assets/lunar_hfield.png'` (meshdir
prepended twice), not a clearer "not found" error. See
`src/lunar_sim/description/husky_scene.xml`'s `<hfield>` element.

### An `<hfield file="...">` PNG is loaded bottom-up: image row 0 lands at maximum y

MuJoCo flips a heightfield PNG's rows on load, so the image's *first* row becomes the *last*
`hfield_data` row - i.e. image row 0 is world `+size_y`, not `-size_y` (image column 0 is world
`-size_x`, unflipped). Verified against MuJoCo 3.6 by loading a ramp PNG and reading
`model.hfield_data`, plus downward `mj_ray` probes on a quadrant-coded hfield. A generator that
writes rows in increasing-y order silently produces terrain mirrored about y=0, which nothing
catches: the model loads, the render looks plausible, but anything else placed from the same
in-memory height array (rocks dropped onto the surface, a spawn pose) sits at the wrong ground
height. Emit the rows reversed - see `generate_terrain.py`'s `height_to_png`.

### `<geom>` has no per-instance mesh `scale` - put it on the `<mesh>` asset instead

MuJoCo's schema rejects `scale` on `<geom type="mesh">` (`Schema violation: unrecognized
attribute: 'scale'`); only the `<mesh>` asset itself takes `scale="x y z"`. To scatter many
instances of the same base mesh at different sizes (e.g. a small procedural rock library) without
duplicating geometry per instance, quantize the desired sizes into a handful of buckets and emit
one `<mesh>` asset per (variant, bucket) combo, each referencing the same STL file with a
different `scale`, then point each `<geom>`'s `mesh=` at the right bucketed asset name. See
`generate_terrain.py`'s `ROCK_SIZE_BUCKETS` / `rocks_assets_generated.xml`.

### Every fixed-mode MJCF camera needs a `<camera_name>_optical_frame` site once `render_publish_rate > 0`

`picknik_mujoco_ros/MujocoSystem` (`cameras.cpp`'s `extract_cameras()`) enumerates **every**
fixed-mode camera in the compiled MJCF model - not just ones referenced from the URDF - and
throws `The MJCF model does not define a site for the camera frame: <name>_optical_frame` at
hardware init if any of them lacks a matching `<site>`. A camera with `mode="targetbody"` (or
any non-fixed mode) is exempt. This bites a debug-only scene camera added for offline rendering
(e.g. `validate_and_render.py`) the moment `render_publish_rate` is turned on for the first time
- it will not surface from reading the xacro/URDF, only from actually running the stack. Fix: add
a `<site name="<camera>_optical_frame" pos="<camera pos>" quat="...">` at the camera's pose, with
the quat rotated 180 deg about the camera's own local X axis (MuJoCo camera convention -> ROS
optical-frame convention: flip Y and Z, keep X) - see `scene_camera_optical_frame` in
`src/lunar_sim/description/husky_scene.xml` or `src/factory_sim/description/scene.xml` for the
established pattern.

### MuJoCo documentation

Refer to [docs.picknik.ai](https://docs.picknik.ai) for MuJoCo configuration guides:
Expand Down
2 changes: 0 additions & 2 deletions CLAUDE.md

This file was deleted.

1 change: 1 addition & 0 deletions CLAUDE.md
1 change: 0 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@ git submodule update --init src/moveit_pro_sam2 src/moveit_pro_sam3
- `kitchen_sim`
- `lab_sim`
- `lunar_sim`
- `phoebe_sim`
- `so101_sim`
- `vla_sim`
- `moveit_pro_franka_configs/franka_base_config`
Expand Down
4 changes: 0 additions & 4 deletions bin/validate_workspace_dependencies.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,10 +63,6 @@
)
CLEARPATH_CONTROLLER_CONFIGS = {
Path("src/hangar_sim/config/control/picknik_ur.ros2_control.yaml"),
Path(
"src/external_dependencies/phoebe_ws/src/phoebe_sim/config/control/"
"dual_arm.ros2_control.yaml"
),
}
UPSTREAM_FETCH_TIMEOUT_SECONDS = 300
UPSTREAM_GIT_OPERATION_TIMEOUT_SECONDS = 300
Expand Down
1 change: 0 additions & 1 deletion colcon-defaults.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ build:
# Enable this for bidirectional syncing from the UI
symlink-install: true
allow-overriding:
- ewellix_description
- franka_description
- kortex_description
- robotiq_description
Expand Down
27 changes: 27 additions & 0 deletions src/external_dependencies/clearpath_common/UPSTREAM.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
upstream:
repository: https://github.com/clearpathrobotics/clearpath_common.git
commit: 811baf06a32747be0653ea18db2c8868860d193f
branch: jazzy
vendored_paths:
- LICENSE
- clearpath_platform_description/CMakeLists.txt
- clearpath_platform_description/CHANGELOG.rst
- clearpath_platform_description/package.xml
- clearpath_platform_description/urdf
- clearpath_platform_description/meshes
pruned_paths:
- other clearpath_common packages (clearpath_control, clearpath_desktop, clearpath_generic_base, clearpath_platform_msgs, clearpath_sensors, etc.) unused by this workspace
- test and CI configuration files
- clearpath_platform_description/launch (unreferenced anywhere in this workspace)
- clearpath_platform_description/urdf and meshes for every platform other than a300 (a200, j100, r100, w200, dd100, dd150, do100, do150)
- clearpath_platform_description/urdf/links (macro helpers only used by non-a300 platforms)
- clearpath_platform_description/urdf/generic/gazebo.urdf.xacro, empty.urdf.xacro, and drivetrain/control/* (unused drivetrain-control variants for this hardware plugin)
- clearpath_platform_description/urdf/a300 unused attachments (spotlight, top_plate, wireless_charger) and their meshes
- clearpath_platform_description/meshes/a300 unused drivetrain meshes (caster bracket/flange, unused wheel-type meshes)
modified_paths:
- clearpath_platform_description/CMakeLists.txt
notes:
- Byte-identical vendor of the clearpath_platform_description package from clearpath_common except for CMakeLists.txt (see modified_paths); not otherwise modified in this workspace.
- CMakeLists.txt: install(DIRECTORY launch meshes urdf ...) narrowed to install(DIRECTORY meshes urdf ...) since the launch/ directory was pruned (unreferenced anywhere in this workspace); a BSD-licensed file, so no PickNik change notice is required in the file itself.
- Binary files matching workspace Git LFS patterns are stored as LFS objects; checked-out bytes match upstream.
- Pruned to the A300 subset needed by lunar_sim/description/husky_a300_mujoco.xacro at upstream commit 811baf06a32747be0653ea18db2c8868860d193f (unchanged from the prior vendor), computed by running xacro over that entry point against a source-built overlay (not the base image's stale install) and tracing every file it opened plus every mesh referenced in the expanded output; see pruned_paths for what was removed.
Loading
Loading