[gareth-morgan-nv]

Summary

Adds a MOCK backend to the SENSING camera plugin so it builds and runs on any machine with an NVIDIA GPU, not only on a Jetson. The mock replaces only SiplCamera and JetsonEncoder; everything above them (SensingCamera, the sinks, the YUV-to-RGBA and RGBA-to-NV12 kernels, and the CUDA IPC publisher) is the code that runs on the rig. It also fixes the capture timestamp of encoded streams, which was always zero.

Stacked on the SENSING camera plugin PR (#998): the base branch is its head branch, so this diff shows only the changes below.

What changed

• SENSING_BACKEND (AUTO, JETSON, MOCK). AUTO picks JETSON on aarch64 and MOCK elsewhere, so a Jetson without the SDK still gets the existing setup instructions instead of a silent mock. The Jetson SDK checks are unchanged apart from sitting inside the JETSON branch.
• Mock camera. Answers --list-sensors and the link masks from the vendor JSON, including the renumbering of sensors to pipeline indices. Builds a synthetic NV12 surface from an animated GIF and runs the real conversion on it. Keeps the three-slot mailbox with its reader lease and a capture TSC shared by every sensor. SENSING_MOCK_FPS, SENSING_MOCK_FAIL_AFTER and SENSING_MOCK_SCALE_DIV control frame rate, an injected link failure and test-size frames.
• GIF source. Each sensor shows an animated GIF fixed by its module name: S56C_HEAD the robot (its two sensors are the left and right eye, offset against each other by an illustrative fixed disparity), SHF3L_LEFT the waving hand, SHF3L_RIGHT the hand flipped left to right. It plays on the capture clock, scaled to cover the sensor's configured size and centre-cropped, never stretched. A module with no picture, or a GIF that cannot be loaded, is an error; there is no fallback. mock_assets/mock_rig.json is a mock-only three-camera rig (two 2560x1984 head eyes, two 1920x1536 wrists).
• Mock encoder. Runs the real RGBA-to-NV12 kernel and device-to-host copy, then encodes with the desktop GPU's NVENC through libavcodec's h264_nvenc, set up as the Jetson encoder is: High profile, Level 5.2, VBR with a peak, an IDR every GOP with SPS/PPS in front of it, no B-frames. The output is real, decodable H.264. It keeps the real encoder's contract: a fixed pool of input buffers that makes submit() refuse frames when behind, microsecond-resolution stamps, and a concatenated tail at end of stream. Bitstream bytes, unit sizes and latency are NVENC's, not the Jetson engine's. Needs libavcodec-dev and a driver with NVENC.
• Timestamp fix. The encoder returns each unit's stamp truncated to microseconds, while SensingCamera matched it against the submitted nanosecond stamp for equality, so capture_tsc_ns was 0 in the metadata of every encoded stream. The comparison is now at microsecond resolution (own commit).
• Bundled GIFs (robot and waving hand) are Animated Noto Emoji, copyright Google, licensed CC BY 4.0, unmodified, with the licence, source and attribution in mock_assets/. LICENSES/CC-BY-4.0.txt is the canonical text and the root README notes the exception. They are stored through Git LFS. Decoding uses stb_image, a public-domain header fetched at configure time at a pinned commit and checked against its hash; only the mock backend uses it.
• Tests (sensing_mock_tests): the camera, the encoder, SensingCamera end to end, and a CUDA IPC consumer that maps the shared allocation and reads published frames. GIF scaling is covered for upscaling, downscaling and mismatched aspect ratios, at native size and at test size. The existing test_sipl_query also runs against the mock.

Validation

• sensing_mock_tests: 37 cases pass; the encoder tests decode the output with libavcodec on a laptop with an NVIDIA GPU (CUDA 12.0 toolkit).
• test_sipl_query passes against the mock.
• The camera_plugin_sensing binary run with defaults captured about 220 frames per sensor in 4 seconds and wrote a metadata file.
• clang-format-14 --dry-run --Werror is clean on every C++ and CUDA file changed.
• pre-commit over the changed files: 10 hooks pass. The REUSE hook still reports pre-existing SPDX-like string literals in scripts/check_licenses.py, src/core/codegen/generate_trackers.py and src/core/codegen/templates.py, none of which this PR touches; nothing in the files changed here is flagged.
• Forcing SENSING_BACKEND=JETSON on x86 stops at the existing "SIPL headers not found" error.
• Not run: the JETSON backend was never built or run, and nothing was exercised against SIPL, the V4L2 encoder or real cameras.

Known limitations / follow-ups

• A pass on the mock says nothing about SIPL, the ISP output format, NvSci buffer lifetime, the V4L2 encoder or real frame rates; those need an Orin with SENSING cameras.
• The mock needs an NVIDIA GPU, the CUDA toolkit and the EGL headers (libegl-dev), libavcodec-dev with a driver that has NVENC, and fetches stb_image at configure time.
• The mock encoder is NVENC, not the Jetson V4L2 engine, so bitrate behaviour and latency differ from the rig.
• The JETSON build files were restructured (wrapped in the backend switch) without being built here.

Test plan

• On an Orin with SENSING cameras: configure and build with the default JETSON backend and confirm the plugin builds and captures as before.
• On a machine with an NVIDIA GPU: cmake -B build -DBUILD_PLUGIN_SENSING=ON -DSENSING_BACKEND=MOCK, then run sensing_mock_tests and camera_plugin_sensing --list-sensors.
• Review the timestamp fix on its own: sensing_camera.cpp.
• CI.

PR description drafted with Claude Sonnet 5.5.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

• Source mode: Auto
• Repository: NVIDIA/IsaacCapture
• Branch: jiwenc/camera-cuda
• File: .coderabbit.yaml
• Commit: 8b39353
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Enterprise
• Run ID: 3bf1b655-0639-46bf-a8b9-2c9f1ff06245

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}