Skip to content

Prototype: CompressedImage camera dumb relay - #11

Open
rosterloh wants to merge 1 commit into
mainfrom
camera-stream-prototype
Open

Prototype: CompressedImage camera dumb relay#11
rosterloh wants to merge 1 commit into
mainfrom
camera-stream-prototype

Conversation

@rosterloh

Copy link
Copy Markdown
Owner

What

Prototype "dumb relay" for camera streams: the agent forwards sensor_msgs/msg/CompressedImage frames without decoding them, inspired by Media over QUIC's "keep the server dumb" model. It rides the existing topic pipeline, so the router, UDS, QUIC per-topic streams, CLI echo, and the TUI all work with no changes.

talos echo /camera/image_raw/compressed prints format: "jpeg" / data: [N bytes], proving end-to-end forwarding without needing a renderer.

Changes (+91 lines, 2 code files)

  • talos-agent/src/conversions.rscompressed_image_to_dynvalue() mapping to { header, format, data: Bytes }, copying the opaque payload verbatim, plus a passthrough unit test.
  • talos-agent/src/bridge.rssensor_msgs/msg/CompressedImage subscription arm (mirrors the LaserScan/Imu arms, honors qos).
  • docs/src/configuration/agent-config.md — camera subscription example + scope note.
  • docs/src/future/roadmap.md — moq-inspired scaling path (dedicated binary frame type, per-keyframe-group streams with stale-frame dropping, codec-free agent, non-terminal viewer).
  • CHANGELOG.md[Unreleased]/Added entry.

Design decisions

  • Reused DynValue::Bytes instead of a dedicated CameraFrame variant — a new variant would change the ProtocolClient::recv_data trait signature and ripple through both transports and both clients. For compressed frames the existing reliable pipeline is fine (within the 16 MiB frame cap). The dedicated-frame + group-rotation design is documented as the scaling step, not built.
  • No TUI video rendering — the terminal can't show real video; the TUI shows frame metadata harmlessly.

Out of scope (documented in roadmap)

Raw sensor_msgs/msg/Image, in-agent transcoding, per-keyframe-group QUIC streams with stale-frame dropping, and a non-terminal viewer.

Verification

⚠️ The agent crate was not compiler-verified in the authoring environment (no built rclrs_ws/install/ ROS message crates). The CompressedImage field types (header: Header, format: String, data: Vec<u8>) were checked against the generated binding, and the code is mechanically identical to the proven LaserScan/Imu arms — type-correct by construction. Please confirm CI / local cargo test -p talos-agent is green before merge.

🤖 Generated with Claude Code

Forward sensor_msgs/msg/CompressedImage frames verbatim (header, format,
opaque data bytes) without decoding, riding the existing topic pipeline so
router, UDS, QUIC, CLI echo, and the TUI work unchanged. Inspired by Media
over QUIC's "dumb relay" model.

Documents the moq-inspired scaling path (dedicated binary frame type,
per-keyframe-group streams with stale-frame dropping, codec-free agent,
non-terminal viewer) in the roadmap as explicitly out of scope for this
prototype.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Base automatically changed from dev to main June 10, 2026 09:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant