Skip to content

fix: preserve camera image channels and BGR colors - #92

Open
chaiyuntian wants to merge 1 commit into
SimWorld-AI:mainfrom
chaiyuntian:fix/camera-image-decoding
Open

chaiyuntian wants to merge 1 commit into
SimWorld-AI:mainfrom
chaiyuntian:fix/camera-image-decoding

Conversation

@chaiyuntian

@chaiyuntian chaiyuntian commented Sep 11, 2026

Copy link
Copy Markdown

RGB PNG camera responses currently lose their blue channel because the decoder unconditionally removes the last channel. Palette and grayscale PNGs also fail, while BMP fast capture returns RGB even though PNG/file capture, display and video recording use BGR. This corrupts RGB observations and segmentation label colors when switching transports.

Normalize PNG inputs to three RGB channels before converting to BGR, and preserve BMP's native BGR order. Keep BMP padding, alpha removal and top-down/bottom-up handling intact.

Validation evidence

The same 12 regression cases were run before and after the fix on Windows:

Revision / environment Result
Upstream 1f588ec, Python 3.12 11 failed, 1 passed
Fixed, Python 3.12 12 passed
Fixed, Python 3.10 12 passed

Coverage includes RGB/RGBA/palette/grayscale PNGs, 24/32-bit BMPs with row padding in both storage orientations, and the public get_image path for both lit and object_mask observations. The PNG payloads are encoded by Pillow and the BMP fixtures have explicit headers and known pixel values. Only the server response is mocked; the production decoders are exercised.

Reproduce from the repository root:

python -m pip install -r tests/requirements.txt
python -m pytest tests/test_camera_image_decoding.py -q

The repository's pre-commit checks pass for changed files (flake8 was limited to one worker locally). The added CI workflow runs on Windows/Linux with Python 3.10/3.12 and uploads JUnit reports.

Remote CI passed on all four combinations for commit 54a0d9804d1c6a9329c67957154a42d1d1f3a93a: workflow run and downloadable JUnit evidence. Each job reports 12 passed.

Found while investigating #88. This PR repairs client image decoding. The runtime actor registration defect is addressed separately in #95, which includes before/after captures from the official UE package, original metric depth payloads, and moving-actor validation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant