Skip to content

feat: export cubemap recordings as equirectangular videos - #93

Open
chaiyuntian wants to merge 2 commits into
SimWorld-AI:mainfrom
chaiyuntian:feat/panoramic-video-export
Open

chaiyuntian wants to merge 2 commits into
SimWorld-AI:mainfrom
chaiyuntian:feat/panoramic-video-export

Conversation

@chaiyuntian

@chaiyuntian chaiyuntian commented Sep 11, 2026

Copy link
Copy Markdown

Add export of six synchronized cubemap views to a full 360-by-180-degree, 2:1 equirectangular (ERP) MP4, addressing the panoramic recording export requested in #85.

CubemapProjector reuses projection maps and save_panorama_video consumes frames incrementally instead of retaining the recording in memory. Inputs use six documented Unreal camera orientations and BGR uint8 arrays. Invalid dimensions, incompatible faces, empty recordings and writer failures are reported explicitly, and the encoder is released if capture or projection fails. This uses the existing NumPy/OpenCV dependencies.

The sensor guide includes the face convention and a runnable example that exports recorded PNG sequences. A single perspective video cannot supply the missing views. Capture must supply synchronized faces; this PR implements the export stage and does not add UE capture commands or spherical-player metadata.

Fixes #85.

Validation evidence

31 tests pass on each of Python 3.10 and 3.12 on Windows, including:

  • All six cardinal directions, poles and the panorama's wrap boundary.
  • Per-pixel comparison against an independently calculated direction-colored sphere, catching mirrored faces and incorrect pole orientation.
  • A real MP4 encode/decode round trip: 128x64 resolution, 12 fps, exactly three frames and preserved red/green/blue colors.
  • A 24-frame 512x256 synthetic ERP MP4 whose decoded frames are compared with the projected reference image.
  • Incremental generator consumption, invalid inputs, source exceptions and encoder cleanup.

Reproduce from the repository root:

python -m pip install -r tests/requirements.txt
python -m pytest tests/test_panorama.py -q --basetemp=panorama-test-output

The CI workflow tests Windows/Linux and Python 3.10/3.12, and uploads JUnit reports plus synthetic-erp-reference.png and synthetic-erp.mp4. These are explicitly synthetic projection/encoding evidence, not recordings from a running SimWorld UE server.

Remote CI passed on all four combinations for the current PR head 540bdad7078abe68a51c9bf79af95e8f112d96c2: workflow results and downloadable evidence. Each job runs all 31 tests. Download the Linux/Python 3.10 JUnit report, ERP PNG and MP4 sample.

The repository's pre-commit checks pass for changed files (flake8 was limited to one worker locally). Sphinx HTML builds successfully; rendered guide and API entries were checked. Its three existing upstream broken-reference warnings are unchanged, with no new warnings or errors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Panoramic video export

1 participant