Skip to content

fix(playback): recognize VR180 tile, fisheye and right-eye-first metadata (#172) - #222

Merged
BGLuis merged 2 commits into
developfrom
fix/172-vr180-tile-fisheye-metadata
Oct 26, 2026
Merged

BGLuis merged 2 commits into
developfrom
fix/172-vr180-tile-fisheye-metadata

Conversation

@BGLuis

@BGLuis BGLuis commented Oct 26, 2026

Copy link
Copy Markdown
Owner

Description

Resolves #172 by recognizing container projection and stereoscopic metadata according to FFmpeg specifications:

  • Recognizes AV_SPHERICAL_EQUIRECTANGULAR_TILE with 0.32 fixed-point boundaries, correctly mapping tiles whose horizontal bounds cover ~50% of the sphere (Google VR180 files) to Vr180Mono or Vr180Sbs, and unbounded tiles to full 360° Equirectangular.
  • Maps AV_SPHERICAL_FISHEYE to Fisheye190.
  • Maps AV_STEREO3D_FLAG_INVERT with Side-by-Side to SideBySideRight and Top-Bottom to TopBottomRight.
  • Extracts initial projection orientation (yaw, pitch, roll) from AVSphericalMapping and carries it across the C ABI via consume_initial_orientation, rotating the initial scene view in both Vulkan and GLES renderers.

Closes #172

Type of change

  • 🐛 Bug fix (non-breaking change which fixes an issue)
  • ✨ New feature (non-breaking change which adds functionality)
  • 🛠️ Technical refactor / Code improvement
  • 📝 Documentation update
  • 🚀 Performance optimization
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)

Affected layers

  • 🟣 Kotlin (app/) — app shell, UI panels, credentials, history, i18n
  • 🔵 C++ (native/) — OpenXR session, Vulkan/GLES render loop, controller input
  • 🦀 Rust (rust/) — demux, decode, audio, network protocols
  • 🔧 Build / CI / scripts
  • 📚 Docs only

How has this been tested?

  • Unit tests (JVM / cargo test): 58 tests in media-logic + 251 tests in workspace passed.
  • Native/Vulkan build compiled without errors (./scripts/test-native-host.sh and ./scripts/check-shaders.sh passed).
  • Physically tested on a device (Quest 3 / Quest 3s) — not tested on physical hardware in this session.

Checklist:

  • My code follows the project's style guidelines (ktlint, rustfmt, clippy -D warnings).
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added new tests that prove my fix is effective or that my feature works.
  • The CI checks (Security & Workflows, Rust, C++, Android and Markdown Lint, gated by CI Gate) are passing for this PR.

If applicable

  • Kotlin never calls Rust directly — the Kotlin → JNI → C++ → C ABI → Rust path is preserved (ADR-002).
  • I changed the screen/stereo mode enum and kept it in sync across all four places.
  • I added or changed user-facing strings and updated all supported locales.
  • Pure logic I added lives in media-logic (host-testable) rather than in core, where it could not be tested.
  • No credential is stored or logged in plain text.

…data (#172)

Parse container projection and stereo metadata according to FFmpeg specifications:
- Add VideoProjection::EquirectangularTile with fixed-point 0.32 bounds, resolving tiles with ~50% horizontal coverage to VR180 (mono/SBS) and ~100% to 360°.
- Map AV_SPHERICAL_FISHEYE to Fisheye190.
- Map AV_STEREO3D_FLAG_INVERT with SBS to SideBySideRight and with TB to TopBottomRight.
- Extract initial projection orientation (yaw, pitch, roll) from AVSphericalMapping and carry it to the native renderer via consume_initial_orientation, rotating the initial scene view.

Closes #172
@BGLuis

BGLuis commented Oct 26, 2026

Copy link
Copy Markdown
Owner Author

PR Review & Fixes Summary

All review findings from the initial inspection have been resolved in commit 05753af:

  1. Decoupled Sphere Yaw Orientation from OpenXR Stage Space:

    • In Vulkan (vr_player_app_vulkan.cpp) and GLES (vr_player_app.cpp), introduced videoSphereYawOffset specifically applied to the spherical mesh (Mat4RotationY(sceneYawOffset + (sphereMode ? videoSphereYawOffset : 0.0f))).
    • The user's calibrated head orientation in Stage space (sceneYawOffset) and the UI controls panel remain stable in front of the viewer.
    • Clarified that pitch and roll from metadata are intentionally ignored to maintain horizon stability and prevent VR motion sickness.
  2. Orientation Reset Lifecycle in Bridge:

    • In rust/bridge/src/lib.rs, apply_screen_mode_after_load and reset_3d_mode now explicitly reset orientation to 0.0f and signal pending orientation when media without spherical pose loads, preventing rotation leakage between files.
  3. Memory Safety & Alignment in Side Data Parsing:

    • In rust/core/src/format3d_detect.rs, replaced unsafe pointer reference casts with std::ptr::read_unaligned for both AVStereo3D and AVSphericalMapping, eliminating potential undefined behavior on unaligned packet side data slices.
  4. Test Coverage:

    • Added unit test resolve_container_hint_fisheye190 in rust/media-logic/src/format3d.rs covering mono and SBS formats.

Verification:

  • CI checks (Security & Workflows, Rust Quality & Tests, C++ Native & Shaders, Build Quest 3 APK, and CI Gate) passed cleanly on GitHub Actions.

@BGLuis
BGLuis merged commit 82c3333 into develop Oct 26, 2026
8 checks passed
@BGLuis
BGLuis deleted the fix/172-vr180-tile-fisheye-metadata branch October 26, 2026 18:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant