Skip to content

FEAT: Add viser-based remote viewer#8

Open
zoe1230 wants to merge 2 commits intofacebookresearch:mainfrom
zoe1230:main
Open

FEAT: Add viser-based remote viewer#8
zoe1230 wants to merge 2 commits intofacebookresearch:mainfrom
zoe1230:main

Conversation

@zoe1230
Copy link
Copy Markdown

@zoe1230 zoe1230 commented Apr 20, 2026

This PR adds a browser-based remote viewer backend for Boxer, enabling interactive Prompt/Fusion/Tracker visualization over HTTP for SSH and headless workflows.

Changes

Viser backend integration:

  • Added utils/viser_viewer.py with unified remote viewer implementations for prompt, fusion, and tracker modes
  • Added shared utilities for OBB line generation, RGB loading, 3D-to-2D RGB overlay projection, trajectory rendering, camera pose lookup, and SDP loading
  • Improved prompt-viewer responsiveness in web mode by forcing immediate websocket flush after detect/clear updates so new results appear without manual browser refresh

CLI integration for existing viewer entry points:

  • Updated view_prompt.py to support remote backend selection and network binding options
  • Updated view_fusion.py to support remote fusion controls, including seq-context loading, RGB panel toggles, SDP controls, and seek debounce
  • Updated view_tracker.py to support remote tracker controls, including freeze mode, parameter apply delay, seek debounce, SDP toggle, and RGB 3D overlay toggle

Documentation updates:

  • Updated README.md with optional viser installation and remote usage examples for tracker, fusion, and prompt viewers

How to Test

Note: The setup was tested on Linux with remote browser access.

  1. Install optional remote viewer dependency
    uv pip install 'viser>=0.2.23'

  2. Generate sample outputs first
    python run_boxer.py --input nym10_gen1 --max_n=90 --track --skip_viz

  3. Tracker viewer over HTTP
    python view_tracker.py --input nym10_gen1 --viewer_backend viser --host 0.0.0.0 --port 8080

  4. Fusion viewer over HTTP
    python view_fusion.py --input nym10_gen1 --viewer_backend viser --host 0.0.0.0 --port 8080 --viser_show_rgb

  5. Prompt viewer over HTTP
    python view_prompt.py --input nym10_gen1 --viewer_backend viser --host 0.0.0.0 --port 8080

  6. Open http://:8080 and verify:

    • timeline playback and frame seeking are stable
    • 3D OBBs render and update correctly
    • RGB panel and 3D overlays work when enabled
    • prompt detect/clear updates appear immediately without browser refresh
    • SDP visibility and point-size controls take effect

Notes

  • Existing local viewer behavior is preserved
  • Remote mode is opt-in through the viewer backend flag

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Apr 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant