Skip to content

Carry an optional camera block through SOG meta.json, and add --camera-from - #328

Open
dfattal wants to merge 2 commits into
playcanvas:mainfrom
dfattal:camera-block
Open

dfattal wants to merge 2 commits into
playcanvas:mainfrom
dfattal:camera-block

Conversation

@dfattal

@dfattal dfattal commented Sep 23, 2026

Copy link
Copy Markdown

Implements the proposal in #319: an optional camera key in meta.json that says how the scene is meant to be opened (one camera: rest pose, pinhole intrinsics, a few viewing hints). It is advisory. Readers that don't use it ignore it, and nothing changes for files without one.

"camera": {
  "convention": "opencv",
  "rig": "camera",
  "rest": { "position": [0, 0, 0], "rotation": [0, 0, 0, 1] },
  "intrinsics": { "fx": 1194.67, "fy": 1194.67, "cx": 1024, "cy": 576, "width": 2048, "height": 1152 },
  "stereo": { "baseline_m": 0.063 },
  "focus": { "point": [0, 0, 1.68], "subject_m": 2.14, "near_m": 0.73, "far_m": 66.2 }
}

What changes

  • ChunkSourceMetadata gets an optional camera next to model, and it travels the same way model does. The SOG reader keeps meta.camera. The SOG writer writes it to bundled and unbundled output. Before this change write-sog.ts built metaObj from scratch, so any .sog → .sog re-encode dropped the block.
  • Streamed SOG: the LOD writer puts the block once at the top level of lod-meta.json (sibling of asset), and the per-unit and env meta.json files don't repeat it. The lod-meta.json reader reads it back.
  • The camera is stored raw, the same way the gaussians are, so bakeTransform moves it: -t/-r/-s apply the full transform to the rest pose and the uniform scale to the focus/stereo distances. Intrinsics and unknown keys pass through untouched. The block is deep-copied on read, so nothing is re-typed or dropped.
  • Other places that build metadata by hand now carry it too: both decimators, concatSource (keeps the first input's camera and warns if another input's differs), the DataTable island in processSourceBridged, and the CLI's mixed-layout combine.
  • New CLI option --camera-from cameras.json[:n] (per input, default n = 0) sets the block from a training camera in the INRIA-style cameras.json that 3DGS trainers write next to the PLY. That file stores position, a camera-to-world rotation given as rows, and pixel fx/fy with width/height, all in the PLY's own coordinates with OpenCV axes, so the pose is used as-is (rows → quaternion). The file has no principal point, so it is set to the image centre. If --camera-from follows the output file, the CLI rejects it.
  • Library exports: SogCamera, sogCameraFromCamerasJson, transformSogCamera, withCamera.
  • README: a short "Scene camera" section next to the antialiased/2DGS one, plus the new option.

What doesn't change

  • version stays 2, and so does every existing key. A file without a camera writes byte-identical meta.json.
  • .ply, .spz and the other outputs have nowhere to put the block, so it is dropped silently, which is how model is handled.

Tests

  • test/sog-camera.test.mjs (new, 11 tests) covers:

    • the cameras.json mapping, including rotation rows → quaternion
    • transformSogCamera
    • meta.json write/read
    • .sog → re-encode, where the camera JSON is byte-equal
    • output without a camera has no camera key
    • rotate+translate actions keep the camera on the splat it sat on and still pointing at the next one
    • the camera sits at the top level of lod-meta.json and not in the unit meta.json files
    • lod-meta.json read-back

    I also checked that the rotate test fails with the bakeTransform change reverted, and that the lod-meta test fails if the units keep the camera.

  • test/cli.test.mjs: in.ply --camera-from cameras.json:1 out/meta.json and … out/lod-meta.json both write the expected block, and --camera-from after the output is rejected. To make the CLI tests able to write SOG from source, the test bootstrap now points WebPCodec.wasmUrl at lib/webp.wasm, the same way the in-process tests do.

  • npm run lint is clean and npm run docs builds. npm test gives 858 tests, 855 pass. The 3 failures also fail on main on my machine (Apple M1 Pro): the mid and tiny-dof render goldens, and "should write unbundled HTML viewer with separate files".

Spec. The SOG spec page is in playcanvas/developer-site (docs/user-manual/gaussian-splatting/formats/sog.md, and streamed-sog.md for lod-meta.json), not in this repo. I can open a PR there with the matching paragraph once the shape here is settled.

Closes #319.

meta.json may hold a `camera` key (rest pose, pinhole intrinsics and a
few viewing hints, see playcanvas#319). Until now the SOG writer rebuilt meta.json
from scratch, so any re-encode dropped it.

The block now lives on ChunkSourceMetadata next to `model`: the SOG and
streamed SOG readers keep it, the SOG writer writes it, and the LOD
writer writes it once at the top level of lod-meta.json (the per-unit
meta.json files don't repeat it). Like the gaussians it is stored raw,
so bakeTransform moves the rest pose with the scene and scales its
distances; other keys in the block pass through untouched.
`--camera-from cameras.json[:n]` records training camera n (default 0)
of the cameras.json that 3DGS trainers write next to the PLY as that
input's camera: position and camera-to-world rotation as the rest pose,
fx/fy/width/height as the intrinsics (principal point at the image
centre). The poses are already in the PLY's coordinates with OpenCV
axes, so they are used as-is.

Also documents the block in the README.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SOG meta.json: optional camera block (rest pose + pixel intrinsics) so a .sog is self-describing

1 participant