Streaming without holes 11/12: a large world loads its object table by distance, without crashing the tab

[pasquelin]

Why

A large compiled world reads its whole object table before its first frame, and the tab crashes.

• Seen: the open world with 2 regions (90 k placed nodes, 7 031 primitives) loads in 9.8 s to a 767 MB JS heap, then the tab crashes on the first frame.
• Fetched and parsed before the first frame: scene-tables.json 72 MB (≈ 800 B per node), source.gltf 24 MB, scene.gltf 20 MB, clusters.json 9.6 MB. The full world holds ≈ 300 k nodes.

To do

• A world partition: object descriptors stored per spatial cell and loaded by distance to the camera.
• A merged, simplified proxy (HLOD) per cell for what is far. Moved to Streaming without holes 10/12: world-scale roots, pinned memory independent of the world size #23 (world-scale roots), 24 Sept., before the merge of Streaming without holes 11/12: world partition, object descriptors in cells read by distance (#404) #555.
• The page keeps loading one world; the engine cuts it.

Files

• Reproduction: site/examples/an-open-world.html, scripts/docs-open-world.ts (branch 332-open-world).

Proof of done

• A lightweight reproduction scene of the repository (comparable node/table shape, seconds to load) opens and draws its first frame without crashing the tab; JS heap and bytes fetched before the first frame published, before and after.
• Proven on one scene never tuned on.
• Verified against the open world in pasquelin/Trillion3D-openworld, once the engine there is pinned on the fixing commit.

Links

• Found by the open world (Open world: a 50 km procedural world to walk, drive and fly — the portal's reference example #332), moved from #358 (defect 2).
• Related: Streaming without holes 10/12: world-scale roots, pinned memory independent of the world size #23 (HLOD inside the DAG for distant instances).
• Parent Streaming without holes: rules and objectives for geometry, memory and shadows #483 (rule 6: bounded by the view, not by the world).

Moved out (CTO, 26 Sept.)

This issue keeps the audit remainders delivered by branch 404-audit-remainders: the dead code in the arrival queue, one frame budget, cellReach following the zoom, the inCellFrame shear, and a scale-down growing rows in place on engines with growPlacements. Moved:

• the paged manifest (Streaming without holes: rules and objectives for geometry, memory and shadows #483 rule 6): The compiler writes the manifest in pages a reader fetches by region #750 (compiler), then The runtime holds only the manifest pages its view needs, and the open world loads #751 (runtime, and the open-world tab crash);
• the scale-down on WebGPU (no growPlacements): Content changes: forgotten-page cleanup and final no-reopen integration proof #572;
• the getObjectByName name buckets, and the compiler refusing a partition that breaks an invariant (rule 3): Mesh pages carry their own tables and clusters.json stays the same size whatever the world #762;
• the runtime carried items (rows in the one memory budget, bare-explorer rows sized for the world, animated-parent placements in core, cells starving the page drains) and the rule 8 and 9 proofs: The runtime holds only the manifest pages its view needs, and the open world loads #751;
• the texture tiles' own per-frame ms ceiling: Texture levels survive a device loss and yield to the pages a frame keeps #745.

[pasquelin]

taken by lead geometry

[pasquelin]

taken by lead bug / audit ko

[pasquelin]

Released by lead bug / audit ko: taken by mistake, the geometry lead holds it. No change made.