Mesh pages carry their own tables and clusters.json stays the same size whatever the world

[pasquelin]

Why

Split from #750 (about 950 lines). After #750 pages the cell index, the per-page tables still sit in one file that grows with the world.

To do

• Per-page tables in the mesh pages: each mesh page carries its own tables; clusters.json becomes a fixed-size root; the format versions bump.

Code context

#750's region tree and fixed-size root (reuse its halving and root layout), the compiler's cluster table writer, docs/FORMAT.md.

Proof

Format goldens updated with the bumps; clusters.json stays the same size whatever the world's size; validate --group native green.

Links

Parent #483. Blocked by #750. Split from #750.

[pasquelin]

To do moved by the CTO from #404 (split again when taken if over ~500 lines): (1) getObjectByName name buckets written by the compiler (format bump, caches regenerated) and read by the runtime; (2) the compiler refuses a partition that breaks an invariant (#483 rule 3).

[pasquelin]

taken by lead compiler

[pasquelin]

Lead compiler: this issue needs a split by the CTO (AGENTS.md rule 5). The coder estimated about 900 hand-written lines before writing any code. in progress is removed.

Why it is bigger than planned:

• The whole clusters.json must leave the root, not only the primitives. selectedTriangles, metrics, coplanar, cutouts, texturePreviews, unsupported and files vary with the world, so they go into a content-addressed head page. The previews become a WGMB with no primitives.
• Every reader of clusters.json/clusters.bin changes in the same pull request as the writer:
  • Rust: reuse proof, reuse, prune, compute_bench, and 26 references in 12 test files;
  • TS: manifestLoad.ts, bench, sdk/common/streaming.ts, installed-package scripts, and six test/fixture files.
• feat(compiler): the scene tables list their cells in region pages under a fixed-size root (#750) #781's pager is hard-wired to cells (scene-page-, cells, PARTITION_VERSION) in Rust and TS. It must take the page kind as a parameter, not be copied.
• Region pages naming mesh pages changes the build order: mesh pages must be written before stage_scene_tables.

Proposed split. Each step keeps develop working.

• 762a — clusters.json becomes the fixed root, read through the page tree (~540).
  • Root: fixed fields, a head slot and 8 slots in feat(compiler): the scene tables list their cells in region pages under a fixed-size root (#750) #781's layout. Head page: the top-level fields and the preview WGMB.
  • Mesh pages: slim primitive JSON plus a WGMB, named by content. In this step there is still one mesh page for every primitive.
  • feat(compiler): the scene tables list their cells in region pages under a fixed-size root (#750) #781's pager is generalised in Rust and TS, and every reader walks index pages (synthetic test).
  • Versions: MANIFEST_BINARY_VERSION 8 → 9, FORMAT_VERSION 7 → 9, blend 8 → 10. manifestLoad.ts merges through decodeManifestBinary; callers and FORMAT.md are updated.
  • Proofs: clusters.json is the same size on grid(48), grid(192) and the open-world cell; the TS round trip gives the same ClusterManifest.
• 762b — mesh pages cut under PAGE_BYTES (~200, blocked by 762a). Primitives are grouped under 128 KiB, with index pages. No format change. A primitive larger than one page is a page of its own. Proof: every mesh page is under the limit in the three worlds.
• 762c — region pages name their mesh pages (~180, blocked by 762b). Mesh pages are written before the scene tables, and leaf region pages list the mesh pages their cells place (the core's on the root region). The partition version bumps. Proof: every region page names exactly the mesh pages its cells place.
• The runtime holds only the manifest pages its view needs, and the open world loads #751 would then be blocked by 762c.

[pasquelin]

Split by the CTO: #762 keeps 762a (fixed root, head page, one mesh page, readers walking the tree, format bumps). 762b and 762c move to 792; #751 now waits on 792.

[pasquelin]

taken by lead compiler (762a, per the CTO's split)

[pasquelin]

Split again by the CTO: the generic pager (no format change) and the goldens marked generated move to 796, taken first. #762 keeps the root, head and mesh pages, every reader (cutout thumbnails, bench cacheManifest), the version bumps and FORMAT.md.

[pasquelin]

Lead compiler, state for a clean restart (the boss's order to stop, 26 Sept.). The 762a work is written and saved, not reviewed, as wip commit 9d45dd878 on the pushed branch 762-manifest-root. It covers the root, head and mesh pages, every reader (including sdk-node/src/cutout/thumb.mts and the now async bench/runner/cacheManifest.ts with its 6 callers), the version bumps (manifest binary 8 → 9, format 7 → 9, blend 8 → 10) and the goldens. Still to do:

1. Rebase it onto One region pager serves every page kind, and the format goldens count as generated #796 (the generic pager) once that merges, and trim it to 600 hand-written lines or fewer (goldens then excluded).
2. Write FORMAT.md and the other docs.
3. Check committed_colliders_hold_their_published_tolerance, which likely only fails for lack of compile:caches in the worktree.
4. Run the gates, both skills, the reviewer, then open the PR.
   in progress is removed.