Skip to content

P1+P2: runnable engine + artifact lock, schedules/drivers/fallback audit (#23–#26)#28

Merged
ebootheee merged 2 commits into
mainfrom
feat/runnable-engine
May 29, 2026
Merged

P1+P2: runnable engine + artifact lock, schedules/drivers/fallback audit (#23–#26)#28
ebootheee merged 2 commits into
mainfrom
feat/runnable-engine

Conversation

@ebootheee
Copy link
Copy Markdown
Owner

Closes #23, #24, #25, #26. Two commits on this branch.

P1 (#23 + #24) — reliably emit a runnable engine.js

A clean ete init on the real models never finished: the chunked emitter built the cell-level dependency graph as a full in-memory map + a second serialized String (OOM on multi-million-cell models), and engine.js was emitted after it, so the runnable engine never landed.

  • Emit engine.js before the dependency graph (it depends only on the sheet DAG + partitions); stream dependency-graph.json to disk one entry at a time (the OOM fix). Output stays deterministic; schema unchanged.
  • ete init --timeout <seconds> (default 1800; 0 disables); verify engine.js after a fresh parse (fail loud, never partial; --reuse-parse exempt).
  • chunked/build-manifest.json (Artifact must be consumable downstream without per-version reconciliation (lock layout + emit content hash + always emit engine.js) #24): locked artifact layout + a stable contentHash over the identity artifacts (engine.js, sheets/, _ground-truth.json, manifest.json) — stable across rebuilds, changes on drift. New lib/build-manifest.mjs, npm run test:runnable + CI step.

P2 (#25 + #26) — contract schedules, drivable inputs, fallback audit

Review-hardening

The #26 gate as first written throwew mid-emit and ete init's try/catch swallowed it — silently dropping the entire contract (named-outputs/inputs/cell-types) while still reporting success, a failure that would fire routinely on the real models. It now emits all maps first, then surfaces the result so it can never be swallowed (reproduced + verified end-to-end).

Tests

Full matrix green: npm test 387 (test-manifest-maps 47→57), smoke 78/78, test:runnable 20, test:depgraph 11, test:engine 21, test:slimming 13, cargo test 11/11 (0 warnings). CI runs ubuntu + windows.

Known follow-ups

  • Drivers under --reuse-parse without the workbook are skipped (work in the normal init path).
  • Schedule scalar baseCaseValue sums across years (right for flows, not balances; perYear is authoritative).

🤖 Generated with Claude Code

ebootheee and others added 2 commits May 28, 2026 21:28
… (P1 #23/#24)

A clean `ete init` on the real PE models never finished: the chunked emitter
built the cell-level dependency graph as a full in-memory map and then
serialized the whole document into a second String, OOM-killing the parser on
multi-million-cell models — and engine.js was emitted *after* that step, so the
runnable engine never landed. The fixed 10-min spawnSync cap compounded it.

Rust (chunked_emitter.rs):
- Emit engine.js BEFORE the dependency graph (it depends only on the sheet DAG
  + partitions), so the runnable engine survives a kill of the later step.
- Stream dependency-graph.json one entry at a time via BufWriter
  (write_dependency_graph) instead of building the full map + a second JSON
  String — the OOM fix. Schema unchanged (cell-dependency-edges-v1; edgeCount
  written last); output deterministic.

init.mjs:
- Configurable --timeout <seconds> (default 600->1800; 0 disables); the fixed
  10-min cap killed legitimate large-model builds mid-emit.
- Fail-loud: verify engine.js landed after a fresh parse before the manifest
  pipeline runs (--reuse-parse exempt, records the incomplete state instead).

#24 (lib/build-manifest.mjs):
- Emit chunked/build-manifest.json: locked artifact layout + a stable
  contentHash over the identity artifacts (engine.js, sheets/, _ground-truth.json,
  manifest.json). Derived contract maps are hashed for integrity but excluded
  from identity, so the hash is stable across rebuilds of the same workbook and
  changes on drift. Also the comprehensive completeness gate: a fresh build
  missing a required artifact hard-fails. --quiet now carries contentHash.

Tests: new `npm run test:runnable` (+ CI step on ubuntu+windows). Full suite
green: npm test 387, smoke 78/78, test:depgraph 11, test:engine 21,
test:slimming 13, cargo test 11.

Docs: CHANGELOG / PLAN / ROADMAP / README / HANDOFF.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… + opt-in gate

Extends the contract maps for the Mippy calibration oracle.

- Schedules/distributions (#25, Req A/B): pin per-year time-series outputs
  (distributionsToEquity, outstandingDebt, equityBase, freeCashFlow, and
  per-class classes[].distributions) into named-outputs.json as cellRange +
  perYear[], gated on manifest.timeline.columnMap. expandRange() brings them
  under the dependency closures (dependsOnNamedInputs / affectsOutputs).
- Drivable named-inputs (#25, Req C): pin exitMultiple, exitYearSelector,
  hurdleRate (source: manifest-driver). Pinned in the normal init path; skipped
  under --reuse-parse without the workbook (follow-up).
- Fallback audit (#26, Req D): scan generated sheet modules -> _fn-fallbacks.json;
  flag any named output/schedule whose dependency closure passes through an
  _fn() stub.

Review-hardening (fixes a blocker in the gate as first written):
- The gate threw mid-emit and `ete init`'s try/catch SWALLOWED it, silently
  dropping the whole contract (named-outputs/inputs/cell-types) while still
  reporting success — and on the real models (~11,813 fallbacks) it would fire
  routinely. It now emits all maps first, annotates affected outputs with
  `resolvesThroughFallback`, and records `stats.fallbackViolations`.
- `ete init` surfaces violations: warns by default, hard-fails under
  `--assert-no-fallbacks` (CI / golden-master gate). Never swallowed.

Tests: test-manifest-maps 47 -> 57 (audit reports without aborting, annotation,
clean-build-stays-clean). Full matrix green: npm test 387, smoke 78/78,
runnable 20, depgraph 11, engine 21, slimming 13, cargo 11.

Docs: CHANGELOG / PLAN / ROADMAP / HANDOFF (gate described as report + opt-in).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@ebootheee ebootheee merged commit 3ad16c0 into main May 29, 2026
2 checks passed
@ebootheee ebootheee deleted the feat/runnable-engine branch May 29, 2026 04:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parser/emitter perf: within-sheet parallelism, streaming writes, init 10-min spawn timeout

1 participant