Skip to content

W5b-5: Engine object (C11/B9/B12) — single-path, properties resolved at load - #55

Merged
bnsd55 merged 3 commits into
mainfrom
w5b-engine
Sep 19, 2026
Merged

bnsd55 merged 3 commits into
mainfrom
w5b-engine

Conversation

@bnsd55

@bnsd55 bnsd55 commented Sep 18, 2026

Copy link
Copy Markdown
Owner

W5b-5: Engine object (review C11/B9/B12)

load_engine returns a frozen Engine dataclass carrying model, tokenizer, and every per-model property resolved once at load. Strict single-path design per review — no dual (model, tokenizer) path, no optional-profile path.

What ships

  • Engine (frozen dataclass): model, tokenizer, model_id (resolved), revision, profile, vocab_size, weight_bytes, cache_capabilities, width_slope. All computed in load_engine; nothing mutates after load.
  • Identity: load_engine("quality") and load_engine(full-id) share one cached Engine (lru on the resolved id, unchanged).
  • Profile resolved exactly once: _resolve_profile is called from load_engine only; _prefill / _get_or_compute_prior take a required profile parameter fed from engine.profile. Engine.resolve_profile does not exist (zero second path).
  • Engine-only entry points: run_parallel_generation(engine, context, schema, ...), run_parallel_generation_batched(engine, contexts, ...), run_naive_generation(engine, ...); calibrate.collect/collect_multi, evalrun.parallel_decide_fn/naive_local_decide_fn, parity.check_scoring_parity/check_batched_parity/parity_report/write_parity_json, serve, bench, and the four benchmark scripts all take an Engine.
  • _PRIOR_CACHE key fix: mlx nn.Module models are unhashable (custom __eq__), so weakrefs can no longer live in the key tuple — the key is ids + prompt/plan hashes; the entry keeps the weakrefs and the live-ref check + finalize eviction is unchanged.
  • conftest.make_engine: the single factory tests use to build Engines (runs _probe_system_role/_profile_for exactly as load_engine does); test fakes collapsed from (model, tokenizer, ...) to (engine, ...).
  • _assemble / _selective_second_pass untouched — no overlap with PR W5-B: shared scalar finalizer, rebuilt dependency second pass, MAP + constraint fixes #45's restructure.

Verification

@bnsd55

bnsd55 commented Sep 19, 2026

Copy link
Copy Markdown
Owner Author

Rebased over main (4a656e6) and pushed — head 991c454.

load_engine returns a frozen Engine dataclass: model, tokenizer, model_id
(resolved), revision, profile, vocab_size, weight_bytes,
cache_capabilities, width_slope — every per-model property resolved ONCE
at load. The prompt profile is probed in load_engine from the RESOLVED
model id (an alias and the full id share the one cached object; is-
identity).

- Generation entry points take an Engine only: run_parallel_generation /
  run_parallel_generation_batched / run_naive_generation; callers swept
  (api, calibrate, cli, evalrun, parity, serve, bench, benchmark scripts).
- _get_or_compute_prior takes the Engine itself (no second Engine
  construction path, no fake per-model properties).
- _prefill takes the engine's profile as a required parameter; the
  profile is resolved in load_engine only (_resolve_profile removed).
- width_slope: carried on the Engine; the process global and accessor are
  gone. _width_bin_max_rows takes the slope as a parameter.
- Tests build engines through the conftest.make_engine factory; the CLI
  smoke suite fakes at the load_engine seam (an Engine in, Engine out).
- New tests: slow-tier load_engine('quality') is load_engine(full-id);
  fake-tier proof that run_parallel_generation runs on the Engine's
  carried properties (vocab/weights/slope) instead of re-introspecting —
  the test calls run_parallel_generation against a model whose true shape
  disagrees with the Engine.
- Docs: ARCHITECTURE.md gains the Engine object section + updated module
  row; CHANGELOG Unreleased entry.

The prior-cache key hunk is NOT here (owned by PR #51).
@bnsd55

bnsd55 commented Sep 19, 2026

Copy link
Copy Markdown
Owner Author

Rebased onto 670909e (main, includes #57 + #58 Ledger) — head 7abae06. GitHub now reports MERGEABLE (BLOCKED only by pending checks).

  • Ticket refs removed from added lines (engine.py _prefill + load_engine docstrings, conftest make_engine) — sentences kept, codes dropped.
  • Ledger plumbing intact: engine.py imports Interval/Ledger and threads the request/per-group/per-context ledgers through the batched path unchanged; test_w5b14_ledger.py passes (14/14).
  • Full suite green except test_api_field_margins_on_real_model, which fails identically on origin/main (pre-existing KeyError 'value'; verified earlier in a clean worktree on main).

@bnsd55
bnsd55 merged commit c9701e7 into main Sep 19, 2026
2 checks passed
@bnsd55
bnsd55 deleted the w5b-engine branch September 19, 2026 08:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant