W5b-5: Engine object (C11/B9/B12) — single-path, properties resolved at load - #55
Merged
Merged
Conversation
Owner
Author
|
Rebased over main (4a656e6) and pushed — head 991c454.
|
load_engine returns a frozen Engine dataclass: model, tokenizer, model_id
(resolved), revision, profile, vocab_size, weight_bytes,
cache_capabilities, width_slope — every per-model property resolved ONCE
at load. The prompt profile is probed in load_engine from the RESOLVED
model id (an alias and the full id share the one cached object; is-
identity).
- Generation entry points take an Engine only: run_parallel_generation /
run_parallel_generation_batched / run_naive_generation; callers swept
(api, calibrate, cli, evalrun, parity, serve, bench, benchmark scripts).
- _get_or_compute_prior takes the Engine itself (no second Engine
construction path, no fake per-model properties).
- _prefill takes the engine's profile as a required parameter; the
profile is resolved in load_engine only (_resolve_profile removed).
- width_slope: carried on the Engine; the process global and accessor are
gone. _width_bin_max_rows takes the slope as a parameter.
- Tests build engines through the conftest.make_engine factory; the CLI
smoke suite fakes at the load_engine seam (an Engine in, Engine out).
- New tests: slow-tier load_engine('quality') is load_engine(full-id);
fake-tier proof that run_parallel_generation runs on the Engine's
carried properties (vocab/weights/slope) instead of re-introspecting —
the test calls run_parallel_generation against a model whose true shape
disagrees with the Engine.
- Docs: ARCHITECTURE.md gains the Engine object section + updated module
row; CHANGELOG Unreleased entry.
The prior-cache key hunk is NOT here (owned by PR #51).
Owner
Author
|
Rebased onto 670909e (main, includes #57 + #58 Ledger) — head 7abae06. GitHub now reports MERGEABLE (BLOCKED only by pending checks).
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
W5b-5: Engine object (review C11/B9/B12)
load_enginereturns a frozen Engine dataclass carrying model, tokenizer, and every per-model property resolved once at load. Strict single-path design per review — no dual (model, tokenizer) path, no optional-profile path.What ships
Engine(frozen dataclass):model,tokenizer,model_id(resolved),revision,profile,vocab_size,weight_bytes,cache_capabilities,width_slope. All computed inload_engine; nothing mutates after load.load_engine("quality")andload_engine(full-id)share one cached Engine (lru on the resolved id, unchanged)._resolve_profileis called fromload_engineonly;_prefill/_get_or_compute_priortake a requiredprofileparameter fed fromengine.profile.Engine.resolve_profiledoes not exist (zero second path).run_parallel_generation(engine, context, schema, ...),run_parallel_generation_batched(engine, contexts, ...),run_naive_generation(engine, ...);calibrate.collect/collect_multi,evalrun.parallel_decide_fn/naive_local_decide_fn,parity.check_scoring_parity/check_batched_parity/parity_report/write_parity_json,serve,bench, and the four benchmark scripts all take an Engine._PRIOR_CACHEkey fix: mlxnn.Modulemodels are unhashable (custom__eq__), so weakrefs can no longer live in the key tuple — the key is ids + prompt/plan hashes; the entry keeps the weakrefs and the live-ref check + finalize eviction is unchanged.conftest.make_engine: the single factory tests use to build Engines (runs_probe_system_role/_profile_forexactly asload_enginedoes); test fakes collapsed from(model, tokenizer, ...)to(engine, ...)._assemble/_selective_second_passuntouched — no overlap with PR W5-B: shared scalar finalizer, rebuilt dependency second pass, MAP + constraint fixes #45's restructure.Verification
slowreal-model tests: engine, smoke, w4b parity, adapters, w1a/w1c).ecbc77b(PR W5b-11: compiled constraint objects — compile_constraints as the only entry #49 merge; includes PRs W6-1: LM-head adapters module (standalone) #42–W5b-12: docs sweep — ARCHITECTURE.md + CHANGELOG.md for #42/#43/#45/#46/#47/#48 (+ leaderboard v2 follow-up) #52).