pool: ablate to what kern-serve uses; one GPU e2e gate over the four spec families - #18
Conversation
…hat it hands out
Trimmed against the one consumer, kern-serve: the make-room loop the
scheduler and agentx_replay each carried (coldest resident → park, drop
the coldest parked until it fits, else drop) is `Prefix::evict`, which
names the entry it took (`Evicted::Parked(key)` / `Dropped { key,
parked }`); `coldest`, `park` and `remove` are private, `Hit::tier /
resident / parked`, `Prefix::resident / parked / is_empty`, `Store::pages
/ paged`, `Host::bytes / pages / page_bytes / slot_bytes`,
`Parked::slot_offset`, `Remap::is_empty`, `Waking::tokens` and
`Tray::fork` had no caller outside tests and are gone; `Pooled::objects`
is private.
The three `mod tests` that read the ledger (chunks, the pool's statuses,
the tree's edges) are gone with the `cfg(test)` accessors they needed.
What they checked is checked one level up, through the public API: the
pool property tests land every planned `Remap` into a model of the
chunks (a chunk is free or at one position, whatever a handle names is
mapped whole, the counters say what the plans said), the prefix model
learns what `evict` took from the key it names, and `runs` is an
integration test. kern-pool: 40 tests, no GPU.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G6hPjvcgRZcdFcymLwFchB
Signed-off-by: JinYan Su <751080330@qq.com>
The e2e driver compares a served answer with kern run's on the same tokens; a prompt given as text tokenizes the same on both sides only until an answer's text is fed back (it need not tokenize to its ids). Given ids, the oracle runs exactly what the server ran. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G6hPjvcgRZcdFcymLwFchB Signed-off-by: JinYan Su <751080330@qq.com>
A speculative round accepts tokens the client never gets: past max_tokens, or past the stop token. With a recurrent state they are in the sequence's state, and the request-end snapshot keyed by them is a prefix no next turn sends; it held a slot until evicted and never hit (e2e turn2 on DSv4.1: 2 of 4 misses, dflash2: 1 of 4). Such a state now returns to the pool at once, with a debug line; a state ending exactly at the stop token is kept, a chat's next turn ends the answer with it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G6hPjvcgRZcdFcymLwFchB Signed-off-by: JinYan Su <751080330@qq.com>
The scheduler thread unwinding left the port open and every request hanging until the client's timeout; the e2e driver sat 15 minutes on one. The process exits 101 instead, the panic message already printed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G6hPjvcgRZcdFcymLwFchB Signed-off-by: JinYan Su <751080330@qq.com>
One driver, one kern.toml, every target through the same scenarios over the binaries a user has: kern test, conc1 against kern run, the hits the checkpoints allow (repeat, turn2 as ids), warm against kern run and against a cold server, twelve at once, a client hanging up mid-stream, a small pool with a host tier filled until the server parks, a stateful manifest growing its slots from two, a speculative manifest at --rows 1. Byte identity gates; a divergence on a single-rank target is excused only by the logits of that very step (kern run --prompt-ids over the shared context, both tokens within 4 ULPs of the top), and a tray target reports what crossed numerical paths instead of gating it. Gate on 2026-09-14 (tray06 / tray07 / tray09): qwen3-4b, its dspark draft, qwen3.8-27b, its dflash2 draft, DeepSeek-V4.1-Flash EP4 and Kimi-K3 4 layers EP4 all pass; numbers in docs/serve.md. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G6hPjvcgRZcdFcymLwFchB Signed-off-by: JinYan Su <751080330@qq.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G6hPjvcgRZcdFcymLwFchB Signed-off-by: JinYan Su <751080330@qq.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 717169b7c0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if a.targets: | ||
| targets = [t for t in targets if t.name in a.targets] |
There was a problem hiding this comment.
When --targets contains a misspelled or renamed target, this filter silently drops it; if none of the names match, reports remains empty and the script writes an empty summary and exits successfully. A CI invocation intended to validate specific models can therefore pass without running any checks, so validate every requested name against the configured targets and fail on unknown names.
Useful? React with 👍 / 👎.
| if self.park(&key, &mut park)? { | ||
| return Ok(Some(Evicted::Parked(key))); |
There was a problem hiding this comment.
When the host is full, this loop can remove one or more parked entries and then successfully park the resident entry, but the Evicted::Parked result discards dropped. Both KernScheduler::make_room and agentx_replay::make_room only add to host_evictions from the Dropped variant, so this common successful-retry path underreports host evictions and corrupts the new gate's reported counters.
Useful? React with 👍 / 👎.
| let visible = s.prompt_len + s.emitted; | ||
| let past = s.history.get(visible..).unwrap_or_default(); | ||
| if s.pos > visible + 1 || past.iter().any(|&t| !self.policy.stop_tokens.contains(&(t as u32))) { |
There was a problem hiding this comment.
For a speculative request that reaches max_tokens before a later accepted draft, this condition treats one unseen draft as valid whenever its ID happens to be a configured stop token, even though emit finished the request with Length (and likewise when EOS handling is ignored). The resulting state is indexed with a token the client never received and cannot include in its next prompt, leaving a useless recurrent-state slot occupied until eviction; the exception needs to depend on the actual finish reason rather than only the token value.
Useful? React with 👍 / 👎.
| cmd = self.command(context) + ["--steps", "2", "--probe-dir", str(d), "--probe-steps", "2", "--probe-labels", ""] | ||
| if rows > 1: | ||
| cmd += ["--rows", "1"] |
There was a problem hiding this comment.
When session_rows1 encounters a divergence, it calls this method with rows == 1, so the probe omits --rows 1; on the speculative targets for which that session runs, kern run then defaults to the widest declared row shape (crates/kern-run/src/run.rs:321). The near-tie verdict is consequently based on a different speculative path rather than the plain path being compared, which can incorrectly excuse or reject a rows-1 regression.
Useful? React with 👍 / 👎.
| key = json.dumps([prompt, steps, rows]) | ||
| if key not in self.cache: |
There was a problem hiding this comment.
The persistent oracle.json key contains only the prompt, step count, and row count. Reusing an output directory after changing the manifest, kernels, weights, or kern binary therefore skips the current oracle run and compares the new server against an answer produced by an older artifact set, making the gate result invalid and commonly producing false failures. Include artifact/binary identity in the cache key or clear the cache for each invocation.
Useful? React with 👍 / 👎.
1 participant
Stacked on #17 (
pool-radix); base moves tomasteronce that merges.Ablation, judged from kern-serve
kern-poolkeeps what its one consumer uses.Prefixmakes room itself (evict(park)→Evicted::Parked/Dropped),coldest/park/removeare no longer public,Storeexposestokens/has_slotonly,Hostis built from bytes,Waking::tokensandTray::forkare gone. The crate has nomod tests; its accounting is tested through what it hands out (tests/pool.rsreferenceMappingmodel landingRemapplans,tests/prefix.rsbrute-force model learning tiers fromEvicted,tests/host.rs).One GPU e2e over the four spec families
tools/e2e/e2e.pydrives every target of a kern.toml through kern-serve the way a client does (README in the dir):kern test, conc1 vskern run, repeat / turn2 hits, warm vskern runand vs a cold server, twelve at once, a client hanging up mid-stream, a small pool with a host tier filled until the server parks, slot growth from two slots,--rows 1. Byte identity gates; a divergence on a single-rank target is excused only by the logits of that step (kern run --prompt-ids <shared context> --rows 1 --probe-dir, both tokens within 4 ULPs of the top); tray targets report cross-path identity instead of gating it.Gate 2026-09-14 (tray06 / tray07 / tray09, 4×GB300 each): qwen3-4b, qwen3-4b-dspark, qwen3.8-27b, qwen3.8-27b-dflash2, DeepSeek-V4.1-Flash EP4, Kimi-K3 4-layer EP4 — all pass. Table in
docs/serve.md"e2e 门禁".Writing the scenarios found three interface problems, fixed here:
kern run --prompt-ids: an answer's text need not tokenize back to its ids, so turn2 is sent as ids and the oracle replays them.max_tokensor the stop left a request-end snapshot keyed by tokens no next turn sends (DSv4.1 2/4, dflash2 1/4 turn2 misses, a slot held until evicted). Not kept any more (not keptdebug line); a state ending exactly at the stop token is.Open, noted in serve.md: K3 4-layer at the default budget is 131k chunks of 2 MiB (chunk = half the smallest object), 40–60 s of mapping per rank and one
cuMemSetAccessOOM — the chunk should scale with the budget.Checks
host:
cargo test --release,cargo clippy --all-targets -D warnings, fmt, hawk; kernel-lab:cargo clippy -p kern-serve,cargo test -p kern-serve; CI's manifest-name grep.🤖 Generated with Claude Code
https://claude.ai/code/session_01G6hPjvcgRZcdFcymLwFchB