P0 calling-layer fixes: one capped verdict everywhere, block identity, machine-readable ladder ceiling, timeout policy, finetext parity, chat formatConstrains - #233
Merged
Conversation
… identity, ladder markers, timeout policy, finetext parity Blueprint P0 (docs/maxusai/ollama-client-extraction-blueprint.md), the harness half. Every change is test-first; test_client 35, test_summarizers 70, test_rescore 24, test_verdicts 60, all green. - ONE capped definition (SPEC H5): the driver's escalation heredoc ignored done_reason (a measured stop-overshoot in cudafull1 escalated a finished cell; a synthetic length below the cap ended with no verdict) — replaced by tested capped_arms(). summarize_reps pooled capped cells into means its whole life; summarize_geometry had no capped concept; summarize_matrix carried a stale arithmetic copy; preflight check_quality counted capped arms as quality failures. All now reach was_capped and only was_capped. - Block identity + provenance (P0-2): client stamps _endpoint/_think_sent, capture_stamps() writes capture_schema/model/tag/endpoint/think_sent/ retries plus the driver's POWERMODE and COLD_START_MECH into every block from both producers. NOT CONVERGED is now machine-readable (mark_not_converged + arm_done honouring the ceiling marker) so ceiling cells stop re-climbing the ladder on every resume. - REFUSING gate (P0-3) now resolves CTX_START_THINKON like the loop it guards; raising the start rung no longer defeats it. - implied_scale (P0-4) is gated on a real-frame best fit: it fabricated a 28% frame error (0.721 / IoU 0.078) on a perfect norm-1000 mlx0330nv cell; test_rescore's baseline list amended deliberately. - finetext_probe (P0-5): error guard (one transport failure aborted whole campaigns), ft_done resume, and ft_block at full schema parity. - Timeout policy (P0-6): budget scales with num_predict at a 20 tok/s floor; a blown timeout is terminal, never retried (was 4 x 1800s). - In passing: test_verdicts' unittest.main() sat mid-file since #230, so PoisonNodeCorroboration's six tests never ran as a script (58 defined, 52 collected). Moved to the end; they run and pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…trigger the two-pass flow
GenerateHandler gates its structured-outputs machinery on
formatConstrains(), which treats JSON null and empty-string as
non-constraining; ChatHandler still tested req.Format != nil at its three
sites, and json.RawMessage("null") is non-nil. A client encoding
"no format" as an explicit null therefore ran the full two-pass flow —
pass one deferred with the format stripped, then a second generation — to
apply a format that constrains nothing (blueprint P0-7, found by the wire
review). TestChatNullFormatIsNotConstraining pins the fix: single
completion call, format passed through undeferred.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…moved to the driver, od leak fixed, timeout taxonomy corrected Max-effort review of the P0 branch (10 Opus finder angles + sweep, findings re-verified by hand) surfaced three regressions in the P0 round itself and an architectural error; all fixed test-first, every suite green. - ARCHITECTURE: arm_done is SPEC H4b verbatim again (capped ALWAYS re-runs — the ceiling-marker exception violated the normative rule and the blueprint's own preserve list, and split run/skip across disagreeing deciders). Skipping a standing NOT-CONVERGED cell is now the DRIVER's decision via ceiling_standing(path, ctx_max[, only]), which knows CTX_MAX: zero restarts and zero probe runs on a resumed ceiling cell, and raising CTX_MAX reopens it at both the cell and arm level. mark_not_converged only raises the ceiling, skips error blocks, never raises (set -eu) and writes atomically via a shared save(); capped_arms is marker-aware, ONLY_TESTS-scoped, escalates blown-budget errors, and drops the wrong-rung cap parameter (the req_num_predict alias moved into was_capped itself, deleting two disagreeing shims). The driver invokes all three as argv subcommands — shell values no longer spliced into python -c source. The CLI smoke caught main() discarding the ceiling-standing exit code; sys.exit(main()) now. - od LEAK (would have corrupted anchor metrics on every finishing norm-dialect bbox arm): the declared-order binding moved back above the implied_scale gate; the gate now suppresses the diagnostic only for norm-family best fits, keeping the total-miss case it exists for. Regression-tested with an anchored norm fixture and a synthesized total-miss response. - TIMEOUTS: only a BARE TimeoutError (read budget) is terminal — a connect timeout arrives as URLError(reason=TimeoutError) and retries again (socket.timeout IS TimeoutError on 3.10+, so the reason clause made server-restart windows terminal); the error carries the stable 'blown generation budget' phrase capped_arms escalates on, so a blown budget climbs the ladder instead of silently ending it; HTTP_TIMEOUT '0'/'' mean unset. - REPORTING: reps cells render '(k capped)' with spread suppressed and the 'identical across all N runs' claim gated; geometry renders capped in rows and both summaries, counts exclusions, survives empty min() lists, and names all-capped models; preflight quality scopes to this run's tests (shared-tag cross-profile FAIL) with the suite-less-lineage skip guard. - PROVENANCE: capture_stamps takes powermode/cold_start as parameters (env reads moved to the campaign layer); driver exports powermode as n/a off-macOS per ADR 0012 conv 1; cold_start stamps only the arm that actually ran cold, warm thereafter and for the finetext leg; blocks gain the resolved think mode (think_sent is None for auto+on); error blocks carry stamps; ft_block records the vision-budget/MTP env and shares client.metrics_block with the suite (round-2 parity); a finetext error preserves the prior block under 'prior'. - Go: the null-format test folded into the adjacent table as TestChatFormatPassthrough (one scaffold, four rows). SPEC conformance rows, README timeout notes, test_rescore docstring and the contract-matrix powermode comment updated to match. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the blueprint's P0 list (docs/maxusai/ollama-client-extraction-blueprint.md, merged in #231). Test-first throughout; all suites green: test_client 35, test_summarizers 70, test_rescore 24, test_verdicts 60, go test ./server/ ok.
Commit 1 — harness (vision-suite):
capped_arms()(tested; the cudafull1 stop-overshoot false-escalate and the synthetic-length false-converge are both pinned).summarize_reps(pooled capped cells into every mean it ever printed),summarize_geometry(no capped concept),summarize_matrix(stale arithmetic copy) and preflightcheck_quality(counted capped arms as quality failures) all now reachwas_cappedand onlywas_capped.capture_stamps()in the one request path writescapture_schema/model/tag/endpoint/think_sent/retries(+ driver-stampedpowermode/cold_start) into every block from both producers;_endpoint/_think_sentare stamped as wire truth.mark_not_converged()+arm_donehonouring the ceiling marker — ceiling cells stop re-climbing the whole ladder on every resume (raising CTX_MAX reopens them, as it must).CTX_START_THINKONlike the loop it guards.test_rescorebaseline amended deliberately with the rationale in-file.ft_doneresume,ft_blockschema parity.test_verdicts.py'sunittest.main()sat mid-file since preflight: corroborate poison_probe at the node when the meter is on #230, so the six PoisonNodeCorroboration tests never ran as a script (58 defined, 52 collected). Moved to the end; they run and pass.Commit 2 — server: ChatHandler's three
req.Format != nilsites now useformatConstrains, soformat: null/""no longer trigger the two-pass structured-outputs flow (P0-7, found by the wire review).TestChatNullFormatIsNotConstrainingpins it.Held-then-released: the driver edits were originally deferred until the campaign finished; the campaign was stopped instead (operator call), so they land here and the resumed campaign runs on the fixed escalation logic.
🤖 Generated with Claude Code