Repository navigation
Add --models: corral proposes seats from the machine ledger, and a person accepts the plan - #562
Merged
Merged
Conversation
… from the operator's pool and the recorded rankings, so a lineup can be explained and reproduced from its input. Pins win, measured seats take the measured best at or above the floor, the deriver and critic take pool order, and a critic or verifier equal to its writer or reviewer refuses the plan. A seat nothing qualifies for is left unfilled with the flag that would fill it, never filled by lowering the floor or by a model outside the pool. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…ord, because a lineup proposed by --models is otherwise indistinguishable from one the operator named by hand. SeatedBy (role -> operator, evidence or order) and SeatEvidence travel the road SeatProviders took in e30847d: RunSpec -> Verdict on both construction paths (they ride signedRoster, so tickAggregate and timeoutVerdict cannot differ, and SeatedOnly drops a seat the roster leaves off), ledger.Row, reviewrecord.Review, and the corral_audits and corral_reviews warehouse columns with their read path. Absent fields are omitempty and NULL, so an entry written without --models is byte-identical to today's, and absence reads as "every seat was named by flag". Tier is declared, not measured: it is corral's statement of how it made a choice, not something execution observed. The in-toto statements are not touched; no gate required it, and who chose a seat is on the ledger row. VerdictGeneration is 48 because the Verdict shape changed on a generation that is on main; a cached "47" document reads both fields as absent. Nothing sets the fields yet outside tests; the cmd/corral doors come in later tasks. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…s into a proposed lineup, so the pool is parsed, resolved, credential-checked, ranked and offered the same way on every verb rather than once per door. Every member is resolved and checked before any evidence is read, so a pool that works today cannot fail on the day a different member wins. The plan prints before anything is spent; a refused lineup exits 2 without asking, a terminal is asked once, and with no terminal the lineup is printed as the explicit seat flags and nothing runs, because there is nobody to accept it. Only an accepted lineup writes the chosen members, as the operator typed them, into the door's own flag variables, so the door's normal resolution runs on them unchanged. The language has no default: a door passes the scan's language or shadowpool.LangAny, and a scan of several languages ranks each seat across all of them as one group. --models is nil until given, so an empty pool is refused rather than read as absent. golang.org/x/term becomes a direct requirement for the terminal check. The printed-metrics gate gains rows for the plan's numbers and for autoseat's floor, which the walk now reaches. No door is wired yet. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…and record on each written entry how every seat was chosen, so a lineup the evidence proposed is never signed as one the operator named. Each door registers the flag through registerModelsFlags and calls seatFromPool once, after flag parsing and before the shadow draw and seat resolution, so a proposed member is drawn against and resolved exactly as the same model typed into its flag would be. - certify --local ranks by the audited file's language (drawLang) and refuses at the door, naming --lang, when none can be detected. The seating travels localAuditInput -> RunSpec -> Verdict -> ledger row. - certify --repo ranks by the scan's language (call C7): the set the evidence pass instruments, from the walk the scan already makes. The seat block moved, unchanged, below that walk rather than walking twice; the --scope-tests refusal moved with it so it still follows seat resolution. Rows carry SeatedBy narrowed to the roster, as providers are. --dry-run spends nothing and asks for no credential, so there the pool is not credential-checked and the plan is shown without a question. - doctor prints the lineup certify would propose, says it ran nothing, and checks those seats. Its one paid check, the derive probe, now runs only for a deriver the operator named, never one a pool proposed. - demo forwards the pool to the real certify --local, which proposes and records; proposing in demo and passing plain flags on would record the seats as operator-named. The production loader and terminal confirm sit behind doorPoolDeps so no door test reads the machine ledger or a terminal. Gate tests require every seat-flag door to register --models (review.go is a named pending exemption that fails once Task 5 wires it) and keep autoseat imported by models_pool.go alone. seatProvidersJSON becomes roleMapJSON, since it now serialises SeatedBy too. A run without --models writes entries shaped as before, asserted by reading them back. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…ver on the row's roster when it runs, and refuse certify --local's flag errors before --models asks anything, so the record matches the plan the operator accepted and no one accepts a lineup for a run that was always going to be refused. The --repo roster (shared by the rows and the --attest statement) now names goal-deriver with its concrete model when no --goals map replaced it, so SeatedBy["goal-deriver"] survives the existing narrowing to the roster and is attributed to a model; with --goals no deriver ran and none is named. advpool.SeatedOnly is unchanged. Every reader of the roster gains the key additively: the statement's models map, SeatTools for an agent deriver, and models rank, which already reports the deriver as "not scored". The verdict cache key does not read the roster, so cache hits are unchanged; the deriver's provider stays unrecorded because the derive seat has no role in certifySeats. certify --local's --local-endpoint parse and its --code, --goal, subject, bind-dir, seed-limit and --ledger-inside-checkout refusals now precede seatFromPool, the shadow draw and seat resolution, the order --repo keeps. TestEveryDoorDrawsFromItsPoolFlags now triggers its after-the-draw refusal with a missing --prior instead of a missing --code. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…the evidence line claimed 'this scan has more than one' for doctor without --code, and would have for a review, where it is false. seatFromPool now takes an explicit reason (anyLangReason, no default) that is required whenever the language is shadowpool.LangAny and is printed on the plan's evidence line and recorded in seat_evidence. certify --repo passes the several-languages wording, doctor passes 'no single language is known for this check', and review will pass 'a review is not ranked by language'. A door that gives no reason is refused before any plan is shown. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
… verifier, so a pool can seat the reviewer and verifier from the review evidence and the entry records how each was chosen. --models proposes the reviewer and the verifier, ranked across all languages (a review is not ranked by language, and the plan says so). Every usage refusal that needs no seat (missing --scope, an unexpected argument, the ledger signer, a non-commit checkout) now precedes the plan, so nobody is asked to accept a run that was always going to be refused; only the missing reviewer is decided after it. review had no way to say 'no verifier' that --models could respect, so --verifier-model off now means exactly that, as --critic-model off does on certify: it is a pin to --models, and is normalised to an empty verifier before any check, so a run with off writes an entry identical in shape to one with no flag and the word is never compared to the reviewer as a model. seated_by and seat_evidence are recorded beside seat_providers, for a seat only when it is on the entry, and a run without --models writes neither. The pending-doors exemption in the gate test is removed now that every seat-flag door is wired, and the CLI reference is regenerated. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…operator named on that run, because "disclosure, never selection" stopped being true when --models gave the ranking a reader that seats models. The package comment, corral models rank's help and printed line, the top-level usage, the design notes and two diagrams now say so, and keep the router incident that the rule exists to prevent; the command itself still seats nothing, and its test now pins the new sentence. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…will change it: how to name a pool, what the plan shows, what refuses and why, what the entry records, and that --autoseat and exploration are not in this release. Every statement was checked against the code on this branch, the sample plan is a test's fixture labelled as one, and no run against a real ledger is claimed because none has happened yet. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
… by the model that would run, because comparing typed spellings let one model grade its own work under a second name: a writer seated as anthropic/claude-sonnet-5-5 beside a critic pinned as claude-code:claude-sonnet-5-5 was not refused, and a writer pinned by alias was offered again as the critic. The proposer now takes resolved pins (autoseat.Pin: the typed text, the resolved name and the provider), compares resolved names in the hard rule and in "the critic is not the writer" and "the verifier is not the reviewer", prefers a different provider than a pinned writer as it did for a member, reports every broken pair in Roles order with the flags that fix it, enforces its own evidence floor instead of trusting Row.Sufficient, and says when the only member with enough evidence is already the counterpart. A pin no source serves is refused with the resolver's own words before any evidence is read. The recorded seat evidence no longer carries the operator's state path: the machine-ledger loader gives a second, path-free name built from the same counts, the plan keeps printing the directory, and a record keeps the evidence line only while a seat it names was seated by evidence (the verdict, the --repo row and the review entry alike). The plan adds one line when there is no evidence at all, the no-terminal path ends by saying nothing was audited, the prompt needs stdout to be a terminal as well as stdin, a member a placement alone serves names --local-endpoint in its refusal, and a scan with no detected language says so instead of "more than one". Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…nly errors before the question, and say plainly what doctor and a run with no terminal do, because the record counted a person reading the plan as scan time and certify --repo could still refuse a lineup the operator had already accepted. certify --local and certify --repo restart started_at, total_millis and the host-load sampler at the answer (review's started_at was already taken after it); a --repo scan's total_millis therefore no longer covers the walk that told it the scan's language. certify --repo now refuses --scope-tests and a malformed --local-endpoint before the walk, hands the placements to --models so a pin served only by its placement resolves there too, and names the zero-language case on the evidence line. The draw test's after-the-draw refusal for --repo is now a primary seat no source serves, which the seat resolution that follows the draw refuses. Doctor no longer claims to run nothing: it asks nothing and spends nothing on a seat it proposes, and a --derive-model the operator names is still probed with one real call. The --models help at each door says that with no terminal it audits nothing and exits 0, so a pipeline must pass the seat flags itself, and the top-level usage now says corral models rank reads this machine's ledger unless --db or --source names another source. The consent property gets door-level tests: a declined run and a run with no terminal write no ledger entry, open no store and make no model call at certify --local, certify --repo, demo (through its forward) and review. The gates now also fail a door that registers --models and never hands it to seatFromPool, and an evidence reader or ranking call outside the files that legitimately hold one. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…use the claims review found sentences that were false or would have become false: the recorded evidence line was said to be the printed one, the critic's provider preference was stated for a pinned writer it did not reach, the machine ledger was said to carry no critic adjudications when its rows do, and an entry without --models was said to carry no seating even where its cached verdict document does. agent-seats.md, AGENTS.md, the CHANGELOG entry and the README now say that pins are resolved and compared by the model each would run, that the prompt needs stdin and stdout to be terminals, that a run with no terminal audits nothing and exits 0 so a pipeline must pass the seat flags, that review --models refuses on a machine without five verdicts unless the verifier is pinned, that a model only a --local-endpoint placement serves cannot be a member, that the recorded SeatEvidence names the source without its directory and only beside an evidence seat, that a cached verdict keeps the seating it was measured under, that the --repo roster names the deriver whenever --goals is absent, that doctor still probes a --derive-model the operator names, and that a run's recorded time starts at the answer. The refusal text the CHANGELOG quotes is marked as an example, model-ranking.md says the proposer is the only code that decides a verdict seat from the ranking and that the shadow draw is separate, and the CHANGELOG records the corrected top-level usage. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
… host figures instead of restarting the clock at the answer, because a restarted clock made total_millis and started_at mean something different on a --models scan (a --repo scan stopped counting its walk), and two entries for one repository stopped being comparable. seatFromPool measures the interval once, from the evidence load to the operator's answer, returns it as poolSeating.PlanWait, and pauses the door's host sampler across exactly that interval. The certify doors keep started_at at the run's real start and subtract PlanWait from total_millis. The new hoststat Pause/Resume takes no load sample while paused and leaves the paused wall time and CPU out of WallMillis and CPUMillis, so LoadStart and MemAvailableStartBytes stay the readings at the run's start; a restart would have moved them to the answer. The test now checks that started_at is the real start, that exactly the plan interval is left out, and that work before the question is still counted. The documents and the ledger field comments say the time spent on the plan is excluded rather than that the clock starts at acceptance. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
… true in most cases and false in one: the critic's provider preference was said to be absent beside any operator-defined agent, the evidence-reader gate was said to cover readers it did not hold, the engine's dependency on the proposer's package was unexplained, and the no-terminal line said "audited" on corral review, which audits nothing. agent-seats.md now gives an agent seat's provider as the code computes it: the vendor of a pinned model the model-name table recognises, else a built-in agent's declared maker, and none for any other agent seat. TestOnlyAutoseatDecidesASeatFromEvidence now also holds scorecardRankEvidence, bugcatchRankEvidence and warehouseRankEvidence to models_rank.go, and AGENTS.md names exactly what the gate holds. The verdict's call to autoseat.EvidenceFor says why the engine imports the proposer's package: the rule has one definition, shared with the doors. The no-terminal line now reads "nothing was run", and the AGENTS.md, CHANGELOG and agent-seats.md sentences that quote it follow. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
Auto-assign phase 2 from the founder-approved design (2026-11-16):
--models <a,b,c>oncertify --local,certify --repo,demo,doctorandreview. The operator names a pool; corral proposes which member sits in which seat from the machine ledger's own measurements, prints the plan, and runs only when a person accepts it on a terminal.internal/autoseat.Proposeis pure and is the one place a seat is decided from evidence (TestOnlyAutoseatDecidesASeatFromEvidence). Pins always win. The measured seats (mutant-generator, test-writer, reviewer, verifier) take the best pool member with at least 5 runs in that role; with none, the seat is refused and the refusal names the flag to pin. The goal-deriver takes the first pool member by order. The critic takes the first member that is not the writer, preferring a different provider. Critic = writer and verifier = reviewer are refused.--models(TestEveryDoorWithSeatFlagsRegistersModelsFlags), through one helper,seatFromPool, which fills the door's existing seat variables before any existing resolution runs.SeatedBy(operator,evidence,order) andSeatEvidence, carried likeSeatProviders, with tier rows.reposcan.VerdictGenerationis now 48. The recorded evidence line carries no host path.run with these seats? [y/N], no--yes. With no terminal, corral prints the plan and the equivalent pinned flags, says nothing was run, and exits 0.doctorandcertify --repo --dry-runprint the plan and ask nothing, because they spend nothing.total_millis, and the host sampler is paused across it.Not in this PR (phase 3):
--autoseat, exploration,--no-explore, shadow results as evidence.Changes that reach runs without
--modelscertify --repoentry now names the goal-deriver's model on its roster whenever a deriver ran.models rankshows those rows as not scored.corral reviewaccepts--verifier-model off.Evidence
go test ./...at 4d6c48b: 105 packages ok, 0 failed; build, vet, gofmt, the four generated-doc checks, licensing andcheck-security.shpass. The one commit after that (e0b7729) changes wording only; the gates and the covering tests were re-run on it, the full suite was not.demo --models anthropic/claude-sonnet-5-5,anthropic/claude-haiku-5-5refuses the mutant-generator seat (no member has 5 runs in that role) and exits 2.--mutant-model anthropic/claude-sonnet-5-5pinned it prints a full plan (writer by evidence, critic by order), the equivalent flags, "nothing was run", and exits 0.demo --models anthropic/claude-sonnet-5-5,anthropic/claude-haiku-5-5 --mutant-model anthropic/claude-sonnet-5-5, answeredy): machine-ledger entry20261119T195844Z-nocommit-131eb36a527e, generation 48, recordsSeatedBy= mutant-generatoroperator, test-writerevidence, test-criticorder, andSeatEvidence= "the machine ledger (40 scan(s), 0 review(s)); language go" with no host path. That matches the printed plan seat for seat, andModelsByRolematches the plan's models. The run planted 11 mutants, graded 7, 5 survived, 5 proven; 25,955 input and 5,201 output tokens.corral verify --ledgerreports 41 entries, chain intact.Open decision
With no terminal,
--modelsexits 0 having run nothing, as the design says. A CI job with a gate flag (--min-kill-rate) would therefore go green without an audit. The output and docs say so plainly; whether it should exit non-zero when a gate flag is set is the founder's call.Known gaps
EvidenceForlives ininternal/autoseatand is imported by the engine;internal/seatrecordis the better home.SeatedByyet, and its values are not validated on read.doctorstops at the first refused pool member.🤖 Generated with Claude Code