Skip to content

A judged withdrawal is recorded with its reason in the intended-actions ledger, and its line does not reach the Ward Room - #1827

Merged
seangalliher merged 13 commits into
mainfrom
fix/withdrawal-reason-ledger-20261117
Nov 17, 2026
Merged

seangalliher merged 13 commits into
mainfrom
fix/withdrawal-reason-ledger-20261117

Conversation

@seangalliher

Copy link
Copy Markdown
Owner

Refs #1807

What changes

The intended-actions ledger can tell a judgement from a loss. Before, the single call's billet told a model that finds a decided proposal or notebook entry unsupported to answer exactly [NO_RESPONSE]. That is the same output as a decision the model forgot, so the ledger recorded both as no_response, and the live re-measure of #1673 could not tell them apart. A [WITHDRAWN] <reason> line written without the token skipped the silence gate and was posted: BF-203's regex removes the tag and keeps the text (measured on the base: the thread title was [Observation] evidence changed). Now:

  • The billet (src/probos/cognitive/action_billets.py) asks for [NO_RESPONSE] and, on the next line, [WITHDRAWN] followed by one plain sentence on why.
  • One parser, withdrawal_reason, reads the first upper-case [WITHDRAWN] anywhere in the text it is given: the reason is the rest of that line, non-printable characters become spaces, whitespace collapses, and it is cut at 200 code points (WITHDRAWAL_REASON_CHARS). It returns None without a marker and "" for a marker with no reason.
  • The ledger (src/probos/cognitive/action_accounting.py): withdrawn joins PRE_EXTRACTION, unaccounted and settle_intended_actions take a keyword-only withdrawal, the dropped row's data.withdrawal carries the text (its detail and the log line carry only withdrawn), and a withdrawal is logged at INFO.
  • The loop (src/probos/proactive.py) treats the marker like the silence token, with or without the token: both silence settles record withdrawn with the reason, a valid block beside the line is still carried out, and the line never reaches the Ward Room (an observation written beside a token-less marker is not posted either). It reads the line outside the output's proposal and notebook blocks (withdrawal_outside_blocks): a marker inside one of those blocks is that block's own text, carried out as the base carries it out. The blocks are found as the extractor finds them (a removed block is whitespace while they are found and a line break for the parser), so no marker forms across a removed block and no reason runs on past one. Review found that reading it anywhere recorded withdrawn for an action the model never withdrew, and that a forgotten action could look judged.
  • Retries need no code change: a control test pins that a retried think writes one declared row per attempt under one correlation id and settles once, which is how BF-934: crew improvement proposals are silently dropped: 31 of 36 intended proposals never posted, mostly because a proposal without a chat action skips the chain and the single call never sees the decision (notebooks likely affected too) #1673's reopening comment counts them.

A bare [NO_RESPONSE] still records no_response. No config field, flag, event type or schema changes; the one new key is optional in a dropped row's data.

One new test file, tests/test_bf976_withdrawal_ledger.py (36 cases: 21 that fail on the base at their own assert, 2 controls, and the thirteen of one function that pins what a marker inside, beside or across a block does, eight of which pass on the base because they pin what the base does), uses a real think cycle, EventLog and Ward Room, and no wall-clock wait. The rest of the diff is the contract (prompts/withdrawal-reason-ledger.md), DECISIONS.md, PROGRESS.md, a dated note under D2 of prompts/bf-934-dropped-crew-proposals.md (its mutation row 46 and L15 stay as that contract's record), and the regenerated ledger pair.

Commits

Commit Subject
fbebfc11 Withdrawal reason ledger (#1807): contract for a judged withdrawal recorded apart from a lost action, with its reason
ae6e457a Withdrawal ledger tests: a judged withdrawal is recorded with its reason and its line is not posted (expected failures until the build)
0489277a A judged withdrawal is recorded with its reason, and its line does not reach the Ward Room
9fba38a3 Trackers for the withdrawal reason in the intended-actions ledger
686acd43 Withdrawal reason ledger R1: the campaign of record and three corrections to the contract's text (#1807)
fb3097c4 Withdrawal reason ledger R2: a marker inside a proposal or notebook block is that block's own text (#1807)
32095ad2 Withdrawal reason ledger R3: the campaign of record for 20 rows replaces the one for 16 (#1807)
705d04d3 Withdrawal reason ledger R4: the case for a marker inside a block is not a built case, and the contract says what it pins on the base (#1807)
16894e46 Withdrawal reason ledger R5: the campaign of record on the head of R4 replaces the one on the head of R2 (#1807)
afda64af Withdrawal reason ledger R6: L7 says what runs when a marker in another kind of block withdraws a turn (#1807)
bcd90e2c Withdrawal reason ledger R7: a removed block leaves a line break, so no marker forms across it (#1807)
df36ded0 Withdrawal reason ledger R8: a removed block is a form feed while the blocks are found, so no strict end tag is made or lost (#1807)
f1d6ee3a Withdrawal reason ledger R9: the campaign of record for 25 rows replaces the one for 20 (#1807)

Review and the broad gate

Three review rounds and two withdrawn attempts. Every review was by the Diff Reviewer on gpt-6.1-sol, a different model family from the author, and the model of each was read back from the session log's usage records, not taken from the request.

  • Round 1: not ready, one finding. F1 (P2, ordinary reach, not a regression): the loop read a [WITHDRAWN] marker anywhere in the output, including inside a proposal or notebook block that it then carries out, so a marker that was that block's own text recorded withdrawn for an action the model never withdrew. The orchestrator reproduced it with a probe whose controls held. The Architect chose the repair (read the line outside those blocks) and rejected stripping every marker line first, which would have refused a proposal whose title holds the marker. Repaired in R2 to R5, with a re-review of the delta only.
  • Two attempts at round 2 were withdrawn by the orchestrator before they completed, and are not counted as rounds. The first because self-review found that the new case b9 had been marked built although five of its seven cases pass on the base (measured: 23 failed and 7 passed for the file on the base source). The second because the orchestrator's own structural probe showed that a sentence of limit L7 was wrong: with a valid block present the extractor runs over the whole output, as it does for the token. R4 and R6 correct both.
  • Round 2: not ready, one finding. F2 (P2, ordinary reach, a regression of the round-1 repair): removing a block joined its neighbours, so [WITH, a proposal block and DRAWN] why read as a marker the model never wrote, and a reason could run on past a removed block. Reproduced by the orchestrator with controls. Repaired in R7 (each removed block leaves a line break for the parser).
  • The orchestrator's own check before round 3. It compared R7's helper with the extractor's own stripping over every sequence of up to six pieces of the block vocabulary and found that the line break itself made a strict proposal end tag that the extractor's joined text lacks, or ended a block early (944 sequences). R8 finds the blocks with a form feed, which is whitespace and not a line break, and only then makes line breaks for the parser; a form feed in the reply is read as a space first. Three cases join b9. R9 is the campaign of record on R8's head.
  • Round 3: ready, no findings. Every property was verified by execution. An independent reference of the extractor's stripping over 1,948,716 combinations and 25,000 more samples differed only in the documented L7 class (a proposal tag completed only by removing a notebook block). No reply without a literal marker withdrew (1,111,110 enumerated). Probes through the real think cycle, Ward Room and EventLog matched. The cost is not of a different order. b9 fails on the base in five cases, on R6 in three, on R7 in two and on the head in none. R17 to R25 and five more mutation rows each failed exactly their named cases, and 391 passed in the ten-file matrix. It did not run the broad gate, a live vessel or the full 1 MiB worst case of an unfinished block.

Limits the author recorded instead of repairing, by decision of the Architect: L1 (BF-204 can replace a withdrawal reason that cites two identifiers with the bare token) as an evidence comment on #1639, and L2 and L3 (the chain emits the bare token, and _actions_lost ignores the original's silence) as one item on the consolidated follow-up #1820 and an evidence comment on #1673, all filed after this pull request merges. L4 to L9 are by design and are named in the contract.

The broad gate. One run of the canonical wrapper on the frozen head f1d6ee3a75072abd1066bee0e60dbdf8ae6d8180 (tree f2e98bcac61e7e9e0c94277e57be3d38280dd3cd), after the last review round and with no change since: scripts/run_test_gate.py --label withdrawal-reason-ledger --receipt logs/gates/withdrawal-reason-ledger.receipt.json, 3 workers, on a host with nothing else running. Result: 54147 passed, 184 skipped, 247 warnings, 7 subtests passed in 1:41:22, exit 0. The receipt validates (54,331 tests collected and reported, 0 failures, 0 errors, 184 skipped) and binds the committed tree; the live-data manifest taken before and after the run is identical (54,107 files, none added, removed or changed). origin/main was 81f646d0 at the start of the run and at the push.

Evidence

  • The new file on the base source: 26 failed and 10 passed. The 21 built cases and five of b9's thirteen fail at their own assert; the two controls and b9's other eight pass, because they pin what the base does with a marker inside or across a block.
  • Focused checks on the head (one process each, -n 0 -p no:randomly): the new file 36 passed; the nine other files of the contract's section 8.1 at their base counts; 391 passed in all. The architecture check (srp-size=92, unchanged) and the config-facade check pass, the ledger test 33 passed and the ledger generator reports it current, and scripts/run_test_gate.py --preflight-only exits 0.
  • Mutation campaign of record on the head of R8 (the runner's own output; the bullet is in the DECISIONS.md entry): 25 of 25 rows killed and none survived, 23 by exactly the cases the contract names and the two "at least" rows (R2 and R12) by supersets. The contract's table of rows is checked mechanically against the runner's rows: files, originals, mutants and killers agree.
  • Differential check of the helper against the extractor's own stripping: over all 5,380,839 sequences of up to seven pieces of the block vocabulary (proposal tags, end tags, a notebook block, a marker, plain text), the helper agrees except in 1,374 sequences where the removal of a notebook block completes a proposal tag or end tag (recorded as L7). The earlier line-break design differed in 944 other sequences of up to six pieces, which is why a removed block is a form feed while the blocks are found.
  • The think cycle end to end (real Ward Room, EventLog and notebook file, with each probe asserting its own controls) on the base, on the line-break design and on the head: a marker split by a block, a reason that runs on past a block, and a marker beside, inside or across a block settle as the contract's tables say.

What this pull request does not do

  • It does not teach the line to COMPOSE, REFLECT, BF-206 or the _actions_lost path: where the chain answers, or where BF-204 replaces a reason that cites two identifiers the call was not shown, a judged withdrawal still records as no_response, which over-counts loss and never hides it (the contract's limits L1 to L3).
  • It does not read a lower-case [withdrawn], and it does not post an observation written beside a marker (L5, L6).
  • It does not change how a marker inside a proposal that the extractor's pattern does not match, or inside any other kind of block, is read: there it still withdraws the turn, as the token does (L7).
  • It does not change the writes of a retried think (L9).

seangalliher and others added 13 commits November 17, 2026 09:13
…corded apart from a lost action, with its reason

Refs #1807.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…son and its line is not posted (expected failures until the build)

Adds 21 cases that fail on the base at their own assert, marked strict expected failures until the build, and two controls that pass on the base: a bare silence after a proposal billet still records no_response, and a retried think writes one declared row per attempt under one correlation id and settles once.

Refs #1807.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…t reach the Ward Room

The billet asks for [NO_RESPONSE] and a [WITHDRAWN] line with one sentence on why. One parser reads the first such line and cuts its reason to 200 printable code points. The proactive loop treats the line like the silence token: the observation is not posted, a valid block beside it is still carried out, and each intended action nothing carried out is recorded with reason withdrawn, logged at INFO, with the reason in the dropped row's data.

Refs #1807.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
PROGRESS.md and DECISIONS.md entries, and a dated note in the earlier ledger contract.

Refs #1807.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ions to the contract's text (#1807)

The campaign of record ran the 16 rows on the M3 head: every mutant was
killed and none survived. Fourteen rows failed exactly the cases the
contract names; rows R2 and R12, which the contract marks "at least", also
failed the controls, as the bullet in DECISIONS.md explains. The contract's
D4 now says which output reaches the raw_intent_json settle (measured by a
probe: a silenced output that carries a valid block and opens with a raw
intents object is settled there, with the line or the token alike), and two
of its section 10 search lists now name every line the command prints. Its
other cites and its four absence searches were run again on the base of this
branch and hold. This commit changes the contract's text, DECISIONS.md and
the two regenerated ledger files only; no source or test line changes.

Refs #1807.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…lock is that block's own text (#1807)

Review round 1 found that the proactive loop read a [WITHDRAWN] marker anywhere in the output, including inside a proposal or notebook block that it then carried out. A marker the model wrote as part of its own block recorded the reason withdrawn, with the block's text as the reason, for an action the model never withdrew, and it withdrew the observation beside the block.

The loop now reads the line from the output without its notebook blocks and its strict proposal blocks, the patterns and the order the extractor strips with. A marker inside one of them is that block's own text and the block is carried out as before. A marker in a proposal that the pattern does not match, or in any other kind of block, still withdraws the turn. The parser is unchanged.

One test function adds seven cases. The contract gains decision D8 and the pairs of this change, and its L3, L7 and L8 and its tables are corrected. The progress and decision entries and the ledger pair follow.

Refs #1807.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ces the one for 16 (#1807)

The runner ran the 20 rows of the contract's section 8.3 on the head of R2, one at a time: every row was killed and none survived, 18 by exactly the cases the contract names and the two "at least" rows (R2 and R12) by supersets. The bullet in the decisions entry is written from the runner's own output and replaces the bullet for 16 rows. The ledger pair is regenerated. No source, test or contract line changes.

Refs #1807.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…not a built case, and the contract says what it pins on the base (#1807)

On the base, the test file of R2 gives 23 failures and 7 passes: five of the seven parameters of the new case pass there, because they pin what the base does with a marker inside a block. The marker built, which says that a case fails on the base at its own assert, is removed from that case, and its docstring and the module's say which of its cases pass on the base and on the tree of M2.

The contract's header, D1, section 4 and 8.3 say the same, and section 6.5 holds the pairs. No source line changes.

Refs #1807.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… replaces the one on the head of R2 (#1807)

The test file's bytes changed in R4, so the runner ran the 20 rows of the contract's section 8.3 again, on the head of R4, one at a time: every row was killed and none survived, 18 by exactly the cases the contract names and the two "at least" rows (R2 and R12) by supersets. The bullet in the decisions entry is written from the runner's own output and replaces the bullet for the head of R2. The ledger pair is regenerated. No source, test or contract line changes.

Refs #1807.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…er kind of block withdraws a turn (#1807)

A probe of the structural cases found that the last clause of L7 said that, when such a marker withdraws the turn, only a valid proposal or notebook block is carried out. When one is present the extractor runs over the whole output, as it does for the token, and a read tag beside it is queued; when none is, nothing is carried out. L7 now says so, with the two differences from the base that the probe measured, and section 4 names R6.

The contract and the ledger pair change; no source, test or tracker line does.

Refs #1807.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…no marker forms across it (#1807)

Review round 2 found that the helper that reads the withdrawal line outside the proposal and notebook blocks joined the text around each block it removed. A reply with no [WITHDRAWN] in it, such as "[WITH", a proposal block and "DRAWN] why", then read as a withdrawal, and a reason could run on past a removed block.

Each removed block now leaves a line break. The parser is unchanged. Three cases join b9, the contract gains the pairs of section 6.6 and corrects D8, L7 and its tables, and the progress and decision entries and the ledger pair follow.

Refs #1807.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… blocks are found, so no strict end tag is made or lost (#1807)

A comparison of the helper that reads the withdrawal line outside the proposal and notebook blocks with the extractor's own stripping found two differences that R7 left. The line break that R7 put in place of a removed notebook block made a strict proposal end tag that the extractor's joined text does not have, and ended a block early that the joined text ends later, so a marker outside every block was hidden and a marker inside one was read.

While the blocks are found, each removed block is now a form feed, which is whitespace and not a line break, and only then a line break for the parser. A form feed in the reply is a space first, as the parser reads it. Three cases join b9, the contract gains the pairs of section 6.7 and corrects D8, L7 and its tables, and the progress and decision entries and the ledger pair follow.

Refs #1807.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ces the one for 20 (#1807)

R8 changed the helper and the test file, so the runner ran the 25 rows of the contract's section 8.3 on the head of R8, one at a time: every row was killed and none survived, 23 by exactly the cases the contract names and the two "at least" rows (R2 and R12) by supersets. The bullet in the decisions entry is written from the runner's own output and replaces the bullet for 20 rows. The ledger pair is regenerated. No source, test or contract line changes.

Refs #1807.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant