feat: a blocked entity says why on disk - #124
Merged
Merged
Conversation
An observation says what a run learned about one property, and a request body says what a create looked like when it worked. Neither says why an entity produced nothing: that sentence reached only the operator's terminal, so an entity blocked by a refusal nobody was watching for left no trace to act on. The run summary is now committed at audit/summary.json, beside the observations and the request bodies the same run wrote, and is written whatever the run did — a run where every entity blocked is exactly the run whose reasons are worth keeping. Each entity result carries the redacted refusal behind a status that is not audited, so the API's own words survive the run. A minimal create that the additive search could not heal now reports the search's own last refusal rather than the earlier one that started it, and names the fields it added: the status alone does not distinguish a document that understates the create from an API refusing it for another reason. The aggregate counts stay as they were. docs/emittance_tracker.md remains the only place a count of what the toolkit emits or refuses may live. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The gap
run.Summaryalready carriesEntityResult{Entity, Status, Reason}and was never serialised.cli/audit.gowrote observations and request bodies only.The consequence on the pilot: 21 of 34 entities have zero refusal excerpts anywhere on disk, 599 blocked observations against 76 confirmed, and the reason for those 21 existed only in an audit stdout that was never redirected.
handoff.mdranks "read the reasons in the audit output" as the highest-value next step; today there is nothing to read.Two narrower losses behind it:
runCreateMinimaladopted the additive search's result only when it produced an object, so a failed search's last refusal was discarded and the excerpt attached to the block was the pre-search one.searchMinimalrecorded nothing at all on give-up — no attempt, no field, no final message.The change
audit/summary.json, committed, written beside the observations wherever--outleaves them, and written whatever the run did.EntityResultgainsRefusal *observe.Excerpt— the redacted request and response behind a status that is not audited. A blocked create now reports the search's own last refusal and names the fields it added:The status alone does not distinguish a document that understates the create from an API refusing it for another reason; that is the distinction an operator needs to act.
Two things I got wrong while planning this, corrected here
Not manifest-covered. The plan said
audit/summary.jsonshould be manifest-covered soprovider verifysees it. It should not: the pilot manifest contains zeroaudit/paths — observations and request bodies are committed but outside the generation manifest, becauseprovider generatenever writes them andverifybyte-compares what it regenerates. The summary follows that precedent.The aggregate counts stay on stdout's side of the line only in spirit — they are on the struct, and serialising it commits them. They are a fact about one run against one document, not about what the toolkit emits or refuses, so
docs/emittance_tracker.mdis untouched and remains the only place the latter may live. Called out in the glossary entry so the distinction is on the record rather than assumed.Naming
audit summarywas already approved vocabulary —rejectsUnknownFields,backoffandrequestAdjustmenteach name it. The pathaudit/summary.jsonis the new decision and is recorded indocs/glossary.md.Verification
make check(90.7% total;audit/run90.4%,cli87.6%) andgolangci-lint run(0 issues). The end-to-end assertion was confirmed to fail with the refusal not carried.No benchmark movement is expected from this PR. It is what makes the next one sighted for the 21 entities that currently record no reason at all.
🤖 Generated with Claude Code