Repository navigation
feat(hooks): tell an unattended session that its kickoff question reaches nobody - #50
Merged
Merged
Conversation
…ches nobody rules/autonomy.md opens every piece of work with three questions, which is right whenever somebody is there to answer them. Measured on a pinned public repository at n=3, a headless run of the scaffolded harness ended every session holding those questions and wrote no code at all, while the same agent with no methodology at all shipped the feature every time. Adding the missing branch to the rule's prose changed nothing — 0 of 3 again — because it asks the agent to act on a condition it cannot observe: from inside the turn an unattended session looks like any other. SessionStart can observe it, so it says so, and the same arm then delivered 5 of 5 with the plan, the ticked box, the branch and a clean scc validate the methodology promises. The signal is undocumented and is treated that way: exactly one spelling is read and anything else, unset included, says nothing. A missed line leaves today's behaviour; a line in front of an attended session would talk somebody out of a question they were right to be asked. The rule carries the branch too, because two of the three harnesses have no hook surface to deliver it. It is paid for with a cut that leaves the file 10 bytes smaller than before, so the standing cost does not move.
… buy design/ponytail-port.md shipped its hook and rule work and skipped its measurement half, leaving the question standing: nothing measures whether the rules earn what they cost. This is that restart at a twentieth of the planned size — two tickets, one pinned target, one model, n=3, 24 headless sessions and 8 blind critics for about $17. What it found: the scaffold costs +14,703 tokens in every request, +42% on the harness's own baseline, with the scoped rules measurably absent from that figure; blind critics picked the scaffolded arm 3 of 3 on a bug fix, twice for the same reason, that it left regression tests the control deleted; and the same scaffold delivered nothing at all on new work, three times, because kickoff asks a question no unattended run can answer. Limitations first, and the section that says what is still unanswered is the longer half: fourteen of the fifteen rules have still never been told apart from their own absence, and the attended session — the one most people are in — is unmeasured.
…t took with it Four findings from the code and security reviews of this branch, all upheld. An environment variable carries no provenance, so a hostile .envrc or devcontainer.json in a clone telling an attended session that nobody is watching would spend the one checkpoint a person gets. That was testable rather than arguable: exported as 1 into the shell that launched `claude -p`, the hook still saw 0, so the harness sets the value rather than inheriting it. One direction of a symmetric mechanism is not a proof of both, so the injected text now ends by saying that a user who asks for something else is here after all and what they say wins — live text is the one input no file in the repository can forge. The docstring quoted its own measurement wrong: 6 of 6, not 5 of 5. SessionStart's one-line description in `scc hooks check` still listed only what the stage did before. And the cut that paid for the new paragraph took a reason with it — why the CI question in particular has to be asked at kickoff — which is back on the question it belongs to, one line, inside the budget at 54.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
SessionStartnow says one thing when the harness reports the session is unattended: nobody will read a question asked this turn, so takeautonomy.md's three kickoff answers asauto/no-wait/ the user's own language, record them in frontmatter as assumed rather than answered, and build. The rule gains the matching branch in prose, because two of the three harnesses have no hook surface to deliver it.Why, and the measurement behind it
design/ponytail-port.mdshipped its hook and rule work and skipped its measurement half, leaving its own §Why standing: nothing measures whether the rules earn what they cost. This is that restart at a twentieth of the planned size — two tickets ongithub.com/dustin/go-humanizepinned at4d1d908, one model, n=3, 24 headless sessions and 8 blind critics for about $17.design/rule-measurement.mdis the writeup, limitations first.What it found:
feat/branch and a cleanscc validate.CLAUDE_CODE_SESSION_ATTENDEDis undocumented, so exactly one spelling is read and everything else, unset included, is silence. A missed line leaves today's behaviour; a line in front of an attended session would talk somebody out of a question they were right to ask.The rule pays for its new paragraph with a cut and lands at 54 lines against a cap of 55, so the standing cost does not move.
Verification
scc validate --checks --pron the branch: 12 validators, 0 findings; 981 tests, coverage 86.0%.go build ./...,golangci-lint run(0 issues),golangci-lint fmt --diff(clean),go test ./...green.Reviews
Both dispatched on the diff, both returned changes-requested, and all findings are addressed in 85556cf. No blocker or critical.
scc-code-review— 1 major, 3 minor. The major was the false-positive direction: a session wrongly read as unattended, with no way for a live user to override an already-injected assumption. The minors: the docstring misquoted its own measurement (5 of 5 for 6 of 6),SessionStart's one-line description inscc hooks checkhad gone stale, and the cut that paid for the new paragraph took with it the reason the CI question in particular belongs at kickoff.scc-security-review— 1 medium. An environment variable has no provenance, so a hostile.envrcordevcontainer.jsonin a clone could tell an attended session nobody is watching and spend the one checkpointgatedexists to provide. It named the unverified link itself, and it was testable: exported as1into the shell that launchedclaude -p, the hook still saw0— the harness sets the value rather than inheriting it. One direction of a symmetric mechanism is not a proof of both, so the injected text now ends by saying the conversation wins.Found, not fixed
One blind critic picked the control on code because the methodology arm had bundled lint-driven rewrites of
bigbytes.go,bytes.goandsi_test.gointo a money-formatting change. The agents recorded the format gate correctly as the check and not the rewrite — the guidance held — but a tree-wide gate on a repository already failing it turns any task into a whole-tree reformat. It happened in 2 of 6 delivering runs and 0 of 3 control runs, which is under the resolution of n=3, so nothing was changed on the strength of it.design/rule-measurement.md§6 records the shape of the fix and what measuring it would cost.Fourteen of the fifteen rules have still never been told apart from their own absence, and the attended session — the one most people are in — is unmeasured. Both are named in the writeup rather than left implied.