Skip to content

feat(hooks): tell an unattended session that its kickoff question reaches nobody - #50

Merged
protonspy merged 3 commits into
mainfrom
feat/unattended-kickoff
Sep 16, 2026
Merged

protonspy merged 3 commits into
mainfrom
feat/unattended-kickoff

Conversation

@protonspy

Copy link
Copy Markdown
Owner

What changed

SessionStart now says one thing when the harness reports the session is unattended: nobody will read a question asked this turn, so take autonomy.md's three kickoff answers as auto / no-wait / the user's own language, record them in frontmatter as assumed rather than answered, and build. The rule gains the matching branch in prose, because two of the three harnesses have no hook surface to deliver it.

Why, and the measurement behind it

design/ponytail-port.md shipped its hook and rule work and skipped its measurement half, leaving its own §Why standing: nothing measures whether the rules earn what they cost. This is that restart at a twentieth of the planned size — two tickets on github.com/dustin/go-humanize pinned at 4d1d908, one model, n=3, 24 headless sessions and 8 blind critics for about $17. design/rule-measurement.md is the writeup, limitations first.

What it found:

  • The scaffold costs +14,703 tokens in every request, +42% on the harness's own baseline, with the three scoped rules measurably absent from that figure.
  • On a bug fix, blind critics picked the scaffolded arm 3 of 3, twice for the same unprompted reason: it left regression tests the control wrote and then deleted.
  • On new work the same scaffold delivered nothing, 3 of 3. It ended every turn holding the kickoff questions and wrote no code, while the control shipped the function every time. Editing the rule's prose changed nothing — 0 of 3 again — because it asks the agent to act on a condition it cannot observe. With this line in front of it: 6 of 6, with the plan, the ticked box, a feat/ branch and a clean scc validate.
  • Regression on the small ticket: 7 of 7, diffs of 8–17 lines against the control's 8–20, five of seven under $0.26. The expensive path fires on the ticket that routes to it.

CLAUDE_CODE_SESSION_ATTENDED is undocumented, so exactly one spelling is read and everything else, unset included, is silence. A missed line leaves today's behaviour; a line in front of an attended session would talk somebody out of a question they were right to ask.

The rule pays for its new paragraph with a cut and lands at 54 lines against a cap of 55, so the standing cost does not move.

Verification

  • scc validate --checks --pr on the branch: 12 validators, 0 findings; 981 tests, coverage 86.0%.
  • go build ./..., golangci-lint run (0 issues), golangci-lint fmt --diff (clean), go test ./... green.

Reviews

Both dispatched on the diff, both returned changes-requested, and all findings are addressed in 85556cf. No blocker or critical.

  • scc-code-review — 1 major, 3 minor. The major was the false-positive direction: a session wrongly read as unattended, with no way for a live user to override an already-injected assumption. The minors: the docstring misquoted its own measurement (5 of 5 for 6 of 6), SessionStart's one-line description in scc hooks check had gone stale, and the cut that paid for the new paragraph took with it the reason the CI question in particular belongs at kickoff.
  • scc-security-review — 1 medium. An environment variable has no provenance, so a hostile .envrc or devcontainer.json in a clone could tell an attended session nobody is watching and spend the one checkpoint gated exists to provide. It named the unverified link itself, and it was testable: exported as 1 into the shell that launched claude -p, the hook still saw 0 — the harness sets the value rather than inheriting it. One direction of a symmetric mechanism is not a proof of both, so the injected text now ends by saying the conversation wins.

Found, not fixed

One blind critic picked the control on code because the methodology arm had bundled lint-driven rewrites of bigbytes.go, bytes.go and si_test.go into a money-formatting change. The agents recorded the format gate correctly as the check and not the rewrite — the guidance held — but a tree-wide gate on a repository already failing it turns any task into a whole-tree reformat. It happened in 2 of 6 delivering runs and 0 of 3 control runs, which is under the resolution of n=3, so nothing was changed on the strength of it. design/rule-measurement.md §6 records the shape of the fix and what measuring it would cost.

Fourteen of the fifteen rules have still never been told apart from their own absence, and the attended session — the one most people are in — is unmeasured. Both are named in the writeup rather than left implied.

…ches nobody

rules/autonomy.md opens every piece of work with three questions, which is right
whenever somebody is there to answer them. Measured on a pinned public repository
at n=3, a headless run of the scaffolded harness ended every session holding those
questions and wrote no code at all, while the same agent with no methodology at all
shipped the feature every time.

Adding the missing branch to the rule's prose changed nothing — 0 of 3 again —
because it asks the agent to act on a condition it cannot observe: from inside the
turn an unattended session looks like any other. SessionStart can observe it, so it
says so, and the same arm then delivered 5 of 5 with the plan, the ticked box, the
branch and a clean scc validate the methodology promises.

The signal is undocumented and is treated that way: exactly one spelling is read
and anything else, unset included, says nothing. A missed line leaves today's
behaviour; a line in front of an attended session would talk somebody out of a
question they were right to be asked.

The rule carries the branch too, because two of the three harnesses have no hook
surface to deliver it. It is paid for with a cut that leaves the file 10 bytes
smaller than before, so the standing cost does not move.
… buy

design/ponytail-port.md shipped its hook and rule work and skipped its measurement
half, leaving the question standing: nothing measures whether the rules earn what
they cost. This is that restart at a twentieth of the planned size — two tickets,
one pinned target, one model, n=3, 24 headless sessions and 8 blind critics for
about $17.

What it found: the scaffold costs +14,703 tokens in every request, +42% on the
harness's own baseline, with the scoped rules measurably absent from that figure;
blind critics picked the scaffolded arm 3 of 3 on a bug fix, twice for the same
reason, that it left regression tests the control deleted; and the same scaffold
delivered nothing at all on new work, three times, because kickoff asks a question
no unattended run can answer.

Limitations first, and the section that says what is still unanswered is the longer
half: fourteen of the fifteen rules have still never been told apart from their own
absence, and the attended session — the one most people are in — is unmeasured.
…t took with it

Four findings from the code and security reviews of this branch, all upheld.

An environment variable carries no provenance, so a hostile .envrc or
devcontainer.json in a clone telling an attended session that nobody is watching
would spend the one checkpoint a person gets. That was testable rather than
arguable: exported as 1 into the shell that launched `claude -p`, the hook still
saw 0, so the harness sets the value rather than inheriting it. One direction of a
symmetric mechanism is not a proof of both, so the injected text now ends by saying
that a user who asks for something else is here after all and what they say wins —
live text is the one input no file in the repository can forge.

The docstring quoted its own measurement wrong: 6 of 6, not 5 of 5. SessionStart's
one-line description in `scc hooks check` still listed only what the stage did
before. And the cut that paid for the new paragraph took a reason with it — why the
CI question in particular has to be asked at kickoff — which is back on the question
it belongs to, one line, inside the budget at 54.
@protonspy
protonspy merged commit 5db5f8f into main Sep 16, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant