Skip to content

fix(mr-hook): no skill hint on background-task notification turns - #71

Merged
dmmdea merged 1 commit into
mainfrom
feat/hint-skip-machine-turns
Sep 23, 2026
Merged

dmmdea merged 1 commit into
mainfrom
feat/hint-skip-machine-turns

Conversation

@dmmdea

@dmmdea dmmdea commented Sep 23, 2026

Copy link
Copy Markdown
Owner

What

mr-hook no longer injects the skill hint on background-task notification turns. Human prompts are unchanged, and so is the quota banner.

Why

Claude Code delivers background-task notifications (a finished background command or subagent, a Monitor event) through UserPromptSubmit, the same event as a typed prompt. The hook ranked them as if they were requests. Measured on one long session: the hint fired 25 times on notifications and 7 times on the operator's own prompts. Every hint stays in context and is re-read on every later call.

The shape, measured on real transcripts

  • Every notification prompt starts with <task-notification>. That holds for all 1,710 queued ones (queued_command, commandMode: "task-notification") as well as direct user turns. Three tag layouts occur: background command, finished agent (with <note>/<result>), and Monitor event (<event>, with no tool-use id).
  • The hook receives queued notifications verbatim: the usage log's prompt_hash equals the sha256 of the transcript's queued prompt, and the lengths match.
  • Other machine-originated shapes exist and are not covered here: <cross-session-message …> and <agent-message …>. Both are queued with commandMode: "prompt", there were 10 in total, and they carry actual requests.

Change

  • cmd/mr-hook/machineturn.go: isMachineTurn returns true when the prompt starts, after leading whitespace, with <task-notification>.
  • cmd/mr-hook/main.go: on a machine turn, retrieval is skipped, so no embedder call is made and the row logs mode:"machine-turn". The index load, the identity check and the quota banner (including the delegate proposal) run exactly as before. This keeps the PR feat: render the quota banner only when it carries news (semantic ChangedAt) #69 semantics: the banner shows on a changed_at within ±15m, a reset within 15m, the first prompt of a session, or the GLM latch.
  • The offload nudge is untouched: 0 of 3,274 unique real notifications can trigger it.
  • There are no new writes of any kind. The hook still writes only its usage row.
  • Consumers of the new mode: mr-outcomes skips rows with no surfaced ids, and a reader of cands gets none, the same as on a gated-empty row.
  • Docs: the README hook table plus one key-property bullet, one SECURITY.md sentence, and the mode list in usagelog.Record.

Tests (red first, then green)

Before implementation, with an inert stub, the notification tests got the gstack-qa hint and failed. The human control passed, and the GLM test passed its banner assertion and failed only on the leaked hint.

test asserts
TestIsMachineTurnShapes three real layouts, leading whitespace, a human prompt quoting a notification, a truncated tag, empty
TestHintSilentOnTaskNotificationE2E built binary, against an embedder that scores every prompt 1.0: no output, 0 embedder calls, mode:"machine-turn"
TestHintSilentOnQueuedNotificationE2E the queued agent-finished layout, with and without leading whitespace
TestHintStillShownOnHumanPromptE2E control arm, same setup: the hint surfaces and mode:"embed"
TestGLMLatchBannerOnNotificationTurnE2E established session, so the first-prompt rule cannot be the cause: banner shown, hint absent, orchestrator state dir byte-identical, no Claude-side state created

Mutation proof

The harness requires each mutant to compile (go test -run '^$') before a failing run counts, and restores the file with a hash check after each mutant.

mutant result
guard never true at the call site caught
machine branch dead caught
retrieval still runs after the machine branch caught
row mislabelled caught
isMachineTurn always false caught
no leading-whitespace trim caught
prefix widened to contains caught
tag widened to <task- caught
quota banner suppressed on machine turns caught (GLM test)
a machine turn writes orchestrator state caught (GLM test)
non-compiling control rejected, not counted

Gates

  • go test -count=1 ./...: all 53 packages ok.
  • go vet ./...: clean.
  • gofmt: clean on the LF blobs of the touched Go files, and every staged blob is LF.
  • semgrep scan --config auto --error <changed files>: exit 0, 0 findings. The new test's launcher of its own freshly built binary carries a targeted nosemgrep with its reason. The existing E2E launchers in untouched test files still trip the same rule 5 times; they are out of scope here.
  • B12 raised consciously: +28 non-test LOC (22 in machineturn.go, 6 in main.go), 30924 → 30952, with the prior note kept as PRIOR.

Deploy (after merge, not done here)

From a clean checkout of merged main: bash scripts/deploy-fleet.sh. It prints built mr-hook @ <rev> for each binary and ends with fleet is uniform at revision <rev>.

Risks

  • A human prompt that itself starts with <task-notification> gets no hint. That is unlikely, and the cost is one missing hint.
  • If a future Claude Code release changes the notification envelope, the hint quietly comes back on those turns. That fails in the direction of today's behaviour, not towards suppressing human hints.
  • Cross-session and agent-message turns still get the hint (see above).

🤖 Generated with Claude Code

Claude Code delivers background-task notifications (a finished background
command or subagent, a Monitor event) through UserPromptSubmit, the same event
as a typed prompt. The hook ranked them like requests. Measured on one long
session: the skill hint fired 25 times on notifications and 7 times on the
operator's own prompts, and every hint stays in context for the rest of the
session.

Shape, measured on real transcripts: every notification prompt starts with
<task-notification>, including all 1,710 queued ones (queued_command,
commandMode "task-notification"). The hook receives the queued text verbatim:
the usage log's prompt_hash equals the sha256 of the transcript's queued prompt.

cmd/mr-hook/machineturn.go adds isMachineTurn: the prompt starts, after leading
whitespace, with <task-notification>. On such a turn main.go skips retrieval,
so nothing reaches the embedder, and the row logs mode "machine-turn". The index
load, the identity check and the quota banner run exactly as before, so the GLM
hard-stop latch still shows on these turns. The offload nudge is left alone:
0 of 3,274 unique real notifications can trigger it. Human prompts are unchanged.

Consumers of the new mode: mr-outcomes skips rows with no surfaced ids, and a
reader of cands gets none, which is the same as a gated-empty row.

Tests, red first against an inert stub, then green:
- TestIsMachineTurnShapes: three real layouts, leading whitespace, a human
  prompt that quotes a notification, a truncated tag, empty.
- TestHintSilentOnTaskNotificationE2E and TestHintSilentOnQueuedNotificationE2E
  run the built binary against an embedder that scores every prompt 1.0, so the
  silence can only come from the guard. They assert no output, zero embedder
  calls, and mode machine-turn.
- TestHintStillShownOnHumanPromptE2E is the control arm.
- TestGLMLatchBannerOnNotificationTurnE2E runs in an established session, so the
  first-prompt rule cannot be what shows the banner. It asserts the banner is
  shown, the hint is absent, and the state dir is byte-identical afterwards.

Mutation-proven with a harness that requires each mutant to compile first: 10
of 10 compiling mutants are caught, and a non-compiling control is rejected
rather than counted. The caught mutants include: guard off at the call site;
retrieval still running; mislabelled row; no whitespace trim; contains instead
of prefix; widened tag; banner suppressed on machine turns; a state write on
machine turns.

B12: +28 non-test LOC, budget raised to 30952 with the prior note kept.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@dmmdea
dmmdea merged commit 5d395c5 into main Sep 23, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant