Skip to content

fix: eleven audit findings, and the standing cost measured and cut - #45

Merged
protonspy merged 2 commits into
mainfrom
fix/audit-findings
Sep 15, 2026
Merged

protonspy merged 2 commits into
mainfrom
fix/audit-findings

Conversation

@protonspy

Copy link
Copy Markdown
Owner

An audit of the CLI, the hook layer, the skills and the rules. Every bug below was reproduced against a built binary before it was fixed; the token numbers are measured on a freshly scaffolded Claude Code workspace.

Bugs

Reproduced against the binary:

  • --help exited 1 on all 52 flag-parsed subcommands. An agent branching on the 0/1/2 contract reads the help it asked for as a command that could not run.
  • A flag after a positional was swallowed as prose. scc notes add --tag gotcha "text" --path internal/x recorded no path, wrote --path internal/x into the note body, and exited 0. Misfiling a record while reporting success is worse than rejecting it.
  • scc patch wrote outside the workspace. scc patch append ../outside.md L3 --text X modified a file outside the repository and exited 0. An artifact address reaches scc from a file the agent read as readily as from the user, so the boundary is in artifact.Resolve rather than in one handler. map show had the read half of the same hole.

Found by reading:

  • core.hooksPath was joined literally, so a configured ~/.githooks became <repo>/~/.githooks: scc wrote three hooks where git never reads them.
  • A shebang with flags was called non-POSIX. #!/bin/sh -e made --force refuse an ordinary husky hook.
  • Stop carried two findings the next turn cannot resolve — a signature in a commit already made needs a rebase, and a finding in a file this branch never touched is not this task's. Both re-fire every turn on a stage where speaking is a continuation, which is the shape of the loop fix(hooks): the Stop stage looped — honour stop_hook_active, move drift to SessionStart #44 fixed.
  • Stop re-indexed the symbol graph every turn, starting a node process on turns that wrote nothing.
  • pre-push ran the full gate on git push --delete and on a push carrying nothing. git says which on stdin and nothing read it.
  • commit-msg assumed #, so a repository setting core.commentChar had git's own template read as message and the whole verbose diff scanned for signatures.
  • scc hooks run <stage> blocked forever on a terminal, which the usage text invites.
  • The undelivered-work line fired from the first commit of a unit of work, asking for a push mid-task on a stage that acts on what it is told. It waits for a clean tree now.
  • A settings entry was claimed by substring, so my-scc hooks run stop read as scc's own.

The standing cost

The entry file plus 15 rules came to 47KB preloaded into every request of the session. CLAUDE.md still said nine rules and ~26KB, and had said so since the set was that size.

Nothing about the methodology changes here. What changes is when three of its files arrive: specs.md, tasks.md and knowledge-base.md carry a paths: header, so the harness loads them when the agent touches the tree they govern.

The bar is mechanical and tested: a rule may be scoped only where a validator reports what it prevents. A scope trades "in context before the decision" for "in context when the file is open", and that is survivable exactly when something else catches the result. The rules that look most scopable are the clearest refusals — artifacts.md would scope to plans/** and its instruction is never open the plan, so it would only ever load to say the thing it prevents has happened.

The skill descriptions were the unmeasured half: a body is paid once by the run that invokes it, a description by every request whether the skill runs or not. Eight came to 4087 characters, mostly enumerating findings the validator already names by rule.

before after
preloaded per request 52,388 B 40,178 B
map tasks --json 2,198 B 1,558 B
clean scc validate 12 lines 1 line

--json is compact off a terminal and indented on one, the same test render makes about color.

Naming

The review agents become scc-code-review and scc-security-review. Claude Code ships its own under the unprefixed names, so delivery.md naming them in prose resolved to whichever the harness picked — a collision nobody would ever see, because both produce a review and the one that ran is the one that answers. scc update turns the old files into deletes or orphans on its own.

--no-verify becomes --no-seal-check. git spells "skip the hooks" the same way, and this skipped a plan's seal check while every validator still ran.

Verification

go build, gofmt -l, go vet, golangci-lint run (0 issues) and the full suite are green. -race could not run on this machine — no gcc for cgo — so CI is the first place it does.

Smoke-tested end to end on a fresh workspace: init, the three git hooks firing on a real commit, the three harness stages, pre-push skipping a deletion and running on a real ref, and the Stop stage silent on a dirty tree and speaking on a clean one with an unpushed commit.

… in the workspace

Eleven findings from an audit of the CLI and the hook layer, each reproduced
before it was fixed.

The three that were measured against a built binary:

- `--help` exited 1 on all 52 flag-parsed subcommands, so an agent branching on
  the 0/1/2 contract read the help it asked for as a command that could not run.
- A flag after a positional was swallowed as prose: `notes add --tag gotcha
  "text" --path x` recorded no path and wrote the flag into the note, exit 0.
  parseFlags now alternates between peeling positionals and parsing flags, and
  honours a bare `--`.
- `scc patch append ../outside.md` wrote to a file outside the repository and
  exited 0. artifact.Resolve now requires the resolved path to sit inside the
  workspace, symlinks evaluated on both sides so a macOS temp dir still passes.

The hook layer:

- core.hooksPath was joined literally, so a configured `~/.githooks` became
  `<repo>/~/.githooks` and git read the real one. rev-parse --git-path already
  answers both halves.
- A shebang with flags (`#!/bin/sh -e`) was called non-POSIX, so --force refused
  an ordinary husky hook. The interpreter is read out of the line now.
- Stop ran the attribution validator, whose finding needs a rebase rather than an
  edit — a line that can never clear on a stage where speaking is a continuation.
  It also reported findings in files this branch never touched. Both are out:
  `validate.WithoutAttribution`, and a filter against `git.Changed`.
- Stop synced the symbol graph on every turn. It now compares the graph's mtime
  against the files this branch changed and skips when nothing is newer.
- pre-push ran the full gate on `git push --delete` and on a push carrying
  nothing. git says which on stdin, and that is now read.
- commit-msg assumed `#`, so a repository setting core.commentChar read git's
  template as message and scanned the whole verbose diff.
- `scc hooks run <stage>` blocked forever on a terminal, which the usage text
  invites people to do.
- The undelivered-work line fired from the first commit of a unit of work,
  asking for a push mid-task on a stage that continues the conversation. It now
  waits for a clean tree.
- A settings entry was claimed by substring, so `my-scc hooks run stop` read as
  scc's own.

Output, where an agent pays per byte: `--json` is compact off a terminal
(`map tasks --json` 2198 -> 1558 bytes) and a clean `scc validate` prints one
line instead of twelve. `--no-verify` is renamed `--no-seal-check`, since git
spells "skip the hooks" the same way and this skipped a plan's seal check.
…the review agents

The standing cost of a scaffolded Claude Code workspace, measured rather than
assumed: the entry file plus 15 rules came to 47KB preloaded into every request
of the session, while CLAUDE.md still said nine rules and ~26KB. Nothing about
the methodology changes here. What changes is when three of its files arrive.

`specs.md`, `tasks.md` and `knowledge-base.md` carry a `paths:` header, so the
harness loads them when the agent touches the tree they govern rather than at
session start. That is 10.4KB off every request, about a quarter.

The bar for scoping a rule is mechanical and tested: a validator has to report
what the rule prevents. A scope trades "in context before the decision" for "in
context when the file is open", and that is survivable exactly when something
else catches the result — a malformed EARS line, a task with no methodology
annotation, a broken wikilink are all exit 2 before the commit. The rules that
look most scopable are the clearest refusals: artifacts.md would scope to
`plans/**` and its instruction is never open the plan, so it would only load to
say the thing it prevents has happened.

The skill descriptions were the unmeasured half. A body is paid once by the run
that invokes it; a description is paid by every request whether the skill runs
or not. Eight came to 4087 characters, mostly enumerating findings the validator
already names by rule — 2084 now, with a budget test so they cannot creep back.

And the two review agents become `scc-code-review` and `scc-security-review`.
Claude Code ships its own under the unprefixed names, so delivery.md naming them
in prose resolved to whichever the harness picked — a collision nobody would
ever see, because both produce a review and the one that ran is the one that
answers. `scc update` turns the old files into deletes or orphans on its own.

Also corrected: the stack skill claimed `scc validate` reads go.mod and
package.json, where it reads seven manifests.

Preloaded, Claude Code, fresh workspace: 52388 -> 40178 bytes.
@protonspy
protonspy merged commit 5c461b2 into main Sep 15, 2026
4 checks passed
@protonspy
protonspy deleted the fix/audit-findings branch September 15, 2026 21:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant