Skip to content

Commit 9cd3f2f

Browse files
GHGuideclaude
andcommitted
feat: deterministic quality hooks + prompt-lint (from autonomous research)
Researched Anthropic's orchestrator-worker system, 2026 parallel-agent orchestrators (Conductor/Claude-Squad/claude-flow), Claude Code hooks, and DSPy/promptfoo. polylane already covers most patterns (council=orchestrator-worker, corpus/brief=memory externalization, tmux+worktrees=parallel isolation). Two genuine gaps, built minimally: 1. verify-gate.sh — a Stop HOOK (reuses the existing hook-install pattern) that BLOCKS a lane claiming DONE without a non-empty docs/verify-<lane>.md. The research lesson: hooks are the deterministic layer under a probabilistic agent — polylane baked quality into prompts (hope); this enforces it (no DONE without evidence). Registered via the existing settings snippet; installed beside graphify-nudge. 2. polylane-promptlint.sh — deterministic gate on GENERATED lane prompts before launch: asserts objective/OWN/FORBIDDEN/nonce-DONE/verify structure. DSPy is over-engineering for prompts generated fresh per lane (nothing to compile); this enforces the empirically-validated shape offline and catches the orchestrator dropping a block (the real marker-drift/missing-boundary bug class) before a pane spawns. +16 tests (442->458). The marker-contract doc guard caught bare-marker drift in my own new doc text (fixed). All 21 scripts shellcheck-clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent b267fcd commit 9cd3f2f

7 files changed

Lines changed: 148 additions & 3 deletions

File tree

assets/settings-hook-snippet.json

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
{
2-
"_comment": "Merge this into <project>/.claude/settings.json (create the file if absent, or merge the hooks.PreToolUse array into an existing one). Registers a non-blocking nudge that fires on Grep/Glob and steers navigation to graphify-out/q.py. The lanes skill CANNOT write this file for you under auto-mode (self-modification guard) — hand this snippet to the user to paste/approve.",
2+
"_comment": "Merge into <project>/.claude/settings.json (create if absent, or merge each array into the existing one). PreToolUse: a non-blocking nudge steering navigation to graphify-out/q.py. Stop: verify-gate BLOCKS a lane that claims DONE without writing its docs/verify-<lane>.md evidence — deterministic verification-before-completion. The lanes skill CANNOT write this file under auto-mode (self-modification guard) — hand this snippet to the user to paste/approve.",
33
"hooks": {
44
"PreToolUse": [
55
{
@@ -8,6 +8,13 @@
88
{ "type": "command", "command": "\"$CLAUDE_PROJECT_DIR/.claude/hooks/graphify-nudge.sh\"" }
99
]
1010
}
11+
],
12+
"Stop": [
13+
{
14+
"hooks": [
15+
{ "type": "command", "command": "\"$CLAUDE_PROJECT_DIR/.claude/hooks/verify-gate.sh\"" }
16+
]
17+
}
1118
]
1219
}
1320
}

assets/verify-gate.sh

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
#!/usr/bin/env bash
2+
# Stop hook — a lane may NOT finish having claimed DONE without leaving evidence.
3+
# If docs/status-<lane>.md says DONE but docs/verify-<lane>.md is missing/empty,
4+
# BLOCK (exit 2) so the agent must write the proof before stopping. The lane prompt
5+
# only ASKS for verification; this hook ENFORCES it deterministically (the research
6+
# lesson: hooks are the deterministic layer under a probabilistic agent).
7+
# Must exit 0 on every non-block path (missing dir, closed stdout, set -e callers).
8+
input=$(cat 2>/dev/null || true)
9+
# already retried once via a stop hook -> don't hard-loop; let the agent stop.
10+
case "$input" in *'"stop_hook_active":true'*) exit 0 ;; esac
11+
12+
DIR="${CLAUDE_PROJECT_DIR:-.}"
13+
for s in "$DIR"/docs/status-*.md; do
14+
[ -f "$s" ] || continue
15+
head -1 "$s" 2>/dev/null | grep -q 'DONE' || continue
16+
lane=$(basename "$s" .md); lane=${lane#status-}
17+
if [ ! -s "$DIR/docs/verify-$lane.md" ]; then
18+
echo "polylane verify-gate: lane '$lane' claims DONE in $s but docs/verify-$lane.md is missing/empty. Write the verification evidence (what you built + proof it works) before finishing." >&2
19+
exit 2
20+
fi
21+
done
22+
exit 0

bin/polylane-promptlint.sh

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
#!/usr/bin/env bash
2+
# polylane-promptlint.sh — deterministic gate on a GENERATED lane prompt before launch.
3+
# The orchestrator writes each .polylane/lanes/<lane>.txt with an LLM, so a block can be
4+
# dropped (the marker-drift + missing-OWN/FORBIDDEN bugs were exactly this). This lints
5+
# for the empirically-validated structure — objective, tool/file boundaries, the nonce
6+
# DONE contract, verify evidence — the way DSPy/promptfoo enforce prompt quality, but
7+
# cheap and offline (the prompts are generated fresh per lane, so there's nothing to
8+
# compile). Reports PROMPT-LINT: <lane> missing <what>; exit 6 if any lane fails.
9+
# lint <lane-prompt-file> [<lane-name>] one prompt
10+
# lint-run <manifest> every lane's prompt in a run.json
11+
# Pure bash-3.2 + jq (jq only for lint-run); main-guarded.
12+
set -euo pipefail
13+
14+
# required token -> human label. A prompt must contain each (case-insensitive).
15+
lint_one() {
16+
local f="$1" lane="${2:-$(basename "$1" .txt)}" miss=""
17+
[ -s "$f" ] || { echo "PROMPT-LINT: $lane empty-or-missing $f"; return 6; }
18+
grep -qiE 'GOAL|/goal' "$f" || miss="$miss objective(GOAL)"
19+
grep -qi 'OWN' "$f" || miss="$miss ownership(OWN)"
20+
grep -qi 'FORBIDDEN' "$f" || miss="$miss boundaries(FORBIDDEN)"
21+
grep -qE 'STATUS:.*DONE' "$f" || miss="$miss done-marker(STATUS:..DONE)"
22+
grep -q 'run=' "$f" || miss="$miss nonce(run=<RUN_ID>)"
23+
grep -qi 'verify' "$f" || miss="$miss verify-evidence"
24+
if [ -n "$miss" ]; then echo "PROMPT-LINT: $lane missing$miss"; return 6; fi
25+
return 0
26+
}
27+
28+
lint_run() {
29+
local mf="$1" rc=0 lane pf dir
30+
command -v jq >/dev/null 2>&1 || { echo "polylane-promptlint: jq required for lint-run" >&2; return 2; }
31+
dir=$(cd "$(dirname "$mf")/.." && pwd) # .polylane/ -> project root
32+
for lane in $(jq -r '.lanes[].name, .integrator.name' "$mf"); do
33+
pf=$(jq -r --arg n "$lane" '(.lanes[],.integrator) | select(.name==$n) | .prompt_file' "$mf" | head -1)
34+
case "$pf" in /*) : ;; *) pf="$dir/$pf" ;; esac
35+
lint_one "$pf" "$lane" || rc=6
36+
done
37+
return $rc
38+
}
39+
40+
if [ "${BASH_SOURCE[0]:-}" = "${0}" ]; then
41+
case "${1:-}" in
42+
lint) shift; lint_one "$@" ;;
43+
lint-run) shift; lint_run "${1:?usage: lint-run <manifest>}" ;;
44+
*) echo "usage: polylane-promptlint.sh lint <prompt-file> [lane] | lint-run <manifest>" >&2; exit 2 ;;
45+
esac
46+
fi

references/install-helpers.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -80,8 +80,13 @@ PROJECT="/absolute/path/to/target/project" # the repo builders will edit
8080

8181
mkdir -p "$PROJECT/.claude/hooks"
8282
cp "$SKILL_DIR/assets/graphify-nudge.sh" "$PROJECT/.claude/hooks/graphify-nudge.sh"
83-
chmod +x "$PROJECT/.claude/hooks/graphify-nudge.sh"
83+
cp "$SKILL_DIR/assets/verify-gate.sh" "$PROJECT/.claude/hooks/verify-gate.sh"
84+
chmod +x "$PROJECT/.claude/hooks/"*.sh
8485
```
86+
`verify-gate.sh` is a **Stop hook**: it BLOCKS a lane that writes `STATUS: <lane> DONE run=<RUN_ID>`
87+
without leaving a non-empty `docs/verify-<lane>.md` — deterministic
88+
verification-before-completion (the prompt asks; the hook enforces). Registered by the
89+
same settings snippet below.
8590
Verify:
8691
```bash
8792
python "$PROJECT/graphify-out/q.py" 2>&1 | head -1 # prints usage → q.py runs

references/planning.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,8 @@ When the run finishes and the integrator issues GO on a **re-merge of current br
9292
- Every generated prompt bakes the done-signal: the lane writes `docs/status-<lane>.md` with first line exactly `STATUS: <lane> DONE run=<RUN_ID>` (per-lane, worktree-safe). `docs/parallel-status.md` is for cross-lane requests only — never the done signal.
9393
- Phase 5 defaults to one git worktree per lane (shared-index race — a shared tree lets one lane's commit bundle another's staged files); shared-tree only on explicit user opt-out.
9494
- The Phase 5 plan gate always shows the rough per-lane + total cost estimate computed from the `references/model-selection.md` price table (canonical for costs) — never ask for approval without the $ visible.
95-
- Phase 6 emits `.polylane/run.json` (frozen schema: `base` · `intensity` · `available_models[]` · `integrator{name,model,effort,branch,worktree,prompt_file}` · `lanes[]{name,model,effort,branch,worktree,prompt_file,own_globs}`) plus `.polylane/lanes/<lane>.txt` per lane, alongside the printed paste blocks. New keys (`intensity`, `available_models`, per-object `effort`) match Lc's `.polylane/SCHEMA.md`.
95+
- Phase 6 emits `.polylane/run.json` (frozen schema: `base` · `run_id` · `intensity` · `available_models[]` · `integrator{name,model,effort,branch,worktree,prompt_file}` · `lanes[]{name,model,effort,branch,worktree,prompt_file,own_globs}`) plus `.polylane/lanes/<lane>.txt` per lane, alongside the printed paste blocks. New keys (`intensity`, `available_models`, per-object `effort`) match Lc's `.polylane/SCHEMA.md`.
96+
- **Lint the generated prompts before launch (deterministic prompt-quality gate):** `bin/polylane-promptlint.sh lint-run .polylane/run.json` — asserts every lane prompt carries the validated structure (objective/GOAL, OWN + FORBIDDEN boundaries, the `STATUS: … DONE run=<RUN_ID>` marker, verify evidence). A `PROMPT-LINT: <lane> missing …` (rc 6) means the generator dropped a block — REGENERATE that prompt before spawning any pane (a malformed prompt otherwise silent-hangs or breaks isolation). This is the offline equivalent of a DSPy/promptfoo check: the prompts are generated fresh per lane, so there's nothing to compile — the lint enforces the empirically-validated shape instead.
9697
- Shared file between lanes → one lane owns it; the other requests edits via `docs/parallel-status.md`, never edits directly.
9798
- Exactly one lane may hold any shared physical resource (device/simulator/deploy target) — mutex via status file.
9899
- Project-specific facts (build recipes, device UDIDs, quirks) come from the project's CLAUDE.md — keep this skill generic.

tests/test-promptlint.sh

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
#!/usr/bin/env bash
2+
# polylane-promptlint.sh — a generated lane prompt must carry the validated structure
3+
# (objective, OWN/FORBIDDEN, nonce DONE marker, verify). Catches the orchestrator
4+
# dropping a block (the real marker-drift / missing-boundary bugs) before launch.
5+
. "$(cd "$(dirname "$0")" && pwd)/helpers.sh"
6+
LINT="$(cd "$(dirname "$RUNNER")" && pwd)/polylane-promptlint.sh"
7+
. "$LINT"
8+
9+
make_tmpdir
10+
GOOD="$TEST_TMPDIR/good.txt"
11+
cat > "$GOOD" <<'P'
12+
/goal build the thing. OWN: src/x. FORBIDDEN: everything else.
13+
DONE-SIGNAL: STATUS: x DONE run=<RUN_ID>. Write docs/verify-x.md with proof.
14+
P
15+
assert_ok "lint-good" lint_one "$GOOD" x
16+
17+
# each missing element fails with a named gap
18+
miss_test() {
19+
local name="$1" drop="$2"
20+
local f="$TEST_TMPDIR/$name.txt"
21+
grep -viE "$drop" "$GOOD" > "$f" || true
22+
assert_fail "lint-missing-$name" lint_one "$f" "$name"
23+
}
24+
miss_test objective 'GOAL|/goal'
25+
miss_test own 'OWN'
26+
miss_test forbidden 'FORBIDDEN'
27+
miss_test nonce 'run='
28+
miss_test verify 'verify'
29+
30+
# the message names what's missing
31+
out=$(lint_one "$TEST_TMPDIR/nonce.txt" nonce 2>&1 || true)
32+
assert_contains "lint-names-gap" "nonce(run=" "$out"
33+
34+
# empty prompt fails
35+
: > "$TEST_TMPDIR/empty.txt"
36+
assert_fail "lint-empty" lint_one "$TEST_TMPDIR/empty.txt" e
37+
38+
finish

tests/test-verify-gate.sh

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
#!/usr/bin/env bash
2+
# assets/verify-gate.sh — Stop hook: block a lane that claims DONE without its
3+
# verify evidence file. Deterministic verification-before-completion.
4+
. "$(cd "$(dirname "$0")" && pwd)/helpers.sh"
5+
GATE="$(cd "$(dirname "$RUNNER")" && pwd)/../assets/verify-gate.sh"
6+
7+
make_tmpdir
8+
mkdir -p "$TEST_TMPDIR/docs"
9+
run_gate() { CLAUDE_PROJECT_DIR="$TEST_TMPDIR" bash "$GATE" <<<'{}'; }
10+
11+
# no status file at all -> allow (nothing claimed)
12+
assert_ok "gate-allows-when-no-status" run_gate
13+
14+
# status DONE but NO verify -> BLOCK (exit 2)
15+
printf 'STATUS: alpha DONE run=r1\n' > "$TEST_TMPDIR/docs/status-alpha.md"
16+
assert_rc "gate-blocks-done-without-verify" 2 env CLAUDE_PROJECT_DIR="$TEST_TMPDIR" bash "$GATE" </dev/null
17+
18+
# add the verify evidence -> allow
19+
printf 'built alpha; tests green\n' > "$TEST_TMPDIR/docs/verify-alpha.md"
20+
assert_ok "gate-allows-with-verify" run_gate
21+
22+
# stop_hook_active -> never hard-loop, allow even if verify missing
23+
rm -f "$TEST_TMPDIR/docs/verify-alpha.md"
24+
assert_ok "gate-no-loop-when-active" sh -c "CLAUDE_PROJECT_DIR='$TEST_TMPDIR' bash '$GATE' <<<'{\"stop_hook_active\":true}'"
25+
26+
finish

0 commit comments

Comments
 (0)