docs: context-closed-tasks experiment — brainstorms v2/v3, pilot protocol, run data, RESULTS (STRONG)

docs/brainstorms/2026-07-06-context-closed-tasks-v3.md

@@ -0,0 +1,115 @@
+---
+date: 2026-07-06
+version: 3
+topic: context-closed-tasks
+status: post-pilot design state
+origin: Jan Skolte × Claude (Fable 5); consolidates v2, the landscape doc, and the Part B pilot outcome
+supersedes: >
+  2026-07-05-context-closed-tasks-v2.md §A (design) is updated by this doc;
+  v2 Part B (pilot protocol) remains the canonical protocol reference and is
+  NOT superseded. The landscape doc (2026-07-06-context-closed-tasks-
+  landscape-and-substrate.md) remains the research record.
+evidence: experiments/ccx/RESULTS.md (branch experiment/ccx-spikes)
+artifact_readiness: design-current; build input for the thin harness
+---
+
+# Context-Closed Tasks v3 — What the Pilot Proved, Changed, and Earned
+
+## §1 Thesis, post-pilot
+
+The v2 thesis stands and now has empirical rails: **agent context managed as
+contract revisions beats history transfer on decomposable implementation
+work.** Pilot (N=8 tasks, 2 arms, blinded scoring, pre-registered): Arm A
+(fresh sessions, ~2k-token byte-stable briefs) — 0 implementation defects,
+3 unlicensed decisions, 8/8 delivered-as-specified, 100% surfaced-vs-guessed;
+Arm B (continuous full-context) — 21+ implementation defects, 40 unlicensed
+decisions, one silent task substitution. Both scorers agreed directionally
+8/8. Every Arm A defect was in the **contracts**, not the implementations —
+risk concentrated into the decomposer exactly as A-CORE predicted. STRONG
+reading met; decision rule → thin harness + pilot 2. Caveats recorded in
+RESULTS.md §Validity (same-model scorers, N=8, one repo, author=operator).
+
+## §2 New design commitments (earned today, not speculated)
+
+Per §A9.1 discipline, each is backed by a named defect/incident:
+
+1. **Exclusion/environment clause on contracts.** The shipped drift guard
+   carried a live-reproduced P1 because the contract was silent on ignore
+   semantics — the implementer hand-built a walk with weaker exclusion
+   semantics than every sibling surface. Contracts that touch filesystem
+   enumeration must state the exclusion contract (policy/.forgeignore/
+   .gitignore) or name the primitive that owns it.
+   (Earned by: review finding #1, run 20260706-145749-963e80e5.)
+2. **Gate layering is a model, not an option.** Contract acceptance
+   (self-run commands) / independent re-verification / layered adversarial
+   review / CI catch DISJOINT failure classes: 11/11 gates and two blinded
+   scorers missed a P1 the persona review reproduced in an hour. Contract
+   green licenses integration, never merge. (Earned by: the entire NER-382
+   promotion arc.)
+3. **Fix set / guard set for `acceptance`** (steal-list item, now earned):
+   five Goodhart cases logged where acceptance passed while intent was
+   violated, incl. one vacuous test filter. Acceptance needs a must-fix set
+   and a must-not-regress set. (Earned by: 4 Arm B verifier defects + B's
+   vacuous `-p forge-store provenance` filter.)
+4. **Facade/wiring allowance in `allowed_changes`.** Every real slice that
+   adds a module needs the facade decl/re-export line; contracts that omit
+   it force either a violation or an unlicensed edit. Standing allowance:
+   facade files, decl+re-export only. (Earned by: A-382-2 blast violation +
+   both arms' cap-collision workarounds on 362-1.)
+5. **Contract lint.** My contracts weren't valid YAML; one was
+   unsatisfiable (file at exactly its line cap + "add lines, don't grow").
+   Lint battery v0: parseable; allowed_changes non-empty and satisfiable;
+   referenced primitives exist AND are visible (pub vs pub(crate) — the
+   382-2 rev-1 contradiction AND the hand-parse both trace to a fenced
+   pub(crate) primitive); acceptance commands match ≥1 test.
+   (Earned by: scorer contract-defect findings, all five.)
+6. **Dependency stacking is a harness primitive, not an afterthought.**
+   Clean-base runs of dependent tasks are physically impossible; the P1
+   amendment (stack predecessors' patches, committed, detached HEAD) is now
+   protocol. The clean-base failure mode doubles as a cheap
+   unknown-surfacing probe. (Earned by: pilot amendment P1.)
+7. **Byte-stable briefs** confirmed as design requirement (v2 §7.3):
+   verified `cmp`-identical emissions; keep canonical serialization
+   mandatory from day one.
+
+## §3 What did NOT survive contact
+
+- **"Contract-author familiarity makes authoring cheap"** — the ~45-min
+  figure is a floor; and familiarity did not prevent five contract defects.
+  Authoring quality, not authoring speed, is the cost center.
+- **"Blast radius as pure allowlist"** — needs the standing facade
+  allowance (§2.4) and, per NER-383, semantics decisions where refusal
+  fires; a bare path list under-specifies.
+- **v2's implicit "acceptance green ≈ done"** — replaced by §2.2 layering.
+
+## §4 Updated roadmap (decision-rule compliant)
+
+1. **Thin harness** (files+scripts, Git-compatible, no Forge objects):
+   productize `brief.sh` (+neighbor graphs), `blast-check.py`, the
+   UNKNOWN.md stop convention + triage flow, dependency-ordered runner
+   (from run-arm-a-stacked.sh), contract lint v0 (§2.5), and the
+   acceptance fix/guard split (§2.3).
+2. **NER-362 completion** through that harness: tip-resolution contract
+   revision (the 362-3 contract defect), affected reruns, promotion round
+   with the full gate stack. Dogfoods post-pilot revision flow.
+3. **Pilot 2 — neighbor ablation** (pre-registered, v2 §A9.3) on the next
+   real feature.
+4. **Pilot 3 — CooperBench** (~10 typst pairs × 3 arms, ~$30–75; T3
+   feasibility note): contracts vs. NL chat, Shepherd's supervisor as the
+   reference point.
+5. **Substrate build** (Forge-native contract records, forge brief/unknown,
+   merge gate) only after 1–4; the substrate map (landscape §4) stands.
+6. **Publish decision** (Jan): RESULTS.md + this arc as a public writeup.
+
+## §5 Open items carried forward
+
+- U4: verified-handoff doc still uncommitted (lineage debt before v4 cites
+  it).
+- U5: harness-side injection surface — prototype exists (pilot runner);
+  design the real one in roadmap item 1.
+- U6: brief cost at neighbor-graph scale — measure in pilot 2.
+- NER-383: four refusal-semantics decisions (defaults proposed).
+- A6.10 review-fatigue mitigations: unexercised (no forge compact yet);
+  the replay-evidence idea (landscape §5.3) remains the strongest candidate.
+- Inter-rater with a HUMAN scorer: still valuable; both scorers were
+  same-model-family agents (recorded caveat, not resolved).

docs/handoffs/2026-07-06-ccx-thin-harness.md

@@ -0,0 +1,83 @@
+# Handoff: CCX thin harness (next session)
+
+Date: 2026-07-06 · From: contract-pilot session (context ~60%+, wrapping)
+Owner: Jan Skolte · Repo: forge (PUBLIC — only Jan-approved merges to main)
+
+## Where things stand (all verified, nothing in-flight)
+
+- **Merged to main:** PR #123 — NER-382 drift guard + review fixes
+  (b82f244). NER-382 is Done in Linear.
+- **Awaiting Jan's review:** PR #124 — the complete experiment record
+  (branch `experiment/ccx-spikes`, 20 commits: brainstorms v2/v3,
+  landscape doc, pilot protocol/rubric/contracts, all run data,
+  RESULTS.md, 3 solution docs). Docs-only.
+- **Pilot outcome:** STRONG pre-registered reading met
+  (`experiments/ccx/RESULTS.md` — read the Validity caveats before
+  quoting). Two blinded scorers, 8/8 directional agreement.
+- **Open tickets:** NER-383 (4 drift-guard semantics decisions, defaults
+  proposed — Jan's call, blocks nothing). NER-362 (intent-aware blame) —
+  implementation EXISTS as pilot Arm A patches
+  (`experiments/ccx/runs/A-362-*/patch.diff`, stacked order 362-1 →
+  362-2-r2 → 362-3-r2 → 362-4-r2 → 362-5-r2, gates verified) but is
+  HELD: the 362-3 contract pinned blame's tip resolution to the native
+  HEAD ref while repo convention resolves the authoritative tip from the
+  ledger (contract defect, scorer-confirmed). Needs a contract revision +
+  targeted fix + its own promotion round (full gate stack incl.
+  /ce-code-review — see gate-layering solution doc for why that is
+  non-negotiable).
+
+## Next objective: build the thin harness (v3 roadmap item 1)
+
+Requirements source: `docs/brainstorms/2026-07-06-context-closed-tasks-v3.md`
+§2 (seven earned commitments) + §4 item 1. Productize the pilot's duct
+tape into a small, Git-compatible toolkit (files + scripts — deliberately
+NO new Forge objects yet; that is the decision rule's explicit boundary):
+
+1. **Brief emitter** — from `experiments/ccx/brief.sh`: byte-stable
+   (canonical ordering, no timestamps — verified requirement), neighbor
+   resolution (mind the BSD-sed `[[:space:]]` lesson), fail-closed when a
+   contract is missing.
+2. **Contract lint v0** (earned, v3 §2.5): YAML-parseable; allowed_changes
+   non-empty AND satisfiable (the 4730-line-cap contradiction); referenced
+   primitives exist AND are visible (pub vs pub(crate) caused two
+   defects); acceptance commands match ≥1 test (vacuous-filter Goodhart);
+   exclusion clause present for any filesystem-enumeration task (the P1).
+3. **Blast-radius check** — from `experiments/ccx/blast-check.py`, plus
+   the standing facade allowance (decl/re-export lines in facade files
+   always permitted — v3 §2.4).
+4. **UNKNOWN.md convention + triage flow** — the stop rule verbatim from
+   `run-arm-a.sh`, plus a triage step (an unanswered unknown teaches
+   guessing — see the stop-on-unknown solution doc).
+5. **Dependency-ordered runner** — from `run-arm-a-stacked.sh`: stack
+   predecessors' patches, committed on a DETACHED head (the pilot-run
+   branch-pointer bug), `--3way` apply, halt-on-unknown.
+6. **Acceptance fix/guard split** (earned, v3 §2.3): must-fix commands +
+   must-not-regress commands per contract.
+
+Process: run `/ce-plan` with v3 as input → doc-review gate → `/ce-work`.
+Dogfood target after the harness: NER-362 completion THROUGH it (contract
+revision flow is itself the thing to exercise).
+
+## Gotchas the fresh session must know
+
+- Public repo; commit docs/experiments to branches; Jan approves merges.
+  (Memory file: forge-main-approval-public-repo.)
+- Dogfood `forge` binary ONLY in /tmp throwaway repos or a temp clone of
+  `~/Github-Private/forge-dogfood` — NEVER from the project root.
+- Verify trio + `rtk bash scripts/ci.sh` before any push; the
+  code-review gate is non-optional (fresh evidence: the gate found a
+  live-reproducible P1 after 11/11 acceptance gates passed —
+  `docs/solutions/conventions/contract-acceptance-is-not-merge-ready.md`).
+- `forge-content-native/src/lib.rs` is allowlisted at exactly 4730 lines
+  and MUST NOT GROW — new code goes in new module files.
+- grep the three new `docs/solutions/` docs before designing anything
+  walker-, gate-, or unknown-shaped.
+- Lineage debt U4: the 2026-07-04 verified-handoff brainstorm is still
+  uncommitted; commit or formally drop the reference before v4 cites it.
+
+## Queue after the harness (v3 §4)
+
+NER-362 completion → pilot 2 (neighbor ablation, next real feature) →
+pilot 3 (CooperBench, ~$30-75, feasibility note in
+`experiments/ccx/T3-cooperbench-feasibility.md`) → substrate build →
+publish decision (Jan).

docs/solutions/architecture-patterns/filesystem-enumeration-shared-exclusion-contract.md

@@ -0,0 +1,83 @@
+---
+title: "Filesystem-enumeration surfaces must share one exclusion contract — never write a second walker"
+date: 2026-07-06
+category: architecture-patterns
+module: forge-content-native
+problem_type: architecture_pattern
+component: workspace-drift-equality-vs-scanner-walk
+severity: high
+applies_when:
+  - Any new code enumerates worktree/workspace files to compare against a recorded native tree (equality checks, drift guards, verification passes)
+  - A check's "actual side" is built with a different ignore/exclusion stack than the walk that produced its "expected side"
+  - forge-store (or any consumer crate) needs tree-vs-filesystem facts that forge-content-native computes internally via pub(crate) primitives
+tags: [exclusion-contract, gitignore, forgeignore, is-ignored-by-policy, walk-worktree, workspace-equality, drift-guard, one-walker-rule, ner-382]
+---
+
+# Filesystem-enumeration surfaces must share one exclusion contract
+
+## Context
+
+The NER-382 drift guard (refuse `attempt attach` when the workspace dir
+drifted from its recorded `materialized_content_ref`) shipped with a
+hand-built workspace walk in `forge-store` that filtered only
+`forge_content::is_ignored_by_policy`. The recorded tree it compared
+against was built by the native scanner (`walk_worktree`), which ALSO
+honors `.gitignore`/`.forgeignore` (`.git_ignore(true)`), and the
+re-materialization deletion pass honors them too. Result — reproduced live
+against the binary: a gitignored build artifact (`target/artifact.o`)
+created by running builds inside a workspace (a documented supported flow)
+triggered `WORKSPACE_DRIFT`, survived `--discard-workspace-changes`
+(re-materialization skips gitignored files), and the very next attach
+refused again. Permanent, unclearable false drift that trains users to
+reflexively pass the override, defeating the guard. The same hand-built
+walk also re-parsed the native tree-object JSON (`entries/name/kind/mode/
+object`) in forge-store, skipping `validate_tree_entry` checks the owning
+crate applies. Root cause of the hand-build: the natural primitive
+(`tree_fingerprints`) was `pub(crate)` in forge-content-native.
+
+## Guidance
+
+One walker, one exclusion contract, owned by forge-content-native:
+
+- Never build a second filesystem walk or tree parse outside the owning
+  crate. If the primitive you need is `pub(crate)`, EXPOSE a purpose-built
+  read-only function from the owning crate rather than reimplementing.
+- Both sides of any tree-vs-filesystem comparison must go through the same
+  exclusion stack: policy filter (`is_ignored_by_policy`) AND the ignore
+  walker semantics (`.forgeignore`, `.gitignore`, rooted at the scanned
+  dir so materialized ignore files apply).
+- The fix shape that worked: `forge_content_native::workspace_equality::
+  tree_equality_drift(repo_root, scan_root, tree, excluded_paths)` —
+  actual side via `walk_worktree` rooted at `scan_root`, expected side via
+  `tree_fingerprints` (which enforces tree schema + entry validation),
+  bytes-only blob-id comparison, symlink targets compared without
+  following, strictly read-only. The forge-store caller shrank ~180 lines.
+- Contracts/specs for enumeration features must state the exclusion
+  contract explicitly or name the owning primitive ("enumerate via
+  walk_worktree semantics"), never leave it implied.
+
+## Why This Matters
+
+A filter divergence between a write path and a read path is a silent
+false-negative or false-positive factory: drift goes undetected, or
+spurious refusals condition users to bypass the guard. The divergence is
+invisible to the feature's own tests (they don't know to create gitignored
+files) and to acceptance gates — it was found only by adversarial review.
+See also the sibling learning: native-worktree-walker-ignore-engine doc
+(2026-05-30) already classified ignore-semantics divergence as a class,
+not a one-off; this is the second confirmed instance.
+
+## When to Apply
+
+Before writing ANY loop over `fs::read_dir` in a crate other than
+forge-content-native, or any `serde_json` parse of a native object
+payload outside it: stop, find the owning primitive, expose it if needed.
+
+## Examples
+
+Anti-pattern (as shipped, commit 158cc65): `collect_workspace_paths` with
+policy-only filtering + `collect_expected_tree_files` hand-parsing tree
+JSON in forge-store. Fix (commit bc2ea57): both deleted; one shared
+read-only primitive in forge-content-native/src/workspace_equality.rs,
+pinned by `attach_drift_check_honors_workspace_gitignore` (gitignored
+artifact is not drift; non-ignored stray still is).

docs/solutions/conventions/contract-acceptance-is-not-merge-ready.md

@@ -0,0 +1,75 @@
+---
+title: "Gate layering: contract acceptance green is not merge-ready — acceptance, review, and CI catch disjoint failure classes"
+date: 2026-07-06
+category: conventions
+module: compound-engineering
+problem_type: convention
+component: verification-gate-layering
+severity: high
+applies_when:
+  - A change passed its declared acceptance commands (forge check gates, task-contract acceptance, self-run test suites) and someone proposes skipping the code-review gate
+  - Designing verification for agent-produced changes (task contracts, forge trust/check policy, CI pipelines)
+  - Interpreting experiment or pilot results where "all gates passed" is used as a quality claim
+tags: [gate-layering, goodhart, acceptance-tests, code-review-gate, verifier-defect, ce-code-review, contract-pilot, ner-382]
+---
+
+# Contract acceptance green is not merge-ready
+
+## Context
+
+In the 2026-07-06 contract pilot + NER-382 promotion arc, the drift-guard
+implementation passed every gate available to it: 11/11 contract
+acceptance commands re-run independently on rebuilt bases, the full
+`scripts/ci.sh` (fmt, workspace tests, clippy, e2e eval), AND two
+independent blinded scorers who rated the patch near-perfect. The layered
+`/ce-code-review` gate (8 personas + per-finding validators) then found 15
+findings — every one sent to validation CONFIRMED — including a P1
+reproduced live against the binary (gitignored artifacts causing
+permanent false drift) and a validated composition where the documented
+override flag would delete private-labeled files.
+
+## Guidance
+
+Treat the gates as CATCHING DISJOINT FAILURE CLASSES, never as redundant
+layers where one green light excuses another:
+
+- **Acceptance commands** verify what the spec's author thought to check.
+  They are blind to everything the spec was silent about (here: ignore
+  semantics, private-label composition, crash windows).
+- **Independent re-verification** (re-running gates on rebuilt bases)
+  catches self-report drift and Goodhart-by-accident — but only within
+  the same command set.
+- **Layered adversarial review + per-finding validation** catches
+  spec-silence failures: composition across features, abuse loops,
+  crash-ordering, platform divergence. This is where all 15 findings came
+  from.
+- **CI** is the post-merge backstop, never a substitute (already repo
+  law in CLAUDE.md — this learning adds the evidence).
+
+Corollary for contract-driven work: "contract green" licenses
+INTEGRATION of a task's output into the stack; only the review gate
+licenses MERGE. Do not weaken CLAUDE.md's two non-optional gates on the
+argument that contracts/acceptance already passed.
+
+## Why This Matters
+
+Five Goodhart cases were logged in one day (acceptance passing while
+intent was violated, including one vacuous test filter that matched zero
+tests). The failure mode is seductive precisely because everything is
+green — the review gate's cost (~1h wall, ~10 subagents) bought a
+reproducible P1 and a private-data-deletion hazard before they reached
+main of a public repo.
+
+## When to Apply
+
+Every non-trivial change, and ESPECIALLY changes whose tests were written
+by the same process that wrote the code (agent-produced patches with
+self-authored acceptance). The more gates a change already passed, the
+more suspicious "skip the review" becomes.
+
+## Examples
+
+Evidence trail: `experiments/ccx/RESULTS.md` (verifier-defect class),
+review run `/tmp/compound-engineering/ce-code-review/20260706-145749-963e80e5/review.json`
+(15/15 validated findings after 11/11 green gates), PR #123 commits
+158cc65 (gates-green with P1s) → bc2ea57 (post-review fixes).