Skip to content

test: realign AGENTS.md contract assertions with the StaffCaptain fork vocabulary - #1

Merged
Freudator86 merged 3 commits into
mainfrom
fm/sc-fork-red-main
Jul 28, 2026
Merged

Freudator86 merged 3 commits into
mainfrom
fm/sc-fork-red-main

Conversation

@Freudator86

Copy link
Copy Markdown
Owner

Intent

Get Freudator86/staffcaptain-firstmate main green again after it went red on 2026-07-24 and sat unnoticed for four days, and do it honestly rather than by making CI pass.

Background: commit 04763e8 forked AGENTS.md into StaffCaptain form, renaming the fleet-level human authority from 'captain' to 'Commodore' and the orchestrator from 'Firstmate' to 'StaffCaptain', in prose only. That rename is deliberate, landed fleet policy (confirmed by the captain 2026-07-28), and is the whole point of the fork. Three inherited behavior tests assert AGENTS.md wording as literal fixed strings and still expected the upstream spelling. The instruction is correct and the tests are the stale artifact, so the fix updates the tests. AGENTS.md is deliberately NOT touched - restoring the captain wording to make a test pass would silently undo the fork decision.

What I changed: 8 drifted assertion sites across tests/fm-captain-translation-contract.test.sh (6), tests/fm-instruction-owners.test.sh (1), and tests/fm-no-mistakes-ownership.test.sh (1). Every one pins a contract that the fork still honours; only the vocabulary moved. Two are not pure renames and were judged individually: 'project outcome' became 'coordination outcome' (StaffCaptain coordinates vessels rather than doing project work, per new AGENTS.md section 1a), and the house-vocabulary sentence additionally gained 'vessel'. I updated the tests to the fork's wording in both cases rather than weakening the assertions.

Deliberate decisions a reviewer reading only the diff would not know:

  • The test FILE NAME fm-captain-translation-contract.test.sh is intentionally left as-is. The fork renamed prose only, not file names or schema tokens; data/captain.md, tasks-axi --kind captain, and the captain-hold verb all keep upstream spelling for the same reason.
  • Roughly 15 other captain/Firstmate assertions across the suite are deliberately left unchanged. They target .agents/skills/** and bin/, which the fork never modified, so those tests correctly describe what those files actually say. Changing them would make the tests wrong.
  • I added an explicit assert_section_9_found guard. The section 9 awk extractor anchors on the heading; once the heading was renamed it returned an empty string, so all 26 assertions in that file failed against empty input and only the first was printed. The guard makes a future rename fail for the real cause. This tightens the file; no assertion was deleted or weakened anywhere.
  • One bullet was added to CONTRIBUTING.md recording the rename and naming the three tests that track it, so a later upstream fast-forward does not silently restore the inherited phrasing.

A previous run of this pipeline reached the review gate and raised three findings. I verified each against the repository independently and accepted two as tightenings, which I still want applied: (1) sibling-extractor-unguarded - validate_contract in tests/fm-no-mistakes-ownership.test.sh has the same silent-empty extractor pattern, anchoring on /^### Validate$/, and needs the same explicit heading-found guard; (2) vacuous-scout-ban-assertion - assert_not_contains "scout, ship" in tests/fm-captain-translation-contract.test.sh can never fire because 'ship' appears zero times in section 9 both today and at 04763e8^, so it must be re-anchored on the actual 'Do not expose internal terms such as ...' ban sentence so that appending scout would genuinely turn it red. A test that can no longer fail is precisely the defect this task exists to remove. The third finding, skills-cross-reference-stale-vocab, is a no-op: it describes the known fork-completeness gap where .agents/skills/** still say captain/Firstmate, which is the halted phase-2 work and is deliberately out of scope. That run failed only because this host had no git author identity configured, which has since been fixed; no code problem was found.

Scope constraints: no fork-wide feature work (phase-2 investment was halted 2026-07-26; the repo is slated for archival with a 30-day rollback window), no history rewrite, no force-push, no change to the vendored/upstream relationship, and no modification of AGENTS.md.

Verification already done locally: each of the 8 changed assertions was mutation-tested to confirm it still fails when the contract it pins is removed from AGENTS.md, and when AGENTS.md is reverted to upstream wording. Full suite 89 pass / 0 fail, bin/fm-lint.sh exit 0, both symlink invariants hold, no personal fleet paths tracked, macOS job counts still exactly 13 and 36.

What Changed

  • Updated 8 drifted fixed-string assertions across tests/fm-captain-translation-contract.test.sh, tests/fm-instruction-owners.test.sh, and tests/fm-no-mistakes-ownership.test.sh to the fork's Commodore/StaffCaptain wording (including project outcome → coordination outcome and the house-vocabulary sentence gaining vessel), and renamed test_firstmate_never_responds_for_crew_run to test_orchestrator_never_responds_for_crew_run. AGENTS.md is untouched, and the test file names, data/captain.md, and other schema tokens keep their upstream spelling.
  • Tightened the same suites rather than only renaming: added assert_section_9_found and assert_validate_contract_found guards so a renamed heading fails on the real cause instead of silently feeding every assertion an empty string; re-anchored the scout/secondmate/vessel ban check on the actual Do not expose internal terms such as sentence via a new internal_term_ban extractor (the previous assert_not_contains "scout, ship" could never fire); and added fork-spelling negative guards for the personal-review prohibition.
  • Documented the divergence so an upstream fast-forward cannot silently restore the inherited phrasing: a CONTRIBUTING.md bullet recording the prose-only rename and naming the three tests that track it, plus a docs/fork-patches.md note marking the rename a standing keep divergence owned by CONTRIBUTING.md.

Full suite passes (89 pass / 0 fail); each changed assertion was mutation-tested to confirm it still fails when the contract it pins is removed or reverted to upstream wording.

Risk Assessment

✅ Low: Test-only change that leaves AGENTS.md and all file names untouched; all three previously accepted findings are implemented as genuine tightenings, and I verified each new guard extracts correctly and passes against the current AGENTS.md while remaining able to fire on the violation it pins.

Testing

The configured full-suite command had already run green as this run's baseline. On top of that I ran the three targeted suites at HEAD (19 ok / 0 not ok), reproduced the original red main by running the pre-fix test files against the fork's AGENTS.md, and then mutation-tested the change in a throwaway symlink sandbox — 15 hostile single-contract edits to AGENTS.md, all 15 caught, including five that revert AGENTS.md to upstream wording. I isolated the review's vacuous-assertion finding and showed the base-commit form stayed green under an edit that contradicts section 9 while the re-anchored form goes red, and confirmed the same for the new section-9 and ### Validate heading guards. Scope checks confirm AGENTS.md is untouched, history is linear, no test was renamed, and the fork commit touched AGENTS.md alone, so the deliberately unchanged captain/Firstmate assertions still describe their targets truthfully. Adjacent AGENTS.md-reading suites still pass; the sandbox was removed and the worktree is clean. No UI surface exists for this change — it is bash behavior tests plus one CONTRIBUTING.md bullet — so no screenshot or rendered artifact is applicable; CLI transcripts are the end-user-equivalent surface here.

Evidence: Evidence summary (start here)
# Local Test evidence — fm/sc-fork-red-main (04763e8 → c18e353)

Intent: get `main` green again **honestly** — the AGENTS.md StaffCaptain/Commodore
rename is landed policy, the three inherited tests are the stale artifact, and
AGENTS.md must not be touched.

| # | Evidence | Result |
|---|---|---|
| 01 | The three targeted suites at HEAD | 19 ok / 0 not ok, exit 0 |
| 02 | Pre-fix (04763e8) test files vs the fork's AGENTS.md | all three RED — reproduces the 2026-07-24 breakage |
| 03 | Mutation matrix, 15 hostile edits to AGENTS.md | **15/15 CAUGHT** — no assertion is vacuous |
| 04 | Old `scout, ship` assertion vs the re-anchored one | old = GREEN under a hostile edit, new = RED |
| 05 | Scope constraints from the intent | AGENTS.md untouched, linear history, no rename |
| 06 | Assertion-count reconciliation | failure sites 28→30, 56→58, 4→5 — tightened everywhere |
| 07 | Fork blast radius + sweep of other AGENTS.md-reading tests | fork touched AGENTS.md only; no other suite affected |
| 08 | Adjacent AGENTS.md-reading suites | fm-bootstrap, fm-stow-contract, fm-supervision-instructions all pass |

## Why the mutation matrix is the load-bearing evidence

Passing tests prove nothing here — the tests were *edited to match the file they
assert on*, which is exactly the shape of a dishonest green. The matrix perturbs
`AGENTS.md` one contract at a time in a throwaway symlink sandbox (the worktree
was never mutated) and requires the corresponding test to go red with the right
message:

- **A1–A6, D1, D2** — the 8 changed assertion sites. Five of them (A2, A5, A6, D1,
  D2) revert AGENTS.md to the *upstream* wording, so they confirm the tests would
  still catch an accidental un-forking.
- **B1, B2** — appending `scout`/`vessels` to section 9's internal-term ban. This
  is the contradiction the assertion exists to catch, and evidence 04 shows the
  base-commit form let it through: the literal `scout, ship` occurs zero times in
  section 9 at both 04763e8 and HEAD, so the old check could never fire.
- **C1–C3** — the new heading guards. Renaming `## 9.` or `### Validate` now fails
  on the real cause instead of feeding empty string to 26 downstream assertions.
- **E1, E2** — the new negative guards, verified against four spellings of the
  renamed orchestrator.

## Not covered

No UI surface exists — this change is bash behavior tests plus one CONTRIBUTING.md
bullet, so there is no rendered artifact to screenshot. Broad regression across the
other ~86 suites is CI's job; the configured full-suite command already ran green
as this run's baseline.
Evidence: Mutation matrix — 15/15 hostile AGENTS.md edits caught

=== A. the 6 changed assertions in fm-captain-translation-contract.test.sh === CAUGHT A1 drop positive translation contract sentence -> not ok - section 9 does not own the positive Commodore-facing translation contract CAUGHT A2 revert Commodore's nouns -> captain's nouns -> not ok - section 9 does not require Commodore-owned nouns (missing: 'Use the Commodore's nouns:') CAUGHT A3 drop the rewrite-mapping lead-in -> not ok - section 9 does not own the rewrite mapping list CAUGHT A4 drop allowed house-vocabulary sentence -> not ok - section 9 does not preserve scout as allowed house vocabulary CAUGHT A5 revert verbatim-evidence ban to captain chat -> not ok - section 9 does not reject verbatim internal evidence in Commodore chat CAUGHT A6 revert chat-summary clause to captain-facing -> not ok - section 9 does not keep chat summaries plain English === B. re-anchored scout ban (the assertion the review found vacuous) === CAUGHT B1 append 'scout' to the internal-term ban sentence -> not ok - section 9 added 'scout' to the internal-vocabulary ban sentence CAUGHT B2 append 'vessels' to the internal-term ban sentence -> not ok - section 9 added 'vessel' to the internal-vocabulary ban sentence === C. new extractor guards (silent-empty-input defect) === CAUGHT C1 rename the section 9 heading -> not ok - AGENTS.md has no '## 9. Escalation and Commodore etiquette' heading to extract CAUGHT C2 delete the internal-term ban sentence -> not ok - AGENTS.md section 9 has no 'Do not expose internal terms such as' sentence to extract CAUGHT C3 rename the '### Validate' heading -> not ok - AGENTS.md has no '### Validate' heading to extract === D. fm-instruction-owners + fm-no-mistakes-ownership === CAUGHT D1 revert separate-review clause to captain -> not ok - compressed AGENTS.md lost safety phrase 'A separate review or audit is allowed only when the Commodore explicitly requests...' CAUGHT D2 revert StaffCaptain-never-responds to Firstmate -> not ok - Validate contract permits StaffCaptain to respond directly for a crew-owned run === E. new negative guards for the renamed orchestrator === CAUGHT E1 reintroduce weak review prohibition as StaffCaptain -> not ok - AGENTS.md retained the weaker duplicate review prohibition as 'StaffCaptain' CAUGHT E2 reintroduce personal branch review as 'staff captain' -> not ok - AGENTS.md retained a personal branch-review requirement as 'staff captain' RESULT: every mutation was CAUGHT - no assertion is vacuous

=== A. the 6 changed assertions in fm-captain-translation-contract.test.sh ===
CAUGHT   A1 drop positive translation contract sentence             -> not ok - section 9 does not own the positive Commodore-facing translation contract (missing: 'Every Commodore-facing message must translate internal state into the coordination outcome, consequence, and next decision.')
CAUGHT   A2 revert Commodore's nouns -> captain's nouns             -> not ok - section 9 does not require Commodore-owned nouns (missing: 'Use the Commodore's nouns:')
CAUGHT   A3 drop the rewrite-mapping lead-in                        -> not ok - section 9 does not own the rewrite mapping list (missing: 'When evidence uses an internal label, rewrite it before sending:')
CAUGHT   A4 drop allowed house-vocabulary sentence                  -> not ok - section 9 does not preserve scout as allowed house vocabulary (missing: 'Scout, secondmate, and vessel are accepted house vocabulary and do not need translation')
CAUGHT   A5 revert verbatim-evidence ban to captain chat            -> not ok - section 9 does not reject verbatim internal evidence in Commodore chat (missing: 'Never relay worker reports, status lines, tool output, validation-state labels, or decision records verbatim into Commodore chat.')
CAUGHT   A6 revert chat-summary clause to captain-facing            -> not ok - section 9 does not keep chat summaries plain English (missing: 'the Commodore-facing chat summary that points to the report still follows this translation rule')

=== B. re-anchored scout ban (the assertion the review found vacuous) ===
CAUGHT   B1 append 'scout' to the internal-term ban sentence        -> not ok - section 9 added 'scout' to the internal-vocabulary ban sentence
CAUGHT   B2 append 'vessels' to the internal-term ban sentence      -> not ok - section 9 added 'vessel' to the internal-vocabulary ban sentence

=== C. new extractor guards (silent-empty-input defect) ===
CAUGHT   C1 rename the section 9 heading                            -> not ok - AGENTS.md has no '## 9. Escalation and Commodore etiquette' heading to extract
CAUGHT   C2 delete the internal-term ban sentence                   -> not ok - AGENTS.md section 9 has no 'Do not expose internal terms such as' sentence to extract
CAUGHT   C3 rename the '### Validate' heading                       -> not ok - AGENTS.md has no '### Validate' heading to extract

=== D. fm-instruction-owners.test.sh + fm-no-mistakes-ownership.test.sh ===
CAUGHT   D1 revert separate-review clause to captain                -> not ok - compressed AGENTS.md lost safety phrase 'A separate review or audit is allowed only when the Commodore explicitly requests that deliverable or the authorized task is a knowledge-only review; one named question remains scoped to that question.'
CAUGHT   D2 revert StaffCaptain-never-responds to Firstmate         -> not ok - Validate contract permits StaffCaptain to respond directly for a crew-owned run (missing: 'StaffCaptain never invokes `no-mistakes axi respond` for a crew-owned run.')

=== E. new negative guards added for the renamed orchestrator ===
CAUGHT   E1 reintroduce weak review prohibition as StaffCaptain     -> not ok - AGENTS.md retained the weaker duplicate review prohibition as 'StaffCaptain'
CAUGHT   E2 reintroduce personal branch review as 'staff captain'   -> not ok - AGENTS.md retained a personal branch-review requirement as 'staff captain'

RESULT: every mutation was CAUGHT - no assertion is vacuous
Evidence: Vacuous-assertion proof: old check green, new check red on the same hostile edit

Hostile edit applied to AGENTS.md section 9: Do not expose internal terms such as scout, startup machinery, locks, watchers, polling, crewmates, ... This edit BANS 'scout' while section 9 still says scout is allowed house vocabulary -- exactly the contradiction the assertion exists to catch. --- OLD assertion (base 04763e8): assert_not_contains "$contract" "scout, ship" --- VERDICT: GREEN (vacuous - hostile edit sails through) occurrences of the literal 'scout, ship' in section 9: 0 occurrences of the word 'ship' anywhere in section 9: 0 ... at base commit 04763e8 as well: 0 --- NEW assertion (target c18e353): re-anchored on the real ban sentence --- VERDICT: RED (section 9 added 'scout' to the internal-vocabulary ban sentence)

Hostile edit applied to AGENTS.md section 9:
6:Do not expose internal terms such as scout, startup machinery, locks, watchers, polling, crewmates, task ids, briefs, worktrees, checkouts

This edit BANS 'scout' while section 9 still says scout is allowed house vocabulary --
exactly the contradiction the assertion exists to catch.

--- OLD assertion (base commit 04763e8): assert_not_contains "$contract" "scout, ship" ---
  VERDICT: GREEN (vacuous - hostile edit sails through)
  occurrences of the literal 'scout, ship' in section 9: 0
  occurrences of the word 'ship' anywhere in section 9:   0
  ... at base commit 04763e8 as well:                     0

--- NEW assertion (target commit c18e353): re-anchored on the real ban sentence ---
  VERDICT: RED   (section 9 added 'scout' to the internal-vocabulary ban sentence)
Evidence: Red baseline: pre-fix tests vs the fork's AGENTS.md

== fm-captain-translation-contract.test.sh (at 04763e8) == not ok - section 9 does not own the positive captain-facing translation contract (missing: 'Every captain-facing message must translate internal state into the project outcome, consequence, and next decision.') --- output --- <empty: the pre-guard extractor silently returned nothing> -- exit 1 -- == fm-instruction-owners.test.sh (at 04763e8) == not ok - compressed AGENTS.md lost safety phrase 'A separate review or audit is allowed only when the captain explicitly requests...' -- exit 1 -- == fm-no-mistakes-ownership.test.sh (at 04763e8) == -- exit 1 --

### Reproducing the red main of 2026-07-24
### Pre-fix test files (from base commit 04763e8) run against the fork's AGENTS.md

== fm-captain-translation-contract.test.sh (at 04763e8) ==
not ok - section 9 does not own the positive captain-facing translation contract (missing: 'Every captain-facing message must translate internal state into the project outcome, consequence, and next decision.')
--- output ---

-- exit 1 --

== fm-instruction-owners.test.sh (at 04763e8) ==
ok - generic effort fallback applies only below captain and standing configuration
ok - firstmate-coding-guidelines owns compatibility review and deterministic enforcement
ok - secondmate registry guidance keeps concise routes and points to the charter
ok - state, startup, and ordinary recovery have focused owners and triggers
ok - compressed AGENTS.md records the approved one-owner map
not ok - compressed AGENTS.md lost safety phrase 'A separate review or audit is allowed only when the captain explicitly requests that deliverable or the authorized task is a knowledge-only review; one named question remains scoped to that question.'
-- exit 1 --

== fm-no-mistakes-ownership.test.sh (at 04763e8) ==
Resume fleet supervision immediately after the decision lands.

Judge validation by the branch-matched run step through `bin/fm-crew-state.sh`, not by shell liveness or the last status event.
Running, fixing, or CI states remain working; parked approval or fix-review states require the worker to follow the active gate help; passed or checks-passed is done; failed or cancelled is failed.
A worker hand-editing, committing, aborting, or restarting during an active validation run duplicates pipeline ownership; steer it back to the gate response flow.
The worker reports the PR when CI first becomes green rather than waiting for merge monitoring to finish.
-- exit 1 --
Evidence: Targeted suites green at HEAD

== tests/fm-captain-translation-contract.test.sh == 7 ok / 0 not ok, exit 0 == tests/fm-instruction-owners.test.sh == 10 ok / 0 not ok, exit 0 == tests/fm-no-mistakes-ownership.test.sh == 2 ok / 0 not ok, exit 0

== tests/fm-captain-translation-contract.test.sh ==
ok - section 9 owns the positive Commodore-facing translation contract
ok - scout remains allowed in private Commodore chat
ok - compressed safety labels require concrete plain renderings
ok - section 9 maps high-risk internal vocabulary families
ok - Commodore chat rejects verbatim internal evidence while private reports stay precise
ok - outward-facing skill handoffs point to the section 9 owner
ok - skills cross-reference section 9 instead of duplicating the mapping list
-- exit 0 --
== tests/fm-instruction-owners.test.sh ==
ok - new internal skills have one precise AGENTS.md trigger each
ok - diagnostic-reasoning owns the approved evidence procedure
ok - project-management owns registry, delivery posture, consent, initialization, and removal safety
ok - secrets-handling owns the dangerous-command doctrine, contained stow-and-clear scope, and escalation triggers
ok - generic effort fallback applies only below captain and standing configuration
ok - firstmate-coding-guidelines owns compatibility review and deterministic enforcement
ok - secondmate registry guidance keeps concise routes and points to the charter
ok - state, startup, and ordinary recovery have focused owners and triggers
ok - compressed AGENTS.md records the approved one-owner map
ok - compressed AGENTS.md retains authority, supervision, AFK, and X safety
-- exit 0 --
== tests/fm-no-mistakes-ownership.test.sh ==
ok - Validate contract assigns the complete synchronous driver loop to the initiating task worker
ok - Validate contract forbids StaffCaptain from responding directly for a crew-owned run
-- exit 0 --
Evidence: Intent scope constraints checked against the diff

-- AGENTS.md must NOT be touched -- OK: AGENTS.md byte-identical between 04763e8 and c18e353 files changed: CONTRIBUTING.md tests/fm-captain-translation-contract.test.sh tests/fm-instruction-owners.test.sh tests/fm-no-mistakes-ownership.test.sh -- no history rewrite / force-push -- OK: 04763e8 is a linear ancestor of c18e353 (2 commits added) -- test file name deliberately left at upstream spelling -- all M, no R: no test file was renamed

### Scope constraints from the stated intent, checked against the diff

-- AGENTS.md must NOT be touched --
   OK: AGENTS.md byte-identical between 04763e8 and c18e353
   files changed: CONTRIBUTING.md tests/fm-captain-translation-contract.test.sh tests/fm-instruction-owners.test.sh tests/fm-no-mistakes-ownership.test.sh 

-- no history rewrite / force-push --
   OK: 04763e8 is a linear ancestor of c18e353 (2 commits added)

-- test file name deliberately left at upstream spelling --
M	tests/fm-captain-translation-contract.test.sh
M	tests/fm-instruction-owners.test.sh
M	tests/fm-no-mistakes-ownership.test.sh
   (all M, no R: no test file was renamed)

-- no assertion deleted or weakened: assertion counts per file --
   fm-captain-translation-contract.test.sh        base=26  target=25  ASSERTIONS LOST
   fm-instruction-owners.test.sh                  base=52  target=54  not weakened
   fm-no-mistakes-ownership.test.sh               base=4   target=4   not weakened

-- schema/flag tokens keep upstream 'captain' spelling (rename is prose-only) --
   literal "captain" occurrences in AGENTS.md: 14
   still upstream-spelled: bin/fm-bearings-snapshot.sh
   still upstream-spelled: bin/fm-decision-hold.sh
   still upstream-spelled: bin/fm-fleet-snapshot.sh
Evidence: Assertion-count reconciliation (26->25 dip explained; coverage rose)

Counting every site that can fail the test (assert_* plus explicit 'fail'): fm-captain-translation-contract.test.sh base=28 target=30 fm-instruction-owners.test.sh base=56 target=58 fm-no-mistakes-ownership.test.sh base=4 target=5 One literal substring check that could never match was replaced by a 3-term, case-insensitive, plural-tolerant word-boundary scan of the real ban sentence. Coverage went up; the raw assert_* count went down by one because the loop calls fail directly.

### Follow-up: the naive assert_* count dipped 26->25 in fm-captain-translation-contract.
### Counting every site that can fail the test (assert_* plus explicit 'fail'):

fm-captain-translation-contract.test.sh        base=28  target=30 
fm-instruction-owners.test.sh                  base=56  target=58 
fm-no-mistakes-ownership.test.sh               base=4   target=5  

### The single 'lost' assert_not_contains, before and after:
--- base 04763e8 ---
46:  assert_not_contains "$contract" "scout, ship" \
47-    "section 9 must not add scout to the internal-vocabulary ban"
--- target c18e353 ---
68:  ban=$(internal_term_ban)
69-  for term in scout secondmate vessel; do
70-    if printf '%s\n' "$ban" | grep -Eqi "(^|[^[:alnum:]])${term}s?([^[:alnum:]]|\$)"; then
71-      fail "section 9 added '$term' to the internal-vocabulary ban sentence"$'\n'"--- ban sentence ---"$'\n'"$ban"
72-    fi
73-  done

One literal substring check that could never match was replaced by a 3-term,
case-insensitive, plural-tolerant word-boundary scan of the real ban sentence.
Coverage went up; the raw assert_* count went down by one because the loop
calls fail directly. Section 04 shows the old form was GREEN under a hostile
edit that the new form catches.
Evidence: Fork blast radius + sweep of other AGENTS.md-reading tests

Files changed by the fork commit 04763e8: AGENTS.md => .agents/skills/** and bin/ were NOT touched by the fork, so the ~15 remaining captain/Firstmate assertions that target them still describe those files truthfully.

### Did the fork touch anything the untouched tests describe?
Files changed by the fork commit 04763e8:
   AGENTS.md

=> .agents/skills/** and bin/ were NOT touched by the fork, so the ~15 remaining
   captain/Firstmate assertions that target them still describe those files truthfully.

### Sweep: any OTHER test asserting AGENTS.md wording that the rename could have made stale or vacuous?
   fm-instruction-owners.test.sh                  AGENTS.md-reading, captain/Firstmate mentions=18
   fm-captain-translation-contract.test.sh        AGENTS.md-reading, captain/Firstmate mentions=8
   fm-no-mistakes-ownership.test.sh               AGENTS.md-reading, captain/Firstmate mentions=1
   fm-bootstrap.test.sh                           AGENTS.md-reading, captain/Firstmate mentions=2
   fm-stow-contract.test.sh                       AGENTS.md-reading, captain/Firstmate mentions=0
   fm-supervision-instructions.test.sh            AGENTS.md-reading, captain/Firstmate mentions=0
Evidence: Adjacent AGENTS.md-reading suites still pass

fm-bootstrap exit=0 (21 ok, 0 not ok) fm-stow-contract exit=0 (2 ok, 0 not ok) fm-supervision-instructions exit=0 (10 ok, 0 not ok)

### The one remaining AGENTS.md-reading test with captain/Firstmate mentions:
6:# all is well. firstmate consumes the exact 'MISSING: treehouse (install: ...)',
670:    printf 'window=firstmate:fm-sm\n'

=> both are fixture/path spellings, not AGENTS.md prose assertions - unaffected by the rename.

### Regression: the other three AGENTS.md-reading suites still pass
   fm-bootstrap                       exit=0  (21 ok, 0 not ok)
   fm-stow-contract                   exit=0  (2 ok, 0 not ok)
   fm-supervision-instructions        exit=0  (10 ok, 0 not ok)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

⚠️ **Review** - 1 info
  • 🚨 tests/fm-no-mistakes-ownership.test.sh:8 - The intent states as required: "(1) sibling-extractor-unguarded - validate_contract in tests/fm-no-mistakes-ownership.test.sh has the same silent-empty extractor pattern, anchoring on /^### Validate$/, and needs the same explicit heading-found guard" — "which I still want applied". The diff does not add it. validate_contract() (lines 8-14) still returns an empty string with exit 0 if the ### Validate heading is ever renamed or removed, so both assert_contains calls would fail with a misleading "Validate contract does not assign..." message instead of the real cause. This is the exact defect assert_section_9_found was added to fix in the sibling file, and the change was made in this same file (test renamed at line 32) without adding the guard.
  • 🚨 tests/fm-captain-translation-contract.test.sh:59 - The intent states as required: "(2) vacuous-scout-ban-assertion - assert_not_contains &#34;scout, ship&#34; ... must be re-anchored on the actual 'Do not expose internal terms such as ...' ban sentence so that appending scout would genuinely turn it red." The diff leaves line 59 untouched. I confirmed the vacuity: awk extraction of section 9 from both the current AGENTS.md and 04763e8^:AGENTS.md contains zero occurrences of the substring "ship", so assert_not_contains &#34;$contract&#34; &#34;scout, ship&#34; can never fire. Adding "scout" to the real ban sentence at AGENTS.md section 9 line 6 ("Do not expose internal terms such as startup machinery, locks, watchers, ...") would produce "scout, locks" or similar and the test would still pass green. An assertion that cannot fail is precisely what this branch exists to remove.
  • ⚠️ tests/fm-instruction-owners.test.sh:234 - Same defect class as the accepted vacuous-scout-ban finding, in a file this change already touches. assert_no_grep &#39;Firstmate does not personally review code or deliverables&#39; &#34;$AGENTS&#34; (line 234) and assert_no_grep &#39;firstmate reviews your branch&#39; &#34;$AGENTS&#34; (line 236) are regression guards against re-introducing a weaker personal-review requirement into AGENTS.md. After the fork, AGENTS.md contains no "Firstmate" at all, so a realistic regression would be written as "StaffCaptain does not personally review code or deliverables" and neither guard would fire. The positive assertion three lines above (line 235 in the diff) was updated to the fork vocabulary; these two negative guards were not, so they now silently protect nothing. Either re-anchor them on the fork's spelling or make them vocabulary-agnostic.

🔧 Fix: tighten three vacuous AGENTS.md contract assertions in tests
1 info still open:

  • ℹ️ tests/fm-instruction-owners.test.sh:239 - The new guard enumerates four fixed-string casings (StaffCaptain, 'Staff Captain', staffcaptain, 'staff captain') and relies on assert_no_grep, which is grep -F and therefore case-sensitive. A regression written as 'Staffcaptain does not personally review code or deliverables' or 'STAFF CAPTAIN reviews your branch' would still slip through. A single case-insensitive check (grep -Eqi 'staff ?captain (does not personally review code or deliverables|reviews your branch)') would close the whole class and replace eight grep invocations with one. Noting it as a tradeoff rather than a defect: the explicit variant list is readable and covers every realistic spelling, and collapsing it would re-word an assertion the captain just directed be added, so leaving it as-is is defensible.
✅ **Test** - passed

✅ No issues found.

  • command -v tmux >/dev/null || { echo "tmux is required for e2e tests" >&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t =="; bash "$t" || rc=1; done; exit "$rc"
  • bash tests/fm-captain-translation-contract.test.sh — 7 ok / 0 not ok
  • bash tests/fm-instruction-owners.test.sh — 10 ok / 0 not ok
  • bash tests/fm-no-mistakes-ownership.test.sh — 2 ok / 0 not ok
  • Red-baseline repro: git show 04763e8:tests/&lt;each&gt;.test.sh run against the fork's AGENTS.md in a symlink sandbox — all three exit 1
  • Mutation matrix of 15 single-contract perturbations of AGENTS.md (A1–A6, D1–D2 = the 8 changed assertion sites; B1–B2 = re-anchored scout/vessel ban; C1–C3 = new heading guards; E1–E2 = new negative guards) — 15/15 caught with the expected failure message
  • Vacuous-assertion before/after: base-commit assert_not_contains &#34;$contract&#34; &#34;scout, ship&#34; vs the re-anchored ban scan, both applied to an AGENTS.md that appends scout to the internal-term ban sentence
  • Scope verification: git diff --quiet 04763e8 c18e353 -- AGENTS.md, git merge-base --is-ancestor, git diff --name-status --find-renames -- tests/, git show --stat --name-only 04763e8
  • Assertion-count reconciliation across both commits (assert_* plus explicit fail sites)
  • Sweep for other AGENTS.md-reading suites the rename could have made stale, then bash tests/fm-bootstrap.test.sh, bash tests/fm-stow-contract.test.sh, bash tests/fm-supervision-instructions.test.sh — all exit 0
  • git status --porcelain after teardown — clean
⚠️ **Document** - 2 infos
  • ℹ️ docs/fork-patches.md:3 - docs/fork-patches.md states "Each fork-only, non-merge commit has one row", but its last row is PR #4 (8f3190b) with ad9f3a7 as the last reviewed upstream tip, while roughly a dozen later fork-only commits (a8ce717 #7 through b9cb8bf #18, plus the three StaffCaptain-fork commits) have no rows. This drift predates this change and reconciling it needs a real upstream-tip comparison via bin/fm-fork-sync-check.sh, which is out of scope here; my edit only makes the vocabulary divergence discoverable from the owner doc. Suggested follow-up: one task that re-runs the fork-sync comparison and refreshes the registry against the current real-upstream tip.
  • ℹ️ .agents/skills/afk/SKILL.md:54 - AGENTS.md now requires addressing the user as "Commodore", while .agents/skills/** (afk, bootstrap-diagnostics, decision-hold-lifecycle, stuck-crewmate-recovery, updatefirstmate, harness-adapters) still instruct the agent to say "Captain" and reference "firstmate". This is the halted phase-2 fork-completeness gap the intent declares out of scope, and the new CONTRIBUTING bullet's "in AGENTS.md prose only" wording correctly bounds the rename, so I left it alone. Noting it only so the contradiction stays visible if phase 2 ever resumes before archival.
✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

Main has been red since 04763e8 forked AGENTS.md into StaffCaptain form.
That commit renamed the fleet-level human authority from captain to
Commodore and the orchestrator from Firstmate to StaffCaptain, in prose
only. Three inherited behavior tests assert AGENTS.md wording verbatim
and still expected the upstream spelling, so the instruction was correct
and the tests were the stale artifact.

Update the eight drifted assertion sites to the fork's own vocabulary,
keeping every contract they pin intact:

- fm-captain-translation-contract: the section 9 heading anchor plus the
  translation contract, nouns line, house-vocabulary line, verbatim-chat
  ban, and chat-summary rule.
- fm-instruction-owners: the separate-review-or-audit authority clause.
- fm-no-mistakes-ownership: the orchestrator-never-responds prohibition.

The section 9 extractor silently returned nothing once the heading was
renamed, which surfaced as a cascade of twenty-six "missing contract"
failures instead of the real cause. Assert the heading was found first so
a future rename fails for the right reason.

Assertions targeting .agents/skills/ and bin/ are deliberately unchanged:
the fork did not touch those files, so they still say captain/Firstmate
and the tests correctly describe them. File names and schema tokens
(data/captain.md, tasks-axi --kind captain, captain-hold) keep their
upstream spelling for the same reason.

Each changed assertion was verified to still fail when the contract it
pins is removed from AGENTS.md, and to fail when AGENTS.md is reverted to
the upstream wording.

Record the rename and its trap in CONTRIBUTING.md so a later upstream
fast-forward does not silently restore the inherited phrasing.
@Freudator86
Freudator86 merged commit cf6085b into main Jul 28, 2026
5 checks passed
@Freudator86
Freudator86 deleted the fm/sc-fork-red-main branch August 6, 2026 03:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant