Skip to content

feat(spawn): add preflight resolver to validate harness and project before worktree creation#7

Merged
ryxli merged 3 commits into
herdrfrom
fm/spawnres-e3
Jun 25, 2026
Merged

feat(spawn): add preflight resolver to validate harness and project before worktree creation#7
ryxli merged 3 commits into
herdrfrom
fm/spawnres-e3

Conversation

@ryxli

@ryxli ryxli commented Jun 25, 2026

Copy link
Copy Markdown
Owner

Intent

The developer wanted to evaluate their AI agent fleet from a productivity standpoint and fix the root cause of agents failing to parallelize file analysis - not through text-instruction band-aids that bloat context windows, but through a structural architectural solution. They opted in permanently to using the Workflow tool for multi-file analysis tasks and asked firstmate to help design a measurable productivity metric framework based on output rate and attention cost (escalations per task). They approved establishing a temporary weekly productivity log at data/productivity-log.md with a schema clean enough to later migrate to Linear or Notion. Finally, they decided to create a personal GitHub fork of the firstmate repository (ryxli/firstmate), directly merge the herdr migration branch into main, and update the local origin remote to point at their fork rather than the upstream kunchenguid/firstmate.

What Changed

  • Added bin/fm-resolve-spawn.sh, a preflight script that verifies the harness binary is on PATH, warns when the target project is not in the registry, and confirms the worktree base is writable - aborting before any git or herdr state is created on failure.
  • Wired the resolver into bin/fm-spawn.sh so every non-secondmate spawn runs the preflight automatically.
  • Added a 119-line test suite (tests/fm-resolve-spawn.test.sh) covering harness-missing, unregistered-project, and happy-path cases; fixed PATH coupling in tests/fm-spawn-batch.test.sh so batch tests no longer assume a specific harness is installed on the host.

Risk Assessment

✅ Low: Both round-1 findings were correctly fixed: the batch tests now create a hermetic fake harness binary instead of relying on omp being installed, and the redundant grep pattern was removed. No new issues introduced.

Testing

All 5 new resolver tests and all 7 batch tests pass. End-to-end verification confirms the preflight catches a missing harness binary with a clear error before any git worktree or herdr pane is created, and that unregistered projects emit a non-fatal warning while valid harness+project pairs pass silently.

Evidence: Spawn preflight CLI transcript
=== fm-resolve-spawn preflight behavior ===

Scenario 1: Missing harness binary is caught before spawn
$ fm-resolve-spawn.sh alpha codex   # codex not on PATH
error: spawn harness binary 'codex' was not found on PATH; check config/crew-harness or pass an available harness override
exit: 1

Scenario 2: Registered project with valid harness passes silently
$ fm-resolve-spawn.sh alpha omp     # omp is on PATH
exit: 0 (no output)

Scenario 3: Unregistered project warns but continues
$ fm-resolve-spawn.sh projects/unknown omp
warn: project 'unknown' does not appear in data/projects.md; continuing because direct paths are allowed
exit: 0

Scenario 4: fm-spawn aborts BEFORE creating worktree when harness missing
$ FM_SPAWN_NO_GUARD=1 fm-spawn.sh mytest-k7 projects/alpha codex   # codex not on PATH
error: spawn harness binary 'codex' was not found on PATH; check config/crew-harness or pass an available harness override
exit: 1
Worktree created: no - correct

=== Test suite results ===

tests/fm-resolve-spawn.test.sh:
ok - missing harness binary blocks before spawn
ok - unregistered project warns but does not block
ok - registered project passes quietly
ok - missing worktree base parent blocks
ok - fm-spawn aborts before creating a worktree or pane

tests/fm-spawn-batch.test.sh:
ok - batch dispatch re-execs and reports every id=repo pair
ok - a single id=repo pair routes through batch dispatch
ok - single-task invocation (no '=') is untouched by batch detection
ok - batch dispatch rejects an argument that is not id=repo
ok - an arg whose id part contains '/' is not treated as a batch pair
ok - FM_HOME scopes projects/ paths for single-task spawn
ok - FM_PROJECTS_OVERRIDE scopes projects/ paths for single-task spawn

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 3 issues found → auto-fixed ✅
  • ⚠️ tests/fm-spawn-batch.test.sh:96 - tests/fm-spawn-batch.test.sh lines 96 and 114 switched from 'codex' to 'omp' as the harness argument to keep reaching the brief-check stage after the new preflight. But unlike the new resolver tests (which create a fakebin directory), these two tests rely on 'omp' being present on PATH at test time. If CI runs without omp installed, the resolver exits early with "spawn harness binary 'omp' was not found on PATH" and the grep for "error: no brief at ..." fails, making the test assertion wrong rather than gracefully skipped. The resolver tests themselves demonstrate the correct pattern: create a fake harness binary in a temp dir and prepend it to PATH.
  • ℹ️ bin/fm-resolve-spawn.sh:52 - fm-resolve-spawn.sh line 52: the second grep -e pattern "-e &fix(watcher): lossless check wakes via watcher-side suppression + enqueue-before-suppress kunchenguid/firstmate#34;- ${project_name} [&fix(watcher): lossless check wakes via watcher-side suppression + enqueue-before-suppress kunchenguid/firstmate#34;" is redundant. Any registry line matching "- name [" also contains the literal "- name " (the space before '[' satisfies the first pattern), so the first -e always catches both cases. Only the first pattern is needed.
  • ℹ️ bin/fm-resolve-spawn.sh:31 - When no harness override is given, fm-harness.sh crew is now invoked twice per non-secondmate spawn: once inside fm-resolve-spawn.sh (line 31) and again inside fm-spawn.sh (line 131). This doubles the subprocess cost for the common code path. Not a correctness issue given spawning is a low-frequency operation.

🔧 Fix: fix batch test PATH coupling and redundant grep -e pattern
✅ Re-checked - no issues remain.

✅ **Test** - passed

✅ No issues found.

  • bash tests/fm-resolve-spawn.test.sh
  • bash tests/fm-spawn-batch.test.sh
  • Manual CLI transcript: fm-resolve-spawn.sh alpha codex with codex absent (exits 1 with actionable error)
  • Manual CLI transcript: fm-resolve-spawn.sh alpha omp with valid harness (exits 0 silently)
  • Manual CLI transcript: fm-resolve-spawn.sh projects/unknown omp unregistered project (exits 0 with warning)
  • Manual CLI transcript: fm-spawn.sh mytest-k7 projects/alpha codex with missing harness confirms no worktree is created on failure
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

@ryxli ryxli merged commit ab1c0ad into herdr Jun 25, 2026
@ryxli ryxli deleted the fm/spawnres-e3 branch June 25, 2026 08:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant