test_runner convention-scope error recommends the blocked scope "all"; the resulting SKIPPED verdict is scored as stage_b_failed and deletes the reviewer approval

[andrewjfiore]

Summary

Two linked problems in the test_engineer path.

1. When test_runner is called with scope: "convention" and no files, its error text recommends scope: "all". But scope: "all" is blocked unless SWARM_ALLOW_FULL_SUITE is set, and the test_engineer prompt itself forbids it. An agent that follows the tool's own advice cannot succeed, and a model that keeps omitting files loops until the repetition breaker fires.
2. The verdict the agent then reports, [TESTED] ... SKIPPED, is treated as stage_b_failed. That clears the reviewer's APPROVED proof and moves the task to rework_required. A tool-argument problem is scored as a code failure and forces a coder rework of code that is correct.

Version

opencode-swarm@7.179.1 on OpenCode 1.18.21, Windows 11. The relevant files are byte-identical on the 7.180.0 release branch.

Reproduction

1. Python project with requirements.txt naming pytest, one passing test file test_calc.py.
2. Dispatch test_engineer with a normal Stage B delegation that names the test file but does not spell out tool arguments.
3. With one model (Kimi K2.7 Code in our runs) the agent calls test_runner with scope: "convention" and no files, 10 to 11 times, until the breaker stops it. Each call returns:

scope "convention" and "graph" require explicit files or targets array - omitting both causes unsafe full-project discovery

with a hint to use scope: "all".

4. The agent reports VERDICT: FAIL [0/0 tests run] or SKIPPED.
5. The task moves to rework_required and the reviewer's approval is deleted, although python -m pytest passes.

Two other models (DeepSeek V4.1 Flash, GLM-5.3 Flash) made one correct call with files: ["test_calc.py"] from the identical prompt, so the loop is model-dependent, but the two plugin behaviors above turn a recoverable argument mistake into a workflow dead end.

Where it happens

• src/tools/test-runner.ts: the guard that blocks scope: "all" without SWARM_ALLOW_FULL_SUITE (around line 3150) and the error text that recommends "all" (around line 3169).
• src/agents/test-engineer.ts (around lines 91, 100, 215): the prompt marks scope: "all" as prohibited.
• src/hooks/delegation-gate.ts (around lines 5952-5966): any non-PASS TESTED verdict is handled as stage_b_failed.
• src/hooks/gate-evidence.ts (around lines 611-628): stage_b_failed clears the reviewer proof and sets rework_required.

Expected

• The error text for a missing files array should say to pass files, not recommend a scope that is blocked.
• SKIPPED (tests not run) should be distinguished from FAIL (tests ran and failed). A not-run result should leave the task in a retryable Stage B state without deleting the reviewer approval, so the architect can re-dispatch the test gate instead of a coder.

Impact

Correct, reviewed code is sent back for rework, and (with issue #2755 below) the task cannot recover from that state without a human command.

Related: #2174 (SKIPPED for named test targets, a different cause with the same symptom).

[zaxbysauce]

@andrewjfiore on it. Thanks!

[zaxbysauce]

Fix published in #2766 (fix/issue-2756-skipped-verdict-retryable, head 13b5e7b1).

What changed

1. test_runner guard-2 remediation (src/tools/test-runner.ts): the missing-files error message now directs the caller to pass a non-empty files array (or targets for framework-native names) with a concrete example, and never mentions scope "all" — the scope the sibling guard blocks and the prompt prohibits. The error field and both guards' semantics are unchanged.
2. TESTED SKIPPED is retryable, not a failure:
   • Foreground (src/hooks/delegation-gate.ts): a [TESTED] | task-N | SKIPPED | ... verdict logs a warn and leaves the task in its Stage B eligible state — no stage_b_failed, no rework_required, reviewer APPROVED proof and completion entry preserved. The architect re-dispatches the test gate instead of a coder rework.
   • Background (src/background/stage-b-gates.ts + completion-observer.ts): SKIPPED classifies as 'skip', the record is consumed with a dedicated skipped (tests not run) — re-dispatch the test gate advisory; no transition, no proof clearing.
3. Genuine FAIL and REVIEWED rejections keep the existing stage_b_failed → rework_required semantics (pinned by new and existing tests).

Validation — frozen acceptance checks C1-C3 RED→GREEN and C4 GREEN→GREEN (replayed independently by a fresh implementation reviewer and a separate final critic, including revert/mutation probes); 11 new regression tests; blast-radius per-file loop over all 146 test files importing the changed modules green; typecheck/biome/test-file-cap/mock-cleanup clean.

Notes — complementary to #2755's PR #2760 (this prevents the wrongful entry into rework_required; #2760 is the audited exit once there). No migration needed; behavior changes only on the SKIPPED verdict path and the guard-2 message text.