follow-up: harden welcome eval relevance and live-signal stability

[Gandy2025]

Context

PR #2170 merged the first onboarding chat into a bounded, value-first in-chat work loop. Its release scope included the reproduced core-contract fixes: premature PR/MR menus, broad post-result scans, and a bridge that must be directly related to the completed result.

This issue tracks post-merge eval hardening. It must not be used to defer a reproduced user-visible safety or core-flow defect from a source fix.

Follow-up scope

• Make broad repository-scan detection aware of tool working directories and shell cd state. Equivalent repo-relative recursive scans such as cd source-repo && rg --files, (cd source-repo; find . -type f), and recursive commands executed with the tool workdir set to source-repo must be classified the same as explicit find source-repo ... commands.
• Generalize post-result bridge relevance coverage beyond checkout/payment fixture wording so generic verbs such as verify cannot satisfy relevance by themselves.
• Stabilize the tree-kickoff live-gate signal that produced one semantically compliant literal-signal false negative.
• Reduce periodic oracle false-positive/false-negative noise while preserving hard failures for broad scans, unauthorized writes, premature PR/setup, role leaks, and first-task fan-out.
• Promote reusable findings to the appropriate committed skill-eval or QA case disposition.

Acceptance

• Add red-to-green controls for explicit-path, shell-cd, subshell-cd, and tool-workdir repo-relative recursive scans; retain passing controls for bounded direct-reference reads.
• Add good, bad, and borderline bridge-relevance controls across at least three distinct repository domains.
• Demonstrate that a directly related next step passes while unrelated or invented adjacent paths fail, without fixture-brand coupling.
• Demonstrate stable tree-kickoff classification for semantically compliant outputs.
• Keep every existing safety and role/setup regression green.
• Document whether each resulting case belongs in deterministic skill eval, live periodic QA, or both.

Related: #2170

Review finding: #2170 (comment)

[Gandy2025]

Two additional post-merge review findings belong in this eval-hardening follow-up:

• Normalize command cwd and shell cd state so repo-relative recursive scans such as cd source-repo && rg --files, (cd source-repo; find . -type f), and recursive search invoked with workdir=source-repo are classified as broad scans.
• Require selected-task result evidence to resolve to fixture files and/or observed reads. Add a negative control for an invented path or an unsupported call-chain step so path-shaped text alone cannot satisfy the reviewable-result gate.

These are deterministic oracle gaps discovered after the phased source merge, not new observed user-flow regressions. They should preserve all existing no-scan, no-write, role/setup, first-menu, and current-chat-result protections while adding the focused red-to-green controls above.

The separate durable Context Tree drift is addressed by first-tree-context#891.