UX: unscoped-vs-scoped e2e mismatch should be reported once at the aggregate level, not per scenario

[vivganes]

Severity: Low · Type: enhancement (not a bug)

Source: PR #5 review — reproduced by running the tool at HEAD (20b0c6b).

Problem

By design, an unscoped result (config key "") never satisfies a module-scoped requirement (@require-e2e:checkout). When a repo registers e2e under "" while scenarios declare scoped requirements, every such scenario shows a red e2e(checkout) Missing chip next to a visible, passing, linked e2e result — with no hint that the unscoped-vs-scoped distinction is the cause.

I verified this by running the tool:

"requirements": [{"layer":"e2e","satisfied":false,"module":"checkout"}]
"results":      [{"layer":"e2e","name":"does something","status":"passed"}]

→ renders e2e (checkout) Missing, with no explanatory text anywhere in the report or JSON.

This is a discoverability gap, not a silent failure (the result and requirement are both visible; only the why is missing). The behavior itself matches the documented design (README:118/122, PRD §9).

Why NOT a per-requirement "reason" tooltip

The unscoped-vs-scoped mismatch is a config-level fact, not a per-scenario one. Across N scenarios it's the same root cause (e2e registered under ""). Emitting a "results exist but are unscoped" hint on every chip would:

• repeat the identical diagnosis N× (e.g. 200×),
• bury the only scenario-specific fact that matters (which module each scenario needs),
• train users to ignore the red chip.

Also, the original Fix proposed "extend the MISSING-reason tooltip (see #6)" — but that tooltip infrastructure does not exist (_layer_satisfied returns a bare bool; the JSON schema has no reason field; #6 is still open/unmerged). So the fix must not depend on #6.

Proposed fix (aggregate only)

Mirror the existing unlinked health check in aggregator.py:health_checks (which already emits one global reason instead of repeating per result):

1. Add a health-check rule that detects scoped requirements unsatisfied solely because matching results exist but are unscoped, and emits a single reason naming the count and affected modules, e.g.:

   "12 scenarios require module-scoped @require-e2e results, but e2e is registered under the unscoped key "". Add module keys to your e2e config (e.g. "checkout": [...]) to satisfy them."

2. Per-scenario chip stays minimal — keep e2e(checkout) Missing as today. Optionally add a hover-only title= attribute for genuinely odd cases (e.g. a single scenario whose module has zero results anywhere), but no visible per-scenario text in the common case.

3. (Nice-to-have) Distinguish in the aggregate reason whether the cause is (a) no results for the layer, (b) results under a different module, or (c) results unscoped — so the single message is precise rather than generic.

Evidence

requirement @require-e2e:checkout, result linked + passing under config key ''
  -> satisfaction: False (MISSING, by design) ; report shows no cause