VisualQAStage cannot support an aesthetic-improvement claim: candidate-only, end-to-end mode only, and silently degrades to a non-failing skip on three paths

[cryptoxdog]

Summary

src/stages/VisualQAStage.ts checks a deployed candidate URL in isolation. It never captures or compares a baseline, so a structural build PASS can be reported as visual improvement with no evidence supporting that claim.

Observed at 70f3f9a4.

Current behavior

ctx.qualityEvidence.visualQa = 'skipped';   // script missing        (line 33)
ctx.qualityEvidence.visualQa = 'skipped';   // dry run               (line 38)
ctx.qualityEvidence.visualQa = 'skipped';   // no deployment URL     (line 46)

All three are non-failing. The stage shells out to scripts/verify-visual-qa.mjs --url <deployUrl> and fails only when stdout contains the literal CRITICAL. A non-BuildError exec failure is caught and downgraded to a logged warning with visualQa = 'failed' (lines 60-66) — it does not stop the build.

It is also registered only for end-to-end mode (src/pipeline/FactoryExecutionPlan.ts:103), so local-proof and publish-proof runs have no visual gate at all.

Gaps

• No baseline capture, so no baseline-vs-candidate comparison.
• No sentinel matrix — a single URL, not a route/viewport set.
• No verdict vocabulary; passed/failed/skipped cannot express "not regressed" or "inconclusive".
• skipped does not block a downstream improvement claim.

Expected

Rendered baseline-vs-candidate comparison must be mandatory before REDESIGN_IMPROVE may emit IMPROVED or REVIEWABLE.

Minimum sentinel coverage: homepage desktop, homepage mobile, representative service page desktop, representative service page mobile, gallery/project-proof route if present, contact/conversion route.

Evaluate at least: visual hierarchy, image quality, authenticity/trust, image density, section rhythm, brand coherence, conversion clarity, CTA salience, mobile composition, and obvious rendering defects.

Verdicts: IMPROVED | NON_REGRESSED | REGRESSED | INCONCLUSIVE. Subjective by design — no fabricated numeric precision.

• Material REGRESSED → REVIEWABLE = false
• Blocking INCONCLUSIVE → REVIEWABLE = false
• Skipped visual QA → cannot yield IMPROVED
• Structural PASS alone → cannot yield IMPROVED

PlaywrightScreenshotCapturer (src/ingestion/ScreenshotCapturer.ts) already exists for baseline capture and should be reused.

Related

Depends on intent threading. The baseline side of the comparison comes from source-site ingestion, which is currently a no-op because no committed spec enables it — see the SourceAssetCorpus issue.