Skip to content

feat: add evidence-aware desktop comparison - #1

Merged
woai3c merged 1 commit into
mainfrom
feat/evidence-aware-comparison
Aug 18, 2026
Merged

woai3c merged 1 commit into
mainfrom
feat/evidence-aware-comparison

Conversation

@woai3c

@woai3c woai3c commented Aug 18, 2026

Copy link
Copy Markdown
Owner

Summary

  • Add a Desktop-first workflow for selecting and comparing any two compatible analysis records from History, without requiring a pinned reference.
  • Persist versioned capture conditions and fail closed when page health, routes, viewports, or capture settings are not comparable.
  • Compare supported design tokens plus conservatively matched layout, interaction-state, and responsive observations, with evidence references and explicit partial-coverage limitations.
  • Add reviewable token governance, privacy-safe Desktop persistence, CLI reliability improvements, and standards-compliant MCP stdio behavior.

Desktop comparison

  • Provide one Compare two analyses action with a clear earlier/later record picker and detected site names.
  • Report each category as changed, unchanged, or inconclusive; partial reliable evidence remains usable instead of invalidating unrelated categories.
  • Link results to captured evidence and use conservative cross-capture entity matching. Ambiguous or unmatched entities are excluded rather than reported as design drift.
  • Treat layout, interaction-state, and responsive comparison as observed evidence, not pixel regression or approved token-contract rules.
  • Allow reviewed, supported token changes to create versioned partial Design Contract revisions with audit history.

Capture reliability and privacy

  • Add a versioned Analysis Request and Capture Manifest covering viewports, access/depth/discovery settings, browser/emulation data, stabilization, coverage, and page health.
  • Treat maxPages as an upper bound: a site that exposes one healthy page with all planned viewports is complete even when the configured limit is higher. Actual selected-page or viewport failures still fail closed.
  • Sanitize persisted, logged, and displayed Desktop URLs while keeping SQLite migrations additive and idempotent.
  • Treat legacy records without the required manifest and health evidence as inconclusive.

Shared entry points

  • Keep the primary comparison workflow in the Desktop app.
  • Improve CLI diagnostics with actual browser-launch validation, explicit browser paths, structured exit codes, cancellation code 130, and parseable stdout.
  • Keep MCP stdio output to one JSON-RPC object per line and share the core analysis request schema.

Verification

  • pnpm test — 46 files / 300 tests passed
  • pnpm test:design-evidence — 2 files / 24 real-browser regression tests passed
  • pnpm test:e2e — Desktop and CLI builds plus 68 E2E tests passed
  • Desktop TypeScript check passed
  • CLI/MCP TypeScript check passed
  • ESLint and Prettier checks passed

Scope and limitations

  • Comparison is not pixel regression and does not classify observed changes as defects.
  • Controlled fixtures validate behavior and fail-closed rules; they do not prove universal accuracy on arbitrary live sites.
  • Section and component identity matching remains conservative. Ambiguous and unmatched evidence is not automatically treated as drift.
  • Layout, interaction-state, and responsive observations are not approved Design Contract rules.
  • Authorized live-site and end-user workflow validation is still required before claiming full product exit criteria.

Reviewer focus

  • Capture comparability and fail-closed behavior, especially page-limit upper-bound semantics
  • SQLite migration and URL-sanitization idempotence
  • Cross-capture entity matching and partial-coverage behavior
  • Desktop selection/dialog UX and preload/IPC boundaries
  • CLI exit behavior and MCP stdio framing

@woai3c
woai3c merged commit ea4e43b into main Aug 18, 2026
4 checks passed
@woai3c
woai3c deleted the feat/evidence-aware-comparison branch August 18, 2026 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant