Canary — Cross-Application Visual Regression Testing

Canary is a visual regression testing harness for desktop and browser applications. It records user interactions, replays them against live applications, captures screenshots at checkpoints, and compares them against verified baselines to catch visual regressions across builds.

Like a canary in a coal mine — it warns you early when something looks wrong.

Architecture

Canary uses a two-process model:

• The Harness (canary.exe) — an external orchestrator that manages test lifecycle, input replay, screenshot comparison, and reporting.
• Workload Agents — thin plugins that live inside each target application and execute commands received over named pipes.

canary.exe (harness)  <-- Named Pipe IPC -->  Agent inside Rhino (.rhp plugin)
                      <-- Named Pipe IPC -->  PenumbraBridgeAgent (.NET) -- CDP --> Chrome

Features

• CLI + GUI: Command-line harness and Avalonia 11 GUI (FluentAvalonia dark theme) — migrated from WinForms 2026-05-27, see docs/features/canary-ui-avalonia.md
• Pixel diff + SSIM: Configurable tolerance, composite diff images
• HTML reports: Self-contained with embedded images, status badges
• JUnit XML: CI-compatible test output
• Input recording: Win32 hooks (mouse/keyboard) or CDP mouse events
• Programmatic camera: Deterministic camera positioning via scripted coordinates
• Supervised sessions: operator-driven debugging mode (canary session start or the Sessions nav tab) — boot a workload's target app under Canary supervision and capture screenshots on demand with Ctrl+Shift+C / Ctrl+Shift+A. See docs/features/supervised-session.md.
• 258+ unit tests across 13+ build phases

Quick Start

# Build
dotnet build Canary.sln

# Run unit tests
dotnet test tests/Canary.Tests/Canary.Tests.csproj --filter "Category=Unit"

# Run visual regression tests for a workload
canary run --workload penumbra

# Approve new baselines after intentional visual changes
canary approve --workload penumbra

# Open the HTML report
canary report

Abort

Press Ctrl+C at any time to kill the test run. The harness terminates all child processes and exits cleanly.

Workloads

Workload	Host Application	Agent Type	Status
Rhino	Rhino 8	RhinoCommon plugin (.rhp)	Done
Penumbra	Chrome via CDP	Browser bridge agent	Active
Qualia	Browser (Vite + Chrome via CDP)	Canary.Agent.Qualia	Active — 6 suites / 78 tests
Grasshopper	Rhino 8 + GH	RhinoCommon plugin (.rhp)	Planned

Project Structure

Canary.sln
src/
  Canary.Core/           # Shared library (comparison, config, orchestration, reporting)
  Canary.Harness/        # CLI entry point
  Canary.UI.Avalonia/    # Avalonia 11 + FluentAvalonia + CommunityToolkit.Mvvm GUI (outputs Canary.UI.exe)
  Canary.Agent/          # Shared agent interface (net8.0 + net48)
  Canary.Agent.Rhino/    # Rhino plugin (net48, outputs .rhp)
  Canary.Agent.Penumbra/ # CDP bridge for browser testing
tests/
  Canary.Tests/          # Unit tests
  Canary.Tests.Integration/
workloads/
  rhino/                 # Rhino smoke test definitions
  penumbra/              # Penumbra visual regression tests (5 test suites)
  qualia/                # Stub
docs/
  bugs/                  # Bug reports with queryable frontmatter
  decisions/             # Architecture Decision Records
  debug-sessions/        # Investigation journals
  features/              # Feature status tracking
  templates/             # Reusable doc templates
spec/                    # Frozen design documents (7 files)

Documentation

• CHANGELOG.md — version history
• BUILD_LOG.md — phase checkpoint records
• Feature Status — living feature tracker
• Creating a Workload — agent development guide
• Architecture Decisions — ADRs

Requirements

• Windows 10/11
• .NET 8.0 SDK
• Chrome or Edge (for Penumbra workload)
• Rhino 8 (for Rhino workload)

License

MIT