Replace Pa11y with axe-core + add MCP elicitation auto-check#91
Merged
lessthanseventy merged 34 commits intoMar 25, 2026
Conversation
- #84: Fix IO.Stream crash in build_markdown_report by capturing System.cmd output as string instead of streaming to IO.stream. Also propagate non-zero exit codes from test runs. - #83: Catch exit signals in LiveView.get_assigns/1 so unsupported :get_state calls (e.g. LiveView 1.1.22+) return {:error, :unsupported} instead of crashing. - #80: Add file size guards to MCP timeline tools to prevent crashes on large timeline files. get_timeline truncates to last 200 events, analyze_timeline rejects files over 512KB with a helpful message. - #85: Add BitString implementation for Excessibility.Source protocol to support component-level snapshot testing with rendered_to_string. Closes #84, closes #83, closes #80, closes #85 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace Pa11y + ChromicPDF with Playwright + axe-core. Simplify MCP from 11 tools to 3, 4 resources to 2, 9 prompts to 0. Add mix excessibility.snapshots task. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
11 tasks across 6 phases: new engine, swap mix tasks, remove Pa11y + ChromicPDF, simplify MCP surface, update docs, verify. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replaces Pa11y CLI with a small Node.js script that uses Playwright to load URLs and @axe-core/playwright to run accessibility checks. Supports file:// and http:// URLs, screenshots, wait-for selectors, and rule disabling. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Wraps assets/axe-runner.js with an Elixir module that supports file:// and http:// URLs, screenshots, wait-for selectors, and rule disabling. Includes tests covering violations, accessible HTML, screenshots, disable_rules, and error handling. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace Pa11y with AxeRunner in the mix excessibility task. The task now calls Excessibility.AxeRunner.run/2 for each snapshot file and formats violations with axe-core impact levels and help URLs. Adds support for :axe_disable_rules app env config. Removes pa11y_path, pa11y_config_args, and dependency_root helpers. Tests updated to use mock axe-runner.js scripts instead of mock Pa11y scripts. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Replace Pa11y npm install with npm install + npx playwright install chromium - Remove ensure_pa11y_config and pa11y.json creation from installer - Update all MCP tools (suggest_fixes, list_violations, check_route, e11y_check) to reference axe-core instead of Pa11y - Update MCP config resource to use axe_runner_path/axe_disable_rules - Update moduledocs and descriptions across all affected files - Maintain backwards compatibility: MCP schemas still accept pa11y_output and run_pa11y keys as fallbacks for existing clients - Update test expectations to match new schema keys Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Simplifies MCP surface. Remaining tools: get_snapshots, get_timeline, generate_test. These will be joined by new a11y_check and debug tools. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Re-add :screenshot? option to html_snapshot using AxeRunner - Format all files with mix format - All 475 tests pass, 0 credo issues Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- axe-runner.js: use real user agent, `load` instead of `networkidle`, SPA content polling for hydration detection - axe-crawl.js: multi-page crawl script for job application flow (careers → job listing → apply page) - AxeRunner: automatic curl fallback when Playwright fails (timeout, WAF block, protocol error). Fetches HTML via curl and scans locally. Result includes :fallback key when curl was used. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Breaking: Pa11y + ChromicPDF replaced with Playwright + axe-core, MCP surface simplified from 11 tools to 5. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Only include package.json, axe-runner.js, and axe-crawl.js from assets/ instead of the entire directory. Users run npm install themselves. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace links to CLAUDE.md and docs/telemetry-analysis.md (not included in hex package) with inline references. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add Sec-Fetch-*, Sec-Ch-Ua, Accept-Language, and other headers that mimic a real Chrome browser fingerprint. Helps bypass WAFs that reject plain curl requests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Design for adding MCP elicitation support to the server and creating an automated a11y/perf checking workflow via CLAUDE.md instructions, replacing the .claude_docs approach. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
7-task TDD plan covering elicitation module, server wiring, threshold-based triage in a11y_check, check_work composite tool, installer CLAUDE.md migration, and README updates. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implements the elicitation protocol (JSON-RPC 2.0) with build_request/3, parse_response/1, and build_callback/2 for tools to ask users questions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add capability negotiation so the server advertises elicitation support
only when the client declares it. Store client support in GenServer state
and pass an elicit callback to tools when available.
- Add client_supports_elicitation field to server struct
- Make handle_call support {response, new_state} tuples from initialize
- Build elicit callback via Elicitation.build_callback for tool opts
- Update call_tool to accept and thread state through
- Document :elicit option in Tool behaviour
- Add 5 tests for elicitation capability negotiation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When critical/serious accessibility violations are found and an elicit callback is available, prompt the user to choose: fix all, fix critical only, show details, or skip. Minor-only violations pass through without elicitation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Runs tests, accessibility checks, and optional performance analysis in a single call. Designed to be invoked by Claude after code changes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the .claude_docs/excessibility.md approach with appending an ## Excessibility section to the project's CLAUDE.md file. This follows the standard Claude Code convention and provides concise guidance for MCP tools and skills. Delete the standalone setup_claude_docs mix task as it is no longer needed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove deprecated Claude Documentation section and setup_claude_docs references. Add check_work to MCP tools table, replace recommended workflow with auto-check workflow description, update setup instructions to reference CLAUDE.md, and add optional hooks section. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extract pattern-matching function heads to replace nested cond+case. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…cit callback - Add 6 tests for CheckWork.extract_violations/1 covering all branches: structured violations, success status, error status, and fallback - Add comment in A11yCheck explaining why elicitation is URL-only (mix task path returns raw text, not structured violation data) - Cache the elicit closure in Server struct during initialize instead of rebuilding it on every tool call Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…on-and-mcp-elicitation # Conflicts: # lib/excessibility/mcp/tools/analyze_timeline.ex
- Add Node.js setup and npm install + playwright install chromium to CI - Fix a11y_check test assertion to match actual error message Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Extract run_with_fallback/3 and process_curl_result/5 from axe_runner to reduce nesting depth - Alias Mix.Tasks.Excessibility.Snapshots as SnapshotsTask in test Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
check_workcomposite tool runs tests + a11y + optional perf analysis. Installer adds CLAUDE.md instructions so Claude runs it automatically after modifying code..claude_docs/approach with direct CLAUDE.md section. Deletemix excessibility.setup_claude_docstask.Closes
Closes #80, closes #83, closes #84, closes #85
What changed
axe-core migration (from PR #90)
mix excessibilityfrom Pa11y to axe-coremix excessibility.checkfor arbitrary URL checkingmix excessibility.snapshotsfor snapshot managementMCP elicitation + auto-check (new)
Excessibility.MCP.Elicitationmodule for the elicitation protocola11y_check— only interrupts for critical/serious violationscheck_workcomposite tool (tests + a11y + optional perf)Test plan
🤖 Generated with Claude Code