Skip to content

Bump axe-core from 4.12.1 to 4.14.0 in /sandbox/browser-web-v1 #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dependabot wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Bump axe-core from 4.12.1 to 4.14.0 in /sandbox/browser-web-v1 #39

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions sandbox/browser-web-v1/package-lock.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion sandbox/browser-web-v1/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"private": true,
"type": "module",
"dependencies": {
"axe-core": "4.12.1",
"axe-core": "4.14.0",

@devin-ai-integration devin-ai-integration Bot Oct 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Existing benchmark runs reject the upgraded evaluator

After rebuilding the template, axe-core changes its hash, but published benchmark versions retain the old environment hash. evaluateFrontendRun rejects their runs with environment_hash_mismatch, preventing evaluation.

Learn more

The evaluator template build hash includes the lockfile, so this dependency change produces a new template hash (template fingerprint). seedRankedCatalog hashes that template with the evaluator policy, but published benchmark versions keep their stored environment hash because catalog seeding uses onConflictDoNothing (benchmark insert). evaluateFrontendRun compares both stored hashes with the current configuration before starting a sandbox. The stale axeCore value in EVALUATION_ENVIRONMENT_V1 also makes the package-to-descriptor test fail.

Example: A benchmark published with template hash A has environment hash H(A). Deployment configures a newly built template with hash B. Its pending run still has H(A), so the evaluator rejects it before execution.

Recommended fix: Keep the existing v1 template available for existing runs, and introduce a new versioned environment policy, template, and benchmark versions for the upgraded axe-core. Route each run to its recorded environment; update the frozen-package assertion to check the corresponding policy.

Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

@devin-ai-integration devin-ai-integration Bot Oct 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Security impact absent from PR body

The PR body lists release notes but does not explain the security impact of changing the evaluator dependency, as CONTRIBUTING.md requires.

Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

"playwright": "1.62.0"
}
}
Loading