Skip to content

feat: add diagnose-test-failure skill#3133

Draft
amanjeetsingh150 wants to merge 13 commits intomainfrom
pr/skill
Draft

feat: add diagnose-test-failure skill#3133
amanjeetsingh150 wants to merge 13 commits intomainfrom
pr/skill

Conversation

@amanjeetsingh150
Copy link
Copy Markdown
Collaborator

@amanjeetsingh150 amanjeetsingh150 commented Apr 4, 2026

Summary

  • Add Claude Code skill for diagnosing Maestro test failures from CI artifacts. This for us to quickly root cause issues on our E2E tests.

The skill requires to submit a job or artifacts directory which would allow claude to go through all the context around failed command and query maestro.log to see the failure, invoking this from maestro codebase also ensure we have context about code.

amanjeetsingh150 and others added 6 commits April 4, 2026 12:23
Embed the demo_app Flutter project directly in the e2e directory so
that it can be built from source in CI instead of downloading
pre-built artifacts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ifacts

- Remove "Set up demo_app workspace" copy steps from all jobs —
  run_tests now references e2e/demo_app/.maestro directly
- Android job builds APK from source via flutter build apk
- iOS job builds .app from source via flutter build ios --simulator
- Remove demo_app entries from manifest.txt (no longer downloaded)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
These workflow files were copied from the demo_app repo and don't
belong in the Maestro tree.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@amanjeetsingh150 amanjeetsingh150 marked this pull request as draft April 4, 2026 08:36
@amanjeetsingh150 amanjeetsingh150 changed the base branch from main to pr/contains-child-fix April 4, 2026 13:59
amanjeetsingh150 and others added 7 commits April 4, 2026 19:36
ls artifacts/commands-*.json

# Find which tests have FAILED commands
grep -l '"FAILED"' artifacts/commands-*.json
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would jq be more efficient in this stage, if it's available?

@amanjeetsingh150 amanjeetsingh150 force-pushed the pr/contains-child-fix branch 2 times, most recently from 1cb6436 to dc8308b Compare April 6, 2026 15:38
Base automatically changed from pr/contains-child-fix to pr/no-ai-artifacts April 6, 2026 19:29
Base automatically changed from pr/no-ai-artifacts to pr/organize-debug-output April 6, 2026 19:30
Base automatically changed from pr/organize-debug-output to pr/demo-app-from-source April 6, 2026 19:30
Base automatically changed from pr/demo-app-from-source to main April 6, 2026 20:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants