AI-powered CI failure classification — understand why your tests failed, not just that they failed.
Retrace analyzes failed CI test output and classifies the failure into categories such as real regressions and external dependencies, helping developers understand what happened without manually digging through logs.
CI failures are noisy.
A failed test does not always mean your code is broken. It could be caused by:
- A real regression in application logic
- An external API or service
- Network problems
- Temporary infrastructure issues
- Other environmental factors
Retrace uses AI to analyze the failure context and provide a clear classification with a confidence score.
Instead of spending time asking "Why did CI fail?", Retrace gives you a starting point immediately.
- 🤖 AI-powered failure classification
- 📊 Confidence scores for classifications
- 🔍 Analyzes test output and failure context
- 💬 Posts classifications directly to GitHub pull requests
- ⚙️ GitHub Actions integration
- 🚀 Designed for automated CI workflows
GitHub Actions
↓
Run tests
↓
Test fails
↓
Capture failure output
↓
Send failure to Retrace
↓
AI analyzes the failure
↓
Classification + confidence
↓
Post result to the Pull Request
Retrace can analyze a deterministic regression such as:
Expected: 1260
Received: 1400
and classify it as:
likely real regression
Confidence: 95%
Example explanation:
The test deterministically expects a total of 1260 but receives 1400, indicating a change in application logic rather than timing or external factors.
Retrace can be integrated directly into a GitHub Actions workflow.
👉 Retrace GitHub Actions Integration
The integration workflow:
- Checks out the repository
- Sets up Node.js
- Installs dependencies
- Runs the tests
- Sends the failure output to Retrace
- Receives the AI classification
- Posts the classification as a PR comment
- Fails the workflow if the tests failed
Workflow file:
.github/workflows/retrace-classify.yml
┌──────────────────┐
│ GitHub PR │
└────────┬─────────┘
│
▼
┌──────────────────┐
│ GitHub Actions │
└────────┬─────────┘
│
▼
┌──────────────────┐
│ Run Tests │
└────────┬─────────┘
│
Test failure
│
▼
┌──────────────────┐
│ Retrace │
│ Failure Analysis│
└────────┬─────────┘
│
▼
┌──────────────────┐
│ AI Classification│
└────────┬─────────┘
│
▼
┌──────────────────┐
│ PR Comment + │
│ Confidence Score │
└──────────────────┘
- JavaScript
- Node.js
- Jest
- GitHub Actions
- GitHub
- REST API
- AI
The companion repository contains a working example of the GitHub Actions integration:
👉 https://github.com/thuluxx/retrace-test
It demonstrates how Retrace can automatically classify a failing test inside a real CI workflow.
Traditional CI tells you:
❌ Test failed
Retrace aims to tell you:
❌ Test failed
Classification:
likely real regression
Confidence:
95%
Reason:
The failure is deterministic and indicates a
change in application logic.
That extra context helps developers decide what to investigate first.
- AI-powered CI failure classification
- GitHub Actions integration
- Automated PR comments
- Confidence scoring
- More failure categories
- Improved failure explanations
- Support for more CI platforms
- Historical failure tracking
- Team-level analytics
Retrace was built as a project focused on making CI failures easier to understand and act on.
For the GitHub Actions integration and demo, see: