ci-detective

CI Failure Pattern Detector - Find recurring issues in your CI pipeline.

The Problem

Developers spend 20-40% of their time debugging broken CI pipelines. The same failures happen again and again:

• Flaky tests that pass on retry
• Dependency download timeouts
• Environment configuration drift
• Race conditions
• Resource exhaustion

But CI logs are cryptic, scattered across builds, and nobody connects the dots.

The Solution

ci-detective analyzes failed CI runs across your repos and identifies:

• Recurring failure patterns
• Flaky tests
• Common error signatures
• Suggested fixes based on similar failures

Status

✅ MVP Complete - Basic analysis working

Installation

# Requires gh CLI
brew install gh  # or see https://cli.github.com

# Run directly
chmod +x ci_detective.py
./ci_detective.py --help

Usage

Analyze failures in current repo

ci-detective

Analyze specific repo

ci-detective --repo owner/repo-name

Limit analysis to recent N failures

ci-detective --limit 100

Example Output

🔍 Analyzing last 50 failed CI runs...

Found 50 failed runs

============================================================
📊 ANALYSIS RESULTS
============================================================

🔴 Top Failure Patterns:
  • Timeout: 12 occurrences
  • Dependency Error: 8 occurrences
  • Assertion Error: 6 occurrences
  • Connection Error: 4 occurrences
  • Flaky Test: 3 occurrences

⚙️  Most Failing Workflows:
  • CI: 25 failures
  • Build and Test: 15 failures
  • Deploy: 6 failures
  • Lint: 4 failures

🌿 Most Failing Branches:
  • main: 18 failures
  • develop: 12 failures
  • feature/api-refactor: 8 failures

============================================================

💡 Analyzed 10 run logs out of 50 failed runs

Detected Patterns

The tool detects these common failure types:

• Flaky Test - Tests that fail intermittently
• Timeout - Operations that exceed time limits
• Dependency Error - Package download/resolution failures
• Out of Memory - Memory exhaustion issues
• Permission Denied - Access control problems
• Connection Error - Network connectivity issues
• Assertion Error - Test assertion failures
• Compilation Error - Build/syntax errors

Features

• GitHub Actions log fetching via gh CLI
• Pattern detection (8 common failure types)
• Workflow failure analysis
• Branch failure analysis
• GitLab CI support
• ML-based pattern detection
• Flaky test identification
• Historical trending (database)
• Suggested fixes database
• Slack/Discord notifications

Architecture (Draft)

ci-detective/
├── collectors/         # Fetch logs from CI providers
│   ├── github_actions.py
│   ├── gitlab_ci.py
│   └── base.py
├── analyzers/          # Pattern detection
│   ├── patterns.py
│   ├── flaky.py
│   └── clustering.py
├── database/           # Storage (SQLite for MVP)
│   └── schema.sql
└── cli.py              # Main CLI interface

Next Steps

1. Implement GitHub Actions log collector
2. Build pattern matching engine
3. Add flaky test detector
4. Create CLI interface
5. Test on real repos

License

MIT (planned)