Agent Performance Report — Week of May 25–31, 2026

[amalgamated-bot]

This report covers 13 agent workflows active in the Biblioteka repository as of 2026-05-31. Data was collected from workflow run history, PR merge rates, issue tracking, and behavioral pattern analysis via the pattern-detector sub-agent.

Key update since yesterday: Code Simplifier has recovered its run success rate (now 10/10 ✅) after being reported as a silent failure. However, PR merge rate remains 0%, indicating a scope-creep pattern. Two new agents joined the ecosystem: Efficiency Improver (first PR merged successfully) and PR Triage Agent (currently in first run).

Executive Summary

• Agents analyzed: 13
• Average quality score: 57/100 (↓ 6pts from May 30 — scope expansion)
• Average effectiveness score: 57/100
• Clean agents: 3/13 — Contribution Check, Go Fan, Efficiency Improver
• Top performers: Contribution Check, Go Fan, Efficiency Improver
• Needs improvement: Architecture Guardian, Code Refiner, Dead Code Remover, Caveman Optimizer

---

Performance Rankings

Top Performing Agents 🏆

1. Contribution Check (Quality: 98/100, Effectiveness: 98/100)

   • 100% run success rate, runs every 4 hours
   • Consistent, well-structured reports with clear pass/fail signals
   • Zero behavioral issues detected
   • Example outputs: [Contribution Check Report] Contribution Check — 2026-05-31 #3122, [Contribution Check Report] Contribution Check — 2026-05-30 #3118, [Contribution Check Report] Contribution Check — 2026-05-30 #3121

2. Go Fan (Quality: 85/100, Effectiveness: 88/100)

   • 100% run success rate, 90% PR merge rate via Copilot SWE agent
   • Previously exhibited over-creation pattern — fully resolved
   • High-value outputs: recent merged PRs include asynq priority queue improvements
   • Example outputs: feat(worker): asynq improvements — priority queues, opt-in Unique, slog adapter, trace propagation, graceful shutdown #3079, refactor(calibre): deduplicate import and web import orchestration #3075, refactor(handlers): deduplicate paginated named-entity list handlers #3055

3. Efficiency Improver (Quality: 80/100, Effectiveness: 85/100)

   • New agent — first PR [efficiency-improver] feat(middleware): add gzip HTTP response compression #3116 merged successfully (gzip HTTP compression)
   • 100% merge rate on first attempt; promising trajectory
   • Too early for full pattern assessment but no negative signals

Agents Needing Improvement 📉

1. Architecture Guardian (Quality: 35/100, Effectiveness: 25/100)

   • Intermittent failure: 3/10 runs succeed (30% success rate)
   • Failing since ~May 15 with no remediation progress (Day 16+)
   • Pattern: inconsistency — alternating success/failure with no apparent trigger
   • Recommendation: Run gh aw compile, check for config drift or external API dependency issues

2. Code Refiner (Quality: 0/100, Effectiveness: 0/100)

   • All runs cancelled or skipped — 0/10 success
   • Triggered on PR events but consistently cancelled before completion
   • Pattern: under-creation — zero effective output despite many trigger attempts
   • [agent-perf] Code Refiner - 100% cancellation rate (3/3 runs) #3111 was closed, but root cause (cancellation trigger) unresolved
   • 1 open PR ([refiner] style(middleware): fix gzip doc comments and use otelkeys.Error (#3116) #3119) created but not merged
   • Recommendation: Investigate cancellation trigger — likely concurrency limit or missing job dependency

3. Dead Code Remover (Quality: 30/100, Effectiveness: 20/100)

   • 0/2 PRs merged — both closed without merge
   • Runs appear to succeed but outputs are consistently rejected
   • Pattern: under-creation — runs but produces no net-effective change
   • Recommendation: Review PR rejection reasons; add quality gate before PR creation

4. Caveman Optimizer (Quality: 45/100, Effectiveness: 30/100)

   • 3/11 PRs merged (27% merge rate) over history
   • Low-quality suggestions frequently rejected
   • Patterns: scope-creep + inconsistency
   • Recommendation: Narrow scope to high-confidence optimizations; add pre-submission check

5. Code Simplifier (Quality: 55/100, Effectiveness: 50/100)

   • ✅ Run success RECOVERED (10/10 success — silent failure resolved)
   • ❌ PR merge rate: 0% (0/2 recent PRs closed without merge)
   • Pattern: scope-creep + inconsistency — changes fall outside project acceptance criteria
   • Recommendation: Review closed PRs [code-simplifier] refactor(worker): use any instead of interface{} in slog adapter #3090, [code-simplifier] refactor(middleware): fix misplaced GzipMiddleware doc and simplify acceptsGzip #3126 to understand rejection patterns; constrain changeset scope

Inactive / New Agents

• PR Triage Agent: Currently in first run — under-creation pattern expected, too early to evaluate
• Agentic Maintenance: Under-creation pattern flagged (run count lower than expected given every-2h cadence) but functional

Behavioral Pattern Analysis

Pattern detection run on 2026-05-31 via pattern-detector sub-agent.

Agent	Patterns
Contribution Check	✅ Clean
Go Fan	✅ Clean
Efficiency Improver	✅ Clean
Testify Expert	repetition
Daily Documentation Updater	scope-creep
Code Simplifier	scope-creep, inconsistency
Caveman Optimizer	scope-creep, inconsistency
Function Namer	inconsistency
Architecture Guardian	inconsistency
Code Refiner	under-creation
Dead Code Remover	under-creation
PR Triage Agent	under-creation (new)
Agentic Maintenance	under-creation

Pattern Summary:

• ✅ Clean: 3 agents (23%)
• ⚠️ Scope-creep: 3 agents — outputs regularly rejected as out-of-scope
• ⚠️ Inconsistency: 4 agents — highly variable success/quality rates
• ⚠️ Under-creation: 4 agents — not producing expected output volume/impact
• ⚠️ Repetition: 1 agent — duplicate task generation

PR Merge Rate Analysis

Agent Group (branch prefix)	PRs	Merged	Closed	Merge Rate
efficiency (Efficiency Improver)	1	1	0	100% ✅
copilot (Go Fan, Testify Expert, Function Namer)	10	9	1	90% ✅
dependabot	13	10	3	77% ✅
docs (Daily Doc Updater)	3	1	2	33% ⚠️
caveman (Caveman Optimizer)	11	3	8	27% ⚠️
code-simplifier (Code Simplifier)	2	0	2	0% ❌
chore (Dead Code Remover)	2	0	2	0% ❌
refiner (Code Refiner)	1	0	0	0% (open) ❌

The Copilot SWE agent (used by Go Fan, Testify Expert, Function Namer) produces the highest quality PRs at 90% merge rate, indicating the agent-as-orchestrator pattern with Copilot execution is highly effective.

Ecosystem Coverage

Well-Covered Areas

• Code quality review (Contribution Check, Code Refiner — though Refiner offline)
• Go module updates (Go Fan)
• Documentation (Daily Documentation Updater)
• Testing quality (Testify Expert)

Gaps

• Security vulnerability tracking (no active security agent)
• Performance optimization (Efficiency Improver is new but shows promise)
• Architecture enforcement (Architecture Guardian intermittent)

New Additions

• Efficiency Improver: Fills performance optimization gap — strong debut
• PR Triage Agent: Fills PR review automation gap — first run today

Behavioral Patterns

Productive Patterns ✅

• Go Fan → Copilot SWE execution: High-quality analysis + execution delivers 90% merge rate
• Contribution Check cadence: Every-4h reporting provides consistent health signal
• Efficiency Improver debut: First PR (gzip compression) merged immediately — right-sized scope

Problematic Patterns ⚠️

• Code Refiner cancellation loop: Triggered on every qualifying PR but always cancelled — consuming trigger quota with no output
• Caveman + Code Simplifier scope-creep: Both agents create PRs that get systematically closed — wasted effort and PR noise
• Architecture Guardian instability: 30% success rate with no apparent fix applied after 16+ days

Recommendations

High Priority

1. Diagnose Code Refiner cancellation — 0% success, all runs cancelled on PR trigger

   • Check concurrency limits on PR-triggered workflows
   • Verify job dependencies and trigger conditions
   • Estimated effort: 1-2 hours | Expected: restore functional agent

2. Scope Architecture Guardian investigation — 30% success, Day 16+ without fix

   • Previous open issue [aw] Architecture Guardian failed #3033; run gh aw compile to check config
   • Add error logging to distinguish flaky API from config issue
   • Estimated effort: 2-4 hours | Expected: restore to 90%+ success

3. Review Code Simplifier PR rejection reasons

   • PRs [code-simplifier] refactor(worker): use any instead of interface{} in slog adapter #3090 and [code-simplifier] refactor(middleware): fix misplaced GzipMiddleware doc and simplify acceptsGzip #3126 closed — understand whether it's code style, scope, or correctness
   • Add pre-submission lint/test validation gate
   • Estimated effort: 1 hour | Expected: improve 0% → 50%+ merge rate

Medium Priority

4. Add Testify Expert deduplication ([testify-expert] Quality Regression: Issue #3046 Output is Empty Placeholder #3052 existing issue)

   • Agent creating duplicate improvement issues for same files
   • Add seen-file tracking between runs
   • Estimated effort: 1-2 hours | Expected: eliminate duplicate issue creation

5. Narrow Caveman Optimizer scope

   • Only 27% merge rate — refine prompt to target high-confidence, low-risk optimizations
   • Estimated effort: 30 min prompt change | Expected: improve to 50%+

6. Trigger Metrics Collector — 57+ days stale (last: 2026-04-04)

   • All meta-orchestrators relying on stale April 4 baseline
   • Critical for accurate trend analysis

Low Priority

7. Review Dead Code Remover PR quality — both closed, investigate whether detecting truly-dead or still-referenced code
8. Close stale recovery issues [aw] Daily Documentation Healer failed #3022, [aw] Daily Documentation Updater failed #3031 — agents stable for 5+ days

Trends (May 30 → May 31)

Metric	May 30	May 31	Δ
Average quality	63/100	57/100	↓ -6 (scope expansion)
Average effectiveness	67/100	57/100	↓ -10 (scope expansion)
Clean agent %	53% (10/19)	23% (3/13)	↓ (new scope — note denominator change)
Architecture Guardian success	3/10	3/10	→ stable (failing)
Code Refiner success	0/10	0/10	→ stable (broken)
Code Simplifier run success	failing	10/10	✅ RECOVERED
Copilot SWE PR merge rate	90%	90%	→ stable

Actions Taken This Run

• Created this performance report discussion
• Pattern detection via pattern-detector sub-agent (13 agents analyzed)
• Updated shared memory (agent-performance-latest.md, shared-alerts.md)
• No new improvement issues created — existing issues [aw] Architecture Guardian failed #3033, [testify-expert] Quality Regression: Issue #3046 Output is Empty Placeholder #3052, [Agent Fix] Add Output Validation Gate to Daily Caveman Optimizer #3062, [Agent Fix] Restrict Function Namer Scope - Part 1: Single-Letter Variables Only #3063, [Agent Fix] Function Namer Part 2: Add Multi-Heuristic Quality Gate #3065, 🚨 [SECURITY] Security Red Team Workflow Configuration Issue - Target Directories Not Found #3066 cover outstanding items

Next Steps

1. 🚨 Diagnose Code Refiner cancellation loop (root cause unknown)
2. 🚨 Fix Architecture Guardian (16+ days intermittent — escalate)
3. ⚠️ Constrain Code Simplifier and Caveman Optimizer scope
4. ⚠️ Add Testify Expert deduplication
5. 📊 Trigger Metrics Collector to refresh 57-day-stale baseline
6. 👀 Monitor PR Triage Agent first run results

---

Analysis period: 2026-05-24 to 2026-05-31
Previous report: §26684844700 (May 30)
Next report: 2026-06-07
Metrics baseline: 2026-04-04 (57 days stale — Metrics Collector needs manual trigger)

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• localhost

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "localhost"

See Network Configuration for more information.

Generated by Agent Performance Analyzer - Meta-Orchestrator · sonnet46 2.8M · ◷

Add this agentic workflows to your repo

To install this agentic workflow, run

gh aw add github/gh-aw/.github/workflows/agent-performance-analyzer.md@c88e1268ab92fe509e85c0bef376884880613be2

• expires on Jun 2, 2026, 1:28 PM UTC