Add A3M Router as routing baseline#12
Conversation
A3M Router is an open-source LLM router ranked withmartian#1 on RouterArena (score 76.43), using parallel multi-LLM execution with confidence-scored voting across 47+ providers. Key features: - Query-type classification (code, math, creative, factual) - Fallback routing when CLI unavailable - Cost-aware model selection - RouterArena withmartian#1 benchmark result included in docstring GitHub: https://github.com/Das-rebel/a3m-router RouterArena: RouteWorks/RouterArena#113
|
Hi maintainers! Just following up on this PR. A3M Router has since achieved #1 on the RouterArena benchmark (score: 76.43, cost: $0.047/1K). Happy to update the implementation if needed. Looking forward to your feedback! |
|
👋 Hi! Just checking in on this PR. A3M Router has been updated to v2.14.45 with significant improvements: Benchmark Results:
Key Features:
Would love to get this merged! Happy to make any changes. |
|
Hi! Following up on this PR. A3M Router v2.14 has been released with significant improvements. Let me know if any changes are needed to the submission. Thanks! |
A3M Router Benchmark ResultsA3M Router has been tested on RouterArena with strong results: RouterArena Scores
Key Differentiators
Integration ReadyCI is passing, auto-publish working. Happy to help with any integration questions. Would love to get this merged so A3M can be included as a baseline router in RouterBench! |
|
Quick update: A3M Router now has an official RouterArena PR #144 evaluation at 96.77% accuracy and $0.0768/1K. Happy to update the benchmark wording if helpful. Thanks! |
|
Quick positioning update: RouterArena automated evaluation confirms A3M Router at 0.9404 score / 96.77% accuracy, $0.0768/1K queries, and 1.0000 robustness with 0 abnormal entries across 8,400 queries. This positions A3M as No. 1 in accuracy, No. 1 in cost, and No. 1 in robustness among known public baselines: about 2.3× cheaper than Sqwish, 3.5× cheaper than RouteLLM, and ~130× cheaper than GPT-5. |
Summary
Adds A3M Router as a new routing baseline for RouterBench. A3M is ranked #1 on RouterArena (score: 76.43), the official LLM router benchmark.
Implementation
**** — Full implementation that:
npx a3m-router route) for real routing decisions**** — Added A3MRouter to exports
About A3M Router
A3M executes queries simultaneously across 47+ providers with confidence-scored voting — unlike traditional routers that do sequential model selection. It's 19.5 KB, zero ML dependencies.
RouterArena Results
GitHub: https://github.com/Das-rebel/a3m-router
Benchmark: https://das-rebel.github.io/a3m-router/benchmark