A Study of Model and Policy Routing for General-Purpose AI Systems
English · 简体中文
This repository holds the experiment code for Route Anything (arXiv:2611.05275). A router chooses which candidate runs on each input: a language model, an image, video, or speech generator, or a robot policy. The paper asks how much routing can gain, how well learned routers work, and when a cheaper route is good enough. Its theory plays the candidate with the highest expected value minus priced cost given what the router observes, with a reference policy and a quality tolerance fixing the price.
Paper-to-code map · Experiment guide · Data · Reproduction notes · Validation
| Setting | What this repository provides | Data availability |
|---|---|---|
| Omni-modal routing, §§3–4 and Appendix B | Released router implementations and evaluation utilities; see implementation coverage | The omni-modal trajectory matrix is proprietary and is not released. |
| Priced code/text testbeds, §5 and Appendix C | Offline replay, quality-floor certification, collection utilities, and archived aggregate results | CodeRouterBench and the existing code/text replay bundle are public; RouterBench is obtained separately from upstream. |
| Historical extensions | Analytical examples, allocation, latency, and optional stepwise experiments | Retained for continuity; not all of them appear in the paper. |
No omni-modal matrix records, task-level derived tables, split files, embeddings, media, or trained checkpoints are included in the code release or uploaded to Hugging Face. The existing Hugging Face replay bundle covers the earlier code/text studies; it is not the paper's 25-benchmark omni-modal matrix. Reproducing the matrix-based empirical scores requires separately authorized data. Synthetic examples check code behavior, not those scores.
Results reported in the paper. The omni-modal numbers need the private matrix to reproduce; the code/text numbers replay from public records.
- Potential gain (§3). On 19,814 test inputs from 24 benchmarks, the per-input oracle scores 87.98 against 82.60 for the best fixed candidate, 5.4 points higher. SciCode, the 25th benchmark, is excluded from this panel.
- Learned routers (§4). A capability-tree router over 16 benchmarks scores 79.98 against 79.68 for the best fixed candidate (+0.3 points; +2.1 on video generation). On robots, an outcome router gains 1.1 points with 24% less policy time; outcome regression reaches 71.55% success against 42.8% for winner-label prediction.
- Cheaper routes (§5). Against the reference, routing raises efficiency by 8.82× on average over four testbeds, including 25.5× on code and 7.1× on text QA, at a quality loss below 0.5 points. Certified on independent data, the zero-loss code route holds the floor on test in all 200 splits at a median 19.4×.
Evaluation panels have different inputs and baselines. The paper map records section and table correspondence, units, and reproduction limits.
Use Python 3.11 or newer for CPU analyses:
git clone https://github.com/InfRec-Cardinal/route-anything.git
cd route-anything
python -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt
python scripts/reproduce.py --outdir runs/analyticalOn Windows, activate the environment with .venv\Scripts\Activate.ps1. The analytical command needs no dataset or API key. It checks finite examples and recurrence formulas and writes figures, four coordinate tables, and a summary. These are mathematical illustrations retained from the earlier release.
For the paper's learned routers and certification studies, follow omni-modal code and quality-floor certification. GPU training dependencies are separate from CPU replay.
Download the existing code/text records, then run an offline experiment:
python scripts/download_data.py
python scripts/run_experiment.py X2
python scripts/run_experiment.py X10RouterBench must be obtained separately as described in data instructions. For an existing data directory:
python scripts/run_experiment.py X2 --data-root /path/to/dataThe expected layout includes data/agent-as-a-router/data/coderouterbench/ and data/routerbench/routerbench_0shot.pkl. Historical scripts write into their experiment folders; use a separate checkout for reruns if you want to preserve the archived outputs. See the experiment guide for inputs and commands supported by each study.
Fresh collection requires an explicitly configured service through ROUTER_API_BASE_URL and ROUTER_API_KEY. Recorded model names, prices, and relay identities describe the historical experiment. Offline replay needs no service account.
experiments/omnimodal/ router code; no proprietary matrix or artifacts
experiments/quality_floor/ independent quality-floor certification on code/text
experiments/learned_price/ CPU learned quality/cost routers and baselines
experiments/disjoint_retry/ disjoint feedback/scoring retry analysis (supplementary)
experiments/ X2–X6 and follow-up code/text analyses
experiments_x/ X7–X13, X16, collection utilities, historical R6
scripts/ analytical checks, code/text download and replay runner
numerical_results/ analytical reference coordinates
docs/ paper map, data scope, reproduction and validation
Efficiency is the ratio of mean quality to mean resource use, not a mean of per-task ratios. Costs in dollars and time are separate resources. A fitted floor and an independently certified floor answer different questions; oracle verification is a replay assumption, not a deployed verifier. The repository preserves negative results and limitations alongside positive ones.
@article{zhou2026routeanything,
title = {{Route Anything}: A Study of Model and Policy Routing for General-Purpose {AI} Systems},
author = {Zhou, Pengfei and Ma, Yixing and Wang, Hexin and Tang, Weidong and Wan, Zhenglin and Zhao, Wangbo and You, Yang},
journal = {arXiv preprint arXiv:2611.05275},
year = {2026}
}GitHub's "Cite this repository" button reads the same metadata from CITATION.cff.
The project code uses the MIT license. Dataset, benchmark, and model licenses remain with their upstream sources. The code license grants no access to the proprietary matrix.