CrossRouter

CrossRouter is an open-source router that balances cost and accuracy. It ranks first🥇 on the RouterArena leaderboard with a RouterArena Score of 76.1, outperforming the second-place vLLM-SR team (vllm-project/semantic-router) by 0.8 points.

CrossRouter uses structural, domain, length, and semantic signals from each query to estimate whether three capability-tiered base models can answer it correctly. It then selects the least expensive model that is likely to be sufficient. Finally, it adjusts the predicted probabilities using broad query characteristics and the relative strengths of the candidate models in different domains.

The project provides a simple, lightweight, and effective routing pipeline. In production settings, it can help reduce inference cost and ease compute pressure while maintaining model quality.

Pipeline overview:

Raw LLMRouterBench evaluations
  -> query table, splits, and three-tier soft correctness targets
  -> structural/domain/length signals + Qwen embeddings
  -> PCA-128 + CrossRouter pretraining
  -> three-tier sufficiency probabilities
  -> LLMRouterBench base policy
  -> base predictions over three models
  -> routing-ratio adjustment + rule-based correction

Implementation

1. Prepare the data and embedding model

Download the LLMRouterBench dataset as an auxiliary dataset and place it in the following directory:

dataset/
├── bench-release/
│   └── <dataset>/<split>/<model>/*.json
└── bench-release-domain/
    └── <dataset>/<split>/domain.json

• bench-release contains per-query scores, costs, and queries from the LLMRouterBench source models.
• bench-release-domain contains the domain category and confidence for each query.

CrossRouter uses Qwen3-Embedding-0.6B to construct query embeddings.

2. Prepare the training data and features

The model uses four broad groups of query information: structural, domain, length, and semantic signals.

Run:

python cross_router_training/build_training_data.py
python cross_router_training/build_training_features.py

The exact structural and domain feature construction is implemented in cross_router_training/build_structural_features.py.

3. Train CrossRouter and predict probabilities

Run:

python cross_router_training/train_cross_router.py

python cross_router_training/select_base_policy.py

4. Prepare RouterArena inference inputs

Prepare the structural, domain, length, and semantic inputs for each query.

Download the public RouterArena splits and create the local dataset files:

python scripts/prepare_routerarena_data.py

Convert the RouterArena embedding/domain cache into the CrossRouter inference format:

python cross_router_training/build_router_features.py \
  --source-cache /path/to/routerarena_source_cache.npz \
  --output router_inference/router/cross_router_assets/cross_router_features_full.npz

Create one feature cache for each of the full, sub_10, and robustness splits.

5. Generate RouterArena predictions

python router_inference/generate_prediction_file.py cross-router sub_10
python router_inference/generate_prediction_file.py cross-router full
python router_inference/generate_prediction_file.py cross-router robustness

6. Adjust routing proportions and apply broad corrections

Adjust the three-tier RouterArena traffic distribution so that it falls approximately around:

T0: about 55%
T1: about 30%
T2: about 15%

This distribution is not a hard constraint and is not fitted on the RouterArena dataset. Its purpose is to prevent the router from sending too many queries to the most expensive model or assigning too many difficult queries to the lowest-cost model.

🌟At this stage, the RouterArena score is approximately 75.

The final stage applies coarse corrections based on observable query characteristics, such as whether a query is related to code, competition mathematics, or long-form content, and redirects the prediction toward a more suitable model.

Citation

If you find CrossRouter useful in your projects or research, please consider citing it and starring the repository:

@misc{crossrouter2026,
  title        = {Cross Router},
  author       = {JiaHg},
  year         = {2026},
  howpublished = {\url{https://github.com/JiaHg/CrossRouter}},
}