Add KT-ModelRouter submission

[DusanBaek]

Reference implementation: KT-A-Autonomous-Tech-Team/ModelRouter

Summary

Adds KT-ModelRouter, which routes each query across a five-model pool using an internally trained routing policy:

• deepseek/deepseek-v4-flash
• deepseek/deepseek-v4-pro
• google/gemma-4-31b-it
• google/gemini-3-flash-preview
• qwen/qwen3-235b-a22b-2507

Evaluation Integrity

• The routing policy was trained independently of RouterArena.
• RouterArena data, labels, reference answers, per-model outcomes, scores, and evaluation results were not used to train, fit, or tune any routing component.

Files Added or Changed

• router_inference/config/KT-ModelRouter.json — router configuration and five-model pool
• router_inference/predictions/KT-ModelRouter.json — full split with 8,400 entries and populated generated_result fields
• router_inference/predictions/KT-ModelRouter-robustness.json — robustness split with 420 entries
• universal_model_names.py — registers google/gemma-4-31b-it and maps google/gemini-3-flash-preview to the existing gemini-3-flash-preview model-cost entry
• model_cost/model_cost.json — adds the public reference price for google/gemma-4-31b-it

Pricing Basis for google/gemma-4-31b-it

The submitted outputs were generated using an internally served instance of google/gemma-4-31b-it. OpenRouter and OpenInference were not used as the inference backend for the submitted generations.

Because KT's internal serving cost is not publicly listed, the submission uses the publicly listed price of the same model served by the OpenInference provider through OpenRouter as a reproducible public reference rate for RouterArena scoring.

This reference price does not imply that KT and OpenInference use identical serving infrastructure, hardware, or quantization. It provides a publicly verifiable token price for the same released model rather than an unverifiable internal cost estimate.

Public Pricing Reference

• Model: google/gemma-4-31b-it
• Architecture: 31B dense, instruction-tuned
• Pricing platform: OpenRouter
• Listed provider: OpenInference
• Listed input price: $0.08 per 1M tokens
• Listed output price: $0.35 per 1M tokens
• Observed effective input price: $0.07986 per 1M tokens
• Observed effective output price: $0.3486 per 1M tokens
• Pricing evidence captured at: 2026-08-20 04:59:57 GMT
• Public model page: OpenRouter – Google Gemma 4 31B
• Provider endpoint data: OpenRouter Gemma 4 31B endpoints
• Actual submission inference backend: KT internal infrastructure

At the time the pricing evidence was captured, OpenRouter listed its OpenInference endpoint for google/gemma-4-31b-it at $0.08/M input tokens and $0.35/M output tokens. The corresponding effective rates observed for that endpoint were $0.07986/M input and $0.3486/M output.

RouterArena's static model-cost configuration uses the publicly displayed list prices:

• Input: $0.08 per 1M tokens
• Output: $0.35 per 1M tokens

The effective rates are included as supporting evidence but are not used directly in the static model-cost entry.

The price is based on a publicly listed endpoint for the same model. It is not estimated from parameter count, copied from a different model, or based on a free or promotional endpoint.

Price Comparison

For context, the proposed rate is within the range of similarly priced models already registered in RouterArena.

Model	Input / 1M	Output / 1M
google/gemma-4-31b-it — OpenInference list price	$0.08	$0.35
qwen/qwen3-30b-a3b-instruct-2507	$0.08	$0.33
Qwen/Qwen3-Coder-Next	$0.07	$0.30
meta-llama/llama-3.3-70b-instruct	$0.10	$0.32
deepseek/deepseek-v4-flash	$0.14	$0.28
gpt-4o-mini	$0.15	$0.60

This comparison is provided only as a reasonableness check. The pricing basis remains the OpenInference provider's published OpenRouter rate for the exact google/gemma-4-31b-it model.

RouterArena price reference: model_cost/model_cost.json

Token Accounting

• Cost is computed from the input and output token counts recorded in the prediction file.
• Reasoning or thinking tokens, where applicable, are counted as output tokens.
• No cache-read discount or promotional pricing is applied.
• The OpenInference list prices of $0.08/M input and $0.35/M output are used.
• The model identifier is consistently recorded as google/gemma-4-31b-it.

Model Cost Entry

"google/gemma-4-31b-it": {
  "input_token_price_per_million": 0.08,
  "output_token_price_per_million": 0.35
}

Validation

• Full prediction entries: 8400
• Full prediction validation with --check-generated-result: passed
• Robustness prediction entries: 420
• Robustness prediction validation: passed
• Pre-commit checks: passed

Leaderboard Metadata

• Display name: KT-ModelRouter
• Affiliation: KT
• Type: Open-source
• GitHub: KT-A-Autonomous-Tech-Team/ModelRouter
• Requested action: Add KT-ModelRouter as a new leaderboard entry
• Prediction artifact key: KT-ModelRouter

[DusanBaek]

/evaluate

[github-actions]

Router Evaluation Results

Router: KT-ModelRouter
Dataset Split: full

RouterArena Metrics

Metric	Value
RouterArena Score	0.7628
Accuracy	78.14%
Total Cost	$2.266668
Avg Cost per Query	$0.000270
Avg Cost per 1K Queries	$0.2698
Number of Queries	8400
Abnormal Entries	0
Robustness Score	0.8048

---

Evaluation completed by RouterArena automated workflow

[yl231]

Merged and live — KT-ModelRouter is #2 on the leaderboard (Arena 76.28, accuracy 78.14%, $0.27/1K, robustness 80.48). Clean audit: 8,400 unique rows, real token accounting, faithfulness gap +0.026 (no sign the trained policy touched the public split). Thanks @DusanBaek!