ModelHub

AI model pricing, capabilities, and benchmarks — from Bifrost, LiteLLM, models.dev, OpenRouter, Artificial Analysis, the BenchLM leaderboard, and the llm-stats.com TrueSkill leaderboard. Zero dependencies.

Install

go install github.com/jelloeater-agent/modelhub/cmd/modelhub@latest

Or grab a binary from releases.

Usage

modelhub refresh           Fetch latest data from all sources
modelhub list [--table]    List models (JSON default)
modelhub show <id>         Show a single model (JSON)
modelhub stats             Aggregate statistics (JSON + summary)

Examples

# Refresh data
modelhub refresh

# Pipe JSON to jq for any query
modelhub list | jq '.[] | select(.provider=="openai") | .name'
modelhub list | jq '.[] | select(.context_window > 128000 and .input_price_per_1m < 1)'
modelhub list | jq '.[] | select(.supports_vision) | .id'

# Quick visual scan
modelhub list --table
modelhub list --table | grep gpt-4

# BenchLM leaderboard order (rank 1 first, unranked last; falls back to llm-stats rank)
modelhub list --table --sort rank

# Single model details
modelhub show openai/gpt-4o | jq .context_window

# Stats
modelhub stats

# Search interactively with fzf
modelhub list | jq -r '.[].id' | fzf --preview 'modelhub show {}'

Config

Set AA_API_KEY env var for Artificial Analysis benchmarks (optional):

export AA_API_KEY=your_key_here

Or create ~/.modelhub/config.json:

{
  "aa_api_key": "your_key_here"
}

Build

go build ./cmd/modelhub

TODO: LiveBench as a source

Investigated 2026-08-05 — doable but not worth it yet. LiveBench (https://livebench.ai) publishes an aggregated per-model table with 23 task scores:

• https://livebench.ai/table_2026_06_25.csv (38 models, snapshot date pinned in their JS bundle — no "latest" alias)
• https://livebench.ai/categories_2026_06_25.json (task → category mapping: Coding, Agentic Coding, Mathematics, Reasoning, Data Analysis, Language, IF)

Why it was deferred:

1. Fragile URL: the snapshot date lives in their frontend bundle (const pe=[...dates...]). We'd pin the latest date in model.Config.BenchLMURL-style defaults and bump it manually each release.
2. ID normalization breaks merging: LiveBench IDs carry effort/config suffixes (claude-opus-5-max-effort, gpt-5.2-2025-12-11-high). NormalizeID strips date/version segments but not -max-effort, -xhigh, -thinking, -auto, so most Claude/GPT rows would NOT merge with anthropic/claude-opus-5, and variants collide under one base ID (gpt-5.2 has 3 rows). Would need a dedicated effort-suffix normalizer plus a collision policy (highest-effort row wins).
3. Redundant signal: BenchLM already covers the same ~40 frontier models with a stable single-URL JSON API and 9 score fields.

If it becomes valuable later, implement as internal/fetch/livebench.go: parse the CSV + categories JSON, aggregate task scores to category averages, normalize effort suffixes, keep top-effort row per base ID, add as SourceLiveBench at bottom priority next to benchlm/llmstats.