MCP Router

An open-source, Jev-based MCP router with edge capacity. It discovers, catalogs, classifies and deduplicates every MCP server and tool you have — then exposes each AI agent only the small, relevant, authorized subset it needs for the task at hand, instead of flooding its context window with the whole catalog.

Jev-based means routing decisions are System One decisions — typed Choice / Score / Noul questions answered with calibrated probabilities in a single forward pass, the paradigm introduced by TypeSafe's Jev — rather than generated text. The decision engine is pluggable: run it at the edge on a laptop GPU or CPU, point it at a hosted endpoint (Jev, or another MCP Router serving its local model), or use Azure OpenAI with a URL and key. See Backends.

• Recommended decision engine: Laya — a 421M open-weight, Jev-compatible decision model (Apache 2.0), opt-in through the [inference] extra. Because a decision model can only select among options supplied by deterministic code, it structurally cannot invent tool names or execute anything. Out of the box the router runs the zero-ML deterministic backend.
• Retrieval: embeddings in PostgreSQL + pgvector (hash projection by default, BGE-small with [inference]), fused with Postgres full-text keyword search (reciprocal-rank fusion).
• Security: a deterministic, deny-by-default policy engine and execution manager. Model scores are relevance data — they can never widen access.
• Exposure budgets + route cache: per-request, per-agent and global caps on exposed tools and distinct servers (a request can lower a budget, never raise it); a generation-invalidated route cache that re-checks authorization on every hit.
• Efficiency analytics (the funnel): every routing decision is followed through surfaced → selected → succeeded, with execution-to-decision attribution, position bias, context-token savings vs. the agent's full authorized catalog, per-agent profiles, wasted-exposure and staleness suggestions, daily rollups and Prometheus totals. Admin simulations are excluded; cache hits count as real traffic.
• Dashboard: a tool playground (schema-generated forms, runs through the same execution manager over POST /api/v1/tools/{id}/execute), an approvals queue, the agent lens (see exactly what one agent would be shown for a query, with every budget clamp and policy filter explained), and the analytics funnel.
• Zero-ML fallback everywhere: the entire stack runs with no torch installed (hash embeddings + lexical decision model), and any model failure or timeout degrades to deterministic retrieval ranking, flagged fallback_used.

Skills

Agent Skills are routed like tools, through one pipeline and one policy engine:

• Sources: register a local directory or an https git repo. Git sources are shallow-cloned with hooks off and size and time caps. Ingest checks each skill against the spec and flags secret-shaped or oversize content without editing it.
• Risk class: each skill is classed read, write, execute or unknown. Shipping scripts/ or exec-like allowed-tools makes a skill execute, and policy treats unknown as execute. Human review is respected, but a content change re-opens it.
• Routing: /route returns skills[] under a separate maxSkills budget. Policy rules use resourceKind: "skill".
• Exposure: a skill is exposed only if it was routed to the agent. Clients receive it as MCP prompts and resources, through the meta-tools router.activate_skill and router.read_skill_resource, or over REST activate or bundle (a zip for ~/.claude/skills). Every activation is policy-checked and audited, and the funnel tracks it as surfaced → activated.
• Operator guide with flow diagram: docs/skills.md.

Target hardware: a single laptop GPU (RTX 4060 Laptop, 8GB) with full CPU fallback. Runs entirely offline; no external inference services are needed.

Backends

The decision model is pluggable via MCPR_DECISION_BACKEND (default deterministic):

• deterministic (default) — no model; also the automatic fallback on any backend error.
• laya — local model, GPU if available. Needs the [inference] extra or the inference image (docker-compose.inference.yml, plus docker-compose.gpu.yml for NVIDIA: deploy.md § Flavors).
• lyla — vendored catalog-selection engine (Amantux/lyla, MIT, stdlib-only — no [inference] extra). Ranks the router's own tool+skill catalog (tags, operation, cost, success rate, connectivity); knobs MCPR_LYLA_K_COARSE, MCPR_LYLA_EPSILON, MCPR_LYLA_MIN_CONFIDENCE.
• remote — hosted Jev (AIML API) or another MCP Router's POST /api/v1/decision/systemone edge endpoint.
• aoai — Azure OpenAI v1 (decision), plus MCPR_EMBEDDING_BACKEND=aoai for embeddings.

Embeddings use MCPR_EMBEDDING_BACKEND (default hash; bge with [inference]; aoai). Outbound endpoints are https-only (http only to localhost), link-local/metadata targets are refused, keys are never logged, and every call has a total deadline. Full matrix, env examples, wire shapes and the edge topology: docs/backends.md.

Architecture

flowchart TD
    A[AI agents / MCP clients] -->|MCP streamable-http /mcp<br/>REST /api/v1| GW[MCP Gateway<br/>per-agent auth · session tool exposure]
    GW --> REG[(Tool Registry<br/>PostgreSQL + pgvector)]
    GW --> POL[Policy Engine<br/>deny-by-default · deterministic]
    Q[task / query] --> RET[Hybrid retrieval<br/>vector + keyword, RRF]
    RET --> LAYA[Decision model<br/>domain choice · candidate scoring · no-match]
    LAYA --> RANK[Ranked candidates]
    RANK --> POL2[Authorization filter<br/>scores never widen access]
    POL2 --> EXP[Dynamic tool exposure<br/>3–8 tools, budget-clamped]
    EXP --> EXEC[Execution manager<br/>schema validation · rate limits ·<br/>timeouts · approvals · audit]
    EXEC --> SRV[Downstream MCP servers<br/>stdio · streamable-http · SSE]
    DISC[Discovery & sync<br/>tools/list · schema versioning ·<br/>health checks] --> REG
    REG --> RET
    SRV -.-> DISC
    EXEC --> TEL[Telemetry & audit<br/>Prometheus /metrics · execution records]

Loading

Routing pipeline (FR-03/FR-04)

sequenceDiagram
    participant Agent
    participant API as /api/v1/route
    participant Scope as Policy scope filter
    participant Ret as Hybrid retriever
    participant Laya as Decision model
    participant Exec as Gateway /mcp

    Agent->>API: query (authenticated; agent_id = principal)
    API->>Scope: pre-filter: which servers/tools is this agent allowed?
    Scope-->>Ret: authorized scope only
    Ret-->>API: 10–20 candidates (vector ∪ keyword, RRF)
    API->>Laya: Choice: which domain? (only domains present)
    API->>Laya: Score each surviving candidate (batched)
    API->>Laya: Noul: does anything fit? → no_match gate
    API->>Scope: re-filter ranked list (defense in depth)
    API-->>Agent: top 3–8 tools + scores (+ fallback_used / no_match)
    Agent->>Exec: tools/list → exposed subset · call_tool → policy + validation + audit

Loading

A hierarchical cascade — never one big classification over the whole catalog. Any model exception or deadline overrun falls back to deterministic retrieval ranking; a sub-threshold "does anything fit?" probability returns an honest no_match instead of garbage. More: docs/architecture.md.

Run it (Docker)

Prerequisites: Docker with Compose v2. Run every command from the repository root.

1. cp .env.example .env
2. Generate secrets and put them in .env:

   openssl rand -hex 32   # -> MCPR_ADMIN_TOKEN=...
   openssl rand -hex 32   # -> POSTGRES_PASSWORD=...

3. docker compose up -d --build --wait (if the build stalls at npm ci, see Troubleshooting)
4. curl -fsS http://localhost:8400/healthz (connection refused while the container shows healthy means MCPR_ADMIN_TOKEN is not set: see Troubleshooting)
5. Browse to http://localhost:8400/ and follow First run.

The API is published on 127.0.0.1:8400 only. Without MCPR_ADMIN_TOKEN the container still starts, but uvicorn binds 127.0.0.1 inside the container and logs a WARNING, so the published port is unreachable from the host until the token is set (the in-container healthcheck stays green). MCPR_ALLOW_OPEN_DEV=1 overrides this for local dev only (open admin API). Other hosts also need MCPR_ALLOWED_HOSTS and MCPR_BIND (deploy.md). Every setting: configuration reference.

Check a running stack end to end:

BASE=http://127.0.0.1:8400 COMPOSE="docker compose" \
  MCPR_ADMIN_TOKEN=... MCPR_AGENT_KEY=<an agent key> scripts/smoke.sh

First run

1. Open http://localhost:8400/, click Connect and paste your MCPR_ADMIN_TOKEN (kept in this browser tab's session only).
2. Then one of two paths:
   • Setup wizard (recommended). With no agents yet, the dashboard opens /setup: add servers and skill sources, create an agent (its key is shown once), optionally grant read-only starter rules, then copy the client snippet. Skip setup returns to the dashboard.
   • Agents from .env. If you set MCPR_AGENT_KEYS=agent:key before starting, that agent already exists, so the wizard is skipped. Add servers and policy rules from the dashboard or the API (INSTALL.md §2–3).
3. Point your coding agent at http://localhost:8400/mcp with its agent key: docs/INSTALL.md has verified configs for Claude Code, GitHub Copilot CLI, Codex CLI, Cursor, VS Code and Gemini CLI.

Imported mcpServers entries that start with npx run inside the container (Node 22 is in the image); uvx entries need an extended image (stdio servers in Docker).

Run from source

Prerequisites: Python 3.12, uv, Node 22 (.nvmrc), Docker Compose v2 for the database. From the repository root:

POSTGRES_PASSWORD=mcprouter docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d --wait db  # dev-only: pgvector on 127.0.0.1:5434
uv sync --locked --extra dev --python 3.12  # zero-ML core, from uv.lock (= make setup)
uv sync --locked --extra dev --extra inference  # optional: real embeddings + Laya
(cd ui && npm ci && npm run build)          # dashboard; build it before starting the server

export MCPR_ADMIN_TOKEN=$(openssl rand -hex 32)
.venv/bin/uvicorn --factory mcprouter.api.app:create_app --port 8400

Compose needs POSTGRES_PASSWORD for every command, even the dev database. Without it the first line fails with

error while interpolating services.api.environment.MCPR_DATABASE_URL: required variable POSTGRES_PASSWORD is missing a value: set POSTGRES_PASSWORD in .env

(the service named varies: the base file alone reports services.db.environment.POSTGRES_PASSWORD; an empty POSTGRES_PASSWORD= in .env fails the same way). Despite the message, the dev database does not need a .env: keep the POSTGRES_PASSWORD=mcprouter prefix shown above (or run make db-up). For the full stack, set it in .env as in Run it (Docker).

The server serves ui/dist relative to the working directory, so start it from the repository root. Contributor setup, gates and tests: CONTRIBUTING.md.

Try it with a synthetic fleet

.venv/bin/python -m testbed.seed --servers 10 --tools 40

This registers ten synthetic servers with overlapping tools across five domains straight into the database, so routing, the agent lens and analytics have something to work with. To execute tools too, serve live fleet servers with .venv/bin/python -m testbed.serve and register their URLs (CLI reference).

Analytics rollups: set MCPR_ANALYTICS_ROLLUP_ENABLED=true to recompute the daily funnel rollups inside the app (a pass at startup, then every 24h), or leave it off and run POST /api/v1/analytics/rollup (admin token) from cron once a day. Either way, days that were never rolled up are computed live from raw rows — rollups only make old windows faster. Run ANALYZE after bulk imports.

Status — honest ledger

Test counts stamped 2026-10-09 from main at 6058a3e (0.6.0) with:

MCPR_DATABASE_URL=postgresql+psycopg://mcprouter:mcprouter@localhost:5434/mcprouter .venv/bin/pytest -q -n 8
cd ui && npx vitest run

Re-stamp both after each full run; CI runs the backend suite in lanes (unit, e2e) and the UI suite on every push. The other rows were measured earlier in the 0.6 cycle and are not re-run by these commands.

Verified (ran here, CPU)	Pending (needs the target GPU)
Backend: 2062 passed, 10 skipped with -n 8 on main (skips are slow, live-model and [inference]-only; the route-coverage gate counts as a skip under xdist). UI: Tests 255 passed (255) in 31 files on Node 22. e2e: discover → route → execute → audit → analytics funnel	CUDA/FP16 paths (written, device-agnostic, unproven)
Laya 0.4.0 loaded on CPU: choice/score/noul with calibrated probs	<150ms warm routing p95 — at risk: CPU measurements put it out of reach; see hardware-validation.md
100 servers / 1,000 tools full refresh in 6.0s (target: <60s)	<4GB VRAM claim
Zero unauthorized executions across the adversarial test battery	Laya candidate-count tuning (score top-5 vs top-20)
Skills e2e (tests/test_e2e_skills.py): source sync → classify → route → MCP prompt → bundle → analytics; synthetic baseline (fallback): skills top-1 1.0 / top-5 1.0, mixed 0.75, 0 unauthorized skill exposures
Security guards mutation-checked (break guard → named test fails)

The GPU validation runbook is docs/hardware-validation.md.

Security model

Authentication (per-agent API keys, hashed at rest) → deny-by-default policy rules with read < write < execute ceilings (an unclassified tool counts as execute — fail closed) → JSON-Schema argument validation → optional human approval gates (single-use, expiring) → timeouts and per-agent rate limits → an audit record for every attempt, including refusals. Secrets are redacted before logs, audit rows and model inputs. Details: docs/security-model.md.

⚠️ Registering a stdio server means the platform will run that command. Server registration is therefore admin-only, and the admin API fails closed when agents exist but no MCPR_ADMIN_TOKEN is configured. Without a token the container binds loopback inside the container, so its published port is unreachable. Don't expose the API beyond localhost without auth configured.

Documentation

Doc	For
docs/INSTALL.md	Connecting coding agents, identities and policy, API conventions
docs/deploy.md	Docker deployment, security posture, limits, troubleshooting
docs/upgrade.md	Upgrades, downgrades, backups
docs/reference/configuration.md	Every MCPR_* setting (generated)
docs/reference/api.md	Every REST route with its auth class (generated)
docs/reference/cli.md	testbed, bench and scripts/ commands
docs/architecture.md	How the router is built
docs/security-model.md	Identities, policy, execution, redaction
docs/backends.md	Embedding and decision backends, edge topology
docs/skills.md	Agent Skills operator guide
docs/analytics.md	The funnel, savings estimates, rollups
docs/hardware-validation.md	GPU validation runbook
docs/SPEC.md, docs/scoping.md	Design baseline (amended)
CONTRIBUTING.md	Development setup and gates
CHANGELOG.md	Release notes

Repository map

src/mcprouter/
  mcpclient/   MCP SDK 2.x connectors (stdio · streamable-http · SSE)
  discovery/   tools/list sync, schema versioning, health, config import
  registry/    catalog search, classification + human review, usage stats
  dedup/       duplicate detection → human-reviewed suggestions (never auto-disable)
  inference/   hash + BGE embeddings · Laya adapter · deterministic fallback · engine
  routing/     hybrid retrieval · hierarchical pipeline · no-match gate ·
               exposure budgets · route cache · simulate traces
  policy/      deny-by-default rules engine + scope filter
  execution/   validation · approvals · rate limits · redaction · audit
  gateway/     per-agent MCP endpoint with dynamic tool exposure
  analytics/   funnel · attribution · context economy · profiles · rollups · metrics
  eval/        routing-quality framework + synthetic dataset (111 cases)
  skills/      skill sources (directory · git) · ingest/validate · safe serving · bundle export
ui/            React + Vite + Fluent UI v9 dashboard: catalog, playground,
               approvals, agent lens, analytics funnel
testbed/       synthetic MCP server fleet with ground-truth labels
  skills/      deterministic Agent Skills generator (near-duplicates, invalid cases, git)
bench/         latency/VRAM benchmark harness + committed CPU baselines
docs/          operator guides, references, design baseline; history/ is archival

License

Apache License 2.0. Laya (Apache 2.0) and BGE (MIT) are downloaded at runtime from their upstream repositories and are not redistributed here.