Skip to content

feat(analytics): add route reliability and latency analytics - #1005

Draft
Wibias wants to merge 4 commits into
lidge-jun:devfrom
Wibias:feat/ri-03-routing-analytics
Draft

feat(analytics): add route reliability and latency analytics#1005
Wibias wants to merge 4 commits into
lidge-jun:devfrom
Wibias:feat/ri-03-routing-analytics

Conversation

@Wibias

@Wibias Wibias commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

RI-03 of the Router Intelligence / Routing Control Plane programme. Adds
source-backed routing analytics computed from the request-history index
(RI-02), never from repeated full JSONL scans.

Read-only: this PR cannot change routing behavior (ADR-10 - no automatic
self-tuning).

Metrics

GET /api/routing-analytics (filters: provider, model, profileId,
surface, from, to) returns:

  • Success / failure / cancelled rates (client cancellations are never
    provider failures)
  • Fallback rate (multi-attempt share) and attempt counts
  • p50/p95/p99 total duration and time-to-first-token (nearest-rank
    percentiles, with TTFT sample count + coverage)
  • Incomplete-stream rate (terminalStatus: "incomplete")
  • Cooldown-triggering failure count (status 429, or attempts with
    rate-limit-429 / key-429 / oauth-401 / anthropic-oauth-429
    recovery kinds)
  • Estimated cost per successful request and total for successful requests
    (reuses usage/cost.ts pricing; unknown prices stay unknown, never zero)
  • Usage coverage and price coverage
  • Provider/model/account breakdown with per-bucket sample counts and p50
  • Profile breakdown (from RI-01 decision traces) with revision
  • Minimum-sample confidence (high >= 100, medium >= 20, else low)
  • Explicit historyTruncated flag when the analysis cap (50,000 rows) is hit

Scope

  • src/routing/analytics.ts - computeRoutingAnalytics() over the SQLite
    index (bounded, deterministic, single SQL pass + JS aggregation).
  • src/server/management/routing-analytics-routes.ts - the endpoint,
    registered in management-api.ts.
  • tests/routing-analytics.test.ts - 8 tests.

Privacy / security

  • No prompts, credentials, or raw bodies: the analysis reads the same
    privacy-bounded columns the index already stores.
  • Cost estimates are derived locally from usage + prices; no new data leaves
    the process.
  • bun run privacy:scan passes.

Compatibility

  • Additive API; /api/logs, /api/request-history, and usage.jsonl
    contracts unchanged.
  • No routing behavior change.

Dependency

Non-goals

  • No routing-decision changes, no self-tuning, no profile weights mutation.
  • No policy profiles (RI-04..08) - the profileId filter/breakdown only
    reads RI-01 traces.
  • No explainability surfaces (RI-09), no GUI (RI-10).

Local verification (exact)

  • bun x tsc --noEmit -> PASSED (0 errors)
  • bun run test tests/routing-analytics.test.ts -> 8/8 pass
  • Focused regression suites (analytics, request-history, route-decision-
    trace, request-log, usage-log, combos) -> 144/144 pass
  • bun run privacy:scan -> passed

Notes for reviewers

  • Classification is explicit and documented in classifyRow(): status 499 /
    client_cancel -> cancelled; terminalStatus: "incomplete" -> failure;
    status >= 400 -> failure; otherwise success.
  • Unknown prices produce null estimates and priceCoverage: 0 - never a
    fake zero cost.

@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 096e4a49-b0f9-441b-9f2c-3e0b4b6ffdd5

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant