Repository navigation
feat: the Mini App admin API, a documented OpenAPI surface, and an HTTP test suite - #13
Merged
Merged
Conversation
`PROMETHEUS_MULTIPROC_DIR` names a directory the client writes one mmap file per metric into, and it must exist before the first metric is *declared* — a `Gauge(..., multiprocess_mode=...)` opens its file in its constructor, at import time. `start_metrics_server` also created it, which is far too late. With the variable set (as `.env.example` sets it) and the directory absent, importing `cb_core.metrics` raised `FileNotFoundError` and every service died during startup. A fresh clone that copied `.env.example` hit exactly that. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uqm1RzvYzYvoFANDDqN22v
The fixture already stripped every `CB_*` variable from the process environment, for exactly this reason. It could not reach the other half: `.env` is a *file* source, and pydantic-settings merges dict-typed fields across sources rather than letting an explicit kwarg win outright. So a developer who has a `.env` — CONTRIBUTING tells them to make one — got its `CB_BOT_TOKENS` merged into the dict these tests pass, and two assertions failed on their machine and nowhere else. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uqm1RzvYzYvoFANDDqN22v
…rs only `x_analytics_api` answers "how is *my group* doing" and refuses anything wider — every query it makes carries a `group_id` and the caller must administer that group. Right for a group admin, useless to the person who runs the bot: they could not see how many groups it is in, which are alive, what the fleet costs in LLM tokens, or which commands anyone uses. Those numbers were in Grafana, which is not something a Mini App can open. Seven owner-only reads, all aggregate: GET /admin/overview reach, the window's totals, the LLM budget GET /admin/analytics/daily every group, summed per day GET /admin/analytics/groups the busiest groups GET /admin/analytics/commands every command, and how many groups use each GET /admin/analytics/llm fleet-wide spend per provider and model GET /admin/groups the directory, keyset-paginated GET /admin/tenant how this deployment is configured Three decisions worth reviewing. **The scope is granted, never assumed.** `/oauth2/token` adds `admin:read` to a session only when its subject is a tenant owner or `CB_OWNER_ID` — the same people the owner-only Telegram commands answer to. A non-owner's token cannot carry it, so it cannot reach these endpoints however the client edits its own request. `/me` reports `is_bot_admin` from the tenant rather than from the token, so a Mini App draws its admin screens without provoking a 403 to find out. A refresh reissues the stored scope rather than re-evaluating it, so an owner removed today keeps it until the session expires or is revoked. **403 here, not the 404 the group endpoints answer.** Those hide behind a 404 because a chat id is worth hiding; `/admin/...` is a fixed path in `/openapi.json`, so there is nothing to hide and a 404 would tell an owner with a stale token exactly what it tells an outsider. **These are the only queries in the codebase that do not filter on `group_id`**, which AGENTS.md §4.1 forbids on the reply path. The departure is confined to `cb_core/platform_analytics.py` and argued in its docstring: they read the daily rollups rather than events, every one aggregates before returning so only the grouped result crosses the network, and nothing on the reply path calls them. If a deployment outgrows it the fix is a `platform_daily_stats` rollup in cb-worker, and these shapes are already what one would return. Reads only, and no member data: no message, no member name, no group's rules. `/admin/tenant` never returns a bot token, and nothing here is writable — every field on it changes how the bot behaves, and a form one mis-tap from emptying `owner_ids` would be an outage nobody could undo through the same API. Also extracts `cb_api/refusals.py`: three routers were each declaring their own near-copy of the same 401/403/404 descriptions. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uqm1RzvYzYvoFANDDqN22v
…m it
FastAPI already builds a full OpenAPI 3.1 document from the same annotations
the handlers run on, so it cannot drift from the code the way a hand-written
API document does. What it *can* be is thin: a route with no summary, a
response typed `object`, a refusal nobody declared. All invisible until someone
tries to write a client or a test against it, which is when being thin costs
most.
`scripts/api_spec.py lint` is the gate — ten rules, each one a thing a client
author needs and cannot recover from the code: a summary, a description, a
named response shape, a body model on every declared refusal, a description on
every query parameter, a docstring on every response model, and the right
refusals declared (401 wherever a token is required, 404 on group-scoped paths,
403 and *not* 404 on the fleet-wide ones).
It found 51 gaps on its first run, all fixed here or in the previous commit:
two endpoints with no description, 23 response models with no docstring, 19
undescribed parameters, two health endpoints answering `dict[str, object]`, and
two tags nobody had described.
`generate` writes two committed artifacts:
docs/site/public/openapi.json client generators, and the contract tests
docs/site/public/api-reference/index.html an offline, searchable reference
`generate --check` fails when either is stale. A generated artifact nobody
regenerated is worse than none: it is wrong and authoritative-looking at once.
v1's `/` and `POST /login` are no longer exempt. They used to be, because
`COOKIEBOT-WebHub` reads their bodies by name and a `response_model` would
filter an unlisted key back out the day one is added. They now declare their
shapes through `responses={200: {"model": ...}}` with `response_model=None`,
which documents the body without touching a byte of it — the pattern to copy
when something has to be compatible and described at once.
The reference is published at `/api-reference/`, not `/api/`, because the docs
site already owns `/api/search`. `.gitignore` gains a negation for it (the
blanket `*.html` is Cython's annotation output) and an entry for `.qa/`, which
the setup script in the next commit writes tokens into.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uqm1RzvYzYvoFANDDqN22v
The API had unit tests with the database faked and core integration tests below HTTP, and nothing in between. Nothing proved a response still matched the document clients are generated from, and nothing proved a *deployment* answered at all. Three files under `qa/api/`, each answering a different question and failing for a different reason: test_smoke.py does the deployment I just started answer? running API test_contract.py does every response match openapi.json? database test_integration.py does it behave against real rows? database **Contract** validates every response — refusals included — against the *committed* `docs/site/public/openapi.json`, never against the document the app would build right now: an app checked against its own live description agrees with itself by construction and can change shape freely. `jsonschema-rs` does the validating (the Rust implementation: one compiled schema, reused across every payload, and it reports every drift rather than the first). `CASES` is a whitelist, so an endpoint added without a row fails collection. **Integration** drives the real app over `httpx.ASGITransport` against a real database — no port, no server. Sessions are minted through the real `/oauth2/token`, so every test also exercises the token path against the keys in `signing_keys`; a break there fails loudly rather than silently downgrading these tests to something weaker. The app's lifespan is deliberately not entered — it would converge the schema, open a second pool, connect Valkey and object storage and bind the metrics port. **Smoke** is the only layer that talks over a socket, and the only one that can catch what sits between the code and a caller: a process that will not boot, a `.env` pointing at the wrong database, a middleware that swallowed the `WWW-Authenticate` header. Every layer skips *loudly*, naming the command that would fix it — a suite that passes quietly when it tested nothing is a green tick over an untested deployment. `initData` and the login widget's payload are signed from Telegram's published algorithm in `qa/api/auth.py`, never by importing `cb_api.miniapp`: a fixture that builds its input by calling the code under test can only ever agree with it. `scripts/qa_setup.py` is the other half — one command from a fresh clone to a running API with data in it: uv run scripts/qa_setup.py It checks the machine, writes a `.env` if there is none, starts Citus and Valkey, migrates, seeds three groups and a cast where every authorisation rule has someone who fails it, starts cb-api in the background, mints three tokens and prints what every endpoint answered next to what it should have. It carries its own dependencies (PEP 723), so `uv run` gives it rich and httpx without touching this workspace, and it is idempotent. It can log in without Telegram because it signs its own `initData` with a local-only bot token it writes to `.env`. Nothing is stubbed: the HMAC is real and verified exactly as Telegram's is. What holding the key buys is a session as *anyone*, which is what makes the refusals testable — and the refusals are most of this API's behaviour. New tasks: `api-lint`, `api-docs`, `api-test`, `setup`. `check` now runs the document gate before the tests, so a route that lost its documentation fails with one line rather than as a contract test about a schema that no longer exists. `test` and `qa` exclude the `smoke` marker so the offline suite stays offline on a machine that happens to have a deployment running. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uqm1RzvYzYvoFANDDqN22v
Six steps, in the order someone actually meets them: stand it up, call it by hand, understand why logging in works without Telegram, run the suite, write your own, keep the document honest. Written for whoever is testing the HTTP API rather than writing it — the rest of this section is written from the inside out, and none of it answers "what do I type first". The half worth reading before writing a test is step 5: which of the layers a case belongs in (two dividing lines settle almost every one), the conventions the suite follows with the bug each prevents, and the fixtures available. The traps are named because both have already bitten this repo — a fleet-wide assertion against a database the rest of the suite is also using, and rollup tables that have no foreign key to `groups`, so deleting a group leaves its statistics behind. AGENTS.md §6 gains the API layer and three rules: an endpoint is not done until the document describes it, adding one means a row in three whitelists, and a fixture that signs a payload is written from the published algorithm rather than by importing the module under test. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uqm1RzvYzYvoFANDDqN22v
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What a group (and whoever runs the bot) would notice
Owners get numbers they never had.
x_analytics_apideliberately refuses to answer anything wider than one group, so the person running the deployment could not see how many groups the bot is in, which are alive, what the fleet costs in LLM tokens, or which commands anyone uses — those lived in Grafana, which a Mini App cannot open. Seven owner-only reads now answer that (/admin/overview,/admin/analytics/{daily,groups,commands,llm},/admin/groups,/admin/tenant).Nothing a group admin can reach changed. Their endpoints answer exactly as before.
Which layer, and why
cb_core/platform_analytics.py— the fleet reads. The only queries in the codebase that do not filter ongroup_id, which AGENTS.md §4.1 forbids on the reply path. Confined to that one module and argued in its docstring: they read the daily rollups rather than events, every one aggregates before returning so only the grouped result crosses the network, and nothing on the reply path calls them. If a deployment outgrows it, the fix is aplatform_daily_statsrollup in cb-worker — these shapes are already what one would return.cb_api/routers/admin.py— HTTP only. Reads only; no member data, no bot tokens, nothing writable.cb_api/refusals.py— extracted; three routers were each declaring a near-copy of the same 401/403/404 set, and two named no model at all.Two decisions worth a look:
admin:readis granted, never requested./oauth2/tokenadds it only when the subject is a tenant owner orCB_OWNER_ID. A non-owner's token cannot carry it, so it cannot reach/admin/...however the client edits its request. A refresh reissues the stored scope rather than re-evaluating it — an owner removed today keeps it until the session expires or is revoked.403 there, not the 404 the group endpoints answer. Group paths hide behind a 404 because a chat id is worth hiding.
/admin/...is a fixed path in/openapi.json: nothing to hide, and a 404 would tell an owner with a stale token exactly what it tells an outsider.The scenario that proves it
Every layer, and three that did not exist before:
packages/cb-api/tests/test_admin_endpoints.py— the boundary, the shapes, a parameterised sweep asserting every route is behind the dependencyqa/integration/test_platform_analytics.py— the six fleet queries against real Citus, plus anEXPLAINasserting the fan-out aggregates rather than shipping rowsqa/api/test_contract.py— every response, refusals included, validated against the committedopenapi.jsonwithjsonschema-rsqa/api/test_integration.py— the real app over ASGI against real rowsqa/api/test_smoke.py— a running deployment, over a socketscripts/api_spec.py lintfound 51 documentation gaps on its first run — all fixed here.refusal-schema(rule 10) exists because the contract tests caught what the linter had missed: two routers declared 401/404 with a description and no body model, so the document promised a status with no shape.v1's
/andPOST /loginare no longer exempt from the shape rules. They declare their bodies throughresponses={200: {"model": ...}}withresponse_model=None, which documents them without filtering a single key —COOKIEBOT-WebHubreads those bodies by name and none of them moved.Two fixes found on the way
cb_core/metrics.pycreatedPROMETHEUS_MULTIPROC_DIRinstart_metrics_server, but the multiproc gauges open their files at import. With the variable set (as.env.examplesets it) and the directory absent, every service died during startup. A fresh clone that copied.env.examplehit exactly that.cb-workersettings tests failed for anyone who has a.env— pydantic-settings merges dict fields across sources, so the dotenvCB_BOT_TOKENSleaked into an explicit kwarg.Docs
Testing the API is a six-step guide for whoever tests this rather than writes it, and the API reference is generated from the spec and committed, so a clone can read the API without building anything.
🤖 Generated with Claude Code
https://claude.ai/code/session_01Uqm1RzvYzYvoFANDDqN22v