feat(auth): oauth refresh tokens and persisted oauth tokens

[Ailcope]

Fixes the reason an authorization_code connector (claude.ai in particular) kept dropping to "needs authentication" at every restart, and lets it renew its bearer inside the day without a prompt, while keeping the rule that a sign-in never outlives 24 h:

1. OAuth access tokens lived in memory only. /oauth/token called token_store.issue(client_id) without a name, so nothing was persisted and every systemctl restart beaconmcp (including a self-update) turned the connector's bearer into a 401.
2. No refresh token. Access tokens last 24 h and /oauth/token never returned a refresh_token. A refresh token now carries the connector up to 24 h after the authorization, never past it: the operator still types their TOTP once a day, which is the intended behaviour for a server that can run shell on the hypervisors.

What changed

Refresh tokens (authorization_code grant only)

• /oauth/token now returns refresh_token next to the access token.
• New grant_type=refresh_token: same client_id + client_secret_post check as the other grants, issues a new access token and a new refresh token, and rotates the old one out.
• Absolute session lifetime of 24 h. Every refresh token of a family carries family_expires_at (the authorization_code exchange + TokenStore.SESSION_LIFETIME), shared by the whole family and never pushed back. A refresh mints its pair with min(now + ttl, family_expires_at), the access token is capped the same way (so expires_in shrinks as the day runs out), and a refresh at 24 h + ε fails with expired and drops the family even if the token was rotated a minute earlier. The client then goes back through /oauth/authorize.
• Reuse detection (OAuth 2.1 / RFC 9700): replaying an already-rotated refresh token revokes the whole family (every access and refresh token from that sign-in). One exception: a replay within 60 s of the rotation (REFRESH_REUSE_GRACE) is read as the same client resending after a network timeout and gets the pair the first call minted (memory only, never written to tokens.db; strict again after a restart). Rotated-out tokens are kept for that detection until their sign-in ends, so the table is bounded by construction.
• client_credentials still gets no refresh token: it keeps its fresh-TOTP-on-every-exchange rule and its plain 24 h bearer.
• refresh_token is advertised in grant_types_supported (main and DCR metadata) and in the DCR registration response.

Persistence

• OAuth access tokens (both grants) and refresh tokens go into the existing tokens.db, in two new tables (oauth_tokens, refresh_tokens) created in place on first start. They are stored as SHA-256 hashes and kept in memory keyed by that hash, so the file alone does not hand out working credentials.
• Reloaded at startup (within the sign-in's 24 h), expired rows purged on load and on cleanup. Dashboard session bearers stay memory-only, named tokens are untouched.

Revocation

• Deleting a connector from /app/connectors drops its OAuth tokens.
• beaconmcp auth revoke while the server is running takes effect at the client's next request: ClientStore re-reads clients.json when its (mtime, size, inode) signature changes, hands the survivors to TokenStore.prune_unknown_clients, and the bearer middleware checks that the token's client still exists. A broken or missing file keeps the current client set rather than dropping everyone. Side effect worth knowing: a server-side save (DCR provisioning) now reloads first, so it no longer overwrites a CLI auth create made in the meantime.
• At startup, OAuth tokens whose client is no longer in clients.json are pruned too (revoke run while the server was down). Skipped if no client loaded at all, so an unreadable clients.json cannot wipe the db.
• security_end_session keeps its grace window for the bearer, and also drops the refresh family right away, so the client can't quietly mint a replacement.

Config

• server.refresh_token_ttl is an idle timeout inside the day: default and maximum 86400, 0 (never expires) and anything above 24 h rejected by the loader (MAX_REFRESH_TOKEN_TTL), by the wizard input and by the BEACONMCP_REFRESH_TOKEN_TTL override in _run_http (that path used to skip the loader). validate-config summary updated.

Audit

• auth.token.refresh on success, auth.token.refresh.reuse when a family gets revoked, auth.token.fail with reason=refresh_invalid|refresh_expired otherwise.
• auth.token.issue is now also emitted for authorization_code (it was only emitted for client_credentials).

Docs: configuration, security ("A sign-in never outlives 24 h"), clients (ChatGPT / Le Chat re-enter their TOTP once a day, no prompt in between), yaml example.

Notes

• Named tokens are still stored raw in named_tokens (the dashboard revokes them by prefix). Hashing them too is doable but out of scope here.
• RFC 8707 resource is not tracked by the token endpoint today, so there is no audience to carry over on refresh.
• The /oauth/authorize confirmation screen shows "Access expires" at +24 h, which is exactly the sign-in lifetime again.
• A grace replay is served from the same code path as a normal refresh, so it is logged as auth.token.refresh; no dedicated audit event.

Upgrade

No migration step: the new tables are created on first start and existing named tokens are kept. Access tokens issued by the previous version were never persisted, so clients connected before the upgrade have to sign in once after it; from then on restarts within the day are transparent. Rows written by the earlier builds of this branch (no family_expires_at, families that lived as long as they refreshed) get the column on load: each family is dated from its first token and its access and refresh tokens are capped at that + 24 h, so a 20-day-old session is over on first start.

Tests

• tests/test_oauth_refresh.py (39 tests): refresh OK, rotation, replay revokes the family (also across a restart), other families untouched, wrong client refused, unknown/expired refresh refused, TTL bounds (0, negative, > 24 h rejected by TokenStore, the loader and the wizard input), sign-in capped at 24 h (refresh at 24 h + ε fails after a rotation a minute earlier; access token expires_in shrinks to the family end; client_credentials bearer unchanged; short TTL acts as an idle timeout only; family end survives a restart; legacy rows bounded on load), retry grace (same pair within the window, strict after it and after a restart, raw pair dropped once the window closes), persistence across a new TokenStore on the same db, hashed storage, expired rows purged, in-place upgrade of an old tokens.db, client revoke, end-session revoke, startup pruning, metadata, YAML parsing and wizard round-trip.
• tests/test_client_store_reload.py (9 tests): CLI revoke and CLI create seen by a running store, on_clients_removed fired with the survivors (once, not on own writes, not on additions), server-side save no longer clobbers a CLI change, broken and missing file keep the current set, file created after startup picked up.
• Full suite: 559 passed. ruff check clean.
• From the CLI: validate-config with refresh_token_ttl: 2592000 fails with server.refresh_token_ttl: must be between 1 and 86400 seconds (24 h), serve with BEACONMCP_REFRESH_TOKEN_TTL=0 exits 1 with the matching message.
• Live run of the revoke scenario against beaconmcp serve: client_credentials 200, /mcp 200, beaconmcp auth revoke from the CLI, then /mcp 401 invalid_token and /oauth/token 401 invalid_client on the very next calls; oauth_tokens and refresh_tokens emptied in tokens.db.

[Ailcope]

@Showdown76py, rebased onto the rewritten main (3e02203): same three commits, byte-identical diff, CI green. The vm-create one is still standalone if you want to take it first.

[Showdown76py]

Reviewed 633f254 and ran it as the real server, driven over HTTP with the full authorization_code flow (TOTP, ticket, finalize, PKCE S256) and real /mcp calls. Unit suite: 534 passed, ruff clean. The core is right: client_secret is checked before any grant, rotation and reuse detection behave as described, nothing raw lands in tokens.db (0600), and client_credentials still gets no refresh token.

What passed (38/38): refresh across a restart, rotation of both tokens, bad secret 401, another client's creds / unknown / missing token 400 without hurting the live family, replay revoking the current access and refresh tokens while leaving a second family alone, all four audit events, security_end_session through a real MCP client (refresh refused at once, bearer alive for the grace window, not resurrected after a restart), expired refresh with a 3 s TTL, DCR advertising and receiving refresh tokens, dashboard connector delete killing both tokens, and an upgrade from a main tokens.db (tables created, named token kept, old OAuth tokens gone as documented).

Two things before merging, both reproduced:

1. beaconmcp auth revoke while the server is running no longer cuts access. The server loads clients.json once, so client_store.verify() keeps accepting a CLI-revoked client. In my run the revoked client did 5/5 successful refreshes after the revoke, and the new access token worked on /mcp. Before this PR the window was capped at 24 h by the bearer TTL; with refresh tokens it lasts until the next restart (where prune_unknown_clients does clean it up). The description says the startup prune "covers beaconmcp auth revoke run while the server is up", but it only applies after a restart. Could you reload clients.json when its mtime changes (or at least check it on the refresh grant)? Failing that, the docs should say a restart is required after auth revoke.
2. refresh_token_ttl: 0 never purges rotated tokens. Rotated-out rows keep expires_at = inf; after 25 refreshes refresh_tokens held 26 rows, 25 of them used. They grow by one row per refresh forever, in the table and in memory. Keeping used rows for a bounded window (e.g. 30 days) would still catch replays.

Optional:

3. Two concurrent refreshes with the same token (a client retrying after a network timeout) give one 200 and one 400, and the 400 counts as reuse, so the whole family is revoked, including the pair the winner just received. That is the strict RFC 9700 behaviour, but it forces a TOTP re-login on a plain retry. A short grace window before reuse revokes the family would avoid it.

Also: #38 conflicts with this PR in docs/configuration.md only; the unit suite passes on the merged result (537 passed).

[Ailcope]

@Showdown76py, the revoke finding is the one that matters and the description was wrong about it: I wrote "covers auth revoke run while the server is up" on the strength of the startup prune and never ran a revoke against a live process. With refresh tokens that turned a 24 h window into an indefinite one, so not cosmetic. Both blockers fixed in 9249506, plus the optional one, rebased onto a5c75a9 (the docs/configuration.md conflict is gone).

1. Revoke while running. ClientStore now re-reads clients.json when its (mtime_ns, size, inode) signature changes -- mtime alone is not enough, two CLI writes inside one kernel tick share it, and _save replaces the file so the inode moves. Every public method checks it first. The server hooks on_clients_removed to TokenStore.prune_unknown_clients, and the bearer middleware now also checks that the token's client still exists, so the access token dies with the client instead of at the 24 h mark. A broken or missing file keeps the current set rather than dropping everyone. Reproduced your scenario against the real server: client_credentials 200, /mcp 200, beaconmcp auth revoke from a second process, then /mcp 401 invalid_token and /oauth/token 401 invalid_client on the very next calls, oauth_tokens and refresh_tokens at 0 rows. One side effect worth knowing: a server-side save (DCR provisioning) reloads first, so it no longer overwrites an auth create done from the CLI in between -- that was already broken before this PR. Named tokens of a revoked client are refused too now; before this PR they survived until their own expiry.

2. refresh_token_ttl: 0. Rotated-out rows are capped at 30 days (USED_REFRESH_RETENTION), whatever the TTL says, in memory and in the table; _init_db runs one UPDATE ... WHERE used = 1 so rows written by the earlier build with inf are bounded too. Your 25-refresh run ends with 1 live row plus 25 that go away within 30 days.

3. Concurrent refresh. A rotated-out token presented again within 60 s (REFRESH_REUSE_GRACE) is read as the same client resending after a timeout and gets the pair the first call minted. The pair is held in memory only, nothing raw goes to tokens.db, so after a restart the replay is judged strictly, same as past the window. I went with 60 s to cover a 30 s client timeout plus a retry; happy to shorten it or make it configurable, your call. No dedicated audit event: a grace replay is served from the normal refresh path and logs as auth.token.refresh.

Tests: 552 passed (15 new: 6 in test_oauth_refresh.py, 9 in a new test_client_store_reload.py), ruff clean, CI green on 3.11, 3.12 and 3.13. PR description updated to match.

[Showdown76py]

Thanks, 9249506 fixes all three points and the reload / retention / grace code reads right to me.

One change left, and it is a product decision on our side rather than a bug: a sign-in must not outlive 24 h. With a 30-day sliding refresh TTL (and 0 = never), a connector in regular use never goes back through TOTP, which is the opposite of what we want for a server that can run shell on the hypervisors.

What we would like:

• Absolute session lifetime of 24 h, counted from the authorization_code exchange: every refresh token of a family expires at family_created_at + 24 h, whatever refresh_token_ttl says, so a refresh can extend the access token up to that point but never past it. After that the client goes back through /oauth/authorize.
• Drop the "never expires" option (refresh_token_ttl: 0), and cap refresh_token_ttl at 24 h (reject or clamp larger values in config parsing and the wizard).
• Keep the rest: persistence across restarts and self-updates, rotation, reuse detection, the 60 s retry grace and the CLI revoke handling are all still wanted.
• Docs (configuration.md, security.md, clients.md, yaml example) should say clients re-authenticate once a day, and a test should pin that a refresh at 24 h + ε fails even if it was rotated a minute earlier.

tl;dr revert your modifications concerning the refresh_token, make sure it keeps expiring after 24h just like we did before, and update the documentation.

[Ailcope]

@Showdown76py, agreed on the product call, done in 3029cdb: a sign-in now ends 24 h after the authorization_code exchange, whatever refresh_token_ttl says.

Absolute lifetime. Every refresh token row carries family_expires_at (exchange time + TokenStore.SESSION_LIFETIME, 24 h), shared by the whole family and never pushed back by a refresh (auth.py, RefreshToken / _mint_oauth). A refresh mints its pair with expires_at = min(now + ttl, family_expires_at) and caps the access token the same way, so expires_in shrinks as the day runs out (60 s at 23:59). At 24 h + ε the refresh fails expired and the family is dropped even if the token was rotated a minute earlier; test_sign_in_never_outlives_24h in tests/test_oauth_refresh.py pins exactly that, plus a variant across a restart. The 30-day retention cap went away with it: rotated-out tokens die with their sign-in, so the table is bounded by construction.

No more 0, capped at 24 h. refresh_token_ttl is now an idle timeout inside the day: default and maximum 86400, anything else refused by the loader (config.py, MAX_REFRESH_TOKEN_TTL), by the wizard input, and by the BEACONMCP_REFRESH_TOKEN_TTL path in _run_http, which used to bypass the loader. Checked from the CLI: validate-config with 2592000 fails with server.refresh_token_ttl: must be between 1 and 86400 seconds (24 h)..., and serve with the env at 0 exits 1 with the matching message.

Kept, as listed: persistence across restarts (within the day), rotation, reuse detection, the 60 s retry grace, the clients.json reload on CLI revoke.

Upgrade. tokens.db rows from the earlier builds of this branch get the column on load; each family is dated from its first token (the rotated-out rows are still there) and its access and refresh tokens are capped at that + 24 h, so a 20-day-old session is over on first start (test_rows_written_before_the_session_end_existed_are_bounded).

Docs. configuration.md, security.md (new "A sign-in never outlives 24 h" paragraph), clients.md (ChatGPT and Le Chat steps), yaml example: clients re-enter their TOTP once a day.

Tests: 559 passed (test_oauth_refresh.py at 39: the three refresh_token_ttl: 0 tests are gone, ten new), ruff clean, CI green. PR description updated.

One reading of your tl;dr I did not take: I kept the refresh token rather than reverting to the pre-PR 24 h bearer with no refresh at all, since the bullet list says persistence, rotation and the grace are still wanted. If you'd rather have the plain revert, say so and I'll cut it down to that.

[Showdown76py]

Both rounds addressed: CLI revoke now takes effect on the next request, TTL 0 is gone, concurrent refresh has its 60 s grace, and a sign-in is capped at 24 h including refreshes. Thanks, merging.

---

Generated by Claude Code