Skip to content

Bill every request through one function that takes the hot locks last - #323

Merged
robkooper merged 2 commits into
lum-157-billing-update-firstfrom
lum-158-bill-request
Oct 29, 2026
Merged

robkooper merged 2 commits into
lum-157-billing-update-firstfrom
lum-158-bill-request

Conversation

@robkooper

Copy link
Copy Markdown
Member

What changed

  • New bill_request in lumen/services/llm.py: the single billing write path. It writes in a fixed order: INSERT request_logs (flushed), UPDATE model_stats, UPDATE entity_stats, UPDATE api_keys (API only), UPDATE entity_balances, then COMMIT. The per-entity balance lock is held only for the commit, and because every billing transaction takes its locks in the same order, two of them cannot deadlock.
  • update_stats now adds and flushes the RequestLog before the counter updates.
  • _record_api_key_usage moves from lumen/blueprints/api/routes.py into lumen/services/llm.py.
  • record_aborted_request / record_stream_abort take api_key_id instead of the record_extra lambda.
  • All 5 call sites use bill_request: chat stream, aborted streams, API non-stream, API stream, audio.
  • Chat route, storage on: the new-conversation counter (entity_stats) is bumped after the message adds, after an explicit flush, right before the commit, so its lock covers only the commit.

Testing

  • New tests/routes/test_billing_queries.py (SQLite): records write statements plus an UPSTREAM marker and a COMMIT marker from an engine commit listener. For API non-stream, API stream, audio, an aborted API stream and the chat stream, the tables written between UPSTREAM and the first COMMIT are exactly request_logs, model_stats, entity_stats, [api_keys], entity_balances. The chat case runs with storage on and also asserts that the conversation transaction writes entity_stats last, before its own COMMIT. I checked that the tests fail if the old order is put back in bill_request or in the chat route.
  • Updated the steady-state statement test from the previous PR for the new order, and moved three billing-failure tests to patch the new seams.
  • uv run pytest passes and ruff check is clean. I could not run the Postgres-gated suites (including tests/integration/test_disconnect.py) locally; CI runs them.

Stacked on #322

Part of #171

@robkooper
robkooper added this pull request to stack #324 October 28, 2026 06:46
@robkooper

Copy link
Copy Markdown
Member Author

How to test
Run Lumen against a configured backend with a finite coin allowance and nonzero model pricing. Send a normal chat in the UI with conversation storage enabled, then make non-streaming and streaming API completions using one API key. Check Usage, the key's totals and the balance: each completion should be recorded once with matching usage/cost, and the UI conversation should contain both messages. Interrupt a longer API stream and confirm one aborted request is recorded and its consumed usage is charged to the same key.

Comment thread lumen/services/llm.py
db.session.flush()


def bill_request(entity_id: int, model_config_id: int, source: str, input_tokens: int, output_tokens: int,

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add something to AGENTS.md to reflect this as well?

All five billing paths (chat stream, aborted streams, API non-stream,
API stream, audio) now call one function that writes in a fixed order:
INSERT request_logs, UPDATE model_stats, UPDATE entity_stats, UPDATE
api_keys (API only), UPDATE entity_balances, COMMIT. The per-entity
balance lock is held only for the commit, and a single lock order means
two billing transactions cannot deadlock.

update_stats now inserts and flushes the RequestLog before the counter
updates, _record_api_key_usage moves into the llm service, and the
abort helpers take api_key_id instead of a record_extra callback. The
chat route bumps the new-conversation counter as the last write before
its commit, so that entity_stats lock also covers only the commit.

Part of #171
@robkooper
robkooper force-pushed the lum-158-bill-request branch from 9360a8d to 9cf5580 Compare October 29, 2026 14:21
@robkooper
robkooper merged commit ee22692 into main Oct 29, 2026
22 checks passed
@robkooper
robkooper deleted the lum-158-bill-request branch October 29, 2026 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant