Repository navigation
Bill every request through one function that takes the hot locks last - #323
Merged
Merged
Conversation
Member
Author
|
How to test |
robkooper
commented
Oct 29, 2026
| db.session.flush() | ||
|
|
||
|
|
||
| def bill_request(entity_id: int, model_config_id: int, source: str, input_tokens: int, output_tokens: int, |
Member
Author
There was a problem hiding this comment.
Should we add something to AGENTS.md to reflect this as well?
All five billing paths (chat stream, aborted streams, API non-stream, API stream, audio) now call one function that writes in a fixed order: INSERT request_logs, UPDATE model_stats, UPDATE entity_stats, UPDATE api_keys (API only), UPDATE entity_balances, COMMIT. The per-entity balance lock is held only for the commit, and a single lock order means two billing transactions cannot deadlock. update_stats now inserts and flushes the RequestLog before the counter updates, _record_api_key_usage moves into the llm service, and the abort helpers take api_key_id instead of a record_extra callback. The chat route bumps the new-conversation counter as the last write before its commit, so that entity_stats lock also covers only the commit. Part of #171
robkooper
force-pushed
the
lum-158-bill-request
branch
from
October 29, 2026 14:21
9360a8d to
9cf5580
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
bill_requestinlumen/services/llm.py: the single billing write path. It writes in a fixed order: INSERTrequest_logs(flushed), UPDATEmodel_stats, UPDATEentity_stats, UPDATEapi_keys(API only), UPDATEentity_balances, then COMMIT. The per-entity balance lock is held only for the commit, and because every billing transaction takes its locks in the same order, two of them cannot deadlock.update_statsnow adds and flushes theRequestLogbefore the counter updates._record_api_key_usagemoves fromlumen/blueprints/api/routes.pyintolumen/services/llm.py.record_aborted_request/record_stream_aborttakeapi_key_idinstead of therecord_extralambda.bill_request: chat stream, aborted streams, API non-stream, API stream, audio.entity_stats) is bumped after the message adds, after an explicit flush, right before the commit, so its lock covers only the commit.Testing
tests/routes/test_billing_queries.py(SQLite): records write statements plus an UPSTREAM marker and a COMMIT marker from an enginecommitlistener. For API non-stream, API stream, audio, an aborted API stream and the chat stream, the tables written between UPSTREAM and the first COMMIT are exactlyrequest_logs, model_stats, entity_stats, [api_keys], entity_balances. The chat case runs with storage on and also asserts that the conversation transaction writesentity_statslast, before its own COMMIT. I checked that the tests fail if the old order is put back inbill_requestor in the chat route.uv run pytestpasses andruff checkis clean. I could not run the Postgres-gated suites (includingtests/integration/test_disconnect.py) locally; CI runs them.Stacked on #322
Part of #171