Skip to content

Billing counters: UPDATE first, INSERT ON CONFLICT only on first use - #322

Open
robkooper wants to merge 1 commit into
mainfrom
lum-157-billing-update-first
Open

robkooper wants to merge 1 commit into
mainfrom
lum-157-billing-update-first

Conversation

@robkooper

Copy link
Copy Markdown
Member

What changed

  • New private helper _bump(model, key, increments, insert_values) in lumen/services/llm.py: runs the atomic UPDATE and, only when it matches no row (an entity's first use), falls back to the dialect INSERT ... ON CONFLICT (key) DO UPDATE with the same increments. Postgres or SQLite insert is picked from db.engine.dialect.
  • update_stats (model_stats, entity_stats), subtract_coins (entity_balances) and the chat route's _count_conversation_started use it. The existence SELECTs, savepoints and swallowed IntegrityErrors are gone.
  • First-use balance is inserted already deducted: coins_left = max(0, starting - cost), last_refill_at = now. The UPDATE and the ON CONFLICT branch both apply GREATEST(0, coins_left - cost) to an existing row.
  • Not a plain upsert on every call: Postgres draws the serial id of model_stats / entity_balances before it detects the conflict, so every request would burn a sequence value.
  • Call order and signatures are unchanged. A stale comment in api/routes.py that pointed at the old pattern is updated.

Steady-state billing statements go from SELECT+UPDATE x3 + INSERT to UPDATE x3 + INSERT.

Testing

  • New tests/unit/test_billing_counters.py (SQLite):
    • steady-state update_stats + subtract_coins issue no SELECT and no SAVEPOINT (statement recorder, same technique as test_preflight_queries.py)
    • first use inserts each row once with the right values; first-use balance is floored at 0
    • fallback: rows pre-seeded and the UPDATE forced to report rowcount == 0; the ON CONFLICT branch adds exactly once (and floors the balance)
    • conversation counter: first use and fallback
  • New tests/integration/test_billing_counters_postgres.py (gated on LUMEN_TEST_POSTGRES_URL): 8 threads bill a fresh entity at once, giving requests == 8, balance == starting - sum(cost), no errors; 10 steady-state bills leave model_stats_id_seq and entity_balances_id_seq unchanged. This runs in CI; no local Postgres was available.
  • Existing test_llm_coins.py, test_entity_stat.py and test_chat_routes.py still pass. The full SQLite suite and ruff check pass.

Part of #171

update_stats, subtract_coins and the conversation counter no longer SELECT
the counter row before bumping it. A new _bump helper runs the atomic
UPDATE and, only when it matches no row, falls back to the dialect
INSERT ... ON CONFLICT DO UPDATE with the same increments, which also
covers concurrent first requests. The savepoints and swallowed
IntegrityErrors go away. A steady-state bill is one UPDATE per counter
and draws no serial id.

Part of #171
@robkooper

Copy link
Copy Markdown
Member Author

How to test
Configure the README's local dev user and dummy backend with a model that has nonzero pricing and a finite coin pool. Send two messages in separate new chats, then open Usage and the user's balance: each request should count once, the balance should fall by the reported costs without going below zero, and the conversation total should increase by two. Repeat for a fresh user to exercise first-use counter creation.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant