fix(rollup): preserve provider through the rollup index#4
Merged
Conversation
query_rollups was hard-coding provider: String::new() in the response because: 1. EngineConfig.rollup_dims didn't include DIM_PROVIDER, so the keplordb rollup index aggregated by (user, api_key, model) and dropped provider entirely from the grouping key. 2. RollupKey in kdb_store.rs only carried four fields and read dim[0..3]; provider had no slot. Both consequences cascaded: /v1/rollups returned provider="" for every row, /v1/stats?group_by=model inherited the empty provider from rollups, and downstream consumers (Obol's BY PROVIDER chart) collapsed every event onto a single empty bucket. Fix: - rollup_dims now [USER_ID, API_KEY_ID, MODEL, PROVIDER] - RollupKey gains a provider field, read from key.dims.get(3) - dim_filters extended from [3] to [4] to align with the index - RollupRow.provider is now k.provider instead of String::new() The rollup index is in-memory and rebuilt from segments on open (rollup_replay_days, default 7). Existing event data is unchanged; restart triggers a one-time re-aggregation with provider as a grouping key. Verified live: 22 mistral + 61 groq events now report their providers correctly. 117 / 117 tests pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
`/v1/rollups` and `/v1/stats?group_by=model` were returning `provider=""` for every row, even though events were ingested with `provider="mistral"` / `"groq"` / etc. Anything downstream that grouped by provider collapsed onto a single empty bucket — Obol's BY PROVIDER chart was the visible victim.
Two-part root cause:
Fix
Compatibility: the rollup index is in-memory and rebuilt from segments on open (`rollup_replay_days`, default 7). Existing event data on disk is unchanged; a restart triggers a one-time re-aggregation with provider as a grouping key. No migration step needed.
Verified live
Deployed manually to a running keplor instance ahead of merge. Before / after:
Test plan