Say why the live feed keys records, and stop citing an upsert - #75
Open
jerome-queck wants to merge 1 commit into
Open
Say why the live feed keys records, and stop citing an upsert#75jerome-queck wants to merge 1 commit into
jerome-queck wants to merge 1 commit into
Conversation
The two comments in live-feed.ts justified keying records in memory by an upsert MongoDB is doing upstream. It is not: upstream's real-time ingestor appends a row per revision (InsertOne, never ReplaceOne) and there is no index on the key it would upsert on. The collapsing to one record per key happens in the query API, on read. The behaviour the comments defend is right, and its real reason is better — this feed is handed the revision stream with no query API in front of it, so it has to collapse the revisions itself in order to agree with what rest-feed.ts is given for the same Session. Comments only; nothing executable changes. Investigated in #74. Assisted-by: Claude Opus 5 (reasoning: high) Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two comments in
server/openf1/live-feed.tsexplained the in-memory keying by an upsert that doesnot exist. Upstream's real-time ingestor writes
InsertOneper emission(
real_time/processing.py:104→util/db.py:348-357), it never calls theupsert_data_syncthatsits next to it, and it creates no index on
_keyfor MongoDB to key on. One record per_keyiswhat the query API answers with, by grouping on
_keyand taking the highest_id(
util/db.py:66-75).The keying itself is correct and stays. Only the reason changes, and the true one is the more
useful: this feed reads the revision stream off MQTT with no query API in front of it, so it must
collapse the revisions itself for the live path to agree with what
rest-feed.tsreceives for thesame Session.
Comments only — no executable change,
test/rungreen.The measured mechanism is in #74, which this does not close: what remains there is whether to record
the rule as an ADR, and that is the author's call.
Assisted-by: Claude Opus 5 (reasoning: high)
Co-authored-by: Claude noreply@anthropic.com