Repository navigation
fix(ranking): sample the rendered feed, not just the ranked window (GEO-2950) - #967
Merged
Merged
Conversation
…EO-2950) The canary measured entities_ranked_for_feed_by_type's output. Nobody sees that. Explore applies read-time reordering in geogenesis before rendering, and on 2026-09-17 the same feed was Claim 24 / Debate 8 / News 34 in the window while readers were served 1.8 / 5.9 / 2.3 per 10. The canary's numbers were true and about a feed nobody looks at -- it would not have moved when Preston reported the claim shortfall. Records BOTH sources, because the gap is the diagnostic: a change in both is a ranking change, a change in rendered alone is read-time reordering, a change in window alone is something no reader saw. That distinction was the whole answer today and was not available. window samples stay in SQL. rendered needs an HTTP call to the app's own feed endpoint, so the binary measures it and record_feed_composition_sample stores it -- one table, one shape. feed_composition_drift now takes a source so the two histories are never compared against each other, which would fire on every run. Verified end to end: the binary's rendered sample independently reproduces the 4/13/5 measured by hand on production.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The canary I shipped this morning was measuring a feed nobody sees.
What went wrong
0091/0092 sample the ranked window — the rows
entities_ranked_for_feed_by_typereturns. Explore then reorders that in geogenesis (applyDiversityCap, and with geogenesis#2456applyTargetMix, thenapplyPerSpaceQuota) before rendering. Measured on production today, featured spaces, the default three types:So its
Claim 24 / Debate 8 / News 34was true and useless. It would not have moved when Preston reported the feed had gone claim-starved, because the window never changed — the reordering did. It took three wrong answers today to find that.Record both, because the gap is the diagnostic
Not a repoint. A composition change visible in both sources is a ranking change; visible in
renderedalone it is read-time reordering; visible inwindowalone it is something no reader ever saw. That distinction was the entire answer today and the canary could not express it.windowsamples stay in SQL (sample_feed_composition, unchanged).renderedneeds an HTTP call to the app's own feed endpoint, so the binary measures it andrecord_feed_composition_samplestores it — one table, one shape, config snapshot taken at write time for both.feed_composition_driftnow takes asource, so the two histories are never compared against each other. They legitimately differ, so a cross-source comparison would fire on every single run and the alert would be worthless within a day.FEED_COMPOSITION_FEED_URLis optional: unset, the job still runs and records the window only.The test harness earned its keep
Adding the
sourceparameter tofeed_composition_driftcreated a sibling overload rather than replacing the function, so every existing one-argument call broke with:The SQL suites caught it immediately — 6 failures across 0091, 0092 and 0093 — before it could reach a deploy. 0077 and 0085 both hit this same trap and document it; the fix is dropping the old signature first, and drizzle runs a migration in one transaction so no caller sees the gap.
Verified
tests/0093: the source guard, verbatim storage with a write-time config snapshot, and the assertion this exists for — window and rendered are separate histories, so two rendered samples plus one window sample yield drift for the first and none for the second.cargo clippy -- -D warningsandcargo fmtclean.Deploy
Migration 0093 applies via the api's
migrateinitContainer; the binary and the new env ship with the nextranking-indexerdeploy. Existing rows default tosource = 'window', which is what they were.