Order the public article listing by publish date - #210
Merged
Conversation
A story published this morning never reached the homepage. It was published, filed under News and served fine at its own URL, but sat at offset 10027 of 10050 in the listing, wedged between rows from 2011. The public listing ordered by `id`, so placement followed insert order rather than publish date. The article held id 77 while everything else published that day held ids above 10082 -- an id the CMS issued before an ETL reseed loaded the legacy archive above it. Ordering by `id` put it ten thousand rows down, well past the 13 slots the homepage news block reads. Sort public reads by `pub_date` DESC, `id` DESC instead. That is one change for the homepage, the section pages and /v1/articles, since all three go through queryArticles, and it matches what search and GetFeaturedArticle already did. `id` stays as the tiebreak because a whole issue publishes on one timestamp and needs a stable order in it. Editors keep `id` DESC: their listing carries drafts, whose `pub_date` is NULL and would sort to the very end, burying a new draft on the last page of a 10k-article list. That is the reasoning articleOrderByClause already encodes for their explicit date sorts. Add idx_articles_pub_date (pub_date, id) so the new ordering walks an index instead of filesorting the corpus on every public request. It is wired in non-fatally, like the search indexes -- the ordering is correct without it, it is just slower. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Erik reported that a story published this morning never appeared on the home page. It was published, filed under News, and served fine at its own URL — it just was not in the listing anywhere a reader would look.
What was wrong
queryArticlesordered the public listing byid, so placement followed insert order rather than publish date.The article held
id77. Everything else published that day held ids above 10082 — 77 is an id the CMS issued before an ETL reseed loaded the legacy archive above it. Ordering byidput a story published at 9am today at offset 10027 of 10050, between rows from 2011:The homepage news block reads 13 slots, so it never had a chance. I swept all 10,050 articles: this is currently the only 2026-dated row with an id below 9000, but any other draft created before the reseed will do the same thing the day it is published.
The fix
Public reads now sort by
pub_date DESC, id DESC. One change covers the homepage, the section pages and/v1/articles, since all three go throughqueryArticles, and it matches what search andGetFeaturedArticlealready did.idstays as the tiebreak — a whole issue publishes on one timestamp and needs a stable order within it.Editors keep
id DESC. Their listing carries drafts, whosepub_dateis NULL and would sort to the very end, burying a new draft on the last page of a 10k-article list. That is the same reasoningarticleOrderByClausealready encodes for their explicit date sorts, and there is a test pinning it so the two branches do not later get collapsed into one.Also adds
idx_articles_pub_date (pub_date, id), wired in non-fatally alongside the other index builds — without it the new ordering filesorts the whole migrated corpus on every public request. Verified against MariaDB 11.8 that the ALTER is idempotent and the planner picks it up (range idx_articles_pub_date, noUsing filesort).Testing
New integration test seeds the exact live shape: a low-id row with today's
pub_dateamong high-id archive rows. I checked it is not vacuous by reverting the fix and confirming it fails with the production symptom:go build,go vetand the fullgo test ./...pass on this branch's merge base, run against a MariaDB 11.8 container so the integration tests actually executed rather than skipping on an absentCMS_TEST_DSN.Notes for review
creation_date. The drafted-before-the-reseed theory fits all the evidence but is unverified. Worth someone with DB access confirming, since it predicts more of these as old drafts get published.🤖 Generated with Claude Code