Skip to content

chore: normalise createdAt before the feed starts ordering on it - #17

Merged
duuyguyiilmaz merged 1 commit into
mainfrom
chore/backfill-created-at
Aug 31, 2026
Merged

duuyguyiilmaz merged 1 commit into
mainfrom
chore/backfill-created-at

Conversation

@duuyguyiilmaz

Copy link
Copy Markdown
Owner

Groundwork for pagination. Paginating the feed and the leaderboard means ordering in the query rather than sorting in the client — and Firestore's ordering has two behaviours that turn old data into damage a user can see:

  • A document that lacks the ordered field is excluded from the query. Not sorted last: absent. A note written before createdAt existed would vanish from the feed with nothing to indicate it ever had.
  • Values sort by type before value, and numbers come before timestamps. A note whose createdAt is epoch millis would land below every timestamped note whatever its date.

toPost() already reads both shapes, and notes from an older version of the app do exist, so both are real. Neither announces itself: the first is a note quietly missing, the second a feed quietly in the wrong order.

The migration

backfill-created-at.js normalises every note to a Timestamp.

  • Reports by default, writes only with --apply
  • Idempotent — notes already carrying a Timestamp are untouched, so a partial failure can simply be re-run
  • Batches writes 450 at a time (Firestore's limit is 500)
  • A note with no date anywhere is dated to the epoch, not to now: inventing a recent date would float undated legacy notes to the top of every feed, which is a louder lie than showing them last. The report says how many fall into that case before anything is written.
cd firestore-tests
npm run backfill:created-at -- --project <id>            # reports only
npm run backfill:created-at -- --project <id> --apply    # writes

Verification

Eight tests run it against the emulator with legacy-shaped documents. The one that carries the argument asserts the hazard directly: an ordered query over two notes, one of them undated, returns one document before the migration and two after.

The CLI path was also run end to end against the emulator — the dry run reported 3 scanned and 2 needing a write and changed nothing; --apply then wrote 2.

Firestore suite is 32 → 40 tests.

Two choices worth naming

This ships before the pagination it enables. Migrating the data and changing the query in one commit would mean the ordering going live against un-normalised documents if the second half failed.

It lives in firestore-tests/ rather than a second Node project: same emulator, same dependencies, same CI job — and the migration needs that emulator to be verified at all. The package description now says so.

🤖 Generated with Claude Code

Pagination needs the feed and the leaderboard to order in the query rather than sort in
the client, and Firestore's ordering has two behaviours that turn old data into damage
a user can see:

- A document that lacks the ordered field is excluded from the query. Not sorted last —
  absent. A note written before createdAt existed would vanish from the feed with
  nothing to indicate it ever had.
- Values sort by type before value, and numbers come before timestamps. A note whose
  createdAt is epoch millis would land below every timestamped note whatever its date.

toPost() already reads both shapes, and the project owner confirms notes from an older
version of the app exist, so both are real. Neither announces itself: the first is a
note quietly missing, the second a feed quietly in the wrong order.

backfill-created-at.js normalises every note to a Timestamp. It reports by default and
writes only with --apply, leaves notes that are already correct alone so a partial
failure can simply be re-run, and batches writes 450 at a time. A note with no date
anywhere is dated to the epoch rather than to now: inventing a recent date would float
undated legacy notes to the top of every feed, which is a louder lie than showing them
last. The report says how many fall into that case before anything is written.

Eight tests run it against the emulator with legacy-shaped documents. The one that
carries the argument asserts the hazard directly: an ordered query over two notes, one
of them undated, returns one document before the migration and two after. The CLI path
was also run end to end against the emulator — dry run reported 3 scanned, 2 needing a
write, and changed nothing; --apply then wrote 2.

This deliberately ships before the pagination it enables. Migrating the data and
changing the query in one commit would mean the ordering going live against
un-normalised documents if the second half failed.

Kept in firestore-tests/ rather than a second Node project: it is the same emulator, the
same dependencies and the same CI job, and the migration needs that emulator to be
verified at all. The package description now says so.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@duuyguyiilmaz
duuyguyiilmaz merged commit 65ff36c into main Aug 31, 2026
2 checks passed
@duuyguyiilmaz
duuyguyiilmaz deleted the chore/backfill-created-at branch August 31, 2026 13:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant