Why
Moving the full refresh/dump to a daily cadence is blocked by timestamp churn.
app.dump is a stateless rebuild: every run stamps created_at/updated_at with the build time, so even when the underlying data/ is unchanged it emits a ~1400-file diff. On a daily schedule this means a giant no-op PR to TechAPI every day, drowning real changes and defeating review. (This is the same churn that was hit and reverted during a manual dump.)
What
Make dump output deterministic so a re-run with unchanged data produces no diff:
- preserve
created_at (never rewrite once set), and
- only bump
updated_at when the record's substantive fields actually change, or
- drop build-time timestamps from the static dump entirely / pin them (e.g.
SOURCE_DATE_EPOCH).
Unblocks
Once dump diffs reflect only real data changes, weekly-refresh (or a split dump-only job) can safely move to daily. Until then it stays weekly.
Related
Scheduling split landed in b53bcab (coverage-report daily; scrape + ingest weekly).
Why
Moving the full refresh/dump to a daily cadence is blocked by timestamp churn.
app.dumpis a stateless rebuild: every run stampscreated_at/updated_atwith the build time, so even when the underlyingdata/is unchanged it emits a ~1400-file diff. On a daily schedule this means a giant no-op PR to TechAPI every day, drowning real changes and defeating review. (This is the same churn that was hit and reverted during a manual dump.)What
Make dump output deterministic so a re-run with unchanged data produces no diff:
created_at(never rewrite once set), andupdated_atwhen the record's substantive fields actually change, orSOURCE_DATE_EPOCH).Unblocks
Once dump diffs reflect only real data changes,
weekly-refresh(or a split dump-only job) can safely move to daily. Until then it stays weekly.Related
Scheduling split landed in b53bcab (coverage-report daily; scrape + ingest weekly).