postgres: opt-in checksum ledger to skip unchanged schema files#1844
Open
ns-kosti wants to merge 2 commits into
Open
postgres: opt-in checksum ledger to skip unchanged schema files#1844ns-kosti wants to merge 2 commits into
ns-kosti wants to merge 2 commits into
Conversation
ns-kosti
force-pushed
the
postgres/skip-unchanged-schema
branch
2 times, most recently
from
June 24, 2026 13:00
de20609 to
6ed5990
Compare
## What Adds an opt-in `track_schema_checksums` intent flag to the OSS Postgres database provider. When enabled, each schema file's checksum is recorded in a `foundation.schema_checksums` table and the file is only re-applied when its contents change, instead of being re-applied on every `ns deploy`. ## Why The provider currently re-applies every `.sql` file on every deploy. Schema files are expected to be idempotent (e.g. `fn_ensure_table`, `CREATE INDEX IF NOT EXISTS`), so this is safe but wasteful. The checksum table lets us skip files that haven't changed. ## Design - **Opt-in, default false** — existing databases are unaffected until they enable it. - **Checksum** = `sha256:` + hex of the raw file bytes. No canonicalization, so any byte change re-applies; this avoids the correctness risk of a SQL-aware normalizer at the cost of an occasional harmless re-run. - **Table** = `foundation.schema_checksums(path TEXT PRIMARY KEY, checksum TEXT, updated_at TIMESTAMPTZ)`. - **Written only after a successful apply**, so `updated_at` stays constant across redeploys when nothing changed. - **Path validation** — empty and duplicate paths are rejected, since `path` is the primary key. - **Single batched read** of stored checksums (`WHERE path = ANY($1)`); reads and writes retry on retryable errors. - **Logs** each schema file it applies (skipped files are silent). ## Caveat The table records that a file's bytes were applied successfully once; it is not an assertion that the live schema matches the file. With the `fn_ensure_*` helpers, editing a definition changes the checksum (so the file re-runs) but the helper may still no-op against an existing object. It's a re-execution cache. ## Tests Unit tests cover checksum determinism, byte-sensitivity, the `sha256:` prefix, and path validation. End-to-end skip/re-apply coverage is tracked in #1845.
- storedChecksums now uses ReturnFromReadWriteTx for retry/backoff instead of a hand-rolled loop with repeated retryable-error classification. - ApplyWithHelpers branches on the positive track-checksums intent for readability; behavior is unchanged.
ns-kosti
force-pushed
the
postgres/skip-unchanged-schema
branch
from
June 24, 2026 14:00
09b7528 to
a46ed8e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds an opt-in
track_schema_checksumsintent flag to the OSS Postgres database provider. When enabled, each schema file's checksum is recorded in afoundation.schema_checksumstable and the file is only re-applied when its contents change, instead of being re-applied on everyns deploy.Why
The provider currently re-applies every
.sqlfile on every deploy. Schema files are expected to be idempotent (e.g.fn_ensure_table,CREATE INDEX IF NOT EXISTS), so this is safe but wasteful. The checksum table lets us skip files that haven't changed.Design
sha256:+ hex of the raw file bytes. No canonicalization, so any byte change re-applies; this avoids the correctness risk of a SQL-aware normalizer at the cost of an occasional harmless re-run.foundation.schema_checksums(path TEXT PRIMARY KEY, checksum TEXT, updated_at TIMESTAMPTZ).updated_atstays constant across redeploys when nothing changed.pathis the primary key.WHERE path = ANY($1)); reads and writes retry on retryable errors.Caveat
The table records that a file's bytes were applied successfully once; it is not an assertion that the live schema matches the file. With the
fn_ensure_*helpers, editing a definition changes the checksum (so the file re-runs) but the helper may still no-op against an existing object. It's a re-execution cache.Tests
Unit tests cover checksum determinism, byte-sensitivity, the
sha256:prefix, and path validation. End-to-end skip/re-apply coverage is tracked in #1845.