Skip to content

Conversation

@samrose
Copy link
Collaborator

@samrose samrose commented Jan 27, 2026

The Problem

Users upgrading via Supabase UI encounter this error:
old cluster uses data checksums but the new one does not

Root Cause

Commit cf9bce1 (Nov 14, 2025) enabled data checksums for new PostgreSQL
installations, but the pg_upgrade scripts were not updated to match.

Affected Files
File: ansible/tasks/setup-postgres.yml
Has --data-checksums?: Yes
Lines: https://github.com/supabase/postgres/blob/develop/ansible/tasks/setup-postgres.yml#L232,
https://github.com/supabase/postgres/blob/develop/ansible/tasks/setup-postgres.yml#L260,
https://github.com/supabase/postgres/blob/develop/ansible/tasks/setup-postgres.yml#L277
────────────────────────────────────────
File: ansible/files/admin_api_scripts/pg_upgrade_scripts/initiate.sh
Has --data-checksums?: No
Lines: https://github.com/supabase/postgres/blob/develop/ansible/files/admin_api_scripts/pg_upgrade_scripts/initiate.sh#L470,
https://github.com/supabase/postgres/blob/develop/ansible/files/admin_api_scripts/pg_upgrade_scripts/initiate.sh#L472,
https://github.com/supabase/postgres/blob/develop/ansible/files/admin_api_scripts/pg_upgrade_scripts/initiate.sh#L475
What Happens

  1. User has a database created after Nov 14, 2025 → checksums enabled via setup-postgres.yml
  2. User initiates upgrade via Supabase UI
  3. initiate.sh runs initdb to create new cluster without --data-checksums
  4. pg_upgrade fails because old cluster has checksums but new cluster doesn't

The Fix

Detect whether the old cluster has checksums enabled using pg_controldata, then conditionally pass --data-checksums to initdb.

Add detection after PGDATAOLD is set (after line 244):

Check if old cluster has data checksums enabled

CHECKSUM_VERSION=$("$PGBINOLD/pg_controldata" "$PGDATAOLD" | grep -i checksum | awk '{print $NF}')
if [ "$CHECKSUM_VERSION" != "0" ]; then
CHECKSUM_FLAG="--data-checksums"
else
CHECKSUM_FLAG=""
fi

Update all three initdb calls to use $CHECKSUM_FLAG:

  • Line 470 (NIX upgrade, PG16+):
    $PGBINNEW/initdb $CHECKSUM_FLAG --encoding=$SERVER_ENCODING --locale-provider=icu ...
  • Line 472 (NIX upgrade, pre-PG16):
    $PGBINNEW/initdb $CHECKSUM_FLAG --encoding=$SERVER_ENCODING --lc-collate=...
  • Line 475 (non-NIX upgrade):
    $PGBINNEW/initdb $CHECKSUM_FLAG -L $PGSHARENEW -D $PGDATANEW/ --username=supabase_admin

Summary by CodeRabbit

  • New Features

    • Database upgrades now detect and preserve data checksum configuration from the source cluster.
  • Chores

    • Updated PostgreSQL versions: postgresorioledb-17 to 17.6.0.032, postgres17 to 17.6.1.075, and postgres15 to 15.14.1.075.

✏️ Tip: You can customize this high-level summary in your review settings.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Jan 27, 2026

Walkthrough

The PR modifies the PostgreSQL upgrade initialization script to detect data checksums on the old cluster and propagate the checksum setting to the new database initialization. Additionally, it updates three PostgreSQL release version strings for different variants.

Changes

Cohort / File(s) Summary
PostgreSQL upgrade checksum propagation
ansible/files/admin_api_scripts/pg_upgrade_scripts/initiate.sh
Introduces detection of data checksums from the old cluster via pg_controldata, sets a CHECKSUM_FLAG variable, and propagates it to all new database initialization invocations in both nix-based and non-nix upgrade paths.
Version updates
ansible/vars.yml
Updates postgres_release version strings: postgresorioledb-17 (17.6.0.031-orioledb → 17.6.0.032), postgres17 (17.6.1.074 → 17.6.1.075), and postgres15 (15.14.1.074 → 15.14.1.075).

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

Suggested reviewers

  • hunleyd
  • LGUG2Z
🚥 Pre-merge checks | ✅ 1 | ❌ 2
❌ Failed checks (1 warning, 1 inconclusive)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ❓ Inconclusive The description comprehensively explains the problem, root cause, affected files, failure scenario, and proposed fix with specific code examples, but references a template system that requires selecting from sub-templates (Default or Extension Upgrade), which was not followed. Clarify which template sub-template (Default or Extension Upgrade) applies to this PR and verify the description structure aligns with the chosen template's required sections.
✅ Passed checks (1 passed)
Check name Status Explanation
Title check ✅ Passed The title 'fix: add checksum to upgrade scripts' directly and clearly summarizes the main change—adding checksum detection and propagation to the PostgreSQL upgrade scripts to resolve the checksum mismatch error.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@samrose samrose marked this pull request as ready for review January 28, 2026 19:16
@samrose samrose requested review from a team as code owners January 28, 2026 19:16
@samrose
Copy link
Collaborator Author

samrose commented Jan 28, 2026

Tested upgrade with various checksum/no checksum permutations, pause and restore and functionality. Ready for review and merge

@hunleyd
Copy link
Contributor

hunleyd commented Jan 28, 2026

@jchancojr take a peek?

Copy link
Contributor

@jchancojr jchancojr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm ty!

@samrose samrose enabled auto-merge January 28, 2026 19:23
@samrose
Copy link
Collaborator Author

samrose commented Jan 28, 2026

============================================================
Supadev PostgreSQL Engine Test Suite
============================================================
[INFO] Testing PostgresEngines: 17
[INFO] Using region: us-east-1
[INFO] Project prefix: supadev-test-1769638075
[INFO] Version: 17.6.1.074-checksum-1


============================================================
Testing PostgreSQL Engine: 17
============================================================
[INFO] Project name: supadev-test-1769638075-pg17
[INFO] Region: us-east-1
[INFO] Start time: 2026-01-28 17:07:55
[INFO] Using explicit version: 17.6.1.074-checksum-1
[STEP] Creating project with PostgresEngine 17...
🧐 🐘 postgres_dba 6.0 installed. Use ":dba" to see menu
UPDATE 0
INSERT 0 0
UPDATE 1
UPDATE 1
  id  |                 version                 |  region   |   provider_image_id   | release_channel | postgres_engine 
------+-----------------------------------------+-----------+-----------------------+-----------------+-----------------
 7719 | supabase-postgres-17.6.1.074-checksum-1 | us-east-1 | ami-0828e1e13bb58e589 | ga              | 17
(1 row)


Creating project on release channel "ga" with version "17.6.1.074-checksum-1"
Project URL: http://localhost:8082/project/ryysurwywricuvahkjfx
Admin URL: http://localhost:4000/projects?identifier=ryysurwywricuvahkjfx
Database status is not yet available, waiting...
Database status is currently UNKNOWN, waiting...
Database status is currently COMING_UP, waiting...
Database status is currently COMING_UP, waiting...
Database status is now ACTIVE_HEALTHY
[SUCCESS] Project supadev-test-1769638075-pg17 created and reached ACTIVE_HEALTHY status
[INFO] Time to ACTIVE_HEALTHY: 201 seconds
[INFO] Sleeping for 10 seconds before checking project health
[STEP] Checking project health for supadev-test-1769638075-pg17...
Error: 
   0: environment variable not found

Location:
   src/platform.rs:806

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ BACKTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
                                ⋮ 6 frames hidden ⋮                               
   7: supadev::platform::Platform::project_health_get::he11e303c297927d1
      at <unknown source file>:<unknown line>
   8: supadev::platform::Platform::project_health::h8c22d87ad38428e7
      at <unknown source file>:<unknown line>
   9: supadev::main::h3ebb20626023f6cb
      at <unknown source file>:<unknown line>
  10: std::sys::backtrace::__rust_begin_short_backtrace::hcfc601fdab066f90
      at <unknown source file>:<unknown line>
  11: std::rt::lang_start::{{closure}}::h0b54837a8fc1253b
      at <unknown source file>:<unknown line>
  12: std::rt::lang_start_internal::hd700ba983d3377dc
      at <unknown source file>:<unknown line>
  13: _main<unknown>
      at <unknown source file>:<unknown line>

Run with COLORBT_SHOW_HIDDEN=1 environment variable to disable frame filtering.
Run with RUST_BACKTRACE=full to include source snippets.
[STEP] Printing project information for supadev-test-1769638075-pg17
{
  "cloud_provider": "AWS",
  "db_host": "db.ryysurwywricuvahkjfx.supabase.red",
  "id": 2,
  "inserted_at": "2026-01-28T22:08:03.203589",
  "name": "supadev-test-1769638075-pg17",
  "organization_id": 1,
  "ref": "ryysurwywricuvahkjfx",
  "region": "us-east-1",
  "status": "ACTIVE_HEALTHY",
  "subscription_id": "my97RENkzngskhj5",
  "connectionString": "U2FsdGVkX1/VAipIZnIXv9BXT4vBVBvDRyWgc/txVO5G6RHHW8JBUCCeFCH0J8hgy6XWlxTxklfuft00zHCMYmCUXSOi2tamRuOAye2voUraau+rEkuCyxZ7qJUJEl9jHkCy8qmc9kuLa+6W1nij5xVysY3RwKDbbwkz/xZ5yNtfatrL4fl8YAHPy/nm+l1b",
  "restUrl": "https://ryysurwywricuvahkjfx.supabase.red/rest/v1/",
  "volumeSizeGb": 8,
  "maxDatabasePreprovisionGb": null,
  "lastDatabaseResizeAt": null,
  "is_branch_enabled": false,
  "is_physical_backups_enabled": true,
  "infra_compute_size": "nano",
  "dbVersion": "17.6.1.074-checksum-1"
}
[STEP] Deleting project supadev-test-1769638075-pg17...
Project deleted
[SUCCESS] Project supadev-test-1769638075-pg17 deleted successfully
[SUCCESS] PostgresEngine 17 test completed successfully
[INFO] Total test duration: 216 seconds


============================================================
Cleaning Up: Withdrawing Inserted Versions
============================================================
[STEP] Withdrawing version 17.6.1.074-checksum-1 for engine 17...
🧐 🐘 postgres_dba 6.0 installed. Use ":dba" to see menu
UPDATE 0
INSERT 0 0
UPDATE 1
UPDATE 1
  id  |                 version                 |  region   |   provider_image_id   | release_channel | postgres_engine 
------+-----------------------------------------+-----------+-----------------------+-----------------+-----------------
 7720 | supabase-postgres-17.6.1.074-checksum-1 | us-east-1 | ami-0828e1e13bb58e589 | withdrawn       | 17
(1 row)


[SUCCESS] Version 17.6.1.074-checksum-1 marked as withdrawn

============================================================
Test Summary
============================================================

ENGINE          RESULT
------          ------
17              PASSED (216s)

============================================================
Total: 1 | Passed: 1 | Failed: 0
============================================================
[SUCCESS] All tests passed!

@samrose
Copy link
Collaborator Author

samrose commented Jan 28, 2026

============================================================
Supadev PostgreSQL Engine Test Suite
============================================================
[INFO] Testing PostgresEngines: 15
[INFO] Using region: us-east-1
[INFO] Project prefix: supadev-test-1769638372
[INFO] Version: 15.14.1.074-checksum-1


============================================================
Testing PostgreSQL Engine: 15
============================================================
[INFO] Project name: supadev-test-1769638372-pg15
[INFO] Region: us-east-1
[INFO] Start time: 2026-01-28 17:12:52
[INFO] Using explicit version: 15.14.1.074-checksum-1
[STEP] Creating project with PostgresEngine 15...
🧐 🐘 postgres_dba 6.0 installed. Use ":dba" to see menu
UPDATE 0
INSERT 0 1
UPDATE 1
UPDATE 1
  id  |                 version                  |  region   |   provider_image_id   | release_channel | postgres_engine 
------+------------------------------------------+-----------+-----------------------+-----------------+-----------------
 7722 | supabase-postgres-15.14.1.074-checksum-1 | us-east-1 | ami-03a4c50234d98e6f6 | ga              | 15
(1 row)


Creating project on release channel "ga" with version "15.14.1.074-checksum-1"
Project URL: http://localhost:8082/project/ilnqwlfmvbopenznwtfc
Admin URL: http://localhost:4000/projects?identifier=ilnqwlfmvbopenznwtfc
Database status is not yet available, waiting...
Database status is not yet available, waiting...
Database status is not yet available, waiting...
Database status is not yet available, waiting...
Database status is not yet available, waiting...
Database status is not yet available, waiting...
Database status is not yet available, waiting...
Database status is not yet available, waiting...
Database status is not yet available, waiting...
Database status is not yet available, waiting...
Database status is not yet available, waiting...
Database status is not yet available, waiting...
Database status is not yet available, waiting...
Database status is not yet available, waiting...
Database status is currently COMING_UP, waiting...
Database status is currently COMING_UP, waiting...
Database status is currently COMING_UP, waiting...
Database status is currently COMING_UP, waiting...
Database status is now ACTIVE_HEALTHY
[SUCCESS] Project supadev-test-1769638372-pg15 created and reached ACTIVE_HEALTHY status
[INFO] Time to ACTIVE_HEALTHY: 393 seconds
[INFO] Sleeping for 10 seconds before checking project health
[STEP] Checking project health for supadev-test-1769638372-pg15...
Error: 
   0: environment variable not found

Location:
   src/platform.rs:806

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ BACKTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
                                ⋮ 6 frames hidden ⋮                               
   7: supadev::platform::Platform::project_health_get::he11e303c297927d1
      at <unknown source file>:<unknown line>
   8: supadev::platform::Platform::project_health::h8c22d87ad38428e7
      at <unknown source file>:<unknown line>
   9: supadev::main::h3ebb20626023f6cb
      at <unknown source file>:<unknown line>
  10: std::sys::backtrace::__rust_begin_short_backtrace::hcfc601fdab066f90
      at <unknown source file>:<unknown line>
  11: std::rt::lang_start::{{closure}}::h0b54837a8fc1253b
      at <unknown source file>:<unknown line>
  12: std::rt::lang_start_internal::hd700ba983d3377dc
      at <unknown source file>:<unknown line>
  13: _main<unknown>
      at <unknown source file>:<unknown line>

Run with COLORBT_SHOW_HIDDEN=1 environment variable to disable frame filtering.
Run with RUST_BACKTRACE=full to include source snippets.
[STEP] Printing project information for supadev-test-1769638372-pg15
{
  "cloud_provider": "AWS",
  "db_host": "db.ilnqwlfmvbopenznwtfc.supabase.red",
  "id": 3,
  "inserted_at": "2026-01-28T22:12:58.846879",
  "name": "supadev-test-1769638372-pg15",
  "organization_id": 1,
  "ref": "ilnqwlfmvbopenznwtfc",
  "region": "us-east-1",
  "status": "ACTIVE_HEALTHY",
  "subscription_id": "my97RENkzngskhj5",
  "connectionString": "U2FsdGVkX1+o1sagoCbwvYDKVOcebvReFc2aNugVtzUEDNHmklZyYPMoQNPOJbdTqZoBr6At/aZrzNeX8j5T0neqGm0jk4o546nC3KVkgxVGRBvcO6G/A1w8JBjQkydxPLdZHgyGaaig1mnumWPlb2RRG6GUfN/y2Wa7gH8/1a0UoxMVb1+CF2b1GDJVH5sm",
  "restUrl": "https://ilnqwlfmvbopenznwtfc.supabase.red/rest/v1/",
  "volumeSizeGb": 8,
  "maxDatabasePreprovisionGb": null,
  "lastDatabaseResizeAt": null,
  "is_branch_enabled": false,
  "is_physical_backups_enabled": true,
  "infra_compute_size": "nano",
  "dbVersion": "15.14.1.074-checksum-1"
}
[STEP] Deleting project supadev-test-1769638372-pg15...
Project deleted
[SUCCESS] Project supadev-test-1769638372-pg15 deleted successfully
[SUCCESS] PostgresEngine 15 test completed successfully
[INFO] Total test duration: 408 seconds


============================================================
Cleaning Up: Withdrawing Inserted Versions
============================================================
[STEP] Withdrawing version 15.14.1.074-checksum-1 for engine 15...
🧐 🐘 postgres_dba 6.0 installed. Use ":dba" to see menu
UPDATE 0
INSERT 0 0
UPDATE 1
UPDATE 1
  id  |                 version                  |  region   |   provider_image_id   | release_channel | postgres_engine 
------+------------------------------------------+-----------+-----------------------+-----------------+-----------------
 7723 | supabase-postgres-15.14.1.074-checksum-1 | us-east-1 | ami-03a4c50234d98e6f6 | withdrawn       | 15
(1 row)


[SUCCESS] Version 15.14.1.074-checksum-1 marked as withdrawn

============================================================
Test Summary
============================================================

ENGINE          RESULT
------          ------
15              PASSED (408s)

============================================================
Total: 1 | Passed: 1 | Failed: 0
============================================================

@samrose
Copy link
Collaborator Author

samrose commented Jan 28, 2026

============================================================
Supadev PostgreSQL Engine Test Suite
============================================================
[INFO] Testing PostgresEngines: 17-oriole
[INFO] Using region: us-east-1
[INFO] Project prefix: supadev-test-1769638932
[INFO] Version: 17.6.0.031-orioledb-checksum-1


============================================================
Testing PostgreSQL Engine: 17-oriole
============================================================
[INFO] Project name: supadev-test-1769638932-pg17_oriole
[INFO] Region: us-east-1
[INFO] Start time: 2026-01-28 17:22:12
[INFO] Using explicit version: 17.6.0.031-orioledb-checksum-1
[STEP] Creating project with PostgresEngine 17-oriole...
🧐 🐘 postgres_dba 6.0 installed. Use ":dba" to see menu
UPDATE 0
INSERT 0 1
UPDATE 1
UPDATE 1
  id  |                     version                      |  region   |   provider_image_id   | release_channel | postgres_engine 
------+--------------------------------------------------+-----------+-----------------------+-----------------+-----------------
 7725 | supabase-postgres-17.6.0.031-orioledb-checksum-1 | us-east-1 | ami-07530d1161ecbb292 | preview         | 17-oriole
(1 row)


Creating project on release channel "preview" with version "17.6.0.031-orioledb-checksum-1"
Project URL: http://localhost:8082/project/mapthmmcbwxiqmnswxam
Admin URL: http://localhost:4000/projects?identifier=mapthmmcbwxiqmnswxam
Database status is currently UNKNOWN, waiting...
Database status is currently COMING_UP, waiting...
Database status is currently COMING_UP, waiting...
Database status is currently COMING_UP, waiting...
Database status is currently COMING_UP, waiting...
Database status is currently COMING_UP, waiting...
Database status is currently COMING_UP, waiting...
Database status is currently COMING_UP, waiting...
Database status is currently COMING_UP, waiting...
Database status is now ACTIVE_HEALTHY
[SUCCESS] Project supadev-test-1769638932-pg17_oriole created and reached ACTIVE_HEALTHY status
[INFO] Time to ACTIVE_HEALTHY: 547 seconds
[INFO] Sleeping for 10 seconds before checking project health
[STEP] Checking project health for supadev-test-1769638932-pg17_oriole...
Error: 
   0: environment variable not found

Location:
   src/platform.rs:806

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ BACKTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
                                ⋮ 6 frames hidden ⋮                               
   7: supadev::platform::Platform::project_health_get::he11e303c297927d1
      at <unknown source file>:<unknown line>
   8: supadev::platform::Platform::project_health::h8c22d87ad38428e7
      at <unknown source file>:<unknown line>
   9: supadev::main::h3ebb20626023f6cb
      at <unknown source file>:<unknown line>
  10: std::sys::backtrace::__rust_begin_short_backtrace::hcfc601fdab066f90
      at <unknown source file>:<unknown line>
  11: std::rt::lang_start::{{closure}}::h0b54837a8fc1253b
      at <unknown source file>:<unknown line>
  12: std::rt::lang_start_internal::hd700ba983d3377dc
      at <unknown source file>:<unknown line>
  13: _main<unknown>
      at <unknown source file>:<unknown line>

Run with COLORBT_SHOW_HIDDEN=1 environment variable to disable frame filtering.
Run with RUST_BACKTRACE=full to include source snippets.
[STEP] Printing project information for supadev-test-1769638932-pg17_oriole
{
  "cloud_provider": "AWS",
  "db_host": "db.mapthmmcbwxiqmnswxam.supabase.red",
  "id": 4,
  "inserted_at": "2026-01-28T22:22:16.181541",
  "name": "supadev-test-1769638932-pg17_oriole",
  "organization_id": 1,
  "ref": "mapthmmcbwxiqmnswxam",
  "region": "us-east-1",
  "status": "ACTIVE_HEALTHY",
  "subscription_id": "my97RENkzngskhj5",
  "connectionString": "U2FsdGVkX1+2eEgJZX8V3fhO7+Q0keuJi25ZRX8XaiE0JzSNmPd4NoeL5Obz6D0JHsZFNyuBw86Fy8NhPrpBn8yHtUQtkQ3NLabGQKiEwuKf9WkIIeSTrvfQACyeLfo5J0vg+X72g2gqeO3JCucTd8J0jR9MXTvaSsISpWOskVEWTaOZEP9JnLXFr7YkuS0J",
  "restUrl": "https://mapthmmcbwxiqmnswxam.supabase.red/rest/v1/",
  "volumeSizeGb": 8,
  "maxDatabasePreprovisionGb": null,
  "lastDatabaseResizeAt": null,
  "is_branch_enabled": false,
  "is_physical_backups_enabled": true,
  "infra_compute_size": "nano",
  "dbVersion": "17.6.0.031-orioledb-checksum-1"
}
[STEP] Deleting project supadev-test-1769638932-pg17_oriole...
Project deleted
[SUCCESS] Project supadev-test-1769638932-pg17_oriole deleted successfully
[SUCCESS] PostgresEngine 17-oriole test completed successfully
[INFO] Total test duration: 563 seconds


============================================================
Cleaning Up: Withdrawing Inserted Versions
============================================================
[STEP] Withdrawing version 17.6.0.031-orioledb-checksum-1 for engine 17-oriole...
🧐 🐘 postgres_dba 6.0 installed. Use ":dba" to see menu
UPDATE 0
INSERT 0 0
UPDATE 1
UPDATE 1
  id  |                     version                      |  region   |   provider_image_id   | release_channel | postgres_engine 
------+--------------------------------------------------+-----------+-----------------------+-----------------+-----------------
 7726 | supabase-postgres-17.6.0.031-orioledb-checksum-1 | us-east-1 | ami-07530d1161ecbb292 | withdrawn       | 17-oriole
(1 row)


[SUCCESS] Version 17.6.0.031-orioledb-checksum-1 marked as withdrawn

============================================================
Test Summary
============================================================

ENGINE          RESULT
------          ------
17-oriole       PASSED (563s)

============================================================
Total: 1 | Passed: 1 | Failed: 0
============================================================
[SUCCESS] All tests passed!

@samrose samrose disabled auto-merge January 28, 2026 22:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants