Skip to content

ci(release): harden integrated upgrade acceptance - #2181

Merged
hidb4ai merged 13 commits into
masterfrom
codex/issue-2161-upgrade-release-gate
Oct 26, 2026
Merged

hidb4ai merged 13 commits into
masterfrom
codex/issue-2161-upgrade-release-gate

Conversation

@Zxf-xufeng

@Zxf-xufeng Zxf-xufeng commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Which issue or RFC does this PR close?

Closes #2161. Related: #2154, #2160, RFC #1771, and the migration framework merged in #2168.

Rationale for this change

Upgrade acceptance must detect real schema and authorization regressions in the installed packages. Raw DDL whitespace normalization can incorrectly treat distinct DEFAULT/CHECK literals as equivalent, and checking a retained grant using the deployment administrator does not prove that an ordinary user's permissions survived migration.

What changes are included in this PR?

  • Integrate the current master migration framework and parallel acceptance workflow. Preserve the fixed 1.3.0 source and all twelve backend/configuration profiles: ten automatic cases plus the two temporarily manual embedded-seekDB Code profiles. Manual cases remain outside advertised supported upgrade combinations; no 1.4.0-source acceptance claim is added.
  • Preserve SQL literals, quoted identifiers, comments and identifier boundaries in raw DDL comparisons. Ignore only unquoted formatting whitespace and table-level auto-increment counters. Keep native logical table reflection and view/trigger comparison for seekDB/OceanBase, including trigger timing (BEFORE/AFTER), event, table and statement.
  • Seed an ordinary viewer grant with the source package and verify allowed reads, denied cross-scope reads, denied writes and unchanged records after rejection with both installed releases. Use the installed Server's real HTTP handlers and persisted builtin authorization. Stop each CLI Server before opening the SDK probe so embedded storage has one owner.
  • Preserve the historical Processing capability declaration in the authorization probe. Disable scheduling and supply a separate unprivileged background identity; the independent background-processing acceptance phase continues to exercise the actual Worker.
  • Run schema-equivalence regressions alongside gate/publication regressions in the shared workflow before the explicit persistence-policy check. Policy failures still block artifact preparation.
  • Reject unsupported hosts before installing packages or creating full-acceptance workspaces. Add regressions for unknown configurations, missing or mismatched configuration evidence during parallel collection, and differences in native logical objects.

The shared workflow builds and validates one wheel/sdist set, runs independent selected cases, and collects evidence bound to that exact preparation. Existing source discovery, process ownership, skipped/missing-case rejection, publication artifact checks, vector/Code validation, background processing and SQLite backup-restoration acceptance remain in place.

Are there any user-facing changes?

The acceptance README documents historical-database, operating-system, engine-dependency and backup/rollback coverage limits. Passing one target build does not establish every future target or production database.

No runtime API, migration resource, persisted schema or data-format change. The policy remains upgrade-required; these changes do not declare release readiness. The ten automatic cases must pass before their acceptance is established. The two deferred seekDB Code profiles remain explicitly unverified until separately exercised.

Administrators must require Upgrade release gate in branch protection. Release publication remains subject to the reviewed support policy and the selected acceptance results.

How was this change tested?

  • Reproduced the raw DDL comparison defect against the merged master implementation: ten schema regressions failed before the fix and passed afterward.
  • uv run --locked pytest -q tests/test_upgrade_release_gate.py tests/test_upgrade_publication.py tests/test_upgrade_schema_equivalence.py tests/test_release_version.py tests/test_ci_release_verification.py: 159 passed, 6 Linux-specific tests skipped on macOS.
  • Reproduced a native-trigger comparison false positive using synthetic driver rows: BEFORE INSERT and AFTER INSERT compared equal before the fix and differ after it. This regression exercises the probe, JSON evidence and comparator; it is not live native-backend acceptance.
  • make check: passed, including pre-commit hooks, formatting, types, lock and policy checks.
  • Built the target wheel/sdist with uv build and installed the official 1.3.0 package and target wheel into separate temporary environments. The actual SQLite full-server-no-vector suite passed all 11 tests in 101 seconds, including migration/repetition, backup restoration, source/target viewer permissions, background processing and fresh/upgraded equivalence. This direct macOS suite execution does not validate the Linux process supervisor or native seekDB/OceanBase behavior.
  • The integrated Linux CI run is required to establish the ten automatic case results. No complete native-backend or manual-case success is claimed from local regression tests.

AI usage statement

Implemented and validated with OpenAI Codex (GPT-6 family), under maintainer direction.

Comment thread scripts/upgrade_release_gate.py Outdated
@AlexStocks AlexStocks added the 1.4.1 v1.4.1 release tracking label Oct 21, 2026

@hidb4ai hidb4ai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One additional P2 in the local publication path. The CRLF digest issue is already covered by the existing review thread.

Comment thread Makefile Outdated
@Zxf-xufeng Zxf-xufeng changed the title ci(release): gate supported database upgrades on installed artifacts ci(release): require positive upgrades from 1.3.0 on all backends Oct 22, 2026

@hidb4ai hidb4ai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed 10efc32f and replayed both counterexamples. The CRLF digest and publication-set P2s are fixed. Publication now validates the wheel/sdist pair and uploads only the accepted snapshot. No new confirmed defects found in this pass.

Merge remains blocked by the Upgrade release gate. I inspected its downloaded evidence: all three 1.3.0 source fixtures completed, but the public migration status command failed on every backend:

  • SQLite: unknown_baseline — the schema is not registered.
  • Embedded seekDB: incompatible_schema — a CHECK expression is reported as having unbalanced parentheses.
  • OceanBase: incompatible_schema — an unsupported schema option is reported.

Post-migration startup, read/write, authorization, history/cursors, and schema equivalence therefore remain unverified. Complete the migration integration and rerun positive acceptance on all three backends before merging.

@Zxf-xufeng
Zxf-xufeng marked this pull request as draft October 22, 2026 08:56
@Zxf-xufeng
Zxf-xufeng marked this pull request as ready for review October 23, 2026 10:07

@hireview001 hireview001 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • CI failed: Upgrade release gate (failure).
  • CI failed: Main (failure). Failed steps: Upgrade release gate / Run declared installed-release acceptance.

@Zxf-xufeng

Copy link
Copy Markdown
Member Author

The upgrade CI failure is expected at this stage. This PR requires successful end-to-end upgrades from PowerContext 1.3.0 on SQLite, embedded seekDB, and OceanBase, but the complete migration framework integration is still pending in PR #2168.

PR #2168 needs to address the migration framework blockers before this gate can pass. Once those changes are integrated, we will rerun acceptance on all three backends and verify the remaining migration and post-upgrade checks. The gate should remain failing until those checks actually succeed; expected failure here is not grounds to bypass the gate.

@Zxf-xufeng

Copy link
Copy Markdown
Member Author

Addressed the remaining local review findings in commit 9f78868:

  • Schema equivalence (P2): DDL comparison now preserves literals, quoted identifiers, comments, and identifier boundaries. It ignores only unquoted formatting whitespace and table-level auto-increment counters. Different DEFAULT/CHECK values such as 'a b' and 'a b' no longer compare equal. Added regression coverage using actual SQLite behavior, plus quoting, escaping, counter, and identifier-boundary cases.
  • Authorization coverage: The source fixture now grants scope.viewer to an ordinary user. Both installed releases must allow that user's reads in the granted scope and reject cross-scope reads and writes. The probe uses the installed Server's real HTTP handlers and persisted builtin authorization; only the fixed authentication identity is supplied. CLI startup, invalid-token rejection, and persisted-grant checks remain mandatory.

Validation: 113 focused tests passed; make check and uv run prek run -a passed. Separate installed 1.3.0 and target SQLite fixtures passed the viewer checks, and revoking the grant correctly caused the probe to fail. These are fixture validations, not evidence of a completed upgrade.

The existing positive upgrade gate remains mandatory and unchanged in its success requirements. The migration framework blockers tracked in #2168 and the related schema-probe defects still need to be resolved before full acceptance can pass on SQLite, embedded seekDB, and OceanBase. No skips or expected-failure bypasses were added.

@hidb4ai

hidb4ai commented Oct 24, 2026

Copy link
Copy Markdown
Contributor

test failed

@hireview001 hireview001 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hidb4ai hidb4ai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 9f7886894489. The branch snapshot passes the focused publication, gate and schema-equivalence checks.

The current Upgrade release gate stops before installed-release acceptance: CI tests merge 53b134d412c8 (this head into 1da997476651), whose compatibility declaration is ca3b475f...37927 while the actual managed-input digest is 59e34e86...1ab95. I reproduced the mismatch from that exact CI tree; the branch's own digest matches. The quality and unit-test failures have the same preflight cause.

Please integrate the migration changes, review/update the declaration for the resulting tree, and rerun positive installed 1.3.0-to-target acceptance on SQLite, seekDB and OceanBase. This CI run has not reached those upgrade checks; its failure should not be treated as an exercised migration rejection.

@hidb4ai

hidb4ai commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

@Zxf-xufeng Please update this PR together with #2168. The current CI failure is a compatibility-declaration mismatch in the merge tree, so the gate stops before installed-release upgrade acceptance.

Please integrate the migration changes, review the resulting persistence changes and refresh the declaration, then rerun positive installed 1.3.0-to-target upgrades on SQLite, seekDB and OceanBase. Please share an ETA and reply with the updated commit and passing CI evidence; an expected gate failure is not sufficient for merge readiness.

@Zxf-xufeng

Copy link
Copy Markdown
Member Author

@hidb4ai Following up on your integration and acceptance request: agreed. The current failure is a compatibility-declaration mismatch in the merge tree, before installed-release upgrade acceptance. My earlier description of the failure as expected was too broad; this run does not demonstrate an exercised migration rejection or merge readiness.

Progress on #2181: commit 52bebcf adds rejection of unimplemented acceptance configurations, verifies configuration evidence, and cleans up detached acceptance processes on timeout, failure and cancellation. All six Linux process-cleanup regressions passed in the latest gate job. The job still failed on the existing persistence-digest mismatch, so full database upgrade acceptance has not run.

#2168 remains open and is the dependency for the complete migration implementation. Once those changes are ready for integration, we will review the resulting persistence changes, refresh the compatibility declaration, and rerun positive installed 1.3.0-to-target acceptance on SQLite, embedded seekDB and OceanBase against the same final build. I will follow up here with the integrated commit and passing CI evidence; this request remains outstanding until then.

I do not yet have a reliable ETA because completion depends on #2168 and the combined three-backend acceptance results. We will not bypass the gate or treat an expected failure as sufficient for merging.

@hireview001 hireview001 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Zxf-xufeng Zxf-xufeng changed the title ci(release): require positive upgrades from 1.3.0 on all backends ci(release): harden integrated upgrade acceptance Oct 26, 2026
@Zxf-xufeng

Copy link
Copy Markdown
Member Author

@hidb4ai Following up on your integration request, the merge-tree digest review, and the positive-acceptance requirement:

Master containing the merged #2168 implementation is integrated in 4879b07. The resulting compatibility declaration passes the local policy check. This supersedes my previous update saying #2168 was still open.

The latest commit is 91eb0fd. It also fixes an inherited schema-equivalence blind spot: native triggers differing only in BEFORE/AFTER timing no longer compare equal. The new regression reproduces the old failure through synthetic driver rows, the database probe and serialized comparison evidence; it does not claim live trigger/backend validation. The README now states the historical-database, platform, engine-dependency and backup/rollback coverage limits.

Validation on this head: 159 focused gate/publication/schema/release tests passed, 6 Linux-only tests were skipped on macOS, and make check passed. The matrix remains fixed at source 1.3.0 with ten automatic cases across SQLite, embedded seekDB and OceanBase, plus the two explicitly manual seekDB Code cases. No acceptance bypass or support-policy relaxation was added.

The new Main run is pending, so passing three-backend CI evidence is still outstanding. I am keeping that requirement open. Now that #2168 is integrated, a new upgrade failure must be investigated rather than explained away as an expected dependency failure. Implementation is pushed; I cannot give a reliable completion ETA while CI is awaiting execution.

@Zxf-xufeng

Copy link
Copy Markdown
Member Author

@hidb4ai Regarding "test failed" and the subsequent CI reports (upgrade acceptance, merge-tree preflight, latest historical failure report): the migration integration and compatibility-declaration updates are now included in this branch. Current head 91eb0fd passes the local policy, quality and focused regression checks.

Those earlier red runs remain failures; local checks do not replace them or establish full unit/upgrade CI success. A fresh Main run for 91eb0fd7 is pending. The complete installed 1.3.0-to-target results still need to pass before this acceptance requirement can be closed. We are no longer treating failures as expected merely because #2168 was a dependency.

@hidb4ai hidb4ai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hidb4ai
hidb4ai merged commit 13cbbaa into master Oct 26, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

1.4.1 v1.4.1 release tracking

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci(release): gate supported upgrades on complete databases from actual prior releases

4 participants