Skip to content

feat(migrate)!: merge parallel migration branches without conflicts - #214

Merged
piwi3910 merged 19 commits into
mainfrom
feat/parallel-migration-branches
Nov 16, 2026
Merged

piwi3910 merged 19 commits into
mainfrom
feat/parallel-migration-branches

Conversation

@piwi3910

@piwi3910 piwi3910 commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

What

Phase 1 of #205.

  • momo.sum format 2: one sorted checksum line per file, no h1: directory hash, lines read in any order. momo migrate hash writes it (and upgrades format 1); --sum-format 1 writes format 1. A momo from before this change stops at the format=2 header with ErrNewerFormat, so it never misreads the file. diff and new --custom keep the format of the file they extend. hash and validate warn when no .gitattributes rule gives momo.sum merge=union.
  • Chain: parent is the migration a snapshot was generated from. The schema after a migration is derived by folding the migrations in version order against each migration's base. Forks validate and apply only when they can be shown independent (different objects; no view reads a table the other changes; no opaque function, trigger or policy next to a table, view or function change; no enum, extension or schema removed or altered next to a table, view or function change; every foreign key still valid); everything else is an ErrBranchConflict at validate, naming both migrations. A linear chain derives to its stored snapshots. A snapshot written after a merge records "base": "derived:v1:<sha256>"; a late merge that changes that state is a conflict naming candidate branches.
  • momo migrate rebase --renumber [--applied V1,V2] moves pending migrations older than the newest applied one to later versions, rewriting file names, parent links and momo.sum; a snapshot written after a merge moves with the branch it includes and gets its marker recomputed; it refuses to touch applied ones, before writing anything. New files are written before old ones are removed.
  • drift compares the database with the applied migrations only.
  • Docs: migrations guide (Parallel branches, .gitattributes momo.sum merge=union), upgrading, supported versions.

Breaking: format 2 is refused by older binaries once hash writes it.

Note for #212 (standalone sequences): whichever of the two lands second must handle schema.Schema.Sequences in splitObjects in migrate/state.go, with an independence rule (a sequence change conflicts with any table change on the other side that may use it through nextval). TestSplitObjectsKeepsEveryObjectKind fails until the new list is handled.

Closes #205

momo.sum format 2 is one checksum line per file with no directory hash, so a git union merge of two branches' lines is a valid file and momo migrate hash sorts it. Format 1 is still read; hash upgrades it and hash --sum-format 1 writes it again. A momo binary from before this change stops at the format=2 header with ErrNewerFormat.

A snapshot's parent is the migration it was generated from. The schema after a migration is derived by folding the migrations in version order, each contributing what it changed relative to its parent. Forks whose branches change different tables validate and apply; a fork that changes the same table stays ErrBranchConflict and now names the table and both migrations. A chain without a fork derives to its stored snapshots. diff, new --custom and drift build on the derived state.

momo migrate rebase --renumber moves pending migrations that sort before the newest applied one to later versions, rewriting file names, parent links and momo.sum, and refuses to touch an applied migration.

BREAKING CHANGE: momo migrate hash writes momo.sum format 2, which momo binaries from before this change refuse with ErrNewerFormat. Use hash --sum-format 1 to keep format 1.

Refs #205
@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

apidiff

Exported API changes against origin/main:

Incompatible changes:
- ./migrate.SumFormat: value changed from 1 to 2
Compatible changes:
- ./migrate.(*Engine).AppliedVersions: added
- ./migrate.(*Engine).Renumber: added
- ./migrate.BaseDerived: added
- ./migrate.LatestState: added
- ./migrate.Migration.Base: added
- ./migrate.Renumbered: added
- ./migrate.SnapshotFile.Base: added
- ./migrate.Sum.Format: added
- ./migrate.SumFormatLegacy: added

Docs gate passed: the change touches docs.

Copilot AI balanced review requested due to automatic review settings November 16, 2026 12:56

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The review found an issue that should be addressed before approval.

8 open issues
What changed in this PR

Implements parallel-branch migration support for #205 through merge-friendly checksums, derived schema state, and pending-migration renumbering.

Changes:

  • Introduces momo.sum format 2 while retaining format 1 compatibility.
  • Folds forked snapshots into derived state for generation, validation, and drift.
  • Adds rebase --renumber, regression tests, and migration documentation.
File Description
migrate/​validate.go Updated as part of this pull request.
migrate/​sum.go Updated as part of this pull request.
migrate/​sum_test.go Updated as part of this pull request.
migrate/​state.go Updated as part of this pull request.
migrate/​state_test.go Updated as part of this pull request.
migrate/​sfake_test.go Updated as part of this pull request.
migrate/​renumber.go Updated as part of this pull request.
migrate/​parallel_branches_test.go Updated as part of this pull request.
migrate/​parallel_branches_integration_test.go Updated as part of this pull request.
migrate/​gomigration.go Updated as part of this pull request.
migrate/​engine.go Updated as part of this pull request.
migrate/​drift.go Updated as part of this pull request.
migrate/​custom.go Updated as part of this pull request.
migrate/​chain.go Updated as part of this pull request.
migrate/​chain_test.go Updated as part of this pull request.
migrate/​apply_safety_test.go Updated as part of this pull request.
docs/​upgrading.md Updated as part of this pull request.
docs/​supported-versions.md Updated as part of this pull request.
docs/​reference.md Updated as part of this pull request.
docs/​migrations.md Updated as part of this pull request.
cli/​migrate.go Updated as part of this pull request.
cli/​migrate_sumformat_test.go Updated as part of this pull request.

Review effort: Balanced

Comment thread migrate/drift.go Outdated
exact := false
if latest != "" {
snap, err := snapshotUpTo(e.dir, latest)
snap, err := stateUpTo(e.dir, latest)
Comment thread migrate/gomigration.go Outdated
continue // reported as a missing parent
}
if !schemaEqual(e.snap.Schema, parent.snap.Schema) {
if !schemaEqual(e.snap.Schema, parent.derived()) {
Comment thread migrate/renumber.go
Comment on lines +91 to +96
for v, p := range parentOf {
if move[p] && !move[v] {
if done[v] {
return nil, nil, nil, fmt.Errorf("migrate: version %s is applied but is built on %s, which is pending and would move; renumber cannot change applied migrations", v, p)
}
move[v], grew = true, true
Comment thread migrate/renumber.go
Comment on lines +160 to +164
kept := &Sum{Format: existing.Format, Files: maps.Clone(existing.Files)}
for _, p := range removed {
delete(kept.Files, p)
}
sum, err := AppendSum(kept, added, addedNames)
Comment thread migrate/state.go Outdated
Comment on lines +214 to +216
if i == 0 || idx[e.snap.ParentVersion] == i-1 {
// The migration was generated from the one before it: its snapshot is the state.
states[i] = e.snap.Schema
Comment thread migrate/state.go Outdated
Comment on lines +227 to +228
if objectsEqual(mine[k], theirs[k]) {
continue // both sides made the same change
Comment thread migrate/state.go Outdated
Comment on lines +259 to +263
own := danglingForeignKeys(e.snap.Schema)
for _, ref := range slices.Sorted(maps.Keys(danglingForeignKeys(s))) {
if !own[ref] {
problems = append(problems, fmt.Errorf("%w: %s: merged with the migrations between it and its parent %s, foreign key %s refers to a table that no longer exists", ErrBranchConflict, e.path, e.snap.ParentVersion, ref))
}
Comment thread migrate/validate.go Outdated
continue // reported as a missing parent
}
if !schemaEqual(e.snap.Schema, parent.snap.Schema) {
if !schemaEqual(e.snap.Schema, parent.derived()) {
These fail on the previous commit: a migration built on one branch after a merge loses the other branch's tables, and an identical change on both branches is accepted although the second SQL would fail on apply.
…orks

A migration built on one branch after a merge no longer loses the other branch's tables: every migration is folded against its base, which is the parent's stored snapshot, or the derived parent for a snapshot marked base=derived (written by diff and new --custom after a merge). Forks are independent only when they change different objects, no view reads a table the other changes, no opaque function, trigger or policy meets a table change, and every foreign key stays valid; identical changes conflict. Renumber names pending Go migrations, applyRebase writes before it removes, and hash and validate warn when no .gitattributes union rule covers momo.sum.
Regenerate docs/reference.md from CI.
A snapshot generated from a merged state records base=derived:<sha256> of that state, and the fold reports a conflict when the parent state has since changed. Removing or altering an enum, extension or schema conflicts with any table, view or function change on the other branch, as does a function change with another function. The fold verifies that no object is dropped by the split or the merge, and a test covers every list in schema.Schema. A marked snapshot must equal the derived parent, an unknown base is refused with ErrNewerFormat, and the .gitattributes check lets the last matching merge setting win.
…t on applied

A snapshot generated from a merged state moves with a pending branch that renumber puts behind it, follows it as its parent and gets its base recomputed; when it is applied, renumber refuses before writing. The base marker is derived:v1:<sha256> over a canonical encoding pinned by a test against a literal hash. The late-merge conflict names candidate branches. Drift compares the database with the applied migrations only.
A snapshot generated from a merged state holds only what was older than itself, so renumber refuses, before writing, when it would have to sort behind a migration that was newer and stays. The fingerprint test covers every field of every object kind, and the late-merge message lists possible causes, recent ones first.
splitObjects and assemble handle schema.Schema.Sequences. A changed sequence conflicts with a table or view change on the other side whose default, generation expression, check or query calls nextval on it by name or on anything unreadable, and with any function, trigger or policy change; different sequences are independent. The full-schema fingerprint fixture now includes a populated Sequence, which changes its pinned hash; the literal is taken from CI, and no marker has shipped.
The full-schema fixture now includes a populated Sequence, so the pinned v1 hash changes; the literal is taken from CI. No marker has shipped, so no version bump. Regenerate docs/reference.md from CI.
@piwi3910
piwi3910 merged commit 47a6e44 into main Nov 16, 2026
32 checks passed
@piwi3910
piwi3910 deleted the feat/parallel-migration-branches branch November 16, 2026 22:27
@github-actions github-actions Bot mentioned this pull request Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

migrate: parallel branches always conflict on momo.sum and the snapshot chain

2 participants