Repository navigation
feat(migrate)!: merge parallel migration branches without conflicts - #214
Merged
Merged
Conversation
momo.sum format 2 is one checksum line per file with no directory hash, so a git union merge of two branches' lines is a valid file and momo migrate hash sorts it. Format 1 is still read; hash upgrades it and hash --sum-format 1 writes it again. A momo binary from before this change stops at the format=2 header with ErrNewerFormat. A snapshot's parent is the migration it was generated from. The schema after a migration is derived by folding the migrations in version order, each contributing what it changed relative to its parent. Forks whose branches change different tables validate and apply; a fork that changes the same table stays ErrBranchConflict and now names the table and both migrations. A chain without a fork derives to its stored snapshots. diff, new --custom and drift build on the derived state. momo migrate rebase --renumber moves pending migrations that sort before the newest applied one to later versions, rewriting file names, parent links and momo.sum, and refuses to touch an applied migration. BREAKING CHANGE: momo migrate hash writes momo.sum format 2, which momo binaries from before this change refuse with ErrNewerFormat. Use hash --sum-format 1 to keep format 1. Refs #205
Contributor
apidiffExported API changes against Docs gate passed: the change touches docs. |
Regenerate docs/reference.md from the CI artifact.
There was a problem hiding this comment.
🟡 Changes recommended
The review found an issue that should be addressed before approval.
8 open issues
The greatest applied version does not imply that every older migration in a merged directory was… · New A registered Go migration created on B before merging independent branch A has a snapshot equal to… · New The descendant closure also selects snapshot-only Go migrations, but renumbering only changes their… · New Renumber silently accepts edits to moved files. existingSum only parses the manifest; deleting the… · New A child of the preceding version is not necessarily a snapshot of the folded state. For root R,… · New Identical resulting objects do not make both migrations executable. The new identical-change… · New Checking only foreign keys accepts other non-executable cross-object forks. For example, from a… · New This rejects a valid custom migration authored on a branch before merging. If A and B independently… · New
What changed in this PR
Implements parallel-branch migration support for #205 through merge-friendly checksums, derived schema state, and pending-migration renumbering.
Changes:
- Introduces
momo.sumformat 2 while retaining format 1 compatibility. - Folds forked snapshots into derived state for generation, validation, and drift.
- Adds
rebase --renumber, regression tests, and migration documentation.
| File | Description |
|---|---|
| migrate/validate.go | Updated as part of this pull request. |
| migrate/sum.go | Updated as part of this pull request. |
| migrate/sum_test.go | Updated as part of this pull request. |
| migrate/state.go | Updated as part of this pull request. |
| migrate/state_test.go | Updated as part of this pull request. |
| migrate/sfake_test.go | Updated as part of this pull request. |
| migrate/renumber.go | Updated as part of this pull request. |
| migrate/parallel_branches_test.go | Updated as part of this pull request. |
| migrate/parallel_branches_integration_test.go | Updated as part of this pull request. |
| migrate/gomigration.go | Updated as part of this pull request. |
| migrate/engine.go | Updated as part of this pull request. |
| migrate/drift.go | Updated as part of this pull request. |
| migrate/custom.go | Updated as part of this pull request. |
| migrate/chain.go | Updated as part of this pull request. |
| migrate/chain_test.go | Updated as part of this pull request. |
| migrate/apply_safety_test.go | Updated as part of this pull request. |
| docs/upgrading.md | Updated as part of this pull request. |
| docs/supported-versions.md | Updated as part of this pull request. |
| docs/reference.md | Updated as part of this pull request. |
| docs/migrations.md | Updated as part of this pull request. |
| cli/migrate.go | Updated as part of this pull request. |
| cli/migrate_sumformat_test.go | Updated as part of this pull request. |
Review effort: Balanced
| exact := false | ||
| if latest != "" { | ||
| snap, err := snapshotUpTo(e.dir, latest) | ||
| snap, err := stateUpTo(e.dir, latest) |
| continue // reported as a missing parent | ||
| } | ||
| if !schemaEqual(e.snap.Schema, parent.snap.Schema) { | ||
| if !schemaEqual(e.snap.Schema, parent.derived()) { |
Comment on lines
+91
to
+96
| for v, p := range parentOf { | ||
| if move[p] && !move[v] { | ||
| if done[v] { | ||
| return nil, nil, nil, fmt.Errorf("migrate: version %s is applied but is built on %s, which is pending and would move; renumber cannot change applied migrations", v, p) | ||
| } | ||
| move[v], grew = true, true |
Comment on lines
+160
to
+164
| kept := &Sum{Format: existing.Format, Files: maps.Clone(existing.Files)} | ||
| for _, p := range removed { | ||
| delete(kept.Files, p) | ||
| } | ||
| sum, err := AppendSum(kept, added, addedNames) |
Comment on lines
+214
to
+216
| if i == 0 || idx[e.snap.ParentVersion] == i-1 { | ||
| // The migration was generated from the one before it: its snapshot is the state. | ||
| states[i] = e.snap.Schema |
Comment on lines
+227
to
+228
| if objectsEqual(mine[k], theirs[k]) { | ||
| continue // both sides made the same change |
Comment on lines
+259
to
+263
| own := danglingForeignKeys(e.snap.Schema) | ||
| for _, ref := range slices.Sorted(maps.Keys(danglingForeignKeys(s))) { | ||
| if !own[ref] { | ||
| problems = append(problems, fmt.Errorf("%w: %s: merged with the migrations between it and its parent %s, foreign key %s refers to a table that no longer exists", ErrBranchConflict, e.path, e.snap.ParentVersion, ref)) | ||
| } |
| continue // reported as a missing parent | ||
| } | ||
| if !schemaEqual(e.snap.Schema, parent.snap.Schema) { | ||
| if !schemaEqual(e.snap.Schema, parent.derived()) { |
These fail on the previous commit: a migration built on one branch after a merge loses the other branch's tables, and an identical change on both branches is accepted although the second SQL would fail on apply.
…orks A migration built on one branch after a merge no longer loses the other branch's tables: every migration is folded against its base, which is the parent's stored snapshot, or the derived parent for a snapshot marked base=derived (written by diff and new --custom after a merge). Forks are independent only when they change different objects, no view reads a table the other changes, no opaque function, trigger or policy meets a table change, and every foreign key stays valid; identical changes conflict. Renumber names pending Go migrations, applyRebase writes before it removes, and hash and validate warn when no .gitattributes union rule covers momo.sum.
Regenerate docs/reference.md from CI.
A snapshot generated from a merged state records base=derived:<sha256> of that state, and the fold reports a conflict when the parent state has since changed. Removing or altering an enum, extension or schema conflicts with any table, view or function change on the other branch, as does a function change with another function. The fold verifies that no object is dropped by the split or the merge, and a test covers every list in schema.Schema. A marked snapshot must equal the derived parent, an unknown base is refused with ErrNewerFormat, and the .gitattributes check lets the last matching merge setting win.
…t on applied A snapshot generated from a merged state moves with a pending branch that renumber puts behind it, follows it as its parent and gets its base recomputed; when it is applied, renumber refuses before writing. The base marker is derived:v1:<sha256> over a canonical encoding pinned by a test against a literal hash. The late-merge conflict names candidate branches. Drift compares the database with the applied migrations only.
A snapshot generated from a merged state holds only what was older than itself, so renumber refuses, before writing, when it would have to sort behind a migration that was newer and stays. The fingerprint test covers every field of every object kind, and the late-merge message lists possible causes, recent ones first.
splitObjects and assemble handle schema.Schema.Sequences. A changed sequence conflicts with a table or view change on the other side whose default, generation expression, check or query calls nextval on it by name or on anything unreadable, and with any function, trigger or policy change; different sequences are independent. The full-schema fingerprint fixture now includes a populated Sequence, which changes its pinned hash; the literal is taken from CI, and no marker has shipped.
The full-schema fixture now includes a populated Sequence, so the pinned v1 hash changes; the literal is taken from CI. No marker has shipped, so no version bump. Regenerate docs/reference.md from CI.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

What
Phase 1 of #205.
momo.sumformat 2: one sorted checksum line per file, noh1:directory hash, lines read in any order.momo migrate hashwrites it (and upgrades format 1);--sum-format 1writes format 1. A momo from before this change stops at theformat=2header withErrNewerFormat, so it never misreads the file.diffandnew --customkeep the format of the file they extend.hashandvalidatewarn when no.gitattributesrule givesmomo.summerge=union.parentis the migration a snapshot was generated from. The schema after a migration is derived by folding the migrations in version order against each migration's base. Forks validate and apply only when they can be shown independent (different objects; no view reads a table the other changes; no opaque function, trigger or policy next to a table, view or function change; no enum, extension or schema removed or altered next to a table, view or function change; every foreign key still valid); everything else is anErrBranchConflictat validate, naming both migrations. A linear chain derives to its stored snapshots. A snapshot written after a merge records"base": "derived:v1:<sha256>"; a late merge that changes that state is a conflict naming candidate branches.momo migrate rebase --renumber [--applied V1,V2]moves pending migrations older than the newest applied one to later versions, rewriting file names, parent links andmomo.sum; a snapshot written after a merge moves with the branch it includes and gets its marker recomputed; it refuses to touch applied ones, before writing anything. New files are written before old ones are removed.driftcompares the database with the applied migrations only..gitattributesmomo.sum merge=union), upgrading, supported versions.Breaking: format 2 is refused by older binaries once
hashwrites it.Note for #212 (standalone sequences): whichever of the two lands second must handle
schema.Schema.SequencesinsplitObjectsinmigrate/state.go, with an independence rule (a sequence change conflicts with any table change on the other side that may use it throughnextval).TestSplitObjectsKeepsEveryObjectKindfails until the new list is handled.Closes #205