Skip to content

docs: PG-Sync V2 design#81

Open
JustMaier wants to merge 1 commit intomainfrom
docs/pg-sync-v2
Open

docs: PG-Sync V2 design#81
JustMaier wants to merge 1 commit intomainfrom
docs/pg-sync-v2

Conversation

@JustMaier
Copy link
Copy Markdown
Contributor

Summary

  • Three design docs for the Sync V2 redesign (ops-based outbox replacing enrichment-heavy poller)
  • pg-sync-v2-final.md — clean final design with all agreed decisions
  • pg-sync-v2.md — working doc with full Justin + Adam discussion trail
  • computed-sort-fields.md — separate prerequisite feature (assigned to Ryan)

Key Decisions

  • JSONB ops array with old+new values (eliminates docstore reads on write path)
  • queryOpSet for fan-outs — bitmap-resolved bulk ops (15M images = 2 bitmap ops)
  • WAL-backed ops endpoint (HTTP ingestion → append-only WAL → reader thread)
  • YAML-driven trigger config with expression interpolation and hash-based reconciliation
  • Unified dump pipeline for initial load (no manual pod teardown)
  • Flat migration: full reload + cutover, no V1 fixes

Test plan

  • Design review by architecture sub-agent (17 findings, all resolved)
  • Implementation tracked in separate PRs

🤖 Generated with Claude Code

…d pipeline

Three design docs for the Sync V2 redesign:

- pg-sync-v2.md: Working design doc with full discussion trail (Justin + Adam)
- pg-sync-v2-final.md: Clean final design — ops-based outbox, JSONB ops with
  old+new values, queryOpSet for bitmap-resolved fan-outs, WAL-backed ops
  endpoint, YAML-driven trigger config, unified dump pipeline, review findings
- computed-sort-fields.md: Prerequisite feature for cross-table sort computation
  (sortAt = GREATEST(existedAt, publishedAt)), separate workstream

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant