Skip to content

feat(phase1-a): migration scaffold + civiccore_0001 baseline - #1

Merged
scottconverse merged 3 commits into
mainfrom
phase1/migrations-scaffold
Apr 24, 2026
Merged

scottconverse merged 3 commits into
mainfrom
phase1/migrations-scaffold

Conversation

@scottconverse

@scottconverse scottconverse commented Apr 24, 2026

Copy link
Copy Markdown
Contributor

Draft. Scope: Phase 1 Part A per ADR-0003 only. No civicrecords-ai changes, no model extraction, no opportunistic cleanup.

Branch state — 3 commits on phase1/migrations-scaffold

SHA Subject
2849b77 feat(phase1-a): migration scaffold + civiccore_0001 baseline (9 files, +962/−44)
63899f7 ci(phase1-a): add CI workflow + fix stale scottconverse/* URLs (5 files, +33/−14)
6fb2c6f chore: untrack .claude/ (local Claude Code tooling, not a project deliverable)

What landed in this PR

Migration scaffold (2849b77)

  • civiccore/migrations/guards.pyidempotent_create_table, idempotent_add_column, idempotent_alter_column, has_table helper
  • civiccore/migrations/runner.pyupgrade_to_head(connection) and current_revision(connection) for consuming modules' env.py
  • civiccore/migrations/alembic.iniversion_table = alembic_version_civiccore so civiccore does not collide with consuming modules' Alembic state
  • civiccore/migrations/env.py — three modes: online-with-connection (runner.py path), online-standalone (CLI), offline. Replaces Phase 0 stub.
  • civiccore/migrations/versions/civiccore_0001_baseline_v1.py — synthetic baseline as-of records HEAD 019_encrypt_connection_config:
    • 1 PostgreSQL extension (vector / pgvector)
    • 4 shared enum types (user_role, source_type, ingestion_status, rule_type); records-only enums (flag_status, inclusion_status, request_status) stay records-side
    • 16 shared tables in dependency order, each guarded
  • civiccore/migrations/versions/__init__.py — lets the test import _SHARED_TABLE_ORDER via standard Python; Alembic's discovery ignores it
  • tests/test_baseline_idempotency.py — pytest using testcontainers[postgres] against pgvector/pgvector:pg17:
    • test_baseline_runs_clean_on_empty_db: empty DB → upgrade → all 16 tables + HEAD stamped
    • test_baseline_is_idempotent: snapshot, upgrade twice, snapshots equal
  • pyproject.toml — adds testcontainers[postgres]>=4.9.0 to dev extras
  • CHANGELOG.md — Unreleased > Added entries

CI + URL drift fix (63899f7)

  • .github/workflows/ci.yml — runs tests/test_smoke.py and tests/test_baseline_idempotency.py on pull_request + push to main. ubuntu-latest, Python 3.13. Pre-pulls pgvector/pgvector:pg17 so testcontainers starts fast. The idempotency test is now an actual enforced merge gate, not just a claim.
  • README.md, CONTRIBUTING.md, pyproject.toml [project.urls], CHANGELOG.md header — corrected stale scottconverse/civiccore and scottconverse/civicsuite URLs to CivicSuite/civiccore and CivicSuite/civicsuite. The scottconverse/civicrecords-ai URLs were intentionally preserved (records repo has not been transferred to the org yet).

Cleanup (6fb2c6f)

git add -A in 63899f7 accidentally swept in 3 local Claude-Code config files under .claude/ (a commit-size-gate hook + settings.json). Verified no secrets (zero ghp_ matches). Untracked + added .claude/ to .gitignore. Also gitignored .schema-dump.sql (regenerable temp pg_dump output). Files remain in history at 63899f7 — no force-push to rewrite, since they're not sensitive and this is a draft PR with no consumers yet.

Verification

Local (Windows 11 Pro, Python 3.13.11, Docker Desktop 29.4.0)

```
$ python -m pytest tests/test_smoke.py -v
tests/test_smoke.py::test_import_civiccore PASSED [100%]
============================== 1 passed in 0.05s ==============================

$ python -m pytest tests/test_baseline_idempotency.py -v
tests/test_baseline_idempotency.py::test_baseline_runs_clean_on_empty_db PASSED [ 50%]
tests/test_baseline_idempotency.py::test_baseline_is_idempotent PASSED [100%]
======================== 2 passed, 1 warning in 4.35s =========================
```

The single warning is SAWarning: Did not recognize type 'vector' of column 'embedding' — cosmetic; SQLAlchemy's introspector lacks a Python adapter for pgvector's vector column, but the DB has the extension and DDL round-trips correctly.

CI (GitHub Actions, ubuntu-latest)

  • 63899f7PASSED
  • 6fb2c6f — in progress (docs/.gitignore-only follow-up to a passing run; expected pass)

What's explicitly NOT in this PR (deferred to Part B)

  • Records-side 14-migration idempotent guard pass (001, 002, 003, 006, 787207afc66a, 011, 012, 013, 014, 015, 016, 017, 018, 019)
  • civicrecords-ai/backend/alembic/env.py 6-line wiring to call civiccore.migrations.runner.upgrade_to_head before records' own chain
  • The three migration gate tests from ADR-0003 §5: fresh-install, upgrade-from-v1.2.x, reapplication-idempotency
  • Records pyproject.toml adding civiccore>=0.1,<0.2
  • Records CHANGELOG.md v1.3.0 entry

Source + provenance

Baseline schema extracted via:

```
docker exec -i civicrecords-ai-postgres-1 pg_dump --schema-only
--no-owner --no-privileges --no-comments
-U civicrecords civicrecords_test
```

at scottconverse/civicrecords-ai origin/master = 3cf7719, records alembic head = 019_encrypt_connection_config, 2026-04-24.

13 shared→shared FKs preserved; zero shared→records-only FKs (none to drop); no cycles.

Merge criteria

  • Both civiccore tests pass locally
  • CI green on phase1/migrations-scaffold (Actions runs)
  • Stale scottconverse/* URLs corrected (audit DOC-001)
  • Test described as "merge gate" is now an actual enforced gate (audit TEST-001)
  • Part B opens and its three migration gate tests pass against civicrecords-ai — the actual Phase 1 release authority
  • Scott's explicit review + approval

Related: extraction inventory PR, ADR-0003, ADR-0002, umbrella docs, civiccore landing.

🤖 Generated with Claude Code

Scott Converse added 3 commits April 24, 2026 10:28
Phase 1 Part A per ADR-0003. One narrow PR adding:
- civiccore.migrations.guards (4 idempotent op.* wrappers + has_table helper)
- civiccore.migrations.runner (upgrade_to_head + current_revision)
- civiccore/migrations/alembic.ini + env.py (alembic_version_civiccore
  version table isolates civiccore from consuming modules' Alembic)
- civiccore_0001_baseline_v1 migration: idempotent snapshot of the
  16 shared tables + 4 shared enum types + pgvector extension as-of
  records HEAD 019_encrypt_connection_config
- tests/test_baseline_idempotency.py: pytest + testcontainers[postgres]
  proving the baseline runs clean on empty DB AND is a no-op on re-run

Source schema: pg_dump --schema-only against running civicrecords-ai
postgres at alembic head 019_encrypt_connection_config (2026-04-24).

Scope: civiccore only. No civicrecords-ai changes. No model extraction.
Phase 1 Part B ships separately: records-side 14-migration guard pass,
env.py wiring (6 lines), and the three migration-gate tests (fresh
install / v1.2.x upgrade / reapplication) from ADR-0003 section 5.
Audit findings on PR #1:
- TEST-001: idempotency test claimed to be 'merge gate' but no CI
  existed. Added .github/workflows/ci.yml that runs both test files
  on PR + push to main; pre-pulls pgvector/pgvector:pg17 to keep
  testcontainers fast.
- DOC-001: README, CONTRIBUTING, pyproject.toml [project.urls],
  and CHANGELOG header still pointed at scottconverse/civiccore
  and scottconverse/civicsuite. Corrected to CivicSuite/* (the
  org Scott created on 2026-04-23). The scottconverse/civicrecords-ai
  URL is intentionally preserved — records repo has not been
  transferred to the CivicSuite org yet.

CHANGELOG.md updated with both entries.
…iverable)

Slip in 63899f7: 'git add -A' swept in a local commit-size-gate hook
config under .claude/ that some skill installed in this working dir.
The files contain no secrets (verified ghp_-grep), but they are
machine-local Claude Code config and don't belong in the public repo.

Adding .claude/ to .gitignore and untracking. Files remain on disk
locally; just no longer tracked. History keeps one commit of them at
63899f7 — no force-push to rewrite, since they're not sensitive and
this is a draft PR with no consumers yet.

Also gitignoring .schema-dump.sql (the temporary pg_dump output used
when authoring the baseline migration; regenerable on demand).
@scottconverse
scottconverse marked this pull request as ready for review April 24, 2026 16:54
@scottconverse
scottconverse merged commit 561f2d7 into main Apr 24, 2026
1 check passed
@scottconverse
scottconverse deleted the phase1/migrations-scaffold branch April 24, 2026 16:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant