Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: civiccore CI

on:
pull_request:
branches: [main]
push:
branches: [main]

jobs:
tests:
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@v5

- name: Set up Python 3.13
uses: actions/setup-python@v6
with:
python-version: "3.13"
cache: pip

- name: Verify Docker is available
run: docker version

- name: Pre-pull pgvector image
run: docker pull pgvector/pgvector:pg17

- name: Install civiccore (editable, dev extras)
run: |
pip install --upgrade pip
pip install -e .[dev]

- name: Run smoke test
run: pytest tests/test_smoke.py -v --tb=short

- name: Run baseline idempotency test
run: pytest tests/test_baseline_idempotency.py -v --tb=short
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -64,3 +64,9 @@ desktop.ini
*.sqlite3
*.db
.alembic-tmp/

# Local Claude Code project tooling (hooks, settings) — machine-local, not project deliverables
.claude/

# Local schema dumps used during baseline authorship — regenerate with pg_dump if needed
.schema-dump.sql
13 changes: 12 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ All notable changes to **civiccore** are documented here. Format follows
adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

CivicCore is the shared platform package for the
[CivicSuite](https://github.com/scottconverse/civicsuite) open-source
[CivicSuite](https://github.com/CivicSuite/civicsuite) open-source
municipal operations suite. Per the CivicCore Extraction Spec section 16,
breaking changes to the public API surface (Appendix A of that spec) ship
as MAJOR releases; new symbols or backward-compatible behavior ship as
Expand Down Expand Up @@ -33,13 +33,24 @@ MINOR; bug fixes ship as PATCH.
- Apache 2.0 `LICENSE`, `README.md`, `CONTRIBUTING.md` (with the
bug-routing decision tree from spec section 18), `.gitignore`, and
placeholder `civiccore-ui/` npm package directory.
- `civiccore.migrations.guards` — three idempotent op wrappers (`idempotent_create_table`, `idempotent_add_column`, `idempotent_alter_column`) plus `has_table` helper.
- `civiccore.migrations.runner` — `upgrade_to_head(connection)` and `current_revision(connection)` entry points for consuming modules' env.py.
- `civiccore/migrations/alembic.ini` + `civiccore/migrations/env.py` — civiccore's own Alembic wiring (`alembic_version_civiccore` version table to avoid collision with consuming modules).
- `civiccore_0001_baseline_v1` migration — idempotent snapshot of the 16 civiccore-owned shared tables at records HEAD `019_encrypt_connection_config`, per ADR-0003.
- `tests/test_baseline_idempotency.py` — pytest asserting the baseline runs clean on an empty DB and is a no-op against an already-populated DB.
- `.github/workflows/ci.yml` — CI workflow on `pull_request`/`push` to `main`. Runs `tests/test_smoke.py` and `tests/test_baseline_idempotency.py` on `ubuntu-latest`/Python 3.13; pre-pulls `pgvector/pgvector:pg17` so the testcontainers-managed Postgres starts cleanly. Makes the idempotency test an actual enforced merge gate (was claim-only before).

### Changed
- License switched from MIT to Apache License 2.0 to match civicrecords-ai
(which is Apache-2.0). Spec doc 02 Appendix D and CONSISTENCY.md
section 6 are being updated in the umbrella repo in the same change.
- `docs/index.html` landing page added to satisfy the project's pre-push
documentation gate.
- README, CONTRIBUTING, pyproject.toml, and CHANGELOG itself: stale
`scottconverse/civiccore` and `scottconverse/civicsuite` URLs corrected
to `CivicSuite/civiccore` and `CivicSuite/civicsuite` (org-hosted as of
2026-04-23). The `scottconverse/civicrecords-ai` URLs are unchanged —
records repo has not yet been transferred to the CivicSuite org.

No release sections yet — `0.1.0` ships with Phase 1 of the CivicCore
extraction (shared models + audit chain), per spec section 12.
12 changes: 6 additions & 6 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Contributing to CivicCore

Thanks for considering a contribution. CivicCore is the shared platform
package for the [CivicSuite](https://github.com/scottconverse/civicsuite)
package for the [CivicSuite](https://github.com/CivicSuite/civicsuite)
open-source municipal operations suite — every line of code here is
consumed by every CivicSuite module, so the bar is high and the surface
is deliberately small.
Expand All @@ -12,7 +12,7 @@ CivicCore is at v0.1 (Phase 0 — package skeleton). Functional code lands
in Phase 1 and beyond per the CivicCore Extraction Spec section 12. If
you want to contribute today, the most useful work is:

- Reviewing the Extraction Spec (in scottconverse/civicsuite) and filing
- Reviewing the Extraction Spec (in CivicSuite/civicsuite) and filing
issues against ambiguous wording.
- Building a Phase 1 prototype against the auth + audit subsystem
contracts in Appendix A of the spec.
Expand All @@ -33,7 +33,7 @@ section 18 ("Contributor confusion about where to file a bug").
notification service, onboarding, the municipal systems catalog, the
50-state exemption engine, sovereignty verification scripts, shared
ORM models, or shared-table Alembic migrations.
➜ **File it here:** https://github.com/scottconverse/civiccore/issues
➜ **File it here:** https://github.com/CivicSuite/civiccore/issues

2. **Is the bug in records-request workflow, response-letter generation,
fee schedules, the records dashboards, or any records-specific UI
Expand All @@ -49,7 +49,7 @@ section 18 ("Contributor confusion about where to file a bug").
4. **Is the bug about how the modules fit together, the suite-wide
roadmap, the module catalog, or cross-module documentation?**
➜ **File it in the CivicSuite umbrella:**
https://github.com/scottconverse/civicsuite/issues
https://github.com/CivicSuite/civicsuite/issues

5. **Are you reporting a security vulnerability?**
Do not file it as a public issue. See "Security advisories" below.
Expand All @@ -72,7 +72,7 @@ Requirements:
Clone and install in editable mode with the dev extras:

```bash
git clone https://github.com/scottconverse/civiccore.git
git clone https://github.com/CivicSuite/civiccore.git
cd civiccore
python -m venv .venv
# macOS / Linux:
Expand Down Expand Up @@ -142,7 +142,7 @@ commit (currently: `pyproject.toml` `[project].version` and
Do not file security issues as public GitHub issues. Use GitHub's
private vulnerability reporting on this repository (Security tab → Report
a vulnerability), or email the maintainer listed on the
[CivicSuite umbrella repo](https://github.com/scottconverse/civicsuite)
[CivicSuite umbrella repo](https://github.com/CivicSuite/civicsuite)
governance page. We will acknowledge within seven days and coordinate a
fix and disclosure timeline with you.

Expand Down
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# CivicCore

Shared platform package for the
[CivicSuite](https://github.com/scottconverse/civicsuite) open-source
[CivicSuite](https://github.com/CivicSuite/civicsuite) open-source
municipal operations suite.

## What this is
Expand Down Expand Up @@ -33,7 +33,7 @@ pip install civiccore
For now (pre-release), install from a clone:

```bash
git clone https://github.com/scottconverse/civiccore.git
git clone https://github.com/CivicSuite/civiccore.git
cd civiccore
pip install -e .[dev]
```
Expand All @@ -44,7 +44,7 @@ CivicCore's v0.1 public API is deliberately lean. The full list of
exported symbols — which is **stable across the v0.x series** per the
spec's semver policy — is published in **Appendix A of the CivicCore
Extraction Spec** in
[scottconverse/civicsuite](https://github.com/scottconverse/civicsuite).
[CivicSuite/civicsuite](https://github.com/CivicSuite/civicsuite).
Refer to that document; this README does not duplicate the list, so the
two cannot drift.

Expand All @@ -54,7 +54,7 @@ Every CivicSuite module's README declares a CivicCore version range
(e.g. `civiccore >= 0.1, < 0.2`). The suite-wide compatibility matrix —
which module versions work with which CivicCore versions — is maintained
at
[scottconverse/civicsuite/docs/compatibility/](https://github.com/scottconverse/civicsuite/tree/main/docs/compatibility).
[CivicSuite/civicsuite/docs/compatibility/](https://github.com/CivicSuite/civicsuite/tree/main/docs/compatibility).

## License

Expand Down
3 changes: 3 additions & 0 deletions civiccore/migrations/alembic.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[alembic]
script_location = %(here)s
version_table = alembic_version_civiccore
131 changes: 88 additions & 43 deletions civiccore/migrations/env.py
Original file line number Diff line number Diff line change
@@ -1,43 +1,88 @@
"""Alembic environment for CivicCore shared-table migrations.

STUB ONLY — Phase 0 scaffold. No real migration revisions are seeded yet.
Phase 1 will baseline this against the latest CivicRecords AI migration that
touches a shared table (users, roles, audit_log, documents, document_chunks,
model_registry, connectors, notification_templates, city_profile,
exemption_rules) and mark it as the CivicCore baseline revision.

Migration-ordering contract (from CivicCore Extraction Spec section 14):

1. CivicCore's migration runner is called FIRST by every consuming
module's runner. The wrapper in each module is a thin shim around
Alembic that calls `civiccore.migrations.run()` before applying its
own revisions.

2. For a fresh install, the sequence is:
(a) civiccore_migrate upgrade head
(b) <module>_migrate upgrade head # records, clerk, code, zone, ...

3. Order is enforced by Alembic's `depends_on` metadata on every
module-side revision that touches a shared table. A module revision
declares `depends_on = ("<civiccore_revision_id>",)` so a fresh DB
cannot race the two runners.

4. CivicCore NEVER imports from any module. Module migrations may
reference shared tables, but the dependency arrow points one way:
module --> civiccore.

5. Shared-table schema changes are MAJOR CivicCore releases
(semver-major). Minor and patch CivicCore releases never alter
shared table schemas. See spec section 16.

This file deliberately does not configure a context, target_metadata, or
run_migrations_online() yet. Phase 1 will fill those in once the shared
models module is populated.
"""

# Phase 1 will add:
# from alembic import context
# from civiccore.models import Base
# target_metadata = Base.metadata
# def run_migrations_online() -> None: ...
# def run_migrations_offline() -> None: ...
"""Alembic env for civiccore. Reads the connection from Config.attributes['connection'] when invoked by runner.py (the normal path). Falls back to DATABASE_URL env var when invoked standalone."""

from __future__ import annotations

import os
from logging.config import fileConfig

from alembic import context
from sqlalchemy import engine_from_config, pool

config = context.config

# Tolerate the minimal alembic.ini which intentionally omits [loggers]/[handlers]/[formatters].
if config.config_file_name is not None:
try:
fileConfig(config.config_file_name)
except KeyError:
# No logging sections configured; skip Python logging setup.
pass

VERSION_TABLE = "alembic_version_civiccore"
target_metadata = None # v0.1 baseline is raw-SQL; no ORM metadata yet.


def run_migrations_offline() -> None:
"""Run migrations in 'offline' mode — emits SQL to stdout / script without a DB connection."""
url = os.environ.get("DATABASE_URL") or config.get_main_option("sqlalchemy.url")
context.configure(
url=url,
target_metadata=target_metadata,
literal_binds=True,
dialect_opts={"paramstyle": "named"},
version_table=VERSION_TABLE,
)
with context.begin_transaction():
context.run_migrations()


def run_migrations_online() -> None:
"""Run migrations in 'online' mode.

Two sub-modes:
- Connection supplied by caller via `config.attributes["connection"]`
(the normal path: records' env.py hands us its live connection through
`runner.upgrade_to_head`). We do not dispose the connection; the caller
owns its lifecycle.
- Standalone CLI (`alembic upgrade head`): no attached connection, so
build an engine from `DATABASE_URL` (or the ini's `sqlalchemy.url`) and
run migrations against it.
"""
connectable = config.attributes.get("connection", None)

if connectable is not None:
context.configure(
connection=connectable,
target_metadata=target_metadata,
version_table=VERSION_TABLE,
)
with context.begin_transaction():
context.run_migrations()
return

# Standalone path: no connection was passed in.
url = os.environ.get("DATABASE_URL")
ini_section = config.get_section(config.config_ini_section) or {}
if url:
ini_section["sqlalchemy.url"] = url

engine = engine_from_config(
ini_section,
prefix="sqlalchemy.",
poolclass=pool.NullPool,
)

with engine.connect() as connection:
context.configure(
connection=connection,
target_metadata=target_metadata,
version_table=VERSION_TABLE,
)
with context.begin_transaction():
context.run_migrations()


if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()
Loading