Skip to content

refactor(core): Postgres template databases for sub-second branch reset - #2

Merged
faculopezscala merged 1 commit into
mainfrom
worktree-agent-a72cc7fb
Apr 6, 2026
Merged

refactor(core): Postgres template databases for sub-second branch reset#2
faculopezscala merged 1 commit into
mainfrom
worktree-agent-a72cc7fb

Conversation

@faculopezscala

Copy link
Copy Markdown
Contributor

Summary

Lane B (Issue #1): refactor the Docker branch provider so `sow branch reset` runs in well under 1.5s instead of the current 5-15s.

Architecture change

Before: One Docker container per branch. Reset = `docker rm -f` + `docker run` + `pg_isready` poll + `psql -f init.sql`. Total: 5-15s.

After: One long-lived Docker container per connector, hosting:

  • a frozen seed database (`sow_seed_`) with `IS_TEMPLATE=true` and `ALLOW_CONNECTIONS=false`
  • N per-branch databases (`sow_`) cloned from the seed via `CREATE DATABASE ... TEMPLATE seed`

Reset is now just:
```sql
DROP DATABASE sow_feature_a WITH (FORCE);
CREATE DATABASE sow_feature_a WITH TEMPLATE sow_seed_myconn OWNER sow;
```
That's ~200-800ms on a 10k-row schema — roughly 10x faster than the old container-recycle path. Container boot cost is amortized across every branch for the connector instead of being paid on every reset.

What changed

  • New low-level helpers in `branching/docker.ts` (`createConnectorContainer`, `waitForConnectorReady`, `execSqlInDb`, `loadInitSqlIntoDb`, `dumpDatabase`, `restoreDumpToDatabase`, `listDatabases`).
  • `DockerBranchProvider` rewritten around per-connector `container.json` metadata (stored at `.sow/snapshots//container.json`).
  • `providerMetaVersion: 2` with explicit, actionable migration error for v1 branches: "Run `sow branch delete ` and recreate the branch." (Option A from the plan — safer than transparent migration for v0.2.)
  • `stopBranch` becomes per-branch connection termination via `pg_terminate_backend` (the container is shared so stopping it would kill sibling branches). `startBranch` verifies the container is up and re-starts it if it was stopped at the docker level.
  • `deleteBranch` drops the database with `WITH (FORCE)` and tears the container down only when the last non-seed branch is gone, so we don't leak idle Postgres processes.
  • `manager.resetBranch` no longer special-cases docker by recreating the branch — the in-place reset preserves the connection string, port, and metadata.
  • All dynamic SQL goes through `quoteIdent`. Connector and branch names are validated against `/^[a-zA-Z0-9_-]{1,40}$/` to prevent injection through the connector name itself. `CREATE/DROP DATABASE` calls are issued outside any transaction (Postgres requires this).
  • CLI `sow branch info` now displays the database and container names so users can understand the new model.

Tests

  • 7 new unit tests in `packages/core/src/branching/providers/docker.test.ts` with mocked docker helpers, covering: first-branch container creation, second-branch container reuse, unsafe-name rejection, reset issues DROP+CREATE, deleteBranch leaves container when other branches remain, deleteBranch removes container when last branch is deleted, v1-meta migration rejection.
  • 1 BUGSTER_DB_URL-gated integration benchmark in `reset-perf.integration.test.ts` asserting that a real reset on the bugsterdb fixture completes in < 1500ms.
  • All 89 pre-existing tests still pass (96/96 total).

Test plan

  • `bunx vitest run` — 96/96 passing
  • `bunx turbo build` — clean
  • `bunx eslint` on touched files — clean
  • `BUGSTER_DB_URL=postgresql://... bunx vitest run --config vitest.integration.config.ts` — manually verify <1500ms reset against a live 10k-row schema

Migration story

Branches created by older sow versions have a v1 `providerMeta` shape (`{ containerId, containerName, pgVersion }` with no `databaseName`). Any operation on such a branch now throws a clear error instructing the user to delete and recreate the branch. This is safer than auto-migrating because the old per-branch containers would need to be torn down and the user's port assignments may shift.

🤖 Generated with Claude Code

Replaces the per-branch container model with one long-lived container per
connector that hosts a frozen seed database (IS_TEMPLATE=true) plus N
branch databases cloned via CREATE DATABASE ... TEMPLATE seed.

Reset goes from "docker rm -f + docker run + waitForReady + psql restore"
(5-15s) to "DROP DATABASE WITH (FORCE); CREATE DATABASE FROM TEMPLATE"
(~200-800ms on a 10k-row schema), an ~10x improvement.

Key changes:
- New low-level helpers in branching/docker.ts: createConnectorContainer,
  waitForConnectorReady, execSqlInDb, loadInitSqlIntoDb, dumpDatabase,
  restoreDumpToDatabase, listDatabases.
- DockerBranchProvider rewritten around container.json (one per connector,
  in .sow/snapshots/<connector>/container.json).
- providerMetaVersion=2 with explicit migration error for v1 branches.
- stopBranch is now per-branch connection termination (the container is
  shared); startBranch verifies the container is running.
- manager.resetBranch no longer recreates the container in the docker case.
- All identifiers go through quoteIdent and connector/branch names are
  regex-validated to prevent injection through the connector name.
- 7 new unit tests with mocked docker helpers; one BUGSTER_DB_URL-gated
  integration benchmark asserting reset < 1500ms.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@faculopezscala
faculopezscala merged commit ac87021 into main Apr 6, 2026
1 check passed
faculopezscala added a commit that referenced this pull request Apr 6, 2026
The 5 code PRs (#2-#6) deliver the engineering for the launch. This PR
delivers everything user-facing: the README pitch, the package metadata,
the docs/ folder, and the CHANGELOG.

This PR depends on PRs #2-#6 being merged first because the new README
references commands those PRs add (sow sandbox, sow doctor <connector>,
the --allow-unsafe flag, the sub-second reset). Land them first, rebase
this against main, then merge.

README.md
  Hero rewritten from "Safe test databases from production Postgres" to
  "Stop letting Claude touch your prod database". Body explains the
  anxiety-reduction pitch: a coding agent is about to do something
  database-adjacent and you feel that quiet pang. sow is the safety
  layer. New "Why sow" section. New "How It Works" diagram showing the
  template-DB shape (one container per connector, N branch DBs, reset
  in <1s). New "Cookbook" stub linking to docs/cookbook.md. New
  "Documentation" section with the docs/ index.

packages/cli/package.json
  Description: "Stop letting Claude touch your prod database. PII-safe
  local Postgres sandbox for coding agents."
  Keywords: added ai-agents, coding-agents, claude-code, cursor,
  sandbox, mcp.

packages/core/package.json
  Description: "sow core engine — analyze, sample, sanitize, and branch
  Postgres databases for safe coding-agent sandboxes"
  Keywords: added ai-agents, coding-agents, sandbox.

packages/mcp/package.json
  Description corrected from "15 tools" to "22 tools" (the actual count
  in packages/mcp/src/index.ts) and repositioned: "sow MCP server — 22
  tools for coding agents (Claude Code, Cursor, Codex) to safely manage
  Postgres sandboxes"
  Keywords: added claude-code, cursor, codex, coding-agents, sandbox.

docs/sandbox.md (new)
  The sow sandbox flagship command — what it does, the flags, the
  .env.local backup/revert flow, when not to use it, and what's
  actually in the sandbox.

docs/sanitization.md (new)
  What sow sanitizes (the PII type table), how JSONB walking works,
  the fail-closed gate, the --allow-unsafe escape hatch, custom rules
  via .sow.yml, what sow does NOT do (free-text NER, etc.), and the
  read-only-on-the-source guarantee.

docs/cookbook.md (new)
  Three end-to-end workflows with concrete prompts:
  1. Let Claude refactor your schema without fear
  2. Let Cursor generate seed data for a new feature
  3. Let your coding agent debug a failing migration
  Plus the "agent reset loop" pattern diagram, the MCP tool list,
  and operational tips (one long-running sandbox per project,
  checkpoints for known-good states, sow doctor as the inspection
  surface, principle of least privilege on the source DB user).

CHANGELOG.md (new)
  Scaffold with three sections:
  - [Unreleased] documenting the planned PR #2-#6 features under
    Added/Changed
  - [0.1.14] documenting the SQL injection security fix that already
    shipped (PR #1, merged earlier in the session)
  - [0.1.13] one-line summary of the initial public release

Test/build/lint all clean (89/89 tests, 3/3 packages built, no source
code changed in this PR).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant