Skip to content

Tshah-95/parallel-worktree-dev

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Conductor + Parallel Git Worktree Dev Setup

A complete, working setup for running multiple coding agents in parallel against the same Next.js codebase, each in its own git worktree with its own database, dev URL, and browser session — fully isolated, no cross-contamination, no manual port shuffling.

Used in production by carlo-finance (a Next.js 16 / Postgres / Vercel app). Designed for Conductor but works equally well from the shell.


What problem does this solve?

You want to run two (or five, or ten) agents in parallel — each editing the same repo, each running their own dev server, each writing to their own DB — without:

  • Port collisions on localhost:3000
  • One agent's DB writes leaking into another agent's view
  • Cookie/auth bleed between dev servers (host-only cookies need distinct origins)
  • Manually keeping track of which env file points where
  • Re-pulling production data from scratch for every new branch

The setup gives every parallel worktree a frozen identity tuple that derives every per-worktree resource from a single source of truth.


Mental model: one frozen identity, every resource derived from it

When you run bun run worktree:up <branch-name>:

branch name "feat/oauth_v2"
        │
        ▼
sanitize → CARLO_WT_SLUG     = "oauth_v2_a1b2c3"   (Postgres-safe, sha1[:6] suffix)
        → CARLO_WT_SUBDOMAIN = "oauth-v2"          (DNS-safe label)
        → CARLO_WT_BRANCH    = "feat/oauth_v2"     (the original, even if you rename later)
        │
        ▼
   ┌────────────────────────────────────────────────────────────┐
   │ Per-worktree resources, all derived from the slug/subdomain │
   ├────────────────────────────────────────────────────────────┤
   │ App DB           carlo_oauth_v2_a1b2c3                       │
   │ Workflow DB      world_oauth_v2_a1b2c3                       │
   │ Dev URL          https://oauth-v2.carlo.localhost            │
   │ Better Auth URL  https://oauth-v2.carlo.localhost            │
   │ Browser session  carlo-oauth_v2_a1b2c3                       │
   │ DATABASE_URL     postgres://carlo:carlo@localhost:5432/      │
   │                  carlo_oauth_v2_a1b2c3                       │
   └────────────────────────────────────────────────────────────┘

The three CARLO_WT_* vars get written into the worktree's .env.development.local. They are frozen at provision timegit branch -m doesn't change them, because the underlying DBs and dev URLs don't move when the branch renames.

Every tool that touches a worktree's resources (db:status, db:psql, db:pull, db:migrate) sources scripts/lib/worktree-identity.sh — the single derivation surface — so they can never disagree about which DB belongs to which worktree.


What's in this bundle

Local-dev worktree lifecycle (the headline)

File Role
scripts/worktree-up.sh Wrapper: collision-checks, creates the git worktree, hands off to setup
scripts/worktree-setup.sh Operates inside an existing worktree: provisions DBs, writes env files, copies .vercel/, writes browser session, bun install, db-pull
scripts/worktree-down.sh Wrapper: stops processes, removes worktree, deletes branch, hands off to teardown
scripts/worktree-teardown.sh DB drop only (separated so Conductor's archive hook can call it without git ops)
scripts/worktree-stop.sh Kills processes whose CWD is rooted in a worktree path; preserves its own ancestor chain
scripts/lib/worktree-naming.sh Pure helpers: sanitize_branch_for_db, portless_subdomain, find_main_worktree
scripts/lib/worktree-identity.sh Single derivation surface for every per-worktree name

DB tooling (worktree-aware)

File Role
scripts/db-setup-world.sh Idempotent bootstrap of the Workflow Postgres World DB used by @workflow/world-postgres
scripts/db-pull.sh pg_dump from prod (Neon, via vercel env pull) → restore into the active worktree's DB
scripts/db-status.sh Prints the active worktree, branch, slug, subdomain, app DB, world DB, masked DATABASE_URL, table count, last migration. Run this first when in doubt.
scripts/db-psql.sh Worktree-aware psql wrapper. Always targets the same DB db:migrate would mutate.

Conductor wiring

File Role
conductor.json Versioned setup/run/archive hooks. Conductor reads this on workspace create.

Local infra

File Role
docker-compose.yml Local Postgres 17 with pg_stat_statements, max_connections=200. Only the main worktree starts it; parallel worktrees share the same container.
.mise.toml Pins Node 24, auto-loads .env.local + .env.development.local on cd (interactive shells)
package.json.snippet The scripts block to drop into your package.json

Env templates

File Role
env-templates/env.local.example Shape of the main .env.local (Vercel-pulled in carlo-finance)
env-templates/env.development.local.template What worktree-setup.sh writes into each parallel worktree

Bonus: production-infra layer (NOT used by the local-dev flow)

File Role
scripts/db-branch.sh.example Sanitized ZFS-backed prod DB branching script. Runs on a real DB host with sudo + ZFS + PgBouncer. Branches a recent prod replica via copy-on-write — instant, cheap, and the clones are reachable by stable PgBouncer hostnames. The dev flow uses db-pull.sh (slower, self-contained) for ergonomics; db-branch.sh is the upgraded path for teams that have a beefy DB host with ZFS. Read its header for the full architecture.

Provisioning flow (worktree:up <branch>)

flowchart TD
    A[bun run worktree:up feat/oauth_v2] --> B{branch / dir / DBs<br/>collision-free?}
    B -->|no| FAIL([Refuse, suggest worktree:down])
    B -->|yes| C[docker compose up -d<br/>wait for pg_isready]
    C --> D[git worktree add<br/>../carlo-finance-feat/oauth_v2 -b feat/oauth_v2 main]
    D --> E[cd into new worktree<br/>+ run worktree-setup.sh]

    subgraph setup [worktree-setup.sh — runs INSIDE new worktree]
      E --> F[derive slug + subdomain<br/>from current branch]
      F --> G[CREATE DATABASE carlo_slug<br/>CREATE DATABASE world_slug]
      G --> H[bunx workflow-postgres-setup<br/>against world_slug]
      H --> I[symlink .env.local from main]
      I --> J[write .env.development.local<br/>with CARLO_WT_* + 3 URL overrides]
      J --> K[cp -R main/.vercel ./.vercel]
      K --> L[write agent-browser.json<br/>sessionName: carlo-slug]
      L --> M[bun install]
      M --> N{app DB empty<br/>or FORCE_DB_PULL=1?}
      N -->|yes| O[db-pull: pg_dump Neon →<br/>restore into carlo_slug]
      N -->|no| P[skip db-pull]
      O --> Q
      P --> Q[Done]
    end

    Q --> R([cd worktree && bun run dev:portless<br/>https://oauth-v2.carlo.localhost])
Loading

Key design choices:

  • Two-phase (worktree-up.sh + worktree-setup.sh) so Conductor — which does its own git worktree add — can call worktree-setup.sh directly via its setup_script hook, bypassing the wrapper. The shell-only path goes through both.
  • Pre-flight collision checks before any work. A failed worktree:up halfway through would leave orphan resources to clean up.
  • Symlink .env.local, override via .env.development.local. Single source of truth for shared secrets; only the three URL keys differ per worktree. Vercel rotates a secret? vercel env pull from main and every worktree picks it up automatically.
  • Skip db-pull if the DB already has tables. Re-running worktree-setup.sh is idempotent; an existing worktree with local edits doesn't get its data wiped.
  • agent-browser.json is gitignored. Per-worktree value (sessionName: carlo-<slug>) is written by worktree-setup.sh; main-worktree value is {"sessionName": "carlo"}.

Teardown flow (worktree:down <branch>)

flowchart TD
    A[bun run worktree:down feat/oauth_v2] --> B{is the worktree<br/>I'm CD'd into?}
    B -->|yes| FAIL([Refuse — cd elsewhere first])
    B -->|no| C[worktree-stop.sh<br/>SIGTERM all procs rooted in WT path<br/>2s grace, SIGKILL stragglers]
    C --> D{git worktree dirty?}
    D -->|yes, no --force| FAIL2([Refuse with hint to use --force])
    D -->|clean or --force| E[git worktree remove]
    E --> F[worktree-teardown.sh:<br/>DROP DATABASE carlo_slug WITH FORCE<br/>DROP DATABASE world_slug WITH FORCE]
    F --> G{branch unmerged?}
    G -->|yes, no --force| FAIL3([Refuse with hint])
    G -->|merged or --force| H[git branch -d/-D]
    H --> I([Done])
Loading

The two-phase split here mirrors up: worktree-down.sh does git-side ops, worktree-teardown.sh does the DB drops. Conductor's archive hook only needs the DB drop.

worktree-stop.sh walks lsof -d cwd to find every process whose working directory is rooted inside the soon-to-be-removed worktree (dev server, Next.js workers, MCP servers, agent shells, watchers). It carefully preserves its own ancestor chain so it doesn't kill the script that's running it.


Resource topology — main vs parallel worktrees

~/Github/your-app/                                       ← main worktree
  ├── .env.local                  Vercel-pulled, source of truth
  ├── .env.development.local      not used in main
  ├── agent-browser.json          {"sessionName": "carlo"}
  └── (your code on `main`)
                                                          ┌──────────┐
docker compose container ────────────────────────────────►│ Postgres │
  ├── DB: carlo                                            │  :5432   │
  ├── DB: world                                            │ shared   │
  ├── DB: carlo_oauth_v2_a1b2c3   ← parallel worktree 1   │  by all  │
  ├── DB: world_oauth_v2_a1b2c3                            │ worktrees│
  ├── DB: carlo_billing_d4e5f6    ← parallel worktree 2   └──────────┘
  └── DB: world_billing_d4e5f6

~/Github/your-app-feat/oauth_v2/                         ← parallel worktree 1
  ├── .env.local → ~/Github/your-app/.env.local          (symlink)
  ├── .env.development.local      CARLO_WT_*, DATABASE_URL=...carlo_oauth_v2_a1b2c3
  ├── agent-browser.json          {"sessionName": "carlo-oauth_v2_a1b2c3"}
  └── (your code on `feat/oauth_v2`)
       │
       └── bun run dev:portless → https://oauth-v2.carlo.localhost

~/Github/your-app-billing/                               ← parallel worktree 2
  ├── .env.local → ~/Github/your-app/.env.local          (symlink — same source!)
  ├── .env.development.local      CARLO_WT_*, DATABASE_URL=...carlo_billing_d4e5f6
  ├── agent-browser.json          {"sessionName": "carlo-billing_d4e5f6"}
  └── (your code on `billing`)
       │
       └── bun run dev:portless → https://billing.carlo.localhost

Three independent agents can each bun run dev:portless simultaneously — distinct DBs, distinct origins, distinct cookies, no port collisions. They share one Postgres container, one Vercel-pulled secret file, one Docker daemon.


How to adapt this to your project

Most of it ports directly. Find-and-replace these to match your project:

Find Replace with Where
carlo (project name) <your-project> scripts (lots of places — DB names, container names, hostnames, browser session)
carlo-finance-db-1 <your-project>-db-1 (or ${DB_CONTAINER} env var — already supported) scripts
carlo-finance- (worktree dir prefix) <your-app>- worktree-up.sh, worktree-down.sh
vercel env pull (in db-pull.sh) your platform's secret-pull command db-pull.sh
NEON_DATABASE_URL_UNPOOLED your prod DB URL var name db-pull.sh
@workflow/world-postgres references drop entirely if you don't use Vercel Workflow worktree-setup.sh, db-setup-world.sh
portless / *.carlo.localhost however you do dev URLs (mkcert + nginx, traefik, etc.) worktree-naming.sh::portless_subdomain, worktree-identity.sh::worktree_better_auth_url
BETTER_AUTH_URL whatever env var your auth library checks for the canonical origin worktree-setup.sh, worktree-identity.sh

What you must keep

The two-tier env file pattern (.env.local symlinked from main + .env.development.local with overrides). It's the cleanest way to keep secrets in one place while letting per-worktree URLs win. Next.js's .env.development.local > .env.local precedence is the load-bearing piece; if you're not on Next.js, check that your framework has a similar override mechanism.

What's optional

  • Workflow Postgres World (the world_<slug> DB and db-setup-world.sh) — only relevant if you use @workflow/world-postgres for durable workflows. Drop these references if not.
  • Portless — for stable HTTPS dev URLs at *.<project>.localhost without port suffixes. If you're fine with localhost:3001, localhost:3002, etc., skip the subdomain machinery; per-worktree dev URL becomes "whatever PORT you set".
  • db-pull.sh — if your prod isn't on a managed Postgres or you don't want every worktree to start with prod data, replace with a fixture-loading script.

Conductor

Conductor is a Mac app that creates parallel git worktrees with a UI. Wiring is via conductor.json:

{
  "scripts": {
    "setup":   "bash scripts/worktree-setup.sh",
    "run":     "bun run dev:portless",
    "archive": "bash scripts/worktree-teardown.sh"
  }
}

Conductor invokes:

  1. setup after creating its own worktree — worktree-setup.sh provisions everything inside it.
  2. run to start the dev server (concurrent with the agent panel).
  3. archive on workspace delete — drops the worktree's DBs.

The factoring (separate worktree-up.sh and worktree-setup.sh) is what makes this dual-mode-friendly: shell users get the wrapper that creates the git worktree; Conductor users skip the wrapper because Conductor handles git worktree add itself.


Gotchas worth knowing

  • The Postgres connection-pool ceiling is max_connections=200 (in docker-compose.yml). At ~10–20 connections per active worktree, that caps you around 10 parallel dev servers. Tune up if you push past it.
  • portless requires sudo on first run to bind 443 and install a local CA. One-time. After that it's a long-lived daemon that survives logout.
  • vercel link changes don't propagate. worktree-setup.sh copies .vercel/ (not symlinks) so each worktree has an independent project link. If you re-link in main, copy the new .vercel/ into every worktree or tear them down.
  • A migration "didn't apply" is almost never the migration script being broken. It's almost always you (or your agent) running raw psql against the default carlo DB instead of the worktree's carlo_<slug>. bun run db:status first; bun run db:psql for ad-hoc queries.
  • Dirty worktree teardown. worktree-down.sh refuses without --force. This catches "I forgot to commit" mistakes.
  • Branch-rename safety. The slug is frozen at provision time. git branch -m updates CARLO_WT_BRANCH (read fresh from git rev-parse) but leaves the DB names alone — the resources don't move.

What this setup deliberately doesn't include

  • No multi-agent merge harness. Branches + GitHub PRs are the coordination surface today; the merge queue + hotspot locks + coordinator-agent layer is a follow-up.
  • No automated branch naming. You (or your agent) pick the name.
  • No cross-worktree IPC. Agents talk via gh pr comment / gh pr review / labels.
  • No Windows support. macOS-first; Linux probably works with minor tweaks (Docker, lsof, shasum). Windows would need WSL.

Files in this gist, by category

.
├── README.md                                  ← you are here
├── conductor.json                             ← Conductor hooks
├── docker-compose.yml                         ← local Postgres
├── .mise.toml                                 ← Node version + env auto-load
├── package.json.snippet                       ← scripts block
├── env-templates/
│   ├── env.local.example                      ← shape of main env
│   └── env.development.local.template         ← shape of per-worktree overrides
└── scripts/
    ├── worktree-up.sh                         ← public CLI: provision
    ├── worktree-down.sh                       ← public CLI: teardown
    ├── worktree-setup.sh                      ← inside-worktree provision (Conductor entry)
    ├── worktree-teardown.sh                   ← DB drop (Conductor archive entry)
    ├── worktree-stop.sh                       ← process killer used by teardown
    ├── db-setup-world.sh                      ← Workflow Postgres bootstrap
    ├── db-pull.sh                             ← Neon prod → local
    ├── db-status.sh                           ← worktree-aware status
    ├── db-psql.sh                             ← worktree-aware psql wrapper
    ├── db-branch.sh.example                   ← BONUS: prod-infra ZFS branching (sanitized)
    └── lib/
        ├── worktree-naming.sh                 ← sanitizers
        └── worktree-identity.sh               ← single source of truth for derived names

If anything in here feels load-bearing but undocumented, ask the person who shared this with you — most of the design decisions live in the script headers.

About

Parallel git worktree dev setup for Next.js + Conductor — multiple coding agents in parallel, each with its own DB, dev URL, browser session. Frozen identity tuple, worktree-aware DB tooling, Conductor wiring.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors