Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion .github/workflows/docker-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ jobs:
runs-on: ubuntu-latest
needs: smoke
permissions:
contents: read
contents: write
packages: write

steps:
Expand Down Expand Up @@ -55,3 +55,8 @@ jobs:
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}

- name: Update GitHub Action version tags
uses: peter-evans/action-update-version-tags@v1
with:
token: ${{ secrets.GITHUB_TOKEN }}
53 changes: 29 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,54 +1,61 @@
# pgsync

One-shot PostgreSQL logical sync: `pg_dump | psql`. rsync-shaped flags, sane defaults (`--no-owner`, `--no-privileges`, `ON_ERROR_STOP` on the target).
**One-shot PostgreSQL logical sync** — copy one database to another with `pg_dump | psql` and opinionated defaults.

CLI • Docker • GitHub Actions

Not incremental replication, CDC, or bidirectional sync — always a full logical dump over the wire.

## Quick start

**Script** — needs Bash 4+ and `pg_dump` / `psql` on `PATH`:
### CLI

Needs Bash 4+ and `pg_dump` / `psql` on `PATH`:

```bash
./pgsync.sh \
-s 'postgresql://user:pass@source:5432/dbname' \
-t 'postgresql://user:pass@target:5432/dbname'
```

**Docker** — no local Postgres client needed; image includes `pg_dump` and `psql`:
### Docker

```bash
docker run --rm ghcr.io/mamahoos/pgsync:latest \
-s 'postgresql://user:pass@source:5432/dbname' \
-t 'postgresql://user:pass@target:5432/dbname'
```

Published on every version tag (`v*`) to [GHCR](https://github.com/mamahoos/pgsync/pkgs/container/pgsync). Pin a release, e.g. `ghcr.io/mamahoos/pgsync:2.0.1`.

Log output goes to **stderr**; stdout stays free for the dump pipe.
Images publish on version tags to [GHCR](https://github.com/mamahoos/pgsync/pkgs/container/pgsync). Pin a release, e.g. `ghcr.io/mamahoos/pgsync:2.1.0`.

## Install
### GitHub Actions

```bash
sudo source ./install.sh # system: /usr/local/bin/pgsync
PREFIX="${HOME}/.local" ./install.sh # user-local, no root
```yaml
- uses: mamahoos/pgsync@v2
with:
source: ${{ secrets.PGSYNC_SOURCE_URI }}
target: ${{ secrets.PGSYNC_TARGET_URI }}
```

Bash tab completion is installed alongside the binary. Open a new shell or `source` the completion file the installer prints.
Prefer a pinned semver (`@v2.1.0`) or commit SHA for production. Do not use `@main`.

## Docker Compose (local demo)
Reference workflow: [`examples/github-action-sync.yml`](examples/github-action-sync.yml).

Two Postgres instances plus a one-shot sync — useful for smoke tests:
Dry-run and verbose output **redact passwords** in connection URIs. Log messages go to **stderr**; stdout stays free for the dump pipe.

## Install

```bash
docker compose run --rm pgsync
sudo source ./install.sh # /usr/local/bin/pgsync
PREFIX="${HOME}/.local" ./install.sh # user-local
```

Custom URIs or a pre-built image:
Bash tab completion ships with the installer.

```bash
docker compose run --rm pgsync \
-s 'postgresql://user:pass@host:5432/src' \
-t 'postgresql://user:pass@host:5432/dst'
## Docker Compose (local demo)

PGSYNC_IMAGE=ghcr.io/mamahoos/pgsync:latest docker compose run --rm pgsync
```bash
docker compose run --rm pgsync
```

## Options
Expand All @@ -60,17 +67,15 @@ PGSYNC_IMAGE=ghcr.io/mamahoos/pgsync:latest docker compose run --rm pgsync
| `-n`, `--dry-run` | Print the pipeline; do not run |
| `--delete` | Drop and recreate `public` on target before restore |
| `--schema-only` | Schema without data |
| `--data-only` | Data only (target table must already exist; use `--no-clean`) |
| `--data-only` | Data only (target table must exist; use `--no-clean`) |
| `--no-clean` | Skip `pg_dump --clean --if-exists` |
| `--single-transaction` | Wrap restore in one transaction (`psql -1`) |
| `-q`, `--quiet` | Suppress the final `pgsync: ok` line |
| `-v`, `--verbose` | Show pipeline and server output |
| `-h`, `--help` | Help |
| `-V`, `--version` | Version |

`--source=URI` and `--target=URI` are accepted. There is no incremental mode — always a full logical dump over the wire.

Standard libpq env vars apply (`PGPASSWORD`, `PGSSLMODE`, …). Non-zero exit on bad args, missing tools, or `psql` errors.
`--source=URI` and `--target=URI` are accepted. Standard libpq env vars apply (`PGPASSWORD`, `PGSSLMODE`, …).

## Tests

Expand Down
77 changes: 77 additions & 0 deletions action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
name: pgsync
description: One-shot PostgreSQL logical sync (pg_dump | psql) with sensible defaults.
branding:
icon: database
color: blue

inputs:
source:
description: Source PostgreSQL connection URI
required: true
target:
description: Target PostgreSQL connection URI
required: true
dry-run:
description: Print the pipeline without running pg_dump or psql
required: false
default: "false"
delete:
description: Drop and recreate the public schema on the target before restore
required: false
default: "false"
schema-only:
description: pg_dump --schema-only
required: false
default: "false"
data-only:
description: pg_dump --data-only
required: false
default: "false"
no-clean:
description: Omit pg_dump --clean --if-exists
required: false
default: "false"
single-transaction:
description: Wrap restore in one transaction (psql -1)
required: false
default: "false"
quiet:
description: Suppress the final pgsync ok line
required: false
default: "false"
verbose:
description: Show pipeline and psql output
required: false
default: "false"

runs:
using: composite
steps:
- name: Ensure PostgreSQL client
shell: bash
run: |
if ! command -v pg_dump >/dev/null 2>&1 || ! command -v psql >/dev/null 2>&1; then
sudo apt-get update -qq
sudo apt-get install -y -qq postgresql-client
fi

- name: Run pgsync
shell: bash
run: |
set -euo pipefail
args=(-s "${{ inputs.source }}" -t "${{ inputs.target }}")

is_true() {
[[ "${1,,}" == "true" ]]
}

is_true "${{ inputs.dry-run }}" && args+=(-n)
is_true "${{ inputs.delete }}" && args+=(--delete)
is_true "${{ inputs.schema-only }}" && args+=(--schema-only)
is_true "${{ inputs.data-only }}" && args+=(--data-only)
is_true "${{ inputs.no-clean }}" && args+=(--no-clean)
is_true "${{ inputs.single-transaction }}" && args+=(--single-transaction)
is_true "${{ inputs.quiet }}" && args+=(-q)
is_true "${{ inputs.verbose }}" && args+=(-v)

"${{ github.action_path }}/pgsync.sh" "${args[@]}"
21 changes: 21 additions & 0 deletions examples/github-action-sync.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Reference workflow — copy into .github/workflows/ in your repo.
#
# Required secrets:
# PGSYNC_SOURCE_URI
# PGSYNC_TARGET_URI

name: Sync PostgreSQL

on:
workflow_dispatch:

jobs:
sync:
runs-on: ubuntu-latest
steps:
- name: Sync source to target
uses: mamahoos/pgsync@v2
with:
source: ${{ secrets.PGSYNC_SOURCE_URI }}
target: ${{ secrets.PGSYNC_TARGET_URI }}
quiet: "true"
23 changes: 20 additions & 3 deletions pgsync.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
#
set -euo pipefail

VERSION="2.0.1"
VERSION="2.1.0"

QUIET=false
VERBOSE=false
Expand Down Expand Up @@ -41,6 +41,16 @@ vmsg() {
say "pgsync: $*"
}

# Hide credentials in logged URIs (dry-run / verbose pipeline).
redact_uri() {
local uri="$1"
if [[ "$uri" =~ ^postgres(ql)?://[^:/@]+:[^@]+@ ]]; then
printf '%s' "$uri" | sed -E 's#^(postgres(ql)?://[^:/@]+):[^@]*@#\1:***@#'
else
printf '%s' "$uri"
fi
}

#############################################
usage() {
cat <<'EOF' >&2
Expand Down Expand Up @@ -116,10 +126,17 @@ PSQL+=("$DST")

#############################################
dry_show() {
local arg
printf ' ' >&2
printf '%q ' "${PGDUMP[@]}" >&2
for arg in "${PGDUMP[@]}"; do
[[ "$arg" == "$SRC" ]] && arg="$(redact_uri "$SRC")"
printf '%q ' "$arg" >&2
done
printf ' | ' >&2
printf '%q ' "${PSQL[@]}" >&2
for arg in "${PSQL[@]}"; do
[[ "$arg" == "$DST" ]] && arg="$(redact_uri "$DST")"
printf '%q ' "$arg" >&2
done
printf '\n' >&2
}

Expand Down
82 changes: 82 additions & 0 deletions tasks/SPEC-github-action.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# Spec: GitHub Action interface for pgsync

## Objective

Add a **composite GitHub Action** in the same repository as a thin wrapper around `pgsync.sh`. The Action is one distribution interface alongside CLI and Docker — not a replacement for them.

**User stories:**
- As a CI maintainer, I can sync two PostgreSQL databases with `uses: mamahoos/pgsync@v2` and secrets for URIs.
- As an operator, dry-run/verbose output never prints database passwords.
- As a new user, README shows CLI, Docker, and GitHub Actions in under 10 seconds.

## Tech Stack

- Bash 4+ (`pgsync.sh` core)
- Composite Action (`action.yml` at repo root)
- Bats for unit/integration tests
- GitHub Actions + GHCR (existing)

## Commands

```bash
./scripts/test.sh # shellcheck + unit tests
INTEGRATION=1 ./scripts/test.sh # + integration tests
docker build -t pgsync:local . # container smoke
```

## Project Structure

```
pgsync.sh # core sync logic
action.yml # composite GitHub Action (wrapper only)
examples/ # reference workflows (not executed by CI)
tasks/ # spec, plan, todo
tests/unit/ # CLI + redaction tests
.github/workflows/ # CI (unchanged triggers except action.yml runs tests)
```

## Code Style

- Single implementation: Action calls `pgsync.sh`; no duplicated pg_dump/psql logic.
- Credentials redacted in any user-visible pipeline output (`dry_show`).
- Action inputs use kebab-case matching CLI flags where possible.

## Testing Strategy

| Level | What | Where |
|-------|------|-------|
| Unit | URI redaction, dry-run output | `tests/unit/redact.bats`, update `dry_run.bats` |
| Unit | CLI unchanged behavior | existing bats |
| Integration | Real sync | existing `tests/integration/` |
| CI | shellcheck includes action if bash embedded | lint job |

## Boundaries

**Always:**
- Redact passwords in dry-run/verbose pipeline display
- Pin Action docs to version tags (`@v2.1.0`), not `@main`
- Keep Action as composite wrapper

**Ask first:**
- Docker-based Action (heavier; defer)
- PostgreSQL version matrix

**Never:**
- Log raw connection URIs with passwords
- Duplicate sync logic in `action.yml`
- Separate repository for the Action

## Success Criteria

- [ ] `action.yml` exists at repo root; composite; all CLI flags exposed as optional inputs
- [ ] `dry_show` redacts `user:password@` in URIs; bats prove password never appears in stderr
- [ ] README repositioned: one-shot sync; CLI • Docker • GitHub Actions; example `uses:` block
- [ ] `examples/github-action-sync.yml` reference workflow
- [ ] Repo description updated on GitHub
- [ ] Version bumped to 2.1.0; unit test updated
- [ ] Tag `v2.1.0` publishes Docker image and floating major tag `v2`
- [ ] `./scripts/test.sh` passes

## Open Questions

- None — same-repo composite Action confirmed by user analysis.
40 changes: 40 additions & 0 deletions tasks/plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Implementation Plan: GitHub Action interface

## Overview

Add composite Action + credential redaction + README/OSS positioning. One core (`pgsync.sh`), three interfaces (CLI, Docker, Action).

## Architecture Decisions

1. **Composite Action** on `ubuntu-latest` — installs `postgresql-client` if missing; delegates to `${{ github.action_path }}/pgsync.sh`.
2. **Credential redaction** in `dry_show` only — runtime still uses real URIs; display uses `redact_uri()`.
3. **Same repo** — Action version tracks git tags (`v2.1.0`); floating `v2` tag updated on release.
4. **examples/** — reference workflow only; not wired into CI triggers.

## Task List

### Phase 1: Security + core display
- [ ] Task 1: `redact_uri()` + update `dry_show` in `pgsync.sh`
- [ ] Task 2: `tests/unit/redact.bats`; fix `dry_run.bats` expectations

### Checkpoint 1
- [ ] `./scripts/test.sh` green

### Phase 2: Action interface
- [ ] Task 3: `action.yml` composite with all inputs
- [ ] Task 4: `examples/github-action-sync.yml`

### Phase 3: OSS packaging
- [ ] Task 5: README reposition + GitHub repo description
- [ ] Task 6: Bump 2.1.0; release workflow floating tag step

### Checkpoint 2
- [ ] Full test suite + tag `v2.1.0` + release

## Risks

| Risk | Mitigation |
|------|------------|
| Password in `%q` escaped output | Test asserts literal password absent |
| Runner lacks pg_dump | Action step installs postgresql-client |
| `@main` usage | README warns; document `@v2` / pinned semver |
Loading
Loading