Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,15 @@

All notable changes to pgclone are documented in this file.

## [4.1.0]

### Added
- **Schema diff (`pgclone.diff(source_conninfo, schema_name)`)** β€” read-only DDL drift detection between source and the local target. Returns a JSON document with summary counts plus per-category arrays of `only_in_source` / `only_in_target` / `modified` for tables (with per-column `type` / `not_null` / `default` drift), indexes (excluding those backing constraints), constraints, user-defined triggers, views and materialized views, and sequences. Both source and local connections run inside `BEGIN ... READ ONLY` transactions; the function never executes DDL or DML on either side.

### Internal
- New isolated translation unit `src/pgclone_diff.c`. The diff feature does not share helpers with `src/pgclone.c`, keeping the surface area additive and trivially auditable.
- Catalog queries explicitly use `ORDER BY ... COLLATE "C"` to guarantee identical sort order on both sides regardless of the local lc_collate setting, so the merge-walk comparison is deterministic.

## [4.0.1]

### Fixed
Expand Down
10 changes: 6 additions & 4 deletions META.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,14 @@
"name": "pgclone",
"abstract": "Clone PostgreSQL databases, schemas, tables between staging, test, dev and prod environments",
"description": "PostgreSQL extension for easily cloning your DB, Schemas, Tables and more between environments",
"version": "4.0.1",
"version": "4.1.0",
"maintainer": "Valeh Agayev <valeh.agayev@gmail.com>",
"license": "postgresql",
"provides": {
"pgclone": {
"abstract": "Clone PostgreSQL databases, schemas, and tables across environments",
"file": "sql/pgclone--4.0.1.sql",
"version": "4.0.1"
"file": "sql/pgclone--4.1.0.sql",
"version": "4.1.0"
}
},
"prereqs": {
Expand Down Expand Up @@ -42,6 +42,8 @@
"database",
"table",
"masking",
"anonymization"
"anonymization",
"diff",
"drift"
]
}
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

EXTENSION = pgclone
MODULE_big = pgclone
OBJS = src/pgclone.o src/pgclone_bgw.o
OBJS = src/pgclone.o src/pgclone_bgw.o src/pgclone_diff.o
DATA = $(wildcard sql/pgclone--*.sql)

PG_CPPFLAGS = -I$(shell $(PG_CONFIG) --includedir) -Isrc
Expand Down
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
[![CI](https://github.com/valehdba/pgclone/actions/workflows/ci.yml/badge.svg)](https://github.com/valehdba/pgclone/actions/workflows/ci.yml)
[![Postgres 14–18](https://img.shields.io/badge/Postgres-14%E2%80%9318-336791?logo=postgresql&logoColor=white)](https://github.com/valehdba/pgclone)
[![License](https://img.shields.io/badge/License-PostgreSQL-blue.svg)](https://github.com/valehdba/pgclone/blob/main/LICENSE)
[![Version](https://img.shields.io/badge/version-4.0.1-orange)](https://github.com/valehdba/pgclone/releases/tag/v4.0.1)
[![Version](https://img.shields.io/badge/version-4.1.0-orange)](https://github.com/valehdba/pgclone/releases/tag/v4.1.0)

A PostgreSQL extension that clones databases, schemas, tables, and functions between PostgreSQL instances β€” directly from SQL. No `pg_dump`, no `pg_restore`, no shell scripts.

Expand Down Expand Up @@ -141,10 +141,10 @@ pgclone uses Unix domain sockets for local loopback connections, so the default
- [x] v3.4.0: Clone roles with permissions and passwords
- [x] v3.5.0: Clone verification - compare row counts across source and target
- [x] v3.6.0: GDPR/Compliance masking report
- [x] v4.0.0: Schema namespace - all functions under `pgclone` schema (`pgclone.table()`, `pgclone.schema()`, etc.)
- [ ] v4.1.0: Schema diff - DDL drift detection between source and target (`pgclone.diff`)
- [ ] v4.2.0: Pre-flight validator - connection, space, permissions, version, name-conflict checks before clone (`pgclone.preflight`)
- [ ] v4.3.0: FK-aware referential sampling - sample N rows and follow foreign keys to keep child rows consistent (`pgclone.table_sample`)
- [x] v4.0.0: Schema namespace β€” all functions under `pgclone` schema (`pgclone.table()`, `pgclone.schema()`, etc.)
- [x] v4.1.0: Schema diff β€” DDL drift detection between source and target (`pgclone.diff`)
- [ ] v4.2.0: Pre-flight validator β€” connection, space, permissions, version, name-conflict checks before clone (`pgclone.preflight`)
- [ ] v4.3.0: FK-aware referential sampling β€” sample N rows and follow foreign keys to keep child rows consistent (`pgclone.table_sample`)

## License

Expand Down
107 changes: 107 additions & 0 deletions docs/USAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -657,6 +657,112 @@ SELECT * FROM pgclone.masking_report('public');

---

## Schema Diff (v4.1.0)

Detect DDL drift between a source database and the local target without
modifying either side. Useful for answering "is dev still in sync with prod?"
before a release, after a hotfix, or as a scheduled health check.

### Usage

```sql
SELECT pgclone.diff(
'host=source-server dbname=prod user=postgres password=secret',
'app_schema'
)::jsonb;
```

The function compares the named schema on the source against the same-named
schema on the database the call is issued from. It returns a single JSON
document β€” pipe through `::jsonb` (as above) or `jsonb_pretty()` for
readability.

### What is compared

| Category | Compared by | Drift detected on |
|-------------|--------------------------------------------------|------------------------------|
| Tables | `relname` | (presence) |
| Columns | `(table, column)` | type, `NOT NULL`, default |
| Indexes | `index name` | `pg_get_indexdef` |
| Constraints | `constraint name` | `pg_get_constraintdef` |
| Triggers | `trigger name` (user-defined only) | `pg_get_triggerdef` |
| Views | `relname` (regular + materialized) | `pg_get_viewdef` |
| Sequences | `relname` | (presence) |

Indexes that back a primary key or unique constraint are reported under the
**constraints** category, not the indexes category, to avoid double-counting.

### Output shape

```json
{
"schema": "app_schema",
"in_sync": false,
"diff_count": 4,
"summary": {
"tables_only_in_source": 1, "tables_only_in_target": 0, "tables_modified": 1,
"indexes_only_in_source": 1, "indexes_only_in_target": 0, "indexes_modified": 0,
"constraints_only_in_source": 0, "constraints_only_in_target": 0, "constraints_modified": 0,
"triggers_only_in_source": 0, "triggers_only_in_target": 0, "triggers_modified": 0,
"views_only_in_source": 0, "views_only_in_target": 0, "views_modified": 0,
"sequences_only_in_source": 1, "sequences_only_in_target": 0
},
"tables": {
"only_in_source": ["audit_log"],
"only_in_target": [],
"modified": [
{
"name": "customers",
"columns_only_in_source": [
{"name": "loyalty_tier", "type": "text", "not_null": false, "default": null}
],
"columns_only_in_target": [],
"columns_drift": [
{"name": "id",
"source_type": "bigint", "target_type": "integer",
"source_not_null": true, "target_not_null": true,
"source_default": "nextval('customers_id_seq'::regclass)",
"target_default": "nextval('customers_id_seq'::regclass)"}
]
}
]
},
"indexes": { "only_in_source": [...], "only_in_target": [], "modified": [] },
"constraints": { "only_in_source": [], "only_in_target": [], "modified": [] },
"triggers": { "only_in_source": [], "only_in_target": [], "modified": [] },
"views": { "only_in_source": [], "only_in_target": [], "modified": [] },
"sequences": { "only_in_source": ["audit_log_seq"], "only_in_target": [] }
}
```

`in_sync` is `true` only when `diff_count` is `0`.

### Quick boolean check

```sql
SELECT (pgclone.diff(:src, 'app_schema')::jsonb ->> 'in_sync')::boolean AS in_sync;
```

### Notes

- **Read-only on both sides.** Both source and local connections run inside a
`BEGIN ISOLATION LEVEL REPEATABLE READ READ ONLY` transaction; the function
never executes DDL or DML.
- **Permissions.** The calling role only needs read access to `pg_catalog`
on both sides β€” the same access required by `\d` in psql.
- **Sort stability.** Catalog rows are ordered with `COLLATE "C"` so the
comparison is deterministic regardless of the cluster's `lc_collate`.
- **Schema must exist on the source.** If it does not exist on the target,
every source object is reported under `only_in_source` (and vice versa).
- **Currently scoped to a single schema per call.** Loop in SQL to compare
multiple schemas:
```sql
SELECT n, pgclone.diff(:src, n)::jsonb
FROM unnest(ARRAY['public','app','reporting']) AS n;
```

---

## JSON Options Reference

| Option | Type | Default | Description |
Expand Down Expand Up @@ -698,6 +804,7 @@ SELECT * FROM pgclone.masking_report('public');
| `pgclone.verify(conninfo)` | table | Compare row counts for all tables across source and target |
| `pgclone.verify(conninfo, schema)` | table | Compare row counts for tables in a specific schema |
| `pgclone.masking_report(schema)` | table | GDPR/compliance audit: sensitive columns, mask status, recommendations |
| `pgclone.diff(conninfo, schema)` | text (JSON) | DDL drift report: tables, columns, indexes, constraints, triggers, views, sequences |
| `pgclone.table_async(...)` | int | Async table clone (returns job_id) |
| `pgclone.schema_async(...)` | int | Async schema clone (returns job_id) |
| `pgclone.progress(job_id)` | json | Job progress as JSON |
Expand Down
2 changes: 1 addition & 1 deletion pgclone.control
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# pgclone extension
comment = 'Clone PostgreSQL databases, schemas, tables, roles and permissions with selective columns, data filtering, data masking/anonymization, async, parallel cloning, materialized views, resume, and conflict resolution'
default_version = '4.0.1'
default_version = '4.1.0'
module_pathname = '$libdir/pgclone'
relocatable = false
superuser = true
19 changes: 19 additions & 0 deletions sql/pgclone--4.0.1--4.1.0.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
/* pgclone--4.0.1--4.1.0.sql */
\echo Use "ALTER EXTENSION pgclone UPDATE" to load this file. \quit

-- v4.1.0: Schema diff β€” DDL drift detection between source and target.
--
-- Adds pgclone.diff(source_conninfo, schema_name): a read-only
-- comparison of catalog metadata across source and local target.
-- Returns a JSON document describing tables/indexes/constraints/
-- triggers/views/sequences that exist on only one side or differ.

CREATE FUNCTION pgclone.diff(source_conninfo TEXT, schema_name TEXT)
RETURNS TEXT AS 'MODULE_PATHNAME', 'pgclone_diff'
LANGUAGE C VOLATILE STRICT;

COMMENT ON FUNCTION pgclone.diff(TEXT, TEXT) IS
'Compare DDL of a schema between source and the local target. '
'Returns JSON drift report listing objects only_in_source / only_in_target / modified '
'across tables (with per-column type/nullability/default drift), indexes, '
'constraints, triggers, views, and sequences. Read-only on both sides.';
Loading
Loading