Skip to content

Import a managed-provider dump without the user knowing pg_dump flags #1132

Description

@geodro

A dump taken from a managed Postgres (Neon, Supabase, RDS) and imported through lerd ends on hundreds of errors that all come down to two causes, neither of which is the user's mistake.

The dump assigns ownership and privileges to roles that only exist at the provider. A real one measured here carried 77 ALTER ... OWNER TO laravel, two ALTER DEFAULT PRIVILEGES ... TO neon_superuser and a GRANT, so 80 of its errors were ownership statements. lerd's postgres has exactly one role, postgres, and every preset injects DB_USERNAME=postgres, so none of those statements mean anything locally. The load already streams through psql, so a COPY-aware filter on that stream can drop them, which is --no-owner --no-privileges applied at restore time instead of at dump time. The filter has to track COPY ... FROM stdin blocks and leave every byte between the header and \. untouched, or a row of data beginning with GRANT gets mangled. What it skipped belongs in the import report next to the errors, so lerd never rewrites someone's SQL silently.

The mysql families have the same problem wearing a different hat, and a nastier failure mode. Views, triggers, routines and events carry a DEFINER=user@host`` clause. Running as root the import creates them without complaint, so the report reads clean, and the failure surfaces later at query time as ERROR 1449: The user specified as a definer does not exist. Silent at import, broken at runtime. `mysqldump` has no flag for it, stripping the clause is the standard remedy, so the same stream filter should cover it.

The other cause is that the target database was not empty, which produced roughly 241 further errors on the postgres dump: schema, tables, sequences, indexes and constraints that already exist, then duplicate keys and foreign key violations once the COPY data landed on populated tables. Nothing can be filtered there. Import needs an option to empty the target first, a checkbox in the modal and a flag on lerd db:import, and the database is left in a mixed state without it. This half matters far less for mysql, where mysqldump writes its own DROP TABLE/DROP VIEW.

Each half fixes half the problem; together a provider dump imports clean with no flags. The engine-specific rules here should be written so they can move into the service store later, since importShellCommand and exportShellCommand are already family switches in Go and that debt is tracked in #1117.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions