Repository navigation
firedrill 0.5.0: pgBackRest and WAL-G backups, webhook alerts - #6
Merged
Merged
Conversation
`firedrill physical pgbackrest|walg` restores with the tool itself into a container of the backup's own major, replays WAL (to --target for PITR) and runs smoke, semantics and integrity. The major version is read from backup.info / the WAL-G stop sentinel; remote repositories use the tools' own PGBACKREST_*/WALG_*/AWS_* variables, passed by name. The restored server is started with -c archive_mode=off and then asked, so production's archive_command can never push WAL into the repository being checked -- tested with a backup carrying archive_mode=on. Tested against repositories made by pgBackRest 2.59.1 and WAL-G v3.0.9 (tests/make_physical.py): full replay, exact PITR boundary, target past the archive, target before every backup (pgBackRest [075] and Postgres's own message), a corrupted file, and --max-age staleness. A restore that ran and broke is FAIL, not COULD NOT VERIFY. --notify ENV_VAR posts the verdict to a Slack/Discord/any webhook on failure or could-not-verify; the Action gains physical and notify inputs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI showed Postgres recovering to a LATER moment than asked: with no commit during the backup it cannot stop before the backup's end, so it promotes there. Backup finish times come from backup.info / the WAL-G sentinel, so a target before every backup is PITR_TARGET_BEFORE_BACKUP up front, and WAL-G now fetches the newest base backup that finished before the target. A server that looked ready (Postgres serves reads during recovery) and then exited because the archive ran out is PITR_TARGET_UNREACHED, not PITR_FAILED. The action-flags test knows the action calls `physical`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`firedrill run` recognises a mysqldump / mariadb-dump file (plain or .gz) by its header, restores it into mysql:<version> / mariadb:<version> from that header, and runs smoke, semantics and CHECK TABLE. A missing `-- Dump completed` trailer is ARCHIVE_TRUNCATED before anything is restored; the client's `ERROR nnnn at line N` becomes a finding. MariaDB 10.5.25+/11.x dumps open with a sandbox-mode line before the header; it is read past. The password reaches the client via MYSQL_PWD inside the container -- exported, since a prefix on a pipeline applies only to its first command and `cat dump | mysql` logged in without one. Tested on datacharmer/test_db's employees database (300,024 employees, 2,844,047 salaries) dumped by MySQL 8.4 and MariaDB 11.4: both pass, gzipped passes, a copy cut at 80 MB fails with the trailer, the client's error line, 7 of 8 tables and both counts wrong. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
firedrill physical pgbackrest|walgrestores with the tool itself intoa container of the backup's own major, replays WAL (to --target for
PITR) and runs smoke, semantics and integrity. The major version is
read from backup.info / the WAL-G stop sentinel; remote repositories
use the tools' own PGBACKREST_/WALG_/AWS_* variables, passed by name.
The restored server is started with -c archive_mode=off and then asked,
so production's archive_command can never push WAL into the repository
being checked -- tested with a backup carrying archive_mode=on.
Tested against repositories made by pgBackRest 2.59.1 and WAL-G v3.0.9
(tests/make_physical.py): full replay, exact PITR boundary, target past
the archive, target before every backup (pgBackRest [075] and Postgres's
own message), a corrupted file, and --max-age staleness. A restore that
ran and broke is FAIL, not COULD NOT VERIFY.
--notify ENV_VAR posts the verdict to a Slack/Discord/any webhook on
failure or could-not-verify; the Action gains physical and notify inputs.
🤖 Generated with Claude Code