Skip to content

README.md

Latest commit

 

History

History
309 lines (230 loc) · 7.51 KB

README.md

File metadata and controls

309 lines (230 loc) · 7.51 KB

BackupForge

Build Status Version License Python Version

BackupForge is a production-focused command-line utility for backing up and restoring databases across multiple engines. It provides a consistent interface for creating reliable backups, storing them locally or on S3, applying compression, and validating artifact integrity with cryptographic manifests.

Repository: https://github.com/RajeshBasnet-dev/Database_Backup_Utility_CLI

Table of Contents

Why BackupForge

BackupForge is designed for teams and builders who need:

  • Multi-database support with one CLI interface.
  • Operational simplicity for backup, restore, and scheduling flows.
  • Reliability controls such as compression and backup integrity verification.
  • Deployment flexibility for local-first workflows and cloud storage (AWS S3).

Key Features

  • 🔥 Multi-Database Support: MySQL, PostgreSQL, MongoDB, SQLite
  • 💾 Storage Backends: Local filesystem and AWS S3
  • 🗜️ Compression: ZIP and GZIP options
  • ⏰ Scheduling: Automated recurring backups
  • 📢 Notifications: Slack webhook integration for backup and restore outcomes
  • 🔐 Integrity Manifests: SHA-256 + size metadata for tamper detection
  • 🌐 Cross-Platform: Linux, macOS, and Windows compatible

Supported Databases

Database Backup Restore Compression S3 Storage Integrity Verify
MySQL ✅ ✅ ✅ ✅ ✅
PostgreSQL ✅ ✅ ✅ ✅ ✅
MongoDB ✅ ✅ ✅ ✅ ✅
SQLite ✅ ✅ ✅ ✅ ✅

Architecture Overview

graph LR
    A[CLI Interface] --> B[Argument Parser]
    B --> C{Database Type}
    C --> D[MySQL Connector]
    C --> E[PostgreSQL Connector]
    C --> F[MongoDB Connector]
    C --> G[SQLite Connector]
    D --> H[Backup / Restore Engine]
    E --> H
    F --> H
    G --> H
    H --> I{Storage}
    I --> J[Local]
    I --> K[S3]
    H --> L[Compression]
    H --> M[Manifest Generation]
    H --> N[Logging + Notifications]
Loading

Installation

1) Clone the repository

git clone https://github.com/RajeshBasnet-dev/Database_Backup_Utility_CLI.git
cd Database_Backup_Utility_CLI

2) Install dependencies

pip install -r requirements.txt

3) Install required database client tools

  • MySQL: mysqldump, mysql
  • PostgreSQL: pg_dump, pg_restore, psql
  • MongoDB: mongodump, mongorestore, mongosh (or mongo)
  • SQLite: bundled with Python (sqlite3 module)

Quick Start

Backup (MySQL)

backupforge backup \
  --db-type mysql \
  --host localhost \
  --port 3306 \
  --username root \
  --password root \
  --database appdb

Restore (PostgreSQL)

backupforge restore \
  --db-type postgres \
  --host localhost \
  --username postgres \
  --password postgres \
  --database appdb \
  --file backup.dump

Backup + Integrity Manifest (SQLite)

backupforge backup --db-type sqlite --database /path/to/app.sqlite --manifest

Command Reference

backup

Create a backup and optionally compress and store it.

backupforge backup --db-type <mysql|postgres|mongodb|sqlite> [options]

Common options:

  • --compress zip|gzip
  • --storage local|s3
  • --manifest (generate SHA-256 sidecar manifest)

restore

Restore a database from a backup artifact.

backupforge restore --db-type <...> --file <backup-file> [options]

test-connection

Validate database connection parameters.

backupforge test-connection --db-type <...> [options]

schedule

Schedule recurring backups.

backupforge schedule --db-type <...> --interval 60 [options]

list

List existing backups from local path or S3.

backupforge list --storage local --path ./backups

verify

Verify backup file integrity against its manifest.

backupforge verify --file ./backups/my_backup.sql.gz

If manifest path is custom:

backupforge verify --file ./backups/my_backup.sql.gz --manifest-file ./backups/custom.manifest.json

Integrity Workflow (Recommended)

  1. Create backup with manifest 
    backupforge backup --db-type sqlite --database /path/to/app.sqlite --manifest
  2. Store/transmit backup artifact (local or S3).
  3. Verify integrity before restore 
    backupforge verify --file ./backups/app_backup.db
  4. Restore only after successful verification.

This workflow reduces risk from accidental corruption and unauthorized tampering.

Configuration

Slack notifications

Set webhook URL:

export SLACK_WEBHOOK_URL="https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK"

AWS credentials for S3

You can provide credentials via:

  1. CLI flags (--access-key, --secret-key)
  2. Environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
  3. Standard AWS credential/config files

Examples

PostgreSQL backup to S3 with compression

backupforge backup \
  --db-type postgres \
  --host localhost \
  --username user \
  --password pass \
  --database mydb \
  --compress gzip \
  --storage s3 \
  --bucket my-backups \
  --region us-east-1 \
  --access-key YOUR_ACCESS_KEY \
  --secret-key YOUR_SECRET_KEY \
  --manifest

MongoDB backup (local)

backupforge backup \
  --db-type mongodb \
  --host localhost \
  --port 27017 \
  --username admin \
  --password admin \
  --database analytics

List local backups

backupforge list --storage local --path ./backups

Operational Notes

  • Verify required database client binaries are installed and available in PATH.
  • For production usage, prefer:
    • storing backups in immutable/object storage,
    • enabling manifests (--manifest),
    • verifying backups before restore,
    • rotating and monitoring backup jobs.
  • Treat credentials securely (avoid exposing secrets in shell history when possible).

Roadmap Ideas

Potential next steps to elevate BackupForge further:

  • End-to-end encryption for backup artifacts
  • Signed manifests (e.g., HMAC or asymmetric signing)
  • Policy-driven retention/pruning
  • Health dashboards and periodic restore drills
  • Pluggable storage targets beyond S3