now uses kopia sync-to for remote repo copies (instead of rclone directly).

Author: rusty-art

.gitignore

@@ -2,8 +2,8 @@
 .env
 .env.local
 
-# User config that might store with passwords
-kopia-configs.yaml
+# User config that might contain passwords
+kopia-helpers.yaml
 
 # Python
 __pycache__/

README.md

@@ -30,21 +30,21 @@ If you only have one backup destination, just use Kopia directly. If you follow
 ## Setup
 
 1. Install [Kopia](https://kopia.io/docs/installation/)
-2. Copy `kopia-configs.template.yaml` to `kopia-configs.yaml`
-3. Edit `kopia-configs.yaml` with your repository paths and sources
+2. Copy `kopia-helpers.template.yaml` to `kopia-helpers.yaml`
+3. Edit `kopia-helpers.yaml` with your repository paths and sources
 4. Set your password (see below)
 5. Run `python kopia-start-backups.py` as Administrator
 6. Run `python kopia-health-check.py --register` to enable backup monitoring
 
-Your `kopia-configs.yaml` is re-read each time the scheduled task runs, so config changes take effect automatically. You can also safely re-run steps 5 and 6 anytime to re-register the tasks (but not required). Uou will receive a Windows Toast Notification if there are backup failures or lack of backup activity. 
+Your `kopia-helpers.yaml` is re-read each time the scheduled task runs, so config changes take effect automatically. You can also safely re-run steps 5 and 6 anytime to re-register the tasks (but not required). Uou will receive a Windows Toast Notification if there are backup failures or lack of backup activity. 
 
 ## Password Configuration
 
 Passwords can be set in multiple ways (checked in this order):
 
 ### Option 1: In config file (simplest)
 ```yaml
-# kopia-configs.yaml
+# kopia-helpers.yaml
 repositories:
   - name: my-backup
     password: your-password-here
@@ -65,7 +65,7 @@ set KOPIA_PASSWORD=your-password
 # .env.local
 KOPIA_PASSWORD_MY_BACKUP1=your-password
 ```
-**Note:** The variable name is based on the repository `name` in your `kopia-configs.yaml`, converted to uppercase with dashes replaced by underscores.
+**Note:** The variable name is based on the repository `name` in your `kopia-helpers.yaml`, converted to uppercase with dashes replaced by underscores.
 Example: `name: my-backup1` → `KOPIA_PASSWORD_MY_BACKUP1`
 
 ## Scheduled Backups
@@ -75,7 +75,7 @@ Run as Administrator to auto-register with Windows Task Scheduler:
 python kopia-start-backups.py
 ```
 
-This creates a task that runs every 15 minutes (configurable in kopia-configs.yaml).
+This creates a task that runs every 15 minutes (configurable in kopia-helpers.yaml).
 
 ## Health Check
 
@@ -84,58 +84,141 @@ Register the health check to alert if backups stop:
 python kopia-health-check.py --register
 ```
 
-This checks every 3 hours and shows a toast notification if no backups for 7 days (configurable in kopia-configs.yaml).
+This checks every 3 hours and shows a toast notification if no backups for 7 days (configurable in kopia-helpers.yaml).
 
-## Cloud Backup (OneDrive, Google Drive, etc.)
+## Syncing to Cloud / Remote Storage
 
-For cloud destinations, these scripts use [rclone](https://rclone.org/) to sync your local Kopia repository to the cloud. This is more reliable than the OneDrive desktop client for automated backups (which warns about bulk deletes, has character encoding issues, etc.).
+After Kopia creates local snapshots, you can sync them to cloud storage or other destinations using Kopia's built-in `repository sync-to` command. This properly handles repository structure (sharding) and supports multiple destinations per repository.
 
 ### How it works
 
-1. Kopia writes snapshots to a local directory repo as per normal
-2. After backup completes, rclone syncs the kopia repo to your cloud destination
-3. The cloud provider's desktop client can sync back for fast local restores
+1. Kopia writes snapshots to a local directory repository
+2. After backup completes, `kopia repository sync-to` syncs to each configured destination
+3. Each destination can have its own sync interval
 
-### Setup
+### Supported backends
+
+| Backend | Description | Setup Required |
+|---------|-------------|----------------|
+| `rclone` | OneDrive, Dropbox, and [50+ providers](https://rclone.org/overview/) | [rclone](https://rclone.org/downloads/) installed + configured |
+| `s3` | Amazon S3 and S3-compatible storage | Access keys |
+| `gcs` | Google Cloud Storage | GCP credentials |
+| `azure` | Azure Blob Storage | Storage account + key |
+| `b2` | Backblaze B2 | Application key |
+| `gdrive` | Google Drive (native) | OAuth credentials |
+| `filesystem` | Local/network paths | None |
+| `sftp` | SSH/SFTP servers | SSH access |
+| `webdav` | WebDAV servers | URL + credentials |
+
+### Configuration
+
+Add a `sync-to` list to your repository in `kopia-helpers.yaml`:
+
+```yaml
+repositories:
+  - name: my-backup
+    repo_destination: C:/kopia-cache/mysrc
+    repo_config: C:/kopia-cache/mysrc/repository.config
+    repo_password: your-password
+
+    sources:
+      - C:/Users/username/Documents
+
+    policies:
+      # ... retention policies ...
+
+    # Sync to one or more destinations
+    sync-to:
+      # OneDrive via rclone
+      - type: rclone
+        remote-path: onedrive:mybackups/kopia
+        interval: 60m
+
+      # Local NAS
+      - type: filesystem
+        path: //nas/backups/kopia
+        interval: 30m
+```
+
+### Default flags
+
+These flags are always applied (you can override via `extra-args`):
+- `--delete` - Mirror behavior (remove files not in source)
+- `--flat` - Flat directory structure for cloud backends
+
+### Using rclone backend (OneDrive, Dropbox, etc.)
 
 1. Install rclone: https://rclone.org/downloads/
-2. Configure your cloud remote:
+2. Configure your remote:
    ```bash
    rclone config
-   # Choose 'n' for new remote
-   # Name it: onedrive (or gdrive, dropbox, etc.)
-   # Follow the browser auth flow
-   ```
-3. Test connectivity:
-   ```bash
-   rclone lsd onedrive:
+   # Choose 'n' for new remote, name it 'onedrive', follow auth flow
    ```
-4. Add a cloud repository to `kopia-configs.yaml`:
+3. Test: `rclone lsd onedrive:`
+4. Add to config:
    ```yaml
-   repositories:
-     - name: cloud-backup
-       local_destination_repo: C:/kopia-cache/mysrc               # Where Kopia writes locally
-       remote_destination_repo: onedrive:mybackups/kopia          # rclone sync destination
-       local_config_file_path: C:/kopia-cache/mysrc/repository.config
-       password: your-password
-       sources:
-         - C:/Users/username/Documents
-         - ...
-       policies:
-         # ... same as local repos
+   sync-to:
+     - type: rclone
+       remote-path: onedrive:mybackups/kopia
+       interval: 60m
    ```
 
-Kopia writes to `local_destination_repo`, then rclone syncs to `remote_destination_repo` (e.g., `onedrive:`, `gdrive:`, `dropbox:`).
+### Using S3 backend
 
-### Supported cloud providers
+```yaml
+sync-to:
+  - type: s3
+    bucket: my-backup-bucket
+    interval: 120m
+    extra-args: ["--access-key=AKIAXXXXXXXX", "--secret-access-key=xxx"]
+```
+
+### YAML parameters vs extra-args
+
+Each backend has a few **required YAML fields** (the destination identifier). Everything else goes in `extra-args`:
+
+| Backend | Required YAML fields | Example extra-args |
+|---------|---------------------|-------------------|
+| `rclone` | `remote-path` | `--rclone-args=...` |
+| `s3` | `bucket` | `--access-key=...`, `--region=...` |
+| `gcs` | `bucket` | `--credentials-file=...` |
+| `azure` | `container`, `storage-account` | `--storage-key=...` |
+| `b2` | `bucket` | `--key-id=...`, `--key=...` |
+| `gdrive` | `folder-id` | `--credentials-file=...` |
+| `filesystem` | `path` | |
+| `sftp` | `path`, `host`, `username` | `--keyfile=...`, `--password=...` |
+| `webdav` | `url` | `--username=...`, `--password=...` |
+
+**Common fields for all backends:**
+- `interval` - how often to sync (e.g., `60m`, `2h`)
+- `extra-args` - list of additional kopia flags
 
-Any [rclone-supported backend](https://rclone.org/overview/) works:
-- OneDrive / OneDrive for Business
-- Google Drive
-- Dropbox
-- Amazon S3
-- Backblaze B2
-- And many more...
+### Using extra-args
+
+```yaml
+sync-to:
+  - type: rclone
+    remote-path: onedrive:mybackups/kopia
+    interval: 60m
+    extra-args: ["--no-delete", "--times"]
+
+  - type: s3
+    bucket: my-bucket
+    interval: 120m
+    extra-args: ["--access-key=AKIA...", "--secret-access-key=...", "--region=us-west-2"]
+
+  - type: sftp
+    path: /backups/kopia
+    host: backup.example.com
+    username: backupuser
+    interval: 30m
+    extra-args: ["--keyfile=/path/to/key"]
+```
+
+To see all available options for a backend:
+```bash
+kopia repository sync-to <type> --help
+```
 
 ## Finding Files
 
@@ -174,7 +257,7 @@ All scripts support:
 - Python 3.8+
 - PyYAML: `pip install pyyaml`
 - python-dotenv (optional): `pip install python-dotenv`
-- rclone (optional, for cloud backup): https://rclone.org/downloads/
+- rclone (for OneDrive/Dropbox/etc sync): https://rclone.org/downloads/
 
 ## License
 

kopia-find-files.py

@@ -30,7 +30,7 @@
     python kopia-find-files.py --mount Z: --repo myrepo   # Mount for browsing
 
 Requires:
-    - kopia-configs.yaml with repository definitions
+    - kopia-helpers.yaml with repository definitions
     - Password configured via yaml, environment variable, or .env file
     - kopia CLI installed and in PATH
     - WinFsp for mount functionality (Windows)
@@ -100,7 +100,7 @@ def parse_ls_line(line: str) -> Optional[Tuple[int, str, str]]:
 def find_in_repo(runner: utils.KopiaRunner, repo_config: Dict[str, Any], filename_pattern: str, max_snapshots: Optional[int] = None, match_path: bool = False, verbose: bool = False) -> List[Dict[str, Any]]:
     """Search for files using kopia ls -l -r (gets file details directly, no mounting)."""
     name = repo_config['name']
-    config_file = repo_config['local_config_file_path']
+    config_file = repo_config['repo_config']
 
     print(f"  Searching in {name}...")
 
@@ -193,7 +193,7 @@ def find_in_repo(runner: utils.KopiaRunner, repo_config: Dict[str, Any], filenam
 
 def mount_repository(runner: utils.KopiaRunner, repo_config: Dict[str, Any], drive_letter: str, verbose: bool = False):
     name = repo_config['name']
-    config_file = repo_config['local_config_file_path']
+    config_file = repo_config['repo_config']
 
     print(f"\nPreparing to mount '{name}' to {drive_letter}...")
 
@@ -278,7 +278,7 @@ def find_free_drive_letter() -> Optional[str]:
 def list_repositories(config: Dict[str, Any]) -> List[Dict[str, Any]]:
     print("\nAvailable Repositories:")
     for i, repo in enumerate(config['repositories']):
-        print(f"[{i}] {repo['name']} ({repo['local_destination_repo']})")
+        print(f"[{i}] {repo['name']} ({repo['repo_destination']})")
     return config['repositories']