Synora

A mirror synchronization engine written in Rust. Synora decides when to sync, where to sync, through which network path, with which tool, and how to recover; the actual data movement is performed by rsync, scripts, or Docker containers.

Rust · Mirror Sync · Cron · Multi Worker · Docker · Proxy · IPv4/IPv6 · ZFS/Btrfs · Prometheus · TUI

Acknowledgements

Synora builds on the ideas and production experience of tunasync and tunasync-scripts (TUNA), Yuki (USTC LUG), and tsumugu: tunasync's Manager + Worker architecture, rsync argument conventions (success exit codes 23/24, --safe-links --timeout=120), mirror_subdir, tunasync.json-compatible status output and its optional-TLS wire mode; Yuki's SQLite storage and reload convention; tsumugu's fault-tolerant HTTP directory mirroring. The scripts in synora-scripts/ started as tunasync-scripts. Thank you to TUNA and to the authors and maintainers of these projects.

Features

• No-drift scheduling: cron / daily / weekly / fixed interval with a persistent anchor — next runs are computed from the wall clock, never from "last run end + interval" (misfire policies: skip / run-immediately / run-next).

• Six providers: rsync (tunasync-aligned defaults, success_exit_codes [24]; exit 23 fails unless explicitly enabled), two-stage-rsync (tunasync two-pass: a fast stage-1 subset by profile, then the full sync), script (SYNORA_* env, SYNORA_SIZE= size reporting; workers always run these in synora-scripts), docker (docker run, storage mounted at /data, optional in-container command), git (clone --mirror + remote update --prune, same synora-scripts image on workers), and HTTP directory mirroring (tsumugu-style: downloaded-file 404/410 responses complete with warnings; other transfer/listing errors fail the run, local symlinks left alone, listing-marked symlinks mirrored as local links, configurable download concurrency, 30 s connect / 120 s idle-read timeout, unlimited run time unless a timeout is set with 1m/1h/1d units).

  Git mirrors allow 6 hours for bootstrap and 1 hour for incremental fetches; override with git -C <mirror> config synora.syncTimeout 12h. Empty mirrors can opt into a direct static-pack seed using git -C <mirror> config synora.bootstrapUrl https://trusted-mirror/repo.git/. Seed packs are checksum-checked and verified with git fsck; the official remote must still finish successfully. Completed packs survive retries; partial transfers resume only when the HTTP server honors Range requests.

• Single machine or distributed: synora start runs standalone (SQLite); synora-manager + N × synora-worker form a pull-model cluster (workers register, heartbeat every 15 s, claim assigned runs). PostgreSQL optional.

• Crash safety: every run has a 60 s lease; lease expiry → LOST → a scheduled-dispatch hold (default on_worker_lost = "fail"; retry is opt-in). Claims issue fencing tokens and workers cancel on lease loss. Shared-storage/container takeover also needs infrastructure fencing; see the configuration reference before enabling automatic recovery.

• Hot reload: manager API / SIGHUP (/run/synora/*.pid) / POST /api/v1/reload — job and schedule changes apply live; invalid or non-reloadable changes are rejected as a whole; changed jobs get a catch-up run queued automatically.

• Delete/size protection: max_delete_files / max_delete_ratio / max_size_drop_ratio measured around every run; a mirror that shrinks too much fails instead of being kept (rsync also gets --max-delete=N).

• Proxies & egress: http / socks5h / command / direct with latency and egress-IP probing (default Cloudflare egress), per-job direct sync with ipv4/ipv6 and bind-address selection; TUI auto-registers CF One / WARP.

• cgroup v2 limits per run (memory.max / cpu.max) plus docker resource flags.

• tunasync.json compatibility: the manager serves TUNA/mirror-web tunasync.json (bare array, success/syncing/failed/paused, last-success last_update, du -h sizes like 1.6T) alongside its native synora.json. Internal states such as queued stay in synora.json.

• Security: Bearer-token API with RBAC permission keys, plain HTTP or TLS/mTLS (tunasync-style [api.tls] + worker ca_cert), constant-time token comparison. The rsync/docker/git providers execute argv arrays (no shell); the script provider runs its command through a shell by design (tunasync-scripts compatibility) — its command string comes from trusted local config only, never from API input.

• Observability: Prometheus metrics, per-run log files, events table, TUI. Run logs use <job>_YYYY-MM-DD_HH_MM.log in UTC; repeated starts in the same minute add _2, _3, etc. current.log points to the latest run. The TUI displays logs chronologically and follows the tail; PgUp/PgDn browse history.

• Config: TOML with include (glob/nested/cycle-detected), ${VAR} expansion, file:line validation via synora check.

• Migration: scripts/tunasync2synora.py and scripts/yuki2synora.py convert existing tunasync / Yuki configs in one shot.

中文说明 · 配置逐项解析 · 生产配置示例 · systemd units

Quick start

# validate a config (file:line errors)
export SYNORA_API_TOKEN=$(openssl rand -hex 32)   # placeholder tokens are rejected
synora check -c examples/simple.toml

# run the standalone daemon (SQLite + scheduler + /metrics on 127.0.0.1:8100)
synora start -c examples/simple.toml

# trigger one job now
synora run ubuntu -c examples/simple.toml

# status / logs / cancel / reload
synora status -c examples/simple.toml
synora logs ubuntu -c examples/simple.toml
synora stop slow -c examples/simple.toml
synora reload -c examples/simple.toml

# distributed mode
synora-manager -c config/synora.toml     # manager (API + scheduler)
synora-worker  -c config/worker1.toml    # one worker per host
synora worker list -c config/synora.toml
synora tui -c config/synora.toml         # terminal console

Configuration

include = ["jobs/*.toml"]        # globs, nesting, cycle detection

[daemon]
max_concurrency = 16
log_dir = "/var/log/synora"

[daemon.db]
kind = "sqlite"                  # or "postgres" + url = "postgres://..."
path = "data/synora.db"

[api]
listen = "127.0.0.1:8100"

[api.tls]                        # optional; client_ca enables mTLS
cert = "/etc/synora/cert.pem"
key  = "/etc/synora/key.pem"
# client_ca = "/etc/synora/client-ca.pem"

[[api.tokens]]
name = "admin"
token = "${SYNORA_API_TOKEN}"    # ${VAR} from the environment; >= 32 bytes
role = "admin"                   # admin | operator | viewer

A job (jobs/*.toml):

[[jobs]]
name = "ubuntu"
enabled = true
schedule = "cron"                # cron | daily | weekly | interval | manual | startup
cron = "0 */6 * * *"
timezone = "Asia/Shanghai"

provider = "rsync"               # rsync | two-stage-rsync | http | git | docker | script
upstream = "rsync://archive.ubuntu.com/ubuntu/"
storage = "/srv/mirror/ubuntu"
options = ["--delete", "--delay-updates"]
success_exit_codes = [24]       # 0 always succeeds; opt into [23,24] only for tunasync compatibility
exclude = ["*.tmp"]             # rsync --exclude=PATTERN entries (tunasync `exclude`)

retry = 3
retry_delay = "5m"
timeout = "2h"
worker = "g1"                    # worker id, group label, or omitted (auto)
resources = ["large-disk"]       # matched against worker labels
statistics = "provider"          # provider | filesystem

[jobs.hooks]
on_success = ["/opt/scripts/publish.sh"]

[jobs.safety]
max_delete_files = 50000
max_delete_ratio = 0.30

Worker (worker1.toml):

[worker]
name = "worker-a"               # optional: friendly id (defaults to the token name)
manager = "https://synora.example.org:8100"
token = "worker-token"
labels = ["g1", "zfs"]
ca_cert = "/etc/synora/ca.pem"   # optional, verifies the manager's TLS
max_concurrency = 8
scripts_image = "ghcr.io/palvef/synora-scripts:latest"  # git/script runtime (always Docker on workers)

REST API

All endpoints under /api/v1, Authorization: Bearer <token>. Roles: admin / operator / viewer; permission keys: jobs.read, jobs.write, runs.manage, workers.read, workers.write, logs.read.

Method	Path	Permission	Purpose
POST	/workers/register	runs.manage	worker registration → worker_id
POST	/workers/{id}/heartbeat	runs.manage	heartbeat + lease refresh; returns run assignment / cancel request
POST	/runs/{id}/claim	runs.manage	atomic claim (409 if taken)
POST	/runs/{id}/complete	runs.manage	success / success_with_warnings / failed / cancelled report
POST	/workers/{id}/retire (alias /drain)	workers.write	stop accepting new runs
DELETE	/workers/{id}	workers.write	unregister (only when idle)
GET	/jobs	jobs.read	jobs with status/next_run/size
POST	/jobs/{name}/run	jobs.write	trigger a run now
POST	/jobs/{name}/stop	jobs.write	cancel (worker cancels on next heartbeat)
GET	/jobs/{name}/history	jobs.read	run history
GET	/jobs/{name}/logs?tail=200	logs.read	tail of current.log
GET	/workers	workers.read	worker list
POST	/reload	config.reload	hot-reload config
GET	/metrics	metrics.read	Prometheus text format

/metrics requires a Bearer token with metrics.read by default; explicitly set [api] metrics_auth = false for a trusted private scraper. /healthz and mirror-web status JSON remain public. Empty JSON paths disable those endpoints. Reload requires config.reload, granted to admins by default.

Metrics

synora_job_status{job,worker} (gauge: 0 pending, 4 running, 5 success, 6 failed, 9 cancelled, 10 lost, 12 completed with warnings), synora_job_runs_total, failures_total, retries_total, duration_seconds, last_success/start/end_timestamp, next_run_timestamp, bytes_transferred_total, repository_size_bytes, synora_worker_status, synora_worker_jobs_running.

Repository size

Priority: provider report (--stats / SYNORA_SIZE=) → script output → filesystem walk when statistics = "filesystem". Stored as raw bytes + human form (KiB/MiB/GiB/TiB).

Design principles

Scheduler ↔ Executor decoupled · Provider ↔ Storage decoupled · Manager ↔ Worker decoupled · DB is the source of truth · all states observable. Proxy/egress, snapshots, HTTP, Manager/Worker and TUI are implemented. ZFS/Btrfs rollback and cross-host shared-storage takeover require deployment validation; HA multi-manager and a general Provider SDK are not production claims. See the configuration reference for new defaults and Manager/Worker upgrade requirements.

Development

scripts/ci.sh               # format, versions, Clippy, tests and build
synora check -c examples/simple.toml

Rust is pinned in rust-toolchain.toml. Set SYNORA_TEST_PG_URL to an empty, disposable PostgreSQL database to include the concurrency integration test locally; GitHub CI always runs it with PostgreSQL 17. CI checks every branch push and PR.

Workspace crates: core (domain types, no-drift schedule math, metrics), config (TOML loader/validator), db (SQLite/PostgreSQL), provider (rsync/script/docker/git/http), engine (scheduler/executor), api (DTOs + client), netroute (proxy probing/serving), manager, worker, cli, tui, nginx (directory-index parser), httpfetch.

License

GPL-3.0

Repository cleanup and package filtering:

• HTTP delete = true removes files and empty directories absent from a complete upstream listing. Failed or incomplete listings and failed transfers suppress cleanup. exclude accepts path globs and protects matching local content; changing an exclusion does not itself purge existing files.
• APT/YUM helpers exclude debug symbols and test packages by default (dbg, debug, dbgsym, debuginfo, debugsource, test, tests, testing, testsuite as package-name segments). Runtime and development packages remain included. SYNC_EXCLUDE_PACKAGES adds package-name globs. APT retains upstream signed indexes; explicitly excluded packages are intentionally unavailable. YUM regenerates metadata for the filtered package set.
• Rsync jobs can share synora-scripts/helpers/package-filters.rules via --filter "merge /path/to/package-filters.rules". Its receiver rules remove existing excluded packages without deleting unrelated protected files. The ftpsync image applies the same package patterns.
• APT only removes retired suites after both upstream Release and InRelease return 404/410; metadata/network failures stop cleanup. Component Release files no longer disable package garbage collection. MongoDB recovery prunes obsolete RPMs only after complete object discovery and successful downloads.

HTTP warn_on_forbidden_files = true makes ordinary file 403 responses warnings (disabled by default). Missing or forbidden repository metadata, failed directory listings, checksum errors, and server/transport failures remain fatal; any warning suppresses destructive cleanup. Python production excludes **/*.rekor and prerelease paths separately.