Skip to content

index/sync: live progress output for long-running operations #95

Description

@mbertschler

Problem

squirrel index and squirrel sync run silently and only print a single summary line when they finish. For large volumes that's a long, blind wait:

  • Syncing a large volume over a slow network connection can take hours with no output at all until the final transferred=… bytes=… run=… line. There's no way to tell how far along it is, the current rate, an ETA, or whether it has stalled.
  • index on a large volume hashes many GB with the same silence — no signal until the run completes.

Desired behavior

Live, throttled progress while the operation runs.

sync

  • bytes transferred / total, transfer rate, ETA, files done/total
  • rclone already computes all of this (its stats / --progress), so this is largely a matter of surfacing rclone's stats stream instead of only reporting the final tally
  • expose via a --progress / -P flag, or auto-enable when stdout is a TTY and stay quiet otherwise

index

  • files hashed / total (and/or bytes hashed), current path, rate
  • emitted periodically (e.g. every N seconds) so it doesn't flood logs

Notes

  • Keep it TTY-aware: the agent and scripted/cron runs should stay quiet (the current single summary line is right for logs and runs). Progress output is for interactive use.
  • The existing summary line (status=… transferred=… checked=… errors=… run=…) should remain — this adds live feedback, it doesn't replace the final report.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions