fix: SFTP sync failures from restricted-shell md5sum and checksum fallback#413
fix: SFTP sync failures from restricted-shell md5sum and checksum fallback#4137MS8 wants to merge 1 commit into
Conversation
…lback On SFTP remotes with restricted shells (e.g. Synology NAS), rclone's --checksum option makes it call md5sum/sha1sum over SSH to verify transfers. Restricted shells often can't see paths that are only exposed via SFTP, so every transfer failed with "corrupted on transfer: md5 hashes differ" / "failed to run md5sum: No such file or directory". Disable the remote hash commands for SFTP remotes (--sftp-md5sum-command/--sftp-sha1sum-command none) so rclone doesn't attempt them. Disabling both, however, leaves rclone with no common hash type when --checksum is still requested; it silently falls back to a size-only comparison and ignores modification time, so a changed file that keeps the same size never gets re-synced. Skip --checksum entirely for SFTP remotes so rclone falls back to its default size+modtime comparison instead. Verified against a live Synology SFTP remote and a physical Pixel 7a (GrapheneOS): transfers that previously failed with the md5sum error now succeed, and a same-size/changed-content file is correctly detected via modtime and re-transferred instead of being silently skipped. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Following up on my concern in #406: I don’t think this should be merged as-is. It silently ignores a user-visible setting and disables checksums for all SFTP remotes, including those where they work correctly. Given that the project is currently in some kind of a maintenance mode, my suggestion would be to keep the fix as small and narrowly scoped as possible: preserve the existing checksum behavior, clearly expose the actual rclone error, and optionally add a short hint suggesting More advanced solutions, such as better capability detection or UI changes, could be added to the wishlist for the planned rewrite. That would minimize the risk of regressions in the current codebase. The final approach is, of course, up to the maintainer. |
Split out of #406 per reviewer request (separate unrelated fixes into their own PRs).
Bug
On SFTP remotes with restricted shells (e.g. Synology NAS), when the task has "use MD5 checksum" enabled, rclone's
--checksumoption makes it callmd5sum/sha1sumover SSH to verify transfers. A restricted shell often can't see paths that are only exposed via SFTP (not a general SSH login), so every transfer fails with:Fix (two parts, kept together because the first alone regresses)
--sftp-md5sum-command none,--sftp-sha1sum-command none) so rclone doesn't attempt them over SSH at all.--checksumis still requested — it silently falls back to a size-only comparison and ignores modification time entirely, so a changed file that happens to keep the same byte size is never re-synced (--checksum is in use but the source and destination have no hashes in common; falling back to --size-onlyinsync.log). Skip--checksumentirely for SFTP remotes so rclone uses its default size+modtime comparison instead.Why this doesn't touch every SFTP remote's ability to verify by hash (open question raised in review)
This trades hash-based transfer verification for size+modtime comparison, for every SFTP remote, not just restricted-shell ones — this was raised as a concern in review of the original bundled PR (#406). A per-remote opt-out toggle (only disable remote hash commands for remotes that actually need it) would be a better long-term fix; that's intentionally left out of this PR to keep it minimal and is a candidate for a focused follow-up if wanted.
Reproduction
md5sum ... No such file or directory/corrupted on transfererrors above.sync.logshows thefalling back to --size-onlymessage), the change is never picked up.sync.log:Copied (replaced existing)).Test plan
./gradlew :app:compileOssDebugSources).