Repository navigation
Let sync apps send files in pieces over WebDAV - #22
Merged
Merged
Conversation
Behind a proxy that limits the size of requests, such as Cloudflare's 100 MB, sync apps could not upload larger files: WebDAV sends each file in one request. WebDAV now also takes resumable uploads (tus, as the browser sends them) that a POST to the folder starts, as ownCloud Infinite Scale does, and rclone with vendor=infinitescale sends every file that way, in 10 MiB pieces. - POST to a folder under /.dav/ with Tus-Resumable starts an upload; it replaces a file of its name, as a PUT does, unless If-None-Match: *. - /.uploads/ takes the app password that the upload was started with. - The creation's Location is absolute, as rclone needs it. - mtime in Upload-Metadata becomes the file's modification time, kept with the upload (migration 008); the browser sends it too, so its uploads keep their times now. - OPTIONS on /.dav/ names the tus version and extensions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cloudflare's 100 MB request limit meant no sync app could upload anything larger to ftp.ssenerg.com: plain WebDAV sends each file in one request. This adds a chunked upload path over WebDAV.
How it works
goftp already speaks tus for browser uploads, and rclone has a tus mode for WebDAV (
vendor=infinitescale, the way ownCloud Infinite Scale takes uploads). I used that rather than Nextcloud's chunking: rclone only does Nextcloud chunking at a URL containing/dav/files/USER, which doesn't fit goftp's hidden/.dav/prefix. This also reuses the existing resumable-upload code, with its quotas, locks, claim leases and expiry.POSTto a folder under/.dav/withTus-Resumablestarts an upload. It replaces a file of the same name, as a PUT does, unlessIf-None-Match: *is sent./.uploads/(HEAD/PATCH/DELETE) now accepts the app password the upload was started with. Another user's app password gets 404, a wrong one 401.Locationis absolute, because rclone builds its PATCH URLs from it. The browser already resolved it as a URL.mtimeinUpload-Metadata(seconds since 1970) becomes the file's modification time. It is kept with the upload in a newuploads.mtimecolumn (migration008_upload_mtime.sql). The browser now sends it too, so web uploads keep their file times.OPTIONS /.dav/lists POST inAllowand names the tus version and extensions.vendor=infinitescalefor rclone.vendor=owncloudstill works, one request per file.Testing
TestDavResumablecovers:If-None-Match: *refusing to (409)mtimeTestUploadschecks the upload's mtime round trip on both the memory and Postgres stores.go test -race ./...with Postgres passes on macOS, exceptTestLongNonASCIIPath, which fails only on macOS (on main too). Everything passes on Linux ingolang:1.27.3-alpine.vendor=infinitescale) against a local server:bisync --resyncof a 25 MB file, a small file and an empty file: the 25 MB file went up as three 10 MiB PATCHes, and no request carried a whole file.checkfound 4 matching files and the server logged no errors.Not covered
Apps other than rclone (FolderSync, GoodSync, Mountain Duck) still send each file in one request, so the 100 MB limit still applies to them.
🤖 Generated with Claude Code