Skip to content

Let sync apps send files in pieces over WebDAV - #22

Merged
ssenerg merged 1 commit into
mainfrom
chunked-dav
Oct 25, 2026
Merged

ssenerg merged 1 commit into
mainfrom
chunked-dav

Conversation

@ssenerg

@ssenerg ssenerg commented Oct 25, 2026

Copy link
Copy Markdown
Owner

Cloudflare's 100 MB request limit meant no sync app could upload anything larger to ftp.ssenerg.com: plain WebDAV sends each file in one request. This adds a chunked upload path over WebDAV.

How it works

goftp already speaks tus for browser uploads, and rclone has a tus mode for WebDAV (vendor=infinitescale, the way ownCloud Infinite Scale takes uploads). I used that rather than Nextcloud's chunking: rclone only does Nextcloud chunking at a URL containing /dav/files/USER, which doesn't fit goftp's hidden /.dav/ prefix. This also reuses the existing resumable-upload code, with its quotas, locks, claim leases and expiry.

  • POST to a folder under /.dav/ with Tus-Resumable starts an upload. It replaces a file of the same name, as a PUT does, unless If-None-Match: * is sent.
  • /.uploads/ (HEAD/PATCH/DELETE) now accepts the app password the upload was started with. Another user's app password gets 404, a wrong one 401.
  • The creation's Location is absolute, because rclone builds its PATCH URLs from it. The browser already resolved it as a URL.
  • mtime in Upload-Metadata (seconds since 1970) becomes the file's modification time. It is kept with the upload in a new uploads.mtime column (migration 008_upload_mtime.sql). The browser now sends it too, so web uploads keep their file times.
  • OPTIONS /.dav/ lists POST in Allow and names the tus version and extensions.
  • The Sync page and README now recommend vendor=infinitescale for rclone. vendor=owncloud still works, one request per file.

Testing

  • TestDavResumable covers:
    • OPTIONS
    • creating an upload with an absolute Location
    • pieces with the right app password; another user's gets 404, a wrong one 401
    • HEAD, completion, content and mtime
    • replacing an existing file, and If-None-Match: * refusing to (409)
    • the home quota (507), hidden folders (403), another user's home
    • POST without tus (405)
    • the site's resumable uploads keeping mtime
  • TestUploads checks the upload's mtime round trip on both the memory and Postgres stores.
  • go test -race ./... with Postgres passes on macOS, except TestLongNonASCIIPath, which fails only on macOS (on main too). Everything passes on Linux in golang:1.27.3-alpine.
  • End to end with rclone v1.75.2 (vendor=infinitescale) against a local server:
    • bisync --resync of a 25 MB file, a small file and an empty file: the 25 MB file went up as three 10 MiB PATCHes, and no request carried a whole file.
    • The content hash matches, and the 2020 modification time was kept.
    • Updating existing files (a 12 MB rewrite, an edited small file, a new file) replaced them in place with nothing put in the trash; check found 4 matching files and the server logged no errors.

Not covered

Apps other than rclone (FolderSync, GoodSync, Mountain Duck) still send each file in one request, so the 100 MB limit still applies to them.

🤖 Generated with Claude Code

Behind a proxy that limits the size of requests, such as Cloudflare's
100 MB, sync apps could not upload larger files: WebDAV sends each file
in one request. WebDAV now also takes resumable uploads (tus, as the
browser sends them) that a POST to the folder starts, as ownCloud
Infinite Scale does, and rclone with vendor=infinitescale sends every
file that way, in 10 MiB pieces.

- POST to a folder under /.dav/ with Tus-Resumable starts an upload; it
  replaces a file of its name, as a PUT does, unless If-None-Match: *.
- /.uploads/ takes the app password that the upload was started with.
- The creation's Location is absolute, as rclone needs it.
- mtime in Upload-Metadata becomes the file's modification time, kept
  with the upload (migration 008); the browser sends it too, so its
  uploads keep their times now.
- OPTIONS on /.dav/ names the tus version and extensions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ssenerg
ssenerg merged commit d0f986b into main Oct 25, 2026
4 checks passed
@ssenerg
ssenerg deleted the chunked-dav branch October 25, 2026 14:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant