Skip to content

khwstolle/git-lfserve

Repository files navigation

Git LFS Client Worker

A Cloudflare Pages middleware that serves Git LFS objects in place of pointer files, overcoming the 25 MiB Pages file-size limit.

Project structure

_middleware.js is a thin entry point over six modules in src/ (handler, pointer, lfsconfig, batch, objects, http), each covered by a node --test suite in test/. Useful commands:

npm test              # run the test suite
npm run lint          # eslint
npm run format        # prettier --write (npm run format:check to verify only)
npm run dev           # serve locally with wrangler pages dev (see wrangler.toml)
npm run check:bundle  # verify the worker bundles (needs ../.lfsconfig; see below)

check:bundle bundles _middleware.js, which imports the .lfsconfig.txt symlink. In a standalone checkout that symlink points outside the repo; create a throwaway target first:

printf '[lfs]\nurl = https://example.invalid/info/lfs\n' > ../.lfsconfig

Configuration

The worker reads your repository's .lfsconfig (via the .lfsconfig.txt symlink) and optional Pages environment bindings.

  • .lfsconfig supplies the upstream LFS server URL. The worker checks, in order, [lfs] url, then [remote "origin"] lfsurl, then any other [remote "..."] lfsurl.
  • LFS_BUCKET + LFS_BUCKET_URL (optional): when both are set, the worker reads objects straight from the bound R2 bucket instead of asking the LFS server for a URL.
  • KEEP_HEADERS (optional, default Cache-Control): a comma-separated list of headers copied from the original page response onto the served object.

If the worker cannot resolve an object, it logs the cause server-side and returns 502 Bad Gateway.

HTTP range requests (e.g., video seeking) work on both paths: the bucket path serves 206 Partial Content directly from R2, and the LFS-server path forwards Range to the object store. Object responses carry Content-Length and Accept-Ranges: bytes.

Usage

Run every command from your repository's root:

cd "$(git rev-parse --show-toplevel)"

Install Git LFS

Confirm Git LFS is installed:

git lfs version

If Git LFS is missing, follow the installation instructions.

Install the smudge and clean filters

The Git LFS binary does not always install the smudge and clean filters. Install them for your user account:

git lfs install

Install the LFS Client Worker

The worker intercepts pointer files the same way the standard Git LFS client does: when Pages is about to serve an LFS pointer, the worker resolves the object it references and serves that object instead. To intercept requests this way, it must run as middleware, so _middleware.js from this repo must become functions/_middleware.js in yours. It also needs a symlink from functions/.lfsconfig.txt to .lfsconfig to read your LFS configuration.

If your site has no functions directory, add this repo as a submodule named functions:

git submodule add https://github.com/khwstolle/git-lfserve functions
git commit -m "Add Git LFS Client Worker"

Disable the smudge and clean filters by default

The worker resolves pointers on the fly, so the standard client must not also do so when Cloudflare Pages clones your repo. (If it does, Pages builds fail even though LFS holds every large file.) Disable the filters in .lfsconfig rather than via environment variables, which do not appear to reach the "Cloning git repository" build step:

git config -f .lfsconfig lfs.fetchexclude '*'
git add .lfsconfig
git commit -m "Disable LFS fetching by default"

After checking out a commit with this change, Git LFS leaves pointers in place. Restore normal behavior in your own clones by overriding lfs.fetchexclude:

git config lfs.fetchexclude ''

Change LFS servers

GitHub and GitLab both run default LFS servers, but neither suits this use case:

  • Storage is limited.
    • GitHub's free tier holds up to 10 GiB across all repositories; more costs $0.07/GiB-month.
    • GitLab's free tier holds up to 5 GB, shared with all other GitLab services; more must be bought in $60/year "packs".
  • Bandwidth is limited.
    • GitHub's free tier serves up to 10 GiB/month across all repositories; more costs $0.0875/GiB.
    • GitLab's free tier serves up to 10 GB/month, shared with all other GitLab services; more must be bought in the same $60/year "packs".
  • Latency is poor.
    • GitHub's LFS server takes ~270 ms to return an object's URL; the object store takes another 20–100 ms.
    • Neither service was built to serve web content.

Consider setting up LFS S3 Proxy backed by R2 instead. On Cloudflare's free tier it serves up to 10 GB with unlimited bandwidth and the lowest possible latency, since your objects sit in the same datacenters as the worker. Beyond 10 GB, storage costs $0.015/GB-month — several times cheaper than GitHub or GitLab.

If you keep a default LFS server, still specify its URL in .lfsconfig: the worker has no other way to find it, because a worker does not know which repo built its Pages site.

Add files to Git LFS

Start using Git LFS. At a minimum, track every file larger than the 25 MiB Pages limit:

find . -type f '!' -path './.*' -size +25M -exec git lfs track {} +
find . -type f '!' -path './.*' -size +25M -exec git add --renormalize {} +
git add .gitattributes
git commit -m "Add files over 25 MiB to Git LFS"

Track each new large file before committing it:

git lfs track bigfile

When a file type consistently exceeds 25 MiB, track it by pattern:

git lfs track '*.mp4'

Create a Pages site

Create a Cloudflare Pages site from your repo, or push to trigger a rebuild:

git push

Optional: bind your R2 bucket to the worker

If you switched to LFS S3 Proxy backed by R2, bind the bucket to your Pages site for better performance:

  • Open the Workers & Pages section of the Cloudflare dashboard.
  • Select your Pages site.
  • Set up LFS_BUCKET:
    • Navigate to Settings > Functions > R2 bucket bindings > Production.
    • Click Add binding.
    • Set Variable name to LFS_BUCKET.
    • For R2 bucket, select the bucket you created for LFS S3 Proxy.
    • Click Save.
  • Set up LFS_BUCKET_URL:
    • Navigate to Settings > Environment variables > Production.
    • Click Add variables.
    • Set Variable name to LFS_BUCKET_URL.
    • Set Value to your LFS server URL without the access key (just https://<INSTANCE>/<ENDPOINT>/<BUCKET>).
  • Redeploy your Pages site:
    • Navigate to Deployments > Production > View details.
    • Click Manage deployment > Retry deployment.

With both variables set, the worker fetches objects straight from the bucket instead of asking LFS S3 Proxy for presigned URLs.

About

Serve large files directory from Git LFS using CloudFlare pages

Topics

Resources

License

Security policy

Stars

Watchers

Forks

Contributors