Livestore-Filesync

File sync for LiveStore apps. This missing piece for local-first apps that need to sync files.

[!] This is still under active development and not yet ready for production. The maintained React/OPFS path and image package are covered by unit and browser E2E tests. The Node example includes an adapter store-creation smoke test. Expo has filesystem/image module contract tests and iOS simulator checks for old-store recovery, image resize and upload. Physical-device and Android behavior are not yet verified. pnpm test includes both maintained React examples in Chromium.

[!] This project currently targets an exact LiveStore main snapshot and Effect 4 RC cohort. The pins in pnpm-workspace.yaml keep local development and CI on one reproducible package graph. These snapshot pins are development inputs, not a published compatibility promise.

Local-first: Files are written to local storage first ensuring best UX and offline support.
Background sync: Files as synced in background by reacting to LiveStore events.
Cross platform support: Web, desktop and mobile support through file system adapters based on the Effect Platform Filesystem interface.
R2 and S3 remote storage: Built in support for Cloudflare R2 and any S3 compatible remote storage service.
Media type support: An optional image package is available for pre-processing helpers and local thumbnail generation. Artchitecture supports extending to other media types.

See the 9 September correctness report for changes, diagrams, validation and remaining limits.

Packages

Package	Description
`@livestore-filesync/core`	Framework-agnostic API and schema helpers
`@livestore-filesync/opfs`	OPFS filesystem adapter for browsers
`@livestore-filesync/expo`	Expo/React Native filesystem and image processing adapters
`@livestore-filesync/r2`	Cloudflare R2 storage handler (Worker-proxied)
`@livestore-filesync/s3-signer`	S3-compatible presigned URL signer (direct-to-storage)
`@livestore-filesync/image`	Image preprocessing and thumbnail generation

Getting Started

Clone the repo and install dependencies:

git clone https://github.com/<org>/livestore-filesync.git
cd livestore-filesync
pnpm install

LiveStore snapshot cohort

This repo follows LiveStore main through exact npm snapshots so installs are reproducible:

LiveStore core, web, React, sync, and development packages use 0.0.0-snapshot-97407c6622c93eb1ae4c02a40c79743426ee101f
Effect packages use 4.0.0-rc.111
@livestore/adapter-node uses published contrib snapshot 0.0.0-snapshot-9fd312cd51d0c38b9e53a78bb4fc211ee4f9c5d2.8aa073fb46b5977e8dbae2be00e8af3b85abbd2f, with transitive core dependencies overridden to the selected core snapshot.

Update the complete LiveStore and Effect catalog cohorts together and run pnpm install whenever intentionally moving to a newer snapshot.

Downstream apps must install the same LiveStore snapshot directly, through their own pnpm catalog, or through equivalent package-manager overrides. This repo forces transitive LiveStore and Effect packages onto the selected cohort to avoid duplicate runtime generations. @livestore-filesync/core and @livestore-filesync/image expose LiveStore as peer dependencies, so consumers remain responsible for resolving a compatible LiveStore version.

Install @livestore/peer-deps from the same snapshot cohort in downstream apps to keep Effect and OpenTelemetry peer versions aligned with LiveStore.

Install

# Web app (React/Vue/etc)
pnpm add @livestore-filesync/core @livestore-filesync/opfs

# Expo/React Native
pnpm add @livestore-filesync/core @livestore-filesync/expo
npx expo install expo-file-system expo-image-manipulator

# Node.js
pnpm add @livestore-filesync/core @effect/platform-node

Quick Start

1. Extend your LiveStore schema

import { createFileSyncSchema } from '@livestore-filesync/core/schema'

const fileSyncSchema = createFileSyncSchema()

export const tables = { ...fileSyncSchema.tables, /* your tables */ }
export const events = { ...fileSyncSchema.events, /* your events */ }

const materializers = State.SQLite.materializers(events, {
  ...fileSyncSchema.createMaterializers(tables)
})

2. Initialize FileSync

import { initFileSync } from '@livestore-filesync/core'
import { layer as opfsLayer } from '@livestore-filesync/opfs'

const dispose = initFileSync(store, {
  fileSystem: opfsLayer(),
  remote: { signerBaseUrl: '/api' }
})

// Later, to clean up:
// await dispose()

For unauthenticated/local-first use, pass remote: false. Files are stored and resolved locally, and FileSync will not call the signer API or enqueue uploads/downloads:

const dispose = initFileSync(store, {
  fileSystem: opfsLayer(),
  remote: false
})

3. Use the API

import { saveFile, resolveFileUrl } from '@livestore-filesync/core'

const result = await saveFile(file)
const url = await resolveFileUrl(result.fileId)

See examples/ for complete implementations:

examples/react-filesync — React example
examples/react-thumbnail — React example with image thumbnails using the canvas processor (no WASM)
examples/node-filesync — Node.js usage
examples/vue-filesync and examples/vue-thumbnail — historical Vue examples; these are not actively maintained

Note on examples: I've been using examples to test out the implementation on different platforms and frameworks. The examples are also used to run the e2e tests against which makes them more complicated than they need to be for testing and debugging purposes. As the project matures the examples will probably migrate towards a few simpler examples and the e2e testing targets into seperate apps.

Filesystem Adapters

The core package has a pluggable filesystem architecture. It expects any layer that provides the required subset of Effect 4's effect/FileSystem interface.

Browser (OPFS): Use the provided @livestore-filesync/opfs package:

import { layer as opfsLayer } from '@livestore-filesync/opfs'
initFileSync(store, { fileSystem: opfsLayer(), ... })

Expo/React Native: Use the provided @livestore-filesync/expo package:

import { layer as expoLayer } from '@livestore-filesync/expo'
initFileSync(store, { fileSystem: expoLayer(), ... })

Node.js: Use @effect/platform-node directly:

import { NodeFileSystem } from '@effect/platform-node'
createFileSync({ fileSystem: NodeFileSystem.layer, ... })

The published Node adapter uses the same Effect RC cohort. Workspace overrides align its core dependencies with the selected snapshot. The Node example smoke test checks startup; its migration harness checks old persisted metadata, cached bytes, and resumed uploads. Node uploads use fetch when XMLHttpRequest is unavailable.

Backend Storage

Backend storage is optional. Use remote: false for local-only apps or guest sessions where files should be saved, read, updated, and deleted only from local storage. In local-only mode, remoteKey stays as an empty string and localFileState reports local files as uploadStatus: "done" and downloadStatus: "done".

The client expects a signer service running in a server-side process that mints short-lived URLs for uploads/downloads and handles deletes. In the examples, we leverage the existing LiveStore Cloudflare Worker to host this signer service alongside LiveStore sync — see the src/cf-worker/ folder in each example for the implementation.

The API contract:

GET /health — Health check
POST /v1/sign/upload — Returns { url, method, headers?, expiresAt }
POST /v1/sign/download — Returns { url, headers?, expiresAt }
POST /v1/delete — Deletes an object (returns 204)

We provide two backend implementations. To switch between them, update the main entry point in your wrangler.toml to point to either index.r2.ts or index.s3.ts — both files are provided in each example's cf-worker/ folder:

R2 Adapter (`@livestore-filesync/r2`)

Files are proxied through your Cloudflare Worker using R2 bucket bindings.

import { createR2Handler, composeFetchHandlers } from '@livestore-filesync/r2'

export default {
  fetch: composeFetchHandlers(
    createR2Handler({
      bucket: (env) => env.FILE_BUCKET,
      // Secret for HMAC-signing presigned URLs
      getSigningSecret: (env) => env.FILE_SIGNING_SECRET,
      // Async auth validation with optional key prefix restrictions
      validateAuth: async (request, env) => {
        const token = request.headers.get("Authorization")?.replace("Bearer ", "")
        if (!token || token !== env.WORKER_AUTH_TOKEN) return null // Deny
        return [] // Allow all keys (or return ["user123/"] to restrict)
      }
    })
  )
}

S3 Signer (`@livestore-filesync/s3-signer`)

Generates presigned URLs — clients upload/download directly to S3-compatible storage.

import { createS3SignerHandler } from '@livestore-filesync/s3-signer'

export default {
  fetch: createS3SignerHandler({
    basePath: '/api',
    // Async auth validation with optional key prefix restrictions
    validateAuth: async (request, env) => {
      const token = request.headers.get("Authorization")?.replace("Bearer ", "")
      if (!token || token !== env.WORKER_AUTH_TOKEN) return null // Deny
      return [] // Allow all keys (or return ["user123/"] to restrict)
    }
  })
}

Requires S3 credentials (S3_ENDPOINT, S3_BUCKET, S3_ACCESS_KEY_ID, S3_SECRET_ACCESS_KEY) and CORS configuration on the bucket for browser uploads.

Comparison

	R2 Adapter	S3 Signer
How it works	Worker proxies all file data	Worker signs URLs, clients transfer directly
Setup	Simple — just R2 binding	Requires S3 credentials + CORS
Best for	Local dev, small deployments	High traffic, large files
Trade-offs	Bandwidth through Worker (~128MB limit)	More complex initial setup

File URL Resolution

Use resolveFileUrl() to get a displayable URL for each file. This approach integrates with the download queue and automatically prioritizes files that are being displayed.

import { resolveFileUrl } from '@livestore-filesync/core'

const url = await resolveFileUrl(file.id)
// Use url in your component

Automatic prioritization: When resolveFileUrl() is called for a file that's queued for download, that file is automatically moved to the front of the queue. This ensures visible files are downloaded before background files.

In local-only mode, resolveFileUrl() only resolves local files and returns null if the local bytes are missing; it never signs a remote download URL.

See examples/react-filesync or examples/vue-filesync for complete implementations.

Handling Upload State

When files sync across clients, the file metadata may arrive before the file content is uploaded. Use getFileDisplayState() to determine if a file can be displayed:

import { getFileDisplayState } from '@livestore-filesync/core'
import { queryDb } from '@livestore/livestore'

// In your React component — query only this file's local state for targeted reactivity
const localFileState = store.useQuery(
  queryDb(tables.localFileState.where({ fileId: file.id }).first())
)
const { canDisplay, isUploading } = getFileDisplayState(file, localFileState ?? undefined)

// canDisplay is true when:
// - The file exists locally (originating client can display immediately)
// - OR the file has been uploaded (remoteKey is set, other clients can download)

const url = await resolveFileUrl(file.id)

return canDisplay && url
  ? <img src={url} />
  : <div>{isUploading ? 'Uploading...' : 'Waiting for file...'}</div>

This ensures a good user experience:

The originating client displays files immediately from local storage
Other clients show a placeholder until the upload completes
After edits, the correct version is displayed (based on content hash matching)

Transfer correctness

Transfers validate the captured content hash, path, remote key, and deletion state before publishing completion. Editing during an upload or download schedules the latest version; a stale attempt cannot attach its remote key or completion state to the replacement. Downloads verify SHA-256 before writing locally. A checksum mismatch follows the normal retry policy and remains an error if retries are exhausted; retryErrors() can retry after the remote content is repaired. Deleting a file cancels its queued or active download.

Shared bytes and cleanup

Records with identical content share a local path and remote key. Deleting or updating one record preserves bytes owned by another live record. Local cleanup also protects active transfers and serializes deletion with writes and metadata publication within a FileSync instance. It rechecks references after asynchronous deletion and restores bytes if a new owner appeared while the adapter was deleting.

resolveFileUrl() returns null for deleted records and rejects results from reads or signing if the source version changes while resolution is in progress. With the core files table configured, thumbnail generation, state, and URL lookup similarly reject deleted or outdated source content; polling generates the current version. See thumbnail guarantees and compatibility limits.

Remote blobs are retained, including replaced content and stale uploads. A client cannot know about references on offline devices, so it cannot safely garbage-collect remote keys. Applications needing reclamation must implement a server-authoritative retention/GC policy; FileSync does not currently provide one. Local cleanup is best effort, and future references arriving after reclamation require a download (or re-saving bytes in local-only mode).

Multi-Tab Support

FileSync is designed to work correctly when multiple browser tabs are open to the same app. It uses LiveStore's built-in leader election (via Web Locks API) to ensure only one tab runs the sync loop at a time. This prevents race conditions and duplicate operations.

Leader tab: Runs the sync loop, handles uploads/downloads
Non-leader tabs: Can still save/update/delete files — operations are synced to the leader via SharedWorker
Automatic failover: If the leader tab closes, another tab automatically becomes leader

No configuration required — this works automatically.

Lifecycle and shared mounts

start() and stop() are awaitable; existing fire-and-forget calls remain supported. dispose() is permanent and idempotent. Overlapping singleton mounts share the first configuration for the same store object, user, and remote mode. Each disposer belongs to its original instance, so an old mount cannot dispose a newer user's instance. Explicitly dispose and reinitialize to change configuration, or supply an authToken getter for same-user token refresh. Failed startup can be retried with start().

Background workers run only while this instance is a running leader. Stop and leadership loss invalidate old transfer progress/completion and interrupt owned work; restart rebuilds persisted work. Fetch/XHR support abort, but shutdown may wait for local storage operations that cannot be cancelled immediately. See lifecycle guarantees.

Startup and Stream Restarts

Startup bootstrap from the files table is conditional. It runs only when the stored cursor is the root cursor or localFileState is empty.
Mid-session restarts (syncNow(), heartbeat recovery) restart eventsStream from the stored cursor and do not rescan every file row.
Startup and each leadership acquisition rebuild executor queues from persisted queued and interrupted rows, even with an advanced cursor. Errors receive one bounded transfer retry cycle per leadership acquisition or explicit retryErrors() call.
Heartbeat and syncNow() restore missing queued work without resetting active attempts or repeatedly retrying terminal errors. A new file event can retry failed work for that file.
Failed file inspections persist targeted repair IDs and attempt counts alongside the cursor before it advances. Startup and heartbeat retry those IDs, up to max(2, maxRetries + 1) inspections including the first failure. retryErrors() resets exhausted inspection attempts. Disabling heartbeat requires startup or retryErrors() to resume these repairs.
Reconciliation batches changed local rows and compares metadata and local state again before committing. Concurrent changes defer that file to durable repair instead of overwriting it. Unchanged done files are neither inspected on warm startup nor reuploaded.

File Preprocessors

FileSync supports preprocessing files before they are saved. This is useful for:

Resizing images to reduce storage and bandwidth
Converting images to more efficient formats (WebP, JPEG)
Applying transformations based on file type

Basic Usage

import { initFileSync, type PreprocessorMap } from '@livestore-filesync/core'
import { layer as opfsLayer } from '@livestore-filesync/opfs'

const preprocessors: PreprocessorMap = {
  // Transform all images
  'image/*': async (file) => {
    // Your transformation logic
    return transformedFile
  },
  // Or return metadata with the final file
  'image/webp': async (file) => ({
    file,
    metadata: {
      mimeType: file.type,
      sizeBytes: file.size,
      image: { width: 1200, height: 800 }
    }
  }),
  // Or specific types
  'image/png': async (file) => convertPngToJpeg(file)
}

initFileSync(store, {
  fileSystem: opfsLayer(),
  remote: { signerBaseUrl: '/api' },
  options: { preprocessors }
})

Pattern Matching

Preprocessor patterns support:

Exact match: 'image/png' matches only PNG files
Wildcard subtype: 'image/*' matches all image types
Universal wildcard: '*' or '*/*' matches any file

Priority order: exact match > wildcard subtype > universal wildcard.

File Metadata

Preprocessors can return { file, metadata } to persist metadata on the synced files row. Metadata is stored with the current contentHash, so updateFile() replaces it when file content changes.

import { getFileMetadata } from '@livestore-filesync/core'

const metadata = getFileMetadata(fileRecord)
const ratio = metadata?.image
  ? `${metadata.image.width} / ${metadata.image.height}`
  : undefined

Supported metadata fields are:

mimeType?: string
sizeBytes?: number
image?: { width: number; height: number }
custom?: Record<string, unknown>

Existing preprocessors that return only File continue to work. Existing rows without metadata read as null through getFileMetadata().

Image Preprocessing Package

For image preprocessing, use the optional @livestore-filesync/image package:

pnpm add @livestore-filesync/image wasm-vips

import { createImagePreprocessor } from '@livestore-filesync/image/preprocessor'

initFileSync(store, {
  fileSystem: opfsLayer(),
  remote: { signerBaseUrl: '/api' },
  options: {
    preprocessors: {
      'image/*': createImagePreprocessor({
        maxDimension: 1500,  // Max width/height in pixels
        quality: 90,         // JPEG/WebP quality (1-100)
        format: 'jpeg'       // Output format: 'jpeg', 'webp', or 'png'
      })
    }
  }
})

Image preprocessors return metadata for the final output image, including MIME type, byte size, and pixel dimensions. When an image is already in the target format and is returned unchanged, dimensions are still extracted once so UIs can reserve layout before resolveFileUrl() returns.

Lightweight Canvas Alternative: If you don't need the full power of wasm-vips (ICC profile preservation, lossless compression), you can use the canvas-based processor:

createImagePreprocessor({
  processor: 'canvas',  // No WASM required
  maxDimension: 1500,
  format: 'webp'
})

See the image package README and image processing docs for setup instructions and full documentation.

Image Thumbnails (Optional)

The @livestore-filesync/image package provides client-side thumbnail generation in a dedicated web worker. Two processing backends are available:

Canvas: Lightweight, zero additional bundle size, works everywhere
Vips (wasm-vips): Higher quality, ICC profile preservation, ~5 MB WASM download

Features

Client-side generation: Thumbnails are generated in the browser
Multiple sizes: Configure named sizes (e.g., small: 128, medium: 256, large: 512)
Local storage: Thumbnails stored in OPFS (not synced between clients)
Automatic generation: Watches for new image files and generates thumbnails automatically
Leader-only: Only the leader tab generates thumbnails to avoid duplicated work
Batched scan state commits: Startup/poll scans commit changed thumbnail state updates in one transaction before enqueueing generation work, reducing sync-queue pressure during large cold starts without re-emitting unchanged state

Quick Start

import { createFileSyncSchema } from '@livestore-filesync/core/schema'
import { createThumbnailSchema } from '@livestore-filesync/image/thumbnails/schema'
import { initThumbnails, resolveThumbnailUrl } from '@livestore-filesync/image/thumbnails'
import { State } from '@livestore/livestore'
import { layer as opfsLayer } from '@livestore-filesync/opfs'

// 1. Merge schemas
const fileSyncSchema = createFileSyncSchema()
const thumbnailSchema = createThumbnailSchema()

const tables = { ...fileSyncSchema.tables, ...thumbnailSchema.tables }
const events = { ...fileSyncSchema.events, ...thumbnailSchema.events }

const materializers = State.SQLite.materializers(events, {
  ...fileSyncSchema.createMaterializers(tables),
  ...thumbnailSchema.createMaterializers(tables)
})

// 2. Initialize (after FileSync is initialized)
const dispose = initThumbnails(store, {
  sizes: { small: 128, medium: 256, large: 512 },
  format: 'webp',
  fileSystem: opfsLayer(),
  workerUrl: new URL('./thumbnail.worker.ts', import.meta.url),
  schema: { tables }
})

// 3. Create your worker file (thumbnail.worker.ts)
// import '@livestore-filesync/image/thumbnails/worker'

// 4. Get thumbnail URLs
const url = await resolveThumbnailUrl(fileId, 'small')

See examples/react-thumbnail for the maintained thumbnail example. Vue thumbnail examples are historical and are not actively maintained.

Requirements

Browser: OPFS support (Chrome 86+, Edge 86+, Firefox 111+, Safari 15.2+)
Effect 4.0.0-rc.111 and the matching Effect platform packages
For image processing with Vips: wasm-vips ^0.0.16 (~5 MB WASM), SharedArrayBuffer support (requires COOP/COEP headers). Alternatively, use the Canvas processor which has no additional requirements.

License

MIT

Name		Name	Last commit message	Last commit date
.vscode		.vscode
docs		docs
examples		examples
packages		packages
tests/e2e		tests/e2e
.gitignore		.gitignore
.gitmodules		.gitmodules
.npmrc		.npmrc
AGENTS.md		AGENTS.md
LICENSE		LICENSE
README.md		README.md
TODO.md		TODO.md
eslint.config.mjs		eslint.config.mjs
package.json		package.json
pnpm-lock.yaml		pnpm-lock.yaml
pnpm-workspace.yaml		pnpm-workspace.yaml
setupTests.ts		setupTests.ts
tsconfig.base.json		tsconfig.base.json
tsconfig.json		tsconfig.json

Navigation Menu

Repository navigation

About

Repository files navigation

Livestore-Filesync

Packages

Getting Started

LiveStore snapshot cohort

Install

Quick Start

1. Extend your LiveStore schema

2. Initialize FileSync

3. Use the API

Filesystem Adapters

Backend Storage

R2 Adapter (`@livestore-filesync/r2`)

S3 Signer (`@livestore-filesync/s3-signer`)

Comparison

File URL Resolution

Handling Upload State

Transfer correctness

Shared bytes and cleanup

Multi-Tab Support

Lifecycle and shared mounts

Startup and Stream Restarts

File Preprocessors

Basic Usage

Pattern Matching

File Metadata

Image Preprocessing Package

Image Thumbnails (Optional)

Features

Quick Start

Requirements

License

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

Livestore-Filesync

Packages

Getting Started

LiveStore snapshot cohort

Install

Quick Start

1. Extend your LiveStore schema

2. Initialize FileSync

3. Use the API

Filesystem Adapters

Backend Storage

R2 Adapter (@livestore-filesync/r2)

S3 Signer (@livestore-filesync/s3-signer)

Comparison

File URL Resolution

Handling Upload State

Transfer correctness

Shared bytes and cleanup

Multi-Tab Support

Lifecycle and shared mounts

Startup and Stream Restarts

File Preprocessors

Basic Usage

Pattern Matching

File Metadata

Image Preprocessing Package

Image Thumbnails (Optional)

Features

Quick Start

Requirements

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

R2 Adapter (`@livestore-filesync/r2`)

S3 Signer (`@livestore-filesync/s3-signer`)