Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions .claude/skills/doc-check/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ Code examples embedded in prose. API changes silently break copy-paste.
| File | Why it needs checking |
| --------------------------------- | ------------------------------------- |
| `website/guide/quick-start.md` | First code users copy-paste |
| `website/guide/databases.md` | Database comparison and setup codes |
| `website/guide/concepts.md` | Core concept explanations with code |
| `website/guide/server-mode.md` | Server mode tutorial |
| `website/guide/fullstack-mode.md` | Fullstack mode tutorial |
Expand All @@ -101,11 +102,11 @@ Code examples embedded in prose. API changes silently break copy-paste.
| `website/guide/multi-tenant.md` | Multi-tenant isolation patterns |
| `website/guide/deployment.md` | Deployment guide with mode comparison |

### Tier 6: Sidebar config
### Tier 6: Sidebar config and meta tags

Menu links and anchors must match actual headings. Mismatches cause 404s.
Menu links, anchors, and OGP/meta descriptions must stay in sync.

- [ ] `website/.vitepress/config.ts` — VitePress slugifies headings for anchors
- [ ] `website/.vitepress/config.ts` — sidebar links, `og:description`, `twitter:description`, site `description`

### Tier 7: Example apps

Expand Down
12 changes: 12 additions & 0 deletions .claude/skills/doc-check/scripts/find-stale.sh
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,18 @@ else
"Prefer await durably.init() in guides and examples" \
"$API_REF_EXCLUDE"

# ── OGP / meta tags ──

check_pattern \
"OGP/meta still says 'Just SQLite'" \
'Just SQLite' \
"PostgreSQL is now supported. Update og:description, twitter:description, and site description"

check_pattern \
"OGP/meta still says 'No Redis'" \
'No Redis' \
"Outdated positioning. Update og:description and twitter:description"

# ── Old terminology ──

check_pattern \
Expand Down
21 changes: 13 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# durably

Step-oriented resumable batch execution for Node.js and browsers using SQLite.
Steps that survive crashes. SQLite to PostgreSQL.

**[Documentation](https://coji.github.io/durably/)** | **[Live Demo](https://durably-demo.vercel.app)**

Expand All @@ -13,16 +13,21 @@ Step-oriented resumable batch execution for Node.js and browsers using SQLite.

## Features

- Resumable batch processing with step-level persistence
- Works in both Node.js and browsers
- Uses SQLite for state management (better-sqlite3/libsql for Node.js, SQLite WASM for browsers)
- Minimal dependencies - just Kysely and Zod as peer dependencies
- Event system for monitoring and extensibility
- Type-safe input/output with Zod schemas
- **Resumable** — each step's result is persisted; interrupted jobs resume from the last successful step
- **Flexible storage** — libSQL/Turso, PostgreSQL, better-sqlite3, or browser OPFS
- **Browser + server** — same API for Node.js and browsers
- **Lease-based recovery** — stale workers are automatically reclaimed via fencing tokens
- **Auto cleanup** — `retainRuns` option purges old completed runs automatically
- **React hooks** — real-time progress via SSE, fullstack and SPA modes
- **Type-safe** — Zod schemas for input/output, labels, and auth context

## Quick Start

See the [Quick Start](https://coji.github.io/durably/guide/quick-start) for installation and usage instructions.
```bash
pnpm add @coji/durably kysely zod @libsql/client @libsql/kysely-libsql
```

See the [Quick Start](https://coji.github.io/durably/guide/quick-start) guide, or [Choosing a Database](https://coji.github.io/durably/guide/databases) for PostgreSQL and other backends.

## License

Expand Down
2 changes: 1 addition & 1 deletion packages/durably-react/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# @coji/durably-react

React bindings for [Durably](https://github.com/coji/durably) - step-oriented resumable batch execution.
React bindings for [Durably](https://github.com/coji/durably) — steps that survive crashes.

**[Documentation](https://coji.github.io/durably/)** | **[GitHub](https://github.com/coji/durably)**

Expand Down
10 changes: 7 additions & 3 deletions packages/durably/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# @coji/durably

Step-oriented resumable batch execution for Node.js and browsers using SQLite.
Steps that survive crashes. SQLite to PostgreSQL.

**[Documentation](https://coji.github.io/durably/)** | **[GitHub](https://github.com/coji/durably)** | **[Live Demo](https://durably-demo.vercel.app)**

Expand All @@ -9,10 +9,14 @@ Step-oriented resumable batch execution for Node.js and browsers using SQLite.
## Installation

```bash
npm install @coji/durably kysely zod better-sqlite3
# libSQL (recommended default)
npm install @coji/durably kysely zod @libsql/client @libsql/kysely-libsql

# PostgreSQL (multi-worker)
npm install @coji/durably kysely zod pg
```

See the [Quick Start](https://coji.github.io/durably/guide/quick-start) for other SQLite backends (libsql, SQLocal for browsers).
See [Choosing a Database](https://coji.github.io/durably/guide/databases) for all backends.

## Quick Start

Expand Down
2 changes: 1 addition & 1 deletion packages/durably/docs/llms.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

## Overview

Durably is a minimal workflow engine that persists step results to SQLite or PostgreSQL. If a job is interrupted (server restart, browser tab close, crash), it automatically resumes from the last successful step. Supports libSQL/Turso (single-server, serverless), PostgreSQL (multi-worker), and SQLocal (browser/OPFS).
Durably is a minimal workflow engine that persists step results to SQLite or PostgreSQL. If a job is interrupted (server restart, browser tab close, crash), it automatically resumes from the last successful step. Supports libSQL/Turso (single-server or serverless), PostgreSQL (recommended for multi-worker), and SQLocal (browser/OPFS).

## Installation

Expand Down
6 changes: 3 additions & 3 deletions website/.vitepress/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import { defineConfig } from 'vitepress'
export default defineConfig({
title: 'Durably',
description:
'Step-oriented resumable batch execution for Node.js and browsers',
'Steps that survive crashes. Resumable batch execution for Node.js and browsers. SQLite to PostgreSQL.',
base: '/durably/',

head: [
Expand All @@ -13,7 +13,7 @@ export default defineConfig({
'meta',
{
property: 'og:description',
content: 'Just SQLite. No Redis required.',
content: 'Steps that survive crashes. SQLite to PostgreSQL.',
},
],
[
Expand All @@ -30,7 +30,7 @@ export default defineConfig({
'meta',
{
name: 'twitter:description',
content: 'Just SQLite. No Redis required.',
content: 'Steps that survive crashes. SQLite to PostgreSQL.',
},
],
[
Expand Down
76 changes: 35 additions & 41 deletions website/guide/databases.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,24 @@ Durably supports multiple database backends through Kysely dialects. This guide

## At a Glance

| Backend | Writers | Serverless | Setup | Performance | Cost |
| ------------------------- | ------------- | ---------- | --------------- | ------------------------------ | ----------------------------- |
| **libSQL** (local) | Single | Limited | Zero config | Fast (local file) | Free |
| **Turso** (remote libSQL) | Single per DB | Yes | Managed | Network latency, edge-friendly | Free tier, then pay-as-you-go |
| **better-sqlite3** | Single | No | Zero config | Fast (local sync) | Free |
| **PostgreSQL** | Multiple | Varies | Server required | Strong under concurrency | Self-hosted or managed |
| **SQLocal** (browser) | Single tab | N/A | Zero config | Browser-local (OPFS) | Free (client-side) |
### SQLite Family (Single-Writer)

| Backend | Serverless | Setup | Performance | Cost |
| --------------------- | --------------------------- | -------------------------------------- | -------------------------------- | ------------------------------------------------ |
| **libSQL / Turso** | Local: Limited, Remote: Yes | Zero config (local) / Managed (remote) | Fast local, edge-friendly remote | Free (local) / Free tier + pay-as-you-go (Turso) |
| **better-sqlite3** | No | Zero config | Fast (synchronous) | Free |
| **SQLocal** (browser) | N/A | Zero config | Browser-local (OPFS) | Free (client-side) |

### PostgreSQL (Multi-Writer)

- **Serverless**: Varies (Neon: yes, RDS: no)
- **Setup**: Server required
- **Performance**: Strong under concurrency (advisory locks + `FOR UPDATE SKIP LOCKED`)
- **Cost**: Self-hosted or managed (Neon, Supabase, RDS)

::: warning
**Local SQLite** (libSQL local, better-sqlite3) is single-writer — multiple workers on the same file will cause lock contention. **Turso remote** accepts multiple connections, but concurrency key enforcement is weaker than PostgreSQL (no advisory locks). For reliable multi-worker setups, use PostgreSQL.
:::

## Decision Flowchart

Expand All @@ -19,21 +30,23 @@ Running in the browser?
Yes → SQLocal (only option)
No ↓

Need multiple workers processing jobs concurrently?
Yes → PostgreSQL
Large volume of jobs, or multiple app servers sharing one DB?
(e.g. high-traffic API, queue with thousands of jobs/hour)
Yes → PostgreSQL (strongest guarantees)
Turso also works (weaker concurrency key enforcement)
No ↓

Deploying to serverless / edge (Vercel, Cloudflare)?
Yes → Turso (remote libSQL)
No ↓

Need a lightweight embedded DB?
Single server or CLI script?
Yes → libSQL (local) or better-sqlite3
```

## libSQL (Local)
## libSQL / Turso

**Recommended default for Node.js.** Zero-config embedded database that works everywhere.
**Recommended default.** libSQL works as a local embedded database and as a managed remote database via [Turso](https://turso.tech). Same `LibsqlDialect` for both — just change the URL.

```bash
pnpm add @libsql/client @libsql/kysely-libsql
Expand All @@ -43,36 +56,21 @@ pnpm add @libsql/client @libsql/kysely-libsql
import { createClient } from '@libsql/client'
import { LibsqlDialect } from '@libsql/kysely-libsql'

// Local (single file on disk)
const client = createClient({ url: 'file:local.db' })
const dialect = new LibsqlDialect({ client })
```

- Single file on disk
- No external dependencies
- Same dialect works with Turso remote (just change the URL)

## Turso (Remote libSQL)

**For serverless and edge deployments.** Managed libSQL database with global replication.

```bash
pnpm add @libsql/client @libsql/kysely-libsql
```
// Turso remote (serverless / edge)
// const client = createClient({
// url: process.env.TURSO_DATABASE_URL!,
// authToken: process.env.TURSO_AUTH_TOKEN!,
// })

```ts
import { createClient } from '@libsql/client'
import { LibsqlDialect } from '@libsql/kysely-libsql'

const client = createClient({
url: process.env.TURSO_DATABASE_URL!,
authToken: process.env.TURSO_AUTH_TOKEN!,
})
const dialect = new LibsqlDialect({ client })
```

- Same `LibsqlDialect` as local — just swap the URL
- **Local**: Single file, zero config, no external dependencies
- **Turso**: Managed service with global replication, free tier available
- Works on Vercel, Cloudflare Workers, Fly.io
- Free tier available at [turso.tech](https://turso.tech)

::: tip
See the [fullstack-vercel-turso example](https://github.com/coji/durably/tree/main/examples/fullstack-vercel-turso) for a complete Vercel + Turso deployment.
Expand All @@ -97,11 +95,11 @@ const dialect = new SqliteDialect({

- Synchronous API (slightly faster for small workloads)
- Good for one-off scripts and CLI tools
- No Turso remote support (use libSQL if you might need remote later)
- No remote support (use libSQL if you might need Turso later)

## PostgreSQL

**For multi-worker production deployments.** The only backend that supports multiple workers processing jobs concurrently from the same database.
**For multi-worker production deployments.** The recommended backend for running multiple workers concurrently, with advisory locks and `FOR UPDATE SKIP LOCKED` for strong concurrency guarantees.

```bash
pnpm add pg
Expand All @@ -122,10 +120,6 @@ const dialect = new PostgresDialect({
- Connection pooling via `pg.Pool`
- Works with any PostgreSQL provider (Neon, Supabase, RDS, self-hosted)

::: warning
SQLite backends (libSQL, better-sqlite3) are single-writer. Running multiple workers against the same SQLite file will cause lock contention. Use PostgreSQL for multi-worker setups.
:::

## SQLocal (Browser)

**For browser-only apps.** Runs SQLite in the browser using OPFS (Origin Private File System).
Expand Down
26 changes: 20 additions & 6 deletions website/guide/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Long-running tasks fail. Networks drop, servers restart, browsers close. Traditi

## The Solution

Durably saves each step's result to SQLite. On resume, completed steps return cached results instantly.
Durably saves each step's result to the database (SQLite or PostgreSQL). On resume, completed steps return cached results instantly.

```ts
const job = defineJob({
Expand All @@ -35,14 +35,28 @@ If the process crashes after importing 500 of 1000 rows, restart picks up at row

## Three Ways to Run

| Mode | Storage | Use Case |
| ------------- | ------------------------------ | ------------------------------------ |
| **Server** | @libsql/client, better-sqlite3 | Cron jobs, data pipelines, CLI tools |
| **Fullstack** | Server DB + SSE to browser | Web apps with real-time progress UI |
| **SPA** | SQLite WASM + OPFS | Offline-capable, local-first apps |
| Mode | Storage | Use Case |
| ------------- | ---------------------------------- | ------------------------------------ |
| **Server** | libSQL, PostgreSQL, better-sqlite3 | Cron jobs, data pipelines, CLI tools |
| **Fullstack** | Server DB + SSE to browser | Web apps with real-time progress UI |
| **SPA** | SQLite WASM + OPFS | Offline-capable, local-first apps |

Same job definition works in all three modes.

## When NOT to Use Durably

Durably is great for batch processing with step-level resumability. It's not the right tool for everything:

| If you need... | Use instead |
| -------------------------------------------------------------- | ------------------------------------------------------ |
| **Sub-second job dispatch** (real-time messaging, webhooks) | BullMQ, SQS, or a message broker |
| **Hundreds of concurrent workers** processing a massive queue | Temporal, Inngest, or a dedicated workflow engine |
| **Simple scheduled tasks** (one cron job, no steps) | Node.js cron (`node-cron`), OS crontab, or Vercel Cron |
| **Stream processing** (continuous event ingestion) | Kafka, Redis Streams |
| **Long-running single operation** (no natural step boundaries) | A plain async function with retry logic |

Durably shines when your task has **multiple discrete steps** that are expensive to redo, and you want **automatic resume** without managing infrastructure.

## Next Step

**[Quick Start](/guide/quick-start)** — Run your first resumable job in under 2 minutes.
20 changes: 13 additions & 7 deletions website/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ layout: home
hero:
name: Durably
text: Resumable Batch Execution
tagline: Just SQLite. No Redis required.
tagline: Steps that survive crashes. SQLite to PostgreSQL.
actions:
- theme: brand
text: Get Started
Expand All @@ -17,16 +17,22 @@ hero:
link: https://github.com/coji/durably

features:
- icon: 🚀
title: Zero Infrastructure
details: SQLite only. No Redis, no Postgres, no config files. Just npm install and go.
- icon: 🔄
title: Resumable Steps
details: Each step auto-saves to SQLite. Interrupted jobs resume exactly where they left off.
details: Each step auto-saves to the database. Interrupted jobs resume exactly where they left off — server restarts, crashes, browser tab closes.
- icon: 🗄️
title: Flexible Storage
details: SQLite, libSQL/Turso, PostgreSQL, or browser OPFS. Pick what fits your deployment — from zero-config local to multi-worker production.
- icon: 🌐
title: Browser + Server
details: Same API for Node.js and browsers. Use OPFS for offline-capable browser apps.
details: Same API for Node.js and browsers. Use OPFS for offline-capable browser apps, or connect to a server via SSE.
- icon: ⚡
title: React Ready
details: Built-in hooks with real-time updates via SSE. Build progress UIs in minutes.
details: Built-in hooks with real-time progress updates via SSE. Fullstack and SPA modes with type-safe APIs.
- icon: 🔒
title: Lease-Based Recovery
details: Workers claim jobs via leases with fencing tokens. Stale leases are automatically reclaimed — no stuck jobs.
- icon: 🧹
title: Auto Cleanup
details: Set retainRuns to automatically purge old completed runs. Or call purgeRuns() for manual batch cleanup.
---
2 changes: 1 addition & 1 deletion website/public/llms.txt
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

## Overview

Durably is a minimal workflow engine that persists step results to SQLite or PostgreSQL. If a job is interrupted (server restart, browser tab close, crash), it automatically resumes from the last successful step. Supports libSQL/Turso (single-server, serverless), PostgreSQL (multi-worker), and SQLocal (browser/OPFS).
Durably is a minimal workflow engine that persists step results to SQLite or PostgreSQL. If a job is interrupted (server restart, browser tab close, crash), it automatically resumes from the last successful step. Supports libSQL/Turso (single-server or serverless), PostgreSQL (recommended for multi-worker), and SQLocal (browser/OPFS).

## Installation

Expand Down