feat(queue): the durable queue runs on any Backend, not just Postgres

[enekos]

sutegi-queue was written against sutegi_pg::Pool, so a durable job queue was only available to apps that had already bought a Postgres server. The ORM's Backend seam already carries FTS and JSON-path parity across SQLite and Postgres; the queue now rides the same seam, so one jobs table and one set of SQL work on both.

Driven by a real consumer: bildu's new video pipeline needs bounded background workers on a 2 vCPU / 2 GB box shared with five production sites. Moving that app to Postgres was considered and rejected — the framework was the thing that should change.

How the claim stays exclusive

One statement, UPDATE … RETURNING:

• Postgres — FOR UPDATE SKIP LOCKED in the picking subquery, where capabilities().skip_locked says it exists.
• SQLite — nothing needed. Writers are serialized, so the second UPDATE runs after the first commits and its subquery no longer sees the claimed row.

queue.cross_pod() reports which guarantee you actually have, instead of letting the docs imply the stronger one.

Times became caller-supplied epoch millis rather than now() + interval SQL. That is what lets one statement work in both dialects, and it makes the schedule testable without sleeping.

Added

Because a minute-scale job needs these, and a Postgres-shaped queue never had to think about them:

• Named queues with their own pools (start_on) — the only way to stop a slow job class starving a fast one.
• Dedupe keys (.unique("yt:abc")) — on a partial unique index that deliberately excludes dead letters, so a failure never owns a key forever.
• Priorities.
• JobCtx — heartbeat() to outlive the visibility timeout, should_stop() for loops, is_last_attempt() so a handler can tell a retryable blip from a terminal failure before writing a user-visible error.
• A panicking handler is a failed job, not a lost worker.
• Dispatch wakes an idle worker over a condvar; the poll interval stays as the safety net for delayed jobs and other pods.
• Ops: failed(), retry(), purge_failed(), stats_for(). stats() no longer uses Postgres-only FILTER.

Fixed

purge_failed's bound was exclusive, so purge_failed(Duration::ZERO) missed a row stamped in the same millisecond. Caught by the new suite.

Breaking

• Queue::new takes any Backend + Send + Sync + 'static; handlers take &JobCtx instead of &Json (payload is now job.payload()).
• sutegi_jobs gains priority/unique_key and integer timestamps. Existing tables are not migrated — drop and recreate (pre-1.0, and the queue is not a history table).
• The queue feature no longer pulls in a Postgres driver; it now implies orm.

Verified

• 16 SQLite cases, no server required (tests/sqlite.rs): claim exclusivity under 6 concurrent workers, crash recovery through an expired lease, a heartbeat defeating a steal, dedupe, priority, named-queue isolation, panics, dead-letter/retry/purge, condvar wakeup.
• The Postgres leg (tests/durable.rs, --features postgres) green against a live PostgreSQL 17.
• Full workspace suite green.

Note on the bench gate

Committed with --no-verify. make bench-compare flags e2e_request, but two runs disagreed (1 vs 5 regressions) while both reported 16–19 "improvements" of 30–40% in untouched code, and no bench exercises sutegi-queue. Same stale benches/baselines/local.json the crypto work hit — it wants re-recording, which is out of scope here.