Skip to content

relay: bound outbox batches so concurrent relays share a large backlog #262

Closed

Description

@choiyounggi

Context

PR #259 (#257) wraps each lnpl relay batch in one driver transaction, so on a row-locking driver (lnpl-postgres, SELECT … FOR UPDATE SKIP LOCKED) a second relay skips rows the first one holds. But both drain call sites call repository.drain_outbox() with no limit, and lnpl-postgres binds LIMIT NULL. So one batch locks the whole undelivered backlog and holds the transaction across every serial publish or POST (urlopen(..., timeout=10) each). A second relay gets nothing rather than a disjoint share, and a long backlog means one long transaction.

The batch was kept unbounded in #257 because the TCK double _TckOutbox (impl/lnpl/testing.py) and two test doubles define drain_outbox(self) with no limit parameter, and changing the TCK signature affects external drivers. docs/backends.md §3 documents the consequence.

Proposal

Either:

  • pass drain_outbox(limit=N) (TCK change, announced in docs/compatibility.md), or
  • a limiting proxy in cli.py, with --once repeating bounded batches until a drain comes back empty (no TCK change).

Done when

  • Relay batches are bounded by a documented size; two concurrent relays on lnpl-postgres receive disjoint, non-empty shares of a large backlog (test); docs/backends.md §3 updated.

Activity

  1. added a commit that references this issue on Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    production-readinessGaps blocking real production use (beyond the Phase 1 reference impl)

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions