Add bounded concurrency control for the preprocessing/indexing workflow pipeline — Closes #45#63

Merged

conradbzura merged 19 commits into

45-bounded-concurrency-dispatch

Jun 23, 2026

Commits on Jun 22, 2026

feat: Add bounded-concurrency env knobs for the workflow pipeline
conradbzura
committed
feat: Add per-worker backpressure for the wool worker pool
conradbzura
committed
feat: Add a priority load balancer for the workflow worker pool
conradbzura
committed
feat: Add JobRecord dispatch-scheduling fields and lock queries
conradbzura
committed
fix: Harden dispatch-scheduling lock helpers per review
conradbzura
committed
feat: Index the durable retry scheduler's due-dispatch lease query
conradbzura
committed
test: Cover the dispatch-scheduling lock helpers and JobRecord fields
conradbzura
committed
test: Harden load-balancer, backpressure, and wiring coverage
conradbzura
committed
feat: Drive workflow dispatch with a durable bounded-concurrency scheduler
conradbzura
committed
feat: Surface admission rejection as 429 and start the retry scheduler
conradbzura
committed
docs: Document bounded-concurrency, durable queuing, and admission control
conradbzura
committed
fix: Mark a workflow RUNNING on worker acceptance to stop double-dispatch
conradbzura
committed

Commits on Jun 23, 2026