Queue io_uring and SPDK submissions under I/O backpressure

[thweetkomputer]

Parallel I/O bursts can exhaust io_uring SQ space or SPDK request descriptors. Queue accepted work until capacity returns, and remove the shared 4096-object SPDK callback pool that could make one busy qpair reject work for another.

Changes

• Keep prepared SQEs in an io_uring FIFO and advance them during submission and polling without dispatching completions inside Submit calls. Preserve queued cancellation, descriptor reuse protection, outstanding work detection, and permanent submission-error shutdown.
• Keep only unsubmitted parameters in a deque per SPDK qpair. Pass the caller's IoCompletion directly as callback context, remove AsyncRequest and its fixed pool, and release pending metadata immediately after successful submission.
• Preserve FIFO ordering, exactly-once completion accounting, callback reentry, and owner-worker shutdown, including deferred open/close/no-VWC flush completions. Route listener stop/close through its owner worker.
• Prepare directly in the ring SQE when capacity is available and the FIFO is empty. Keep temporary SQEs and queue growth in a non-inlined slow path; preserve borrowed timeout/sendmsg arguments and recv accounting. Ordinary operations no longer check for cancel opcodes.
• Outline queue/error work from common submission paths and synchronize the architecture and API lifetime comments.

Each unfinished storage operation requires its own completion tag; callbacks may reuse or destroy it. The successful byte count now lives in IoCompletion, so consumers must rebuild. On the tested AArch64 layout, common storage awaitables grow by 8 bytes. Software queues grow with available memory.

Validation

All checks passed using this checkout in the Lavik workspace:

ctest --test-dir build_debug -R '^bycorf_(io_uring_backpressure|connect_timer|connection_storage|tcp_buffered_read|backend_selection|task_transfer)_check$' --output-on-failure --no-tests=error
ctest --test-dir /tmp/bycorf-direct-frame-build -R '^bycorf_(spdk_backpressure_stub|io_uring_backpressure|backend_selection)_check$' --output-on-failure --no-tests=error

The first build is Debug; the second is RelWithDebInfo with BYCORF_KERNEL_BYPASS=ON. The SPDK regression runs the production backend against link-time device stubs and covers 8193-request queued/in-flight bursts, independent qpairs, FIFO retries, callback tag reuse/self-destruction, channel changes, and shutdown draining. No NVMe hardware throughput or tail-latency benchmark was run.

The io_uring regression also mixes send and sendmsg operations across direct SQ admission and the pending FIFO, checking completion counts, payload order, and retained caller-owned arguments.

An ARM64 GCC 13.3 Release disassembly comparison against b47fb60, using both a non-LTO object and a small LTO probe, confirms that Send/SendMsg no longer construct or copy a temporary 64-byte SQE on the normal path. Those paths also have no helper calls, cancel-opcode checks, or stack-canary overhead. The probe was used only for disassembly; this does not establish an end-to-end QPS gain.

The changed C++ files also passed clang-format 23.1.1 through pre-commit, and the diff passed whitespace checks.

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: a22b6c1b-22e0-4984-a676-a1f14b096d8b

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}