Skip to content

Fail adaptive DPDK mode when RX notifications are unavailable - #10

Merged
thweetkomputer merged 1 commit into
mainfrom
codex/adaptive-require-rx-notifications
Oct 10, 2026
Merged

thweetkomputer merged 1 commit into
mainfrom
codex/adaptive-require-rx-notifications

Conversation

@thweetkomputer

Copy link
Copy Markdown
Contributor

Adaptive mode can currently turn into polling when a PMD rejects RX interrupts or a queue cannot provide a usable notification descriptor. On a 16-worker Azure NetVSC run this made an idle adaptive server consume 16 CPUs while reporting per-queue fallback messages.

This change keeps both poll and adaptive, including the existing default. Adaptive startup validates every hardware RX queue and fails with an explicit BYCORF_DPDK_MODE=poll suggestion if notifications are unavailable. Notification registration and interrupt failures during execution terminate the server instead of silently spinning; peers are stopped before the all-worker teardown barriers.

The mlx5 PMD acknowledges RDMA event-channel records itself. Its empty-channel EAGAIN on interrupt disable is valid after startup probes or timer/mailbox wakes, and is not treated as unsupported hardware. TAP packet descriptors and mlx5 event channels are not read as ordinary eventfds.

Validation on Ubuntu 24.04, GCC 13, Release, DPDK/SPDK enabled:

  • Link-time fault injection with the ring PMD covers configuration rejection without a retry, missing notification descriptors, interrupt enable/disable rejection, runtime enable/disable failure with two workers, explicit poll startup, and mlx5 empty-channel handling.
  • TAP smoke covers both modes, 100 streams, half-close, 80 idle wakeups, cross-worker RX/TX forwarding, and shutdown with open sockets.
  • Lavik runtime-backend smoke covers data, recovery, kernel/DPDK switches, and clean shutdown.
  • Formatting uses clang-format 23.1.1; git diff --check passes.

Runtime and networking architecture documentation and the mode configuration guide are updated. On the Azure ConnectX-5 VF (mlx5_pci), 16 adaptive workers and 16 RX/TX queues start successfully, recover a six-device SPDK dataset with one billion keys, and answer remote PING/DBSIZE. After recovery, idle process CPU averages 11.4% of one CPU (0.114 CPU) across five one-second samples; the previous NetVSC fallback consumed about 16 CPUs. These are idle-mode checks, not throughput results.

@coderabbitai

coderabbitai Bot commented Oct 10, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 04753bb5-5a0f-4dcd-ad8f-62c1d54ab1ce

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@thweetkomputer
thweetkomputer merged commit 7904686 into main Oct 10, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant