A Full-Stack Open-Source High-Performance RDMA Hardware and Software Implementation, Focused on AI Applications, Committed to Surpassing Commercial Solutions
English | 简体中文
Ideal for Researchers, University Students, and RDMA Beginners
With the rapid development of AI large model technology, the high-performance networking provided by RDMA has become a critical component of AI infrastructure. However, RDMA technology born in the last century is difficult to fully adapt to today's AI computing scenarios: existing commercial RDMA solutions are constrained by legacy compatibility burdens and struggle to iterate quickly to meet AI demands; RDMA network cards as closed-source black-box products form a significant contradiction with the open-source AI large model software ecosystem. At a time when hardware-software co-optimization is increasingly important, black-box RDMA network cards have become a key bottleneck restricting global optimization of AI systems.
To address these issues, Pazhou Laboratory (Huangpu) and Datanlord jointly initiated the Open-RDMA open-source project. We deeply recognize that relying solely on a single research institution's strength is insufficient to complete the development and debugging of a full-stack system, let alone change the existing industry landscape. Therefore, we chose a full-stack open-source approach, starting from the dimensions of academic research and talent cultivation, leveraging open-source community power to lower the technical barrier of this specialized RDMA field, cultivate RDMA technical talent, enhance the brand influence of the Open-RDMA project community, and gradually expand Open-RDMA's industry influence in the entire RDMA field.
Whether you are a practitioner or student in hardware (FPGA, ASIC), software information technology (driver development, training/inference frameworks, communication protocols), or algorithms (GPU Kernel), you can find a technical direction aligned with your field in the Open-RDMA open-source project.
Break the Black Box, Master the Network | Complete Open-Source RDMA Technology Stack from RTL to Drivers
🌟 Star Us · 📖 Documentation · 🚀 Quick Start · 🤝 Contributing
OpenRDMA is a GPU-friendly RDMA design: the GPU submits only a short instruction, the NIC builds the full request, and data is transferred through the Multipath Reliable Connection (MRC) protocol. The figure below compares how OpenRDMA and existing RDMA solutions submit communication requests:
- GPUDirect RDMA (GDR), orange path: The CPU builds and submits the request.
- InfiniBand GPUDirect Async (IBGDA), blue path: The GPU builds and submits the request.
- OpenRDMA, dark-blue path: The GPU sends only a minimal 8-byte Doorbell to select a preconfigured operation; the NIC parses it and builds the full request.
GDR, IBGDA, and OpenRDMA control paths for submitting communication requests
Multipath Reliable Connection (MRC) is an open network protocol jointly developed by OpenAI, AMD, Broadcom, Microsoft, and NVIDIA. It allows packets from the same QP to use multiple Ethernet paths and handles out-of-order delivery, loss recovery, congestion, and path failures. OpenRDMA draws on these mechanisms to provide reliable multipath transport.
|
|
From RTL hardware design to Linux user-space drivers, every line of code is transparent and public |
Inspired by the MRC protocol, Ethernet-based with hardware-software co-design for ultra-low latency |
|
FPGA-based, freely optimized for AI clusters, GPU communication, and other scenarios—no vendor lock-in |
From verification testing to AI-assisted development tools, accelerating AI infrastructure iteration through AI technology |
|
Researchers can quickly customize experimental platforms based on this project to test novel congestion control algorithms and communication protocols |
All materials publicly available with introductory guides, helping more newcomers become RDMA experts |
|
Hardware RTL code for the open-rdma project |
Driver code including user-space and kernel-space drivers |
Setting up an environment from hardware simulation to software driver debugging and application development is no easy task, so we provide an out-of-the-box development environment. |
|
Language Server for Bluespec SystemVerilog, providing a smooth development experience for BSV projects. |
An AI Agent-driven RTL verification framework launched by Beijing Institute of Open Source Chip, we added support for Cocotb test environment. |
PCIe behavioral and simulation models based on Cocotb, we added Altera RTile support. |
- A Practical Tutorial on PCIe for Total Beginners on Windows
- Although using Windows as an example, most content is platform-agnostic.
- PCIe Deep Dive (Chinese)
We provide a ready-to-use Docker image for new users, containing all development tools and dependencies:
# Pull and start the image
docker run -it open-rdma/dev-env:latest
# Or use docker-compose
docker-compose up -d📖 Detailed Installation Steps
# 1. Clone the repository
git clone https://github.com/open-rdma/open-rdma.git
cd open-rdma
# 2. Install dependencies
make deps
# 3. Build
make build
# 4. Run tests
make testWe welcome all forms of contributions!
- 🐛 Report a Bug
- 💡 Request a Feature
- 📖 Improve documentation
- 🔧 Submit a Pull Request
See Contributing Guide for more details.
- 💬 GitHub Discussions - Questions and discussions
This project is licensed under the Apache 2.0 License.
