RDMA Programming: InfiniBand & RoCE

High-Performance Networking & Cloud-Native · 30 illustrated lessons (text/HTML)

Outcomes

• Use the verbs API for send/recv, RDMA read/write and atomics
• Tune RDMA performance and flow control
• Benchmark and troubleshoot RDMA applications

Where we teach this

• AIC-210 · RDMA & AI Cluster Networking live cohort

The lessons

Every lesson in this curriculum, in the order we teach it.

• 1RDMA overview
• 2InfiniBand architecture and layering
• 3The RDMA hardware and software ecosystem
• 4Setting up an RDMA development environment
• 5RDMA core concepts and resources explained
• 6The verbs API programming model
• 7Memory registration and DMA
• 8Queue pair (QP) state machine
• 9RDMA operation types explained
• 10Connection management: establishing connections with the CM
• 11Your first RDMA program: Hello World
• 12One-sided operations: RDMA Write in practice, remote memory access semantics and consistency models
• 13Two-sided operations: Send/Recv in practice, the messaging model and buffer management
• 14Completion event handling
• 15RDMA atomic operations
• 16Multiple queue pairs and concurrency
• 17SRQ (shared receive queue)
• 18Memory windows: principles, binding and unbinding, and flexible access control
• 19RDMA flow control and reliability
• 20RDMA performance tuning basics
• 21Advanced RDMA performance tuning
• 22RoCEv2-specific configuration: PFC, ECN, DCQCN, and IP/UDP layer setup
• 23RDMA with multi-tenancy and virtualisation
• 24RDMA in storage
• 25RDMA in AI and HPC
• 26RDMA troubleshooting and debugging
• 27RDMA security considerations
• 28Advanced RDMA programming patterns
• 29RDMA testing and benchmarking
• 30Project: designing a simple RDMA key-value store