RDM-02 · RDMA and HPC Interconnect
A lossless network that carries RDMA and TCP side by side.
Design, configure and troubleshoot an Ethernet fabric where RDMA is protected and everything else still flows.
You build: a lossless fabric design with a runbook for the failures you planned for.
In final review — the chapters for this course are still being finalised.
Promise
Design, configure, and troubleshoot a lossless Ethernet fabric that carries RDMA alongside TCP/IP.
Prerequisites
basic switching; RDM-01 recommended.
Units
1. Foundations ch 01–04
network review, RoCEv2 in depth, converged-deployment challenges.
2. Fabric design ch 05–10
topology, switch basics, priority classes, PFC, ECN, data-center bridging.
3. Host configuration ch 11–15
host setup, kernel tuning, communication libraries, connectivity tests, benchmarks.
4. Congestion and isolation ch 16–19
congestion-control choice, multi-tenancy, storage and compute integration.
5. Virtual and container deployment ch 20–21
SR-IOV in hypervisors and RDMA in Kubernetes.
6. Operations ch 22–24
monitoring, troubleshooting method, link problems.
7. Security, HA, automation ch 25–30
hardening, redundancy, automated deployment, summary.
Labs
Each lab names its environment and a pass check. Labs are not marked complete on "it ran" — the pass check has to hold.
| Lab | What you do | Environment |
|---|---|---|
| RDM-02ach 08–09 | Configure PFC and ECN on a switch or a simulatorPass check: Lossless class shows no drops under load | hardware90 min |
| RDM-02bch 14 | Connectivity test across the fabricPass check: All pairs reachable | hardware60 min |
| RDM-02cch 15 | Benchmark with standard toolsPass check: Report with throughput and latency | hardware90 min |
| RDM-02dch 24 | Find a link fault from countersPass check: Fault named with the counter that shows it | sim60 min |
Capstone project · CAP-03
MPI Application Benchmark and Tuning · 3 weeks
A research team's MPI job runs slower on the RoCEv2 fabric than on the old InfiniBand cluster.
Benchmark the application's communication, find the dominant pattern, tune transport and collective choices, and report the gain with its uncertainty.
Deliverables: Benchmark scripts, before-and-after table, a recommendation with risks.
Outcomes
- I can explain why PFC alone is not enough, and how ECN fits in.
- I can write a configuration for a lossless class and verify it.
- I can find a loss problem with counters before users report it.
- I can automate the host setup with a playbook.
Also in
How it runs
Work through the units in order. Each unit groups the chapters of the study guide with its share of the labs. You can start any time; there is no cohort date and nothing expires.
Labs run in a VM, a container, a simulator or on real hardware, and the card above names which. Where a lab needs hardware — an RDMA NIC, a programmable switch model, an FPGA — there is a simulator fallback so you are never blocked by equipment.
Questions
Do I get a certificate?
You get a completion record for the course. Path badges and project credentials are earned by passing the labs and the project review, not by attendance.
Can my team take this together?
Yes — and if you would rather have it delivered live, with the instructor in the room, this course also exists as a scheduled live cohort. See the live dates.
How long does it take?
≈ 30 h of study including the labs. Most engineers take it part-time over a few weeks.