Train your team

AIC-101 · GPU & AI Compute

Foundations of AI Compute & GPU Architecture

The architecture, memory hierarchies and fabrics underpinning modern accelerated computing: CPU-GPU topologies, SIMT execution, HBM, NVLink, and datacenter networking.

Foundation 4 days in person8 half-days online Max 14 in person

Prepares for the NVIDIA NCA-AIIO certification.

Who this course is for

Engineers moving into GPU/AI infrastructure who need deep, defensible systems fundamentals: how CPUs, GPUs, memory hierarchies, and high-speed fabrics actually behave underneath PyTorch, vLLM, and distributed training jobs.

Prerequisites

Basic Linux command-line proficiencySome exposure to compiled code (C/C++ helpful)No prior GPU programming required

Course outline

Day 1 — Host architecture & OS fundamentals for AI

  • Fetch/decode/execute and why memory bandwidth dominates modern workloads
  • CPU cache hierarchies (L1/L2/L3), miss patterns, and cacheline false sharing
  • SIMD vector processing: AVX-512, NEON, and Arm SVE execution
  • Multi-socket NUMA topologies, distance matrices, and CPU-GPU affinity
  • Processes, threads, CPU scheduling, and virtual memory page table mechanics
  • Direct I/O, page cache, huge pages (2MB/1GB), and the Linux OOM killer

Day 2 — GPU microarchitecture & memory systems

  • CPU vs GPU architecture philosophy: low-latency cores vs massive SIMT throughput
  • SIMT execution model: warps, wavefronts, branch divergence, and warp scheduling
  • GPU memory hierarchy: registers, shared memory, L2 cache, GDDR, and HBM3/HBM3e
  • Memory coalescing rules, bank conflicts, and achieving theoretical DRAM bandwidth
  • Architecture progression: Tensor Cores (Volta to Blackwell), TF32, Transformer Engine, FP4/FP8
  • AMD CDNA / MI300X architecture: unified APU memory, matrix cores, and Infinity Fabric

Day 3 — Interconnects & multi-GPU topologies

  • PCIe Gen4 through Gen6: lane bandwidth, root complexes, switches, and tree depth
  • NVLink 3.0/4.0/5.0 and NVSwitch crossbars: HGX and DGX architecture topologies
  • Compute Express Link (CXL): CXL.io, CXL.cache, and pooled CXL.mem fabrics
  • Pinned (page-locked) host memory, GPUDirect P2P, and DMA transfer mechanics
  • Unified Memory architectures: CUDA UMA, ROCm UMA, and hardware page faulting

Day 4 — Cluster networking & node procurement audit

  • Datacenter network fabrics for AI: spine-leaf topologies and non-blocking fat trees
  • InfiniBand vs RoCEv2: flow control (PFC), congestion management (ECN/DCQCN)
  • First look at kernel bypass, RDMA queue pairs, and NCCL collective primitives
  • Sizing math for AI infrastructure: balancing compute, HBM capacity, and network bandwidth

What you build

Not exercises with a known answer. Each project is built, measured and defended by you, on hardware and a kernel version you actually run — so the result is something you can take back to your team.

The server silicon & topology audit

Map an active GPU server down to the silicon: CPU NUMA layout, cache boundaries, PCIe switches, GPU-NIC affinity, and HBM topology, identifying three configuration bugs that degrade GPU throughput.

You leave witha reproducible topology discovery script and an executive architecture audit report

The memory tier roofline model

Measure achievable throughput across all memory tiers (system DRAM, PCIe bus, NVLink, and GPU HBM) on your platform, plotting an empirical roofline chart and comparing against vendor specs.

You leave withan empirical bandwidth roofline report with validated kernel execution time predictions

8-GPU node procurement specification

Produce a detailed specification for an 8-GPU training node targeting a 70B parameter model: GPU generation, HBM capacity, host RAM sizing, NIC count, and NVLink topology, defended with sizing math.

You leave witha complete procurement-ready hardware specification and cost-performance trade-off analysis

Hands-on labs

Labs follow the academy model — 35% principles, 20% guided investigation, 45% engineering studio. Every claim you make in a lab is backed by a trace, a counter or a measurement you captured yourself. How we teach

  1. Lab: map server topology using lscpu, numactl, nvidia-smi topo -m, and lspci -tv to trace the full PCIe/NVLink graph
  2. Lab: benchmark memory bandwidth across DRAM, PCIe, and GPU HBM, identifying the NUMA cross-traffic cliff
  3. Lab: measure warp divergence and uncoalesced memory penalties on real GPU kernels with Nsight Compute
  4. Lab: configure huge pages and pinned host memory, measuring the DMA throughput difference to GPU memory
  5. Lab: capture and analyze inter-node RDMA traffic and measure collective latency with iperf3 and perftest

Capstone project

Perform a comprehensive architectural audit and procurement specification for an 8-GPU AI training node: map the physical topology, calculate the bandwidth roofline across each memory tier, identify configuration bottlenecks that starve GPUs, and deliver a procurement-ready specification with bandwidth math attached.

What you leave with

  • A rigorous mental model of the hardware and OS stack underneath PyTorch and vLLM
  • Fluency with topology and profiling tools: nvidia-smi, lstopo, lspci, and Nsight
  • The ability to read GPU whitepapers and calculate realistic throughput ceilings
  • Preparation toward the NVIDIA Certified Associate: AI Infrastructure and Operations (NCA-AIIO)

How it runs

Every course follows the same model: 35% principles, 20% guided investigation, 45% engineering studio. You leave with working code, raw measurements and an evidence-based report — not a certificate of attendance. Read the methodology or see a full sample lesson.

Material is adapted to your kernel version, hardware and workload before a private delivery. For public cohorts, the environment is provided and configured.

Questions

Who is this course for?

Engineers moving into GPU/AI infrastructure who need deep, defensible systems fundamentals: how CPUs, GPUs, memory hierarchies, and high-speed fabrics actually behave underneath PyTorch, vLLM, and distributed training jobs. It sits at foundation level within the GPU & AI Compute track.

What do I need to know already?

Specific prerequisites for this course: Basic Linux command-line proficiency; Some exposure to compiled code (C/C++ helpful); No prior GPU programming required. We confirm levels before the cohort starts and adapt if a group is stronger or weaker than expected.

Can this run privately for my team?

Yes. Any course runs on-site at your offices anywhere, or live online for a distributed team, with labs adapted to your hardware and codebase.

What is the difference between in-person and online?

In person is 4 full days with hardware on your desk, capped at 14. Online is 8 half-day sessions across about two weeks so you can keep working, capped at 20, with remote lab access.

Do you invoice companies?

Yes. Purchase orders are accepted and invoicing is available in USD, EUR, GBP, SAR and CAD.

Upcoming dates

DatesWhereSeatsEarly birdRegular
25 Oct – 28 Oct 20264 full days RiyadhIn person · KAFD Conference Centre 11 of 14 —SAR 9,750
1 Nov – 4 Nov 20264 full days Kuwait CityIn person · Al Hamra Tower 6 of 14 —USD 2,600
8 Nov – 11 Nov 20264 full days MuscatIn person · Knowledge Oasis Muscat 11 of 14 —USD 2,600
8 Nov – 17 Nov 20268 half-days Gulf bandLive online · 09:00–13:00 GMT+3 3 of 20 —US$1,850
9 Nov – 12 Nov 20264 full days OttawaIn person · Kanata North Tech Park 6 of 14 —CAD 3,540
16 Nov – 19 Nov 20264 full days TorontoIn person · MaRS Discovery District 11 of 14 CAD 3,190until 17 OctCAD 3,540
16 Nov – 25 Nov 20268 half-days Europe bandLive online · 09:00–13:00 CET 8 of 20 US$1,660until 17 OctUS$1,850
16 Nov – 25 Nov 20268 half-days Americas bandLive online · 13:00–17:00 ET 13 of 20 US$1,660until 17 OctUS$1,850
23 Nov – 26 Nov 20264 full days LondonIn person · Shoreditch Works 6 of 14 GBP 1,830until 24 OctGBP 2,030
23 Nov – 26 Nov 20264 full days BerlinIn person · Factory Görlitzer Park 11 of 14 EUR 2,150until 24 OctEUR 2,390

Dates shown for the next few months. If nothing fits, tell us where and when — cohorts are added on demand, and private delivery can be scheduled any week.

More in GPU & AI Compute

AIC-2004 days Linux for GPU Systems What the kernel is doing underneath your training job, and how to tune it. The layer almost nobody teaches. Practitioner Practitioner-taught SAR 12,000Next 15 Nov AIC-2104 days RDMA & AI Cluster Networking Build and debug the fabric distributed training runs on, from queue pairs up to a tuned NCCL all-reduce. PractitionerNCP-AIN Practitioner-taught SAR 12,000Next 1 Nov AIC-2205 days CUDA & HIP Programming Write, profile and optimise GPU kernels on both vendors, including the CUDA-to-HIP porting path. Practitioner Practitioner-taught SAR 15,000Next 11 Oct AIC-2304 days Distributed Training with PyTorch From a single-GPU training loop to sharded multi-node training that survives a node failure. Practitioner Practitioner-taught SAR 12,000Next 15 Nov AIC-3004 days Containers, Kubernetes & GPU Schedulers Run a shared GPU cluster multiple teams can actually use: partitioning, scheduling, quotas and isolation. PractitionerNCP-AII Practitioner-taught SAR 12,000Next 18 Oct AIC-3103 days Storage & Data Pipelines for AI Stop starving your GPUs: parallel filesystems, GPUDirect Storage and pipelines built for sustained throughput. Practitioner Practitioner-taught SAR 9,000Next 22 Nov AIC-3203 days MLOps & Inference Serving Get models off a laptop and onto a GPU endpoint that scales, with the pipeline machinery around them. PractitionerNCP-AIO Practitioner-taught SAR 9,000Next 1 Nov AIC-3302 days AI Infrastructure Security & Observability Harden a multi-tenant GPU platform and see what it is doing before users report a problem. Advanced Practitioner-taught SAR 7,500Next 18 Oct AIC-4003 days Large-Scale Training & Datacenter Architecture The thousand-GPU conversation: 3D parallelism, reference architectures, TCO and where the hardware is heading. Advanced Practitioner-taught SAR 11,250Next 8 Nov