CUDA Kernel Academy

English

简体中文

English

简体中文

Theme

CUDA Kernel AcademySystematic CUDA kernel engineering

A repository-specific reading map from SGEMM fundamentals to reusable inference components.

Visit GitHub
Start Reading
01

SGEMM Tutorial

Learn the kernel optimization ladder: naive, tiling, bank-conflict cleanup, double buffering, vectorization, and Tensor Core variants.

02

TensorCraft Core

See how tutorial kernels become reusable library components with modern C++ CUDA interfaces, error handling, and memory management.

03

HPC Advanced

Study the module that pushes toward production-grade performance with CUTLASS, FlashAttention, register tiling, and newer CUDA features.

04

Inference Engine

Follow the end-to-end system path where optimized kernels become a schedulable inference pipeline with memory pools and streams.

4
Core modules
3
Reading lenses
2
Build systems
1
Integrated learning path

Who this landing page is for ​

This page is the fastest orientation pass for readers who want more than a repo brochure.

  • New CUDA learners can use it as a staged reading order instead of opening modules at random.
  • Practitioners tuning kernels can jump straight to the module and benchmark evidence most relevant to GEMM performance.
  • Systems builders can see how tutorial kernels, reusable infrastructure, and inference integration relate inside one repository.

Read the repository like an academy, not a brochure ​

Treat the repository as three connected views of the same work:

  1. Module pages explain what each directory teaches and what code to inspect first.
  2. Whitepapers explain why the modules are shaped the way they are and how the pieces connect.
  3. Benchmarks and roadmap tell you how to validate the results and what to study next.

Recommended first pass

If you are new to this repo, read Getting Started, then skim 01-SGEMM Tutorial, and only then come back to the broader architecture whitepaper.

Choose your first module by goal ​

Your goalStart hereWhy this is the right entryRead next
Understand CUDA performance basics01-SGEMM TutorialIt shows the optimization ladder in the smallest possible teaching surface.Benchmarks → SGEMM whitepaper
Build reusable CUDA operators02-TensorCraft CoreIt turns isolated kernels into reusable abstractions, RAII-based resource handling, and library boundaries.Architecture → TensorCraft design
Study advanced kernel techniques03-HPC AdvancedIt is where register tiling, FlashAttention, CUTLASS, and newer CUDA features are discussed in repo context.Advanced showcase → Benchmarks
Understand end-to-end deployment shape04-Inference EngineIt shows how optimized kernels feed stream scheduling, memory pools, and inference-oriented execution.Inference engine design → Roadmap

How the modules connect technically ​

System Architecture

SGEMM
01-SGEMM Tutorial
TENSORCRAFT
02-TensorCraft Core
HPC
03-HPC Advanced
INFERENCE
04-Inference Engine

Click a module to see details

The repo is intentionally layered.

  • Module 01 teaches the mechanics of a fast kernel in the most explicit form.
  • Module 02 captures the reusable patterns that survive beyond a one-off tutorial kernel.
  • Module 03 explores techniques that matter when you are chasing architectural limits rather than just correctness.
  • Module 04 proves that the earlier modules are useful only if they can be integrated into a whole inference pipeline.

Build reality before you dive deeper ​

  • The root CMake graph covers 02-tensorcraft-core, 03-hpc-advanced, 04-inference-engine, common, and examples.
  • 01-sgemm-tutorial stays intentionally separate behind its own Makefile.
  • GitHub-hosted automation is CPU-safe only; real CUDA build and runtime validation belong on a local GPU machine.

What each documentation area gives you ​

Documentation areaBest used forMain question it answers
Module docsClose reading of code and concepts“What does this directory teach, and where should I open the code first?”
WhitepapersCross-module reasoning“Why is the repo structured this way, and how do the modules feed one another?”
BenchmarksEvidence and limits“What performance story is the repo claiming, and how should I interpret it?”
RoadmapSequencing your study“What should I read next for my current goal and experience level?”

Foundational references for the first pass ​

  1. [1]Volkov, V. and Demmel, J. W.. Benchmarking GPUs to Tune Dense Linear Algebra.SC, 2008.
  2. [2]Boehm, Simon. How to Optimize a CUDA Matmul Kernel. Technical article, 2022.
  3. [3]Dao, Tri et al.. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness. NeurIPS, 2022.
  4. [4]NVIDIA. CUDA C++ Programming Guide. CUDA Toolkit documentation, 2024.

Study next ​

  • Want the system view first? Read the architecture whitepaper.
  • Want evidence before theory? Read benchmarks with the SGEMM module open beside it.
  • Want a guided route by goal? Open Reading Paths.
  • Want a guided sequence? Follow the roadmap and choose the track that matches your goal.

Released under the MIT License.

Copyright © 2026 CUDA Kernel Academy Contributors