Roadmap

Use this roadmap to choose a learning track, not just to see a chronological list of topics.

If you want one fast win, use the entry table and stop after the first stage that matches your goal.
If you want a full academy path, follow the four stages in order.
If you want system intuition, read the module, benchmark, and whitepaper links together instead of treating this page as standalone.

Choose your entry point

If your immediate goal is...	Start with	Why	Then read
Get comfortable reading CUDA kernels	01-SGEMM Tutorial	It keeps the optimization story explicit and local.	Benchmarks
Build reusable CUDA abstractions	02-TensorCraft Core	It moves from kernel tricks to reusable interfaces and runtime helpers.	TensorCraft design
Chase higher-performance techniques	03-HPC Advanced	It is where the repo starts to discuss more architecture-sensitive ideas.	Advanced showcase
Understand end-to-end inference assembly	04-Inference Engine	It connects optimized kernels to memory pools, streams, and runtime orchestration.	Inference engine design

Stage map

What each stage is trying to teach

Stage	Main question	Primary repo pages	Evidence to look for	Foundational paper or reference
1. Fundamentals	How does a fast kernel emerge from a simple one?	01-SGEMM Tutorial, Benchmarks	Stepwise gains in TFLOPS, bandwidth, and correctness confidence	Boehm matmul article; CUDA Programming Guide
2. Core Library	Which optimization ideas deserve reusable abstractions?	02-TensorCraft Core, Architecture, TensorCraft design	Library boundaries that stay readable without losing the optimization story	CUDA C++ Programming Guide; CUTLASS mental model
3. Advanced HPC	How do you move closer to hardware ceilings?	03-HPC Advanced, Advanced showcase	Larger gains from register tiling, WMMA, FlashAttention-like ideas, or newer CUDA features	FlashAttention paper; CUDA feature docs
4. Inference System	How do optimized kernels survive contact with a real runtime?	04-Inference Engine, Inference engine design	Throughput and orchestration evidence that the system still benefits from earlier kernel work	System design docs plus benchmark interpretation

How the modules connect over time

The point of the roadmap is not merely sequential difficulty. It reflects a change in what “success” means:

In Stage 1, success means understanding why each optimization step exists.
In Stage 2, success means keeping those ideas reusable without losing clarity.
In Stage 3, success means knowing when architecture-specific techniques are worth the complexity.
In Stage 4, success means proving the earlier work survives system integration.

Suggested study plans

3-day orientation pass

Day 1: Getting Started + 01-SGEMM Tutorial
Day 2: Benchmarks + System Architecture
Day 3: Choose either 02-TensorCraft Core or 04-Inference Engine based on whether you care more about library design or runtime integration.

2-week kernel engineering track

Week 1: Finish Stage 1 and capture the optimization ladder in your own notes.
Week 2: Read Stage 2 plus the relevant whitepapers, then sample Stage 3 topics that match your GPU and interests.

Full academy path

Move through all four stages in order.
After each stage, verify your understanding against the linked benchmark or whitepaper page before advancing.
Do not skip directly from Stage 1 to Stage 4 unless your goal is purely system orientation; you will miss why the abstractions exist.

Foundational references by stage

[1]Boehm, Simon. How to Optimize a CUDA Matmul Kernel. Technical article, 2022.
[2]Volkov, V. and Demmel, J. W.. Benchmarking GPUs to Tune Dense Linear Algebra.SC, 2008.
[3]Dao, Tri et al.. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness. NeurIPS, 2022.
[4]NVIDIA. CUDA C++ Programming Guide. CUDA Toolkit documentation, 2024.

What to study next after this page

Need the big-picture rationale? Open System Architecture.
Need performance evidence? Pair this page with Benchmarks.
Need to act on a concrete goal? Jump to the module linked in the entry table above.