Benchmarks
This page explains what benchmark evidence in CUDA Kernel Academy is meant to prove, and what it is not meant to prove.
- For kernel readers: use it to connect optimization steps to measured deltas.
- For library and systems readers: use it to see which claims are local kernel wins versus end-to-end integration signals.
- For cautious readers: read the caveats first; some values are placeholder reference numbers rather than a universal performance promise.
How to read benchmark evidence in this repo
Benchmark data in this repository should be interpreted as teaching evidence, not marketing copy.
- SGEMM ladders show the direction and relative impact of optimization steps.
- Module-level tables show what each module is trying to validate.
- Comparisons to cuBLAS are a sanity check, not a claim that the repo should replace cuBLAS in every case.
Limits matter
Some values on this page are placeholder reference numbers for an RTX 4090-class GPU. They are useful for understanding the shape of the optimization ladder, but real validation still belongs on your own GPU, with your own compiler, clocks, and CUDA version.
SGEMM optimization ladder
| Kernel | TFLOPS (FP32) | Bandwidth (GB/s) | vs cuBLAS |
|---|---|---|---|
| Naive | 0.5 | 20 | 2% |
| Tiled | 2.1 | 85 | 10% |
| Coalesced | 4.5 | 180 | 22% |
| Double Buffer | 8.2 | 320 | 40% |
| Vectorized | 12.5 | 480 | 60% |
| Tensor Core | 18.0 | 700 | 85% |
| cuBLAS | 21.0 | 820 | 100% |
What each module's benchmarks are trying to prove
| Module | Typical evidence | What that evidence means | What it does not mean |
|---|---|---|---|
| 01-SGEMM Tutorial | Step-by-step TFLOPS and bandwidth gains | Individual kernel transformations are doing real work. | A single tuned SGEMM generalizes to every operator. |
| 02-TensorCraft Core | Correctness, reusable operator behavior, library-level overhead checks | Abstractions can remain lightweight enough for performance-sensitive code. | Every abstraction is free on every architecture. |
| 03-HPC Advanced | Higher ceilings from register tiling, WMMA, CUTLASS-style ideas, or FlashAttention-inspired kernels | Advanced techniques can move the ceiling closer to hardware limits. | The same technique is always the best choice for every GPU or problem size. |
| 04-Inference Engine | Throughput, latency, and runtime behavior under streams or memory-pool reuse | Kernel work still matters when placed inside a larger execution pipeline. | End-to-end speedup comes only from the kernel; orchestration matters too. |
How to interpret the SGEMM ladder
Read the ladder as a chain of questions:
- Naive → Tiled: did shared-memory reuse remove the most obvious waste?
- Tiled → Coalesced: are memory transactions better aligned with how the GPU wants to fetch data?
- Coalesced → Double Buffer: are compute and data movement overlapped more effectively?
- Double Buffer → Vectorized / Tensor Core: are you now benefiting from hardware-specific throughput features rather than only generic cleanup?
The value of this ladder is that it matches the repository structure: the early modules teach the steps, and the later modules show how those steps evolve inside more realistic systems.
Common mistakes when reading these numbers
- Mistake: treating cuBLAS percentage as the only score. In this repo, the learning value is often in understanding why each jump happened.
- Mistake: comparing across architectures without context. The same kernel can move very differently on Volta, Ampere, Ada, or Hopper.
- Mistake: ignoring correctness and integration cost. A faster kernel that does not compose cleanly with the later modules is not a meaningful repo-wide win.
Benchmark workflow inside this repository
This workflow is why the benchmark page belongs in an academy-style site: it teaches readers how to connect code changes, architectural reasoning, and evidence.
Foundational references for interpreting results
- [1]. Benchmarking GPUs to Tune Dense Linear Algebra.SC, 2008.
- [2]. How to Optimize a CUDA Matmul Kernel. Technical article, 2022.
- [3]. CUDA C++ Best Practices Guide. CUDA Toolkit documentation, 2024.
Study next
- To understand why these numbers are arranged in this order, read 01-SGEMM Tutorial.
- To understand how these optimizations feed system design, read System Architecture.
- To decide which benchmark lens matters for your current goal, follow the Roadmap.