GEMM Optimization

Detailed walkthrough of the 7-step GEMM optimization path.

Optimization Path

Summary

Step	Technique	TFLOPS (FP32)	Speedup
1	Naive	0.5	1.0×
2	Shared Memory	2.0	4.0×
3	Double Buffer	3.5	7.0×
4	Register Tiling	6.0	12.0×
5	WMMA	50+	100×
6	MMA PTX	60+	120×
7	Pipeline	70+	140×

References