03-HPC Advanced

Advanced CUDA / HPC tutorial module focusing on complex operator organization, GEMM optimization paths, property-based testing, and new architecture experiments.

Prerequisites

Completion of 01-sgemm-tutorial and 02-tensorcraft-core is recommended.

Contents

• GEMM optimization path (Naive → Pipeline)
• Reduction / Attention / Convolution / Quantization examples
• GoogleTest + RapidCheck testing
• Optional Python bindings
• CUDA 12/13 experimental features

Sub-topics

• GEMM Optimization
• Memory Optimization
• Reduction Optimization
• Flash Attention
• CUDA 13 Features

References