GitHub repository:
cuda-foundations📚 Portfolio map: https://github.com/open-infra-ai/open-infra-ai
Systematic CUDA kernel engineering from SGEMM fundamentals to reusable inference components
从 SGEMM 基础到可复用推理组件的系统性 CUDA 算子工程学习路径
A repository for people who want to understand how CUDA kernels evolve from a first correct GEMM into reusable kernels, advanced optimization experiments, and lightweight inference plumbing.
Most CUDA learning material is either too small to feel like engineering or too large to understand end to end. cuda-foundations sits in the middle:
- module 01 teaches the optimization ladder directly on SGEMM
- module 02 turns those ideas into a reusable kernel library shape
- module 03 explores more advanced CUDA and HPC patterns
- module 04 shows how kernels, memory, streams, and configuration fit into a small inference-oriented system
Teaching repository, status stable (correctness fixes and teaching
refinements only; no new modules). The four modules are complete. On the
2026-08-23 test machine (RTX 3060 Laptop, sm_86, CUDA 12.0),
ctest --preset default reported 261/261 passed. No new modules
will be added here. Follow-up
work lives in LEARNING_PATH.md (e.g. tiny-llm
and paged-serving).
cuda-foundations is the maintained successor for the overlapping learning paths that previously lived in sgemm-optimization, modern-ai-kernels, hpc-ai-optimization-lab, and mini-inference-engine. Useful implementations and verification now live in the matching modules; the duplicate repositories have been retired instead of remaining as parallel maintenance surfaces.
The former llm-speed GEMM learning path is covered here, while the maintained deep FlashAttention implementation lives in cuflash. The repository intentionally keeps only one implementation per teaching role instead of importing every historical variant.
IN（本仓库负责）：
- CUDA 编程模型与 kernel 编写（线程/块/共享内存/同步）
- GEMM 优化阶梯（naive → tiled → bank-conflict-free → double buffering → WMMA）
- 算子库设计（header-only 布局、operator surface）
- profiling 与性能分析方法
OUT（明确不做，见对应仓库）：
- 生产级推理运行时（模型加载/采样/生成）→ tiny-llm
- 完整 FlashAttention 前后向与优化 → cuflash
- Serving / 分页 KV / continuous batching → paged-serving
- 本仓库 04-inference-engine 为教学预览，非独立作品
- 本仓库 03-hpc-advanced 的 FlashAttention 核是教学简化版，权威实现见 cuflash
| Module | What you learn | Build path |
|---|---|---|
| 01-sgemm-tutorial | tiled SGEMM, bank conflicts, double buffering, WMMA | root CMake + optional Makefile |
| 02-tensorcraft-core | reusable kernel APIs, header-only layout, operator surface（历史模块名 TensorCraft，不是独立产品） | root/module CMake |
| 03-hpc-advanced | advanced optimization topics, experiments, CUDA 12+ features（内含教学用 FA 简化核） | root/module CMake |
| 04-inference-engine（教学预览） | tensor plumbing, memory pools, streams, lightweight inference flow | root/module CMake |
| If you want to... | Go to... |
|---|---|
| understand CUDA optimization from first principles | 01-sgemm-tutorial |
| inspect a reusable kernel library layout | 02-tensorcraft-core |
| study advanced CUDA/HPC experiments | 03-hpc-advanced |
| see kernels embedded in a tiny system | 04-inference-engine |
| understand how to build, verify, and contribute | docs/en/index.md |
For the repository sequence from CUDA fundamentals to LLM serving, see the Chinese AI Infra beginner learning path. It also records the shared optimization and invariant-testing method used across the five retained projects.
git clone https://github.com/open-infra-ai/cuda-foundations.git
cd cuda-foundations
cmake --list-presets
cmake --preset default
cmake --build --preset default
ctest --preset defaultFor the standalone tutorial:
cd 01-sgemm-tutorial
make GPU_ARCH=sm_86
make test- the root CMake graph covers
01-sgemm-tutorial,02-tensorcraft-core,03-hpc-advanced,04-inference-engine,common, andexamples - the
01-sgemm-tutorial/Makefileremains available for quick standalone runs, but tests and CI use the root CMake graph - GitHub Actions only runs CPU-safe checks
- real CUDA build and runtime validation should happen on a local GPU machine
- Documentation index
- Getting started
- Development workflow
- Coding style
- Troubleshooting
- Changelog
- 中文文档首页
- Contributing
- 📖 技术文档站 由 VitePress 驱动，支持 NVIDIA 风格深色主题、交互式图表和学术论文引用。
| Component | Minimum | Recommended |
|---|---|---|
| CUDA Toolkit | 12.0 | 12.x |
| CMake | 3.20 | 3.24+ |
| Compiler | GCC 9 / Clang 10 | GCC 11+ |
| GPU | Volta (sm_70) | Ampere / Ada / Hopper |
@misc{cuda-foundations,
author = {cuda-foundations Contributors},
title = {cuda-foundations},
year = {2026},
publisher = {GitHub},
url = {https://github.com/open-infra-ai/cuda-foundations}
}
MIT