Performance-focused CUDA kernels for batch normalization and depthwise conv2d inference, built to showcase low-level optimization skills and measurable GPU speedups.
- Warp-level reductions and shared-memory staging for batch norm statistics
- Coalesced memory access patterns and arithmetic caching
- Unrolled inner loops and fused multiply-add in depthwise conv2d
- Clean, reproducible benchmarks with scripts and report artifacts
| Kernel | Tests | Total runtime | Hardware |
|---|---|---|---|
| Batch Norm (NCHW) | 4 | 1.125 ms | RTX 5090 |
| Depthwise Conv2D (NCHW) | 5 | 0.104 ms | RTX 5090 |
bash scripts/run_batchnorm.sh
bash scripts/run_depthwise_conv.sh- src/batchnorm - batch norm kernel, reference, and tests
- src/depthwise_conv - depthwise conv kernel, reference, and tests
- scripts - build and run helpers
- assets/performance - benchmark plots
- docs - report and assignment specs
- The scripts target
sm_89by default. Update the-archflag to match your GPU. - Profiling commands are included in the scripts but commented out for safety.

