Skip to content

About

Custom CUDA kernels from scratch for Batch Normalization and 2D Depthwise Convolution

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

1 Commit

Folders and files

Repository files navigation

CUDA CNN Kernels

Performance-focused CUDA kernels for batch normalization and depthwise conv2d inference, built to showcase low-level optimization skills and measurable GPU speedups.

Highlights

  • Warp-level reductions and shared-memory staging for batch norm statistics
  • Coalesced memory access patterns and arithmetic caching
  • Unrolled inner loops and fused multiply-add in depthwise conv2d
  • Clean, reproducible benchmarks with scripts and report artifacts

Results

Kernel Tests Total runtime Hardware
Batch Norm (NCHW) 4 1.125 ms RTX 5090
Depthwise Conv2D (NCHW) 5 0.104 ms RTX 5090

Batch norm performance Depthwise conv performance

Quick Start

bash scripts/run_batchnorm.sh
bash scripts/run_depthwise_conv.sh

Project Structure

Report and Specs

Notes

  • The scripts target sm_89 by default. Update the -arch flag to match your GPU.
  • Profiling commands are included in the scripts but commented out for safety.

About

Custom CUDA kernels from scratch for Batch Normalization and 2D Depthwise Convolution

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages