Warpline is a measured GPU performance project for three Triton kernels on an NVIDIA Blackwell laptop GPU. It combines explicit numerical contracts, fair PyTorch baselines, CUDA-event benchmarking, an empirical roofline, and Nsight Compute evidence. Performance claims come from versioned JSON artifacts and checksummed manifests rather than product-sheet peaks or one-off timings.
The accepted run is
rtx-pro-3000-20260812. Its
validator covers 120 benchmark artifacts, four plots, six K1 shapes, six K2
shapes, and two K3 shapes in both cold headline and warm diagnostic lanes. The
accepted Nsight run is
20260812-blackwell-ncu-2026.2.1;
its manifest is complete and its checksums pass.
One Triton program handles one row. It adds the input and residual in FP32, rounds that sum once to the input dtype, computes the RMS reduction in FP32, applies the weight, and writes the final FP16 or BF16 result. Fusion removes a materialized residual tensor. At batch 64 and hidden size 8192, the analytical model removes 2,097,152 requested intermediate bytes.
Rows up to 8192 columns use a resident one-pass path. Wider rows use a stable online max and sum recurrence in 1024-element chunks, followed by a second pass that normalizes and writes the output. Both paths keep reduction state in FP32 and support masked irregular widths.
Weights use symmetric signed INT4 with one BF16 scale per 128 values along K. Two two's-complement values are packed into each byte in group-major layout. The monolithic kernel decodes, scales, and accumulates four output rows in FP32. A deterministic split-K fallback writes FP32 partials and reduces them in a fixed order without atomics.
See kernel design for the numerical and byte-model contracts.
Every provider for a shape receives the same seeded inputs and must pass the same reference check before timing starts.
- K1 eager and compiled providers execute the same low-precision residual rounding semantics as Triton.
- K2 eager and compiled providers call row-wise PyTorch softmax with explicit FP32 computation and one cast to the input dtype. The reference computes in FP64 before that final cast.
- K3 dense providers use a BF16 matrix dequantized from the same packed values
and scales as the INT4 providers. Dense eager uses
torch.mv; compiled lanes compile that same operation. The unfused INT4 lane materializes BF16 weights beforetorch.mv. - Compiled results include
torch.compilewith Inductor,mode="max-autotune", plus a sensitivity lane usingmode="max-autotune-no-cudagraphs". - Compilation and Triton autotuning finish before measured samples.
These values are recorded in
ceilings.json and the
Nsight compatibility record.
| Component | Recorded value |
|---|---|
| GPU | NVIDIA RTX PRO 3000 Blackwell Generation Laptop GPU |
| Architecture | Compute capability 12.0 (sm_120), 46 SMs |
| Device memory | 12,227 MiB reported by nvidia-smi; 12,820,480,000 bytes reported by PyTorch |
| Memory terminology | GDDR7 device DRAM |
| Driver | 595.79 |
| Host | WSL2, Linux 6.6.114.1-microsoft-standard-WSL2, x86_64 |
| Python | 3.12.13 |
| PyTorch | 2.12.0+cu130, package version 2.12.0 |
| Torch-bundled CUDA | 13.0 |
| cuDNN | Runtime-reported integer 92000 |
| Triton | 3.7.0 |
| Nsight Compute | 2026.2.1.0, build 38283040 |
The benchmark environment did not find nvcc or ncu on PATH. Profiling
later used the explicit executable
/opt/nvidia/nsight-compute/2026.2.1/ncu.
The accepted suite uses BF16 inputs, seed 2026, 10 warmups, and 100 timed samples per provider and shape. Providers are interleaved with a rotating order to reduce fixed ordering bias. Each sample uses CUDA events and stores the raw latency plus median, p10, p90, MAD, minimum, and maximum.
Correctness is checked before warmup or timing:
abs(actual - reference) <= atol + rtol * abs(reference)
The harness also records maximum absolute error and thresholded relative error so values near zero do not produce meaningless relative-error claims.
| Kernel | BF16 atol |
BF16 rtol |
Accepted artifacts |
|---|---|---|---|
| K1 RMSNorm plus residual | 0.02 | 0.02 | 48 of 48 passed |
| K2 softmax | 0.004 | 0.004 | 48 of 48 passed |
| K3 INT4 GEMV | 0.5 | 0.02 | 24 of 24 passed |
Cold is the headline lane. Before each timed provider call, the accepted run touches a 256 MiB device buffer outside the event interval. Warm is a diagnostic lane with no flush hook. These labels describe harness policy; they do not prove that every transaction missed or hit a particular cache.
More detail is in benchmark methodology and artifacts and data model.
All latencies below are timing.statistics.median_ms. Speedup is the lowest
same-shape PyTorch median divided by the Triton median, so values above 1 mean
Triton was faster. "No graphs" is the compiled sensitivity lane. The
evidence-gated analysis carries the full interpretation and
claim record.
| Batch x hidden | Eager ms | Compiled ms | No graphs ms | Triton ms | Triton vs best PyTorch |
|---|---|---|---|---|---|
| 1 x 1024 | 0.14656 | 0.01331 | 0.00778 | 0.00794 | 0.98x |
| 8 x 1024 | 0.14704 | 0.01834 | 0.00960 | 0.00952 | 1.01x |
| 16 x 2048 | 0.17282 | 0.02208 | 0.01019 | 0.00979 | 1.04x |
| 32 x 4096 | 0.16517 | 0.02883 | 0.01206 | 0.00994 | 1.21x |
| 64 x 4096 | 0.17998 | 0.03094 | 0.01382 | 0.01040 | 1.33x |
| 64 x 8192 | 0.17150 | 0.03693 | 0.01814 | 0.01197 | 1.52x |
At the largest shape, Triton was 14.33x faster than eager and 1.52x faster than the best PyTorch lane. The byte saving is analytical requested traffic, not a physical DRAM claim.
| Rows x columns | Path | Eager ms | Compiled ms | No graphs ms | Triton ms | Triton vs best PyTorch |
|---|---|---|---|---|---|---|
| 128 x 1024 | Resident | 0.02614 | 0.03488 | 0.01022 | 0.01003 | 1.02x |
| 64 x 4096 | Resident | 0.03011 | 0.03099 | 0.01406 | 0.01030 | 1.36x |
| 32 x 8192 | Resident | 0.02837 | 0.03294 | 0.01603 | 0.01181 | 1.36x |
| 16 x 8193 | Online | 0.02797 | 0.03347 | 0.01682 | 0.02062 | 0.82x |
| 8 x 16384 | Online | 0.03040 | 0.03941 | 0.02235 | 0.02656 | 0.84x |
| 4 x 32768 | Online | 0.03824 | 0.05200 | 0.03486 | 0.04757 | 0.73x |
The resident path won at all three tested widths. The online path remained correct at 8193 and larger columns, but its second input pass did not beat the best PyTorch lane in these shapes.
| Square shape | Dense eager ms | Dense compiled ms | No graphs ms | Unfused INT4 ms | Triton ms | Split-K ms | Triton vs best PyTorch |
|---|---|---|---|---|---|---|---|
| 4096 | 0.12395 | 0.26802 | 0.10856 | 2.61342 | 0.08930 | 0.09755 | 1.22x |
| 8192 | 0.40390 | 1.11203 | 0.40763 | 11.08125 | 0.27590 | 0.29646 | 1.46x |
The monolithic kernel beat the best PyTorch lane at both shapes and beat the deterministic split-K fallback by 1.09x at 4096 and 1.07x at 8192. It was 29.27x and 40.16x faster than materializing BF16 weights in the unfused INT4 lane.
The artifacts model 33,570,816 dense compulsory bytes versus 8,667,136 fused INT4 bytes at 4096, a 3.873346x reduction. At 8192 the corresponding values are 134,250,496 and 34,635,776 bytes, a 3.876064x reduction. These ratios include weights, scales, input, and output. They are not measured DRAM ratios.
Warpline calibrates candidates, selects on calibration data, then reports the median and range of seven independent confirmation-run medians.
| Ceiling | Median | Seven-run minimum | Seven-run maximum | Selected candidate |
|---|---|---|---|---|
| Measured GDDR7 device-memory bandwidth | 381.53 GB/s | 345.02 GB/s | 384.27 GB/s | 512 MiB payload, block 512, 8 warps |
Measured BF16 torch.mm tensor throughput |
26.49 TFLOP/s | 23.53 TFLOP/s | 29.64 TFLOP/s | 4096 x 4096 x 4096 |
The median BF16 ridge point is 69.4232 FLOP per requested byte. K2 does not use that compute coordinate. Its work model counts scalar operations, including exponentials, comparisons, and divisions. The BF16 tensor roof is device context only, not a scalar, reduction, reciprocal-square-root, exponential, or SFU roof.
| Largest cold Triton point | Intensity | Useful throughput | Memory branch | Model-relative fraction |
|---|---|---|---|---|
| K1, 64 x 8192 | 0.6250 FLOP/requested byte | 0.2190 TFLOP/s | 0.2385 TFLOP/s | 91.9% |
| K2, 4 x 32768 | 1.1672 ops/requested byte | 0.0193 TOP/s | 0.4453 TOP/s | 4.3% |
| K3, 8192 square | 3.8751 FLOP/requested byte | 0.4865 TFLOP/s | 1.4785 TFLOP/s | 32.9% |
These fractions use each provider's analytical requested-byte convention. They are not measured DRAM-bandwidth utilization.
See roofline methodology for the ceiling selection and work conventions.
Nsight Compute 2026.2.1 profiled one exact function launch per logical kernel
with kernel replay, cache control all, and clock control none.
| Profile target | Occupancy | Overall throughput | DRAM throughput | L2 throughput | Observed SM clock | Leading warp stall |
|---|---|---|---|---|---|---|
K1 _rmsnorm_residual_kernel |
8.26% | 4.30% | 4.30% | 1.30% | 2.7645 GHz | Long scoreboard, 67.45% |
K2 _online_softmax_kernel |
8.05% | 0.87% | 0.87% | 0.41% | 2.7741 GHz | Long scoreboard, 36.27% |
K3 _int4_gemv_kernel |
86.95% | 73.50% | 32.54% | 17.82% | 1.3908 GHz | Long scoreboard, 49.26% |
The captured targets were the K1 2 x 8192 FP16 test, the K2 2 x 16381 FP16 online test, and the K3 4096-square BF16 test. K3 reported 8,663,296 DRAM read bytes. Its analytical read-side model is 8,658,944 bytes, so the observed value was 0.05% higher for that launch. No dense K3 profile was captured, so an observed dense-to-INT4 DRAM ratio is not available. See the profiling workflow for metric discovery, capture policy, commands, and interpretation boundaries.
The benchmark artifacts record clocks.locked: null with the note
not probed by the environment snapshot. No clock lock is claimed. The
largest-shape cold snapshots alone show different states: K1 recorded P4 at
1357 MHz SM, K2 recorded P4 at 1350 MHz, and K3 recorded P1 at 2190 MHz. Nsight
used --clock-control none.
Telemetry is a point-in-time snapshot, not a continuous trace. Laptop power, temperature, and performance state can change between shapes. This also means the warm lane is a cache diagnostic, not a controlled cache-only comparison.
The run environment records branch main, no resolved commit, and a dirty
initial tree. The manifest and checksums identify the artifacts, but the run
cannot be attributed to a committed source revision.
Prerequisites are WSL2, a Windows NVIDIA driver that exposes CUDA to WSL, and
uv. From the repository root:
bash scripts/setup_wsl.shThe script verifies WSL2, installs Python 3.12 through uv, syncs uv.lock
without updating it, and runs eager CUDA, Triton, and torch.compile preflight
checks for sm_120.
Equivalent manual commands:
uv python install 3.12
uv sync --locked
uv run --no-sync python scripts/preflight.py \
--require-wsl \
--require-gpu \
--require-sm120For an optional Windows-host clock probe and lock attempt, run the suite through the PowerShell wrapper. Omit the clock arguments to probe support and record telemetry without requesting a lock:
.\scripts\run_locked.ps1 `
-GraphicsClockMHz <supported-sm-clock> `
-MemoryClockMHz <supported-memory-clock> `
-WslCommand "uv run --no-sync python scripts/run_roofline.py suite --run-id <new-run-id> --config configs/roofline.json --results-root results --repository-root ." `
-Output ".preflight/<new-run-id>-host-clocks.json"The wrapper records every host command and its exit code, always attempts to
reset each requested clock in finally, and reports lock_state: false when a
request fails. It never substitutes a synthetic clock value.
Run the full ceiling and benchmark suite with a new ID:
uv run --no-sync python scripts/run_roofline.py suite \
--run-id <new-run-id> \
--config configs/roofline.json \
--results-root results \
--repository-root .
uv run --no-sync python scripts/validate_roofline_results.py \
results/<new-run-id>Regenerate plots without a GPU rerun:
uv run --no-sync python scripts/run_roofline.py plot-run \
--run-directory results/<run-id>Run tests and static checks:
uv run --no-sync pytest tests/cpu
uv run --no-sync pytest tests/unit
uv run --no-sync pytest tests/gpu
uv run --no-sync ruff check .
uv run --no-sync ruff format --check .Reproduce the three profile targets with a new profile ID:
export NCU_BIN=/opt/nvidia/nsight-compute/2026.2.1/ncu
bash scripts/profile.sh \
--run-id <new-profile-run-id> \
--kernel rmsnorm_residual \
--kernel-filter _rmsnorm_residual_kernel \
-- .venv/bin/python -m pytest \
tests/gpu/test_rmsnorm_residual.py -k 8192 -q
bash scripts/profile.sh \
--run-id <new-profile-run-id> \
--kernel softmax \
--kernel-filter _online_softmax_kernel \
-- .venv/bin/python -m pytest \
tests/gpu/test_softmax.py -k shape3 -q
bash scripts/profile.sh \
--run-id <new-profile-run-id> \
--kernel int4_gemv \
--kernel-filter _int4_gemv_kernel \
-- .venv/bin/python -m pytest \
tests/gpu/test_int4_gemv.py -k primary_4096 -qwarpline/kernels/: Triton kernels and public work models.warpline/baselines/: references, eager providers, and compiled providers.warpline/bench/: correctness gates, provider preparation, CUDA-event timing, and statistics.warpline/roofline/: ceiling measurement, point adapters, CLI, and plotting.warpline/quantization.py: signed INT4 packing, dequantization, and byte models.warpline/artifacts.pyandwarpline/environment.py: schema validation, atomic writes, provenance, and environment capture.configs/roofline.json: accepted suite coverage and measurement counts.scripts/: setup, preflight, suite, validation, plotting, and profiling entry points.tests/: CPU harness tests, quantization unit tests, and GPU kernel tests.results/: benchmark, ceiling, and roofline artifacts.profiles/: Nsight reports, extracted summaries, commands, and checksums.docs/: design and methodology documentation.
GitHub Actions runs on Ubuntu 24.04 for pushes and pull requests. It installs
Python 3.12 with uv, grants read-only repository contents permission, and
runs uv lock --check before installing the project and development group with
uv sync --locked --group dev. It then runs:
ruff check .ruff format --check .pytest tests/cpupytest tests/unitpytest tests/gpu, which must skip cleanly without CUDA- the checked-in roofline validator and profile checksum verification
python scripts/preflight.py --cpu-only
Hosted CI does not run GPU kernels or Nsight profiling. It also does not enforce Markdown style or link checks.
- Results apply to the recorded RTX PRO 3000 laptop GPU, driver, WSL kernel, software stack, shapes, BF16 benchmark dtype, and observed power states.
- Clock-lock state was not probed, no lock is claimed, and telemetry was not continuously monitored.
- Cold and warm cache labels are harness policies, not cache hit-rate measurements.
- Analytical requested bytes are not physical DRAM traffic. Only matching profiler counters can support a physical traffic statement.
- The BF16 matrix-multiply ceiling is not an applicable compute roof for these scalar, reduction, and SFU-heavy kernels.
- Nsight profiles cover one function launch per kernel, not every provider, shape, or K2 and K3 variant.
- The run has no source commit and records a dirty tree.
- No end-to-end model latency, model quality, production readiness, or cross-GPU generalization is claimed.
The complete claim policy is in limitations and claims.
- Documentation index: task-oriented entry points and evidence order.
- Architecture: package boundaries and control flow.
- Kernel design: numerical contracts and work models.
- Benchmark methodology: providers, correctness, timing, cache lanes, and statistics.
- Artifacts and data model: schemas, provenance, hashes, and lineage.
- Environment and reproducibility: WSL2 setup, preflight, and replay checks.
- Roofline methodology: measured ceilings, intensity conventions, and plotting policy.
- Profiling workflow: Nsight metric discovery, capture, extraction, and measured summaries.
- Testing and CI: CPU, unit, GPU, and hosted workflow scope.
- Limitations and claims: publication rules and evidence boundaries.
- Evidence-gated analysis: complete artifact-derived interpretation and final claim record.
Apache-2.0. See LICENSE.