LaunchLatency

A small HIP micro-benchmark that measures kernel-launch and graph-launch latency on AMD GPUs. It reports launch overhead three complementary ways:

• enqueue only (wall): CPU cost of issuing the launch, with the stream drained beforehand so it is not inflated by GPU queue backpressure. This is independent of how long the kernel actually runs.
• launch + sync (wall): full host-visible round trip — issue the launch then hipStreamSynchronize. Scales with kernel/graph execution time.
• execution (event): GPU-side elapsed time measured with hipEvent.

Building

CMake (recommended)

cmake -B build                 # uses /opt/rocm and hipcc by default
cmake --build build -j
./build/hip_launch_latency -k -g

Override the ROCm location if needed:

cmake -B build -DROCM_PATH=/opt/rocm-7.0

Direct hipcc

/opt/rocm/bin/hipcc -O3 hip_launch_latency.cpp -o hip_launch_latency

Usage

Usage: hip_launch_latency [options]

Benchmarks:
  -k                 run kernel-launch latency benchmark
  -g                 run graph-launch latency benchmark
                     (give both to run both; at least one is required)

Timing mode (pick any; default is both):
  --wall             measure CPU walltime (chrono around launch + sync)
  --event            measure GPU event elapsed time (hipEvent)

Kernel:
  -t, --timer <us>   use timing kernel that busy-waits <us> microseconds
                     on device (0 = empty kernel, default 0)

Graph:
  -N, --nodes <n>    kernel nodes captured per graph (default 100)

Iterations:
  -i, --iters <n>    iteration count (default 1000); applies to single
                     launches, number of batches, and graph launches
  -b, --batch <n>    kernel launches per batch before one sync (default 100)

Misc:
  -d, --device <id>  device id (default 0)
  -h, --help         show this help

Tests performed

• -k kernel
  • single: one launch followed by a synchronize, repeated --iters times.
  • batch : --batch launches then one synchronize, repeated --iters times.
• -g graph
  • one launch of a graph holding --nodes kernel nodes, followed by a synchronize, repeated --iters times.

The optional timing kernel (-t <us>) busy-waits on the device using wall_clock64() (constant-frequency wall clock, queryable via hipDeviceAttributeWallClockRate), so you can see how launch+sync/event scale with kernel duration while enqueue only stays flat.

Example

$ ./hip_launch_latency -g -t 5 -i 500
========================================
hip_launch_latency  v0.5
Device 0: AMD Instinct MI300X
Fixed wall-clock frequency : 100.000 MHz (100000 kHz)
Peak shader clock          : 2100.000 MHz (2100000 kHz)
----------------------------------------
Timing modes : wall event
Kernel       : timing (5 us busy-wait)
warmup=10  iters=500  nodes/graph=100
========================================

========================================
Graph-launch latency (100 kernel nodes per graph)
========================================
Single graph launch [wall ]:     9.4484 us/launch  (500 iters, enqueue only)
Single graph+sync   [wall ]:   586.8914 us/launch  (500 iters)
Single graph        [event]:   577.9984 us/launch  (500 iters)

Done.

Notes

• Graph results are per graph launch (not divided by node count); kernel batch results are per kernel launch (divided by --batch).
• A fixed warmup (10 iterations) runs before every measured loop.
• --wall/--event only gate the single-launch kernel section; the batch and graph sections always report both wall and event metrics from a single pass.