Benchmarking XDG's GPU API

[Waqar-ukaea]

I need a place to store and track these benchmark results, so I am creating this issue.

I previously benchmarked GPRT across several hardware platforms and have now started doing the same for cuBQL. The existing GPRT results could probably do with being refreshed, particularly because the GPU-facing API has been more strictly defined during development of the cuBQL backend.

Benchmark parameters

Model	Volume	No. of Elements	No. of Rays	Location	No. of Runs
simple_tokamak	2	280K	80M	(180, 250, -27)	100

A render of the simple_tokamak model [1] used in these preliminary benchmarks is shown below. The image also depicts a significantly smaller number of the launched rays, with the queried volume highlighted in blue.

Ray tracer performance (trace-only)

Baseline = Embree running on a complete dual-socket CPU node containing 2× Intel Xeon Platinum 8480+ processors and 112 physical cores in total.

The node exposes one hardware thread per physical core:

• 56 physical cores per socket
• 2 sockets
• 112 physical cores in total
• 1 hardware thread per core

Unless otherwise noted, times and throughput are averaged over 100 runs.

Ray Tracer Backend	Hardware	Trace Time (s)	Throughput (ray/s)	Speedup vs 2× 8480+ (112 physical cores)	Peak FP32/FP64 (TFLOPS) + RT Cores	Node Fraction
Embree	2× Intel Xeon Platinum 8480+ (Sapphire Rapids), 112 physical cores	0.189140	4.22967e+08	1× (baseline)	N/A	100%
GPRT (mixed FP64)	NVIDIA RTX 2000 Ada	1.112610	7.19033e+07	~0.17×	FP32: 12.0 FP64: 0.19 RT cores: 22	Not HPC
GPRT (FP32 + RT cores)	NVIDIA RTX 2000 Ada	0.0302956	2.64065e+09	~6.24×	FP32: 12.0 FP64: 0.19 RT cores: 22	Not HPC
cuBQL (FP64)	NVIDIA RTX 2000 Ada	~1.188013*	6.73393e+07	~0.16×	FP32: 12.0 FP64: 0.19 RT cores: 22 (unused)	Not HPC
GPRT (mixed FP64)	NVIDIA L40	0.185930	4.31900e+08	~1.02×	FP32: 90.5 FP64: 1.41 RT cores: 142	25%
GPRT (FP32 + RT cores)	NVIDIA L40	0.008051	9.95400e+09	~23.5×	FP32: 90.5 FP64: 1.41 RT cores: 142	25%
GPRT (mixed FP64)	NVIDIA A100	0.183721	4.50600e+08	~1.07×	FP32: 19.5 FP64: 9.7 RT cores: N/A	25%
cuBQL (FP64)	AMD MI300X	0.075946	1.05338e+09	~2.49×	FP32: 163.4 FP64: 81.7 RT cores: N/A	12.5%

cuBQL results for the NVIDIA A100 and L40 have not yet been collected.

References

[1] Valentine, A., Berry, T., Bradnam, S., Hagues, J., & Hodson, J. (2022). Benchmarking of emergent radiation transport codes for fusion neutronics applications. Fusion Engineering and Design, 180, 113197. https://doi.org/10.1016/j.fusengdes.2022.113197