I spent some time with claude and got xla 0.10 working with rocm 7.2 using the r9700 ai toolbox https://github.com/kyuz0/amd-r9700-ai-toolboxes
If anyone wants i had claude make a script to patch it as well as document the issues.
some benchmarks, no idea how these compare to nvda
--- Softmax (f32 / bf16 / f16) ---
1024x1024: f32: 154 µs bf16: 147 µs f16: 148 µs
4096x1024: f32: 208 µs bf16: 168 µs f16: 175 µs
1024x32768: f32: 1.3 ms bf16: 697 µs f16: 689 µs
--- Element-wise (16M elements) — add bandwidth by precision ---
f32 : 270 µs 248.6 GB/s
bf16 : 90 µs 372.8 GB/s
f16 : 93 µs 360.8 GB/s
f8 : 76 µs 220.8 GB/s
fp4 : UNSUPPORTED (invalid numerical type: {:f, 4} (see Nx.Type docs )
int8 : 79 µs 212.4 GB/s
int4 : 67 µs 250.4 GB/s
int1 : UNSUPPORTED (invalid numerical type: {:u, 1} (see Nx.Type docs )
--- Element-wise (f32, 16M elements) ---
exp : 271 µs 247.6 GB/s effective
sqrt : 540 µs 124.3 GB/s effective
add : 269 µs 249.5 GB/s effective
I spent some time with claude and got xla 0.10 working with rocm 7.2 using the r9700 ai toolbox https://github.com/kyuz0/amd-r9700-ai-toolboxes
If anyone wants i had claude make a script to patch it as well as document the issues.
some benchmarks, no idea how these compare to nvda
--- Softmax (f32 / bf16 / f16) --- 1024x1024: f32: 154 µs bf16: 147 µs f16: 148 µs 4096x1024: f32: 208 µs bf16: 168 µs f16: 175 µs 1024x32768: f32: 1.3 ms bf16: 697 µs f16: 689 µs --- Element-wise (16M elements) — add bandwidth by precision --- f32 : 270 µs 248.6 GB/s bf16 : 90 µs 372.8 GB/s f16 : 93 µs 360.8 GB/s f8 : 76 µs 220.8 GB/s fp4 : UNSUPPORTED (invalid numerical type: {:f, 4} (see Nx.Type docs ) int8 : 79 µs 212.4 GB/s int4 : 67 µs 250.4 GB/s int1 : UNSUPPORTED (invalid numerical type: {:u, 1} (see Nx.Type docs ) --- Element-wise (f32, 16M elements) --- exp : 271 µs 247.6 GB/s effective sqrt : 540 µs 124.3 GB/s effective add : 269 µs 249.5 GB/s effective