[rocWMMA] GFX Targets not supported

[bsyrowik]

The following GFX architectures are not currently supported by the rocWMMA library:

• gfx906 (incompatible)
• gfx101X (incompatible)
• gfx103X (incompatible)
• gfx1103 (added in Added initial support for gfx1103 rocm-libraries#2301; availability expected in a future ROCm release)
• gfx1152 (added in [rocWMMA] Add gfx1152 and gfx1153 rocm-libraries#2850; availability expected in ROCm 7.11)
• gfx1153 (added in [rocWMMA] Add gfx1152 and gfx1153 rocm-libraries#2850; availability expected in ROCm 7.11)

The gfx906 (MI50), gfx101X, and gfx103X devices are fundamentally incompatible with rocWMMA, and as such will not be supported. Support for gfx1103 may be added some time in the future.

[oscarbg]

curious, I assume gfx115x and gfx120x are already supported right?

[bsyrowik]

curious, I assume gfx115x and gfx120x are already supported right?

Yes, that is correct.

[Djip007]

gfx1103 was added a few day ago: https://github.com/ROCm/TheRock/blob/main/RELEASES.md#rocm-for-gfx110x-all

(for now there is a probleme with pytorch distribution)

[edit]: if most rocm lib look to work , rocWMMA still not usable with gfx1103 (at least not with llama.cpp)...)

[it1984-tea]

Hi @bsyrowik

Firstly, nice work with the team on therock, it's shaping up nicely.

Perhaps you are already aware, a massive number of gfx906 GPUs ended up with homelabbers. I'm guessing some datacenter in china offloaded their cards or something and now many people have them. They make excellent inference cards.

MI50s are mentioned almost weekly on local LLM communities online.

I understand what I said above might be irrelevant to AMD, but look at this:
https://www.reddit.com/r/LocalLLaMA/comments/1ns2fbl/for_llamacppggml_amd_mi50s_are_now_universally/

Because these cards are everywhere, users contribute optimizations in projects that indirectly improve performance for other AMD cards. By extending the usefulness/unlocking the full performance of these cards, you make them more relevant = more code contributions. I hope this will nudge the decision in the right direction. Please add rocWMMA support for gfx906.

[bsyrowik]

Hi @bsyrowik

Firstly, nice work with the team on therock, it's shaping up nicely.

Perhaps you are already aware, a massive number of gfx906 GPUs ended up with homelabbers. I'm guessing some datacenter in china offloaded their cards or something and now many people have them. They make excellent inference cards.

MI50s are mentioned almost weekly on local LLM communities online.

I understand what I said above might be irrelevant to AMD, but look at this: https://www.reddit.com/r/LocalLLaMA/comments/1ns2fbl/for_llamacppggml_amd_mi50s_are_now_universally/

Because these cards are everywhere, users contribute optimizations in projects that indirectly improve performance for other AMD cards. By extending the usefulness/unlocking the full performance of these cards, you make them more relevant = more code contributions. I hope this will nudge the decision in the right direction. Please add rocWMMA support for gfx906.

Hi @it1984-tea,
Unfortunately the gfx906 device doesn't support the type of matrix-multiply instructions that rocWMMA targets, so AMD will likely not add support for it in rocWMMA. I will update my initial description to make this more clear. Thank you for your continued interest in ROCm!

[IMbackK]

@it1984-tea to expand that a little more: rocWMMA on gfx906 makes little sense since rocWMMA exists to expose wavefront wide matrix multiplications, which gfx906 dosent have. Now you could simply emulate the behavior of those functions using the valu V_FMA* , V_DOT* instructions gfx906 dose have, but that would end up slower than the valu kernels machine learning frameworks already have so it would be work for a negative result.

AMD often makes terrible support decisions but this is not one of them. I would recommend complaining about the narrow support window for tensilelite/hipblaslt this is a real travesty.

[it1984-tea]

Hi,
I see. Thanks for the clarification.

In any case, it seems to me that things are moving in the right direction, gfx906 was deprecated in ROCm 6.4 but this was reversed and now it's fully supported again with ROCm 7.9/therock, please keep the momentum!

[bsyrowik]

#2650 will add gfx1152/3 to TheRock.

[bsyrowik]

GFX1103 support was integrated into rocWMMA, but is not officially supported yet. Waiting on CI flows in TheRock on gfx1103 before advertising support.