The only option currently is for JIT kernel compilation. This will create issues if known model(s) are used and a compilation environment (e.g., IRON) is not present.
We need a tool that scans a config file and pregenerates the kernels defined in that config file. The kernels can be output in a directory that can be give at run-time using the GGML_HSA_KERNEL_DIR environment variable.
The only option currently is for JIT kernel compilation. This will create issues if known model(s) are used and a compilation environment (e.g., IRON) is not present.
We need a tool that scans a config file and pregenerates the kernels defined in that config file. The kernels can be output in a directory that can be give at run-time using the
GGML_HSA_KERNEL_DIRenvironment variable.