Emitter & bootstrapping approaches

[bryjen]

Find a suitable approach for generating some sort of output representation that supports native GPU/NPU targeting, without needing changes in language syntax. Plan to enable acceleration backend via CLI flags.

[bryjen]

Approach as of right now is to emit normal MLIR for CPU bound code, and emit TOSA MLIR for hardware accelerated (HA) code. HA code will most likely be separate to CPU code and will function analogous to compute shaders, having CPU code call and pass parameters to HA code that runs on a separate runtime.

Although support for GPUs and NPUs would be nice, due to the lack of infrastructure on NPUs - compared to GPUs - focusing on finding a solution for GPU accelerated code would be ideal. Although not completely finalized right now, a potential approach would be to lower to SPIR-V, bootstrap the Vulkan runtime, register the emitted IR as a compute shader (somehow), then dispatch that compute shader via a function call in CPU code.

Furthermore, it would be nice to have support for backend selection. For example, if we detect that the user does not have a backend (ex. no GPU or NPU), we could lower everything to CPU code (LLVM) and have it function as any normal application.