idea: embeddings should be generated using llama.cpp

[daboe01]

i think that this might speed up indexing of large documents and/or a high volume of documents

[daboe01]

https://generativeai.pub/llama-cpp-one-man-band-embeddings-llm-for-a-full-rag-stack-435be8e05b2b

[daboe01]

ggml-org/llama.cpp#5423

[abgulati]

moving to discussions

Hi @daboe01,
Thanks for bringing this up. In fact, ggreganov had already suggested this to me in a recent chat I had with him about LARS. I've tabled it for now as a topic to explore in the future as it's non-urgent.