Problem
QMD is currently centered around local llama.cpp model access.
That makes it harder to use QMD with a local or self-hosted OpenAI-compatible server when a user wants to route:
- query expansion / generation
- embeddings
- reranking
through the same remote service.
There are already requests for remote embeddings, but the broader problem is allowing QMD to work with a compatible server across the whole retrieval flow.
Related issues:
Solution
Add support for an OpenAI-compatible backend that can be configured in QMD and used for:
- generation
- embeddings
- reranking
This should allow users to point QMD at a compatible local or self-hosted server, configure model aliases, and run the normal QMD flow without requiring direct local model access.
Reference
PR #619 implements this feature:
#619
Problem
QMD is currently centered around local
llama.cppmodel access.That makes it harder to use QMD with a local or self-hosted OpenAI-compatible server when a user wants to route:
through the same remote service.
There are already requests for remote embeddings, but the broader problem is allowing QMD to work with a compatible server across the whole retrieval flow.
Related issues:
Solution
Add support for an OpenAI-compatible backend that can be configured in QMD and used for:
This should allow users to point QMD at a compatible local or self-hosted server, configure model aliases, and run the normal QMD flow without requiring direct local model access.
Reference
PR #619 implements this feature:
#619