Thank you for creating this plugin. It looks promising but I've not been able to get started with it yet.
I have Qwen operating in 760K context on llamacpp. Unfortunately I can't see any way to tell your plugin that actually I can run past 128K (which assumes I'm on LMStudio). generic-openai works even worse with hardcoded 32K limit.
I'm exposing /props /slots endpoints that could be queried for accurate readings but in the absence of that, I'm also happy to just provide these values myself. I understand that an average user isn't going to know these details, but since you have an "Advanced" section, perhaps an override can be declared there.
I'd also like to avoid putting any sort of limits on my LLM (they're controlled on server side already), which again can be toggled through an Advanced box to say, only send in raw inference request and ignore sitting any reasoning effort or max token settings.
Secondly, for RAG, i have the same problem effectively - I can't provide you with an embedding URL to use. I already have Jina v5 and LaSER Qwen3 8b embedding models that will happily provide vectoring support if only I could offer the embedding endpoint (also hosted by a second llamacpp instance).
Bonus: I'm not sure how you're handling RAG storage in background, but perhaps a future feature could take in some of the established providers for storage (for example, i have Qdrant as my vector store available). This one is just an idea :)
Bonus idea 2: since you're using embeddings, you could also optionally take a reranking model on another advanced url section that helps rerank semantic searches within the context.
Thank you for creating this plugin. It looks promising but I've not been able to get started with it yet.
I have Qwen operating in 760K context on llamacpp. Unfortunately I can't see any way to tell your plugin that actually I can run past 128K (which assumes I'm on LMStudio). generic-openai works even worse with hardcoded 32K limit.
I'm exposing /props /slots endpoints that could be queried for accurate readings but in the absence of that, I'm also happy to just provide these values myself. I understand that an average user isn't going to know these details, but since you have an "Advanced" section, perhaps an override can be declared there.
I'd also like to avoid putting any sort of limits on my LLM (they're controlled on server side already), which again can be toggled through an Advanced box to say, only send in raw inference request and ignore sitting any reasoning effort or max token settings.
Secondly, for RAG, i have the same problem effectively - I can't provide you with an embedding URL to use. I already have Jina v5 and LaSER Qwen3 8b embedding models that will happily provide vectoring support if only I could offer the embedding endpoint (also hosted by a second llamacpp instance).
Bonus: I'm not sure how you're handling RAG storage in background, but perhaps a future feature could take in some of the established providers for storage (for example, i have Qdrant as my vector store available). This one is just an idea :)
Bonus idea 2: since you're using embeddings, you could also optionally take a reranking model on another advanced url section that helps rerank semantic searches within the context.