Llama & Mistral integration

[Teicu]

Hey, thanks for this tool.

I was wondering if it might be possible for this extension to utilise local LLMs such as Ollama or Mistral? They could be really useful as they offer free, offline alternatives to cloud-based models. Running a local model allows you to avoid sending data to cloud services, which can be beneficial for security.

Cheers

[richeeta]

Hey, thanks for this tool.

I was wondering if it might be possible for this extension to utilise local LLMs such as Ollama or Mistral? They could be really useful as they offer free, offline alternatives to cloud-based models. Running a local model allows you to avoid sending data to cloud services, which can be beneficial for security.

Cheers

@Teicu Hey there! I totally agree and love that idea. I don't have sufficient GPU to run LLaMa or Mistral atm (I'm waiting til NVIDIA's CES 2025 before I upgrade), but I would be very happy to collaborate with you on this.

Between LLaMa and Mistral: do you have a preference?

[Teicu]

Hi, I use LM Studio, which offers a wide range of LM models in its catalog. The choice depends on my needs and the situation. You can easily select the model you want to use—it's straightforward.

https://lmstudio.ai/models

[V9Y1nf0S3C]

Hey, thanks for this tool.
I was wondering if it might be possible for this extension to utilise local LLMs such as Ollama or Mistral? They could be really useful as they offer free, offline alternatives to cloud-based models. Running a local model allows you to avoid sending data to cloud services, which can be beneficial for security.
Cheers

@Teicu Hey there! I totally agree and love that idea. I don't have sufficient GPU to run LLaMa or Mistral atm (I'm waiting til NVIDIA's CES 2025 before I upgrade), but I would be very happy to collaborate with you on this.

Between LLaMa and Mistral: do you have a preference?

Hi @richeeta ,

I’m also interested in the ability to integrate local LLM tools with the extension—this would help ensure client data remains secure and allow us to flexibly use the most suitable model available via tools like LM Studio.

Regarding your question about LLaMa and Mistral—both models are excellent in their own ways. Perhaps we could support both to offer broader compatibility and let users decide based on their needs.

I’d be glad to contribute and collaborate on this enhancement. Let me know your thoughts and how you'd prefer to proceed.

Cheers

[richeeta]

Hey, thanks for this tool.
I was wondering if it might be possible for this extension to utilise local LLMs such as Ollama or Mistral? They could be really useful as they offer free, offline alternatives to cloud-based models. Running a local model allows you to avoid sending data to cloud services, which can be beneficial for security.
Cheers

@Teicu Hey there! I totally agree and love that idea. I don't have sufficient GPU to run LLaMa or Mistral atm (I'm waiting til NVIDIA's CES 2025 before I upgrade), but I would be very happy to collaborate with you on this.

Between LLaMa and Mistral: do you have a preference?

Hi @richeeta ,

I’m also interested in the ability to integrate local LLM tools with the extension—this would help ensure client data remains secure and allow us to flexibly use the most suitable model available via tools like LM Studio.

Regarding your question about LLaMa and Mistral—both models are excellent in their own ways. Perhaps we could support both to offer broader compatibility and let users decide based on their needs.

I’d be glad to contribute and collaborate on this enhancement. Let me know your thoughts and how you'd prefer to proceed.

Cheers

Hi @V9Y1nf0S3C:

Thanks so much for your feedback, and I would certainly be open to collaborating with you. Could you email me at richard.im AT owasp.org with what you have in mind? I'd love to explore this with you privately.

Thanks,
Richard

[V9Y1nf0S3C]

Hey Richard,

I just sent you an email with my detailed proposal - sorry for the delay in responding, I was caught up with work and missed your message. Looking forward to hearing your thoughts once you’ve had a chance to review it!

Thanks for your patience and talk soon 🙂