no local llm support

[jmanhype]

THIS IS NOT OPEN SOURCED PLEASE ALLOW US TO USE 7b models!!!

[Forest-Person]

I would second this issue. Came here to give your framework a shot. If you add local LLM support you will have a tsunami of interested collaborators. Congrats on your progress so far.

[jmlago]

Hey guys,

This is something I had in mind since very early, but seeing the results of gpt-3.5 last version (are not promising), I've been testing 7B models and I'm not sure if these are going to be able to handle the decision making correctly to be able to navigate the simulation, and to format the events in socket payload format.

That being said, as it is a reality that LLMs and open source LLMs would eventually outperform GPT-4, I want to add open source LLM support into the framework. I'm looking at openchat 3.5 but I want to integrate a solution that can somehow evolve with time without much modifications. Are you aware of some library that works as an aggregator or similar?

So essentially, the goal is to find a solution that can adapt and evolve over time with minimal modifications, keeping pace with these rapid advancements. I'm currently exploring options like OpenChat 3.5, but I'm also very open to suggestions from the community. If you know of any libraries or tools that could function as an aggregator or offer similar future-proofing capabilities, please do share.

Your insights and contributions are invaluable as we navigate these exciting yet challenging times in LLM development. I look forward to your suggestions and to collaborating on enhancing the framework.

[ilyalasy]

Hey guys,

This is something I had in mind since very early, but seeing the results of gpt-3.5 last version (are not promising), I've been testing 7B models and I'm not sure if these are going to be able to handle the decision making correctly to be able to navigate the simulation, and to format the events in socket payload format.

That being said, as it is a reality that LLMs and open source LLMs would eventually outperform GPT-4, I want to add open source LLM support into the framework. I'm looking at openchat 3.5 but I want to integrate a solution that can somehow evolve with time without much modifications. Are you aware of some library that works as an aggregator or similar?

So essentially, the goal is to find a solution that can adapt and evolve over time with minimal modifications, keeping pace with these rapid advancements. I'm currently exploring options like OpenChat 3.5, but I'm also very open to suggestions from the community. If you know of any libraries or tools that could function as an aggregator or offer similar future-proofing capabilities, please do share.

Your insights and contributions are invaluable as we navigate these exciting yet challenging times in LLM development. I look forward to your suggestions and to collaborating on enhancing the framework.

LocalAI might be worth to look at.