Support for local LLMs

[spicoflorin]

Hello!

In my opinion, this tool could be very helpful for Data Engineering performing the time consuming such as data cleaning and data preparation.
I have observed that currently supported LLM are the ones from OpenAI. This approach might involve costs from the business perspective.
Therefore I have the following questions:

Is there any plan to support open source LLM as llama?

Thanks, Florin

[zachary62]

Hi Florin,

Yes! The extension would be easy. This is the function for different LLM APIs:

cocoon/cocoon_data/llm.py

Line 108 in 1790375

def call_llm_chat(messages, temperature=0.1, top_p=0.1, use_cache=True):

Do you have any open-source LLMs in mind?

From my experiments, LLMs comparable to GPT-4 are preferable (e.g., Claude 3 and Gemini Ultra would also be good). Additionally, the cost is relatively low even with GPT-4. For example, using Cocoon to clean and profile a table costs ~20 cents.