jLLM (Java Large Language Models SDK)

About jLLM

Welcome to jLLM, an open-source SDK for Java that makes it easy to integrate large language models (LLMs) into your applications. Whether you're working with OpenAI, Hugging Face, or Groq, jLLM gives you the tools you need to interact with these powerful AI models in a simple and efficient way.

The SDK is designed with flexibility in mind, allowing you to seamlessly send prompts, handle responses, and even stream data in real time. We’ve built it to be easy to extend, so if you have a new LLM provider in mind, adding it is a breeze. With built-in retry policies, rate limit handling, and error management, jLLM helps you focus on building your app rather than managing the complexities of interacting with LLMs.

What Makes jLLM Awesome

• Modular and Extensible: jLLM is all about flexibility. The modular design means you can easily add new providers and extend its functionality as your needs evolve.

• Support for Multiple Providers: Right now, jLLM works with OpenAI, but we’re planning to expand support to other providers like Hugging Face and Groq in the near future.

• Prompt Management Made Easy: Send prompts to LLMs, retrieve their responses, and process them for use in your app—all in a straightforward, easy-to-understand API.

• Real-Time Streaming: If you need live token streaming (e.g., for real-time apps), jLLM has you covered. Stream responses from the model and handle them as they come in.

• Built-in Error Handling & Retries: No one likes dealing with errors, so we’ve made sure jLLM comes with sensible retry policies and error handling out of the box, so you don’t have to.

• Simple Integration: Whether you’re new to Java or a seasoned developer, you’ll find that integrating jLLM into your application is a smooth experience.

How to Get Started

1. Clone the repo:

   git clone https://github.com/NoroSaroyan/JLLM-Connect
   

2. Install the dependencies: Make sure you’ve got the right dependencies in your pom.xml or build.gradle file (like OkHttp, Jackson, etc.).

3. Set up your OpenAI API key: To use the OpenAI provider, you’ll need to grab an API key from OpenAI. Once you’ve got that, just set it up in your app when you initialize the OpenAIProvider.

4. Run the example:

   • Send a prompt to OpenAI and get a response.
   • Try out the streaming feature and get tokens as they’re generated by the model.

Project Structure

• src/main/java/com/jllm/core/provider/: Contains the core interface and the OpenAI provider implementation.
• src/main/java/com/jllm/core/model/: Data models like PromptRequest, PromptResponse, and TokenUsage to handle the LLM interaction.
• src/main/java/com/jllm/core/: Utility classes for retry policies, error handling, and more.
• src/main/resources/: Configuration files, API keys, and other settings.

Contributing

I’d love for fellow devs to contribute! Here’s how you can help the right way:

1. Fork the repo.
2. Create a new branch for your feature or fix.
3. Push your changes.
4. Submit a pull request with a detailed description of your changes.

Feel free to ask questions, open issues, or submit bug reports. We’re all in this together!

License

This project is licensed under the MIT License. See the LICENSE file for details.