-
Notifications
You must be signed in to change notification settings - Fork 621
Description
Is your feature request related to a problem? Please describe.
Currently, Genkit (Go SDK) does not have built-in support for Cerebras, an AI provider known for its extremely fast inference speeds. Developers who want to use Cerebras models (like llama3.1-8b, llama3.1-70b) have to manually configure the generic OpenAI client or write their own implementation, which is not the best developer experience.
Describe the solution you'd like
I would like to implement a dedicated plugin for Cerebras in the Go SDK.
Since Cerebras is OpenAI-compatible, I plan to leverage the existing compat_oai package to create a new plugin at go/plugins/cerebras.
The plugin will:
- Set the correct Base URL for Cerebras (
https://api.cerebras.ai/v1). - Provide a predefined list of supported models (e.g., Llama 3.1 variants).
- Simplify the configuration process for users.
I am willing to submit a PR for this implementation.
Describe alternatives you've considered
The alternative is using the generic OpenAI plugin and manually overriding the baseUrl and model names. However, having a first-party provider package (genkitx-cerebras or similar) provides better type safety, auto-completion for models, and a smoother "out-of-the-box" experience for Genkit users.
Additional context
Cerebras API documentation: https://inference-docs.cerebras.ai/
I have checked the codebase and go/plugins/compat_oai seems perfect for this integration.
Metadata
Metadata
Assignees
Labels
Type
Projects
Status