Proposal: `opengatellm-rs`, an unofficial Rust client for OpenGateLLM

[Adelagric]

Context

There is currently no Rust client — official or community — for OpenGateLLM. Rust consumers of an OGL instance (agent frameworks, internal services, government tools targeting Albert API) today have to either:

• use a generic OpenAI-compatible crate such as async-openai, which does not cover OGL's distinctive endpoints (rerank, OCR, audio, files, collections, search RAG, environmental impact, etc.); or
• write raw reqwest against the OpenAPI spec.

This is friction for Rust adoption around Albert / OGL.

What I built

A first iteration of an unofficial Rust client, scoped to the stable, OpenAI-compatible surface of OGL — to keep the v0.1 small enough to survive the current beta period without churn.

Repository (MIT OR Apache-2.0): https://github.com/Adelagric/opengatellm-rs

Scope v0.1

Endpoint	Method
GET /v1/models	client.models()
GET /v1/models/{id}	client.model(id)
POST /v1/embeddings	client.embeddings(req)
POST /v1/chat/completions (JSON)	client.chat_completion(req)
POST /v1/chat/completions (SSE)	client.chat_completion_stream(req)

Out of scope for v0.1 (explicitly): admin/*, files/*, collections/*, rerank, audio/*, ocr, usage. The intent is to revisit these once the API surface stabilises post-beta.

Design choices

• Compatibility window: OGL >=0.4.5 <0.5.0 (your README warns of breaking changes; I pin to a specific spec version).
• The reference spec/openapi-0.4.5.json is committed and used as ground truth.
• Hand-written types (rather than codegen): progenitor does not yet support OpenAPI 3.1, and openapi-down-convert + progenitor adds friction for the small v0.1 surface. I'll reconsider codegen if scope grows significantly.
• Types of response bodies are #[non_exhaustive] so server-side additions can land without breaking changes.
• Streaming uses eventsource-stream; the data: [DONE] sentinel is detected before deserialization (covered by a dedicated test).
• unwrap_used, expect_used, panic, unsafe_code are all denied in library code.
• Multimodal content (text + image_url) is supported on input via #[serde(untagged)] to stay OpenAI-compatible.
• 20 serde / SSE-parser tests run in CI. 6 integration tests are gated behind OGL_TEST_URL and a compose.test.yml that wires Ollama as the upstream provider — runnable locally via make test-integration.

Quickstart

use opengatellm::{Client, ChatMessage, CreateChatCompletion};

let client = Client::new("http://localhost:8000", Some("TOKEN"))?;
let req = CreateChatCompletion::new(
    vec![ChatMessage::user("Quelle est la capitale de la France ?")],
    "qwen3-coder",
).max_completion_tokens(64);
let resp = client.chat_completion(&req).await?;
println!("{:?}", resp.choices[0].message.content);

Streaming and embeddings examples are runnable under examples/ against any OGL instance via OGL_URL + OGL_TOKEN.

What I would like from the OpenGateLLM team

1. Signal of interest: is a Rust client something you want to see exist around OGL? If not, I'll keep it as a standalone community crate; no harm done.
2. Hosting: if there is interest, would you be open to hosting it under etalab-ia/opengatellm-rs (donation under MIT OR Apache-2.0, same dual-license as the Rust ecosystem norm)? I'd remain the primary maintainer initially.
3. Naming: I'd publish on crates.io as opengatellm (currently unclaimed). Happy to use a different name if you prefer (opengatellm-client, etc.) — I haven't published yet so anything is still movable.
4. API surface feedback: any endpoints you'd consider higher-priority than what I picked for v0.1?

I am holding the crates.io publication until I hear back, to make sure the first published version is one we both agree on.

Notes

• The crate is marked experimental in its README; consumers are told to expect breaking changes during the OGL beta.
• No automatic retry/backoff on 429/5xx — explicitly documented as caller's responsibility for v0.1.
• No telemetry, no callbacks, no implicit logging.

Thanks for OGL and for keeping it open. Happy to adjust any of the above.