fix: accept OpenAI-chat shape under /gemini and /openai prefixes (#1)#3
Merged
Merged
Conversation
OpenAI-chat clients (e.g. Hermes) build their request URL by appending
/chat/completions onto base_url, and probe /v1/models for discovery. A
user pointing such a client at the /gemini base therefore called
/gemini/chat/completions, which had no handler under the native
generateContent route prefix and returned 404 {"detail":"Not Found"}.
Mount the OpenAI-compat handler and the model-discovery endpoints under
the /gemini and /openai prefixes as well as the bare root. Gemini already
routes through Vertex's OpenAI-compat endpoint inside _handle_openai
(keyed off the request body's model), so the URL prefix the client
happens to use no longer matters.
Also corrects the README recipe and Hermes example, which previously
recommended the broken /gemini + openai_chat pairing.
Fixes #1
4b36107 to
a6ed129
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Reported in #1: an OpenAI-chat client (Hermes, or any client using an
openai_chattransport) configured with a/geminibase URL fails with:even though
curlagainst the native Gemini route works fine.Root cause
OpenAI-style clients build their final request URL by appending a fixed suffix onto
base_url:/chat/completionsfor inference/models,/v1/models,/v1/models/{id}for model discoverySo a
base_urlofhttp://127.0.0.1:8787/geminimakes the client callPOST /gemini/chat/completions. But/geminiis the proxy's nativegenerateContentroute prefix, which has nochat/completionshandler. The OpenAI-compat handler was only mounted at/openai/v1/chat/completions,/v1/chat/completions, and/chat/completions. The proxy log in the issue shows exactly this: every/gemini/chat/completionsPOST and every/gemini/.../modelsdiscovery probe returning 404.curlworked because it used the native shape (/gemini/v1beta/models/...:generateContent); the OpenAI-chat client uses a different shape against the same prefix.Fix
Mount the OpenAI-compat chat handler and the model-discovery endpoints under the
/geminiand/openaiprefixes as well as the bare root. Gemini already routes through Vertex's OpenAI-compat endpoint inside_handle_openai(which dispatches by the request body'smodel, not the URL prefix), so it does not matter which prefix the client appends to.Newly accepted shapes:
POST/gemini/chat/completions,/gemini/v1/chat/completions,/openai/chat/completionsGET/models,/openai/v1/models,/openai/models,/gemini/v1/models,/gemini/models,/api/v1/models, plus/{openai,gemini}/v1/models/{id}probesThese join the existing
/{,openai/}v1/chat/completions,/chat/completions, and/v1/modelsroutes, matching the project's existing "accept client URL variants" approach.Docs
examples/hermes-config-snippet.yamlpreviously recommended the broken/gemini+openai_chatpairing. Updated to point theopenai_chatGemini provider at/openai(clearest match for the shape) and note that/geminiand the bare root now work too.404 {'detail': 'Not Found'}symptom.Tests
test_gemini_prefix_accepts_openai_chat_shape: regression for Not able to use proxy for gemini-2.5-flash #1. The three new chat paths return 200 and forward to Vertex's OpenAI-compat endpoint withmodel: google/gemini-2.5-flash.test_model_discovery_under_client_prefixes: discovery probes resolve under each client prefix; unknown-model probe still 404s.Full suite: 22 passed,
ruff check+ruff format --checkclean.Closes #1