feat: add OrcaRouter provider with API-key and OAuth 2.0 PKCE login - #129
kuswardhanietidims-svg wants to merge 2 commits into
Conversation
Signed-off-by: kuswardhanietidims-svg <kuswardhanietidims-svg@users.noreply.github.com>
HenryNdubuaku
left a comment
There was a problem hiding this comment.
Happy to take OrcaRouter in. Two things before it can merge: README, doc/finetuning.md and llms.txt conflict with main, so please rebase; and tests/test_gui_evidence.py imports playwright at module level, which errors out the whole collection on a machine without it (pytest.importorskip("playwright") fixes that). Everything else passed locally on top of main, 231 tests.
Rebased onto `main` (fc5bae0): upstream removed the whole `doc/` tree (`doc/finetuning.md` is gone) and rewrote `README.md` and `llms.txt`, so the OrcaRouter documentation is re-applied against the new structure rather than resurrecting `doc/finetuning.md`. `main` also renamed the OpenRouter default model, which the new provider registry had reverted; that default now matches main again. `tests/test_gui_evidence.py` imported `playwright.sync_api`, `PIL` and `numpy` at module scope, so a checkout without them turned the whole pytest collection into an error instead of skipping the one test that needs a browser. The three imports now go through `pytest.importorskip`, matching how `tests/test_packaging.py` already guards an optional import, and the module imports `pytest` explicitly instead of relying on injection. The evidence test still runs wherever the browser stack is present; only a machine that cannot run it is skipped. Signed-off-by: kuswardhanietidims-svg <kuswardhanietidims-svg@users.noreply.github.com>
|
Both points from your review are addressed on Rebase. The branch is rebased onto current While resolving that, I found a real regression the rebase had hidden: playwright. Adds OrcaRouter as a first-class provider for Needle's tool-calling data synthesis, with two explicit ways to authenticate and a model selector driven by the gateway's own catalog. OrcaRouter is an OpenAI-compatible AI gateway that routes many providers behind one endpoint. I'm an engineer on the OrcaRouter team. What this changes
Two files carry the bulk of it:
AI input surfaces covered
The playground is the only renderable surface; there is no second model picker to keep in sync, and the CLI shares Two authentication methods
API key. A pasted OAuth 2.0 + PKCE. PKCE details:
Credential lifecycleThe exchange returns a durable OrcaRouter API key, not a refresh token, so there is no refresh grant and no re-authorisation on start: the stored key is reused until it is revoked. It is written to the project's existing cache directory ( A 401 from the relay calls Playground connect state is generation-guarded; Catalog and capability filteringThe catalog is
Changing the provider, the modality toggle, the attachment type or the task recomputes the options; a selected model that is no longer compatible is cleared with a prompt to re-pick rather than silently kept. Live discovery is authoritative, and the seed is not merged into a successful live result. When discovery fails, a last-known-good list is reused and reported as degraded; if there is none, the verified cold-start seed is used and labelled as such. TestingRe-run on this head ( The seven live tests go through the implemented provider path — Full suite on this head: 230 passed, 6 skipped, 8 failed, 15 errors. Every failing node is a missing optional dependency in this sandbox (
Compliance greps run before committing:
Why the diff also touches
|
Adds OrcaRouter as a first-class provider for Needle's
tool-calling data synthesis, with two explicit ways to authenticate and a model
selector driven by the gateway's own catalog.
OrcaRouter is an OpenAI-compatible AI gateway built for both models and agents, with adaptive routing,
automatic failover, zero-markup inference, observability, guardrails, and agent-tool governance.
It also runs gateway-level, zero-trust security for AI agents on the same endpoint — screening every
prompt/response and governing every tool call on a default-deny basis, with no application code changes.
Discord: discord.gg/YEubt8enRA · X: https://x.com/OrcaRouter
I'm an engineer on the OrcaRouter team.
What this changes
needle generate-data --provider orcarouter(and the playground's Dataset tab)synthesise tool-calling examples through OrcaRouter, using OrcaRouter's own
/v1/modelscatalog instead of a free-text model field. Nothing about theexisting OpenRouter path changes:
get_provider(None)still resolves to it, andOPENROUTER_URLkeeps its historical meaning of "the completions URL itself".Two files carry the bulk of it:
needle/model/providers.py— provider registry, origin policy, catalogdiscovery and capability filtering.
needle/model/credentials.py— one credential interface with two adapters.AI input surfaces covered
needle generate-data(CLI synthesis)--provider orcarouter--api-key,ORCAROUTER_API_KEY, or a stored login--model, filtered by--input-modalityneedle connect/needle models--provider--oobselects Flow Bneedle models --capabilitylists the filtered catalog<select>, re-filtered on toggleThe playground is the only renderable surface; there is no second model picker
to keep in sync, and the CLI shares
filter_modelswith it rather thanre-implementing the rules.
Two authentication methods
needle/model/credentials.pyexposes one seam —credential_for(provider, api_key, store)returning aCredentialResult— with two adapters behind it.generate_examples, the catalog fetcher and the model selector consume theresult; none of them can tell which adapter produced it.
API key. A pasted
sk-orca-…key, orORCAROUTER_API_KEY, or a storedrecord.
credential_from_api_keymasks it in every message and log line viamask_secret,store.clear()removes it, and a 401 from the relay surfaces asCredentialErrorwith a message naming the account rather than echoing the key.OAuth 2.0 + PKCE.
needle connectruns Flow A (loopback redirect) witha
127.0.0.1callback;--oobruns Flow B (out-of-band code) for aplayground bound to a non-loopback host where no redirect can be received.
Flow A is the default because the CLI and a loopback-bound playground can both
receive the redirect, which avoids asking the user to copy a code by hand; it is
not available for a remote-bound playground, which is exactly when
--oobisoffered, and the playground's connect dialog picks between them automatically
based on the bind address.
PKCE details:
S256; verifier andstatecome fromsecrets.token_urlsafeper attempt andnever leave the process — they are not in the URL, the logs or telemetry.
base64url(sha256(verifier))with padding stripped.statewithhmac.compare_digestbefore the code is touched.errors all end with an actionable message; none of them retry or hang.
https://www.orcarouter.ai/auth+/api/v1/auth/keys; inference and thecatalog are
https://api.orcarouter.ai/v1. The wrong/v1/auth/keyspathdoes not appear anywhere in the implementation.
ORCA_BASE_URLis the sharedself-hosted origin, with
ORCA_AUTH_BASE_URL/ORCA_API_BASE_URLasexplicit overrides that win over it. HTTPS is enforced for anything remote;
plain HTTP is only accepted for loopback.
scopeis read and checked against what the callerneeds. A requested scope is never treated as a granted one.
Credential lifecycle
The exchange returns a durable OrcaRouter API key, not a refresh token, so
there is no refresh grant and no re-authorisation on start: the stored key is
reused until it is revoked. It is written to the project's existing cache
directory (
~/.cache/cactus-needle/orcarouter/credentials.json, orNEEDLE_CACHE_DIR) with mode0600— no new secret store, no new dependency.A 401 from the relay calls
mark_rejected_generation, which marks exactly theaccount and credential generation that sent the rejected request. An
ORCAROUTER_API_KEYfrom the environment that the store never issued cannotinvalidate a stored record, and a late failure from an older generation cannot
disturb a credential that already replaced it. Nothing is deleted before a new
login succeeds.
Playground connect state is generation-guarded;
pagehideclears the busy flagand hint synchronously and then cancels the server-side task with
keepalive,rather than relying on a
finallythe guard would swallow.Catalog and capability filtering
The catalog is
GET /v1/modelson the configured origin, bounded in time(20 s), bytes (2 MiB) and item count (2000). Filtering is driven only by
supported_endpoint_typesandarchitecture.input_modalities— never by modelname, which is preserved verbatim including the
vendor/modelnamespace:?capability=chat, plus an endpoint type out ofopenai / anthropic / gemini / openai-response, which excludes
image-generation, openai-video and jina-rerank models.
architecture.input_modalitiesmust explicitly contain the modality the surface actually uploads. An
undeclared modality fails closed — it is not shown.
endpoint type.
Changing the provider, the modality toggle, the attachment type or the task
recomputes the options; a selected model that is no longer compatible is
cleared with a prompt to re-pick rather than silently kept.
Live discovery is authoritative, and the seed is not merged into a
successful live result. When discovery fails, a last-known-good list is reused
and reported as degraded; if there is none, the verified cold-start seed is used
and labelled as such.
SEED_CATALOGinneedle/model/providers.pyis the onlyplace in the tree carrying model ids: five entries with endpoint types, context
lengths and input modalities, plus the low/medium/high/xhigh reasoning ladder
for
openai/gpt-5.5, so a fallback entry filters exactly like a discovered one.A restored model id is re-checked against the compatible list before it is
reselected.
Real results
Run from a checkout of this branch on 2026-09-17, with
ORCAROUTER_API_KEYset:Those seven go through the implemented provider path —
providers.discover_modelsand
finetune._openrouter— against the live endpoints, not through a separatecurl. The catalog returned 16 chat-capable models and 2 image-generationmodels for this workspace, and a real chat completion came back through
chat_completions_url.tests/test_gui_evidence.pydrives the real playgroundin Chromium, expands the selector against that same live catalog and writes
orca-evidence/manifest.jsonwithcatalog_model_count: 16,image_model_count: 2.Focused unit/integration, with no network and only fake credentials:
Why the diff also touches
needle/model/tokenizer.pyIndependent verification of this change installed the package on a minimal
environment and hit
ModuleNotFoundError: No module named 'sentencepiece'whilecollecting
tests/test_orcarouter_integration.py— it importsneedle.model.finetune, which importsneedle.model.tokenizer, which importedsentencepieceat module scope.sentencepieceis declared under thetrainextra (
pyproject.toml), while the marker and id constants in that module arealso pulled in by inference, export, rendering and dataset-synthesis paths. The
import is now soft and
SANTokenizer.__init__raises aRuntimeErrornamingcactus-needle[train], so a runtime-only install can still reach the providerand catalog code.
tests/test_tokenizer.pycovers both halves. This is aprerequisite, not scope creep: without it the OrcaRouter tests cannot even be
collected on
pip install -e ..Testing
tests/test_credentials.py— save/read/clear, masking, update, missing-key errortests/test_credentials.py— verifier/challenge/state, S256, unpadded base64url, authorize URL shapetests/test_credentials.py— correct path and body against a local fake auth server, success persists, denial, Flow A state mismatch, reused and expired code, scope downgradetests/test_credentials.py— corrupt key classified terminal, 401 marks only the exact account/generation, 429 and network failure are retryabletests/test_providers.py— registry, alias, self-hosted base, auth/API overrides winning, HTTPS enforcement, HTTP allowed on loopback onlytests/test_providers.py— parse, bounds, degraded fallback, live result authoritative and never merged with the seedtests/test_providers.py— fixtures for text-only, image-input chat, embedding, image generation, video and rerank; multimodal fail-closedtests/test_orcarouter_integration.py— the real URL, headers and body fromgenerate_examples->_openrouter, with a stubbed transporttests/test_playground_providers.py— server-side provider/model routes, key never leaves the backend, generation guard, pagehide cleanuptests/test_gui_evidence.py— real Chromium run, screenshots, measured panel geometry and catalog-driven listCompliance greps run before committing:
git grep -n "v1/auth/keys" -- needle/matches only the docstring inproviders.py:260that warns against that exact path.git grep -niE "client_secret|code_verifier=\"|sk-orca-[0-9a-f]{16}" -- needle/is empty — no client secret, no fixed verifier, no real key in the tree.
git grep -nE "gpt-5\.5|claude-opus-4\.8|gemini-3\.5-flash|deepseek-v4-pro|orcarouter/auto"matches only
SEED_CATALOG/SEED_MODEL_IDSand the two references to theseed in the provider definition.
Full-suite note:
tests/test_render.pyandtests/test_run.pyhave 3 errors and1 failure in this sandbox, all of them
RuntimeError: No pretraining tokenizer … HF download failed (SOCKS proxy …). They reproduce identically on the unmodifiedbase commit with
needle/model/tokenizer.pychecked out from base, so they areenvironmental (the Hugging Face tokenizer cannot be downloaded from here), not
caused by this change.
Evidence and verification
tests/test_credentials.py, through the real connect adapter.tests/test_orcarouter_live.py, through the implemented providertests/test_credentials.pywith a stubbed 401; no real key was revokedorca-evidence/auth-methods.png,orca-evidence/text-model-dropdown.png,orca-evidence/multimodal-model-dropdown.png(generated bytests/test_gui_evidence.py, manifest with sha256 and measured UI assertions alongside)unsponsored_surface-style gate, a maintainer has to review the security boundary and apply the sponsorship label — please review the exact head commit rather than the branch tip at any earlier time.Not implemented
Flow C (device grant) is not wired up; the spec allows it as an extra but it
cannot replace PKCE, and
--oobalready covers the no-redirect case for thisrepository's two surfaces. No new dependency was added for SHA-256 or base64url.