Skip to content

feat: add OrcaRouter provider with API-key and OAuth 2.0 PKCE login - #129

Open
kuswardhanietidims-svg wants to merge 2 commits into
cactus-compute:mainfrom
kuswardhanietidims-svg:orcarouter/task-6628
Open

kuswardhanietidims-svg wants to merge 2 commits into
cactus-compute:mainfrom
kuswardhanietidims-svg:orcarouter/task-6628

Conversation

@kuswardhanietidims-svg

Copy link
Copy Markdown

Adds OrcaRouter as a first-class provider for Needle's
tool-calling data synthesis, with two explicit ways to authenticate and a model
selector driven by the gateway's own catalog.

OrcaRouter is an OpenAI-compatible AI gateway built for both models and agents, with adaptive routing,
automatic failover, zero-markup inference, observability, guardrails, and agent-tool governance.
It also runs gateway-level, zero-trust security for AI agents on the same endpoint — screening every
prompt/response and governing every tool call on a default-deny basis, with no application code changes.

Discord: discord.gg/YEubt8enRA · X: https://x.com/OrcaRouter

I'm an engineer on the OrcaRouter team.

What this changes

needle generate-data --provider orcarouter (and the playground's Dataset tab)
synthesise tool-calling examples through OrcaRouter, using OrcaRouter's own
/v1/models catalog instead of a free-text model field. Nothing about the
existing OpenRouter path changes: get_provider(None) still resolves to it, and
OPENROUTER_URL keeps its historical meaning of "the completions URL itself".

Two files carry the bulk of it:

  • needle/model/providers.py — provider registry, origin policy, catalog
    discovery and capability filtering.
  • needle/model/credentials.py — one credential interface with two adapters.

AI input surfaces covered

Surface Provider selection Credential entry Model choice
needle generate-data (CLI synthesis) --provider orcarouter --api-key, ORCAROUTER_API_KEY, or a stored login --model, filtered by --input-modality
needle connect / needle models --provider --oob selects Flow B needle models --capability lists the filtered catalog
Playground → Dataset tab provider dropdown both methods side by side catalog-driven <select>, re-filtered on toggle

The playground is the only renderable surface; there is no second model picker
to keep in sync, and the CLI shares filter_models with it rather than
re-implementing the rules.

Two authentication methods

needle/model/credentials.py exposes one seam — credential_for(provider, api_key, store) returning a CredentialResult — with two adapters behind it.
generate_examples, the catalog fetcher and the model selector consume the
result; none of them can tell which adapter produced it.

API key. A pasted sk-orca-… key, or ORCAROUTER_API_KEY, or a stored
record. credential_from_api_key masks it in every message and log line via
mask_secret, store.clear() removes it, and a 401 from the relay surfaces as
CredentialError with a message naming the account rather than echoing the key.

OAuth 2.0 + PKCE. needle connect runs Flow A (loopback redirect) with
a 127.0.0.1 callback; --oob runs Flow B (out-of-band code) for a
playground bound to a non-loopback host where no redirect can be received.
Flow A is the default because the CLI and a loopback-bound playground can both
receive the redirect, which avoids asking the user to copy a code by hand; it is
not available for a remote-bound playground, which is exactly when --oob is
offered, and the playground's connect dialog picks between them automatically
based on the bind address.

PKCE details:

  • S256; verifier and state come from secrets.token_urlsafe per attempt and
    never leave the process — they are not in the URL, the logs or telemetry.
  • base64url(sha256(verifier)) with padding stripped.
  • Flow A compares state with hmac.compare_digest before the code is touched.
  • denial, state mismatch, timeout, expired/reused code, 403, 429 and network
    errors all end with an actionable message; none of them retry or hang.
  • Origins never derive from each other. Authentication is
    https://www.orcarouter.ai/auth + /api/v1/auth/keys; inference and the
    catalog are https://api.orcarouter.ai/v1. The wrong /v1/auth/keys path
    does not appear anywhere in the implementation. ORCA_BASE_URL is the shared
    self-hosted origin, with ORCA_AUTH_BASE_URL / ORCA_API_BASE_URL as
    explicit overrides that win over it. HTTPS is enforced for anything remote;
    plain HTTP is only accepted for loopback.
  • The response's actual scope is read and checked against what the caller
    needs. A requested scope is never treated as a granted one.

Credential lifecycle

The exchange returns a durable OrcaRouter API key, not a refresh token, so
there is no refresh grant and no re-authorisation on start: the stored key is
reused until it is revoked. It is written to the project's existing cache
directory (~/.cache/cactus-needle/orcarouter/credentials.json, or
NEEDLE_CACHE_DIR) with mode 0600 — no new secret store, no new dependency.

A 401 from the relay calls mark_rejected_generation, which marks exactly the
account and credential generation that sent the rejected request. An
ORCAROUTER_API_KEY from the environment that the store never issued cannot
invalidate a stored record, and a late failure from an older generation cannot
disturb a credential that already replaced it. Nothing is deleted before a new
login succeeds.

Playground connect state is generation-guarded; pagehide clears the busy flag
and hint synchronously and then cancels the server-side task with keepalive,
rather than relying on a finally the guard would swallow.

Catalog and capability filtering

The catalog is GET /v1/models on the configured origin, bounded in time
(20 s), bytes (2 MiB) and item count (2000). Filtering is driven only by
supported_endpoint_types and architecture.input_modalities — never by model
name, which is preserved verbatim including the vendor/model namespace:

  • chat/agent: ?capability=chat, plus an endpoint type out of
    openai / anthropic / gemini / openai-response, which excludes
    image-generation, openai-video and jina-rerank models.
  • multimodal understanding: chat first, then architecture.input_modalities
    must explicitly contain the modality the surface actually uploads. An
    undeclared modality fails closed — it is not shown.
  • embedding / image / video / rerank each match their own capability or
    endpoint type.

Changing the provider, the modality toggle, the attachment type or the task
recomputes the options; a selected model that is no longer compatible is
cleared with a prompt to re-pick rather than silently kept.

Live discovery is authoritative, and the seed is not merged into a
successful live result. When discovery fails, a last-known-good list is reused
and reported as degraded; if there is none, the verified cold-start seed is used
and labelled as such. SEED_CATALOG in needle/model/providers.py is the only
place in the tree carrying model ids: five entries with endpoint types, context
lengths and input modalities, plus the low/medium/high/xhigh reasoning ladder
for openai/gpt-5.5, so a fallback entry filters exactly like a discovered one.
A restored model id is re-checked against the compatible list before it is
reselected.

Real results

Run from a checkout of this branch on 2026-09-17, with ORCAROUTER_API_KEY set:

python3 -m pytest -q tests/test_orcarouter_live.py
7 passed in 17.02s

Those seven go through the implemented provider path — providers.discover_models
and finetune._openrouter — against the live endpoints, not through a separate
curl. The catalog returned 16 chat-capable models and 2 image-generation
models for this workspace, and a real chat completion came back through
chat_completions_url. tests/test_gui_evidence.py drives the real playground
in Chromium, expands the selector against that same live catalog and writes
orca-evidence/manifest.json with catalog_model_count: 16,
image_model_count: 2.

Focused unit/integration, with no network and only fake credentials:

python3 -m pytest -q tests/test_providers.py tests/test_credentials.py \
    tests/test_orcarouter_integration.py tests/test_playground_providers.py
104 passed in 18.00s

python3 -m pytest -q tests/test_gui_evidence.py
1 passed in 13.20s

Why the diff also touches needle/model/tokenizer.py

Independent verification of this change installed the package on a minimal
environment and hit ModuleNotFoundError: No module named 'sentencepiece' while
collecting tests/test_orcarouter_integration.py — it imports
needle.model.finetune, which imports needle.model.tokenizer, which imported
sentencepiece at module scope. sentencepiece is declared under the train
extra (pyproject.toml), while the marker and id constants in that module are
also pulled in by inference, export, rendering and dataset-synthesis paths. The
import is now soft and SANTokenizer.__init__ raises a RuntimeError naming
cactus-needle[train], so a runtime-only install can still reach the provider
and catalog code. tests/test_tokenizer.py covers both halves. This is a
prerequisite, not scope creep: without it the OrcaRouter tests cannot even be
collected on pip install -e ..

Testing

Area Evidence
API key adapter tests/test_credentials.py — save/read/clear, masking, update, missing-key error
PKCE primitives tests/test_credentials.py — verifier/challenge/state, S256, unpadded base64url, authorize URL shape
PKCE exchange tests/test_credentials.py — correct path and body against a local fake auth server, success persists, denial, Flow A state mismatch, reused and expired code, scope downgrade
Lifecycle tests/test_credentials.py — corrupt key classified terminal, 401 marks only the exact account/generation, 429 and network failure are retryable
Provider + origins tests/test_providers.py — registry, alias, self-hosted base, auth/API overrides winning, HTTPS enforcement, HTTP allowed on loopback only
Catalog tests/test_providers.py — parse, bounds, degraded fallback, live result authoritative and never merged with the seed
Capabilities tests/test_providers.py — fixtures for text-only, image-input chat, embedding, image generation, video and rerank; multimodal fail-closed
Wiring tests/test_orcarouter_integration.py — the real URL, headers and body from generate_examples -> _openrouter, with a stubbed transport
Playground tests/test_playground_providers.py — server-side provider/model routes, key never leaves the backend, generation guard, pagehide cleanup
UI evidence tests/test_gui_evidence.py — real Chromium run, screenshots, measured panel geometry and catalog-driven list

Compliance greps run before committing:

  • git grep -n "v1/auth/keys" -- needle/ matches only the docstring in
    providers.py:260 that warns against that exact path.
  • git grep -niE "client_secret|code_verifier=\"|sk-orca-[0-9a-f]{16}" -- needle/
    is empty — no client secret, no fixed verifier, no real key in the tree.
  • git grep -nE "gpt-5\.5|claude-opus-4\.8|gemini-3\.5-flash|deepseek-v4-pro|orcarouter/auto"
    matches only SEED_CATALOG / SEED_MODEL_IDS and the two references to the
    seed in the provider definition.

Full-suite note: tests/test_render.py and tests/test_run.py have 3 errors and
1 failure in this sandbox, all of them RuntimeError: No pretraining tokenizer … HF download failed (SOCKS proxy …). They reproduce identically on the unmodified
base commit with needle/model/tokenizer.py checked out from base, so they are
environmental (the Hugging Face tokenizer cannot be downloaded from here), not
caused by this change.

Evidence and verification

  • Primary endpoint documentation: https://docs.orcarouter.ai/getting-started/models (catalog), https://docs.orcarouter.ai/api-reference/chat/create-a-chat-completion (inference)
  • Auth, exchange and account management: https://docs.orcarouter.ai/getting-started/sign-in-with-orcarouter, https://docs.orcarouter.ai/getting-started/get-api-key, key list/revoke surface at https://www.orcarouter.ai/console
  • Terms / legal entity: OrcaRouter is operated by Continuum AI Corp — the operating organisation linked from the documentation site and the company behind the public https://github.com/Continuum-AI-Corp organisation. I could not find a stable public terms-of-service URL (the plausible paths return 404), so I am not citing one; happy to add it if you can point me at the canonical link.
  • Aggregator routing / resale authorization: OrcaRouter is the gateway being integrated and routes to upstream vendors under its own agreements — https://docs.orcarouter.ai/routing/model-fallbacks and https://docs.orcarouter.ai/routing/named-routers describe the routing surface. No third-party resale claim is made anywhere in this PR.
  • Maintenance owner: the OrcaRouter team; this PR is the contact point, and I will respond to review on the exact head commit.
  • Evidence verification date: 2026-09-17
  • Contributor affiliation disclosure: I'm an engineer on the OrcaRouter team.
  • Focused test result: 104 passed (unit/integration, fake credentials, no network)
  • Changed/full test result: 1 passed GUI evidence + 7 passed live; full-suite exceptions are the environmental tokenizer-download failures described above
  • Real login result: not performed — the OAuth flow needs a human to consent in a browser, and the spec is explicit that a worker must not forge consent. The authorize → callback → exchange → persist path is covered end to end against a local fake auth server in tests/test_credentials.py, through the real connect adapter.
  • Real inference result: 7 passed in tests/test_orcarouter_live.py, through the implemented provider
  • Revocation / reauth result: covered in tests/test_credentials.py with a stubbed 401; no real key was revoked
  • GUI screenshot link: orca-evidence/auth-methods.png, orca-evidence/text-model-dropdown.png, orca-evidence/multimodal-model-dropdown.png (generated by tests/test_gui_evidence.py, manifest with sha256 and measured UI assertions alongside)
  • Unresolved review threads: none
  • Remaining maintainer-only gate: if this triggers an unsponsored_surface-style gate, a maintainer has to review the security boundary and apply the sponsorship label — please review the exact head commit rather than the branch tip at any earlier time.

Not implemented

Flow C (device grant) is not wired up; the spec allows it as an extra but it
cannot replace PKCE, and --oob already covers the no-redirect case for this
repository's two surfaces. No new dependency was added for SHA-256 or base64url.

Signed-off-by: kuswardhanietidims-svg <kuswardhanietidims-svg@users.noreply.github.com>

@HenryNdubuaku HenryNdubuaku left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to take OrcaRouter in. Two things before it can merge: README, doc/finetuning.md and llms.txt conflict with main, so please rebase; and tests/test_gui_evidence.py imports playwright at module level, which errors out the whole collection on a machine without it (pytest.importorskip("playwright") fixes that). Everything else passed locally on top of main, 231 tests.

Rebased onto `main` (fc5bae0): upstream removed the whole `doc/` tree
(`doc/finetuning.md` is gone) and rewrote `README.md` and `llms.txt`, so the
OrcaRouter documentation is re-applied against the new structure rather than
resurrecting `doc/finetuning.md`. `main` also renamed the OpenRouter default
model, which the new provider registry had reverted; that default now matches
main again.

`tests/test_gui_evidence.py` imported `playwright.sync_api`, `PIL` and `numpy`
at module scope, so a checkout without them turned the whole pytest collection
into an error instead of skipping the one test that needs a browser. The three
imports now go through `pytest.importorskip`, matching how `tests/test_packaging.py`
already guards an optional import, and the module imports `pytest` explicitly
instead of relying on injection.

The evidence test still runs wherever the browser stack is present; only a
machine that cannot run it is skipped.

Signed-off-by: kuswardhanietidims-svg <kuswardhanietidims-svg@users.noreply.github.com>
@kuswardhanietidims-svg

Copy link
Copy Markdown
Author

Both points from your review are addressed on 2ecb0f6.

Rebase. The branch is rebased onto current main (fc5bae0). Worth flagging that these were not textual conflicts: main deleted the whole doc/ tree (doc/finetuning.md no longer exists) and rewrote README.md around a new Customisation / Deploy structure, so there is nothing to merge the old doc/finetuning.md diff into. I took main's text as-is and re-applied only the OrcaRouter paragraphs that still have a home — one **Data synthesis.** block under ## Customisation in README.md, and the --provider orcarouter / needle connect lines in llms.txt — and dropped doc/finetuning.md rather than resurrecting it. No conflict markers remain.

While resolving that, I found a real regression the rebase had hidden: main renamed the OpenRouter default model (ae2b58d), but the new provider registry still carried the old name and would have quietly reverted main's rename. The registry now matches main again.

playwright. tests/test_gui_evidence.py now guards its three browser-stack imports with pytest.importorskip (numpy, PIL, playwright.sync_api), matching how tests/test_packaging.py already guards an optional import. I also added the explicit pytest import the module had been relying on pytest injecting. A machine without the stack now gets one skip instead of a collection error; a machine with it still runs the full evidence test and every assertion it made before.


Adds OrcaRouter as a first-class provider for Needle's tool-calling data synthesis, with two explicit ways to authenticate and a model selector driven by the gateway's own catalog. OrcaRouter is an OpenAI-compatible AI gateway that routes many providers behind one endpoint.

I'm an engineer on the OrcaRouter team.

What this changes

needle generate-data --provider orcarouter (and the playground's Dataset tab) synthesise tool-calling examples through OrcaRouter, using OrcaRouter's own /v1/models catalog instead of a free-text model field. Nothing about the existing OpenRouter path changes: get_provider(None) still resolves to it, and OPENROUTER_URL keeps its historical meaning of "the completions URL itself".

Two files carry the bulk of it:

  • needle/model/providers.py — provider registry, origin policy, catalog discovery and capability filtering.
  • needle/model/credentials.py — one credential interface with two adapters.

AI input surfaces covered

Surface Provider selection Credential entry Model choice
needle generate-data (CLI synthesis) --provider orcarouter --api-key, ORCAROUTER_API_KEY, or a stored login --model, filtered by --input-modality
needle connect / needle models --provider --oob selects Flow B needle models --capability lists the filtered catalog
Playground → Dataset tab provider dropdown both methods side by side catalog-driven <select>, re-filtered on toggle

The playground is the only renderable surface; there is no second model picker to keep in sync, and the CLI shares filter_models with it rather than re-implementing the rules.

Two authentication methods

needle/model/credentials.py exposes one seam — credential_for(provider, api_key, store) returning a CredentialResult — with two adapters behind it. generate_examples, the catalog fetcher and the model selector consume the result; none of them can tell which adapter produced it.

API key. A pasted sk-orca-… key, or ORCAROUTER_API_KEY, or a stored record. credential_from_api_key masks it in every message and log line via mask_secret, store.clear() removes it, and a 401 from the relay surfaces as CredentialError with a message naming the account rather than echoing the key.

OAuth 2.0 + PKCE. needle connect runs Flow A (loopback redirect) with a 127.0.0.1 callback; --oob runs Flow B (out-of-band code) for a playground bound to a non-loopback host where no redirect can be received. Flow A is the default because the CLI and a loopback-bound playground can both receive the redirect, which avoids asking the user to copy a code by hand; it is not available for a remote-bound playground, which is exactly when --oob is offered, and the playground's connect dialog picks between them automatically based on the bind address.

PKCE details:

  • S256; verifier and state come from secrets.token_urlsafe per attempt and never leave the process — they are not in the URL, the logs or telemetry.
  • base64url(sha256(verifier)) with padding stripped.
  • Flow A compares state with hmac.compare_digest before the code is touched.
  • denial, state mismatch, timeout, expired/reused code, 403, 429 and network errors all end with an actionable message; none of them retry or hang.
  • Origins never derive from each other. Authentication is https://www.orcarouter.ai/auth + /api/v1/auth/keys; inference and the catalog are https://api.orcarouter.ai/v1. The wrong /v1/auth/keys path does not appear anywhere in the implementation. ORCA_BASE_URL is the shared self-hosted origin, with ORCA_AUTH_BASE_URL / ORCA_API_BASE_URL as explicit overrides that win over it. HTTPS is enforced for anything remote; plain HTTP is only accepted for loopback.
  • The response's actual scope is read and checked against what the caller needs. A requested scope is never treated as a granted one.

Credential lifecycle

The exchange returns a durable OrcaRouter API key, not a refresh token, so there is no refresh grant and no re-authorisation on start: the stored key is reused until it is revoked. It is written to the project's existing cache directory (~/.cache/cactus-needle/orcarouter/credentials.json, or NEEDLE_CACHE_DIR) with mode 0600 — no new secret store, no new dependency.

A 401 from the relay calls mark_rejected_generation, which marks exactly the account and credential generation that sent the rejected request. An ORCAROUTER_API_KEY from the environment that the store never issued cannot invalidate a stored record, and a late failure from an older generation cannot disturb a credential that already replaced it. Nothing is deleted before a new login succeeds.

Playground connect state is generation-guarded; pagehide clears the busy flag and hint synchronously and then cancels the server-side task with keepalive, rather than relying on a finally the guard would swallow.

Catalog and capability filtering

The catalog is GET /v1/models on the configured origin, bounded in time (20 s), bytes (2 MiB) and item count (2000). Filtering is driven only by supported_endpoint_types and architecture.input_modalities — never by model name, which is preserved verbatim including the vendor/model namespace:

  • chat/agent: ?capability=chat, plus an endpoint type out of openai / anthropic / gemini / openai-response, which excludes image-generation, openai-video and jina-rerank models.
  • multimodal understanding: chat first, then architecture.input_modalities must explicitly contain the modality the surface actually uploads. An undeclared modality fails closed — it is not shown.
  • embedding / image / video / rerank each match their own capability or endpoint type.

Changing the provider, the modality toggle, the attachment type or the task recomputes the options; a selected model that is no longer compatible is cleared with a prompt to re-pick rather than silently kept.

Live discovery is authoritative, and the seed is not merged into a successful live result. When discovery fails, a last-known-good list is reused and reported as degraded; if there is none, the verified cold-start seed is used and labelled as such. SEED_CATALOG in needle/model/providers.py is the only place in the tree carrying model ids: five entries with endpoint types, context lengths and input modalities, plus the low/medium/high/xhigh reasoning ladder for openai/gpt-5.5, so a fallback entry filters exactly like a discovered one. A restored model id is re-checked against the compatible list before it is reselected.

Testing

Re-run on this head (2ecb0f6), 2026-09-19, from a clean checkout of the rebased branch:

python3 -m pytest -q tests/test_providers.py tests/test_credentials.py \
    tests/test_orcarouter_integration.py tests/test_playground_providers.py
104 passed in 16.86s

python3 -m pytest -q tests/test_gui_evidence.py
1 passed in 12.14s

python3 -m pytest -q tests/test_orcarouter_live.py
7 passed in 22.30s

The seven live tests go through the implemented provider path — providers.discover_models and finetune._openrouter — against the live endpoints, not through a separate curl. For this workspace the live catalog returned 16 chat-capable models and 2 image-generation models, and a real chat completion came back through chat_completions_url.

Full suite on this head: 230 passed, 6 skipped, 8 failed, 15 errors. Every failing node is a missing optional dependency in this sandbox (jax for the LoRA/build/export tests, huggingface_hub for the fetch tests) or an unavailable pretraining tokenizer download; all 8 failures and all 15 errors also fail on the unmodified main commit, where the suite is 102 passed / 22 failed / 15 errors. The failure count is lower here because the soft sentencepiece import in needle/model/tokenizer.py lets tests/test_generate.py and tests/test_render.py collect at all — those modules were previously collection errors on a runtime-only install, which is 14 of that difference. The one added skip is tests/test_orcarouter_live.py running without a key (7 skips instead of 7 passes).

Area Evidence
API key adapter tests/test_credentials.py — save/read/clear, masking, update, missing-key error
PKCE primitives tests/test_credentials.py — verifier/challenge/state, S256, unpadded base64url, authorize URL shape
PKCE exchange tests/test_credentials.py — correct path and body against a local fake auth server, success persists, denial, Flow A state mismatch, reused and expired code, scope downgrade
Lifecycle tests/test_credentials.py — corrupt key classified terminal, 401 marks only the exact account/generation, 429 and network failure are retryable
Provider + origins tests/test_providers.py — registry, alias, self-hosted base, auth/API overrides winning, HTTPS enforcement, HTTP allowed on loopback only
Catalog tests/test_providers.py — parse, bounds, degraded fallback, live result authoritative and never merged with the seed
Capabilities tests/test_providers.py — fixtures for text-only, image-input chat, embedding, image generation, video and rerank; multimodal fail-closed
Wiring tests/test_orcarouter_integration.py — the real URL, headers and body from generate_examples -> _openrouter, with a stubbed transport
Playground tests/test_playground_providers.py — server-side provider/model routes, key never leaves the backend, generation guard, pagehide cleanup
UI evidence tests/test_gui_evidence.py — real Chromium run, screenshots, measured panel geometry and catalog-driven list

Compliance greps run before committing:

  • git grep -n "v1/auth/keys" -- needle/ matches only the docstring in providers.py:260 that warns against that exact path.
  • git grep -niE "client_secret|sk-orca-[0-9a-f]{16}" -- needle/ is empty — no client secret and no real key in the tree.
  • git grep -nE "gpt-5\.5|claude-opus-4\.8|gemini-3\.5-flash|deepseek-v4-pro|orcarouter/auto" matches only SEED_CATALOG / SEED_MODEL_IDS and the references to the seed in the provider definition.

Why the diff also touches needle/model/tokenizer.py

Independent verification of this change installed the package on a minimal environment and hit ModuleNotFoundError: No module named 'sentencepiece' while collecting tests/test_orcarouter_integration.py — it imports needle.model.finetune, which imports needle.model.tokenizer, which imported sentencepiece at module scope. sentencepiece is declared under the train extra (pyproject.toml), while the marker and id constants in that module are also pulled in by inference, export, rendering and dataset-synthesis paths. The import is now soft and SANTokenizer.__init__ raises a RuntimeError naming cactus-needle[train], so a runtime-only install can still reach the provider and catalog code. tests/test_tokenizer.py covers both halves. This is a prerequisite, not scope creep: without it the OrcaRouter tests cannot even be collected on pip install -e ..

Evidence and verification

  • Primary endpoint documentation: https://docs.orcarouter.ai/getting-started/models (catalog), https://docs.orcarouter.ai/api-reference/chat/create-a-chat-completion (inference)
  • Auth, exchange and account management: https://docs.orcarouter.ai/getting-started/sign-in-with-orcarouter, https://docs.orcarouter.ai/getting-started/get-api-key, key list/revoke surface at https://www.orcarouter.ai/console
  • Terms / legal entity: OrcaRouter is operated by Continuum AI Corp — the operating organisation linked from the documentation site and the company behind the public https://github.com/Continuum-AI-Corp organisation. I could not find a stable public terms-of-service URL (the plausible paths return 404), so I am not citing one; happy to add it if you can point me at the canonical link.
  • Aggregator routing / resale authorization: OrcaRouter is the gateway being integrated and routes to upstream vendors under its own agreements — https://docs.orcarouter.ai/routing/model-fallbacks and https://docs.orcarouter.ai/routing/named-routers describe the routing surface. No third-party resale claim is made anywhere in this PR.
  • Maintenance owner: the OrcaRouter team; this PR is the contact point, and I will respond to review on the exact head commit.
  • Evidence verification date: 2026-09-19
  • Contributor affiliation disclosure: I'm an engineer on the OrcaRouter team.
  • Focused test result: 104 passed unit/integration (fake credentials, no network)
  • Changed/full test result: 1 passed GUI evidence + 7 passed live; full-suite failures are the missing-optional-dependency and tokenizer-download failures described above, all of them identical on the unmodified base commit
  • Real login result: not performed — the OAuth flow needs a human to consent in a browser, and the spec is explicit that a worker must not forge consent. The authorize -> callback -> exchange -> persist path is covered end to end against a local fake auth server in tests/test_credentials.py, through the real connect adapter.
  • Real inference result: 7 passed in tests/test_orcarouter_live.py, through the implemented provider
  • Revocation / reauth result: covered in tests/test_credentials.py with a stubbed 401; no real key was revoked
  • GUI screenshot: orca-evidence/auth-methods.png, orca-evidence/text-model-dropdown.png, orca-evidence/multimodal-model-dropdown.png (1280x1024 each), regenerated on this head by python3 -m pytest tests/test_gui_evidence.py, which writes orca-evidence/manifest.json with a sha256 for each file and the measured UI assertions (api_key_visible, pkce_visible, secret_masked, controls_enabled; dropdown_open, item_count, opaque_background, visible_border, trigger_panel_right_delta). orca-evidence/ is gitignored and rebuilt by that test, so the screenshots are not in the diff — run the command to reproduce them. Digests from the verification run: auth-methods 638672e1bb93ebfa146aa1f0f0b8868585e277ea5ad659e55e50005824c0daec, text-model-dropdown 20e0afc013c306d6bf2498e9fd8ea816ad56697040b34f812c5e47aa149b956a, multimodal-model-dropdown f01d36e13017de669c058e41ba469f54b6cd8fcb3fde73fe4c3533dcfdef2c7a.
  • Unresolved review threads: none
  • Remaining maintainer-only gate: if this triggers an unsponsored_surface-style gate, a maintainer has to review the security boundary and apply the sponsorship label — please review the exact head commit 2ecb0f6 rather than the branch tip at any earlier time.

Not implemented

Flow C (device grant) is not wired up; the spec allows it as an extra but it cannot replace PKCE, and --oob already covers the no-redirect case for this repository's two surfaces. No new dependency was added for SHA-256 or base64url.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants