Skip to content

feat(provider): add built-in LLMTR gateway provider - #10

Open
knowhycodata wants to merge 4 commits into
s0ld13rr:mainfrom
knowhycodata:feat/llmtr-provider
Open

feat(provider): add built-in LLMTR gateway provider#10
knowhycodata wants to merge 4 commits into
s0ld13rr:mainfrom
knowhycodata:feat/llmtr-provider

Conversation

@knowhycodata

@knowhycodata knowhycodata commented Aug 21, 2026

Copy link
Copy Markdown

Closes #9

What

Adds LLMTR — a Turkey-hosted, OpenAI-compatible AI gateway (200+ models, global + Turkey-hosted with data-residency guarantee) — as a built-in provider. It plugs in like the other gateway providers (openrouter/nvidia/zenmux): an @ai-sdk/openai-compatible provider at https://llmtr.com/v1 whose bearer key is resolved from a new llmtr integration.

Changes

  • packages/core/src/plugin/provider/llmtr.ts — new provider plugin:
    • Injects the llmtr provider into the catalog (@ai-sdk/openai-compatible, url https://llmtr.com/v1, branding headers).
    • Registers an llmtr integration with API-key + env (LLMTR_API_KEY) auth methods.
    • Discovers the model catalog live from the public /v1/models endpoint (OpenRouter-style schema): converts per-token pricing → per-1M cost, maps modalities, tool support (supported_parameters), and context/output limits.
    • Best-effort, forked fetch (never blocks registration) with a curated offline seed of Turkey-hosted flagship models.
    • LLMTR_BASE_URL overrides the endpoint; LLMTR_SKIP_REMOTE_MODELS=1 disables the live fetch (used by the test).
  • packages/core/src/plugin/provider.ts — register LLMTRPlugin in ProviderPlugins.
  • packages/core/test/plugin/provider-llmtr.test.ts — hermetic tests: registration, provider api/headers, integration key+env methods, seed model pricing conversion & tool detection.
  • README.md — LLMTR configuration section.

Usage

pentestcode auth login    # pick "LLMTR", paste API key   (or export LLMTR_API_KEY=...)
{ "provider": { "llmtr": { "model": "openai/gpt-5.5" } } }

Verification

  • tsgo --noEmit (core) — clean.
  • bun test test/plugin/217 pass / 0 fail (incl. 4 new).

knowhycodata and others added 3 commits August 21, 2026 18:10
LLMTR (https://llmtr.com) is a Turkey-hosted, OpenAI-compatible AI gateway
fronting 200+ models (global providers + Turkey-hosted models with a
data-residency guarantee) behind a single endpoint.

- New `llmtr` provider plugin injects an `@ai-sdk/openai-compatible` provider
  into the catalog (url https://llmtr.com/v1), mirroring the openrouter/nvidia/
  zenmux gateway pattern. Bearer key resolves from the `llmtr` integration.
- Registers an `llmtr` integration with API-key + env (`LLMTR_API_KEY`) auth.
- Discovers the model catalog live from the public `/v1/models` endpoint
  (OpenRouter-style schema), converting per-token pricing to per-1M cost and
  mapping modalities/tool support/limits. Curated offline seed of Turkey-hosted
  flagships keeps the provider usable without network. `LLMTR_BASE_URL` overrides
  the endpoint; `LLMTR_SKIP_REMOTE_MODELS=1` disables the live fetch.
- Adds hermetic test coverage and a README section.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
LLMTR models (gemma-4, qwen3, muse-glimmer) expose thinking as an on/off
toggle (`reasoning: true`, or a `:think` model-id suffix), not OpenAI-style
graded reasoning_effort. An active effort variant — or one carried over from
another model — sent `reasoning_effort` and LLMTR rejected the whole request.

Rewrite the outgoing request body for the LLMTR provider so any graded
reasoning_effort becomes `reasoning: true` (an explicit off/none/minimal/
disabled keeps thinking off, and an existing reasoning flag is preserved),
matching what LLMTR accepts. Add rewriteLLMTRReasoningBody plus unit tests.
Seed the GLM 5.x family (5.3, 5.2, 5.1, 5) and the DeepSeek family (V4 Pro,
V4 Flash, Chat, R1) — strong tool-use / reasoning models better suited to
security work than the existing Turkey-hosted seeds. Metadata comes from the
live LLMTR catalog; the forked live fetch still wins on id collisions so
pricing stays current.

These upstreams expose different thinking controls, so make the reasoning
normalization per-model instead of blanket: classify each model as
effort/boolean/none from its advertised parameters (in a dependency-free
module to avoid a plugin-registry import cycle), and only rewrite an outgoing
reasoning_effort for on/off ("boolean") or no-reasoning ("none") models.
Graded-effort models (GLM 5.2/5.3, proxied OpenAI, ...) keep their native
reasoning_effort untouched. Add classifier and seed-catalog tests.
@s0ld13rr

Copy link
Copy Markdown
Owner

Hey, thanks for the PR — went through the diff.

The code itself is solid and well-tested, no issues there. But I have a few concerns about merging it as a built-in provider:

  1. fetchModels() fires on every startup regardless of whether anyone has an LLMTR key configured. None of the other providers (groq, nvidia, openrouter) do remote fetches at init. At minimum this needs a guard — skip the fetch entirely if no API key is present.

  2. The rewriteLLMTRReasoningBody logic lives in the shared provider.ts, so every request to any provider now passes through if (model.providerID === "llmtr"). This should live inside the plugin itself (via ctx.aisdk.sdk hook, same way openrouter does its SDK override), not in the common path.

  3. The seed list includes DeepSeek and GLM models that already have their own provider plugins — could cause duplicate/conflicting entries in the catalog.

  4. Broader question: LLMTR is OpenAI-compatible, so users can already connect to it with ~3 lines of config in pentestcode.jsonc without any code changes. Adding a built-in provider means we commit to maintaining it long-term.

Two paths forward:

A) I can add a "Custom providers" section to the README showing how to point at LLMTR (or any OAI-compatible gateway) via existing config. Zero code, works today.

B) If you want first-class support — move the reasoning rewrite into the plugin, gate the fetch behind API key presence, and drop the seed models that overlap with existing providers. Happy to review a v2.

What do you think?

…e reasoning

Responds to PR review feedback:

- Gate the live `/v1/models` fetch behind API-key presence (env var or an active
  llmtr integration connection). No key → no request at init, matching how the
  other providers behave. Refetch on ConnectionUpdated so `auth login` mid-session
  still populates the full catalog.
- Remove the `if (model.providerID === "llmtr")` branch from the shared request
  path (opencode/provider.ts). Introduce a generic per-provider outgoing-body
  rewriter registry (request-transform.ts); the LLMTR reasoning rewrite now lives
  entirely in its own core module and registers itself. The interactive path uses
  BUNDLED_PROVIDERS (not the V2 ctx.aisdk.sdk hook), so the rewrite stays in that
  path but is no longer provider-specific there.
- Move rewriteLLMTRReasoningBody + its tests into core alongside the reasoning
  module; consolidate reasoning unit tests into packages/core.

DeepSeek/GLM seed models are intentionally kept (LLMTR prices them below list /
periodically free, and they live under the `llmtr` namespace — no catalog collision).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@knowhycodata

Copy link
Copy Markdown
Author

Hey, thank you so much for taking the time to go through this so carefully — genuinely helpful feedback, and you're right on the money with the first two. I pushed a v2 (64cb77d) that takes care of them:

1. Startup fetch — totally agree, it shouldn't do anything before there's a key. It's now gated on API-key presence (env var or an active LLMTR connection), so a user with no key pays zero at init and just gets the seed list. It also re-fetches on auth login so the full catalog fills in the moment a key is added. 👍

2. Reasoning rewrite in the shared path — 100% agree it didn't belong there as a providerID === "llmtr" branch. I pulled it out into a generic per-provider body-rewriter registry, so the shared path no longer knows anything about LLMTR — a provider just registers its own rewriter and the request path looks it up generically. One thing worth flagging (took me a bit to trace): the interactive path builds its SDK through BUNDLED_PROVIDERS, not the V2 ctx.aisdk.sdk hook — so if I'd moved the rewrite into the aisdk hook like openrouter, it'd silently stop firing in the actual TUI. So it stays on that path, just no longer provider-specific. Very open to a better home for it if you know one I missed!

3. DeepSeek / GLM overlap — fair concern, let me make the case for keeping them and you tell me if you disagree. They register under the llmtr namespace (llmtrdeepseek/…, zai/glm-…), so they're separate catalog entries rather than collisions with the models.dev providers. The reason they earn their spot is pricing: LLMTR serves the GLM 5.x family ~10% under list and the DeepSeek family is periodically free, so reaching them through LLMTR is a real cost win, not just a duplicate. That said — totally happy to trim to only the ones with a meaningful price delta if you'd rather keep the list lean.

4. Maintenance — we've got this one long-term, no worries: I work at LLMTR, so keeping the catalog and pricing current is on us rather than the project. And if you'd like to kick the tires end-to-end, I'd be glad to drop some test credit on an LLMTR account for you — just send me the email you'd sign up with. 🙂

Thanks again — really appreciate the thoughtful review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add built-in LLMTR (Turkey-hosted, OpenAI-compatible) gateway provider

2 participants