Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion kubernetes/apps/base/ai/litellm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ from the CRs here.
| File | What it declares |
| --- | --- |
| [`app/litellmproxy.yaml`](app/litellmproxy.yaml) | The `LiteLLMProxy` - image, probes, envFrom, non-secret SSO `env`, admin-API access, `litellmSettings`, `routerSettings` (incl. `redis_host`/`redis_port` against `litellm-dragonfly` - see `docs/ai-system/litellm/README.md#why-dragonfly-redis`). Deliberately **no** `spec.route`. |
| [`app/models/`](app/models/) | 36 `LiteLLMModel` CRs, one per model. Five of them are the SAME local B70 backend under different aliases, each carrying one property the others must not: `qwen3.6-35b-a3b` (terminal, **synthetically priced** - reached only by the `demo` budget test), `chat-local` (terminal, **zero-priced** - what real traffic runs on), `chat-ha` (**cloud fallback** for entitled keys), `qwen3.6-35b-a3b-classifier` (thinking disabled, separate metrics series), `pr-review-local` (thinking disabled, AI PR reviewer only - `docs/ai-system/litellm/pr-reviewer.md`). Plus `auto` (the D3 router), `claude-opus-5-metered`, `claude-sonnet-5-metered`, the 2026-08-27 `indydevdan-model-stack` batch - see [Model catalog](#model-catalog) below - and `claude-sonnet-5` + `claude-opus-5` (renamed 2026-08-31 from `claude-code-subscription` / `claude-code-subscription-opus`), the odd ones out: the proxy holds **no credential** for either - see [Claude Code subscription pass-through](#claude-code-subscription-pass-through). |
| [`app/models/`](app/models/) | 38 `LiteLLMModel` CRs, one per model. Five of them are the SAME local B70 backend under different aliases, each carrying one property the others must not: `qwen3.6-35b-a3b` (terminal, **synthetically priced** - reached only by the `demo` budget test), `chat-local` (terminal, **zero-priced** - what real traffic runs on), `chat-ha` (**cloud fallback** for entitled keys), `qwen3.6-35b-a3b-classifier` (thinking disabled, separate metrics series), `pr-review-local` (thinking disabled, AI PR reviewer only - `docs/ai-system/litellm/pr-reviewer.md`). Plus `auto` (the D3 router), `claude-opus-5-metered`, `claude-sonnet-5-metered`, the 2026-08-27 `indydevdan-model-stack` batch - see [Model catalog](#model-catalog) below - and `claude-sonnet-5` + `claude-opus-5` (renamed 2026-08-31 from `claude-code-subscription` / `claude-code-subscription-opus`), the odd ones out: the proxy holds **no credential** for either - see [Claude Code subscription pass-through](#claude-code-subscription-pass-through). |
| [`app/virtualkeys/`](app/virtualkeys/) | One `LiteLLMVirtualKey` + its `PushSecret` per consumer (D4). |
| [`app/httproute-internal.yaml`](app/httproute-internal.yaml) | Standalone internal `HTTPRoute` named `litellm-internal` (not `litellm`) - the operator deletes any route whose name matches the proxy CR when `spec.route` is absent. |
| [`app/dbinit.yaml`](app/dbinit.yaml) | `postgres-init` Job creating the role + database in the shared `postgres-17` cluster. |
Expand Down
27 changes: 27 additions & 0 deletions kubernetes/apps/base/ai/litellm/app/models/gpt-6-astra-pro.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
---
# yaml-language-server: $schema=https://kubernetes-schemas.pages.dev/litellm.home-operations.com/litellmmodel_v1alpha1.json
# Tier S - SOTA (frontier ceiling). Released 2026-09-04.
#
# OpenRouter is the only credential this repo holds that reaches this
# vendor, so it is the route by necessity, not preference.
# Model id verified against the live provider catalog:
# GET https://openrouter.ai/api/v1/models (2026-09-09): same 1,050,000
# context and $10/M input, $50/M output pricing as gpt-6-astra.yaml. A
# `:batch` version also exists and is not registered.
#
# COST WARNING: this is a $50-per-million-output model, roughly 50x the
# output price of the local/cheap cloud tiers already on this proxy. Do not
# add it to any auto-router tier or fallback chain.
#
# REGISTRATION ONLY - see ./kustomization.yaml for what that does and does
# not grant, and for why no `info.extra` prices appear here.
apiVersion: litellm.home-operations.com/v1alpha1
kind: LiteLLMModel
metadata:
name: gpt-6-astra-pro
spec:
modelName: gpt-6-astra-pro
proxyRef: litellm
params:
model: openrouter/openai/gpt-6-astra-pro
apiKey: os.environ/OPENROUTER_API_KEY
27 changes: 27 additions & 0 deletions kubernetes/apps/base/ai/litellm/app/models/gpt-6-astra.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
---
# yaml-language-server: $schema=https://kubernetes-schemas.pages.dev/litellm.home-operations.com/litellmmodel_v1alpha1.json
# Tier S - SOTA (frontier ceiling). Released 2026-09-04.
#
# OpenRouter is the only credential this repo holds that reaches this
# vendor, so it is the route by necessity, not preference.
# Model id verified against the live provider catalog:
# GET https://openrouter.ai/api/v1/models (2026-09-09): 1,050,000 context,
# $10/M input, $50/M output. A `-pro` variant (gpt-6-astra-pro.yaml) and
# `:batch` versions of both also exist; the `:batch` ids are not registered.
#
# COST WARNING: this is a $50-per-million-output model, roughly 50x the
# output price of the local/cheap cloud tiers already on this proxy. Do not
# add it to any auto-router tier or fallback chain.
#
# REGISTRATION ONLY - see ./kustomization.yaml for what that does and does
# not grant, and for why no `info.extra` prices appear here.
apiVersion: litellm.home-operations.com/v1alpha1
kind: LiteLLMModel
metadata:
name: gpt-6-astra
spec:
modelName: gpt-6-astra
proxyRef: litellm
params:
model: openrouter/openai/gpt-6-astra
apiKey: os.environ/OPENROUTER_API_KEY
2 changes: 2 additions & 0 deletions kubernetes/apps/base/ai/litellm/app/models/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,8 @@ resources:
- ./gpt-5.6-luna.yaml
- ./gpt-5.6-sol.yaml
- ./gpt-5.6-terra.yaml
- ./gpt-6-astra-pro.yaml
- ./gpt-6-astra.yaml
- ./gpt-oss-20b.yaml
- ./grok-4.5.yaml
- ./grok-4.6.yaml
Expand Down
Loading