feat: modelCost — because the same paid fallback was found in three apps - #3
Merged
Conversation
One mistake, three codebases, one day. Each was a FALLBACK that began spending
the moment the free tier ran dry — the path nobody watches, because it only runs
when something else has already broken:
botsmann OPENROUTER_DEFAULT_MODEL = anthropic/claude-sonnet-5 (premium)
kivvi chain ended at Anthropic; OpenRouter default was
google/gemini-2.0-flash-001, the paid twin of a :free id
evig meta-llama/llama-3.3-70b-instruct — reads free, bills at 1e-7/token,
and its :free sibling has been retired from the catalogue
A fallback is a reliability mechanism; paying is a business decision. Wiring the
second to the first lets an outage make the decision, at the worst moment,
without anyone choosing it.
The rule is deliberately narrow and says so. A ROUTED id (vendor/model) is free
only with the :free suffix and paid without it — that suffix is the entire
difference between free routing and a per-call charge for identical weights. A
bare id (llama-3.1-8b-instant) returns "unknown": whether it costs depends on
the account tier at that vendor, which no string can answer. Guessing "free"
there is the direction that let all three through review, so it is not the
default.
paidModelsIn() applies it to a whole chain, returning offenders rather than
throwing — the caller knows whether a paid link is a bug or a deliberate,
opted-in upgrade, and a library that throws on the second case gets routed
around.
35 tests green, including the case that keeps this package honest: its own
shipped free chain must contain no paid model.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
One mistake, three codebases, one day. Each was a fallback that began spending the moment the free tier ran dry — the path nobody watches, because it only runs when something else has already broken.
anthropic/claude-sonnet-5— a premium model as the OpenRouter fallbackgoogle/gemini-2.0-flash-001, the paid twin of a:freeidmeta-llama/llama-3.3-70b-instruct— reads free, bills at 1e-7/token, and its:freesibling has been retired from the catalogueA fallback is a reliability mechanism; paying is a business decision. Wiring the second to the first lets an outage make the decision, at the worst moment, without anyone choosing it.
The rule is narrow, and says so
A routed id (
vendor/model) is free only with:freeand paid without it — that suffix is the entire difference between free routing and a per-call charge for identical weights.A bare id (
llama-3.1-8b-instant) returns"unknown": whether it costs depends on the account tier at that vendor, which no string can answer. Guessing free there is the direction that let all three through review, so it is not the default.paidModelsIn()applies it to a whole chain and returns offenders rather than throwing — the caller knows whether a paid link is a bug or a deliberate opted-in upgrade, and a library that throws on the second case gets routed around.35 tests green, including the one that keeps this package honest: its own shipped free chain must contain no paid model.
🤖 Generated with Claude Code