diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml new file mode 100644 index 0000000..f78236a --- /dev/null +++ b/.github/workflows/publish.yml @@ -0,0 +1,42 @@ +name: Publish + +# Publishing is driven by a version tag, so the released artifact is always +# traceable to a commit. `npm version` creates the tag; pushing it ships. +on: + push: + tags: ['v*'] + +jobs: + publish: + runs-on: ubuntu-latest + permissions: + contents: read + # Required for npm provenance — proves on the registry that this tarball + # was built by this workflow from this commit. + id-token: write + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-node@v4 + with: + node-version: '22' + registry-url: 'https://registry.npmjs.org' + + - run: npm ci --ignore-scripts + + # Never publish something that would not have passed CI. + - run: npm run verify + + # Refuse to publish a tag whose version does not match package.json, + # rather than silently shipping the wrong number. + - name: Check tag matches package version + run: | + tag="${GITHUB_REF_NAME#v}" + pkg=$(node -p "require('./package.json').version") + if [ "$tag" != "$pkg" ]; then + echo "Tag v$tag does not match package.json version $pkg" >&2 + exit 1 + fi + + - run: npm publish + env: + NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }} diff --git a/README.md b/README.md index a7c270f..b0c0aa6 100644 --- a/README.md +++ b/README.md @@ -1,129 +1,172 @@ -# ai-ration +# ai-kit -Keep an LLM app on free tiers, and share what is free fairly. - -Three small, pure modules. No HTTP client, no SDK, no framework — every app -already has its own calling conventions, and replacing those is a rewrite rather -than an adoption. **This package supplies the decisions; you keep the fetch.** +**The AI layer of an app, in one install.** Which model to call, what to do when +the vendor deletes it, what to do when you're going too fast, how to share a free +tier fairly between users, and how to fill a form from plain language. ```bash -npm install github:maonakamoto/ai-ration#v0.1.0 +npm install ai-kit ``` -## Why +--- + +## Why this is one package and not four + +Adding an AI feature looks like one decision and is actually four. Get any of +them wrong and the app fails **identically** from the outside: the assistant is +broken, and the error usually blames the wrong thing. + +On 2026-08-26 that stopped being hypothetical. Groq retired its entire +`llama-3.x` family. Every app in this fleet that had picked a model by hand went +down at the same moment — five repos, three of them serving live traffic — and +the one app that had adopted the fallback chain was unaffected. One of the broken +ones reported *"AI assistant not configured, please set GROQ_API_KEY"* on a +deployment whose key was perfectly valid, so the first hour of the investigation +went into checking a credential that was never the problem. -Free LLM tiers fail in three specific ways, and each one has been mistaken for an -application bug at least once: +That app had already adopted the form-filling half. It hand-rolled the other +half, because that was a second decision and nobody made it. -| Failure | What it looks like | What actually fixes it | -|---|---|---| -| A pinned free model is retired | "the assistant is broken" | a chain, not a pin | -| The vendor's daily budget is gone | a 429 that retrying never clears | a **different vendor** | -| One eager user spends the day's tokens by 10am | everyone else meets a wall | rationing | +So the four decisions ship together now. Adding AI is one install. -The third is the one that costs you users, because the person who hits the wall -is usually the one trying the product for the first time. +> **Renamed from `ai-ration` in v0.3.0.** The old name described one of its five +> modules and hid the other four, and the person deciding whether to install it +> could not tell what it did. An unreadable name is a cost paid at every install +> decision — and this package had a single adopter while five repos that skipped +> it were taken down together by exactly the failure it prevents. -## `chain` — a fallback list across vendors +--- + +## What's in it + +### Which model — a list, never a pin ```ts -import { freeChain, usableChain, chainFrom } from 'ai-ration'; +import { freeChain, usableChain, chainFrom } from 'ai-kit'; + +const providers = freeChain('MYAPP'); // groq → openrouter +const links = usableChain(providers, process.env); // drops vendors with no key -const providers = freeChain('MYAPP'); // groq → openrouter, free models -const links = usableChain(providers, process.env); // drops vendors with no key for (const { provider, model } of chainFrom(process.env.MYAPP_MODEL, links)) { // POST `${provider.baseUrl}/chat/completions` with `model` // on failure, continue — that is the whole point } ``` -Both properties are load-bearing. Falling back to a **smaller model at the same -vendor** buys nothing: it draws on the same org-wide daily budget, so when the -day runs dry every link in that "fallback" is already dead. Only a different -vendor has a different meter. +Falling back to a **smaller model at the same vendor buys nothing**: it draws on +the same org-wide daily budget, so when the day runs dry every link in that +"fallback" is already dead. Only a different vendor has a different meter. + +**Probe before you pin.** Of nine free models probed live, **five** answered only +via a text tool protocol, not native `tool_calls`. A native-only client would +have silently lost most of the chain. + +### Still there? — catch a retirement before a user does + +```ts +import { freeChain, checkCatalog, hasRot, catalogReport } from 'ai-kit'; + +const verdicts = await checkCatalog(freeChain('MYAPP')); +if (hasRot(verdicts)) console.warn(catalogReport(verdicts)); +``` -**Probe before you pin.** Of nine free models probed live for the default chain, -**five** answered only via a text tool protocol, not native `tool_calls`. A -native-only client would have silently lost most of the chain. The shipped list -is evidence from one day, not a constant — free catalogues rot, so re-probe. +One `GET /models` per vendor. **Zero tokens**, which is what makes it +schedulable — and "somebody is supposed to remember" is precisely what failed. -## `limits` — the three kinds of 429 +Three states, not two: a catalogue that could not be read reports **unchecked**, +never *gone*. Treating "I could not look" as "nothing is there" marks every model +retired and invents an outage someone then acts on. -They share a status code, a `type`, and a `code`. Only the body tells them apart, -and they need **opposite** responses: +> This fleet runs it daily across every repo from +> [`dotfiles/scripts/ci/model-pin-audit.mjs`](https://github.com/maonakamoto/dotfiles). + +### Too fast? — the three kinds of 429 ```ts -import { classifyRateLimit, rateLimitMessage } from 'ai-ration'; +import { classifyRateLimit, rateLimitMessage } from 'ai-kit'; classifyRateLimit(body); // 'capacity' | 'size' | 'daily' ``` -- **capacity** — the per-minute window is spent. Wait, or step down. -- **size** — one request exceeds the whole window. Waiting *never* helps, and - stepping down makes it strictly worse (the cheaper model has a smaller - ceiling). Only a smaller prompt helps. -- **daily** — the day's budget is gone, org-wide. Every response that works for - capacity is actively harmful: a step-down draws on the same exhausted budget, - and a 25-second wait is nothing against a reset measured in tens of minutes. +They share a status code, a `type` and a `code`. Only the body tells them apart, +and they need **opposite** responses: retry shortly, shrink the request, or give +up on this vendor until tomorrow. + +`retryAfterSeconds` is present only for the refusal a wait actually fixes. +Telling someone whose daily quota is gone to try again in 20 minutes is a lie. + +### Who gets it — fair shares of a free tier + +```ts +import { fairShare, utcDayElapsed } from 'ai-kit'; +``` -`rateLimitMessage(body)` returns a clause you embed in your own framing. It says -whether waiting can help and when, because "try again shortly" on an exhausted -day invites exactly the retry that is guaranteed to fail for the next hour. +A free tier grants roughly 100k tokens **per day for an entire org**, and one +measured tool-calling turn cost ~16k — about six turns a day. Divided badly, the +first enthusiastic user spends it before lunch and everyone after them meets a +wall, including the person trying the product for the first time, who concludes +it is broken and never comes back. -An unrecognised body degrades to `capacity` — the safe guess, since its response -is harmless when wrong. +Shares are `capacity / active users`, recomputed per request, where *active* +means users who actually drew today — one user on a quiet day correctly gets +everything. The allowance unlocks gradually through the day, with a **one-turn +floor** so nobody's first question of the morning is refused. -## `fair-share` — divide a fixed daily pool +Whatever you pass as `costTokens` must err **high**: under-estimating admits +turns the pool cannot cover, draining the day while the gate still believes there +is room. -Pure policy: no database, no clock, no provider. You own *"what has this user -spent today"*; it owns *"may they spend more"*. +### Filling forms — from prose, then by talking to it ```ts -import { fairShare, utcDayElapsed } from 'ai-ration'; - -const decision = fairShare({ - dayCapacityTokens: dayCapacityTokens(providers, process.env), - activeUsers, // distinct users who drew TODAY, including this one - userSpentTokens, - costTokens: ESTIMATED_TURN, - dayElapsed: utcDayElapsed(new Date()), -}); -// decision.reason: 'ok' | 'paced' | 'share-spent' | 'no-capacity' +import { runFormAssist } from 'ai-kit'; +import { useAiForm } from 'ai-kit/react'; +import { createFormAssistHandler } from 'ai-kit/server'; ``` -Two ideas, both needed: +Re-exported from [`ai-forms`](https://github.com/maonakamoto/ai-forms), which +stays its own package — it works, four apps run it, and it is useful well outside +this fleet. Swallowing it would have broken those four for the sake of a filing +system. -1. **Share** = capacity ÷ users *active today*. Recomputed per request, so a new - user is counted the moment they arrive. Counting dormant accounts would ration - a quiet day down to nothing. -2. **Pacing** — a share is a whole-day allowance, and the day is consumed in - order. Without pacing, three users legitimately spend their full shares by - 09:00 and the fourth finds nothing left. So the allowance unlocks gradually. +React lives on its own subpath and is an **optional** peer, so importing `ai-kit` +on a server never pulls in a UI library. -Plus a **one-turn floor**: the allowance never sits below the cost of a single -turn, so nobody's first question of the morning is refused. Without it, pure -pacing tells a first-time user to come back in three hours — indistinguishable -from broken. +--- -`retryAfterSeconds` is present **only** for `paced`, the one refusal a wait -actually fixes. Telling someone whose share is spent to try again in 20 minutes -is the same lie as "try again shortly" on an exhausted daily quota. +## What it deliberately does not ship -No clawback: a user who spent under a larger share when they were alone is not -punished when a second user appears — their allowance simply stops growing. +**An HTTP client.** Every app has its own calling conventions, retries and +logging, and replacing those is a rewrite rather than an adoption. This supplies +the decisions; you keep the fetch. -## Estimating a turn +That rule is under review, and honestly. `ai-forms` is the most-adopted package +in this fleet and it is the one that broke the rule, by shipping a route factory +and a React hook. A package that hands you a working route gets installed; one +that hands you advice about routes does not. -Whatever you pass as `costTokens` must err **high**. Under-estimating admits -turns the pool cannot cover, draining the day while the gate still believes there -is room. One measured turn on a free model — a single tool call — cost **~16k -tokens**; an estimate of 10k had been documented as "deliberately on the high -side" and was 37% below it. Measure, then replace the estimate with the mean. +**Model values.** Which ids are free, which are billed, and which your account +may use are properties of *your* deployment. Centralise the rule, assert it +locally. + +--- + +## Related + +| Package | For | +|---|---| +| [`ai-forms`](https://github.com/maonakamoto/ai-forms) | Form filling on its own, without the model layer | +| [`threadkit`](https://github.com/maonakamoto/threadkit) | Messages between people, and who may see them | +| [`limitkit`](https://github.com/maonakamoto/limitkit) | Stopping someone doing something too often | + +`threadkit` and `limitkit` are **not** merged in here, on purpose: neither has +anything to do with AI. An app that throttles its login form should not install a +model catalogue to do it. ## Development ```bash -npm run verify # build + test +npm run verify # lint + typecheck + build + test ``` MIT. diff --git a/package-lock.json b/package-lock.json index 4574f10..aa43049 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,13 +1,16 @@ { - "name": "ai-ration", - "version": "0.1.0", + "name": "ai-kit", + "version": "0.3.0", "lockfileVersion": 3, "requires": true, "packages": { "": { - "name": "ai-ration", - "version": "0.1.0", + "name": "ai-kit", + "version": "0.3.0", "license": "MIT", + "dependencies": { + "ai-forms": "^0.1.2" + }, "devDependencies": { "@eslint/js": "^9.39.5", "@types/node": "^22.10.2", @@ -18,6 +21,14 @@ }, "engines": { "node": ">=18" + }, + "peerDependencies": { + "react": ">=18" + }, + "peerDependenciesMeta": { + "react": { + "optional": true + } } }, "node_modules/@eslint-community/eslint-utils": { @@ -572,6 +583,23 @@ "acorn": "^6.0.0 || ^7.0.0 || ^8.0.0" } }, + "node_modules/ai-forms": { + "version": "0.1.2", + "resolved": "https://registry.npmjs.org/ai-forms/-/ai-forms-0.1.2.tgz", + "integrity": "sha512-agadca4pN0iGlZTlNy1Vo2SmnQB+0dNkrQSDE3wCJYeaVxKDs5KaxiwGj5+GL6mVYcI1xZ8pwnT9kL9WcxnBHA==", + "license": "MIT", + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "react": ">=18" + }, + "peerDependenciesMeta": { + "react": { + "optional": true + } + } + }, "node_modules/ajv": { "version": "6.15.0", "resolved": "https://registry.npmjs.org/ajv/-/ajv-6.15.0.tgz", diff --git a/package.json b/package.json index 0ddcb14..d34c531 100644 --- a/package.json +++ b/package.json @@ -1,25 +1,25 @@ { - "name": "ai-ration", - "version": "0.2.1", - "description": "Stay on free LLM tiers: a multi-vendor fallback chain, rate-limit classification that tells the three kinds of 429 apart, and fair-share rationing of a shared daily pool across users.", + "name": "ai-kit", + "version": "0.3.0", + "description": "One install for the AI layer of an app: which model to call and what to do when the vendor retires it, how to read the three kinds of 429, a fair daily budget across users, and headless AI form filling.", "license": "MIT", "author": "Mao Nakamoto", - "homepage": "https://github.com/maonakamoto/ai-ration#readme", + "homepage": "https://github.com/maonakamoto/ai-kit#readme", "repository": { "type": "git", - "url": "git+https://github.com/maonakamoto/ai-ration.git" + "url": "git+https://github.com/maonakamoto/ai-kit.git" }, "bugs": { - "url": "https://github.com/maonakamoto/ai-ration/issues" + "url": "https://github.com/maonakamoto/ai-kit/issues" }, "keywords": [ "ai", "llm", + "form-fill", + "fallback", "free-tier", "rate-limit", - "fallback", "quota", - "rationing", "fair-share", "groq", "openrouter" @@ -40,6 +40,18 @@ ".": { "types": "./dist/index.d.ts", "default": "./dist/index.js" + }, + "./forms": { + "types": "./dist/forms.d.ts", + "default": "./dist/forms.js" + }, + "./react": { + "types": "./dist/react.d.ts", + "default": "./dist/react.js" + }, + "./server": { + "types": "./dist/server.d.ts", + "default": "./dist/server.js" } }, "scripts": { @@ -58,5 +70,16 @@ "globals": "^15.15.0", "typescript": "^5.8.2", "typescript-eslint": "^8.67.0" + }, + "dependencies": { + "ai-forms": "^0.1.2" + }, + "peerDependencies": { + "react": ">=18" + }, + "peerDependenciesMeta": { + "react": { + "optional": true + } } } diff --git a/src/forms.ts b/src/forms.ts new file mode 100644 index 0000000..48d2c60 --- /dev/null +++ b/src/forms.ts @@ -0,0 +1,15 @@ +/** + * Form filling, re-exported from `ai-forms`. + * + * `ai-forms` is NOT absorbed. It stays its own package: it works, four apps run + * it, and it is a genuinely general-purpose thing that people outside this fleet + * can use. Swallowing it would break four repos and delete a good name off the + * registry to satisfy a filing system. + * + * What this subpath buys is that an app adding AI installs ONE thing. AOZ is + * the argument: it adopted `ai-forms`, then hand-rolled a provider layer and a + * chat loop, because those were two further decisions nobody made. Filling a + * form from prose and choosing which model fills it are the same feature to the + * app, so they should be one install. + */ +export * from "ai-forms"; diff --git a/src/index.ts b/src/index.ts index 5be19c1..f1df465 100644 --- a/src/index.ts +++ b/src/index.ts @@ -1,24 +1,40 @@ /** - * ai-ration — keep an LLM app on free tiers, and share what is free fairly. + * ai-kit — one install for the AI layer of an app. * - * Four pieces that each solve a distinct failure, and are useful separately: + * WHAT IT IS FOR + * -------------- + * An app that wants an AI feature needs four unrelated-looking decisions to go + * right, and getting any one wrong looks identical from the outside: the + * assistant is broken. This package holds all four, so adding AI is one + * decision instead of four. * - * chain — a fallback list ACROSS VENDORS, because a single pinned free - * model is a scheduled outage and a smaller model at the same - * vendor draws on the same exhausted daily budget. - * catalog — has the vendor retired an id the chain still asks for? The - * chain is itself a list of pins, so it rots too; on 2026-08-25 - * four of nine default ids were already gone, including an - * entire vendor. Zero tokens, so it can run on a schedule. - * limits — tell the three kinds of 429 apart, because they need opposite - * responses and only the body distinguishes them. - * fair-share — divide a fixed daily pool across active users so the person who - * arrives at 4pm still gets a turn. + * which model — a fallback list ACROSS VENDORS, because a single pinned free + * model is a scheduled outage, and a smaller model at the same + * vendor draws on the same exhausted daily budget. + * still there? — has the vendor retired an id we still ask for? The list is + * itself a list of pins, so it rots too. Zero tokens, so it can + * run on a schedule instead of being remembered. + * too fast? — tell the three kinds of 429 apart. They share a status code + * and need opposite responses; only the body distinguishes them. + * who gets it — divide a fixed daily pool across active users, so the person + * who arrives at 4pm still gets a turn. + * filling forms— fill a form from prose and keep talking to it, re-exported + * from `ai-forms` (see ./forms.ts for why it stays separate). * - * Deliberately NOT included: an HTTP client. Every app here already has its own - * calling conventions, retries, and logging, and replacing those is a rewrite - * rather than an adoption. This package supplies the decisions; the caller keeps - * the fetch. + * WHY IT WAS RENAMED FROM ai-ration + * --------------------------------- + * Because the owner of this fleet read the name and could not tell what it did. + * That is not a cosmetic complaint: an unreadable name is an adoption cost paid + * on every single install decision, and this package had ONE adopter while the + * five repos that skipped it were all taken down together on 2026-08-26 by a + * retired model id — the exact failure the `chain` and `catalog` modules exist + * to prevent. "Ration" described one of five modules and buried the other four. + * + * STILL NOT INCLUDED: an HTTP client. Every app has its own calling conventions, + * retries and logging, and replacing those is a rewrite rather than an adoption. + * This supplies the decisions; the caller keeps the fetch. That rule is under + * review — `ai-forms`, the most-adopted package in this fleet, is the one that + * broke it by shipping a route factory and a hook. */ export { @@ -64,3 +80,7 @@ export { utcDayElapsed, utcDayKey, } from "./fair-share.js"; + +// Form filling. Re-exported so that adding AI to an app is one install; the +// package itself stays independent and separately useful. +export * from "./forms.js"; diff --git a/src/react.ts b/src/react.ts new file mode 100644 index 0000000..b16464e --- /dev/null +++ b/src/react.ts @@ -0,0 +1,8 @@ +/** + * The React form hook, re-exported from `ai-forms/react`. + * + * Kept on its own subpath so importing `ai-kit` on a server never pulls React + * in. `react` is an OPTIONAL peer for exactly this reason: an app using only + * the provider chain should not be asked to install a UI library. + */ +export * from "ai-forms/react"; diff --git a/src/server.ts b/src/server.ts new file mode 100644 index 0000000..beede65 --- /dev/null +++ b/src/server.ts @@ -0,0 +1,10 @@ +/** + * The form-assist route factory, re-exported from `ai-forms/server`. + * + * This is the piece that explains the fleet's adoption numbers. `ai-forms` is + * the most-adopted shared package here, and it is also the only one that ships + * real machinery rather than decisions alone — AOZ imports this factory and the + * React hook, and nothing else. A package that hands you a working route gets + * installed; one that hands you advice about routes does not. + */ +export * from "ai-forms/server"; diff --git a/test/catalog.test.js b/test/catalog.test.js index f1f9aab..c9ebf26 100644 --- a/test/catalog.test.js +++ b/test/catalog.test.js @@ -15,7 +15,7 @@ import { deadProviders, catalogReport, withEnvPrefix, -} from 'ai-ration'; +} from 'ai-kit'; const provider = (id, models, keyEnv) => withEnvPrefix('T', { id, baseUrl: `https://${id}.test/v1`, keyEnv, models, dailyTokens: 1000 }); diff --git a/test/chain.test.js b/test/chain.test.js index 2f1b16c..1c8fe13 100644 --- a/test/chain.test.js +++ b/test/chain.test.js @@ -12,7 +12,7 @@ import { dayCapacityTokens, usableChain, chainFrom, -} from 'ai-ration'; +} from 'ai-kit'; const CHAIN = freeChain('LOKI'); diff --git a/test/cost.test.js b/test/cost.test.js index 8a9ab4f..2f55db0 100644 --- a/test/cost.test.js +++ b/test/cost.test.js @@ -6,7 +6,7 @@ import { test } from 'node:test'; import assert from 'node:assert/strict'; -import { modelCost, modelCostAt, paidModelsIn, freeChain } from 'ai-ration'; +import { modelCost, modelCostAt, paidModelsIn, freeChain } from 'ai-kit'; test('the three ids that were actually billing are all caught', () => { assert.equal(modelCost('anthropic/claude-sonnet-5'), 'paid'); diff --git a/test/exports.test.js b/test/exports.test.js index c174b7e..f35ff55 100644 --- a/test/exports.test.js +++ b/test/exports.test.js @@ -11,7 +11,7 @@ import { test } from 'node:test'; import assert from 'node:assert/strict'; -import * as pkg from 'ai-ration'; +import * as pkg from 'ai-kit'; test('the package exports its public surface through the exports map', () => { const expected = [ @@ -26,3 +26,36 @@ test('the package exports its public surface through the exports map', () => { assert.ok(name in pkg, `missing export: ${name}`); } }); + +/** + * The merge is the feature, so it needs a test. + * + * `ai-kit` exists so that adding AI to an app is ONE install rather than four + * separate decisions — AOZ made one of those decisions (ai-forms), skipped the + * other two, and was taken down by the one it skipped. If the form-filling + * re-export silently stops resolving, the package quietly becomes the old + * ai-ration again under a friendlier name, and nothing else here would notice. + */ +test('form filling is reachable from the root, so one install covers it', async () => { + const pkg = await import('ai-kit'); + for (const name of ['runFormAssist', 'defineFields', 'mergeValues', 'sanitizeValues']) { + assert.equal(typeof pkg[name], 'function', `missing re-export: ${name}`); + } +}); + +test('the model layer and the form layer coexist without shadowing', async () => { + const pkg = await import('ai-kit'); + // One from each half. A collision would drop one silently at build time. + assert.equal(typeof pkg.freeChain, 'function'); + assert.equal(typeof pkg.runFormAssist, 'function'); +}); + +test('./forms resolves through the exports map', async () => { + const forms = await import('ai-kit/forms'); + assert.equal(typeof forms.runFormAssist, 'function'); +}); + +test('./server resolves, and is what the most-adopted package actually ships', async () => { + const server = await import('ai-kit/server'); + assert.equal(typeof server.createFormAssistHandler, 'function'); +}); diff --git a/test/fair-share.test.js b/test/fair-share.test.js index 2f2d147..83a70b7 100644 --- a/test/fair-share.test.js +++ b/test/fair-share.test.js @@ -10,7 +10,7 @@ import { test } from 'node:test'; import assert from 'node:assert/strict'; -import { fairShare, utcDayElapsed, utcDayKey, DAY_SECONDS, DEFAULT_BURST } from 'ai-ration'; +import { fairShare, utcDayElapsed, utcDayKey, DAY_SECONDS, DEFAULT_BURST } from 'ai-kit'; const TURN = 20_000; const NOON = 0.5; diff --git a/test/limits.test.js b/test/limits.test.js index 4f442f9..a072c31 100644 --- a/test/limits.test.js +++ b/test/limits.test.js @@ -6,7 +6,7 @@ import { test } from 'node:test'; import assert from 'node:assert/strict'; -import { classifyRateLimit, retryAfterSeconds, humanizeWait, rateLimitMessage } from 'ai-ration'; +import { classifyRateLimit, retryAfterSeconds, humanizeWait, rateLimitMessage } from 'ai-kit'; const CAPACITY = 'Rate limit reached for model `llama-3.3-70b-versatile` on tokens per minute (TPM): Limit 12000, Used 11800, Requested 400. Please try again in 3.6s.';