Skip to content

feat!: become ai-kit — the AI layer in one install #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
catomean merged 1 commit into from
Aug 26, 2026
Merged

feat!: become ai-kit — the AI layer in one install #5

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions .github/workflows/publish.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: Publish

# Publishing is driven by a version tag, so the released artifact is always
# traceable to a commit. `npm version` creates the tag; pushing it ships.
on:
push:
tags: ['v*']

jobs:
publish:
runs-on: ubuntu-latest
permissions:
contents: read
# Required for npm provenance — proves on the registry that this tarball
# was built by this workflow from this commit.
id-token: write
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '22'
registry-url: 'https://registry.npmjs.org'

- run: npm ci --ignore-scripts

# Never publish something that would not have passed CI.
- run: npm run verify

# Refuse to publish a tag whose version does not match package.json,
# rather than silently shipping the wrong number.
- name: Check tag matches package version
run: |
tag="${GITHUB_REF_NAME#v}"
pkg=$(node -p "require('./package.json').version")
if [ "$tag" != "$pkg" ]; then
echo "Tag v$tag does not match package.json version $pkg" >&2
exit 1
fi

- run: npm publish
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
205 changes: 124 additions & 81 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,129 +1,172 @@
# ai-ration
# ai-kit

Keep an LLM app on free tiers, and share what is free fairly.

Three small, pure modules. No HTTP client, no SDK, no framework — every app
already has its own calling conventions, and replacing those is a rewrite rather
than an adoption. **This package supplies the decisions; you keep the fetch.**
**The AI layer of an app, in one install.** Which model to call, what to do when
the vendor deletes it, what to do when you're going too fast, how to share a free
tier fairly between users, and how to fill a form from plain language.

```bash
npm install github:maonakamoto/ai-ration#v0.1.0
npm install ai-kit
```

## Why
---

## Why this is one package and not four

Adding an AI feature looks like one decision and is actually four. Get any of
them wrong and the app fails **identically** from the outside: the assistant is
broken, and the error usually blames the wrong thing.

On 2026-08-26 that stopped being hypothetical. Groq retired its entire
`llama-3.x` family. Every app in this fleet that had picked a model by hand went
down at the same moment — five repos, three of them serving live traffic — and
the one app that had adopted the fallback chain was unaffected. One of the broken
ones reported *"AI assistant not configured, please set GROQ_API_KEY"* on a
deployment whose key was perfectly valid, so the first hour of the investigation
went into checking a credential that was never the problem.

Free LLM tiers fail in three specific ways, and each one has been mistaken for an
application bug at least once:
That app had already adopted the form-filling half. It hand-rolled the other
half, because that was a second decision and nobody made it.

| Failure | What it looks like | What actually fixes it |
|---|---|---|
| A pinned free model is retired | "the assistant is broken" | a chain, not a pin |
| The vendor's daily budget is gone | a 429 that retrying never clears | a **different vendor** |
| One eager user spends the day's tokens by 10am | everyone else meets a wall | rationing |
So the four decisions ship together now. Adding AI is one install.

The third is the one that costs you users, because the person who hits the wall
is usually the one trying the product for the first time.
> **Renamed from `ai-ration` in v0.3.0.** The old name described one of its five
> modules and hid the other four, and the person deciding whether to install it
> could not tell what it did. An unreadable name is a cost paid at every install
> decision — and this package had a single adopter while five repos that skipped
> it were taken down together by exactly the failure it prevents.

## `chain` — a fallback list across vendors
---

## What's in it

### Which model — a list, never a pin

```ts
import { freeChain, usableChain, chainFrom } from 'ai-ration';
import { freeChain, usableChain, chainFrom } from 'ai-kit';

const providers = freeChain('MYAPP'); // groq → openrouter
const links = usableChain(providers, process.env); // drops vendors with no key

const providers = freeChain('MYAPP'); // groq → openrouter, free models
const links = usableChain(providers, process.env); // drops vendors with no key
for (const { provider, model } of chainFrom(process.env.MYAPP_MODEL, links)) {
// POST `${provider.baseUrl}/chat/completions` with `model`
// on failure, continue — that is the whole point
}
```

Both properties are load-bearing. Falling back to a **smaller model at the same
vendor** buys nothing: it draws on the same org-wide daily budget, so when the
day runs dry every link in that "fallback" is already dead. Only a different
vendor has a different meter.
Falling back to a **smaller model at the same vendor buys nothing**: it draws on
the same org-wide daily budget, so when the day runs dry every link in that
"fallback" is already dead. Only a different vendor has a different meter.

**Probe before you pin.** Of nine free models probed live, **five** answered only
via a text tool protocol, not native `tool_calls`. A native-only client would
have silently lost most of the chain.

### Still there? — catch a retirement before a user does

```ts
import { freeChain, checkCatalog, hasRot, catalogReport } from 'ai-kit';

const verdicts = await checkCatalog(freeChain('MYAPP'));
if (hasRot(verdicts)) console.warn(catalogReport(verdicts));
```

**Probe before you pin.** Of nine free models probed live for the default chain,
**five** answered only via a text tool protocol, not native `tool_calls`. A
native-only client would have silently lost most of the chain. The shipped list
is evidence from one day, not a constant — free catalogues rot, so re-probe.
One `GET /models` per vendor. **Zero tokens**, which is what makes it
schedulable — and "somebody is supposed to remember" is precisely what failed.

## `limits` — the three kinds of 429
Three states, not two: a catalogue that could not be read reports **unchecked**,
never *gone*. Treating "I could not look" as "nothing is there" marks every model
retired and invents an outage someone then acts on.

They share a status code, a `type`, and a `code`. Only the body tells them apart,
and they need **opposite** responses:
> This fleet runs it daily across every repo from
> [`dotfiles/scripts/ci/model-pin-audit.mjs`](https://github.com/maonakamoto/dotfiles).

### Too fast? — the three kinds of 429

```ts
import { classifyRateLimit, rateLimitMessage } from 'ai-ration';
import { classifyRateLimit, rateLimitMessage } from 'ai-kit';

classifyRateLimit(body); // 'capacity' | 'size' | 'daily'
```

- **capacity** — the per-minute window is spent. Wait, or step down.
- **size** — one request exceeds the whole window. Waiting *never* helps, and
stepping down makes it strictly worse (the cheaper model has a smaller
ceiling). Only a smaller prompt helps.
- **daily** — the day's budget is gone, org-wide. Every response that works for
capacity is actively harmful: a step-down draws on the same exhausted budget,
and a 25-second wait is nothing against a reset measured in tens of minutes.
They share a status code, a `type` and a `code`. Only the body tells them apart,
and they need **opposite** responses: retry shortly, shrink the request, or give
up on this vendor until tomorrow.

`retryAfterSeconds` is present only for the refusal a wait actually fixes.
Telling someone whose daily quota is gone to try again in 20 minutes is a lie.

### Who gets it — fair shares of a free tier

```ts
import { fairShare, utcDayElapsed } from 'ai-kit';
```

`rateLimitMessage(body)` returns a clause you embed in your own framing. It says
whether waiting can help and when, because "try again shortly" on an exhausted
day invites exactly the retry that is guaranteed to fail for the next hour.
A free tier grants roughly 100k tokens **per day for an entire org**, and one
measured tool-calling turn cost ~16k — about six turns a day. Divided badly, the
first enthusiastic user spends it before lunch and everyone after them meets a
wall, including the person trying the product for the first time, who concludes
it is broken and never comes back.

An unrecognised body degrades to `capacity` — the safe guess, since its response
is harmless when wrong.
Shares are `capacity / active users`, recomputed per request, where *active*
means users who actually drew today — one user on a quiet day correctly gets
everything. The allowance unlocks gradually through the day, with a **one-turn
floor** so nobody's first question of the morning is refused.

## `fair-share` — divide a fixed daily pool
Whatever you pass as `costTokens` must err **high**: under-estimating admits
turns the pool cannot cover, draining the day while the gate still believes there
is room.

Pure policy: no database, no clock, no provider. You own *"what has this user
spent today"*; it owns *"may they spend more"*.
### Filling forms — from prose, then by talking to it

```ts
import { fairShare, utcDayElapsed } from 'ai-ration';

const decision = fairShare({
dayCapacityTokens: dayCapacityTokens(providers, process.env),
activeUsers, // distinct users who drew TODAY, including this one
userSpentTokens,
costTokens: ESTIMATED_TURN,
dayElapsed: utcDayElapsed(new Date()),
});
// decision.reason: 'ok' | 'paced' | 'share-spent' | 'no-capacity'
import { runFormAssist } from 'ai-kit';
import { useAiForm } from 'ai-kit/react';
import { createFormAssistHandler } from 'ai-kit/server';
```

Two ideas, both needed:
Re-exported from [`ai-forms`](https://github.com/maonakamoto/ai-forms), which
stays its own package — it works, four apps run it, and it is useful well outside
this fleet. Swallowing it would have broken those four for the sake of a filing
system.

1. **Share** = capacity ÷ users *active today*. Recomputed per request, so a new
user is counted the moment they arrive. Counting dormant accounts would ration
a quiet day down to nothing.
2. **Pacing** — a share is a whole-day allowance, and the day is consumed in
order. Without pacing, three users legitimately spend their full shares by
09:00 and the fourth finds nothing left. So the allowance unlocks gradually.
React lives on its own subpath and is an **optional** peer, so importing `ai-kit`
on a server never pulls in a UI library.

Plus a **one-turn floor**: the allowance never sits below the cost of a single
turn, so nobody's first question of the morning is refused. Without it, pure
pacing tells a first-time user to come back in three hours — indistinguishable
from broken.
---

`retryAfterSeconds` is present **only** for `paced`, the one refusal a wait
actually fixes. Telling someone whose share is spent to try again in 20 minutes
is the same lie as "try again shortly" on an exhausted daily quota.
## What it deliberately does not ship

No clawback: a user who spent under a larger share when they were alone is not
punished when a second user appears — their allowance simply stops growing.
**An HTTP client.** Every app has its own calling conventions, retries and
logging, and replacing those is a rewrite rather than an adoption. This supplies
the decisions; you keep the fetch.

## Estimating a turn
That rule is under review, and honestly. `ai-forms` is the most-adopted package
in this fleet and it is the one that broke the rule, by shipping a route factory
and a React hook. A package that hands you a working route gets installed; one
that hands you advice about routes does not.

Whatever you pass as `costTokens` must err **high**. Under-estimating admits
turns the pool cannot cover, draining the day while the gate still believes there
is room. One measured turn on a free model — a single tool call — cost **~16k
tokens**; an estimate of 10k had been documented as "deliberately on the high
side" and was 37% below it. Measure, then replace the estimate with the mean.
**Model values.** Which ids are free, which are billed, and which your account
may use are properties of *your* deployment. Centralise the rule, assert it
locally.

---

## Related

| Package | For |
|---|---|
| [`ai-forms`](https://github.com/maonakamoto/ai-forms) | Form filling on its own, without the model layer |
| [`threadkit`](https://github.com/maonakamoto/threadkit) | Messages between people, and who may see them |
| [`limitkit`](https://github.com/maonakamoto/limitkit) | Stopping someone doing something too often |

`threadkit` and `limitkit` are **not** merged in here, on purpose: neither has
anything to do with AI. An app that throttles its login form should not install a
model catalogue to do it.

## Development

```bash
npm run verify # build + test
npm run verify # lint + typecheck + build + test
```

MIT.
36 changes: 32 additions & 4 deletions package-lock.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading