Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 11 additions & 7 deletions .github/workflows/model-pins.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,21 +53,25 @@ jobs:
- name: Self-test the detector
run: node scripts/ci/test-model-pin-audit.mjs

# dotfiles has no package.json on purpose. Install ai-ration into a
# scratch dir and point the audit at it — the same shape ui-defects.yml
# uses for playwright. The vendor query lives in ai-ration precisely so
# this repo does not grow a second copy of it.
- name: Install ai-ration
# dotfiles has no package.json on purpose. Install ai-kit into a scratch
# dir and point the audit at it — the same shape ui-defects.yml uses for
# playwright. The vendor query lives in ai-kit precisely so this repo does
# not grow a second copy of it.
#
# Installed from GitHub rather than npm: ai-kit v0.3.0 is tagged and built
# but not on the registry, because the repo has no NPM_TOKEN. Switch this
# to `npm i ai-kit` once it does.
- name: Install ai-kit
if: github.event_name != 'pull_request'
run: |
mkdir -p "$RUNNER_TEMP/air" && cd "$RUNNER_TEMP/air"
npm init -y >/dev/null
npm i --no-audit --no-fund github:maonakamoto/ai-ration#v0.2.1 >/dev/null
npm i --no-audit --no-fund github:maonakamoto/ai-kit#v0.3.0 >/dev/null

- name: Audit the fleet's pins
if: github.event_name != 'pull_request'
env:
AI_RATION_FROM: ${{ runner.temp }}/air
AI_KIT_FROM: ${{ runner.temp }}/air
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Absent keys are handled, not fatal: the vendor is reported UNCHECKED
# rather than clean. "I could not look" is not "nothing is wrong".
Expand Down
14 changes: 11 additions & 3 deletions SHARED.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,22 @@ the inventory underneath it is **generated**, and the number it produces is a
| Package | Install | Replaces |
|---|---|---|
| [`ai-forms`](https://github.com/maonakamoto/ai-forms) | `npm i github:maonakamoto/ai-forms#v0.1.0` | per-app "fill this form from prose" + conversational refinement. Headless — ships **no markup**, so each app keeps its own styling. |
| [`ai-ration`](https://github.com/maonakamoto/ai-ration) | `npm i github:maonakamoto/ai-ration#v0.2.0` | LLM free-tier survival: multi-vendor fallback chain, the three kinds of 429, per-user fair-share rationing, `modelCost()` so a fallback can never silently bill. |
| [`ai-kit`](https://github.com/maonakamoto/ai-kit) | `npm i github:maonakamoto/ai-kit#v0.3.0` | **the AI layer, in one install** — which model to call, whether the vendor still lists it, the three kinds of 429, fair-share of a free tier, and (re-exported) `ai-forms`. Renamed from `ai-ration` 2026-08-26: the name described one of five modules, and the package had one adopter while five repos that skipped it went down together to a retired model id. |
| [`threadkit`](https://github.com/maonakamoto/threadkit) | `npm i threadkit` | multi-participant message threads where *permission is participation*, not a role or an ownership column. Headless pure functions, so "who may read this" is unit-testable instead of buried in a `WHERE` clause. AI participants obey the same visibility rules. **ESM-only.** |
| [`limitkit`](https://github.com/maonakamoto/limitkit) | `npm i github:maonakamoto/limitkit#v0.1.0` | the fleet's **12 hand-rolled rate limiters** (this file's own "next extraction" row). Sliding/fixed windows over an injectable two-method `Store`; **bounded** memory default (the unbounded-Map leak is impossible by construction); standard `X-RateLimit-*` + `Retry-After` headers — what orangecat's ADR-0002 specified seven months before anything enforced it; `clientIp()`. Refusals count nothing, so a hammered key recovers. Ships no middleware and **no limit values** — how many attempts a route allows is app semantics, asserted locally. |

**Adopted:** `ai-forms` — fleetcrown, evig, aoz-housing, surf-your-life.
`ai-ration` — fleetcrown, **and this repo** (`model-pin-audit.mjs` calls
`ai-kit` — fleetcrown, **and this repo** (`model-pin-audit.mjs` calls
`checkCatalog`; the audit needed exactly the vendor query the package owns, so
writing a second one here would have been this file's own sin). `threadkit` — **nobody yet**.
writing a second one here would have been this file's own sin).

**On merging packages.** `ai-kit` absorbed `ai-ration` and re-exports
`ai-forms`, because to an app "which model do we call", "AI chat" and "AI form
fill" are ONE feature — AOZ adopted the form half, hand-rolled the rest, and was
taken down by the half it skipped. `threadkit` and `limitkit` are deliberately
NOT merged in: neither is about AI, and an app throttling its login form should
not install a model catalogue to do it. Merge by what a consumer needs together,
never by "these are all shared utilities". `threadkit` — **nobody yet**.
`limitkit` — fleetcrown (proving consumer; its old limiter had the unbounded
Map). **Next adopter should be orangecat** — it closes ADR-0002 by making its
Upstash client a 12-line `Store` adapter and deleting three of its four
Expand Down
26 changes: 16 additions & 10 deletions scripts/ci/model-pin-audit.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@
* runs in repos somebody still touches, and the repos that rot quietly are
* precisely the ones nobody touches. This one needs no adoption at all.
*
* WHY IT REUSES ai-ration
* -----------------------
* WHY IT REUSES ai-kit
* --------------------
* `checkCatalog` already answers this, already distinguishes the three states
* that matter, and already carries the scars — its own docstring records four
* of nine default pins gone and a consumer silently failing for eight days.
Expand All @@ -46,7 +46,7 @@
* ----------------------
* That a listed model WORKS. Existence is cheap; capability is not. A model
* can be listed and still refuse tool calls — of nine free models probed for
* ai-ration's default chain, five answered only via a text protocol. If the
* ai-kit's default chain, five answered only via a text protocol. If the
* surface is a tool loop, probe with a real tool call before pinning. This
* audit catches the retirement, not the mismatch.
*
Expand All @@ -61,7 +61,7 @@
*
* Env: GH_OWNER (default maonakamoto), GH_LIMIT (default 100),
* FLEET_ROOT (default ~/dev, --local only),
* AI_RATION_FROM (path to a repo that installs ai-ration),
* AI_KIT_FROM (path to a repo that installs ai-kit),
* GROQ_API_KEY / OPENROUTER_API_KEY (to read the catalogues).
*/
import { execFile } from "node:child_process";
Expand Down Expand Up @@ -410,26 +410,32 @@ function localFiles(repo) {
}));
}

// ── The catalogue, via ai-ration ─────────────────────────────────────────────
// ── The catalogue, via ai-kit ─────────────────────────────────────────────

function loadAiRation() {
function loadAiKit() {
const candidates = [
process.env.AI_KIT_FROM,
// Renamed from ai-ration in v0.3.0. The old paths stay in the list so a
// checkout that has not been renamed still resolves — a rename should not
// turn a working audit into a silent exit 2.
process.env.AI_RATION_FROM,
join(homedir(), "dev", "fleetcrown"),
join(homedir(), "dev", "ai-kit"),
join(homedir(), "dev", "ai-ration"),
].filter(Boolean);

for (const root of candidates) {
for (const entry of [
join(root, "node_modules", "ai-kit", "dist", "index.js"),
join(root, "node_modules", "ai-ration", "dist", "index.js"),
join(root, "dist", "index.js"),
]) {
if (existsSync(entry)) return import(entry);
}
}
console.error(
" ai-ration not found. Set AI_RATION_FROM=/path/to/a/repo that installs it,\n" +
" or build it once: (cd ~/dev/ai-ration && npm i && npm run build)",
"\u2717 ai-kit not found. Set AI_KIT_FROM=/path/to/a/repo that installs it,\n" +
" or build it once: (cd ~/dev/ai-kit && npm i && npm run build)",
);
process.exit(2);
}
Expand Down Expand Up @@ -525,7 +531,7 @@ function report(judged) {
if (gone.length) {
lines.push("");
lines.push("A pin is a scheduled outage. The durable fix is a chain across VENDORS —");
lines.push("see SHARED.md → ai-ration. Repinning buys time until the next retirement.");
lines.push("see SHARED.md → ai-kit. Repinning buys time until the next retirement.");
}

return lines.join("\n");
Expand All @@ -534,7 +540,7 @@ function report(judged) {
// ── Main ─────────────────────────────────────────────────────────────────────

export async function main() {
const { checkCatalog } = await loadAiRation();
const { checkCatalog } = await loadAiKit();

const repos = LOCAL ? localRepos() : await remoteRepos();
const findings = [];
Expand Down