feat: Wave 2 v1.0.0 rollout — config, SARIF, vision, per-file diff, auto-detect

[faraa2m]

Summary

Wave 2 of the v1.0.0 rollout. Pure additions across @tokenometer/core, the GitHub Action, and the CLI. No breaking changes.

What's in here

Core lib (@tokenometer/core)

• loadConfig / parseConfig — .tokenometer.yml loader with walk-up directory search. Halts at filesystem root or git root. Validates models against KNOWN_MODELS, formats against the supported set, budgets non-negative, paths array-of-strings.
• toSarif — SARIF 2.1.0 emitter. One result per (file × model × format) cell with prompt-cost ruleId.
• anthropicVisionTokens, openaiVisionTokens, googleVisionTokens — documented per-provider formulas for image tokenization.

GitHub Action (packages/action)

• Sticky PR comment now includes a top-N changed-file table with per-file Δ tokens and Δ USD, plus a collapsible "all files" details block.
• New optional top-n-files input (range 1–20, default 5).
• Per-file aggregator extracted to per-file-diff.ts for unit tests (11 tests + 4 markdown snapshots).
• dist/index.cjs rebuilt.

CLI (tokenometer)

• Auto provider detection: when --model is omitted, picks based on which of ANTHROPIC_API_KEY, OPENAI_API_KEY, GOOGLE_API_KEY / GEMINI_API_KEY is set. Stderr note when multiple are present (defaults to claude-opus-4-7).
• .tokenometer.yml config loading; --no-config to skip; --config <path> to point at a specific file. User flags always override config.
• --by-file: per-file token/cost attribution table (no-op for single file).
• --output table|json|sarif: machine-readable output. JSON emits TokenometerResult; SARIF passes through toSarif from core.
• --image <path> (repeatable): vision-token cost estimation. Each image becomes a virtual file row in the result; provider is inferred from the model.

Release workflow

• Chained npm run lint:fix after npx changeset version so Biome compact arrays survive Changesets' package.json rewrites.
• Direct fix for the lint failure pattern that broke PR chore(release): version packages #12 — going forward, the version PR's CI stays green without manual intervention.

Mistral integration research

• Memo at .planning/research/mistral-integration.md informs Wave 3 Phase H impl: ship mistral-tokenizer-js for SentencePiece family with approximate: true, fall back to char-ratio heuristic for Tekken models (NeMo / Pixtral / Devstral / Mistral Medium 2505+), defer empirical mode (no public token-count endpoint).

Why now

• Plan said Wave 2 = Phase C (feature wins) + Phase D (vision) + Phase H prep. All landed.
• The release workflow lint chain wasn't in the original plan but is required to keep the auto-bump pipeline working past Wave 1's keywords additions.

Test plan

• npm run lint — 61 files clean
• npm run typecheck — clean
• npm test — 150/150 across 16 files (was 48/6 before)
• npm run build — clean
• Action dist/index.cjs rebuilt and contains the new aggregatePerFileDiff / renderPerFileMarkdown symbols
• CLI smoke: --output table, --output json | jq '.files[0].path', --output sarif | jq '.version' (returns "2.1.0"), --by-file with multiple input files
• Action smoke (post-merge): tokenometer's own prompt-cost.yml runs against this PR and sticky comment renders the new per-file table
• Add a changeset for any follow-up tweaks before merging the resulting Version Packages PR

Out of scope (Wave 3 / 4)

• VS Code / Cursor extension (E.1)
• Claude Code skill (E.2 — confirmed this round)
• Playground showcase pages for the new features (F)
• --latency flag
• Mistral + Cohere implementation (per memo)
• Unified release pipeline phase I (Marketplace verify, VS Code publish, smoke test job)

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
tokenometer	Ready	Preview, Comment	May 10, 2026 3:14am