feat: Wave 3 v1.0.0 rollout — VS Code ext, Claude Code skill, Mistral+Cohere, --latency, playground showcase#14
Merged
Merged
Conversation
…ere, latency, playground showcase Adds the headline differentiation features for v1.0.0. No breaking changes to existing flags or output paths. VS Code / Cursor extension (`packages/vscode/`, new package): - Status bar shows live token count + USD cost for the active editor file across Claude, GPT-4o, and Gemini. - Settings: model, format, warnOnCostAbove threshold (turns the bar background warning-yellow when exceeded). - Commands: Tokenometer: switch model, Tokenometer: show info. - Debounced 200ms recompute on document changes; 1MB file-size gate. - Reuses @tokenometer/core via esbuild bundle. Marketplace publish in Wave 4 Phase I. Claude Code skill (`packages/claude-code-skill/`, new package): - SKILL.md with `name: tokenometer-cost-check` + multi-trigger description. - 13-section body teaching Claude Code agents when and how to invoke `npx tokenometer` for prompt cost analysis: offline + empirical mode, --output json/sarif, --by-file, --image, auto provider detection, .tokenometer.yml config, CI guardrail via `tokenometer init`. - One-line install via `~/.claude/skills/tokenometer/SKILL.md`. Mistral + Cohere providers (`@tokenometer/core`): - Mistral: `mistral-tokenizer-js` for SentencePiece family (Mistral 7B, Mixtral 8x7B/8x22B, Mistral Large 2407, Codestral 22B); chars/4 heuristic for Tekken-tokenizer models (NeMo, Pixtral, Mistral Small 2409+, Devstral, Mistral Medium 2505+, Magistral, Ministral). All marked `approximate: true`. Empirical mode unsupported — Mistral has no public token-count API. - Cohere: chars/4 heuristic offline; empirical via POST /v1/tokenize using bare fetch (Cohere SDK ships only a REST wrapper, no offline tokenizer). - Pricing: Mistral auto-sourced from @tokenlens/models; Cohere via LOCAL_OVERRIDES until tokenlens adds upstream. Total KNOWN_MODELS: 42 → 63. Latency mode (`--latency` flag): - Measures real generation latency: TTFT (time to first token) + total wall-clock + tokens/sec. Reports p50/p95/mean over n trials (default 3, configurable 1-10 via --latency-trials). - Implies --empirical. Bumps default --max-spend ceiling to $0.25 to cover ~200-token generations. - Streamed via SDK for Anthropic/Google; bare fetch SSE/NDJSON for OpenAI/Mistral/Cohere (no extra SDK install footprint). - New table columns + JSON/SARIF carry the latency block. - Differentiation pitch: only LLM cost CLI that also reports latency. Playground showcase (`packages/web/`): - 11 new routes: /diff, /by-file, /sarif, /vision, /config-builder, /init, /models (Cost Atlas with sort/search/filter), /models/<id> (per-model SEO pages), /editor, /claude-code, plus / and /calculator for the existing landing. - react-router-dom 7, top nav with Tools dropdown, footer linking repo + npm + marketplace placeholders. - Cross-page provider robustness: Map<Provider, ...> instead of static Records so playground gracefully extends as new providers land. - Static robots.txt + sitemap.xml; per-page document.title via usePageTitle hook (no react-helmet dependency). Test count: 150 → 227 (across 20 files). Lint, typecheck, root + web builds all clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
…itions Wave 3 added 21 new model ids (Mistral × 19 + Cohere × 2), bringing KNOWN_MODELS from 42 to 63. Each prompt × format × new-model combination adds a fresh cell to the benchmark sweep — the drift detector caught 210 new cells (10 prompts × 21 models × 1 format check) and rightly failed CI. Re-ran `npm run benchmarks:regenerate` to refresh the committed dataset to 10 × 63 × 5 = 3,150 cells. No existing cells changed; only additions. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Action bundles @tokenometer/core via esbuild. Wave 3 added Mistral and Cohere providers (LOCAL_OVERRIDES entries, tokenize-mistral.ts, tokenize-cohere.ts), but the bundle wasn't re-emitted, so CI's bundle sync check rightly failed. Re-ran `npm run build --workspace=packages/action`. Bundle now reflects the new providers (+857 / -482 lines, mostly the new tokenize-cohere heuristic path and the LOCAL_OVERRIDES additions for command-r and command-r-plus). No source changes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
faraa2m
added a commit
that referenced
this pull request
May 11, 2026
…+Cohere, --latency, playground showcase (#14) * feat: Wave 3 v1.0.0 rollout — VS Code, Claude Code skill, Mistral+Cohere, latency, playground showcase Adds the headline differentiation features for v1.0.0. No breaking changes to existing flags or output paths. VS Code / Cursor extension (`packages/vscode/`, new package): - Status bar shows live token count + USD cost for the active editor file across Claude, GPT-4o, and Gemini. - Settings: model, format, warnOnCostAbove threshold (turns the bar background warning-yellow when exceeded). - Commands: Tokenometer: switch model, Tokenometer: show info. - Debounced 200ms recompute on document changes; 1MB file-size gate. - Reuses @tokenometer/core via esbuild bundle. Marketplace publish in Wave 4 Phase I. Claude Code skill (`packages/claude-code-skill/`, new package): - SKILL.md with `name: tokenometer-cost-check` + multi-trigger description. - 13-section body teaching Claude Code agents when and how to invoke `npx tokenometer` for prompt cost analysis: offline + empirical mode, --output json/sarif, --by-file, --image, auto provider detection, .tokenometer.yml config, CI guardrail via `tokenometer init`. - One-line install via `~/.claude/skills/tokenometer/SKILL.md`. Mistral + Cohere providers (`@tokenometer/core`): - Mistral: `mistral-tokenizer-js` for SentencePiece family (Mistral 7B, Mixtral 8x7B/8x22B, Mistral Large 2407, Codestral 22B); chars/4 heuristic for Tekken-tokenizer models (NeMo, Pixtral, Mistral Small 2409+, Devstral, Mistral Medium 2505+, Magistral, Ministral). All marked `approximate: true`. Empirical mode unsupported — Mistral has no public token-count API. - Cohere: chars/4 heuristic offline; empirical via POST /v1/tokenize using bare fetch (Cohere SDK ships only a REST wrapper, no offline tokenizer). - Pricing: Mistral auto-sourced from @tokenlens/models; Cohere via LOCAL_OVERRIDES until tokenlens adds upstream. Total KNOWN_MODELS: 42 → 63. Latency mode (`--latency` flag): - Measures real generation latency: TTFT (time to first token) + total wall-clock + tokens/sec. Reports p50/p95/mean over n trials (default 3, configurable 1-10 via --latency-trials). - Implies --empirical. Bumps default --max-spend ceiling to $0.25 to cover ~200-token generations. - Streamed via SDK for Anthropic/Google; bare fetch SSE/NDJSON for OpenAI/Mistral/Cohere (no extra SDK install footprint). - New table columns + JSON/SARIF carry the latency block. - Differentiation pitch: only LLM cost CLI that also reports latency. Playground showcase (`packages/web/`): - 11 new routes: /diff, /by-file, /sarif, /vision, /config-builder, /init, /models (Cost Atlas with sort/search/filter), /models/<id> (per-model SEO pages), /editor, /claude-code, plus / and /calculator for the existing landing. - react-router-dom 7, top nav with Tools dropdown, footer linking repo + npm + marketplace placeholders. - Cross-page provider robustness: Map<Provider, ...> instead of static Records so playground gracefully extends as new providers land. - Static robots.txt + sitemap.xml; per-page document.title via usePageTitle hook (no react-helmet dependency). Test count: 150 → 227 (across 20 files). Lint, typecheck, root + web builds all clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(benchmarks): regenerate results.json after Mistral + Cohere additions Wave 3 added 21 new model ids (Mistral × 19 + Cohere × 2), bringing KNOWN_MODELS from 42 to 63. Each prompt × format × new-model combination adds a fresh cell to the benchmark sweep — the drift detector caught 210 new cells (10 prompts × 21 models × 1 format check) and rightly failed CI. Re-ran `npm run benchmarks:regenerate` to refresh the committed dataset to 10 × 63 × 5 = 3,150 cells. No existing cells changed; only additions. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(action): rebuild dist/index.cjs after Mistral + Cohere additions The Action bundles @tokenometer/core via esbuild. Wave 3 added Mistral and Cohere providers (LOCAL_OVERRIDES entries, tokenize-mistral.ts, tokenize-cohere.ts), but the bundle wasn't re-emitted, so CI's bundle sync check rightly failed. Re-ran `npm run build --workspace=packages/action`. Bundle now reflects the new providers (+857 / -482 lines, mostly the new tokenize-cohere heuristic path and the LOCAL_OVERRIDES additions for command-r and command-r-plus). No source changes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Wave 3 of the v1.0.0 rollout. Five workstreams, all additive — no breaking changes.
What's in here
VS Code / Cursor extension (
packages/vscode/, new package)<model> · <tokens> tok · $<cost>for the active editor file.tokenometer.model,tokenometer.format,tokenometer.warnOnCostAbove(turns the bar warning-yellow when exceeded).Tokenometer: switch model(quick-pick acrossKNOWN_MODELS),Tokenometer: show info..md/.txt/.json/.yaml/.yml/.xml.@tokenometer/corevia esbuild bundle. Marketplace publish lands in Wave 4 Phase I.Claude Code skill (
packages/claude-code-skill/, new package)SKILL.mdwithname: tokenometer-cost-check+ multi-triggerdescription.npx tokenometer: offline + empirical mode,--output json/sarif,--by-file,--image, auto provider detection,.tokenometer.yml,tokenometer initfor CI scaffold.~/.claude/skills/tokenometer/SKILL.md. Convenienceinstall.shincluded.Mistral + Cohere providers (
@tokenometer/core)mistral-tokenizer-jsfor SentencePiece family (Mistral 7B, Mixtral 8x7B/8x22B, Mistral Large 2407, Codestral 22B);chars/4heuristic for Tekken-tokenizer models (NeMo, Pixtral, Mistral Small 2409+, Devstral, Mistral Medium 2505+, Magistral, Ministral). All markedapproximate: true. Empirical mode unsupported — Mistral has no public token-count API; throws a clear error pointing at the metered chat-completion fallback.chars/4heuristic offline. Empirical viaPOST /v1/tokenizeusing barefetch(Cohere SDK ships only a REST wrapper, no offline tokenizer — saved 4.9 MB install).@tokenlens/models/mistral(19 models). Cohere viaLOCAL_OVERRIDES(command-r,command-r-plus) until tokenlens adds upstream.KNOWN_MODELScount: 42 → 63.--latencyflag (CLI + core)--latency-trials).--empirical. Bumps default--max-spendceiling to $0.25 to cover ~200-token generations.fetchSSE/NDJSON for OpenAI + Mistral + Cohere (avoids extra SDK install footprint).latencyblock.tokencost,tiktoken,gpt-token-counter-live) ships latency.Playground showcase (
packages/web/)react-router-dom@7:/diff,/by-file,/sarif,/vision,/config-builder,/init,/models(Cost Atlas: sortable + searchable + filter-by-provider),/models/<id>(per-model SEO pages),/editor,/claude-code, plus/and/calculatorfor the existing landing.Map<Provider, ...>everywhere instead of staticRecords — playground gracefully extends as more providers land.robots.txt+sitemap.xml; per-pagedocument.titleviausePageTitlehook (noreact-helmetdependency).Why now
Test plan
npm run lint— 89 files cleannpm run typecheck— cleannpm test— 227/227 tests across 20 files (was 150/16 before Wave 3)npm run build— cleannpm run build -w @tokenometer/web— clean (4.43 MB main JS, 2.01 MB gzipped, larger-than-500kB warning is acknowledged + accepted for now)version: "2.1.0";--by-file/--output json/--imagepaths exercised by integration testsprompt-cost.ymlruns and the per-file Action comment renders.vsix(built locally vianpm run package:vsix --workspace=packages/vscode) and confirm status bar renders. Marketplace publish deferred to Phase I.cp packages/claude-code-skill/SKILL.md ~/.claude/skills/tokenometer/then issue a test prompt to confirm Claude Code triggers on itOut of scope (Wave 4)
vsce, Open VSX publish, post-publish smoke test job, Vercel deploy hook, Marketplace verify HTTP probe. Right now the release workflow only handles npm via Changesets.~/Downloads/tokenometer-content-todo.md; manual posting after Wave 4 is green.🤖 Generated with Claude Code