diff --git a/.changeset/legit-docs-and-seo.md b/.changeset/legit-docs-and-seo.md deleted file mode 100644 index 4892a46..0000000 --- a/.changeset/legit-docs-and-seo.md +++ /dev/null @@ -1,13 +0,0 @@ ---- -"tokenometer": minor -"@tokenometer/core": minor ---- - -Project legitimacy and SEO rollout (Wave 1 of v1.0.0): - -- Added `CODE_OF_CONDUCT.md` (Contributor Covenant 2.1), `CONTRIBUTING.md`, `SECURITY.md` (GitHub Private Vulnerability Reporting only — no email exposure), `CHANGELOG.md`. -- Added `.github/copilot-instructions.md` for `gh copilot review`, plus `PULL_REQUEST_TEMPLATE.md`, issue forms (`bug_report.yml`, `feature_request.yml`, `config.yml`), and `FUNDING.yml`. -- Rewrote root `README.md` with badges row, "Why Tokenometer vs alternatives" comparison table (vs `tokencost`, `tiktoken`, `gpt-tokenizer`, `promptfoo`, `gpt-token-counter-live`), ASCII demo of CLI table output, and project-health checklist. -- Extended npm `keywords` and `description` across `tokenometer` (CLI), `@tokenometer/core`, and `@tokenometer/action` for SEO. Root description now reads: "Tokenometer — LLM cost calculator, token counter, and CI cost-guardrail Action for Claude, GPT-4o, Gemini." -- Added Marketplace publish prep (`.github/release-notes-v1.0.0.md`) and awesome-list PR templates (`.github/awesome-list-prs/`). -- Initialized Changesets for auto-bump + auto-CHANGELOG-generation on release. Replaced the manual `release.yml` workflow with the Changesets-driven version-PR + publish flow. diff --git a/.changeset/wave-2-action-per-file-diff.md b/.changeset/wave-2-action-per-file-diff.md deleted file mode 100644 index 8add080..0000000 --- a/.changeset/wave-2-action-per-file-diff.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -"tokenometer": minor -"@tokenometer/core": minor ---- - -Action sticky PR comment now includes a top-N changed-file table with per-file -Δ tokens and Δ USD, plus a collapsible "all files" section. New optional -`top-n-files` input controls N (default 5). diff --git a/.changeset/wave-2-cli-wiring.md b/.changeset/wave-2-cli-wiring.md deleted file mode 100644 index 869dda6..0000000 --- a/.changeset/wave-2-cli-wiring.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -"tokenometer": minor -"@tokenometer/core": minor ---- - -CLI gains: -- Auto provider detection when `--model` is omitted (picks based on which `*_API_KEY` env is set). -- `.tokenometer.yml` config loading (walk-up); `--no-config` and `--config ` overrides. -- `--by-file` per-file token/cost attribution table. -- `--output table|json|sarif` for machine-readable output. -- `--image ` (repeatable) for vision-token cost estimation across Claude / GPT-4o / Gemini. diff --git a/.changeset/wave-2-core-additions.md b/.changeset/wave-2-core-additions.md deleted file mode 100644 index 6f981b9..0000000 --- a/.changeset/wave-2-core-additions.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -"tokenometer": minor -"@tokenometer/core": minor ---- - -Core lib additions for v1.0.0: - -- `loadConfig` / `parseConfig` for `.tokenometer.yml`. -- `toSarif` SARIF 2.1.0 output. -- Vision token estimators for Anthropic, OpenAI, and Google. diff --git a/.changeset/wave-3-claude-code-skill.md b/.changeset/wave-3-claude-code-skill.md deleted file mode 100644 index 75f48a6..0000000 --- a/.changeset/wave-3-claude-code-skill.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -"tokenometer": minor -"@tokenometer/core": minor ---- - -Add `@tokenometer/claude-code-skill` — a Claude Code skill that teaches -Claude Code agents to invoke `npx tokenometer` for prompt-cost analysis. -Install via `~/.claude/skills/tokenometer/SKILL.md`. Submission to -community skill registry tracked separately. diff --git a/.changeset/wave-3-latency.md b/.changeset/wave-3-latency.md deleted file mode 100644 index 45b5aab..0000000 --- a/.changeset/wave-3-latency.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -"tokenometer": minor -"@tokenometer/core": minor ---- - -Add `--latency` flag — measures real generation latency (TTFT + total ms + -tokens/sec, p50/p95/mean over n trials) alongside token cost. Implies -`--empirical`. Default trials = 3, configurable via `--latency-trials ` -(1-10). Bumps default `--max-spend` to $0.25 to cover the n × 200-token -generations. Supported providers: Anthropic, OpenAI, Google, Cohere, -Mistral (latter two are metered). diff --git a/.changeset/wave-3-mistral-cohere.md b/.changeset/wave-3-mistral-cohere.md deleted file mode 100644 index 2e3cbf1..0000000 --- a/.changeset/wave-3-mistral-cohere.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -"tokenometer": minor -"@tokenometer/core": minor ---- - -Add Mistral and Cohere providers. - -- Mistral: `mistral-tokenizer-js` for SentencePiece family (Mistral 7B, - Mixtral, Mistral Large 2407, Codestral); `chars/4` heuristic for Tekken - models (NeMo, Pixtral, Mistral Small 2409+, Devstral, Mistral Medium - 2505+, Magistral, Ministral). All marked `approximate: true`. Empirical - mode unsupported (Mistral has no public token-count API). -- Cohere: offline heuristic `chars/4` (Cohere SDK is REST-only; no offline - tokenizer ships in JS). Empirical via `POST /v1/tokenize` when - `COHERE_API_KEY` is set. - -Pricing for Mistral auto-sourced from `@tokenlens/models/mistral`. Cohere -pricing comes from `LOCAL_OVERRIDES` (`command-r`, `command-r-plus`) -because `@tokenlens/models` does not yet ship a Cohere catalog at v1.3.0. diff --git a/.changeset/wave-3-playground-showcase.md b/.changeset/wave-3-playground-showcase.md deleted file mode 100644 index 9023fe2..0000000 --- a/.changeset/wave-3-playground-showcase.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -"tokenometer": minor -"@tokenometer/core": minor ---- - -Playground (`https://tokenometer.vercel.app`) gains showcase pages for -every Wave 2 feature: `/diff`, `/by-file`, `/sarif`, `/vision`, -`/config-builder`, `/init`, `/models` (Cost Atlas + per-model SEO pages), -plus placeholder pages for the VS Code extension and Claude Code skill. - - \ No newline at end of file diff --git a/.changeset/wave-3-vscode-extension.md b/.changeset/wave-3-vscode-extension.md deleted file mode 100644 index 76f09d9..0000000 --- a/.changeset/wave-3-vscode-extension.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -"tokenometer": minor -"@tokenometer/core": minor ---- - -Add VS Code / Cursor extension (`@tokenometer/vscode`). Status bar shows -live token count + USD cost for the active editor file across Claude, -GPT-4o, and Gemini. Reuses `@tokenometer/core`. Marketplace publish -follows in Phase I. diff --git a/.changeset/wave-4-docs-seo-sweep.md b/.changeset/wave-4-docs-seo-sweep.md deleted file mode 100644 index a69f9c4..0000000 --- a/.changeset/wave-4-docs-seo-sweep.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -"tokenometer": minor -"@tokenometer/core": minor ---- - -Documentation + SEO sweep: README, per-package READMEs, package.json -descriptions and keywords updated to reflect every shipped feature -(CLI flags, GitHub Action, VS Code extension, Claude Code skill, -Mistral + Cohere providers, latency benchmarking, vision tokens, SARIF). -Repo metadata + awesome-list templates regenerated. No source code -changes. diff --git a/.changeset/wave-4-phase-i-release-pipeline.md b/.changeset/wave-4-phase-i-release-pipeline.md deleted file mode 100644 index 67f8bcc..0000000 --- a/.changeset/wave-4-phase-i-release-pipeline.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -"tokenometer": minor -"@tokenometer/core": minor ---- - -Unified release pipeline (Phase I): one merge of the Version Packages PR -publishes tokenometer + @tokenometer/core to npm with provenance, creates -the GitHub Release (which republishes the Action to GitHub Marketplace), -publishes the VS Code extension to VS Code Marketplace + Open VSX, runs -a post-publish smoke test, verifies the Marketplace listing, and triggers -the Vercel deploy hook. Local `npm run smoke` runs the full sweep (lint, -typecheck, test, build, benchmarks, CLI smoke). diff --git a/packages/action/package.json b/packages/action/package.json index aa67089..d5018c6 100644 --- a/packages/action/package.json +++ b/packages/action/package.json @@ -30,7 +30,7 @@ "@actions/core": "^1.11.1", "@actions/exec": "^1.1.1", "@actions/github": "^6.0.0", - "@tokenometer/core": "0.0.2", + "@tokenometer/core": "0.1.0", "minimatch": "^10.2.5" }, "devDependencies": { diff --git a/packages/cli/CHANGELOG.md b/packages/cli/CHANGELOG.md new file mode 100644 index 0000000..e6c04c9 --- /dev/null +++ b/packages/cli/CHANGELOG.md @@ -0,0 +1,91 @@ +# tokenometer + +## 0.1.0 + +### Minor Changes + +- [#11](https://github.com/faraa2m/tokenometer/pull/11) [`a864454`](https://github.com/faraa2m/tokenometer/commit/a8644542d65641a4d8804ec4c4153bcfb2ccbf10) Thanks [@faraa2m](https://github.com/faraa2m)! - Project legitimacy and SEO rollout (Wave 1 of v1.0.0): + + - Added `CODE_OF_CONDUCT.md` (Contributor Covenant 2.1), `CONTRIBUTING.md`, `SECURITY.md` (GitHub Private Vulnerability Reporting only — no email exposure), `CHANGELOG.md`. + - Added `.github/copilot-instructions.md` for `gh copilot review`, plus `PULL_REQUEST_TEMPLATE.md`, issue forms (`bug_report.yml`, `feature_request.yml`, `config.yml`), and `FUNDING.yml`. + - Rewrote root `README.md` with badges row, "Why Tokenometer vs alternatives" comparison table (vs `tokencost`, `tiktoken`, `gpt-tokenizer`, `promptfoo`, `gpt-token-counter-live`), ASCII demo of CLI table output, and project-health checklist. + - Extended npm `keywords` and `description` across `tokenometer` (CLI), `@tokenometer/core`, and `@tokenometer/action` for SEO. Root description now reads: "Tokenometer — LLM cost calculator, token counter, and CI cost-guardrail Action for Claude, GPT-4o, Gemini." + - Added Marketplace publish prep (`.github/release-notes-v1.0.0.md`) and awesome-list PR templates (`.github/awesome-list-prs/`). + - Initialized Changesets for auto-bump + auto-CHANGELOG-generation on release. Replaced the manual `release.yml` workflow with the Changesets-driven version-PR + publish flow. + +- [#13](https://github.com/faraa2m/tokenometer/pull/13) [`cc745d5`](https://github.com/faraa2m/tokenometer/commit/cc745d5701bec2e9fd4f52afd008b87e63f06802) Thanks [@faraa2m](https://github.com/faraa2m)! - Action sticky PR comment now includes a top-N changed-file table with per-file + Δ tokens and Δ USD, plus a collapsible "all files" section. New optional + `top-n-files` input controls N (default 5). + +- [#13](https://github.com/faraa2m/tokenometer/pull/13) [`cc745d5`](https://github.com/faraa2m/tokenometer/commit/cc745d5701bec2e9fd4f52afd008b87e63f06802) Thanks [@faraa2m](https://github.com/faraa2m)! - CLI gains: + + - Auto provider detection when `--model` is omitted (picks based on which `*_API_KEY` env is set). + - `.tokenometer.yml` config loading (walk-up); `--no-config` and `--config ` overrides. + - `--by-file` per-file token/cost attribution table. + - `--output table|json|sarif` for machine-readable output. + - `--image ` (repeatable) for vision-token cost estimation across Claude / GPT-4o / Gemini. + +- [#13](https://github.com/faraa2m/tokenometer/pull/13) [`cc745d5`](https://github.com/faraa2m/tokenometer/commit/cc745d5701bec2e9fd4f52afd008b87e63f06802) Thanks [@faraa2m](https://github.com/faraa2m)! - Core lib additions for v1.0.0: + + - `loadConfig` / `parseConfig` for `.tokenometer.yml`. + - `toSarif` SARIF 2.1.0 output. + - Vision token estimators for Anthropic, OpenAI, and Google. + +- [#14](https://github.com/faraa2m/tokenometer/pull/14) [`061e0c5`](https://github.com/faraa2m/tokenometer/commit/061e0c577fce3ecc09c6a201a408cd1a360d5587) Thanks [@faraa2m](https://github.com/faraa2m)! - Add `@tokenometer/claude-code-skill` — a Claude Code skill that teaches + Claude Code agents to invoke `npx tokenometer` for prompt-cost analysis. + Install via `~/.claude/skills/tokenometer/SKILL.md`. Submission to + community skill registry tracked separately. + +- [#14](https://github.com/faraa2m/tokenometer/pull/14) [`061e0c5`](https://github.com/faraa2m/tokenometer/commit/061e0c577fce3ecc09c6a201a408cd1a360d5587) Thanks [@faraa2m](https://github.com/faraa2m)! - Add `--latency` flag — measures real generation latency (TTFT + total ms + + tokens/sec, p50/p95/mean over n trials) alongside token cost. Implies + `--empirical`. Default trials = 3, configurable via `--latency-trials ` + (1-10). Bumps default `--max-spend` to $0.25 to cover the n × 200-token + generations. Supported providers: Anthropic, OpenAI, Google, Cohere, + Mistral (latter two are metered). + +- [#14](https://github.com/faraa2m/tokenometer/pull/14) [`061e0c5`](https://github.com/faraa2m/tokenometer/commit/061e0c577fce3ecc09c6a201a408cd1a360d5587) Thanks [@faraa2m](https://github.com/faraa2m)! - Add Mistral and Cohere providers. + + - Mistral: `mistral-tokenizer-js` for SentencePiece family (Mistral 7B, + Mixtral, Mistral Large 2407, Codestral); `chars/4` heuristic for Tekken + models (NeMo, Pixtral, Mistral Small 2409+, Devstral, Mistral Medium + 2505+, Magistral, Ministral). All marked `approximate: true`. Empirical + mode unsupported (Mistral has no public token-count API). + - Cohere: offline heuristic `chars/4` (Cohere SDK is REST-only; no offline + tokenizer ships in JS). Empirical via `POST /v1/tokenize` when + `COHERE_API_KEY` is set. + + Pricing for Mistral auto-sourced from `@tokenlens/models/mistral`. Cohere + pricing comes from `LOCAL_OVERRIDES` (`command-r`, `command-r-plus`) + because `@tokenlens/models` does not yet ship a Cohere catalog at v1.3.0. + +- [#14](https://github.com/faraa2m/tokenometer/pull/14) [`061e0c5`](https://github.com/faraa2m/tokenometer/commit/061e0c577fce3ecc09c6a201a408cd1a360d5587) Thanks [@faraa2m](https://github.com/faraa2m)! - Playground (`https://tokenometer.vercel.app`) gains showcase pages for + every Wave 2 feature: `/diff`, `/by-file`, `/sarif`, `/vision`, + `/config-builder`, `/init`, `/models` (Cost Atlas + per-model SEO pages), + plus placeholder pages for the VS Code extension and Claude Code skill. + + + +- [#14](https://github.com/faraa2m/tokenometer/pull/14) [`061e0c5`](https://github.com/faraa2m/tokenometer/commit/061e0c577fce3ecc09c6a201a408cd1a360d5587) Thanks [@faraa2m](https://github.com/faraa2m)! - Add VS Code / Cursor extension (`@tokenometer/vscode`). Status bar shows + live token count + USD cost for the active editor file across Claude, + GPT-4o, and Gemini. Reuses `@tokenometer/core`. Marketplace publish + follows in Phase I. + +- [#15](https://github.com/faraa2m/tokenometer/pull/15) [`3681ae3`](https://github.com/faraa2m/tokenometer/commit/3681ae35d021a22a300271e2ea945cfeaeca9a0e) Thanks [@faraa2m](https://github.com/faraa2m)! - Documentation + SEO sweep: README, per-package READMEs, package.json + descriptions and keywords updated to reflect every shipped feature + (CLI flags, GitHub Action, VS Code extension, Claude Code skill, + Mistral + Cohere providers, latency benchmarking, vision tokens, SARIF). + Repo metadata + awesome-list templates regenerated. No source code + changes. + +- [#16](https://github.com/faraa2m/tokenometer/pull/16) [`dd9dab3`](https://github.com/faraa2m/tokenometer/commit/dd9dab3e06bbaf374b72d1903055b8a22472e5a1) Thanks [@faraa2m](https://github.com/faraa2m)! - Unified release pipeline (Phase I): one merge of the Version Packages PR + publishes tokenometer + @tokenometer/core to npm with provenance, creates + the GitHub Release (which republishes the Action to GitHub Marketplace), + publishes the VS Code extension to VS Code Marketplace + Open VSX, runs + a post-publish smoke test, verifies the Marketplace listing, and triggers + the Vercel deploy hook. Local `npm run smoke` runs the full sweep (lint, + typecheck, test, build, benchmarks, CLI smoke). + +### Patch Changes + +- Updated dependencies [[`a864454`](https://github.com/faraa2m/tokenometer/commit/a8644542d65641a4d8804ec4c4153bcfb2ccbf10), [`cc745d5`](https://github.com/faraa2m/tokenometer/commit/cc745d5701bec2e9fd4f52afd008b87e63f06802), [`cc745d5`](https://github.com/faraa2m/tokenometer/commit/cc745d5701bec2e9fd4f52afd008b87e63f06802), [`cc745d5`](https://github.com/faraa2m/tokenometer/commit/cc745d5701bec2e9fd4f52afd008b87e63f06802), [`061e0c5`](https://github.com/faraa2m/tokenometer/commit/061e0c577fce3ecc09c6a201a408cd1a360d5587), [`061e0c5`](https://github.com/faraa2m/tokenometer/commit/061e0c577fce3ecc09c6a201a408cd1a360d5587), [`061e0c5`](https://github.com/faraa2m/tokenometer/commit/061e0c577fce3ecc09c6a201a408cd1a360d5587), [`061e0c5`](https://github.com/faraa2m/tokenometer/commit/061e0c577fce3ecc09c6a201a408cd1a360d5587), [`061e0c5`](https://github.com/faraa2m/tokenometer/commit/061e0c577fce3ecc09c6a201a408cd1a360d5587), [`3681ae3`](https://github.com/faraa2m/tokenometer/commit/3681ae35d021a22a300271e2ea945cfeaeca9a0e), [`dd9dab3`](https://github.com/faraa2m/tokenometer/commit/dd9dab3e06bbaf374b72d1903055b8a22472e5a1)]: + - @tokenometer/core@0.1.0 diff --git a/packages/cli/package.json b/packages/cli/package.json index 9d271cd..42ce7bb 100644 --- a/packages/cli/package.json +++ b/packages/cli/package.json @@ -1,6 +1,6 @@ { "name": "tokenometer", - "version": "0.0.2", + "version": "0.1.0", "description": "Tokenometer CLI — LLM token cost + latency benchmarking across Claude, GPT-4o, Gemini, Mistral, and Cohere. Multi-format, empirical mode, vision tokens, SARIF output.", "license": "MIT", "author": "Faraazuddin Mohammed ", @@ -79,7 +79,7 @@ "clean": "rm -rf dist" }, "dependencies": { - "@tokenometer/core": "0.0.2", + "@tokenometer/core": "0.1.0", "image-size": "^2.0.2" }, "devDependencies": { diff --git a/packages/core/CHANGELOG.md b/packages/core/CHANGELOG.md new file mode 100644 index 0000000..78b8d6d --- /dev/null +++ b/packages/core/CHANGELOG.md @@ -0,0 +1,86 @@ +# @tokenometer/core + +## 0.1.0 + +### Minor Changes + +- [#11](https://github.com/faraa2m/tokenometer/pull/11) [`a864454`](https://github.com/faraa2m/tokenometer/commit/a8644542d65641a4d8804ec4c4153bcfb2ccbf10) Thanks [@faraa2m](https://github.com/faraa2m)! - Project legitimacy and SEO rollout (Wave 1 of v1.0.0): + + - Added `CODE_OF_CONDUCT.md` (Contributor Covenant 2.1), `CONTRIBUTING.md`, `SECURITY.md` (GitHub Private Vulnerability Reporting only — no email exposure), `CHANGELOG.md`. + - Added `.github/copilot-instructions.md` for `gh copilot review`, plus `PULL_REQUEST_TEMPLATE.md`, issue forms (`bug_report.yml`, `feature_request.yml`, `config.yml`), and `FUNDING.yml`. + - Rewrote root `README.md` with badges row, "Why Tokenometer vs alternatives" comparison table (vs `tokencost`, `tiktoken`, `gpt-tokenizer`, `promptfoo`, `gpt-token-counter-live`), ASCII demo of CLI table output, and project-health checklist. + - Extended npm `keywords` and `description` across `tokenometer` (CLI), `@tokenometer/core`, and `@tokenometer/action` for SEO. Root description now reads: "Tokenometer — LLM cost calculator, token counter, and CI cost-guardrail Action for Claude, GPT-4o, Gemini." + - Added Marketplace publish prep (`.github/release-notes-v1.0.0.md`) and awesome-list PR templates (`.github/awesome-list-prs/`). + - Initialized Changesets for auto-bump + auto-CHANGELOG-generation on release. Replaced the manual `release.yml` workflow with the Changesets-driven version-PR + publish flow. + +- [#13](https://github.com/faraa2m/tokenometer/pull/13) [`cc745d5`](https://github.com/faraa2m/tokenometer/commit/cc745d5701bec2e9fd4f52afd008b87e63f06802) Thanks [@faraa2m](https://github.com/faraa2m)! - Action sticky PR comment now includes a top-N changed-file table with per-file + Δ tokens and Δ USD, plus a collapsible "all files" section. New optional + `top-n-files` input controls N (default 5). + +- [#13](https://github.com/faraa2m/tokenometer/pull/13) [`cc745d5`](https://github.com/faraa2m/tokenometer/commit/cc745d5701bec2e9fd4f52afd008b87e63f06802) Thanks [@faraa2m](https://github.com/faraa2m)! - CLI gains: + + - Auto provider detection when `--model` is omitted (picks based on which `*_API_KEY` env is set). + - `.tokenometer.yml` config loading (walk-up); `--no-config` and `--config ` overrides. + - `--by-file` per-file token/cost attribution table. + - `--output table|json|sarif` for machine-readable output. + - `--image ` (repeatable) for vision-token cost estimation across Claude / GPT-4o / Gemini. + +- [#13](https://github.com/faraa2m/tokenometer/pull/13) [`cc745d5`](https://github.com/faraa2m/tokenometer/commit/cc745d5701bec2e9fd4f52afd008b87e63f06802) Thanks [@faraa2m](https://github.com/faraa2m)! - Core lib additions for v1.0.0: + + - `loadConfig` / `parseConfig` for `.tokenometer.yml`. + - `toSarif` SARIF 2.1.0 output. + - Vision token estimators for Anthropic, OpenAI, and Google. + +- [#14](https://github.com/faraa2m/tokenometer/pull/14) [`061e0c5`](https://github.com/faraa2m/tokenometer/commit/061e0c577fce3ecc09c6a201a408cd1a360d5587) Thanks [@faraa2m](https://github.com/faraa2m)! - Add `@tokenometer/claude-code-skill` — a Claude Code skill that teaches + Claude Code agents to invoke `npx tokenometer` for prompt-cost analysis. + Install via `~/.claude/skills/tokenometer/SKILL.md`. Submission to + community skill registry tracked separately. + +- [#14](https://github.com/faraa2m/tokenometer/pull/14) [`061e0c5`](https://github.com/faraa2m/tokenometer/commit/061e0c577fce3ecc09c6a201a408cd1a360d5587) Thanks [@faraa2m](https://github.com/faraa2m)! - Add `--latency` flag — measures real generation latency (TTFT + total ms + + tokens/sec, p50/p95/mean over n trials) alongside token cost. Implies + `--empirical`. Default trials = 3, configurable via `--latency-trials ` + (1-10). Bumps default `--max-spend` to $0.25 to cover the n × 200-token + generations. Supported providers: Anthropic, OpenAI, Google, Cohere, + Mistral (latter two are metered). + +- [#14](https://github.com/faraa2m/tokenometer/pull/14) [`061e0c5`](https://github.com/faraa2m/tokenometer/commit/061e0c577fce3ecc09c6a201a408cd1a360d5587) Thanks [@faraa2m](https://github.com/faraa2m)! - Add Mistral and Cohere providers. + + - Mistral: `mistral-tokenizer-js` for SentencePiece family (Mistral 7B, + Mixtral, Mistral Large 2407, Codestral); `chars/4` heuristic for Tekken + models (NeMo, Pixtral, Mistral Small 2409+, Devstral, Mistral Medium + 2505+, Magistral, Ministral). All marked `approximate: true`. Empirical + mode unsupported (Mistral has no public token-count API). + - Cohere: offline heuristic `chars/4` (Cohere SDK is REST-only; no offline + tokenizer ships in JS). Empirical via `POST /v1/tokenize` when + `COHERE_API_KEY` is set. + + Pricing for Mistral auto-sourced from `@tokenlens/models/mistral`. Cohere + pricing comes from `LOCAL_OVERRIDES` (`command-r`, `command-r-plus`) + because `@tokenlens/models` does not yet ship a Cohere catalog at v1.3.0. + +- [#14](https://github.com/faraa2m/tokenometer/pull/14) [`061e0c5`](https://github.com/faraa2m/tokenometer/commit/061e0c577fce3ecc09c6a201a408cd1a360d5587) Thanks [@faraa2m](https://github.com/faraa2m)! - Playground (`https://tokenometer.vercel.app`) gains showcase pages for + every Wave 2 feature: `/diff`, `/by-file`, `/sarif`, `/vision`, + `/config-builder`, `/init`, `/models` (Cost Atlas + per-model SEO pages), + plus placeholder pages for the VS Code extension and Claude Code skill. + + + +- [#14](https://github.com/faraa2m/tokenometer/pull/14) [`061e0c5`](https://github.com/faraa2m/tokenometer/commit/061e0c577fce3ecc09c6a201a408cd1a360d5587) Thanks [@faraa2m](https://github.com/faraa2m)! - Add VS Code / Cursor extension (`@tokenometer/vscode`). Status bar shows + live token count + USD cost for the active editor file across Claude, + GPT-4o, and Gemini. Reuses `@tokenometer/core`. Marketplace publish + follows in Phase I. + +- [#15](https://github.com/faraa2m/tokenometer/pull/15) [`3681ae3`](https://github.com/faraa2m/tokenometer/commit/3681ae35d021a22a300271e2ea945cfeaeca9a0e) Thanks [@faraa2m](https://github.com/faraa2m)! - Documentation + SEO sweep: README, per-package READMEs, package.json + descriptions and keywords updated to reflect every shipped feature + (CLI flags, GitHub Action, VS Code extension, Claude Code skill, + Mistral + Cohere providers, latency benchmarking, vision tokens, SARIF). + Repo metadata + awesome-list templates regenerated. No source code + changes. + +- [#16](https://github.com/faraa2m/tokenometer/pull/16) [`dd9dab3`](https://github.com/faraa2m/tokenometer/commit/dd9dab3e06bbaf374b72d1903055b8a22472e5a1) Thanks [@faraa2m](https://github.com/faraa2m)! - Unified release pipeline (Phase I): one merge of the Version Packages PR + publishes tokenometer + @tokenometer/core to npm with provenance, creates + the GitHub Release (which republishes the Action to GitHub Marketplace), + publishes the VS Code extension to VS Code Marketplace + Open VSX, runs + a post-publish smoke test, verifies the Marketplace listing, and triggers + the Vercel deploy hook. Local `npm run smoke` runs the full sweep (lint, + typecheck, test, build, benchmarks, CLI smoke). diff --git a/packages/core/package.json b/packages/core/package.json index 9d5c87b..aea1462 100644 --- a/packages/core/package.json +++ b/packages/core/package.json @@ -1,6 +1,6 @@ { "name": "@tokenometer/core", - "version": "0.0.2", + "version": "0.1.0", "description": "Tokenometer core library — multi-provider LLM token cost, latency, and vision-token estimation with empirical countTokens fallback (Anthropic, OpenAI, Google, Mistral, Cohere).", "license": "MIT", "author": "Faraazuddin Mohammed ", diff --git a/packages/vscode/CHANGELOG.md b/packages/vscode/CHANGELOG.md index 2057992..c8720a9 100644 --- a/packages/vscode/CHANGELOG.md +++ b/packages/vscode/CHANGELOG.md @@ -1,5 +1,12 @@ # Changelog +## 0.0.3 + +### Patch Changes + +- Updated dependencies [[`a864454`](https://github.com/faraa2m/tokenometer/commit/a8644542d65641a4d8804ec4c4153bcfb2ccbf10), [`cc745d5`](https://github.com/faraa2m/tokenometer/commit/cc745d5701bec2e9fd4f52afd008b87e63f06802), [`cc745d5`](https://github.com/faraa2m/tokenometer/commit/cc745d5701bec2e9fd4f52afd008b87e63f06802), [`cc745d5`](https://github.com/faraa2m/tokenometer/commit/cc745d5701bec2e9fd4f52afd008b87e63f06802), [`061e0c5`](https://github.com/faraa2m/tokenometer/commit/061e0c577fce3ecc09c6a201a408cd1a360d5587), [`061e0c5`](https://github.com/faraa2m/tokenometer/commit/061e0c577fce3ecc09c6a201a408cd1a360d5587), [`061e0c5`](https://github.com/faraa2m/tokenometer/commit/061e0c577fce3ecc09c6a201a408cd1a360d5587), [`061e0c5`](https://github.com/faraa2m/tokenometer/commit/061e0c577fce3ecc09c6a201a408cd1a360d5587), [`061e0c5`](https://github.com/faraa2m/tokenometer/commit/061e0c577fce3ecc09c6a201a408cd1a360d5587), [`3681ae3`](https://github.com/faraa2m/tokenometer/commit/3681ae35d021a22a300271e2ea945cfeaeca9a0e), [`dd9dab3`](https://github.com/faraa2m/tokenometer/commit/dd9dab3e06bbaf374b72d1903055b8a22472e5a1)]: + - @tokenometer/core@0.1.0 + ## Unreleased - Initial scaffold. Status bar shows live token count and USD cost for the diff --git a/packages/vscode/package.json b/packages/vscode/package.json index 6a6d97a..24dc137 100644 --- a/packages/vscode/package.json +++ b/packages/vscode/package.json @@ -1,6 +1,6 @@ { "name": "@tokenometer/vscode", - "version": "0.0.2", + "version": "0.0.3", "displayName": "Tokenometer", "description": "Tokenometer for VS Code / Cursor — live token count and USD cost in the status bar for LLM prompts (Claude, GPT-4o, Gemini, Mistral, Cohere).", "publisher": "faraa2m", @@ -105,7 +105,7 @@ "package:vsix": "npm run build && vsce package --no-dependencies" }, "dependencies": { - "@tokenometer/core": "0.0.2" + "@tokenometer/core": "0.1.0" }, "devDependencies": { "@biomejs/biome": "^1.9.4", diff --git a/packages/web/package.json b/packages/web/package.json index 46757f8..91be376 100644 --- a/packages/web/package.json +++ b/packages/web/package.json @@ -13,7 +13,7 @@ "dependencies": { "@anthropic-ai/sdk": "^0.40.0", "@google/generative-ai": "^0.21.0", - "@tokenometer/core": "0.0.2", + "@tokenometer/core": "0.1.0", "react": "^19.0.0", "react-dom": "^19.0.0", "react-router-dom": "^7.15.0"