diff --git a/CHANGELOG.md b/CHANGELOG.md index 80dade6..8fd4d2b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,32 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.2.1] - 2026-09-02 + +Two ways a run could report `looks safe to adopt` over something it never measured are +closed, and the first error a stranger hits no longer hands them the command they just ran. + +**A green exit code is still not proof of full coverage.** A scenario with no recorded +baseline is disclosed everywhere a reviewer looks, and costs the run its affirmative +clearance, but it does not by itself fail the build (ADR-0033). A published Report's headline +also does not yet qualify itself when scenarios were skipped. Read the coverage disclosure, +not the exit code. + +### Fixed +- **User-facing error text is text, not markup.** `[M]` A user whose install lacked the SDK + extra was told to run `pip install 'modelpin'` — the command they had just run — because + rich parsed `[providers]` as a style tag and ate it. It is the first error such a user + hits. The same defect had a quieter half: a message containing anything bracket-shaped was + silently rewritten, with no exception, naming something that did not exist. Errors are now + escaped once at the root rather than at each call site, where the class had been closed + five separate times. +- **`modelpin scan` reported file paths that do not exist**, and `modelpin init` misdescribed + the directory you had just named. `[M]` A folder called `src [experimental]` rendered as + `src ` — a rich table parses its cells as markup, and those cells are paths discovered in + your repo. Any project using bracketed directory names, such as Next.js route folders, saw + every affected row corrupted. + + ### Added - **The semantic judge runs cross-vendor.** It accepted OpenAI judge models only, which left a hole with teeth: cross-vendor is a core promise, and for a suite that declares no `tools` the @@ -870,7 +896,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 opinion-framed Markdown + JSON report. - BYO-key throughout, with key-shaped-secret scrubbing on all output. -[Unreleased]: https://github.com/samarthputhraya/modelpin/compare/v0.2.0...HEAD +[Unreleased]: https://github.com/samarthputhraya/modelpin/compare/v0.2.1...HEAD +[0.2.1]: https://github.com/samarthputhraya/modelpin/compare/v0.2.0...v0.2.1 [0.2.0]: https://github.com/samarthputhraya/modelpin/compare/v0.1.2...v0.2.0 [0.1.2]: https://github.com/samarthputhraya/modelpin/compare/v0.1.1...v0.1.2 [0.1.1]: https://github.com/samarthputhraya/modelpin/compare/v0.1.0...v0.1.1 diff --git a/README.md b/README.md index a2101eb..2336e02 100644 --- a/README.md +++ b/README.md @@ -37,7 +37,7 @@ Install (Python 3.12+): ```bash pip install "modelpin[providers]" # or: pipx install "modelpin[providers]" -modelpin version # -> modelpin 0.2.0 +modelpin version # -> modelpin 0.2.1 ``` > **Windows PowerShell:** run `modelpin …`, not `mp …`. PowerShell ships a built-in `mp` alias @@ -73,10 +73,13 @@ dated copy under `.modelpin/runs/` that the next run will not overwrite, for cit | `invoice_parse` | `changed_minor` | `"Total: $5"` → `"Total: 5"` breaks the scenario's assertion, but nothing refused and no tool moved | `modelpin check` exits **1** only on a real `regression` — that's the CI gate, and it is why -the demo exits 1. It also exits **3** when a scenario it compared could not be measured, or when -nothing was compared at all — a different claim from "it broke". A scenario with no recorded -baseline is named in the report and costs the run its clearance, but does not fail the build. Then edit `traces.json`, re-run, and watch the verdict move: -the answer is computed from the traces, not baked in. +the demo exits 1. It also exits **3** when a scenario it compared could not be measured, when the +provider rejected one, or when nothing could be compared at all — a different claim from "it +broke". A scenario with no recorded baseline is named in the report and costs the run its +clearance, but does not by itself fail the build. + +Then edit `traces.json`, re-run, and watch the verdict move: the answer is computed from the +traces, not baked in. None of this is bundled inside the installed package — the wheel is code only, and the demo is generated on your machine. That is deliberate: shipping them would mean the quickstart depends @@ -518,7 +521,7 @@ is what keeps the false-positive promise honest and the tool small enough to tru ## Status **Phase 0 (core engine MVP) — detection demonstrated but NOT characterised; the false-positive half is NOT met** -(see [`docs/fp-measurement.md`](https://github.com/samarthputhraya/modelpin/blob/main/docs/fp-measurement.md)); `v0.2.0` live on PyPI. Live-validated cross-vendor +(see [`docs/fp-measurement.md`](https://github.com/samarthputhraya/modelpin/blob/main/docs/fp-measurement.md)); `v0.2.1` live on PyPI. Live-validated cross-vendor (OpenAI ↔ Google ↔ Groq/Llama); **false-positive rate not established** (the "0 in 8 held-out trials" claim is withdrawn — those 8 could not have fired, so the honest score is 0/0); multi-turn replay; a real GitHub Action; the public-report engine (`mp report`) + the open suite (in this repo, not diff --git a/modelpin/__init__.py b/modelpin/__init__.py index 9844e13..0ea6c2a 100644 --- a/modelpin/__init__.py +++ b/modelpin/__init__.py @@ -1,3 +1,3 @@ """Modelpin — Dependabot for AI models. Know before the model breaks you.""" -__version__ = "0.2.0" +__version__ = "0.2.1" diff --git a/pyproject.toml b/pyproject.toml index 1604685..85515cc 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "modelpin" -version = "0.2.0" +version = "0.2.1" description = "Dependabot for AI models — catch model-migration regressions before they hit production." readme = "README.md" requires-python = ">=3.12"