From 54794788a2588cb42f78c976a7358eab5d949b9c Mon Sep 17 00:00:00 2001 From: Samartha Puthraya Date: Wed, 2 Sep 2026 16:14:13 +0530 Subject: [PATCH 1/2] chore(release): 0.2.1 The first release in which a GREEN result is evidence. Two ways a run could report `looks safe to adopt` over something it never measured are closed (MP-159, MP-160), and the first error a stranger hits no longer hands them the command they just ran (MP-161/MP-170). `[M]` 749 passed + 4 xfailed (753 collected), 95% of 2085, ruff + black clean. `[M]` Verified from a clean-venv wheel install: init --demo / baseline / check all behave as the README documents. --- CHANGELOG.md | 7 +++++++ README.md | 4 ++-- modelpin/__init__.py | 2 +- pyproject.toml | 2 +- 4 files changed, 11 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 80dade6..c78f2e6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.2.1] - 2026-09-02 + +The first release in which a GREEN result is evidence. Two ways a run could report +`looks safe to adopt` over something it never measured are closed, and the error a +stranger hits first no longer hands them the command they just ran. + + ### Added - **The semantic judge runs cross-vendor.** It accepted OpenAI judge models only, which left a hole with teeth: cross-vendor is a core promise, and for a suite that declares no `tools` the diff --git a/README.md b/README.md index a2101eb..f091bd2 100644 --- a/README.md +++ b/README.md @@ -37,7 +37,7 @@ Install (Python 3.12+): ```bash pip install "modelpin[providers]" # or: pipx install "modelpin[providers]" -modelpin version # -> modelpin 0.2.0 +modelpin version # -> modelpin 0.2.1 ``` > **Windows PowerShell:** run `modelpin …`, not `mp …`. PowerShell ships a built-in `mp` alias @@ -518,7 +518,7 @@ is what keeps the false-positive promise honest and the tool small enough to tru ## Status **Phase 0 (core engine MVP) — detection demonstrated but NOT characterised; the false-positive half is NOT met** -(see [`docs/fp-measurement.md`](https://github.com/samarthputhraya/modelpin/blob/main/docs/fp-measurement.md)); `v0.2.0` live on PyPI. Live-validated cross-vendor +(see [`docs/fp-measurement.md`](https://github.com/samarthputhraya/modelpin/blob/main/docs/fp-measurement.md)); `v0.2.1` live on PyPI. Live-validated cross-vendor (OpenAI ↔ Google ↔ Groq/Llama); **false-positive rate not established** (the "0 in 8 held-out trials" claim is withdrawn — those 8 could not have fired, so the honest score is 0/0); multi-turn replay; a real GitHub Action; the public-report engine (`mp report`) + the open suite (in this repo, not diff --git a/modelpin/__init__.py b/modelpin/__init__.py index 9844e13..0ea6c2a 100644 --- a/modelpin/__init__.py +++ b/modelpin/__init__.py @@ -1,3 +1,3 @@ """Modelpin — Dependabot for AI models. Know before the model breaks you.""" -__version__ = "0.2.0" +__version__ = "0.2.1" diff --git a/pyproject.toml b/pyproject.toml index 1604685..85515cc 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "modelpin" -version = "0.2.0" +version = "0.2.1" description = "Dependabot for AI models — catch model-migration regressions before they hit production." readme = "README.md" requires-python = ">=3.12" From 75090bd3943ee14eab52ed8d3393e11a14838c75 Mon Sep 17 00:00:00 2001 From: Samartha Puthraya Date: Wed, 2 Sep 2026 20:16:52 +0530 Subject: [PATCH 2/2] docs(release): the claims gate's two blockers, and a claim of mine that was wrong a fifth time The pre-release claims audit BLOCKED 0.2.1 on two doc defects, neither in code. 1. CHANGELOG link definitions were stale: `[Unreleased]` still compared from v0.2.0, and there was no `[0.2.1]` definition at all, so the new heading rendered unlinked. 2. The 0.2.1 summary claimed 'the first release in which a GREEN result is evidence'. Refuted three ways: ADR-0033's own stated accepted failure mode (a run exits 0 over a scenario it never measured), the strict xfail pinning MP-165, and MP-44 -- which the auditor proved live by RENDERING a Report, showing `_report_header` branches on five conditions and never on `meta.skipped`, so a document that measured 1 of 14 still leads with 'No behavioral change observed'. The entry now says what is true: two specific holes are closed, and a green exit code is still not proof of full coverage. Also fixed, and it is the same mistake a fifth time: README's exit-code paragraph omitted provider rejection, though `check` exits 3 whenever `unmeasured or rejected`. The MP-160 commit claimed the contract was 'restored on all four' surfaces. There were five. Added the missing CHANGELOG entry for MP-161/MP-170 -- the summary advertised the fix while the document carried no record of it, including the user-visible `scan` and `init` path corruption. `[M]` 749 passed + 4 xfailed; README's 749 / 753 / 95% independently reconfirmed by the auditor, along with both suite hashes. --- CHANGELOG.md | 28 ++++++++++++++++++++++++---- README.md | 11 +++++++---- 2 files changed, 31 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c78f2e6..8fd4d2b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,9 +8,28 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [0.2.1] - 2026-09-02 -The first release in which a GREEN result is evidence. Two ways a run could report -`looks safe to adopt` over something it never measured are closed, and the error a -stranger hits first no longer hands them the command they just ran. +Two ways a run could report `looks safe to adopt` over something it never measured are +closed, and the first error a stranger hits no longer hands them the command they just ran. + +**A green exit code is still not proof of full coverage.** A scenario with no recorded +baseline is disclosed everywhere a reviewer looks, and costs the run its affirmative +clearance, but it does not by itself fail the build (ADR-0033). A published Report's headline +also does not yet qualify itself when scenarios were skipped. Read the coverage disclosure, +not the exit code. + +### Fixed +- **User-facing error text is text, not markup.** `[M]` A user whose install lacked the SDK + extra was told to run `pip install 'modelpin'` — the command they had just run — because + rich parsed `[providers]` as a style tag and ate it. It is the first error such a user + hits. The same defect had a quieter half: a message containing anything bracket-shaped was + silently rewritten, with no exception, naming something that did not exist. Errors are now + escaped once at the root rather than at each call site, where the class had been closed + five separate times. +- **`modelpin scan` reported file paths that do not exist**, and `modelpin init` misdescribed + the directory you had just named. `[M]` A folder called `src [experimental]` rendered as + `src ` — a rich table parses its cells as markup, and those cells are paths discovered in + your repo. Any project using bracketed directory names, such as Next.js route folders, saw + every affected row corrupted. ### Added @@ -877,7 +896,8 @@ stranger hits first no longer hands them the command they just ran. opinion-framed Markdown + JSON report. - BYO-key throughout, with key-shaped-secret scrubbing on all output. -[Unreleased]: https://github.com/samarthputhraya/modelpin/compare/v0.2.0...HEAD +[Unreleased]: https://github.com/samarthputhraya/modelpin/compare/v0.2.1...HEAD +[0.2.1]: https://github.com/samarthputhraya/modelpin/compare/v0.2.0...v0.2.1 [0.2.0]: https://github.com/samarthputhraya/modelpin/compare/v0.1.2...v0.2.0 [0.1.2]: https://github.com/samarthputhraya/modelpin/compare/v0.1.1...v0.1.2 [0.1.1]: https://github.com/samarthputhraya/modelpin/compare/v0.1.0...v0.1.1 diff --git a/README.md b/README.md index f091bd2..2336e02 100644 --- a/README.md +++ b/README.md @@ -73,10 +73,13 @@ dated copy under `.modelpin/runs/` that the next run will not overwrite, for cit | `invoice_parse` | `changed_minor` | `"Total: $5"` → `"Total: 5"` breaks the scenario's assertion, but nothing refused and no tool moved | `modelpin check` exits **1** only on a real `regression` — that's the CI gate, and it is why -the demo exits 1. It also exits **3** when a scenario it compared could not be measured, or when -nothing was compared at all — a different claim from "it broke". A scenario with no recorded -baseline is named in the report and costs the run its clearance, but does not fail the build. Then edit `traces.json`, re-run, and watch the verdict move: -the answer is computed from the traces, not baked in. +the demo exits 1. It also exits **3** when a scenario it compared could not be measured, when the +provider rejected one, or when nothing could be compared at all — a different claim from "it +broke". A scenario with no recorded baseline is named in the report and costs the run its +clearance, but does not by itself fail the build. + +Then edit `traces.json`, re-run, and watch the verdict move: the answer is computed from the +traces, not baked in. None of this is bundled inside the installed package — the wheel is code only, and the demo is generated on your machine. That is deliberate: shipping them would mean the quickstart depends