Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
481d329
docs: design adapter conformance hardening
thomaswillner Aug 13, 2026
333ffcc
docs: align ownership design with implementation grant
thomaswillner Aug 13, 2026
93e59a1
docs: plan adapter conformance release
thomaswillner Aug 13, 2026
7e4b506
feat: define adapter conformance corpus
thomaswillner Aug 13, 2026
426f8bc
feat: trace exact adapter conformance outcomes
thomaswillner Aug 13, 2026
42e7685
feat: reject vacuous conformance passes
thomaswillner Aug 13, 2026
84beaa0
feat: publish adapter conformance command
thomaswillner Aug 13, 2026
495e4bd
release: prepare 0.4.0 adapter conformance hardening
thomaswillner Aug 13, 2026
514ce29
fix: harden adapter conformance evidence
thomaswillner Aug 13, 2026
143846e
fix: scope multiplicity to preserved identities
thomaswillner Aug 13, 2026
59d0f91
fix: retain zero-count proposition identities
thomaswillner Aug 13, 2026
0ddef19
test: require bounded binding construction
thomaswillner Aug 13, 2026
046dcaa
fix: bound binding construction time
thomaswillner Aug 13, 2026
b76814b
fix: validate pre-release security transition
thomaswillner Aug 13, 2026
d79bfd8
test: isolate dirty-tree conformance fixture
thomaswillner Aug 13, 2026
0eff23e
test: bind receipt and metadata acceptance seams
thomaswillner Aug 13, 2026
73a2b9d
fix: share receipt acceptance and bound metadata
thomaswillner Aug 13, 2026
120cda8
test: reject malformed receipt JSON safely
thomaswillner Aug 13, 2026
3d5dc89
fix: refuse unreadable receipt evidence
thomaswillner Aug 13, 2026
4ab1a0f
ci: fetch immutable conformance history
thomaswillner Aug 13, 2026
1fa500c
test: bind history guard to checkout step
thomaswillner Aug 13, 2026
1aa9db8
fix: close conformance review findings
thomaswillner Aug 13, 2026
03d1a23
fix: bound external binding imports
thomaswillner Aug 13, 2026
41fb070
fix: execute admitted binding bytes
thomaswillner Aug 13, 2026
08e3f59
fix: isolate binding package imports
thomaswillner Aug 13, 2026
c7fa670
fix: isolate dotted binding namespaces
thomaswillner Aug 13, 2026
d7f5a45
fix: isolate binding imports for full run
thomaswillner Aug 13, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/validate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,10 @@ jobs:
steps:
- name: Check out the repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1, Node 24
with:
# Conformance binds historical immutable normative commits. A shallow
# checkout cannot verify those Git objects and must not run the suite.
fetch-depth: 0

- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0, Node 24
Expand Down
39 changes: 38 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,44 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),

Changes to the bounded novelty statement or the source comparison are recorded here even when they narrow or retire part of the claim. That is the intended direction of travel, not an exception.

## [Unreleased](https://github.com/thomaswillner/llm-errata/compare/v0.3.0...HEAD)
## [Unreleased](https://github.com/thomaswillner/llm-errata/compare/v0.4.0...HEAD)

## [0.4.0](https://github.com/thomaswillner/llm-errata/releases/tag/v0.4.0) - 2026-08-13

### Added — adapter conformance and validator hardening

- Added five provider-neutral adapter cases with complete checkpoint,
aggregate, triad, store, and receipt outcomes. Calls are traced through the
exact target adapter instance rather than matched globally by function name.
- Added bounded proposition-multiplicity preservation. Synthetic conformance
bindings expose stable provider-local proposition identity and active count;
an unobservable count is `unknown`, not a pass.
- Added exact semantic mutation controls. A mutation must complete and produce
its declared counter-result; an exception or different failure cannot earn
credit.
- Added three executable validator attacks covering empty receipts, incomplete
no-op feed acceptance, and constant-`unknown` semantic aggregation.
- Added `errata adapter-conformance`, canonical JSON reporting, and distinct
pass, failed-control, and invalid-evidence exit codes.

Rastislav Drahos/DanceNitra materially advanced this release by reporting the
adapter-coverage gap, a real duplicate-preservation defect, candidate adapter
behaviors, and the three anti-vacuity attacks. The immutable MIT-licensed source
artifact is
`DanceNitra/agora@2ba1e299b3483b9038d03387345702427608b90b`; Inspeximus is a G4
candidate and the source commit discloses Claude Opus 5 co-authorship, so the
input remains interested-party evidence. LLM Errata independently authored its
corpus and validator without copying or vendoring that runner or fixture.

### Status and ownership

- Repository files authored here remain copyright Thomas Rainer Willner.
Contributor credit and MIT permission do not transfer copyright.
- Existing licence terms are unchanged: independent commercial and
non-commercial specification implementations are permitted under the
attributed implementation grant; Reference Code remains restricted.
- Verdict remains `NOT_PROD_READY`. G2 through G6 remain blocked by their
existing external evidence requirements.

### Added — production-readiness evidence

Expand Down
5 changes: 2 additions & 3 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ type: software
authors:
- family-names: "Willner"
given-names: "Thomas Rainer"
version: 0.3.0
date-released: 2026-08-07
version: 0.4.0
date-released: 2026-08-13
license: LicenseRef-LLM-Errata-Personal-Use
license-url: "https://github.com/thomaswillner/llm-errata/blob/main/LICENSE"
abstract: >-
Expand All @@ -21,4 +21,3 @@ keywords:
- provenance
- conformance
- verifiable repair

7 changes: 7 additions & 0 deletions IDEA.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,13 @@ Every repair uses a three-way check—the **repair triad**:
2. **Positive:** the replacement influenced the cases in which it should.
3. **Preservation:** nearby facts that were not changed still worked.

Within an inspectable adapter scope, preservation also requires bounded
proposition multiplicity: repair must not increase active assertions of a
preserved proposition unless the erratum requires another assertion. The
adapter supplies stable provider-local proposition identity and count for the
synthetic conformance fixture. Text similarity is not identity; an adapter
that cannot expose this observation reports it as `unknown`.

Erasure has no positive replacement, but still needs negative and preservation checks. The triad defeats two cheap tricks: adding a new fact while still retrieving the old one, and “fixing” the problem by wiping the whole profile.

## Three operations, three meanings
Expand Down
8 changes: 8 additions & 0 deletions INDEPENDENT_IMPLEMENTATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,14 @@ the adapter cannot retire one input while preserving the others without an
additional decomposition or rebuild contract; it must report that limit rather
than fabricate complete repair.

Run the adapter corpus against an isolated synthetic store with
`errata adapter-conformance --binding module:factory`. The binding must expose
stable provider-local proposition identities and active counts for that
fixture. A store unable to observe multiplicity reports `unknown`; text
normalization or embedding similarity cannot substitute for identity. A pass
is candidate internal evidence only and still requires a separate producer to
validate both implementations and the validator.

## Independence and evidence

An implementation report must name its authors, repository and commit, supported
Expand Down
4 changes: 2 additions & 2 deletions PRODUCTION_READINESS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

| Field | Value |
|---|---|
| Version | 0.3.0 |
| Version | 0.4.0 |
| Verdict | **NOT_PROD_READY** |
| Ledger | `readiness/production-readiness.json` |

Expand All @@ -15,7 +15,7 @@ Build health means the repository's deterministic structure/metadata checks, bou
| Gate | Criterion | Current status | Current evidence | Next evidence required |
|---|---|---|---|---|
| G1 | VERSION, SECURITY support policy, readiness matrix, and check documentation remain aligned; negative tests protect every machine-enforced binding. | `PASS` | `VERSION`, `README.md`, `AGENTS.md`, `CONTRIBUTING.md`, `SECURITY.md`, `PRODUCTION_READINESS.md`, readiness ledger, both checkers, and their focused tests. | Maintain document, ledger, matrix, checker, and test consistency with each release. |
| G2 | Complete Phase 2 implementation, including provider-neutral semantic probes, and an independent reviewer evaluates the complete conformance surface. | `BLOCKED` | Phase 2 implementation includes conflict-disclosed remediation for split-view equivocation, unsupported empty enumeration, checkpoint coverage, and adapter-contract completeness, plus schemas, semantic probes, key rotation, invalid-target, confidentiality, and receipt binding; no qualifying independent review is recorded. | Dated independent external conformance-review result covering the exact complete Phase 2 surface after remediation. |
| G2 | Complete Phase 2 implementation, including provider-neutral semantic probes, and an independent reviewer evaluates the complete conformance surface. | `BLOCKED` | Phase 2 implementation includes conflict-disclosed remediation for split-view equivocation, unsupported empty enumeration, checkpoint coverage, and adapter-contract completeness, plus schemas, semantic probes, adapter-level conformance, validator anti-vacuity controls, key rotation, invalid-target, confidentiality, and receipt binding; no qualifying independent review is recorded. | Dated independent external conformance-review result covering the exact complete Phase 2 surface after remediation. |
| G3 | Production signing uses an audited constant-time library through the Signer seam, with independent security review of key lifecycle. | `BLOCKED` | `THREAT_MODEL.md` and `docs/CRYPTOGRAPHY_QUALIFICATION.md` record the internal candidate assessment. PyCA passed wire-compatibility checks but documents no external project audit; libsodium has audited lineage only for older versions. No production signer or qualifying independent lifecycle review exists. | Qualify exact current library, binding, build, and platforms; implement rotation, recovery, revocation, and delegation; obtain dated independent security review. |
| G4 | Two independently authored adapters consume the same erratum and a third-party validator evaluates their receipts consistently. | `BLOCKED` | Inspeximus `v2.7.0` is one tagged externally authored adapter candidate with disclosed v2.6.1 reference-code contamination and a claimed clean-room rewrite. It targets historical commit `a477fe4f5c86730031b6285d9505778fb8eec060`; provenance, current-target behavior, a second candidate, and a third-party validator result remain unverified. | Rebind candidates to the current immutable target; obtain dated evidence from two independently authored adapters, including separate provenance review where needed, and a separately produced third-party validator result. |
| G5 | One user-controlled synthetic root completes declared experiment across three independently operated memory systems. | `BLOCKED` | `ROADMAP.md` records interoperability experiment requirement; no approved systems or measured result are recorded. | Approved third-party systems, authorized synthetic-data experiment, and measured report. |
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
| Field | Value |
|---|---|
| Author | Thomas Rainer Willner |
| Version | 0.3.0 |
| Version | 0.4.0 |
| Status | Public concept proposal / Request for Comment |
| Published | 2026-08-07 |
| Research reviewed through | 2026-08-01 |
Expand Down Expand Up @@ -179,7 +179,7 @@ cannot be read as a bug. See [prototype/README.md](prototype/README.md).

## Current maturity

Version 0.3.0 is an experimental conformance proposal and tested reference implementation, not a production protocol or proof of interoperability. Phase 1 and the internal Phase 2 conformance surface include conflict-disclosed external remediation for split-view limitations, empty-enumeration truthfulness, phase-specific checkpoint coverage, complete adapter call-surface documentation, and removal of hidden reference-ledger coupling. Phase 2 also includes provider-neutral semantic probes, durable `errata quarantine` checkpoints required by CLI repair, owner-key rotation schedules, same-view conflict and invalid-target cases, content-free confidentiality evidence, and mutation coverage for every signed receipt field. G2 remains `BLOCKED`: the reviewer disclosed a commercial conflict and did not cover the full required conformance scope. G4 also remains `BLOCKED`: one tagged external adapter candidate exists, but its clean-room provenance and behavior are not third-party validated and no separate validator result exists.
Version 0.4.0 is an experimental conformance proposal and tested reference implementation, not a production protocol or proof of interoperability. Phase 1 and the internal Phase 2 conformance surface include conflict-disclosed external remediation for split-view limitations, empty-enumeration truthfulness, phase-specific checkpoint coverage, complete adapter call-surface documentation, and removal of hidden reference-ledger coupling. Phase 2 also includes provider-neutral semantic probes, durable `errata quarantine` checkpoints required by CLI repair, owner-key rotation schedules, same-view conflict and invalid-target cases, content-free confidentiality evidence, mutation coverage for every signed receipt field, and independently authored adapter-level cases with target-instance tracing, complete outcomes, bounded proposition multiplicity, exact semantic mutations, and executable validator anti-vacuity controls. G2 remains `BLOCKED`: interested-party findings and internal remediation do not replace a complete independent review of the current surface. G4 also remains `BLOCKED`: one externally authored adapter candidate exists, but two independent implementations and a separately produced third-party validator result are not established.

Current production-readiness verdict: **NOT_PROD_READY**. [ROADMAP.md](ROADMAP.md) defines implementation and kill criteria. [PRODUCTION_READINESS.md](PRODUCTION_READINESS.md) records the human evidence matrix and continuous enforcement boundaries.

Expand Down
6 changes: 6 additions & 0 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,12 @@ Only after the file-backed proof passes:
4. Define the complete adapter interface for enumeration, root-specific lineage-completeness evidence, quarantine and quarantine-phase coverage, store-owned repair inputs, retirement, reconstruction, recall probes, final dispositions, and coverage reporting. Empty enumeration or missing phase evidence is `unknown`; final success cannot overwrite a worse durable checkpoint.
5. Publish conformance vectors for signatures, sequencing, key rotation, concurrent events, invalid targets, receipt binding, and confidentiality.
6. Add model-assisted semantic probes behind a provider-neutral interface with deterministic fixtures and recorded verifier configuration. **Implemented internally:** [`prototype/semantic.py`](prototype/semantic.py), [`spec/semantic/`](spec/semantic/), and `errata semantic-test` record only structured, configuration-bound observations; inconclusive, malformed, missing, duplicate, or drifted required evidence is not success.
7. Publish adapter-level conformance with exact target-instance call controls,
complete outcomes, bounded proposition multiplicity, exact semantic
mutations, and executable validator anti-vacuity attacks. **Implemented
internally:** [`prototype/conformance.py`](prototype/conformance.py),
[`spec/adapter-conformance.json`](spec/adapter-conformance.json), and
`errata adapter-conformance`. Passing is internal evidence, not G2 or G4.

The CLI quarantine checkpoint is also implemented internally. `errata quarantine`
authenticates exactly the next pending erratum, gates enumerable descendants,
Expand Down
8 changes: 6 additions & 2 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,15 @@ LLM Errata is currently a public concept proposal and may later include schemas,

## Supported versions

Until a later policy states otherwise, only the latest versioned release is eligible for security fixes. Development revisions after that release receive fixes at maintainer discretion and are not represented as supported releases.
Until a later policy states otherwise, only the latest versioned release is
eligible for security fixes. Support moves from 0.3.x to 0.4.x beginning with the immutable `v0.4.0` release. Development revisions after the latest release
receive fixes at maintainer discretion and are not represented as supported
releases.

| Version | Supported |
|---|---|
| 0.3.x | Yes |
| 0.4.x | Yes, after `v0.4.0` is published |
| 0.3.x | Yes, until `v0.4.0` is published |
| 0.2.x and earlier | No |
| Unreleased development revisions | No |

Expand Down
1 change: 1 addition & 0 deletions SOURCES.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ the default branch and will move.
| Source | Pin | Verified 2026-08-07 |
|---|---|---|
| [Inspeximus](https://github.com/DanceNitra/inspeximus) | `4c711f2982911841d86d7ac1989b0ffb866dc891` (review baseline, 2026-08-12); `v2.6.0` / `36611027a463a8e526e23baf2d6bb8d9797b67ac`; `v2.7.0` / `ccdb30de5a32896fafcb4ed18a7a6f228691e078` (2026-08-13) | Baseline confirms `retract_lineage`, `include_superseded`, `needs_rederivation`, and `rederive`. v2.6.0 makes known unresolved derivation holes `partially_audited` and reports `subject_reachable_records`. v2.7.0 adds a tagged LLM Errata adapter candidate and preserves explicit provenance disclosure: v2.6.1 copied reference structure and is disqualified; v2.7.0 claims a clean-room rewrite and reports adapter-contract defects reproduced locally here. Tags and source version are verified; GitHub Releases has no v2.6.0–v2.7.0 Release entries. The adapter targets historical LLM Errata commit `a477fe4f5c86730031b6285d9505778fb8eec060` and must rebind to the current immutable target. Candidate implementation and interested-party evidence only, not established independence, third-party validation, or G4 completion. |
| [DanceNitra adapter-conformance candidate](https://github.com/DanceNitra/agora/tree/2ba1e299b3483b9038d03387345702427608b90b/contrib/llm-errata-adapter-conformance) | `2ba1e299b3483b9038d03387345702427608b90b` (2026-08-13), MIT | Rastislav Drahos reported five adapter behaviors, three validator anti-vacuity attacks, and a duplicate-preservation counterexample. Inspeximus is a G4 candidate and the source commit discloses Claude Opus 5 co-authorship, so this is interested-party input rather than independent evidence. LLM Errata independently authored its corpus and validator without copying or vendoring the source runner or fixtures. |
| [jedisct1/ed25519.py](https://github.com/jedisct1/ed25519.py) | `67902d339ea47418a60fb7684255b81bc4f6d46e` (2026-08-13) | Maintainer-provided pure-Python Ed25519 comparison implementation. README claims RFC 8032 vectors, canonical/small-order rejection, batch verification, and optional randomized signing countermeasures. Repository supplies no detected licence file or independent audit/build qualification, so it is a comparison oracle only and does not satisfy G3. |
| [Doyle, A Truth Maintenance System](https://dblp.org/rec/journals/ai/Doyle79.html) | Artificial Intelligence 12(3), 1979; DBLP `journals/ai/Doyle79` | Stable bibliographic record confirms title and publication. Cited by Inspeximus as prior retract-and-retain lineage. Included to make historical dependency-retraction collision explicit; LLM Errata claims no invention of this mechanism. |
| [IAB Data Deletion Request Framework](https://github.com/InteractiveAdvertisingBureau/Data-Subject-Rights/blob/main/Data%20Deletion%20Request%20Framework.md) | `b9418f5394ca91193181a61c567ffbbdff79cdef` (2025-02-06) | Repository unchanged since well before the cutoff |
Expand Down
7 changes: 7 additions & 0 deletions THREAT_MODEL.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ mitigation or an admitted limit.
| Erasure smuggling content back | An erasure carrying a replacement is refused | `errata._check_shape` |
| Signature malleability | Non-canonical scalars (`S >= L`) rejected, so a receipt cannot be altered and still verify | `ed25519.verify` |
| Repair that destroys retained memory | Preservation probe | `controller._run_triad` |
| Repair that duplicates retained propositions | Stable synthetic proposition identity/count; exact mutation control | `conformance.validate_adapter_conformance` |
| Repair that adds the new value and keeps serving the old | Negative probe | `controller._run_triad` |
| Half-repaired state being served | Quarantine completes before any rebuild; an interrupted repair leaves state gated | `controller.repair` |
| Stale export undoing a repair | A re-import at or below the applied sequence is refused | `controller.reimport` |
Expand All @@ -54,6 +55,12 @@ offers. Mitigations are outside the contract: owner-issued spot probes, an
independent verifier, or a transparency log that makes a false claim durable
and attributable.

**A lying conformance binding.** Target-instance tracing proves the controller
called the supplied adapter, and exact mutations prove the case can reject a
named flattering behavior. Neither proves the adapter truthfully exposed all
substrate state or proposition multiplicity. A third producer remains required
to validate independent implementations and the validator.

**A store whose substrate contradicts its API.**
[Ghost Vectors](https://arxiv.org/abs/2606.18497v1) demonstrates that
soft-deleted embeddings remain reconstructible from HNSW index files in
Expand Down
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0.3.0
0.4.0
18 changes: 18 additions & 0 deletions docs/PUBLICATION_LOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,23 @@
# Publication log

## 2026-08-13 — adapter-conformance feedback incorporated for 0.4.0

Rastislav Drahos/DanceNitra published an MIT-licensed candidate fixture at
`DanceNitra/agora@2ba1e299b3483b9038d03387345702427608b90b`. Review reproduced
the reported 5/5 result and found false-pass paths in exception handling,
global name tracing, source binding, citation validation, declarative-only
anti-vacuity cases, and partial expected outcomes. LLM Errata independently
implemented the accepted behaviors with exact target-instance tracing,
complete outcomes, bounded proposition multiplicity, exact mutation results,
and executable validator attacks. The external runner and fixture were not
copied or vendored.

This is internal remediation from interested-party feedback. It does not
satisfy G2 or G4. Repository ownership and licence terms are unchanged:
Thomas Rainer Willner retains copyright in repository-authored files;
independent specification implementations remain permitted under the
attributed implementation grant; Reference Code remains separately restricted.

This log records publication and outreach evidence. It does not upgrade a
readiness gate. Last verified: 2026-08-13.

Expand Down
Loading