Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 39 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,45 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),

Changes to the bounded novelty statement or the source comparison are recorded here even when they narrow or retire part of the claim. That is the intended direction of travel, not an exception.

## [Unreleased](https://github.com/thomaswillner/llm-errata/compare/v0.4.0...HEAD)
## [Unreleased](https://github.com/thomaswillner/llm-errata/compare/v0.4.1...HEAD)

## [0.4.1](https://github.com/thomaswillner/llm-errata/releases/tag/v0.4.1) - 2026-08-14

Integrity and conformance hardening release following an authorized read-only
Claude Opus 5 adversarial review of v0.4.0. The review recommended keeping
v0.4.0 published and issuing this patch release rather than withdrawing it.

- Separated the immutable normative review target from the metadata-only
packaging/release commit. Publication and adapter-conformance metadata now
share one target; tag-time validation binds the final release commit without
attempting a self-referential Git hash.
- Scoped the offline publication checker to claims it can prove and made live
GitHub verification explicitly inconclusive without a freshness receipt.
- Enforced ledger-to-matrix criterion equality for all six readiness gates and
added gate-specific, commit-bound independent evidence contracts for G3-G5.
- Made the G3 scope parser reject malformed or unhashable tokens without a
traceback and added complete positive records for the G3-G5 contracts.
- Bound G4 validation to one shared erratum and the exact IDs and SHA-256
digests of both adapter receipts; unrelated validator evidence cannot qualify.
- Required G5 evidence to include correction, supersession, erasure, all nine
measurements, three distinct operators, the deliberately nonconforming arm,
incomplete or opaque coverage, and mixed-artifact lineage.
- Enforced publication history against committed Git revisions so valid-looking
coordinated rewrites, deletions, and arbitrary active-surface replacement fail.
- Widened receipt state roots from truncated 128-bit values to full SHA-256 and
required the same width in durable quarantine checkpoints and vectors.
- Disclosed state-root non-binding for every snapshot-less adapter path,
including opaque and lineage-incomplete stores.
- Clarified every named normative Markdown contract included in Specification
Materials and that conformance bindings execute trusted code only.
- Replaced the silent pre-corpus digest skip with an explicit failure and bound
the adapter corpus into G2 digests. One shared readiness/conformance module
now normalizes only the two unavoidable self-target scalar values and keeps
every other corpus byte, including whitespace and escape spelling, bound.

The production verdict remains **NOT_PROD_READY**. G2-G6 remain `BLOCKED`; this
release adds no external review, independent implementation, operated-system,
cryptography, or operational-readiness evidence.

## [0.4.0](https://github.com/thomaswillner/llm-errata/releases/tag/v0.4.0) - 2026-08-14

Expand Down
2 changes: 1 addition & 1 deletion CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ type: software
authors:
- family-names: "Willner"
given-names: "Thomas Rainer"
version: 0.4.0
version: 0.4.1
date-released: 2026-08-14
license: LicenseRef-LLM-Errata-Personal-Use
license-url: "https://github.com/thomaswillner/llm-errata/blob/main/LICENSE"
Expand Down
15 changes: 15 additions & 0 deletions INDEPENDENT_IMPLEMENTATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,14 @@ normalization or embedding similarity cannot substitute for identity. A pass
is candidate internal evidence only and still requires a separate producer to
validate both implementations and the validator.

The binding command executes imported Python inside the invoking process. It is
for **trusted code only** from a reviewed, clean checkout. Git identity,
tracked-byte binding, namespace isolation, and timeouts improve evidence
provenance; they are not an operating-system sandbox and do not make untrusted
candidate code safe. Production evaluation of untrusted submissions requires a
separate least-privilege process or container with explicit filesystem,
network, secret, CPU, and memory limits.

## Independence and evidence

An implementation report must name its authors, repository and commit, supported
Expand All @@ -86,6 +94,13 @@ cannot also occupy the separately authored third-party validator role for its
own implementation. Commercial interest and other conflicts must be disclosed;
they do not erase technical evidence, but they control how it can satisfy G4.

Qualifying G4 ledger records must bind both adapter receipts to the same
erratum, immutable commit, and canonical surface digest. The validator must
name both adapter implementation IDs and exact receipt IDs and SHA-256 digests;
an unrelated validator result cannot be combined with otherwise valid adapter
records. See [`docs/READINESS_EVIDENCE_SCHEMAS.md`](docs/READINESS_EVIDENCE_SCHEMAS.md)
for the exact role-specific fields.

No per-implementer permission is required for an independently authored
commercial or non-commercial implementation of the specification. The
irrevocable implementation grant requires every product or service to credit
Expand Down
9 changes: 6 additions & 3 deletions LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,15 @@ versions permanently. This licence does not, and cannot, withdraw it.
1. DEFINITIONS

"Specification Materials" means the normative requirements incorporated by
reference from README.md, IDEA.md, ROADMAP.md, SECURITY.md, and
THREAT_MODEL.md, plus files under `spec/` other than `spec/vendor/`.
reference from README.md, IDEA.md, ROADMAP.md, SECURITY.md, THREAT_MODEL.md,
HARD_PROBLEMS.md, PRIOR_ART.md, RESEARCH.md, PRODUCTION_READINESS.md,
INDEPENDENT_IMPLEMENTATION.md, and PHASE3_SYSTEMS.md, plus files under
`spec/` other than `spec/vendor/`.

"Reference Code" means files under `prototype/`, `scripts/`, and `tests/`,
together with other repository software not expressly included in the
Specification Materials.
Specification Materials. Markdown documentation is not Reference Code merely
because it is not listed as a Specification Material.

"Implementation" means independently authored software or services that
implement a material part of the Specification Materials without copying
Expand Down
7 changes: 7 additions & 0 deletions PHASE3_SYSTEMS.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,13 @@ Measurements include observation-to-quarantine time, known-descendant coverage,
stale-behavior rate, replacement activation, collateral retention, stale-reimport
resistance, opaque coverage, operator effort, and user-visible friction.

A qualifying report records all three operations and every measurement for
each system with public raw-evidence references. It also marks the intentionally
nonconforming importer, `incomplete` or `opaque` coverage system, and
mixed-artifact-lineage system explicitly. Three generic system-level pass
statements cannot satisfy G5. Exact ledger fields are defined in
[`docs/READINESS_EVIDENCE_SCHEMAS.md`](docs/READINESS_EVIDENCE_SCHEMAS.md).

## Nomination requirements

Please name:
Expand Down
6 changes: 3 additions & 3 deletions PRODUCTION_READINESS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

| Field | Value |
|---|---|
| Version | 0.4.0 |
| Version | 0.4.1 |
| Verdict | **NOT_PROD_READY** |
| Ledger | `readiness/production-readiness.json` |

Expand All @@ -23,9 +23,9 @@ requirement to wait indefinitely for unsolicited external reviewers.
|---|---|---|---|---|
| G1 | VERSION, SECURITY support policy, readiness matrix, and check documentation remain aligned; negative tests protect every machine-enforced binding. | `PASS` | `VERSION`, `README.md`, `AGENTS.md`, `CONTRIBUTING.md`, `SECURITY.md`, `PRODUCTION_READINESS.md`, readiness ledger, both checkers, and their focused tests. | Maintain document, ledger, matrix, checker, and test consistency with each release. |
| G2 | Complete Phase 2 implementation, including provider-neutral semantic probes, and an independent reviewer evaluates the complete conformance surface. | `BLOCKED` | Phase 2 implementation includes conflict-disclosed remediation for split-view equivocation, unsupported empty enumeration, checkpoint coverage, and adapter-contract completeness, plus schemas, semantic probes, adapter-level conformance, validator anti-vacuity controls, key rotation, invalid-target, confidentiality, and receipt binding; no qualifying independent review is recorded. | Dated independent external conformance-review result covering the exact complete Phase 2 surface after remediation. |
| G3 | Production signing uses an audited constant-time library through the Signer seam, with independent security review of key lifecycle. | `BLOCKED` | `THREAT_MODEL.md` and `docs/CRYPTOGRAPHY_QUALIFICATION.md` record the internal candidate assessment. PyCA passed wire-compatibility checks but documents no external project audit; libsodium has audited lineage only for older versions. No production signer or qualifying independent lifecycle review exists. | Qualify exact current library, binding, build, and platforms; implement rotation, recovery, revocation, and delegation; obtain dated independent security review. |
| G3 | Production signing uses an audited constant-time library through the Signer seam and independent security review covers key lifecycle. | `BLOCKED` | `THREAT_MODEL.md` and `docs/CRYPTOGRAPHY_QUALIFICATION.md` record the internal candidate assessment. PyCA passed wire-compatibility checks but documents no external project audit; libsodium has audited lineage only for older versions. No production signer or qualifying independent lifecycle review exists. | Qualify exact current library, binding, build, and platforms; implement rotation, recovery, revocation, and delegation; obtain dated independent security review. |
| G4 | Two independently authored adapters consume the same erratum and a third-party validator evaluates their receipts consistently. | `BLOCKED` | Inspeximus `v2.7.0` is one tagged externally authored adapter candidate with disclosed v2.6.1 reference-code contamination and a claimed clean-room rewrite. It targets historical commit `a477fe4f5c86730031b6285d9505778fb8eec060`; provenance, current-target behavior, a second candidate, and a third-party validator result remain unverified. | Rebind candidates to the current immutable target; obtain dated evidence from two independently authored adapters, including separate provenance review where needed, and a separately produced third-party validator result. |
| G5 | One user-controlled synthetic root completes declared experiment across three independently operated memory systems. | `BLOCKED` | `ROADMAP.md` records interoperability experiment requirement; no approved systems or measured result are recorded. | Approved third-party systems, authorized synthetic-data experiment, and measured report. |
| G5 | One user-controlled synthetic root completes the declared experiment across three independently operated memory systems. | `BLOCKED` | `ROADMAP.md` records interoperability experiment requirement; no approved systems or measured result are recorded. | Approved third-party systems, authorized synthetic-data experiment, and measured report. |
| G6 | All ten operational scopes pass from one independent report bound to exact commit and deployment, with declared thresholds and measured comparators. | `BLOCKED` | No independent report binds an exact commit and deployment to passing measured comparators for all ten operational scopes. | One qualifying independent report with declared workload, platform, failure domain, observation window, numeric thresholds, raw artifacts, and passing measurements for every scope. |

## Approval boundaries
Expand Down
14 changes: 13 additions & 1 deletion PUBLISHING.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,20 @@ Do not call the repository a standard, certified protocol, proven deletion syste

## Before making the repository public

LLM Errata uses two non-interchangeable commit identities. The immutable
`review_target` is the normative source commit. A later packaging commit may
change only `publication/active-surfaces.json` and
`spec/adapter-conformance.json` to point back to that source. The release tag
binds the packaging commit. This avoids requiring a Git commit to contain its
own SHA while keeping normative and release identities auditable.

1. Run `make check` from the repository root. It must exit zero. Exit `2` from the claim guard is *inconclusive*, not a pass.
2. Run `make links`. Every cited URL must resolve or be reported as `blocked`, never `dead`.
3. Review the author name, date, independent-publication disclaimer, and AI-assisted research disclosure.
4. Check every claim in `PRIOR_ART.md` against the cited primary or authoritative source.
5. Request a public archive snapshot for each source listed as unpinned in `SOURCES.md`, then replace `none` with the snapshot URL. These are the sources the collision matrix depends on most and the ones most likely to change.
6. Confirm that no employer, customer, personal, confidential, or credential material is present.
7. Confirm that `VERSION`, `CITATION.cff`, `CHANGELOG.md`, and the release tag agree with each other. Never re-tag existing content with an older version: the prior-art claim is dated, and a release tag that back-dates it corrupts the only thing this repository is for.
7. Confirm that `VERSION`, `CITATION.cff`, `CHANGELOG.md`, and the release tag agree with each other. Run `python3 scripts/check_publication.py --tag v$(cat VERSION)` after creating the local annotated tag and before pushing it. Never re-tag existing content with an older version: the prior-art claim is dated, and a release tag that back-dates it corrupts the only thing this repository is for.
8. Enable GitHub Issues.
9. Enable GitHub private vulnerability reporting before pointing readers to `SECURITY.md`.
10. Decide whether GitHub Discussions should be enabled for design debate; keep factual corrections and prior-art challenges in Issues so they remain traceable.
Expand All @@ -32,6 +39,11 @@ Do not call the repository a standard, certified protocol, proven deletion syste
15. Create the `prior-art`, `correction`, `conformance`, `implementation`, and `maintenance` labels used by the issue forms and Dependabot.
16. Publish [REVIEW_REQUEST.md](REVIEW_REQUEST.md), [INDEPENDENT_IMPLEMENTATION.md](INDEPENDENT_IMPLEMENTATION.md), and [PHASE3_SYSTEMS.md](PHASE3_SYSTEMS.md) only as calls for evidence. Record an external review, independent implementation, or system experiment in the readiness ledger only after its dated, independently produced result exists.

The default publication checker is deliberately offline. It validates tracked
metadata and local Git bindings; it does not prove GitHub comments still exist
or remain unchanged. Live publication claims require a fresh repository-wide
GitHub inventory and schema-v2 freshness receipt after the last mutation.

## Experimental release policy

Publishing an experimental version and declaring production readiness are
Expand Down
14 changes: 10 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,10 @@
| Field | Value |
|---|---|
| Author | Thomas Rainer Willner |
| Version | 0.4.0 |
| Version | 0.4.1 |
| Status | Public concept proposal / Request for Comment |
| Published | 2026-08-07 |
| Latest release | v0.4.0first materially improved experimental release |
| Latest release | v0.4.1integrity and conformance hardening release |
| Research reviewed through | 2026-08-01 |
| License | Attributed specification implementations permitted; reference code remains personal-use. See [LICENSE](LICENSE). |

Expand Down Expand Up @@ -129,6 +129,7 @@ See [PRIOR_ART.md](PRIOR_ART.md) for the feature-level comparison and [RESEARCH.
| [PHASE3_SYSTEMS.md](PHASE3_SYSTEMS.md) | Nominations for future authorized three-system synthetic-data experiment. |
| [docs/PUBLICATION_STRATEGY.md](docs/PUBLICATION_STRATEGY.md) | Evidence-bounded publication channels and canonical announcement copy. |
| [docs/PUBLICATION_LOG.md](docs/PUBLICATION_LOG.md) | Public GitHub calls, blocked external-channel attempts, and readiness boundary. |
| [docs/READINESS_EVIDENCE_SCHEMAS.md](docs/READINESS_EVIDENCE_SCHEMAS.md) | Exact commit-bound G3-G5 external evidence records and qualification boundaries. |

## Verifying this repository

Expand All @@ -148,6 +149,11 @@ make links # liveness of every cited external URL (needs network)
| `make readiness` | Is the recorded readiness evidence structurally honest and synchronized with the human matrix? | The ledger is malformed, evidence is insufficient for a recorded status, or the matrix contradicts the ledger. |
| `make test` | Do those checkers reject what they claim to reject? | A checker has stopped catching a fault it is supposed to catch. |

`make publication` proves offline manifest consistency and local Git binding
only. It prints remote GitHub state as unverified; a live publication or latest
feedback claim additionally requires the repository-wide freshness receipt
described in `PUBLISHING.md`.

The self-tests exist because a check that has never failed has not been shown to
work. The self-tests build corpora that misstate the proposal — an inverted
quarantine ordering, an asserted world first — and require the guard to reject
Expand Down Expand Up @@ -180,12 +186,12 @@ cannot be read as a bug. See [prototype/README.md](prototype/README.md).

## Current maturity

Version 0.4.0 is an experimental conformance proposal and tested reference implementation, not a production protocol or proof of interoperability. Phase 1 and the internal Phase 2 conformance surface include conflict-disclosed external remediation for split-view limitations, empty-enumeration truthfulness, phase-specific checkpoint coverage, complete adapter call-surface documentation, and removal of hidden reference-ledger coupling. Phase 2 also includes provider-neutral semantic probes, durable `errata quarantine` checkpoints required by CLI repair, owner-key rotation schedules, same-view conflict and invalid-target cases, content-free confidentiality evidence, mutation coverage for every signed receipt field, and independently authored adapter-level cases with target-instance tracing, complete outcomes, bounded proposition multiplicity, exact semantic mutations, and executable validator anti-vacuity controls. G2 remains `BLOCKED`: interested-party findings and internal remediation do not replace a complete independent review of the current surface. G4 also remains `BLOCKED`: one externally authored adapter candidate exists, but two independent implementations and a separately produced third-party validator result are not established.
Version 0.4.1 is an experimental conformance proposal and tested reference implementation, not a production protocol or proof of interoperability. It hardens immutable review/release binding, append-only publication history, readiness matrix equality, gate-specific external-evidence independence, full-width state roots, snapshot limitation disclosure, licence scope, and byte-preserving conformance-corpus digests. G4 validator reports must bind the same erratum and exact receipts from both adapters; G5 reports must record the complete declared three-system experiment rather than generic system passes. Phase 1 and the internal Phase 2 conformance surface retain conflict-disclosed external remediation for split-view limitations, empty-enumeration truthfulness, phase-specific checkpoint coverage, complete adapter call-surface documentation, and removal of hidden reference-ledger coupling. G2 remains `BLOCKED`: interested-party findings and internal remediation do not replace a complete independent review of the current surface. G4 also remains `BLOCKED`: one externally authored adapter candidate exists, but two independent implementations and a separately produced third-party validator result are not established.

Current production-readiness verdict: **NOT_PROD_READY**. [ROADMAP.md](ROADMAP.md) defines implementation and kill criteria. [PRODUCTION_READINESS.md](PRODUCTION_READINESS.md) records the human evidence matrix and continuous enforcement boundaries.

Experimental release readiness is separate from production readiness. Version
0.4.0 is published so implementers can evaluate and extend a materially better
0.4.1 is published so implementers can evaluate and extend a materially better
baseline while G2–G6 remain an explicit backlog. External challenge is welcome
whenever users or reviewers encounter the project, but no release claims those
gates passed merely because a reviewer did not appear.
Expand Down
Loading