From b9bc681f1597f05b81c0da1c9b1d27ba4ea378b4 Mon Sep 17 00:00:00 2001 From: DeviousCardi <115358213+DeviousCardi@users.noreply.github.com> Date: Wed, 9 Sep 2026 13:30:52 +0530 Subject: [PATCH] Governance and adjudication documents, before calling it 1.0 (Part J) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `GOVERNANCE.md`: who maintains (currently one — @DeviousCardi — with a process for how another is added or removed), and the rule that a maintainer employed by or paid by a vendor in the matrix declares it in the file and recuses from adjudicating, reviewing, or merging anything about that vendor. `ADJUDICATION.md`: makes `AGENTS.md`'s three-tier precedence operational — the process for the one tier that reaches an issue (spec silent, no reference implementation to defer to), what the upstream question has to establish, and what "promote the check" actually changes (a `present` case never fails a backend for what it records; an `exact` one does, because a rule now exists for it to violate). Worked example: `otlp-logs/timestamp-nanosecond-precision`, whose own notes already say "Adjudication open." Deviates from the plan's literal `docs/ADJUDICATION.md` path: `docs/` is gitignored in this repository (`CLAUDE.md`: "Planning documents are not in this repository"), and the operational content this project has already moved out of `docs/DESIGN.md` lives in `AGENTS.md`, a tracked file — so `ADJUDICATION.md`, which exists for outside readers to act on, goes at the repository root beside `GOVERNANCE.md`, not into the directory nothing else publishes from. `.github/ISSUE_TEMPLATE/adjudication.md` and `divergence.md` already existed from earlier work and needed no changes; `README.md` and `CONTRIBUTING.md` now link both new documents, and README gains the `Funding` section the plan asks for — none currently, stated rather than left silent. Ran the adjudication process through the template end to end for the timestamp-nanosecond-precision case: opened https://github.com/DeviousCardi/specmatrix/issues/12 (the `adjudication` label did not exist yet either; created it), filled in every section the process currently allows — what five implementations do, the specification's own text and exactly where it stops, and the question as it would be put upstream — and left the "question put upstream" and "their answer" sections open rather than guessed at, per this project's own standing policy of not filing issues against repositories it does not maintain without approval; `ADJUDICATION.md` says plainly that an issue with no upstream question linked stays open with nothing to promote, which is exactly this issue's state. Linked from the case's own `notes:`. 166 tests, 0 clippy warnings, corpus gate passes. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_018k65nFSzuwsHwYnpSHNaeK --- ADJUDICATION.md | 71 +++++++++++++++++++ AGENTS.md | 3 + CONTRIBUTING.md | 7 +- GOVERNANCE.md | 63 ++++++++++++++++ README.md | 11 ++- .../timestamp-nanosecond-precision.yaml | 6 ++ 6 files changed, 158 insertions(+), 3 deletions(-) create mode 100644 ADJUDICATION.md create mode 100644 GOVERNANCE.md diff --git a/ADJUDICATION.md b/ADJUDICATION.md new file mode 100644 index 0000000..c65380d --- /dev/null +++ b/ADJUDICATION.md @@ -0,0 +1,71 @@ +# Adjudication + +`AGENTS.md`'s "Adjudication" section gives the order of precedence a check's +verdict rests on: + +1. **The specification says so.** Cite the clause. Done — a check written + this way never needs what follows. +2. **The specification is silent but a reference implementation is the de + facto definition.** Mark the check `basis: de-facto`, name the reference, + and it is settled the same way. +3. **Genuinely ambiguous** — the specification is silent and there is no + reference implementation to defer to. This is the only case that reaches + the rest of this document. + +A check in the third case ships as `match: present`: it records what each +store did without judging it, because there is no rule yet to judge it +against. This file is how one gets a rule, so the check can be promoted to +`match: exact`. + +## The process + +1. **The disagreement is real, not a harness bug.** Before anything else, + rule out the alternative `CLAUDE.md` names: a poll shorter than a store's + commit interval, a read-back selector that assumed a label, a case + measured before a store finished starting. If ruling it out is not + possible from the case's own run, it is not ready for adjudication. +2. **Open an issue from the `Adjudication` template** + (`.github/ISSUE_TEMPLATE/adjudication.md`), naming the check, what every + implementation that has run it did, and precisely what the specification + does not settle — quoting the clause and saying where it stops, not + paraphrasing it. +3. **Put the question to the specification's owners** — the standards body, + or the project that maintains the de facto definition when there is no + formal spec — and link that discussion from the issue. An issue with no + upstream question linked stays open with nothing to promote: recording a + divergence is the point, and guessing at what the specification's authors + would say is exactly what this process exists to avoid. +4. **Record the answer verbatim in the issue and in the check's `notes:`** + when it arrives, with the date and who gave it. +5. **Promote the check only once a rule exists.** Change `match: present` to + `match: exact`, cite the answer in `rule:`, and re-run the suite: every + implementation that disagreed with the new rule is now `ALTER`, not + `present`, and that is the adjudication doing its job — a `present` check + never fails a backend for the divergence it records; an `exact` one does, + because there is now a rule it violates. + +A check does not sit at `present` forever by default. Every one gets an +adjudication issue, even if the issue stays open for years waiting on an +answer — `AGENTS.md`'s "Never" section is explicit that a `present` case is +not simply left as the final answer. + +## Two implementations disagreeing is not, on its own, a rule + +The order of precedence above is deliberately not "whichever behaviour is +more common" or "whichever behaviour the newest implementation chose." A +specification that leaves something unsettled leaves it unsettled regardless +of how many stores converge on one answer by accident; convergence is +evidence worth citing in the issue, not a substitute for an answer from the +people who own the specification. + +## Worked example + +`cases/otlp-logs/timestamp-nanosecond-precision.yaml` is the first case this +process was written for. Five stores answered five different ways for how +much of `time_unix_nano`'s precision survives a round trip — millisecond +string, microsecond integer, a nanosecond integer that turned out to hold +only whole microseconds, and two that kept every digit. The OpenTelemetry +logs specification defines what the field means and says nothing about what a +receiver must retain, so no rule exists yet to promote the check with, no +matter how the five stores split. The case's own `notes:` record each +measurement as it was made; see there for the reproductions. diff --git a/AGENTS.md b/AGENTS.md index 54b201a..b0de28a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -73,6 +73,9 @@ check exists to find. Two implementations disagreeing is not a rule. A check moves from `present` to `exact` only when a rule exists, not when the disagreement gets interesting. +[`ADJUDICATION.md`](ADJUDICATION.md) makes step 3 operational: the issue +template to use, what the upstream question has to establish, and what +"promote the check" actually changes. ### Do not tune a backend to fit the corpus diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 69c83cf..8ea813d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -19,8 +19,11 @@ value is entirely in every entry being defensible. ### If two backends disagree and you are unsure who is right -Say so in the pull request and leave the verdict out. Open an issue upstream, or -with whoever owns the specification, and let it be settled there. +Say so in the pull request and leave the verdict out — ship the check as +`match: present`. Then open an issue from the `Adjudication` template and work +through [`ADJUDICATION.md`](ADJUDICATION.md#the-process): the question goes to +whoever owns the specification, and the check is promoted to `match: exact` +only once they answer. A wrong verdict published under this project's name damages someone's reputation unfairly and destroys the project's own. Being slow is much cheaper than being diff --git a/GOVERNANCE.md b/GOVERNANCE.md new file mode 100644 index 0000000..c7558ae --- /dev/null +++ b/GOVERNANCE.md @@ -0,0 +1,63 @@ +# Governance + +The results are only worth reading if they are not for sale. This is written +down before anyone has a reason to ask. + +## Maintainers + +The project currently has one maintainer, [@DeviousCardi](https://github.com/DeviousCardi), +who has commit access and decides what merges. That is a starting point, not +a target: a project with one maintainer has a bus factor of one, and this file +exists so growing past that happens by a stated process rather than by +whoever shows up first getting write access. + +### How a maintainer is added + +1. Several substantive pull requests merged — a check with a real citation, an + adapter re-confirmed by hand, or a runner fix, not typo corrections. +2. A public proposal, as an issue on this repository, naming the person and + what they have contributed. +3. No standing maintainer objects within two weeks. + +Removal follows the same shape: a public issue, a reason, two weeks for +objection. A maintainer who has been inactive for two consecutive quarterly +reruns (`docs/BACKENDS.md`'s checklist, run by `.github/workflows/matrix.yml`) +is asked directly before being proposed for removal, not removed by default — +absence is not the same as disagreement. + +## Vendor conflict of interest + +A maintainer employed by, or paid by, a vendor whose backend appears in the +matrix **declares it in this file**, in the table below. A maintainer with a +declared interest in a vendor: + +- Does not adjudicate a case (`ADJUDICATION.md`) where that vendor's backend + is the reference implementation or the backend under dispute. +- Does not merge a pull request that changes that vendor's adapter, files a + finding against it, or resolves a `divergence` issue about it — review and + approval come from another maintainer. +- May still write cases, fix the runner, and do everything else a maintainer + does; the recusal is scoped to decisions about the vendor they are + affiliated with, not a blanket exclusion. + +An affiliation is declared before the maintainer takes any of the actions +above, not after being asked about it. + +| Maintainer | Vendor | Relationship | +| --- | --- | --- | +| _(none declared)_ | | | + +## Funding + +None. If that changes, the amount and source are added to +[`README.md`](README.md#funding), not only here — a reader deciding whether +to trust a result should not have to find this file to learn who paid for it. + +## Scope of this document + +This covers who decides what merges and who may declare a vendor conflict of +interest away from a decision. It does not cover the rule for what a check +may assert, which is `AGENTS.md`'s "Adjudication" section and made +operational in [`ADJUDICATION.md`](ADJUDICATION.md); it does not cover how a +backend maintainer disputes a specific published verdict, which is +`.github/ISSUE_TEMPLATE/divergence.md`. diff --git a/README.md b/README.md index 4932bec..59fa050 100644 --- a/README.md +++ b/README.md @@ -79,6 +79,8 @@ static `matrix.html` under `results///`. | `src/` | The runner. Sends cases, reads back, compares, decides verdicts. | | `tools/` | The corpus gate CI runs. | | [`AGENTS.md`](AGENTS.md) | The design, and how to add a check or a backend. | +| [`GOVERNANCE.md`](GOVERNANCE.md) | Who maintains this project, and vendor conflicts of interest. | +| [`ADJUDICATION.md`](ADJUDICATION.md) | How a `present` check earns a rule and becomes `exact`. | The corpus is the asset. The runner is plumbing and could be rewritten in a weekend; a corpus of checks that each trace to a line of a specification takes @@ -200,7 +202,14 @@ The results are only worth reading if they are not for sale. contribute to. A comparison written by an interested party is an advertisement. This should not -become one. +become one. [`GOVERNANCE.md`](GOVERNANCE.md) has who maintains this project and +the rule for a maintainer with a vendor conflict of interest; +[`ADJUDICATION.md`](ADJUDICATION.md) has how a check earns the right to fail a +backend rather than only record what it did. + +### Funding + +None. If that changes, the amount and source are listed here. ## Licence diff --git a/cases/otlp-logs/timestamp-nanosecond-precision.yaml b/cases/otlp-logs/timestamp-nanosecond-precision.yaml index d36d1ab..0ef157e 100644 --- a/cases/otlp-logs/timestamp-nanosecond-precision.yaml +++ b/cases/otlp-logs/timestamp-nanosecond-precision.yaml @@ -110,3 +110,9 @@ notes: > is not the same as a rule saying it must be kept, and the specification still does not give one. Recorded, not adjudicated — but this is now a strong question to put to the specification's owners rather than a curiosity. + + Adjudication opened 2026-09-09: + https://github.com/DeviousCardi/specmatrix/issues/12, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the OpenTelemetry + specification's owners — this check stays `present` until that answer + arrives, whichever way it goes.