diff --git a/ADJUDICATION.md b/ADJUDICATION.md new file mode 100644 index 0000000..c65380d --- /dev/null +++ b/ADJUDICATION.md @@ -0,0 +1,71 @@ +# Adjudication + +`AGENTS.md`'s "Adjudication" section gives the order of precedence a check's +verdict rests on: + +1. **The specification says so.** Cite the clause. Done — a check written + this way never needs what follows. +2. **The specification is silent but a reference implementation is the de + facto definition.** Mark the check `basis: de-facto`, name the reference, + and it is settled the same way. +3. **Genuinely ambiguous** — the specification is silent and there is no + reference implementation to defer to. This is the only case that reaches + the rest of this document. + +A check in the third case ships as `match: present`: it records what each +store did without judging it, because there is no rule yet to judge it +against. This file is how one gets a rule, so the check can be promoted to +`match: exact`. + +## The process + +1. **The disagreement is real, not a harness bug.** Before anything else, + rule out the alternative `CLAUDE.md` names: a poll shorter than a store's + commit interval, a read-back selector that assumed a label, a case + measured before a store finished starting. If ruling it out is not + possible from the case's own run, it is not ready for adjudication. +2. **Open an issue from the `Adjudication` template** + (`.github/ISSUE_TEMPLATE/adjudication.md`), naming the check, what every + implementation that has run it did, and precisely what the specification + does not settle — quoting the clause and saying where it stops, not + paraphrasing it. +3. **Put the question to the specification's owners** — the standards body, + or the project that maintains the de facto definition when there is no + formal spec — and link that discussion from the issue. An issue with no + upstream question linked stays open with nothing to promote: recording a + divergence is the point, and guessing at what the specification's authors + would say is exactly what this process exists to avoid. +4. **Record the answer verbatim in the issue and in the check's `notes:`** + when it arrives, with the date and who gave it. +5. **Promote the check only once a rule exists.** Change `match: present` to + `match: exact`, cite the answer in `rule:`, and re-run the suite: every + implementation that disagreed with the new rule is now `ALTER`, not + `present`, and that is the adjudication doing its job — a `present` check + never fails a backend for the divergence it records; an `exact` one does, + because there is now a rule it violates. + +A check does not sit at `present` forever by default. Every one gets an +adjudication issue, even if the issue stays open for years waiting on an +answer — `AGENTS.md`'s "Never" section is explicit that a `present` case is +not simply left as the final answer. + +## Two implementations disagreeing is not, on its own, a rule + +The order of precedence above is deliberately not "whichever behaviour is +more common" or "whichever behaviour the newest implementation chose." A +specification that leaves something unsettled leaves it unsettled regardless +of how many stores converge on one answer by accident; convergence is +evidence worth citing in the issue, not a substitute for an answer from the +people who own the specification. + +## Worked example + +`cases/otlp-logs/timestamp-nanosecond-precision.yaml` is the first case this +process was written for. Five stores answered five different ways for how +much of `time_unix_nano`'s precision survives a round trip — millisecond +string, microsecond integer, a nanosecond integer that turned out to hold +only whole microseconds, and two that kept every digit. The OpenTelemetry +logs specification defines what the field means and says nothing about what a +receiver must retain, so no rule exists yet to promote the check with, no +matter how the five stores split. The case's own `notes:` record each +measurement as it was made; see there for the reproductions. diff --git a/AGENTS.md b/AGENTS.md index 54b201a..b0de28a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -73,6 +73,9 @@ check exists to find. Two implementations disagreeing is not a rule. A check moves from `present` to `exact` only when a rule exists, not when the disagreement gets interesting. +[`ADJUDICATION.md`](ADJUDICATION.md) makes step 3 operational: the issue +template to use, what the upstream question has to establish, and what +"promote the check" actually changes. ### Do not tune a backend to fit the corpus diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 69c83cf..8ea813d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -19,8 +19,11 @@ value is entirely in every entry being defensible. ### If two backends disagree and you are unsure who is right -Say so in the pull request and leave the verdict out. Open an issue upstream, or -with whoever owns the specification, and let it be settled there. +Say so in the pull request and leave the verdict out — ship the check as +`match: present`. Then open an issue from the `Adjudication` template and work +through [`ADJUDICATION.md`](ADJUDICATION.md#the-process): the question goes to +whoever owns the specification, and the check is promoted to `match: exact` +only once they answer. A wrong verdict published under this project's name damages someone's reputation unfairly and destroys the project's own. Being slow is much cheaper than being diff --git a/GOVERNANCE.md b/GOVERNANCE.md new file mode 100644 index 0000000..c7558ae --- /dev/null +++ b/GOVERNANCE.md @@ -0,0 +1,63 @@ +# Governance + +The results are only worth reading if they are not for sale. This is written +down before anyone has a reason to ask. + +## Maintainers + +The project currently has one maintainer, [@DeviousCardi](https://github.com/DeviousCardi), +who has commit access and decides what merges. That is a starting point, not +a target: a project with one maintainer has a bus factor of one, and this file +exists so growing past that happens by a stated process rather than by +whoever shows up first getting write access. + +### How a maintainer is added + +1. Several substantive pull requests merged — a check with a real citation, an + adapter re-confirmed by hand, or a runner fix, not typo corrections. +2. A public proposal, as an issue on this repository, naming the person and + what they have contributed. +3. No standing maintainer objects within two weeks. + +Removal follows the same shape: a public issue, a reason, two weeks for +objection. A maintainer who has been inactive for two consecutive quarterly +reruns (`docs/BACKENDS.md`'s checklist, run by `.github/workflows/matrix.yml`) +is asked directly before being proposed for removal, not removed by default — +absence is not the same as disagreement. + +## Vendor conflict of interest + +A maintainer employed by, or paid by, a vendor whose backend appears in the +matrix **declares it in this file**, in the table below. A maintainer with a +declared interest in a vendor: + +- Does not adjudicate a case (`ADJUDICATION.md`) where that vendor's backend + is the reference implementation or the backend under dispute. +- Does not merge a pull request that changes that vendor's adapter, files a + finding against it, or resolves a `divergence` issue about it — review and + approval come from another maintainer. +- May still write cases, fix the runner, and do everything else a maintainer + does; the recusal is scoped to decisions about the vendor they are + affiliated with, not a blanket exclusion. + +An affiliation is declared before the maintainer takes any of the actions +above, not after being asked about it. + +| Maintainer | Vendor | Relationship | +| --- | --- | --- | +| _(none declared)_ | | | + +## Funding + +None. If that changes, the amount and source are added to +[`README.md`](README.md#funding), not only here — a reader deciding whether +to trust a result should not have to find this file to learn who paid for it. + +## Scope of this document + +This covers who decides what merges and who may declare a vendor conflict of +interest away from a decision. It does not cover the rule for what a check +may assert, which is `AGENTS.md`'s "Adjudication" section and made +operational in [`ADJUDICATION.md`](ADJUDICATION.md); it does not cover how a +backend maintainer disputes a specific published verdict, which is +`.github/ISSUE_TEMPLATE/divergence.md`. diff --git a/README.md b/README.md index 4932bec..59fa050 100644 --- a/README.md +++ b/README.md @@ -79,6 +79,8 @@ static `matrix.html` under `results///`. | `src/` | The runner. Sends cases, reads back, compares, decides verdicts. | | `tools/` | The corpus gate CI runs. | | [`AGENTS.md`](AGENTS.md) | The design, and how to add a check or a backend. | +| [`GOVERNANCE.md`](GOVERNANCE.md) | Who maintains this project, and vendor conflicts of interest. | +| [`ADJUDICATION.md`](ADJUDICATION.md) | How a `present` check earns a rule and becomes `exact`. | The corpus is the asset. The runner is plumbing and could be rewritten in a weekend; a corpus of checks that each trace to a line of a specification takes @@ -200,7 +202,14 @@ The results are only worth reading if they are not for sale. contribute to. A comparison written by an interested party is an advertisement. This should not -become one. +become one. [`GOVERNANCE.md`](GOVERNANCE.md) has who maintains this project and +the rule for a maintainer with a vendor conflict of interest; +[`ADJUDICATION.md`](ADJUDICATION.md) has how a check earns the right to fail a +backend rather than only record what it did. + +### Funding + +None. If that changes, the amount and source are listed here. ## Licence diff --git a/cases/otlp-logs/timestamp-nanosecond-precision.yaml b/cases/otlp-logs/timestamp-nanosecond-precision.yaml index d36d1ab..0ef157e 100644 --- a/cases/otlp-logs/timestamp-nanosecond-precision.yaml +++ b/cases/otlp-logs/timestamp-nanosecond-precision.yaml @@ -110,3 +110,9 @@ notes: > is not the same as a rule saying it must be kept, and the specification still does not give one. Recorded, not adjudicated — but this is now a strong question to put to the specification's owners rather than a curiosity. + + Adjudication opened 2026-09-09: + https://github.com/DeviousCardi/specmatrix/issues/12, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the OpenTelemetry + specification's owners — this check stays `present` until that answer + arrives, whichever way it goes.