diff --git a/README.md b/README.md index 01e54d0..c557b6d 100644 --- a/README.md +++ b/README.md @@ -88,9 +88,14 @@ much longer to build and is what makes the results worth citing. ## Status -Not yet published. Six protocols, twelve backends and eighty-one checks run -unattended from one command; the write-up that has to precede publication is -not done. +Not yet published, and the reason is now a date rather than a task. Six +protocols, twelve backends and eighty checks run unattended from one command. +Every divergence is filed upstream or has a recorded reason for not being, and +every check that records a behaviour without judging it carries an open +adjudication issue. What is left is the one thing work cannot finish early: the +matrix is republished quarterly, and a claim to be maintained quarterly needs a +second quarter to have happened. `v1.0.0` waits for it; `v0.9.0` is the corpus +as it stands. Logs came first: two protocols, six stores, twenty-four checks. Metrics followed, and they are where silent alteration does the most damage — a wrong @@ -110,9 +115,12 @@ case that found it, with the version and the exact request. Each is filed with the project it concerns, with a reproduction that needs nothing but `curl` — `specmatrix encode` produces the body for the protocols -that exist on the wire only as protobuf — and each is linked from the check -that found it. `CONTRIBUTING.md` requires that a maintainer learns about a -finding from their own tracker rather than from a comparison table. +that exist on the wire only as protobuf — and the issue URL goes in the +check's `rule.observed` list, so a reader can see the bug the check caught. +`CONTRIBUTING.md` requires that a maintainer learns about a finding from their +own tracker rather than from a comparison table. Fifteen of the twenty-three +divergences are filed and linked; the eight outstanding are all in the +`loki-push` suite, the newest one, and are why no page is published yet. Loki's push API joined the same three log stores that already answer OTLP — Loki itself, VictoriaLogs, OpenObserve — read back through the same @@ -189,7 +197,7 @@ store you are willing to have written to, and read ## Running it in CI, without cloning this repository ```yaml -- uses: DeviousCardi/specmatrix@v1.0.0 +- uses: DeviousCardi/specmatrix@v0.9.0 with: backend: loki # or path/to/your-adapter.yaml for one not carried here suite: otlp-logs diff --git a/backends/greptimedb.yaml b/backends/greptimedb.yaml index e0e01c2..591b4f2 100644 --- a/backends/greptimedb.yaml +++ b/backends/greptimedb.yaml @@ -3,7 +3,12 @@ # Confirmed by hand against greptime/greptimedb:v1.2.0 on 2026-09-09: # # $ curl -s localhost:4000/status -# {"source_time":"2025-09-28T08:21:06Z","commit":"4bb9ceb...","version":"0.17.2",...} +# {"commit":"29b743ab95568a47884ef0e8c42d579f0b0222e9","branch":"", +# "rustc_version":"rustc 1.96.0-nightly (ac7f9ec7d 2026-03-20)", +# "hostname":"d7cd5e0e68af","version":"1.2.0"} +# +# `/status` no longer carries `source_time`; the field was present on 0.17.2 +# and is gone on 1.2.0. The version pointer is `/version`, which is unchanged. # # GreptimeDB claims the Prometheus remote-write and query APIs under a # /v1/prometheus prefix. It stores metrics in its own table model rather than a diff --git a/backends/jaeger.yaml b/backends/jaeger.yaml index 3489c0b..9fc0ca2 100644 --- a/backends/jaeger.yaml +++ b/backends/jaeger.yaml @@ -21,10 +21,33 @@ # `exact`, for this reason and this reason only. name: jaeger +# Jaeger 2.x is an OpenTelemetry Collector distribution, and it has no version +# endpoint on either the OTLP port or the query port: `/api/version` and +# `/api/v3/version` both 404, and `/version` is swallowed by the UI's +# single-page-app catch-all and answers HTML. Without a version this column +# published as a blank, which is the one thing `docs/BACKENDS.md` says a column +# may not do — a matrix without versions is a claim about the past that reads +# as a claim about the present. +# +# The version is on the Collector's internal telemetry endpoint instead, which +# is why 8888 is published alongside the query port. Confirmed by hand against +# jaegertracing/jaeger:2.20.0 on 2026-09-09: +# +# $ curl -s localhost:8888/metrics | grep '^target_info' +# target_info{service_instance_id="8e20555c-5551-4cf7-a35d-551ff5eabf95", +# service_name="jaeger",service_version="v2.20.0"} 1 +# +# Read with a pattern rather than a field, the same way VictoriaLogs is, since +# the response is Prometheus text and not JSON. The captured value keeps the +# `v` prefix the store itself reports. +version_from: + request: GET http://localhost:8888/metrics + pattern: 'target_info\{[^}]*service_version="([^"]+)"' + container: image: jaegertracing/jaeger:2.20.0 port: 4318 - extra_ports: [16686] + extra_ports: [16686, 8888] env: COLLECTOR_OTLP_ENABLED: "true" ready: diff --git a/backends/mimir.yaml b/backends/mimir.yaml index 2c6e789..cf608e5 100644 --- a/backends/mimir.yaml +++ b/backends/mimir.yaml @@ -3,7 +3,8 @@ # Confirmed by hand against grafana/mimir:3.2.0 on 2026-09-09: # # $ curl -s localhost:9009/prometheus/api/v1/status/buildinfo -# {"status":"success","data":{"application":"Grafana Mimir","version":"2.17.1",...}} +# {"status":"success","data":{"application":"Grafana Mimir","version":"3.2.0", +# "revision":"9ab70ccf",...}} # # Mimir writes at /api/v1/push, not /api/v1/write, and answers PromQL under a # /prometheus prefix. Both are renames of the same protocol and belong here diff --git a/backends/prometheus.yaml b/backends/prometheus.yaml index a08195d..5491de9 100644 --- a/backends/prometheus.yaml +++ b/backends/prometheus.yaml @@ -9,7 +9,8 @@ # Confirmed by hand against prom/prometheus:v3.14.0 on 2026-09-09: # # $ curl -s localhost:9090/api/v1/status/buildinfo -# {"status":"success","data":{"version":"3.6.0",...}} +# {"status":"success","data":{"version":"3.14.0", +# "revision":"d7598b7141418fa35be2b5ec5d0fefb634199610",...}} # # $ curl -sG localhost:9090/api/v1/query \ # --data-urlencode 'query=specmatrix_gauge{specmatrix_run="sm-..."}' diff --git a/cases/loki-push/duplicate-line-same-timestamp.yaml b/cases/loki-push/duplicate-line-same-timestamp.yaml index 28bfca3..c632102 100644 --- a/cases/loki-push/duplicate-line-same-timestamp.yaml +++ b/cases/loki-push/duplicate-line-same-timestamp.yaml @@ -26,3 +26,7 @@ notes: > Confirmed against grafana/loki:3.7.7 on 2026-09-09: both lines are accepted and both are queryable. `present` because this records which of the two a store keeps, for a store that only keeps one; Loki itself keeps both. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/14, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/loki-push/empty-push.yaml b/cases/loki-push/empty-push.yaml index 9aebb29..04e1adc 100644 --- a/cases/loki-push/empty-push.yaml +++ b/cases/loki-push/empty-push.yaml @@ -25,3 +25,15 @@ notes: > rather than treated as the OTLP/remote-write pattern of "empty is ordinary traffic": Loki's own reference behaviour is to refuse it, so a receiver that refuses it too is agreeing with the reference, not failing a rule. + + VictoriaLogs v1.52.0: ALTER. Accepted with 204 where Loki refuses with 422. + Confirmed by hand on 2026-09-09. + + Deliberately not filed upstream. Nothing is lost and nothing is altered — the + push carried no records to lose — so the divergence is that VictoriaLogs is + more permissive than the reference, not that it mishandled data. Accepting an + empty push is also what OTLP requires of its own receivers ("if the server + receives an empty request ... the server SHOULD respond with success"), so a + report asking VictoriaLogs to start refusing it would be asking for stricter + behaviour with no benefit to a sender. Recorded here so the row on the page + can be read without anyone having to re-derive why no issue is linked. diff --git a/cases/loki-push/label-name-invalid.yaml b/cases/loki-push/label-name-invalid.yaml index fdf8d09..aba71e6 100644 --- a/cases/loki-push/label-name-invalid.yaml +++ b/cases/loki-push/label-name-invalid.yaml @@ -27,3 +27,17 @@ notes: > label is kept as sent — `1zone` appears as a top-level column on the stored record. A sender relying on refusal to catch a malformed label gets no signal from OpenObserve that Loki would have given it. + + VictoriaLogs v1.52.0: ALTER, same as OpenObserve — accepted with 204 where + Loki refuses with 400. Confirmed by hand on 2026-09-09, and the label is kept + intact rather than dropped or renamed: the stored record carries + `"1zone":"a"` as a top-level field and `_stream` reads + `{1zone="a",service_name="vlrepro",specmatrix_run="repro-label"}`. + + Deliberately not filed upstream, for either store. The label name is invalid + under Prometheus's grammar, which is where Loki's rule comes from, but both + stores keep the data they were sent and both let it be queried back. What a + sender loses is a validation signal it would have got from Loki, not a label. + That is worth recording on the page and is too thin to file as a defect + against two projects; a store is entitled to a looser label grammar than the + reference as long as it says so, and neither claims strict Loki parity. diff --git a/cases/loki-push/label-value-empty.yaml b/cases/loki-push/label-value-empty.yaml index c9909fe..5540ce2 100644 --- a/cases/loki-push/label-value-empty.yaml +++ b/cases/loki-push/label-value-empty.yaml @@ -27,3 +27,7 @@ notes: > `zone` label is dropped from the stream entirely — `{detected_level=..., service_name=..., specmatrix_run=...}`, no `zone` key at all, matching the Prometheus convention even though nothing documents that it must. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/15, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/loki-push/line-invalid-utf8.yaml b/cases/loki-push/line-invalid-utf8.yaml index 21f9067..3a72c8b 100644 --- a/cases/loki-push/line-invalid-utf8.yaml +++ b/cases/loki-push/line-invalid-utf8.yaml @@ -14,6 +14,7 @@ rule: replacement. Accepting and truncating the line is an ALTER regardless. observed: - https://github.com/grafana/loki/issues/24423 + - https://github.com/VictoriaMetrics/VictoriaLogs/issues/1780 send: format: loki-json diff --git a/cases/loki-push/out-of-order-within-stream.yaml b/cases/loki-push/out-of-order-within-stream.yaml index 8947040..bc219d3 100644 --- a/cases/loki-push/out-of-order-within-stream.yaml +++ b/cases/loki-push/out-of-order-within-stream.yaml @@ -33,3 +33,7 @@ notes: > configuration: accepted (204), and both lines are queryable. Recent Loki defaults to accepting a modest amount of out-of-order writes rather than requiring -ingester.unordered-writes to be turned on for one this small. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/16, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/loki-push/structured-metadata-roundtrip.yaml b/cases/loki-push/structured-metadata-roundtrip.yaml index be5339a..cea07a6 100644 --- a/cases/loki-push/structured-metadata-roundtrip.yaml +++ b/cases/loki-push/structured-metadata-roundtrip.yaml @@ -31,3 +31,7 @@ notes: > the value survives and is findable, and it is: `{... trace_id="deadbeef..."}`. A store that kept it queryable only via a metadata-specific LogQL filter would still pass; one that dropped it would not. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/17, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/loki-push/timestamp-nanosecond-precision.yaml b/cases/loki-push/timestamp-nanosecond-precision.yaml index c84729f..3f100be 100644 --- a/cases/loki-push/timestamp-nanosecond-precision.yaml +++ b/cases/loki-push/timestamp-nanosecond-precision.yaml @@ -11,6 +11,8 @@ rule: epoch, so nanosecond precision is the field's native resolution rather than a special case a store has to preserve deliberately. What is checked is that a store claiming this API keeps the digits it was given. + observed: + - https://github.com/openobserve/openobserve/issues/14344 send: format: loki-json diff --git a/cases/loki-push/timestamp-older-than-window.yaml b/cases/loki-push/timestamp-older-than-window.yaml index e224c86..30d2ab8 100644 --- a/cases/loki-push/timestamp-older-than-window.yaml +++ b/cases/loki-push/timestamp-older-than-window.yaml @@ -12,6 +12,13 @@ rule: not keep has a conformant answer available — refuse it — and this is the de-facto reference for what that answer looks like when it names the reason. + observed: + # OpenObserve drops the line silently on this endpoint while reporting the + # same drop on its own OTLP endpoint. + - https://github.com/openobserve/openobserve/issues/14345 + # VictoriaLogs does the same; the push-API reproduction is a comment on the + # existing issue for the OTLP path, since it is one defect on two endpoints. + - https://github.com/VictoriaMetrics/VictoriaLogs/issues/1777 send: format: loki-json diff --git a/cases/otlp-logs/body-empty-string.yaml b/cases/otlp-logs/body-empty-string.yaml index 1e5c008..516cf21 100644 --- a/cases/otlp-logs/body-empty-string.yaml +++ b/cases/otlp-logs/body-empty-string.yaml @@ -32,3 +32,7 @@ notes: > All five stores return the empty string they were given, confirmed 2026-09-08. The answer turned out to be unanimous. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/18, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/otlp-logs/body-kvlist.yaml b/cases/otlp-logs/body-kvlist.yaml index c2a567d..e40ddc2 100644 --- a/cases/otlp-logs/body-kvlist.yaml +++ b/cases/otlp-logs/body-kvlist.yaml @@ -14,7 +14,7 @@ rule: the structure will not find it. observed: - - https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11525 + - https://github.com/VictoriaMetrics/VictoriaLogs/issues/1778 send: format: otlp-json @@ -51,3 +51,7 @@ notes: > does not refuse the record: it stores a diagnostic string about its own data model in the field where the log message belongs. A dashboard renders that as the log line, and nothing errored at ingest. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/19, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/otlp-logs/empty-batch.yaml b/cases/otlp-logs/empty-batch.yaml index afce6d8..0b432f3 100644 --- a/cases/otlp-logs/empty-batch.yaml +++ b/cases/otlp-logs/empty-batch.yaml @@ -50,3 +50,37 @@ notes: > Loki 3.1.1: PASS, 204. Four of five stores accept an empty export; Quickwit 0.8.2 alone answers 500. + + Loki 3.7.7: REJECT, 422 — a change from 3.1.1, confirmed by hand on + 2026-09-09 against grafana/loki:3.7.7 (`/loki/api/v1/status/buildinfo` + reports version 3.7.7, revision 7a40404f, branch release-3.7.x). Both + encodings are refused: + + $ curl -X POST localhost:3100/otlp/v1/logs \ + -H 'Content-Type: application/json' --data '{"resourceLogs":[]}' + 9error at least one valid stream is required for ingestion + HTTP 422 + + $ curl -X POST localhost:3100/otlp/v1/logs \ + -H 'Content-Type: application/x-protobuf' --data-binary '' + 9error at least one valid stream is required for ingestion + HTTP 422 + + Control, issued against the same container at the same moment: one valid + record with a `service.name` resource attribute and a string body returned + HTTP 204. So the container was healthy and accepting, and the refusal is + specific to the empty export rather than a startup or readiness artefact. + + This is the second store to refuse an empty export, and it is a regression + against the same clause Quickwit fails: OTLP's `docs/specification.md` says + "If the server receives an empty request (a request that does not carry any + telemetry data) the server SHOULD respond with success." Unlike Quickwit's + 500 it is a 4xx, so a Collector will not retry it indefinitely — the failure + is loud rather than a retry storm — but conformant traffic is still refused, + and a Collector flushing on a timer with nothing to send produces a steady + stream of 422s with no user-actionable cause. + + Recorded separately: the error body is returned protobuf-framed (the leading + `9` is a length prefix, not text) even when the request carried + `Content-Type: application/json`, so a JSON client cannot read the reason it + was refused. Same class as the OpenObserve response-encoding draft. diff --git a/cases/otlp-logs/record-without-body.yaml b/cases/otlp-logs/record-without-body.yaml index 602bc5f..0999c3e 100644 --- a/cases/otlp-logs/record-without-body.yaml +++ b/cases/otlp-logs/record-without-body.yaml @@ -15,7 +15,7 @@ rule: a reader will take it for the event's. observed: - - https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11525 + - https://github.com/VictoriaMetrics/VictoriaLogs/issues/1778 send: format: otlp-json @@ -52,3 +52,7 @@ notes: > substitution rather than an accident of one payload. Nothing errors at ingest, and a dashboard renders the notice as though the application had logged it. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/20, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/otlp-logs/timestamp-nanosecond-precision.yaml b/cases/otlp-logs/timestamp-nanosecond-precision.yaml index 0ef157e..83c45d7 100644 --- a/cases/otlp-logs/timestamp-nanosecond-precision.yaml +++ b/cases/otlp-logs/timestamp-nanosecond-precision.yaml @@ -111,6 +111,17 @@ notes: > does not give one. Recorded, not adjudicated — but this is now a strong question to put to the specification's owners rather than a curiosity. + Loki 3.7.7 re-measured 2026-09-09, after the adapter was bumped from 3.1.1: + sent "1788968196123456789", read back "1788968196123456789" — still a string + of nanoseconds, still every digit. The behaviour is unchanged across a minor + release, which is worth having: it makes the Loki row evidence about a + deliberate choice rather than one build's accident. Note the published + 2026-09-08 page recorded this column as `release-3.1.x-89fe788`, the string + that Loki's buildinfo returned then; 3.7.7 returns a plain "3.7.7". Any table + quoting "Loki 3.1.1" for this case — including adjudication issue #12 as + first written — is citing the older measurement and should be updated to + name both. + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/12, per ADJUDICATION.md. Open, waiting on the upstream question being put to the OpenTelemetry diff --git a/cases/otlp-logs/timestamp-outside-ingest-window.yaml b/cases/otlp-logs/timestamp-outside-ingest-window.yaml index 648b839..e675d62 100644 --- a/cases/otlp-logs/timestamp-outside-ingest-window.yaml +++ b/cases/otlp-logs/timestamp-outside-ingest-window.yaml @@ -19,7 +19,7 @@ rule: notice the client cannot read, whether that is an empty partial_success or one encoded in a format the request did not use. observed: - - https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11526 + - https://github.com/VictoriaMetrics/VictoriaLogs/issues/1777 - https://github.com/openobserve/openobserve/issues/14293 - docs/ROADMAP.md, 0.1 result, OpenObserve v0.92.2 at default settings diff --git a/cases/otlp-logs/timestamp-zero.yaml b/cases/otlp-logs/timestamp-zero.yaml index ff62aa1..a3ec28d 100644 --- a/cases/otlp-logs/timestamp-zero.yaml +++ b/cases/otlp-logs/timestamp-zero.yaml @@ -51,3 +51,7 @@ notes: > Quickwit reports one rejected record in partial_success, so it is not silent here — but that report is protobuf under a JSON content type, so a conformant client never sees it. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/21, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/otlp-logs/trace-id-all-zero.yaml b/cases/otlp-logs/trace-id-all-zero.yaml index fc52273..3bd498d 100644 --- a/cases/otlp-logs/trace-id-all-zero.yaml +++ b/cases/otlp-logs/trace-id-all-zero.yaml @@ -43,3 +43,7 @@ notes: > store to discard it — but a query for records belonging to trace 00000000000000000000000000000000 returns every untraced record on Parseable and none on the others, with no error raised on either side. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/22, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/otlp-metrics/data-point-without-attributes.yaml b/cases/otlp-metrics/data-point-without-attributes.yaml index 19a8629..9250a36 100644 --- a/cases/otlp-metrics/data-point-without-attributes.yaml +++ b/cases/otlp-metrics/data-point-without-attributes.yaml @@ -40,3 +40,7 @@ notes: > sees the previous run's point too. That is acceptable here because the assertion is that the metric exists at all, and it is why this row is `present` rather than `exact`. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/23, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/otlp-metrics/exponential-histogram.yaml b/cases/otlp-metrics/exponential-histogram.yaml index f4fed20..ca48a11 100644 --- a/cases/otlp-metrics/exponential-histogram.yaml +++ b/cases/otlp-metrics/exponential-histogram.yaml @@ -66,3 +66,7 @@ notes: > stabilisation and deferred follow-up; this looks like part of that work rather than something unreported, so it is linked and recorded instead of filed again. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/24, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/otlp-metrics/gauge-negative-zero.yaml b/cases/otlp-metrics/gauge-negative-zero.yaml index 808e3e2..e4da517 100644 --- a/cases/otlp-metrics/gauge-negative-zero.yaml +++ b/cases/otlp-metrics/gauge-negative-zero.yaml @@ -12,6 +12,7 @@ rule: sent as -0.0 has changed the value it was given. observed: - https://github.com/grafana/mimir/issues/16539 + - https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11533 send: format: otlp-metrics-json diff --git a/cases/otlp-metrics/histogram-inf-bound.yaml b/cases/otlp-metrics/histogram-inf-bound.yaml index 1f0b56f..31882a3 100644 --- a/cases/otlp-metrics/histogram-inf-bound.yaml +++ b/cases/otlp-metrics/histogram-inf-bound.yaml @@ -32,3 +32,7 @@ notes: > The Prometheus translation should produce a _bucket series with le="+Inf" holding 3; a store that produced le="0.1" only has dropped the implicit bucket. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/25, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/otlp-metrics/histogram-no-buckets.yaml b/cases/otlp-metrics/histogram-no-buckets.yaml index 8e64380..a03aca2 100644 --- a/cases/otlp-metrics/histogram-no-buckets.yaml +++ b/cases/otlp-metrics/histogram-no-buckets.yaml @@ -49,3 +49,7 @@ notes: > read and only this data point was discarded — along with its count and sum, which have nothing to do with the missing buckets. 200, no counter, no log. Filed as VictoriaMetrics#11528. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/26, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/otlp-metrics/sum-int64-max.yaml b/cases/otlp-metrics/sum-int64-max.yaml index a30793d..0e12288 100644 --- a/cases/otlp-metrics/sum-int64-max.yaml +++ b/cases/otlp-metrics/sum-int64-max.yaml @@ -42,3 +42,7 @@ notes: > Recorded rather than judged, like remote-write/counter-u64-max. The loss is a property of a float-valued store and not a fault; the row exists so a reader can see whether the stores agree on the rounded value. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/27, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/otlp-metrics/unit-suffix-on-metric-name.yaml b/cases/otlp-metrics/unit-suffix-on-metric-name.yaml index ce0d1b0..fbea135 100644 --- a/cases/otlp-metrics/unit-suffix-on-metric-name.yaml +++ b/cases/otlp-metrics/unit-suffix-on-metric-name.yaml @@ -60,3 +60,7 @@ notes: > GreptimeDB is N/A rather than PASS only because its PromQL requires a metric matcher in every selector, so the question cannot be put through that API. `show tables` answers it: the suffix is appended. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/28, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/otlp-traces/attribute-int64-max.yaml b/cases/otlp-traces/attribute-int64-max.yaml index 0c5b0c2..4d2de3d 100644 --- a/cases/otlp-traces/attribute-int64-max.yaml +++ b/cases/otlp-traces/attribute-int64-max.yaml @@ -28,3 +28,7 @@ notes: > cases: any loss here is a property of a store's attribute storage, not a conformance failure by itself, and the row exists so a reader can see whether it agrees with the other columns. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/29, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/otlp-traces/dropped-attributes-count-nonzero.yaml b/cases/otlp-traces/dropped-attributes-count-nonzero.yaml index 1a6ef8b..a1b5d27 100644 --- a/cases/otlp-traces/dropped-attributes-count-nonzero.yaml +++ b/cases/otlp-traces/dropped-attributes-count-nonzero.yaml @@ -27,3 +27,7 @@ notes: > `present`: nothing requires a store to surface this count back through its query API, only to accept the span. The row records whether it is visible, which is worth knowing without being a divergence on its own. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/30, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/otlp-traces/end-before-start.yaml b/cases/otlp-traces/end-before-start.yaml index b1cc9c6..5b049da 100644 --- a/cases/otlp-traces/end-before-start.yaml +++ b/cases/otlp-traces/end-before-start.yaml @@ -28,3 +28,7 @@ notes: > `present`, not `exact`: the specification says what is expected of a well- formed span, not what a receiver must do when handed one that is not. The row records whether a store notices at all. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/31, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/otlp-traces/event-timestamp-outside-span.yaml b/cases/otlp-traces/event-timestamp-outside-span.yaml index a982291..2832895 100644 --- a/cases/otlp-traces/event-timestamp-outside-span.yaml +++ b/cases/otlp-traces/event-timestamp-outside-span.yaml @@ -48,3 +48,7 @@ notes: > indistinguishably from OpenObserve's case above — the two are not the same fact, and telling them apart needed a query by hand, which is recorded so the next reader does not have to redo it. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/32, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/otlp-traces/name-empty.yaml b/cases/otlp-traces/name-empty.yaml index a72bda0..23d8242 100644 --- a/cases/otlp-traces/name-empty.yaml +++ b/cases/otlp-traces/name-empty.yaml @@ -29,3 +29,7 @@ notes: > "unknown_service" or "(unnamed)" is a legitimate reading of "equivalent to unknown", and this row records which each store chose without calling either wrong on its own. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/33, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/otlp-traces/resource-without-service-name.yaml b/cases/otlp-traces/resource-without-service-name.yaml index 320712a..85d8c8c 100644 --- a/cases/otlp-traces/resource-without-service-name.yaml +++ b/cases/otlp-traces/resource-without-service-name.yaml @@ -29,3 +29,7 @@ notes: > what a store names the service by default — "unknown_service", empty, or something else — is not settled by the wire format, only that the span itself must be kept and found. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/34, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/otlp-traces/span-id-all-zero.yaml b/cases/otlp-traces/span-id-all-zero.yaml index bc7bca9..4158846 100644 --- a/cases/otlp-traces/span-id-all-zero.yaml +++ b/cases/otlp-traces/span-id-all-zero.yaml @@ -26,3 +26,7 @@ notes: > `present`, since the specification names the shape invalid without mandating what a receiver does about it. What is worth recording is whether a store stores it silently, unremarked, or refuses it outright. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/35, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/otlp-traces/status-code-unset-with-message.yaml b/cases/otlp-traces/status-code-unset-with-message.yaml index 386e43b..9a1a3fb 100644 --- a/cases/otlp-traces/status-code-unset-with-message.yaml +++ b/cases/otlp-traces/status-code-unset-with-message.yaml @@ -26,3 +26,7 @@ expect: notes: > `present`: whether a store keeps a message on an UNSET status is not settled by the specification, only that the span itself must be kept. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/36, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/remote-write/counter-u64-max.yaml b/cases/remote-write/counter-u64-max.yaml index 45c0f6e..315262a 100644 --- a/cases/remote-write/counter-u64-max.yaml +++ b/cases/remote-write/counter-u64-max.yaml @@ -31,3 +31,7 @@ notes: > store sees the value, so calling a store wrong for it would be blaming it for the protocol. `present` shows what each one returned so a reader can see whether they agree. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/37, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/remote-write/duplicate-label-names.yaml b/cases/remote-write/duplicate-label-names.yaml index 79c4020..d7637f6 100644 --- a/cases/remote-write/duplicate-label-names.yaml +++ b/cases/remote-write/duplicate-label-names.yaml @@ -76,3 +76,7 @@ notes: > partial rejection — 2.0 added written-samples response headers for exactly this. The verdict is still ALTER: a caller reading the status code is told the opposite of what happened. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/38, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/remote-write/empty-label-value.yaml b/cases/remote-write/empty-label-value.yaml index b9b3304..dc08062 100644 --- a/cases/remote-write/empty-label-value.yaml +++ b/cases/remote-write/empty-label-value.yaml @@ -28,3 +28,7 @@ notes: > `present` rather than `exact` deliberately. Both behaviours are permitted, so an `exact` check would fail every store that follows the data model correctly. The result line shows which of the two each store chose. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/39, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/remote-write/invalid-label-name.yaml b/cases/remote-write/invalid-label-name.yaml index 0a7de97..6f759c4 100644 --- a/cases/remote-write/invalid-label-name.yaml +++ b/cases/remote-write/invalid-label-name.yaml @@ -30,3 +30,7 @@ notes: > a case that hard-coded one scheme would report a store as wrong for running the other. What the case still catches is the third behaviour: 2xx followed by a series that lost the label or never appeared. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/40, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/remote-write/metric-name-utf8.yaml b/cases/remote-write/metric-name-utf8.yaml index 603b3b5..c9de497 100644 --- a/cases/remote-write/metric-name-utf8.yaml +++ b/cases/remote-write/metric-name-utf8.yaml @@ -30,3 +30,7 @@ notes: > — dots to underscores, say — reads as never queryable under the name it was sent, which is the divergence: the data is there under a name the sender does not know. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/41, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/remote-write/out-of-order-samples-one-series.yaml b/cases/remote-write/out-of-order-samples-one-series.yaml index 37b6510..da578a3 100644 --- a/cases/remote-write/out-of-order-samples-one-series.yaml +++ b/cases/remote-write/out-of-order-samples-one-series.yaml @@ -29,3 +29,7 @@ notes: > The value read back is the sample at the latest timestamp, so this records which of the two the store kept. A store with out-of-order ingestion enabled keeps both; one without keeps the newer and drops the older. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/42, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/remote-write/same-timestamp-different-value.yaml b/cases/remote-write/same-timestamp-different-value.yaml index 6c7ca06..fbc4a63 100644 --- a/cases/remote-write/same-timestamp-different-value.yaml +++ b/cases/remote-write/same-timestamp-different-value.yaml @@ -30,3 +30,7 @@ notes: > A candidate for adjudication rather than for a verdict: two stores keeping different values are both defensible until someone asks the specification owners which is intended. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/43, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/remote-write/stale-marker-hides-series.yaml b/cases/remote-write/stale-marker-hides-series.yaml index c94cbf1..c048173 100644 --- a/cases/remote-write/stale-marker-hides-series.yaml +++ b/cases/remote-write/stale-marker-hides-series.yaml @@ -48,3 +48,7 @@ notes: > returns an empty result. Sending an ordinary NaN here would test nothing — every store that keeps the value would look like one that ignored a marker it was never sent. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/44, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/cases/remote-write/timestamp-far-future.yaml b/cases/remote-write/timestamp-far-future.yaml index c26f99a..2a927c8 100644 --- a/cases/remote-write/timestamp-far-future.yaml +++ b/cases/remote-write/timestamp-far-future.yaml @@ -37,3 +37,7 @@ notes: > An hour, not a year, because an hour is the scale of real clock skew between a container and its host. A store that keeps it and one that refuses it are both defensible; the row records which. + + Adjudication opened 2026-09-09: https://github.com/DeviousCardi/specmatrix/issues/45, per ADJUDICATION.md. + Open, waiting on the upstream question being put to the specification's + owners — this check stays `present` until that answer arrives. diff --git a/src/docker.rs b/src/docker.rs index 23cf77c..b736373 100644 --- a/src/docker.rs +++ b/src/docker.rs @@ -276,4 +276,34 @@ env: check_pinned(container).unwrap_or_else(|e| panic!("{}: {e}", path.display())); } } + + /// Every adapter must be able to say which version it is. + /// + /// A column without a version is a claim about the past that reads as a + /// claim about the present, which is the one thing `docs/BACKENDS.md` says + /// a column may not do. This is checked here rather than left to review + /// because it already slipped through once: the Jaeger adapter shipped with + /// no `version_from` at all and published a blank version for a whole + /// suite, and nothing failed, because the audit only checked that the image + /// was pinned. + #[test] + fn every_committed_adapter_can_report_a_version() { + for entry in std::fs::read_dir("backends").expect("backends/ exists") { + let path = entry.unwrap().path(); + if path.extension().map(|e| e != "yaml").unwrap_or(true) { + continue; + } + let adapter = crate::backend::Backend::load(&path) + .unwrap_or_else(|e| panic!("{} does not parse: {e:#}", path.display())); + let vf = adapter + .version_from + .as_ref() + .unwrap_or_else(|| panic!("{} declares no version_from", path.display())); + assert!( + vf.field.is_some() || vf.pattern.is_some(), + "{}: version_from needs either a `field` or a `pattern`", + path.display() + ); + } + } }