0.4 — OTLP traces: Jaeger, Tempo, Quickwit, OpenObserve - #9
Merged
Merged
Conversation
…(Part G)
Fifteen checks in a new `otlp-traces` suite, run against four backends that
each represent a span in a different shape than OTLP's own — which forced
several runner fixes that generalize past traces:
- `Runner::send()` and `docker::wait_ready()` now treat a rendered request
path starting with `http://`/`https://` as an absolute URL and use it
verbatim, bypassing `base_url`. Needed because Jaeger and Tempo split
ingest and query across two ports of one container; regression-tested with
two separate stub servers so neither endpoint can silently fall back to
the other.
- `Container` gained `extra_ports` (publish additional container ports) and
`config` (an inline file `specmatrix up` writes to a temp path and mounts
at a fixed path), for settings with no CLI-flag equivalent — confirmed by
reading each binary's own `-help` first. Tempo needs this: its OTLP
receiver binds to 127.0.0.1 inside the container unless the config sets an
explicit 0.0.0.0 endpoint, and nothing in the API says so, only
`docker logs`.
- `export_report()` was comparing `request_encoding == "otlp-json"` literally,
so it was silently wrong for every OTLP metrics/traces JSON export
(`otlp-metrics-json`/`otlp-traces-json` never matched) and its protobuf
decode path was hardcoded to the logs response type — a genuine traces or
metrics partial-success report decoded to `None` and was invisibly
dropped. Now takes `protocol: &str` and branches field names and decode
type per protocol; confirmed against Jaeger, and against Quickwit's own
genuine json-response-to-protobuf-request mismatch, which survives
correctly.
- `field_of()` gained a fallback chain for fields with no adapter mapping:
`attributes.<key>` searches the record's own OTLP-shaped attributes array,
and any other dotted field with no mapping is tried as a JSON pointer.
Verified against Tempo, isolating cases via move-aside/restore to confirm
each fix in a clean run.
Found and fixed a real regression along the way: an earlier edit meant to add
`trace_id_base64` to `vars_for()` silently failed to save (a multi-assert
Python script whose last assert failed, so the final write never ran), while
a separately-applied edit to `read_back()` for the same feature did save —
masking the omission. Combined with `first_record()`'s `.contains(run_key)`
check, where `"".contains("")` is `true` in Rust, the always-empty
`trace_id_base64` spuriously matched the first record in any response,
breaking `a_leftover_record_from_another_run_does_not_count`. Fixed by
inserting the missing variable and adding a permanent guard —
`first_record()` now refuses to match an empty key — plus a regression test.
Adapter notes, each confirmed against a running container:
- Tempo answers real OTLP JSON on read-back, unlike every other store here,
except trace/span IDs come back base64 (protobuf JSON's own bytes-field
encoding) where this project's encoder uses hex — compared as bytes, not
equal strings.
- Jaeger answers only its own query model: `startTime`/`duration` in
microseconds, not OTLP's nanoseconds, and a span event becomes a `logs[]`
entry keyed by `fields[].key == "event"`.
- Quickwit and OpenObserve both flatten spans into search columns; OpenObserve
stores a span's events as a JSON string rather than structured JSON, so no
JSON-pointer mapping can reach it at all.
`event-timestamp-outside-span` is `match: present` rather than `exact` for
this reason: no store here keeps OTLP's own field shape for a span event, so
the check records where each one put the data rather than judging a shape
none of them chose to keep.
All 15 cases pass on all 4 columns (60/60). 183 unit tests, 0 clippy warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018k65nFSzuwsHwYnpSHNaeK
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Part G of PLAN-1.0.md: a new
otlp-tracessuite (15 checks) run against fourbackends — Jaeger, Grafana Tempo, Quickwit, OpenObserve — each of which
represents a span in a genuinely different shape than OTLP's own.
All 15 cases pass on all 4 columns (60/60). 183 unit tests, 0 clippy warnings.
Runner changes (generalize past traces)
Runner::send()anddocker::wait_ready(): arendered request path starting with
http:///https://is used verbatim,bypassing
base_url. Needed because Jaeger and Tempo split ingest andquery across two ports of one container. Regression-tested with two
separate stub servers so neither endpoint can fall back to the other.
Container.extra_ports/Container.config: publish additionalcontainer ports, and mount an inline config file at a fixed path, for
settings with no CLI-flag equivalent. Tempo needs this — its OTLP receiver
binds to
127.0.0.1inside the container unless the config sets anexplicit
0.0.0.0endpoint, confirmed only viadocker logs, not the API.export_report()made protocol-aware: it was comparingrequest_encoding == "otlp-json"literally, silently wrong for every OTLPmetrics/traces JSON export, and its protobuf decode path was hardcoded to
the logs response type — a genuine traces/metrics partial-success report
decoded to
Noneand was invisibly dropped before this fix.field_of()fallback chain extended:attributes.<key>searches therecord's own OTLP-shaped attributes array; any other dotted field with no
adapter mapping is tried as a JSON pointer.
A regression found and fixed along the way
An earlier edit meant to add
trace_id_base64tovars_for()silentlyfailed to save (a multi-assert Python script whose last assert failed, so the
final file write never ran) while a separately-applied edit to
read_back()for the same feature did save — masking the omission. The always-empty
trace_id_base64, combined withfirst_record()'s.contains(run_key)check (
"".contains("")istruein Rust), spuriously matched the firstrecord in any response. Fixed by inserting the missing variable and adding a
permanent guard:
first_record()now refuses to match an empty key, with aregression test.
Adapter notes (confirmed against running containers)
back base64 (protobuf JSON's bytes-field encoding) where this project's
encoder uses hex — compared as bytes, not equal strings.
span event becomes a
logs[]entry.OpenObserve stores events as a JSON string, unreachable by any field
pointer.
event-timestamp-outside-spanismatch: presentrather thanexactbecause no store here keeps OTLP's own field shape for a span event — the
check records where each one put the data.
🤖 Generated with Claude Code
https://claude.ai/code/session_018k65nFSzuwsHwYnpSHNaeK