Bring the server to Content Telemetry v1.0 conformance

[jalexspringer]

Implements the v1.0 conformance worklist from the spec review, against the normative spec at SPUR-Coalition/telemetry (main = v1.0).

Compatibility posture

Live member Cloudflare workers still emit v0.1 edge batches (schema_version: "0.1", data.bot_category, source_role: edge); no agents are integrated yet. So:

• Both "0.1" and "1.0" are accepted. Documents declaring "1.0" (or nothing) get full v1 strictness; documents declaring "0.1" are normalised per spec §12.1 where a rule exists (absent data.citation_type → unclassified; absent content_grounded data.scope → turn/session by turn_id presence; data.bot_category read as purpose) and tolerated otherwise.
• Materialised session documents stamp schema_version: "1.0" — truthful because stored rows are read under the same §12.1 migration rules, and rows that still fail the v1 shape are quarantined under extensions.events.

Changes

• content_reproduced excised as a core type (it existed only on the pre-release v1-draft line, §12.1): removed from the core sets, enum rules, structural rules, error text, docs and tests. Stored draft rows self-quarantine into extensions.events — covered by a new test.
• Missing v1 structural enforcement on "1.0" documents: content_grounded requires closed data.scope (§6.4) and consistent provenance/cached pairing (agent_fetched⇒cached:false, agent_cached⇒cached:true); content_cited requires data.citation_type (§6.5); content_retrieved requires source_role (§5.2.2).
• Field-placement rules (§5.7.5): presentation_id and event-level ctx_token only on content_engaged; citation_id only on content_presented; turn only on turn events. The envelope ctx_token-only-with-engagements check is hoisted out of resolve_binding so it applies even when a session_id is present.
• content_fingerprint.preserved_in_output is stripped wherever withdrawn fields are stripped (it lives one level down; the old sweep only caught top-level keys).
• Migration 0003 adds ctx_token and terms_ref columns to events. terms_ref (§5.2.4) passes through byte-for-byte, never rewritten; the event-level ctx_token materialises on content_engaged only.
• ctx tokens (§7.4.1): minted as ct_ + 32 CSPRNG bytes base64url (256 bits vs the 96-bit floor); a ctx_token_well_formed() predicate (^ct_[A-Za-z0-9_-]{16,240}$) is applied at mint (including caller-supplied overrides), at event binding, and on recorded event-level tokens.
• bot_category → purpose in the read layer (§12.1): every filter and grouping reads COALESCE(event_data->>'purpose', event_data->>'bot_category'); query params take purpose with bot_category as a deprecated serde alias; the agent breakdown returns the value under both names during the transition.
• Funnel/report surfaces: the reproduced series is dropped; stored v0.1 content_displayed rows keep counting inside the presented stage as an explicit legacy branch, and the presented count is returned under both presented and displayed keys (the website dashboard still reads displayed; a coordinated rename comes later).
• Docs sweep: README example and smoke script on the "1.0" line with scope/citation_type; "click manifest" → "click context"; RSL profile spec_refs point at §6.4/§6.5.

Deliberately left out (separate PRs)

• The §7.4.4 four-component click-context response (the resolver still returns the v0.1 whole-session manifest shape under the existing consent gates) — TODO(spec 7.4.4) markers at routes.rs::lookup_ctx and click_tokens.rs::lookup_by_token.
• content_id-prefix owner resolution (§7.3) — TODO(spec 7.3) in click_tokens.rs.

Testing

• cargo test --workspace green (64 unit + 12 DB integration tests, against Postgres 17) — includes new tests for version acceptance, §12.1 migration defaults, the new structural and placement rules, token minting/grammar, and extension quarantine.
• cargo clippy --workspace --all-targets and cargo fmt --check clean.
• scripts/smoke.sh passes end-to-end against a live build.
• A materialised session document containing a v1 funnel, a normalised v0.1 batch, and a quarantined draft content_reproduced row validates against the normative telemetry-session.json (jsonschema 4.26, Draft 2020-12).

🤖 Generated with Claude Code