v1 identity, spec corrections, and conformance-suite hardening - #40
Merged
Merged
Conversation
The spec body already makes v1 normative claims (ip_hash withdrawal in 9.1, the content_displayed rename in 12.1) while the header still said 0.1/Preview and every schema pinned schema_version to "0.1" under a /schema/v0.1/ $id, so a conforming v1 emitter had to declare "0.1" and the two versions were indistinguishable on the wire. - Header: Version 1.0 (release candidate draft), status release candidate in preparation, feature freeze 21 August 2026. - All four schemas: $id path v0.1 -> v1, schema_version const -> "1.0"; manifest const description now says v1 emitters MUST use "1.0". - 5.7.4 and 12.1 now state explicitly that v1 documents declare "1.0", that a v0.1 consumer rejects them under the preview rule, and that a v1 consumer rejects "0.1". - Every fixture and every inline example in SPECIFICATION.md and README.md now declares "1.0". invalid/invalid-schema-version.json keeps its deliberately wrong "2.0" (still != the new const). - validate.py / tests/README.md docstrings updated; no hardcoded versions or v0.1 paths remain in the test scripts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The table said id was optional and generated by the server if not provided. The schema requires id on content_cited, content_reproduced and content_presented, and server-side generation is incompatible with citation_id and presentation_id, which reference an id the emitter must already hold when it constructs the referencing event. Reworded to the table's conditional style (as used by output_id and presentation_id): required for reproduced/cited/presented, optional elsewhere, emitter-assigned. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Section 5.7.3 lists data.citation_type among the Citation-emitter requirements marked schema-enforced, but the content_cited conditional required only id and output_id - unlike content_reproduced and content_presented, whose conditionals require their data objects. The content_cited conditional now requires data with citation_type, mirroring the reproduced (data.reproduction_type) and presented (data.presentation_kind/presentation_type) pattern. Two cited events carried no data and gained citation_type: "reference" (each is paired with a link presentation of the same source): the 7.1 event-batch example and tests/valid/event-batch-agent.json, plus the same event in tests/invalid/batch-missing-session-and-ctx-token.json so that fixture still passes the schema and fails at the application layer as its description intends. All other cited fixtures and examples already carried citation_type. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- 8.3: revert 'Presentation name' to 'Display name' for operator.name. The display->presentation rename applies to the presentation event, not ordinary UI terminology; manifest.json already says Display name. - telemetry-session.json: ad_rendered description now says 'rendered', matching the field name and 5.4. - 6.5: restate the excerpt pair on the v1 chars-primary hierarchy - excerpt_chars is the portable primary measurement under 6.4's counting rule, excerpt_tokens the agent-native supplementary one - mirroring 6.4's chars_ingested/tokens_ingested wording so 6.6's 'same pairing' cross-reference holds. - 6 intro: the profiles are no longer in lifecycle order (6.5 Citation precedes 6.6 Reproduction while the lifecycle runs Reproduced then Cited), so the intro no longer claims they are; sections keep their numbers. - 5.7: the optional-signals sentence now names reproduction alongside presentation and engagement - it likewise sits outside the Retrieval/Grounding/Citation ladder as a SHOULD. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…shapes Mutation-verified review findings, all previously undetected: - Build every validator with format_checker so format: uuid / date-time / uri assertions enforce instead of annotate; both runners hard-error at startup if the checker lacks uuid or date-time. Install line becomes pip install "jsonschema[format-nongpl]". - Require _expected_error on every invalid fixture: a substring that must appear in the actual error (first schema error message and JSON pointer, or the application-layer violation text). A fixture that fails for the wrong reason, or carries no pin, now fails the run. - Reconcile APPLICATION_LAYER_VIOLATIONS keys against invalid/: an entry with no matching file fails the run instead of silently dropping the expectation. - Apply privacy field gating (5.5) to turns wherever they appear: session documents, event batches, and standalone event envelopes, not only session events lists. - Add referential integrity checks within a session document (6.6-6.8): content_engaged.presentation_id must match a content_presented event id, and citation_id on content_presented/content_reproduced must match a content_cited event id. Standalone envelopes and batch members are exempt; the corroborating click-out flow is out of scope here. - check_examples.py: validate complete bare event objects (type + timestamp) against the TelemetryEvent definition instead of skipping them; 3 of the 7 skipped fragments are now validated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New invalid fixtures, each pinned by _expected_error:
- presented-missing-id, presented-missing-output-id, cited-missing-id:
the required event and output identifiers of sections 6.5 and 6.7.
- cited-missing-citation-type: the schema now requires
data.citation_type on content_cited.
- reproduction-type-invalid: reproduction_type 'paraphrase' - section
6.6 says paraphrase is not reproduction.
- presentation-kind-invalid: presentation_kind is a closed two-value
enum.
- grounded-negative-chars-ingested, reproduced-negative-chars: count
fields carry minimum 0.
- malformed-parent-session-id: format: uuid, caught only now that the
runners enforce format assertions.
- privacy-violation-{response-text,query-intent,topics,response-type,
response-mode,model-id}-at-minimal: one fixture per remaining field
forbidden at minimal privacy (section 5.5).
- presented/retrieved/engaged-missing-identifier: the section 5.7.5
identifier rule, previously only tested via content_grounded.
- engaged-presentation-id-unmatched, presented-citation-id-unmatched:
the new intra-session referential integrity checks (sections 6.7,
6.8).
New valid fixture:
- session-reproduction-no-grounding: the fifth funnel departure of
section 4.3, reproduction of memorised content with no grounding
event.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
mutation_smoke.py copies the schemas and tests/ into a temp directory, applies each known suite-weakening mutation - dropping the format checker, gutting the withdrawn-ip-hash fixture, shrinking CONTENT_EVENT_TYPES and PRIVACY_FORBIDDEN_FIELDS, pointing an engagement at an all-zeros presentation_id - and confirms validate.py fails under every one. Each of these previously went undetected. The working tree is never modified; a surviving mutation fails the script. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…s branch The rebase onto v1-draft brought in fixtures and an inline example added by PRs #38, #41 and #45, none of which were swept by the v1 identity commit or carry the pinned violation this branch now requires. - CI installed plain `jsonschema`, so the hardened runners' format-checker guard aborted the job. The workflow now matches tests/README.md and installs `jsonschema[format-nongpl]`. - 17 fixtures and the §5.1.3 access_context example still declared schema_version 0.1. Bumped to 1.0; invalid-schema-version.json keeps its deliberately wrong value. - 8 invalid fixtures had no `_expected_error`. Each now pins its own violation, including the two application-layer provenance rules and the ctx_token pattern. Suite: 107/107 fixtures, 13/13 examples, 5/5 mutations caught. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
jalexspringer
force-pushed
the
v1-rc-fixes
branch
from
August 22, 2026 11:28
922ae27 to
4b4946b
Compare
…ick-context gate, migration, suite hardening Prose (SPECIFICATION.md, README, tests/README) - schema_version is major.minor; 5.7.4, 8.7 and 12 state the one-schema-per-minor rule; the 1.0.0-style examples are gone and the const "1.0" stays - source_role is a MUST on content_retrieved everywhere (5.2.2 aligned with 5.7.1) - data.scope and citation_type stated as required in 6, 6.4, 6.5; 5.7.1 says "every content event" - 5.7.5 gains the field-placement, distinct-id, same-content join, one-token-one- presentation, envelope-token and manifest placement rules - 7.4.1 token minimum 16 characters and an unguessability MUST; 7.4.4 resolution gate defined on the click turn's privacy_level (minimal: engagement and lineage only; no turn fields at any level); 7.4.5 destination-owner caveat; 7.4.6 records token lifetime and requester authentication as v1 limits - 9.1 states that privacy_level gates the named turn fields only and that opaque and extension fields must not carry identity or withheld text - 12.1: the "URL-carried presentation_id" and published-v0.1 preserved_in_output paragraphs corrected; migration notes for citation_type, scope, the non-null reference rule, ip_hash, source_role and license_ref semantics added - Section 8 written in v1 tense; manifest id and endpoint are https-only and id sits at the well-known path; tokens_ingested at minimal; share and user-supplied agent_navigate engagements; Annex B.5 multi-owner catalogue (#32) Schemas - content_grounded requires data.scope; content_depth typed on retrievals; ct_ pattern {16,240}; manifest_ref format uri; envelope agent_id nullable like the session field; manifest id/endpoint https patterns Conformance suite - validate.py: source_role, field placement, duplicate ids, same-content joins, shared tokens, envelope ctx_token, root-manifest domains, agent/platform ctx_resolution; check_examples.py runs the application-layer rules too - 66 new invalid and 7 new valid fixtures: every closed enum, every format assertion, every cross-shape rule in all three document shapes, origin and index retrievals, platform manifest, all engagement/citation/presentation types, a destination engagement batch, the v0.1 wire version rejected - every schema pin is pointer-prefixed; session-minimal carries source_role; the one fixture without a description has one - mutation_smoke.py replays 59 mutations (54 new) and runs in CI Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The v1 lifecycle keeps the content_displayed -> content_presented rename and drops the separate reproduction event. Output-side reuse reporting (reproduction_type, reproduced_chars/tokens/hash, the reproduced funnel departures and per-reproduction counting) leaves core; a future version or profile can reintroduce it on implementation evidence. The migration note records that the event existed only on the pre-release draft line. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Discovery/outcome-layer clarification in 1.4 and the consumer vs index-emitter role separation in 7.3 (#12) - Optional manifest identifier_schemes block and co-primary URL / content_id owner resolution with defined conflict and failure behaviour (#17, #31), with fixtures and placement checks - The manifest identifies a domain, not a legal person; party-identity declarations recorded as deferred (#46) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…aveat - Version 1.0 / current-specification framing in the spec header, README and GOVERNANCE; the consultation section becomes a past-tense record and the checkbox list goes away - GOVERNANCE gains the decision process owed before 1.0 and loses the broken request-for-comment anchor - Cross-language citation guidance in 6.5 (#24 fallback) and the bounded-portion caveat restated in 6.6 (#25) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- bot_category becomes purpose: an open enum classifying the access (training, inference, search, advertising), with the vendor signal mappings moved to informative Annex C and a migration note (#7) - Generic evidence reference slot on content events (6.8): scheme, detached ref plus digest, no status escalation; the status vocabulary and trust policy stay with the evidence profile (#8) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Review fixes from the 12 Aug v1-draft review, in three groups.
v1 version identity
The spec header still declared Version 0.1 / Preview while the body made v1 normative claims (§9.1 ip_hash withdrawal, §12.1 event rename), and the schemas kept
/schema/v0.1/$ids withschema_versionconst"0.1"— so the "v0.1" schemas rejected valid v0.1 documents, a conforming v1 emitter had to declare0.1, and v1/v0.1 were indistinguishable on the wire. Now: header is 1.0 (release candidate draft) with the 21 Aug freeze noted, schemas declare/schema/v1/andschema_version "1.0", §5.7.4/§12.1 state the negotiation explicitly, and every fixture and inline example is swept (invalid-schema-version.jsondeliberately keeps a wrong version).Spec/schema corrections
idrow said optional, "generated by server if not provided" — contradicting the schema (required on reproduced/cited/presented) and incompatible withcitation_id/presentation_idreferences. Now conditional and emitter-assigned.data.citation_typeis schema-enforced; the schema didn't require it. Thecontent_citedconditional now requiresdata.citation_type, mirroring reproduced/presented.ad_rendereddescription aligned to "rendered"; §6.5 updated to the chars-primary/tokens-supplementary hierarchy that §6.6 already cites; §6 intro no longer claims lifecycle ordering; §5.7 overview now names reproduction among the optional lifecycle signals.Conformance-suite hardening (review findings were mutation-verified)
"not-a-uuid"session ids validated) plus a startup guard; install line ispip install "jsonschema[format-nongpl]"._expected_error; a fixture failing for the wrong reason fails the run (previously, gutting a fixture left the suite green). OrphanedAPPLICATION_LAYER_VIOLATIONSkeys fail the run.content_engaged.presentation_idmust match acontent_presented.id,citation_idmust match acontent_cited.id.tests/mutation_smoke.pyreplays the review's five suite-weakening mutations; all now caught.Rebased onto
v1-draft(22 Aug)Rebased past #38, #41 and #45. Two conflicts, both resolved in favour of
v1-draft's content: thesession-cached-groundingfixture description, and thevalidate.pyrule-list comment (both rules kept, referential integrity renumbered to 6).A final commit extends this branch's two invariants to everything that landed after it was opened:
jsonschema, so the new format-checker guard aborted the job (ERROR: format checker cannot enforce date-time)..github/workflows/ci.ymlnow installsjsonschema[format-nongpl], matching whattests/README.mdalready documented.access_contextexample still declaredschema_version0.1; bumped to1.0.invalid-schema-version.jsonkeeps its deliberately wrong value._expected_errorand so failed this branch's own pinning rule. Each now pins its own violation — including the two application-layer provenance rules and thectx_tokenpattern.The click-context work this PR originally deferred to
v1-click-contexthas since merged; §7.1 andpresentation_idconditionality are no longer open here. The §6.7/§6.8 document scope ambiguity forcitation_id/presentation_idreferences still stands: the suite enforces them within session documents only, and mixed per-session delivery remains undefined in the spec.Remaining editorial follow-up (deliberately untouched): prose that discusses v0.1 as a historical version (§8.4 signing "informational in v0.1", §12.1 migration narrative, README "Current spec version: 0.1") needs an editorial pass deciding which policies carry into v1.
🤖 Generated with Claude Code