✨ Add continuity-eval telemetry and ADR-I-0019 harness placement#59
Merged
Conversation
Additive-only public telemetry for the continuity evaluation harness: FanoutUtilizationTrace (per-relation/object retained and omitted-by-fanout counts around existing graph-expansion pruning) and RationaleCategory (stable enum populated from existing structural decisions). All new fields are optional and serde-defaulted; no facade, ranking, or default changes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The skip-if-unavailable predicate now matches the error shape the Qdrant client actually emits, so integration tests skip cleanly when the local service is down instead of failing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…repo Resolves the ADR-I-0018 "no unambiguous home" revisit trigger: the continuity evaluation harness lives in the private companion CharacterMemoryEvals repository as a dataset crate consuming this crate strictly through the public API. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plan for the v0.1.4 continuity evaluation harness milestone (harness hosted in the private evals repo; Tasks 1-3 complete with findings and validation evidence logged). Rules: sibling-repo reference and public-reader wording rules in common.md; diff-scoped ADR-I-0018 dependency-direction audit guidance in orchestrator.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…evision Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR lays groundwork for the v0.1.4 continuity evaluation milestone by (1) recording the harness-placement decision via ADR-I-0019 and (2) adding additive retrieval telemetry needed by continuity metrics (fanout utilization + stable rationale categories), alongside a small integration-test skip predicate fix and planning/rules bookkeeping updates.
Changes:
- Add fanout utilization telemetry (
FanoutUtilizationTrace) to retrieval traces and plumb utilization collection through bounded graph expansion. - Add stable rationale category taxonomy (
RationaleCategory) and populate it across section assignments and stale-candidate omissions. - Document harness placement via ADR-I-0019 and refresh coding-agent rules/plan/lessons; adjust Qdrant “skip-if-unavailable” detection.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/support/base.rs | Updates Qdrant-unavailable predicate to match observed error shapes. |
| src/usecases/retrieve.rs | Emits new telemetry in retrieval traces and adds rationale category population. |
| src/ports/graph_authority.rs | Extends GraphExpansion to carry fanout utilization details from policy to usecases. |
| src/policy/graph_expansion.rs | Collects per-(relation, object_type) fanout utilization during bounded expansion. |
| src/lib.rs | Re-exports new public telemetry types from the crate root. |
| src/api/types/retrieval.rs | Adds new telemetry types/fields and serde-defaulting tests for legacy payloads. |
| src/api/types.rs | Re-exports new retrieval telemetry types through crate::api::types. |
| docs/decisions/README.md | Adds ADR-I-0019 to the decisions index. |
| docs/decisions/implementation/ADR-I-0019-continuity-eval-harness-placement.md | New ADR documenting private evals-repo harness placement and boundaries. |
| docs/coding-agent/rules/orchestrator.md | Updates orchestrator rules (including diff-scoped dependency-direction audit guidance). |
| docs/coding-agent/rules/common.md | Adds repo wording rules for the private evals repo + no-hard-wrap prose rule. |
| docs/coding-agent/plans/active/v0-1-4-continuity-evaluation-harness-plan.md | Adds the v0.1.4 continuity harness execution plan and progress/findings log. |
| docs/coding-agent/lessons.md | Records a new lesson about avoiding hard-wrapped prose in committed docs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…teria Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…s candidate Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Groundwork for the v0.1.4 continuity evaluation harness milestone. The harness itself lives in the private companion evals repository (a development aid, not core library functionality); this PR delivers everything the milestone needs from this crate: the placement decision record, the additive telemetry the continuity metrics consume, and the plan/rules bookkeeping.
Description
FanoutUtilizationTrace(per-relation/object retained and omitted-by-fanout counts wrapped around the existing graph-expansion pruning — the pruned link set is byte-identical, produced by delegation to the unmodifiedapply_fanout_limits) andRationaleCategory(stable enum populated from existing structural decisions). All new fields optional and serde-defaulted; legacy payloads covered by a deserialization test; no facade, ranking, or default changes.Reviewer evidence (independent): additive-only diff review + ADR-I-0018 dependency-direction audit clean; fmt/check/clippy
-D warnings/test all green (339 lib + 25 integration tests).Checklist
cargo checkcargo test --no-runcargo clippy --all-targets -- -D warningscargo fmt --checkcargo test --verboseAdditional Notes (optional)
The three MINOR follow-ups from review are logged in the plan's cleanup backlog (trace-demand gating of fanout utilization, a one-time
crate::domainimport sweep in ports/policy, a doc comment onFanoutUtilizationTrace::selected_cap).🤖 Generated with Claude Code