Skip to content

Commit e2e5948

Browse files
ADR 0007: status Proposed -> Accepted (5 OQs resolved with defaults)
User confirmed acceptance of the revised ADR (commit c4ce331) on 2026-05-31, including all five recommended-default resolutions for the Open Questions in §7. Status changes: - Header: Proposed (in flight) -> Accepted (2026-05-31, blocking v0.3.0 GA) - README index: Proposed -> Accepted OQ resolutions recorded inline in §7 for traceability: OQ-1 System-prompt change -> new-session path uniformly OQ-2 Determinism strictness -> bit-identical by default; any relaxation must be written into this ADR explicitly (auto-relaxing tests are forbidden) OQ-3 --no-cross-request-reuse flag -> no OQ-4 matched-prefix in usage block -> no for v0.3 OQ-5 idle timeout in single-tenant -> no Implementation work (PR 7-1 ~ 7-7) starts immediately. Per §5 plan, PR 7-1 ships the SinkWindowKVCache parallel token sequence + INV-1 assertions. Co-authored-by: FluffyAIcode <FluffyAIcode@users.noreply.github.com>
1 parent c4ce331 commit e2e5948

2 files changed

Lines changed: 51 additions & 32 deletions

File tree

docs/adr/0007-cross-request-kv-reuse.md

Lines changed: 50 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# ADR 0007 — Cross-request KV cache reuse for long sessions
22

3-
- **Status**: Proposed (in flight, blocking v0.3.0 GA)
3+
- **Status**: Accepted (2026-05-31, blocking v0.3.0 GA)
44
- **Date**: 2026-05-31
55
- **Decision drivers**: ADR 0006 §2.3.b empirical evidence (4-hour Mac M4
66
run); the gap between the "local agent infrastructure" framing and the
@@ -649,36 +649,55 @@ This ADR is considered validated when:
649649
Items 2–5 are GA gates; v0.3.0 cannot promote to GA without all of
650650
them.
651651

652-
## 7. Open questions (require decision before implementation)
653-
654-
These are decisions the ADR author is *not* taking unilaterally and
655-
need explicit approval before PR 7-1 starts:
656-
657-
- **OQ-1**: System-prompt change handling. What if the user's system
658-
prompt rotates mid-session (an agent's "tool" message inserted
659-
ahead of the conversation history)? This shifts every position by
660-
N, breaking the prefix. Default proposal: treat as a full reset.
661-
Alternative: detect the system-prompt shift specifically and
662-
handle. **Recommend default until production data shows otherwise.**
663-
- **OQ-2**: Numerical determinism strictness on Mac M4. Strict
664-
bit-identical or relaxed-to-ULP-equivalent? **Recommend strict
665-
first, relax only if tests fail.**
666-
- **OQ-3**: Should the `--no-cross-request-reuse` server flag exist
667-
for debugging? **Recommend no — keep the protocol surface
668-
minimal. Operators debugging can compare old behavior by checking
669-
out v0.3.0-rc1.**
670-
- **OQ-4**: Should we also expose the matched-prefix length as a
671-
per-response field (in the `usage` block)? **Recommend no for
672-
v0.3** — it leaks server implementation detail. Add only if a
673-
user need surfaces.
674-
- **OQ-5**: Eviction in v0.3 is "implicit replacement" (next
675-
non-matching request resets). Is that adequate, or do we need a
676-
"stale cache idle timeout" even in single-tenant? **Recommend
677-
no idle timeout for v0.3** — single-tenant servers are usually
678-
long-running with one client.
679-
680-
Each OQ has a recommended default. If you accept all defaults, this
681-
ADR is ready to move from `Proposed` to `Accepted` and PR 7-1 starts.
652+
## 7. Open questions — resolved 2026-05-31
653+
654+
All five OQs were resolved with the recommended-default answers
655+
on 2026-05-31. The ADR moved from Proposed to Accepted on the
656+
same date. The original questions and resolutions are recorded
657+
below for traceability:
658+
659+
- **OQ-1 (RESOLVED, default accepted)**: System-prompt change
660+
handling. What if the user's system prompt rotates mid-session
661+
(an agent's "tool" message inserted ahead of the conversation
662+
history)? This shifts every position by N, breaking the prefix.
663+
**Resolution**: take the new-session path uniformly. Do not
664+
attempt to detect "system prompt only changed" specifically.
665+
Revisit only if production data shows the simpler behavior is
666+
costly enough to justify the detection logic.
667+
668+
- **OQ-2 (RESOLVED, default accepted, strengthened)**: Numerical
669+
determinism strictness on Mac M4. **Resolution**: bit-identical
670+
by default. If a future test cycle shows the strict gate is
671+
unreachable on Metal, the relaxation must be **written into this
672+
ADR explicitly** (as an amendment) before the gate is changed.
673+
Tests that auto-relax based on whether the strict path passes
674+
are explicitly forbidden — that pattern is a fallback in
675+
disguise.
676+
677+
- **OQ-3 (RESOLVED, default accepted)**: Should the
678+
`--no-cross-request-reuse` server flag exist for debugging?
679+
**Resolution**: no. Path selection (§2.4) is total; there is
680+
no failure mode that requires a runtime toggle. Operators who
681+
want to compare against pre-cross-request-reuse behavior can
682+
check out the v0.3.0-rc1 tag.
683+
684+
- **OQ-4 (RESOLVED, default accepted)**: Expose matched-prefix
685+
length in the `usage` block of the response? **Resolution**: no
686+
for v0.3. `usage` is a public OpenAI-compatible field; adding
687+
Kakeya-specific keys leaks implementation detail to clients
688+
that don't need it. The same information is available via
689+
Prometheus metrics (§2.10) for operators. Revisit only if a
690+
user-facing need surfaces.
691+
692+
- **OQ-5 (RESOLVED, default accepted)**: Eviction in v0.3 is
693+
"implicit replacement" (next non-matching request takes the
694+
new-session path and overwrites the cache). Adequate, or need a
695+
"stale cache idle timeout" in single-tenant scope?
696+
**Resolution**: no idle timeout for v0.3. Single-tenant servers
697+
are typically long-running with one client; stale-cache-on-idle
698+
has no concrete operational benefit at this scope. v0.4
699+
multi-tenant ADR will define LRU + idle-timeout policy at that
700+
scope.
682701

683702
## 8. References
684703

docs/adr/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ reader what was *not* chosen.
3838
| 0004 | [Alignment training data preparation policy (Nemotron-informed)](0004-alignment-training-data-preparation-policy.md) | Accepted |
3939
| 0005 | Personal layer / personal data store | Planned |
4040
| 0006 | [Project positioning as local agent infrastructure](0006-local-agent-infrastructure-positioning.md) | Accepted |
41-
| 0007 | [Cross-request KV cache reuse for long sessions](0007-cross-request-kv-reuse.md) | Proposed |
41+
| 0007 | [Cross-request KV cache reuse for long sessions](0007-cross-request-kv-reuse.md) | Accepted |
4242

4343
Note: ADR numbering is monotonically increasing; in-flight or
4444
planned numbers (0005) appear in the index so readers can

0 commit comments

Comments
 (0)