evaluation(handoff): benchmark rollover handoffs against compaction-style summaries #1791

Feature description

Benchmark Rollover Handoff against transcript compaction and informal summaries on long tasks.

Codex's context-management work is motivated by long-session degradation and token-budget pressure. PowerContext should evaluate whether evidence-backed Rollover Handoff improves continuation quality compared with common alternatives.

Codex prior art to review:

• [codex] Add new context window tool openai/codex#27488 - [codex] Add new context window tool
• Add history and notes tools for token-budget sessions openai/codex#39827 - Add history and notes tools for token-budget sessions
• Inject history notes hints into context windows openai/codex#40539 - Inject history notes hints into context windows
• Allow models to enable token budgeting by default openai/codex#41803 - Allow models to enable token budgeting by default

Problem and proposed solution

Rollover Handoff is promising, but PowerContext should validate the claim with product-level evaluation rather than assuming it works.

Proposed evaluation:

• Choose long coding/documentation tasks that require multiple turns, file reads, and validation.
• Compare at least three continuation methods:
  • full or compacted transcript continuation;
  • informal human/model summary;
  • PowerContext Rollover Handoff plus bounded continuation context.
• Measure task success, time to recover state, incorrect assumptions, missing evidence, injected bytes, and user correction burden.
• Include cases with unavailable evidence and known omissions.
• Run across at least two host integrations if available.

Alternatives considered

• Ship rollover based only on qualitative intuition. That risks adding complexity without proving better outcomes.
• Evaluate only retrieval precision/recall. This misses work-continuity behavior such as next-action correctness and stale-plan avoidance.

Additional context

Related PowerContext work:

• PR RFC: Handoff session rollover #1783: Rollover Handoff RFC and initial implementation.
• feat(evaluation): benchmark PowerContext across agent harnesses and models #1705: benchmark PowerContext across agent harnesses and models.
• Tracking Issue: build a governed evaluation and improvement loop #1422: governed evaluation and improvement loop.
• Tracking Issue: expand benchmark coverage #1263: expand benchmark coverage.

Acceptance criteria:

• Define benchmark tasks, baselines, and scoring rubric.
• Report injected bytes separately from task success.
• Include failure analysis for cases where Rollover Handoff underperforms.
• Use findings to update Handoff quality requirements or continuation assembly.