Research context strategies and success measures against the Darrow delivery baseline

[BjRo]

Outcome

Research whether smart-zone context strategies improve Darrow delivery compared with the current Adaptive Delivery baseline, and establish how to evaluate that claim before changing delivery behavior.

The motivating idea is to keep working context focused through coherent implementation slices, a compact project map, explicit checkpoints, and selective fresh contexts. Treat any fixed token threshold as a hypothesis to evaluate, not a universal quality boundary.

Current Darrow already launches a delivery owner with fresh context (fork_turns: none) and a self-contained contract. That owner retains responsibility for the run and may delegate bounded subtasks. Replacing the owner after every slice would change the current ownership invariant. The research should distinguish improvements compatible with this design from proposals requiring an architectural change.

Done criteria

• Document which parts of the smart-zone/Ralph approach remain useful for Darrow, their tradeoffs, and the evidence or uncertainty behind the recommendation.
• Define a reproducible comparison against the current Darrow baseline, including workload selection, controlled conditions, outcome measures, and the rule for judging success.
• State whether the evidence supports changing the strategy, retaining the baseline, or conducting further experiments; distinguish proposed benefits from measured results.

This issue covers research and evaluation design; it does not commit to mandatory resets, a fixed token cutoff, or a new orchestration runtime.

Open questions

• How do we evaluate success of a changed strategy against the Darrow baseline? Which baseline revision and configuration should be pinned, and how do we keep fixtures, prompts, acceptance checks, harness, effective model/effort, and resource limits comparable?
• Which representative tasks expose context-related failures, while also revealing reset overhead or lost continuity on smaller and tightly coupled tasks?
• Which outcomes determine success: acceptance rate, escaped defects, missed constraints, rework, human interruptions, tokens/cost, or elapsed time? What improvement or non-regression thresholds and repeated-trial evidence are sufficient?
• How can comparisons separate the effects of slicing, checkpoints, context isolation, and resets from changes in model routing or additional compute?
• When does context renewal help more than it costs in rediscovery and lost reasoning, and does the answer vary by model, host, or task?

References

• Current ownership and launch contracts: docs/specs/adaptive-delivery.md; plugins/orchestration/darrow-adaptive-delivery/skills/adaptive-delivery/references/codex-launch.md.
• Existing comparative-evaluation guidance: docs/design.md; docs/eval-development.md (matched controls, native/passive owner evaluation, effective-route evidence, and reporting limitations).
• Advanced Context Engineering for Coding Agents.
• Long-Context Isn't the Answer.
• Context Rot research.
• Ralph Wiggum as a software engineer.