Skip to content

docs: research plan — Artifact-Summary Handoff (context-bounded state transfer in branching LLM code agents) - #31

Open
kotama7 wants to merge 5 commits into
mainfrom
plan-artifact-summary-handoff
Open

kotama7 wants to merge 5 commits into
mainfrom
plan-artifact-summary-handoff

Conversation

@kotama7

@kotama7 kotama7 commented Jun 12, 2026

Copy link
Copy Markdown
Owner

概要

分岐型LLMコード生成エージェントにおけるノード間状態受け渡し(handoff)を研究対象として定式化し、ARI を reference implementation として ablation 実験を行う研究計画 ari-core/PLAN_artifact_summary_handoff.md を追加します。

提案手法 Artifact-Summary Handoff は、親ノードの全対話履歴・全ログを子へ渡さず、code artifact state(work_dir 経由のファイル継承)operational summary state(実験状態の構造化要約) のみを渡すことで、探索深度とプロンプト長を切り離します。

計画の骨子

  • 比較 handoff 方式: code_only / summary_only / code_plus_summary(提案) / code_plus_full_log / code_plus_truncated_log / rolling_summary / failure_only_summary
  • summary schema の field ablation(metrics / known_failures / next_steps / delta_vs_parent 等の要素別寄与)
  • 主タスク: CSR SpMV 最適化(行列ファミリ別)、拡張で stencil / FFT / GEMM micro-kernel
  • 主評価: deterministic evaluator による best valid geomean speedup @ N nodes + cost_trace.jsonl ベースのトークン効率指標(LLM Judge は主評価から除外)
  • 投稿戦略: ASE 系(LLM code agent / automated SE の論文として、HPC kernel 論文にしない)

実装監査(計画 §0 に固定)

計画の前提を現行コードに対して監査済み。一致点(work_dir 継承 + _OUTPUT_BLACKLIST、node_report.json の全フィールド、cost_trace.jsonl の per-call prompt_tokens など)に加え、計画が依拠してはいけない7つの食い違いを §0.2 に明記しています:

  1. linear refinement は未実装(BFTS のみ。frontier_score は fallback スコアリング)
  2. search_trace.jsonl は存在しない
  3. 親 summary の注入面は planner(expand prompt)と memory 経由の2面で、子 agent 自身のプロンプトへの注入は新規実装
  4. memory 完全 off スイッチは未実装(tool trace が memory 経由で子に漏れるため code_only 条件に必須)
  5. timeout_per_node は実行時非強制(プロンプト文言のみ)
  6. experiments/ は gitignore 済みのため harness は tracked な置き場所が必要
  7. context overflow の明示的検出は存在しない(指標は計測器を定義して測る)

変更内容

  • ari-core/PLAN_artifact_summary_handoff.md 追加のみ(コード変更なし)
  • 既存 PLAN_*.md 規約に準拠(冒頭に Status と削除条件、evidence-based の file:line 引用)
  • scripts/readme_sync.py --check 通過確認済み

🤖 Generated with Claude Code

kotama7 and others added 2 commits June 12, 2026 17:25
…unded state transfer in branching code agents)

Adds ari-core/PLAN_artifact_summary_handoff.md: a research plan that
formalizes ARI's parent->child node state transfer (work_dir artifact
inheritance + structured node_report summaries) as a controllable
handoff policy, and designs an ablation study (code_only / summary_only
/ code_plus_summary / full_log / truncated_log / rolling / failure-only)
on CSR SpMV optimization with deterministic evaluation.

Section 0 pins a verified audit of the current implementation
(bfts_loop.py work_dir copy + _OUTPUT_BLACKLIST, node_report builder,
_format_parent_report_block planner injection, cost_trace.jsonl token
records) including seven divergences the plan must account for (no
linear search, no search_trace.jsonl, planner-vs-agent injection
surfaces, no memory-off switch, advisory-only node timeout, gitignored
experiments/, no overflow detection).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Update PLAN_artifact_summary_handoff.md to the controlled artifact/history
study (RQ-A scaffold; RQ-B field-level content; RQ-C cross-node transfer;
RQ-D capability x handoff), and fix the §0.2 premises that block a clean
comparison: B1 memory third-channel, B2 deterministic evaluator drives
selection, B3 exogenize the LLM-self-determined metric contract, plus
sterile-gate asymmetry, dead timeout, full_log overflow, seed/digest.

Add MASTER_PLAN_handoff_impl.md (dependency DAG, critical path, MVP cut)
and a per-subtask Plan.md under each affected directory. Each Plan.md
records its inter-plan dependencies and deletion requirements. Per-dir
README Contents indexes synced for the new files.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kotama7
kotama7 force-pushed the plan-artifact-summary-handoff branch from 7f5d571 to f179b7f Compare June 13, 2026 18:17
kotama7 and others added 3 commits June 14, 2026 03:23
Bring the handoff-plan branch up to date with main (claim_gate package,
metric_contract, routing, web_provenance, docs tooling). README ## Contents
conflicts resolved by regenerating with scripts/readme_sync.py so main's new
modules and the new per-subtask Plan.md files are both listed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Close the two coverage gaps in the per-subtask implementation plans:
- ari-core/ari/Plan.md: root-file edits (core.py evaluator dispatch,
  cost_tracker.py provenance) that had only been cross-referenced.
- ari-skill-memory/Plan.md: memory backend side of RQ-C (push/pull
  surface, ancestor-scoped topology, retrieved-text logging; Phase C).
Both carry inter-plan dependencies and deletion requirements. MASTER
subtask index + completion log updated; ari/ README Contents synced.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adopt the more-detailed-implementation options on delegation: task=SpMM,
implementation branch=bfts_compare. Freeze valid predicate, epsilon model
(C=8), _scientific_score normalization (TARGET=4.0x), N=10, failure codebook,
primary contrast, H-B/H-C/H-D directions, qwen3 8/14/32B gradient, run-level
stats (n>=10, cluster bootstrap, TOST margin=log(1.05)), timing W=3/R=10.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant