Skip to content

FileTree asks the reflector for a rationale and then drops it: carry Diff.rationale into the ledger commit and the rejected-evidence note #247

Closed

Description

@Birfy

背景 / Motivation

EDIT_PROTOCOL in agentdescent/treestrategy.py asks every reflector for a rationale:

{"rationale": "<one sentence: what was wrong and why this fixes it>", "edits": [...]}

but parse_edits only returns {path: content}, so the rationale is paid for in tokens on every proposal and then thrown away. Downstream, nothing can recover why a change was made:

  • Diff (agentdescent/evolvable.py) has no field for it.
  • The ledger commit for an accepted diff is just merge {diff.diff_id} -> {artifact_id} (aggregator.py), so git log on the ledger — our lineage — has ids and versions but no reasons.
  • ReplayAwareProposal (agentdescent/replay.py, from feat(replay): settled-evidence pool gets a consumer - ReplaySampler + ProposalContext.rejected #193) shows the inner policy "Recently discarded for this task", but its default note can only render touched paths, base version and measured delta. The reflector learns that an edit to SKILL.md was rejected, not what idea was rejected, so "do not re-propose what was just rejected" is weaker than it could be.

Industry signal points the same way: GEPA v0.1.3 now logs reflection prompts/outputs to a proposals table "for run archaeology" and exposes them in on_proposal_end; WikiSkill's ablation finds that a persistent record of past proposals and their accept/reject outcomes (skill-impact.md, read by the proposer so rejected interventions are not proposed again) is critical to its gains.

参考链接 / References

建议实现路径 / Proposed approach

  1. Diff gains rationale: str = "" (default keeps every existing constructor and pickle/replay path working). It must not enter diff_id / the fingerprint, so dedup and eval caching are unchanged.
  2. treestrategy.py: add parse_proposal(proposal) -> tuple[dict, str] returning edits plus the rationale (stripped, capped at e.g. 500 chars, non-string → ""); keep parse_edits as a thin wrapper for backward compatibility. FileTree.to_diff sets Diff.rationale.
  3. aggregator.py: when diff.rationale is non-empty, append it to the commit message body (merge … -> …\n\n<rationale>), sanitised to a single paragraph. Fused diffs (built in aggregator.py keyed union and fusion.py) concatenate their parents' rationales (deduped, capped).
  4. replay.py::_default_note renders card.diff.rationale when present.
  5. Optional, same PR if small: record rationale in MergeReport entries so a run's report can be grepped by reason.

验收标准 / Acceptance criteria

  • parse_proposal returns the rationale for the <EDITS>, fenced-JSON and bare-object wrappers; parse_edits output is byte-for-byte unchanged (existing tests/test_tree_strategy.py passes untouched).
  • Two proposals with identical edits but different rationales get the same diff_id.
  • An accepted FileTree diff's ledger commit message contains its rationale (test reads it back via git log -1 --format=%B on a scratch ledger).
  • ReplayAwareProposal with the default renderer includes the rejected card's rationale in the note.
  • Diffs without a rationale (all non-FileTree strategies) produce exactly today's commit message.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions