Skip to content

Digest stale tool results before the next model turn #383

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
tylerreckart wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Digest stale tool results before the next model turn #383

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,15 @@ loosely while pre-1.0 (breaking changes can land on minor bumps).

## [Unreleased]

- **Stale tool results stay out of the next prompt.** The model view
keeps the newest tool batch in full and replaces older bulky envelopes
with the writ line, an `ok` / `ERR:` status, and any presence note or
loop warning. Short results (under 4 KB) stay. Stored history and
replay are unchanged. Compaction also recognizes live `[END TOOL RESULTS]`
envelopes when choosing the kept tail, and the summarize call uses the
same digest so it does not re-send every file body. Disable with
`ARBITER_TOOL_ELIDE_DISABLED=1`. See [Sessions](docs/tui/sessions.md).

## [0.13.11] — 2026-09-21

- **Fleet dashboard pane (Phase C1).** The TUI consumes fleet SSE (`stream_id` +
Expand Down
1 change: 1 addition & 0 deletions docs/cli/environment.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ See [`docs/cli/connect.md`](connect.md). Local provider keys (`OPENROUTER_API_KE
| `ARBITER_AUTOSAVE_INTERVAL_SEC` | Periodic dirty flush for conversation files. `0` disables the timer (post-turn and mid-turn `save_async` still run). | `30` |
| `ARBITER_COMPACT_THRESHOLD` | Integer percent (1–100) of the model context window that triggers auto-compaction. | `75` |
| `ARBITER_COMPACT_DISABLED` | When set to a non-empty, non-`0` value, disables automatic compaction. `/compact` still works. | unset |
| `ARBITER_TOOL_ELIDE_DISABLED` | When set to a non-empty, non-`0` value, the model view keeps every earlier tool-result body. By default only the newest batch is sent in full; older bulky bodies become a writ/status digest. | unset |

See [`docs/tui/sessions.md`](../tui/sessions.md).

Expand Down
14 changes: 13 additions & 1 deletion docs/tui/sessions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,19 @@ to force it. `/reset` clears both history and compaction state for that agent.

The summary call uses `constitution.advisor.model` when set, otherwise the
executor model. Failures are fail-open: the turn proceeds with the uncompacted
view and a warning is logged.
view and a warning is logged. The summarize prompt itself drops bulky tool
bodies (same digest as below) so that call is not a second copy of every file
the agent already read.

Before each model request, tool-result envelopes older than the newest batch
are replaced in that view with a short digest: the writ line (`[/read path]`,
`/exec`, …), `ok, N bytes omitted` or the first `ERR:` line, plus any
presence note or loop warning that was attached. The newest tool batch is
sent in full, so the model still has the evidence for the turn it is about
to take. File bodies under 4 KB stay as well — a short search hit is the
signal, not the cost. Image bytes on an older tool turn are dropped with a
one-line stand-in. Replay and session JSON keep the original envelopes.
Set `ARBITER_TOOL_ELIDE_DISABLED=1` to send every body.

## Sessions vs the structured memory graph

Expand Down
18 changes: 18 additions & 0 deletions include/context_compaction.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,24 @@ CompactionConfig compaction_config_from_env();

[[nodiscard]] bool is_tool_results_message(const Message& m);

// Newest tool-result messages left verbatim in the model view. Older bulky
// envelopes become a writ/status digest. The live view keeps one so the
// model still sees the batch it is about to act on.
inline constexpr int kKeepRecentToolResults = 1;

// Short search hits and errors stay whole. File reads, maps, and long
// command output are what blow the prompt up on the next turns.
inline constexpr size_t kToolElideMinBytes = 4096;

// ARBITER_TOOL_ELIDE_DISABLED set to anything other than empty or `0`
// keeps every tool body in the model view and in the compaction prompt.
[[nodiscard]] bool tool_result_elision_enabled();

// Digest tool-result messages except the last `keep_recent` ones.
// Non-tool messages are left alone. Image parts on a digested message
// are dropped. Does not read the env flag — callers decide.
void elide_stale_tool_results(std::vector<Message>& messages, int keep_recent);

// Cut index so histories_[cut …] is the recent window (keep last N), snapped
// so the kept tail starts on a real user turn when possible.
[[nodiscard]] size_t compute_cut_index(const std::vector<Message>& history,
Expand Down
Loading
Loading