Skip to content

deepagent: runtime_loop drops final assistant message when agent terminates without calling submit_tool #280

@fede-kamel

Description

@fede-kamel

TL;DR

create_deepagent-built agents return AgentResult.text='' when the agent terminates via MaxIterations (or any non-submit-tool path) without explicitly calling the submit_tool, even when the model emitted real completion tokens. The model's output is in metrics.completion_tokens but never lands on AgentResult.text / AgentResult.last_assistant_message.

This surfaced as part of #278 (the max_tokens rename) — discovered during live testing against OCI Gemini 2.5 Flash. The two bugs are separate but contribute to the same operator-facing "empty output" UX.

Repro

Live, against OCI Gemini 2.5 Flash via API_FREE_TIER. The xfailed live test in #278's PR captures the exact shape:

agent = create_deepagent(
    model="oci:google.gemini-2.5-flash",
    tools=[some_tool],
    system_prompt="...long system prompt...",
    reflexion=False,
    grounding=False,
    # NO total_token_budget — should be bounded only by MaxIterations
    max_iterations=6,
    max_output_tokens=2048,
)
result = agent.run_sync("...prompt...")

# OBSERVED:
result.text                      == ""
result.last_assistant_message    == ""
result.tool_executions           == []
result.metrics.iterations        == 1
result.metrics.completion_tokens == 5    # ← MODEL WROTE THIS, lost
result.metrics.total_tokens      == 2905

Reference test asserting the bug shape:
tests/integration/test_deepagent_token_budget_live.py::test_long_prompt_with_default_none_produces_real_output
(currently @pytest.mark.xfail(strict=False) pointing here).

Why

Looking at src/locus/agent/runtime_loop.py the TerminateEvent.final_message is sourced from _last_assistant_content. On a path where:

  1. The model returns content without tool calls
  2. The next iteration's termination check fires (e.g., MaxIterations)
  3. The agent yields TerminateEvent with final_message=_last_assistant_content

…the assistant content might not have been flushed into _last_assistant_content yet, OR the final_message doesn't end up on AgentResult.message. The submit_tool exit branch flushes correctly (because the tool's result is captured); the MaxIterations branch silently loses the trailing message.

Need to trace exactly which path zeroes out the content. Candidates:

  • _last_assistant_content not updated after the FINAL model response that triggered MaxIterations
  • TerminateEvent → AgentResult conversion writing message="" when no submit_tool was called
  • "Summary request" injection path (line ~270 in runtime_loop) re-asking the model and discarding the first message

Impact

  • Every create_deepagent user who doesn't enforce a submit_tool contract loses the agent's final response when MaxIterations triggers
  • Particularly bad for long-form research (the deepagent's primary use case) where the model writes a 25K-char narrative on the LAST iteration without calling submit_tool, then hits the iteration limit, and the narrative is silently dropped
  • Was the second-highest contributor (after deepagent: max_tokens parameter is the run-level budget but reads like the per-completion cap — empty-output bug #278) to the "empty output" UX in observai/optic AFS DeepAgent integration

Acceptance criteria

  • AgentResult.text returns the model's final-iteration content even when termination wasn't via submit_tool
  • Repro from the xfailed test above passes (text non-empty when iterations > 0 and completion_tokens > 0)
  • Unit test pinning the contract: "an agent that terminates via MaxIterations with a model response must preserve that response in result.text"
  • No regression to existing submit_tool-exit happy path

Refs

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions