Skip to content

What an agent received lives on the agent - #121

Draft
lloyal-research wants to merge 1 commit into
feat/documentsfrom
feat/documents-pr-feedback
Draft

What an agent received lives on the agent#121
lloyal-research wants to merge 1 commit into
feat/documentsfrom
feat/documents-pr-feedback

Conversation

@lloyal-research

Copy link
Copy Markdown
Contributor

Stacked on #120. Base is feat/documents, so this reads as its own diff.

The review of #120 found view_page recording a page as viewed before the pool had admitted it, so a settle rejection followed by a retry left the model blind. Tracing that finding showed it was one instance of a pattern rather than a bug in one tool: three tools each kept a private per-agent map, keyed by ToolContext.agentId and written when the tool returned rather than when the result landed.

agentId and branch were duplicates. CallingAgent has carried the live agent, its branch and its history since 2026-03-27; the id predates it by three weeks and was never retired. An id sitting on the port is what invited three separate authors to keep private per-agent state.

What changed

  • Agent.attendedResults(tool) — the calls of one tool whose RESULTS this agent attends over, its own and its callers'. One walk of the booked history, no new state. landed stays the word for the event a prefill was admitted; attends names the KV state, which is the question a tool is actually asking.
  • ToolHistoryEntry.outcome, taken from the prefill item's own kind. This was a hole underneath the finding: a settle nudge's replacement item carries the ORIGINAL call's name and args, so toolHistory claimed a nudged call had landed, and any guard reading history alone would have inherited the same blindness the private maps had.
  • ToolContext loses agentId and branch. The port carries the values of the call; who is calling is ambient.
  • The three tools drop their maps — corpus read_file, documents read_document and view_page.
  • corpus search leaves the global score floor for top-K within a token budget, and always scores in explore mode. Both were measured on the pharmacology thread: the floor at zero returned nothing precisely when the agent most needed the best available passages, and exploit's min() against the original question took the answer-bearing passage from +5.5 to −5.0.
  • The delegate echo threshold is named and documented in LOGITS, the unit scoreSimilarityBatch actually returns. It was written as a 0–1 similarity and compared against logits, so a guard meant to catch paraphrases would have rejected every legitimate sub-question. Default 7, with the measurement in the comment.
  • corpus 2.0.3.

Tests

They exercise the real thing rather than a restatement of the rule. The settle-reject-then-retry case runs through agentPool: the page defers over headroom, the nudge lands, the retry lands the page, and the nudged call is absent from what the agent attends over. The ability tests construct real Agent instances and book history through recordToolResult, so the forked-child case proves Agent.walkAncestors rather than a walk the test wrote itself.

Deliberately not in this PR

shouldProceed and the entailment floor stay. Nested pools still copy their own available list, so a child's admissions never reach the run that spawned it; that gap is on record as an it.fails test rather than left unsaid, and scope contexts close it in the next agents major.

Gates

Full suite 160 files, 1146 passed, 1 expected fail (the nested-pool gap above), 2 skipped. tsc -p tsconfig.test.json clean. verify:packed and verify:oci verified. Casework 45/45, web typecheck 0.

Three tools kept private per-agent maps of what they had returned, keyed by
ToolContext.agentId and written when the tool returned rather than when the
result landed. A settle rejection followed by a retry therefore left the model
blind: the tool had already recorded the page as delivered.

agentId and branch were duplicates. CallingAgent has carried the live agent,
its branch and its history since 2026-03-27; the id predates it by three weeks
and was never retired. An id on the port is what invited the private state.

- Agent.attendedResults(tool) — the calls of one tool whose RESULTS this agent
  attends over, its own and its callers'. One walk of the booked history.
  "landed" stays the word for the event; "attends" names the KV state, which is
  the question a tool is actually asking.
- ToolHistoryEntry.outcome, taken from the prefill item's own kind. A settle
  nudge's replacement item carries the ORIGINAL call's name and args, so history
  claimed a nudged call had landed and any guard reading it inherited that.
- ToolContext loses agentId and branch.
- corpus read_file, documents read_document and view_page drop their maps.
- corpus search leaves the global score floor for top-K within a token budget,
  and always scores in explore mode: exploit's min() against the original
  question was measured taking the answer-bearing passage from +5.5 to -5.0.
- The delegate echo threshold is named and documented in LOGITS, the unit
  scoreSimilarityBatch returns. Written as a 0-1 similarity and compared against
  logits, it would have rejected every legitimate sub-question.
- corpus 2.0.3.

The tests exercise the real thing: the settle-reject retry runs through
agentPool, and the ability tests construct Agent instances and book history
through recordToolResult instead of stubbing the lineage walk.

Nested pools still copy their own available list, so a child's admissions never
reach the run that spawned it. That gap is on record as an it.fails test rather
than left unsaid; scope contexts close it in the next agents major.

@lloyal-research lloyal-research left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the stacked delta against 5d7e97b, including the base fixes responding to #120. I would fix the two inline findings before merging.

The simplification has merit: admission owns bookkeeping, three speculative per-agent maps disappear, and nudge/tool-result discrimination is explicit. The remaining problem is semantic: attendedResults() turns admitted call arguments plus caller genealogy into a claim about evidence physically present in KV. Those are different facts, and both mismatches reproduce below. The tests using real Agent instances with cast branches verify the history walk; they do not establish its correspondence to actual branch inheritance.

I would retain admission-owned bookkeeping, but derive deduplication from the successfully delivered resource identity/ranges (the landed result), and inherit that evidence only along the actual forked prefix. Reuse the existing admitted records where practical; another independently maintained tool cache would recreate the original problem. If that contract is not ready, restricting deduplication to evidence provably present is preferable to withholding unseen content.

Two non-blocking design observations: removing identity from ToolContext eliminates duplicate access paths, but CallingAgent still exposes the mutable Agent, so this is API consolidation rather than an ownership fence. Also, always-explore corpus retrieval and the echo threshold change are behavior/calibration changes, not prerequisites for the history refactor. Their synthetic-score tests verify the chosen rule, not its generality across research tasks; keep that distinction explicit.

Validation: workspace build and test typecheck pass. 228 existing tests passed across 16 focused files, with one separately declared expected failure for nested-pool asset propagation; additional filters skipped unrelated cases. Two added local regression probes fail with the withheld-content behavior described inline. I did not rerun real-model calibration or packaging gates. A remaining PDF inspection-limit bypass in the base is posted on #120: #120 (review)

const prev = agent
? mergeRanges(
agent.attendedResults(this.name)
.filter((a) => index.find(String(a.document ?? ''))?.id === doc.id)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Do not turn a landed error response into an already-read span

outcome === 'toolResult' means the response prefilled, not that the requested document text was returned. This code re-resolves the original arguments against today's available documents and treats them as delivered content. I reproduced the failure through the real pool with the mock native context: read_document(id, page: 2) returns NO_DOCUMENTS; another tool admits that document root; the agent repeats the same read. The second response is Lines 7-12 already read, with no content, even though the first response contained only the error. Both entries are present in attendedResults('read_document').

Book/derive what the successful response actually delivered (resolved document identity and returned ranges), rather than inferring it from requested arguments and the prefill kind. Error/note responses must not acquire evidence coverage. Add the read-before-admission → admit-root → retry sequence as a pool-level regression; the existing nudge test does not cover an error that legitimately lands on the tool-result rail.

* against its caller would withhold content the child never saw.
*/
attendedResults(tool: string): Record<string, unknown>[] {
return this.walkAncestors((a) => a.toolHistory)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Derive evidence inheritance from the forked prefix, not the caller chain

setupAgent(parentBranch, ...) assigns Agent.parent from ambient CallingAgent independently of parentBranch, and the pool/task API permits forking from the spine or another branch. Consequently this walk can return results that are absent from the child's KV. I reproduced it using actual Branch objects and setupAgent: caller A has a booked read of lines 1–20 on A's branch; with A as CallingAgent, create a child from the spine. The child's branch.parent is the spine while Agent.parent is A. A child read of lines 1–25 returns only 21–25, withholding twenty unseen lines.

The doc comment acknowledges this assumption, but the new deduplication behavior relies on it without enforcement. DelegateTool's caller-branch path is valid; the general Agent API cannot infer all attention ancestry from it. Track/inherit evidence for the actual forked prefix (including its fork boundary), enforce the required relationship, or omit ancestor subtraction where inheritance is unproven. Add a real-branch test for both fork origins; manually setting Agent.parent with cast branches cannot validate this invariant.

@lloyal-research
lloyal-research marked this pull request as draft September 7, 2026 10:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant