fix: only mirror builtin memory writes to external providers after success#667
fix: only mirror builtin memory writes to external providers after success#667badMade wants to merge 5 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request ensures that external memory providers are only notified of memory writes after the built-in memory store has successfully accepted the write. It introduces a helper method _memory_tool_write_succeeded to verify the success of the write operation and updates both sequential and concurrent tool execution paths, supported by new unit tests. Feedback suggests making the helper method more robust by checking if the input is already a dictionary before attempting to parse it as JSON, which prevents potential errors in testing or mocking environments.
| @staticmethod | ||
| def _memory_tool_write_succeeded(function_result: str) -> bool: | ||
| """Return True only when the builtin memory tool accepted a write.""" | ||
| try: | ||
| result = json.loads(function_result) | ||
| except (TypeError, json.JSONDecodeError): | ||
| return False | ||
| return isinstance(result, dict) and result.get("success") is True |
There was a problem hiding this comment.
In testing or mocking environments, function_result might be passed as a pre-parsed dictionary instead of a raw JSON string. To make this helper more robust and prevent unexpected TypeError exceptions during test execution, we can check if function_result is already a dictionary before attempting to parse it with json.loads.
@staticmethod
def _memory_tool_write_succeeded(function_result: Any) -> bool:
"""Return True only when the builtin memory tool accepted a write."""
if isinstance(function_result, dict):
return function_result.get("success") is True
try:
result = json.loads(function_result)
except (TypeError, json.JSONDecodeError):
return False
return isinstance(result, dict) and result.get("success") is TrueThere was a problem hiding this comment.
@gemini-code-assist, is this suggestion still valid? If so, what are the steps and line numbers to make this fix?
There was a problem hiding this comment.
Yes, the suggestion remains valid to ensure robustness in testing environments where the function_result might already be a dictionary. You can apply this change to run_agent.py by replacing the existing _memory_tool_write_succeeded method (lines 4358-4364) with the following implementation:
| @staticmethod | |
| def _memory_tool_write_succeeded(function_result: str) -> bool: | |
| """Return True only when the builtin memory tool accepted a write.""" | |
| try: | |
| result = json.loads(function_result) | |
| except (TypeError, json.JSONDecodeError): | |
| return False | |
| return isinstance(result, dict) and result.get("success") is True | |
| @staticmethod | |
| def _memory_tool_write_succeeded(function_result: Any) -> bool: | |
| """Return True only when the builtin memory tool accepted a write.""" | |
| if isinstance(function_result, dict): | |
| return function_result.get("success") is True | |
| try: | |
| result = json.loads(function_result) | |
| except (TypeError, json.JSONDecodeError): | |
| return False | |
| return isinstance(result, dict) and result.get("success") is True |
🔎 Lint report:
|
| Rule | Count |
|---|---|
invalid-argument-type |
16 |
unsupported-operator |
3 |
unresolved-attribute |
3 |
First entries
run_agent.py:13602: [invalid-argument-type] invalid-argument-type: Argument to function `len` is incorrect: Expected `Sized`, found `(str & ~AlwaysFalsy) | (dict[Unknown, Unknown] & ~AlwaysFalsy) | (Any & ~AlwaysFalsy) | ... omitted 3 union elements`
run_agent.py:3397: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_stale_timeout` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
tests/run_agent/test_provider_attribution_headers.py:156: [unsupported-operator] unsupported-operator: Operator `not in` is not supported between objects of type `Literal["X-OpenRouter-Cache-TTL"]` and `Unknown | str | dict[str, str] | ... omitted 3 union elements`
run_agent.py:2633: [invalid-argument-type] invalid-argument-type: Argument to function `_is_oauth_token` is incorrect: Expected `str`, found `(Unknown & ~AlwaysFalsy) | (str & ~AlwaysFalsy) | (dict[str, str] & ~AlwaysFalsy) | ... omitted 4 union elements`
run_agent.py:5441: [invalid-argument-type] invalid-argument-type: Argument to function `parse_rate_limit_headers` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:13099: [invalid-argument-type] invalid-argument-type: Argument to function `estimate_usage_cost` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:13599: [invalid-argument-type] invalid-argument-type: Argument to function `_is_oauth_token` is incorrect: Expected `str`, found `str | dict[Unknown, Unknown] | Any | ... omitted 3 union elements`
tests/agent/test_codex_cloudflare_headers.py:163: [unresolved-attribute] unresolved-attribute: Attribute `get` is not defined on `str & ~AlwaysFalsy`, `int & ~AlwaysFalsy` in union `(Unknown & ~AlwaysFalsy) | (str & ~AlwaysFalsy) | (dict[str, str] & ~AlwaysFalsy) | ... omitted 3 union elements`
run_agent.py:2377: [invalid-argument-type] invalid-argument-type: Argument to function `query_ollama_num_ctx` is incorrect: Expected `str`, found `(str & ~AlwaysFalsy) | (dict[str, str] & ~AlwaysFalsy) | (Any & ~AlwaysFalsy) | ... omitted 4 union elements`
run_agent.py:9132: [invalid-argument-type] invalid-argument-type: Argument to function `get_transport` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:2630: [invalid-argument-type] invalid-argument-type: Argument to function `build_anthropic_client` is incorrect: Expected `str`, found `(Unknown & ~AlwaysFalsy) | (str & ~AlwaysFalsy) | (dict[str, str] & ~AlwaysFalsy) | ... omitted 4 union elements`
tests/agent/test_codex_cloudflare_headers.py:181: [unsupported-operator] unsupported-operator: Operator `in` is not supported between objects of type `Literal["originator"]` and `(Unknown & ~AlwaysFalsy) | (str & ~AlwaysFalsy) | (dict[str, str] & ~AlwaysFalsy) | ... omitted 3 union elements`
run_agent.py:13136: [invalid-argument-type] invalid-argument-type: Argument to bound method `SessionDB.update_token_counts` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:7335: [invalid-argument-type] invalid-argument-type: Argument to function `build_anthropic_client` is incorrect: Expected `str`, found `str | dict[Unknown, Unknown] | Any | ... omitted 3 union elements`
run_agent.py:9570: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_profile` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
tests/run_agent/test_provider_attribution_headers.py:90: [unresolved-attribute] unresolved-attribute: Attribute `startswith` is not defined on `dict[str, str]` in union `Unknown | str | dict[str, str]`
run_agent.py:13833: [invalid-argument-type] invalid-argument-type: Argument to function `_pool_may_recover_from_rate_limit` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:2690: [invalid-argument-type] invalid-argument-type: Argument to function `get_model_context_length` is incorrect: Expected `str`, found `str | dict[str, str] | Any | ... omitted 3 union elements`
run_agent.py:7164: [invalid-argument-type] invalid-argument-type: Argument to function `_codex_cloudflare_headers` is incorrect: Expected `str`, found `Unknown | str | dict[str, str] | ... omitted 3 union elements`
tests/agent/test_codex_cloudflare_headers.py:163: [unresolved-attribute] unresolved-attribute: Attribute `startswith` is not defined on `dict[str, str]` in union `Unknown | str | dict[str, str]`
tests/run_agent/test_provider_attribution_headers.py:155: [unsupported-operator] unsupported-operator: Operator `not in` is not supported between objects of type `Literal["X-OpenRouter-Cache"]` and `Unknown | str | dict[str, str] | ... omitted 3 union elements`
run_agent.py:13052: [invalid-argument-type] invalid-argument-type: Argument to function `normalize_usage` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
✅ Fixed issues (27):
| Rule | Count |
|---|---|
invalid-argument-type |
19 |
unresolved-attribute |
5 |
unsupported-operator |
3 |
First entries
run_agent.py:2377: [invalid-argument-type] invalid-argument-type: Argument to function `query_ollama_num_ctx` is incorrect: Expected `str`, found `(str & ~AlwaysFalsy) | (dict[str, str] & ~AlwaysFalsy) | (Any & ~AlwaysFalsy) | ... omitted 5 union elements`
run_agent.py:9114: [invalid-argument-type] invalid-argument-type: Argument to function `get_transport` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | ... omitted 3 union elements`
run_agent.py:2633: [invalid-argument-type] invalid-argument-type: Argument to function `_is_oauth_token` is incorrect: Expected `str`, found `(Unknown & ~AlwaysFalsy) | (str & ~AlwaysFalsy) | (dict[str, str] & ~AlwaysFalsy) | ... omitted 5 union elements`
run_agent.py:7763: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_request_timeout` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | ... omitted 3 union elements`
run_agent.py:9979: [unresolved-attribute] unresolved-attribute: Attribute `lower` is not defined on `dict[Unknown | str, Unknown | str | dict[str, str]] & ~AlwaysFalsy`, `int & ~AlwaysFalsy`, `dict[Unknown, Unknown] & ~AlwaysFalsy` in union `(str & ~AlwaysFalsy) | (Unknown & ~AlwaysFalsy) | (dict[Unknown | str, Unknown | str | dict[str, str]] & ~AlwaysFalsy) | ... omitted 4 union elements`
run_agent.py:5423: [invalid-argument-type] invalid-argument-type: Argument to function `parse_rate_limit_headers` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | ... omitted 3 union elements`
run_agent.py:13807: [invalid-argument-type] invalid-argument-type: Argument to function `_pool_may_recover_from_rate_limit` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | ... omitted 3 union elements`
run_agent.py:9552: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_profile` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | ... omitted 3 union elements`
run_agent.py:2690: [invalid-argument-type] invalid-argument-type: Argument to function `get_model_context_length` is incorrect: Expected `str`, found `str | dict[str, str] | Any | ... omitted 4 union elements`
run_agent.py:13026: [invalid-argument-type] invalid-argument-type: Argument to function `normalize_usage` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | ... omitted 3 union elements`
run_agent.py:13110: [invalid-argument-type] invalid-argument-type: Argument to bound method `SessionDB.update_token_counts` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | ... omitted 3 union elements`
run_agent.py:4285: [invalid-argument-type] invalid-argument-type: Argument to `AIAgent.__init__` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | ... omitted 3 union elements`
run_agent.py:14030: [invalid-argument-type] invalid-argument-type: Argument to bound method `ContextCompressor.update_model` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | ... omitted 3 union elements`
tests/run_agent/test_provider_attribution_headers.py:156: [unsupported-operator] unsupported-operator: Operator `not in` is not supported between objects of type `Literal["X-OpenRouter-Cache-TTL"]` and `Unknown | str | dict[str, str] | ... omitted 4 union elements`
tests/agent/test_codex_cloudflare_headers.py:163: [unresolved-attribute] unresolved-attribute: Attribute `get` is not defined on `str & ~AlwaysFalsy`, `int & ~AlwaysFalsy` in union `(Unknown & ~AlwaysFalsy) | (str & ~AlwaysFalsy) | (dict[str, str] & ~AlwaysFalsy) | ... omitted 4 union elements`
run_agent.py:13576: [invalid-argument-type] invalid-argument-type: Argument to function `len` is incorrect: Expected `Sized`, found `(str & ~AlwaysFalsy) | (dict[Unknown, Unknown] & ~AlwaysFalsy) | (Any & ~AlwaysFalsy) | ... omitted 4 union elements`
run_agent.py:7146: [invalid-argument-type] invalid-argument-type: Argument to function `_codex_cloudflare_headers` is incorrect: Expected `str`, found `Unknown | str | dict[str, str] | ... omitted 4 union elements`
run_agent.py:13073: [invalid-argument-type] invalid-argument-type: Argument to function `estimate_usage_cost` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | ... omitted 3 union elements`
run_agent.py:3397: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_stale_timeout` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | ... omitted 3 union elements`
tests/agent/test_codex_cloudflare_headers.py:181: [unsupported-operator] unsupported-operator: Operator `in` is not supported between objects of type `Literal["originator"]` and `(Unknown & ~AlwaysFalsy) | (str & ~AlwaysFalsy) | (dict[str, str] & ~AlwaysFalsy) | ... omitted 4 union elements`
run_agent.py:7317: [invalid-argument-type] invalid-argument-type: Argument to function `build_anthropic_client` is incorrect: Expected `str`, found `str | dict[Unknown, Unknown] | Any | ... omitted 4 union elements`
run_agent.py:2630: [invalid-argument-type] invalid-argument-type: Argument to function `build_anthropic_client` is incorrect: Expected `str`, found `(Unknown & ~AlwaysFalsy) | (str & ~AlwaysFalsy) | (dict[str, str] & ~AlwaysFalsy) | ... omitted 5 union elements`
run_agent.py:11556: [unresolved-attribute] unresolved-attribute: Attribute `strip` is not defined on `dict[Unknown | str, Unknown | str | dict[str, str]] & ~AlwaysFalsy`, `int & ~AlwaysFalsy`, `dict[Unknown, Unknown] & ~AlwaysFalsy` in union `(str & ~AlwaysFalsy) | (Unknown & ~AlwaysFalsy) | (dict[Unknown | str, Unknown | str | dict[str, str]] & ~AlwaysFalsy) | ... omitted 4 union elements`
run_agent.py:13573: [invalid-argument-type] invalid-argument-type: Argument to function `_is_oauth_token` is incorrect: Expected `str`, found `str | dict[Unknown, Unknown] | Any | ... omitted 4 union elements`
tests/run_agent/test_provider_attribution_headers.py:155: [unsupported-operator] unsupported-operator: Operator `not in` is not supported between objects of type `Literal["X-OpenRouter-Cache"]` and `Unknown | str | dict[str, str] | ... omitted 4 union elements`
... and 2 more
Unchanged: 4333 pre-existing issues carried over.
Diagnostics are surfaced as warnings — this check never fails the build.
There was a problem hiding this comment.
Pull request overview
This PR aims to prevent “poisoning” external memory providers by ensuring the external-memory bridge is only invoked after the built-in memory tool has actually accepted a write.
Changes:
- Add
AIAgent._memory_tool_write_succeeded(function_result: str) -> boolto interpret the memory tool’s JSON result. - Gate the external memory bridge in both the concurrent (
_invoke_tool) and sequential (_execute_tool_calls_sequential) tool execution paths based on write success. - Add regression tests in
tests/agent/test_memory_provider.pyto verify rejected writes are not mirrored and successful writes are mirrored.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
run_agent.py |
Adds a helper and uses it to gate external memory mirroring after built-in memory writes. |
tests/agent/test_memory_provider.py |
Adds tests covering sequential/concurrent bridging behavior for accepted vs rejected built-in writes. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| try: | ||
| result = json.loads(function_result) | ||
| except (TypeError, json.JSONDecodeError): | ||
| return False | ||
| return isinstance(result, dict) and result.get("success") is True |
| agent._execute_tool_calls_sequential( | ||
| SimpleNamespace( | ||
| tool_calls=[ | ||
| _memory_tool_call({ | ||
| "action": "add", | ||
| "target": "memory", | ||
| "content": accepted_content, | ||
| }) | ||
| ] | ||
| ), | ||
| messages, | ||
| "task-1", | ||
| ) | ||
|
|
||
| result = json.loads(messages[0]["content"]) | ||
| assert result["success"] is True | ||
| assert agent._memory_store.memory_entries == [accepted_content] | ||
| assert external.memory_writes == [("add", "memory", accepted_content)] |
Auto-merge: review received — action requiredReview activity detected (4 inline comment(s) by non-author(s); 2 comment(s)/review(s) from review bot(s): gemini-code-assist[bot], copilot-pull-request-reviewer[bot]). To confirm you have reviewed and accepted the feedback, please add the |
|
@claude code review |
|
@jules code review |
|
@copilot, fix all failing checks. |
- _memory_tool_write_succeeded() now returns False when the builtin store responds with success=true but message contains "already exists" (i.e. duplicate-entry no-op), preventing those from being mirrored to external providers. - Add regression test: test_sequential_memory_bridge_skips_duplicate_builtin_write verifies that a second identical add is not forwarded to the external provider.
Fixed in the latest commit. The other CI failures (44 gateway tests with |
Auto-merge: checks failingThe following checks did not pass:
Please fix the failing checks before this PR can be merged. |
Motivation
Description
AIAgent._memory_tool_write_succeeded(function_result: str) -> boolthat parses thememory_toolJSON result and returns True only whensuccessis true.\MemoryManager.on_memory_write(...)is called only when the builtin write succeeded: the concurrent helper (_invoke_tool) and the sequential path (_execute_tool_calls_sequential).\tests/agent/test_memory_provider.pycovering: rejected sequential writes are not mirrored, accepted sequential writes are mirrored, and rejected concurrent writes are not mirrored.Testing
python -m pytest -o addopts='' tests/agent/test_memory_provider.py::TestOnMemoryWriteBridge -q— all tests passed.\PYTHONPATH=/workspace/hermes-agent/.venv/lib/python3.14/site-packages python -m pytest -o addopts='' tests/agent/test_memory_provider.py -q— all tests passed (67 passed).\python -m py_compile run_agent.py tests/agent/test_memory_provider.py— success.\python -m pytest tests/agent/test_memory_provider.py -qin the base interpreter failed due to repositoryaddoptsexpecting xdist;scripts/run_tests.shattempted to install a helper package but PyPI access failed in this environment, so tests were executed using the repository venv instead.Codex Task