Skip to content

feat: add pre/post call hooks to ToolSimulator (#167)#168

Open
kaghatim wants to merge 1 commit intostrands-agents:mainfrom
kaghatim:feat/tool-simulator-hooks
Open

feat: add pre/post call hooks to ToolSimulator (#167)#168
kaghatim wants to merge 1 commit intostrands-agents:mainfrom
kaghatim:feat/tool-simulator-hooks

Conversation

@kaghatim
Copy link
Copy Markdown
Contributor

Description

Add optional pre_call_hook and post_call_hook parameters to ToolSimulator.__init__ for fault injection and response modification during tool simulation.

  • pre_call_hook: invoked before the LLM generates a response. Receives a PreCallHookEvent dataclass. Returns a dict to short-circuit the LLM call, or None to proceed normally. Short-circuited responses are cached in the state registry.
  • post_call_hook: invoked after the LLM generates a response but before caching. Receives a PostCallHookEvent dataclass. Returns a (possibly modified) response dict.

Hook events follow the Strands SDK pattern (@dataclass events, before/after separation) and live in types/simulation/hook_events.py.

Both hooks are optional and default to None, making this fully backward compatible.

Related Issues

Closes #167

Documentation PR

N/A - No new docs needed for this change. Hook usage is documented in the __init__ docstring.

Type of Change

New feature

Testing

  • Added 12 unit tests covering: short-circuit, passthrough, response modification, caching of hook responses, call history accumulation, shared state, both hooks together, post-hook skipped on short-circuit, and backward compatibility.
  • All 24 tests pass via hatch test tests/strands_evals/simulation/test_tool_simulator.py
  • I ran hatch run prepare

Checklist

  • I have read the CONTRIBUTING document
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Comment thread src/strands_evals/simulation/tool_simulator.py
Add optional pre_call_hook and post_call_hook parameters to ToolSimulator
for fault injection and response modification during tool simulation.

- pre_call_hook: intercepts before LLM call, can short-circuit with a
  custom response (e.g., rate limit errors, timeouts)
- post_call_hook: modifies LLM-generated response before caching
  (e.g., inject metadata, corrupt fields for robustness testing)

Hooks receive typed dataclass events (PreCallHookEvent, PostCallHookEvent)
following the Strands SDK hook pattern for extensibility.

Closes strands-agents#167
@kaghatim kaghatim force-pushed the feat/tool-simulator-hooks branch from cfdcc34 to 2314f7f Compare March 18, 2026 22:32
@kaghatim kaghatim requested a review from ybdarrenwang March 19, 2026 14:37
Comment thread src/strands_evals/simulation/tool_simulator.py
Comment thread src/strands_evals/types/simulation/hook_events.py
@afarntrog
Copy link
Copy Markdown
Contributor

Thanks for creating this PR @kaghatim!

Have you considered taking the Plugin approach? I think that would be cleaner.

You would pass in the plugin here: https://github.com/kaghatim/evals/blob/2314f7f66d6b33c0bb72b8301b65296a310a07fd/src/strands_evals/simulation/tool_simulator.py#L243-L250

and you would do:

simulator = ToolSimulator(
    model=model,
    plugin=my_hook_plugin, <=== new
)

See https://strandsagents.com/docs/user-guide/concepts/plugins/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Pre/post call hooks on ToolSimulator for fault injection and response modification

3 participants