docs(evals): updated examples to be async#2846
Conversation
github-actions
Bot
added
bug
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| async def main(): | ||
| report = await experiment.run_evaluations_async(my_agent_task, max_workers=1) | ||
|
|
||
| asyncio.run(main()) |
There was a problem hiding this comment.
Issue: report is now scoped inside main() and discarded, but the following snippets ("Viewing Recommendations" and "Accessing Full Diagnosis Data") still reference report.display(...) and report.recommendations at module level. A reader copy-pasting these blocks in sequence will hit a NameError.
Suggestion: Either return report from main() and bind it (report = asyncio.run(main())), or move the display/recommendation calls inside main() — the way the second full example further down (L254–261) already does. Worth keeping the two examples in this same file consistent.
| async def main(): | ||
| report = await experiment.run_evaluations_async(task_function) | ||
|
|
||
| asyncio.run(main()) |
There was a problem hiding this comment.
Issue: report is assigned but never used here (no run_display()/return), so the snippet runs the evaluation and silently throws the result away.
Suggestion: Either drop the assignment, or add report.run_display() inside main() for consistency with the first example on this page. Same pattern appears in evaluators/index.mdx (L297–300).
|
Assessment: Comment Clean, consistent conversion of the evals examples to Review themes
Nice, low-risk improvement that gets the docs aligned with the async-first API. |
1 participant
Description
Updated most of the examples and code snippet to use async.
Related Issues
N/A
Documentation PR
N/A
Type of Change
Documentation update
Other (please describe):
Testing
How have you tested the change? Verify that the changes do not break functionality or introduce new warnings.
hatch run prepareChecklist
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.