Repository navigation
Conversation
…rewAIInc#8186) * fix(core): send AMP requests through the proxy the environment names PlusAPI built its httpx clients with trust_env=False, so HTTP(S)_PROXY, NO_PROXY and SSL_CERT_FILE were ignored and every AMP call tried a direct connection. On a corporate network or inside a coding-agent sandbox (Claude Code, Codex), which only allow traffic through that proxy, deploy, eval, org, triggers, trace upload, skills and repository agents all failed with "[Errno 8] nodename nor servname provided", while `crewai login` (module-level httpx.post) worked. trust_env=False dates from crewAIInc#1455, when the client used requests, where it also stopped ~/.netrc credentials from replacing the AMP bearer token. httpx 0.28 no longer reads .netrc, so the flag now only disables proxy and CA-bundle variables. Remove it from the three clients (core request and multipart, and the CLI's multipart override). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(core): keep the direct-request proxy test off the OS proxy settings With no proxy variables set, httpx falls back to urllib's getproxies(), which reads the macOS or Windows system proxy settings. On a machine with a system proxy configured, the "no proxy variables" test would have routed through it. Model a machine with none configured. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(cli): keep a project's .env from choosing how crewai eval sends the login crewai eval already refuses to send the saved login to an AMP host that only the project's .env names. With AMP requests now honoring proxy and CA variables, the same .env could still pick the route: its own proxy and its own CA bundle, past that check. eval and eval --models now leave *_proxy, SSL_CERT_FILE and SSL_CERT_DIR out when they load the project's .env, say so, and keep the machine's own settings (shell exports, OS proxy). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(core): take AMP's proxy and CA from the machine, not a project's .env AMP requests carry the saved login. Honoring HTTP(S)_PROXY and SSL_CERT_FILE from the live environment let a cloned project's .env choose both: `crewai run` on a JSON crew or a declarative flow loads it with override=True, and `crewai.llm` loads it again on import, which also undid the filter `crewai eval` applied to its own loader. PlusAPI now pins the proxy and CA variables when crewai_core.plus_api is first imported, which the CLI does at startup, and hands them to httpx explicitly with trust_env=False. With none set, the OS proxy settings apply, read live. The eval loader goes back to main's, since the client now covers every command. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(cli): pin AMP's proxy and CA in the JSON-crew runner too `crewai run` runs a JSON crew in a fresh `uv run python -c` interpreter that never imports cli.py, and `_run_json_crew` loads the project's .env with override=True before tracing first imports crewai_core.plus_api, so the pin recorded the project's proxy and CA. The runner now imports plus_api before anything else. Declarative flows re-run through `uv run crewai run`, which already pins in cli.py. Also keeps `_requests_sent_by` off machine and OS proxies, so a developer's proxy can't mount a real transport ahead of its MockTransport. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(core): keep urllib's CGI proxy rule, and surface a broken plus_api import The pinned proxy map dropped urllib's REQUEST_METHOD rule: in a CGI process an uppercase HTTP_PROXY can come from a client's Proxy header, and AMP's login and deployment ZIPs would follow it. REQUEST_METHOD is now pinned with the other transport variables, and the map is read exactly as urllib reads it. The JSON-crew runner skipped any ImportError from crewai_core.plus_api. It now skips only a crewai-core too old to have the module; any other failure stops the run with its own error instead of running with nothing pinned. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(cli): import the proxy pin with the project folder off sys.path `python -c` puts the working directory, the project, first on sys.path, so a project folder named crewai_core could supply the module that pins where the saved login travels. The JSON-crew runner now imports the pin with the project off the path and restores the path right after, so project code (custom tools importing their neighbours) still finds its own modules. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(core): pin AMP's proxy and CA first thing in `import crewai` A crew run in its own process (`uv run run_crew`) never imports the CLI, so `import crewai` is where the pin happens there. It already preceded the project's .env, loaded by crewai.llm on import, but only by import order. The pin is now the first import in crewai/__init__.py, and a test runs a fresh interpreter in a project whose .env names a proxy and CA: the .env loads, and the pin still holds only what the machine set. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(cli): keep the machine's proxy and CA for the crew `crewai eval` runs `crewai eval` loads the project's .env into its own process and then runs the crew in a child that inherits that environment. The child pins what it inherits, so a .env naming HTTPS_PROXY and SSL_CERT_FILE became the run's machine transport, and that run's traces carry the saved login. The proxy and CA variables go back to the machine's pin right after the .env loads; the rest of the .env still applies, and the run loads the .env itself after its own pin, as under `crewai run`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(core): keep a project's UV_ENV_FILE out of the crew `crewai eval` runs `crewai eval` loads the project's .env and starts the crew with `uv run`. A .env naming itself as UV_ENV_FILE had uv load it into the run before Python started, so the run pinned the project's proxy and CA anyway. UV_ENV_FILE now counts among the variables restored to the machine's. The pin test starts the run through `uv run`, as the CLI does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…rewAIInc#8278) * Preserve non-ASCII characters in JSON loading Updated json_loader.py to ensure non-ASCII characters are preserved when converting JSON data to text. * Fix BOM handling in JSON loader tests * Add tests for JSONLoader with non-ASCII characters Add tests to verify JSONLoader handles non-ASCII text correctly. * Preserve lone surrogate escapes in JSONLoader output Added encoding to keep lone surrogates escaped in JSON output. * Test JSONLoader scalar Unicode and surrogate handling
* fix: store deferred tool results in tmp * chore: remove tool result artifact diagnostic
* feat(rag): add standard-library jsonl loader * fix(rag): enforce strict jsonl record validation * docs(rag): document jsonl loader functions and tests --------- Co-authored-by: Lorenze Jay <63378463+lorenzejay@users.noreply.github.com>
* feat(rag): add read-only notebook text loader * fix(rag): validate notebook cells and metadata * docs(rag): document notebook loader functions and tests * fix(rag): accept utf-8 bom in notebook files --------- Co-authored-by: Lorenze Jay <63378463+lorenzejay@users.noreply.github.com>
…c#8273) * feat(tools): add Azure DocumentDB vector search integration Add native vector search, document ingestion, IVF indexing, and renewable Entra ID authentication for Azure DocumentDB and Azure OpenAI. Include optional dependencies, public exports, tool metadata, documentation, and unit, Podman, and live Azure tests. Refs crewAIInc#7302 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs(tools): complete Azure DocumentDB function docstrings Document tool helpers, OIDC callbacks, and tests to satisfy the PR docstring coverage threshold. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat(flow): add opt-in per-turn conversation tracing * fix(flow): avoid completing turns paused for human feedback
…oding agents less (crewAIInc#8298) * feat(cli): rewrite the scaffolded AGENTS.md around the CrewAI + AMP workflow The template opened with an API reference and mentioned traces, eval and deployment as optional extras, so coding agents rarely used them. It now opens with the steps an agent follows in a session (set up, reuse what the organization has on AMP, run, trace and evaluate every change, deploy when the work has to run elsewhere), then a short reference of the facts models get wrong. About 3.3K tokens instead of 11.9K, with outdated claims corrected. Tests assert the behaviours the old wording carried, check every documented `crewai` command against the CLI, and keep numbered lists gapless with and without the evaluation blocks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(cli): index the features AGENTS.md no longer details, and guard the file The workflow rewrite dropped references to guardrails, knowledge, MCP, conversational and declarative flows, checkpoints and more. An "Also available" index under Reference names each one with its API and docs path. Tests now guard the shipped file on every PR: a reference manifest, a token budget, text that is wrong on main, imports that must resolve, python blocks that must parse, and docs paths that must exist. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(cli): add a core API section to AGENTS.md for models that don't know CrewAI Measured with OpenAI models, the workflow rewrite lost the basics they don't know without the file: flow decorators and state, the @crewbase class, task fields, CrewOutput and custom tools. A short Core API section restores them and folds in the two crew and router facts it would otherwise repeat. The reference manifest now guards those core references too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: run the AGENTS.md guards on PRs that touch the template The guards live in lib/cli/tests, which the main test workflow doesn't run. A path-filtered job runs them when the template, its tests or the docs pages it links change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: run the AGENTS.md workflow only on pull requests against main Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(cli): close the gaps where AGENTS.md taught less than the file it replaces Question by question against today's file, V6 trailed on facts it had cut or worded loosely: the context window and date options, task output folders and human review, the guardrail's return shape, test targets before writes, the eval's structural fixes, logged-in trace visibility, partial selector misses, written IDs for silent failures, ZIP deploys with an origin, the flow memory root, and LiteLLM for providers outside the native set. Each is back, verified on main. Measured on Claude Code (Haiku, Sonnet) and Codex, no question now scores below today's file. The token budget rises to 4,800 to hold them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(cli): teach the hooks and planning that telemetry counts, and three facts the old file had The index named kickoff hooks and Crew(planning=True), but the features most projects use are interception hooks (@on(InterceptionPoint...) in crewai.hooks) and agent planning (Agent(planning_config=PlanningConfig(...))). Both lines now lead with those. Three facts today's file taught come back: OpenAI's Responses API, agents judge while code computes, and answering a paused deployed flow through AMP's human_feedback_requests API. Measured on 221 questions in Claude Code (Haiku, Sonnet) and Codex, no question scores below today's file. The budget rises to 5,000 tokens, guards cover the new lines, and the never-teach list allows naming reasoning=True as replaced. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(devtools): agents-md-eval fails a PR whose AGENTS.md teaches coding agents less `devtools agents-md-eval` asks a 218-question bank in fresh, tool-less Claude Code and Codex sessions, once with the base branch's AGENTS.md and once with this branch's, grades each answer by its key, and exits non-zero when either agent scores more than 10% below the base. The AGENTS.md workflow runs it on PRs that change the template, the eval or its questions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(devtools): run the AGENTS.md eval per model and count sessions with no answer The action asks Claude Code with the latest Haiku and Sonnet and Codex with gpt-5.6-luna and gpt-5.6-sol, one job per model. A session that hangs, fails or returns nothing is retried once and then counted; past 2% unanswered the comparison is called unreliable instead of scored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: run the AGENTS.md eval only on pull requests against main Drops the manual trigger and the conditions it needed; a test pins the trigger. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(devtools): say why AGENTS.md eval sessions gave no answer Each unanswered session records its exit code and last error line (keys redacted) or a timeout, and the command prints the most common reasons, so a run called unreliable can be diagnosed from the CI log. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(devtools): grade 'is a different command' as a warning, accept 'call the agent directly' Two answer-key misses found reading V6's answers: a correct answer that warns off `crewai update` tripped its trap, and "No. Call the agent directly" didn't count as needing no crew. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(devtools): B-065's trap fires only on advice to run crewai update Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(devtools): read a failed session's reason from stdout when stderr is empty Claude Code prints some failures, such as 'Not logged in', on stdout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(devtools): accept 'never a fix' in C-018's key Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(devtools): read typographic apostrophes and 'replaces' as warnings in traps Codex writes "don’t"; the negation check knew only "don't", so its warnings counted as recommendations. "X replaces reasoning=True" now reads as a warning too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(devtools): add hooks and agent planning questions; D-018 needs the tools, not the package name Three questions for features telemetry counts and the bank missed: tool-call hooks, LLM-call hooks and agent planning (221 questions). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(cli): AGENTS.md teaches that crewai eval runs the crew itself, logged in or out Since crewAIInc#8270, `crewai eval` runs the crew, traced, when the project changed since its last traced run. The file still said to `crewai run` first, and that a run without a terminal is traced only when logged in, so logged-out coding agents logged in or stopped instead of evaluating. It now says `crewai eval` works logged in or out, with or without a terminal, with no `crewai run` between rounds and no wait on a login. The CLI test's command parser stops at a group that runs without a subcommand (`crewai eval`, `crewai eval --models ...`), which `crewai eval` became when it gained `export`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(devtools): ask whether crewai eval needs a login, and read "doesn't" as a warning Two questions join the bank: can a logged-out coding agent with no terminal get a verdict, and does it need `crewai run` first. Three keys follow main: C-004 and B-051 accept `crewai eval` (it runs and traces the crew), C-032 accepts "flow code". Negations now include doesn't, didn't, aren't and can't, so "evaluation doesn't require an account" no longer trips the trap it warns against. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(ci): run the AGENTS.md eval from the base commit, with the PR's files as data The eval job checked out the pull request and ran its devtools with both API keys in the environment, so a branch could print or send them. It now checks out the base commit, fetches the pull request's commit only to read its AGENTS.md and CLAUDE.md (`agents-md-eval --head <sha>`), and runs in the `agents-md-eval` environment, where the keys can sit behind a required reviewer. When the base branch has no eval yet, the job skips with a notice. The eval also names a session's real error: Claude Code can end its output with a JSON log line after the message ("Invalid API key"), which hid every Haiku failure. Item text and reasons print as text, so `[tool.crewai]` isn't dropped and `[/...]` can't raise MarkupError after the paid runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(cli): AGENTS.md says deploys build from any git origin, not only GitHub `crewai deploy create` sends whatever `origin` URL the project has, and AMP builds from GitHub or another git host (`any_git`). The file said an origin meant GitHub, which pointed GitLab and other users at the wrong fix. The GitHub integration note now applies to GitHub only, and a guard keeps it so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(ci): run the AGENTS.md eval on pull_request_target, so its keys reach only reviewed code Under pull_request a branch's own copy of the workflow runs, so a pull request could edit the eval's steps and still get the keys. The eval now has its own workflow on pull_request_target: GitHub runs it from the base branch, the job checks out the base commit, and the pull request's AGENTS.md and CLAUDE.md are fetched and read as data, never run. It runs for this repository's branches only, since pull_request_target would hand the keys to forks too, and it fetches the pull request's commit only once it knows it will run, so a skip can't fail on the fetch. The guards stay on pull_request, with no secrets, and no longer persist the checkout's token for the pull request's code to read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(devtools): accept "go-ahead" as asking first in C-052's key "Publishing writes to your organization, so it needs your go-ahead first" asks before publishing, but the key only matched ask, approve, confirm or permission, so correct answers scored wrong. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
… it asks (crewAIInc#8301) At a terminal, the first time tracing is turned on, `crewai eval` showed crewAI's trace consent screen ("Detailed execution traces are available! ... Share this execution trace with CrewAI?") with nothing before it, though no crew had run yet. It now says first why the crew runs and where its trace goes ("No traced run of this crew yet. To evaluate it, the crew runs now and its trace goes to CrewAI AMP, which grades it."), and does not repeat the run line after a yes. Without a terminal nothing changes: no screen, no extra line. `crewai eval --help` still said "Nothing is asked, with or without a terminal". It now says a person at a terminal is asked the first time tracing is turned on, and that a project that turned tracing off is asked at a terminal and told how without one. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(tools): default-deny non-global IPs in SSRF check is_blocked_ip used hand-written denylists that missed CGNAT (100.64.0.0/10, incl. Alibaba metadata 100.100.100.200), 198.18.0.0/15, Azure WireServer (168.63.129.16), and IPv6 forms wrapping private IPv4 (NAT64, 6to4). Block anything that is not is_global, keep explicit entries for ranges is_global may allow or that vary by Python version, and unwrap IPv4-mapped and NAT64 (64:ff9b::/96) addresses so the embedded IPv4 is checked. 6to4, Teredo and local-use NAT64 are non-global and blocked. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * docs(tools): note CGNAT is blocked by SSRF-safe scrape tools Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * fix(tools): block IPv6 transition ranges explicitly in is_blocked_ip 6to4 and local-use NAT64 were is_global=True before the CVE-2024-4032 fix (Python <3.10.15 / <3.11.10), so they relied on the interpreter to be blocked. Add them, Teredo, IPv4-compatible (::/96), IPv4-translated (::ffff:0:0:0/96) and deprecated site-local (fec0::/10) to the explicit IPv6 list, which also closes forms is_global allows on every version. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
…Inc#8303) Serialize replacement rows before deleting saved memories so invalid metadata cannot erase the original record. Add regression coverage for serialization failures and successful replacement. Fixes crewAIInc#8302 Signed-off-by: vatsalyd <vatsal.y.official@gmail.com>
JsonProvider.prune() and the checkpoint CLI joined the checkpoint location (and branch) with "*.json" or "**/*.json" and passed the result to glob.glob(), so glob characters in the path were read as a pattern. For "runs[1]", pruning deleted the checkpoints in "runs1" and kept its own, and the CLI list, info and prune commands read "runs1". Escape the directory part, so that only the file name part is a pattern.
…c#8282) Newer Claude models reject a forced named tool_choice, and one executable tool should not require the model to call it.
…to a log (crewAIInc#8325) * feat(cli): with no terminal, crewai eval sends the crew's own output to a log A coding agent reads every line `crewai eval` prints, and pays for it. When `crewai eval` ran the crew itself, the crew's panels, final answer and tool calls came first: 341 of 378 lines in a measured run, all before the verdict, and an agent that cuts long output could lose the verdict and the brief at the end. With no terminal, what the crew prints while it runs now goes to `.crewai/last_run.log`, and the agent gets one line saying so, one saying how long the run took, then the verdict and brief as before: 40 lines for the same crew. A run that leaves no trace to evaluate shows its last 40 lines, which usually hold the error. At a terminal nothing changes. It moves this process's stdout and stderr while the crew runs, which every child inherits, so every way a project runs is covered; anything that keeps them from moving leaves the output on screen. `.crewai/` is already left out of the project-changed check, so the log never triggers a re-run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(cli): the run log is the owner's alone, and both streams always come back Review on crewAIInc#8325: - the run log holds final answers, tool output and tracebacks that can carry a credential: it is now opened owner-only (0600) and never through a link, and a log an earlier run left is emptied and tightened; - both descriptors are saved before either moves, and a move that fails halfway puts back what it moved, so the verdict still reaches the screen; - putting them back is best effort for each on its own and never raises, so the run's own error is the one that surfaces; - whether to keep the transcript off screen is decided by stdout, as for the brief (`nobody_watching()`): an agent can hand the command a terminal for stdin and still read a captured stdout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(cli): the run log is a file object closed by its with; a test's raise moves into a helper Code-quality review on crewAIInc#8325: the log descriptor was closed by hand on every path; it is now a file object (os.fdopen) the redirect holds in a with, so it is closed however the block ends. The put-back test raised inline inside pytest.raises, which reads as unreachable code after it; the raise is now in a helper. No behaviour changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…project (crewAIInc#8326) * feat(cli): every crewai eval keeps the evaluation's full data in the project An evaluation's report opens for one five-minute sitting after it is first read, and the verdict `crewai eval` waits for is that read. A coding agent that wants a check's quotes, the outputs or the trace after its edits ran `crewai eval export` and found it closed ("has been read already"). Right after the verdict, `crewai eval` now saves the export into `.crewai/evaluations/<id>/` (report.md, run/checks.md, run/outputs.md, run/trace.json, beside a `.gitignore` of `*`) and says where. Graded again with nothing changed, it points at what it saved instead of asking AMP a second time. Only inside a project; the ten newest are kept. A service that does not export yet says nothing, and any other failure is one line: the verdict and the exit code are never the save's. The export writer now raises an error its callers decide about instead of exiting, and writes into a nested folder the same way it wrote into one: each folder opened inside the one before it, never through a link. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(cli): a saved evaluation is marked whole, pruned only by its owner, and fetched briefly Review on crewAIInc#8326: - the save asks AMP with its own short timeout (20 s to read) instead of export's two minutes: the verdict is already printed, and CI waits on the command (`PlusAPI.export_evaluation(..., timeout=)`); - a marker is written last, and only when every file was; it, not report.md, says an evaluation is saved, so a save cut short is tried again; - a save that skipped a file says "Part of its data was saved … N skipped" and leaves no marker, instead of claiming the full data; - only a folder this command saved whole (the marker, a plain file) is ever removed to keep the newest ten; never a link or a junction, and nothing at all when `.crewai/` or `.crewai/evaluations/` is a link; - the path writer (Windows) refuses a link anywhere on the folder's way before making anything through it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(cli): AMP cannot send a saved evaluation's marker Review on crewAIInc#8326: `_export_target` reserved the folder's `.gitignore` but not the marker that says a save was whole. A payload carrying `.crewai-eval-saved` and one file that is skipped would have left that marker on a partial save: read as whole, never tried again, and kept over a complete one. The marker is now reserved the same way, compared as a case-insensitive filesystem that drops trailing dots sees it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(deps): update vulnerable audit dependencies * fix(ci): update uv bootstrap pins
… ai mind guides (crewAIInc#8356) Co-authored-by: leom1919 <leom1919@users.noreply.github.com>
…trace guides (crewAIInc#8359) Co-authored-by: leom1919 <leom1919@users.noreply.github.com>
…ctly (crewAIInc#8366) Fixes crewAIInc#8365 Change - replay() now emits CrewKickoffStartedEvent(crew_name, inputs=replay_inputs) after input interpolation and stores its event_id in _kickoff_event_id, mirroring prepare_kickoff. The emit future is awaited like the kickoff path does. Why - replay() skipped prepare_kickoff but its execution tail still emitted CrewKickoffCompletedEvent(started_event_id=self._kickoff_event_id): the bus warned about an unpaired ending event and the completion referenced the original kickoff's start id, corrupting event pairing for listeners. Proof - New test test_replay_emits_paired_kickoff_events: after kickoff(), replay() must surface its own CrewKickoffStartedEvent and leave _kickoff_event_id pointing at it (fails on main; completion events are constructed from _kickoff_event_id in _create_crew_output, so the id is the pairing invariant). All 14 existing replay tests and the event bus suites stay green (135 passed; the single usage-metrics failure reproduces on unmodified main, env-only). Risk - Listeners that assume exactly one CrewKickoffStartedEvent per Crew instance now see one per execution run (kickoff + each replay), which is the documented lifecycle semantics. 🤖 Generated with ZCode (AI agent) on behalf of Wilson0511 Co-authored-by: Wilson0511 <wilsonjiang0511@gmail.com> Co-authored-by: ZCode <agent@zcode.dev>
…Inc#8230) * feat(tools): add offline jev routing and guardrail evaluation runner * fix(tools): record malformed jev responses as failed and split jsonl on newlines only * docs(tools): note jev evaluation differs from runtime guardrail outcomes * fix(tools): fail jev answers whose type does not match the question * fix(tools): fail jev choice answers with non-mapping probabilities * fix(tools): keep falsy non-mapping jev probabilities so they fail validation * refactor: move jev evaluation runner to scripts/ --------- Co-authored-by: Lorenze Jay <63378463+lorenzejay@users.noreply.github.com>
…ation (crewAIInc#8393) The crewAI-examples repo was reorganized (crewAIInc#277): flows moved under flows/, lead_score_flow renamed to lead-score-flow, crewai-langgraph renamed to CrewAI-LangGraph, Notebooks renamed to notebooks. Several docs pages still point to the old paths, which now 404. Fixes, all verified against the live repo tree: - tree/main/email_auto_responder_flow -> tree/main/flows/email_auto_responder_flow - tree/main/lead-score-flow -> tree/main/flows/lead-score-flow - tree/main/meeting_assistant_flow -> tree/main/flows/meeting_assistant_flow - tree/main/write_a_book_with_flows -> tree/main/flows/write_a_book_with_flows - tree/main/flows/lead_score_flow -> tree/main/flows/lead-score-flow - tree/main/integrations/crewai-langgraph -> tree/main/integrations/CrewAI-LangGraph - tree/main/Notebooks -> tree/main/notebooks (incl. Simple QA Crew + Flow) Scope: docs/edge and docs/v1.15.28, en/ko/pt-BR/ar. 68 link replacements across 20 files. The flows/content_creator_flow link is left untouched on purpose: that path currently exists as a broken submodule in crewAI-examples (gitlink without .gitmodules), so it needs fixing on the repo side first.
* fix: resolve pip-audit findings * fix: align opentelemetry package versions
A second kickoff of a hierarchical crew broke in two ways. With manager_llm the manager built on the first run kept its delegation tools, so _create_manager_agent raised "Manager agent should not have tools". With manager_agent the first run left the manager assigned to the task, and _update_manager_tools then offered it only itself as a coworker. Only reject tools that are not BaseAgentTool delegation tools, and don't treat the manager as the task's worker. This also fixes copy() after a run, kickoff_for_each() after a run and train(n_iterations > 1).
…nc#8403) Change: process_guardrail now emits LLMGuardrailCompletedEvent(success=False, error='<Type>: <msg>') and re-raises when the guardrail callable raises a non-HookAborted exception, mirroring the existing HookAborted branch. Why: LLMGuardrailStartedEvent is emitted before the call, but an ordinary exception left it without a terminal event. The tracing listener's 'evaluate guardrail' span then stayed open until the end-of-execution sweep closed it as 'orphaned', discarding the real error from the trace; retries of unrelated guardrails stacked more orphaned spans. Proof: - tests/test_guardrail_events_on_exception.py: raising guardrail -> exactly one started + one completed(success=False) event, exception unchanged for the caller (red before the fix: completed list empty). - hooks/test_hook_abort_propagation.py + test_task_guardrails.py: 48 passed (deny path and task retry behavior unchanged). Risk: Low. Purely additive event emission on a path that previously raised immediately; HookAborted keeps its exact legacy error string. 🤖 Generated with ZCode (AI agent) on behalf of Wilson0511 Co-authored-by: Wilson0511 <wilsonjiang0511@gmail.com> Co-authored-by: ZCode <agent@zcode.dev>
…rewAIInc#8418) Change: Crew.replay's execution tail now wraps _execute_tasks in try/except, emitting CrewKickoffFailedEvent(started_event_id=self._kickoff_event_id) before re-raising — mirroring kickoff()'s except-clause. Why: crewAIInc#8366 taught replay() to emit CrewKickoffStartedEvent so the success path's completion pairs correctly, but the failure path had no terminal: a task exception raised straight out of replay left the start event dangling — no CrewKickoffFailedEvent for listeners, and a leaked crew_kickoff_started frame on the event-scope stack that parented every later event in the same context (the next kickoff's start event got the leaked frame as parent_event_id; 100 failed replays on one thread hit the scope stack depth limit). Proof: - test_crew.py::test_replay_failure_emits_kickoff_failed_event: a failing replay emits a Failed event scoped to the replay's own start id (red: no Failed event at all). - test_crew.py::test_replay_failure_does_not_leak_kickoff_scope_into_next_kickoff: after a failed replay, the next kickoff's start event has parent_event_id None (red: parented to the leaked replay scope). - test_crew.py -k 'replay or kickoff_event': 16 passed; full test_crew.py: 137 passed, 1 skipped (env-only exclusions unchanged from main). Risk: Low. Purely additive terminal event on a path that previously raised immediately; success path unchanged. 🤖 Generated with ZCode (AI agent) on behalf of Wilson0511 Co-authored-by: Wilson0511 <wilsonjiang0511@gmail.com> Co-authored-by: ZCode <agent@zcode.dev>
…airs (crewAIInc#8420) Change: Crew.test emits CrewTestStartedEvent before create_llm (eval_llm now carries the requested spec, which the field's type already allows) instead of after it, so a setup failure's CrewTestFailedEvent has its opener on the scope stack. Why: crew_test_failed is scope-ending (VALID_EVENT_PAIRS -> crew_test_started). The start event was emitted only after create_llm succeeded, so a create_llm failure (bad model spec, missing key) emitted an unpaired Failed that popped the caller's enclosing event scope and corrupted later event parentage — the same theft shape as the a2a delegation fix (crewAIInc#8407), in the test harness path. Proof: - test_crew.py::test_crew_test_setup_failure_pairs_events: with create_llm raising, test() emits Started then a Failed whose started_event_id pairs with it (red: no Started at all, Failed unpaired). - test_crew.py -k 'test_crew or replay': 136 passed, 1 skipped, plus the known env-only test_crew_kickoff_streaming_usage_metrics that fails identically on clean main. The existing success-path test (test_crew_testing_function) still passes. Risk: Low. Success-path payload change is eval_llm now carrying the requested spec instead of the built instance; no in-repo listener reads the field (telemetry has no crew_test handler). 🤖 Generated with ZCode (AI agent) on behalf of Wilson0511 Co-authored-by: Wilson0511 <wilsonjiang0511@gmail.com> Co-authored-by: ZCode <agent@zcode.dev>
…nc#8424) Tool actions build their tool instances once per flow object (_flow_post_init) and reuse them across kickoffs, but nothing reset current_usage_count at the kickoff boundary. A max_usage_count budget exhausted in an earlier kickoff therefore made every later kickoff of the same flow object fail: BaseTool.run answered with ToolFailure(USAGE_LIMIT) and ToolAction.run (PR crewAIInc#8367) turned that into a fatal ToolExecutionFailedError. Agents already zero the counter every time they prepare tools (parse_tools); flows now reset at the kickoff boundary instead, in kickoff_async - the single seam every run goes through (kickoff, kickoff_async, astream, checkpoint resume). Change: - _actions.py (ToolAction.__init__): register the built tool on the flow instance (_tool_action_tools) so kickoffs can find it. - runtime/__init__.py (Flow._reset_tool_action_usage_budgets): new helper calling tool.reset_usage_count() for every registered tool; invoked at the top of kickoff_async's execution path. Proof: - tests/test_flow_tool_action_usage.py::test_tool_action_usage_budget_resets_between_kickoffs fails on main and passes with the fix. - test_tool_action_usage_budget_still_enforced_within_a_kickoff pins that the budget remains per-kickoff, not per-call. - tests/test_flow_from_definition.py (includes PR crewAIInc#8367's suite): 173 passed. Risk: - Low: reset only touches tools built by flow tool actions; the budget still applies within a single kickoff. 🤖 Generated with ZCode (AI agent) on behalf of Wilson0511 Co-authored-by: Wilson0511 <wilsonjiang0511@gmail.com> Co-authored-by: ZCode <agent@zcode.dev>
…nc#8426) ToolAction.run (PR crewAIInc#8367) raised ToolExecutionFailedError on every detected ToolFailure, ignoring BaseTool.tool_failure_policy - whose contract is "Overrides the agent's and task's tool_failure_policy for this tool only". A tool explicitly configured WARN (or IGNORE) still killed the whole flow, and because the raise bypassed handle_tool_failure entirely, not a single ToolFailureDetectedEvent reached the event bus (zero observability for flow tool failures). Change: - _actions.py (ToolAction.run): when the tool declares an explicit tool_failure_policy, route the detected failure through handle_tool_failure (tool=...): WARN records + emits ToolFailureDetectedEvent and the flow continues with the soft-failure output; IGNORE stays silent; RAISE aborts. A tool with no policy set keeps flow actions' fail-hard default, preserving crewAIInc#8367's behavior for unconfigured tools. Proof: - tests/test_flow_tool_action_failure_policy.py::test_warn_policy_tool_does_not_kill_the_flow fails on main (ToolExecutionFailedError + zero events) and passes with the fix; test_raise_policy_tool_still_aborts_the_flow pins RAISE. - tests/test_flow_from_definition.py (includes crewAIInc#8367's suite): 173 passed; tests/test_flow_tool_action_usage.py: 2 passed. Risk: - Low: only tools that explicitly set tool_failure_policy change behavior, in the direction their own configuration asks for. 🤖 Generated with ZCode (AI agent) on behalf of Wilson0511 Co-authored-by: Wilson0511 <wilsonjiang0511@gmail.com> Co-authored-by: ZCode <agent@zcode.dev>
Change: AgentExecutor.execute_tool_action (the ReAct text-based tool route) caught every Exception from execute_tool_and_check_finality and folded it into an "Observation: Error executing tool: ..." recovery message for the LLM. That handler also swallowed HookAborted raised by a POST_TOOL_CALL policy hook. Add an `except HookAborted: raise` clause ahead of the generic handler, mirroring the existing ToolExecutionFailedError passthrough. Why: The hook contract makes HookAborted a deliberate stop, not a recoverable tool error: _invoke_hook (hooks/dispatch.py) re-raises only HookAborted and fails open on everything else, and the documented way to veto a tool call is `raise HookAborted(...)` inside an @on(...) hook. The native tool-call path (execute_native_tool), the parallel todo executor and LiteAgent all re-raise HookAborted so the run terminates. Only the ReAct text path downgraded the abort into a tool-error observation, letting the agent keep iterating (and re-run the tool) after a policy veto, with inconsistent behavior between the two tool-call modes. Proof: - New lib/crewai/tests/agents/test_react_tool_hook_contract.py: a POST_TOOL_CALL hook raising HookAborted now propagates out of execute_tool_action (red on main: DID NOT RAISE), while a plain RuntimeError from tool execution still takes the observation recovery path and returns "tool_completed". - lib/crewai/tests/agents/test_native_tool_hook_contract.py, lib/crewai/tests/tools/test_tool_failure.py: 122 passed. - lib/crewai/tests/agents/test_agent_executor.py: same 12 pre-existing failures before and after this change (verified via stash), 109 passed. Risk: Minimal. HookAborted is already imported in this module and re-raised at four other sites in the same file; only deliberate policy aborts change behavior, and plain exceptions keep the existing recovery path. 🤖 Generated with ZCode (AI agent) on behalf of Wilson0511 Co-authored-by: Wilson0511 <wilsonjiang0511@gmail.com> Co-authored-by: ZCode <agent@zcode.dev>
Guardrail retries rebuilt the prompt from only the validation-error template, so declared upstream task output was dropped. Preserve the context the agent already received, including PRE_STEP rewrites. Co-authored-by: wannT-1 <wannT-1@users.noreply.github.com>
cursor
Bot
force-pushed
the
cursor/fix-guardrail-retry-context-68a0
branch
from
October 23, 2026 08:15
8497449 to
13b2b63
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related issue
Fixes crewAIInc#8219
Summary
When a task with declared upstream context fails a guardrail, the retry rebuilt the prompt from only the validation-error template. Outputs from tasks listed in
context(for example an account service code produced by an earlier research task) were dropped, so the retried agent could not cite them.Task._invoke_guardrail_functionandTask._ainvoke_guardrail_functionnow freeze the context the agent already received and prepend it to the validation error on every retry.prompt_contextis updated afterPRE_STEP, so a hook that rewrites context (for example redaction) is what the retry reuses. If there was no upstream context, the retry still sends only the validation error.Verification
test_guardrail_retry_preserves_upstream_contextand its async counterpart assert that each retry prompt still contains the upstream facts (ACCT-8841) plus the guardrail error. A no-context retry stays validation-only, and aPRE_STEPrewrite is preserved instead of the pre-hook text.lib/crewai/tests/test_task_guardrails.pyandlib/crewai/tests/task/test_async_task.pypass (44 tests).Additional context
None