Skip to content

docs(network): add the missing VAI-011 rationale doc #10

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jhumel-code wants to merge 1 commit into from
Closed

docs(network): add the missing VAI-011 rationale doc #10

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 16 additions & 15 deletions POLICY_INDEX.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,9 +47,9 @@ Users can contribute their own policies by:
| LangChain / LangGraph | 11 | 3 | 0 | 1 | 15 | [langchain/POLICY_INDEX.md](langchain/POLICY_INDEX.md) |
| CrewAI | 7 | 6 | 0 | 1 | 14 | [crewai/POLICY_INDEX.md](crewai/POLICY_INDEX.md) |
| AutoGen / AG2 | 6 | 5 | 0 | 1 | 12 | [autogen/POLICY_INDEX.md](autogen/POLICY_INDEX.md) |
| Vercel AI SDK | 5 | 3 | 0 | 1 | 9 | [vercel_ai/POLICY_INDEX.md](vercel_ai/POLICY_INDEX.md) |
| Vercel AI SDK | 6 | 3 | 0 | 1 | 10 | [vercel_ai/POLICY_INDEX.md](vercel_ai/POLICY_INDEX.md) |
| Pydantic AI | 7 | 4 | 0 | 1 | 12 | [pydantic_ai/POLICY_INDEX.md](pydantic_ai/POLICY_INDEX.md) |
| **All** | **102** | **49** | **2** | **11** | **164** | |
| **All** | **103** | **49** | **2** | **11** | **165** | |

## All rules

Expand Down Expand Up @@ -206,16 +206,17 @@ Users can contribute their own policies by:
| 149 | VAI-006 | Vercel AI | agent | vercel_ai_agent | Vercel AI agent wires a provider shell / computer / code-execution tool | high | 0.85 | 59.5 | [vercel_ai/agent_safety.yaml](https://github.com/trustabl/trustabl-rules/blob/main/vercel_ai/agent_safety.yaml) |
| 150 | VAI-007 | Vercel AI | agent | vercel_ai_agent | Vercel AI agent tool loop has no step bound | medium | 0.60 | 24.0 | [vercel_ai/agent_safety.yaml](https://github.com/trustabl/trustabl-rules/blob/main/vercel_ai/agent_safety.yaml) |
| 151 | VAI-008 | Vercel AI | agent | vercel_ai_agent | Vercel AI agent forces a provider execution tool every step | medium | 0.65 | 26.0 | [vercel_ai/agent_safety.yaml](https://github.com/trustabl/trustabl-rules/blob/main/vercel_ai/agent_safety.yaml) |
| 152 | VAI-012 | Vercel AI | repo | vercel_ai | Vercel AI project ships no agent-guidance doc (AGENTS.md/CLAUDE.md) | low | 0.90 | 13.5 | [vercel_ai/repo_hygiene.yaml](https://github.com/trustabl/trustabl-rules/blob/main/vercel_ai/repo_hygiene.yaml) |
| 153 | PYD-001 | Pydantic AI | tool | pydantic_ai_tool | Pydantic AI tool has no description | low | 0.90 | 13.5 | [pydantic_ai/tool_definition.yaml](https://github.com/trustabl/trustabl-rules/blob/main/pydantic_ai/tool_definition.yaml) |
| 154 | PYD-002 | Pydantic AI | tool | pydantic_ai_tool | Pydantic AI tool parameters are not type-annotated | medium | 0.85 | 34.0 | [pydantic_ai/tool_definition.yaml](https://github.com/trustabl/trustabl-rules/blob/main/pydantic_ai/tool_definition.yaml) |
| 155 | PYD-003 | Pydantic AI | tool | pydantic_ai_tool | Pydantic AI tool body spawns a subprocess | high | 0.85 | 59.5 | [pydantic_ai/shell_safety.yaml](https://github.com/trustabl/trustabl-rules/blob/main/pydantic_ai/shell_safety.yaml) |
| 156 | PYD-004 | Pydantic AI | tool | pydantic_ai_tool | Pydantic AI tool body evaluates dynamic code | high | 0.85 | 59.5 | [pydantic_ai/code_execution.yaml](https://github.com/trustabl/trustabl-rules/blob/main/pydantic_ai/code_execution.yaml) |
| 157 | PYD-005 | Pydantic AI | tool | pydantic_ai_tool | Pydantic AI tool fetches a caller-controlled URL (SSRF) | high | 0.80 | 56.0 | [pydantic_ai/ssrf.yaml](https://github.com/trustabl/trustabl-rules/blob/main/pydantic_ai/ssrf.yaml) |
| 158 | PYD-006 | Pydantic AI | tool | pydantic_ai_tool | Pydantic AI tool network call has no timeout | high | 0.85 | 59.5 | [pydantic_ai/network.yaml](https://github.com/trustabl/trustabl-rules/blob/main/pydantic_ai/network.yaml) |
| 159 | PYD-007 | Pydantic AI | tool | pydantic_ai_tool | Mutating Pydantic AI tool has no idempotency key | medium | 0.55 | 22.0 | [pydantic_ai/idempotency.yaml](https://github.com/trustabl/trustabl-rules/blob/main/pydantic_ai/idempotency.yaml) |
| 160 | PYD-101 | Pydantic AI | agent | pydantic_ai_agent | Pydantic AI agent has no structured output validation | low | 0.70 | 10.5 | [pydantic_ai/agent_safety.yaml](https://github.com/trustabl/trustabl-rules/blob/main/pydantic_ai/agent_safety.yaml) |
| 161 | PYD-102 | Pydantic AI | agent | pydantic_ai_agent | Pydantic AI agent wires the code-execution native tool | high | 0.85 | 59.5 | [pydantic_ai/agent_safety.yaml](https://github.com/trustabl/trustabl-rules/blob/main/pydantic_ai/agent_safety.yaml) |
| 162 | PYD-103 | Pydantic AI | agent | pydantic_ai_agent | Pydantic AI agent wires a model-driven URL-fetching native tool | medium | 0.75 | 30.0 | [pydantic_ai/agent_safety.yaml](https://github.com/trustabl/trustabl-rules/blob/main/pydantic_ai/agent_safety.yaml) |
| 163 | PYD-105 | Pydantic AI | agent | pydantic_ai_agent | Pydantic AI agent retries with the exhaustive end strategy | low | 0.70 | 10.5 | [pydantic_ai/agent_safety.yaml](https://github.com/trustabl/trustabl-rules/blob/main/pydantic_ai/agent_safety.yaml) |
| 164 | PYD-201 | Pydantic AI | repo | pydantic_ai | Pydantic AI project ships no agent-guidance doc (AGENTS.md/CLAUDE.md) | low | 0.90 | 13.5 | [pydantic_ai/repo_hygiene.yaml](https://github.com/trustabl/trustabl-rules/blob/main/pydantic_ai/repo_hygiene.yaml) |
| 152 | VAI-011 | Vercel AI | tool | vercel_ai_tool | Vercel AI tool HTTP call has no timeout | high | 0.60 | 42.0 | [vercel_ai/network.yaml](https://github.com/trustabl/trustabl-rules/blob/main/vercel_ai/network.yaml) |
| 153 | VAI-012 | Vercel AI | repo | vercel_ai | Vercel AI project ships no agent-guidance doc (AGENTS.md/CLAUDE.md) | low | 0.90 | 13.5 | [vercel_ai/repo_hygiene.yaml](https://github.com/trustabl/trustabl-rules/blob/main/vercel_ai/repo_hygiene.yaml) |
| 154 | PYD-001 | Pydantic AI | tool | pydantic_ai_tool | Pydantic AI tool has no description | low | 0.90 | 13.5 | [pydantic_ai/tool_definition.yaml](https://github.com/trustabl/trustabl-rules/blob/main/pydantic_ai/tool_definition.yaml) |
| 155 | PYD-002 | Pydantic AI | tool | pydantic_ai_tool | Pydantic AI tool parameters are not type-annotated | medium | 0.85 | 34.0 | [pydantic_ai/tool_definition.yaml](https://github.com/trustabl/trustabl-rules/blob/main/pydantic_ai/tool_definition.yaml) |
| 156 | PYD-003 | Pydantic AI | tool | pydantic_ai_tool | Pydantic AI tool body spawns a subprocess | high | 0.85 | 59.5 | [pydantic_ai/shell_safety.yaml](https://github.com/trustabl/trustabl-rules/blob/main/pydantic_ai/shell_safety.yaml) |
| 157 | PYD-004 | Pydantic AI | tool | pydantic_ai_tool | Pydantic AI tool body evaluates dynamic code | high | 0.85 | 59.5 | [pydantic_ai/code_execution.yaml](https://github.com/trustabl/trustabl-rules/blob/main/pydantic_ai/code_execution.yaml) |
| 158 | PYD-005 | Pydantic AI | tool | pydantic_ai_tool | Pydantic AI tool fetches a caller-controlled URL (SSRF) | high | 0.80 | 56.0 | [pydantic_ai/ssrf.yaml](https://github.com/trustabl/trustabl-rules/blob/main/pydantic_ai/ssrf.yaml) |
| 159 | PYD-006 | Pydantic AI | tool | pydantic_ai_tool | Pydantic AI tool network call has no timeout | high | 0.85 | 59.5 | [pydantic_ai/network.yaml](https://github.com/trustabl/trustabl-rules/blob/main/pydantic_ai/network.yaml) |
| 160 | PYD-007 | Pydantic AI | tool | pydantic_ai_tool | Mutating Pydantic AI tool has no idempotency key | medium | 0.55 | 22.0 | [pydantic_ai/idempotency.yaml](https://github.com/trustabl/trustabl-rules/blob/main/pydantic_ai/idempotency.yaml) |
| 161 | PYD-101 | Pydantic AI | agent | pydantic_ai_agent | Pydantic AI agent has no structured output validation | low | 0.70 | 10.5 | [pydantic_ai/agent_safety.yaml](https://github.com/trustabl/trustabl-rules/blob/main/pydantic_ai/agent_safety.yaml) |
| 162 | PYD-102 | Pydantic AI | agent | pydantic_ai_agent | Pydantic AI agent wires the code-execution native tool | high | 0.85 | 59.5 | [pydantic_ai/agent_safety.yaml](https://github.com/trustabl/trustabl-rules/blob/main/pydantic_ai/agent_safety.yaml) |
| 163 | PYD-103 | Pydantic AI | agent | pydantic_ai_agent | Pydantic AI agent wires a model-driven URL-fetching native tool | medium | 0.75 | 30.0 | [pydantic_ai/agent_safety.yaml](https://github.com/trustabl/trustabl-rules/blob/main/pydantic_ai/agent_safety.yaml) |
| 164 | PYD-105 | Pydantic AI | agent | pydantic_ai_agent | Pydantic AI agent retries with the exhaustive end strategy | low | 0.70 | 10.5 | [pydantic_ai/agent_safety.yaml](https://github.com/trustabl/trustabl-rules/blob/main/pydantic_ai/agent_safety.yaml) |
| 165 | PYD-201 | Pydantic AI | repo | pydantic_ai | Pydantic AI project ships no agent-guidance doc (AGENTS.md/CLAUDE.md) | low | 0.90 | 13.5 | [pydantic_ai/repo_hygiene.yaml](https://github.com/trustabl/trustabl-rules/blob/main/pydantic_ai/repo_hygiene.yaml) |
148 changes: 148 additions & 0 deletions docs/Policy/vercel_ai/network.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
---
policy_id: vercel_ai_network
category: vercel_ai
topic: network
rules:
- id: VAI-011
severity: high
confidence: 0.6
scope: tool
fix_type: code
references: [LLM10, LLM06]
---

# Policy Rationale: Vercel AI SDK Network Safety

**Policy ID:** `vercel_ai_network`
**File:** `vercel_ai/network.yaml`
**Rules:** VAI-011
**Severities:** high
**Fix types:** code
**References:** LLM10 (Unbounded Consumption), LLM06 (Excessive Agency)

---

## What this policy covers

Vercel AI SDK tools whose `execute()` body makes an outbound HTTP call with no
timeout. **VAI-011** fires on the `has_http_call_without_timeout` fact: a handler
that calls `fetch` / `axios` / `got` / `undici` and passes no options object
carrying a `signal`, `timeout`, or `abortSignal` key. A call that attaches any of
those — `fetch(url, { signal: AbortSignal.timeout(ms) })`, `axios.get(url, {
timeout: ms })` — does not fire. The fact is structural: discovery walks the
`execute` handler, recognizes the HTTP-client call, and inspects its argument
objects for a timeout-bearing key.

---

## Why an unbounded HTTP call is a distinct concern in Vercel AI tools

Node's `fetch` (and the browser's) has no implicit timeout: with no `signal`, a
request waits until the OS gives up on the socket, which on Linux is the
`tcp_syn_retries` window — minutes. In a normal web handler a parent request can
be cancelled; inside a Vercel AI agent the tool call sits in the model's tool
loop, and there is no external caller to abort it. While the call hangs, the turn
cannot advance: the user sees nothing, no other tool runs, and the whole turn's
wall-clock and token budget drains against one stalled endpoint. The serverless
runtime's own per-invocation limit may then kill the function mid-turn, losing
all progress.

The hazard compounds with two others. Agents retry: when the call finally errors,
the model often calls the tool again, multiplying the stall and threatening the
request-worker pool. And it stacks with SSRF (VAI-003): a model-controlled URL
that *also* cannot time out is the ideal target for an injected instruction that
points the fetch at an internal host which simply never answers — a slow-loris
into the agent's own network.

---

## Rule-by-rule defense

### VAI-011 — Tool HTTP call has no timeout (Severity: high, Confidence: 0.6, Fix type: code)

**What we detect:** a Vercel AI tool `execute()` handler that calls
`fetch`/`axios`/`got`/`undici` with no options object carrying a `signal`,
`timeout`, or `abortSignal` key (`has_http_call_without_timeout`). A bare
`fetch(url)` or `fetch(url, { method })` fires; any of the three timeout keys
clears it.

**Why it is flaggable:** the SDK and the runtime inject no timeout, so a missing
one is observable in source and means the call can block the tool loop until the
OS or the platform kills it.

**Real-world consequence:** a `fetchUrl({ url })` tool that does `await
fetch(url)` is handed a URL on a sinkhole host; the turn hangs until the
serverless function's wall-clock limit terminates it, and the user's request is
lost with no partial answer.

**Why severity is high and not medium:** the failure denies the agent loop, not
just one response — there is no in-band mitigation once the call is in flight, and
the kernel default is "minutes." Medium is reserved for output-quality issues;
this freezes the turn.

**Fix type — code:** the fix is a source edit on the call —
`signal: AbortSignal.timeout(ms)` for `fetch`, or `timeout:` for axios/got. No
guardrail, hook, or runtime config can inject a deadline into an in-flight call.

**Confidence 0.6:** the check is precise about the call shape (no substring
guesswork), but it cannot follow indirection. **False positives** (fires though
bounded): an options object passed by identifier (`fetch(url, opts)`), a
`signal`/`AbortController` defined on a separate line, a `Promise.race([fetch(url),
timeout])`, or an `axios.create({ timeout })` instance whose per-call site shows
no `timeout` key. **False negatives** (silent though unbounded): `signal:
req.signal` with no deadline behind it, or an axios `timeout: 0` (which means "no
timeout"). The rule shares its predicate and its 0.6 calibration with the OpenAI
TS sibling OAI-016.

---

## What this policy does not cover

- A timeout reached indirectly — options passed by identifier, a signal/controller
bound on a separate line, a `Promise.race` deadline, or an `axios.create({
timeout })` instance — is not seen, so the rule fires on some already-bounded
calls.
- A non-deadline `signal: req.signal`, or an axios `timeout: 0`, is treated as a
timeout and does not fire.
- HTTP clients outside the recognized `fetch`/`axios`/`got`/`undici` set, or a
call made in a helper in another module (discovery sees the `execute` body, not
a wrapper elsewhere).
- TypeScript only: a tool defined in plain `.js`/`.mjs` is not AST-parsed, so its
`execute()` HTTP calls are a coverage gap.
- Retry-without-backoff and unbounded response-body reads are separate
budget-exhaustion hazards this rule does not model.

---

## Recommendations beyond the fix

```typescript
import { tool } from "ai";
import { z } from "zod";

export const getStatus = tool({
description: "Fetch a status path from the vetted API host.",
inputSchema: z.object({ path: z.string() }),
execute: async ({ path }) => {
const url = new URL(`/${path.replace(/^\/+/, "")}`, "https://api.example.com");
// Bound the call: abort after 10s so a slow host cannot hang the turn.
const res = await fetch(url, {
redirect: "error",
signal: AbortSignal.timeout(10_000),
});
// Cap how much we read so a slow-drip body cannot drain the budget either.
const body = (await res.text()).slice(0, 500_000);
return { status: res.status, body };
},
});
```

1. Attach `AbortSignal.timeout(ms)` (5–30s) to every `fetch`; for runtimes without
it, drive an `AbortController` from a `setTimeout` and clear it in a `finally`.
axios and got take a `timeout` option directly.
2. Surface the resulting `AbortError` / `TimeoutError` as a structured tool result
the model can branch on, rather than letting the promise hang or throwing raw.
3. Cap the response body size so a timely-but-slow-drip server cannot exhaust the
budget the timeout was meant to protect.
4. Pair with VAI-003: validate the destination host so a model-controlled URL
cannot aim the (now time-bounded) request at an internal address.
Loading
Loading