Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 85 additions & 0 deletions docs/REFERENCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -415,6 +415,91 @@ A contract violation on a dependency stops the chain immediately and raises `CON

---

## Immutable Inference Sandboxing (IIS)

OAS gives every spec a `sandbox:` block that defines **what a task is mechanically permitted to do**. The runner enforces these constraints _before_ any tool call reaches the I/O layer — no network connection is opened, no file handle is created, no exception needs to be caught.

### The boundary

| Layer | Concern | Mechanism |
|-------|---------|-----------|
| **OAS `sandbox:`** | Hard execution constraints | Runner blocks before dispatch |
| **BCE `behavioural_contract:`** | Policy / quality contract | Validated after the run |

OAS controls **what a task can do**. BCE controls **what a task should do**. They are complementary and never overlap.

### Root-level sandbox (applies to all tasks)

```yaml
sandbox:
tools:
allow: [file.read, http.get] # allowlist — anything else is SANDBOX_TOOL_VIOLATION
# deny: [file.write] # denylist alternative (use one or the other)
http:
allow_domains:
- api.example.com # exact match or any subdomain
file:
allow_paths:
- ./data/ # resolved to absolute paths at check time
```

### Per-task sandbox (overrides root)

A `sandbox:` key inside a task definition completely overrides the root sandbox for that task. Use this to _tighten_ constraints on sensitive tasks without changing the global default:

```yaml
sandbox:
tools:
allow: [file.read, http.get]

tasks:
check_status:
sandbox:
tools:
allow: [http.get] # file.read now also blocked for this task
http:
allow_domains: [status.openai.com]
```

### Error codes

All sandbox violations raise `OARunError` immediately with one of three structured codes:

| Code | Trigger |
|------|---------|
| `SANDBOX_TOOL_VIOLATION` | Tool name not in `allow` list, or in `deny` list |
| `SANDBOX_DOMAIN_VIOLATION` | HTTP host not in `allow_domains` (for `http.get` / `http.post`) |
| `SANDBOX_PATH_VIOLATION` | File path outside `allow_paths` (for `file.read` / `file.write`) |

Path traversal (`../../`) is caught automatically — paths are resolved to absolute before comparison.

### Input immutability

Every task receives a **deep copy** of its input. Chain outputs merged into downstream inputs never mutate the caller's original dict. This is enforced at three levels:

1. Entry to `run_task_from_spec` (public API boundary)
2. Each dependency call in `_resolve_chain`
3. Start of `_run_single_task` (guards delegated specs too)

### What IIS deliberately excludes (belongs in BCE)

| Concern | Why it's BCE |
|---------|-------------|
| Prompt injection detection | Requires interpretation — subjective, evolving |
| PII scanning | Team-specific policy |
| Compliance tagging | Audit concern, not execution constraint |
| Session / memory management | Out of scope for a stateless runner |

### Future BCE rename note

The BCE library currently uses `allowed_tools` in `behavioural_contract` as an audit field. A future BCE release will rename this to `expected_tools` to make it unambiguous that this is a _post-run audit assertion_, not an enforcement rule. The OAS `sandbox.tools.allow` list is the enforcement mechanism; BCE's audit field is observational only.

### Example

See [`examples/sandboxed-agent/`](../examples/sandboxed-agent/) for a working demo that exercises tool allowlists, domain restrictions, file path restrictions, and per-task sandbox overrides.

---

## Engines (quick)

| Engine | Env var | Notes |
Expand Down
70 changes: 70 additions & 0 deletions examples/sandboxed-agent/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Sandboxed Agent — IIS Demo

Demonstrates **Immutable Inference Sandboxing (IIS)**: the runner enforces hard
execution constraints _before_ any tool call reaches the I/O layer.

## What IIS is

| Layer | Concern |
|-------|---------|
| **OAS `sandbox:`** | Hard mechanical block — binary allow/deny enforced by the runner |
| **BCE `behavioural_contract:`** | Soft policy audit — validated after the run |

OAS owns **what a task can do**. BCE owns **what a task should do**. They never overlap.

## Sandbox keys

```yaml
sandbox:
tools:
allow: [file.read, http.get] # allowlist (mutually exclusive with deny)
# deny: [file.write] # denylist alternative
http:
allow_domains: [api.example.com] # subdomains are also allowed
file:
allow_paths: [./data/] # resolved to absolute paths at check time
```

A **root-level** `sandbox:` applies to every task. A **task-level** `sandbox:`
completely overrides the root for that task, so you can tighten constraints
per-task without relaxing the global defaults.

## Error codes

| Code | Trigger |
|------|---------|
| `SANDBOX_TOOL_VIOLATION` | Tool name blocked by `allow` / `deny` |
| `SANDBOX_DOMAIN_VIOLATION` | HTTP host not in `allow_domains` |
| `SANDBOX_PATH_VIOLATION` | File path outside `allow_paths` |

## Running the demo

```bash
# Summarise the sample report (file.read allowed inside data/)
oa run spec.yaml --task summarise --input '{"filename": "report.txt"}'

# Check OpenAI status (http.get allowed to status.openai.com)
oa run spec.yaml --task check_status --input '{}'
```

## Seeing a violation

Try reading a file outside the allow_paths:

```bash
oa run spec.yaml --task summarise --input '{"filename": "../../README.md"}'
# → OARunError: SANDBOX_PATH_VIOLATION
```

Or call a blocked domain (by temporarily editing the prompt):

```bash
# The runner blocks the call before it reaches the network.
```

## What IIS deliberately does NOT do

- **Prompt injection detection** — subjective, evolving, belongs in BCE
- **PII scanning** — policy-level concern, belongs in BCE
- **Memory / session management** — out of scope for a stateless runner
- **Parallel execution or retry logic** — not OAS concerns
3 changes: 3 additions & 0 deletions examples/sandboxed-agent/data/report.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Q3 2026 Revenue: $4.2M (+18% YoY)
Key drivers: enterprise subscriptions up 32%, SMB churn reduced to 4.1%.
Outlook: Q4 pipeline strong at $6.1M weighted, 3 enterprise renewals pending signature.
118 changes: 118 additions & 0 deletions examples/sandboxed-agent/spec.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
open_agent_spec: "1.4.0"

agent:
name: sandboxed-doc-reader
description: >
A read-only document assistant that can only access approved local files
and call approved APIs. Demonstrates OAS Immutable Inference Sandboxing (IIS):
the runner blocks any tool call that violates these constraints before it
reaches the I/O layer — no network call, no file open, no exception to catch.
role: analyst

intelligence:
type: llm
engine: openai
model: gpt-4o-mini

# ── Root-level sandbox ────────────────────────────────────────────────────────
# Applies to every task in this spec unless a task declares its own sandbox block.
sandbox:
tools:
# Only these two tools are permitted. Any other tool (file.write, http.post,
# env.read, …) triggers SANDBOX_TOOL_VIOLATION before the model call.
allow:
- file.read
- http.get
http:
# http.get may only contact the official OpenAI status page.
# Any other domain triggers SANDBOX_DOMAIN_VIOLATION.
allow_domains:
- status.openai.com
file:
# file.read may only access the data/ subdirectory next to this spec.
# Any path outside that prefix triggers SANDBOX_PATH_VIOLATION.
allow_paths:
- ./data/

tools:
reader:
type: native
native: file.read
description: Read a local document file and return its text content.

api_status:
type: native
native: http.get
description: Check the status of an external API endpoint.

tasks:
summarise:
description: >
Read a document from the approved data/ directory and produce a concise summary.
The sandbox ensures the task cannot write files, call unapproved APIs, or
read files outside data/ — even if the model attempts to do so.
tools:
- reader
input:
type: object
properties:
filename:
type: string
description: "Filename inside the data/ directory to summarise (e.g. 'report.txt')."
required:
- filename
output:
type: object
properties:
summary:
type: string
description: A concise summary of the document.
word_count:
type: integer
description: Approximate word count of the original document.
required:
- summary
- word_count
prompts:
system: >
You are a document summariser. You will be given a file to read.
After reading it, return JSON with two fields: "summary" (a concise
paragraph) and "word_count" (integer). Return valid JSON only.
user: "Summarise the file: data/{filename}"

check_status:
description: >
Fetch the OpenAI status page and report whether the API is operational.
This task has a stricter per-task sandbox that also disables file.read,
demonstrating how task-level sandbox overrides the root-level one.
# Per-task sandbox — completely replaces the root sandbox for this task.
sandbox:
tools:
allow:
- api_status
http:
allow_domains:
- status.openai.com
tools:
- api_status
input:
type: object
properties: {}
output:
type: object
properties:
status:
type: string
description: "Current API status (e.g. 'operational', 'degraded', 'outage')."
message:
type: string
description: Human-readable status summary.
required:
- status
- message
prompts:
system: >
You are a system health checker. Fetch the OpenAI status page and return
JSON with "status" (one of: operational, degraded, outage) and "message"
(a one-sentence plain-English summary). Return valid JSON only.
user: "Check https://status.openai.com/api/v2/status.json and report the status."
Loading