Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions developer-guide/protect/observability.mdx
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
---
title: 'Observability'
description: 'Monitor Dome Guardrails with OpenTelemetry, tracing, and logging.'

Check warning on line 3 in developer-guide/protect/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (vijil) - vale-spellcheck

developer-guide/protect/observability.mdx#L3

'Monitor' is too wordy.
---

Dome is OpenTelemetry-compliant and integrates with popular observability platforms. This guide covers tracing, metrics, and logging setup.

If you use [Trust Runtime](/developer-guide/protect/trust-runtime), its structured audit events complement Dome telemetry with Agent identity, tool access, attestation, downgrade, and enforcement-heartbeat decisions. Every event includes the configured `agent_id`; cryptographic identity fields are available only when the runtime is attested.

Check warning on line 8 in developer-guide/protect/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (vijil) - vale-spellcheck

developer-guide/protect/observability.mdx#L8

'is attested' may be passive voice. Use active voice if you can.

## Quick Setup

Install the [OpenTelemetry](https://opentelemetry.io/) support version of Dome using pip or your package manager.
Expand All @@ -15,7 +17,7 @@

[Weave](https://weave-docs.wandb.ai/) is W&B's toolkit for tracing LLM applications. You can use it with Dome to trace all inputs and outputs by wrapping Dome with Weave.

In your Weave dashboard, you'll see:
In your Weave dashboard, you will see:
- All `guard_input` and `guard_output` calls
- Individual Guard execution within each Guardrail
- Detector-level details for each Guard
Expand All @@ -26,7 +28,7 @@

## OpenTelemetry Integration

Dome works with any OpenTelemetry-compliant platform (Jaeger, Uptrace, Signoz, etc.).
Dome works with OpenTelemetry-compliant platforms such as Jaeger, Uptrace, Signoz, etc.

Check warning on line 31 in developer-guide/protect/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (vijil) - vale-spellcheck

developer-guide/protect/observability.mdx#L31

Did you really mean 'Uptrace'?

Check warning on line 31 in developer-guide/protect/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (vijil) - vale-spellcheck

developer-guide/protect/observability.mdx#L31

Did you really mean 'Signoz'?

### Basic Setup

Expand All @@ -48,7 +50,7 @@
- Input and output data
- Execution time
- Error information (if any)
- Whether content was flagged

Check warning on line 53 in developer-guide/protect/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (vijil) - vale-spellcheck

developer-guide/protect/observability.mdx#L53

'was flagged' may be passive voice. Use active voice if you can.

### Span Hierarchy

Expand Down Expand Up @@ -91,11 +93,11 @@

### Log Levels

| Level | What is Logged |

Check warning on line 96 in developer-guide/protect/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (vijil) - vale-spellcheck

developer-guide/protect/observability.mdx#L96

'is Logged' may be passive voice. Use active voice if you can.
|-------|---------------|
| `DEBUG` | Initialization details, parameters |
| `INFO` | Inputs and outputs from each component |
| `WARNING` | Missing config, context window exceeded, timeouts |
| `WARNING` | Missing configuration, context window exceeded, timeouts |
| `ERROR` | Handled exceptions during execution |
| `CRITICAL` | Initialization failures, unhandled exceptions |

Expand All @@ -109,7 +111,7 @@

## Dashboard Examples

### Grafana Dashboard

Check warning on line 114 in developer-guide/protect/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (vijil) - vale-spellcheck

developer-guide/protect/observability.mdx#L114

Did you really mean 'Grafana'?

You can track key metrics such as Blocked Request Rate, Average Latency, and Error Rates.

Expand All @@ -121,9 +123,9 @@

1. **Always instrument in production**: Visibility is essential for security
2. **Set appropriate log levels**: Use INFO for production, DEBUG for troubleshooting
3. **Monitor block rates**: Sudden increases may indicate attacks

Check warning on line 126 in developer-guide/protect/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (vijil) - vale-spellcheck

developer-guide/protect/observability.mdx#L126

'Monitor' is too wordy.

Check warning on line 126 in developer-guide/protect/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (vijil) - vale-spellcheck

developer-guide/protect/observability.mdx#L126

'indicate' is too wordy.
4. **Track latency**: Ensure Guards do not impact user experience
5. **Alert on errors**: Guard failures should be investigated immediately

Check warning on line 128 in developer-guide/protect/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (vijil) - vale-spellcheck

developer-guide/protect/observability.mdx#L128

'be investigated' may be passive voice. Use active voice if you can.

<Card title="Work in Progress" icon="pickaxe" badge="Private preview">
The programmatic observability capabilities and integrations are currently in private preview and subject to change.
Expand All @@ -132,6 +134,9 @@
## Next Steps

<CardGroup cols={2}>
<Card title="Trust Runtime" icon="shield-check" href="/developer-guide/protect/trust-runtime">
Capture identity and policy decisions
</Card>
<Card title="Configure Guardrails" icon="sliders-horizontal" href="/developer-guide/protect/configuring-guardrails">
Guard configuration options
</Card>
Expand Down
194 changes: 110 additions & 84 deletions developer-guide/protect/overview.mdx
Original file line number Diff line number Diff line change
@@ -1,150 +1,176 @@
---
title: 'Protection Overview'
description: 'Guard your agent at runtime with Dome Guardrails.'
description: 'Protect Agent inputs, outputs, and tool use with Dome.'
---

<Tip>
**TL;DR:** Dome wraps your [Agent](/owner-guide/register-agents/what-is-an-agent) with configurable [Guardrails](/concepts/defense/guardrail) that intercept every input and output, blocking attacks before they reach your agent or your users. Configure which [Guards](/concepts/defense/guard) run via your registered Vijil Agent, a TOML file, or a Python dict, then choose early-exit (fast) or parallel execution (thorough) mode.
**TL;DR:** Dome scans [Agent](/owner-guide/register-agents/what-is-an-agent) inputs and outputs with configurable Guardrails. Call `Dome` directly or use an integration to apply the resulting decision. Dome marks flagged content for enforcement in enforce mode, while shadow mode reports detected violations without marking them for enforcement.
</Tip>

Evaluation catches vulnerabilities you know to test for. But attackers will try things you did not anticipate like new prompt injection techniques, novel encoding tricks, social engineering patterns that emerge after your last evaluation.
Evaluation catches vulnerabilities you know to test for. Attackers will also try techniques you did not expect, such as new prompt injections, encoding tricks, and social engineering patterns that emerge after your last evaluation.

Dome is Vijil's runtime protection system. It intercepts every input and output, applies configurable Guardrails, and blocks attacks before they reach your agent or your users. When Diamond identifies vulnerabilities you cannot immediately fix, Dome provides defense-in-depth while you remediate.
Dome protects Agents at runtime. Its content Guardrails scan data before and after Agent execution. Direct `Dome` calls return a `ScanResult`; your application or framework integration decides how to apply the result, such as stopping execution or returning blocked or sanitized content.

For tool-using Agents, [Trust Runtime](/developer-guide/protect/trust-runtime) adds trust controls when you configure them. These controls include SPIFFE workload identity, Console or local constraints, tool-level mandatory access control, signed tool manifest support, tool attestation, structured audit events, and optional enforcement heartbeats. Use `secure_agent()` with LangGraph, Google ADK, or Strands, or use `TrustRuntime` directly for a custom integration.

## Choose a Protection Interface

| Interface | Use It When | Protection |
|-----------|-------------|------------|
| `Dome` | You need to scan Agent inputs and outputs | Content Guardrails |
| `secure_agent()` | You use LangGraph, Google ADK, or Strands | Content Guardrails plus configured identity, framework-dependent tool access control, attestation, and audit events |
| `TrustRuntime` | You need direct control or a custom integration | Individual methods for each configured trust enforcement point |

## How Dome Works

Dome wraps your agent with configurable Guardrails:
A Guardrail contains one or more Guards, and each Guard invokes one or more Detectors. Input and output Guardrails run on opposite sides of Agent execution:

```mermaid actions={false}
%%{init: {'theme':'base', 'themeVariables': {'fontFamily':'Futura Medium, Futura, sans-serif','fontSize':'13px'}, 'flowchart': {'nodeSpacing':25,'rankSpacing':40,'padding':6}}}%%
flowchart LR
subgraph Guardrails[Guardrails]
GuardIn[Guard]
DetectorIn[Detector]
ModelIn[Model]
ActionIn[Action]
ActionOut[Action]
ModelOut[Model]
DetectorOut[Detector]
GuardOut[Guard]
Request[Request] --> InputGuard

subgraph InputGuardrail[Input Guardrail]
direction LR
InputGuard[Guard] --> InputDetector[Detector]
end

Agent((Trusted Agent))
InputDetector --> Agent((Agent))
Agent --> OutputGuard

GuardIn --> DetectorIn --> ModelIn --> ActionIn --> Agent
Agent --> ActionOut --> ModelOut --> DetectorOut --> GuardOut
subgraph OutputGuardrail[Output Guardrail]
direction LR
OutputGuard[Guard] --> OutputDetector[Detector]
end

classDef guard fill:#DE1616,stroke:#2B0C0C,color:#FFFFFF,stroke-width:1px;
classDef redLine fill:#FFFFFF,stroke:#DE1616,color:#DE1616,stroke-width:1.5px;
classDef agent fill:#0247A9,stroke:#DE1616,color:#FFFFFF,stroke-width:3px;
classDef railbox fill:none,stroke:#DE1616,color:#DE1616,stroke-width:1px;
OutputDetector --> Response[Response]

class GuardIn,GuardOut guard;
class DetectorIn,DetectorOut,ModelIn,ModelOut,ActionIn,ActionOut redLine;
classDef endpoint fill:#FFFFFF,stroke:#2B0C0C,color:#2B0C0C,stroke-width:1.5px;
classDef guard fill:#DE1616,stroke:#2B0C0C,color:#FFFFFF,stroke-width:1px;
classDef detector fill:#FFFFFF,stroke:#DE1616,color:#DE1616,stroke-width:1.5px;
classDef agent fill:#0247A9,stroke:#2B0C0C,color:#FFFFFF,stroke-width:1.5px;
classDef railbox fill:none,stroke:#DE1616,color:#DE1616,stroke-width:1px;

class Request,Response endpoint;
class InputGuard,OutputGuard guard;
class InputDetector,OutputDetector detector;
class Agent agent;
class Guardrails railbox;
class InputGuardrail,OutputGuardrail railbox;
```

| Component | Purpose |
|-----------|---------|
| **[Guardrail](/concepts/defense/guardrail)** | Pipeline of Guards (input or output) |
| **[Guard](/concepts/defense/guard)** | Group of Detectors of one type |
| **[Guardrail](/concepts/defense/guardrail)** | Input or output pipeline containing Guards |
| **[Guard](/concepts/defense/guard)** | Group of Detectors for one protection category |
| **[Detector](/concepts/defense/detector)** | Individual detection method |

## Protection Types

### Security Guards

Detect and block adversarial attacks:

| Detector | What It Catches |
|----------|-----------------|
| `prompt-injection-mbert` | Injected instructions in user input |
| `prompt-injection-deberta-v3-base` | Advanced injection attempts |
| `encoding-heuristics` | Base64, Unicode, and encoding attacks |
| `security-embeddings` | Semantic similarity to known attacks |

### Moderation Guards
Dome supports six Guard categories. The following table shows representative registered Detectors rather than the complete catalog:

Filter harmful and inappropriate content:
| Guard Category | Purpose | Representative Detectors |
|----------------|---------|--------------------------|
| **Security** | Detect prompt injection, jailbreaks, and obfuscated payloads | `prompt-injection-mbert`, `encoding-heuristics` |
| **Moderation** | Detect harmful content and match configured keyword lists | `moderation-deberta`, `moderation-flashtext` |
| **Privacy** | Detect PII and credentials | `privacy-presidio`, `detect-secrets` |
| **Integrity** | Check factual consistency and hallucinations | `hhem-hallucination`, `fact-check-roberta` |
| **Generic** | Apply a configurable LLM classifier | `generic-llm` |
| **Policy** | Check content against configured policy sections | `policy-sections` |

| Detector | What It Catches |
|----------|-----------------|
| `moderation-flashtext` | Fast keyword-based toxicity |
| `moderation-deberta` | Neural toxicity classification |
| `moderations-oai-api` | OpenAI Moderation API |
| `moderation-llamaguard` | Llama Guard safety model |
<Info>
See [Detection Methods](/developer-guide/protect/detection-methods) for the full Detector catalog and configuration parameters.
</Info>

### Privacy Guards
## Default Guardrails

Prevent exposure of sensitive data:
`Dome()` creates input and output Guardrails with the following default checks:

| Detector | What It Catches |
|----------|-----------------|
| `privacy-presidio` | PII (names, emails, SSN, etc.) |
| `detect-secrets` | API keys, passwords, credentials |
| Direction | Default Checks |
|-----------|----------------|
| **Input** | Encoding heuristics, prompt injection detection, moderation |
| **Output** | Moderation, PII detection |

## Quick Start

You can protect your agents with default Guards. The default configuration includes:
- **Input**: Prompt injection detection, encoding heuristics, moderation
- **Output**: Moderation, PII detection
The default prompt injection and moderation models require the `local` extra when you run them locally, and Presidio PII detection requires the `pii` extra. You can instead set `DOME_INFERENCE_URL` to route supported Detectors, including the default model and PII Detectors, to an inference service.

## Configuration Sources

Dome accepts configuration from three sources:
Choose a configuration source that matches how you operate Dome:

| Source | How to Use | Best For |
|--------|-----------|----------|
| **Registered Agent** | Pull from your Vijil Console agent definition | Production: keeps configuration in sync with your policy |
| **TOML file** | Reference a local `.toml` file at initialization | Version-controlled configuration in CI/CD |
| **Python dict** | Pass a `dict` at runtime | Dynamic configuration, testing, prototyping |
|--------|------------|----------|
| **Built-in defaults** | Initialize `Dome()` without a configuration | Initial setup and evaluation |
| **Local configuration** | Pass a Python dictionary or TOML path to `Dome` | Version-controlled or dynamic configuration |
| **Registered Agent** | Use `Dome.create_from_vijil_agent()` | Console-managed Agent configuration |
| **Evaluation recommendation** | Use `Dome.create_from_vijil_evaluation()` | Guardrails derived from evaluation results |
| **S3** | Use `Dome.create_from_s3()` with the `s3` extra | Remotely managed configuration in AWS |

For the full list of configuration options, see [Configure Guardrails](/developer-guide/protect/configuring-guardrails).
<Info>
For the available settings, see [Configure Guardrails](/developer-guide/protect/configuring-guardrails).

Check warning on line 108 in developer-guide/protect/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (vijil) - vale-spellcheck

developer-guide/protect/overview.mdx#L108

Always capitalize this term: write 'guardrails' with an uppercase first letter.
</Info>

## Scan Results

Every scan returns a result object with the following fields:
Every input or output scan returns a `ScanResult` with these key members:

| Field | Type | Description |
|-------|------|-------------|
| `is_safe` | Boolean | `True` if content passed all Guards |
| `flagged` | Boolean | `True` if any Guard flagged the content |
| `response_string` | String | Original content if safe; fallback message if blocked |
| `exec_time` | Float | Scan duration in milliseconds |
| `trace` | Dict | Per-Guard and per-Detector execution details |
| Member | Type | Description |
|--------|------|-------------|
| `flagged` | `bool` | Whether any Guard flagged the content |
| `enforced` | `bool` | Whether Dome marks a flagged result for enforcement |
| `response_string` | `str` | Original, blocked, or sanitized content returned by the Guardrail |
| `detection_score` | `float` | Highest triggered detection score, clamped between `0.0` and `1.0` |
| `triggered_methods` | `list[str]` | Detectors that flagged the content |
| `errored_methods` | `list[str]` | Detectors or Guard tasks that returned errors or timed out |
| `trace` | `dict` | Per-Guard and per-Detector execution details |
| `exec_time` | `float` | Total scan duration in seconds |
| `is_safe()` | Method returning `bool` | Returns `True` when `flagged` is `False`, regardless of enforcement mode |

## Framework Integrations

Dome is designed to be integrable with popular frameworks and runtimes. You can see the specific framework developer guides for integration patterns during the developer access phase.
You can call `Dome.guard_input()` and `Dome.guard_output()` from any Agent framework. For the complete trust stack, `secure_agent()` provides built-in adapters for LangGraph, Google ADK, and Strands.

Tool enforcement depends on the integration points exposed by each framework. LangGraph tool access control is best-effort because the compiled graph must expose tool functions that the adapter can discover and wrap. LangGraph checks streaming output only after it yields the chunks, so an output Guard cannot retract delivered chunks.

<Info>
See [Trust Runtime](/developer-guide/protect/trust-runtime) for integration examples and constraints.
</Info>

## Execution and Failure Options

## Performance Options
Early exit and parallel execution are independent options at both the Guardrail and Guard levels:

| Mode | Behavior | Best For |
|------|----------|----------|
| **Early exit** (default) | Stops after the first Guard flags content | Rejecting clearly malicious input with minimal latency |
| **Complete execution** | Runs all Guards regardless of flags | Comprehensive audit trails, maximum detection coverage |
| **Parallel execution** | Runs Guards concurrently | High-throughput production environments |
| Option | Behavior |
|--------|----------|
| **Early exit** | Stops after the first configured Guard or Detector flags content |
| **Parallel execution** | Runs configured Guards or Detectors concurrently and can reduce scan wall-clock latency |

See [Configure Guardrails](/developer-guide/protect/configuring-guardrails) for the configuration options for each mode.
By default, Guards and Guardrails use early exit and do not use parallel execution. Disable early exit to run every configured check. You can also combine early exit with parallel execution; Dome cancels pending checks after one flags content.

When configuration omits `on-error` settings, `Dome(enforce=True)` defaults to `fail_closed`, while `Dome(enforce=False)` defaults to `fail_open`. Explicit Guardrail-level or Guard-level `on-error` values override the inherited default. Failed methods remain available in `errored_methods` for inspection.

<Info>
See [Configure Guardrails](/developer-guide/protect/configuring-guardrails) for the settings at each level.

Check warning on line 151 in developer-guide/protect/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (vijil) - vale-spellcheck

developer-guide/protect/overview.mdx#L151

Always capitalize this term: write 'guardrails' with an uppercase first letter.
</Info>

<Card title="Work in Progress" icon="pickaxe" badge="Private preview">
The programmatic protection capabilities are currently in private preview and subject to change.
</Card>

## Next Steps

<CardGroup cols={2}>
<Card title="Trust Runtime" icon="shield-check" href="/developer-guide/protect/trust-runtime">
Add identity, tool access control, attestation, and audit events
</Card>
<Card title="Configure Guardrails" icon="sliders-horizontal" href="/developer-guide/protect/configuring-guardrails">
Detailed Guard configuration options
Configure Guards, Detectors, execution, and failure behavior
</Card>
<Card title="Use Guardrails" icon="train-track" href="/developer-guide/protect/using-guardrails">
Runtime patterns and best practices
Apply Guardrails at runtime
</Card>
<Card title="Detection Methods" icon="radar" href="/developer-guide/protect/detection-methods">
Review built-in Detectors and their parameters
</Card>
<Card title="Custom Detectors" icon="wrench" href="/developer-guide/protect/custom-detectors">
Build your own detection methods
</Card>
<Card title="Observability" icon="eye" href="/developer-guide/protect/observability">
Monitoring and tracing setup
Track Guardrail decisions and traces
</Card>
</CardGroup>
Loading