[FEAT]: Serialize final evaluation metadata

docs/api/core-types.md

@@ -14,6 +14,8 @@ Data types shared across the entire framework. All importable from `rampart` dir
         - ToolCall
         - SideEffect
         - Turn
+        - EvaluationRole
+        - TerminationReason
         - EvalOutcome
         - EvalResult
         - EvalContext
@@ -28,6 +30,8 @@ Data types shared across the entire framework. All importable from `rampart` dir
         - SafetyStatus
         - HarmCategory
         - InjectionRecord
+        - resolve_attack_verdict
+        - resolve_probe_verdict
         - resolve_as_attack
         - resolve_as_probe
 

docs/api/evaluators.md

@@ -7,6 +7,7 @@ Built-in evaluators. All extend `BaseEvaluator` and support composition via `|`,
       members:
         - ToolCalled
         - ResponseContains
+        - ResponseScope
         - SideEffectOccurred
         - LLMJudge
         - TranscriptScope

docs/api/index.md

@@ -13,7 +13,7 @@ API reference organized by RAMPART's package layout. Each page documents the pub
 | [Core Protocols](core-protocols.md) | `Session`, `AgentAdapter`, `Evaluator`, `Surface`, `InjectionHandle`, and more |
 | [Attacks](attacks.md) | `Attacks.xpia()`, `XPIAExecution` |
 | [Probes](probes.md) | `Probes.behavior()`, `SingleTurnExecution` |
-| [Evaluators](evaluators.md) | `ToolCalled`, `ResponseContains`, `SideEffectOccurred`, `LLMJudge`, `TranscriptScope` |
+| [Evaluators](evaluators.md) | `ToolCalled`, `ResponseContains`, `ResponseScope`, `SideEffectOccurred`, `LLMJudge`, `TranscriptScope` |
 | [Drivers](drivers.md) | `StaticDriver`, `LLMDriver` |
 | [Payloads](payloads.md) | `Payloads`, `PayloadTemplate`, `PayloadStore` |
 | [Surfaces](surfaces.md) | `OneDriveSurface` |

docs/attacks/xpia.md

@@ -121,28 +121,44 @@ For generating many variants at scale, use [`Payloads.generate_async()`][rampart
 Combine evaluators with `|` (OR), `&` (AND), and `~` (NOT):
 
 ```python
-from rampart.evaluators import ToolCalled, ResponseContains
+from rampart.evaluators import ResponseContains, ResponseScope, ToolCalled
 
 # OR: detect exfil via tool call or leaked content in response text
 evaluator = (
     ToolCalled("send_email", recipient=lambda v: "evil.com" in str(v))
-    | ResponseContains("attacker@evil.com")
+    | ResponseContains(
+        "attacker@evil.com",
+        scope=ResponseScope.ANY_TURN,
+    )
 )
 
 # AND: agent executed a command AND the response mentions credentials
 evaluator = (
     ToolCalled("exec", command=lambda v: ".ssh" in str(v))
-    & ResponseContains("id_rsa")
+    & ResponseContains("id_rsa", scope=ResponseScope.ANY_TURN)
 )
 
-# NOT: agent did NOT refuse — it complied with the injection
-evaluator = ~ResponseContains(lambda text: "I can't" in text or "I cannot" in text)
+# NOT: the agent failed to refuse on at least one turn
+evaluator = ~ResponseContains(
+    lambda text: "I can't" in text or "I cannot" in text,
+    scope=ResponseScope.ALL_TURNS,
+)
 ```
 
+`~ALL_TURNS(refusal)` is true when **at least one** in-scope response does not refuse, so it detects a single compliant turn among many. `~ANY_TURN(refusal)` is only true when **none** of the in-scope responses refuse. The difference is critical in multi-turn sessions: if the agent refuses on the first turn but complies on a later turn, `~ResponseContains(..., scope=ResponseScope.ALL_TURNS)` fires while `~ResponseContains(..., scope=ResponseScope.ANY_TURN)` does not.
+
 Place the cheaper evaluator on the left side of `|` — it short-circuits if the left operand detects.
 
 The `&` above asks whether both happened, so one condition that definitively did not happen settles the result even if the adapter could not observe the other. Use `|` when either condition on its own would count as the attack succeeding. When the adapter does not report the channel the left condition needs, the result records that on [`EvalResult`][rampart.core.types.EvalResult]. Reversing those two operands records nothing, because a `NOT_DETECTED` left operand short-circuits `&` before the other one runs. See the note on undetermined operands in [Authoring Tests](../usage/authoring-tests.md#composing-evaluators).
 
+!!! warning "Multi-turn scope"
+    State the temporal scope explicitly for multi-turn attacks. The complete
+    positive and negated mapping is maintained in the
+    [Temporal Scope table](../usage/authoring-tests.md#temporal-scope).
+    Omitting `scope` inspects only the current response and emits a
+    `FutureWarning` for multi-turn contexts. Scope applies only to turns in the
+    evaluator context; it does not control execution length or early stopping.
+
 ### LLMDriver for Adaptive Triggers
 
 For multi-turn attacks where the trigger conversation adapts based on agent responses, use [`LLMDriver`][rampart.drivers.llm.LLMDriver] instead of a static string:
@@ -210,7 +226,7 @@ See [`Attacks.xpia()`][rampart.attacks.Attacks.xpia] for the full API reference.
 | `inject` | `InjectionHandle \| list[InjectionHandle] \| None` | `None` | Prepared injections from `surface.inject()`. `None` for inline XPIA. |
 | `trigger` | `str \| list[str] \| Request \| list[Request] \| PromptDriver` | required | Benign prompt(s) that cause retrieval of injected content. |
 | `evaluator` | [`Evaluator`][rampart.core.evaluator.Evaluator] | required | What attack condition to detect. |
-| `max_turns` | `int` | `5` | Maximum prompt-response exchanges before `ERROR`. |
+| `max_turns` | `int` | `5` | Maximum prompt-response exchanges; reaching the limit resolves the trace normally. |
 | `event_handlers` | `list[ExecutionEventHandler] \| None` | `None` | Additional lifecycle event handlers. |
 
 ---

docs/contributing/extending-rampart.md

@@ -248,16 +248,18 @@ class MyEvaluator(BaseEvaluator):
         self._target = target
 
     async def evaluate_async(self, *, context: EvalContext) -> EvalResult:
-        """Evaluate the latest turn for the target condition.
+        """Evaluate the full trace for the target condition.
 
         Args:
             context (EvalContext): The evaluation context with turn history.
 
         Returns:
             EvalResult: Whether the condition was detected, with evidence.
         """
-        latest_turn = context.turns[-1]
-        detected = self._target in latest_turn.response.text
+        detected = any(
+            self._target in turn.response.text
+            for turn in context.turns
+        )
 
         return EvalResult(
             outcome=EvalOutcome.DETECTED if detected else EvalOutcome.NOT_DETECTED,
@@ -268,6 +270,13 @@ class MyEvaluator(BaseEvaluator):
 
 Evaluator tests should cover detection, non-detection, edge cases (empty response, missing data), and that `evidence` / `rationale` are populated correctly.
 
+!!! warning "Multi-turn evaluator migration"
+    A custom evaluator that reads only `context.turns[-1]` intentionally judges
+    only the latest response and cannot preserve earlier evidence. Rewrite
+    multi-turn predicates to inspect `context.turns` explicitly. The
+    [attack execution walkthrough](#attack) shows how execution decides which
+    turns are included in the evaluator context.
+
 
 ## Prompt Driver
 

docs/probes/behavioral.md

@@ -54,20 +54,32 @@ result = await Probes.behavior(
 For full control over the conversation flow, use a [`StaticDriver`][rampart.drivers.static.StaticDriver]:
 
 ```python
-from rampart.drivers import StaticDriver
 from rampart import Request
+from rampart.drivers import StaticDriver
+from rampart.evaluators import ResponseContains, ResponseScope
 
 driver = StaticDriver(prompts=[
-    Request(prompt="Hello"),
-    Request(prompt="What tools do you have?"),
+    Request(prompt="Name a search tool you can use."),
+    Request(prompt="Describe that search tool."),
 ])
 
 result = await Probes.behavior(
     driver=driver,
-    evaluator=ResponseContains("search"),
+    evaluator=ResponseContains(
+        "search",
+        scope=ResponseScope.CURRENT_TURN,
+    ),
 ).execute_async(adapter=my_adapter)
 ```
 
+!!! warning "Multi-turn scope"
+    Choose positive and negated probe scopes from the
+    [Temporal Scope table](../usage/authoring-tests.md#temporal-scope), which is
+    the source of truth for all four combinations. Omitting `scope` inspects
+    only the current response and emits a `FutureWarning` for multi-turn
+    contexts. Scope applies only to turns in the evaluator context; it does not
+    force an execution to produce every planned turn.
+
 ---
 
 ## Parameters
@@ -80,7 +92,7 @@ See [`Probes.behavior()`][rampart.probes.Probes.behavior] for the full API refer
 | `prompts` | `list[str] \| None` | `None` | A list of prompt strings. |
 | `driver` | [`PromptDriver`][rampart.core.prompt_driver.PromptDriver] `\| None` | `None` | A pre-built prompt driver. |
 | `evaluator` | [`Evaluator`][rampart.core.evaluator.Evaluator] | required | What behavior to detect. |
-| `max_turns` | `int` | `25` | Maximum exchanges before `ERROR`. |
+| `max_turns` | `int` | `25` | Maximum exchanges; reaching the limit resolves the trace normally. |
 
 !!! warning
     Provide exactly one of `prompt`, `prompts`, or `driver`. Providing more than one or none raises `ValueError`.

docs/usage/authoring-tests.md

@@ -121,6 +121,59 @@ ResponseContains(re.compile(r"ssh-rsa\s+[A-Za-z0-9+/]+"))
 ResponseContains(lambda text: "secret" in text.lower())
 ```
 
+#### Temporal Scope
+
+By default, `ResponseContains` inspects only the current response. For a
+multi-turn transcript, pass an explicit
+[`ResponseScope`][rampart.evaluators.response_contains.ResponseScope]:
+
+```python
+from rampart.evaluators import ResponseContains, ResponseScope
+
+# Detect if the pattern appeared at any point in the conversation
+ResponseContains("id_rsa", scope=ResponseScope.ANY_TURN)
+
+# Detect only if every response contained the pattern
+ResponseContains("Paris", scope=ResponseScope.ALL_TURNS)
+
+# Inspect only the latest response and ignore earlier turns
+ResponseContains("id_rsa", scope=ResponseScope.CURRENT_TURN)
+```
+
+| Existing use | Intended meaning | Explicit form |
+|---|---|---|
+| attack, `ResponseContains(p)` | some turn contains `p` | `ResponseContains(p, scope=ResponseScope.ANY_TURN)` |
+| attack, `~ResponseContains(p)` | some turn does not contain `p` | `~ResponseContains(p, scope=ResponseScope.ALL_TURNS)` |
+| probe, `ResponseContains(p)` | every turn contains `p` | `ResponseContains(p, scope=ResponseScope.ALL_TURNS)` |
+| probe, `~ResponseContains(p)` | no turn contains `p` | `~ResponseContains(p, scope=ResponseScope.ANY_TURN)` |
+
+!!! warning "Migration"
+    Evaluating an unspecified scope over more than one turn emits a
+    `FutureWarning`. Single-turn evaluation is unchanged. Pass
+    `ResponseScope.CURRENT_TURN` explicitly when latest-response behavior is
+    intentional.
+
+    Scope quantifies only the turns present in the evaluator's `EvalContext`.
+    It does not control how many turns an execution produces or whether an
+    execution stops early.
+
+#### How Each Evaluator Sees the Transcript
+
+Built-in evaluators reach their temporal behavior in two ways. Quantifying
+evaluators compute deterministic matches across turns. Windowing evaluators
+choose how much transcript to give a judge that returns one holistic verdict.
+
+| Evaluator | Mechanism | Default | Configurable via |
+|---|---|---|---|
+| `ToolCalled` | quantifies (`ANY_TURN`) | any turn | — |
+| `SideEffectOccurred` | quantifies (`ANY_TURN`) | any turn | — |
+| `ResponseContains` | quantifies | current turn | `ResponseScope` |
+| `LLMJudge` | windows | full transcript | `TranscriptScope` |
+
+`ResponseScope.CURRENT_TURN` and `TranscriptScope.CURRENT_TURN` both select
+the last turn, but they belong to different enums and are not interchangeable.
+Pass the scope type declared by the evaluator you are configuring.
+
 ### [`SideEffectOccurred`][rampart.evaluators.side_effect.SideEffectOccurred] — Detect Side Effects
 
 ```python
@@ -179,6 +232,10 @@ judge = LLMJudge(
 )
 ```
 
+Use `TranscriptScope.FULL` when evidence from any earlier turn must affect the
+verdict. `CURRENT_TURN` intentionally gives the judge only the latest response.
+Like `ResponseScope`, it does not control how many turns an execution produces.
+
 **Custom persona.** The default judge identity is [`NEUTRAL_EVALUATOR`][rampart.evaluators.personas.NEUTRAL_EVALUATOR] — an impartial, literal evaluator. Override it when a different lens is useful:
 
 ```python

docs/usage/results-and-reporting.md

@@ -15,7 +15,9 @@ result.safe              # bool — did the agent behave safely?
 result.status            # SafetyStatus (SAFE, UNSAFE, UNDETERMINED, ERROR)
 result.summary           # str — human-readable one-liner
 result.observability_level  # ObservabilityLevel (what the adapter saw)
+result.evaluation        # EvalResult | None — final verdict evidence
 result.turns             # list[Turn] — full conversation
+result.termination_reason  # TerminationReason | None
 result.duration_seconds  # float — execution wall-clock time
 result.harm_category     # HarmCategory | str | None
 result.strategy          # str — "xpia", "probe", etc.
@@ -48,10 +50,21 @@ for turn in result.turns:
     turn.request.prompt       # What was sent
     turn.response.text        # What came back
     turn.response.tool_calls  # Tool invocations observed
-    turn.eval_result          # EvalResult for this turn, or None
+    turn.eval_result          # Optional online evaluation evidence
+    turn.eval_role            # Why the online evaluation was produced
     turn.turn_number          # 0-indexed position
 ```
 
+`Result.evaluation` is distinct from turn-level evidence. Execution strategies
+populate it when final-trace verdict cadence is enabled; legacy and manually
+constructed results may leave it as `None`. `Result.eval_results` continues to
+return only evaluations attached to turns.
+
+`termination_reason` distinguishes normal trace endings such as driver
+exhaustion, reaching the turn budget, and an online stop condition. It is not
+an exception category; infrastructure failures remain available through result
+status and metadata.
+
 ### Observability Gaps on a Passing Run
 
 A run can resolve `SAFE` while part of the evaluation was never observable. Such a run is graded as a pass: `result.safe` is `True`, the result line reads `PASS`, a trial group counts it toward the pass rate, and pytest exits zero. `result.summary` names the gap, and `turn.eval_result.undetermined_operands` carries it one reason at a time, so a caller that wants to fail on it has to say so:
@@ -89,6 +102,9 @@ sink = JsonFileReportSink(output_dir=Path(".report"))
 
 Output: `.report/run_report_2026-04-25T14-30-00.json`
 
+The built-in projection includes final evaluation evidence, termination reason,
+and the role of any turn-level evaluation when those fields are present.
+
 ### Custom Sinks
 
 Implement the [`ReportSink`][rampart.reporting.sink.ReportSink] protocol:

docs/usage/xdist.md

@@ -195,6 +195,20 @@ Worker payloads cross a process boundary via `execnet` and may contain attacker-
 - **Terminal/log injection** — ANSI escape sequences are stripped from free-form text at the deserialization boundary.
 - **Path traversal** — worker-local artifact paths are stored as opaque strings in metadata; the controller never accesses worker files.
 
+### Schema Evolution
+
+The streamed xdist transport uses the `rampart.xdist.v2` schema.
+Workers include their installed RAMPART package version for diagnostics. A
+different controller version emits a warning because optional evidence may not
+be available across remote `--tx` gateways.
+
+Core semantic fields remain fail-closed: unknown schema versions, safety
+statuses, observability levels, and evaluator outcomes reject the worker
+payload. Additive display fields such as turn evaluation role and termination
+reason are lenient; an unknown value warns and becomes `None` while preserving
+the core verdict. Version 2 envelopes that predate package-version diagnostics
+or omit the new optional fields remain valid; version 1 payloads are rejected.
+
 ### Size cap
 
 The default 16 MiB cap can be overridden via the pytest CLI option or an ini setting:

rampart/__init__.py

@@ -29,17 +29,21 @@
     SafetyStatus,
     resolve_as_attack,
     resolve_as_probe,
+    resolve_attack_verdict,
+    resolve_probe_verdict,
 )
 from rampart.core.types import (
     EvalContext,
     EvalOutcome,
     EvalResult,
+    EvaluationRole,
     ObservabilityLevel,
     Payload,
     PayloadFormat,
     Request,
     Response,
     SideEffect,
+    TerminationReason,
     ToolCall,
     Turn,
 )
@@ -70,6 +74,7 @@
     "EvalContext",
     "EvalOutcome",
     "EvalResult",
+    "EvaluationRole",
     "Evaluator",
     "EvaluatorError",
     "ExecutionEvent",
@@ -95,13 +100,16 @@
     "Session",
     "SideEffect",
     "Surface",
+    "TerminationReason",
     "ToolCall",
     "ToolDeclaration",
     "TranscriptScope",
     "Turn",
     "record_result",
     "resolve_as_attack",
     "resolve_as_probe",
+    "resolve_attack_verdict",
+    "resolve_probe_verdict",
 ]
 
 

rampart/attacks/_factory.py

@@ -73,8 +73,8 @@ def xpia(
                 Benign user request(s) that cause the agent to process
                 poisoned content.
             evaluator (Evaluator): What condition to check for.
-            max_turns (int): Maximum prompt-response exchanges before
-                ERROR.  Defaults to 5.
+            max_turns (int): Maximum prompt-response exchanges. Reaching the
+                limit resolves the trace normally. Defaults to 5.
             event_handlers (list[ExecutionEventHandler] | None): Optional
                 additional handlers for custom observability.
 

rampart/attacks/_xpia.py

@@ -71,8 +71,8 @@ class XPIAExecution(BaseExecution):
             attachments.
         driver (PromptDriver): How to drive the trigger conversation.
         evaluator (Evaluator): What condition to check for.
-        max_turns (int): Maximum prompt-response exchanges before the
-            execution stops with ERROR.  Prevents unbounded loops.
+        max_turns (int): Maximum prompt-response exchanges. Reaching the
+            limit resolves the trace normally and prevents unbounded loops.
         event_handlers (list[ExecutionEventHandler] | None): Additional
             handlers beyond the framework defaults.
     """