feat(evaluation): add artifact evolution recording

Makefile

@@ -60,6 +60,7 @@ e2e-test: ## Run CLI to Client SDK to Server end-to-end tests.
 evaluation-unit-test: ## Run the evaluation project's unit tests, the work-continuity benchmark included.
 	@uv sync --project evaluation --frozen
 	@uv run --project evaluation pytest -c evaluation/pyproject.toml \
+		evaluation/artifact_evolution/tests \
 		evaluation/coding/swebench_pro/tests/unit \
 		evaluation/coding/work_continuity/tests/unit \
 		evaluation/memory/longmemeval_v2/tests -m "not live" -q

evaluation/README.md

@@ -10,11 +10,16 @@ Python environment and build configuration.
 | [`memory/`](memory/README.md) | Memory retrieval, answer quality, and write-gate evidence | [LoCoMo](memory/locomo/README.md), [LoCoMo-Plus](memory/locomo_plus/README.md), [LongMemEval-V2](memory/longmemeval_v2/README.md), [Memory write gate](memory/write_gate/README.md) |
 | [`performance/`](performance/README.md) | Load, latency, storage growth, and capacity limits | None yet |
 | [`skills/`](skills/README.md) | Agent instruction routing and authorization regressions | [Claude Code skill-up](skills/skill-up/README.md), [ZCode native CLI](zcode_guidance/README.md) |
+| [`artifact_evolution/`](artifact_evolution/README.md) | Evaluation-owned Dream recording and faithful replay | [Artifact evolution](artifact_evolution/README.md) |
 
 ```text
 evaluation/
 ├── pyproject.toml                 # Shared powercontext-eval environment and build
 ├── uv.lock
+├── artifact_evolution/             # Artifact-evolution recording and faithful replay
+│   ├── src/powercontext_eval_artifact_evolution/
+│   ├── tests/
+│   └── README.md
 ├── coding/
 │   ├── swebench_pro/
 │   │   ├── src/powercontext_eval_swebench_pro/
@@ -89,9 +94,10 @@ uv run --directory evaluation ty check
 uv build --project evaluation
 ```
 
-The `powercontext-eval` distribution contains three packages: `powercontext_eval_swebench_pro`,
-`powercontext_eval_work_continuity`, and `powercontext_eval_longmemeval_v2`. Editable installs and wheels expose the
-independent `swebench-pro`, `work-continuity`, and `longmemeval-v2` commands and their suite modules.
+The `powercontext-eval` distribution contains four packages: `powercontext_eval_artifact_evolution`,
+`powercontext_eval_swebench_pro`, `powercontext_eval_work_continuity`, and `powercontext_eval_longmemeval_v2`.
+Editable installs and wheels expose the independent `artifact-evolution`, `swebench-pro`, `work-continuity`, and
+`longmemeval-v2` commands and their suite modules.
 LoCoMo behavior tests remain under [`tests/`](../tests/), including [`tests/evaluation/`](../tests/evaluation/).
 Skill-up and ZCode guidance keep their own validation tests. The [Bub replay harness](../e2e/bub/README.md) and
 [product E2E tests](../tests/e2e/) cover cross-component acceptance.

evaluation/artifact_evolution/README.md

@@ -0,0 +1,60 @@
+# Artifact-evolution evaluation
+
+This suite owns the evaluation-only recording and faithful replay contract for
+Experience Dream evolution. It does not extend Runtime, add database tables, or
+create Candidates. The bundle stores exact references, digests, realized
+results, deterministic validation results, nomination state, and saved cost.
+
+The versioned bundle is `powercontext.artifact-evolution/v1`. Controlled capture
+is explicit; ordinary production capture is not enabled by this suite. Prompt,
+task, and evidence bodies are not part of the contract. Optional output payloads
+require either the deterministic `redacted` policy (sensitive keys and
+credential-shaped strings are replaced before persistence) or an `approved`
+policy with an explicit `approval_ref`. The recorder stores a structured
+`scope_id`/`run_id` reference to the terminal Dream run; it does not copy the
+Runtime row or create a Candidate.
+
+## Commands
+
+Validate a bundle without calling a model or writing Runtime state:
+
+```bash
+uv run --project evaluation artifact-evolution validate path/to/bundle.json
+```
+
+Replay one exact action using the saved result only:
+
+```bash
+uv run --project evaluation artifact-evolution replay path/to/bundle.json \
+  --decision-id decision-1 \
+  --trajectory-id trajectory-1 \
+  --observations-digest 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef \
+  --action-signature refine-experience:v1 \
+  --ordering 0
+```
+
+Report exact replay coverage and saved generation cost:
+
+```bash
+uv run --project evaluation artifact-evolution coverage path/to/bundle.json
+```
+
+Record one controlled terminal `refine_experience` Dream snapshot into a new
+bundle. The input is a JSON serialization of the existing Runtime `DreamRun`
+shape; the evaluation package only reads the fields it needs and computes the
+observation and output digests.
+
+```bash
+uv run --project evaluation artifact-evolution record \
+  base-bundle.json dream-run.json recorded-bundle.json \
+  --action-signature refine-experience:v1 \
+  --policy-id baseline-policy-v1 \
+  --generator-id dream.refine_experience \
+  --runtime-version runtime-v1
+```
+
+An absent or mismatched action returns `out_of_support`. Replay never runs a
+generator or evaluator and never mutates Runtime, Candidate, Review, or Artifact
+state. Exact replay support is identity-level: the trajectory, decision id,
+observation digest, action signature, and ordering must all match. Coverage
+therefore reports unsupported decision ids and is not a cross-policy comparison.

evaluation/artifact_evolution/ruff.toml

@@ -0,0 +1,16 @@
+# Copyright (c) 2026 OceanBase.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+target-version = "py311"
+line-length = 120

evaluation/artifact_evolution/src/powercontext_eval_artifact_evolution/__init__.py

@@ -0,0 +1,59 @@
+# Copyright (c) 2026 OceanBase.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Evaluation-owned recording and faithful replay for Artifact evolution."""
+
+from powercontext_eval_artifact_evolution.bundle import (
+    BundleIOError,
+    BundleValidationError,
+    load_bundle,
+    record_attempt,
+    write_bundle,
+)
+from powercontext_eval_artifact_evolution.models import (
+    ArtifactEvolutionBundle,
+    DecisionRequest,
+    DreamRunRef,
+    ProposalAttempt,
+)
+from powercontext_eval_artifact_evolution.recording import (
+    DreamRecordingError,
+    DreamRunSnapshot,
+    decision_for_dream_run,
+    record_dream_attempt,
+    snapshot_from_dream_run,
+)
+from powercontext_eval_artifact_evolution.replay import (
+    ReplayEngine,
+    ReplayResult,
+)
+
+__all__ = [
+    "ArtifactEvolutionBundle",
+    "BundleIOError",
+    "BundleValidationError",
+    "DecisionRequest",
+    "DreamRecordingError",
+    "DreamRunRef",
+    "DreamRunSnapshot",
+    "ProposalAttempt",
+    "ReplayEngine",
+    "ReplayResult",
+    "decision_for_dream_run",
+    "load_bundle",
+    "record_attempt",
+    "record_dream_attempt",
+    "snapshot_from_dream_run",
+    "write_bundle",
+]

evaluation/artifact_evolution/src/powercontext_eval_artifact_evolution/bundle.py

@@ -0,0 +1,116 @@
+# Copyright (c) 2026 OceanBase.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Read, write, and append operations for artifact-evolution bundles."""
+
+from __future__ import annotations
+
+import json
+import os
+import tempfile
+from pathlib import Path
+
+from pydantic import ValidationError
+
+from powercontext_eval_artifact_evolution.models import ArtifactEvolutionBundle, ProposalAttempt
+from powercontext_eval_artifact_evolution.privacy import digest_json, redact_payload
+
+
+class BundleIOError(ValueError):
+    """A bundle could not be read or written without losing its contract."""
+
+
+class BundleValidationError(BundleIOError):
+    """A bundle is not valid `powercontext.artifact-evolution/v1` JSON."""
+
+
+def load_bundle(path: Path) -> ArtifactEvolutionBundle:
+    """Load and validate a UTF-8 JSON bundle without invoking any evaluator."""
+
+    resolved = path.resolve()
+    try:
+        raw = resolved.read_bytes()
+    except OSError as error:
+        raise BundleIOError(f"cannot read artifact-evolution bundle: {resolved}") from error
+    if not raw.strip():
+        raise BundleValidationError(f"artifact-evolution bundle is blank: {resolved}")
+    try:
+        value = json.loads(raw.decode("utf-8"))
+    except (UnicodeDecodeError, json.JSONDecodeError) as error:
+        raise BundleValidationError("artifact-evolution bundle must be UTF-8 JSON") from error
+    if not isinstance(value, dict):
+        raise BundleValidationError("artifact-evolution bundle must be a JSON object")
+    try:
+        return ArtifactEvolutionBundle.model_validate_json(raw)
+    except ValidationError as error:
+        raise BundleValidationError(f"invalid artifact-evolution bundle: {error}") from error
+
+
+def write_bundle(bundle: ArtifactEvolutionBundle, path: Path) -> Path:
+    """Write one bundle, refusing to overwrite a previously recorded artifact."""
+
+    target = path.resolve()
+    try:
+        bundle = ArtifactEvolutionBundle.model_validate(bundle.model_dump(mode="python", by_alias=True))
+    except ValidationError as error:
+        raise BundleValidationError(f"cannot write invalid artifact-evolution bundle: {error}") from error
+    temporary: Path | None = None
+    try:
+        target.parent.mkdir(parents=True, exist_ok=True)
+        descriptor, temporary_name = tempfile.mkstemp(prefix=f".{target.name}.", dir=target.parent)
+        temporary = Path(temporary_name)
+        with os.fdopen(descriptor, "w", encoding="utf-8") as stream:
+            stream.write(_render(bundle))
+            stream.flush()
+            os.fsync(stream.fileno())
+        # Hard-linking the completed file gives create-without-overwrite semantics.
+        os.link(temporary, target)
+    except FileExistsError as error:
+        raise BundleIOError(f"refusing to overwrite artifact-evolution bundle: {target}") from error
+    except OSError as error:
+        raise BundleIOError(f"cannot write artifact-evolution bundle: {target}") from error
+    finally:
+        if temporary is not None:
+            try:
+                temporary.unlink()
+            except FileNotFoundError:
+                pass
+    return target
+
+
+def record_attempt(bundle: ArtifactEvolutionBundle, attempt: ProposalAttempt) -> ArtifactEvolutionBundle:
+    """Return a validated bundle with one additional realized attempt.
+
+    This pure operation makes recording explicit and keeps the bundle immutable;
+    persistence remains the caller's choice through :func:`write_bundle`.
+    """
+
+    if attempt.result.output_payload is not None:
+        payload = attempt.result.output_payload
+        if bundle.capture_policy.payloads == "redacted":
+            payload = redact_payload(payload)
+        attempt = attempt.model_copy(
+            update={
+                "result": attempt.result.model_copy(
+                    update={"output_payload": payload, "output_digest": digest_json(payload)}
+                )
+            }
+        )
+    updated = bundle.model_copy(update={"attempts": (*bundle.attempts, attempt)})
+    return ArtifactEvolutionBundle.model_validate(updated.model_dump(mode="python", by_alias=True))
+
+
+def _render(bundle: ArtifactEvolutionBundle) -> str:
+    payload = bundle.model_dump(mode="json", by_alias=True, exclude_none=True)
+    return json.dumps(payload, ensure_ascii=False, indent=2, sort_keys=True) + "\n"