Review AgentCore exports and skill deliveries offline

README.md

@@ -115,6 +115,10 @@ also requires all checks to pass. [Verification and limits](docs/verification.md
 
 **Inspect coverage before trusting a perfect score.** The website now lists all three task packs and links each single-case dependency directly to its recorded checks and seeds. Offline audit reports provide the same disclosures without scripts or remote assets. These are new views of the original records, not new model runs.
 
+## Trace Workbench — 0.11.0
+
+Import saved AgentCore Evaluate responses, versioned golden cases and Skills Anywhere delivery receipts. Inspect zero scores, skipped judges, missing results and missed skills separately. Compare matching datasets/rubrics and verify preserved input bytes offline. [Try the authored controls](https://noteflowai.github.io/evalarc/trace-workbench/) · [Actual local MCP delivery](https://noteflowai.github.io/evalarc/trace-mcp/) · [Input contract](docs/trace-workbench.md). No live AWS evaluation is claimed.
+
 ## New in 0.9.0: research you can inspect
 
 [Explore all 27 real GPU skill trials](https://noteflowai.github.io/evalarc/skill-impact/) and [the research pilots](docs/research-pilots.md). Robot Reel's [captured-scene editor](https://noteflowai.github.io/robot-reel/scene-lab/) and [official LIBERO-Plus replay](https://noteflowai.github.io/robot-reel/libero-plus/) connect real source records with portable skill delivery and independent grading. Every failed attempt stays visible; no skill efficacy, full-benchmark or real-hardware result is implied.
@@ -247,7 +251,7 @@ publication status.
 EvalArc's intended place is an **audit layer alongside existing environment
 and training frameworks**. Harbor already supports multi-step tasks and separate
 verifier environments; neither is claimed as an invention here.
-Harbor and Prime Intellect adapters are roadmap items, not current integrations.
+Native Harbor task export, oracle/NOP execution and ATIF 1.8 records are available as [bounded research integrations](docs/research-pilots.md). A general production adapter and Prime Intellect integration remain future work.
 
 ## Project name
 

README.zh-CN.md

@@ -98,6 +98,10 @@ Docker 评测：参考策略 3/3 轮完全通过，重复写入策略虽然平
 
 **在信任满分前，先检查覆盖薄弱点。** 网页现展示全部三个任务包，可从仅靠一个用例检出的缺陷直接定位到原始检查与种子记录。离线审计报告也提供相同的证据展开入口，不依赖脚本或远程资源；新增界面沿用原始数据，不冒充新模型运行。
 
+## 0.11.0：运行记录评估工作台
+
+导入已保存的 AgentCore Evaluate 结果、版本化黄金案例和 Skills Anywhere 加载回执，分别查看有效零分、评估跳过、缺少结果及漏调用技能。支持相同测试集与评分规则下的对比，以及原始输入的离线复核。[交互示例](https://noteflowai.github.io/evalarc/trace-workbench/) · [真实本地 MCP 加载](https://noteflowai.github.io/evalarc/trace-mcp/) · [数据契约](docs/trace-workbench.md)。示例明确区分合成评分与真实加载记录，未运行云端评估。
+
 ## 0.9.0：有原始证据的研究场景
 
 [查看 27 次真实 GPU 技能评测](https://noteflowai.github.io/evalarc/skill-impact/)，并阅读[完整方法与限制](docs/research-pilots.md)。新增[实景 Blender 编辑](https://noteflowai.github.io/robot-reel/scene-lab/)与[官方 LIBERO-Plus 子集回放](https://noteflowai.github.io/robot-reel/libero-plus/)，把原始记录、技能交付与独立验收连接起来。失败尝试全部保留；不宣称技能提分、完整基准成绩或真机效果。

docs/architecture.md

@@ -1,8 +1,10 @@
 # Architecture and domain expansion
 
 EvalArc's direction is open environments and auditable evaluations for
-software agents. This document distinguishes the working v0.6 implementation
-from interfaces proposed for subsequent releases.
+software agents. This document distinguishes current implementation from proposed interfaces.
+The foundational coding/support architecture below is supplemented by the
+robot evidence task, Harbor/ATIF research adapters and the offline
+[Trace Workbench](trace-workbench.md) in v0.11.0.
 
 ## Current implementation
 
@@ -20,8 +22,11 @@ use `evalarc.audit.v2` for declared controls. Metadata includes domain, task
 version, command, fingerprints, runtime limits, outcomes, and validity. Checks
 and evidence remain domain-specific.
 
-The current CLI does not invoke a model or expose a browser. The tool policy is
-an external program; Python and JavaScript examples are scripted controls.
+The core task CLI executes external candidate programs. Separate research
+scripts include recorded local-model pilots and native Harbor execution.
+`trace-import` reviews supplied AgentCore Evaluate responses offline; it does not
+invoke a cloud model. A general model-provider or browser-task adapter is not
+implemented. See [research scope](research-pilots.md).
 
 `repeat` executes fresh attempts from a frozen candidate and summarizes
 case/check variability. `suite` coordinates repetitions across declared jobs:

docs/roadmap.md

@@ -19,11 +19,13 @@ question needs substantially more evidence.
 | v0.4, implemented | Fixed-candidate repeated evaluation, per-check variability, case deadlines, progress events, process diagnostics | Fresh state per attempt, explicit invalid denominators, shared restart budgets, cancellation and cleanup tests |
 | v0.5, implemented | Declarative suites, per-job acceptance gates, HTML/JSON and JUnit export | All-candidate preflight, workload preview, protected dimensions, distinct invalid outcomes, no cross-domain score average |
 | v0.6, implemented | Python/JavaScript templates and independent controls for both tasks | The same 15 declared faults, Node Docker execution, lossless numeric handling, process-crash recovery, atomic workspace initialization |
+| v0.11, implemented | Offline AgentCore trace review and skill receipt linkage | Preserved bytes, versioned golden cases/rubrics, separate missing/skipped/zero results and synthetic vs recorded provenance |
+| Next | Judge calibration and broader trace collection adapters | Live upstream runs, frozen judge repetitions and independent calibration cases |
 | Next | Broader language coverage for coding | A non-Python Durable KV submission passes the same full contract |
 | Next | Independent defect packs and task authors | Reviewer-authored faults and a third independently authored task |
 | Next | Real model-provider adapter | End-to-end run with measured usage, task evidence, repeated attempts |
 | Later | Browser environment adapter | Reproducible initial state, backend outcome checks, isolated sessions |
-| Next | Harbor integration using supported verifier boundaries | Actual upstream task execution, pinned compatibility version |
+| Research pilot, implemented | Native Harbor task export, oracle/NOP execution and ATIF 1.8 | Pinned Harbor 0.23.0; upstream reward separate from independent checks |
 | Next | Procedural task variants with separated public and private acceptance | Measured exposure controls; distribution-level split |
 | Research pilot | Three deep task families | Human baselines, repeated real agent attempts, failure taxonomy |
 | Research study | Grader-audit ablations and downstream training | Unseen-family transfer, matched budgets, uncertainty intervals |

docs/trace-workbench.md

@@ -0,0 +1,135 @@
+# Trace Workbench
+
+`trace-import` turns a bounded export into a local, filterable report. It checks
+case/session/trace/span linkage, applies explicitly supplied evaluator scales,
+matches Skills Anywhere delivery receipts to reviewed bundle hashes, and keeps
+missing results distinct from assessed zero scores.
+
+This is **review of imported judgments**, separate from EvalArc's independent
+executable task graders. The importer does not call AWS, run a model, authenticate
+the producer or prove that the supplied traces cover every action.
+
+## Start with the authored controls
+
+```bash
+evalarc trace-import examples/trace-workbench/current.json \
+  --baseline examples/trace-workbench/baseline.json \
+  --output runs/trace-review-001
+evalarc trace-verify runs/trace-review-001
+```
+
+Open `runs/trace-review-001/index.html` directly, including offline. All rendering
+and filtering are local. The folder includes exact original input bytes,
+optional baseline bytes and a recomputable `review.json`.
+
+The five **synthetic** controls demonstrate an accepted case, assessed zero,
+skipped evaluation, missing result and missed required skill. Their placeholder
+hashes and authored scores are not cloud/model measurements.
+
+## Prepare a real export
+
+Use the example JSON as the wrapper. Fill in:
+
+| Field | Contract |
+| --- | --- |
+| `schema_version` | `evalarc.trace-input.v1` |
+| `run_id` | Your run identity; one attempt per input |
+| `provenance` | `kind: recorded` and a description of collection method and limitations |
+| `dataset` | Stable ID, version and ordered cases: `id`, `goal`, `expected_skills` |
+| `configuration` | Model identity/parameters, prompt and tool-schema SHA-256, `skills` mapping names to reviewed bundle SHA-256 (or `null` when unknown) |
+| `evaluators` | IDs, frozen revision descriptions, target `level`, and explicitly declared rating/gate rules |
+| `cases` | Exactly one recording per golden case, in the same order |
+
+A case contains `case_id`, `session_id`, `trace_ids`, `spans`,
+`skill_observation_complete`, `skill_calls` and `evaluation_response`.
+
+Copy flat OpenTelemetry spans with `traceId` and `spanId` into `spans`.
+Attributes may be a plain object or OTLP key/value array. When `session.id` is
+present it must match the case. Every declared trace must have a span. This first
+adapter accepts a flat span array, not arbitrary OTLP envelopes or CloudWatch
+query output wrappers; extract the native span objects first.
+
+Copy the native AgentCore **Evaluate API** response into `evaluation_response`.
+Preserve `evaluationResults` and original fields. Result targets use
+`context.spanContext.sessionId`, with `traceId` and `spanId` according to level.
+The Workshop CLI's `run.results[].sessionScores` output is a different envelope
+and is not accepted by this adapter.
+
+- `session`: one result for this case's session; no trace/span target.
+- `trace`: one result for each declared trace; no span target.
+- `skill`: one result for each explicitly annotated skill call's trace/span.
+  This level intentionally covers skill calls, not arbitrary tool-call evaluators.
+- Declare at least one session evaluator so every golden case has an expected
+  result even when no skill was called.
+
+Numeric ratings declare `min`, `max`, `pass_at_least`; categorical ratings
+declare `labels` and `pass_labels`. Never infer a scale or freeze a built-in
+evaluator version from its name alone. Record the configuration and collection
+date actually used; the revision field is caller-supplied metadata.
+
+Unknown IDs, duplicate evaluator/target results, wrong sessions and out-of-range
+ratings are rejected. Numeric zero is assessed. Error and skipped results are
+unassessed even if they carry a raw number; that number remains in `input.json`.
+`Skipped` in `errorCode`, or in a label outside a declared categorical scale, is
+recognized; this is not a promise that every service uses that spelling.
+Other errors retain their code. Empty responses produce missing expected results.
+
+## Associate an actual skill load
+
+Skills Anywhere 0.12+ returns `structuredContent.receipt` from successful
+`open_skill` calls. Your collector should preserve the response and annotate its
+actual tool span:
+
+```json
+{
+  "name": "evidence-review",
+  "trace_id": "YOUR_TRACE_ID",
+  "span_id": "YOUR_TOOL_SPAN_ID",
+  "receipt": {"schema": "skills-anywhere-load-1", "...": "copy the complete receipt"}
+}
+```
+
+The snippet is schematic; use the complete returned object. The receipt's
+`bundle_sha256` is available when `include_bundle: true` or
+`expected_bundle_sha256` is requested. The wrapper's `configuration.skills` must
+contain the bundle hashes reviewed before the run.
+
+This annotation is explicit: the importer does not guess skill names from
+arbitrary tool arguments. Set `skill_observation_complete: true` only if your
+collector observed all skill loads for the case. Absent required calls are
+`not_called` only with declared complete coverage; otherwise they remain
+unknown. A matching receipt identifies delivered content, not followed
+instructions or completed work. Multiple loads need distinct receipt IDs.
+
+## Compare and verify
+
+Comparison requires identical complete golden-set contents and evaluator
+definitions, including revision and rating rules. It shows configuration changes
+and case transitions without pooling evaluator scores or claiming causal gains.
+Synthetic and recorded inputs cannot be compared with each other.
+
+`trace-verify` reconstructs the review from preserved bytes and compares all
+fields. It detects changed input or summary bytes, but does not authenticate a
+producer or verify HTML. It does not execute any imported content.
+
+By default successful import exits 0 even when gates reject cases. Add
+`--require-accepted` for CI: 0 = every case accepted, 1 = complete results with
+rejected gates, 2 = incomplete results or invalid input. The complete report is
+still written for well-formed input with a rejected/incomplete gate.
+
+## Scope and sources
+
+Inputs are bounded to 4 MiB, 200 cases, 512 spans per case, and 32 evaluators.
+Duplicate JSON keys, nonfinite numbers and invalid Unicode are rejected. Existing
+output folders are preserved. There is no browser file upload, automatic
+collection, cloud deployment, judge calibration or statistical significance
+calculation in this first release. Review private trace content before sharing.
+
+- [AgentCore Evaluate API](https://docs.aws.amazon.com/bedrock-agentcore/latest/APIReference/API_Evaluate.html)
+- [Evaluation result fields](https://docs.aws.amazon.com/bedrock-agentcore/latest/APIReference/API_EvaluationResultContent.html)
+- [Skill evaluators](https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/skill-evaluators.html)
+- [Eval-First Workshop](https://catalog.us-east-1.prod.workshops.aws/workshops/bdb5c2fd-86cc-4a86-b55f-fbc2a81c001a/zh-CN/010-introduction)
+
+The AWS adapter has been tested against authored API-shaped controls. The
+separate MCP recording exercises actual local instruction delivery with missing
+evaluator results; no successful live AgentCore evaluation is claimed.