Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/apps-module-migrations.json
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
"tests/test_apps_prepare.py",
"tests/test_apps_publication.py",
"tests/test_apps_replay.py",
"tests/test_benchmark_preparation.py",
"tests/test_result_delivery.py"
]
},
Expand Down Expand Up @@ -137,7 +138,8 @@
"tests/test_apps_evidence.py",
"tests/test_apps_execution_binding.py",
"tests/test_apps_generation_plan.py",
"tests/test_apps_module_migrations.py"
"tests/test_apps_module_migrations.py",
"tests/test_benchmark_preparation.py"
]
},
{
Expand Down
99 changes: 99 additions & 0 deletions docs/contracts/production-prepare.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# Production P1 preparation

`inferrsi.benchmark.preparation.PreparationService.prepare` implements the
Benchmark preparation port for the accepted D023 / `t108-v4` task family:
`p1-fixture@1`, `p1-rules@1`, operation `evaluate`, scope `p1`.

The trusted host configures a **Benchmark-owned frozen ProtocolV1** and a
read-only resource capability. Neither is supplied inside PrepareRequest.
The task configuration includes an explicit resource role for the actual worker
code bytes. Its ArtifactRef digest must equal `worker_recipe.code_sha256`.
Preparation verifies the exact supplied bytes; it never treats the historical
documentation worker digest as an available executable.

```python
from inferrsi.benchmark.preparation import (
PreparationService, encode, decode, validate_prepared,
)

service = PreparationService(
frozen_protocol, resources=open_fixed_resource, worker_role="worker-code",
)
prepared, protocol_bytes, open_resource = service.prepare(request)
```

`open_fixed_resource(role)` is a host-supplied capability returning bytes or a
fresh binary stream for an authorized role. Preparation closes each stream it
opens for verification; after prepare, the closure's consumer (normally Apps)
owns closing each stream it opens. The opener does not interpret a
request-provided location as a filesystem path.
The frozen resource metadata is verified during preparation. The returned
`open_resource` accepts only roles in that preparation's closure and reopens the
resource through the same capability. It does not write or archive bytes.
Provisional source locations may alias: byte authority is the configured role,
not a lookup or de-duplication by that location. Apps assigns a distinct final
archive path to each role and verifies the bytes against that role's metadata.

The first task keeps the P1 candidate, claims, evidence rules, worker recipe
shape and sampling conditions. PrepareRequest can bind its candidate expectation,
P1 task parameters, budget references and additional resource pins. A supplied
pin cannot replace a frozen role's hash, format or required use. The first task
does allow that pin to replace the role's provisional source location; this
changes the frozen protocol bytes, with identity effects described below. The first task
accepts an empty `comparison_conditions`: comparison conditions are configured
in the frozen task definition, rather than silently accepting unknown overrides.
Unknown task, grader, worker or claim-rule revisions are refused.

The protocol contains its declared resources and optional baseline/reference/
quality-reference resources. PreparedEvaluation carries the complete closure,
sorted by unique role. Duplicate roles across any of these positions are refused.

## Identity and pure codecs

`encode(prepared)`, `decode(bytes)` and `validate_prepared(prepared,
protocol_bytes)` perform no artifact reads, worker imports, execution or writes.
They preserve the canonical `inferrsi-json-v1` byte contract.

- `prepared_id` excludes its own value, `created_at`, `display_notes`, and
locations in its schema-declared outer ArtifactRefs. Moving both the outer
protocol and outer resource references preserves this ID.
- `protocol_digest` excludes protocol display notes and locations only in the
declared resource references, including baseline/reference/quality-reference.
It preserves an arbitrary ref-shaped value inside Parameters.
- Moving a reference **inside** the protocol changes the protocol's canonical
bytes and byte digest. The changed digest changes `prepared_id`, even when
`protocol_digest` remains equal. Other semantic changes affect both identities.

## Apps boundary and failures

Inject the bound port and pure functions into the existing Apps API:

```python
from inferrsi.apps.api import prepare

output = prepare(
{"schema_version": 1, "request": request},
workspace=workspace, storage_config=storage_config,
prepare_benchmark=service.prepare,
encode_prepared=encode, validate_prepared=validate_prepared,
)
```

Apps assigns its final outer locations, invokes the Benchmark pure codecs,
verifies bytes independently, then commits the full closure before publishing
a PreparedRef. Benchmark imports neither Apps nor Harness.

The prepare port returns a portable PublicError wire object for a declared
rejection. Missing resources produce `ArtifactUnavailable` at `prepare`;
incorrect retained metadata produces `ArtifactIntegrityError`. No preparation
is archived by Benchmark. If the resource disappears after preparation, the
Apps archive reports `ArtifactUnavailable` at `prepared_archive`; changed bytes
are rejected by its independent integrity verifier. Neither case publishes a
PreparedRef or a partial prepared closure.

This port does not register the configured worker with a Harness target or
execute/assess it. The H evaluation-mode path and production offline assessor
remain separate deliveries under #429 and #434. The packaged engineering-task
registry is unchanged. A host must supply the reviewed P1 task definition and
actual worker/resource capability; no default fixture backend is silently
selected. Ordinary CPU regressions do not establish a model/service gain.
1 change: 1 addition & 0 deletions docs/operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ Effect classes used below:
| Rebuild scoped legacy evidence views | `python tools/rebuild_model_evidence.py <manifest> --archive <path> --output <new-directory>` | CPU-only; creates a new output directory | Authenticates original bytes and pinned projection/code; no extraction, network, archived-code execution or campaign qualification |
| Inspect evidence inventory / generate budget input | `python tools/validate_evidence.py [--budget-assets]` | Read-only | Offline layout/hash/schema checks; optional disposable #328 metadata view derived from campaign authority; no downloads |
| Execute one ordinary local CPU command with a fixed record | Python `inferrsi.harness.api.Harness.allocate` / `execute` / `read` | Experiment-executing / record-writing | One explicit RunStore root and one trusted local CPU target; ordinary command mode only; a record holds execution facts, so exit 0 is not a verdict; no device, model or network fallback |
| Freeze a configured P1 preparation | Python `benchmark.preparation.PreparationService.prepare`; [production prepare](contracts/production-prepare.md) | Read-only resource access / pure encoding | Verifies supplied frozen worker/resource bytes and returns a preparation, protocol and role-keyed closure; Apps separately archives them before publishing a Ref |
| Visualize a recorded run or campaign | CLI `dashboard`; [dashboard](dashboard.md) | Report-generating / local read-only serving | Writes an offline HTML snapshot or serves an optional live view on loopback; does not run an agent or validate a claim |
| Capture a knowledge context | CLI `campaign capture-context`; [offline dashboard](dashboard.md#record-the-optimization-story-campaign-trace) | Local evidence-writing | Saves assembled context and resource hashes in a new `knowledge/<id>.json`. Supports ordinary questions and explicit IDs. The only accepted explicit `--relevance-method` value is `find-lexical-v1` |
| Retain an executed retrieval | Python `apps.workflows.campaign.save_retrieval`; [capture contract](knowledge-retrieval-trace.md#retain-a-standalone-result) | Local evidence-writing | Saves an existing catalog/collect result under a fresh campaign capture ID; no new search or automatic publication |
Expand Down
Loading
Loading