Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
name: CI

on:
pull_request:
push:
branches:
- main

permissions:
contents: read

jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- name: Fetch pinned public dependencies
run: |
git clone \
https://github.com/opsle/context-firewall.git \
../context-firewall
git -C ../context-firewall checkout \
953c48f1cfd154d6b7ed10b51b87fe54e4df45f2
git clone \
https://github.com/opsle/decision-evidence-protocol.git \
../decision-evidence-protocol
git -C ../decision-evidence-protocol checkout \
b17ae3b41cea7cb0b9e0befe43e885b5aa0e4a09
- run: npm run verify
29 changes: 28 additions & 1 deletion ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,12 @@ initial model-visible exposure
zero or more escalation transitions
↓
deterministic measurement projection
↓
opsle.value-receipt.v1 ingestion
↓
observational run record
↓
per-run or cumulative Opsle Value summary
```

## Ports
Expand All @@ -19,7 +25,9 @@ deterministic measurement projection
- Core: validates append-only identities and exact byte/event accounting, then
derives initial and effective reduction.
- Evidence store: immutable/content-addressed fixtures or caller-owned artifacts.
- Output adapter: canonical machine JSON or a concise human summary.
- Output adapter: canonical machine JSON on stdout.
- Operator adapter: exactly one stably named completion indicator on stderr,
separate from canonical machine output.

## Independence

Expand All @@ -34,3 +42,22 @@ Canonical semantic results contain no generated timestamp, random identity,
wall-clock latency, ambient filesystem path, or environment state. Event order
is supplied source data; object keys use canonical serialization. Runtime
profiling latency, if measured later, belongs outside semantic results.

Caller-supplied receipt timestamps are retained for display but removed from
receipt, run-record, and summary semantic identity. Receipt and measurement
ordering is canonicalized before summary hashing.

## Visible Value aggregation

The receipt validator is a dependency-free implementation of the normative
program contract, not a runtime import of the research repository. The run
record binds optional ordinary-use telemetry to one or more receipts. The
summary projection retains every operator-displayable measurement and creates
totals only for measurements explicitly marked safe for `SUM`.

Aggregation keys include mechanism revision, configuration, policy,
measurement identity, unit, class, direction, source verification, and material
evidence trust. Result-only measurements retain null baseline and delta totals.
Mixing result-only and baseline/result/delta shapes fails closed. Ordinary run
records reject `EXPERIMENTAL` measurements; controlled experiments remain a
separate evidence surface.
13 changes: 13 additions & 0 deletions BENCHMARK.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,19 @@ Bytes, tokens, events, lines, characters, latency, and cost are separate units.
Payload reduction is not equivalent to token reduction unless provider-recorded
token usage is separately supplied.

## Observational production telemetry

Ordinary run records may accumulate invocation, exact byte/event, validation,
escalation, failure/recovery, tool-call, child-execution, polling, passive-wait,
provider-token, and outcome observations when those values are actually
available. Missing fields remain missing. This corpus describes normal usage;
it does not establish that a mechanism caused lower cost, fewer tokens, lower
latency, preserved correctness, or an avoided failure.

Measurement classes remain part of every value. Cumulative summaries never
flatten exact, observed, estimated, modeled, or experimental evidence. Ordinary
observational records reject experimental measurements.

## Repetition and reporting

Record model, provider, model version, reasoning effort, tool versions, fixture, prompt, environment/hardware, repetition count, observable tool activity, final result, correctness, cost/tokens when available, and known confounders. Report distributions and raw observations; never invent missing values.
Expand Down
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,17 @@
# Changelog

## 0.3.0 - 2026-08-25

- Added dependency-free `opsle.value-receipt.v1` validation and semantic
identity with caller timestamp exclusion.
- Added observational run records and deterministic per-run/cumulative Opsle
Value summaries with class-, unit-, revision-, configuration-, and trust-aware
safe aggregation.
- Added Context Firewall and Decision Evidence receipt support without circular
runtime dependencies or experimental evidence.
- Separated canonical stdout from one named `[Trajectory Profiler]` stderr
indicator and added local static/determinism verification scripts.

## 0.2.0 - 2026-08-25

- Added packet-v1 raw/reduced, suppression, escalation, final-visible, and
Expand Down
44 changes: 37 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,12 @@ and tool-evidence exposure from observable execution artifacts. Context
Firewall packet-v1 receipts become append-only trajectory events without either
upstream package becoming a runtime dependency.

The profiler also ingests the program-wide `opsle.value-receipt.v1` contract
from Context Firewall, Decision Evidence, and other conforming mechanisms. It
preserves measurement class and trust, builds deterministic per-run or
cumulative Opsle Value summaries, and aggregates only explicitly summable,
compatible observations.

## Why it matters

The Opsle thesis asks: **What if we stopped using intelligence for work that doesn’t require intelligence?** This project isolates one candidate boundary so it can be falsified and measured independently.
Expand Down Expand Up @@ -47,26 +53,49 @@ expansion directly.
The dependency-free CLI accepts a bare trajectory or a fixture wrapper:

```bash
node bin/agent-trajectory-profiler.js \
./bin/agent-trajectory-profiler.js \
profile \
fixtures/context-firewall/high-reduction-success.json
```

Use `--summary` for concise output. `npm test` and `npm run conformance`
require no sibling repository or mutable external state. `npm run interop` is a
development proof against exact sibling Context Firewall and Decision Evidence
Protocol checkouts.
Canonical machine JSON is always written to stdout. One concise
`[Trajectory Profiler]` completion indicator is written to stderr; `--quiet`
suppresses only that indicator. The legacy `--summary` flag is accepted as a
compatibility alias but no longer replaces machine output.

Profile an observational value record with:

```bash
./bin/agent-trajectory-profiler.js \
value-summary \
run-record.json
```

Pass multiple records, or `--cumulative`, for a cumulative summary. Use
`validate-record` for validation without summary projection. `npm test` and
`npm run conformance` require no sibling repository or mutable external state.
`npm run interop` remains the development proof against exact sibling Context
Firewall and Decision Evidence Protocol checkouts.

The package API exports packet/raw adapters, escalation-event construction,
trajectory validation/profiling/comparison, protocol constants, and canonical
measurement serialization from `src/index.js`.

The value API additionally exports receipt validation/identity, observational
run-record validation/identity, deterministic summary functions, and named
operator-indicator formatters. See [Visible Value telemetry](docs/VISIBLE_VALUE.md).

## Units and interpretation

Payload values are exact UTF-8 bytes. Evidence values are event counts. Neither
is a character, line, token, latency, or cost count. Optional token fields are
accepted only when labeled `PROVIDER_RECORDED` and remain separate.

Value-receipt aggregation partitions by mechanism, revision, configuration,
policy, measurement identity, unit, class, direction, and evidence trust.
Ratios, percentages, booleans, states, estimates, models, and experiments are
never directly summed. Missing observational fields remain missing.

**Payload reduction is not equivalent to token reduction unless token usage is
separately measured.**

Expand Down Expand Up @@ -132,8 +161,9 @@ A small dependency-free reference prototype is included for falsification and in
Deduplication is exact only within an operation when exposures share an explicit
content identity and byte size. Overlap with a different identity cannot be
proved safely and is counted in full. The adapter supports Context Firewall
packet-v1 only. Correctness preservation, token/cost savings, and a safe
reduction frontier remain unmeasured.
packet-v1 only for trajectory-event construction; the Visible Value path accepts
any conforming Opsle receipt. Correctness preservation, token/cost savings, and
a safe reduction frontier remain unmeasured.

## License

Expand Down
56 changes: 56 additions & 0 deletions SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,24 @@ Trajectory protocol: `opsle.agent-trajectory-profiler.trajectory/v1`.

Measurement protocol: `opsle.agent-trajectory-profiler.measurement/v1`.

Value receipt protocol: `opsle.value-receipt.v1`.

Observational run-record protocol:
`opsle.agent-trajectory-profiler.run-record/v1`.

Value-summary protocol: `opsle.agent-trajectory-profiler.value-summary/v1`.

## Compatibility boundary

The primitive accepts generic structured input and emits generic structured
output. It requires no Taslos Tasks database, worker, scheduler, private
service, model, provider, or network. Context Firewall and Decision Evidence
compatibility is field-based; normal runtime code imports neither package.

Visible Value compatibility is likewise field-based. The profiler implements
the normative receipt invariants without importing the research repository at
runtime.

## Trajectory input

A trajectory contains an explicit run, task, experimental-arm, and
Expand Down Expand Up @@ -104,6 +115,51 @@ Bytes, characters, lines, events, tokens, latency, and cost are not
interchangeable. Optional token measurements require source
`PROVIDER_RECORDED` and separate initial, escalated, and final values.

## Visible Value receipts

A receipt must conform to `opsle.value-receipt.v1`. Its deterministic semantic
identity excludes only caller-supplied `observed_at`. All other mechanism,
operation, configuration, policy, measurement, evidence, limitation, and
extension fields remain identity-bearing.

The validator enforces finite typed values, `delta = result - baseline` when a
numeric baseline is present, exact source verification, explicit assumptions
for estimates/models, controlled identity for experimental claims,
evidence-reference resolution, and the contract's counterfactual claim
ceilings. Bytes alone cannot become an estimated monetary value, and exact or
observed `failures_prevented` claims are rejected.

Only numeric-result `EXACT` or `OBSERVED` measurements with
`aggregation.safe = true` and `method = SUM` are aggregated. Baseline and delta
must either both be numeric and coherent or both be null. Ratio, percent,
boolean, state, estimated, modeled, and experimental measurements cannot be
directly summed.

## Observational run records

A run record requires a stable run identity and a value-receipt array. Task,
work, repository, project, model, reasoning effort, enabled mechanism,
telemetry, event, outcome, and evidence-reference fields are optional. Omitted
fields stay omitted in the summary.

When mechanism declarations are supplied, receipt version, revision,
configuration, and policy must match them exactly. Receipt run identities must
match the record. Duplicate semantic receipts and experimental measurements are
rejected. Provider-token telemetry must explicitly say `PROVIDER_RECORDED`.

## Value summaries

Per-run and cumulative summaries preserve displayable values and their quality
class. Safe totals are partitioned by mechanism, version, revision,
configuration, policy, measurement identity, unit, class, direction, source
verification, and referenced-evidence trust. Duplicate run or receipt identity
cannot enter a cumulative total. Result-only totals keep baseline and delta
null; they are never coerced to zero.

Canonical summary JSON is stdout data. A single deterministic
`[Trajectory Profiler]` indicator is stderr operator telemetry and is not part
of model-visible context unless a caller deliberately forwards it.

## Versioning

Breaking semantic changes require a new protocol version. New optional fields
Expand Down
3 changes: 3 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,6 @@ Key interpretation rules are normative in `SPEC.md`: initial reduction precedes
escalation; effective reduction follows all exposures; suppressed evidence is
not model-visible; tokens are never inferred from bytes; and validation trust is
preserved from Decision Evidence rather than upgraded locally.

See `VISIBLE_VALUE.md` for value-receipt ingestion, observational run records,
safe aggregation, and the operator-visible channel.
66 changes: 66 additions & 0 deletions docs/VISIBLE_VALUE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Visible Value telemetry

The profiler implements the program contract identified by
`opsle.value-receipt.v1`. Runtime code remains dependency-free: interoperability
is field-based, while exact cross-repository fixture generation remains a
development-only tool.

## Observational run record

`opsle.agent-trajectory-profiler.run-record/v1` binds a stable `run.id` and a
`value_receipts` array. The following surfaces are optional and retained only
when supplied:

- task and work classification;
- repository and project;
- model and reasoning effort;
- enabled mechanisms with exact revision, configuration, and policy;
- raw, initially visible, and finally visible evidence bytes;
- provider-recorded input, output, and total tokens;
- tool-call, child-execution, and polling-turn counts;
- passive-wait duration;
- escalation, failure, and recovery observations;
- acceptance and review outcomes;
- evidence references.

An omitted field is not normalized to zero or `null`. A receipt with a non-null
run identity must match the record. If mechanism declarations are supplied,
their revision/configuration/policy identity must match each receipt exactly.

Ordinary records are observational. They reject `EXPERIMENTAL` measurements even
when a receipt otherwise has controlled-comparison fields. Controlled experiment
records will use a separate future protocol.

## Summary behavior

`summarizeRunRecord` retains all operator-displayable values, evidence, class,
unit, direction, trust, and limitations. `summarizeCumulativeValue` combines
multiple unique runs.

Only measurements satisfying all of these conditions enter totals:

- class is `EXACT` or `OBSERVED`;
- result is numeric;
- `aggregation.safe` is `true`;
- aggregation method is `SUM`;
- the unit is directly summable;
- baseline and delta are both numeric or both null.

Totals are partitioned by every compatibility dimension named in the program
contract. Result-only totals retain null baseline and delta. Mixed result-only
and delta-bearing values in one semantic group fail closed. Unsafe values remain
visible in `excluded_from_aggregation`; they are not relabeled, coerced, or
dropped without explanation.

## CLI channel separation

Machine consumers read newline-terminated canonical JSON from stdout. Operators
receive exactly one concise named line on stderr:

```text
[Trajectory Profiler] run run-001 | 2 receipts ingested | 3 visible measurements | 2 safe aggregates
```

`--quiet` suppresses only the stderr line. It cannot alter canonical stdout.
Invocation errors remain machine-readable errors on stderr and do not emit a
completion indicator.
Loading
Loading