Skip to content

feat(pipeline): define a provisional stage result contract - #176

Open
avalla wants to merge 14 commits into
mainfrom
feat/m11-1-stage-result-contract
Open

avalla wants to merge 14 commits into
mainfrom
feat/m11-1-stage-result-contract

Conversation

@avalla

@avalla avalla commented Oct 15, 2026 •

Copy link
Copy Markdown
Owner

Summary

Adds a provisional, versioned contract for the structured outcome of a pipeline stage attempt, with producer provenance and a pure delivery assessment. It is run evidence only: no persistence and no wiring into the stage completion path.

Task

  • Task: 04478a78-d9b9-4409-a05b-ab05d54b9278 — M11-1 — Definire il contratto provvisorio degli artifact
  • Acceptance criteria:
    • Valid bounded results pass
    • Malformed values fail (unknown/missing version, missing field, malformed producer, oversized payload) with sanitized errors
    • No M11.6 aggregate/review semantics exist in this task

Scope

  • In scope: domain envelope contract and parser (packages/domain/src/pipeline/stage-result.ts), application delivery assessment (accepted / duplicate / stale / conflict / producer_mismatch, plus a preemptsBoundRun flag), tests, documentation in docs/development/pipeline-enforcement.md.
  • The diff touches exactly these four files. No migration, no repository or port, no new dependency, no capability or approval field accepted by the envelope (unknown fields are rejected).
  • Non-goals: persistent Artifact/ArtifactVersion aggregate, review lifecycle, stable public schema, ReviewRequest/ReviewResult, approval, automatic retry authority.
  • Deferred integration requirement: the task clarification says the envelope "travels in the existing run result and sanitized audit". The authorizer's same clarification requires no wiring into the AgentRun-to-stage completion path in M11-1, so that transport is a deferred integration requirement, not missing M11-1 scope. Ownership is now explicit: task 19328245-e21c-48ce-9732-13fd1a99ef4c ("M11/M12 — Trasportare l'envelope stage result nel run result e nell'audit sanitizzato"), with verifiable acceptance criteria, depends on M12-1 (76202083-16b6-47e5-b5ed-710b516b42d6, whose normalized result is the carrier) and on M11-1. M12-1 itself stays port-only (no persistence, no audit wiring), so it does not own the transport.

Design notes

  • Domain owns the provider-neutral contract; application classifies deliveries; no adapter is touched.
  • The producer in the payload is a claim; it must equal the deliverer the Runtime authenticated. An operator label is self-declared and is not authenticated against another same-UID process.
  • Duplicate detection compares canonical forms (canonicalStringify), and a recorded result is re-validated first. A different result for the same attempt is a conflict and is never overwritten; a result from an AgentRun no longer bound is stale. An accepted operator delivery while a run is bound sets preemptsBoundRun: a caller must record it through the existing attributed override path with a reason.
  • Untrusted input is read through own data descriptors (accessors refused, never invoked) into a snapshot; length, line-break and character checks run on the raw string before trimming; tab/LF/CRLF are allowed only in summary and data string values; Unicode format, private-use and surrogate code points are refused; only the parser's own rejections pass through the boundary, errors thrown by hostile traps are replaced; errors name the field only.
  • Sensitive data keys are refused (substring match on the existing sensitive names), except token-usage metering keys, which need a mandatory prefix (inputTokens, reasoning_tokens, cache_read_input_tokens, ...); a bare tokens is refused.
  • Limits: summary 2000 B, error message 500 B, data 16384 B, identifiers and data keys 128 B, depth 8, 512 nodes.

Verification (HEAD dc1618a)

Check Command Result
Targeted bun test tests/unit/stage-result-contract.test.ts 113 passed, 0 failed
Full bun run check on dc1618a exit 0, 3568 passed / 239 skipped / 0 failed
Integration with main unpublished merge of origin/main (0c324b9) into dc1618a in a temporary worktree, bun run check exit 0, 3644 passed / 251 skipped / 0 failed; PR history unchanged
Diff git diff --check origin/main...HEAD, Prettier, eslint, tsc clean
Mutation restored old behavior at the error-classification, raw-trim and metering-prefix points the matching regression test fails each time
GitHub CI 13 checks on dc1618a all green (one earlier macOS validate failure in filesystem-connector.test.ts, a file this PR does not touch, passed on rerun)

Review

Reviews are by subagents of the same model family as the author; they are not independent in the strong sense and none is a Codex review.

  • Architecture and requirements review (on c988426): no P1; one P2 (operator pre-emption) addressed by preemptsBoundRun; doc-accuracy P3s corrected.
  • Security and correctness review (on c988426): 1 P2 (invisible characters) and 12 P3; the in-scope ones are fixed in 258796a. Not fixed, by decision: homoglyph/fullwidth data keys bypass the sensitive-name check because normalizeSensitiveFieldKey (shared with capability authorization) drops non-ASCII characters; this belongs in a separate change.
  • QA (on c988426): about 190 own assertions against the final code, no defect.
  • Incremental review of dadf87b..dc1618a (metering allowlist and its test): APPROVE, no P1/P2; mutation confirmed the new test kills the old regex.
  • Incremental review of c988426..dadf87b (258796a, dadf87b: hostile input, Proxy/accessors, typed sanitized errors, credentials, hidden properties, Unicode, data limits, delivery classification): APPROVE, 0 P1, 0 P2, 4 P3; 17 of 18 mutations were caught by the tests; 113 tests pass. The 4 P3 are deferred, not blocking, and tracked in task 3ec1344e-221e-4468-86f2-2a77428f86d7: (1) deliveredBy is read more than once, so a nested accessor could flip the decision (assess-stage-result-delivery.ts:59-88); (2) JWT and URL-userinfo patterns are quadratic, ~38 ms at 16000 chars, bounded by the data budget (stage-result.ts:120,122); (3) credential scanner gaps (redis://:pass@host, token_eyJ..., whsec_) and false positive pwd=/home/user1; (4) the object width-cap line is not proven by a test (stage-result.ts:345-346).
  • Coverage on final head dc1618a: ..c988426 architecture, security and QA; c988426..dadf87b and dadf87b..dc1618a incremental reviews (both APPROVE). Every range has a non-Codex review; no range has a Codex review after 258796a.
  • ChatGPT (not Codex) review of dadf87b: one P2 (bare tokens bypass), fixed in dc1618a, thread resolved.
  • Codex findings: the two original P2s (descriptors, control characters) were already fixed and are covered by tests; three more (raw-text validation, trap-thrown errors, hidden array indices) were fixed in 258796a and dadf87b.

External review (Codex)

Required: external_review.command is configured in .task-delivery.yaml. Not satisfied on the final head. The last Codex review that completed ran on 258796a. Codex CLI and the GitHub Codex connector both report that the usage limit is reached (available again on 2026-10-22), so no Codex review exists for dadf87b or dc1618a. This is a failed gate, not a pass, and no other review stands in for it.

Limitations

  • The envelope is not consumed anywhere yet; M11-2 uses its fields in conditions, M12 and M11.6 are the explicit consumers.
  • Where a recorded result lives (for duplicate/conflict comparison) is left to the caller.
  • The credential check is conservative and is not a complete secret scanner; homoglyph data keys are not caught (see above).

Merge

Do not merge without explicit authorization. READY FOR MERGE is not DONE: post-merge verification is still required.

https://claude.ai/code/session_01EVFrKKyq46D9yGUtsWrPNY

Hardening update (2026-10-19) - HEAD 7cf05cc

All P3 left by the earlier reviews were re-verified on the remote head and the confirmed in-scope ones fixed:

  • Deliverer and bound AgentRun id are validated through the producer boundary (own data properties read once; accessors, extra fields, exotic prototypes, throwing getters and revoked proxies give the typed, sanitized DomainValidationError, never a silent decision).
  • Identifiers are compared exactly: padding is refused, not trimmed.
  • The URL-userinfo check matches :// first and checks the scheme backwards: about 41 ms to under 2 ms at 16 KB (worst case of every pattern measured); a scaling test pins it. Equivalence with the old pattern checked on 300k random strings; the only difference is the intended empty user.
  • Scanner now also refuses redis://:pw@host, whsec_ and token_eyJ... (credential exposure, P3).

Evidence: bun run check exit 0 on 7cf05cc (3597 passed, 239 skipped); GitHub CI 13/13 green; new tests fail on the previous code. Independent read-only review of c88c3fd..fd0d5bd: 2 P3 (deliverer read outside the parser boundary; no test for the linear cost), both fixed in 7cf05cc.

Not in this PR, tracked in task 3ec1344e-221e-4468-86f2-2a77428f86d7: homoglyph/fullwidth data keys (the shared normalizeSensitiveFieldKey also serves capability authorization, so a fix changes authorization behavior); pwd=/home/user1 over-refusal (fail-closed, not an exposure); no roadmap entry for M11-1.

External review (Codex): not satisfied on the final head. The Codex CLI and the GitHub connector are unavailable (usage limit); no other review stands in for it. The independent review below is by a subagent of the same model family, not Codex.

Add a versioned, bounded envelope for the structured outcome of a pipeline
stage attempt, with producer provenance (AgentRun or operator), typed
validation and a pure delivery assessment that classifies a delivery as
accepted, duplicate, stale or conflict without granting retry authority.

The contract is provisional run evidence: no persistence, no wiring into the
stage completion path, and no Artifact, review or approval semantics.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018V8SDpmPqvp85atzRorQoT
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-16T17:43:04.952627Z dadf87b Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 28e0de53b0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/domain/src/pipeline/stage-result.ts Outdated
Comment thread packages/domain/src/pipeline/stage-result.ts Outdated
avalla and others added 7 commits October 15, 2026 18:28
Authenticate the producer: the assessment now compares the payload's claimed
producer with the deliverer established by the Runtime, so a superseded
AgentRun can no longer pass as an operator. A redelivered recorded result is
a duplicate even after its run is unbound.

Bound the work before normalizing untrusted data (depth, node budget and
cheap length checks), reject credential-looking text and any data key that
contains a sensitive name, forbid control and bidi/line-separator characters
and line breaks outside the summary, and read each input field once.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018V8SDpmPqvp85atzRorQoT
Stop rejecting ordinary prose: credential detection now looks for
token-shaped material (long or digit-bearing bearer values, base64-shaped
basic credentials, provider token prefixes, key=value with letters and
digits) instead of words such as "basic" or "token:". Token-usage metering
keys such as totalTokens are accepted; a bare token key is still rejected.

Apply the character and credential checks to data keys, reject zero-width
marks, byte-order marks and lone surrogates, bound the cumulative size of
data strings and keys before normalizing, and normalize a plain snapshot
taken in the guard so a changing object cannot bypass the checks.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018V8SDpmPqvp85atzRorQoT
…tors

A throwing accessor on the envelope, producer or error object escaped as an
arbitrary Error and could run caller-controlled code. Fields are now read once
through own data descriptors into a plain snapshot, an accessor is refused
without being invoked, and hostile input that throws while it is read (throwing
or revoked Proxy) surfaces as the typed validation error.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018V8SDpmPqvp85atzRorQoT
… bytes

Snapshotting turned a Map, Set, Date, RegExp, typed array or class instance in
data into a plain object of its own fields, so different values could become
canonically equal. Only plain and null-prototype objects are accepted now. Data
keys are measured in UTF-8 bytes as documented, and the key checks assert the
rejection reason.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018V8SDpmPqvp85atzRorQoT
The data snapshot silently discarded extra array properties, symbol keys and
non-enumerable properties, which canonical JSON would have rejected. They are
refused now. The remaining realm and reserved-key limits are documented.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018V8SDpmPqvp85atzRorQoT
…perator pre-emption

Check length, line breaks and characters on the raw string before trimming,
reject every Unicode format, private-use and surrogate code point and a lone
CR, and pass through only the parser's own rejections so an error thrown by a
hostile trap is replaced. Refuse non-enumerable array indices, non-plain
envelope records and arrays, and bound object width. Anchor credential
patterns with lookarounds, cover JWT, Stripe, GitLab and URL userinfo, keep
provider usage metering keys. The delivery assessment revalidates the recorded
result, reads its context once and marks an operator delivery that pre-empts a
bound AgentRun.

Claude-Session: https://claude.ai/code/session_01EVFrKKyq46D9yGUtsWrPNY
@avalla

avalla commented Oct 16, 2026

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@avalla

avalla commented Oct 16, 2026

Copy link
Copy Markdown
Owner Author

Both P2 findings are fixed and covered by regression tests: accessor/descriptor refusal without invoking getters (snapshotRecord, guardData; tests 'does not invoke accessors', 'reads fields through descriptors'), and the CR/LF/tab exception limited to summary and data string values (boundedText multiline option; test 'rejects line breaks in single-line fields but not in the summary'). Follow-up hardening is in dadf87b (raw-text validation before trim, Unicode format characters, hidden properties, sanitized trap errors, credential patterns). Local bun run check on dadf87b: exit 0, 3567 passed, 239 skipped, 0 failed.

@avalla avalla left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review indipendente su dadf87b (ChatGPT, non Codex CLI/cloud). Ho esaminato i 4 file della PR, il comportamento del parser e la classificazione delle delivery, il feedback Codex precedente e i check GitHub. 1 finding P2 nella whitelist dei campi sensibili (commento inline). Non ho identificato altri P1/P2 confermati. Prima di dichiarare READY FOR MERGE: correggere il P2 e aggiungere test di regressione; verificare macOS validate (in progress all'ultima lettura); aggiornare il branch rispetto a main (79 commit più recente alla verifica) e rieseguire i gate sull'HEAD finale. L'autore dichiara bun run check verde (3567 passed, 239 skipped), ma non ho eseguito personalmente la suite locale. Questa review non è una review eseguita da Codex e non sostituisce automaticamente un eventuale gate formale che la richieda.

Comment thread packages/domain/src/pipeline/stage-result.ts Outdated
A bare tokens, tokensUsed or tokensCount key bypassed the sensitive-name check.

Claude-Session: https://claude.ai/code/session_01EVFrKKyq46D9yGUtsWrPNY
@avalla

avalla commented Oct 16, 2026

Copy link
Copy Markdown
Owner Author

Re the ChatGPT review on dadf87b: the P2 (optional prefix let a bare tokens/tokensUsed/tokensCount key bypass the sensitive-name check) was valid and is fixed in dc1618a. The metering prefix is now mandatory, with a regression test for tokens, tokensUsed, tokens_count, Tokens and token_used. Local bun run check on dc1618a: exit 0, 3568 passed, 239 skipped, 0 failed. The branch merges cleanly with current main (verified with merge-tree; no merge commit added). macOS validate on dadf87b failed once on an unrelated filesystem-connector test and passed on rerun.

A nested accessor on deliveredBy could return different producers to the
producer comparison and the stale check. The deliverer is now copied into a
plain value before either runs. Adds the regression test and a test that
pins the object width cap against the remaining node budget.

Claude-Session: https://claude.ai/code/session_01Q21NUKiVwrCvpZ5YgzjM5W
Accept-at-limit and reject-at-limit-plus-one coverage for summary, error
message, identifier, data key, depth and node count, so an off-by-one in any
bound is caught. Test-only change.

Claude-Session: https://claude.ai/code/session_01FKM37BV4LS8Rc8ny5GCwXy
…ined error

A hostile trap that held an earlier parser rejection could rewrite its message
and rethrow it; the WeakSet membership test then forwarded the mutated text.
Rejections are now tracked with their reason in a WeakMap and a fresh,
sanitized rejection is thrown from that private reason.

Claude-Session: https://claude.ai/code/session_01FKM37BV4LS8Rc8ny5GCwXy
@avalla

avalla commented Oct 19, 2026

Copy link
Copy Markdown
Owner Author

Update for head c88c3fd: added exact-boundary limit tests (all limits) and fixed a Codex P2: a retained parser rejection whose message a hostile trap mutated was forwarded; rejections are now rebuilt from a private reason (regression test added). Evidence: bun run check exit 0, CI green (13/13 incl. macOS), independent review (no P1; P2 limit-test gap fixed). Codex static review of the diff text: no substantiated findings (an earlier P1 on a private-key fixture was a diff-transport artifact; the file typechecks and 118 tests pass). Known residual P3s not changed: URL-userinfo regex cost on long dotted strings, deliverer context not re-validated, identifiers trimmed before comparison, no M11-1 roadmap entry.

@avalla avalla left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent ChatGPT review — HEAD c88c3fd (static source review, not Codex CLI)

Technical verdict: no newly confirmed P1/P2 findings in the reviewed code. Reviewed the stage-result parser, delivery assessment, documentation and relevant regression tests. The parser snapshots untrusted own data properties, bounds data, rejects unsupported shapes, canonicalizes results and now reconstructs validation errors from privately retained reasons rather than rethrowing mutable error objects. The delivery assessment preserves producer-mismatch / duplicate / stale / conflict precedence and flags operator pre-emption.

Existing lower-severity limitations: the scanner intentionally does not guarantee exhaustive secret detection; malformed trusted delivery context is not revalidated; identifiers are trimmed before equality; performance follow-ups are tracked. These are not newly proven merge blockers for this provisional, non-wired contract.

Evidence: GitHub workflows on the published HEAD are green; I did not execute Bun or perform mutation tests in this review. The PR description's consolidation note still names an earlier HEAD and should be brought up to date (P3 documentation hygiene).

Gate: This is a separate static ChatGPT review; it does not attest that codex review --base origin/main ran successfully and must not be counted as that configured gate.

…exactly

Close the remaining P3 findings of the stage result contract review:

- The authenticated deliverer and the bound AgentRun id are validated with the
  producer rules, so a malformed context is a typed DomainValidationError
  instead of a silent producer_mismatch or stale decision.
- Producer identifiers are compared exactly: padding is refused rather than
  trimmed, so " run-1 " and "run-1" can no longer be treated as one producer.
- The URL-userinfo credential pattern matches "://" first and checks the
  scheme backwards, which removes its quadratic cost on long scheme-looking
  runs (about 41 ms to 2 ms at 16 KB).
- The credential scanner now also refuses an empty URL user (redis://:pw@),
  the whsec_ prefix and a JWT glued to a word (token_eyJ...).

Claude-Session: https://claude.ai/code/session_01FKM37BV4LS8Rc8ny5GCwXy
The deliverer was destructured before validation, so extra fields, inherited
properties, a throwing getter or a revoked proxy bypassed the producer rules.
Pass it to parseStageResultProducer as is: own data properties are read once,
and anything else is a typed, sanitized DomainValidationError.

Also pin the linear cost of the URL credential check with a scaling test, so
the quadratic pattern cannot return unnoticed.

Claude-Session: https://claude.ai/code/session_01FKM37BV4LS8Rc8ny5GCwXy
@avalla

avalla commented Oct 19, 2026

Copy link
Copy Markdown
Owner Author

Hardening cycle closed on 7cf05cc: both remaining reviewer P3 (deliverer parsed outside the producer boundary; no test pinning the linear URL check) are fixed with regression tests. bun run check exit 0 (3597 passed), CI 13/13. Residual out-of-scope items are in task 3ec1344e with reasons (shared normalizeSensitiveFieldKey also serves authorization; pwd= over-refusal is fail-closed). Codex gate: not satisfied, not replaced by this static same-family review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant