Skip to content

Add fuzz coverage for coordinator-only inference error isolation - #2

Merged
numinousmuses merged 1 commit into
masterfrom
forge/task-87364f7dbf3d9d72eac9596d34d4625a
Sep 5, 2026
Merged

numinousmuses merged 1 commit into
masterfrom
forge/task-87364f7dbf3d9d72eac9596d34d4625a

Conversation

@numinousmuses

@numinousmuses numinousmuses commented Sep 5, 2026

Copy link
Copy Markdown

Add fuzz coverage for InferenceErrorMessage so public fields survive JSON roundtrips while the coordinator-only cause cannot enter or leave through wire JSON. Production behavior is unchanged.

The test covers the failure-code vocabulary, quotes, Unicode, empty values, duplicate keys, and attempts to inject coordinator-only fields. Exact string comparisons account for Go's documented replacement of invalid UTF-8.

Validation on fresh Forge workers:

  • Existing protocol tests passed.
  • FuzzInferenceErrorWireIsolation seeds and a bounded five-second fuzz run passed.
  • Both checks verified the same unchanged source tree.
  • Independent review mutation: changing CoordinatorCause from json:"-" to json:"coordinator_cause" makes the new test fail; the original candidate passes.

This is the prompted-work demonstration for Numinous Forge. The first agent attempt lost its model connection; the recorded revision produced this candidate. CI on this PR separately verifies its merge with the current default branch.

This change was prepared by Numinous Forge and independently verified on a fresh worker.

Task: 87364f7dbf3d9d72eac9596d34d4625a
Base: bbf6f83d4bbe66ae1a78c8f5cec898e3fbff5783
Verified tree: bfa23b3e397301e8e5ff31eabb456efb856be996
Policy: 0899a75bbab3c18283279614b63cba0ec5867316cda98dce3f2a032bbea420f7
Verification runs: 53d4185ee124317e6dedb2901b8fd27e, 75cc23da57389b7d1b4f289b82483f8e

Export full evidence with numinous-forge task export 87364f7dbf3d9d72eac9596d34d4625a ./evidence.

Before:

flowchart LR
  M[InferenceErrorMessage] --> J[JSON marshal and unmarshal]
  J --> W[Public wire fields; CoordinatorCause excluded]
  S[Existing fixed protocol tests] --> C[Limited input coverage]
Loading

After:

flowchart LR
  F[FuzzInferenceErrorWireIsolation] --> M[InferenceErrorMessage]
  M --> J[JSON roundtrip and injected private-field inputs]
  J --> A[Assert public fields survive and CoordinatorCause stays private]
  A --> G[Regression guard; production behavior unchanged]
Loading

Verified tree: bfa23b3
Policy: 0899a75bbab3c18283279614b63cba0ec5867316cda98dce3f2a032bbea420f7
@numinousmuses
numinousmuses marked this pull request as ready for review September 5, 2026 23:19
@numinousmuses
numinousmuses merged commit 69bf5fc into master Sep 5, 2026
2 checks passed
@numinousmuses

Copy link
Copy Markdown
Author

Numinous Forge · Change and verification

Result: new test coverage, merged into this fork. This example starts from a requested improvement rather than a reported bug.

What changed: Added fuzz coverage for inference-error messages. Public error fields must survive JSON encoding and decoding; coordinator-only diagnostic details must stay off the wire. Production behavior is unchanged.

What we proved:

  • Seed cases and a bounded five-second fuzz run passed, including quotes, Unicode, empty values, duplicate keys, and attempts to inject private fields.
  • Deliberately allowing the private diagnostic field into JSON makes the new test fail. This demonstrates that the test detects the behavior it is meant to prevent.
  • Independent protocol and fuzz checks passed. Separate PR CI passed protocol, coordinator, and documentation checks on the merge.

Review focus: The test should preserve the public error contract while rejecting diagnostic leakage, including malformed input handling.

After merge: We made the new regression a required check. A scheduled protocol run passed on the improved source under the updated check definitions. Both scheduled occurrences kept separate results; the later run did not overwrite the earlier one. The demonstration schedule is currently paused.

Read the test changes

Review summary for fdb9a4572c3d. Live task and CI progress are reported separately.

@numinousmuses numinousmuses added the trigger:instructed Work was explicitly requested; generated PRs inherit this trigger. label Sep 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

trigger:instructed Work was explicitly requested; generated PRs inherit this trigger.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant