Audit unresolved native robustness review findings #176

Activity

1. Anionix

   added

   bugSomething isn't workingSomething isn't working

   on Jul 16, 2026
2. Anionix commented on Jul 16, 2026

   

   Anionix

   on Jul 16, 2026

   OwnerAuthor

   More actions

   Additional merged-review findings

   The audit also found these older robustness findings. They remain follow-up work and do not change published rankings until fixed and re-run:

   • Add bounded artifact mutations and budget accounting #112: reject evidence sources that contain their destination before copytree to prevent recursive copying and budget bypass.
   • Add bounded artifact mutations and budget accounting #112: avoid no-op truncate mutation recipes for non-empty artifacts.
   • Isolate robustness cases in child processes #113: normalize bytes returned by TimeoutExpired before combining captured output.
   • Isolate robustness cases in child processes #113: classify canonical verification failures as VALUE_MISMATCH rather than REJECTED.
   • Aggregate bounded robustness evidence #126: resolve relative run_dir before deriving child paths.
   • Aggregate bounded robustness evidence #126: include robustness/ artifacts in release packaging.
   • Aggregate bounded robustness evidence #126: restore the full lifecycle contract comment at the robustness transition site.
   • Aggregate bounded robustness evidence #126: wire robustness results into report generation.
3. Anionix commented on Jul 16, 2026

   

   Anionix

   on Jul 16, 2026

   OwnerAuthor

   More actions

   More historical robustness findings

   • Add deterministic bounded robustness cases #111: preserve the requested row count when generated cardinality exceeds the sampled row count.
   • Add robustness evidence contracts #110: normalize serialized expectation/observed strings before deriving robustness verdicts.
   • Separate claims research summaries from robustness attempts #109: keep old claims results reportable, or reject them with a clear schema-version error instead of KeyError.
4. Anionix commented on Jul 16, 2026

   

   Anionix

   on Jul 16, 2026

   OwnerAuthor

   More actions

   Post-merge review findings to carry into follow-up fixes:

   • PR Harden robustness evidence contracts #178: verify_roundtrip() operational exceptions such as OSError, RuntimeError, or KeyError are currently classified as VALUE_MISMATCH. Only a confirmed unequal comparison should use that outcome; unexpected adapter/verifier failures need incomplete/harness evidence.
   • PR Harden robustness evidence contracts #178: seeded generated cases with rows == 0 currently record cardinality == 1, although the materialized table has zero values. Persist zero or mark the dimension not applicable.
   • PR Classify native target failures separately #177: the native workflow's failure gate must include the new TARGET_FAILED outcome, or gate directly on the recorded FAIL verdict, so project-seeded target failures cannot publish as green.

   These are tracked before resolving the merged review threads; each will be fixed in a fresh PR from origin/main.
5. Anionix commented on Jul 16, 2026

   

   Anionix

   on Jul 16, 2026

   OwnerAuthor

   More actions

   Native target failure classification and gate

   PR #177 and its follow-up PR #179 are merged on main.

   • FastLanes/project-seeded non-zero exit is now TARGET_FAILED with verdict FAIL.
   • Signal and timeout remain CRASHED/TIMED_OUT.
   • Arrow/Vortex coverage-guided runner/setup errors remain HARNESS_FAILED/INCOMPLETE.
   • The Linux native workflow now includes TARGET_FAILED in its final failure gate, after evidence upload.

   Verification passed: actionlint, Ruff, Pyright, full pytest, and CI verify. The remaining #176 candidate is the corpus-path behavior for non-fixture Arrow/Vortex native runs; it should be handled in a separate fresh PR.
6. Anionix commented on Jul 16, 2026

   

   Anionix

   on Jul 16, 2026

   OwnerAuthor

   More actions

   Post-merge review follow-up for PR #179:

   • The native workflow treated every TARGET_FAILED as a job failure, including fastlanes-quick-fuzz, whose target tier is EXPERIMENTAL.
   • PR Keep experimental native failures non-blocking #182 changes the workflow gate to fail only for CORE cases with a bad observed outcome or FAIL verdict. Experimental failures remain durable evidence without blocking publication.

   The merged review thread is being resolved with this follow-up linked.
7. Anionix commented on Jul 16, 2026

   

   Anionix

   on Jul 16, 2026

   OwnerAuthor

   More actions

   Native corpus directory fixed

   PR #181 is merged on main.

   • Arrow binary-backed coverage-guided targets now receive a dedicated seed directory instead of a single seed file.
   • The exact copied seed directory is retained as corpus_records with SHA-256 values in native evidence.
   • The source seed path and evidence directory are recorded separately.
   • Vortex remains on its official cargo fuzz run path.

   Verification passed: Ruff, Pyright, full pytest, and CI verify. The two concrete implementation issues identified in #176 are now fixed. The Issue remains open for any future native-runner findings, but no further bounded fix is pending from the current audit.
8. Anionix commented on Jul 16, 2026

   

   Anionix

   on Jul 16, 2026

   OwnerAuthor

   More actions

   Audit complete

   PR #182 is merged on main (82be13e). The native publication gate is now tier-aware:

   • CORE targets with UNSUPPORTED, CRASHED, TIMED_OUT, TARGET_FAILED, HARNESS_FAILED, BUDGET_EXHAUSTED, or FAIL evidence block the workflow.
   • EXPERIMENTAL targets such as FastLanes retain failure evidence but do not block publication.
   • The review threads on Classify native target failures separately #177 and Gate native target failures in workflow #179 are resolved.

   Together with #177, #179, and #181, the concrete findings audited by this Issue are fixed and verified by CI. Closing #176.
9. Anionix

   closed this as completedon Jul 16, 2026
10. Anionix commented on Jul 16, 2026

    

    Anionix

    on Jul 16, 2026

    OwnerAuthor

    More actions

    Post-merge review follow-ups:

    • PR Use dedicated native fuzz corpus directories #181 corpus evidence: corpus_records was not included in release reference validation. PR Validate robustness corpus release evidence #183 adds the references and a missing-record regression test.
    • PR Keep experimental native failures non-blocking #182 native workflow: the unconditional source/build/runner outcome check runs before the tier-aware case filter, so an experimental FastLanes setup failure still fails the matrix job. After Validate robustness corpus release evidence #183 merges, a fresh Harden robustness boundary evidence #184 will make that early outcome gate experimental-aware while preserving CORE failures.

    Both findings remain tracked here before resolving the merged review threads.
11. Anionix commented on Jul 16, 2026

    

    Anionix

    on Jul 16, 2026

    OwnerAuthor

    More actions

    Post-merge review follow-up for PR #183:

    • The empty-row cardinality fix changed the seeded PRNG sequence by skipping the cardinality draw when rows == 0.
    • PR Preserve generated case seed sequence #186 will consume the original cardinality draw and then overwrite only the empty case with zero, preserving all subsequent generated cases for the same seed/count.

    This is recorded before resolving the merged review thread.
12. Anionix commented on Jul 16, 2026

    

    Anionix

    on Jul 16, 2026

    OwnerAuthor

    More actions

    Audit closeout

    PR #186 is merged on main (97c49d3). The generated-case fix now consumes the cardinality PRNG draw even for zero-row cases, so later cases remain identical for the same seed/count.

    The concrete post-merge findings tracked here from #177 through #183 are implemented and verified. Merged review threads are resolved or outdated. The remaining open format work is limited to the explicit reproducibility blockers in #38, #39, and #40.