Skip to content

Gate actual CSV contents and metric regressions before promotion - #3

Merged
matchalatteguy merged 8 commits into
mainfrom
evidence-gate-standout-20261002
Oct 2, 2026
Merged

matchalatteguy merged 8 commits into
mainfrom
evidence-gate-standout-20261002

Conversation

@matchalatteguy

Copy link
Copy Markdown
Owner

What changes

Turn Evidence Gate into a practical local promotion gate for actual export contents and changes against a preserved reference run.

  • Opt-in streaming CSV validation: strict header/record structure, finite typed cells, exact ranges, observed row limits, and agreement with producer counts. Diagnostics aggregate errors with bounded row samples and no raw cell values.
  • Explicit baseline comparisons: absolute increase/decrease tolerances, dataset/configuration identity matching, exact decimal deltas, and rejection of self-comparison (including case/Unicode directory aliases).
  • One validation invocation emits JSON, Markdown, and JUnit with the same decision. Add strict warning policy, contract-only preflight, package version, and named bundled examples.
  • Public helpers hash outputs in chunks and atomically write finite JSON producer reports.
  • Protect CLI input evidence from output aliases/run-tree destinations, reject named-pipe report/status inputs, escape Markdown/XML, preserve repeated JUnit checks, and fail closed on numeric decoding that would erase declared precision.
  • Add a runnable classifier promotion example with actually computed predictions/accuracy: regressed metrics, freshly hashed truncated CSV, invalid cells, then an approved improvement.
  • Update contract/API/CI/producer/failure guides and migration notes; remove an obsolete internal publication assessment.

Validation

  • 321 core/example tests passed on Python 3.14; independent full Python 3.12 suite passed.
  • 12 pinned replay integration tests passed with a freshly built noneditable 0.4.0 package and unchanged companion commits.
  • Ruff lint/format, diff checks, wheel and source builds passed.
  • Independent reviews found and verified fixes for float literal collisions, named-pipe hangs, case/Unicode output corruption and reference self-comparison.
  • Hosted Python 3.11–3.14, installed-wheel examples, and the pinned replay job run for this PR.

Compatibility and limits

Contract/status schema remains 1. Existing unconfigured contracts and header-only csv_columns keep their behavior. New keys require 0.4+. CLI output destinations must be distinct outside run roots; case/Unicode-equivalent spellings and existing symlink/hardlink destinations are refused. Ignore all outputs after exit 2; publication is atomic per file, not a multi-file transaction.

Producer metrics, identities, expected hashes, and reference selection remain trusted inputs. The gate does not authenticate sources, independently recompute accuracy, validate arbitrary JSON Schema, check key uniqueness, or establish statistical/scientific validity. No new runtime dependencies, hosted state, network validation, or scheduler.

@matchalatteguy
matchalatteguy marked this pull request as ready for review October 2, 2026 12:27
@matchalatteguy
matchalatteguy merged commit 83d5109 into main Oct 2, 2026
10 checks passed
@matchalatteguy
matchalatteguy deleted the evidence-gate-standout-20261002 branch October 2, 2026 12:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant