Skip to content

fix(statistics): reject duplicate paired CSV headers - #463

Merged
mikamikasuki merged 1 commit into
mikamikasuki:mainfrom
Ehsan-hash:fix/paired-csv-headers
Oct 27, 2026
Merged

mikamikasuki merged 1 commit into
mikamikasuki:mainfrom
Ehsan-hash:fix/paired-csv-headers

Conversation

@Ehsan-hash

Copy link
Copy Markdown
Contributor

Fixes #405.

csv.DictReader silently keeps the last value for a repeated header. In paired statistics this could reverse the reported comparison. Reject duplicate decoded column names before reading observations, running inference, or writing results, matching the existing paired-verification boundary.

Validation:

  • 12 focused checks passed for path/stream callers, duplicate score/unit/metadata/quoted headers, output preservation, ordinary numerical controls, and the actual API queue plus direct worker execution.
  • 54 affected handoff and evidence checks passed.
  • The pristine implementation analyzed the reported duplicate input as improvement −90; the unique-column control gives +1. The original-source regression run has 6 expected failures and 5 passing controls. One duplicate-unit case already rejected for too few distinct units; it now rejects the ambiguous header earlier.

The worker checks use independent SQLite/data/temp roots. Ambiguous input retains its exact source bytes but publishes no metrics or Analysis; the valid control retains the expected result and provenance. These checks exercise the direct worker branch, not daemon lifecycle transitions. No live provider, research experiment, PostgreSQL, browser, or full-suite validation is claimed.

@mikamikasuki
mikamikasuki merged commit a7775bc into mikamikasuki:main Oct 27, 2026
2 checks passed
@Ehsan-hash

Copy link
Copy Markdown
Contributor Author

Thanks for merging these fixes, @mikamikasuki!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Paired statistics can invert results with duplicate CSV headers

2 participants