PR #127 modified faircode/loaders.py, a file explicitly named on CLAUDE.md's "DO NOT TOUCH" list (the frozen faircode/ core protecting the paper's numbers). It was merged and only caught afterward on manual review (see CHANGELOG [2.0.8]) - verified harmless in that case (the benchmark never imports loaders.py), but nothing stopped a genuinely results-affecting change from landing the same way.
Request: a CI check on pull_request that diffs the changed files against CLAUDE.md §1's frozen list - paper/results-frozen/, results/, faircode/{benchmark,metrics,strategies,models,manifest,loaders,significance,figures}.py, any audit.yaml, any dataset CSV in an audit folder - and fails with a message pointing at CLAUDE.md's escape hatch (§6: flag it, don't silently fix it) instead of relying on a human catching it after merge.
PR #127 modified
faircode/loaders.py, a file explicitly named on CLAUDE.md's "DO NOT TOUCH" list (the frozenfaircode/core protecting the paper's numbers). It was merged and only caught afterward on manual review (see CHANGELOG[2.0.8]) - verified harmless in that case (the benchmark never importsloaders.py), but nothing stopped a genuinely results-affecting change from landing the same way.Request: a CI check on
pull_requestthat diffs the changed files against CLAUDE.md §1's frozen list -paper/results-frozen/,results/,faircode/{benchmark,metrics,strategies,models,manifest,loaders,significance,figures}.py, anyaudit.yaml, any dataset CSV in an audit folder - and fails with a message pointing at CLAUDE.md's escape hatch (§6: flag it, don't silently fix it) instead of relying on a human catching it after merge.