Prepare the paper and analysis package for preprint release

[simonaszilinskas]

Summary

This PR turns the analysis into a complete preprint release candidate: a typeset paper, regenerated results and figures, a reproducible analysis environment, and a documented validation trail.

What changed

• Added the LaTeX manuscript and rendered 19-page PDF with the agreed author order and correspondence address.
• Corrected reasoning-model response handling and added an aggregate audit of hidden reasoning versus visible final content. The corrected parser retains 79 additional valid battles overall.
• Rebuilt the derived datasets and reran the core, joint, depth, topic, task, temporal, MATTR, robustness, ranking-shift, and external-alignment analyses.
• Tightened the statistical documentation around complete-case support, bootstrap counts, multiple comparisons, coefficient centering, and observational limitations.
• Added deterministic checkpoints with provenance manifests, local-source hash verification, a locked uv environment, Make targets, and CI checks.
• Added publication consistency tests, checkpoint tests, citation metadata, contributing guidance, a data codebook, artifact SHA-256 hashes, and a preprint readiness checklist.
• Regenerated every figure and corrected the reported PDF layout issues, including overflow, duplicated numbering, overlaps, excessive whitespace, and orphaned table headings.

Deliverables

• Rendered preprint PDF
• LaTeX source
• Reproduction instructions
• Preprint readiness review
• Artifact hash manifest

Validation

• Ruff passed.
• 35 tests and 11 subtests passed.
• Python compilation and Git whitespace checks passed.
• Two-pass XeLaTeX build completed without layout or package warnings.
• All 19 pages were visually reviewed; extracted text stays within the page bounds.

Before public deposit

The remaining items are author confirmation of affiliations, contributions, use of the correspondence address, and an immutable repository release or DOI.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 0b7d0b1a-2a18-4cf4-a29f-154ea6c52303

📥 Commits

Reviewing files that changed from the base of the PR and between 7a67539 and b3b1861.

⛔ Files ignored due to path filters (13)

• data/battle_tasks.parquet is excluded by !**/*.parquet
• data/fr_battles.parquet is excluded by !**/*.parquet
• data/mattr_alt.parquet is excluded by !**/*.parquet
• figures/fig10_reading_depth.pdf is excluded by !**/*.pdf
• figures/fig10_reading_depth.png is excluded by !**/*.png
• figures/fig11_topic_controls.pdf is excluded by !**/*.pdf
• figures/fig11_topic_controls.png is excluded by !**/*.png
• figures/fig12_external_alignment.pdf is excluded by !**/*.pdf
• figures/fig12_external_alignment.png is excluded by !**/*.png
• figures/fig9_linguistic.pdf is excluded by !**/*.pdf
• figures/fig9_linguistic.png is excluded by !**/*.png
• manuscript/paper.pdf is excluded by !**/*.pdf
• uv.lock is excluded by !**/*.lock

📒 Files selected for processing (53)

• .github/workflows/quality.yml
• .gitignore
• CITATION.cff
• CONTRIBUTING.md
• Makefile
• README.md
• data/README.md
• feedback.md
• manuscript/README.md
• manuscript/REVIEW.md
• manuscript/header.tex
• manuscript/paper.tex
• paper_draft.md
• pyproject.toml
• results/artifact_manifest.json
• results/core_results.json
• results/endogeneity_results.json
• results/external_leaderboard_results.json
• results/leaderboard_shift_results.json
• results/linguistic_results.json
• results/mattr_alt_results.json
• results/mattr_stress_results.json
• results/qualitative_results.json
• results/reasoning_content_audit_results.json
• results/robustness_random_results.json
• results/task_results.json
• results/time_block_results.json
• results/topic_results.json
• results/turn_depth_results.json
• results/vote_timing_audit_results.json
• run.py
• src/analyze_core.py
• src/analyze_mattr_alt.py
• src/audit_reasoning_content.py
• src/build_fr_arena.py
• src/checkpoints.py
• src/external_leaderboard_analysis.py
• src/extract_prompts.py
• src/generate_linguistic_figure.py
• src/generate_polish_figures.py
• src/linguistic_analysis.py
• src/mattr_alt.py
• src/mattr_stress.py
• src/robustness_random.py
• src/task_analysis.py
• src/task_classify.py
• src/time_block_bootstrap.py
• src/topic_analysis.py
• src/turn_depth_analysis.py
• src/write_artifact_manifest.py
• tests/test_checkpoints.py
• tests/test_publication_consistency.py
• tests/test_vote_truncation.py

---

📝 Walkthrough

Walkthrough

The PR adds reproducibility infrastructure, checkpointed dataset processing, profile-based pipeline execution, revised statistical analyses and result artifacts, a complete LaTeX manuscript, contributor and data documentation, artifact hashing, and CI checks for Python and LaTeX builds.

Changes

Reproducibility and project tooling

Layer / File(s)	Summary
Environment, automation, and repository guidance .github/workflows/*, Makefile, pyproject.toml, src/write_artifact_manifest.py, README.md, CONTRIBUTING.md, data/README.md, manuscript/REVIEW.md	Adds locked uv setup, Make targets, CI lint/test/LaTeX jobs, artifact SHA-256 manifests, citation metadata, data boundaries, and reproducibility instructions.

Checkpointed data and analysis pipeline

Layer / File(s)	Summary
Integrity-checked data construction src/checkpoints.py, src/build_fr_arena.py, src/extract_prompts.py, src/mattr_alt.py, src/audit_reasoning_content.py	Adds manifest-validated checkpoints, local SHA-256 verification, hidden-reasoning parsing, resumable extraction, and reasoning-content audits.
Profile orchestration and analyses run.py, src/analyze_mattr_alt.py, src/task_analysis.py, src/topic_analysis.py, src/turn_depth_analysis.py, src/external_leaderboard_analysis.py	Adds core, extended, and full execution profiles, new lexical-diversity analysis, deterministic task bootstrapping, revised depth outputs, and audited external-cache reuse.

Publication outputs

Layer / File(s)	Summary
Manuscript and published result artifacts manuscript/paper.tex, paper_draft.md, results/*.json	Updates methodology, statistical findings, robustness analyses, external-alignment tables, appendices, and generated result payloads.
Consistency and behavior tests tests/*	Adds checkpoint safety, hidden-reasoning parsing, publication consistency, bootstrap metadata, depth-model, and LaTeX formatting checks.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant run.py
  participant DataBuilder
  participant AnalysisScripts
  participant ArtifactManifest
  User->>run.py: select core, extended, or full profile
  run.py->>DataBuilder: execute raw build steps for full profile
  DataBuilder->>DataBuilder: validate checkpoints and source hashes
  run.py->>AnalysisScripts: execute selected analyses
  AnalysisScripts->>ArtifactManifest: produce result artifacts
  ArtifactManifest->>User: write hashed artifact inventory

Loading

Possibly related PRs

• simonaszilinskas/style-control-analysis#1: Covers related vote-time reconstruction, truncation tests, and checkpoint handling.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch preprint-paper-and-code

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}