Implement final statistical reporting standard for the R analysis suite

[rickyjreyes]

Goal

Extend the R analysis and reporting layer so the NIST atomic-spectra study reports a complete statistical audit rather than only peak locations and Monte Carlo p-values.

Required reporting standard

Component	Statistics to report	Recommended final standard
Dataset accounting	Raw rows, retained rows, excluded rows, duplicates, missing values, observed versus Ritz counts, wavelength/wavenumber range	Exact flow table
Main detected peak	Peak frequency f* or omega*, log-period Delta log x, scale ratio, peak power/amplitude, phase	Estimate plus 95% CI
Effect size	Null-standardized z-score, excess power over null, sinusoidal amplitude	Do not report only a p-value
Pointwise significance	Empirical Monte Carlo p-value	p = (r + 1) / (B + 1)
Global significance	Maximum-statistic p-value across the entire frequency scan	Essential; controls the look-elsewhere effect
Multiple analyses	Correction across ions, bin counts, transforms, filters, and other searched variants	Family-wise max-statistic or Benjamini-Hochberg FDR
Uncertainty	Bootstrap CI for frequency, amplitude, period, and scale ratio	At least 2,000–5,000 resamples
Peak stability	Percentage of resamples selecting the same predefined peak region; median and IQR of peak position	Preferably at least 80% in a predefined tolerance
Bin stability	Results across a predefined bin grid, e.g. 60–240 bins	Show frequency, effect, and global p-value, not merely whether a peak appears
Model comparison	Smooth-null model versus smooth-plus-log-periodic model	Delta AIC, Delta BIC, and preferably held-out predictive score
Replication	Observed versus Ritz wavelengths; training/holdout subsets; related independent line sets	Same peak location within uncertainty
Calibration	False-positive rate under synthetic nulls	Approximately nominal 1% or 5% rate
Sensitivity	Injection-recovery simulations	Detection probability versus injected amplitude

Implementation requirements

• Keep the current Python and R scanners unchanged as canonical reference implementations.
• Add new R scripts for audit analyses rather than overloading the scanner.
• Use deterministic seeds and record all analysis settings.
• Distinguish pointwise and scan-global significance explicitly.
• Treat zero Monte Carlo exceedances as a resolution-limited bound, not an exact p-value below 1/(B+1).
• Predefine peak-region tolerances before computing peak-selection stability.
• Avoid comparing raw deviance improvements across species without accounting for different line counts and bin counts.
• Report both observed and Ritz wavelength analyses when both are available.
• Add held-out or blocked validation in which the frequency is locked from training data before evaluation on the test subset.
• Include synthetic null calibration and injection-recovery power curves.
• Apply family-wise or FDR correction to the full declared analysis family.
• Produce machine-readable CSV/JSON outputs and publication-quality R figures/tables.

Suggested R modules

• R/build_dataset_flow.R
• R/bootstrap_peak_uncertainty.R
• R/peak_stability.R
• R/run_bin_grid.R
• R/model_comparison.R
• R/run_holdout_replication.R
• R/global_multiple_testing.R
• R/calibrate_false_positive_rate.R
• R/run_injection_recovery.R
• R/render_statistical_audit.R

Acceptance criteria

1. Exact dataset flow table is generated for every ion/run.
2. Peak frequency, log-period, scale ratio, amplitude, and phase have 95% intervals.
3. Both pointwise and full-scan maximum-statistic p-values are reported.
4. The declared analysis family receives multiplicity correction.
5. Peak-region selection frequency, median, and IQR are reported from at least 2,000 resamples.
6. A predefined 60–240 bin grid is summarized with frequency, effect size, and global p-value.
7. Smooth-null and smooth-plus-periodic models are compared using AIC/BIC and held-out score where possible.
8. Observed-versus-Ritz and training-versus-holdout comparisons are included.
9. Synthetic null false-positive calibration is shown against nominal alpha.
10. Injection-recovery detection curves are produced over a predefined amplitude grid.
11. All outputs are reproducible from one documented PowerShell/R command.
12. The final report clearly separates computational reproduction from independent physical validation.