Harden final JDS Gaussian-BIC validation

README.md

@@ -33,11 +33,11 @@ delete edge iff partial R^2 < 1 - n^(-1/n).
 
 DAGGuard is a score-based refinement procedure, not a finite-sample nominal FDR method. Its formal guarantees are conditional on a fixed or independently learned candidate. The current end-to-end empirical validation is specifically for NOTEARS-generated candidates.
 
-For conventional Gaussian BIC, the public API requires each child's centered full candidate-parent design to have full column rank; rank-deficient candidate designs are rejected. Before numerical rank assessment, nonconstant candidate-parent columns are normalized by their Euclidean norms, so the validation is stable to changes of measurement units. `globally_optimal=True` means the exact search established the minimum score within the documented numerical tolerance and did not hit its search limit; it does not imply a unique representative when multiple subsets are numerically tied.
+For conventional Gaussian BIC, every public refinement entry point requires each child's centered full candidate-parent design to have full column rank, `q_j < n - 1`, and strictly positive, numerically nondegenerate full-model residual variance. Rank-deficient, saturated, constant-response, and degenerate candidate regressions are rejected with informative exceptions. Before numerical rank assessment, nonconstant candidate-parent columns are normalized by their Euclidean norms, so validation is stable to changes of measurement units. `globally_optimal=True` means the exact search established the minimum score within the documented numerical tolerance and did not hit its search limit; it does not imply a unique representative when multiple subsets are numerically tied.
 
 ## Installation
 
-Python 3.12 was used for the reported analyses.
+Python 3.12 was used for the reported analyses. Core package versions are pinned in `requirements.txt`.
 
 ```bash
 python -m pip install -r requirements.txt
@@ -57,14 +57,14 @@ print(exact.globally_optimal)
 print(exact.total_bic)
 ```
 
+The lower-level names `exact_refine_dag` and `greedy_refine_dag` exported by `dagguard.py` use the same validation policy. `local_bic_refinement.py` remains the tested numerical engine for backward compatibility with the earlier reproducibility commit.
+
 Run the minimal example:
 
 ```bash
 python -m examples.dagguard_quickstart
 ```
 
-`local_bic_refinement.py` remains the tested numerical engine for backward compatibility with the earlier reproducibility commit.
-
 ## Main empirical results
 
 ### Controlled fixed-candidate experiments
@@ -89,18 +89,19 @@ The proprietary row-level data are not distributed. The real-data workflow recor
 
 ## Repository map
 
-- `dagguard.py` - public DAGGuard API.
+- `dagguard.py` - validated public DAGGuard API.
 - `local_bic_refinement.py` - backward-compatible numerical engine: local BIC, exact search, greedy search, pruning pressure, and graph metrics.
 - `candidate_contamination_simulations.py` - 12-setting fixed-candidate experiment.
 - `reproduce_simulations.py` - primary NOTEARS simulation workflow.
 - `additional_noise_sensitivity.py` - Gaussian/exponential/Gumbel sensitivity analysis.
 - `notears_tuning_sensitivity.py` - NOTEARS penalty/threshold sensitivity analysis.
 - `realdata_postselection_diagnostics.py` - authorized swine-data analysis.
 - `synthetic_application_twin.py` - public 37-variable workflow without proprietary observations.
-- `reproduce_submission.sh` - staged reproduction entrypoint.
+- `reproduce_submission.sh` - staged reproduction entry point.
 - `benchmarks/seven_method/` - audited competitor implementations and real-data benchmark runner.
+- `benchmarks/seven_method/SOURCE_AUDIT.md` - source-by-source comparator audit and documented adaptation choices.
 - `results/seven_method_benchmark/` - audited benchmark summary tables (no proprietary observations).
-- `tests/` - deterministic unit and numerical-policy tests.
+- `tests/` - deterministic numerical, public-API, and benchmark source-audit tests.
 - `REAL_DATA_SCHEMA.md` - construction of the 37 application variables.
 
 ## Reproduce the main DAGGuard analyses
@@ -128,9 +129,9 @@ python realdata_postselection_diagnostics.py \
 
 ## Seven-method benchmark provenance
 
-The benchmark distinguishes the methods' inferential targets. PC-family procedures are compared by skeleton adjacency because they need not return a uniquely oriented DAG. The Wang et al. (2026) structural method is transparently adapted to continuous variables by empirical tertiles and is not represented as official author software. PC-p is a source-audited Python port of the authors' official MATLAB code because MATLAB/Octave was unavailable in the benchmark runtime.
+The benchmark distinguishes the methods' inferential targets and implementation status. NOTEARS was checked against the public `xunzheng/notears` linear implementation. Li & Wang PC-FDR is an independent implementation of the published Algorithm 3. PC-p is a source-audited Python port of the authors' official MATLAB code because MATLAB/Octave was unavailable in the benchmark runtime. The Wang et al. (2026) structural method is a published-parameter adaptation to continuous variables by empirical tertiles and is not represented as official author software. PC-family procedures and the Wang adaptation are compared by skeleton adjacency because they need not return a uniquely oriented DAG comparable to NOTEARS/DAGGuard.
 
-See `benchmarks/seven_method/README.md` and `results/seven_method_benchmark/method_implementation_provenance.csv`.
+See `benchmarks/seven_method/SOURCE_AUDIT.md`, `benchmarks/seven_method/README.md`, `results/seven_method_benchmark/method_implementation_provenance.csv`, and `results/seven_method_benchmark/AUDIT_NOTES.md`.
 
 ## Tests
 
@@ -140,7 +141,7 @@ python -m examples.dagguard_quickstart
 python synthetic_application_twin.py --out results/synthetic_application_twin
 ```
 
-The public API tests include scale-invariance checks for exact refinement, greedy refinement, and pruning pressure, together with duplicate-column, near-collinearity, and near-tie cases.
+The public API tests cover scale invariance for exact refinement, greedy refinement, and pruning pressure; duplicate columns; near-collinearity; near ties; saturated local models; constant responses; and exact or numerically near-exact fits. Additional regression tests check the Gaussian Fisher-z formula, PC-FDR step-up rule, PC-p BY estimator, and the discrete-BIC parameter count used in the Wang adaptation.
 
 ## Data availability
 

RELEASE_NOTES.md

@@ -4,18 +4,21 @@ This repository state accompanies the JDS submission **“DAGGuard: Exact and Gr
 
 ## Reproducibility scope
 
-- Public refinement API: `dagguard.py`.
+- Validated public refinement API: `dagguard.py`.
 - Historical numerical engine and pinned application audit trail: `local_bic_refinement.py` and commit `509cb29c24967d12b99e2b53641349ce7bb470ed`.
 - Controlled candidate-contamination simulations: 1,200 replicates.
 - NOTEARS end-to-end Gaussian/non-Gaussian simulations: 240 datasets.
 - NOTEARS penalty/threshold sensitivity addressing dependence on upstream tuning.
 - Seven-method benchmark provenance and non-row-level summaries: `results/seven_method_benchmark/`.
+- External-method source audit: `benchmarks/seven_method/SOURCE_AUDIT.md`.
 - Proprietary swine observations are not distributed; the authorized workflow verifies the pinned input SHA-256.
 - `synthetic_application_twin.py` provides a public 37-variable end-to-end example without confidential observations.
-- `reproduce_submission.sh` is the staged reproduction entrypoint.
+- `reproduce_submission.sh` is the staged reproduction entry point.
 
 ## Numerical policy
 
-The public API requires each centered full candidate-parent design to have full column rank for conventional Gaussian BIC. Rank-deficient candidates are rejected. `globally_optimal=True` means that the exact search established the minimum objective value within the documented numerical tolerance and did not hit its search limit; it does not imply uniqueness among numerically tied subsets.
+For conventional Gaussian BIC, every public refinement entry point requires each centered full candidate-parent design to have full column rank, `q_j < n - 1`, and strictly positive, numerically nondegenerate full-model residual variance. Rank-deficient, saturated, constant-response, and degenerate candidate regressions are rejected rather than silently scored. `globally_optimal=True` means that the exact search established the minimum objective value within the documented numerical tolerance and did not hit its search limit; it does not imply uniqueness among numerically tied subsets.
 
-A GitHub tag/release should point to the final merged submission commit after the referee-revision branch is approved.
+The deterministic test suite covers the public numerical policy, exact-versus-greedy invariants, branch-and-bound certification, and targeted source-audit checks for the comparator calculations.
+
+A GitHub tag/release should point to the final merged submission commit after the final acceptance-focused pull request passes CI and is merged.

benchmarks/seven_method/README.md

@@ -1,20 +1,22 @@
 # Seven-method benchmark
 
-This directory contains the comparator implementations used for the DAGGuard JDS benchmark. The common simulation endpoint is skeleton adjacency because PC, PC-FDR, and PC-p do not necessarily return a uniquely oriented DAG.
+This directory contains the comparator implementations used for the DAGGuard JDS benchmark. The common simulation endpoint is skeleton adjacency because PC, PC-FDR, PC-p, and the adapted Wang procedure do not necessarily return a uniquely oriented DAG comparable to NOTEARS/DAGGuard.
 
 Primary methods:
 
-1. NOTEARS (existing project implementation)
+1. NOTEARS (project implementation checked against the public `xunzheng/notears` linear source)
 2. NOTEARS + DAGGuard-Greedy
 3. NOTEARS + DAGGuard-Exact
 4. Wang et al. (2026) hybrid structural pipeline, transparently adapted to continuous variables by empirical-tertile discretization
-5. Li & Wang (2009) PC-FDR
-6. Strobl, Spirtes & Visweswaran (2019) PC-p
+5. Li & Wang (2009) PC-FDR, independently implemented from published Algorithm 3
+6. Strobl, Spirtes & Visweswaran (2019) PC-p, source-audited Python port of the authors' MATLAB code
 7. ordinary PC
 
-`pcp_faithful.py` is a Python port audited against the authors' official MATLAB source (`ericstrobl/PCp`, master tree SHA `42a179d7305641dbe6f8926e46a67ba622c66524`). It is labeled as a port, not official author software.
+`SOURCE_AUDIT.md` records the source-by-source audit, including which components are direct source ports, which are independent implementations, and which are adaptations. This distinction is intentional: none of the independently implemented or adapted comparators is represented as official author software.
 
-The Wang et al. implementation reproduces structural Steps 1-3 with the published threshold setting A `(0.008, 0.005, 0.009)` and empirical-tertile discretization for continuous inputs. Step 4 only orients the resulting skeleton and was not used for the common skeleton benchmark because the publication does not fully specify a generic multilevel adaptation of that orientation rule. All four threshold settings tied as optimal in their application were checked as sensitivity analyses.
+`pcp_faithful.py` was audited against the authors' official MATLAB repository (`ericstrobl/PCp`, master tree SHA `42a179d7305641dbe6f8926e46a67ba622c66524`). The port preserves the official adaptive PC threshold, PC-stable neighborhood snapshots, p-value propagation, and BY FDR procedure. A rare conflict-indexing ambiguity in the MATLAB source is documented in `SOURCE_AUDIT.md`; no orientation conflicts occurred in the 240 primary simulation runs, so it cannot affect the primary benchmark table.
+
+The Wang et al. implementation reproduces the published structural Steps 1-3 as a generic continuous-data adaptation using empirical-tertile discretization. The primary threshold setting is A `(0.008, 0.005, 0.009)` and all four source-paper settings tied as optimal in their application are checked as sensitivities. Source-application root-node labels are not transferred to the generic simulation benchmark. Step 4 only orients the retained skeleton and was not used for the common skeleton endpoint because the publication does not fully specify a generic multilevel adaptation of that orientation rule.
 
 For the proprietary swine data, run only with an authorized local copy:
 
@@ -26,4 +28,4 @@ python -m benchmarks.seven_method.swine_benchmark \
 
 The script writes only non-row-level adjacency matrices and summaries. The expected SHA256 for the analysis file used in the manuscript is `b933fd66f49fd381bb9698ee2b3f5835d0db8d01820a01d5e45c9ac3a7bf5156`.
 
-See `results/seven_method_benchmark/` for the audited primary summary tables included with the repository.
+See `SOURCE_AUDIT.md` and `results/seven_method_benchmark/` for the audit record and primary summary tables.

benchmarks/seven_method/SOURCE_AUDIT.md

@@ -0,0 +1,33 @@
+# External-method source audit
+
+This benchmark distinguishes direct source ports from published-parameter adaptations. The purpose of this file is to make the external-method comparisons auditable without implying that independently implemented or adapted methods are official author software.
+
+## NOTEARS
+
+The project linear NOTEARS implementation was checked against the public `xunzheng/notears` linear least-squares implementation. The objective, positive/negative variable split for the L1 penalty, matrix-exponential acyclicity function and gradient, augmented-Lagrangian updates, zero-diagonal bounds, `h_tol=1e-8`, and `rho_max=1e16` agree in substance. The reported primary settings `lambda1=0.1` and coefficient threshold `0.3` match the standard public example. The project adds a deterministic safeguard that drops threshold-passing edges only if a residual numerical cycle would otherwise remain; this has no effect when the thresholded solution is already acyclic.
+
+## Li and Wang (2009) PC-FDR
+
+`pc_fdr.py` is an independent implementation of the paper's Algorithm 3 for the skeleton. The source audit checked the following features against Algorithm 3 and its Gaussian appendix: ordered-pair testing with current neighborhoods, maximum conditional-independence p-value (`pmax`) updates, repeated FDR application after valid p-values have been accumulated, theorem-version retention of `pmax` values for removed edges, and Fisher-z testing with the `sqrt(n-|C|-3)` factor. The primary benchmark uses the paper's practical `H*=H` step-up choice at `q=0.05`; `q=0.10` and a Benjamini-Yekutieli sensitivity are reported separately. Finite-sample FDP in these simulations is treated as an empirical operating characteristic, not as a test of the paper's asymptotic guarantee.
+
+## Strobl, Spirtes, and Visweswaran (2019) PC-p
+
+`pcp_faithful.py` is a source-audited Python port, not official author software. The audit used the authors' public MATLAB repository `ericstrobl/PCp`, master tree SHA `42a179d7305641dbe6f8926e46a67ba622c66524`, and checked the workflow corresponding to `PC_with_pval.m`, `get_skeleton_stable.m`, `get_v_structures2.m`, `clamp_edges.m`, `orientation_rules.m`, `control_FDR.m`, `binary_search.m`, `get_BY_FDR.m`, and `gaussCItest.m`. The port preserves the official adaptive initial PC threshold, PC-stable neighborhood snapshots, p-value propagation, and Benjamini-Yekutieli FDR step.
+
+One rare source-code ambiguity is documented rather than hidden: the MATLAB conflict-handling code contains expressions that can be read literally as linear indexing although the surrounding logic indicates edge-coordinate indexing. The primary Python port uses the coordinate interpretation and exposes `literal_conflict_indexing=True` for audit purposes. No orientation conflicts occurred in any of the 240 primary simulation runs, so this ambiguity cannot affect the primary simulation table.
+
+Earlier exploratory PC-p results from an incomplete port were discarded and are not part of the manuscript or archived primary results.
+
+## Wang et al. (2026) hybrid structural pipeline
+
+No official public implementation was located. The benchmark therefore labels this method as a transparent published-parameter adaptation rather than an exact reproduction. `wang_full.py`/`wang_sensitivity.py` implement the published structural Steps 1-3: mutual-information skeleton screening, conditional-mutual-information collider identification and pruning, and local discrete-BIC parent pruning. The primary thresholds are the paper's setting A `(0.008, 0.005, 0.009)`; the other three settings tied for best BIC in the source application are included as sensitivity analyses.
+
+The source application is discrete and includes domain-specific root-node restrictions. For the generic continuous simulation benchmark, variables are discretized by empirical tertiles and no source-application root labels are transferred. The paper does not fully formalize the phrase describing exclusion of collider-related nodes in the Step-3 conditioning set, so the implementation uses a conservative documented interpretation. Step 4 is not benchmarked because it only orients the retained skeleton and the common endpoint is skeleton adjacency; its state-wise orientation rule is also not sufficiently specified for a generic continuous-variable adaptation. These choices are limitations of comparability, not claimed features of the authors' original implementation.
+
+## Ordinary PC
+
+The ordinary-PC baseline uses an original-style ordered-pair skeleton search with immediate graph updates and two-sided Gaussian Fisher-z tests at `alpha=0.05`. It is presented as a conventional PC skeleton baseline, not as a reproduction of a particular software package.
+
+## Common endpoint and data checks
+
+PC-family methods and the adapted Wang procedure need not produce the same type of fully oriented DAG as NOTEARS/DAGGuard. The primary seven-method comparison therefore uses skeleton adjacency for all methods. Simulation comparisons use the same 240 `(d, s, noise, rep, seed)` keys. On the authorized commercial data, the comparator methods were rerun from the pinned input bytes and their adjacency matrices matched the archived results entry-for-entry.