Skip to content

Commit f63f40b

Browse files
igerberclaude
andcommitted
Address PR #389 R2 (2 P1 + 1 P2): qug_test scope, design-detection rule, DIDHAD claim
P1 (qug_test in array-in pretest helper list): `docs/api/had.rst:67-72` listed `qug_test` alongside `stute_test` / `yatchew_hr_test` / `stute_joint_pretest` as accepting `survey_design=make_pweight_design(weights)`. Per `had_pretests.py:1236-1255` and the methodology REGISTRY (Phase 4.5 C0 decision gate), `qug_test` permanently raises `NotImplementedError` on any of `survey_design=` / `survey=` / `weights=` - there is no migration target for survey-aware QUG, and `make_pweight_design()` is explicitly NOT a valid QUG migration target. The composite workflow `did_had_pretest_workflow` handles weighted dispatch by skipping QUG with a `UserWarning`. Removed `qug_test` from the array-in helper list and added an explicit permanent-rejection note pointing to the workflow's skip behavior. P1 (estimand-resolution rule misstatement): `docs/troubleshooting.rst` "Resolved estimand" subsection said "no exact `dose == 0` => Design 1". Per `had.py:1932-1987` `_detect_design()` resolves to Design 1' when EITHER `d.min() == 0` OR `d.min() < 0.01 * median(|d|)` (small-share-of-treated escape clause). Rewrote the cause to spell out both sub-cases and clarify that Design 1 only fires when `d.min()` is meaningfully positive relative to the dose scale. Updated the inspection snippet to compute and print the `0.01 * median(|d|)` threshold instead of just counting `dose == 0` rows. P2 (DIDHAD event-study overstatement): `docs/r_comparison.rst` Heterogeneous Adoption section, R-equivalents note, and Migration Tips bullet claimed diff-diff additionally covers "the multi-period event-study extension (paper Appendix B.2)" beyond `DIDHAD`. The `DIDHAD` package already exposes dynamic effects / placebo / event-study output in the QUG case, so this overstates the gap. Narrowed all three locations to the documented differences: Design 1 (no QUG, `WAS_{d_lower}`) and survey-design integration via Binder TSL. Sphinx build clean (0 warnings in edited files; the unrelated `tutorials/18_geo_experiments.ipynb:61` "File not found: practitioner_decision_tree.html#few-test-markets" warning is pre-existing on origin/main and not introduced here). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 9a62e92 commit f63f40b

3 files changed

Lines changed: 37 additions & 25 deletions

File tree

docs/api/had.rst

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -65,11 +65,16 @@ Unit Remains Untreated" (arXiv:2405.04465v6), which:
6565
single SE contract under ``survey_design=`` lands. (Tracked in
6666
``TODO.md``; the deprecation warning emitted by ``HeterogeneousAdoptionDiD.fit``
6767
spells the migration out per call site.) On array-in HAD pretest
68-
helpers (``stute_test``, ``yatchew_hr_test``, ``stute_joint_pretest``,
69-
``qug_test``) the pweight-only shortcut is
68+
helpers (``stute_test``, ``yatchew_hr_test``, ``stute_joint_pretest``)
69+
the pweight-only shortcut is
7070
``survey_design=make_pweight_design(weights)``; data-in surfaces use
7171
``survey_design=SurveyDesign(weights="col_name", ...)`` against
72-
``data`` instead.
72+
``data`` instead. ``qug_test`` is the exception: the QUG step has no
73+
survey-aware migration target (Phase 4.5 C0 decision; see methodology
74+
REGISTRY) and permanently raises ``NotImplementedError`` on any of
75+
``survey_design=`` / ``survey=`` / ``weights=``. The composite
76+
workflow ``did_had_pretest_workflow`` handles this by skipping QUG
77+
under survey/weighted dispatch and emitting a ``UserWarning``.
7378

7479
A simultaneous confidence band (sup-t) is available only on the
7580
**weighted event-study path** via ``cband=True``. Joint cross-horizon

docs/r_comparison.rst

Lines changed: 11 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -225,14 +225,14 @@ August 2025) covers the QUG case (Design 1', ``d_lower = 0``) from the
225225
same arXiv paper.
226226

227227
``diff-diff`` ships :class:`~diff_diff.HeterogeneousAdoptionDiD`, which
228-
implements the broader surface of de Chaisemartin, Ciccia, D'Haultfoeuille
229-
and Knau (2026, arXiv:2405.04465v6): both Design 1' (QUG case, targets
230-
**WAS**) **and** Design 1 (no QUG, ``d_lower > 0``, targets
231-
``WAS_{d_lower}`` under Assumption 6 or sign-only under Assumption 5), the
232-
multi-period event-study extension (paper Appendix B.2), and survey-design
233-
integration via Binder (1983) Taylor-series linearization. The pretest
234-
battery :func:`~diff_diff.did_had_pretest_workflow` adjudicates the design
235-
path and surfaces assumption violations.
228+
implements de Chaisemartin, Ciccia, D'Haultfoeuille and Knau (2026,
229+
arXiv:2405.04465v6) and adds two surfaces beyond the QUG-focused R
230+
package: Design 1 (no QUG, ``d_lower > 0``, targets ``WAS_{d_lower}`` under
231+
Assumption 6 or sign-only under Assumption 5), and survey-design
232+
integration via Binder (1983) Taylor-series linearization (sampling weights
233+
+ optional strata / PSU / FPC). The pretest battery
234+
:func:`~diff_diff.did_had_pretest_workflow` adjudicates the design path
235+
and surfaces assumption violations.
236236

237237
.. code-block:: python
238238
@@ -420,8 +420,7 @@ Feature Comparison Table
420420
HeterogeneousAdoptionDiD (dCDH 2026) overlaps with the dedicated R
421421
package ``DIDHAD`` (de Chaisemartin et al., 2025), which covers the
422422
QUG case (Design 1'); diff-diff additionally covers Design 1 (no QUG,
423-
``WAS_{d_lower}``), the multi-period event-study extension (paper
424-
Appendix B.2), and survey-design integration via Binder TSL.
423+
``WAS_{d_lower}``) and survey-design integration via Binder TSL.
425424

426425
Migration Tips
427426
--------------
@@ -439,9 +438,8 @@ Migration Tips
439438
5. **Missing data**: diff-diff requires complete data; use ``balance_panel()``
440439
or ``dropna()`` first
441440

442-
6. **Heterogeneous Adoption (HAD)**: If you need the broader HAD surface
443-
beyond the QUG case that the R ``DIDHAD`` package covers - Design 1
444-
(no QUG, ``WAS_{d_lower}``), the multi-period event-study extension, or
441+
6. **Heterogeneous Adoption (HAD)**: If you need surfaces the R ``DIDHAD``
442+
package does not cover - Design 1 (no QUG, ``WAS_{d_lower}``) or
445443
survey-design integration - reach for
446444
:class:`~diff_diff.HeterogeneousAdoptionDiD`. See the
447445
`Heterogeneous Adoption (HAD)`_ section above for the migration pattern.

docs/troubleshooting.rst

Lines changed: 18 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -483,28 +483,37 @@ HeterogeneousAdoptionDiD (HAD) Issues
483483
**Problem:** ``HeterogeneousAdoptionDiD`` resolves ``target_parameter`` to
484484
``"WAS_d_lower"`` when you expected ``"WAS"`` (or vice versa).
485485

486-
**Cause:** HAD auto-detects the design path from the dose distribution. Design
487-
1' (QUG case, ``d_lower = 0``) targets WAS by treating the smallest-dose
488-
units as a quasi-untreated anchor; Design 1 (no QUG, ``d_lower > 0``) targets
489-
``WAS_{d_lower}``. If your data has no observations at ``dose = 0`` the
490-
estimator routes to Design 1 even when you intend a WAS interpretation.
486+
**Cause:** HAD auto-detects the design path from the dose distribution. The
487+
``_detect_design`` rule resolves to Design 1' (``continuous_at_zero``,
488+
targets WAS) when EITHER ``d.min() == 0`` exactly OR ``d.min()`` is a small
489+
positive value below ``0.01 * median(|d|)`` (the small-share-of-treated
490+
escape clause). Otherwise (``d.min()`` larger than that threshold) the
491+
estimator routes to Design 1, with a further check for mass-point structure
492+
(modal fraction at ``d.min()`` exceeding 2% routes to ``mass_point``;
493+
otherwise ``continuous_near_d_lower``); both Design 1 paths target
494+
``WAS_{d_lower}``. So a Design 1 resolution only fires when ``d.min()``
495+
is meaningfully positive relative to the dose scale.
491496

492497
**Solutions:**
493498

494499
.. code-block:: python
495500
496501
# Inspect the dose support before fitting
502+
import numpy as np
503+
d = data['dose'].to_numpy()
497504
print(data['dose'].describe())
498-
print((data['dose'] == 0).sum(), "observations at dose=0")
505+
print(f"d.min() = {d.min():.6g}; "
506+
f"0.01 * median(|d|) = {0.01 * np.median(np.abs(d)):.6g}; "
507+
f"d.min() < threshold => Design 1' (WAS)")
499508
500509
# Check the resolved estimand after fitting
501510
results = est.fit(data, outcome_col='y', unit_col='unit',
502511
time_col='period', dose_col='dose')
503512
print(f"Resolved: {results.target_parameter}")
504513
505-
# If you genuinely have a Design 1' panel but lack dose=0 rows, verify
506-
# the dose variable encoding (e.g. log-transformed doses where 0 was
507-
# mapped to a small positive value)
514+
# If you intend Design 1' but `d.min()` exceeds the threshold, verify
515+
# the dose-variable encoding (e.g. log-transformed doses where 0 was
516+
# mapped to a small positive value larger than 1% of the median).
508517
509518
"Mass-point fit fallback"
510519
~~~~~~~~~~~~~~~~~~~~~~~~~

0 commit comments

Comments
 (0)