Skip to content

Preserve aggregate inference across sampling-weight units - #280

Open
pedrohcgs wants to merge 1 commit into
masterfrom
fix/aggregation-weight-scale
Open

pedrohcgs wants to merge 1 commit into
masterfrom
fix/aggregation-weight-scale

Conversation

@pedrohcgs

@pedrohcgs pedrohcgs commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Changing only the common units of valid sampling weights can change aggte() standard errors in the slower estimation path. For example, scaling weights by 1e307 preserves ATT(g,t) and its influence function, but changes a single-cohort aggregate SE from 0.1630024 to 0.1909288. In this case every aggregate must reproduce the original cell IF. The faster path normalizes weights earlier and remains correct.

wif() forms pg / sum(pg)^2. Squaring the cohort masses can overflow, underflow, or enter the subnormal range and silently corrupt the derivative of estimated aggregation weights. This uses an equivalent ratio formula in those cases and also handles overflow in sums of repeated selected masses. Ordinary arithmetic and existing malformed-input behavior are preserved.

Validation:

  • 5,896 assertions pass across all 404 test blocks, with heavy tests enabled (NOT_CRAN=true), and zero failures, warnings, errors or skips. The focused aggregation/IF subset also passes all 2,446 assertions.
  • New tests cover all four aggregations, DR/IPW/regression, panel/repeated-cross-section data, both estimation modes, and common weight scales from 1e-200 through 1e307, including finite subnormal squared masses at 1e-160. They compare complete aggregate influence vectors and inference.
  • An independent empirical ratio derivative verifies the weight IF, and ordinary-scale results are checked for exact identity with the prior formula.
  • Substituting only the baseline wif() into these tests produces 332 assertion failures, identifying the corrected arithmetic rather than a change in estimator or sample.

This was found while checking alignment with Python CSDID and standalone Python DRDID PR #21. The coordinated Python correction is CSDID #88; 552 strict complete-output cross-language/mode/unit comparisons pass against this reviewed R source. No reference tolerances or statistical methods are changed.

All GitHub checks pass at 7516845: R-CMD-check succeeds on macOS, Windows, Linux release, Linux old-release and Linux devel; build-check also succeeds. R-devel reports Status: OK.

Copilot AI balanced review requested due to automatic review settings October 27, 2026 04:17

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants