Consider standardizing and/or centering inputs to `glm`

[svteichman]

Consider standardizing and/or centering inputs to glm, based on @adw96's finding in PR #17, described below:

"I thought I had discovered an asymptotic bias issue in the cluster correlated case... but it was an artifact of numerical instability arising from generating data with outrageous true beta's. I will keep in mind for future that eg beta2 = 5 is just really hard to estimate with this data and can induce the appearance of issues with the actual effect size we care about. So, accuracy continues to be good."

[adw96]

Centering

To elaborate, calculations of the form $X^TX$ (or $X^TWX$), which come up in fitting GLMs, can be stabilized by centering covariates $X_{ik} \rightarrow X_{ik} - mean_{i'} X_{i'k}$.

The impact of this is that

1. The interpretation of the corresponding coefficient is unchanged (along the lines of ``difference in ... for a one-unit change in ...''), and its estimate is not changed (though we should confirm this before merging any PR)
2. The intercept may change (as that mean center goes into it)... but in this particular package we are disregarding interceptors uninterpretable anyway 🥳 So I think that we should have unchanged estimates but more stability

Standardizing

One of the challenges with standardization $X_{ik} \rightarrow ( X_{ik} - mean_{i'} X_{i'k}) / sd_{i'} X_{i'k})$ is that the interpretation of the parameter changes (instead of one unit changes, it's one sd changes). I don't think we want this change in interpretation, but it should be an active decision alongside centering.

[adw96]

Inference

I don't think that doing centering should impact inference results (in places where estimates are stable), but I have less intuition for robust score tests, so I could be wrong. It would be good to understand either way.

Suggestions

A dream PR implementing this could

• check that for reasonable effect sizes, parameter estimates don't change
  • perhaps implementing tests to do this
• check that for reasonably large (positive or negative) effect sizes, parameter estimates change, but be closer to the true value (under simulation)
• understand the impact of centering on testing, and make sure it fully makes sense

With other software packages, we've found that giving the users a huge number of options becomes unwieldy. Wrapping an option center = TRUE in a list of options could be a good place to start allowing the user to play with these choices. "standardize" could also be an option here, FALSE by default.

[svteichman]

Centering is considered in PR #24. While the option is included, I think that by default it should be off. In my investigations, in most settings centering does not change the results. However there are some settings, with large B values or large covariate values that the estimation procedure seems to be less stable when centering compared to not centering.

[svteichman]

Closing this issue because we've explored this option and made it a possibility (but not the default).