BLR transfer silently ignores new dataset missing batch effects

[avchavanne]

I ran the BLR transfer of my trained models on a new dataset with batch effects specified in a different way by accident at first, and then to check further. My trained model has 2 batch effects : sex encoded as M/F, and site. For the transfer norm_data objects, I tried :

1. 2 batch effects : sex encoded as 1/0 and site
2. 1 batch effect : site

Both ran without error nor warnings.
Looking at the design matrices, it seems like the toolkit silently assigns batch effects compatible with the trained model - specifically, it assigns to all subjects the first occuring combination of batch effects in trained model/training data (e.g. all become sex 1 from training site 1).
When I try to transfer using correctly encoded sex in a dummy dataset with a previously-seen site, all subjects still get assigned sex 1 from training site 1 in Phi.
So, it seems like the toolkit might not actually be transferring.

This is probably related to the temporary fix we had by commenting out line 208 of normative_models.py :
#new_model[responsevar].be_maps = copy.deepcopy(be_maps)
But leaving that in gave a dimension mismatch between the dimensions of transfer data Phi (if memory serves) and those expected by the model.

[AuguB]

Hi Alice, thanks for reporting this.

1. 2 batch effects : sex encoded as 1/0 and site: this succeeds because the model is not aware the M and F are the only allowed sexes. It just assumes 1 and 0 are unobserved sexes, just like you could have unobserved sites. This can be fixed using something like a 'transferable' flag on one of the batch effects, and is related to: Intern: Re-estimate a subset of batch effects in transfer #315

2. 1 batch effect : site: This behavior is allowed by the current implementation of BLR transfer is implemented here.
   A set of pseudo-(warped)centiles is computed using the provided X and a synthesized batch_effects matrix, which is what the design matrices that you see are used for. Then, if applicable, the warp is applied to y. Then, for each batch effect present in the transfer data, a set of correction coefficients (corresponding to mean and variance) is computed that correct the warped data to align optimally with the pseudo-centiles. These coefficients are stored and used to correct data from new batches. This means that at inference time, all data is first simulated to have come from a certain batch effect (the 1,1 you see), and then corrected in warped space, using the coefficients that are estimated for the particular batch effect.

So rest assured, the transfer definitely happens.

Although this behavior can easily be changed, it is also not immediately clear to me that it is undesired. A big reference model with more batch effects than sex and site can be imagined to be transferred to a smaller subset of batch effects.

[avchavanne]

Hi Stijn,

Good to hear, thanks for the clarification - it seems i was mixing up the synthetic batch construction/prediction, and the correction steps. I now realise that case 1 was expected. From case 3 (transferring on a dummy dataset with previously seen sex and site values), what I understood is that all values in batch_effects of transfer data are considered unobserved values, whether they were observed in the training data or not.

But in that case it might indeed be worth considering ways to (optionally) set transferable batch effect values and/or, as in case 2, constrain the required batch effect variables for transfer. As you've said, current behavior could very well be desired in some cases, but perhaps in some other cases (e.g. missing some batch effects with expected large impact on mean offset + small transfer datasets / transfer datasets with potentially different distributions of these batch effects w.r.t to training data), some additional options or warnings could be helpful. For instance, with my models, I'm not sure a transfer with only site as batch effect should be possible without at least a warning.

[AuguB]

Yes, I completely agree. Transferring with fewer batch effects is something that should at least trigger a warning. Additional configuration of non-transferable batch effects is best tackled together with aforementioned issue. I'd gladly provide my thoughts about that, if necessary.

…

On Wed, Feb 25, 2026, 19:26 avchavanne ***@***.***> wrote: *avchavanne* left a comment (amarquand/PCNtoolkit#382) <#382 (comment)> Hi Stijn, Good to hear, thanks for the clarification - it seems i was mixing up the synthetic batch construction/prediction, and the correction steps. I now realise that case 1 was expected. From case 3 (transferring on a dummy dataset with previously seen sex and site values), what I understood is that all values in batch_effects of transfer data are considered unobserved values, whether they were observed in the training data or not. But in that case it might indeed be worth considering ways to (optionally) set transferable batch effect values and/or, as in case 2, constrain the required batch effect variables for transfer. As you've said, current behavior could very well be desired in some cases, but perhaps in some other cases (e.g. missing some batch effects with expected large impact on mean offset + small transfer datasets / transfer datasets with potentially different distributions of these batch effects w.r.t to training data), some additional options or warnings could be helpful. For instance, with my models, I'm not sure a transfer with only site as batch effect should be possible without at least a warning. — Reply to this email directly, view it on GitHub <#382 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEWL6V2NOCV75IZCMH2X2OT4NXSO3AVCNFSM6AAAAACV7MEBWOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTSNRRGE3TENRWGM> . You are receiving this because you commented.Message ID: ***@***.***>

[contsili]

I had a discussion with @avchavanne today and I understand that two issues pop up:

1. During transfer there is a re-estimation of the batch effects (b.e.). That might be a problem as when we transfer from a big to a small dataset we dont want the well estimated from the large dataset to be lost and b.e. re-estimated only on the small dataset. Related issue:Intern: Re-estimate a subset of batch effects in transfer #315
   What we can do is:
   a. allow the user to decide
   b. the code decides based on the datasets
   c. we dont allow transferable b.e. at all

2. transferring from a training dataset that has 2 b.e. to a transfer dataset that has only one of the two. In that case the b.e. that is not accounted for might affect the other b.e. (especially if the two have different distributions eg 50/50 sex in the training dataset to 70/30 sex in the transfer dataset) An idea here is to give a warning to the user