Repository navigation
Add latent infection class "MobilitySubpopulationInfections" for contact-mixing models #901
Description
Estimate time-varying
The new class implements Equation 2 of Roy, Clapham, and Mishra (2025),
Incorporating human mobility to enhance epidemic response and estimate real-time reproduction numbers.
This issue states the computation the class performs, the inputs it accepts, and what it does not do. The tests listed at the end check that contract.
Prerequisites
These merge in this order:
- PR #897: time-varying ascertainment. It changes no code in
pyrenew/latent/. - #903, fixed by PR #904: subpopulation count observations are scaled by the subpopulation's population (see Infection counts).
-
#902, fixed by PR #906: removes
infection_process. Infection feedback becomes a constructor option ofPopulationInfectionsandSubpopulationInfections. -
#905, fixed by PR #907:
log_rt_time_0_rvcan give each subpopulation its own starting$\mathcal{R}(t)$ .
Background
SubpopulationInfections gives each subpopulation its own renewal equation.
Subpopulation
MobilitySubpopulationInfections keeps the same
such as one derived from census commuting data.
The model: Roy et al. Equation 2
For home region
with
The paper writes the Eq 1 sum from
-
$m_{jk}$ : fraction of residents of home region$k$ present at location$j$ -
$N_k$ : number of residents of region$k$ -
$R_{t,j}$ : reproduction number at location$j$ , where transmission happens -
$g$ : generation interval
The paper writes M is the matrix, n is the number of regions, and in this version the matrix is fixed.
Mobility matrix convention
M[j, k] is the fraction of residents of home region k who are present at location j. Columns are home regions (origins), rows are locations (destinations), and each column sums to 1. This is the paper's
A matrix with home regions in its rows (rows sum to 1) must be transposed before use.
Regions must be in the same order in M, subpop_fractions, a vector-valued I0_rv, a vector-valued log_rt_time_0_rv, and every observation's subpop_indices.
The computation, one day at a time
PyRenew's latent infections are per resident: infections[t, k] is the number of new infections among residents of home region k on day t, divided by the number of residents of k. Dividing Eq 2 by population_size * p[k], where p is subpop_fractions, the total population cancels.
The share of the people present at each location who live in each home region depends only on M and p, so it is computed once, before the daily loop:
share_present = M * p[None, :] / (M @ p)[:, None]One day of the renewal is:
infectiousness = reversed_gen_int @ history
infectiousness_at_location = share_present @ infectiousness
susceptible = 1 - cumulative
new_infections = susceptible * (M.T @ (rt * infectiousness_at_location))| Code | Roy et al. Eq 2 |
|---|---|
history |
recent |
infectiousness[k] |
|
share_present[j, k] |
|
infectiousness_at_location[j] |
infectiousness per person present at |
rt[j] |
|
M.T @ (...) |
the sum over |
cumulative[i] |
|
susceptible[i] |
|
new_infections[i] |
The order of operations follows Eq 2:
-
rtmultiplies at the location, beforeM.T, because$R_{t,j}$ describes transmission where it happens. -
M.Tassigns infections acquired at each location back to the home region of the person infected. -
susceptiblemultiplies afterM.T, because$S_{t,i}$ does not depend on$j$ : residents of region$i$ lose susceptibility wherever they were infected. - The population present at a location counts all residents present, susceptible or not, as in the denominator of Eq 2.
-
cumulativecounts every latent infection since the first day of the initialization period, as in$\sum_{s<t}$ .
The aggregate is the population-weighted sum, aggregate = p @ new_infections, which is total new infections divided by the total population.
When M is the identity matrix, each region follows Eq 1 on its own. With susceptible depletion turned off, the result is identical to SubpopulationInfections.
Reproduction numbers
Subpopulation SubpopulationInfections:
log_rt_subpop = log_rt_baseline + deviationswhere deviations sum to zero across subpopulations on each day. Under mobility, index j of rt_subpop is a location: rt_subpop[t, j] is
The recorded values (under the latent process name prefix) are those of SubpopulationInfections, plus susceptible_fraction. Under mobility they mean:
| Name | Meaning |
|---|---|
rt_baseline |
geometric mean of rt_subpop across locations |
rt_subpop |
j when everyone is susceptible |
rt_subpop_adjusted |
rt_subpop * (share_present @ susceptible_fraction) after the initialization period, and rt_subpop during it: reproduction number at location j after depletion, which is new infections caused per unit of infectiousness present at j; equal to rt_subpop when depletion is off |
infections_aggregate |
aggregate new infections per resident of the jurisdiction |
susceptible_fraction |
susceptible_fraction[t, i] is i not infected before day t, on the full time axis; recorded with depletion on or off, and used by the renewal only with depletion on |
With susceptible depletion on, rt_subpop is the paper's rt_subpop absorbs depletion, as in SubpopulationInfections.
Public API
from pyrenew.deterministic import DeterministicVariable
from pyrenew.latent import MobilitySubpopulationInfections
builder.configure_latent(
MobilitySubpopulationInfections,
gen_int_rv=gen_int_rv,
I0_rv=I0_rv,
log_rt_time_0_rv=log_rt_time_0_rv,
baseline_rt_process=baseline_rt_process,
subpop_rt_deviation_process=subpop_rt_deviation_process,
mobility_matrix_rv=DeterministicVariable("mobility_matrix", mobility_matrix),
susceptible_depletion=True,
)
model = builder.build()
model.run(
...,
population_size=population_size,
subpop_fractions=subpop_fractions,
**signal_data,
)-
mobility_matrix_rvis required. It is the matrix used on every day. In this version it must be aDeterministicVariablewhose value is an(n, n)numeric array. A data frame must be converted with.to_numpy(), with rows and columns in subpopulation order. -
susceptible_depletiondefaults toTrue, which is Eq 2 as published.Falsedrops the$S_{t,i}$ term. -
I0_rvreturns new infections per resident on the first day of the initialization period, the earliest day on the time axis, either one value for all regions or one value per region. It is not prevalence. New infections grow from that value over the initialization period, and every initialization day counts toward depletion. -
log_rt_time_0_rvreturns the starting log$\mathcal{R}(t)$ , either one value for all regions or one value per region (Letlog_rt_time_0_rvgive each subpopulation its own starting Rt #905). Roy et al. give each region its own starting value. -
model.run()takes the same arguments as forSubpopulationInfections. The matrix is part of the model definition, not run-time data. - Fitting and forecasting use the same matrix. Prediction with NumPyro's
Predictive, includingmodel.posterior_predictive(), runs the fitted model, so it uses the matrix the model was built with. No run-time argument can replace it. - To model another jurisdiction, build a model with that jurisdiction's matrix.
-
pyrenew.latentalso exportsBaseSubpopulationInfections(see Class structure) andcompute_infections_from_rt_with_mobility, the renewal function the class uses, next tocompute_infections_from_rt_with_feedback.
Validation
When the latent process is constructed, the matrix must be:
- two-dimensional and square;
- finite and non-negative;
- column-stochastic: each column sums to 1, within
1e-6. If the columns do not but the rows do, the error message says to pass the transpose; - such that every location has at least one positive entry in its row.
A three-dimensional array is rejected: mobility_matrix_rv must be a single (n, n) matrix. A mobility_matrix_rv that is not a DeterministicVariable is rejected.
Before fitting (MultiSignalModel.validate_data, which run() calls):
- the matrix has one row and one column per entry of
subpop_fractions; - every location has a positive population present:
M @ subpop_fractions > 0.
The size check also runs whenever the model is sampled, including prediction with Predictive. The population check does not: under Predictive and during MCMC the model is traced, and checks on values are skipped.
When the latent process is sampled:
- a vector-valued initial infection level has one value per subpopulation;
- a vector-valued starting log
$\mathcal{R}(t)$ has one value per subpopulation (Letlog_rt_time_0_rvgive each subpopulation its own starting Rt #905).
Infection counts
This class depends on the fix for subpopulation count scaling (#903, PR #904). With that fix:
- count observations of subpopulation
kreceiveinfections[:, k] * population_size * subpop_fractions[k]; - measurement observations, such as wastewater concentrations, receive infections per resident;
- aggregate count observations receive
aggregate * population_size, as now.
What this version does not do
- Time-varying mobility. One fixed matrix. A later version is planned to accept a list of dated matrices, each replacing the fixed matrix on its dates: for example, school holidays, or the identity during lockdowns as in Roy et al. The matrices are data and are never estimated. That version would run one fixed-matrix renewal per date range, carrying the state from one range to the next.
- Estimating the matrix. The matrix is data and is never estimated. A random matrix is rejected.
-
Scenario projections. This version uses one matrix on every day, so a projection that switches to another matrix from the forecast start (for example a lockdown) is not supported. That needs a matrix that changes on a date, as planned for dated matrices. A counterfactual with another matrix on every day is possible: build a second model with that matrix and pass the fitted model's posterior samples to NumPyro's
Predictive, asPredictive(model2.model, posterior_samples=model1.mcmc.get_samples()).model2.posterior_predictive()does not work, becausemodel2has not been fitted. -
Recording the matrix. The matrix is constant, so it is not recorded with every draw. It is available as
model.latent.mobility_matrix_rv.value. -
Mobility that responds to the epidemic, for example people travelling less when local transmission is high. That would need its own renewal step, computing each day's matrix from recent infections, and data on mobility: with count data alone, a mobility response trades off against
$\mathcal{R}(t)$ . -
Infection feedback. This class has no feedback option. In
PopulationInfectionsandSubpopulationInfections(Make infection feedback a constructor option of the latent classes; removeinfection_process#902), infection feedback is an approximate way to account for susceptible depletion. Here Eq 2 accounts for depletion directly, throughsusceptible_depletion. - Prior immunity. All residents are susceptible at the start of the initialization period, as in Roy et al.
-
Coupled initialization. Initialization-period infections grow at a rate computed from each region's own initial
$\mathcal{R}(t)$ , as inSubpopulationInfections. Under coupling this can produce a short adjustment over the first generation interval after initialization. -
Infection sources. The split of infections into local, incoming, and outgoing sources (Roy et al. Eqs 3 to 5) is not recorded. It can be computed after fitting from
latent_infections_by_subpop,rt_subpop, andsusceptible_fraction, with the matrix,population_size,subpop_fractions, and the generation interval. The generation interval must be fixed, because a sampled one is not recorded. With depletion off, use 1 in place ofsusceptible_fraction. - Matrix orientation conversion. A row-stochastic matrix is rejected, not transposed.
- Other kinds of contact mixing, such as age-structured contact matrices. Those are different models.
Deliberate differences from Roy et al.
-
$\mathcal{R}(t)$ structure. Roy et al. give each region an independent random walk on$\log R_{t,i}$ (simulation study) or a covariate model (England case study). This class uses PyRenew's shared baseline plus sum-to-zero deviations, with anyTemporalProcess. As in Roy et al., each region can start at its own$\mathcal{R}(t)$ by givinglog_rt_time_0_rvone value per region (Letlog_rt_time_0_rvgive each subpopulation its own starting Rt #905). -
Initialization. Roy et al. seed a constant number of infections for six days. This class starts from
I0on the first day of the initialization period and grows infections exponentially over that period, asSubpopulationInfectionsdoes. -
Overshoot. On a day with extreme
$\mathcal{R}(t)$ , Eq 2 as written can infect more residents than remain susceptible. This class does not cap that day's infections, so cumulative infections can exceed the number of residents. It floors the susceptible fraction at zero, so infections on later days stay non-negative. At realistic incidence the floor has no effect. - Observation model. Delays, ascertainment, and noise are PyRenew's observation processes. They are not part of Eq 2.
Class structure
BaseSubpopulationInfections holds what both subpopulation models share: the hierarchical
SubpopulationInfections: each subpopulation renews independently, with optional infection feedback. Its constructor, behavior, and recorded values do not change.MobilitySubpopulationInfections: the coupled renewal above, with optional susceptible depletion.
Susceptible depletion is part of the coupled renewal step in Eq 2, so it is a constructor option of MobilitySubpopulationInfections. PopulationInfections and SubpopulationInfections have no depletion option. Independent regions with depletion (Eq 1 for each region) are MobilitySubpopulationInfections with the identity matrix and depletion on. This is the model Roy et al. compare against for regions not linked by mobility.
Required checks
-
Eq 2 reference. Infections match a direct transcription of the time loop in the authors' Stan model (
uk_spatial/simulated_data/simulated_fitting_region.stanin mlgh-sg/Spatial_transmission). The comparison uses the Fig 2b mobility matrix, unequal regional$\mathcal{R}(t)$ , depletion on, and enough days for substantial depletion, and agrees to floating-point precision. -
Identity, no depletion. With the identity matrix and depletion off, every value
SubpopulationInfectionsrecords agrees to floating-point precision (allclose) for the same draws, with one starting log$\mathcal{R}(t)$ or one per region. - Identity, with depletion. With the identity matrix and depletion on, each region follows Eq 1.
-
Adjusted Rt. With depletion off,
rt_subpop_adjustedequalsrt_subpop. -
Conservation. Total new infections equal the sum over locations of
rt_subpop_adjustedtimes the infectiousness present there. -
Aggregate.
aggregateequalsall_subpops @ p. - Running total. Every initialization day counts toward depletion, including days older than the generation interval.
-
Susceptible fraction.
susceptible_fractionstarts at 1 and never increases. With depletion on,rt_subpop_adjustedequalsrt_subpop * (share_present @ susceptible_fraction)after the initialization period. - Extreme Rt. Infections stay non-negative and finite, and gradients stay finite.
-
Construction checks. Matrices that are non-square, negative, non-finite, row-stochastic but not column-stochastic, three-dimensional, or have an empty row are rejected. So is a non-deterministic
mobility_matrix_rv. -
SubpopulationInfectionsunchanged. Its tests, as Make infection feedback a constructor option of the latent classes; removeinfection_process#902 and Letlog_rt_time_0_rvgive each subpopulation its own starting Rt #905 leave them, pass after the base class is extracted, including those for infection feedback and per-region starting$\mathcal{R}(t)$ . -
Pre-fit checks. A matrix whose size does not match
subpop_fractions, or a location with no population present, is rejected byvalidate_databefore MCMC starts. -
End to end. Prior predictive sampling through
PyrenewBuilderandMultiSignalModelworks. -
Counterfactual with another matrix. Samples drawn from a model, passed to
Predictivefor the same model, reproduce its latent infections. Passed toPredictivefor a model built with the identity matrix, they give different latent infections.
Activity
cdc-mitzimorris commentedon Oct 6, 2026
@dylanhmorris, @damonbayer, @KOVALW after review of vhf-metapop-renew and the current PyRenew API I think that adding this to PyRenew will provide a clean way to specify these models - what do y'all think?
KOVALW commentedon Oct 6, 2026
Thanks @cdc-mitzimorris! This is looking great. Some thoughts:
Implementation
For the matrix multiplication, I think up until$R_t$ information earlier, with
prevalence_at_locationmakes sense. However, I want to clarify one piece of the implementation. Because you are generating exposure within a home region as a function of where those individuals were exposed, through the transposed mobility matrix, I think you need to incorporate theExplanation
Let's define some terms with math and code fro some subpopulation$x$
number of subpopulations =$\equiv n$
$\equiv M \times \vec{N}$ , where population present in subpop $j \equiv \hat{N}_j$ and
len(subpop_fractions)population_present=M @ pand in this case, because we're dealing with proportions,$\sum N_k=1$
We can define an expected infection pressure of previous infected individuals to be
$\equiv \lambda_{t,j} = \sum_{k=1}^n m_{jk} \sum_{\tau=0}^{t-1} I_{\tau,k} w(t-\tau)$
infectiousness_present = M @ (p * infectiousness)The contribution of infections from all subpopulations into subpopulation$j$ is
prevalence_at_location = infectiousness_present / population_present=Here, is where the change comes in to differentiate what I've written above. We want to now multiply by the effective reproductive number in each source subpopulation$j$ , not in the origin $x$ , because you are multiplying by the transposed matrix $M^T$
new_infections_by_source_subpop=M.T @ (Rt_subpop * prevalence_at_location)=The original implementation would have instead yielded
Then, as you show above, subpopulation counts corrects for the scaling, multiplying$N_xI_{t,x}$ to yield the observed incidence scaled appropriately. I think this also leaves a hook open where we could include a feedback on the current susceptible population instead of the subpopulation fraction size to arrive at infection incidence, but please let me know if I'm missing a reason why that wouldn't be able to be implemented later on, like in Subekshya's single population example.
We could consider an alternative model where we don't do this extra level of abstraction, and instead incidence is calculated from infection pressure entering origin$x$ , including from itself.
The overall point is that$R_{t,\text{subpop}}$ and $\lambda_{t,\text{subpop}}$ can't be separated across a multiplication step because the infection pressure needs to be tethered to the reproduction.
API
MetapopulationInfectionsas the name here, but would change my position if we want to persistSubpopulationInfectionsas a legacy class and not the default implementation ofMetapopulationInfectionswhen the mobility matrix is the identity matrix. I would likely suggestMixingSubpopulationInfectionsif this is a new class. These could both be derived from someMetapopulationInfectionsclass, however.latent_datashould contain themobility_matrixand notsubpop_fractions. I can understand that leavingsubpop_fractionsat the higher level makes for an easier implementation, but maybe in that casemobility_matrixshould also be an expected keyword argument?dylanhmorris commentedon Oct 7, 2026
Thanks both! A few comments:
Name/hierarchy of the classes
Agree with @KOVALW that mixing and non-mixing {meta,sub}population models should have a shared multi-population base class.
FWIW, I think the name
SubpopulationInfectionsdoesn't imply anything one way or the other about mixing. Given that, I propose the following:SubpopulationInfectionsbecomes the name of a factory function that returns either an instance of:DisconnectedSubpopulationInfections(no mixing, renamed version of the existingSubpopulationInfectionsclassConnectedSubpopulationInfections(new class, which permits mixing/connections.Argument structure
I agree with this. IMO it's fine to have both as top-level kwargs.
Matrix parameter name
Suggest
mixing_matrixovermobility_matrixon the grounds there are other interpretations of this mathematical model (e.g. subpopulations that represent age groups) in which everything is the same except that the biological interpretation of (and data source for) the matrix that connects the subpopulations is no longer "mobility"cdc-mitzimorris commentedon Oct 7, 2026
@KOVALW Both Claude and codex have flagged this equation as having an indexing error:
The coefficient should be$m_{jx}$ . do you agree?
cdc-mitzimorris commentedon Oct 7, 2026
We need to add a class
BaseSubpopulationInfectionProcessand then bothSubpopulationInfectionsandMixingSubpopulationInfectionswould inherit from it.regarding names: "connected" and "disconnected" can mean many different things:
Those are distinct model features.
The existing
SubpopulationInfectionsis not fully “disconnected” in a probabilistic sense. The regions share a baseline trajectory, so their reproduction numbers are related even though their renewal equations are dynamically uncoupled. Likewise, aConnectedSubpopulationInfectionsinstance could receive an identity matrix or block-diagonal matrix and thus be disconnected.For now, I would prefer to keep "SubpopulationInfections" and add "MixingSubpopulationInfections".
Also, a factory function isn't useful here - it would require the mixing matrix to be supplied at construction time, not sample time.
cdc-mitzimorris commentedon Oct 7, 2026
The above 2 comments aside, I completely agree with @KOVALW's main point w/r/t where the$R_t$ belongs in the computation and I will revise the above proposal accordingly.
dylanhmorris commentedon Oct 7, 2026
Good point and agree. Happy with
MixingSubpopulationInfections(orSubpopulationInfectionsMixingClarification: I was envisioning just specifying at construction whether you wanted to allow mixing or not, not passing the matrix. But I'm happy to have users just pick a class.
I still think that for readability and lack-of-user-surprise I'd prefer that
SubpopulationInfectionsbe a base class name (or a factory name) rather than implicitly meaningNoMixingSubpopulationInfections. Do you think that's too much of a breaking change, @cdc-mitzimorris?If so, we could go with
SubpopulationInfectionsBase,SubpopulationInfectionsMixingandSubpopulationInfectionsNoMixingor similar, and makeSubpopulationInfectionsan alias forSubpopulationInfectionsNoMixingwith a deprecation warning.cdc-mitzimorris commentedon Oct 8, 2026
SubpopulationInfectionsis a simple hierarchical model. if you want,HierarchicalSubpopulationInfectionorHierSubpopulationInfection.MixingSubpopulationInfectionstakes a mixing matrixin the future we might also want
SmoothingSubpopulationInfectionsfor spatial smoothing using a CAR or ICAR component.for consistency of naming, I would prefer to keep
Infectionsas the last part of the name.KOVALW commentedon Oct 8, 2026
Yes, the Roy et al paper has the flux of$x$ mixing with the infectiousness in $j$ , rather than the flux of infectiousness in $j$ mixing in $x$ . This is because we ultimately multiply by the population in $x$ , not the population in $j$ . I'll update the above comment accordingly to be correct
[-]Add latent infection class "MetapopulationPopulations" for contact-mixing models[/-][+]Add latent infection class "MixingSubpopulationsPopulations" for contact-mixing models[/+][-]Add latent infection class "MixingSubpopulationsPopulations" for contact-mixing models[/-][+]Add latent infection class "MixingSubpopulationInfections" for contact-mixing models[/+][-]Add latent infection class "MixingSubpopulationInfections" for contact-mixing models[/-][+]Add latent infection class "MobilitySubpopulationInfections" for contact-mixing models[/+]cdc-mitzimorris commentedon Oct 8, 2026
@dylanhmorris and @KOVALW and @paigemiller I have updated the issue (and its title).
I hope that the proposed computation is the model that you want!
Regrading what to name things, after trying out many alternative names, I think that the proposed names are reasonable. I've gone with "mobility" over "mixing" because this is a specific kind of matrix structure and computation and it jibes with the title of the Roy paper.
Regarding
SubpopulationInfections- it's a simple name for a simple model. Keeping it has the advantage that if anyone outside of CDC is using it, (which would be thrilling!), things won't break and we won't need to write the added code to warn now and remove later. in the end, I just couldn't find a better name - the chatbots objected to "HierarhcialSubpopulationModel" on the grounds that the mobility-matrix model is also hierarchical. The alternatives they suggested were long and ugly.KOVALW commentedon Oct 9, 2026
Thanks @cdc-mitzimorris! The new issue looks good to me to start implementation. I agree that mobility is probably the better term, but my brain has a hard time parsing
MobilitySubpopulationInfections. Let's keep it for now and use documentation to elaborate with something like "mobility-adjusted subpopulation infections"cdc-mitzimorris commentedon Oct 13, 2026
implemented as #909