Background
Issue #38 (double interval censoring, type 6) addresses estimation from historical datasets where all outcomes have been observed. For real-time outbreak response, a further correction is needed: right truncation.
During an ongoing epidemic, individuals who experienced the primary event recently may not yet have experienced the secondary event, so they do not appear in the line list. Ignoring this causes the observed delay distribution to be systematically shorter than the true forward distribution — an effect that grows with the epidemic growth rate and is quantitatively equivalent to dynamical bias (Park et al., 2024, doi:10.1101/2024.01.12.24301247).
This issue proposes a new type 7 data format that extends type 6 by including a per-dataset observation time T, enabling unbiased estimation from line-list data during an ongoing outbreak.
Mathematical formulation
For an individual with exposure window $[E_L, E_R]$, event window $[O_L, O_R]$, and analysis date $T$ (i.e. secondary event must satisfy $S < T$), the likelihood contribution is:
$$L_i = \frac{\displaystyle\int_{E_L}^{E_R} \bigl[F_D(O_R - e) - F_D(O_L - e)\bigr],\mathrm{d}e}{\displaystyle\int_{E_L}^{E_R} F_D(T - e),\mathrm{d}e}$$
The numerator is the type 6 double-censoring likelihood (issue #38). The denominator is the probability that the secondary event is observed before $T$, integrated over uncertainty in the primary event time. Both integrals share the same Gauss-Legendre nodes over $[E_L, E_R]$, so no additional quadrature cost is incurred.
This ratio is numerically equivalent to the approximate latent variable method of Ward et al. (2022), the best-performing method in Park et al. (2024), but implemented via deterministic quadrature rather than MCMC latent variables.
Key equivalences:
- Setting $T = \infty$ (i.e. $F_D(T - e) = 1$ for all $e$) recovers the type 6 likelihood exactly, so type 7 is a strict generalisation of type 6.
- When $E_L = E_R = e_0$ and $O_L = O_R = o_0$, the likelihood reduces to $f_D(o_0 - e_0) / F_D(T - e_0)$, the standard right-truncation correction for point-observed delays.
Relationship to dynamical bias
Park et al. (2024) show that during exponential growth, right truncation of the forward distribution and dynamical bias on the backward distribution are mathematically equivalent. Correcting for right truncation via the denominator above therefore also resolves dynamical bias; the two should not both be corrected for simultaneously.
Proposed implementation
Stan data block — extend the existing type 6 arrays with one new scalar per dataset:
array[n_datasets] real truncation_time; // analysis date T; set to large value to disable
truncation_time[d] is ignored for datasets with summary_type != 7.
Stan likelihood — within the summary_type == 7 branch, evaluate the GL quadrature over $[E_L, E_R]$ twice using the same nodes and weights:
real log_numerator = log(gl_sum(cdf_diff_nodes)); // F(O_R - e) - F(O_L - e)
real log_denominator = log(gl_sum(trunc_cdf_nodes)); // F(T - e)
target += log_numerator - log_denominator;
R interface — new input format branch in prepare_stan_data_from_datasets() accepting a data frame with columns expo_lower, expo_upper, event_lower, event_upper, and a scalar truncation_time per dataset. Individual-level line list data (one row per case) and pre-aggregated frequency tables should both be supported.
Use cases
Standalone line list (single outbreak): pass n_datasets = 1 with summary_type = 7. The between-study heterogeneity parameter tau is prior-dominated for a single dataset (as per the existing n_datasets < 5 handling) and propagated harmlessly.
Combined with historical data: the line list is treated as one dataset alongside published summary entries (types 1–6). The shared phi (shape) parameter implies the current outbreak and historical literature share the same distributional shape — consistent with the existing synthesis assumption, but worth noting in documentation.
Special cases handled
| Observation type |
Likelihood |
| Both endpoints point-observed, no truncation |
$f_D(o_0 - e_0)$ |
| Both interval-censored, no truncation (type 6) |
$\int_{E_L}^{E_R}[F_D(O_R-e)-F_D(O_L-e)],\mathrm{d}e$ |
| Both endpoints point-observed, right-truncated |
$f_D(o_0-e_0) / F_D(T-e_0)$ |
| Both interval-censored, right-truncated (proposed type 7) |
$\int_{E_L}^{E_R}[F_D(O_R-e)-F_D(O_L-e)]\mathrm{d}e \Big/ \int_{E_L}^{E_R}F_D(T-e)\mathrm{d}e$ |
Dependencies
References
- Park et al. (2024). Estimating epidemiological delay distributions for infectious diseases. medRxiv. doi:10.1101/2024.01.12.24301247
- Ward et al. (2022). Transmission dynamics of monkeypox in the United Kingdom. BMJ. doi:10.1136/bmj-2022-073153
Background
Issue #38 (double interval censoring, type 6) addresses estimation from historical datasets where all outcomes have been observed. For real-time outbreak response, a further correction is needed: right truncation.
During an ongoing epidemic, individuals who experienced the primary event recently may not yet have experienced the secondary event, so they do not appear in the line list. Ignoring this causes the observed delay distribution to be systematically shorter than the true forward distribution — an effect that grows with the epidemic growth rate and is quantitatively equivalent to dynamical bias (Park et al., 2024, doi:10.1101/2024.01.12.24301247).
This issue proposes a new type 7 data format that extends type 6 by including a per-dataset observation time
T, enabling unbiased estimation from line-list data during an ongoing outbreak.Mathematical formulation
For an individual with exposure window$[E_L, E_R]$ , event window $[O_L, O_R]$ , and analysis date $T$ (i.e. secondary event must satisfy $S < T$ ), the likelihood contribution is:
The numerator is the type 6 double-censoring likelihood (issue #38). The denominator is the probability that the secondary event is observed before$T$ , integrated over uncertainty in the primary event time. Both integrals share the same Gauss-Legendre nodes over $[E_L, E_R]$ , so no additional quadrature cost is incurred.
This ratio is numerically equivalent to the approximate latent variable method of Ward et al. (2022), the best-performing method in Park et al. (2024), but implemented via deterministic quadrature rather than MCMC latent variables.
Key equivalences:
Relationship to dynamical bias
Park et al. (2024) show that during exponential growth, right truncation of the forward distribution and dynamical bias on the backward distribution are mathematically equivalent. Correcting for right truncation via the denominator above therefore also resolves dynamical bias; the two should not both be corrected for simultaneously.
Proposed implementation
Stan data block — extend the existing type 6 arrays with one new scalar per dataset:
truncation_time[d]is ignored for datasets withsummary_type != 7.Stan likelihood — within the$[E_L, E_R]$ twice using the same nodes and weights:
summary_type == 7branch, evaluate the GL quadrature overR interface — new input format branch in
prepare_stan_data_from_datasets()accepting a data frame with columnsexpo_lower,expo_upper,event_lower,event_upper, and a scalartruncation_timeper dataset. Individual-level line list data (one row per case) and pre-aggregated frequency tables should both be supported.Use cases
Standalone line list (single outbreak): pass
n_datasets = 1withsummary_type = 7. The between-study heterogeneity parametertauis prior-dominated for a single dataset (as per the existingn_datasets < 5handling) and propagated harmlessly.Combined with historical data: the line list is treated as one dataset alongside published summary entries (types 1–6). The shared
phi(shape) parameter implies the current outbreak and historical literature share the same distributional shape — consistent with the existing synthesis assumption, but worth noting in documentation.Special cases handled
Dependencies
References