Skip to content

Compare strategies with a reference in calculate_icers() and print.icers() (#209, #200) - #227

Open
feralaes wants to merge 20 commits into
mainfrom
209-reference-strategy-core
Open

feralaes wants to merge 20 commits into
mainfrom
209-reference-strategy-core

Conversation

@feralaes

@feralaes feralaes commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

First PR of the reference-strategy series (#209), the second headline feature of v1.1.0. This one covers the ICER table: calculate_icers() and print.icers().

Part of #209 (next: plot.icers()/plot.psa(), then the PSA summaries). Closes #200.

This branch now also contains #228 (plots), #229 (PSA summaries), #242 (currency), #250 (log axes) and #251 (API harmonization), which were merged into it. So merging this PR also closes:

#209 stays open for its vignette section (#216).

Example

calculate_icers(cost, effect, strategies,
                ref_str = "Standard of care",   # compare every strategy with this one
                wtp = 50000,                    # adds NMB (and iNMB vs the reference)
                currency = "€")
print(x)                     # the usual sequential table (default)
print(x, ref = "reference")  # every strategy vs the reference
print(x, ref = "both")

On the Hunink example (hund_strat, first 5 strategies, strategy "1" as reference):

  Strategy Costs ($) QALYs Incremental Costs vs 1 ($) Incremental QALYs vs 1 ICER vs 1 ($/QALY)                 Status vs 1 NMB ($) iNMB vs 1 ($)
1        2     1,681 0.295                     -4,501                 -0.412             10,922 Less costly, less effective  13,067       -16,104
2        1     6,181 0.707                          -                      -                  -                   Reference  29,172             -
3        3     9,485 0.724                      3,303                  0.017            192,109 More costly, more effective  26,728        -2,443
5        5     6,774 0.117                        593                 -0.590                  -                   Dominated    -909       -30,081

Design, as agreed on #209

  • The frontier is never changed. Status, Inc_Cost, Inc_Effect and ICER are computed exactly as before. The comparison with the reference is added afterwards as separate columns. This avoids the problem with the 2019 ref_strat option, which re-sorted strategies before the dominance loop.
  • Reference columns appear only when ref_str is given:
    • Inc_Cost_ref, Inc_Effect_ref.
    • ICER_ref: numeric, and NA where a ratio isn't meaningful (the reference itself, and strategies that dominate, are dominated, or are equivalent).
    • Status_ref: one of "Reference", "Dominant", "Dominated", "More costly, more effective", "Less costly, less effective".
  • Net benefit with wtp: NMB by default, plus iNMB_ref with a reference. nhb = TRUE adds NHB and iNHB_ref.
  • print.icers():
    • Shows the sequential table by default. ref = "reference" / "both" show the comparison, and the headers name the reference strategy.
    • Uses the currency stored by calculate_icers(currency = ).
    • New arguments are added after the existing ones, so existing calls keep working.

⚠️ One label to confirm: "Equivalent"

A strategy with exactly the same cost and effect as the reference fits neither "Dominant" nor "Dominated". I labeled it "Equivalent" (with ICER_ref = NA). Please confirm, or suggest a different label.

Backward compatibility

  • Unchanged output: with the default arguments, calculate_icers() returns exactly what it did before, with no new columns or attributes. The outputs of every vignette and help-page example are identical to main; the one exception is the new example added to ?calculate_icers.
  • Same printed tables: print.icers() prints identical tables to main's version for existing objects (standard tables, PSA tables with uncertainty columns, custom currency and digits). I compared them directly.

Tests

New test_icers_reference.R:

  • Hand-calculated values for a toy example with one strategy in every Status_ref category.
  • A dominated reference, and the most expensive strategy as reference. The frontier doesn't change with the reference.
  • hund_strat.
  • NMB by default, and NHB only with nhb = TRUE.
  • One and two strategies, and names with spaces.
  • Errors for an unknown ref_str, several ref_str, a vector or negative wtp, and nhb without wtp.
  • The three print views, and the error when no reference was set.

Verification

calculate_icers() gains ref_str, wtp, nhb, and currency (added after the
existing arguments, so positional calls keep working). The frontier
(Status, Inc_Cost, Inc_Effect, ICER) is computed exactly as before and is
never changed; the new columns are added afterwards by
add_reference_and_nmb(), and only for the arguments that are given, so
with the defaults the object is identical to before.

- ref_str: Inc_Cost_ref, Inc_Effect_ref, ICER_ref (numeric, NA where a
  ratio is not meaningful), and Status_ref ('Reference', 'Dominant',
  'Dominated', 'More costly, more effective', 'Less costly, less
  effective', or 'Equivalent' for the same cost and effect) (#209).
- wtp: NMB, plus iNMB_ref with a reference (#200); nhb = TRUE also adds
  NHB and iNHB_ref.
- currency: stored on the object and used by print.icers().
- print.icers() gains ref = c('sequential', 'reference', 'both'); the
  default prints the same table as before. Headers name the reference
  ('Incremental Costs vs Standard of care ($)').
- Validation helpers check_ref_str() and check_wtp_nhb() with clear
  errors; a new example in ?calculate_icers.

Tests (test_icers_reference.R) check hand-calculated values for every
Status_ref category, a dominated and a most-expensive reference,
hund_strat, NMB/NHB, one and two strategies, names with spaces, the
errors, and the print views. The default printed tables are identical to
main's print.icers().

Part of #209. Closes #200
…lines

- plot.icers() gains ref_str, wtp, ref_axis, and ref_caption (after the
  existing arguments). ref_str shifts the plot so the reference strategy
  is at the origin (the frontier and statuses are unchanged); wtp draws a
  dotted line with slope wtp through the reference, or without ref_str
  through the least costly frontier strategy. The currency defaults to
  the one stored by calculate_icers() (#209, #202).
- plot.psa() gains wtp, which draws the same line on its incremental
  plane (requires ref_str) (#202).
- plot.psa(ref_str = ) with two strategies: keep a data frame when one
  comparator is left (drop = FALSE); it failed with "'dims' cannot be of
  length 0". ref_str is now checked with check_ref_str().
- Captions name the reference and the WTP threshold; this also removes a
  double space in plot.psa()'s 'Reference Strategy:  X' caption.
- Tests (test_reference_plots.R) check the plotted coordinates, axis
  labels, captions, WTP line slope and intercept, currency, the
  two-strategy case, and the errors. Default plots are unchanged.

Part of #209
…_icers_psa()

- summary.psa() gains ref_str and wtp (after calc_sds). With ref_str it
  adds meanIncCost, meanIncEffect, and the share of PSA samples in each
  quadrant of the incremental CE plane (propNE, propSE, propSW, propNW;
  NA for the reference). With wtp it adds meanNMB, and with ref_str also
  meanIncNMB and propCostEffective (share of samples with a higher NMB
  than the reference) (#209, #202).
- calculate_icers_psa() gains ref_str, wtp, and nhb, passed to
  calculate_icers(). With uncertainty = TRUE and ref_str, it adds 95%
  intervals of the sample-by-sample differences from the reference
  (Lower_95_/Upper_95_ for Inc_Cost_ref, Inc_Effect_ref, iNMB_ref). The
  icers class and attributes are now kept through the uncertainty joins.
- print.icers(): every *_ref column belongs to the reference view.
- Docs: escape % in roxygen. An unescaped % starts an Rd comment, which
  had silently truncated ?calculate_icers_psa ('whether or not 95') and
  ?predict.metamodel.

calculate_icers_psa() output for existing calls is identical to main's
(objects, class, and printed tables, with and without uncertainty).

Part of #209. Closes #202
Incremental CE plane and WTP lines in plot.icers() and plot.psa() (#209, #202)
Reference strategy and quadrant shares in summary.psa() and calculate_icers_psa() (#209, #202)
@feralaes
feralaes requested review from krijkamp and mbkauf November 12, 2026 22:59
…fix entry first in Bug fixes so the open PRs merge in any order
- ceac(), calc_exp_loss(), calc_evpi(), calc_evppi(), calc_evsi(), and
  calculate_icers_psa() store psa$currency on their results.
- Their plot() methods (and print.icers()) use it: currency defaults to
  NULL, resolved by resolve_currency() as the argument if given, else the
  result's currency, else '$'. Existing calls keep working.
- plot.ceac() gains effect_units (the x-axis label hard-coded QALY).
- EVPPI and EVSI computed from net health benefit are in health units:
  calc_evppi() and calc_evsi() store the outcome, and their plots are
  labeled 'EVPPI (QALY)' / 'EVSI (QALY)' instead of a currency.
  calc_evsi() defaults to outcome = 'nhb', so its default plot was
  mislabeled in dollars.
- Tests (test_currency.R) with a euro PSA for every function, overrides,
  effect_units, and dollar defaults; two existing tests now build the
  expected object with the same currency.

Outputs of all vignettes and examples change only by the new currency
attribute; values, plot data, and labels are unchanged.

Closes #203
Carry the currency of a psa object through to its analyses (#203)

@mbkauf mbkauf left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

icers.R

  1. Printing with reference strategy has the strategies in the same order as without a reference strategy. So, the example code in the roxygen has the reference strategy is "1", but it ends up being the 55th strategy printed in the table. It should reorder with the reference strategy at the top? Or at least with some other ordering with respect to the reference strategy.
  2. In calculate_icers() lines 117 to 124, there is a chunk of code for if only 1 strategy is provided. It returns a data frame, but should it just return an error that more than 1 strategy must be provided?
  3. Are lines 103, 369-370 necessary?
  4. General naming conventions as it relates to capitalization. Should everything be lower case unless it's for formatting purposes to display.
  5. Line 406, what is everything() doing? Is it necessary?
  6. Add comments to explain code in add_reference_and_nmb() and add_reference_intervals() functions. In general, more commenting with newly written code would be good.
  7. Do we want incremental net monetary and incremental net health benefit to have the lower case "i" in the name (e.g. iNMB and iNHB)?

psa.R

  1. Lines 219-222, why does having a WTP threshold require a reference strategy be supplied?

…ng, wtp without ref_str in plot.psa

- print.icers(ref = "reference") lists the reference first and the other
  strategies by increasing cost; "sequential" and "both" keep the frontier order.
- Rename iNMB_ref/iNHB_ref to Inc_NMB_ref/Inc_NHB_ref, matching Inc_Cost_ref,
  Inc_Effect_ref and summary.psa()'s meanIncNMB. Print headers say "Incremental NMB/NHB".
- plot.psa(wtp = ) no longer requires ref_str: without one, the line goes through the
  mean of the least costly strategy, as plot.icers() already did.
- Comment add_reference_and_nmb(), add_reference_intervals(), the global-variable
  declarations, and the everything() in calculate_icers_psa().
@feralaes

Copy link
Copy Markdown
Collaborator Author

@mbkauf thanks for the careful review. Point by point; the changes are in 308850f.

icers.R

  1. Reference strategy at the top. Agreed. print(x, ref = "reference") now lists the reference first and the other strategies by increasing cost. Only the printed table is reordered, not the object. ref = "sequential" and ref = "both" keep the frontier order, because the sequential ICERs are read down that order. There's a new test using the roxygen example (reference "1", which was row 55 of 100).
  2. One strategy. Kept on purpose. Returning a valid icers object for a single strategy is the fix for Return a valid icers object from single-strategy calculate_icers #183, so code that filters down to one strategy (a subgroup, or a single-strategy PSA) can still print and plot the result instead of failing. A test covers it. If the group would rather have an error, that reverses Return a valid icers object from single-strategy calculate_icers #183, so I'd raise it there.
  3. Lines 103 and 369–370. These are needed. They declare the column names used unquoted in the dplyr code; without them R CMD check reports "no visible binding for global variable". I checked that every name in both lists is used that way. I added a comment saying so. Moving them into one utils::globalVariables() call would be a reasonable cleanup later.
  4. Capitalization. Internal variables follow the DARTH lowercase prefixes. The column names users see (Strategy, Cost, Inc_Cost, ICER, Status, meanCost) were all released in 1.0, so renaming them would break users' scripts. The new columns follow the existing style of each object. I'd put lowercase column names on the list for the 2.0 naming decision rather than change them in 1.1.0.
  5. everything() on line 406. Needed. It keeps the reference and net-benefit columns after the listed ones; select() drops them otherwise (I checked). There's now a comment saying that.
  6. Comments. Added throughout add_reference_and_nmb() and add_reference_intervals().
  7. iNMB / iNHB. Agreed. They're now Inc_NMB_ref and Inc_NHB_ref, which matches Inc_Cost_ref, Inc_Effect_ref, and meanIncNMB in summary.psa(). The printed headers say "Incremental NMB" and "Incremental NHB". Nothing has been released, so the rename costs nothing.

psa.R

  1. WTP requiring a reference strategy. You were right, and it was also inconsistent: plot.icers(wtp = ) already worked without ref_str, drawing the line through the least costly strategy on the frontier. plot.psa() now does the same. Without ref_str the line goes through the mean cost and effect of the least costly strategy, and the docs and NEWS say so. With ref_str it still goes through the origin of the incremental plane. Test added.

Tests: 343 passing, 0 lints, R CMD check --as-cran with only the development-version NOTE. Apart from the renamed column, example and vignette output is unchanged.

…rtainty; drop context() for testthat 3e

- plot.icers() now always reduces x to its seven plot columns as a plain data frame.
  dplyr::select() had dropped the stored currency attribute only when uncertainty
  bounds were present, so the two plots differed under testthat's edition-3 comparison.
- Replace context() in the four test files added by this PR (deprecated in edition 3).
@codecov

codecov Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.25094% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.83%. Comparing base (70fda2e) to head (6afb4a7).
⚠️ Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
R/add_common_aes.R 80.00% 1 Missing ⚠️
R/icers.R 99.34% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #227      +/-   ##
==========================================
+ Coverage   93.09%   93.83%   +0.74%     
==========================================
  Files          18       18              
  Lines        1824     2028     +204     
==========================================
+ Hits         1698     1903     +205     
+ Misses        126      125       -1     
Files with missing lines Coverage Δ
R/ceac.R 90.90% <100.00%> (+0.58%) ⬆️
R/evpi.R 96.15% <100.00%> (+0.32%) ⬆️
R/evppi.R 95.45% <100.00%> (+0.58%) ⬆️
R/evsi.R 87.69% <100.00%> (+0.49%) ⬆️
R/exp_loss.R 100.00% <100.00%> (ø)
R/metamodel.R 93.26% <ø> (ø)
R/owsa.R 93.05% <100.00%> (+0.13%) ⬆️
R/psa.R 96.95% <100.00%> (+2.91%) ⬆️
R/twsa.R 93.44% <100.00%> (+0.10%) ⬆️
R/add_common_aes.R 88.81% <80.00%> (-0.32%) ⬇️
... and 1 more
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…scale (#201)

The ellipses were computed on the data scale, so on log axes (added with scale_*_log10())
they lost their shape and any part below zero became NaN. plot.psa() now takes xtrans and
ytrans, passes them to its own scales, and computes each ellipse from the transformed
samples before mapping it back. Values a transformation cannot show (zero or negative on a
log scale, common on the incremental plane) give an informative error. With the default
identity transformations the plots are unchanged. The duplicated ellipse code of the two
planes is now one helper, psa_ellipses().
…idth, axis limits, effect_units

- ceac() takes the psa object first, like calc_exp_loss() and calc_evpi(). The old positional
  ceac(wtp, psa) still works with lifecycle::deprecate_soft(); named calls work in any order.
- plot.owsa() and plot.exp_loss() take linewidth; size and lsize still work, deprecated.
- plot.owsa() and plot.twsa() gain xlim/ylim, owsa_opt_strat() gains xlim.
- effect_units defaults to "QALYs" in every plot; per_unit() gives the singular for
  willingness-to-pay labels, so default labels are unchanged ("$/QALY").
- lifecycle (>= 1.0.0) added to Imports.
- API style decision record in .github/CONTRIBUTING.md, including the open 2.0 questions.
feralaes and others added 5 commits November 13, 2026 17:37
plot.psa(): log axes with correctly shaped ellipses (xtrans, ytrans) (#201)
The minimum-dependency CI job could not install ggplot2@3.5.0 with lifecycle@1.0.0.
Current dplyr needs lifecycle (>= 1.0.5) and ggplot2 3.5.0 needs lifecycle (> 1.0.1), so
any lower pin makes the minimum-dependency environment unsolvable. dampack only uses
deprecate_soft(), deprecated() and is_present(), which predate lifecycle 1.0.
Harmonize the API with soft deprecation: ceac(psa, wtp), linewidth, axis limits, effect_units (#214)
feralaes added a commit that referenced this pull request Nov 14, 2026
Resolve R/ceac.R: keep #251's argument order (psa, then wtp) and this PR's @concept in
place of the multi-word @Keywords. Examples now show the documented ceac(psa, wtp) order.
@feralaes

Copy link
Copy Markdown
Collaborator Author

Hi @mbkauf and @krijkamp, Fernando asked me (Claude, working in his session) to send a short guide for reviewing this PR. It's the one most other work is waiting on: three more PRs are built on top of it. GitHub won't let Fernando approve his own PR, so it needs an approval from one of you.

You don't need to read all the code. The most useful review is to run it and check that the results and wording make sense.

1. Try it (about 5 minutes)

In R:

remotes::install_github("DARTH-git/dampack", ref = "209-reference-strategy-core")
library(dampack)
data(cancer_psa_data)
psa <- make_psa_obj(cost = cancer_psa_data$cost,
                    effectiveness = cancer_psa_data$effectiveness,
                    parameters = cancer_psa_data$parameters,
                    strategies = cancer_psa_data$strategies)

icers <- calculate_icers_psa(psa, ref_str = "Radio", wtp = 100000)
print(icers)                    # the usual sequential table (unchanged)
print(icers, ref = "reference") # every strategy vs Radio, Radio listed first
plot(icers, ref_str = "Radio", wtp = 100000)

summary(psa, ref_str = "Radio", wtp = 100000)  # mean differences and quadrant shares
plot(psa, ref_str = "Radio", wtp = 100000)     # incremental CE plane with a WTP line

2. What to look at

Matt:

  • I answered your 8 points in this reply. Five are changed in the code and two are explained. The wtp point turned out to be right, so it's changed too.
  • Please check that you're happy with how each was handled.
  • Then check the plots. You built the original plot.psa(ref_str).

Eline: the methods, as agreed on #209:

  • ICER_ref is NA for the reference and for dominant, dominated and equivalent strategies.
  • Status_ref uses these labels: Reference, Dominant, Dominated, "More costly, more effective", "Less costly, less effective", Equivalent.
  • NMB and Inc_NMB_ref appear only when wtp is given. NHB appears only with nhb = TRUE.
  • In summary.psa(), propNE/propSE/propSW/propNW are the shares of PSA samples in each quadrant of the incremental plane, and propCostEffective is the share with a positive incremental NMB.
  • If anything looks wrong to you as a health economist, that's exactly what we need to hear.

Both:

  • The sequential ICERs (print(icers) without ref =) should look exactly as before. The new features are only added on top.

3. How to leave your review on GitHub

  1. Click the Files changed tab at the top of this PR.
  2. (Optional) To comment on a specific line, hover over it, click the blue +, type your comment and click Start a review.
  3. Click the green Review changes button at the top right, and choose:
    • Approve if it's fine, or fine with small comments;
    • Request changes if something must be fixed before merging;
    • Comment for questions only.
  4. Click Submit review.

A plain comment at the bottom of this page also works. Fernando or I will turn it into changes.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants