Skip to content

feat: dcc_ date consistency comparison suite - #33

Merged
danalvarez merged 9 commits into
developfrom
feature/dcc_date_consistency_comparison
Jun 1, 2026
Merged

danalvarez merged 9 commits into
developfrom
feature/dcc_date_consistency_comparison

Conversation

@ixaegir

@ixaegir ixaegir commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds four new dcc_ functions for comparing chronological consistency between two date variables in the EIR
  • Adds English vignette documenting the full workflow
  • Adds CHANGELOG.md following the Keep a Changelog format

Changes

Added

  • dcc_rate() — consistency rates (CONSISTENT / INCONSISTENT / DATE_MISSING) disaggregated by ADM0/ADM1/ADM2
  • dcc_barplot() — stacked bar chart of rates; date_1_name / date_2_name params for subtitle
  • dcc_inconsistent() — record-level table of inconsistent and missing entries with day difference
  • dcc_inconsistent_plot() — histogram of absolute day differences; customizable bins via day_groups list of c(lower, upper) pairs, default 10-day bins 0–100 with 100+ catch-all
  • vignettes/date_consistency_comparison_en.Rmd — full workflow vignette
  • CHANGELOG.md — Keep a Changelog format

Test plan

  • dcc_rate() returns correct counts and rates for all three consistency categories
  • dcc_rate() geo_level ADM0 / ADM1 / ADM2 all produce expected groupings
  • dcc_barplot() renders correctly; subtitle reflects date_1_name / date_2_name; plot_missing = FALSE excludes grey segment
  • dcc_inconsistent() returns only INCONSISTENT and DATE_MISSING rows; diff column sign is correct
  • dcc_inconsistent_plot() default bins render; custom day_groups override renders correctly
  • Vignette knits without errors

🤖 Generated with Claude Code

ixaegir and others added 5 commits May 19, 2026 16:18
@ixaegir
ixaegir requested a review from danalvarez June 1, 2026 17:31
ixaegir and others added 2 commits June 1, 2026 11:36
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Comment thread R/validation.R Outdated
if(!all(required_columns %in% names(data.EIR))) {
stop("Error: data.EIR should contain the following columns: ID, date_birth, date_vax, ADM1_residence, ADM2_residence, ADM1_occurrence, ADM2_occurrence, dose.")
}
if(!is.null(indicator) && indicator == "dcc"){

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be nicer to make the print be built automatically from the vector of expected columns. Especially now that we might define different required columns for each module.

Comment thread R/dcc_rate.R Outdated
@@ -0,0 +1,55 @@
#' (D)ate (C)onsistency (C)omparison Rate
#'
#' This function calculates the consistency comparison rate between two date variables, further disagreggating by geographic level, using data from the electronic immunization registry (EIR).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think "consistency comparison rate" sounds a little weird? Maybe just "consistency rate"?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clarifying: The dcc name seems fine to me. But in the docs when you say "This function calculates the consistency comparison rate...", maybe it would be best to say "This function calculates the rate of consistency between two date variables, further disaggregating..."

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

# Glossary

## Political Geographic Boundaries

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For discussion: the other vignettes used a slightly expanded version of this section. Should we normalize to match the other vignettes? (i.e. copy-paste)


#### Interpretation

The x-axis shows the absolute number of days between the two dates, regardless of direction (i.e., whether `date_1` precedes `date_2` or vice versa). Records concentrated in the first bins (0–9 days) are likely simple data entry errors, while records in the 100+ category may indicate systematic record linkage problems.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Records with 100+ days could also be typos. For example, if someone types "2022-02-25" instead of "2022-12-25".

I would expand this explanation to: "The horizontal axis shows the number of days between inconsistent dates. An investigation into the most common causes for the inconsistencies in each bin should be performed. For example, records with large differences can be caused by typos during data entry (e.g. when a date is written as 2022-02-25 instead of 2022-12-25. But, they can also point to more serious systematic problems."

Comment thread R/dcc_inconsistent.R Outdated
@@ -0,0 +1,34 @@
#' (D)ate (C)onsistency (C)omparison Inconsistent Registriest

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: Regisitries. Also, would call it "Inconsistent Records" instead of "Registries"

Comment thread R/dcc_inconsistent.R Outdated
@@ -0,0 +1,34 @@
#' (D)ate (C)onsistency (C)omparison Inconsistent Registriest
#'
#' This function lists the inconsistent records between two date variables, further disagreggating by geographic level, using data from the electronic immunization registry (EIR).

@danalvarez danalvarez Jun 1, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would improve docs to match what was written in the vignette: "This function returns a record-level table of all inconsistent and "date missing" entries, including the day difference between the two dates."

Comment thread CHANGELOG.md Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets keep the changelog in the gh releases for now. Not in a file within the repo


## Date Consistency Comparison Analysis

### Expected Workflow

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing the "expected workflow" block diagram we have in the other vignettes. Can be put after the bullets and just say "The figure below shows the expected workflow".

@danalvarez

Copy link
Copy Markdown
Contributor

Aside from the comments I alread put in, the rest LGTM. Will have to update README to include this new module, but that can be done in develop without the need for a PR.

- Auto-build validation error message from required_columns vector
- Fix dcc_rate description wording
- Fix dcc_inconsistent title typo and improve description
- Expand vignette glossary to match other vignettes
- Add expected workflow diagram reference to vignette
- Update inconsistent_plot interpretation text
- Remove CHANGELOG.md (use GH releases instead)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@danalvarez

Copy link
Copy Markdown
Contributor

Channeling my stackoverflow admin energy: Missing .svg for dcc_workflow but I'll let it slide for now. Must be added before next release.

LGTM. Merged.

@danalvarez
danalvarez merged commit d4d03f1 into develop Jun 1, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants