Add cohort health board breakdown commands (final-list + build tree)#2367
Closed
mtinti wants to merge 5 commits into
Closed
Add cohort health board breakdown commands (final-list + build tree)#2367mtinti wants to merge 5 commits into
mtinti wants to merge 5 commits into
Conversation
Two related commands that break an RDMP cohort down by Scottish health board, using a demography catalogue's region cipher column (e.g. SHARE_Demography.Region), plus a shared Region -> board/node lookup. - HealthBoardLookup: hardcoded Region cipher -> health board (name, HB code) and safe-haven node; unmapped/NULL regions resolve to an Unknown node so counts are never silently dropped. - ExecuteCommandExportCohortHealthBoardBreakdown (+ CIC overload): per-board COUNT(DISTINCT chi) of a cohort's final inclusion list, joined to the demography catalogue, with per-node subtotals, an Unknown bucket and a grand total. Works from a committed ExtractableCohort or a CohortIdentificationConfiguration. - ExecuteCommandExportCohortBuildHealthBoardBreakdown: reproduces the Cohort Builder's per-set/per-container FinalCount and cumulative running totals split by health board (long-format CSV). Operates purely on the query cache: builds once to populate the per-set cache tables, then recomposes each count point from those tables and splits it by region (one GROUP BY per node, all boards at once) - no per-board rebuild, no source-server hits after the single build. Requires a query caching server and the demography catalogue on the cache server. - Tests: no-DB unit tests for the lookups/reports/SQL plus database-backed end-to-end tests (committed cohort, CIC, and a deterministic build-tree fixture asserting the national + per-board cumulative tree reconciles at every node). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0169JCnaL3fhhZjseDx2XXT2
Reshape the build-breakdown output from long to a wide matrix: one row per (count-point x metric), container/set name once, a Total column (RDMP's own national count), one column per Scottish health board, plus an Other column (present non-Scottish / unmapped region codes, previously hidden inside Unknown), a NotKnown residual (not-in-demography + NULL region), a Metric column (Final + Cumulative) and a bottom "% of final cohort" row. Boards + Other + NotKnown reconcile to Total on every row. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0169JCnaL3fhhZjseDx2XXT2
RDMP prefixes cohort set names with cic_<ID>_ and cloning across CICs stacks them (e.g. cic_18286_cic_18284_cic_17950_People in SHARE...). The build breakdown now strips those prefixes for display, so the report shows the real name (e.g. "People in SHARE Current For Contact [Recruitment]", "Excl Grp 1 and 2"). Adds a unit test for the cleaner, moves the no-DB report tests into a plain (non-database) fixture, and adds a Retry + clear guard on the DB integration test to absorb transient local-SQL errors. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0169JCnaL3fhhZjseDx2XXT2
The wide build-breakdown output now repeats the column header just above the "% of final cohort" row, so each percentage lines up under its health board. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0169JCnaL3fhhZjseDx2XXT2
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| cacheServer.SetProperties(cacheDb); | ||
|
|
||
| // ---- synthetic data (see fixture doc) ---- | ||
| var demog = new DataTable(); |
| [Test] | ||
| public void BuildRecords_GroupsByNode_Subtotals_UnknownLast_AndGrandTotal() | ||
| { | ||
| var dt = new DataTable(); |
| { | ||
| // cohort private ids seeded by TestsRequiringACohort: | ||
| // Priv_12345, Priv_66666, Priv_54321, Priv_66999, Priv_14722, Priv_wtf11 (6 patients) | ||
| var demo = new DataTable(); |
| var db = GetCleanedServer(FAnsi.DatabaseType.MicrosoftSQLServer); | ||
|
|
||
| // patients catalogue = the cohort source (everyone here is in the cohort) | ||
| var patients = new DataTable(); |
| patientChi.SaveToDatabase(); | ||
|
|
||
| // demography: maps some patients to boards; c6 absent (-> shortfall), cX not in cohort | ||
| var demo = new DataTable(); |
Comment on lines
+212
to
+221
| foreach (var line in lines.Skip(1)) | ||
| { | ||
| var c = line.Split(','); | ||
| if (c[0] == "Order") | ||
| continue; // the header is repeated just above the percentage row | ||
| if (c[_col["Metric"]] == CohortBuildHealthBoardBreakdownReport.PercentMetric) | ||
| _percent = c; | ||
| else | ||
| _data.Add(c); | ||
| } |
Comment on lines
+215
to
+220
| foreach (var line in lines.Skip(1)) // skip header | ||
| { | ||
| var cells = line.Split(','); | ||
| // Region,HBCode,HBName,Node,Count | ||
| result[cells[2]] = int.Parse(cells[4]); | ||
| } |
Comment on lines
+81
to
+84
| if (boards.TryGetValue(key, out var existing)) | ||
| boards[key] = (existing.Hb, existing.Count + count); | ||
| else | ||
| boards[key] = (hb, count); |
|
|
||
| _toFile ??= BasicActivator.IsInteractive | ||
| ? BasicActivator.SelectFile("Path to write build health board breakdown to", "Build health board breakdown", "*.csv") | ||
| : new FileInfo(Path.Combine(System.Environment.CurrentDirectory, $"{Sanitise(_cic.Name)}-build-healthboard.csv")); |
|
|
||
| ToFile ??= BasicActivator.IsInteractive | ||
| ? BasicActivator.SelectFile("Path to write health board breakdown to", "Health board breakdown", "*.csv") | ||
| : new FileInfo(Path.Combine(System.Environment.CurrentDirectory, $"{Sanitise(SourceName)}-healthboard.csv")); |
11 tasks
Contributor
Author
|
Closing in favour of the plugin package PR #2368 (built .rdmp + docs). The source remains on the branch if a source contribution is preferred later. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed Change
Adds atomic commands that break an RDMP cohort down by Scottish health board, using a demography
catalogue's region-cipher column (e.g.
SHARE_Demography.Region), plus a shared region → board/nodelookup. Two related capabilities:
Final-list breakdown —
ExecuteCommandExportCohortHealthBoardBreakdown(a committedExtractableCohort) andExecuteCommandExportCicHealthBoardBreakdown(aCohortIdentificationConfiguration): per-boardCOUNT(DISTINCT chi)of the final inclusion list,with per-node subtotals, an
Unknownbucket and a grand total.Build-tree breakdown —
ExecuteCommandExportCohortBuildHealthBoardBreakdown: reproduces theCohort Builder's per-set / per-container
FinalCountand cumulative running totals split byhealth board (wide CSV). It builds the cohort once to populate the query cache, then recomposes
every count point purely from the cached per-set identifier tables and splits each by region with
one
GROUP BYper node — no per-board rebuild, no source-catalogue hits after the single build.HealthBoardLookupmaps each region cipher to a board (name, HB code) and safe-haven node; unmapped /NULL regions resolve to an
Unknown/Otherbucket so counts always reconcile to the national total.Notes for reviewers: this is domain-specific (Scottish health boards;
SHARE_Demography/Regionare defaults, overridable via command arguments). Happy to move it behind a plugin instead of
Rdmp.Coreif that's preferred — opening as a draft to discuss placement. Both commands work from theCLI and the GUI right-click menus. Validated against deterministic synthetic fixtures (see the DB tests).
Type of change
What types of changes does your code introduce? Tick all that apply.
Checklist
By opening this PR, I confirm that I have:
Rdmp.Core.Tests/CohortCreation)Documentation/if the feature stays in core