Skip to content

CoMorph KGO-breaking refactoring - #776

Open
MichaelWhitall wants to merge 36 commits into
MetOffice:mainfrom
MichaelWhitall:comorph_refact3
Open

MichaelWhitall wants to merge 36 commits into
MetOffice:mainfrom
MichaelWhitall:comorph_refact3

Conversation

@MichaelWhitall

@MichaelWhitall MichaelWhitall commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

PR Summary

Sci/Tech Reviewer:
Code Reviewer:

The changes under Issue #178 / PR #292 and Issue #713 / PR #717 left various lines of "temporary code" in place so-as to avoid changing known-good-output under the wholesale refactoring of the CoMorph convection scheme. This included many lines that do nothing scientifically, but are just hacks to force calculations to yield the same floating-point rounding-errors as before. Now we remove these, with no expected science impact, but a KGO-change in CoMorph due to changing rounding-errors in various calculations.

For a detailed description of the changes, see the Issue page: #251

Note: the branch for this PR includes all the changes from #717 as well, since they could not be cleanly separated. Until #717 has been lodged and the head of main has been merged into this PR, the "Files Changed" tab will show the combined diff of this PR and #717. In the mean-time, please review the diff relative to the #717 baseline, not the one in the "Files Changed" tab:
comorph_refact2 -> comorph_refact3

Since we've altered the namelists, this PR requires an upgrade macro to apply the namelist changes to existing apps, and a test branch in-which the upgrade macro has been applied to rose-stem: comorph_refact3 -> comorph_refact3_test

Note since the branch for this PR currently includes namelist changes from both this and #717, there are 2 upgrade macros added in um-convection/versions.py. But I found the apply_macros.py script fails if applying 2 separate macros. Hence the test branch has this file modified to combine the two macros into one. This can be ironed-out once #717 is merged into main and its macro is already applied to all the apps.

closes #251
is blocked-by #717

Code Quality Checklist

  • I have performed a self-review of my own code
  • My code follows the project's style guidelines
  • Comments have been included that aid understanding and enhance the readability of the code
  • My changes generate no new warnings
  • All automated checks in the CI pipeline have completed successfully *

* The check-cla test is failing, even though I have not modified the contributors.md file in this PR (and my name is already added to it on the trunk). This test just seems to be broken.

Testing

  • I have tested this change locally, using the LFRic Apps rose-stem suite
  • If any tests fail (rose-stem or CI) the reason is understood and acceptable (e.g. kgo changes) *
  • I have added tests to cover new functionality as appropriate (e.g. system tests, unit tests, etc.) NA
  • Any new tests have been assigned an appropriate amount of compute resource and have been allocated to an appropriate testing group (i.e. the developer tests are for jobs which use a small amount of compute resource and complete in a matter of minutes) NA

* rose-stem KGO-checks fail for all the comorph runs, as expected.

Tests of the impact of the changes in UM 10-day AMIP runs are documented here:
Testing CoMorph refactoring and bug-fixes
(equivalent UM branches have been used, so not the actual branch for this lfric_apps PR, but the whole comorph directory is identical between the UM and lfric_apps branches).

The differences for the run with the changes from this PR (vn14.2refact3_n96_coma9_tb) look like science-neutral / chaotic small deviations in model-evolution as far as I can tell.

trac.log

Test Suite Results - lfric_apps - comorph_refact3_test/run1

Suite Information

Item Value
Suite Name comorph_refact3_test/run1
Suite User michael.whitall
Workflow Start 2026-09-11T17:04:11
Groups Run developer', 'lfric_atm_nwp_ex1a_extra', 'lfric_atm_nwp_azspice_extra
Dependency Reference Main Like
casim MetOffice/casim@2026.07.1 True
jules MetOffice/jules@2026.07.1 True
lfric_apps MichaelWhitall/lfric_apps@comorph_refact3_test False
lfric_core MetOffice/lfric_core@60fc29a True
moci MetOffice/moci@2026.07.1 True
SimSys_Scripts MetOffice/SimSys_Scripts@77a5166 True
socrates MetOffice/socrates@2026.07.1 True
socrates-spectral MetOffice/socrates-spectral@2026.07.1 True
ukca MetOffice/ukca@9fc2b6d True

Task Information

❌ failed tasks - 14
Task State
check_lfric_atm_nwp_coma9-C12_azspice_gnu_fast-debug-32bit-crun1 failed
check_lfric_atm_nwp_coma9-C12_ex1a_cce_fast-debug-32bit-crun1 failed
check_lfric_atm_nwp_comorph_dev-C12_azspice_gnu_fast-debug-32bit-crun1 failed
check_lfric_atm_nwp_comorph_dev-C12_ex1a_cce_fast-debug-32bit-crun1 failed
check_lfric_atm_nwp_comorph_tb-C12_ex1a_cce_fast-debug-32bit-crun1 failed
check_lfric_atm_scm_coma9_bomex-BiP2x2-50000x50000_azspice_gnu_fast-debug-32bit failed
check_lfric_atm_scm_coma9_bomex-BiP2x2-50000x50000_ex1a_cce_fast-debug-32bit failed
check_lfric_atm_scm_coma9_toga-BiP2x2-50000x50000_azspice_gnu_fast-debug-32bit failed
check_lfric_atm_scm_coma9_toga-BiP2x2-50000x50000_ex1a_cce_fast-debug-32bit failed
check_lfric_atm_scm_comorph_dev_bomex-BiP2x2-50000x50000_azspice_gnu_fast-debug-32bit failed
check_lfric_atm_scm_comorph_dev_bomex-BiP2x2-50000x50000_ex1a_cce_fast-debug-32bit failed
check_lfric_atm_scm_comorph_dev_toga-BiP2x2-50000x50000_azspice_gnu_fast-debug-32bit failed
check_lfric_atm_scm_comorph_dev_toga-BiP2x2-50000x50000_ex1a_cce_fast-debug-32bit failed
kgo_groups_checker failed
✅ succeeded tasks - 1294
⌛ waiting tasks - 2
Task State
housekeep_azspice waiting
housekeep_ex1a waiting

Security Considerations

  • I have reviewed my changes for potential security issues
  • Sensitive data is properly handled (if applicable) NA
  • Authentication and authorisation are properly implemented (if applicable) NA

Performance Impact

  • Performance of the code has been considered and, if applicable, suitable performance measurements have been conducted

No change in model performance expected.

AI Assistance and Attribution

  • Some of the content of this change has been produced with the assistance of Generative AI tool name (e.g., Met Office Github Copilot Enterprise, Github Copilot Personal, ChatGPT GPT-4, etc) and I have followed the Simulation Systems AI policy (including attribution labels)

A Met Office Github Copilot Enterprise AI agent was used in visual studios code to do extract the relevant existing changes from the UM comorph_dev branch into smaller separate UM branches (vn14.2_comorph_refact2 for #717, vn14.2_comorph_refact3 for #717 + this PR) containing only the refactoring changes (however the agent made many errors so heavy manual intervention has been required after running the agent).

Documentation

  • Where appropriate I have updated documentation related to this change and confirmed that it builds correctly

Porting comorph documentation to lfric_apps is still tbd, see #359.

PSyclone Approval

  • If you have edited any PSyclone-related code (e.g. PSyKAl-lite, Kernel interface, optimisation scripts, LFRic data structure code) then please contact the HPC Optimisation Team NA

Sci/Tech Review

  • I understand this area of code and the changes being added
  • The proposed changes correspond to the pull request description
  • Documentation is sufficient (do documentation papers need updating)
  • Sufficient testing has been completed

(Please alert the code reviewer via a tag when you have approved the SR)

Code Review

  • All dependencies have been resolved
  • Related Issues have been properly linked and addressed
  • CLA compliance has been confirmed
  • Code quality standards have been met
  • Tests are adequate and have passed
  • Documentation is complete and accurate
  • Security considerations have been addressed
  • Performance impact is acceptable

…ph_refact2 and ran the lfric_styling.py script.
…d parcel radius option, correct sub-grid fraction treatment for graupel in the stand-alone comoprh test.
…ck. Try use-ing them from comorph_config_mod?
… case where namelist:convection (and hence namelist:comorph too) are not in brackets as well as the existing brackets case (the ngarch rose-stem job doesn't have the brackets so the existing macro breaks it).
… the match as that's not there in source after parsing by self.get_setting_value).
…the various example configuration .nml files (none of these use comorph so the new namelist doesn't need to be added to any of them.
…ackets even if namelist:convection isn't, to indicate that its optional.
… and just specify the whole string to be added (1 preceding space at start, newline at end) instead of matching the previous line.
…core calculations inside subroutine core_combine, reverted AI changes to comments).
…e newly-added lfric_apps comorph namelist (VScode github copilot helped with this).
…o the namelist (par_radius_knob slightly changes value when read in as 0.45 from the namelist at 32-bit and then converted to 64-bit, compared to when it was hardwired to 0.45 directly at 64-bit).
…ained air properties inside set_ent, renamed hydrometeor species indices in collision_ctl, harmonised some more comments.
…set kind of par_radius_knob to double in the first place, which should mean we no-longer need the check to set it to exactly 0.45 in um_physics_init_mod (so deleted that).
…or code-style / comment changes, (b) moved autoconversion code from microphysics_2 inside its own subroutine, (c) moved setting of core_ent diagnostic from conv_level_step inside set_ent, (d) added / corrected chk_var calls in comorph_um_namelist_mod...
…ibility of multiple condensate species contributing to the cloud-fractions (needed when switching on the 2nd ice category for use with CASIM). Also deleted stray module use of set_par_cloudfrac from conv_level_step (it isn't used in that routine yet).
…t test for phase_change_solve, and a few more code-style changes to harmonize with the comorph_dev branch.
… double to reproduce KGO in MetOffice#717; now we're breaking the KGOs anyway we may-as-well set this the same as the rest of the comorph namelist for consistency.
…omorph namelist, with sort-keys set to put them next to ass_min_radius.
@MichaelWhitall MichaelWhitall added this to the Autumn 2026 milestone Sep 11, 2026
@MichaelWhitall MichaelWhitall self-assigned this Sep 11, 2026
@MichaelWhitall MichaelWhitall added KGO This PR contains changes to KGO macro This PR contains a metadata upgrade macro labels Sep 11, 2026
@github-actions github-actions Bot added the cla-modified The CLA has been modified as part of this PR - added by GA label Sep 11, 2026
@github-actions

Copy link
Copy Markdown

⚠️ Hello MichaelWhitall!

Your CLA signature was found on the base branch, but you appear to have modified the CONTRIBUTORS.md file in this PR.

Please do not edit the CONTRIBUTORS.md file. If you have already signed the CLA, revert changes to the file and your signature will be picked up.

@DanStoneMO DanStoneMO left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested with JEDI and it works fine, no linked PR needed for it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

KGO This PR contains changes to KGO macro This PR contains a metadata upgrade macro

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CoMorph KGO-breaking refactoring

4 participants