Skip to content

CoMorph further KGO-preserving refactoring - #717

Open
MichaelWhitall wants to merge 38 commits into
MetOffice:mainfrom
MichaelWhitall:comorph_refact2
Open

MichaelWhitall wants to merge 38 commits into
MetOffice:mainfrom
MichaelWhitall:comorph_refact2

Conversation

@MichaelWhitall

@MichaelWhitall MichaelWhitall commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

PR Summary

Sci/Tech Reviewer: AlisonStirling
Code Reviewer: Ed Hone (@EdHone)

The bulk of the refactoring of CoMorph in the comorph_dev branch was lodged in the lfric_apps trunk in Issue #178 / PR #292. But some more KGO-preserving refactoring has been done in the comorph_dev branch after the branch lodged for #292 was split-off from it. We now need to lodge these extra refactoring changes. We also need to add more of CoMorph's tune-able constants to the LFRic namelist.

For a detailed description of the changes, see the Issue page: #713.

Since we've altered the namelists, this PR requires an upgrade macro to apply the namelist changes to existing apps, and a test branch in-which the upgrade macro has been applied to rose-stem: comorph_refact2 -> comorph_refact2_test

closes #713
blocks #776

Code Quality Checklist

  • I have performed a self-review of my own code
  • My code follows the project's style guidelines
  • Comments have been included that aid understanding and enhance the readability of the code
  • My changes generate no new warnings
  • All automated checks in the CI pipeline have completed successfully *

* The check-cla test is failing, even though I have not modified the contributors.md file in this PR (and my name is already added to it on the trunk). This test just seems to be broken.

Testing

  • I have tested this change locally, using the LFRic Apps rose-stem suite
  • If any tests fail (rose-stem or CI) the reason is understood and acceptable (e.g. kgo changes) NA
  • I have added tests to cover new functionality as appropriate (e.g. system tests, unit tests, etc.) NA
  • Any new tests have been assigned an appropriate amount of compute resource and have been allocated to an appropriate testing group (i.e. the developer tests are for jobs which use a small amount of compute resource and complete in a matter of minutes) NA

Note: while all the comorph apps in the lfric_apps rose-stem tests preserved KGO exactly (as did all the UM rose-stem tests with the same changes to comorph in that model), when I tested the same changes in UM N96 AMIP runs (outside of rose-stem) there was in-fact a change in answers for the CoMA9_tb configuration. The difference appears to be insignificant / science-neutral, and is presumably due to a change in compiler optimisation behaviour (the AMIP run uses different compiler optimisation settings to the rose-stem tests). More detail of the tests performed and the results can be seen here:
Testing CoMorph refactoring and bug-fixes

trac.log

Test Suite Results - lfric_apps - comorph_refact2_test/run3

Suite Information

Item Value
Suite Name comorph_refact2_test/run3
Suite User michael.whitall
Workflow Start 2026-09-09T14:30:19
Groups Run developer', 'lfric_atm_nwp_ex1a_extra', 'lfric_atm_nwp_azspice_extra
Dependency Reference Main Like
casim MetOffice/casim@2026.07.1 True
jules MetOffice/jules@2026.07.1 True
lfric_apps MichaelWhitall/lfric_apps@comorph_refact2_test False
lfric_core MetOffice/lfric_core@60fc29a True
moci MetOffice/moci@2026.07.1 True
SimSys_Scripts MetOffice/SimSys_Scripts@77a5166 True
socrates MetOffice/socrates@2026.07.1 True
socrates-spectral MetOffice/socrates-spectral@2026.07.1 True
ukca MetOffice/ukca@9fc2b6d True

Task Information

✅ succeeded tasks - 1309

Security Considerations

  • I have reviewed my changes for potential security issues
  • Sensitive data is properly handled (if applicable) NA
  • Authentication and authorisation are properly implemented (if applicable) NA

Performance Impact

  • Performance of the code has been considered and, if applicable, suitable performance measurements have been conducted

No change in model performance expected.

AI Assistance and Attribution

  • Some of the content of this change has been produced with the assistance of Generative AI tool name (e.g., Met Office Github Copilot Enterprise, Github Copilot Personal, ChatGPT GPT-4, etc) and I have followed the Simulation Systems AI policy (including attribution labels)

A Met Office Github Copilot Enterprise AI agent was used in visual studios code to do the following:

  • Extract the relevant existing changes from the UM comorph_dev branch into a smaller separate UM branch containing only the KGO-preserving refactoring changes (however the agent made many errors so heavy manual intervention has been required after running the agent).
  • Migrate the existing comorph namelist meta-data from the UM into the lfric_apps convection meta-data (but again many manual corrections were required).

Documentation

  • Where appropriate I have updated documentation related to this change and confirmed that it builds correctly

Porting comorph documentation to lfric_apps is still tbd, see #359.

PSyclone Approval

  • If you have edited any PSyclone-related code (e.g. PSyKAl-lite, Kernel interface, optimisation scripts, LFRic data structure code) then please contact the TCD Team NA

Sci/Tech Review

  • I understand this area of code and the changes being added
  • The proposed changes correspond to the pull request description
  • Documentation is sufficient (do documentation papers need updating)
  • Sufficient testing has been completed

(Please alert the code reviewer via a tag when you have approved the SR)

Code Review

  • All dependencies have been resolved
  • Related Issues have been properly linked and addressed
  • CLA compliance has been confirmed
  • Code quality standards have been met
  • Tests are adequate and have passed
  • Documentation is complete and accurate
  • Security considerations have been addressed
  • Performance impact is acceptable

@github-actions github-actions Bot added the cla-modified The CLA has been modified as part of this PR - added by GA label Aug 14, 2026
@MichaelWhitall MichaelWhitall added this to the Autumn 2026 milestone Aug 14, 2026
… the fix which stops the check-cla test from repeatedly spuriously failing, as it annoyingly spams my email every time this happens).
@github-actions github-actions Bot removed the cla-modified The CLA has been modified as part of this PR - added by GA label Aug 14, 2026
@AlisonStirling

Copy link
Copy Markdown

Sci tech review approved

…d parcel radius option, correct sub-grid fraction treatment for graupel in the stand-alone comoprh test.
@github-actions github-actions Bot added the cla-modified The CLA has been modified as part of this PR - added by GA label Aug 26, 2026
…ck. Try use-ing them from comorph_config_mod?
… case where namelist:convection (and hence namelist:comorph too) are not in brackets as well as the existing brackets case (the ngarch rose-stem job doesn't have the brackets so the existing macro breaks it).
… the match as that's not there in source after parsing by self.get_setting_value).
…the various example configuration .nml files (none of these use comorph so the new namelist doesn't need to be added to any of them.
…ackets even if namelist:convection isn't, to indicate that its optional.
… and just specify the whole string to be added (1 preceding space at start, newline at end) instead of matching the previous line.
…or code-style / comment changes, (b) moved autoconversion code from microphysics_2 inside its own subroutine, (c) moved setting of core_ent diagnostic from conv_level_step inside set_ent, (d) added / corrected chk_var calls in comorph_um_namelist_mod...

@DrTVockerodtMO DrTVockerodtMO left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adjoint config changes are fine, thank you!

Comment thread science/physics_schemes/source/convection/comorph/microphysics/autoconversion.F90 Outdated
…ibility of multiple condensate species contributing to the cloud-fractions (needed when switching on the 2nd ice category for use with CASIM). Also deleted stray module use of set_par_cloudfrac from conv_level_step (it isn't used in that routine yet).
@MichaelWhitall

Copy link
Copy Markdown
Contributor Author

Rose-stem test results / trac.log output on the PR summary now updated with a run from the latest branch revision; all tests still pass :)

@AlisonStirling AlisonStirling left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sci/Tech reviewed after additional code added since 21st Aug 2026

…t test for phase_change_solve, and a few more code-style changes to harmonize with the comorph_dev branch.
@github-actions

Copy link
Copy Markdown

⚠️ Hello MichaelWhitall!

Your CLA signature was found on the base branch, but you appear to have modified the CONTRIBUTORS.md file in this PR.

Please do not edit the CONTRIBUTORS.md file. If you have already signed the CLA, revert changes to the file and your signature will be picked up.

@MichaelWhitall

Copy link
Copy Markdown
Contributor Author

Sorry just added a last-minute extra commit:

  • A few more trivial code-style changes.
  • Added an accidentally-missed unit test for the in-plume microphysics implicit phase-change solver (this was added to the comorph_dev branch a while ago where it was used to debug a failure in there once; would like to port this across just in case we need to use it again).

MichaelWhitall added a commit to MichaelWhitall/lfric_apps that referenced this pull request Sep 11, 2026
… double to reproduce KGO in MetOffice#717; now we're breaking the KGOs anyway we may-as-well set this the same as the rest of the comorph namelist for consistency.
MichaelWhitall added a commit to MichaelWhitall/lfric_apps that referenced this pull request Sep 11, 2026
…an the Issue number, for consistency with MetOffice#717 (now that we have the PR number for this one).
@MichaelWhitall MichaelWhitall removed the cla-modified The CLA has been modified as part of this PR - added by GA label Sep 11, 2026

@AlisonStirling AlisonStirling left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sci /tech approved extra submission from Friday 11th September.

@MichaelWhitall

Copy link
Copy Markdown
Contributor Author

James Bruten (@james-bruten-mo), cjohnson-pi, Hacka Fett (@christophermaynard), Steven Sandbach (@ss421), Matt Shin (@matthewrmshin), any chance you can look at this PR Re code-owner approval? For most of you, its just that it modifies an example configuration file you own to keep it consistent with the namelists / meta-data. Among other things, this PR moves some comorph convection scheme settings from the "convection" namelist to a new "comorph" namelist. It turns out none of the example configurations use comorph, so rather than adding the comorph namelist to the .nml files, the correct change was just to delete the existing comorph settings (they would've been trigger-ignored in an equivalent rose-app so technically shouldn't been added to these example config files in the first place).

Note I'd like to get this PR lodged soon if possible as it blocks another PR...

Cheers!
Mike

@MichaelWhitall

Copy link
Copy Markdown
Contributor Author

Ed Hone (@EdHone), this PR is ready for code-review to begin :)

Note this one blocks PR #776 which is also aimed at the next release so it'd be nice to get it lodged relatively soon if that's possible? (still pending some code-owner approvals but I've sent a reminder to the affected code-owners just now...)

Cheers!
Mike

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Macro approved

@ss421

Copy link
Copy Markdown
Contributor

It's because of the namelist updates to applications/jedi_lfric_tests/*/configuration.nml, there's an equivalent set of namelists in metoffice/lfric-jedi and metoffice/jjdocs which need to be kept consistent with these ones.

Does this mean that pretty-much any lfric_apps PR which updates the namelists has to have a linked JEDI PR?? And its just because of the "example configuration" .nml files which annoyingly the upgrade macro doesn't update?

If that's so, surely we ought to urgently either: a) Add some technical wizardry to make the apply_macros.py script apply the macro changes to the "example configuration" files even though they aren't in rose, or b) Migrate all the example configurations to rose so that the upgrade macros update them in the usual way.

Otherwise the overheads from you having to update them in JEDI every time someone adds something to the lfric_apps namelist is going to get silly!

We store namelist configurations in the JEDI testing and these are based on the configurations in LFRic which is ultimately stored in the rose configuration files. Its easier to see what has changed in the namelist files but these should mirror the ones in the rose configuration. I agree that having a mechanism to automatically do that via the rose configs but we are not able to do that just yet so we have this pain for now.

From what I understand in the PR, it seems that while the namelist files have been updated that is related to changes that had not been applied previously and not this PR?

@MichaelWhitall

Copy link
Copy Markdown
Contributor Author

From what I understand in the PR, it seems that while the namelist files have been updated that is related to changes that had not been applied previously and not this PR?

Thanks; the changes to the example configurations in this PR actually just undo the changes from a previous PR which unnecessarily (technically wrongly) added the convection namelist items resdep_precip_ramp and dx_ref to all the example configurations. I now realise that PR shouldn't have added them because they would've been trigger-ignored in those configurations so wouldn't have been added to the namelist file by rose.

I had to remove resdep_precip_ramp and dx_ref from the example configs to get the tests from this PR to pass, because this PR moves those items out of the convection namelist, so the namelist read fails if it finds them in there.

Also, I now realise most of the example config files (and those in JEDI) don't use most of the physics. As long as namelist / meta-data changes only affect things that would be trigger-ignored anyway when not using the relevant physics, then no update to the example configs in JEDI will be needed.

Cheers!
Mike

@ss421 Steven Sandbach (ss421) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Dan has checked and everything works for us :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

macro This PR contains a metadata upgrade macro

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CoMorph further KGO-preserving refactoring

8 participants