Skip to content

Realistic lfric2lfric configs - #660

Open
cjohnson-pi wants to merge 21 commits into
MetOffice:mainfrom
cjohnson-pi:realistic
Open

cjohnson-pi wants to merge 21 commits into
MetOffice:mainfrom
cjohnson-pi:realistic

Conversation

@cjohnson-pi

@cjohnson-pi cjohnson-pi commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

PR Summary

  • C896 to Melbourne
  • C896 to UKV
  • C224 to Melbourne

Also clearing the global mesh (which releases more memory for larger jobs such as these).

Tested by creating a test_branch, merging on the branch from #655, applying upgrade macro, merging on this branch and then running the lfric2lfric_ex1a_weekly group

Sci/Tech Reviewer: Juan Manuel Castillo Sanchez (@ukmo-juan-castillo)
Code Reviewer: mo-lucy-gordon

blocked by #655
split from #530
#440

Code Quality Checklist

  • I have performed a self-review of my own code
  • My code follows the project's style guidelines
  • Comments have been included that aid understanding and enhance the readability of the code
  • My changes generate no new warnings
  • All automated checks in the CI pipeline have completed successfully

Testing

  • I have tested this change locally, using the LFRic Apps rose-stem suite
  • If any tests fail (rose-stem or CI) the reason is understood and acceptable (e.g. kgo changes)
  • I have added tests to cover new functionality as appropriate (e.g. system tests, unit tests, etc.)
  • Any new tests have been assigned an appropriate amount of compute resource and have been allocated to an appropriate testing group (i.e. the developer tests are for jobs which use a small amount of compute resource and complete in a matter of minutes)

trac.log

Test Suite Results - lfric_apps - realistic/run18

Suite Information

Item Value
Suite Name realistic/run18
Suite User christine.johnson
Workflow Start 2026-09-07T13:15:37
Groups Run suite_default
Dependency Reference Main Like
casim MetOffice/casim@2026.07.1 True
jules MetOffice/jules@2026.07.1 True
lfric_apps cjohnson-pi/lfric_apps@realistic False
lfric_core MetOffice/lfric_core@3e965e8 True
moci MetOffice/moci@2026.07.1 True
SimSys_Scripts MetOffice/SimSys_Scripts@77a5166 True
socrates MetOffice/socrates@2026.07.1 True
socrates-spectral MetOffice/socrates-spectral@2026.07.1 True
ukca MetOffice/ukca@9fc2b6d True

Task Information

✅ succeeded tasks - 1216

Security Considerations

  • I have reviewed my changes for potential security issues
  • Sensitive data is properly handled (if applicable)
  • Authentication and authorisation are properly implemented (if applicable)

Performance Impact

  • Performance of the code has been considered and, if applicable, suitable performance measurements have been conducted

AI Assistance and Attribution

  • Some of the content of this change has been produced with the assistance of Generative AI tool name (e.g., Met Office Github Copilot Enterprise, Github Copilot Personal, ChatGPT GPT-4, etc) and I have followed the Simulation Systems AI policy (including attribution labels)

Documentation

  • Where appropriate I have updated documentation related to this change and confirmed that it builds correctly

PSyclone Approval

  • If you have edited any PSyclone-related code (e.g. PSyKAl-lite, Kernel interface, optimisation scripts, LFRic data structure code) then please contact the TCD Team

Sci/Tech Review

  • I understand this area of code and the changes being added
  • The proposed changes correspond to the pull request description
  • Documentation is sufficient (do documentation papers need updating)
  • Sufficient testing has been completed

(Please alert the code reviewer via a tag when you have approved the SR)

Code Review

  • All dependencies have been resolved
  • Related Issues have been properly linked and addressed
  • CLA compliance has been confirmed
  • Code quality standards have been met
  • Tests are adequate and have passed
  • Documentation is complete and accurate
  • Security considerations have been addressed
  • Performance impact is acceptable

@github-actions github-actions Bot added the cla-modified The CLA has been modified as part of this PR - added by GA label Jul 23, 2026
partitioner_dst, &
enforce_constraints = .false. )

call global_mesh_collection%clear()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this line is here because the global mesh is not needed by lfric2lfric. Why is it not the case when the source and destination meshes are read from the same file? Shouldn't be this line outside the if sentence?

A comment in the PR indicating that this change is taking place, apart from the addition of the new tests, would be useful.

I could not find an associated issue where the details of the PR are discussed, although I know this is part of larger work.

@cjohnson-pi cjohnson-pi Jul 27, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue is #440 - now also added above.

@cjohnson-pi cjohnson-pi Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've adjusted the code to also clear the global mesh for the case when the source and destination mesh are from the same file.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I meant is that the clear global mesh could be done outside the
if (mesh_file(dst) == mesh_file(src)) then
condition, but it is also possible to do it like you did.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Juan. Sorry I didn't explain this. Yes I did consider putting the clear_global_mesh outside the if as you had suggested. But the reason I didn't is that when the dst is not the same as src, 2 clear_global_meshes are required - one after each create_local_mesh. So it seemed a better code structure to me to add the clear_global_mesh immediately after the create_local_mesh or create_local_maps. i.e. read global, create local_from_global, clear global. Repeat.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved, although the last comment should be considered.

@github-actions
github-actions Bot requested a review from mo-lucy-gordon July 28, 2026 11:13
@mo-lucy-gordon mo-lucy-gordon added the macro This PR contains a metadata upgrade macro label Aug 25, 2026

@mo-lucy-gordon mo-lucy-gordon left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy with this code but I think another macro upgrade is needed for this PR as well, once 655 has gone on and this can be tested separately.

@github-actions github-actions Bot removed the cla-modified The CLA has been modified as part of this PR - added by GA label Sep 7, 2026
@cjohnson-pi

Copy link
Copy Markdown
Contributor Author

main has been merged (including changes from #655). trac.log has been regenerated.

There is data to be copied to BIG_DATA_DIR. We need to agree on how to deal with the MetOffice/simulation-systems#660. The choices are:

  1. Copy the files as is and add a file detailing where the ancil has come from (I have asked around and there doesn't seem to be an agreement on what this should look like yet).
  2. Copy the files and add a file stipulating that they are only for internal use (as they are only for the weekly tests which are not run by partners).
  3. Reduce the set of data contained in the files so that there are only the fields required (and hence removing the ancil data that requires licensing).

@cjohnson-pi

cjohnson-pi commented Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

I suggest that extracting the essential data is the best option at present. Therefore the files to be copied are (on exa and exc):

/data/users/christine.johnson/lfric_data/lfric2lfric/rho_theta_C896_t1.nc to /data/users/lfricadmin/data/lfric2lfric/C896/PR660/rho_theta_C896_t1.nc

/data/users/christine.johnson/lfric_data/lfric2lfric/rho_theta_zh_C224.nc to /data/users/lfricadmin/data/lfric2lfric/C224/PR660/rho_theta_zh_C224.nc

This has been tested at exa:/home/users/christine.johnson/cylc-run/realistic/run21/share/output/lfric2lfric

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

macro This PR contains a metadata upgrade macro

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants