Skip to content

Add postprocessing for 4D variables (lat, lon, time, pressure level)#1094

Merged
ph-kev merged 3 commits into
mainfrom
kp/pressure-postprocessing
Dec 6, 2024
Merged

Add postprocessing for 4D variables (lat, lon, time, pressure level)#1094
ph-kev merged 3 commits into
mainfrom
kp/pressure-postprocessing

Conversation

@ph-kev
Copy link
Copy Markdown
Member

@ph-kev ph-kev commented Nov 21, 2024

Purpose

closes #1093 - This PR adds postprocessing for 4D variables (lat, lon, time, pressure level). See plots below.

To-do

  • Make new release of ClimaAnalysis and update here in ClimaCoupler

Content

  • Add bias plots after taking time average and slicing at specific pressure levels (850, 500, 250 hPA)

  • Add bias plots over latitude and pressure level after taking average over time and longitude

  • I have read and checked the items on the review checklist.
    bias_vars_in_pfull
    lat_pfull_heatmaps

@ph-kev ph-kev force-pushed the kp/pressure-postprocessing branch from 4461dfe to 97ff662 Compare November 21, 2024 19:40
Comment thread experiments/ClimaEarth/leaderboard/data_sources.jl Outdated
Comment thread experiments/ClimaEarth/leaderboard/leaderboard.jl Outdated
Comment thread experiments/ClimaEarth/leaderboard/leaderboard.jl Outdated
Copy link
Copy Markdown
Member

@Sbozzolo Sbozzolo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thank you!

Copy link
Copy Markdown
Member

@juliasloan25 juliasloan25 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! Thank you Kevin

Comment thread docs/src/leaderboard.md
"MAM", "JJA", "SON", and "ANN" in that order. The file `leaderboard.jl` will load the
appropriate data into the `RMSEVariable`.

### Add a new variable to compare against observations in pressure coordinates
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to also add a description of the new feature in NEWS.md

Comment thread experiments/ClimaEarth/leaderboard/leaderboard.jl
function that returns a `OutputVar`. The function must takes in a start date
which is used to align the times in the observational data to match the
simulation data. The short name must be the same as in `sim_var_dict` in the
function `sim_var_dict`. Any preprocessing is done in the function which
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
function `sim_var_dict`. Any preprocessing is done in the function which
function `get_sim_var_dict`. Any preprocessing is done in the function which

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which function does the preprocessing + unit conversion + date shifting? It's not totally clear to me here

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am going to rewrite this, but the function that does the post processing is the anonymous function defined in the dictionaries. For example, get_sim_var_dict returns a dictionary that map short names to functions and those functions are the one that do the preprocessing (unit conversion and data shifting).

Comment thread experiments/ClimaEarth/leaderboard/leaderboard.jl Outdated
@ph-kev ph-kev force-pushed the kp/pressure-postprocessing branch 3 times, most recently from 4f11200 to fdc3a5d Compare November 26, 2024 00:31
@ph-kev ph-kev force-pushed the kp/pressure-postprocessing branch from fdc3a5d to 1cd54c5 Compare December 5, 2024 01:40
@ph-kev
Copy link
Copy Markdown
Member Author

ph-kev commented Dec 5, 2024

The plots now look like this
bias_lat_pfull_heatmaps
bias_vars_in_pfull

@ph-kev ph-kev force-pushed the kp/pressure-postprocessing branch from 1cd54c5 to a69d6ef Compare December 5, 2024 01:44
Copy link
Copy Markdown
Member

@juliasloan25 juliasloan25 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@ph-kev
Copy link
Copy Markdown
Member Author

ph-kev commented Dec 6, 2024

Going to merge this in, but it will be slow. This part of the postprocessing pipeline will be faster once this PR in ClimaAnalysis is merged in.

The plots added are bias plots at 850 hPa, 500 hPa, and 250 hPa and
lat - pressure plots.
@ph-kev ph-kev force-pushed the kp/pressure-postprocessing branch from a69d6ef to 6bbd53a Compare December 6, 2024 00:16
@ph-kev ph-kev merged commit f66d5df into main Dec 6, 2024
@ph-kev ph-kev deleted the kp/pressure-postprocessing branch December 6, 2024 04:35
Sbozzolo added a commit that referenced this pull request Dec 9, 2024
PR #1103 and #1094 are inconsistent. This uses OrderedDict directly from
ClimaAnalysis
@Sbozzolo Sbozzolo mentioned this pull request Dec 9, 2024
Sbozzolo added a commit that referenced this pull request Dec 11, 2024
PR #1103 and #1094 are inconsistent. This uses OrderedDict directly from
ClimaAnalysis
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add postprocessing for variables in pressure coordinates

3 participants