Skip to content

shorten CI#1937

Merged
akshaysridhar merged 1 commit into
mainfrom
js/ci-shorter
May 20, 2026
Merged

shorten CI#1937
akshaysridhar merged 1 commit into
mainfrom
js/ci-shorter

Conversation

@juliasloan25
Copy link
Copy Markdown
Member

@juliasloan25 juliasloan25 commented May 19, 2026

Purpose

CI has gotten long again. This PR shortens the runtime of three of the longest runs, which currently take:

  1. AMIP - Float64 + hourly checkpoint: ~2 hours 5 min
  2. AMIP - Component dts test: ~1 hour 15 min
  3. Unthreaded AMIP FINE: 43 min
    (taken from this build on main)

After this PR, the times are:

  1. AMIP - Float64 + hourly checkpoint: N/A
  2. AMIP - Component dts test: ~30 mins
  3. Unthreaded AMIP FINE: N/A
    (taken from this build)

The changes to each config are the following:

  1. Shorten simulation length from 1days to 3hours, which is enough to test the hourly checkpointing Removed this run. It wasn't even saving checkpoints, and checkpointing is tested more thoroughly by the restarts test.
  2. Shorten simulation length from 600secs to 300secs; increase timesteps of some components to reduce computation (if this isn't enough, we can shorten it to 150secs). Note that even though we'll realistically want to run sims with the atmosphere having the smallest timestep, dt_atmos is the largest here because it is the most computationally demanding.
  3. Removed this run. It was originally added for performance monitoring, but we have since removed the flamegraphs it used to generate. The profiling option of the benchmarks pipeline can now be used for more informative GPU profiling.

To-do

Content


  • I have read and checked the items on the review checklist.

Copy link
Copy Markdown
Member

@akshaysridhar akshaysridhar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI time following this PR is <1hr. Thanks @juliasloan25.

@akshaysridhar akshaysridhar merged commit 1f60191 into main May 20, 2026
14 checks passed
@akshaysridhar akshaysridhar deleted the js/ci-shorter branch May 20, 2026 16:27
@hgpeterson
Copy link
Copy Markdown
Member

Any thoughts on shortening the github actions CI now that that seems to be the bottleneck?

@juliasloan25
Copy link
Copy Markdown
Member Author

juliasloan25 commented May 21, 2026

Any thoughts on shortening the github actions CI now that that seems to be the bottleneck?

Good point. Right now the GHA run test/runtests.jl (10 mins) and also experiments/test/runtests.jl (50 mins). We could split these into two different workflows so they can run in parallel. The only thing to be mindful of there is that there is a quota for how many github runners we can use at once across the entire CliMA org.

Actually, we run experiments/test/runtests.jl in buildkite already (here), so we don't really need to also run it in GHA. I'll open a PR to remove it from GHA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants