Update EKAT kokkos for Aurora

[amametjanov]

Add PALS_LOCAL_RANKID to the list of recognized env-vars
in Kokkos::Impl::mpi_local_rank_on_node() at core/src/impl/Kokkos_CPUDiscovery.cpp#L35 .

PALS_LOCAL_RANKID on Aurora has the expected local on-node MPI task id.

None of the other env-vars in the Kokkos function are defined on Aurora.

So, E3SM has been using a local shell script gpu_tile_compact.sh in the mpiexec launch: e.g.

      mpiexec -np 48 --label -ppn 12 --cpu-bind list:1-8:9-16:17-24:25-32:33-40:41-48:53-60:61-68:69-76:77-84:85-92:93-100
--gpu-bind list:0.0:0.1:1.0:1.1:2.0:2.1:3.0:3.1:4.0:4.1:5.0:5.1
--mem-bind list:0:0:0:0:0:0:1:1:1:1:1:1 -d 1   
/lus/flare/projects/E3SMinput/tools/mpi_wrapper_utils/gpu_tile_compact.sh
/lus/flare/projects/E3SM_Dec/azamat/scratch/SMS.ne30pg2_EC30to60E2r2.WCYCLXX2010.aurora_oneapi-ifxgpu.JNextEamxx_v1_release20260211_082112/bld/e3sm.exe   >> e3sm.log.$LID 2>&1

which copies value of $PALS_LOCAL_RANKID into MPI_LOCALRANKID (which is recognized) as a work-around.
Without this work-around, e3sm.log has numerous per-mpi-process warnings during kokkos-init like

Warning: unable to detect local MPI rank. Falling back to the first GPU available for execution. Raised by Kokkos::initialize()

raised from here .

By adding this branch on top of e3sm's kokkos, we can remove gpu_tile_compact.sh and clean up the mpiexec launch.

[BFB]

[welcome]

Thanks for opening this pull request! Please check out our contributing guidelines.

[amametjanov]

Conrad (@tcclevenger), please take a look at this PR. I added more details in the description about why this is still needed. If it's easier, this can be rolled onto kokkos-5 PR: adds 1-line.

[Copilot]

Pull request overview

Updates the EKAT Kokkos submodule to a newer upstream commit that includes Aurora support (recognizing PALS_LOCAL_RANKID for local MPI rank detection).

Changes:

• Bumps extern/kokkos submodule pointer to a newer commit.

Comments suppressed due to low confidence (1)

extern/kokkos:1

• Since this PR is a submodule SHA bump only, it’s hard to review/trace the exact Kokkos change being pulled in. Consider adding to the PR description (or a short note in the commit message) a direct link to the upstream Kokkos commit/PR that introduces PALS_LOCAL_RANKID, and briefly summarize any other notable upstream changes included between the old and new SHA (to help future debugging/regression triage).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[rljacob]

@amametjanov don't ask for copilot review for a PR thats just a submodule update. It can't really review anything.

[amametjanov]

yes, a better review is at E3SM-Project/kokkos#2

[tcclevenger]

Apologies, Az, this completely fell off my radar. I see (and approved) your Kokkos PR, but we can merge this as well if you want to have the change in immediately

[amametjanov]

Thanks for reviewing this.
The change (adding that PALS env-var) has now been made in upstream Kokkos develop repo: PR 8895.

I'd rather have this change in sooner than later (just to avoid forgetting about it again). I'm not sure how soon the kokkos-5 update will make it onto E3SM-master (if 8895 is also added to the update). Pushing this forward along with -restrict cxx-flag update in EKAT would help me make more progress on Aurora.

[tcclevenger]

Pushing this forward along with -restrict cxx-flag update in EKAT would help me make more progress on Aurora.

Are you referring to this commit? Go ahead and add that change in this PR and I'll merge and then open an EKAT update PR in E3SM repo.

Edit: Never mind, I see #413 which merged. I'll merge this PR and then update EKAT in E3SM

[tcclevenger]

EKAT update PR E3SM-Project/E3SM#8088