Fix/gpu elecstate pw op spin stride - #7776
Open
chengleizheng wants to merge 4 commits into
Open
Conversation
The GPU kernel indexed the spin components of rho with spin*nrxx, where nrxx is the wavefunction real-space grid size, while rho is allocated with charge->nrxx (density grid) as the per-spin stride. For USPP tests using the double grid the two grids differ, so the spin-down density was written to the wrong offset and effectively lost. Pass an explicit rho_stride parameter instead. Fixes 007_PW_UPF201_USPP_Fe GPU SCF etot being off by 6.07 eV.
The elecstate_pw_op operator signature gained a rho_stride parameter in bc21c53, but the unit test still called it with the old signatures, breaking the test build. Pass this->nrxx, which matches the stride of the rho layout used in the test. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reminder
AGENTS.mdanddocs/developers_guide/agent_governance.md.source/changes.Linked Issue
Fix #7773
Case Tests for my changes
tests/01_PW/007_PW_UPF201_USPP_Fe(PW SCF, USPP,nspin=2,ecutwfc=8,ecutrho=60— the dense FFT grid is 4× larger per dimension than the smooth grid, which is exactly the configuration that exposed the bug).abacuson 007 →FINAL_ETOT_IS = -673.8349347015964668 eV.FINAL_ETOT_IS = -673.8349347005279242 eV(GPU kernels active, e.g.PW_Basis_K recip_to_real gpuin timing).What's changed?
elecstate_pwinsource/source_estate/kernels/cuda/elecstate_op.cuwrote charge density withrho[spin * basis->nrxx + idx], using the wavefunction grid stride (basis->nrxx, the smooth grid). But therhoarray is laid out per spin on the density grid (charge->nrxx, the dense grid). For USPP (double grid) the two differ (e.g. 9³ vs 12³ on 007), so the spin-down component was written at a wrong offset and effectively lost → spin-down charge density missing → etot off by 6.07 eV on GPU.rho_strideparameter, and every caller passescharge->nrxx:elecstate_op.cu/elecstate_op.h/elecstate_op.cpp: kernel andoperator()signatures extended withconst int& rho_stride(CPU ignores it; its per-spin pointer indexing was already correct).elecstate_pw.cpp: rho accumulation site (rhoBandK) andkin_rsite now passcharge->nrxx.elecstate_pw_cal_tau.cpp: passescharge->nrxxfor the kinetic energy density.read_wf2rho_pw.cpp(2 sites) andsto_iter.cpp: passcharge->nrxx.Governance Notes
source_estate/kernelsis extended with one parameter; all GPU call sites within this repo are updated in this PR. CPU path behavior is unchanged.