Make the IB force-reduction receive buffers device resident

[sbryngelson]

Fixes #1840.

s_communicate_ib_forces hands recv_forces_snap, recv_torques_snap, recv_ids and recv_ft to GPU kernels via copy/copyin, but they are allocated with a plain allocate and so are never present on the device. Under Cray OpenMP offload this aborts at the first time step for any moving immersed boundary run on more than one rank:

ACC: find_in_present_table failed for 'recv_forces_snap(:,:)' from .../m_ibm.fpp:1338
ACC: libcrayacc/acc_runtime.c:703 CRAY_ACC_ERROR - Variable not found in present table
srun: error: frontier10177: tasks 1-3: Exited with exit code 1

The send-side twins (send_ids, send_ft) are already @:ALLOCATEd and pushed with GPU_UPDATE, so this only makes the receive side consistent with them: @:ALLOCATE/@:DEALLOCATE, an explicit GPU_UPDATE(device=...) after each host write, and the corresponding copy/copyin clauses dropped.

Testing. Frontier, --gpu mp (cpe/25.03, rocm/6.3.1), 2D moving flat plate with ib_state_wrt, 4 ranks: aborts at step 1 on master, runs to completion with this patch. An OpenACC build of the same case was unaffected before and after. ./mfc.sh precheck passes.

One file, 8 insertions, 6 deletions.

https://claude.ai/code/session_01HMJ7cycfo7kTFSFq5yhHLG

[Copilot]

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Pull request overview

This PR fixes a GPU offload crash in multi-rank immersed boundary runs by ensuring receive-side force/torque buffers are device-resident and explicitly synchronized for OpenMP offload runtimes (e.g., Cray).

Changes:

• Switch receive buffers to @:ALLOCATE/@:DEALLOCATE so they become device-resident like the send-side twins.
• Add GPU_UPDATE(device=...) after host writes to recv_* receive buffers.
• Remove copy/copyin clauses for receive buffers in GPU parallel loops to avoid present-table failures.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[codecov]

Codecov Report

❌ Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 61.26%. Comparing base (e7139bc) to head (99df96c).
⚠️ Report is 3 commits behind head on master.

Files with missing lines	Patch %	Lines
src/simulation/m_ibm.fpp	0.00%	2 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1843   +/-   ##
=======================================
  Coverage   61.26%   61.26%           
=======================================
  Files          84       84           
  Lines       22330    22329    -1     
  Branches     3265     3265           
=======================================
  Hits        13680    13680           
+ Misses       6207     6206    -1     
  Partials     2443     2443           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[sbryngelson]

Worked through the three Copilot comments.

The two full-array update comments are right and are fixed in 29034b0e0. size(patch_ib) is num_local_ibs_max = 2000, so each GPU_UPDATE pushed about 104 KB (recv_ft at 6x2000x8 B plus recv_ids at 8 KB) no matter how many bodies actually arrived, once per neighbour per direction per step. Both sites now update what was written:

$:GPU_UPDATE(device='[recv_ids(1:recv_count), recv_ft(:,1:recv_count)]')

Safe because the kernel below only reads 1:recv_count; the untouched tail is never referenced.

I did not add the suggested recv_count == 0 guard. The loop is already a no-op at zero, a zero-length section is legal, and the surrounding code does not guard its own slices (GPU_UPDATE(host='[patch_ib(1:num_ibs)]')), so the branch would only be there for a case that cannot misbehave.

The recv_forces_snap / recv_torques_snap comment does not apply. It is conditional on host code reading those arrays after the kernel, and nothing does: they are module-private, appear nowhere outside m_ibm.fpp, and their only accesses are the host zeroing plus device update at 1311-1313 and the read/write inside the kernel at 1345-1348. There is no host read between the kernel and the @:DEALLOCATE. Keeping them device resident is the point - the running snapshot now accumulates across receive iterations instead of round-tripping through the host on every launch. Adding a GPU_UPDATE(host=...) would just reintroduce the transfer the PR removes.

Verified on AFBACA70 (3D -> 2 MPI Ranks -> IBM Sphere) plus CE232828, A2036630 and 16574715, amdflang OpenMP offload on MI210: 4 passed, 0 failed. AFBACA70 is the case that matters - this path only runs when num_procs > 1, so the single-rank IB tests never touch it.

b2ef85321 is unrelated: master's Lint Toolchain has been red since #1855 and this branch has that merge, so the three doc-reference fixes are here to get the lint gate green. Same change as #1861; it will reconcile when that lands.

[github-actions]

Lines of Code

File	Lines	Diff
src/simulation/m_ibm.fpp	1255	+2

Directory	Lines	Diff
simulation	28088	+2
total	46342	+2