Skip to content

[ROCCLR] Mark VirtualMapCommand event complete after synchronous submit #6594

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
BertanDogancay wants to merge 1 commit into
base: develop
Choose a base branch
from
+12 −0
Open

[ROCCLR] Mark VirtualMapCommand event complete after synchronous submit #6594

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions projects/clr/rocclr/device/rocm/rocvirtual.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3653,8 +3653,18 @@ void VirtualGPU::submitVirtualMap(amd::VirtualMapCommand& vcmd) {
if (phys_mem_obj->getMemFlags() & ROCCLR_MEM_INTERPROCESS) {
vaddr_sub_obj->setVmmImported(true);
}
// hsa_amd_vmem_map is synchronous on the calling host thread; once it
// returns HSA_STATUS_SUCCESS the GPU page tables are already updated
// and the command has no asynchronous GPU completion to wait on. Mark
// the event CL_COMPLETE here so that a subsequent awaitCompletion()
// short-circuits and never enqueues a Marker on the device's NullStream.
// Without this, the Marker's releaseGpuMemoryFence() / HwQueueTracker::WaitCurrent()
// can transitively block on an unrelated in-flight collective kernel,
// producing a distributed deadlock under multi-rank IPC registration.
vcmd.setStatus(CL_COMPLETE);
} else {
LogError("HSA Command: hsa_amd_vmem_map failed!");
vcmd.setStatus(CL_INVALID_OPERATION);
}
} else {
dispatchBarrierPacket(kBarrierPacketHeader, false);
Expand All @@ -3676,8 +3686,10 @@ void VirtualGPU::submitVirtualMap(amd::VirtualMapCommand& vcmd) {
// Release sub_obj now that HW unmap is complete.
// ~Memory releases parent va_ via parent_->release().
vaddr_sub_obj->release();
vcmd.setStatus(CL_COMPLETE);
} else {
LogError("HSA Command: hsa_amd_vmem_unmap failed");
vcmd.setStatus(CL_INVALID_OPERATION);
}
}

Expand Down
Loading