Skip to content

Support transfers for layouts with multiple id-indexed fields - #284

Merged
apryakhin merged 17 commits into
mainfrom
mdery/apriakhin-uniform-fields
Apr 7, 2026
Merged

Support transfers for layouts with multiple id-indexed fields#284
apryakhin merged 17 commits into
mainfrom
mdery/apriakhin-uniform-fields

Conversation

@2dm

@2dm 2dm commented Sep 3, 2025

Copy link
Copy Markdown
Collaborator

This patch adds a fast‐path for multi‐field copies when source and destination instances store fields consecutively in memory with strictly increasing IDs. Instead of the generic per‐field iterator, Realm switches to an IDIndexedIterator and issues the entire copy as a single batched DMA operation.

Fast‐Path Mechanics

  • Layout Detection: During instance construction, Realm identifies when field IDs form a consecutive sequence and marks the instances for the optimized path.
  • IDIndexedIterator: Attaches a single FieldBlock (allocated from the replicated heap) to the address list, skipping per‐field offset lookups.
  • Affine Collapse: Treats the multi‐field operation as one affine rectangle with N attached fields, collapsing what would be N separate copies into a single batched command.
  • Flow‐Control Splitting: The DMA engine automatically splits the batch according to internal flow‐control limits and IB fragment sizes (for cross‐node transfers).

Testing

  • Integration Test: Verifies end-to-end multi-field copy behavior on real instances.
  • Unit Tests: Covers iterator construction, address-list setup, FieldBlock allocation, and DMA splitting logic.
src_inst{ 0: [..field_size..], 1: [..field_size..], 2: [..field_size..], N-1: [..field_size..] }
dst_inst{ 0: [..field_size..], 1: [..field_size..], 2: [..field_size..], N-1: [..field_size..] }
index_space.copy(src_fids[32, 16, 612, 0..], dst_fids[17, 3, 999, 2..])

Authored by @apryakhin

@2dm
2dm requested a review from apryakhin September 3, 2025 00:19
@github-actions github-actions Bot added the chore label Sep 3, 2025
@codecov

codecov Bot commented Sep 3, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 39.81191% with 192 lines in your changes missing coverage. Please review.
✅ Project coverage is 29.20%. Comparing base (f337379) to head (30251c5).
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/realm/transfer/transfer.cc 23.87% 160 Missing and 9 partials ⚠️
src/realm/inst_layout.inl 65.11% 15 Missing ⚠️
src/realm/runtime_impl.cc 0.00% 3 Missing ⚠️
src/realm/transfer/channel.h 0.00% 2 Missing ⚠️
tests/unit_tests/idindexed_fields_iterator_test.cc 95.83% 2 Missing ⚠️
src/realm/transfer/channel.cc 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #284      +/-   ##
==========================================
+ Coverage   29.07%   29.20%   +0.12%     
==========================================
  Files         194      195       +1     
  Lines       40229    40493     +264     
  Branches    14458    14693     +235     
==========================================
+ Hits        11697    11826     +129     
- Misses      27598    28242     +644     
+ Partials      934      425     -509     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@2dm
2dm force-pushed the mdery/apriakhin-uniform-fields branch 2 times, most recently from a1c10a5 to ed1820b Compare September 3, 2025 02:11
@apryakhin

Copy link
Copy Markdown
Contributor

@2dm Thanks for adding this PR to the github. What's our strategy here? Are you proposing to carry it through the review process or do we plan to split it up?

Comment thread src/realm/transfer/address_list.h
Comment thread src/realm/transfer/transfer.cc Outdated
Comment thread src/realm/transfer/transfer_utils.h
Comment thread src/realm/inst_layout.h
Comment thread src/realm/cuda/cuda_internal.cc
Comment thread src/realm/transfer/transfer.cc Outdated
@apryakhin

Copy link
Copy Markdown
Contributor

@2dm The OG gitlab CI was passing for all tests (except clang-format):

It's either some regression with the additional changes we make to reduce copy launch overhead or there are different settings/tests in github CI compared to gitlab.

Comment thread src/realm/cuda/cuda_internal.cc Outdated
Comment thread src/realm/cuda/cuda_internal.h
Comment thread src/realm/inst_layout.inl
@2dm
2dm force-pushed the mdery/apriakhin-uniform-fields branch 5 times, most recently from 917b308 to 3bca93c Compare September 12, 2025 00:44
@2dm
2dm changed the base branch from main to mdery/addr_list_refactor September 21, 2025 20:01
@2dm
2dm force-pushed the mdery/apriakhin-uniform-fields branch from 3bca93c to 9e84214 Compare September 21, 2025 20:01
@2dm
2dm force-pushed the mdery/addr_list_refactor branch from c032b92 to 06e5ba4 Compare September 21, 2025 20:05
@2dm
2dm force-pushed the mdery/apriakhin-uniform-fields branch from 9e84214 to 9a0df71 Compare September 21, 2025 20:12
@2dm
2dm force-pushed the mdery/addr_list_refactor branch from 06e5ba4 to 792909a Compare November 5, 2025 22:36
@2dm
2dm force-pushed the mdery/apriakhin-uniform-fields branch from 9a0df71 to 6ab1e8a Compare November 5, 2025 22:39
@apryakhin
apryakhin marked this pull request as ready for review November 6, 2025 14:36
@2dm
2dm force-pushed the mdery/apriakhin-uniform-fields branch from 6cc0068 to aff2822 Compare November 6, 2025 16:33
@2dm
2dm force-pushed the mdery/apriakhin-uniform-fields branch from d71688e to 47d7e74 Compare December 2, 2025 21:57
Comment thread src/realm/transfer/transfer.cc Outdated
@apryakhin

Copy link
Copy Markdown
Contributor

overall LGTM, left just one comment

Comment thread tests/multifield_transfer.cc
Comment thread src/realm/transfer/transfer.h Outdated
@apryakhin
apryakhin requested a review from eddy16112 April 7, 2026 16:10
@apryakhin apryakhin assigned apryakhin and unassigned 2dm Apr 7, 2026
Comment thread src/realm/transfer/channel.cc Outdated
Comment thread src/realm/transfer/transfer.h Outdated
Comment thread src/realm/transfer/transfer.cc Outdated
Comment thread src/realm/runtime_impl.h
@apryakhin
apryakhin force-pushed the mdery/apriakhin-uniform-fields branch from 15359ca to 722811f Compare April 7, 2026 17:41
@apryakhin
apryakhin merged commit 442cab5 into main Apr 7, 2026
41 checks passed
@apryakhin
apryakhin deleted the mdery/apriakhin-uniform-fields branch April 7, 2026 18:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

chore enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants