Implement P2300 bulk adapter for HPX executors by shivansh023023 · Pull Request #7240 · TheHPXProject/hpx

shivansh023023 · 2026-05-02T17:39:21Z

P2300 `bulk` Integration for HPX Executors

Description

This PR implements the P2300 bulk sender adapter specifically for HPX's legacy executors, including the parallel_executor and sequenced_executor. By bridging the hpx::execution::experimental::bulk algorithm with HPX's internal bulk_sync_execute mechanism, this change ensures that data-parallel workloads are properly load-balanced and optimized using HPX's high-performance partitioners.

Key Technical Improvements:

Algorithmic Bridging: Successfully mapped the C++23 bulk sender to the underlying HPX execution engine for native parallel performance.
Robust tag_invoke Overloads: Implemented a full suite of connect_t overloads (supporting &&, &, and const&) to ensure compatibility with consumer algorithms like sync_wait.
Reference Stability: Utilized std::decay_t within the bulk_receiver to prevent reference collapsing and ensure the safety of deferred functional object execution.
Architectural Integrity: Resolved a potential circular dependency by situating the implementation within the executors module rather than the core execution module.

Proposed Changes

[New] libs/core/executors/include/hpx/executors/executor_scheduler_bulk.hpp: Implements executor_bulk_sender and executor_bulk_receiver.
[Modified] libs/core/executors/include/hpx/executors/executor_scheduler.hpp: Exposes get_completion_scheduler_t for ADL discovery.
[Modified] libs/core/executors/include/hpx/executors/parallel_executor.hpp & sequenced_executor.hpp: Integrated the new bulk headers.
[New Test] libs/core/executors/tests/unit/executor_algorithm_bulk.cpp: Comprehensive validation for sequential and parallel policies.

Background context

This work is part of the ongoing effort to modernize HPX's execution model to align with the C++23 Sender/Receiver (P2300) standard. Implementing bulk is a "Big Impact" milestone because it allows modern asynchronous pipelines to tap into the mature, multi-threaded performance of the HPX runtime.

Checklist

I have added a new feature and have added tests to go along with it.
I have verified the implementation with a local build of the executor_algorithm_bulk_test target.
I have ensured all includes are strictly alphabetically sorted to comply with the HPX inspect tool.

StellarBot · 2026-05-02T17:40:04Z

Can one of the admins verify this patch?

codacy-production · 2026-05-02T17:42:17Z

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

_{NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer}
_{TIP This summary will be updated as you push new changes.}

hkaiser · 2026-05-03T15:11:47Z

@shivansh023023 Just FYI, before you invest more time into our old sender/receiver implementation: #7123 will soon remove almost all of our code related to this. Please focus your efforts on whatever stays after this has been merged.

shivansh023023 · 2026-05-05T19:30:04Z

@hkaiser

Thank you for the heads-up regarding #7123. I appreciate the guidance to avoid spending effort on code that will soon be deprecated.

I will pause my work on the current P2300 bulk adapter and follow the progress of #7123 closely. Once that is merged, I’ll re-evaluate how to implement the bulk functionality within the new framework.

In the meantime, I’ll focus on finishing the review items for PR #7070 (local convenience header) and cleaning up any remaining issues on my other active PRs that are unaffected by the sender/receiver removal.

shivansh023023 · 2026-05-06T21:15:13Z

Hi @hkaiser

Thank you for the feedback. I have refactored the PR to align with the latest stdexec changes:

Purged Legacy Code: Removed all !defined(HPX_HAVE_STDEXEC) blocks from executor_scheduler_bulk.hpp.

Header Optimization: Removed the inclusions of executor_scheduler.hpp and executor_scheduler_bulk.hpp from the parallel_executor and sequenced_executor headers.

Forward Declarations: Implemented forward declarations for the returned scheduler types in the executor headers to prevent bloat.

Fixed Includes: Removed the reference to the deprecated get_scheduler.hpp header.

The headers are now much cleaner. Let me know if there are any other areas you'd like me to slim down!

charan-003

nice work!!!

charan-003

nice work!!!

…nder/receiver concepts, NVCC guard, dedup get_scheduler, fix post() UB, use get_scheduler in tests

shivansh023023 · 2026-05-07T07:00:03Z

Hi @hkaiser

Thank you for the architectural guidance! I have implemented your suggestions to clean up the header dependencies and tag_invoke logic:

Forwarding Header: I created executor_scheduler_fwd.hpp and moved the declarations there, which allowed me to strip the redundant forward declarations from the individual executor headers.

Tag-Invoke Overloads: I removed the default template arguments and duplicate overloads in parallel_executor and sequenced_executor, replacing them with the concrete friend functions you suggested.

NVCC Fix: I ensured the NVCC/CUDACC guards for the destructors were correctly restored.

Standard Test Patterns: I updated the unit tests to use the standard ex::get_scheduler(exec) call instead of manual constructor calls.

The structure feels much more consistent with the rest of the HPX core modules now. Thanks!

shivansh023023 · 2026-05-07T07:00:30Z

Hi @charan-003

Thanks for the catch on the stdexec compliance and the safety issues! I’ve updated the implementation as follows:

Sender/Receiver Concepts: I added sender_concept to executor_sender and receiver_concept to executor_bulk_receiver. These are now correctly picked up by the stdexec concept checks.

UB Fix in post: I refactored the lambda capture in executor_operation_state::start(). It now captures the state by reference and only moves the receiver inside the task body. This ensures that if hpx::parallel::execution::post throws, the receiver is still valid for the set_error call in the catch block.

I really appreciate the eye for detail on that potential move-before-post UB,definitely a safer implementation now!

shivansh023023 · 2026-05-08T12:51:01Z

@hkaiser
I have just pushed the latest updates for PR #7240, and I’m happy to report that all CI checks are passing! I’ve implemented your latest suggestions to fully modernize the implementation:

C++20 Concepts: I’ve replaced the legacy std::enable_if_t in the executor_scheduler constructor with a clean C++20 requires clause and included the header.

Purged Redundant Guards: Removed the #if defined(HPX_HAVE_STDEXEC) guards in executor_scheduler_bulk.hpp, as we now rely strictly on the modern Sender/Receiver path.

Header Optimization: I have completely removed the implementation headers (executor_scheduler.hpp and executor_scheduler_bulk.hpp) from parallel_executor.hpp and sequenced_executor.hpp. These files now rely solely on the lightweight executor_scheduler_fwd.hpp header, significantly reducing the include bloat for the core executors.

The code is now much cleaner and follows the forwarding pattern we discussed. It should be ready for a final review!

hkaiser · 2026-05-08T13:52:43Z

@shivansh023023 Thanks for the updates. I'd like to wait for #7257 to be finalized before merging anything else related to s&r. Please be patient as this may take a moment.

shivansh023023 · 2026-05-08T13:55:48Z

@shivansh023023 Thanks for the updates. I'd like to wait for #7257 to be finalized before merging anything else related to s&r. Please be patient as this may take a moment.

Sure sir

shivansh023023 · 2026-05-16T08:51:36Z

Hey @hkaiser , all the CI checks are passing and I've implemented all your feedback
from the last review. Can you review it again and tell me if it needs any improvements or can it be merged

hkaiser · 2026-05-16T15:10:23Z

Hey @hkaiser , all the CI checks are passing and I've implemented all your feedback from the last review. Can you review it again and tell me if it needs any improvements or can it be merged

Could you please look into the conflicts? Those will have to be resolved before moving on.

@charan-003 Could you please have a look here to see how that interplays with #6655?

charan-003 · 2026-05-16T21:45:18Z

@charan-003 Could you please have a look here to see how that interplays with #6655?

executor_scheduler adapter for legacy executors with bulk via bulk_sync_execute. My #6655 adds the P2079R7 parallel_scheduler with policy-aware bulk in thread_pool_scheduler_bulk. Different scheduler types, no architectural conflict.

This PR helps older HPX executors (parallel_executor, sequenced_executor) use P2300 bulk in sender pipelines. My #6655 covers the new scheduler side. Same user goal (bulk support), different backend paths by scheduler

I just need to update my PR so it follows the same style(P2300)

codacy-production · 2026-05-17T09:53:53Z

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

_{NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer}
_{TIP This summary will be updated as you push new changes.}

…nder/receiver concepts, NVCC guard, dedup get_scheduler, fix post() UB, use get_scheduler in tests Signed-off-by: Shivansh <singhshivansh023@gmail.com>

- Integrated hpx::execution::experimental::bulk with internal bulk_sync_execute - Implemented robust tag_invoke(connect_t) with full const-correctness - Fixed receiver reference collapsing using std::decay_t - Added executor_algorithm_bulk unit tests for parallel and sequenced execution Signed-off-by: Shivansh <singhshivansh023@gmail.com>

…nder/receiver concepts, NVCC guard, dedup get_scheduler, fix post() UB, use get_scheduler in tests Signed-off-by: Shivansh <singhshivansh023@gmail.com>

Signed-off-by: Shivansh <singhshivansh023@gmail.com>

…rm_completion_signatures Signed-off-by: Shivansh <singhshivansh023@gmail.com>

…cutor includes Signed-off-by: Shivansh <singhshivansh023@gmail.com>

The requires clause in both the sequential and parallel tag_fallback_invoke overloads of hpx::unique_copy_t incorrectly constrained the predicate against the output iterator's value type: is_invocable_v<Pred, iter_value_t<InIter>, iter_value_t<OutIter>> // wrong is_invocable_v<Pred, iter_value_t<FwdIter1>, iter_value_t<FwdIter2>> // wrong unique_copy compares consecutive elements of the *input* range only ([alg.unique]). The output iterator is write-only; its value type is irrelevant to the predicate. The correct form is: is_invocable_v<Pred, iter_value_t<InIter>, iter_value_t<InIter>> This is already the form used in hpx::ranges::unique_copy (via is_indirect_callable_v with projected<Proj, InIter> for both arguments), making this fix also restore consistency between the two API surfaces. Add test_unique_copy_constraint<IteratorTag>() to unique_copy_tests.hpp covering all execution policies (seq, par, par_unseq, seq(task), par(task)) with a strongly-typed binary predicate, plus seven edge cases: empty range, single element, all-identical, no-duplicates, and two-element identical/distinct variants. All verified against std::unique_copy. Signed-off-by: Aneek22112007 <das.aneek007@gmail.com> Signed-off-by: Shivansh <singhshivansh023@gmail.com>

The existing godbolt-minimal preset (added in TheHPXProject#7063) is missing several flags required for a correct Compiler Explorer integration: 1. Add HPX_WITH_DISTRIBUTED_RUNTIME=OFF: HPX_WITH_NETWORKING=OFF alone does not disable the distributed runtime -- AGAS, actions, and the parcelset infrastructure are still compiled, wasting ~30% build time and producing libraries that CE will never use. 2. Add HPX_WITH_CXX_STANDARD=20: pins the C++ standard explicitly so builds are deterministic across GCC/Clang versions. Uses HPX's own cache variable (not CMAKE_CXX_STANDARD, which HPX rejects unless HPX_USE_CMAKE_CXX_STANDARD is also set). 3. Add HPX_WITH_TOOLS=OFF: CE only needs libraries and headers; building HPX tools adds unnecessary build time. 4. Improve description: document the produced static libraries (libhpx_wrap.a, libhpx_init.a, libhpx.a, libhpx_core.a) and the required -Wl,-wrap=main linker flag for non-CMake consumers. 5. Reorder cache variables to group related flags logically: standard -> linking -> runtime -> dependencies -> build targets. Signed-off-by: Shivansh <singhshivansh023@gmail.com>

Signed-off-by: Shivansh <singhshivansh023@gmail.com>

hkaiser · 2026-05-17T16:36:14Z

These changes are unrelated to this PR. Please remove.

hkaiser · 2026-05-17T16:36:26Z

Same here, these changes are unrelated.

hkaiser · 2026-05-17T16:36:44Z

Unrelated changes, please remove.

hkaiser · 2026-05-17T16:37:15Z

These changes are unrelated as well.

…tions Signed-off-by: Shivansh Singh <singhshivansh023@gmail.com>

Signed-off-by: Shivansh Singh <singhshivansh023@gmail.com>

hkaiser

This PR still has a couple of unrelated changes, please remove those. Also, I'd like for this to be compatible with #6655, let's wait for that to be merged to revisit things here.

… modifiers Signed-off-by: Shivansh Singh <singhshivansh023@gmail.com>

shivansh023023 · 2026-05-18T18:46:50Z

This PR still has a couple of unrelated changes, please remove those. Also, I'd like for this to be compatible with #6655, let's wait for that to be merged to revisit things here.

Ok sir , I will fix my other PR's till then

shivansh023023 requested a review from hkaiser as a code owner May 2, 2026 17:39

shivansh023023 force-pushed the feat/p2300-bulk-integration branch from 5cdcfbd to 7b3fc5f Compare May 2, 2026 17:50

hkaiser added category: senders/receivers Implementations of the p0443r14 / p2300 + p1897 proposals type: enhancement type: compatibility issue labels May 3, 2026

hkaiser reviewed May 5, 2026

View reviewed changes

Comment thread libs/core/executors/include/hpx/executors/executor_scheduler_bulk.hpp Outdated

hkaiser reviewed May 5, 2026

View reviewed changes

Comment thread libs/core/executors/include/hpx/executors/parallel_executor.hpp Outdated

Comment thread libs/core/executors/include/hpx/executors/parallel_executor.hpp Outdated

Comment thread libs/core/executors/include/hpx/executors/sequenced_executor.hpp Outdated

hkaiser mentioned this pull request May 5, 2026

Implement P2300 get_scheduler bridge for executors #7238

Open

3 tasks

shivansh023023 force-pushed the feat/p2300-bulk-integration branch from 7b3fc5f to 70c4501 Compare May 6, 2026 21:14

hkaiser reviewed May 6, 2026

View reviewed changes

Comment thread libs/core/executors/include/hpx/executors/parallel_executor.hpp Outdated

hkaiser reviewed May 6, 2026

View reviewed changes

charan-003 reviewed May 6, 2026

View reviewed changes

Comment thread libs/core/executors/include/hpx/executors/executor_scheduler.hpp

Comment thread libs/core/executors/include/hpx/executors/executor_scheduler_bulk.hpp

charan-003 reviewed May 6, 2026

View reviewed changes

Comment thread libs/core/executors/include/hpx/executors/executor_scheduler.hpp Outdated

shivansh023023 pushed a commit to shivansh023023/hpx that referenced this pull request May 7, 2026

Address maintainer feedback for PR TheHPXProject#7240: fwd header, se…

1f4aeee

…nder/receiver concepts, NVCC guard, dedup get_scheduler, fix post() UB, use get_scheduler in tests

shivansh023023 force-pushed the feat/p2300-bulk-integration branch from bbfc7db to 1f4aeee Compare May 7, 2026 06:35

hkaiser reviewed May 7, 2026

View reviewed changes

Comment thread libs/core/executors/include/hpx/executors/executor_scheduler.hpp Outdated

Comment thread libs/core/executors/include/hpx/executors/executor_scheduler_bulk.hpp Outdated

Comment thread libs/core/executors/include/hpx/executors/parallel_executor.hpp Outdated

shivansh023023 force-pushed the feat/p2300-bulk-integration branch from d0b7f07 to 7ab5571 Compare May 8, 2026 07:39

shivansh023023 force-pushed the feat/p2300-bulk-integration branch 2 times, most recently from 01f7224 to 371a9ee Compare May 17, 2026 10:09

shivansh023023 force-pushed the feat/p2300-bulk-integration branch from 371a9ee to 6a61c4e Compare May 17, 2026 14:11

Hackathon User and others added 8 commits May 17, 2026 19:44

Address maintainer feedback for PR TheHPXProject#7240: fwd header, se…

671729c

…nder/receiver concepts, NVCC guard, dedup get_scheduler, fix post() UB, use get_scheduler in tests Signed-off-by: Shivansh <singhshivansh023@gmail.com>

Fix formatting

c0948c7

Signed-off-by: Shivansh <singhshivansh023@gmail.com>

Migrate from deprecated transform_completion_signatures_of to transfo…

ecd87f7

…rm_completion_signatures Signed-off-by: Shivansh <singhshivansh023@gmail.com>

Modernize PR 7240: Use C++20 requires clause and remove redundant exe…

d7fedc0

…cutor includes Signed-off-by: Shivansh <singhshivansh023@gmail.com>

style: apply clang-format to executor_scheduler_bulk

993005b

Signed-off-by: Shivansh <singhshivansh023@gmail.com>

shivansh023023 force-pushed the feat/p2300-bulk-integration branch from 6a61c4e to 993005b Compare May 17, 2026 14:14

hkaiser reviewed May 17, 2026

View reviewed changes

shivansh023023 added 2 commits May 18, 2026 14:36

executors: address review feedback and migrate to stdexec member func…

7be80ea

…tions Signed-off-by: Shivansh Singh <singhshivansh023@gmail.com>

style: apply clang-format to executors module

e1cde7c

Signed-off-by: Shivansh Singh <singhshivansh023@gmail.com>

hkaiser reviewed May 18, 2026

View reviewed changes

executors: fix stdexec SFINAE constraints, noexcept specs, and access…

d357952

… modifiers Signed-off-by: Shivansh Singh <singhshivansh023@gmail.com>

Uh oh!

Conversation

shivansh023023 commented May 2, 2026

P2300 bulk Integration for HPX Executors

Description

Key Technical Improvements:

Proposed Changes

Background context

Checklist

Uh oh!

StellarBot commented May 2, 2026

Uh oh!

codacy-production Bot commented May 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Up to standards ✅

Uh oh!

hkaiser commented May 3, 2026

Uh oh!

shivansh023023 commented May 5, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

shivansh023023 commented May 6, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

charan-003 left a comment

Choose a reason for hiding this comment

Uh oh!

charan-003 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

shivansh023023 commented May 7, 2026

Uh oh!

shivansh023023 commented May 7, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

shivansh023023 commented May 8, 2026

Uh oh!

hkaiser commented May 8, 2026

Uh oh!

shivansh023023 commented May 8, 2026

Uh oh!

shivansh023023 commented May 16, 2026

Uh oh!

hkaiser commented May 16, 2026

Uh oh!

charan-003 commented May 16, 2026

Uh oh!

codacy-production Bot commented May 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Up to standards ✅

Uh oh!

hkaiser May 17, 2026

Choose a reason for hiding this comment

Uh oh!

hkaiser May 17, 2026

Choose a reason for hiding this comment

Uh oh!

hkaiser May 17, 2026

Choose a reason for hiding this comment

Uh oh!

hkaiser May 17, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

hkaiser left a comment

P2300 `bulk` Integration for HPX Executors

codacy-production Bot commented May 2, 2026 •

edited

Loading

codacy-production Bot commented May 17, 2026 •

edited

Loading

shivansh023023 commented May 18, 2026 •

edited

Loading