Skip to content

feat: standalone dvm-rt packaging and eager fusion for PyTorch - #37

Draft
GinFungYJF wants to merge 2 commits into
mindspore-ai:masterfrom
GinFungYJF:feature/pytorch-eager-fusion
Draft

GinFungYJF wants to merge 2 commits into
mindspore-ai:masterfrom
GinFungYJF:feature/pytorch-eager-fusion

Conversation

@GinFungYJF

@GinFungYJF GinFungYJF commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Paired: GitHub #37 ↔ GitCode !939

What type of PR is this?

/kind task

What does this PR do / why do we need it:

  • Need to describe clearly

Make DVM distributable on its own and add graph-less operator fusion for native PyTorch on CUDA.

Which issue(s) this PR fixes:

Fixes #1

What scenarios were tested, and what were the verification results(Function, performance, reliability, etc.):

  • Need to describe clearly

Self-checklist:(Please check carefully,and mark an x in the [] brackets. We will review your completion status.)

  • Design: Has the solution corresponding to the PR been reviewed by the Maintainer, and have all review comments been replied to and revised
  • Test: Has the code in the PR been fully covered by UT/ST test cases, and have the newly added test cases been uploaded to the repository along with this PR or already uploaded.
  • Verification: Does the PR description contains a detailed description of the verification results regarding the achievement of the expected goals for the Feature, Refactor, and Bugfix to this PR.
  • Interface: Does it involve changes to external interfaces? The corresponding changes have been approved by the interface review organization, and the annotation information for the API has been correctly refreshed.
  • Document: Does it involve modifications to the official website documentation? If so, please submit the materials to the Doc repository in a timely manner.

@GinFungYJF
GinFungYJF marked this pull request as draft November 17, 2026 07:57
@GinFungYJF
GinFungYJF force-pushed the feature/pytorch-eager-fusion branch 7 times, most recently from b2123ed to d20fb74 Compare November 19, 2026 03:00
Package DVM as an independent runtime and integrate eager operator fusion
with native PyTorch on CUDA, including the GPU ReduceSum support in aba9572.

- Build libdvm.a and libdvm.so.1, export and verify public shared-library
  symbols, and provide stable-address IntArrayRef storage.
- Build dvm-rt independently of torch and link dvm-torch to the runtime and
  ATen CUDA libraries. Load PyTorch before importing the extension.
- Enable CUDA overrides explicitly through the environment or enable().
  Capture original kernels before registration and retain their exact
  argument types for unboxed fallback calls.
- Capture elementwise operations, scalar operands and in-place variants.
  Register maximum/minimum under their default overload and propagate NaNs.
- Capture sum and mean with full or selected dimensions, negative axes,
  keepdim and explicit dtype conversion. Accumulate FP16/BF16 inputs in
  FP32 and use native kernels for unsupported inputs and deterministic mode.
- Use kEager with unified workspace for reduction consumers, broadcasts
  and branches. Own shape and dimension arrays throughout kernel execution
  and trim graphs within the runtime dependency-mask capacity.
- Track pending values by storage and match shape, dtype and offset before
  reusing nodes. Defer stores, eliminate dead outputs and require complete
  native replay records before replaying small graphs.
- Flush on stream/thread changes, aliases, host reads and unsupported ops.
  Register coverage guards for aten, c10d and _c10d_functional CUDA and
  CompositeExplicitAutograd operators.
- Update operator coverage documentation and add reduction regressions.

Validated on V100 with PyTorch 2.14.1+cu126 and CUDA 12.6. Both wheels build,
246 fusion tests pass, and 66 GPU reduction/eager-runtime tests pass.
Keep large tensor chains fused beyond 256 MiB while preserving operator-count and dependency-mask bounds. Add coverage for a large chain with dead intermediate stores.

Validation: 301 fusion tests passed; BERT and large-shape forward/backward checks passed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant