Repository navigation
feat: standalone dvm-rt packaging and eager fusion for PyTorch - #37
Draft
GinFungYJF wants to merge 2 commits into
Draft
GinFungYJF wants to merge 2 commits into
GinFungYJF wants to merge 2 commits into
Conversation
GinFungYJF
marked this pull request as draft
November 17, 2026 07:57
GinFungYJF
force-pushed
the
feature/pytorch-eager-fusion
branch
7 times, most recently
from
November 19, 2026 03:00
b2123ed to
d20fb74
Compare
Package DVM as an independent runtime and integrate eager operator fusion with native PyTorch on CUDA, including the GPU ReduceSum support in aba9572. - Build libdvm.a and libdvm.so.1, export and verify public shared-library symbols, and provide stable-address IntArrayRef storage. - Build dvm-rt independently of torch and link dvm-torch to the runtime and ATen CUDA libraries. Load PyTorch before importing the extension. - Enable CUDA overrides explicitly through the environment or enable(). Capture original kernels before registration and retain their exact argument types for unboxed fallback calls. - Capture elementwise operations, scalar operands and in-place variants. Register maximum/minimum under their default overload and propagate NaNs. - Capture sum and mean with full or selected dimensions, negative axes, keepdim and explicit dtype conversion. Accumulate FP16/BF16 inputs in FP32 and use native kernels for unsupported inputs and deterministic mode. - Use kEager with unified workspace for reduction consumers, broadcasts and branches. Own shape and dimension arrays throughout kernel execution and trim graphs within the runtime dependency-mask capacity. - Track pending values by storage and match shape, dtype and offset before reusing nodes. Defer stores, eliminate dead outputs and require complete native replay records before replaying small graphs. - Flush on stream/thread changes, aliases, host reads and unsupported ops. Register coverage guards for aten, c10d and _c10d_functional CUDA and CompositeExplicitAutograd operators. - Update operator coverage documentation and add reduction regressions. Validated on V100 with PyTorch 2.14.1+cu126 and CUDA 12.6. Both wheels build, 246 fusion tests pass, and 66 GPU reduction/eager-runtime tests pass.
Keep large tensor chains fused beyond 256 MiB while preserving operator-count and dependency-mask bounds. Add coverage for a large chain with dead intermediate stores. Validation: 301 fusion tests passed; BERT and large-shape forward/backward checks passed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Paired: GitHub #37 ↔ GitCode !939
What type of PR is this?
/kind task
What does this PR do / why do we need it:
Make DVM distributable on its own and add graph-less operator fusion for native PyTorch on CUDA.
Which issue(s) this PR fixes:
Fixes #1
What scenarios were tested, and what were the verification results(Function, performance, reliability, etc.):
Self-checklist:(Please check carefully,and mark an x in the [] brackets. We will review your completion status.)