Skip to content

feat: add PT2 and TorchScript backend plugins - #36

Open
malwan-nv wants to merge 36 commits into
leapp-runtimefrom
malwan-nv/leapp-runtime-plugins
Open

malwan-nv wants to merge 36 commits into
leapp-runtimefrom
malwan-nv/leapp-runtime-plugins

Conversation

@malwan-nv

Copy link
Copy Markdown
Collaborator

Adds Triton backend plugins for PT2 execution through NativeRT and for TorchScript execution, with shared Torch configuration and tensor-handling utilities.
Includes build and installation instructions, a pinned source-build recipe for NativeRT, and CPU/CUDA coverage. Both plugins built successfully; 14 Rust integration checks and the C++ input-analysis and tensor-ownership checks passed.

Signed-off-by: Marc Alwan <malwan@nvidia.com>
Signed-off-by: Marc Alwan <malwan@nvidia.com>
Signed-off-by: Marc Alwan <malwan@nvidia.com>
Signed-off-by: Marc Alwan <malwan@nvidia.com>
Signed-off-by: Marc Alwan <malwan@nvidia.com>
Signed-off-by: Marc Alwan <malwan@nvidia.com>
Signed-off-by: Marc Alwan <malwan@nvidia.com>
Signed-off-by: Marc Alwan <malwan@nvidia.com>
Signed-off-by: Marc Alwan <malwan@nvidia.com>
Signed-off-by: Marc Alwan <malwan@nvidia.com>
Signed-off-by: Marc Alwan <malwan@nvidia.com>
Signed-off-by: Marc Alwan <malwan@nvidia.com>
Signed-off-by: Marc Alwan <malwan@nvidia.com>
Signed-off-by: Marc Alwan <malwan@nvidia.com>
Signed-off-by: Marc Alwan <malwan@nvidia.com>
Signed-off-by: Marc Alwan <malwan@nvidia.com>
Signed-off-by: Marc Alwan <malwan@nvidia.com>
Signed-off-by: Marc Alwan <malwan@nvidia.com>
Signed-off-by: Marc Alwan <malwan@nvidia.com>
Signed-off-by: Marc Alwan <malwan@nvidia.com>
@malwan-nv
malwan-nv added this pull request to stack #35 October 19, 2026 21:45
Move the bundle regression target under tests/unit_tests and remove the redundant C++ fixture-loading check. Keep the inspection-only build instructions aligned with the test layout.

Signed-off-by: Marc Alwan <malwan@nvidia.com>
Activate the exported Session contract while retaining legacy implementation sources for the following engine and binding commits. These sources are temporarily excluded from build targets so the merge compiles against the new API and preserves their subsequent renames.

Signed-off-by: Marc Alwan <malwan@nvidia.com>
Export Session, construction options, result/error types, and backend-independent DLPack ownership. Compile the common binding wrapper against the real core API. Execution returns Unsupported in this contract revision and is supplied by the runtime integration.

Signed-off-by: Marc Alwan <malwan@nvidia.com>
Signed-off-by: Marc Alwan <malwan@nvidia.com>
Signed-off-by: Marc Alwan <malwan@nvidia.com>
Signed-off-by: Marc Alwan <malwan@nvidia.com>
Bring the latest develop fixes and formatting requirements through the PR stack while preserving published commit history.

Apply required Ruff formatting without changing Python syntax trees.

Signed-off-by: Marc Alwan <malwan@nvidia.com>
Bring the latest develop fixes and formatting requirements through the PR stack while preserving published commit history.

Apply required Ruff formatting without changing Python syntax trees.

Signed-off-by: Marc Alwan <malwan@nvidia.com>
Bring the latest develop fixes and formatting requirements through the PR stack while preserving published commit history.

Apply required Ruff formatting without changing Python syntax trees.

Signed-off-by: Marc Alwan <malwan@nvidia.com>
@malwan-nv
malwan-nv removed this pull request from stack #35 October 21, 2026 23:09
@malwan-nv
malwan-nv added this pull request to stack #43 October 21, 2026 23:09
Signed-off-by: Marc Alwan <malwan@nvidia.com>
Signed-off-by: Marc Alwan <malwan@nvidia.com>
@malwan-nv
malwan-nv removed this pull request from stack #43 October 26, 2026 18:43
@malwan-nv
malwan-nv added this pull request to stack #47 October 26, 2026 18:44
Expose the public API as lrt across Rust, Python, and C++. Accept borrowed input maps and define session-owned output-cache retention with bundle-derived byte sizing. Update bindings, documentation, and API checks.

Signed-off-by: Marc Alwan <malwan@nvidia.com>
Merge the reviewed public API updates and use borrowed input maps throughout
execution and bindings. Move CPU/CUDA allocations and output pooling into shared
engine memory helpers, with independent session caches configured once through
SessionOptions and byte budgets derived from bundle and dynamic output sizes.

Align Rust, Python, and C++ APIs under lrt, preserve retained output ownership,
and simplify runtime documentation. Keep editor settings local and excluded
from version control.

Validated desktop CPU/GPU execution, pool ownership and retention, Python and
C++ bindings, and the installed C++ SDK consumer. Clippy, Rustdoc, and pinned
Ruff formatting checks pass.

Signed-off-by: Marc Alwan <malwan@nvidia.com>
Merge the reviewed runtime updates from PR #34. Adapt TorchScript callers to
borrowed input maps and lrt naming, and pass caller-owned output pools through
NativeRT integration tests. Document initialization-only pool configuration
and verify retained outputs, independent peers, and failure isolation without
changing the plugin implementation or ABI.

Validated 15 NativeRT/TorchScript integration tests on CPU and CUDA, rebuilt
both plugins, and passed the C++ CPU/CUDA ownership checks. Clippy, Rustdoc,
and pinned Ruff formatting checks pass.

Signed-off-by: Marc Alwan <malwan@nvidia.com>
Signed-off-by: Marc Alwan <malwan@nvidia.com>
Signed-off-by: Marc Alwan <malwan@nvidia.com>
Base automatically changed from malwan-nv/leapp-runtime-integration to leapp-compiled-runtime October 29, 2026 21:20
@malwan-nv
malwan-nv removed this pull request from stack #47 October 29, 2026 22:14
@malwan-nv
malwan-nv changed the base branch from leapp-compiled-runtime to malwan-nv/leapp-runtime-execution October 29, 2026 22:14
@malwan-nv
malwan-nv added this pull request to stack #53 October 29, 2026 22:15
@malwan-nv
malwan-nv removed this pull request from stack #53 October 29, 2026 22:33
@malwan-nv
malwan-nv changed the base branch from malwan-nv/leapp-runtime-execution to malwan-nv/leapp-runtime-bundle October 29, 2026 22:34
@malwan-nv
malwan-nv added this pull request to stack #54 October 29, 2026 22:34
Base automatically changed from malwan-nv/leapp-runtime-bundle to leapp-runtime October 29, 2026 23:15

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant