[Task][Megatron] 4.4: CUDA-target Megatron compatibility

[lvyufeng]

Parent Issue

Part of #206 — Megatron Roadmap, Phase 4 (Advanced Features / Stretch): Task 4.4

Description

Extend Megatron compatibility to CUDA backend. This is a stretch goal that depends on the CUDA backend and NCCL support being mature.

Requirements

• NCCL backend implementation
• CUDA compute kernels for Megatron workloads
• CUDA graph capture for training
• Parity with NPU path for Megatron operations
• Multi-GPU CUDA training validated

Blocked By

• CUDA backend maturity
• NCCL backend implementation
• All NPU-first Megatron tasks

Blocks

• CUDA-based Megatron training