Skip to content

[RFC]: External routing and MXFP8 pass-through for synchronous AFD on Ascend 950 (A5) #522

Open

Description

@Pingzii

This RFC proposes an optional Attention-side router/shared-expert placement for synchronous A5 AFD, together with MXFP8 activation pass-through to the FFN role.

Collaborators: @ksiyuan, @wangyao-i, @cqq70605.

1. Motivation

AFD allows Attention (A) and FFN (F) resources to be provisioned independently. An important deployment pattern is to have multiple Attention instances feed a smaller shared FFN pool. Aggregating tokens from the Attention DP replicas gives FFN workers larger expert-computation batches, creating an opportunity to improve model FLOPs utilization (MFU) and amortize expert-weight reads over more tokens, while Attention instances retain distributed request and KV-cache capacity.

Realizing this benefit requires delivering the aggregated activations from A to F. As more Attention instances contribute tokens, A2E activation traffic grows. If A and F are placed on different hosts, that traffic also crosses the inter-node fabric. Both token aggregation and cross-host placement can put pressure on activation delivery; when available bandwidth limits execution, transfer time can reduce the benefit of larger FFN batches. Reducing A2E bytes is therefore particularly valuable for bandwidth-constrained, cross-host A/F deployments.

This RFC proposes MXFP8 activation transport and direct consumption on Ascend 950 (A5), reducing A2E activation-data bytes relative to BF16 while carrying the associated E8M0 scale. The initial implementation targets the synchronous CAMP2pAFDConnector. To preserve the transport saving through the FFN input path, it also moves router and shared-expert execution to A, where model-dtype activations are already available. F receives the quantized activation, scale, and top-k, and runs only the routed experts, removing the need to recover BF16 activations on F for router/shared-expert computation.

The initial measurements cover single-host 4A4F. Higher A:F ratios and cross-host bandwidth constraints motivate the design and the scaling evaluation described in section 5.6; they are not yet measured deployment results.

2. Proposed execution

flowchart LR
    H["A: attention output h"] --> G["A: router / top-k"]
    H --> S["A: shared experts"]
    H --> Q["A: optional MXFP8 quantization"]
    G --> T["A2E: activation + top-k + optional MX scale"]
    Q --> T
    T --> F["F: align and run routed experts"]
    F --> R["E2A: routed output"]
    R --> ADD["A: add outputs"]
    S --> ADD
Loading

This role split follows the existing AFD asynchronous CAM implementation, adapting Attention-side routing and shared-expert execution to the synchronous CAMP2P path.

The goal is to reduce A2E activation traffic on Ascend 950 (A5) while allowing F to consume the transported MXFP8 activation directly. Keeping router and shared-expert computation on F requires model-dtype activations for those operations. Recovering BF16 activations adds device work and memory traffic, with larger token batches requiring more data to be processed. Moving both operations to A removes that recovery requirement and allows F to consume the received FP8 activation and scale directly for routed-expert computation.

The proposal therefore moves both the router and shared experts to A, where the model-dtype activation is already available. A computes top-k before transport quantization and computes the shared-expert output from that model-dtype input. With MXFP8 enabled, it sends the FP8 activation, E8M0 scale, and top-k IDs/weights to F, then adds the returned routed output to its shared-expert output. BF16 external routing uses the same role split without transport quantization.

F consumes the received expert IDs and weights and runs only the routed experts. The model helper preserves the target model's routing and scaling semantics: top-k weights already include model-level routed scaling, so F does not apply that factor again. Finalize returns the routed result in the original token order, with communication-specific reduction handled by the MoE implementation.

Router and shared-expert placement share one opt-in switch because both must move to enable direct consumption. Moving only the router would leave the shared experts needing a BF16 activation on F and retain the activation-recovery step.

With external routing enabled, MoE weight placement and execution are:

Component A weights F weights Computation
Router Loaded Retained for the native MoE structure A computes top-k; F does not run its router
Shared experts Loaded when present in the model Not constructed or loaded A computes the shared output
Routed experts Not loaded Loaded and expert-parallel sharded F computes the routed output

This table covers MoE weights, not the full checkpoint. Moving router execution does not mean removing every router parameter from F: the current native MoE still constructs and loads its copy. The proposal avoids duplicating shared and routed experts across roles. Device-memory impact remains to be measured. With the option disabled, the original router/shared/routed computation stays on F.

3. Transfer and consumption contract

3.1 Payload and configuration

Let M be the receiving F rank's local row count before MoE prepare, H the hidden width, K the model's top-k, and C = ceil(H / 32).

Payload field BF16 external routing MXFP8 pass-through
hidden_states BF16 [M, H] E4M3 FP8 [M, H]
topk_ids int32 [M, K] int32 [M, K]
topk_weights float32 [M, K] float32 [M, K]
activation_scale absent E8M0 [M, C]

These fields describe the same rows on the same device. Expert IDs and weights arrive together. A quantized activation and its activation scale also arrive together; a missing half, wrong format, or mismatched shape/device is an error. Validation checks metadata without reading tensor contents back to the host.

The existing A2E IDs/weights slots carry either token IDs for the original router-on-F protocol or expert IDs/weights for external routing. They do not carry both meanings in one transfer. The wire slot named scales carries routing weights; it is distinct from the E8M0 activation scale.

The payload maps to the A2E operator as follows:

Logical field A producer A2E input and corresponding received output F consumer
hidden_states Model-dtype activation, optionally quantized x → received activation Prepare, then routed-expert computation
topk_ids Router's expert selection expert_ids → received expert IDs Prepare and expert selection
topk_weights Router weights including routed scaling scales → received routing weights Prepare and weighted expert computation
activation_scale MXFP8 quantization of that activation x_scale → received MX scale Prepare's pertoken_scale, then the path-specific expert consumer

The F connector constructs the payload from these received tensors; the runner forwards the fields under the same names. The scale then follows section 3.2's alignment and layout rules. In particular, A2E's scales means routing weights, whereas MC2 dispatch's scales means activation scale. These operator parameters share a name but do not carry interchangeable data.

The proposed configuration matrix is:

afd.compute_gate_on_attention afd.connector_extra_config.quant_mode Meaning
false or omitted 0 or omitted Original router-on-F path
true 0 BF16 external routing
true 4 MXFP8 external routing and pass-through
false 4 Outside this proposal; reject rather than silently dequantize

The default stays unchanged. Both roles use matching gate and transport modes. The gate switch belongs in the common afd block, not connector_extra_config. For example, with the role-specific fields supplied separately:

{
  "afd": {
    "role": "attention",
    "connector": "CAMP2pAFDConnector",
    "host": "127.0.0.1",
    "port": 6239,
    "num_attention_ranks": 4,
    "num_ffn_ranks": 4,
    "compute_gate_on_attention": true,
    "connector_extra_config": {"quant_mode": 4}
  }
}

F uses the same settings with role: "ffn". This is a configuration fragment, not a complete launch recipe.

Gate placement and transport mode select the transfer's meaning, not just a local optimization. A and F need matching gate/transport modes, activation and scale formats, model dimensions, and a consistent rank topology. A mismatch is invalid; it is not a request to reinterpret the payload or fall back to another mode. Local shape/dtype checks cannot establish agreement with the peer.

Current code validates local settings but does not negotiate these fields across roles. Deployment therefore requires a consistent configuration for both roles, checked before sending traffic. This RFC defines that compatibility requirement without introducing a new version-negotiation protocol. A future startup check could use the existing control plane to reject disagreement before data transfer.

3.2 Alignment and scale layout

Transfer scale is always two-dimensional. Prepare and dispatch preserve row correspondence across activations, routing, and scale; their transformations are not independent choices.

Path Prepare and dispatch Scale at consumption
ALLGATHER Pad each rank to the DP-wide maximum, then gather activation, routing, and scale in the same rank order Normalize the gathered scale at the expert boundary for the native GMM path
MC2 Keep local rows aligned with the active mask and prepare padding; dispatch activation and scale to the selected experts together Pass a two-dimensional scale into dispatch; the native downstream helper normalizes its output for GMM

Padding uses zero activations and routing weights, a valid placeholder expert ID, and E8M0 byte 127 (scale 1). E8M0 padding and collectives can use a byte view to preserve the stored exponents. Finalize removes padding under the same token metadata used by prepare.

For the validated H=4096 model, transfer scale is [M,128]. After alignment or dispatch, the native GMM layout is [R,64,2], where R is that stage's row count, not necessarily M. The connector and runner do not perform expert reordering or convert the transfer scale into this three-dimensional layout.

MC2 has a separate quantization control from A2E transport. For E4M3 input with an E8M0 scale, dispatch receives scales=<scale> and quant_mode=0: communicate the existing quantized values and scale without dynamic quantization. An explicit incompatible comm_quant_mode is rejected. With no external scale, dispatch retains its original quantization selection. This contract depends on the installed operator implementation, not merely on the presence of a Python argument; see the pinned dispatch operator contract.

3.3 Graph and call lifetime

Capture records receives and subsequent device operations. Replay uses the storage bound to that captured graph: the receive operations refresh activation, top-k, and scale there before expert computation reads them. Python attribute assignments made during capture are not a refresh mechanism during replay.

F caches graphs by its existing per-rank token-count key. Each graph retains its own bound tensor views and lifetimes; this proposal does not introduce one mutable top-k buffer shared across graph keys. Any change that moves a receive outside capture needs an explicit update of the captured input storage before replay. Temporary Python routing/scale/mask references are cleared after the call, including exceptional exits.

Validation checks actual replay and changed inputs at a fixed graph key, then switches keys. Successful capture or higher throughput alone does not establish correct replay.

4. Plugin boundary and supported scope

The model wrappers own router/shared placement and weight loading. The connector owns transport and wire formats. The runner forwards the payload. MoE prepare, dispatch, and quantization own row alignment and kernel-specific scale layout.

Three pinned compatibility patches bridge missing upstream interfaces:

Patch Why needed Removal condition
ALLGATHER prepare Native prepare does not align externally supplied top-k and activation scale Upstream prepare accepts and aligns that bundle
MC2 prepare External routing and scale need the activation's padding and active-row layout Upstream prepare carries the bundle through those transformations
MC2 dispatch kwargs Native kwargs do not forward this external scale or select the pre-quantized mode Upstream dispatch supports the same external-scale contract

The patches preserve upstream behavior when no external routing/scale is provided. Installation needs compatibility checks against the pinned upstream implementation and idempotence. It does not modify the vLLM source checkout. Plugin-owned extension points are preferred when an equivalent hook is available.

The initial target is A5, DeepSeek-V4-Flash W4A8MXFP, synchronous CAMP2P, TP1, without sequence parallelism, DBO, or ubatching. MoE ALLGATHER and MC2 are the supported paths; ALLTOALL and FUSED_MC2 are outside the proposal. Eager and FULL_DECODE_ONLY are the execution modes under evaluation. MXFP8 additionally requires the target A2E operator's equal A/F rank-count topology and E4M3/E8M0 format; device evidence covers single-node 4A4F only.

The proposed guardrail requirement is to reject unsupported configurations as early as the relevant facts are available, before entering an incompatible device operation. This includes unsupported communication policies, incompatible A/F settings, and platform requirements. These are design requirements; completion of the corresponding checks is part of implementation review before merge.

The MXFP8 operator build needs the A2E extensions described in #444, including x_scale and quant_mode. BF16 keeps the original operator call. Async CAM, wider TP, new communication paths, multi-node qualification, and changes to the default mode are outside this RFC.

5. Evidence and alternatives

5.1 Measurement conditions and interpretation

Two datasets are reported separately: the current BF16 split-branch session and the earlier MXFP8 development-branch experiments. Comparisons stay within each dataset. Section 5.5 summarizes the validation status and measurement scope.

Setting Recorded value
Hardware and topology Single-node A5 4A4F; A DP4/TP1 on devices 0–3, F DP4/TP1 on devices 4–7
Model DeepSeek-V4-Flash, W4A8MXFP
Runtime limits max-model-len=9216, max-num-seqs=128, max-num-batched-tokens=3072
KV settings kv-cache-dtype=fp8, indexer_kv_dtype=fp8
MoE communication Automatic selection for every configuration below
Load 128 requests, target concurrency 128, synthetic input setting 8192, output 32
Reported token counts 1,059,200 input / 4,096 output tokens in the cited benchmark logs; reported input averages 8275 per request
Execution Eager uses --enforce-eager; Graph uses FULL_DECODE_ONLY, capture sizes [8,16,32,64,128]
Throughput metric Benchmark-reported Total Token Throughput, input plus output tokens per second

The recorded benchmark runs completed with 128 successes and no failed requests. Graph windows were approximately 3 seconds; eager windows were roughly 5–7 seconds. The configured input length and reported token count are shown separately. All means and ratios below are recomputed from the listed throughput values; ratios use unrounded means. n counts benchmark runs, not requests.

5.2 Current BF16 placement comparison

The BF16 comparison uses 828582a for both gate placements and bcc0126 as the clean baseline, with BF16 transport (quant_mode=0) and automatic communication selection.

Mode Placement / code Each run (token/s) Mean (token/s) n
Eager Gate on A, 828582a 192,374 / 193,400 192,887 2
Eager Gate on F, 828582a 166,567 / 161,427 163,997 2
Eager Clean baseline, bcc0126 165,081 / 164,005 164,543 2
Graph Gate on A, 828582a 327,970 / 324,830 326,400 2
Graph Gate on F, 828582a 333,602 / 329,309 331,455.5 2
Graph Clean baseline, bcc0126 334,084 / 332,730 333,407 2
Comparison Eager Graph (FULL_DECODE_ONLY)
Router/shared on A versus on F, same commit +17.6% throughput -1.5% throughput
Router/shared on F versus clean baseline -0.3% -0.6%

Graph gate-on-A GSM8K was 90.60% over 1319 questions. These results are also recorded in #503. They support offering a placement choice while retaining the current default.

5.3 Historical MXFP8 performance: Graph

The historical experiment used vLLM ced6857, vLLM-Ascend 8d4409d6, MXFP8 development code f683eb1, and clean baseline bcc0126. BF16 controls come from earlier development revisions; the gate-on-A control was recorded as 52a292d. The cross-revision comparison is qualified in section 5.5.

Configuration Each run (token/s) Mean (token/s) n Observed range (token/s)
Clean baseline, gate on F, BF16 343,163 / 340,709 / 338,660 340,844 3 338,660–343,163
Development control, gate on F, BF16 341,995 / 341,185 341,590 2 341,185–341,995
Historical control, gate on A, BF16 336,305 / 334,131 335,218 2 334,131–336,305
Gate on A, MXFP8 347,730 / 351,374 / 367,132 / 356,085 355,580.25 4 347,730–367,132
MXFP8 comparison Ratio of means minus one Interpretation
Gate-on-A MXFP8 versus clean BF16 baseline +4.32% Entire proposal versus a fixed clean baseline; includes placement and transport changes
Gate-on-A MXFP8 versus gate-on-A BF16 +6.07% Historical cross-revision comparison; does not isolate quantization alone

The observed ranges and individual runs are retained above so readers can assess the aggregate alongside run-to-run variation.

5.4 Historical MXFP8 performance: eager

Configuration Each run (token/s) Mean (token/s) n Observed range (token/s)
Clean baseline bcc0126, gate on F, BF16 177,273 177,273 1 177,273
Development control, gate on F, BF16 191,229 / 188,364 189,796.5 2 188,364–191,229
Historical control, gate on A, BF16 226,743 / 217,608 222,175.5 2 217,608–226,743
Gate on A, MXFP8 225,276 / 224,819 225,047.5 2 224,819–225,276

The eager measurements provide workload context. Historical full GSM8K for MXFP8 was 90.30% in eager and 90.14% in Graph, each over 1319 questions.

The tables above summarize historical development measurements. The Graph aggregate was also reported in #497. These measurements span development revisions and motivate a controlled comparison on the final implementation.

5.5 Validation status, measurement scope, and alternatives

BF16 external routing has device evidence at 828582a. Historical MXFP8 measurements provide preliminary performance and full-dataset task-accuracy evidence for the development stack. The current MXFP8 revision 4d2d890 has passed CPU checks; device validation of the final stack remains pending. The original profile fix was exercised in the BF16 integration stack; its replacement described in section 6.1 requires revalidation.

Performance observations use small samples and short batch-completion windows; no statistical significance test was performed. The historical MXFP8 comparison spans development revisions, with incomplete per-run attribution for the BF16 control, and does not isolate quantization alone. Complete dependency versions were not captured for the current BF16 session, so that comparison is limited to its within-session arms.

Graph versus eager changes compilation settings as well as ACL Graph replay. Neither the mode-dependent performance mechanism nor an A2E bandwidth bottleneck has been established by profiling. Final-stack validation should record exact versions, communication-path execution, and capture/replay with changing inputs; the BF16 split-branch session did not retain per-path instrumentation. Task accuracy, throughput, and CPU interface checks serve different validation goals.

Keeping routing on F avoids the added A work and remains the default. A transport-only alternative can retain the received FP8 activation and scale for routed experts while recovering model-dtype activations for router/shared computation. Moving both operations to A removes that recovery requirement. The proposed capability supports the runtime's automatic selection between ALLGATHER and MC2.

5.6 Potential for larger gains at scale

Attention-side routing and MXFP8 pass-through provide two complementary ways to adapt AFD to deployment needs: placing router/shared computation on A creates an opportunity to improve A/F load balance, while quantized transport reduces the activation payload delivered to F. With suitable A/F resource allocation and interconnect characteristics, these capabilities offer potential for larger gains as deployments scale.

The historical MXFP8 Graph measurements above used one eight-device host, with A and F communicating through intra-node HCCL. Mean throughput was 355,580.25 token/s: +4.32% versus the clean BF16 baseline and +6.07% versus the earlier gate-on-A BF16 control. The measurement limits in section 5.5 apply. These observations motivate investigating larger deployments, where aggregation and interconnect bandwidth may make A2E quantization more valuable.

Higher A:F ratios can increase pressure on activation delivery. AFD allows multiple Attention instances to feed fewer FFN instances, combining distributed KV-cache capacity with larger expert-computation batches. As more Attention instances contribute tokens, incoming activation volume and FFN work both grow. If expert computation remains efficient while activation delivery becomes a limiting resource, a smaller A2E payload may help preserve the benefits of aggregation. This is a promising extension beyond the initial equal-rank MXFP8 target; higher A:F device ratios need corresponding operator and alignment support, not only a configuration change.

Bandwidth-constrained links can increase the value of payload reduction. When placement requires A/F traffic to cross an interconnect with lower effective bandwidth, activation delivery can consume more of the critical path. MXFP8 roughly halves the activation-data bytes relative to BF16. Activation scales, routing metadata, and other communication costs remain, so this does not halve total traffic or imply twice the throughput.

Larger gains are therefore plausible when A2E delivery limits execution, provided A-side quantization and added router/shared work do not create another bottleneck. E2A returns and expert communication are not compressed by this proposal and may limit the gain instead. This is an expectation, not a measured scaling result. Validation should vary A:F ratios, deployment size, interconnect bandwidth, and workload, and measure sustained throughput together with A2E time on the critical path.

6. Implementation PRs and validation

6.1 What is implemented

The proposal is implemented as a stack of three PRs, all open as of 2026-11-02. The PR stack is being updated to place profile-dependent expert-ID balancing behind a narrow opaque custom-op boundary, preserving the upstream profiling and compilation lifecycle. The updated stack will be restacked and revalidated before merge. The table records the posted revisions before this update.

#500 profile/compilation fix
  └─ #503 BF16 external routing
       └─ #506 MXFP8 pass-through
PR Branch Recorded revision Implemented scope
#500 pr1/profile-pass-compile 1da04df Keep profile execution out of compilation; guarded installation and tests
#503 pr2/external-routing 828582a A-side router/shared execution, top-k transport, F-side consumption, BF16 alignment on ALLGATHER and MC2
#506 pr3/mxfp8-passthrough 4d2d890 FP8/E8M0 transport, scale-carrying payload, per-path scale layout, pre-quantized MC2 dispatch

All PRs target main, so GitHub's displayed diff includes unmerged dependencies; their file and line counts are not independent changes to add together. Review the incremental ranges for #503 and #506. After a dependency merges, rebase and recheck the next PR's own diff and validation.

6.2 How the CPU checks are run

CPU tests use real PyTorch tensors for dtype, layout, padding, and scale-byte checks. Stand-ins replace vLLM/Ascend interfaces and operator calls that need the NPU environment. The maintained suite runs test files in isolated processes to avoid leaking stand-in modules between files. Representative focused commands are:

python -m pytest -q tests/unit/connectors/test_external_routing.py
python -m pytest -q tests/unit/compat/patches/npu/test_mc2_dispatch_prequant.py

The checks cover paired fields and rejection paths, connector arguments and received payloads, inert padding and rank order, scale contents, ALLGATHER versus MC2 layout at the expert boundary, dispatch mode conflicts, installer behavior, and preservation of the BF16 path.

PR Reported CPU result
#500 Maintained suite: 226 passed, 0 skipped, 0 failed
#503 99 focused cases; maintained suite: 235 passed, 0 skipped, 0 failed
#506 Maintained suite: 235 passed, 0 skipped, 0 failed

Counts are those reported by the linked PRs for the listed revisions. Device-validation status is summarized in section 5.5.

6.3 Device test scenarios and method

Device validation starts matching A/F services, waits for readiness, checks ordinary generation, and then runs ais_bench through the Attention role's vLLM API for synthetic throughput and GSM8K accuracy. Service startup time is outside the benchmark duration.

Scenario Procedure What it establishes
BF16 role-placement comparison On the same node/session, run gate-on-A, gate-on-F, and a clean baseline in eager and FULL_DECODE_ONLY; two benchmark runs for each of six cells The placement comparison in section 5.2; gate-on-A and gate-on-F share 828582a and differ in the gate switch
Synthetic throughput Use 128 requests/concurrency target 128, input setting 8192, output 32, and the runtime limits in section 5.1; retain reported token counts, duration, and success/failure counts Batch-completion throughput for this workload, including the observed effective input count
BF16 task accuracy Run full GSM8K; gate-on-A scored 91.13% in eager and 90.60% in Graph Task-level accuracy for the recorded BF16 configurations
Historical MXFP8 evaluation Run the pre-quantized path in eager and Graph with automatic communication selection; benchmark values are in sections 5.3–5.4 and full GSM8K scores were 90.30% / 90.14% Development-stack performance and full-dataset task accuracy

For final-stack validation, run the BF16 control and MXFP8 candidate at the same final commit under matching environment and execution settings. Record task accuracy, communication-path selection, and capture/replay with changing inputs at a fixed graph key and across key changes. Include service-restart repetitions and longer runs to assess reproducibility and sustained throughput.

If accepted, this proposal updates the existing Attention, FFN, model integration, connector, and patch documents and the CAMP2P user guide.

7. Planned model adaptations

The next model-adaptation targets are MiniMax-3.1-Flash and
DeepSeek-V4.1. The aim is to extend the same A-side routing and MXFP8 A2E
pass-through capability through model-specific adapters, reusing the common
connector, payload, and MoE alignment contracts.

7.1 Model-specific integration

Target model Adaptation work
MiniMax-3.1-Flash Add a plugin-owned adapter for role construction, checkpoint loading, and Attention-side routing. Preserve the native model's gate input, expert selection, top-k weighting, and output scaling. Place shared experts on A when the model has them, and connect the routed-expert backend to transported activations and scales.
DeepSeek-V4.1 Extend the DeepSeek adapter after checking the native model's construction, checkpoint layout, routing behavior, and quantization interfaces against the existing DeepSeek-V4 integration. Reuse compatible components while preserving V4.1-specific behavior, including token-dependent routing where applicable.

Each adapter obtains the hidden width, expert count, top-k, and quantization
layout from its target model and checkpoint. The DeepSeek-V4-Flash example
shapes in section 3 are not constants for these additional models. Shared experts
are constructed only when present in the model.

Router execution and any shared-expert computation use the model-dtype
activation on A. The connector carries the resulting top-k together with the
activation and, for MXFP8, its E8M0 scale. Model-specific routing and weight
semantics remain in the adapter; transport continues to use the common contract.
Direct MXFP8 consumption is enabled with the corresponding native expert-backend
support for the received activation and scale format.

7.2 Integration and validation sequence

For each model:

  1. Establish the native and router-on-F BF16 reference, including checkpoint
    loading and end-to-end generation.
  2. Integrate BF16 external routing, checking expert IDs, routing weights, routed
    scaling, and shared/routed output composition against the native path.
  3. Enable MXFP8 transport and direct consumption, preserving activation/scale row
    correspondence through ALLGATHER and MC2 automatic selection.
  4. Validate eager execution, then graph capture and replay with changing inputs
    and batch sizes, using the narrow routing custom-op boundary for
    profile-dependent behavior.
  5. Record model-appropriate task accuracy and same-revision BF16/MXFP8 throughput,
    including prefill-heavy and decode-heavy workloads, with exact model,
    checkpoint, software, and execution configurations.

These adaptations are follow-up work. The performance and accuracy results in
section 5 remain specific to the recorded DeepSeek-V4-Flash configurations;
model-specific results will accompany each adaptation.

Activity

  1. changed the title [-][RFC]: External routing and MXFP8 pass-through for synchronous A5 AFD[/-] [+][RFC]: External routing and MXFP8 pass-through for synchronous AFD on Ascend 950 (A5)[/+] on Nov 2, 2026
  2. added
    RFCRequest for comments
    AscendAscend NPU platform and related changes
    on Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

AscendAscend NPU platform and related changesRFCRequest for comments

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions