Skip to content

Add core operator documentation guides and update README#2002

Open
sunway513 wants to merge 1 commit intoROCm:mainfrom
sunway513:docs/core-operator-guides
Open

Add core operator documentation guides and update README#2002
sunway513 wants to merge 1 commit intoROCm:mainfrom
sunway513:docs/core-operator-guides

Conversation

@sunway513
Copy link
Collaborator

Summary

  • Add comprehensive documentation guides for 10 core AITER operators: Attention (MHA/PA), MLA, Fused MOE, GEMM, Quantization, Normalization, RoPE, KV-Cache, Elementwise/Activations, and Sampling
  • Each guide covers available variants, backend support (ASM / CK / Triton), Python API examples, and performance tuning advice
  • Update README with improved installation instructions (JIT dev mode, precompiled kernels, build variables) and a new Supported Operators table linking to each guide

Files

Guide Operator
docs/attention_variants_guide.md Multi-Head Attention, Paged Attention, Unified Attention, GQA/MQA
docs/mla_kernel_support_report.md Multi-head Latent Attention — Triton vs ASM backends
docs/moe_variants_guide.md Fused MOE — A8W8, A16W8, FP8 block-scale, MXFP4, 2-stage
docs/gemm_variants_guide.md GEMM — A8W8, A16W16, A4W4, batched, DeepGEMM, Triton FFN
docs/quantization_guide.md Quantization — FP8/MXFP4/INT4, per-tensor/token/block, SmoothQuant
docs/normalization_guide.md RMSNorm, LayerNorm, GroupNorm — fused add/quant variants
docs/rope_guide.md Rotary Position Embedding — SBHD/THD/2D/3D, NeoX & GPT-J
docs/kv_cache_guide.md KV-Cache — paged/flash/MLA layouts, FP8/INT8, fused RoPE
docs/elementwise_activation_guide.md Activations & elementwise — SiLU/GELU/SwiGLU, fused quant
docs/sampling_guide.md Token sampling — greedy, random, top-k, top-p
README.md Updated installation + Supported Operators table

Guide Structure

Each guide follows a consistent format:

  1. Quick Reference table for fast lookup
  2. Numbered sections covering variants, API, backends, data types
  3. Decision tree for variant selection
  4. Source files and test files tables for code navigation

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds operator-focused documentation guides for core AITER components and refreshes the main README to improve installation guidance and provide a linked “Supported Operators” index.

Changes:

  • Added multiple operator guides under docs/ covering API usage, backend support, and tuning notes.
  • Added a consolidated “Supported Operators” table to the README linking to the new guides.
  • Expanded README installation instructions (JIT dev mode, prebuild options, environment variables, Iris comms).

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
docs/attention_variants_guide.md New guide documenting attention variants and backend selection.
docs/mla_kernel_support_report.md New MLA variants/support report and usage guidance.
docs/moe_variants_guide.md New MOE variants guide including routing/quantization/backend notes.
docs/gemm_variants_guide.md New GEMM variants/tuning guide including configs and backends.
docs/quantization_guide.md New quantization guide (FP8/MXFP4/INT4, fused ops, KV cache).
docs/normalization_guide.md New normalization guide (RMSNorm/LayerNorm/GroupNorm + fusions).
docs/rope_guide.md New RoPE guide (formats, backends, env selection, fused paths).
docs/kv_cache_guide.md New KV-cache management guide (layouts, quantization, fused cache ops).
docs/elementwise_activation_guide.md New elementwise/activation guide including fused quantization paths.
docs/sampling_guide.md New sampling guide (greedy/random/mixed, top-k/top-p ops).
README.md Updated installation instructions and added supported-operators index.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@sunway513 sunway513 force-pushed the docs/core-operator-guides branch 2 times, most recently from f36aac8 to c28edc3 Compare February 8, 2026 02:12
Add comprehensive documentation guides for 10 core AITER operators:
Attention (MHA/PA), MLA, Fused MOE, GEMM, Quantization, Normalization,
RoPE, KV-Cache, Elementwise/Activations, and Sampling.

Each guide covers available variants, backend support (ASM/CK/Triton),
Python API examples, and performance tuning advice.

Update README with improved installation instructions (JIT dev mode,
precompiled kernels, build variables) and a new Supported Operators
table linking to each guide.
@sunway513 sunway513 force-pushed the docs/core-operator-guides branch from c28edc3 to 50f7727 Compare February 8, 2026 02:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant