feat: add sglang_trace_analyze task for GPU profiler trace analysis - #24
Merged
myrfy001 merged 15 commits intoAug 25, 2026
Merged
Conversation
… profiler analysis Adds a new task type that profiles models across multiple batch sizes using sglang's bench_one_batch_server with torch profiler, then analyzes the traces for kernel hotspots, TFLOPS/MFU, operator-to-model-structure mapping, fuse opportunities, and LLM-powered optimization hints. 5-phase linear pipeline: MAPPING -> BENCHMARK -> ANALYZE -> HINTS -> SUMMARIZE. Design doc: docs/sglang_trace_analyze-design.md (grilled by architecture review). Co-Authored-By: deepseek-v4-pro[1m] <deepseek-ai@claude-code-best.win>
…table schema - Wire structure_mapper and flops_calculator into pipeline ANALYZE phase - Add all 17 design fields to kernel_table.json (model_layer, tflops_actual, mfu, bound, bandwidth_gb_s, input_dims, confidence) - Fix classifier priority for HIP/CK kernel names (CK-GEMM, CustomAllReduce, MLA, MoE, ElementWise) - Add CPU-op-based model layer inference fallback (no call stacks in trace) - Extract CK GEMM tile dimensions (MT<N>x<N>x<N>) for FLOPs estimation - Add 3-tab frontend: Summary Overview / Batch Detail / Optimization Hints - Register detail_view_module + extra_stylesheets in WebPlugin - Fix trace_parser to accept str paths (not just Path) End-to-end verified: DeepSeek V4 INT8 TP8 BS=8 decode analysis on K100. Co-Authored-By: deepseek-v4-pro[1m] <deepseek-ai@claude-code-best.win>
Add --reasoning-parser deepseek-v4 and --tool-call-parser deepseekv4 to match upstream /workspace/sglang/scripts/run_traces.py params. Without these, sglang may use incorrect model config parser on startup. Verified: upstream run_traces.sh also SIGSEGVs with CUDA Graph ON on K100 — this is a sglang fork bug, not a parameter mismatch. Co-Authored-By: deepseek-v4-pro[1m] <deepseek-ai@claude-code-best.win>
- Real _build_mapping: parse trace with CPU op correlation, call structure_mapper to classify all kernel→layer mappings - Configurable profile steps in bench_config.json (not hardcoded 500/50) - BENCHMARK failure is non-fatal: ANALYZE falls back to mapping traces - ANALYZE auto-discovers traces in sglang timestamp subdirectories - run_benchmark.py uses configurable profile_start_step/profile_steps Verified: full 5-phase pipeline (MAPPING→BENCHMARK→ANALYZE→HINTS→ SUMMARIZE) completes with final_status=success even when formal benchmark SIGSEGVs (sglang K100 fork CUDA Graph bug). Co-Authored-By: deepseek-v4-pro[1m] <deepseek-ai@claude-code-best.win>
…ph state Always appended "graph" regardless of --disable-cuda-graph. Now uses "nograph" for mapping runs, matching upstream run_traces.py behavior. Co-Authored-By: deepseek-v4-pro[1m] <deepseek-ai@claude-code-best.win>
vllm::cross_device_reduce_2stage_pcie is the TP allreduce kernel on K100 HIP. Previously classified as Other (63.8% of GPU time with CUDA Graph ON), now correctly classified as Reduce. Verified: CUDA Graph bug fixed upstream, formal run produces traces. GPU time drops from 7.09s (no graph) to 0.57s (graph ON, 12.4x), throughput 5.63 → 19.71 tok/s (3.5x). Bottleneck shifts from Reduce (71.9%) to GEMM (46.5%). Co-Authored-By: deepseek-v4-pro[1m] <deepseek-ai@claude-code-best.win>
Previously required caller to source HIP/K100 environment variables externally. Now _setup_env() applies them at startup, matching /workspace/sglang/scripts/run_traces.sh exactly. Key fixes: - SGLANG_OPT_USE_HIP_INT8_SCALED_MM: true → 0 - Added SGLANG_OPT_USE_LMSLIM_INT8_QUANT=1 - Added SGLANG_OPT_USE_W8A8_MARLIN_GEMM=1 Co-Authored-By: deepseek-v4-pro[1m] <deepseek-ai@claude-code-best.win>
…hart, search Replace single-page layout with 3-tab dashboard: - Dashboard: stat cards (GPU time, bottleneck %, MFU, CUDA Graph), CSS donut chart for category breakdown, bottleneck detail card, compute/memory bound visualization, overlap status, top kernels preview - Kernel Table: search bar + category filter, all 11 columns with confidence badges, sortable and filterable - Hints: bottleneck analysis with auto-generated suggestions, fuse pattern matches, AI optimization hints Co-Authored-By: deepseek-v4-pro[1m] <deepseek-ai@claude-code-best.win>
…e tiering Kernel names that unambiguously identify op type (CK GEMM, flash_attn, fused_moe, NCCL, w8a8, cross_device_reduce, topk radix/gather) now get "high" confidence without requiring call stacks. Result: 82.4% of GPU time covered by high-confidence mappings. Low confidence restricted to generic elementwise/memory kernels (17.6%). Co-Authored-By: deepseek-v4-pro[1m] <deepseek-ai@claude-code-best.win>
… fuse panels Dashboard now includes six analysis sections: - TFLOPS & Bandwidth table: actual vs theoretical peak per kernel - Model Structure → Operator Mapping: layer↔kernel groupings with confidence distribution, showing which model layers produce which GPU operators - Fuse Opportunities: rule-based pattern matches with estimated savings - Inefficiency Radar: kernels with high time + low MFU ranked by waste - Roofline Analysis: ops/byte vs ridge point visualization - Category donut chart + Compute/Memory bound + Bottleneck detail Mapping data fetched via /mapping API, confidence stats shown inline. Co-Authored-By: deepseek-v4-pro[1m] <deepseek-ai@claude-code-best.win>
…om filename - _find_trace_dir: search formal traces (bs_N/timestamp/) before mapping fallback, so ANALYZE uses CUDA Graph ON traces when available - Detect CUDA Graph from trace filename (_graph_ vs _nograph_) instead of relying on gap count heuristic in overlap detector Result: Dashboard shows CUDA Graph: ON (green) when formal traces are used, OFF (red) only for mapping-only runs. Co-Authored-By: deepseek-v4-pro[1m] <deepseek-ai@claude-code-best.win>
…ncy analysis Dashboard additions: - MFU Distribution: histogram across 7 buckets (0-5%, 5-10%, ..., 90-100%) with avg/median stats, showing how efficiently the GPU is used - Top by Invocation Count: kernels ranked by call frequency, helping identify "death by a thousand cuts" patterns where many small invocations could be batched Co-Authored-By: deepseek-v4-pro[1m] <deepseek-ai@claude-code-best.win>
…ing trace - _merge_mapping_tflops: cross-reference formal trace kernel table with mapping trace (CUDA Graph OFF) to fill in tflops_actual, mfu, bound, bandwidth_gb_s per kernel by name matching - Fix flops_calculator to preserve small TFLOPS values from CK GEMM tiles - _is_formal_trace: detect CUDA Graph status from trace filename Co-Authored-By: deepseek-v4-pro[1m] <deepseek-ai@claude-code-best.win>
Auto-generates 5 insight cards from analysis data: - Dominant kernel alert (single kernel >30% GPU time) - CUDA Graph status assessment - Category concentration warning (>50% in one category) - Top-3 kernels summary with category + time_pct - MFU data availability note with actionable next step Cards use icon + color coding (red/yellow/green) for quick scanning. Co-Authored-By: deepseek-v4-pro[1m] <deepseek-ai@claude-code-best.win>
- Replace LLM stub with rule-based hint generation from kernel table, overlap, and fuse data. Generates 2-5 suggestions with difficulty rating, estimated saving %, and category. - Add executive summary banner at top of Dashboard: one-line summary of CUDA Graph status, bottleneck, and top optimization opportunities. - Fix hint collection to use full kernel list (not just top-3) for accurate category aggregation. Co-Authored-By: deepseek-v4-pro[1m] <deepseek-ai@claude-code-best.win>
Foreverhighness
force-pushed
the
feat/sglang-trace-analyze
branch
from
August 6, 2026 02:30
f44bc64 to
ecf3d4c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
新增 sglang_trace_analyze 任务类型，使用 sglang torch profiler 对模型进行多 batch size profiling，自动分析 trace 文件，并通过 WebUI Dashboard 展示瓶颈定位、性能分析和优化建议。
Test plan