Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
7041366
优化汇总表说明
chopper0126 Apr 20, 2026
6de386f
add level430
chopper0126 Apr 20, 2026
853be8f
Merge pull request #103 from chopper0126/br_ascend_optimization
Just-it Apr 21, 2026
18c7726
add level30 allcase json
chopper0126 Apr 21, 2026
c9ce8c5
add relubackward and fix realdiv
chopper0126 Apr 22, 2026
f137f93
Merge pull request #110 from chopper0126/br_ascend_optimization
Just-it Apr 22, 2026
6b42fa5
[triton] 添加report功能,新增430格式的benchmark评测
ElleElleWu Apr 22, 2026
0f7be33
add per-case speedup
chopper0126 Apr 22, 2026
fb0c2f2
用profile方式验证性能,并保存每个case的加速比
chopper0126 Apr 22, 2026
e396a91
[triton] Phase 4 结果判定以主线最新为准
ElleElleWu Apr 22, 2026
a18a6a1
[triton] 多shape benchmark输出报告和评测方式修正
ElleElleWu Apr 22, 2026
04993f3
修改精度验证方式
chopper0126 Apr 22, 2026
71e4510
[triton] op-task-extractor适配多shape benchmark
ElleElleWu Apr 22, 2026
7bdd379
Merge pull request #113 from ElleElleWu/br_430_2
Just-it Apr 22, 2026
f4d890d
fix elu_grad's model.py
sdshao Apr 22, 2026
8aabab2
Merge pull request #115 from sdshao/br_430
Just-it Apr 23, 2026
e9becba
优化utils/verification_ascendc.py和performance.py
chopper0126 Apr 23, 2026
261f55f
保存每个case加速比
chopper0126 Apr 23, 2026
0197c71
fix realdiv's reference code from tf to torch
sdshao Apr 23, 2026
67cbbed
Merge pull request #118 from sdshao/br_430
Just-it Apr 23, 2026
0c89a5c
Merge remote-tracking branch 'origin/main' into br_430
Just-it Apr 23, 2026
4b42d46
feat:精度测试参考AscendCOPBenchMark修改。
201111007-max Apr 23, 2026
20fcf0f
feat:精度测试参考AscendCOPBenchMark修改。
201111007-max Apr 23, 2026
ce2d203
[benchmark] 修复17_logaddexp用例类型问题
ElleElleWu Apr 24, 2026
40715aa
添加Inf 处理
chopper0126 Apr 24, 2026
3a54f30
[triton] 基线验证和性能评测改为复用基线结果
ElleElleWu Apr 24, 2026
9bf0cb8
Merge pull request #116 from chopper0126/br_ascend_optimization
Just-it Apr 24, 2026
f02dc54
[triton] 新增约束,性能评测之前必须全量shape精度验证通过
ElleElleWu Apr 24, 2026
3269aa1
Merge pull request #122 from ElleElleWu/br_430_2
Just-it Apr 24, 2026
146ccb3
Merge pull request #123 from 201111007-max/br_430
Just-it Apr 25, 2026
ac247af
Merge pull request #124 from ElleElleWu/br_430_workflow
Just-it Apr 25, 2026
8df1790
[triton] benchmark评测方式变更,由延时求和计算加速比更改为几何平均计算加速比
ElleElleWu Apr 25, 2026
56b7d99
Merge pull request #129 from ElleElleWu/br_430_geometric
Just-it Apr 25, 2026
f24df03
Merge pull request #131 from Just-it/br_debug
Just-it Apr 25, 2026
7e05a78
add temp
chopper0126 Apr 25, 2026
f50c4cb
修复提前退出
chopper0126 Apr 27, 2026
1f3c8b9
Merge pull request #132 from chopper0126/br_ascend_optimization
Just-it Apr 27, 2026
0267062
fix logaddexp op
jqliu42 Apr 27, 2026
bbc2b8f
Merge pull request #134 from jqliu42/br_fix_logaddexp
Just-it Apr 27, 2026
519efe5
更新部分uint8用例,避免出现负值入参
JMakit Apr 27, 2026
e4eba40
Merge pull request #135 from JMakit/br_430
Just-it Apr 27, 2026
6a9b1f2
kv rsnorm rope cache benchmark fix
justice-dance Apr 28, 2026
3031889
add kv ref task
justice-dance Apr 29, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.en.md
Original file line number Diff line number Diff line change
Expand Up @@ -536,7 +536,7 @@ def get_init_inputs():
| `framework.peak_memory_mb` | `float` | PyTorch peak memory (MB), average across all Shapes |
| `implementation.avg_latency_ms` | `float` | Implementation average latency (ms), average across all Shapes |
| `implementation.peak_memory_mb` | `float` | Implementation peak memory (MB), average across all Shapes |
| `speedup_vs_torch` | `float` | Speedup over PyTorch (average of all Shape speedups) |
| `speedup_vs_torch` | `float\|null` | Geometric mean speedup `(∏ s_i)^(1/n)` over PyTorch (across pass shapes with finite positive `s_i`); `null` when all shapes are abnormal |
| `perf_method` | `str` | Profiling method: "profiler" (torch_npu.profiler) or "fallback" (time.perf_counter) |
| `skill_path` | `str` | Path to the benchmark skill used |
| `per_shape_results` | `List[Dict]` | Multi-Shape details (present when `total_cases > 1`) |
Expand Down
120 changes: 72 additions & 48 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -451,17 +451,36 @@ def get_init_inputs():
"warmup": 5,
"repeats": 50,
"total_cases": 1,
"passed_cases": 1,
"failed_cases": 0,
"nan_indices": [],
"inf_indices": [],
"zero_indices": [],
"negative_indices": [],
"none_indices": [],
"framework": {
"avg_latency_ms": 0.2345,
"peak_memory_mb": 2.50
"peak_memory_mb": 2.50,
"operators": {}
},
"implementation": {
"avg_latency_ms": 0.1567,
"peak_memory_mb": 1.25
"peak_memory_mb": 1.25,
"operators": {}
},
"speedup_vs_torch": 1.5000,
"perf_method": "profiler",
"skill_path": "/path/to/.claude/skills/kernel-verifier"
"speedup_vs_torch": 1.4965,
"per_shape_results": [
{
"case_idx": 1,
"input_desc": [{"type":"tensor","shape":[1024,1024],"dtype":"torch.float16"}],
"status": "pass",
"framework": {"avg_latency_ms": 0.2345, "peak_memory_mb": 2.50},
"implementation": {"avg_latency_ms": 0.1567, "peak_memory_mb": 1.25},
"speedup_vs_torch": 1.4965,
"error_type": null,
"error_msg": null
}
]
}
```

Expand All @@ -473,53 +492,54 @@ def get_init_inputs():
"warmup": 5,
"repeats": 50,
"total_cases": 3,
"passed_cases": 3,
"failed_cases": 0,
"nan_indices": [],
"inf_indices": [],
"zero_indices": [],
"negative_indices": [],
"none_indices": [],
"framework": {
"avg_latency_ms": 0.4567,
"peak_memory_mb": 8.50
"peak_memory_mb": 8.50,
"operators": {}
},
"implementation": {
"avg_latency_ms": 0.3123,
"peak_memory_mb": 4.25
"peak_memory_mb": 4.25,
"operators": {}
},
"speedup_vs_torch": 1.4600,
"perf_method": "profiler",
"skill_path": "/path/to/.claude/skills/kernel-verifier",
"speedup_vs_torch": 1.4910,
"per_shape_results": [
{
"shape": [128, 128],
"framework": {
"avg_latency_ms": 0.0234,
"peak_memory_mb": 0.50
},
"implementation": {
"avg_latency_ms": 0.0156,
"peak_memory_mb": 0.25
},
"speedup_vs_torch": 1.5000
"case_idx": 1,
"input_desc": [{"type":"tensor","shape":[128,128],"dtype":"torch.float16"}],
"status": "pass",
"framework": {"avg_latency_ms": 0.0234, "peak_memory_mb": 0.50},
"implementation": {"avg_latency_ms": 0.0156, "peak_memory_mb": 0.25},
"speedup_vs_torch": 1.5000,
"error_type": null,
"error_msg": null
},
{
"shape": [256, 256],
"framework": {
"avg_latency_ms": 0.0891,
"peak_memory_mb": 2.00
},
"implementation": {
"avg_latency_ms": 0.0588,
"peak_memory_mb": 1.00
},
"speedup_vs_torch": 1.5200
"case_idx": 2,
"input_desc": [{"type":"tensor","shape":[256,256],"dtype":"torch.float16"}],
"status": "pass",
"framework": {"avg_latency_ms": 0.0891, "peak_memory_mb": 2.00},
"implementation": {"avg_latency_ms": 0.0588, "peak_memory_mb": 1.00},
"speedup_vs_torch": 1.5153,
"error_type": null,
"error_msg": null
},
{
"shape": [1024, 1024],
"framework": {
"avg_latency_ms": 1.2577,
"peak_memory_mb": 8.00
},
"implementation": {
"avg_latency_ms": 0.8625,
"peak_memory_mb": 12.50
},
"speedup_vs_torch": 1.4600
"case_idx": 3,
"input_desc": [{"type":"tensor","shape":[1024,1024],"dtype":"torch.float16"}],
"status": "pass",
"framework": {"avg_latency_ms": 1.2577, "peak_memory_mb": 8.00},
"implementation": {"avg_latency_ms": 0.8625, "peak_memory_mb": 12.50},
"speedup_vs_torch": 1.4582,
"error_type": null,
"error_msg": null
}
]
}
Expand All @@ -533,23 +553,27 @@ def get_init_inputs():
| `warmup` | `int` | 预热次数 |
| `repeats` | `int` | 正式测试次数 |
| `total_cases` | `int` | 测试的 Shape 数量(单 Shape 为 1,多 Shape ≥2) |
| `framework.avg_latency_ms` | `float` | PyTorch 实现平均延迟(毫秒)各 Shape 平均 |
| `passed_cases` / `failed_cases` | `int` | 多 Shape 通过 / 失败用例数(异常 `s_i` 的 shape 仍计入 `passed_cases`)|
| `nan_indices` / `inf_indices` / `zero_indices` / `negative_indices` / `none_indices` | `List[int]` | 各类异常 `s_i` 的 case_idx 列表(从 1 开始,不进入几何平均);无异常时为 `[]` |
| `framework.avg_latency_ms` | `float` | PyTorch 实现平均延迟(毫秒),各 Shape 算术平均(兼容语义)|
| `framework.peak_memory_mb` | `float` | PyTorch 峰值内存(MB)各 Shape 平均 |
| `implementation.avg_latency_ms` | `float` | 实现平均延迟(毫秒)各 Shape 平均 |
| `implementation.avg_latency_ms` | `float` | 实现平均延迟(毫秒)各 Shape 算术平均(兼容语义)|
| `implementation.peak_memory_mb` | `float` | 实现峰值内存(MB)各 Shape 平均 |
| `speedup_vs_torch` | `float` | 相比 PyTorch 的加速比(各 Shape 加速比的平均值) |
| `speedup_vs_torch` | `float\|null` | **几何平均加速比** = `(∏ s_i)^(1/n)`,仅对 status==pass 且 `s_i` 为有限正数的 Shape;全部异常时为 `null` |
| `perf_method` | `str` | 评测方式:"profiler"(torch_npu.profiler)或 "fallback"(time.perf_counter 兜底) |
| `skill_path` | `str` | 使用的 benchmark skill 路径 |
| `per_shape_results` | `List[Dict]` | Shape 明细数据(当 `total_cases > 1` 时出现) |
| `per_shape_results` | `List[Dict]` | Shape 明细数据(永远存在,含失败用例)|

**per_shape_results 元素说明**:

| 字段 | 类型 | 说明 |
|------|------|------|
| `shape` | `List[int]` | 主要输入张量的形状 |
| `framework.avg_latency_ms` | `float` | 该 Shape 的 PyTorch 延迟 |
| `implementation.avg_latency_ms` | `float` | 该 Shape 的实现延迟 |
| `speedup_vs_torch` | `float` | 该 Shape 的加速比 |
| `case_idx` | `int` | 用例序号(从 1 开始)|
| `input_desc` | `List[Dict]` | 输入结构化描述(tensor: shape+dtype;scalar: value)|
| `status` | `str` | `"pass"` 或 `"fail"` |
| `framework` / `implementation` | `Dict\|null` | pass 时含 `avg_latency_ms`、`peak_memory_mb`;fail 时为 null |
| `speedup_vs_torch` | `float\|null` | 该 Shape 的加速比;fail 或 `s_i` 异常(NaN/Inf/0/负数/None)时为 null |
| `error_type` / `error_msg` | `str\|null` | fail 时记录异常类型与堆栈(截断 2000 字符)|

### 适用场景

Expand Down
2 changes: 1 addition & 1 deletion agents/ascend-kernel-developer.md
Original file line number Diff line number Diff line change
Expand Up @@ -437,7 +437,7 @@ while ac_iteration < max_ac_iterations:
2. **执行性能测试**:默认测试 `reference` 和 `ascendc`,使用 `@references/performance.py` 进行对比测试;只有用户明确要求时才额外纳入 `tilelang`
3. **获取性能报告**:记录各实现的耗时和加速比

**产出**:性能分析报告(markdown 格式,包含在 trace 中或直接输出)
**产出**:性能分析报告(markdown 格式,包含在 trace 中或直接输出),`preformance.json`,用于记录每个case的加速比

---

Expand Down
Loading