[Performance] `--enable-insert-sync` on a dynamic GEMM example generates 10% slower kernel than manual sync version

[learning-chip]

Summary

Record a practical use case where ptoas --enable-insert-sync still has ~10% room for performance improvement, compared to a known manual-sync plan.

Background

I wrote a dynamic-shape matmul that is 2x faster than original pto-isa gemm_performance example and 0.9~1.1x of aclnnMatmul in CANN 8.5.0. See matmul_swizzle/simple_demo to reproduce.

The auto-sync version is only ~100 lines of Python, and reaching 90% of manual-sync is quite decent. I just wonder if the last 10% perf gap can be filled.

Command line

ptoas --enable-insert-sync simple_matmul_auto_sync.pto -o simple_matmul_auto_sync.cpp
ptoas simple_matmul_manual_sync.pto -o simple_matmul_manual_sync.cpp

Reproduction input

pto_matmul.zip

contains both inputs:

• simple_matmul_auto_sync.pto
• simple_matmul_manual_sync.pto

and outputs:

• simple_matmul_auto_sync.cpp
• simple_matmul_manual_sync.cpp

Expected performance

Auto-sync should be ideally as fast as manual sync version. (or discover even faster pipelining?)

Actual performance

Auto-sync is 5~15% slower than manual-sync, see the detailed PRs below (contains full code with kernel launch, and on-device performance measurement on 910B2:):

• Directly written in PTO-ISA C++: PTO-ISA implementation of Matmul with L2 cache locality optimization huawei-csl/pto-kernels#26
• Python, manual-sync, equal performance as C++ version above Matmul swizzle in Python with manual sync (cleaned-up for merge) huawei-csl/pto-dsl#72
• Python, using ptoas auto-sync, 10% slower than manual sync Test auto-sync on general-shape matmul with swizzle, compare to manual sync performance. huawei-csl/pto-dsl#73

Git commit

29ed536

[fiskrt]

Here is a visualization of the timeline between using automatic sync (top), and manually syncing (bottom). Timelines are generated with msprof op simulator

[learning-chip]

Here is a visualization of the timeline between using automatic sync (top), and manually syncing (bottom). T

From this profile trace, the "L0 loops" are already quite optimal, but the manual sync has smaller bubbles across the "L1 loops", most likely due to manual prefetching.