Buddy-MLIR Gemmini performance benchmarks: kernel + ResNet50 validation#11
Open
ashvin-verma wants to merge 2 commits intomainfrom
Open
Buddy-MLIR Gemmini performance benchmarks: kernel + ResNet50 validation#11ashvin-verma wants to merge 2 commits intomainfrom
ashvin-verma wants to merge 2 commits intomainfrom
Conversation
Add reproducible benchmarks comparing Buddy-MLIR's Gemmini dialect backend against the Gemmini C reference on Spike simulator. Kernel benchmarks: conv, conv+pool, MLP2 (WS/OS), MLP1, softmax matmul, iGELU matmul. ResNet50 conv1 layer validation with intentional bad case for test methodology verification. Conv benchmarks require buddy-compiler/buddy-mlir#689 (conv encoding fix) for correct im2col lowering.
Step-by-step guide from Gemmini dialect MLIR through buddy-opt, buddy-translate, buddy-llc, bare-metal linking, to Spike execution. Includes setup instructions for all prerequisites.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tiled_matmul_auto/tiled_conv_auto)Conv encoding bug fix
We found and fixed a bug in Buddy-MLIR's Gemmini conv lowering: the
im2colencoding was producing incorrect weight matrix layouts, causing checksum mismatches against the Gemmini C reference. Fix contributed upstream as buddy-compiler/buddy-mlir#689. All conv benchmarks here require this fix. Theconv1-bad-buddytest case (intentionally wrong stride) is included to verify the validation methodology catches such errors.Key Results
Matmul Workloads
Conv Workloads (post conv-encoding fix)
ResNet50 Conv1 Layer
Note on cycle counts:
rdcyclemeasures CPU instructions — Buddy's compile-time loop unrolling reduces host-side orchestration overhead, so speedups reflect less host-side work, not necessarily faster Gemmini hardware throughput.What's included
experiments/buddy-benchmarks/kernels/— 7 kernel benchmarks (.mlir + .c harnesses) with Makefileexperiments/buddy-benchmarks/resnet50/— ResNet50 conv1 validation (Buddy vs Gemmini C + intentional bad case)experiments/buddy-benchmarks/scripts/run_benchmark.sh— Single script to build and run everythingexperiments/buddy-benchmarks/logs/— Reference Spike output logsexperiments/buddy-benchmarks/README.md— Full results, methodology, reproduction instructionsexperiments/buddy-benchmarks/WORKFLOW.md— Complete lowering pipeline documentation (MLIR → buddy-opt → buddy-translate → buddy-llc → gcc link → Spike)Related
Test plan
make -C experiments/buddy-benchmarks/kernels allexperiments/buddy-benchmarks/scripts/run_benchmark.shend-to-endmake -C experiments/buddy-benchmarks/resnet50 validate