fix: correct benchmark paths and enhance notebook analysis#16
Merged
Conversation
**Benchmark Script Fixes:** - Fixed benchmark test path to target ./tests/benchmark directory - Previously was running against main package with no benchmarks - Now properly executes 30+ benchmarks from test suite - Both full suite and profiled benchmarks now work correctly **Jupyter Notebook Enhancements:** - Added SIMD comparison file parsing (simd_comparison.txt) - Updated to support dual-profile CPU profiling structure - Main benchmarks: cpu_profile_main.prof - SIMD benchmarks: cpu_profile_simd.prof - Enhanced cell-5: Added file validation for all profile types - Enhanced cell-11: Parse separate SIMD comparison file - Enhanced cell-25: Dual-profile analysis with distinct visualizations - Main benchmarks: Blue color scheme (steelblue/coral) - SIMD benchmarks: Green color scheme (mediumseagreen/lightcoral) - Removed all problematic UTF-8 encoding and emoji characters - Replaced symbols with simple text markers: [OK], [ERROR], [SKIP], [WARN] **Results:** - Main CPU profile: 38KB (vs 840 bytes before) - SIMD CPU profile: 8.4KB - SIMD analysis now properly displays speedup metrics - All encoding issues resolved
|
📋 Version Check: This PR contains changes. Consider creating a new version tag after merging. Semantic Versioning Guide:
To create a release after merging: git tag v0.1.0
git push origin v0.1.0 |
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR updates the benchmark infrastructure to target the new ./tests/benchmark directory structure instead of running benchmarks from the root package. Additionally, it modernizes status indicators in the Jupyter notebook analyzer by replacing Unicode checkmark/error symbols (✓/✗) with ASCII alternatives ([OK], [ERROR], [SKIP], [WARN]) and refactors the notebook to handle dual-profile CPU profiling (separate main and SIMD profiles).
Key changes:
- Updated benchmark script paths to
./tests/benchmarkfrom root package - Replaced Unicode symbols with ASCII status indicators in the Jupyter notebook
- Enhanced SIMD comparison analysis with separate file handling and improved error reporting
- Restructured CPU profile analysis to handle dual profiles (main and SIMD separately)
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| scripts/benchmark.sh | Updated go test commands to target ./tests/benchmark directory |
| results/benchmark_analyzer.ipynb | Replaced Unicode symbols with ASCII indicators and refactored for dual-profile structure with separate SIMD comparison file handling |
Comments suppressed due to low confidence (23)
results/benchmark_analyzer.ipynb:1
- The error message contains a redundant 'ERROR' - it appears both in the prefix and in the message text. Should be either
[ERROR] Benchmark file not found:or keep the prefix and remove 'ERROR:' from the message body.
{
results/benchmark_analyzer.ipynb:1
- Extra space after '[WARN]' - should be consistent with other status indicators which use single space.
{
results/benchmark_analyzer.ipynb:1
- Missing status indicator prefix. This line had a checkmark removed but no replacement added. Should likely be
[OK]or[INFO]to be consistent with other status messages.
{
results/benchmark_analyzer.ipynb:1
- Missing status indicator prefix. Should likely be
[OK]or similar to match the pattern used elsewhere in the file.
{
results/benchmark_analyzer.ipynb:1
- Inconsistent status indicator usage across multiple graph save messages. Some have leading space, others don't, and none have status indicators like
[OK]. These should be consistent with the new ASCII indicator pattern.
{
results/benchmark_analyzer.ipynb:1
- Inconsistent status indicator usage across multiple graph save messages. Some have leading space, others don't, and none have status indicators like
[OK]. These should be consistent with the new ASCII indicator pattern.
{
results/benchmark_analyzer.ipynb:1
- Inconsistent status indicator usage across multiple graph save messages. Some have leading space, others don't, and none have status indicators like
[OK]. These should be consistent with the new ASCII indicator pattern.
{
results/benchmark_analyzer.ipynb:1
- Inconsistent status indicator usage across multiple graph save messages. Some have leading space, others don't, and none have status indicators like
[OK]. These should be consistent with the new ASCII indicator pattern.
{
results/benchmark_analyzer.ipynb:1
- Inconsistent status indicator usage across multiple graph save messages. Some have leading space, others don't, and none have status indicators like
[OK]. These should be consistent with the new ASCII indicator pattern.
{
results/benchmark_analyzer.ipynb:1
- Inconsistent status indicator usage across multiple graph save messages. Some have leading space, others don't, and none have status indicators like
[OK]. These should be consistent with the new ASCII indicator pattern.
{
results/benchmark_analyzer.ipynb:1
- Inconsistent status indicator usage across multiple graph save messages. Some have leading space, others don't, and none have status indicators like
[OK]. These should be consistent with the new ASCII indicator pattern.
{
results/benchmark_analyzer.ipynb:1
- Multiple export success messages missing status indicators. These should be prefixed with
[OK]to be consistent with the new ASCII indicator pattern used elsewhere in the file.
{
results/benchmark_analyzer.ipynb:1
- Multiple export success messages missing status indicators. These should be prefixed with
[OK]to be consistent with the new ASCII indicator pattern used elsewhere in the file.
{
results/benchmark_analyzer.ipynb:1
- Multiple export success messages missing status indicators. These should be prefixed with
[OK]to be consistent with the new ASCII indicator pattern used elsewhere in the file.
{
results/benchmark_analyzer.ipynb:1
- Multiple export success messages missing status indicators. These should be prefixed with
[OK]to be consistent with the new ASCII indicator pattern used elsewhere in the file.
{
results/benchmark_analyzer.ipynb:1
- Multiple export success messages missing status indicators. These should be prefixed with
[OK]to be consistent with the new ASCII indicator pattern used elsewhere in the file.
{
results/benchmark_analyzer.ipynb:1
- Multiple export success messages missing status indicators. These should be prefixed with
[OK]to be consistent with the new ASCII indicator pattern used elsewhere in the file.
{
results/benchmark_analyzer.ipynb:1
- Multiple export success messages missing status indicators. These should be prefixed with
[OK]to be consistent with the new ASCII indicator pattern used elsewhere in the file.
{
results/benchmark_analyzer.ipynb:1
- Multiple export success messages missing status indicators. These should be prefixed with
[OK]to be consistent with the new ASCII indicator pattern used elsewhere in the file.
{
results/benchmark_analyzer.ipynb:1
- Profile analysis messages missing status indicators. Should be prefixed with
[OK]for consistency with the ASCII indicator pattern.
{
results/benchmark_analyzer.ipynb:1
- Profile analysis messages missing status indicators. Should be prefixed with
[OK]for consistency with the ASCII indicator pattern.
{
results/benchmark_analyzer.ipynb:1
- Profile analysis messages missing status indicators. Should be prefixed with
[OK]for consistency with the ASCII indicator pattern.
{
results/benchmark_analyzer.ipynb:1
- Profile analysis messages missing status indicators. Should be prefixed with
[OK]for consistency with the ASCII indicator pattern.
{
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Benchmark Script Fixes:
Jupyter Notebook Enhancements:
Results: