add automatic tiling with myrtle #152
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds the following changes
1. Automatic Tiling
Adds automatic tiling for all
linalg.matmul_tranpose_boperations detected as root operations of IREE dispatches. Users pass as compiler command line options--iree-quidditch-myrtle--iree-quidditch-myrtle-mode--iree-quidditch-myrtle-outwith resepctive values of the path to myrtle.y, the mode of tile selection desired, and desired output path. Alternatively, users can manually specify the tile sizes for each dispatch using the flag--iree-quidditch-import-tiles, passing it the path to a json file containing tiling schemes. If both--iree-quidditch-myrtleand--iree-quidditch-import-tilesare set, Quidditch first tiles all dispatches requested byiree-quidditch-importtiles, and then uses myrtle to tile any remaining dispatches.Relevant Repo Changes:
.gitmodulesandrequirements.txtto include a new submodule, myrtleConfigureForSnitchpass and replaces it with a new MLIR function interface pass calledConfigureTiles.ConfigureForSnitch pass, which gets populated with (dispatch name, tiling scheme) pairs, after reading json output from myrtle. SeeQuidditchTarget.cpp,TilingScheme.handTilingScheme.cppfor more details on this hash map.2. Automatic Dispatch Timing
Adds automatic insertion of timing operations at the beginning and end of select dispatches. This timing option is enabled with compiler command line argument
--iree-quidditch-time-disp. Valid values passed to the flag are either "grapeFruit" (which refers to the new GrapeFruit test case) or "fakennMxNxK" where M, N, and K are the dimensions of the new fakeNN test case'smatmul_transpose_boperation.Relevant Repo Changes:
QuidditchSnitch_Dialectcalledmyrtle_record_cycles.timeDisp, which enables or disables insertion of themyrtle_record_cyclesoperation at the beginning and end of select dispatches.3. Sensitivity Analysis
Adds scripts to automate NsNet2 Sensitivity Analysis as well new test case GrapeFruit (a copy of NsNet2)
and for future automation, the test case FakeNN (a standalone
matmul_transpose_boperation).comparing-tile-sizesto build different tiled versions of NsNet2.torch.nn.linear) wrapped in a Neural Network Pytorch Object (nn.Module). Its input and output sizes can be easily changed by modifying its CMakeLists.txt and FakeNN.h header file. In a later PR, I will add scripts to automate compilation, running, and exporting of cycles for the FakeNN test case.Relevant Repo Changes:
comparing-tile-sizeswhich contains automation scripts to compile, run and export per-dispatch cycle count for runs of NsNet2 (really using GrapeFruit test case)runtime/samples, GrapeFruit and FakeNN.