Skip to content

Disabled work remap on low sequence length - #2

Merged
tarik-sarac merged 1 commit into
mainfrom
tarik/remap-threshold-256
Mar 2, 2026
Merged

Disabled work remap on low sequence length#2
tarik-sarac merged 1 commit into
mainfrom
tarik/remap-threshold-256

Conversation

@tarik-sarac

Copy link
Copy Markdown
Collaborator

Disabled work remap on short sequence lengths since it doesn't have any additional advantage on lower sequences and additional overhead of work remap slows down short sequences further.

  • Added a new helper function bwd_sorted_remap_min_col_tiles() in flash_api.cpp that reads the minimum column tiles threshold from the BACK_LITE_BWD_SORT_REMAP_MIN_COL_TILES environment variable, enabling runtime configuration of when sorted remapping is applied.
  • Updated the backward pass logic to conditionally enable sorted column remapping only when params.num_col_tiles meets or exceeds the configured threshold; otherwise, disables remapping by setting params.work_remap to nullptr.

@tarik-sarac
tarik-sarac merged commit 4d70ad5 into main Mar 2, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant