Tropical algebra for attention mechanisms — replaces softmax with max-plus algebra for provably sparse, interpretable attention patterns.
- Tropical softmax — max-plus semiring alternative to classical softmax
- Sparse attention — provable sparsity guarantees from tropical geometry
- Interpretable weights — tropical attention maps have clear geometric meaning
- Pure Python — implementations of tropical matrix operations
Part of the SuperInstance mathematical research into alternative attention mechanisms. Connects to info-geo (information geometry) and pareto-tournament (Pareto optimization).
MIT