Skip to content

Latest commit

 

History

5 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 

Repository files navigation

cross-attn-isolation-proof

A PyTorch experiment demonstrating the core lookup capability of the Transformer architecture. By isolating Cross-Attention, this project generates a clean heatmap visualization that shows a perfect, targeted information retrieval from the Encoder's knowledge base.

About

A PyTorch experiment demonstrating the core lookup capability of the Transformer architecture. By isolating Cross-Attention, this project generates a clean heatmap visualization that shows a perfect, targeted information retrieval from the Encoder's knowledge base.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages