Attention, Transformers

[seanshpark]

Where to look at...

[seanshpark]

https://machinelearningmastery.com/what-is-attention/
https://www.frontiersin.org/articles/10.3389/fncom.2020.00029/full
https://towardsdatascience.com/transformer-neural-network-step-by-step-breakdown-of-the-beast-b3e096dc857f

https://towardsdatascience.com/transformers-explained-visually-part-1-overview-of-functionality-95a6dd460452
https://towardsdatascience.com/transformers-explained-visually-part-2-how-it-works-step-by-step-b49fa4a64f34
https://towardsdatascience.com/transformers-explained-visually-part-3-multi-head-attention-deep-dive-1c1ff1024853

[seanshpark]

LSTM

• https://machinelearningmastery.com/gentle-introduction-long-short-term-memory-networks-experts/

[seanshpark]

Vision Transformers

MobileViT: https://arxiv.org/abs/2110.02178

• https://keras.io/examples/vision/mobilevit/

[seanshpark]

Papers

• https://arxiv.org/pdf/1409.0473.pdf
• https://arxiv.org/pdf/1508.04025.pdf