Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention | VitaVision
PaperICML 2020 · 2020

Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention

Angelos KatharopoulosApoorv VyasNikolaos PappasFrançois Fleuret
arXiv 2006.16236
0pages built on it
3more pages cite it
2narratives
0registry papers cite it

In the Atlas

Cited as a reference by 3 pages, grouped by domain

Representation learning 2
Attention MechanismTransformer
Features 1
LoFTR

Appears in 2 narratives

NarrativeFoundation Models for VisionNarrativeThe Geometry Pipeline Collapses

Lineage

Papers in the Atlas registry only · one dot per paper
Built upon by · 0
’17’18’19’20Attention Is All You Need (2017)
Built upon by · 0