An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale | VitaVision

PaperICLR 2021 (arXiv 2020) · 2020

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai Thomas Unterthiner Mostafa Dehghani Matthias Minderer Georg Heigold Sylvain Gelly Jakob Uszkoreit Neil Houlsby

arXiv 2010.11929 DOI View in graph

1page built on it

9more pages cite it

6narratives

10registry papers cite it

In the Atlas

ModelPrimary source of

Vision Transformer — a pure-transformer image classification backbone that treats an image as a sequence of fixed-size patches: split RGB image into.

Cited as a reference by 9 pages, grouped by domain

Representation learning 6

CLIP DeiT DINOv2 MAE Positional Encoding Transformer

Depth 1

DPT (Dense Prediction Transformer)

Detection 1

Segmentation 1

Appears in 6 narratives

NarrativeDepth Becomes General Geometry NarrativeDetection, Removing the Machinery NarrativeFoundation Models for Vision NarrativeSegmentation, from Energy to Prompt NarrativeVision Under a Latency Budget NarrativeWhere Did the Inductive Bias Go?

Lineage

Papers in the Atlas registry only · one dot per paper

Built upon by · 10

Builds on · 0

Built upon by · 10

2020Training data-efficient image transformers & distillation through attentionTouvron, Cord et al.2021Emerging Properties in Self-Supervised Vision TransformersCaron, Touvron et al.2021Masked Autoencoders Are Scalable Vision LearnersHe, Chen et al.2021Swin Transformer: Hierarchical Vision Transformer using Shifted WindowsLiu, Lin et al.2021Learning Transferable Visual Models From Natural Language SupervisionRadford, Kim et al.2021Vision Transformers for Dense PredictionRanftl, Bochkovskiy, Koltun 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with TransformersXie, Wang et al.