PaperICCV 2021 · 2021

Emerging Properties in Self-Supervised Vision Transformers

Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin

arXiv 2104.14294 DOI View in graph

1page built on it

4more pages cite it

3narratives

1registry paper cites it

In the Atlas

ModelPrimary source of

DINO

Self-distillation with no labels: an EMA teacher, multi-crop training, and centering+sharpening yield ViT features whose frozen k-NN accuracy nearly matches a linear probe and whose attention maps segment objects without supervision.

Cited as a reference by 4 pages, grouped by domain

Representation learning 4

DINOv2 DINOv3 Knowledge Distillation Self-Supervised Learning

Appears in 3 narratives

NarrativeDepth Becomes General Geometry NarrativeFoundation Models for Vision NarrativeWhere Did the Inductive Bias Go?

Lineage

Papers in the Atlas registry only · one dot per paper

Built upon by · 1

Builds on · 6

2016Deep Residual Learning for Image RecognitionHe, Zhang et al.2019Momentum Contrast for Unsupervised Visual Representation LearningHe, Fan et al.2020A Simple Framework for Contrastive Learning of Visual RepresentationsChen, Kornblith et al.2020An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleDosovitskiy, Beyer et al.2020Bootstrap your own latent: A new approach to self-supervised LearningGrill, Strub et al.2020Training data-efficient image transformers & distillation through attentionTouvron, Cord et al.

Built upon by · 1

2025DINOv3Siméoni, Vo et al.