The Ambiguity That Never Left
A single image fixes scene geometry only up to degrees of freedom no photometric analysis can resolve. Monocular Depth Estimation states the problem precisely: infinitely many world-scale configurations produce identical pixel values, so no model recovers absolute distance from a photograph alone without camera intrinsics, a physical scale reference, or metric supervision during fine-tuning. Practical systems split into two regimes: relative depth, defined only up to an unknown per-image scale and shift, and metric depth, which carries absolute scale and needs information the image itself does not contain.
MiDaS commits to the relative regime and turns the concession into a training strategy. It mixes five mutually incompatible datasets, LiDAR-derived metric depth, unknown-baseline stereo disparity, and scale-only structure-from-motion depth, under a scale-and-shift-invariant loss that aligns each prediction to its own ground truth by least squares, then trims the twenty percent largest residuals so annotation outliers never enter back-propagation. A Pareto-optimal criterion governs the mixing, and the resulting model transfers zero-shot to datasets held out entirely from training.
What the loss does not touch is the network carrying it: an encoder pretrained on web images for classification, not depth, and simply the best one available in 2019. Scaling the loss further is one problem; scaling the features underneath it is another, and that is where the story turns next.
Borrowed Features
DPT changes the encoder first and nothing else. It keeps the MiDaS protocol, the scale-and-shift-invariant trimmed loss and the dataset mixing, and replaces the convolutional encoder with a Vision Transformer whose tokens a Reassemble step turns back into image-like feature maps at four resolutions for a convolutional decoder. Under that identical training signal, zero-shot transfer improves on MiDaS by more than 23 percent for DPT-Hybrid and 28 percent for DPT-Large. The encoder is still pretrained for image classification; what later work keeps from DPT is the decoder, not the backbone.
DINOv2 was built to solve a different problem than depth: general-purpose visual representation. It trains a Vision Transformer through self-supervised distillation on a corpus assembled automatically, with no human annotation anywhere in the pipeline, selecting 142 million images for diversity and domain coverage. The resulting backbone is used frozen at evaluation time, with no finetuning and no task-specific pretraining. Its features match or exceed text-supervised models on classification and retrieval, and decisively outperform them on dense tasks such as semantic segmentation and monocular depth.
That last property turns DINOv2 into shared infrastructure rather than a one-off encoder. Depth Anything initialises its encoder directly from DINOv2 checkpoints. Depth Anything V2's synthetic-only teacher is itself built on the same backbone at its largest scale. Depth Anything 3 and VGGT both patchify their input through a frozen DINOv2 tokenizer rather than training one from scratch. None was designed around DINOv2's original classification objective; each simply adopts the frozen backbone.
What DINOv2 supplies is a substrate, not an answer. Its features encode enough scene geometry to be useful for depth, but nothing in a self-supervised run on unlabeled images teaches a network to produce a depth map. That training signal, and how far it can be pushed, is the next constraint.
Data Breadth as Signal
Depth Anything keeps MiDaS's loss and DINOv2's encoder fixed and changes only how much data trains the network built on top of them. A teacher trained on six labeled datasets totalling 1.5 million images pseudo-labels 62 million unlabeled images from eight public corpora; a student then trains on labeled and pseudo-labeled data together. Two refinements stop the student from simply copying the teacher. Its inputs are strongly perturbed, including CutMix blending of two unlabeled images at fifty percent probability, forcing it to resolve boundaries neither source image shows on its own: without this step, mean error across evaluation domains does not improve at all with the extra data; the CutMix challenge brings it down measurably, and a further loss aligning student and frozen-encoder features, active only where the two disagree, lowers it again.
The payoff shows on the same backbone MiDaS used: relative-depth error on KITTI drops from 0.127 to 0.076, a forty percent reduction, with no change to the network. Data breadth, not a new model, produced the gain.
The teacher generating all sixty-two million pseudo-labels is itself trained on real sensor, stereo, and structure-from-motion annotations, the kind that carries systematic noise on transparent objects, repetitive textures, and dynamic regions. Scaling the data engine also scales whatever the teacher gets wrong.
Synthetic Teachers
Depth Anything V2 locates the remaining error in the labels themselves, not the network or the data volume. Sensors fail on transparent glass, stereo matching fails on repetitive textures, and structure-from-motion is unstable around dynamic objects, so a teacher trained on any of these real annotations inherits their mistakes before a single pseudo-label leaves it. V2 removes real images from teacher training entirely: the teacher trains exclusively on synthetic images from five rendered datasets, where every thin structure and reflective surface carries exact, renderer-generated ground truth. Among every backbone family tested for this synthetic-only stage, only the largest DINOv2 variant achieves a satisfying transfer to real-world inference; the rest fail outright.
That synthetic-trained teacher then pseudo-labels the same real unlabeled images Depth Anything used, and the resulting labels beat manual annotation on every measured metric. Ground truth, in this pipeline, is now a model's synthetic-trained output rather than a sensor reading or a human label.
Every model in this lineage, from MiDaS through V2, still takes exactly one image and returns exactly one depth map. How several views of the same scene relate to each other is left to different models entirely, with different training pipelines of their own.
Geometry Converges
VGGT is one such separate pipeline, built for multi-view geometry. It takes a sequence of images ranging from one to hundreds and, in a single forward pass, returns per-frame cameras, depth maps, and point maps in one shared coordinate frame, eliminating the pairwise matching and global alignment earlier methods required. Its patch tokens come from a frozen DINOv2 encoder, the same substrate the monocular depth models share.
Depth Anything 3 closes the remaining gap: instead of a depth map alone, it predicts depth plus a per-pixel camera ray map, one target from which a 3D point and, via a lightweight head, a camera pose can both be recovered. The same DINOv2 backbone handles any view count, monocular to thousands of images, through an input-adaptive attention rearrangement, with or without known poses, from one set of weights. Distilled from a Depth Anything V2 teacher, it surpasses that teacher across five depth benchmarks, and on multi-view geometry surpasses VGGT itself, by 44.3 percent in camera pose accuracy and 25.1 percent in geometric accuracy.
The same architecture now handles the single-image case and the any-view case with identical weights, dissolving a boundary that once separated two model families. Across five models the architecture never changed, only the target, the data, or the teacher supplying the label. When the labels are synthetic and the teacher is another model, not a sensor, what does ground truth depth still mean?