The Shared Substrate
Every method that follows in this account produces or consumes one of five quantities. Feature matching turns two sets of descriptors into a consistent partial assignment of correspondences. Epipolar geometry constrains a point seen in one image to a specific line in the other, fixed entirely by the two camera positions, reducing a two-dimensional correspondence search to a one-dimensional one. RANSAC draws minimal random subsets, instantiates a candidate model from each, and keeps the one whose consensus set of inliers is largest, underwriting every stage that follows it. Pose estimation recovers the six degrees of freedom relating a camera to a scene or to a second camera, either from 2-D/3-D correspondences by the Perspective-n-Point problem or from calibrated 2-D/2-D correspondences via the essential matrix. Bundle adjustment refines every camera parameter, and every 3-D point, by minimising the total reprojection error across every observation.
Whether a human derived the estimator that produces these quantities or a network learned it does not change what the quantity is. What changes over the following two decades is who computes the first of them. Until 2018, correspondences come from a detector and a descriptor built entirely by hand.
Detectors and Descriptors, by Hand
SIFT detects keypoints as extrema of a Difference-of-Gaussian pyramid, refines their location to sub-pixel accuracy by quadratic interpolation, and describes each with a 128-dimensional gradient-histogram vector invariant to scale and rotation. On CPU the full pipeline costs roughly 5.2 seconds per frame. ORB instead runs FAST-9 across a five-level pyramid, ranks candidates by Harris cornerness, and packs a 256-bit binary descriptor from greedily selected, rotation-steered pixel-pair tests, matched by Hamming distance rather than Euclidean distance. The same frame that costs SIFT 5.2 seconds costs ORB roughly 15 milliseconds, about two orders of magnitude faster, bought by trading SIFT's 512-byte float descriptor for ORB's 32-byte binary one.
Both detectors are the product of hand-engineered choices: the Difference-of-Gaussian scale ratio, the Harris response threshold, the correlation cutoff that selects ORB's 256 test pairs. Nothing in either pipeline is fit to data. The parameters are set once, by a person, and never revisited during training.
The First Stage Falls to Learning
SuperPoint collapses detection and description into a single fully-convolutional forward pass, trained without any human keypoint annotations through Homographic Adaptation: detections are aggregated across many random warps of the same image until only the repeatable ones survive. SuperPoint keypoints and descriptors are the front end both SuperGlue and LightGlue typically consume. XFeat keeps SuperPoint's shared-encoder, two-head design but decouples the keypoint head onto unfolded 8x8 raw-pixel blocks rather than the deep encoder output, reaching roughly nine times SuperPoint's throughput on the same CPU.
The first stage boundary in the classical pipeline, the line between a hand-designed detector and a hand-designed descriptor, is gone: one learned encoder now produces both, and the data that shapes it is unlabelled images rather than a closed-form rule. What is still hand-designed is the second stage. Turning two sets of descriptors into a consistent set of correspondences is still the nearest-neighbour ratio test.
Matching Becomes a Learned Assignment
SuperGlue replaces the ratio test with a graph neural network trained end to end: nine alternating layers of self- and cross-attention enrich each keypoint's descriptor with context from both images, and a Sinkhorn optimal-transport solver turns the resulting score matrix into a soft partial assignment, with a learned dustbin absorbing points that have no counterpart. LightGlue keeps the attention framework but replaces the Sinkhorn solver with a dual-softmax assignment gated by a per-point matchability score, and adds a confidence head that lets the network exit early once enough points are certain. On MegaDepth the adaptive variant matches SuperGlue's pose accuracy at 31.4 milliseconds against SuperGlue's 70.0, less than half the latency for an equivalent result.
The second stage boundary is now absorbed: matching itself is a differentiable, trained assignment rather than a fixed heuristic. What both matchers still assume is that a detector has already reduced each image to a sparse set of keypoints before matching begins.
Matching Goes Dense and Detector-Free
LoFTR removes the keypoint detector from the pipeline entirely. A shared CNN backbone produces coarse and fine feature maps for both images, an attention-based transformer enriches the coarse maps with global context, and a confidence threshold combined with mutual-nearest-neighbour selection establishes correspondences directly on the dense maps, later refined to sub-pixel accuracy by a local correlation window. Because no detector fires first, LoFTR finds matches in low-texture regions where a keypoint detector would find no repeatable interest points at all. MASt3R reuses this dense, detector-free paradigm but grounds it differently: it extends DUSt3R's pointmap regression with a dense descriptor head trained by an InfoNCE contrastive loss, so correspondences are pixels observing the same physical 3-D point rather than pixels with similar 2-D appearance.
Matching survives as a stage in both cases, but its substrate has shifted, from a fixed descriptor comparison to a dense attention map, and now to a 3-D point shared between two views. What has not yet changed is that matching still hands its output to a separate epipolar-geometry, pose, and bundle-adjustment chain downstream.
The Whole Chain Becomes One Network
COLMAP is the mature classical endpoint. It keeps the standard incremental structure-from-motion pipeline and replaces four of its stages with more robust variants: scene-graph verification after matching, next-best-view selection, recursive RANSAC triangulation, and an iterative bundle-adjustment loop with redundant-view mining. It consumes pose estimation and bundle adjustment as unmodified components: it assembles the classical stages more robustly rather than replacing any of them. DUSt3R collapses the entire downstream chain, matching, epipolar geometry, triangulation, and bundle adjustment, into one feed-forward pass: a shared-frame pointmap regression that jointly encodes correspondence, relative pose and depth for an uncalibrated, unposed image pair. Its global-alignment step for more than two views minimises 3-D projection error rather than 2-D reprojection error, and is faster, seconds rather than minutes. VGGT removes the remaining seam: a single alternating-attention transformer processes one to hundreds of views in one pass, predicting cameras and point maps jointly, with no pairwise-plus-global-alignment post-processing left to run.
What has not yet been questioned is the architecture doing the collapsing. DUSt3R and VGGT are both bespoke, multi-headed transformer stacks purpose-built for this one task.
What Survives Collapse
Depth Anything 3 needs none of that bespoke machinery. It is a plain DINOv2 vision transformer, architecturally identical to the standard backbone, with no separate pose head and no explicit point-map regression. Instead of a pointmap it predicts a depth map and a per-pixel ray map, a single unified target from which camera pose and 3-D position follow by direct combination. On the same any-view geometry benchmark, this plain backbone surpasses VGGT by roughly 44% in camera pose accuracy and 25% in geometric accuracy. Feed-forward 3-D reconstruction is the paradigm DUSt3R founded and MASt3R, VGGT and Depth Anything 3 each generalise, and at every step the module doing the enforcing has shrunk: a hand-built detector, then a learned matcher, then a purpose-built multi-headed transformer, and now a generic backbone with one prediction target.
Bundle adjustment is the stage this trend converges on. It was the classical pipeline's final, most accurate refinement, the joint minimisation of reprojection error over every camera and every point. If a generic transformer with no explicit reprojection term recovers what that refinement used to compute, one question remains: Which parts of bundle adjustment are fundamental, and which are artefacts of the old decomposition into matching, pose and refinement?