One Constraint, Four Answers
Epipolar Geometry ties two views of a scene to one fact: a point visible in one image must lie on a specific line in the other, fixed entirely by where the two cameras sit. Searching along that line already turns correspondence from a 2-D scan into a 1-D scan of a line, but the line is slanted and tilts differently for every query point, so the search still touches the whole second image.
Stereo Rectification removes the slant. It resamples the pair so that every pair of conjugate epipolar lines maps to a single common image row, and once that holds, corresponding points fall on the same image row and dense correspondence becomes a same-row scanline search. Four methods solve exactly this resampling problem, published within a sixteen-month window between 1999 and 2000.
None of the four supersedes another, and the gap between their publication dates is too short to explain why more than one is still in use. What separates them is what each is willing to assume about the cameras before it starts: nothing beyond a fundamental matrix, that matrix plus one extra correspondence for orientation, or full metric calibration with both perspective projection matrices known. That assumption, not the calendar, decides which of the four applies to a given rig.
Hartley: Least Disparity
Hartley Projective Rectification needs nothing beyond the fundamental matrix; no camera matrices or calibration are used, the construction depends on the fundamental matrix alone. A translation sends a chosen reference point to the origin, a rotation about the origin aligns the translated epipole with the x-axis, and a perspectivity sends that aligned epipole to infinity, a construction the method describes as sending the epipole to infinity with a quasi-affine perspectivity. Fixing the matching transform for the other image is then a genuine least-squares problem, minimising horizontal disparity between conjugate points, and restricting that transform to an affine form is what keeps this fit linear.
The construction is not universal. Its correctness proof needs the epipole outside the view window; when the epipole lies inside it instead, forward or near-forward motion, the paper's only stated remedy is to shrink the view window or choose a different reference point, with no guarantee of a valid full-window solution.
That unresolved case, an epipole that will not leave the window, is exactly where the next two methods diverge from Hartley and from each other.
Loop-Zhang: Least Distortion
Loop-Zhang Rectifying Homographies starts from the same fundamental matrix as Hartley but asks a different question, not least disparity but least distortion. Each homography is factored as a projective component chosen from a quantifiable 2-D distortion-minimization criterion, a similarity component that aligns epipolar lines with the horizontal axis, and a shearing component that spends the remaining degrees of freedom to further reduce distortion.
The projective component is the hard part. Its criterion has no closed form; it requires an iterative degree-7 polynomial root-find, with no proven bound on distance from the optimum, and the seed for that iteration comes from solving the two single-image terms separately in closed form.
The method still sends the epipole toward infinity to build the projective component, so an epipole located inside the image frame drives part of the rectified image toward infinity, forcing cropping in practice, the same limit Hartley's construction hits. Neither of these two homography-based methods has an answer for an epipole that stays inside the frame.
Pollefeys: Polar Reparametrisation
Pollefeys Polar Rectification refuses the planar rectified image both earlier methods depend on. Instead of sending the epipole toward infinity, it reparametrizes each image in polar coordinates centered on its own epipole, so the epipole becomes the pole of the coordinate system rather than a point that must be pushed outward. Orientation is fixed from the usual seven or more correspondences used to estimate the fundamental matrix, plus one additional correspondence used solely to fix orientation.
Because nothing is ever sent to infinity, the output-size bound is combinatorial, not data-dependent: it depends only on the source dimensions, unlike planar rectification's output size, which grows without bound as the epipole approaches the image. An epipole sitting inside the frame, the exact case that breaks Hartley and Loop-Zhang, is simply not a problem here.
The price is shape: the rectified images are non-rectangular and visibly distorted compared to a planar-homography-rectified pair. None of these first three methods can recover metric depth from its output; all three are projective, working from the fundamental matrix alone.
Fusiello: Compact Closed Form
Fusiello Compact Stereo Rectification trades the other three methods' minimal inputs for a stronger one: the two known perspective projection matrices of the original cameras, intrinsics and extrinsics included. In exchange it gets a direct construction whose shared new orientation is assembled row-by-row as an orthonormal frame from three geometric constraints tied to the baseline, and the entire construction is 22 lines of MATLAB.
Because both new projection matrices carry the same intrinsics and an orientation anchored to the real baseline, the result is a Euclidean rectification: disparity between rectified images inverts metrically as , and 3-D points can be triangulated from the rectified images with the new PPMs. That is the one result none of the other three methods can offer from a fundamental matrix alone.
The construction fails outright when the optical axis is parallel to the baseline, pure forward motion, since the cross product defining the new axis degenerates to zero. Outside that single case, four methods answer the same rectification problem to four different specifications, and the choice between them is settled by which assumption about the cameras already holds, not by which paper came first.