Goal

Given $n \geq 3$ images of a planar target with known metric coordinates, compute the camera intrinsic matrix $A \in \mathbb{R}^{3 \times 3}$, two radial distortion coefficients $(k_1, k_2)$, and the per-view rigid pose $(R_i, t_i) \in SO(3) \times \mathbb{R}^3$. No 3D calibration object is required and the motion between views need not be known; the pattern and the camera may move freely relative to each other. The defining property is a closed-form linear initialization that turns each view's plane-to-image homography into two linear constraints on the image of the absolute conic.

Algorithm

Let $I$ be an image of a planar target. Let $M = [X, Y, 0]^T$ denote a target point in the world frame (plane on $Z = 0$) and $m = [u, v]^T$ its image projection; their homogeneous forms are $\tilde M$ and $\tilde m$.

The pinhole model with intrinsic matrix $A$ and extrinsic $[R \mid t]$ reads

where $r_i$ is the $i$-th column of $R$, $(u_0, v_0)$ is the principal point, $(\alpha, \beta)$ are axis scale factors, and $\gamma$ is the skew. Since $Z = 0$, the third column of $R$ drops out and the model collapses to a $3 \times 3$ homography

with $\lambda$ an unknown non-zero scalar absorbing the homography's scale ambiguity.

Orthonormality of $r_1$ and $r_2$ yields two linear constraints on $b$ per view:

Stacking the rows from $n$ views produces a $2n \times 6$ matrix $V$ and the homogeneous system $V b = 0$.

Implementation

The linear initialization — building $V$ from homographies and extracting $A$ from the right null vector — in Rust:

use nalgebra::{Matrix3, Matrix6xX, Vector6, DMatrix};

fn v_row(h: &Matrix3<f64>, i: usize, j: usize) -> Vector6<f64> {
    let (hi, hj) = (h.column(i), h.column(j));
    Vector6::new(
        hi[0]*hj[0],
        hi[0]*hj[1] + hi[1]*hj[0],
        hi[1]*hj[1],
        hi[2]*hj[0] + hi[0]*hj[2],
        hi[2]*hj[1] + hi[1]*hj[2],
        hi[2]*hj[2],
    )
}

fn intrinsics_from_homographies(hs: &[Matrix3<f64>]) -> Matrix3<f64> {
    let mut v = DMatrix::<f64>::zeros(2 * hs.len(), 6);
    for (i, h) in hs.iter().enumerate() {
        let v12 = v_row(h, 0, 1);
        let d   = v_row(h, 0, 0) - v_row(h, 1, 1);
        v.row_mut(2*i    ).copy_from(&v12.transpose());
        v.row_mut(2*i + 1).copy_from(&d.transpose());
    }
    let svd = v.svd(false, true);
    let vt  = svd.v_t.expect("right singular vectors");
    let b   = vt.row(vt.nrows() - 1);
    let (b11, b12, b22, b13, b23, b33) =
        (b[0], b[1], b[2], b[3], b[4], b[5]);

    let den  = b11 * b22 - b12 * b12;
    let v0   = (b12 * b13 - b11 * b23) / den;
    let eta  = b33 - (b13 * b13 + v0 * (b12 * b13 - b11 * b23)) / b11;
    let alpha = (eta / b11).sqrt();
    let beta  = (eta * b11 / den).sqrt();
    let gamma = -b12 * alpha * alpha * beta / eta;
    let u0    = gamma * v0 / beta - b13 * alpha * alpha / eta;

    Matrix3::new(
        alpha, gamma, u0,
        0.0,   beta,  v0,
        0.0,   0.0,   1.0,
    )
}

The per-view pose from $(A, H)$, with the final rotation projected to $SO(3)$:

fn pose_from_homography(a_inv: &Matrix3<f64>, h: &Matrix3<f64>)
    -> (Matrix3<f64>, nalgebra::Vector3<f64>)
{
    let lambda = 1.0 / (a_inv * h.column(0)).norm();
    let r1 = lambda * (a_inv * h.column(0));
    let r2 = lambda * (a_inv * h.column(1));
    let r3 = r1.cross(&r2);
    let t  = lambda * (a_inv * h.column(2));
    let q  = Matrix3::from_columns(&[r1, r2, r3]);
    let svd = q.svd(true, true);
    let r = svd.u.unwrap() * svd.v_t.unwrap();
    (r, t)
}

Nonlinear refinement of $(A, k_1, k_2, \{R_i, t_i\})$ over the total reprojection error is standard Levenberg-Marquardt and is not reproduced here; the rotations are parameterised by the Rodrigues 3-vector to keep the Jacobian unconstrained.

Remarks

• Complexity per calibration is dominated by the per-view homography fit ($O(N)$ in the number of correspondences) and the closed-form $2n \times 6$ SVD of $V$; the LM refinement is $O(n \cdot m)$ per iteration in the image count $n$ and target-point count $m$.
• The linear step minimises an algebraic distance on $b$ and does not enforce positive definiteness of $B$; the recovered $\alpha, \beta$ can turn imaginary under heavy noise. The LM stage on the geometric reprojection error is what delivers calibration-grade accuracy — the closed form supplies the initial guess only.
• Parallel orientations of the planar target are degenerate: a second view obtained from the first by a rotation about the plane normal (and any translation) contributes constraints linearly dependent on those of the first view.
• Minimum view count depends on which intrinsics are fixed: $n \geq 3$ for the full five-parameter $A$, $n \geq 2$ with skew fixed to $\gamma = 0$, and $n = 1$ can only solve two parameters (typical choice: $(\alpha, \beta)$ with principal point at the image centre).
• Distortion scope is two-term radial only; tangential (decentering) and thin-prism terms belong to the Brown-Conrady / Weng model and are a drop-in extension of the projection function $\breve m$ and its Jacobian in the LM step.
• Compared with Tsai 1987: see When to choose Tsai over Zhang on the Tsai page, which hosts the comparison per the older-paper-hosts rule.
• Image-level RANSAC extension: CCS (Zhang et al., RA-L 2022) augments the LM stage with a view-selection RANSAC — randomly sample a subset of views, estimate intrinsics via the linear + LM steps above, score by reprojection error on all views, retain views below an inlier threshold, repeat until inlier count is sufficient (§III-C). View-level (rather than corner-level) RANSAC suffices because the upstream UNet detector already rejects per-corner outliers via the heatmap variance $\sigma$.
• The CCS RANSAC extension reduces real-data reprojection error from 0.45 px (STD 0.10) for a Matlab Zhang implementation to 0.37 px (STD 0.02) on a HIKROBOT 1440×1080 sensor (Table II) — a five-fold reduction in run-to-run standard deviation attributed to the view-selection step.

When to choose Zhang over Sturm-Maybank

Sturm-Maybank (CVPR 1999) and Zhang (ICCV 1999 / TPAMI 2000) are concurrent derivations of the same two linear IAC constraints per planar homography. Neither paper predates the other; both share the core algorithm. Per the "same year → more general scope" tiebreaker, Zhang hosts this comparison: Zhang's page covers the end-to-end calibration pipeline (homography fit, IAC linear initialisation, intrinsic + extrinsic recovery, two-term radial distortion, LM refinement) while Sturm-Maybank focuses on the linear IAC step plus singularity analysis and the variable-intrinsics generalisation.

Choose Zhang when (1) you want a complete calibration pipeline including distortion and LM refinement — Zhang's page is the practical end-to-end recipe used by OpenCV, MATLAB, and Kalibr; (2) the camera intrinsics are constant across views (the standard machine-vision case). Choose Sturm-Maybank when (1) the camera intrinsics vary per view (zooming or focusing camera) — this is Sturm-Maybank's unique extension; (2) you need to diagnose a calibration failure and want the explicit singularity catalogue (Tables 1 and 2) to identify which plane orientations make a parameter unrecoverable; (3) you want the cleaner geometric framing via circular-point projections to the IAC, which makes the singularity arguments transparent.

References

1. Z. Zhang. A Flexible New Technique for Camera Calibration. MSR-TR-98-71 (also IEEE TPAMI 22(11), 2000). pdf
2. R. Y. Tsai. A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE Journal on Robotics and Automation 3(4):323–344, 1987. pdf
3. P. Sturm and S. J. Maybank. On plane-based camera calibration: A general algorithm, singularities, applications. IEEE CVPR, 1999. pdf
4. J. Weng, P. Cohen, and M. Herniou. Camera calibration with distortion models and accuracy evaluation. IEEE TPAMI 14(10):965–980, 1992. pdf
5. Y. Zhang, X. Zhao, D. Qian. Learning-Based Distortion Correction and Feature Detection for High Precision and Robust Camera Calibration. IEEE Robotics and Automation Letters 7(4):10470–10477, 2022. arXiv