narrativeFinding a Chessboard

Twenty-five years of chessboard corner detection form one long argument about where the decisive evidence should live. A checkerboard X-corner looks, to a generic detector, like any other place where intensity varies in two directions — the detector cannot by itself tell a true target junction from a blob, an edge crossing, or background clutter. The methods in this story respond by relocating the evidence: first from the raw pixel response to the shape of its local neighbourhood, then from the neighbourhood to the topology of the whole grid, then from hand-crafted topology to a learned response function, and finally into the target itself, redesigned to announce its own position. Each move repairs a specific failure the previous stage could not fix, and each fix exposes the next one.

The X-corner is not a generic corner

Harris Corner Detector scores every pixel by $R = \det(M) - k \cdot \operatorname{tr}(M)^2$, where $M$ is the gradient covariance matrix summed over a local window. The response is general-purpose: it fires at corners, blobs, and edge intersections without any notion of X-junction geometry, and under an exposure change $I \to \rho I$ it scales as $\rho^4$, forcing a fresh detection threshold for every image. Chessboard detection needs something narrower. The Hessian Saddle Response supplies it: instead of the gradient-covariance matrix $M$, it uses the image Hessian $H$, whose determinant is negative at a saddle point and near zero at flat regions, edges, and blobs. Harris's structure tensor is positive semi-definite by construction and cannot separate a saddle from a blob; the Hessian's signed eigenvalues can. The operator traces back to a 2005 Taylor-expansion solve for X-corners, and the atlas concept page describes it as a direct variant of the Harris structure-tensor idea — the same appeal to a $2\times2$ local matrix and its determinant, aimed at a different local shape. That single operator becomes the shared per-pixel discriminator for nearly every full-pattern detector later in this story.

Hand-crafted responses compete on speed and blur

ChESS Corners narrows the response to the X-junction itself: 16 fixed integer offsets on a radius-5 ring feed three difference sums, combined into a single score with no interpolation and no trigonometry — a domain-specific alternative to the Harris structure tensor, tuned to alternating bright-dark quadrants rather than general intensity variation. Geiger Chessboard Corner Detector makes a different bet: a four-quadrant convolution likelihood evaluated at three fixed window sizes, $4 \times 4$, $8 \times 8$, $12 \times 12$, then verified against a 32-bin gradient-orientation histogram. The three scales are chosen empirically, and the design stalls when blur falls between them. Localized Radon Checkerboard Corners replaces both ring sampling and quadrant convolution with 1-D box-filter line integrals along four discrete angles; because box sums attenuate noise while gradients amplify it, the response trades ChESS's per-pixel simplicity for robustness under blur and low contrast.

Pyramidal Blur-Aware X-Corner Chessboard Detector attacks the scale problem directly: it runs a ChESS-style 16-sample template at every level of a full image pyramid and keeps, per corner, the level that maximizes intensity divided by resolution. On the standard 2021 benchmark this beats Geiger's four-quadrant likelihood, F1 0.97 against 0.92. Its edge-validation pass also inherits Duda's box-filter idea directly — the paper describes the spoke pass as conceptually similar to the approximated Radon transform. None of the four hand-crafted responses dominates every regime: each is a different bet on what a single-pixel score should measure, and each still hands its candidates to a separate stage before a full board can be trusted.

When the response is not enough, ask topology

When a per-pixel response cannot separate every true corner from every false one, the next move is to stop trusting individual pixels and verify candidates against the graph they form. Topological Grid Recovery names this shift: a permissive detector first over-generates candidates, then a graph built over them — Delaunay triangulation, a k-nearest-neighbour graph, or a subgraph search — accepts only configurations matching chessboard topology, decoupling recall from precision. Topological Grid Finding is the first instance: Delaunay-triangulate the candidate corners, merge each triangle with its one same-colour edge-neighbour into a quadrilateral, then drop any quad whose vertex degree exceeds four at two or more nodes, or whose opposite-edge length ratios do not stay below ten. What survives is flood-filled with integer grid coordinates from a seed quad, and three marker circles on the pattern fix the origin and x-axis.

Chessboard Detection via X-Corners and Topology keeps the Delaunay backbone but moves the filter down to individual triangles: a triangle is legal only if its interior is uniform and it has at most two edge-neighbours with the same colour, iterated to a fixed point. The paper positions itself explicitly as extending the Shu et al. pipeline with a per-pixel detector and a refined topological filter. Subpixel refinement — the Chen-Zhang Hessian solve — runs only at vertices that already passed the topology test, so refinement compute is never spent on a false candidate.

Occlusion breaks the grid, not the corner

Topology filtering assumes the candidate graph is roughly right; it does not repair a graph that is missing a large piece. ROCHADE pushes robustness further by reducing the gradient-magnitude edge map to a single-pixel centreline graph and reading off inner corners as saddle points — vertices of degree at least three — refined by fitting a bivariate quadratic to a cone-filtered neighbourhood. The design still requires the full pattern: verification checks that a connected component contains exactly the expected number of saddle clusters in the right adjacency, and any occluded or cropped corner fails that check outright. The page notes that this same stage-1 graph is the input required by OCPAD to recover partially occluded patterns, and that the rest of the pipeline is reused verbatim — only the strict full-pattern check needs replacing.

OCPAD does exactly that: it inherits ROCHADE's early stages and swaps the strict corner-count check for VF2 subgraph isomorphism against a model graph, driven by a binary-search driver over how many candidate vertices to include. GP Checkerboard Enhancement repairs the same failure by an unrelated mechanism: it trains two independent Gaussian processes on whatever board-to-pixel pairs an upstream detector already allocated, then uses the posterior mean to predict pixel coordinates for grid positions with no detection at all — occluded, out of frame, or simply missed. The two repairs are complementary by construction: OCPAD recovers a partial visible subgraph, while GP enhancement fills in occluded or out-of-frame corners with smooth refinement.

Detectors that learn the response

A fifth move replaces the hand-crafted response function itself with one learned from data. MATE is the first such detector: a three-convolution CNN with 2,939 parameters, trained by mean-squared error between the predicted response map and a binary corner mask, and the atlas marks it a learned alternative to ChESS's hand-crafted ring score. Its post-processing is minimal — a fixed 0.5 threshold, no non-maximum suppression, no clustering — and on the ROCHADE uEye benchmark that produces 492 false positives. CCDN doubles MATE's depth to six convolutional layers, replaces the mean-squared error with a weighted cross-entropy loss, and stacks an adaptive threshold, non-maximum suppression, and k-means++ clustering on top; the same benchmark's false-positive count drops from 492 (MATE) to 93.

CCS widens the scope from detector to pipeline. It embeds a UNet 2D-Gaussian heatmap detector, extracts the subpixel mean by an SVD-based surface fit, and wraps the result in a CNN distortion-correction stage upstream and a RANSAC-stabilised Zhang calibration downstream, reaching 0.37 px real-data reprojection error against a MATLAB Zhang baseline's 0.45 px. The distinction from CCDN is one of scope, not of accuracy alone: CCDN is a standalone per-pixel corner detector, where CCS is a complete calibration pipeline. Three papers, three answers, but the same substitution at their core: a trained function stands in for the ring score, the quadrant convolution, or the centreline graph that came before it.

The target identifies itself

PuzzleBoard closes the loop by redesigning the target rather than the detector. It finds saddle-point corners from a Hessian response, the same family used by ROCHADE and Laureano, then builds a 9-nearest-neighbour graph over them, disambiguates direct grid edges from diagonals using the Hessian eigenvector directions at each corner, and reconstructs the grid with a Kruskal minimum spanning forest. What is new is what happens next: every checkerboard edge carries one bit, encoded as a small circle at the edge midpoint, and the full $501 \times 501$ pattern is the superposition of two binary de Bruijn factor maps. Reading the circles near a detected corner and cross-correlating against the two factor maps recovers that corner's absolute grid coordinate directly — no boundary marker, no full-pattern requirement. Majority voting across the pattern's built-in bit repetition tolerates up to 40% of the raw observed bits being corrupted before decoding fails. The circles are strictly additive: a generic checkerboard corner detector still locates corners on a PuzzleBoard, since the position decoding rides on top of the saddle test without interfering with it.

What remains for the detector to be robust to

Chessboard X-Corner Detection, the survey that ties this whole family together, records two open problems that none of the preceding six moves has closed. No current method handles fisheye-grade lens distortion combined with extreme tilt without an explicit pre-rectification step; PuzzleBoard's de-Bruijn decoding tolerates more distortion than the grid-topology methods, but only by requiring its own specific target rather than an ordinary checkerboard. And occlusion combined with low contrast defeats both repair mechanisms from earlier in the story at once: OCPAD requires a strong upstream graph, GP enhancement requires a partial board fragment, and neither bootstraps from a detector that returns almost nothing.

PuzzleBoard's own saddle detector is a reminder of how narrow the actual advance is. Its response is the same Harris-penalised Hessian form, $s = f_{xy}^2 - f_{xx}f_{yy} - k(f_{xx}+f_{yy})^2$, that ROCHADE and Laureano build on in other guises — self-identification changes what happens after a corner is found, not whether the per-pixel discriminator can find one in the first place. The survey's own closing question asks whether the field even needs to keep improving that discriminator: does the small specialised target make pretraining unnecessary, or would a larger model improve robustness on the multispectral and low-contrast cases no hand-crafted response yet reaches. Once the target identifies itself, what is left for the detector to be robust to, and does the corner response still matter?