Camera calibration

Camera calibration recovers the intrinsic matrix $\mathbf{K}$ and the lens distortion parameters of a camera from images of a known calibration target. Without accurate calibration, every downstream geometric computation — triangulation, pose estimation, epipolar search — is systematically wrong.

Zhang’s Method

Zhang’s method (1999) is the standard calibration procedure. It uses images of a planar checkerboard taken from multiple viewpoints. Each image provides correspondences between known 3D board corners and detected 2D image corners, giving constraints on both the homography (image-to-board) and the intrinsic parameters. The procedure:

Detect checkerboard corners in $N \geq 3$ images at different orientations.
Compute homographies $H_i$ between the board plane and each image.
Extract constraints on $\mathbf{K}$ from each $H_i$ , using $H = \lambda\mathbf{K}[r_1, r_2, \mathbf{t}]$ with $r_1 \perp r_2$ and $\|r_1\| = \|r_2\|$ .
Solve a linear system for the intrinsic parameters.
Refine all parameters (intrinsic + extrinsic + distortion) via nonlinear least squares.

Where the constraints come from. Write $H = [\mathbf{h}_1, \mathbf{h}_2, \mathbf{h}_3]$ . Since $\mathbf{h}_1 \propto \mathbf{K}r_1$ and $\mathbf{h}_2 \propto \mathbf{K}r_2$ with $r_1, r_2$ orthonormal, each view yields two linear equations in the entries of the symmetric matrix $B = \mathbf{K}^{-T}\mathbf{K}^{-1}$ :

$\mathbf{h}_1^T B\, \mathbf{h}_2 = 0, \qquad \mathbf{h}_1^T B\, \mathbf{h}_1 = \mathbf{h}_2^T B\, \mathbf{h}_2$

$B$ has 6 unknowns (5 if skew is assumed zero), so 3 views suffice; $\mathbf{K}$ is then recovered from $B$ by Cholesky-style decomposition. This is why the board must be shown at different orientations — parallel views give redundant constraints.

Lens Distortion Models

Real lenses deviate from the ideal pinhole model. Two standard models:

Radial distortion causes straight lines to appear curved:

$x_d = x'(1 + k_1 r^2 + k_2 r^4 + k_3 r^6), \qquad y_d = y'(1 + k_1 r^2 + k_2 r^4 + k_3 r^6)$

where $r^2 = x'^2 + y'^2$ is the squared distance from the principal point. $k_1 > 0$ gives barrel distortion; $k_1 < 0$ gives pincushion distortion. Wide-angle lenses have large $|k_1|$ .

Tangential distortion is caused by the lens not being perfectly parallel to the image plane:

$x_d = x' + 2p_1 x'y' + p_2(r^2 + 2x'^2), \qquad y_d = y' + p_1(r^2 + 2y'^2) + 2p_2 x'y'$

In practice, $k_1, k_2$ (and sometimes $k_3, p_1, p_2$ ) are estimated jointly with $\mathbf{K}$ . OpenCV implements this directly in cv::calibrateCamera().

Judging calibration quality

The standard quality metric is the RMS reprojection error returned by the calibration: reproject the board corners with the estimated parameters and measure the pixel residuals. Beyond the single number, check that:

Residuals are evenly distributed across the image — a bias in the corners indicates an inadequate distortion model (e.g., a fisheye lens forced into the radial-tangential model).
The board appeared in all regions of the image, including corners and edges, where distortion is strongest and otherwise unconstrained.
Views included substantial tilt (not just frontal shots) so that focal length and principal point decorrelate.

Common pitfalls

Motion blur and rolling shutter during capture corrupt corner detection; hold the board (or camera) still for each shot.
A non-flat board: printed paper taped to cardboard bends; distortion estimates absorb the bending. Use a rigid, flat target.
Too few poses near the image border — the high-order radial terms then overfit and extrapolate wildly outside the covered region.
Calibrating at one focus/zoom setting and running at another: intrinsics change with focus; lock the lens and recalibrate whenever hardware settings change.

Why it matters for SLAM

SLAM systems assume that undistorted, calibrated observations feed their geometric solvers; a few pixels of uncorrected distortion at the image border can dominate reprojection error and corrupt the map. Calibration is also the template for harder problems you will meet later — camera-IMU and camera-LiDAR extrinsic calibration follow the same “known target + nonlinear refinement” pattern. Practically, calibrating your own camera with a checkerboard is one of the best first exercises in the field.