Given a pair of stereo images, rectification determines a transformation of each image plane such that pairs of conjugate epipolar lines become collinear and parallel to one of the image axes. The important advantage of rectification is that computing correspondences, a 2-D search problem in general, is reduced to a 1-D search problem, typically along the horizontal raster lines of the rectified images [ 6 , 3 , 7 , 8 , 13 ]. The rectified images can be thought of as acquired by a new stereo rig, obtained by rotating the original cameras. This is indeed the basis for computing the rectifying transformation , as well as the perspective projection matrices rectifying the images ( rectifying projection matrices ). This paper presents a novel algorithm rectifying a calibrated stereo rig of unconstrained geometry and mounting general cameras. The only input required is the pair of projection matrices of the two cameras (but not their individual intrinsic or extrinsic parameters), which can be estimated using one of the many existing calibration methods [ 5 , 14 , 2 , 10 , 12 ]. The output is the pair of rectified projection matrices, which can be used to compute the rectified images. Reconstruction can be performed directly from the rectified images and projection matrices. Given the importance of rectification as a module for stereo systems, and the shortage of easily reproducible, easily accessible and clearly stated algorithms, we have made a ``rectification kit'' (code, examples, data and user's manual) available on line. Our work improves and extends [ 1 ], in which a matrix satisfying the constraints sufficient to guarantee rectification is hand-crafted, not derived, and the constraints necessary to guarantee a unique solution are left unspecified. Instead, we enforce explicitly all the constraints necessary and sufficient to derive a unique rectification matrix, and obtain the latter as the solution of the resulting system of simultaneous equations. Some authors report rectification under restrictive assumptions; for instance, [ 9 ] assumes a very restrictive geometry (parallel vertical axes of the camera reference frames). Recently, [ 11 ] have introduced an algorithm which performs rectification with general stereo geometry given a weakly calibrated stereo rig (fundamental matrix and three conjugate pairs). We do require strong calibration, which however can be achieved in many practical situations and by several algorithms.
This paper is organised as follows. Section 2 introduces our notations and summarises some necessary mathematics of perspective projections. Section 3 expresses the rectifying image transformation in terms of projections matrices. Section 4 and 5 derive the algorithm for computing the rectifying projection matrices. Section 6 gives the compact (22 lines), working MATLAB code for our algorithm, and indicates where to find our rectification kit on line. Section 7 reports tests on synthetic and real data. Section 8 is a brief discussion of our work.
Adrian F Clark