Image and Vision Computing 19 (2001) 1099±1110 www.elsevier.com/locate/imavis Simple and ef®cient method of calibrating a motorized zoom lens Yong-Sheng Chen a,c,1, Sheng-Wen Shih b,2, Yi-Ping Hung a,c,*, Chiou-Shann Fuh a,3 a Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan Department of Computer Science and Information Engineering, National Chi Nan University, Nantou, Taiwan c Institute of Information Science 20, Academia Sinica, 128 Academia Road, Section 2, Nankang, Taipei 115, Taiwan b Received 12 May 2000; received in revised form 13 April 2001; accepted 6 June 2001 Abstract In this work, three servo motors are used to independently control the aperture, zoom, and focus of our zoom lens. Our goal is to calibrate, ef®ciently, the camera parameters for all the possible con®gurations of lens settings. We use a calibration object suitable for zoom lens calibration to deal with the defocusing problem. Instead of calibrating the zoom lens with respect to the three lens settings simultaneously, we perform the monofocal camera calibration, adaptively, over the ranges of the zoom and focus settings while ®xing the aperture setting at a preset value. Bilinear interpolation is used to provide the values of the camera parameters for those lens settings where no observations are taken. The adaptive strategy requires the monofocal camera calibration only for the lens settings where the interpolated camera parameters are not accurate enough, and is hence referred to as the calibration-on-demand method. Our experiments show that the proposed calibrationon-demand method can provide accurate camera parameters for all the lens settings of a motorized zoom lens, even though the camera calibration is performed only for a few sampled lens settings. q 2001 Elsevier Science B.V. All rights reserved. Keywords: Camera calibration; Lens calibration; Motorized zoom lens; Camera parameters; Active vision 1. Introduction Motorized zoom lenses have great potential in the applications of active vision [1±3], three-dimensional reconstruction [4±6], and visual tracking [7,8]. In such applications, the aperture, zoom, and focus of the lens can be adjusted to different lighting conditions or to the desired ®eld of view, depth of ®eld, spatial resolution, or focused distance. Although a motorized zoom lens is more ¯exible and useful than a monofocal lens, it is not an easy job, in general, to calibrate a motorized zoom lens. The goal of the motorized zoom lens calibration is to determine the relationship between the lens settings (control parameters for the driving motors) and the camera parameters (CPs). Unfortunately, a motorized zoom lens usually consists of some compound lens groups, such as the focusing lens group, the variator lens group, the * Corresponding author. Address: Institute of Information Science 20, Academia Sinica, 128 Academia Road, Section 2, Nankang, Taipei 115, Taiwan. Tel.: 1886-2-27883799 ext. 1718; fax: 1886-227824814. E-mail addresses: firstname.lastname@example.org (Y.-S. Chen), stone@ csie.ncnu.edu.tw (S.-W. Shih), email@example.com (Y.-P. Hung), firstname.lastname@example.org (C.-S. Fuh). 1 Tel.: 1886-2-27883799 ext. 1518; fax: 1886-2-27824814. 2 Tel.: 1886-49-2910960 ext. 4832; fax: 1886-49-2915226. 3 Tel.: 1886-2-23625336 ext. 327; fax: 1886-2-23628167. compensator lens group, and the relay lens group [9±11]. There is no accurate model that describes the paths of the rays passing through the lens. Moreover, various mechanical assembly and driving motors can be used to adjust the aperture, zoom and focus. Consequently, the relationship between the lens settings and the CPs becomes quite complicated . One way to determine the relationship between the lens settings and the CPs is to perform the monofocal camera calibration for each lens setting. That is, each con®guration of a zoom lens can be treated as a monofocal lens and its CPs can be calibrated by using well-known camera calibration techniques, such as  and . However, this method is extremely inef®cient and hardly ever feasible because a motorized zoom lens can usually have a large number of con®gurations. For example, there will be one million con®gurations when there are one hundred settings for each of the three motors controlling the aperture, zoom, and focus. In the past, Tarabanis et al.  used a special optical bench to calibrate a zoom lens. They constructed a sparse table storing the CPs calibrated for the sampled lens settings. CPs for other lens settings can be obtained via interpolation. The zoom lens they used was mounted on a robot gripper. In Ref. , they also presented a hand-eye calibration method. 0262-8856/01/$ - see front matter q 2001 Elsevier Science B.V. All rights reserved. PII: S 0262-885 6(01)00069-5 1100 Y.-S. Chen et al. / Image and Vision Computing 19 (2001) 1099±1110 Willson and Shafer [12,16] also developed zoom lens calibration techniques. They calibrated two zoom lenses driven by microstepping motors. First, they used an autocollimated laser for locating the image center. Then, Tsai's method  was applied to calibrate the eleven CPs for 25 and 121 regularly sampled lens settings of the two zoom lenses, respectively. For each CP, they approximately modeled the relationship between the CP and the lens settings (zoom and focus) with a bivariate polynomial function. The calibrated CP values of the sampled lens settings were then used to determine the coef®cients of the polynomial functions. These polynomial functions can provide CPs for continuous ranges of the lens settings. The order of the polynomial function depends on the lens used and it has been determined empirically in Willson and Shafer's work (from order zero to order ®ve) . One of the major advantages of this method is that the memory consumption is very ef®cient. Only the coef®cients of the polynomial functions need to be stored. Moreover, these functions could give very accurate CPs according to their experiments, in which average prediction error of less than 0.14 pixels was achieved. In this work, we designed and built a motorized zoom lens with computer-controlled aperture, zoom, and focus. In order to have faster response for the visual surveillance application, we used servo motors, instead of microstepping motors as in Refs. [3,12], for driving the aperture, zoom, and focus of the zoom lens. Also, we designed a calibration object to accommodate various spatial resolutions and ®eld of views, which are unique to zoom lens calibration. The image of the calibration object is often blurred, especially when the depth of ®eld is small. Centroid estimation for the image coordinates of the calibration points was used for dealing with this defocusing problem. In this paper, we ®xed the aperture setting and considered only the zoom and the focus settings when determining the relationship between the lens settings and the CPs. For representing this relationship, a two-dimensional table was adaptively constructed by using a simple and ef®cient method. This table is sparse and each entry stores the calibrated values of the variable CPs as well as the corresponding residual error of calibration. For those lens settings with respect to which the table has not recorded the corresponding CPs, the desired values of the CPs can be obtained via interpolation. To reduce the calibration time, the lens settings for which the camera calibration needs to be performed were adaptively determined according to the accuracy of the CP values interpolated by using the table. Our experimental results have demonstrated that the table constructed by using our calibration-on-demand method can well represent the relationship between the lens settings and the CPs. This table provides accurate CPs of any lens setting for many applications. This paper is organized as follows. We introduce the camera models of the monofocal and zoom lenses in Sections 2.1 and 2.2, respectively. In Section 3, we propose a simple and ef®cient adaptive method, the calibration-on- Fig. 1. The perspective projection geometry with the pinhole camera model. demand method, for calibrating a motorized zoom lens. Section 4 shows the experimental results, and Section 5 gives the conclusions. 2. Camera model 2.1. Monofocal lens Given a 3-D point in the world coordinate system, the camera model and the associated CPs can predict the projected 2-D image coordinates of the 3-D point in the image coordinate system. For a monofocal lens, the pinhole camera model with radial distortion is considered here. As shown in Fig. 1, the 3-D world coordinates xw ; yw ; zw of a point P can be transformed into the 3-D camera coordinates xc ; yc ; zc as follows: 2 3 2 3 xc xw 6 7 6 7 6 yc 7 R6 yw 7 1 t; 1 4 5 4 5 zc zw where R is a 3 £ 3 rotation matrix that is a function of the three CPs: u x, u y, and u z (X±Y±Z Euler angles), and t tx ; ty ; tz T is a translation vector from the origin of the world coordinate system to the origin of the camera coordinate system. In the camera coordinate system, the origin, Oc, is located at the perspective center and the Z axis is aligned with the optical axis of the lens. By using perspective projection, the coordinates xI ; yI in the image plane is given by the equations x xI f c ; 2 zc yI f yc ; zc 3 where f is the effective focal length. Then, without considering lens distortion, the coordinates xI ; yI can be transformed into uI ; vI in the image coordinate system by using the equations uI 2 u0 su xI ; 4 vI 2 v0 sv yI ; 5 where su and sv are the horizontal and vertical pixel width Y.-S. Chen et al. / Image and Vision Computing 19 (2001) 1099±1110 and u0 ; v0 is the image coordinates of the piercing point where the optical axis pierces the image plane. If the lens distortion is considered, Eqs. (4) and (5) become nonlinear. In this work, only the ®rst coef®cient of radial distortion, k , is considered [13,14], and Eqs. (4) and (5) should be replaced by: 1 2 kr2 u^ I 2 u0 su xI ; 2 1 2 kr v^I 2 v0 sv yI ; 6 7 where u^ I ; v^I is the actual image coordinates measured in the image and r2 u^I 2 u0 su 2 1 v^I 2 v0 sv 2 : Totally, there are twelve CPs in this camera model: u0, v0, su, sv, f, k , u x, u y, u z, tx, ty ; and tz : Among these CPs, the vertical pixel width, sv, can be obtained from the speci®cation of the CCD camera. The remaining eleven CPs can be estimated by using Weng et al.'s method , provided that a set of known calibration points are observed in the image. In the following, the 3-D world coordinates of the calibration points, xw ; yw ; zw ; and their corresponding image measurements, u^I ; v^I ; will be referred to as the calibration data. 2.2. Zoom lens 2.2.1. Aperture setting Ideally, we would like to have a zoom lens whose aperture setting does not affect the physical values of the CPs. However, practical lenses suffer a variety of aberrations. Some of the aberrations, such as spherical aberration, coma, and astigmatism, become more serious when the size of the aperture stop is larger [9±11,17]. Furthermore, trying to correct the spherical aberration will usually result in the zonal aberration, which may lead to the focus shift phenomenon where the focal plane moves along the optical axis when adjusting the aperture setting [10,11]. Therefore, changing the aperture setting alone may alter the value of the focal length, even though the zoom and the focal settings remain unchanged. Also, the aberrations may affect the quality of the acquired image and hence the accuracy of the estimated image coordinates of the calibration points. As a result, the aperture setting may indeed in¯uence the estimated values of the CPs. However, some researchers have found that the in¯uence of the aperture setting is negligible for many applications. For example, Li and Lavest  reported that the aperture setting does not change the focal length signi®cantly. Also, our experiment described in Ref.  showed that the aperture setting did not much affect the estimates of the CPs. Hence, in this paper, we have chosen to ®x the aperture setting while calibrating our motorized zoom lens. With this simpli®cation, the obtained CPs can be stored in a two-dimensional table indexed by the zoom and focus settings, instead of a three-dimensional table indexed by the zoom, focus, and aperture settings. Even if the in¯uence of the aperture setting is not negligible, the variation of the 1101 CPs, when adjusting the aperture setting, is usually quite smooth, as described in Ref. . In that case, we can simply choose a couple of aperture settings and repeat our adaptive calibration method described in Section 3. Another possibility is to extend the proposed method to the additional axis of the aperture setting. 2.2.2. Zoom and focus settings Once an aperture setting is chosen and ®xed, our goal is to determine the relationship between the CPs and the zoomfocus setting. For each combination of the zoom and focus settings, the zoom lens can be treated as a monofocal lens and its CPs can be calibrated individually. When we adjust the zoom or the focus setting, some of the CPs remain unchanged during the adjustment. For example, the pixel width, su and sv, and the extrinsic CPs, u x, u y, u z, tx, ty ; and tz, are not supposed to vary. Hence, once these CPs are determined with the initial zoom lens setting, they can be ®xed during the remaining procedure of zoom lens calibration. When adjusting the zoom or the focus setting, the focal length will change accordingly. Thus, f and k should be estimated for each combination of the zoom and focus settings. Furthermore, the lens groups of the zoom lens are moved forward and backward along the optical axis when we adjust the zoom or the focus setting. This longitudinal movement will make the perspective center, i.e. the origin of the camera coordinate system move along the optical axis relative to the ®xed image plane. Consequently, we have to estimate the displacement, Dtz ; of the perspective center on the Z axis for different zoom and focus settings. Besides, the lens groups will be moved back and forth by gradual rotation during the adjustment. If the optical axis deviates from the rotation axis, the image coordinates of the piercing point, u0 and v0, may change and they should also be estimated for each combination of the zoom and focus settings. To achieve a smaller 2-D residual error, all the twelve CPs can be freely optimized during the calibration procedure. However, this will increase the calibration effort and the memory storage of the table. Furthermore, smaller 2-D residual errors do not necessarily result in more accurate estimates of the physical camera parameters. Therefore, our method estimates only ®ve camera parameters, u0, v0, f, k , and Dtz ; in the zoom lens calibration procedure, which will be referred to as the variable camera parameters (VCPs), and ®xes all the remaining CPs to be those determined with the initial zoom lens setting. 3. Zoom lens calibration 3.1. Calibration object There are two major issues when designing the calibration object: the structure of the calibration object and the 1102 Y.-S. Chen et al. / Image and Vision Computing 19 (2001) 1099±1110 calibration patterns to be measured. To accurately estimate the CPs, the calibration data has to be obtained by having the calibration points at different distances from the lens, rather than just at the precisely focused distance. Therefore, the calibration object is usually made of a cube containing calibration patterns on several of its surfaces [5,20] or a calibration plate mounted on a translation stage [13,14,21]. The calibration patterns on the calibration object can be grid lines , squares [13,14], or circles [20,21]. Image coordinates of the calibration points can be obtained by measuring the intersections, corners, or centroids of the calibration patterns in the image. 3.1.1. Defocusing problem caused by aperture stop Without considering aberrations, the points sitting on the focused plane can be projected onto sharp image points in the image plane. Other points deviated from the focused plane will be imaged as ®lled circles (or, to be more precise, as the shape of the aperture stop) and the acquired image is blurred. In this case, it is not easy to accurately estimate the image coordinates, u^ I ; v^I : This defocusing problem is more serious when we use a larger focal length or a larger size of the aperture stop, due to the resulted smaller depth of ®eld. For zoom lens calibration, this defocusing problem becomes even more serious due to the combination of the following three requirements: (1) accurate CP estimation requires calibration data obtained at different distances, rather than just at the focused distance; (2) camera calibration has to be performed for different focus settings, which will make the focused distances deviate from the calibration object and result in blurred images; and (3) camera calibration has to be performed for different zoom settings, while larger focal length will result in shorter depth of ®eld, thus blur the image. When the blurring is symmetric, we can still obtain accurate image coordinates, u^ I ; v^I ; of the calibration points from the blurred image. Let u^I ; v^I be where in the image plane the chief ray projects. The chief ray is de®ned as the ray passing through the center of the aperture stop. Starting from an out-of-focus point object, the chief ray always project on the center of the circle in the image plane when the aperture stop is at its ideal location. That is, the aperture stop is perpendicular to the optical axis and its center is located at the optical center. For different aperture settings, the projected circles are of different sizes but are all concentric. If the center of the circle is measured as the image coordinates, u^I ; v^I ; these coordinates will not vary with the size of the circle. Consequently, the defocusing problem due to the aperture stop will not in¯uence the calibration result, when the aperture stop is ideally located and none of the aberrations are considered. For a practical lens, the center of the aperture stop may not locate at the optical center. The estimates of the CPs can be slightly changed when adjusting the aperture stop. More discussions is given in Appendix A. Fig. 2. This ®gure shows the calibration object we used in this work, which is a calibration plate mounted on a translation stage. The translation direction of the stage is perpendicular to the calibration plate. The calibration plate is a dark board containing 25 £ 25 white circles. The diameters of the circles are 15 mm except for the 24 smaller circles in the middle, whose diameters are 10.5 mm. The distances between the centers of the neighboring circles are 25 mm. 3.1.2. Design of calibration object In this work, we constructed a black calibration plate mounted on a computer-controlled translation stage, as shown in Fig. 2. The normal direction of the calibration plate, the translation direction of the stage, and the direction of the optical axis of the lens are aligned to be parallel. The calibration plate can be moved along the direction of the stage such that the calibration data measured at different distances can be obtained. On the calibration plate, there are many circles used as the calibration patterns. Each circle projected in the image plane will be symmetrically blurred if the circle is out of focus. Edge detection followed by circle ®tting is not suitable to estimate the circle center because there is no sharp edge around the blurred circle. Fortunately, accurate image coordinates of the circle center can be obtained by measuring the centroid of the circle in the image, as mentioned in Section 3.1.1. For better accuracy of camera calibration, the calibration patterns had better be distributed all over the image and the number of the calibration patterns appearing in the image should be as many as possible . For a zoom lens, it is hard to meet these requirements because of the varying ®eld of view and spatial resolution during the adjustment of the Y.-S. Chen et al. / Image and Vision Computing 19 (2001) 1099±1110 1103 of the circle, can be easily estimated by using the following image processing techniques: image binarization, blob analysis, and centroid estimation. As the ®eld of view varies, part of the calibration plate may become invisible in the image. However, we still have to identify the circles appearing in the image in order to obtain their corresponding 3-D world coordinates. For easy identi®cation, the larger circle in the middle, which is surrounded by twenty-four smaller circles, is chosen to be the ®ducial circle. The origin of the world coordinate system, Ow, is located at the center of the ®ducial circle. The 3-D coordinates of the other circles are then described with respect to this origin. If the camera is properly aligned such that the ®ducial circle always appear in the image for all the zoom settings, the task of locating the ®ducial circle in the image can be easily accomplished by utilizing the sizes of the circles. Each circle appearing in the image can then be identi®ed and the calibration data, including 3-D world coordinates and 2-D image coordinates, can be obtained. Notice that if the lens distortion is large enough to severely bias the estimate of the centroid position, one should rectify the image by using the initially estimated lens distortion coef®cients before estimating the centroid position. The initial estimates of the lens distortion coef®cient, together with those of the other CPs, are obtained by using the estimates of the centroid positions without rectifying the images. The recti®cation±estimation procedure may have to be repeated a few times until convergence. However, for the motorized zoom lens used in our experiments, we simply use the initial estimates of the CPs because the lens distortion is not severe enough to adopt the above iterative procedure. Fig. 3. Images of the calibration plate acquired with a zoom lens when its zoom is set to be (a) wide-angle and (b) telephoto. zoom setting. To remedy this problem, circles of two different sizes were used as shown in Fig. 2. When the zoom of the lens is set to be wide-angle, many larger circles as well as the smaller circles in the middle of the calibration plate appear in the image, as shown in Fig. 3(a). These larger circles can provide more accurate centroid positions estimated in the image. When the zoom of the lens, on the other hand, is set to be telephoto, only a few smaller circles in the middle of the calibration plate appear in the image, as shown in Fig. 3(b). These smaller circles still can provide accurate centroid positions due to larger magni®cation. Instead of crowding more small circles in the middle region of the calibration plate, we put the same number of circles, per unit area, as what was put in the outside region, to leave larger gap between the circles. This will prevent the neighboring circles from overlapping each other when the out-offocus circles are seriously blurred in the image. 3.1.3. Image coordinates estimation for calibration patterns Image coordinates of the calibration point, i.e. the center 3.2. Zoom lens calibration procedure This section describes the proposed procedure of motorized zoom lens calibration. For the zoom lens used in our experiments, there are three thousand steps for both the zoom and the focus motors, and totally nine million lens settings in combination. To reduce the number of the monofocal camera calibration performed and the storage required to record the VCPs, we have adopted a sampling and interpolation technique over the entire working range of the lens settings. Our goal is then to create a two-dimensional table, indexed by the values of the lens settings, to store the VCPs estimated at the sampled lens settings. The VCPs for unsampled lens settings, that is, where the monofocal camera calibration has not been performed, can be approximated via bilinear interpolation. Because the values of the VCPs usually vary nonlinearly with respect to the lens settings, a uniformly sampled table might not be able to ef®ciently represent the relationship between the VCPs and the lens settings. A more ef®cient sampling strategy is to perform the monofocal camera calibration only when the bilinear interpolation cannot give an accurate enough result. This strategy leads to an adaptive 1104 Y.-S. Chen et al. / Image and Vision Computing 19 (2001) 1099±1110 Fig. 4. This ®gure shows an example of the sample positions. The number in the table means the order at which the camera calibration is performed. method, referred to as the calibration-on-demand method, which allows adaptive sample positions and locally varying sampling rates. That is, each entry of the VCP table will be adaptively created, by trial and error, to store the VCPs estimated at the sampled lens setting and the corresponding calibration residual error. Our criterion for entry creation is based on the residual error. In the following, we denote the residual error, e, as the function g(´), which calculates the average difference between the predicted and measured image coordinates of the calibration points. That is, e g I; b; z; f ; where z denotes the zoom setting, f the focus setting, I the image set used for measuring the image coordinates of the calibration points, and b denotes the VCPs used for predicting the image coordinates of the calibration points. There are three kinds of residual error used in our zoom lens ^ calibration procedure. The ®rst one is the residual error e; where e^ g I; b^ ; z; f : In this case, the VCPs b^ are calculated by performing the monofocal camera calibration in which the required image coordinates of the calibration points are measured in the image set I. The second one is the residual error e~ g I; b~ ; z; f : Here, the VCPs b~ are interpolated values computed by using the values of the VCPs that have already been stored in the table. The last one is the residual which is the interpolated values by directly using the error e; values of the residual error e^ stored in the table. Fig. 4 gives an example of how we determine the sample positions. At ®rst, we adjust the aperture stop to match the lighting condition and then ®x it in the subsequent calibration. Then, we set the zoom setting to be the middle value within its range, ªZSTART ; ZEND ;º that is, ZSTART 1 ZEND =2: Next, the calibration plate is placed at an appropriate distance and the focus setting is adjusted within its entire working range, FSTART ; FEND ; until the image of the calibration plate becomes the sharpest. This focus setting is referred to as FREF. A monofocal camera calibration is then performed (as illustrated by sample position number 0 in Fig. 4) to obtain all the CPs. During the following zoom lens calibration procedure, eight CPs, that is, su, sv, u x, u y, u z, tx, ty, and tz, are ®xed and the remaining ®ve VCPs, u0, v0, f, k , Dtz ; are to be calibrated. For each zoom setting to be calibrated, the monofocal camera calibration is ®rst performed at the start and end of the focus setting. For example, when the zoom setting is set at ZSTART at the next stage, the start and end of the focus setting are FSTART and FEND, as illustrated by sample positions number 1 and 2 in Fig. 4. Both the obtained VCPs b^ and the residual error e^ will be stored in the table. Next, the focus setting is set to the middle value of its range. The VCPs b~ at this focus setting can be computed by interpolation with the VCPs obtained previously at the start and end of the focus setting, but we have to evaluate whether this interpolated VCPs, b~ ; are accurate enough. To evaluate the accuracy of the interpolation, we ®rst acquire the images of the calibration plate and estimate the 2-D image coordinates of the calibration points observed in those images. Then, we compute the residual error e~ between the observed image coordinates and the predicted positions calculated by using the interpolated VCPs, b~ ; that is, e~ g I; b~ ; z; f ; where z and f are set at sample position number 3 shown in Fig. 4. Large e~ implies that the interpolated VCPs, b~ ; may not be accurate enough, which means a monofocal camera calibration had better be performed at sample position number 3. In our calibration-on-demand method, the monofocal camera calibration is performed only when the residual error e~ is larger than where kf is a scaling factor and e is the a threshold, kf e; residual error computed via interpolation with the values of the residual error e^ obtained at the start and end focus settings. The above procedure is recursively repeated for each middle focus setting between two calibrated focus settings (for example, see sample positions number 4±8 in Fig. 4, where the zoom setting is set to be ZEND) until the interpolated VCPs, b~ ; are accurate enough, that is, e~ is small enough. Notice that the decision of whether e~ is which varies for different small enough depends on e; focus settings. For the dimension of the VCP table along the zoom setting, we adopt an adaptive sampling strategy similar to the one used for the focus setting. Initially, the VCPs corresponding to all the focus settings at z ZSTART and z ZEND are calibrated by using the procedure described in the previous paragraph, which is illustrated by the sample positions in the ®rst and the last columns shown in Fig. 4. Next, the zoom setting is set to be ªzSTART 1 zEND =2º and the focus setting is initially set to be FREF. Then, the images of the calibration plate are acquired and the observed 2-D image coordinates of the calibration points are estimated. With the observed 2-D image coordinates, the residual error e~ can be computed by using the interpolated VCPs, b~ ; computed by using the VCPs recorded in the ®rst and last columns in the table. The interpolated VCPs, b~ ; will be considered as accurate where kz is a scaling enough if e~ is small, that is, e~ , kz e; factor. Otherwise, a monofocal camera calibration will be performed to obtain more accurate VCPs. Again, the Y.-S. Chen et al. / Image and Vision Computing 19 (2001) 1099±1110 1105 above procedure is repeated recursively for each middle zoom setting between two calibrated zoom settings until the interpolated VCPs are accurate enough, that is, e~ is small enough. The proposed procedure for motorized zoom lens calibration is summarized below. Zoom lens camera calibration procedure intrinsic-camera-calibration z; f begin set the lens settings z; f and perform the monofocal camera calibration record the VCPs b^ (u0, v0, f, k , and Dtz ) and residual error e^ in the table with indices z; f end procedure focus-calibration fstart ; fend begin focus setting f : fstart 1 fend =2 interpolate the VCPs, b~ ; from those of fstart and of fend evaluate the residual error e~ if e~ is not small enough begin intrinsic-camera-calibration z; f focus-calibration fstart ; f focus-calibration f ; fend end end procedure zoom-calibration zstart ; zend begin focus setting f : FREF zoom setting z : zstart 1 zend =2 interpolate the VCPs, b~ ; from those of zstart and of zend evaluate the residual error e~ if e~ is not small enough begin intrinsic-camera-calibration z; FSTART intrinsic-camera-calibration z; FEND focus-calibration FSTART ; FEND zoom-calibration zstart ; z zoom-calibration z; zend end end begin perform the initial camera calibration to obtain the CPs to be ®xed: su, sv, u x, u y, u z, tx, ty ; and tz intrinsic-camera-calibration ZSTART ; FSTART intrinsic-camera-calibration ZSTART ; FEND focus-calibration FSTART ; FEND intrinsic-camera-calibration ZEND ; FSTART intrinsic-camera-calibration ZEND ; FEND focus-calibration FSTART ; FEND zoom-calibration ZSTART ; ZEND end Fig. 5. This ®gure shows the motorized zoom lens used in our experiments, which is a Fujinon TV zoom lens H6 £ 12.5R with its aperture, zoom, and focus driven by three servo motors through pushrod links. 4. Experimental results 4.1. Motorized zoom lens In this work, we constructed a motorized zoom lens by using a Fujinon TV zoom lens H6 £ 12.5R, as shown in Fig. 5. Three servo motors were used for driving the aperture, zoom, and focus of the lens through pushrod links. We adopted servo motor because of its faster response. Each servo motor contains three thousand steps, indicated by the optical encoder. We controlled these servo motors with a PC through the Advantech PCL-832 control cards. To avoid the hysteresis problem  due to the backlash, we always set the motor to the desired setting in the same direction. That is, we ®rst drive the motor to the setting a little smaller than the desired one and then drive it to the desired setting. 4.2. Constructing and assessing the table In our experiments, the proposed calibration-on-demand method was used to calibrate the motorized zoom lens described in Section 4.1. In the ®rst experiment, the parameters kf and kz, which determine the sampling density in the focus and zoom axes, were set to be 1.5 and 2, respectively. A table of 120 entries was created and the sample positions are shown in Fig. 6. The VCPs stored in the table are shown in Fig. 7, where tz0 is the value of the ®xed CP, tz, obtained in the initial camera calibration. The mean residual error e^ for the acquired table is 0.19 pixels. To evaluate the repeatability of the motorized zoom lens, we repeatedly set the zoom and focus settings to each sampled settings previously recorded in the table and then took the images of the calibration palate to check how large the residual error e^ became. In this repeatability experiment, the mean residual error e~ did increase to 0.44 pixels. However, if we randomly set the focus and zoom settings, which are unlikely to be the sample positions recorded in the table, the residual error e~ does not increase much further. For example, Fig. 8 shows the residual error e~ of four 1106 Y.-S. Chen et al. / Image and Vision Computing 19 (2001) 1099±1110 Fig. 6. The sample positions with respect to the zoom and focus settings. Fig. 7. The VCPs b^ and the residual error eÃ stored in the table. Y.-S. Chen et al. / Image and Vision Computing 19 (2001) 1099±1110 1107 Fig. 8. The residual error e~ of repeatability and interpolation are shown in dotted and solid lines, respectively. hundred random trials, and the mean residual error e^ is 0.475 pixels, which is only 8% increase compared to the residual error e~ caused by repeatability error. Another experiment was conducted by using different set of kf and kz, which were set to be 2 and 3, respectively. In this case, a table of smaller size (52 entries) was created at the expense of larger error, where the residual error e~ was 0.54 pixels (that is, a 22.7% increase). 5. Conclusions In this paper, we have proposed a calibration-on-demand procedure for calibrating a motorized zoom lens, which can adaptively create a table for representing the relationship between the lens settings and the camera parameters. The major advantage of the proposed method is in its ability to reduce, in an adaptive manner, the amount of image acquisition and monofocal camera calibration, which is a timeconsuming task, while maintaining the required calibration accuracy. Our experiments have shown that the proposed method can provide accurate intrinsic camera parameters for each lens setting of a motorized zoom lens by performing a much smaller number of monofocal camera calibrations. Our experiment has shown that the average residual error of the camera parameters given by the acquired table (having only 120 entries) is less than half pixels. Another contribution of this work is that we have proposed a calibration object which consists of circles distributing on a calibration plate. These circles are of different sizes and have been designed to accommodate various spatial resolutions and ®eld of views, which are unique problems to zoom lens calibration. Another advantage of this calibration object is that accurate image coordinates of the calibration points can be obtained by estimating the centroids of the circles, even when the circles are blurred due to the defocusing problem. Appendix A Here, we analyze the in¯uence of the aperture stop on the resulted images. For an out-of-focus point object, the position of the aperture stop and the aperture setting in¯uence the size and position of the resulted circle in at least three ways: 1. Off optical center. A zoom lens consists of a few lens groups, and the aperture stop is located somewhere between lens groups. As shown in Fig. 9, if the center of the aperture stop is away from the optical center but still on the optical axis, the aperture stop may asymmetrically shield the pencil of rays, which may then shift the center of the circle. 2. Off optical axis. As shown in Fig. 10, if the optical axis does not pass through the center of the aperture stop, the 1108 Y.-S. Chen et al. / Image and Vision Computing 19 (2001) 1099±1110 Fig. 9. When the center of the aperture stop is away from the optical center but still on the optical axis, the aperture stop asymmetrically shields the pencil of rays and the center of the resulted circle is shifted. Fig. 10. When the center of the aperture stop does not pass the optical axis, the aperture stop asymmetrically shields the pencil of rays and the center of the resulted circle is shifted. aperture stop may asymmetrically shield the pencil of rays, which may then shift the center of the circle. 3. Vignetting. As shown in Fig. 11, if the size of the aperture stop is too large, the lens mount or other lens groups may asymmetrically shield the pencil of rays. This phenomenon is called vignetting [9,11]. Not only the center but also the shape of the resulted circle are changed. To sum up, each out-of-focus point is projected onto a ®lled circle in the image plane. If the center of the aperture stop is not ideally located at the optical center, or if the vignetting phenomenon occurs, the center of the circle will not be the image position where the chief ray projects. Moreover, the shape of the circle will not be symmetric when the vignetting phenomenon occurs. In the following, we will quantitatively analyze the in¯uence of the aperture stop on the projected image position, without considering vignetting. As shown in Fig. 12, pencil of rays from a point light source pass through the lens with Fig. 11. When the size of the aperture stop is too large, the lens mount may asymmetrically shields the pencil of rays. Not only the center but also the shape of the resulted circle are changed. Y.-S. Chen et al. / Image and Vision Computing 19 (2001) 1099±1110 1109 coordinates of the calibration data be enlarged or shrunk with a constant ratio k. This is equivalent to elongating or shortening the effective focal length, f, of the CPs. However, this ratio k depends on c. For the points at different distances, the positions of their projected circles are shifted in different ratios. This makes zoom lens calibration harder when the calibration data is obtained at different distances. References Fig. 12. We quantitatively analyze the movement of the projected circles with or without shielding by the aperture stop. optical center O. When the aperture stop is open, this point light source projects in the image plane as a circle between p1 and p5 centered at p3. After shielding by the aperture stop, the resulted circle is between p2 and p4. Let the origin be the piercing point in the image plane. The vertical coordinates, yp1 ; ¼; yp5 ; of the points p1 ; ¼; p5 are: yp1 c tan u 2 c tan u 1 R c 2 b; c yp2 c tan u 2 c tan u 1 r1 c 2 b; c2a yp3 c tan u 2 tan u c 2 b b tan u; yp4 c tan u 2 ctan u 2 r2 c 2 b; c2a yp5 c tan u 2 c tan u 2 R c 2 b; c where R is the radius of the lens, r1 and r2 are the distances from the top and the bottom of the aperture stop to the optical axis, u is the angle between the optical axis and the ray emerging from O, a, b, and c are the distances from the lens to the aperture stop, the image plane, and the converging point of the pencil of rays from the point light source, respectively. Assume that the center of the aperture stop is on the optical axis, i.e. r1 r2: The center p 03 of the circle between p2 and p4 can be estimated in the image as the middle point between p2 and p4: y 0p3 c b 2 a tan u k £ yp3 ; c2a where k c b 2 a=b c 2 a: Notice that k is irrelevant to u . That is, the aperture stop will cause the estimated image  E.P. Krotkov, Exploratory Visual Sensing for Determining Spatial Layout with an Agile Stereo Camera System, PhD thesis, University of Pennsylvania, Pennsylvania, April 1987, Technical Report MS-CIS-87-29.  K. Pahlavan, Active Robot Vision and Primary Ocular Processes, PhD thesis, Royal Institute of Technology, Stockholm, Sweden, May 1993.  S.-W. Shih, Y.-P. Hung, W.-S. Lin, Calibration of an active binocular head, IEEE Transactions on Systems, Man, and Cybernetics 28 (4) (1998) 426±442.  A.L. Abbott, Dynamic Integration of Depth Cues for Surface Reconstruction from Stereo Images, PhD thesis, University of Illinois at Urbana-Champaign, Illinois, January 1991.  J.-M. Lavest, G. Rives, M. Dhome, Three-dimensional reconstruction by zooming, IEEE Transactions on Robotics and Automation 9 (2) (1993) 196±207.  G. Surya, M. Subbarao, Depth from defocus by changing camera aperture: A spatial domain approach, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, New York, June 1993, pp. 61±67.  K. Hosoda, H. Moriyama, M. Asada, Visual servoing utilizing zoom mechanism, Proceedings of the International Conference on Robotics and Automation, Nagoya, Aichi, Japan, May 1995, pp. 178±183.  J.A. Fayman, O. Sudarsky, E. Rivlin, Zoom tracking, Proceedings of the International Conference on Robotics and Automation, Leuven, Belgium, May 1998, pp. 2783±2788.  R. Kingslake, Lens Design Fundamentals, Academic Press, San Diego, 1978.  R. Kingslake, A History of the Photographic Lens, Academic Press, San Diego, 1989.  W.J. Smith, Modern Lens Design, McGraw-Hill, New York, 1992.  R.G. Willson, Modeling and Calibration of Automated Zoom Lenses, PhD thesis, Carnegie Mellon University, Pittsburgh, Pennsylvania, January 1994, Technical Report CMU-RI-TR-94-03.  R.Y. Tsai, A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses, IEEE Journal of Robotics and Automation RA-3 (4) (1987) 323±344.  J. Weng, P. Cohen, M. Herniou, Camera calibration with distortion models and accuracy evaluation, IEEE Transactions on Pattern Analysis and Machine Intelligence 14 (10) (1992) 965±980.  K. Tarabanis, R.Y. Tsai, D.S. Goodman, Calibration of a computer controlled robotic vision sensor with a zoom lens, CVGIP: Image Understanding 59 (2) (1994) 226±241.  R.G. Willson, S.A. Shafer, Precision imaging and control for machine vision research at Carnegie Mellon University, Proceedings of SPIE Conference on High-Resolution Sensors and Hybrid Systems, San Jose, CA, February 1992, vol. 1656, pp. 297±314.  C.C. Slama (Ed.), Manual of Photogrammetry 4th ed, American Society of Photogrammetry, 1980. Location: Falls Church, Virginia.  M. Li, J.-M. Lavest, Some aspects of zoom lens camera calibration, IEEE Transactions on Pattern Analysis and Machine Intelligence 18 (11) (1996) 1105±1110.  Y.-S. Chen, S.-W. Shih, Y.-P. Hung, C.-S. Fuh, Camera calibration 1110 Y.-S. Chen et al. / Image and Vision Computing 19 (2001) 1099±1110 with a motorized zoom lens, Proceedings of the International Conference on Pattern Recognition, Barcelona, Spain, September 2000, vol. 4, pp. 495±498.  J. HeikkilaÈ, O. SilveÂn, A four-step camera calibration procedure with implicit image correction, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Puerto Rico, June 1997, pp. 1106±1112.  S.-W. Shih, Kinematic and Camera Calibration of Recon®gurable Binocular Vision Systems, PhD thesis, National Taiwan University, Taipei, Taiwan, June 1996.