Simple and efficient method of calibrating a motorized zoom lens

Image and Vision Computing 19 (2001) 1099±1110
www.elsevier.com/locate/imavis
Simple and ef®cient method of calibrating a motorized zoom lens
Yong-Sheng Chen a,c,1, Sheng-Wen Shih b,2, Yi-Ping Hung a,c,*, Chiou-Shann Fuh a,3
a
Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan
Department of Computer Science and Information Engineering, National Chi Nan University, Nantou, Taiwan
c
Institute of Information Science 20, Academia Sinica, 128 Academia Road, Section 2, Nankang, Taipei 115, Taiwan
b
Received 12 May 2000; received in revised form 13 April 2001; accepted 6 June 2001
Abstract
In this work, three servo motors are used to independently control the aperture, zoom, and focus of our zoom lens. Our goal is to calibrate,
ef®ciently, the camera parameters for all the possible con®gurations of lens settings. We use a calibration object suitable for zoom lens
calibration to deal with the defocusing problem. Instead of calibrating the zoom lens with respect to the three lens settings simultaneously, we
perform the monofocal camera calibration, adaptively, over the ranges of the zoom and focus settings while ®xing the aperture setting at a
preset value. Bilinear interpolation is used to provide the values of the camera parameters for those lens settings where no observations are
taken. The adaptive strategy requires the monofocal camera calibration only for the lens settings where the interpolated camera parameters
are not accurate enough, and is hence referred to as the calibration-on-demand method. Our experiments show that the proposed calibrationon-demand method can provide accurate camera parameters for all the lens settings of a motorized zoom lens, even though the camera
calibration is performed only for a few sampled lens settings. q 2001 Elsevier Science B.V. All rights reserved.
Keywords: Camera calibration; Lens calibration; Motorized zoom lens; Camera parameters; Active vision
1. Introduction
Motorized zoom lenses have great potential in the applications of active vision [1±3], three-dimensional reconstruction [4±6], and visual tracking [7,8]. In such
applications, the aperture, zoom, and focus of the lens can
be adjusted to different lighting conditions or to the desired
®eld of view, depth of ®eld, spatial resolution, or focused
distance. Although a motorized zoom lens is more ¯exible
and useful than a monofocal lens, it is not an easy job, in
general, to calibrate a motorized zoom lens.
The goal of the motorized zoom lens calibration is to
determine the relationship between the lens settings (control
parameters for the driving motors) and the camera
parameters (CPs). Unfortunately, a motorized zoom lens
usually consists of some compound lens groups, such as
the focusing lens group, the variator lens group, the
* Corresponding author. Address: Institute of Information Science 20,
Academia Sinica, 128 Academia Road, Section 2, Nankang, Taipei 115,
Taiwan. Tel.: 1886-2-27883799 ext. 1718; fax: 1886-227824814.
E-mail addresses: yschen@iis.sinica.edu.tw (Y.-S. Chen), stone@
csie.ncnu.edu.tw (S.-W. Shih), hung@iis.sinica.edu.tw (Y.-P. Hung),
fuh@csie.ntu.edu.tw (C.-S. Fuh).
1
Tel.: 1886-2-27883799 ext. 1518; fax: 1886-2-27824814.
2
Tel.: 1886-49-2910960 ext. 4832; fax: 1886-49-2915226.
3
Tel.: 1886-2-23625336 ext. 327; fax: 1886-2-23628167.
compensator lens group, and the relay lens group [9±11].
There is no accurate model that describes the paths of the
rays passing through the lens. Moreover, various mechanical assembly and driving motors can be used to adjust the
aperture, zoom and focus. Consequently, the relationship
between the lens settings and the CPs becomes quite
complicated [12].
One way to determine the relationship between the lens
settings and the CPs is to perform the monofocal camera
calibration for each lens setting. That is, each con®guration
of a zoom lens can be treated as a monofocal lens and its
CPs can be calibrated by using well-known camera calibration techniques, such as [13] and [14]. However, this
method is extremely inef®cient and hardly ever feasible
because a motorized zoom lens can usually have a large
number of con®gurations. For example, there will be one
million con®gurations when there are one hundred settings
for each of the three motors controlling the aperture, zoom,
and focus.
In the past, Tarabanis et al. [15] used a special optical
bench to calibrate a zoom lens. They constructed a sparse
table storing the CPs calibrated for the sampled lens
settings. CPs for other lens settings can be obtained via
interpolation. The zoom lens they used was mounted on a
robot gripper. In Ref. [15], they also presented a hand-eye
calibration method.
0262-8856/01/$ - see front matter q 2001 Elsevier Science B.V. All rights reserved.
PII: S 0262-885 6(01)00069-5
1100
Y.-S. Chen et al. / Image and Vision Computing 19 (2001) 1099±1110
Willson and Shafer [12,16] also developed zoom lens
calibration techniques. They calibrated two zoom lenses
driven by microstepping motors. First, they used an autocollimated laser for locating the image center. Then, Tsai's
method [13] was applied to calibrate the eleven CPs for 25
and 121 regularly sampled lens settings of the two zoom
lenses, respectively. For each CP, they approximately
modeled the relationship between the CP and the lens
settings (zoom and focus) with a bivariate polynomial function. The calibrated CP values of the sampled lens settings
were then used to determine the coef®cients of the polynomial functions. These polynomial functions can provide
CPs for continuous ranges of the lens settings. The order of
the polynomial function depends on the lens used and it has
been determined empirically in Willson and Shafer's work
(from order zero to order ®ve) [12]. One of the major advantages of this method is that the memory consumption is very
ef®cient. Only the coef®cients of the polynomial functions
need to be stored. Moreover, these functions could give very
accurate CPs according to their experiments, in which average prediction error of less than 0.14 pixels was achieved.
In this work, we designed and built a motorized zoom
lens with computer-controlled aperture, zoom, and focus.
In order to have faster response for the visual surveillance
application, we used servo motors, instead of microstepping
motors as in Refs. [3,12], for driving the aperture, zoom, and
focus of the zoom lens. Also, we designed a calibration
object to accommodate various spatial resolutions and
®eld of views, which are unique to zoom lens calibration.
The image of the calibration object is often blurred, especially when the depth of ®eld is small. Centroid estimation
for the image coordinates of the calibration points was used
for dealing with this defocusing problem. In this paper, we
®xed the aperture setting and considered only the zoom and
the focus settings when determining the relationship
between the lens settings and the CPs. For representing
this relationship, a two-dimensional table was adaptively
constructed by using a simple and ef®cient method. This
table is sparse and each entry stores the calibrated values
of the variable CPs as well as the corresponding residual
error of calibration. For those lens settings with respect to
which the table has not recorded the corresponding CPs, the
desired values of the CPs can be obtained via interpolation.
To reduce the calibration time, the lens settings for which
the camera calibration needs to be performed were
adaptively determined according to the accuracy of the CP
values interpolated by using the table. Our experimental
results have demonstrated that the table constructed by
using our calibration-on-demand method can well represent
the relationship between the lens settings and the CPs. This
table provides accurate CPs of any lens setting for many
applications.
This paper is organized as follows. We introduce the
camera models of the monofocal and zoom lenses in
Sections 2.1 and 2.2, respectively. In Section 3, we propose
a simple and ef®cient adaptive method, the calibration-on-
Fig. 1. The perspective projection geometry with the pinhole camera model.
demand method, for calibrating a motorized zoom lens.
Section 4 shows the experimental results, and Section 5
gives the conclusions.
2. Camera model
2.1. Monofocal lens
Given a 3-D point in the world coordinate system, the
camera model and the associated CPs can predict the
projected 2-D image coordinates of the 3-D point in the
image coordinate system. For a monofocal lens, the pinhole
camera model with radial distortion is considered here. As
shown in Fig. 1, the 3-D world coordinates …xw ; yw ; zw † of a
point P can be transformed into the 3-D camera coordinates
…xc ; yc ; zc † as follows:
2 3
2 3
xc
xw
6 7
6 7
6 yc 7 ˆ R6 yw 7 1 t;
…1†
4 5
4 5
zc
zw
where R is a 3 £ 3 rotation matrix that is a function of the
three CPs: u x, u y, and u z (X±Y±Z Euler angles), and t ˆ
‰tx ; ty ; tz ŠT is a translation vector from the origin of the world
coordinate system to the origin of the camera coordinate
system. In the camera coordinate system, the origin, Oc, is
located at the perspective center and the Z axis is aligned
with the optical axis of the lens. By using perspective
projection, the coordinates …xI ; yI † in the image plane is
given by the equations
x
xI ˆ f c ;
…2†
zc
yI ˆ f
yc
;
zc
…3†
where f is the effective focal length. Then, without considering lens distortion, the coordinates …xI ; yI † can be transformed into …uI ; vI † in the image coordinate system by
using the equations
…uI 2 u0 †su ˆ xI ;
…4†
…vI 2 v0 †sv ˆ yI ;
…5†
where su and sv are the horizontal and vertical pixel width
Y.-S. Chen et al. / Image and Vision Computing 19 (2001) 1099±1110
and …u0 ; v0 † is the image coordinates of the piercing point
where the optical axis pierces the image plane. If the lens
distortion is considered, Eqs. (4) and (5) become nonlinear.
In this work, only the ®rst coef®cient of radial distortion, k ,
is considered [13,14], and Eqs. (4) and (5) should be
replaced by:
…1 2 kr2 †…u^ I 2 u0 †su ˆ xI ;
2
…1 2 kr †…v^I 2 v0 †sv ˆ yI ;
…6†
…7†
where …u^ I ; v^I † is the actual image coordinates measured in
the image and r2 ˆ ‰…u^I 2 u0 †su Š2 1 ‰…v^I 2 v0 †sv Š2 :
Totally, there are twelve CPs in this camera model: u0, v0,
su, sv, f, k , u x, u y, u z, tx, ty ; and tz : Among these CPs, the
vertical pixel width, sv, can be obtained from the speci®cation of the CCD camera. The remaining eleven CPs can be
estimated by using Weng et al.'s method [14], provided that
a set of known calibration points are observed in the image.
In the following, the 3-D world coordinates of the calibration points, …xw ; yw ; zw †; and their corresponding image
measurements, …u^I ; v^I †; will be referred to as the calibration
data.
2.2. Zoom lens
2.2.1. Aperture setting
Ideally, we would like to have a zoom lens whose aperture setting does not affect the physical values of the CPs.
However, practical lenses suffer a variety of aberrations.
Some of the aberrations, such as spherical aberration,
coma, and astigmatism, become more serious when the
size of the aperture stop is larger [9±11,17]. Furthermore,
trying to correct the spherical aberration will usually result
in the zonal aberration, which may lead to the focus shift
phenomenon where the focal plane moves along the optical
axis when adjusting the aperture setting [10,11]. Therefore,
changing the aperture setting alone may alter the value of
the focal length, even though the zoom and the focal settings
remain unchanged. Also, the aberrations may affect the
quality of the acquired image and hence the accuracy of
the estimated image coordinates of the calibration points.
As a result, the aperture setting may indeed in¯uence the
estimated values of the CPs.
However, some researchers have found that the in¯uence
of the aperture setting is negligible for many applications.
For example, Li and Lavest [18] reported that the aperture
setting does not change the focal length signi®cantly. Also,
our experiment described in Ref. [19] showed that the aperture setting did not much affect the estimates of the CPs.
Hence, in this paper, we have chosen to ®x the aperture
setting while calibrating our motorized zoom lens. With
this simpli®cation, the obtained CPs can be stored in a
two-dimensional table indexed by the zoom and focus
settings, instead of a three-dimensional table indexed by
the zoom, focus, and aperture settings. Even if the in¯uence
of the aperture setting is not negligible, the variation of the
1101
CPs, when adjusting the aperture setting, is usually quite
smooth, as described in Ref. [12]. In that case, we can
simply choose a couple of aperture settings and repeat our
adaptive calibration method described in Section 3. Another
possibility is to extend the proposed method to the additional axis of the aperture setting.
2.2.2. Zoom and focus settings
Once an aperture setting is chosen and ®xed, our goal is to
determine the relationship between the CPs and the zoomfocus setting. For each combination of the zoom and focus
settings, the zoom lens can be treated as a monofocal lens
and its CPs can be calibrated individually. When we adjust
the zoom or the focus setting, some of the CPs remain
unchanged during the adjustment. For example, the pixel
width, su and sv, and the extrinsic CPs, u x, u y, u z, tx, ty ;
and tz, are not supposed to vary. Hence, once these CPs
are determined with the initial zoom lens setting, they can
be ®xed during the remaining procedure of zoom lens calibration.
When adjusting the zoom or the focus setting, the focal
length will change accordingly. Thus, f and k should be
estimated for each combination of the zoom and focus
settings. Furthermore, the lens groups of the zoom lens
are moved forward and backward along the optical axis
when we adjust the zoom or the focus setting. This longitudinal movement will make the perspective center, i.e. the
origin of the camera coordinate system move along the
optical axis relative to the ®xed image plane. Consequently,
we have to estimate the displacement, Dtz ; of the perspective
center on the Z axis for different zoom and focus settings.
Besides, the lens groups will be moved back and forth by
gradual rotation during the adjustment. If the optical axis
deviates from the rotation axis, the image coordinates of the
piercing point, u0 and v0, may change and they should also
be estimated for each combination of the zoom and focus
settings.
To achieve a smaller 2-D residual error, all the twelve
CPs can be freely optimized during the calibration procedure. However, this will increase the calibration effort and
the memory storage of the table. Furthermore, smaller 2-D
residual errors do not necessarily result in more accurate
estimates of the physical camera parameters. Therefore,
our method estimates only ®ve camera parameters, u0, v0,
f, k , and Dtz ; in the zoom lens calibration procedure, which
will be referred to as the variable camera parameters
(VCPs), and ®xes all the remaining CPs to be those determined with the initial zoom lens setting.
3. Zoom lens calibration
3.1. Calibration object
There are two major issues when designing the calibration object: the structure of the calibration object and the
1102
Y.-S. Chen et al. / Image and Vision Computing 19 (2001) 1099±1110
calibration patterns to be measured. To accurately estimate
the CPs, the calibration data has to be obtained by having
the calibration points at different distances from the lens,
rather than just at the precisely focused distance. Therefore,
the calibration object is usually made of a cube containing
calibration patterns on several of its surfaces [5,20] or a
calibration plate mounted on a translation stage
[13,14,21]. The calibration patterns on the calibration object
can be grid lines [5], squares [13,14], or circles [20,21].
Image coordinates of the calibration points can be obtained
by measuring the intersections, corners, or centroids of the
calibration patterns in the image.
3.1.1. Defocusing problem caused by aperture stop
Without considering aberrations, the points sitting on the
focused plane can be projected onto sharp image points in
the image plane. Other points deviated from the focused
plane will be imaged as ®lled circles (or, to be more precise,
as the shape of the aperture stop) and the acquired image is
blurred. In this case, it is not easy to accurately estimate the
image coordinates, …u^ I ; v^I †: This defocusing problem is more
serious when we use a larger focal length or a larger size of
the aperture stop, due to the resulted smaller depth of ®eld.
For zoom lens calibration, this defocusing problem becomes
even more serious due to the combination of the following
three requirements: (1) accurate CP estimation requires calibration data obtained at different distances, rather than just
at the focused distance; (2) camera calibration has to be
performed for different focus settings, which will make
the focused distances deviate from the calibration object
and result in blurred images; and (3) camera calibration
has to be performed for different zoom settings, while larger
focal length will result in shorter depth of ®eld, thus blur the
image.
When the blurring is symmetric, we can still obtain accurate image coordinates, …u^ I ; v^I †; of the calibration points
from the blurred image. Let …u^I ; v^I † be where in the image
plane the chief ray projects. The chief ray is de®ned as the
ray passing through the center of the aperture stop. Starting
from an out-of-focus point object, the chief ray always
project on the center of the circle in the image plane when
the aperture stop is at its ideal location. That is, the aperture
stop is perpendicular to the optical axis and its center is
located at the optical center. For different aperture settings,
the projected circles are of different sizes but are all
concentric. If the center of the circle is measured as the
image coordinates, …u^I ; v^I †; these coordinates will not vary
with the size of the circle. Consequently, the defocusing
problem due to the aperture stop will not in¯uence the calibration result, when the aperture stop is ideally located and
none of the aberrations are considered.
For a practical lens, the center of the aperture stop may
not locate at the optical center. The estimates of the CPs can
be slightly changed when adjusting the aperture stop. More
discussions is given in Appendix A.
Fig. 2. This ®gure shows the calibration object we used in this work, which
is a calibration plate mounted on a translation stage. The translation direction of the stage is perpendicular to the calibration plate. The calibration
plate is a dark board containing 25 £ 25 white circles. The diameters of the
circles are 15 mm except for the 24 smaller circles in the middle, whose
diameters are 10.5 mm. The distances between the centers of the neighboring circles are 25 mm.
3.1.2. Design of calibration object
In this work, we constructed a black calibration plate
mounted on a computer-controlled translation stage, as
shown in Fig. 2. The normal direction of the calibration
plate, the translation direction of the stage, and the direction
of the optical axis of the lens are aligned to be parallel. The
calibration plate can be moved along the direction of the
stage such that the calibration data measured at different
distances can be obtained. On the calibration plate, there
are many circles used as the calibration patterns. Each circle
projected in the image plane will be symmetrically blurred if
the circle is out of focus. Edge detection followed by circle
®tting is not suitable to estimate the circle center because
there is no sharp edge around the blurred circle. Fortunately,
accurate image coordinates of the circle center can be
obtained by measuring the centroid of the circle in the
image, as mentioned in Section 3.1.1.
For better accuracy of camera calibration, the calibration
patterns had better be distributed all over the image and the
number of the calibration patterns appearing in the image
should be as many as possible [18]. For a zoom lens, it is
hard to meet these requirements because of the varying ®eld
of view and spatial resolution during the adjustment of the
Y.-S. Chen et al. / Image and Vision Computing 19 (2001) 1099±1110
1103
of the circle, can be easily estimated by using the following
image processing techniques: image binarization, blob
analysis, and centroid estimation. As the ®eld of view
varies, part of the calibration plate may become invisible
in the image. However, we still have to identify the circles
appearing in the image in order to obtain their corresponding 3-D world coordinates. For easy identi®cation, the larger
circle in the middle, which is surrounded by twenty-four
smaller circles, is chosen to be the ®ducial circle.
The origin of the world coordinate system, Ow, is located
at the center of the ®ducial circle. The 3-D coordinates of
the other circles are then described with respect to this
origin. If the camera is properly aligned such that the ®ducial circle always appear in the image for all the zoom
settings, the task of locating the ®ducial circle in the
image can be easily accomplished by utilizing the sizes of
the circles. Each circle appearing in the image can then be
identi®ed and the calibration data, including 3-D world
coordinates and 2-D image coordinates, can be obtained.
Notice that if the lens distortion is large enough to
severely bias the estimate of the centroid position, one
should rectify the image by using the initially estimated
lens distortion coef®cients before estimating the centroid
position. The initial estimates of the lens distortion coef®cient, together with those of the other CPs, are obtained by
using the estimates of the centroid positions without rectifying the images. The recti®cation±estimation procedure may
have to be repeated a few times until convergence.
However, for the motorized zoom lens used in our experiments, we simply use the initial estimates of the CPs
because the lens distortion is not severe enough to adopt
the above iterative procedure.
Fig. 3. Images of the calibration plate acquired with a zoom lens when its
zoom is set to be (a) wide-angle and (b) telephoto.
zoom setting. To remedy this problem, circles of two different sizes were used as shown in Fig. 2. When the zoom of
the lens is set to be wide-angle, many larger circles as well
as the smaller circles in the middle of the calibration plate
appear in the image, as shown in Fig. 3(a). These larger
circles can provide more accurate centroid positions estimated in the image. When the zoom of the lens, on the
other hand, is set to be telephoto, only a few smaller circles
in the middle of the calibration plate appear in the image, as
shown in Fig. 3(b). These smaller circles still can provide
accurate centroid positions due to larger magni®cation.
Instead of crowding more small circles in the middle region
of the calibration plate, we put the same number of circles,
per unit area, as what was put in the outside region, to leave
larger gap between the circles. This will prevent the neighboring circles from overlapping each other when the out-offocus circles are seriously blurred in the image.
3.1.3. Image coordinates estimation for calibration patterns
Image coordinates of the calibration point, i.e. the center
3.2. Zoom lens calibration procedure
This section describes the proposed procedure of motorized zoom lens calibration. For the zoom lens used in our
experiments, there are three thousand steps for both the
zoom and the focus motors, and totally nine million lens
settings in combination. To reduce the number of the monofocal camera calibration performed and the storage required
to record the VCPs, we have adopted a sampling and interpolation technique over the entire working range of the lens
settings. Our goal is then to create a two-dimensional table,
indexed by the values of the lens settings, to store the VCPs
estimated at the sampled lens settings. The VCPs for
unsampled lens settings, that is, where the monofocal
camera calibration has not been performed, can be approximated via bilinear interpolation.
Because the values of the VCPs usually vary nonlinearly
with respect to the lens settings, a uniformly sampled table
might not be able to ef®ciently represent the relationship
between the VCPs and the lens settings. A more ef®cient
sampling strategy is to perform the monofocal camera calibration only when the bilinear interpolation cannot give an
accurate enough result. This strategy leads to an adaptive
1104
Y.-S. Chen et al. / Image and Vision Computing 19 (2001) 1099±1110
Fig. 4. This ®gure shows an example of the sample positions. The number in
the table means the order at which the camera calibration is performed.
method, referred to as the calibration-on-demand method,
which allows adaptive sample positions and locally varying
sampling rates. That is, each entry of the VCP table will be
adaptively created, by trial and error, to store the VCPs
estimated at the sampled lens setting and the corresponding
calibration residual error. Our criterion for entry creation is
based on the residual error. In the following, we denote the
residual error, e, as the function g(´), which calculates the
average difference between the predicted and measured
image coordinates of the calibration points. That is,
e ˆ g…I; b; z; f †;
where z denotes the zoom setting, f the focus setting, I the
image set used for measuring the image coordinates of the
calibration points, and b denotes the VCPs used for predicting the image coordinates of the calibration points.
There are three kinds of residual error used in our zoom lens
^
calibration procedure. The ®rst one is the residual error e;
where e^ ˆ g…I; b^ ; z; f †: In this case, the VCPs b^ are calculated
by performing the monofocal camera calibration in which the
required image coordinates of the calibration points are
measured in the image set I. The second one is the residual
error e~ ˆ g…I; b~ ; z; f †: Here, the VCPs b~ are interpolated
values computed by using the values of the VCPs that have
already been stored in the table. The last one is the residual
which is the interpolated values by directly using the
error e;
values of the residual error e^ stored in the table.
Fig. 4 gives an example of how we determine the sample
positions. At ®rst, we adjust the aperture stop to match the
lighting condition and then ®x it in the subsequent calibration. Then, we set the zoom setting to be the middle value
within its range, ª‰ZSTART ; ZEND Š;º that is, ZSTART 1 ZEND =2:
Next, the calibration plate is placed at an appropriate
distance and the focus setting is adjusted within its entire
working range, ‰FSTART ; FEND Š; until the image of the calibration plate becomes the sharpest. This focus setting is
referred to as FREF. A monofocal camera calibration is
then performed (as illustrated by sample position number
0 in Fig. 4) to obtain all the CPs. During the following zoom
lens calibration procedure, eight CPs, that is, su, sv, u x, u y, u z,
tx, ty, and tz, are ®xed and the remaining ®ve VCPs, u0, v0, f,
k , Dtz ; are to be calibrated.
For each zoom setting to be calibrated, the monofocal
camera calibration is ®rst performed at the start and end
of the focus setting. For example, when the zoom setting
is set at ZSTART at the next stage, the start and end of the
focus setting are FSTART and FEND, as illustrated by sample
positions number 1 and 2 in Fig. 4. Both the obtained
VCPs b^ and the residual error e^ will be stored in the
table. Next, the focus setting is set to the middle value
of its range. The VCPs b~ at this focus setting can be
computed by interpolation with the VCPs obtained
previously at the start and end of the focus setting, but
we have to evaluate whether this interpolated VCPs, b~ ;
are accurate enough. To evaluate the accuracy of the
interpolation, we ®rst acquire the images of the calibration
plate and estimate the 2-D image coordinates of the calibration points observed in those images. Then, we
compute the residual error e~ between the observed
image coordinates and the predicted positions calculated
by using the interpolated VCPs, b~ ; that is, e~ ˆ
g…I; b~ ; z; f †; where z and f are set at sample position
number 3 shown in Fig. 4. Large e~ implies that the interpolated VCPs, b~ ; may not be accurate enough, which
means a monofocal camera calibration had better be
performed at sample position number 3. In our calibration-on-demand method, the monofocal camera calibration
is performed only when the residual error e~ is larger than
where kf is a scaling factor and e is the
a threshold, kf e;
residual error computed via interpolation with the values
of the residual error e^ obtained at the start and end focus
settings. The above procedure is recursively repeated for
each middle focus setting between two calibrated focus
settings (for example, see sample positions number 4±8
in Fig. 4, where the zoom setting is set to be ZEND) until
the interpolated VCPs, b~ ; are accurate enough, that is, e~ is
small enough. Notice that the decision of whether e~ is
which varies for different
small enough depends on e;
focus settings.
For the dimension of the VCP table along the zoom
setting, we adopt an adaptive sampling strategy similar
to the one used for the focus setting. Initially, the VCPs
corresponding to all the focus settings at z ˆ ZSTART and
z ˆ ZEND are calibrated by using the procedure described
in the previous paragraph, which is illustrated by the
sample positions in the ®rst and the last columns shown
in Fig. 4. Next, the zoom setting is set to be
ªzSTART 1 zEND =2º and the focus setting is initially set to
be FREF. Then, the images of the calibration plate are
acquired and the observed 2-D image coordinates of the
calibration points are estimated. With the observed 2-D
image coordinates, the residual error e~ can be computed
by using the interpolated VCPs, b~ ; computed by using the
VCPs recorded in the ®rst and last columns in the table.
The interpolated VCPs, b~ ; will be considered as accurate
where kz is a scaling
enough if e~ is small, that is, e~ , kz e;
factor. Otherwise, a monofocal camera calibration will be
performed to obtain more accurate VCPs. Again, the
Y.-S. Chen et al. / Image and Vision Computing 19 (2001) 1099±1110
1105
above procedure is repeated recursively for each middle
zoom setting between two calibrated zoom settings until
the interpolated VCPs are accurate enough, that is, e~ is
small enough.
The proposed procedure for motorized zoom lens calibration is summarized below.
Zoom lens camera calibration
procedure intrinsic-camera-calibration …z; f †
begin
set the lens settings …z; f † and perform the monofocal
camera calibration
record the VCPs b^ (u0, v0, f, k , and Dtz ) and residual
error e^ in the table with indices …z; f †
end
procedure focus-calibration … fstart ; fend †
begin
focus setting f :ˆ … fstart 1 fend †=2
interpolate the VCPs, b~ ; from those of fstart and of fend
evaluate the residual error e~
if e~ is not small enough
begin
intrinsic-camera-calibration …z; f †
focus-calibration … fstart ; f †
focus-calibration … f ; fend †
end
end
procedure zoom-calibration …zstart ; zend †
begin
focus setting f :ˆ FREF
zoom setting z :ˆ …zstart 1 zend †=2
interpolate the VCPs, b~ ; from those of zstart and of
zend
evaluate the residual error e~
if e~ is not small enough
begin
intrinsic-camera-calibration …z; FSTART †
intrinsic-camera-calibration …z; FEND †
focus-calibration …FSTART ; FEND †
zoom-calibration …zstart ; z†
zoom-calibration …z; zend †
end
end
begin
perform the initial camera calibration to obtain the
CPs to be ®xed: su, sv, u x, u y, u z, tx, ty ; and tz
intrinsic-camera-calibration …ZSTART ; FSTART †
intrinsic-camera-calibration …ZSTART ; FEND †
focus-calibration …FSTART ; FEND †
intrinsic-camera-calibration …ZEND ; FSTART †
intrinsic-camera-calibration …ZEND ; FEND †
focus-calibration …FSTART ; FEND †
zoom-calibration …ZSTART ; ZEND †
end
Fig. 5. This ®gure shows the motorized zoom lens used in our experiments,
which is a Fujinon TV zoom lens H6 £ 12.5R with its aperture, zoom, and
focus driven by three servo motors through pushrod links.
4. Experimental results
4.1. Motorized zoom lens
In this work, we constructed a motorized zoom lens by
using a Fujinon TV zoom lens H6 £ 12.5R, as shown in
Fig. 5. Three servo motors were used for driving the aperture, zoom, and focus of the lens through pushrod links. We
adopted servo motor because of its faster response. Each
servo motor contains three thousand steps, indicated by
the optical encoder. We controlled these servo motors
with a PC through the Advantech PCL-832 control cards.
To avoid the hysteresis problem [12] due to the backlash,
we always set the motor to the desired setting in the same
direction. That is, we ®rst drive the motor to the setting a
little smaller than the desired one and then drive it to the
desired setting.
4.2. Constructing and assessing the table
In our experiments, the proposed calibration-on-demand
method was used to calibrate the motorized zoom lens
described in Section 4.1. In the ®rst experiment, the parameters kf and kz, which determine the sampling density in
the focus and zoom axes, were set to be 1.5 and 2, respectively. A table of 120 entries was created and the sample
positions are shown in Fig. 6. The VCPs stored in the table
are shown in Fig. 7, where tz0 is the value of the ®xed CP, tz,
obtained in the initial camera calibration. The mean residual
error e^ for the acquired table is 0.19 pixels.
To evaluate the repeatability of the motorized zoom lens,
we repeatedly set the zoom and focus settings to each
sampled settings previously recorded in the table and then
took the images of the calibration palate to check how large
the residual error e^ became. In this repeatability experiment,
the mean residual error e~ did increase to 0.44 pixels.
However, if we randomly set the focus and zoom settings,
which are unlikely to be the sample positions recorded in the
table, the residual error e~ does not increase much further.
For example, Fig. 8 shows the residual error e~ of four
1106
Y.-S. Chen et al. / Image and Vision Computing 19 (2001) 1099±1110
Fig. 6. The sample positions with respect to the zoom and focus settings.
Fig. 7. The VCPs b^ and the residual error eà stored in the table.
Y.-S. Chen et al. / Image and Vision Computing 19 (2001) 1099±1110
1107
Fig. 8. The residual error e~ of repeatability and interpolation are shown in dotted and solid lines, respectively.
hundred random trials, and the mean residual error e^ is 0.475
pixels, which is only 8% increase compared to the residual
error e~ caused by repeatability error.
Another experiment was conducted by using different set
of kf and kz, which were set to be 2 and 3, respectively. In
this case, a table of smaller size (52 entries) was created at
the expense of larger error, where the residual error e~ was
0.54 pixels (that is, a 22.7% increase).
5. Conclusions
In this paper, we have proposed a calibration-on-demand
procedure for calibrating a motorized zoom lens, which can
adaptively create a table for representing the relationship
between the lens settings and the camera parameters. The
major advantage of the proposed method is in its ability to
reduce, in an adaptive manner, the amount of image acquisition and monofocal camera calibration, which is a timeconsuming task, while maintaining the required calibration
accuracy. Our experiments have shown that the proposed
method can provide accurate intrinsic camera parameters
for each lens setting of a motorized zoom lens by performing a much smaller number of monofocal camera calibrations. Our experiment has shown that the average residual
error of the camera parameters given by the acquired table
(having only 120 entries) is less than half pixels.
Another contribution of this work is that we have
proposed a calibration object which consists of circles distributing on a calibration plate. These circles are of different
sizes and have been designed to accommodate various
spatial resolutions and ®eld of views, which are unique
problems to zoom lens calibration. Another advantage of
this calibration object is that accurate image coordinates
of the calibration points can be obtained by estimating the
centroids of the circles, even when the circles are blurred
due to the defocusing problem.
Appendix A
Here, we analyze the in¯uence of the aperture stop on the
resulted images. For an out-of-focus point object, the position of the aperture stop and the aperture setting in¯uence
the size and position of the resulted circle in at least three
ways:
1. Off optical center. A zoom lens consists of a few lens
groups, and the aperture stop is located somewhere
between lens groups. As shown in Fig. 9, if the center
of the aperture stop is away from the optical center but
still on the optical axis, the aperture stop may asymmetrically shield the pencil of rays, which may then shift the
center of the circle.
2. Off optical axis. As shown in Fig. 10, if the optical axis
does not pass through the center of the aperture stop, the
1108
Y.-S. Chen et al. / Image and Vision Computing 19 (2001) 1099±1110
Fig. 9. When the center of the aperture stop is away from the optical center but still on the optical axis, the aperture stop asymmetrically shields the pencil of
rays and the center of the resulted circle is shifted.
Fig. 10. When the center of the aperture stop does not pass the optical axis, the aperture stop asymmetrically shields the pencil of rays and the center of the
resulted circle is shifted.
aperture stop may asymmetrically shield the pencil of
rays, which may then shift the center of the circle.
3. Vignetting. As shown in Fig. 11, if the size of the aperture
stop is too large, the lens mount or other lens groups may
asymmetrically shield the pencil of rays. This phenomenon is called vignetting [9,11]. Not only the center but
also the shape of the resulted circle are changed.
To sum up, each out-of-focus point is projected onto a
®lled circle in the image plane. If the center of the aperture
stop is not ideally located at the optical center, or if the
vignetting phenomenon occurs, the center of the circle
will not be the image position where the chief ray projects.
Moreover, the shape of the circle will not be symmetric
when the vignetting phenomenon occurs.
In the following, we will quantitatively analyze the in¯uence of the aperture stop on the projected image position,
without considering vignetting. As shown in Fig. 12, pencil
of rays from a point light source pass through the lens with
Fig. 11. When the size of the aperture stop is too large, the lens mount may
asymmetrically shields the pencil of rays. Not only the center but also the
shape of the resulted circle are changed.
Y.-S. Chen et al. / Image and Vision Computing 19 (2001) 1099±1110
1109
coordinates of the calibration data be enlarged or shrunk
with a constant ratio k. This is equivalent to elongating or
shortening the effective focal length, f, of the CPs.
However, this ratio k depends on c. For the points at different distances, the positions of their projected circles are
shifted in different ratios. This makes zoom lens calibration
harder when the calibration data is obtained at different
distances.
References
Fig. 12. We quantitatively analyze the movement of the projected circles
with or without shielding by the aperture stop.
optical center O. When the aperture stop is open, this point
light source projects in the image plane as a circle between
p1 and p5 centered at p3. After shielding by the aperture stop,
the resulted circle is between p2 and p4. Let the origin be the
piercing point in the image plane. The vertical coordinates,
yp1 ; ¼; yp5 ; of the points p1 ; ¼; p5 are:
yp1 ˆ c tan…u† 2
c tan…u† 1 R
…c 2 b†;
c
yp2 ˆ c tan…u† 2
c tan…u† 1 r1
…c 2 b†;
c2a
yp3 ˆ c tan…u† 2 tan…u†…c 2 b† ˆ b tan…u†;
yp4 ˆ c tan…u† 2
ctan…u† 2 r2
…c 2 b†;
c2a
yp5 ˆ c tan…u† 2
c tan…u† 2 R
…c 2 b†;
c
where R is the radius of the lens, r1 and r2 are the distances
from the top and the bottom of the aperture stop to the
optical axis, u is the angle between the optical axis and
the ray emerging from O, a, b, and c are the distances
from the lens to the aperture stop, the image plane, and
the converging point of the pencil of rays from the point
light source, respectively.
Assume that the center of the aperture stop is on the
optical axis, i.e. r1 ˆ r2: The center p 03 of the circle between
p2 and p4 can be estimated in the image as the middle point
between p2 and p4:
y 0p3 ˆ
c…b 2 a†
tan…u† ˆ k £ yp3 ;
c2a
where k ˆ c…b 2 a†=b…c 2 a†: Notice that k is irrelevant to
u . That is, the aperture stop will cause the estimated image
[1] E.P. Krotkov, Exploratory Visual Sensing for Determining Spatial
Layout with an Agile Stereo Camera System, PhD thesis, University
of Pennsylvania, Pennsylvania, April 1987, Technical Report
MS-CIS-87-29.
[2] K. Pahlavan, Active Robot Vision and Primary Ocular Processes, PhD
thesis, Royal Institute of Technology, Stockholm, Sweden, May 1993.
[3] S.-W. Shih, Y.-P. Hung, W.-S. Lin, Calibration of an active binocular
head, IEEE Transactions on Systems, Man, and Cybernetics 28 (4)
(1998) 426±442.
[4] A.L. Abbott, Dynamic Integration of Depth Cues for Surface Reconstruction from Stereo Images, PhD thesis, University of Illinois at
Urbana-Champaign, Illinois, January 1991.
[5] J.-M. Lavest, G. Rives, M. Dhome, Three-dimensional reconstruction
by zooming, IEEE Transactions on Robotics and Automation 9 (2)
(1993) 196±207.
[6] G. Surya, M. Subbarao, Depth from defocus by changing camera
aperture: A spatial domain approach, Proceedings of the IEEE
Conference on Computer Vision and Pattern Recognition, New
York, June 1993, pp. 61±67.
[7] K. Hosoda, H. Moriyama, M. Asada, Visual servoing utilizing zoom
mechanism, Proceedings of the International Conference on Robotics
and Automation, Nagoya, Aichi, Japan, May 1995, pp. 178±183.
[8] J.A. Fayman, O. Sudarsky, E. Rivlin, Zoom tracking, Proceedings of
the International Conference on Robotics and Automation, Leuven,
Belgium, May 1998, pp. 2783±2788.
[9] R. Kingslake, Lens Design Fundamentals, Academic Press, San
Diego, 1978.
[10] R. Kingslake, A History of the Photographic Lens, Academic Press,
San Diego, 1989.
[11] W.J. Smith, Modern Lens Design, McGraw-Hill, New York, 1992.
[12] R.G. Willson, Modeling and Calibration of Automated Zoom Lenses,
PhD thesis, Carnegie Mellon University, Pittsburgh, Pennsylvania,
January 1994, Technical Report CMU-RI-TR-94-03.
[13] R.Y. Tsai, A versatile camera calibration technique for high-accuracy
3D machine vision metrology using off-the-shelf TV cameras and
lenses, IEEE Journal of Robotics and Automation RA-3 (4) (1987)
323±344.
[14] J. Weng, P. Cohen, M. Herniou, Camera calibration with distortion
models and accuracy evaluation, IEEE Transactions on Pattern
Analysis and Machine Intelligence 14 (10) (1992) 965±980.
[15] K. Tarabanis, R.Y. Tsai, D.S. Goodman, Calibration of a computer
controlled robotic vision sensor with a zoom lens, CVGIP: Image
Understanding 59 (2) (1994) 226±241.
[16] R.G. Willson, S.A. Shafer, Precision imaging and control for machine
vision research at Carnegie Mellon University, Proceedings of SPIE
Conference on High-Resolution Sensors and Hybrid Systems, San
Jose, CA, February 1992, vol. 1656, pp. 297±314.
[17] C.C. Slama (Ed.), Manual of Photogrammetry 4th ed, American
Society of Photogrammetry, 1980. Location: Falls Church, Virginia.
[18] M. Li, J.-M. Lavest, Some aspects of zoom lens camera calibration,
IEEE Transactions on Pattern Analysis and Machine Intelligence 18
(11) (1996) 1105±1110.
[19] Y.-S. Chen, S.-W. Shih, Y.-P. Hung, C.-S. Fuh, Camera calibration
1110
Y.-S. Chen et al. / Image and Vision Computing 19 (2001) 1099±1110
with a motorized zoom lens, Proceedings of the International
Conference on Pattern Recognition, Barcelona, Spain, September
2000, vol. 4, pp. 495±498.
[20] J. HeikkilaÈ, O. SilveÂn, A four-step camera calibration procedure with
implicit image correction, Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition, Puerto Rico, June 1997,
pp. 1106±1112.
[21] S.-W. Shih, Kinematic and Camera Calibration of Recon®gurable
Binocular Vision Systems, PhD thesis, National Taiwan University,
Taipei, Taiwan, June 1996.