Technical report CVAP 55, ISRN KTH/NA/P--88/08--SE, 1988. KTH Royal Institute of Technology, Stockholm, Sweden. Scale-Space for Discrete In1ages Tony Lindeberg Computer Vision and Associative Pattern Processing Laboratory (CVAP ) Depar tment of Numerical Analysis and Computing Science Royal Institute of Technology S-100 44 Stockholm, Sweden 28 October 1988 1 Abstract In this paper we address the formulation of a scale-space theory for discrete images . We denote a one-dimensional kernel a scale-space kern el if it reduces the number of local extrema and discuss which discrete kernels are possible scale-space kernels. Unimodality and pos itivity properties are shown to hold for such kernels as well as thei r Fourier transforms. An explicit expression characterizing a ll discrete scale-space kernels is given. \Ve propose that there is only one reaso nable way to define a scale space family of images L : Z x R+ --. R for a one-dimensional discrete image f : Z --. R, namely by convolution with the family of discrete kernels T : Z x R+ --. R 00 L(x; t) = L T(n; t)f(x- n) (xEZ,t2:,0) n= - oo where T(n; t) = e-t In(t) and I,. is the modified Bessel function of order n. \Vith this representation, comprising a continuous scale par ameter, we are no longer restricted to specific predetermined levels of scale. Further, T( n; t) appears naturally in the solution of a discretized version of the heat equation , both in one and two dimensions. The family {T(n ; t)} (t 2: 0) is the only one-parameter family of discrete symmetric shift-invariant kernels satisfying both necessary scale-space requirements and the semigroup property T(.; s)*T(.; t) T(.; s + t). Similar arguments appl ied in the continuous case uniquely lead to the family of gaussian kemels. The commonly adapted technique with a sampled gaussian pro duces undesirable effects. It is shown that scale-space violations might occur in the family of functions generated by convolution with the sam pl ed gaussian kernel. The result exemplifies that properties derived in the continuous case might be violated after discretization. A discussion about the numerical im plementation is performed and an algorithm generating the filter coefficients is supplied. = 2 1 Introduction It is well known that objects in the world and details in an image exist only over a limited range of resolution. A classical example is the concept of a branch of a tree which makes sense only on the scale say from a few centimeters to at most a few meters. It is meaningless to discuss the tree concept at the nanometer or t he kilometer level. At those levels of scale it is more relevant to talk about the molecules , which form the leaves of the tree, or the forest, in which the tree grows. If one aims at describing the structure of an image , a multiresolution representation is of crucial importance. Then a mechanism, which systematically removes finer details or high-frequency information from an image, is required. This smoothing must be available at any level of scale. A method proposed by Witkin [25] and Koenderink, van Doorn [11] is to embed the original image in a one-parameter family of derived images, the scale-space, where the parameter t describes the current level of scale resolution. Let us briefly develop the procedure as it is formulated for one dimensional continuous images: Given an image function (a signal) f: R-+ R a function 1 L: R X R+ - R is defined by L(x; 0) = f(x) and convolution with the gaussian kernel g : R x R+\ {0} --+- R L(x; t) = (g(-; t) * f (·))(x; t ) = l oo (=-oo 1 -.-e-~ / 2 t f(x- Od~ 2 .j2i'i (1) if t > 0. Equivalently the family can be regarded as defined by the heat equation (2) with appropriate boundary conditions. This family possesses some attrac tive properties. • As the scale p arameter t is increased additional local extrema or ad ditional zero crossings caJlnot appear. • Causality in the sense that L( x; t2) depends exclusively on L(x; tt) if t2 > tl (tt,t2;::::: 0). • The blurring is space invariant and does not depend upon the image values. 1 R+ de notes the set of real non-negative numbers including zero. 3 coarser levels of scale increasing t Figure 1: A scale-space is an ordered set of derived im ages intended to represent the original image at various levels of scale. It has been shown by Babaud et.al. [3] that the gaussian function is the only kernel in a broad class of functions which satisfies adequate scale-space conditions. The theory has b een developed and well-established for continuous im ages. However , it does not tell us at all about how the implementation should be performed computationally for real-life i.e. discrete images. In principle, we feel that there are two approaches possible. • Apply the results obtained from the continuous scale-space theory by discretizing the occurring equations. For instance the convolution in tegral (1) can be approximated by a sum using customary numerical methods. Or, the heat equation (2) can be discretized in space with the ordinary fiv e-point Laplace operator forming a set of coupled or dinary differential equations, which can be further discretized in scale. If the numerical methods are chosen with care we will certainly get an approximation of the continuous numerical values . Dut we are not guaranteed that the original scale-space conditions, however formu lated , will be preserved . • Define a genuine discrete theory by postulating suitable axioms. In this paper we will develop the second item and address the formulation of a scale-space theory for discrete images. We will mainly consider one dimen sional images since, at the moment, the continuous scale-space theory seems well-analyzed only in the one-dimensional case. 4 With a scale-space we will mean a family of derived images meant to represent the original image at various levels of scale. Each member of the family should be associated with a value of a scale parameter intended to somehow describe the current level of scale. This scale parameter, here denoted by t, may be either discrete (t E Z+) or continuous (t E R+) and we will obtain two different types of discrete scale-spaces - discrete images with a discrete scale parameter and discrete images with a continuous scale parameter. In both cases we will start from the following basic assumptions: • All representations should be generated by (linear) convolution of the original image with a shift-invariant kernel. • An increasing value of the scale parameter t should correspond to coarser levels of scale and images with less structure. Particularly, t = 0 should represent to the original image. • All images should be real-valued functions : Z -+ R defined on the same infinite grid, in other words no pyramid representations will be used. It will be natural to study linear transformations. A one-dimensional kernel will be denoted a scale-space kernel if, for all images, the number of lo cal extrema in the con volved image does not exceed the number of local extrema in the original image. In order to get familiar with the consequences of the definition we will start by illustrating what this scale-space property means. In a straightfor ward and constructive manner we will derive some qualitative requirements of the kernel that are necessarily induced by the a...x.iom. They can be ex pressed both in the spatial and the frequency domain. It will be easy to show that a class of kernels, of which the binomial coefficients are a special case, possesses scale-space properties. Later , a complete characterization of a scale-space kernel will be given using existing results from the theory of total positivity. The characteriza tion can b e formulated either in terms of minors of an associated Toeplitz matrix or as an ex plicit expression for the generating function. An alternative formulation of the scale-space for continuous functions will be p erformed based on the scale-space property combined with shift invariance and a semi -group requirement posed on the occurring family of kernels. Using the same argument we will construct the scale-space for discrete images . A one-parameter family of discrete kernels, which in a sense 5 is the discrete analog of the family of gaussian kernels, will be presented. We propose that this fan1ily is unique and that it is the canonical one fulfilli ng reasonable scale-space requirements, including a. continuous scale parameter. The numerical implementation will be treated and an algorithm gener ating the necessary filt er coefficients will be supplied . vVe end the paper by discussing discrete scale-space properties of some obvious discretiza.tions of the continuous convolution integral and the heat equation. It will be shown that convolution with the proposed family of kernels appears naturally as the solution of a. discretized version of the heat equation. We will also show that the commonly adapted technique with a. sampled ga.ussia.n kernel might lead to undesirable effects. The results presented should h ave implications for image analysis as well as other disciplines of digital signal processing. 2 Discrete Kernels with Scale-Space Properties In this section we will consider discrete linear shift-invariant image transfor mations expressed on the general form ! out = ]{ 00 f ou.t(x) = L n=- oo * f in (3) 00 L K(n)fin(x- n) = K (x - n)fin(n) (x E Z) n=-oo where fin : Z - R is a. discrete image and J( : Z - R is a. discrete convolution kernel. As a n underlying motivation we imagine fin and !out as different components of a. multiresolution representation of a. discrete fun ction. The intention is that the representation at a. coarser scale !out should be possible to derive from the representation at a. finer level fin using convolution. The values K(n) of kernel fun ction]( will sometimes be referred to as filter coefficients, Cn = K( n). If fin and ]( have finite support we can form the vectors fin and fout determined by (fi n)n = fin( n) and (fout)n = !out( n ). Then the convolution transform (3) can be expressed on matrix form: fout = C fin (4) The convolution matrix C is a. matrix with constant values along the diagonals C i,.i = ci- j (i,j = l..N). Such a matrix is called a Toeplitz 6 matrix. The generating function 2 associated with a kernel ]( with filter coefficients Cn is 00 2: <PK(z) = (5) CnZn n =-oo If we replace z by ei0 we obtain the Fourier spectrum 00 '1/JK(O) = 2: (6) CneiniJ n=-oo 2.1 Basic Requirements of Scale-Space Kernels By going from a fine scale to a coarser scale in scale-space the high frequency components of the image should in some sense be suppressed. Formulated in another way we can say that the convolved image in some sense ought to be smoother than the input image. If local maxima and minima are regarded as one measure of smoothness, it is necessary that the number of local extrema in space does not increase from a fin er to a coarser scale. As will be shown later, the famil y of functions generated by the gaussian kernel possesses this quality in the continuous case. In a real implementation one often wants to trace the local extrema when the blurring proceeds from a finer to a coarser scale. This problem does for instance involve pract ical complications about how far extrema can move in space. However, in the present discussion we confine ourselves to the first requirement and state it as a basic axiom. The remaining part of this section will be devoted to investigations of how it limits the class of possible kernels. Definition 1 We denote a one-dimensional kernel Z -+ R a scale space kernel if for all images fin : Z -+ R the number of local extrema in the convolved image !out = ](*fin does not exceeil the numbe7' of local extrema in the original image fin. J( : A complication is involved. If either fin or ! out would happen to have a plateau the question must be raised about how many local extrema the plateau sh ould be counted as. At this moment we will not go into the details of those dubious cases. We count a plateau as one local maximum (minimum) if there are strictly smaller (larger) values bounding it both at the left and at the right, see Fig 2. A more accurate treatment will be performed in Section 3. 2 In some llterature this fun ction is referred to as as the characteristic polynomial. 7 (a) (b) ·f ' . r \ Figure 2: Examples illustrating the definition of local extremum. (a) A local maximum (generic case). (b) A plateau counted as one local maximum. (c) A plateau not counted as a local extremum. As a lo cal extremum in a discrete function f is equivalent to a zero crossing in its first difference !J.f defined by (LJ.f)(x) = f(x + 1) - f(x) the definition can be expressed in terms of zero-crossings by replacement of the string "local extrema" with "zero-crossings" . This result follows since the difference operator commutes with the convolution operator. 2.2 Results from the Impulse Response By considering the impulse response it is possible to draw some qualitati ve conclusions about the properties of a scale-space kernel. Let 1 if X= 0 f in (X ) = {j (X ) = { 0 otherwise (7) Then fout(x) =(I( *ti)(x) = K(x) (8) o( x) has exactly one local maximum and no zero-crossings. Therefore in order to be a scale-space kernel J( must not have more than one extremum and no zero-crossings. Thus, • All coefficients must have the same sign. • The coefficient sequence {K(n)}~=-oo must be unimodal3 . We summarize the results in 3 A real sequence is called unimodal if it is first ascending (descending) and then de scending (ascending). 8 • • • • • • • • n Figure 3: The filter coefficient sequence {Cn}~=-oo of a discrete scale-space kernel must be positive and unimodal. Proposition 1 Let {en}~=- oo be the coefficients of a discrete kernel with fi nite support. Then a necessary condition for the kernel to be a discrete scale space kernel is that all Cn have same sign and that the sequence {en}~=-oo is unimodal. Without loss of generality we therefore can restrict the rest of the treat ment to positive sequences where all Cn 2: 0. It seems reasonable to require that ]( E it, i.e. that L~ - oo Jcnl is finite . If f in is bounded and ]( E l 1 then the convolution is well-defined. If, in addition, fin E l1 then also ! out Eh. Further, this assumption guarantees that the Fourier transform of the filter coefficient sequence exists. It also allows u s to normalize the coefficien ts such that L~=-oo en = 1. Particularly the filter coefficients en must then tend to zero as n goes to infinity. 2 .3 A C lass of D iscrete Scal e-Space Kerne ls with Finite S u pport Consider a two-kernel with only two4 non-zero filt er coefficients: ](( 2 )(n) = if n = 0 q ifn = 1 { 0 ot herwise Assume that p 2: 0, q > 0 and p rpJ((2)(x) = p + qz . p +q (9) 1. The generating function is 4 The superscript within parentheses used in the notation for kernels stands for the number of non-ze ro filter coefficients. 9 fin(x) r••,(x) x-1 _q_ _q_ p-+<j p-+<j x+l X Figure 4: To convolve an image fin with a two-kernel J( (2 l (n) is equivalent to to form a weighted average of the sequen ce {f;n(x )}~_ 00 • It is easy to verify that the number of zero-crossings (local extrema) in ! out = ]((2 ) * fin cannot exceed the number of zero crossings (local ex trema) in fi n · This result follows from the fact that convolution of f in with 1((2) is equivalent to the formation a weighted average of the sequence {f;n(x)}~:::::- oo' see Fig 4. The values of the output image can be constructed geometri cally and will fall on straight lines connecting the values of the in put image. The offset along the x- a..x.is is determined by the ratio qf(p + q). One reali zes that no additional zero-crossings can be introduced by this transformation . By repeated application this result we can formulate Proposition 2 All ker·nels with a generating funct ion on the form N cp(z) = c zk I1 (Pi + q;z) (10) i:::::l where P·i > 0 and q; > 0 are discrete scale-space ke rnels. Proof: lly direct application of Definition 1 we can state a lemma. Lemma 1 If two kernels K a. and J(b are scale-space kernels then the com posed kernel K a. * J(b is also a scale-space kernel. Then take an arbitrary image f in and convolve it with a series of two-kernels 2 K f > (i = l.. N), each one having a generating fun ction cpK(2)(z) =Pi + 'JiZ I 10 (Pi > 0 and qi > 0). (11) Ki 2 Let J((N +l) = ) * ](~ ) *... * ]{~) . As linear convolution is an associ ative operation repeated convolution of fin with the kernels ](?) is equivalent to 2 ) is a scale-space kernel J((N+l) convolution of f in with J((N +I). As each must also be a scale-space kernel according to the lemma above. Further, as convolution in the spatial domain is equivalent to multiplication in the domain of the generating func tions it follows that 2 Jd (12) is the generating function of a scale-space kernel. A constant scaling-factor cor a translation IPtranst(z) = zk cannot affect the number of local extrema. Therefore these factors can b e multiplied onto IPJ<{N+t>(z) without changing the the scale-space properties. 0 T he filter coefficien ts ge nerated by (10) can be regarded as a kind of gener alized binomial coefficients. T he ordinary binomial coefficients a re obtained, except for a scali ng-factor , as a special case if all Pi and qi are equal. Another formulation of Proposition 2 is also possible. Proposition 3 Let c_m, ... , c_I, eo, c1, ...en be the coefficients of a discrete kernel with finite suppor·t. Then a sufficient condition for the kernel to be a scale-space kernel is that all roots of the genemting function cp(z) = c_mz - m + ... + c_lz- 1 + eo + c1z + ... + CnZn (13) are real and non-positive. Proof: Let k = - m, N = n+ m in (10). If all roots of c,o(z) are real and negative then (10) in Proposition 2 must be the factori zation of ( 13). 0 T he Fourier spectrum of a symmetric sequence on the form (10) has some interesting properties . The most general generating function of such a sequence can be written as N cp(z) = C IT (Pv + qv z)(Pv + qvz- 1 ) v=l 11 (14) Consider one factor (p11 + q11 z)(p11 t.p 11 ( + q z-1 ). Its Fourier sp ectrum is ei0 ) = (Pv + qvei 8 )(Pv + qve-i8 ) = p~ + q~ + 2pvqv cos 0 11 (15) On the interval [-1r, 1r] this function is non-negative. It assumes its maxi mum value (p 11 +qv )2 for() = 0 and its minimum value (Pv- qv) 2 for() = ±1r. c.p 11 ( ei 8 ) is monotonically increasing on [-1r, 0) and monotonically decreasing on [0, 1r), i.e. unimodal. It is easy to show that any finite product of non-negative increasing (de creasing) fun ctions is also increasing (decreasing). Subsequently, the Fourier spectrum c.p( ei8) of a symmetric kernel on the form (10) is non-negative and unimodal on the interval [-7r, 1r]. In the following three sections we will show that the Fourier spectrum of any symmetric scale-space kernel must possess these qualities. 2.4 A Restriction on the Sign of the E igenvalues of the Convolution Matrix If the convolution matrix has negative eigenvalues then the corresponding kernel cannot be a scale-space kernel. Proposition 4 Let c_m , ... ,c_1,Co,C1, ...cn (c_m =f 0, Cn =f 0} be the coef ficients of a discrete kernel with finite support. Then a necessary condition for the kernel to be a discrete scale-space kernel is that no convolution ma trix c(N) = banddiagNxN((c_m, .. . ,eo, ... ,en)) of any dimension N > 0 has a real negative eigenvalue with a correspondi ng real eigenvector. Co C-1 C1 Co c(N) = C_m C-1 C-m Cn C_m Cn (16) C- m Cn Cn eo C1 c_1 eo Pm·ticularly if the kern el is symmetric c_i = Ci V i then all eigenvalues must be real and non-negative. 12 (a) (b) (c) ~ , _l_ \ Figure 5: (a) The eigenvector V. (b) The components of indices l..N. (c) The corn ponen ts of ]{ * f in . c(N) V having Proof: Because of Proposition 2 it is sufficient to study kernels with only non negative filter coefficients. Assume that C(N) has a real negative eigenvalue for some dimension N with corresponding eigenvector v. Let a and f3 be the indices of the first and the last non-zero components of v. Construct an extended input image fin such that (see Fig 5): fin(x)= { 0 if x <a Vx ifa$x${3 0 if X > (17) f3 Then the first N components (with indices l..N) of C(N)v and]{ *fin will be pairwise equal. As v is an eigenvector with a negative eigenvalue it follows that the components of c(N) v and v have opposite signs. This means that v, C(N) v and ]( * fin all have the same number of internal zero-crossings provided that we observe only the components with indices l..N . The reversal of the components with indices a and f3 guarantee that an additional zero-crossing will occur. If fin(a) is positive (negative) (K * f in)(a) will be negative (positive). As all filter coefficients are positive it is always possible to find at least one positive (negative) component of ]( *fin for a sufficiently small index value. The component with subscript a- m- n+ 1 does certainly serve our purpose, since it is equal to c_mfin( a). Consequently, we have found an additional zero-crossing between this posi tion and position et. The same argument can be carried out at the right end point producing another scale-space violation. 0 13 2.5 A Restriction on the Sign of the Fourier Spectrum A theorem by Toeplitz [23](8] relates the eigenvalues5 of an infinite Toeplitz matrix C with elements Ci,j = Ci-j to the the values of the generating function associated with the sequence of filter weights. Assume that <p(z) = 2::~= - oo CnZn is convergent in the ring r < lzl < R, where 0 < r < 1 < R. Then the eigenvalues of C coincide with the set of complex values that <p(z) assumes on the unit circle lzl = 1. It allows us to derive an interesting Corollary from Proposition 4. Proposition 5 Letc_n , ... ,c_l,co,cl,···cn whereC±n ::/; 0 be the coefficients of a symmetric d iscrete kernel with finite support. Then a necessary condi tion for the kernel to be a discrete scale-space kernel is that the generating function <p(z) = 2::~- oo CnZn assumes no real negative values on the unit circle in the complex plane. In other words, the Fourier spectrum must be non-negative. Proof: Let >.~N) denote the smallest eigenvalue of the convolution matrix of dimension N and let m denote the minimum value6 the generating function <p(z) assumes on the unit circle in the complex plane. As a consequence of a theorem by Grenander [8] p65 about the asymptotic distribution of eigenvalucs of a finite Toeplitz matrix it follows that lim ),(N) N -.oo 1 = m >. (N) > m 1 - (18) Assume that <p assumes a strictly negative value on the unit circle. Then m is strictly negative. As limN-.oo >.~N) = m it follows that for any e > 0 >.iN) must be contained in I= [m, m+ e] when N exceeds some sufficiently large integer Ne. e is chosen small enough such that m+ e < 0. According to Proposition 4 the kernel cannot be a scale-space kernel. 0 2.6 A Unimodality Requirement for the Fourier Spectrum of the Coefficient Sequence In this section we will show that the Fourier spectrum of a symmetric scale space kernel is unimodal on the interval [-rr, rr] . The core of t he proof lies in 5 A is called an eigenvalue of an infinite matrix C if the matrix C-A] has no bounded inverse. I denotes the unit matrix. 6 Due to symmetry of the kernel <,:>(z) assumes only real values on the unit circle. The minimum value d oes certainly exist since <,:>(ei8 ) is a conti nuous function and the interval [-rr,rr] is compact. 14 Lemma 3. The remaining technicalities are all aimed at one thing, namely to avoid problems at the boundaries of the matrix representation of the convolution transformation. We start with the an important observation concerning eigenvectors and eigenvalues. In order to express it clearly we need to distinguish between different kinds of zero-crossings. A zero-crossing is said to be simple if it corresponds to one of the sign sequences ( + - ), (- +), (+ 0 -) and (- 0 +). The number of zero-crossings in a vector x will be denoted V(x). Lemma 2 Let A1 and A2 be real eigenvalues of an x n matrix A and let v1 and v2 be the corresponding eigenvectors. Assume that V(v2) > V(vt) and that v 1 and v 2 have only simple zero-crossings and no zero end elements. Then in order· for y = Ax to be a scale-space transformation for all x it is necessary that IA2I ~ IAtl· Proof: Assume that A2 > A1 . The scale-space properties are not affected by a scaling-factor. Therefore, we can equivalently study B = ~A. For both eigenvectors we define the the largest and smallest absolute values v(absmin) and v(absmax) by v(absmin) =. min jv;j ; v(absmax) = . max t=l..N, l=l..N, v;;o!'O lvd (19) v;;o!'O Let x = v2 + a:v 1 where a: is chosen large enough such that V(x) = V(vt)· This can always be achieved if la:! v~absmin) > v~absmax). Let Iv 1 be the set of all integers i E [1, n] satisfying i E Iv 1 => (vi )i f. 0. Then V i E Iv 1 the signs of (x)i and (v 1 )i will be equal. As all zero-crossings in v 1 are simple and no end elements are zero it follows that no additional zero crossings can be introduced in x. Nor can any zero-crossings be deleted. Thus, V(x) = V(vt). Then consider Bx = ]2 ( a:A 1 v 1 + A2v2) and study Bkx =a:(~:) kv + v2 1 (20) For a fixed value of a: we can always find a sufficiently large value of k such that V(Bkx) = V(v2 ). In a similar manner to above one verifies that the condition la:lj~jk viabsmax) < v~absmin) suffices. Subsequently, V(Bkx) > V(x) which shows that the transformation induced by Bk is not a scale space transformation. Therefore, B cannot be a scale-space kernel since at 15 (a) (b) Figure 6: (a) Input im age consisting of a low frequency component of high amplitude and a high frequency component of low amplitude. (b) In the output image the low fr equency component h as been suppressed while the high frequency component remains unchanged. As we see, additional zero crossings have been introduced. least one scale-space violation must have occurred in the series of k successive transform ations. 0 T he essence of this result is that eigenvectors with strong variation must correspond to small eigenvalues. The next step is to relate the Fourier sp ec trum of a kernel to the eigen values and the eigenvectors of the corresponding convolution matrix. We start by showing that a convolution transformation equivalently can be described as a multiplication with a circulant1 matrix provided that both the kernel and th e input image h ave finite support. Let ]( b e a given symmetric ke rnel with Ci = 0 if li l > n and let f in be an arbitrary im age with fi n(x) = 0 if lxl > m. Defin e vectors fin and fout by (fin )i = fin(i) (i = O,±l , ... , ±M) and (fout)i = f out (i) (i = O,±l, ... , ± Af) for some M > m+ n. Then , the effect of the con volution transformation ! out = ]( * fin can equivalently be expressed on matrix form fout = CTfin > where CT is a Toeplitz matrix with compo nents (CT)i ,j = Ci-j (l:,j = O, ±l, ... ,±M). T his follows as the components with indices O, ±l, .. ., ±M of ! out and fout are equal and fout(x) = 0 when lxl >M. However, since th e first and last n components of fin are zero; t he values of the matrix elements in the upper right and lower left corners do not affect fout· Therefore, we can replace them with some appropriately selected filter 7 In a circulanL maLrix each row is a circular s hin of Lhe preceding row except for the firsL row which is a circular shin of Lhe last row. 16 coefficients such that the matrix becomes circulant8 . Hence, the effect of the convolution transformation can also be written fout = Ccfin • where Cc is the (2M + 1) X (2M + 1) circulant matrix: eo Ct Ct eo Cn Cn Ct Cn Ct Cn Cn Cc= Cn (21) Cn Cn Cn Cn Ct Cn Cn eo Ct Ct eo It is easy to establish the unimodality property for transformations expressed with circulant matrices since the eigenvalues and eigenvectors can be deter mined analytically. Lemma 3 Let {cJ.L}~=-oo be the filter coefficients of a symmetric discrete kernel with cJ.L = 0 if IJ.LI > n. For all integers M 2: n it is required that the transformation given by multiplication with the (2M +1) X (2M +1) symmet ric circulant c~n, defined by (CbM))i,j = Cj- j (i,j = o..M) and circulant extension {21}, should be a scale-space transformation. Then, necessarily the Fottrier spectrum 1/J( 0) = 2:~=-oo cJ.LeiJ.LB must be unimodal on [-rr, n"]. Proof: vVe will use the explicit expressions for the eigenvalues >. m and eigenvec tors V m of cbM). By verification one shows that (m= -M..O..M) (m,k = -M..O..M) (22 ) (23) As the kernel is symmetric all eigenvalues except for >. 0 are pairwise degen erated P-m = >.m)· Hence, we can choose a new basis of eigenvectors by 8 To express the convolution transformation with a circulant matrix is equivalent to periodic extension of the input function 17 1/;(0) 0 Figure 7: If the Fourier spectrum is not unimodal on [-1r, 1r], i.e if there exist (}2 > 01 in [0, 1r] such that 7f;(02) > 7f;(01 ) then the corresponding transformation cannot be a scale-space transformation. linear combination of the old ones. Let Vm = -tCY-m - Ym ) (m= -M.. -1) and Vm = HY-m + Ym) (m= o.. M). Then, (vm)k . ( 21r1nk ) =Sill M 2 +1 21rmk ) (Vm)k = cos ( 2 M +1 (m= -M.. -1 ,k = -M..O..M) (24) (m= O.. M,k = -M..O..M) We note that the number of zero-crossings in an eigenvector increases as lml increases. Subsequently, V(vm 2 ) > V(vm 1 ) if lm2l > lm1l· Introduce the notation 7/J( B) = <p( ei8 ) for the Fourier spectrum. By comparison of (5) and (22) we see that Am= 7f;( 2~Y~\), i.e. the eigenvalues of CbM) are given directly by the Fourier spectrum and a larger value of lml corresponds to a larger absolute value of the argument () to 7/J. Now , assume that the Fourier spectrum is not unimodal i.e that there exist some 82 > (} 1 in [0, 1r] such that 7f;(B2) > 7f;(B 1 ), see Fig 7. (Without loss of generality we can presuppose that 1/J is non-negative on [-1r,1r], because otherwise , according to Proposition 5, J( cannot be a scale-space kernel.) Then , as 1/J is a continuous function of() it is possible to find some sufficiently large integer M such that there exist ()-1 = ;;,~11 and 01 = ;tt~11 satisfying 7f;(02) > 'lj;(0-1 ) for some integers ni2 > ni1 . To summarize, CbM) has eigenvalues Am2 > Am1 and corresponding eigenvectors with V(v 1112 ) > V(v1111 ) since ni2 > nit. From (24) we see that all zero-crossings in the eigenvectors are simple and that no end elements in them are zero. Consequently, according to Lemma 2 the transformation given by c2J"1) cannot be a scale-space transformation. () 18 The last step is to generalize this result to general (non-circulant) con volution transformations. Note that Lemma 3 cannot be used without mod ification, since the eigenvectors involved are non-zero in the region where f in is required to be zero. If f in is non-zero in that region the description with a circulant matrix is no longer equi valent to the description with a Toeplitz matrix. This means that the superposition of eigenvectors used in the proof of Lemma 2 is not feasible as input image for the transformation fout = Crfin· Actually, what Lemma 3 expresses is the desire that if fin is a periodic fun ction then the number of local extrema in one period should not increase from a finer to a coarser level of scale. In view of the last observation we can reformulate Lemma 3 in terms of p eriodic functions. With the number of local extrema (zero-crossings) in one period of a periodic function f with period T we mean the number of local extrema (zero-crossings) in the sequence f(O),J(1), ... , f(T). Lemma 4 Let ]( be a symmet1·ic kernel with finite support, i.e. c"' = 0 if IJ.LI > n. A ssume that for all periodic images fin with period larger than 2n + 1 the number of local extrema in one period of the convolved image ]( * fin does not exceed the number of local extrema in one period of the original image. Then, the Fourier spectrum 1/;(0) = L:::::"=-oo c"' eip.O must be unimodal on [-1r, 7i). Proof: As f in is periodic !out = J( *fin will also be periodic with the same period. Subsequently, it suffices to study one p eriod of each function. We only need to create a counter-example. Therefore, we can restrict the analysis to the case where the period T is odd, i.e. T = 2M + 1 for some integer M. Define vectors fin and fout by (fin)i fi n(i) (i 0,±1, ...,±M) and (fout)i = f out(i) (i = 0, ±1 , ... , ±M). Then, the effect of the convolution transformation ! out = ](*fin can equivalently b e expressed on matrix form fout = Ccfi 0 , where Cc is the (2M + 1) X (2M + 1) circulant m atrix with components (Cc )i,j = Ci-j ( i, j = 0, 1, .. . , M) and circulant extension (21). Assume that th e Fourier spectrum is not unimoda l on [-7r,7r). Hence, according to Lemma 3 it is possible to find some sufficiently large M such that the transformation given by multiplication with is not a scale space tra nsformation. T here is one difference between the non-p eriodic a nd the periodic case that must b e noted. Namely t hat the number of local extrema in one pe riod of a periodic function includes "wrap-around". However, the proof of Lemma 3 can b e carried out in this case as well. All we have to do is to = = eft> 19 (b) , ~A~ '\['r'\)V n .. w • T (2n+l)T \~t\~ '\}'f '\} ~ n Figure 8: Construction of the image fin from the periodic fun ction /per · replace the definition of V(x) and remove the requirement "no zero end ele ments from Lemma 2. Consequently, if the Fourier spectrum is not unimodal the number of local extrema in one period will increase. 0 From this result the generalization to general convolution transforma tions can be obtained by construction of a counter-example. Proposition 6 Let {c'"'}~=-oo be the filter coefficients of a symmetric dis crete kernel ]( with finite support. Then a necessary condition for the kP.r nel to be a discrete scale-space kernel is that the Fourier spectrum tj;(O) = L~=-oocl-'ei 0 ~-' is tmimodal on [-rr,rr]. Proof: Assume that]( is a symmetric discrete kernel with non-unimodal Fourier spectrum and finite support, c'"' = 0 if IJ.LI > n. Then, according to Lemma 4 it is possible to find a periodic function /per of some period T such that the number of local extrema in one period of the convolved image exceeds the number of local extrema in one period of the original image. Let I2n+t be an interval with 2n + 1 consecutive periods and construct a new function fin, which is equal to /per on l2n+t and at then closest points at each boundary, see Fig 8 . At all other points f in should be zero. Due to the construction of fin and the finiteness of ]( it follows that ]( * fin and ](*/per will be equal on hn+l· Thus, provided that we only count the points in I2n+t we have introduced at least 2n + 1 additional zero-crossings. Outside hn+t we might expect to find more zero-crossings in ](*fin· The support region of J( *fin is general larger than the support region of fin· 20 However, fi n cannot have more than a total of 2n additional zero-crossings since fin is non-zero only at 2n points outside h n+l· Consequently, ]( * fin contains at least one zero-crossing more than fin, which shows that]( cannot be a scale-space kernel. 0 2.7 A Result for Kernels with Three Non-Zero Elements For a three-kernel ]((3) with exactly three non-zero consecutive elements c_ 1 > 0, eo > 0 and c1 > 0 it is possible to determine the eigenvalues of the convolution matrix and the roots of the characteristic equation analytically. It is easy to verify that the eigenvalues >.,_. of the convolution matrix are all real and equal to fl'lr >.JJ. =eo- 2Jc_tcl cos( - N ) +1 (IL = l..N) (26) and that the roots of generating function <p[((3)(z) = c_ 1 z- 1 +eo+ c1 z are Z1 2 ' = -eo± Jc6- 4c_l c1 2Ct (27) From (26) we deduce that the eigenvalues of a convolution matrix of arbi trary dimension are all non-negative if and only if eo~ 2jc_ 1c1 . (27) says that the roots of the characteristic equation are real a nd non -positive if and only if c6 ~ 4c_ 1 c1 . Thus, the necessary condition in Proposition 4 and the sufficient condition in Proposition 3 both lead to the condition c6 ~ 4c_ 1 c1 and we obtain a complete classification for all possible values of c_ 1 , c0 and c1 . We conclude that a three-kernel with positive elements is a scale-space kernel if and only if it can b e written as the convolution of two two-kernels with positive elements. At this moment one could ask one-self if the result can be generalized to hold for kernels with arbitrary numbers of non-zero filter coefficients. I.e. if all discrete scale-space kernels with finite support have a generating function on the form (10). This question will be answered in the next section. 21 3 Classification of Discrete Scale-Space Kernels Until now we have postulated an axiom in terms of local extrema or equiv alently zero-crossings and investigated some of its consequences for image transformations expressed as linear convolution with a shift-invariant kernel. We have seen t h at the sequence of filter coefficients must be positive and unimodal a nd that its sum should be convergent. For symmetric kernels the Fourier spectrum must be positive and unimodal on (-1r, 1r]. In this section we will perform a complete characterization of the scale space kernels. We have studied the literature and seen that several interest ing results can be derived from the theory of total positivity. Sign-regular kernels and Polya frequency sequences turn out to be important concepts as they characterize the interesting kernels completely. The proofs of the im porta nt theorems are sometimes of a rather complicated nature for a reader with an engineering background. We will not burden the presentation with them but confine ourselves to references. The pioneer in the subject of va riation-diminishing transforms was I.J. Sch oenberg. He studied t he subject in a series of papers from 1930 to 1953 . Later the theory of total positivity has been covered in a monumental mono graph by Karlin [1 3]. A recent paper by Ando [2] reviews the field using skew-symmetric vector products and Schur complements of matrices as ma jor tools. T he questions issued in the pap er constitute a new application to these not too well-known but very powerful results. 3.1 Definitions We will firs t consider general linear transformations of discrete images where t he kernel does not need to be shift-invariant . 00 f out(x) = L K(x, y )f in(Y) (x E Z) (28) y=-oo Two notions of sign changes in vectors will be used. Let x = (xbx2, ... , xn) be a vector of n real numbers. We denote by v-(x) the number of sign cl1 anges obtained in the seque nce x 1,x2, ...,xn if all zero terms are deleted and by v+(x ) t he maximum number of sign ch anges possible in the sequence XI, x2, ... , Xn if each zero value is allowed to be replaced by either + 1 or -1. We use a special convention saying that the number of sign changes in the null vector is - 1. 22 The interesting sequences and kernels will defined in terms of minors. Given a kernel ]( : X x Y ~ R we form minors of arbitrary order r by selections of x 1 < x2 < ... < Xr from X and of YI < Y2 < ... < Yr from Y. The determinant of t he resulting matrix with components {K(xi, Yi )}i,i=l..r will be called "a minor of order r" and denoted by K(x1,Y2) K(x2, Y2) K(xi. Y1) K(x2, Yl) ]( ( X1, X2, ... ,Xr ) YI. Y2, ... , Yr K(xl,Yr) K(x2,Yr) (29) K(xr, Yl) K(xn Yr) K(xr, Y2) The single most important concept is sign-regularity. Y ~ R is sign-regular (SRoo) if all its r-order minors have same sign for every order r from 1 through oo, i.e. if there exists a sequence of constants e 1, e2, ... each +1 or -1 such that Definition 2 We say that]{: X er]( ( for all choices of X1 X X2, ... ,Xr Y1, Y2, ... , Yr X1, < X2 < ... < ) ~O Xr from X and Y1 (30) < Y2 < ... < Yr from Y. In other words sign-regularity means that it is impossible to find two minors of same order having opposite signs. If strict inequality holds for all r then ](is said to be strictly sign-regular (SSR 00 ). 3.2 Characterization Theorem for General Linear Trans formations with Scale-Space Properties General linear transformations having scale-space properties can be fully characterized in terms of sign-regularity. X m real matrix with n ~ m. Then the linear map A from Rm to Rn diminishes vm·iations in sign in the sense that Theorem 1 Let A be an n for all x E Rm x -::/= 0 (31) if and only if A is strictly sign-regular {SSR 00 ). The original proof of this powerful theorem, forming the foundation of the theory for variation-diminishing transforms, can be found in Schoenberg [22]. Ando [2] p201 derives it using skew-symmetric vector products. Another formulation is possible [2] p202 if A is known to be of full rank. 23 Theorem 2 Let A be an n X m real matrix of rank m. Then (32) holds for all x E Rm (x =P 0) if and only if A is sign-regular (SR=)· We note that the condition (32) is equivalent to the formulation we expressed in Definition 1. Consequently, sign-regularity and full rank are the necessary and sufficient conditions for a kernel to be a potential scale-space kernel. 3.3 Characterization Theorem for Convolution Transfor mations with Scale-Space Properties A narrower class of kernels is obtained if all minors are required to be non negative. Definition 3 A kernel]( : X X Y --t R is said to be totally positive (T P=) if all its minors are nonnegative; i.e. if Xt,X2, .•• ,Xp Y1, Y2, ... , Yp Xl < X2 < ... < Xp; Yl < ) ~ 0 < ... < Yp; Y2 (33) p = 1, 2, ... , 00 An important case ofTP 00 -sequences appears if the discrete kernel is shift invariant i.e. if K(x , y) can be written as k(x- y) = Cx -y· Definition 4 We say that {en} ~= -= is a P6lya frequ ency sequence if any minor of the infinite Toeplitz matrix C= eo c1 c- 1 eo c-2 c_ l C2 C1 Co (34) is nonnegative. The importance of the Polya frequency sequences becomes apparent when we require tha t the generating function con verges, which for instance holds if the sum of the filter coefficients is convergent. 24 Definition 5 A P 6lya f requency sequence {cn }~=- oo having a generating function <p(z) = 2:::~=-oo CnZn which converges in an annulus r < lzl < R (0 < r < 1 < R) such that <p(z) =J 0 is called a normalized P6lya frequency sequence. According to a theorem by Schoenberg [21] p363 sign-regularity com bined with the Toeplitz struct ure implies total positivity. Subsequently, Theorem 3 The convolut ion transformation 00 f out(x) = L cn f in(x- n) n= - oo is variation-diminishing i.e. holds for all fin if and only if the sequence of filte r coefficients a normalized P6lya frequency sequence. {cn } ~= - oo is In other words, each shift-invariant discrete scale-space kernel corresponds to a normalized Polya fr·e.quency sequence. There exists a remarkably explicit characterization theorem for the gen erating function of a P F00-sequ ence. It has been proved in several steps b y Edrei and Schoenberg, see [22] or [13]. Theorem 4 An infinite sequence { cn } ~=-oo is a P6lya frequency sequence if and only if its generating fu nction <p( z) = 2:::~=-oo CnZn is of the form (35) (X) c > 0; k; E Z q_b q1 , a;,{3i,/i, 6; ~ 0 ; L (a; + {3; + li + 6;) < oo i=l T he sequence {en } ~=-= is normalized if a nd only if it in addition holds that {3; < 1 and li = 1, see [13] p423. For kernels with finite support q_ 1 , qt, f3i and /i must be zero and the infinite product must be replaced with a finite one. Thus the class of ker nels we arrived at in Section 2.3 (Proposition 2) is exactly t he set of P F00 sequences having finite support, i.e the kernels wit h a generating function 25 on the form (10) are the only finite scale-space kernels. An immediate con sequence of this characterization is that all finite scale-space kernels can be decomposed into kernels with two real strictly positive consecutive filter coefficients. The representation (35) can sometimes be very convenient for further analysis. For example, starting from (35) it is almost trivial to show that the Fourier spectrum of a symmetric P F 00 -sequence, i.e. a general symmetric scale-space kernel, is unimodal and non-negative on the interval [-7r,7r]. Due to the symmetry q_1 = q1, a 11 = 011 and f3v = 'Yv· As a first step one replaces z with iB and shows that each one of the factors e(q- 1 z-t+qtz), (1 +a 11 z)(1 +011 z- 1 ) and (1- /3 11 z )(1- -y11 z- 1 ) is a non-negative and unimodal function of 8 on [-71" , 7r]. The rest is left as an exercise for the reader. As we see, the theory of total positivity provides us with a complete classification as well as a simple and powerful representation of the inter esting kernels. The key result (Theorem 3) from this discussion is worth restatement: Proposition 7 A discrete kernel]( : Z - R is a scale-space kernel if and only if the corresponding sequence of filter coefficients {K(n)}~=-oo is a normalized Polya frequency sequence. 4 Axiomatic Construction of the Discrete Scale Space In last section we studied linear transformations expressed as convolutions and concluded that a discrete kernel is a scale-space kernel if and only if the sequence of filter coefficients {K(n)}~=-oo is a normalized P6lya frequency sequence. \Vith this result in mind , an apparent way to get a multireso lution representation of an image f is to define a set of discrete functions Li (i = 0.. n) 'vhere Lo = f and each coarser level is calculated by con vo lution from the previous one Li = J(ir-i- 1 * Li-1 (i = l..n). The kernels J(i +-i- l should be approp riately selected scale-space kernels corresponding to suitable amounts of blurring. The scale-space condition for each kernel guarantees that the number of extrema at a coarser level (larger value of i) does not exceed the number of local extrema at a finer level. From empir ical results [16] it is known that not more than approximately seven levels of scale are relevant in human perception of typical real-life images. This provides a good motivation for this so-called sampled scale-space with a dis 26 crete scale parameter. However, in general the locations of those levels are not known a priori. The goal in this section is to tie together scale-space kernels correspond ing to different degree of smoothing in a systematic manner such that a continuous resolution parameter can be introduced. The concept of a con tinuous scale parameter is of considerable importance since, we will no longer be locked to fixed discrete levels of scale. It allows us to defocus images with arbitrary amount of blurring, which will certainly make it easier to locate and trace events in scale-space. Of course, it is impracticable to generate the representations at all levels of scale in a real implementation. However, the important idea is that any representation can be calculated if desired. We will not consider the question about how to choose a suitable set of scale levels in a practical case. Imagine for instance that we want to trace events, like local extrema or convex and concave regions, as the blurring proceeds in scale-space. In order to analyze the behaviour in scale-space, the continuum of multiresolution representations must be sampled at some levels of scale. It is certainly a non-trivial problem to make an appropriate selection of these levels. It seems very reasonable that the sampling rate along the scale direction should depend on the behaviour of the image as a function of the space coordinates. If the representation at some level of scale is relatively smooth we should be able to allow a larger scale step than if it were strongly varying. We will thus be lead to methods that automatically regulate the scale step, based on the interconnection between the appearance of the family of images as a function of the space coordinate and as a function of the scale parameter. The point with a scale-space with a continuous scale parameter is that it provides a theoretical framework for the development of such algorithms. We do not need to select the scale levels a priori, but can leave the decision open to the actual situation. As an illustrative background we start by deriving the continuous scale space for one-dimensional im ages starting from the variation-diminishing property. Then, the discrete scale-space will be constructed with a. similar approach. 4.1 A Continuous Formulation which Leads to the Gaus sian Kernel Koenderink, van Doorn [11] derive the heat equation, or equivalently the gaussian kernel, from three assumptions- causality, homogeneity and isotropy - using differential geometry. Another way to reach to the same conclu 27 sion is to postulate that the scale-space for a continuous image should be constructed by convolution with a one-parameter family of kernels . The parameter should, of course, describe resolution such that a higher param eter value corresponds to a more blurred representation of the original im age. A representation at a coarser scale t2 should be possible to calculate from the rep resentation at a fin er scale t 1 (t 2 > t 1 ) by convolution with a kernel from the family. This can b e formulated as a semi-group property T (·; s) * T(·; t) = T(·; s + t). We require each kernel to be shift-invariant and to possess the same variation diminishing property as a discrete scale space kernel i.e. we require that the number of local extrema9 in a convolved image must not exceed the number of local extrema in the original image. The latter two conditions lead us to study the Polya frequency fun ctions10 , the continuous correspondence to the P6lya frequency sequences. As in the discrete case this class of function s does precisely consist of continuous shift invariant scale-space kernels, see [9] p83-84. A theorem by Karlin [13] p354 shows that these conditions uniquely define the gaussian family of kernels. Theorem 5 Let kt(x): X X R+--+ R (0 < t < oo) denote a one-parameter family of P6lya frequency functions integrable on the real axis and fulfilling the semi-group property (36) Suppose also that kt( x) is Borel-measurable as a function oft. Then, nec essarily - oo < x < oo; t > 0 6 E R (37) Consequently, this theorem provides both a new formulation of the one dimensional scale-space theory for continuous images as well as a further justification of the selection of the gaussian kernel as the canonical contin uous scale-space kernel. The assumption of Borel-measurability means no 9 In the continuous case the va riation-diminish.i ng p roperty is normally expressed in terms of zero-crossings. Thus, this formulation is valid only if the differentiation oper ator commutes with the convolution operator. In problems occur we prefer to base the discussion on zero-crossings instead. 10 A continuous function k : R - R is said to be a Polya frequency function if the function f( : R x R - R defined by K(x , y) = k(x- y) is totally positive i.e. if all minors on the form (29) are non-negative for arbitrary selections of Xt < X2 < ... < Xr and Yt < Y2 < ... < Yr from R. 28 important restriction. It is well known that all continuous functions are Borel-measurable. In this one-dimensional case we based the foundation on the following fundamental assumptions: • As we go from a finer to a coarser level of scale (increasing t) the number of local extrema must not increase. • The smoothing method must be independent of both position and the image values (linear shift-invariant filtering). • The scale-space family of functions should be constructed from a one parameter family of kernels possessing a semi-group property. The last condition makes it possible to calculate the representation L(·; t2) at a coarser level t 2 from the representation L(·; ti) at a finer level t 1 (t 2 > ti) using convolution with a kernel from the one-parameter family. In summary, L(·; t2) ={definition}= T(·; t2) * f ={semi-group}= (38) (T(·; t2- tt) * T(·; t1)) * f = {associativity} = T(·; t2- ti) * (T( ·; tl) *f)= {definition}= T(· ; t2- t 1 ) 4.2 * L(·; t 1) The Discrete Analog of the Gaussian Kernel When we construct a scale-space for discrete images we proceed in the same way as in last section. Let us be a little more formal this time. Given any one-dimensional discrete image f : Z - R we define a derived one parameter family of discrete images L : Z x R+ - R. It is this entire family of functions regarded as one unit that we denote as the scale-space. The parameter t is meant to somehow describe the resolution at the current level. We start at the finest resolution level available and let L(x; 0) = J(x) (x E Z) . The last two assumptions mentioned in the end of last section lead us to postulate that the family should be generated by convolution with a one-parameter family of di screte kernels T: Z x R+ - R, 00 L(x; t) = L T(n; t)f(x- n) (x E Z, t > 0) (39) n =-oo and that the family must possess the semi-group property T ( ·; t ) *T(-; s) = T(· ; t + s). The firs t two assumptions from last section combined wit h the 29 complete characterization of shift-invariant scale-space kernels in Proposi tion 7 leads to the requirement that each kernel in the family must be equivalent to a normalized P F 00 -sequence. A discrete version of Theorem 5, also by Karlin [13] p450, says that under these conditions only one class of discrete functions is possible. Theorem 6 The only semi-group of normalized P6lya frequency sequences has a genera ting function on the form (40) Another reasonable requirement is that the kernel in no way should prefer left to right or the other way round. Thus, we only consider symmetric kernels where c_i = Ci V i. Then the generating function must necessarily satisfy cp( z- 1 ) = cp( z) which in our case leads to q_ 1 = q1 . For simplicity we let q_ 1 = q1 = ~. This means no serious restriction since it only affects the scaling of the scale parameter t. Then, 'Pt(z) = ef(z-l+z} = 00 L In(t)zn ( 41) n=-oo As filter coefficients In we recognize the modified Bessel functions of integer order n, which are the solutions to the differential equation: d2w t2 dt2 dw + tdt- (t2 + n2)w = 0 (42) These functions with real arguments are except for an alternating sign se quence equal to the ordinary Bessel functions l n of integer order with purely imaginary arguments. They can also be expressed as an infinite sum. In this context i denotes the imaginary unit. n . ln(t) = L n(t) = ( -1) ln(zt) = oo L k=O (tj2)n+2k k' ( k)' (n 2:: 0, t > 0) • n + · (43) It has been pointed out by Norman [17] that this kernel is the proper di screte analog of the gaussian kernel. We obtain a normalized kernel if we let T : Z x R+ - R be d efined by (44) This is easily understood by setting z to 1 in the generating function ef (z - l+z} = L~= -oo ln(t)zn . Then it follows that L~=-ooln(t) =et, which means that 30 L~=-oo T(n; t) = 1. The semi-group p roperty is trivially preserved after normalization. For the special case t = 0 it holds that 1 if n = 0 In(O) = Ln(O) = h(n) = { tl . 0 o 1erw1se (n 2: 0) Thus, the convolution expression (39) is valid for t = 0 as well. We summarize this very important conclusion in the following proposi tion. Prop o s ition 8 Given any one-dimensional image f : Z --+ R let L : Z X R+ --+ R be a one-pammeter family of func tions defined by L(x; 0) = f(x) (x E Z) and L(x; t) = L~= -oo T(n; t)f(x- n) {x E Z, t > 0), where T : Z X R+ --+ R is a one-pammeter family of symmetric functions satisfying the semi-group property T(·; s)*T( ·; t) = T( ·; s+t) and the normalization cr·iterion L~=-oo T(n; t) = 1. For all images f it is required that if t2 > t1 then the number of local extrema in L(x; t2) must not exceed the number of local extr·ema in L(x; t1). Then necessarily, T(n; t) = e-atln(at) for some non-negative real a, where I n are the modified Bessel f unctions of integer order. Conseq uently, these continuous and discrete results provide us with explicit controlled methods to preserve structure in the spatial domain as we let the original image erode by blurring it to coarser levels of resolution through scale-space. 5 Numerical Imple me ntation of the Discrete Scale Space According to the definition of the scale-space for discrete images the repre sentation of an image f at a scale-level t is given by, 00 L(x; t) = I: T(n; t) f (x - n) n=-oo (x E Z, t > 0) ( 45) where T(n; t) = e-tin(t). When this transformation is to be implemented computationally a few numerical problems must be considered: • The infinite convolution sum must be replaced with a finite one. 31 • Normally, the modified Bessel functions are not available as standard library routines. Therefore, we must design an algorithm to generate the required filter coefficients T( n; t) for a given value oft. • A realistic image is finite, but a finite approximation of (45) might need additional values. In this section we will discuss the first two items. We will not go into the complications11 , which arise from finite images. Instead we assume that f is defined for all those integers, where image values are required for our algori tluns. 5.1 Truncation of the Infinite Convolution Sum A reasonable approach to approximate (45) is to truncate the infinite sum for some sufficiently large value of N, N L(x; t) ~ L T(n; t)f(x- n) (x E Z, t > 0) ( 46) n=-N chosen such that the absolute error in L due to truncation does not exceed a given error limit C:trunc· If we assume that f is bounded (lf(x)l ~M) we get the sufficient condition 00 2M L T(n; t) :S C:trunc (47) n=N+1 Formally, if f is not defined for all integers the convolution {45) is undefined. If we instead use the finite approximation ( 46) the values of L become undefined as soon as a part of the convolution mask reaches outside the given image. Thus, the size of a blurred image will diminish when we go to coarser levels of scale. However , in general the filter coefficients in the outer regions of the kernel are relatively small compared to the central ones, which means that the value of L should not be very much affected if only a minor part of the convolution mask stretches outside the given image. With this motivation in mind it is possible that we could get a better result by extending f in some way. Several ad hoc extension methods have been invented, but they seem to suffer from the intrinsic disadvantage that the quality of result depends very much upon the image. It is also possible to use a genuine finite approach. A main issue, which then has to be answered, is how the finiteness should be treated i.e. if a finite image should be regarded as a fixed-size photograph or a window to the infinite real world. In some sense the problems due to infinite images seem somewhat artificial. One way to avoid them could be to use an alternative camera geometry, where the image values are mapped onto a sphere/circle instead of an infinite plane/line. 11 32 In order to estimate N as a function of t and eiXfc we need a simpler expression for the remainder. In [15] Section A.2 it is shown that (48) where oo SN+ 1 = 1 "" +~ 'YN+1 n _ 'YN+1 fli==l (N + 1 + i) - l + N + 2 + (N 2 'YN+1 + 2)(N + 3) +... ( 49 ) and I+~ t IN+ 1 = e \lh;~ 2 . _(_1_+-~V-N+_\_1)-N=-:-+-:-1 ' 1 aN+1 = --;===::===== 2 (Ztt) + (N~1f (50) If fS'.t~ < 1 the sum ( 49) can be estimated by a comparison with a geometric series. Then, (51) Otherwise it can be calculated from the recurrence relation n+l Sn+1 = - - (Sn- 1) 'YN+l So= e'YN+I (52) Starting from these expressions it is not difficult to write an algorithm , which for a given value oft returns a sufficiently large value of N. The criterion is not sharp 12 and will in general overestimate N. However , once a sufficient number of filter coefficients has been calculated, it is easy to determine how many that are actually needed from the condition L,~==- N T(n; t) 2: 1- !wL= 2M · 12 We do not claim that the estim ation given by (47) and (48) is optimal in any way. However, it is sufficient for our purpose of implementation. 33 5.2 Calculation of the Filter Coefficients Given N, the next step is to calculate T( n; t) (n = - N .. N). It is not particularly efficient to use the series expression ( 43) since the sum converges slowly for large values oft. Instead we make use of recurrence equation. 2n l n- I(t)- ln+I(t) = -tl n(t) (53) This relation is always unstable for upward recurrence , but stable in the reverse direction. We use Miller's algorithm [18] p142 and start the recur rence with an arbitrary seed IN.tart = 1 and I N.tart+ I = 0 for a sufficiently large start index Nstart· As n decreases the iterates obtained from (53) will successively approach the correct solution. One can show (see [18] p175,711) that if d significant digits are required in IN it is enough to start at Nstart = 2(N + ch/N) (54) The sequence of iterates can be normalized if ! 0 ( t) is calculated by a separate routine. A sketch of an algorithm generating the filter coefficients is given in Appendix A.2. Tables with coefficient values for a few values of t are supplied in Appendix A.l. 5 .3 Asymptotic expressions for T(n; t) We close this section with some asymptotic expressions for the filter coeffi cients. 1 T(n; t) ~ J7C; (55) (n < < t) y 27rt T(n; t) 6 ~ -e-t1 n. (t) - 2 n (n > > t) (56) Discrete Scale-Space Properties of Some Nu merical Approximations of the Continuous Scale Space Theory In this section we will consider some numerical approximations, which are close at hand for the convolution integral (1) and the heat equation (2). Using the results derived in previous sections we will investigate if the oc curring transformations possess scale-space properties in the discrete sense. 34 The aim is to analyze the previously commonly adapted approach where the filter coefficients are set to sampled values of the gaussian kernel. We show that some undesired effects occur, mainly due to the fact that the semi group property does not hold after discretization. We also show that the transformation obtained by convolution with the presented discrete analog of the gaussian kernel is equivalent to the solution of a discretized version of the heat equation. This provides another motivation for the selection of T as the canonical discrete scale-space kernel. The rendering is of necessity somewhat technical and can be skipped by the hasty reader. 6.1 The Rectangle Rule of Integration Maybe the most obvious way to approximate the convolution integral L(x;t) = l oo 1 2 r;c:;e-f. / 2tf(x- Od~ f.=-oo V 27rt (x E R,t > 0) (57) numerically is to first truncate the interval of integration at sufficiently large values and then apply the rectangle rule of integration to the remaining part. L(x;t) ~ f N+J.2 jf.=-(N+~) g(~;t)f(x- Odf. (x E R ,t > 0) Preferably N is chosen as an integer. If it is known that (58) f is bounded (lf(x)l :S M) we can choseN such that the error13 in L(x; t) due to trun cation of the integral does not exceed a given error limit E:trunc · r-(N+~) }f.=-oo I r oo I g(~; t)f(x- Od~ + }f.=N+~ g(~; t)f(x- ~)d~ :S E:trunc (59) This inequality does certainly hold if 2 *M* erf( -(N + !)f,fi) :Se:, where erf denotes the error function 14 . The rectangle rule of integration then gives 13 Approximating Lex; t) numerically raises another question, which we however will make no attempt to answer. Given that ftn is bounded ll(x)l $ M and afflicted with an error not exceeding t:in and that we accept errors of magnitude smaller that t:out in l out· What are then the requirements of the kernel if we demand that the transformation l ou t = K * hn must b e a scale-space transformation modulo local extrema within the error margins. 2 14 erf(x) = 1 f:r: e-~ f 2 d{ ~ J~=- 00 35 the following approximation formula N L(x; t) ~ L g(n; t)J;n(x- n) (x E Z, t > 0) (6 0) n=-N which we recognize as a discrete convolution. The filter coefficients g(i; t) are values of the gaussian kernel sampled at the node points. Ci = g(i; t) = -..[2ii -1 e - •2/2 t 1 Ci (61) From the definitions of PF00 -functions and PF00 -sequences it is clear that uniform sampling of a P F 00 -function will produce a P F 00 -sequence. Sub sequently, if N goes to infinity the approximating transformation (60) pos sesses scale-space properties. However, the semi-group property will not hold exactly after discretiza tion. Therefore, if we calculate the representation at a level t 2 > 0 from the original image we will not obtain the same result as we get if L(·; t 2 ) is calculated via an intermediate level t 1 (0 < t 1 < t2) applying the approxi mation formula (60) in two steps. Of course, we can use the formula (60) to generate the discrete representations at all levels of scale directly from the original image. But then we are not guaranteed that new local extrema cannot be created from a level t 1 f. 0 to a coarser level t2. In Appendix B.1 we show that the transformation between these levels does in general not possess scale-space properties, i.e. the representation L(x; t 2 ) might have more local extrema than L(x; t 1 ) although t2 > t1. Another minor disadvantage with this approximation appears for small values of t (t ~< 1). Then the coefficient sequence is dominated by a very sharp peak at eo . Even thought the integral of the continuous kernel is normalized to one the central peak can drive the sum of the filter coefficients to a value substantially greater than one. In some way the sampled gaussian kernel appears as having a smaller sigma-value than it should, see Fig 9. A renormalization does of course not solve the problem since the mutual relation between the filter coefficients remains unchanged. 6.2 Integrated Values of the Gaussian Kernel One a d hoc way to avoid that the central coefficient c0 becomes too dominant for small t-values is to use integrated values of the gaussian over the pixel 36 -1 1 2 3 Figure 9: The sampled gaussian kernel is donilnated by the central coefficient for small values oft. The peak value is greater than one when t is smaller than 1/2rr. support region instead of sampled values . One can for example let j'+2 _1_ e-e2 i-~ yl2it . ci = I /2t d~ (62) This choke of filter coefficients is equivalent to the continuous formulation (57) if we let the continuous image f be a piecewise constant function, which is equal to the discrete pixel value over each pixel support region. One can show, [15] Section 6.2, that the kernel given by (62) is a scale-space kernel. This means that the transformation from original image (t = 0) to an arbitrary level of scale (t 1 > 0) is always a scale-space transformation. However, we cannot expect a semigroup property to hold exactly and will probably arrive at similar scale-space problems as with the sampled gaussian kernel. 6.3 Discretizing the Heat Equation in Space Consider the one-dimensional heat equation (63) with initial condition L(x ; 0) = f(x) for some given image f. We discretize it in space at an equidistant grid Xi= i (i E Z) with the usual approximation fx¥(xi) ~ ~(Li+1 - 2Li + Li- 1) = {h = 1} = Li+l - 2Li + Li-1· Li 37 does of course denote L(xi; t). This leads to a system of coupled ordinary differential equations (64) with initial conditions Li( 0) = J( i). These equations can be further dis cretized in scale. Using Eulers explicit method with uniform step length D.t (tk = ktlt; *(tk) ~ lt(Li,k+l- Li,k)) we get the recursion formula, (65) which we recognize as a discrete convolution in space. The kernel Kstep has the filter coefficients c_ 1 = ~t, eo = 1 - D.t and c1 = ~t and the generating function IPstep(z) = ~t z- 1 + (1 - D.t) + ~t z. From the scale space requirement for three-kernels 2: 4c_ ICI derived in Section 2.7 it follows that the transformation (65) possesses scale-space properties if and only if c5 D.t 1 < - 2 (66) It is not too difficu lt to get the analytical solution to the system of scale-continuous equations (64). Assume that we want to calculate the so lution for a fixed t-value. We can use the discretization (65) with n steps in the scale-direction such that the step length D.t = tfn satisfies (66) . As each iteration step consists of a linear convolution the final solution achieved can equivalently be obtained by convolution with the composed kernel K composed = *~I Kstep· Let us derive an asymptotic expression for its generating function. As the kernel describing one iteration is a symmetric scale-space kernel it can be decomposed into two-kernels with non-negative coefficients such that IPstep(z) = (p + qz-1 )(p + qz). By identification of the two representations of cp8 tep(z) we get the equation system D.t pq = - 2 ; (67) One verifies that the only non-negative solutions are p = J1- tlt +2v'1- 26,t = 1- -D.t + O( ut2) 2 A 38 (68) q= 1 - /J.t - .J1 - 2/J.t /J.t =2 2 V + 0( ut 2) A and the solution corresponding to interchanged values of p and q. The generating function of the composed kernel then becomes, IPcomposed,n( Z) = (lPstep( Z) t = (p + qzt(P + qz- 1 )n ( 69) which can be written as lPcomposed n( Z) = (1- _!_(1- z- 1 ) + 0( ...!:_2 )t(l- _!_(1- Z) + 0( ...!:_2 ))n (70) ' 2n n 2n n after substitution of -k for !J.t in the Taylor expansions for p and q. Since limn--.. 00 (1 + ~ )n = e01 if limn--.. 00 Cl'n =a it follows that (71) We recognize the generating function of the family of discrete kernels we arrived at when we constructed the discrete scale-space in Section 4.2. e-t is the normalization factor. Consequently, the transformation obtained by convolution with the discrete analog of the gaussian is equivalent15 to the analytical solution of the system of equations obtained by discretizing the heat equation on a fixed equidistant grid in space. In other words (39) is the solution of (64). This is not surprising bearing Theorem 6 in mind. 7 7.1 Summary and Discussion The One-Dimensional Case The aim of this paper has been to investigate the discrete aspects of the one-dimensional scale-space theory. We have studied linear shift-invariant transformations and stated a requirement on kernels saying that the num ber of local extrema in a convolved image must not exceed the number of local extrema in the original image. As an immediate consequence we saw that the coefficient sequence must be non-negative and unimodal. For sym metric kernels the same requirements hold for the Fourier spectrum. We showed that the interesting kernels could be completely classified in terms 15 The observant reader notes that the conclusion is valid only if the discretization (65) converges to the solution of the continuous equations (64) when b.t _. 0. This does for instance hold if f E l 1 or f E l2. 39 of sign-regularity - all shift-invariant scale-space kernels are equivalent to normalized P6lya frequency sequences. The generating function of such a sequence/kernel possesses a very simple characterization. Then we introduced a continuous scale parameter and showed that the only reasonable way to define a scale-space for discrete images is by convolu tion with the one-parameter family of kernels T(n; t) = e - ti n(t), where In arc the modified Bessel functions of integer order. Similar arguments applied in the continuous case uniquely lead to the gaussian kernel. The kernel T does also have the attractive property that it is equivalent to the limit case of a certain di scretization of the heat equation. The idea of a continuous scale parameter even for discrete images is of considerable importance, since it p ermits arbitrary degrees of smoothing, i.e. we are no longer restricted to specific predetermined levels of scale. Due to the semi-group property the scale-space condition holds between any two levels of representation. \Ve showed that the commonly used technique, where the "scale-space" is constructed by convolution with the sampled gaussian kernel, might lead to undesirable effects, since in general the transformation from an arbitrary fine level to a randomly selected coarser level is not a scale-space transformation. An important point with t his result is that it constitutes an example of a property, which has been derived in the continuous case, but does not hold after di scn~tization. The treatment concentrates on one-dimensional images defmed for all in tegers. \Ve have not gone into the complications that occur at the boundary if the image function is defined only for a subset of the integers. However, there is one thing we want to emphasize if the proposed method is applied to a finite image, using some extension method. If one wants the semi-group property to hold exactly (except for rounding errors) it is necessary that all reprcsen tations at all levels are generated di1·eclly from the original extended image using the approximation (46). If several small steps are taken then the intermediate representations must be truncated, which means that we might get a different result, since the semi-group property no longer holds exactly. Of course, a genuine finite approach is also possible. In this presentation we have chosen not to develop the subject, since the associated problems are somehow artificial and difficult to handl e in a consistent manner. There is no getting away from the fact that all finite images have boundaries and that problems arise if one tries to analyze objects near them. Dy necessity, the peripherical image values of a smoothed finit e image will be less reliable than the central ones. However, in a practical case additional data can often 40 be acquired simply by moving the camera such that image values become available in a sufficiently large neighbourhood of the object of interest. For some very simple cases it might be enough do an ad hoc extension. But this requires a priori information about the scene. One way to avoid both the infiniteness and the boundary problems could be to use an alternative camera geometry, where the image values are mapped onto a sphere/circle instead of aJl inftnite plane/line. The idea seem s rather natural from a biological point of view. Then, the ordinary pla nar camera geometry would appear as an approximate description for foveal vision, i.e. small solid angles in the central field of vision. This approach is very closely related to projective geometry. 7.2 Extension to Two Dimensions The extension to two dimensions is not obvious since it is possible to show that there does not exist any non-trivial kernel on R 2 or Z 2 having the property that for all images the number of local extrema in the convolved image does not exceed the number of local extrema in the original image. Pizer and Lifshitz [14] present an illuminating counter-example: Imagine a two-dimensional image function consisting of two hills, one of them somewhat higher than the other one. Assume that they are smooth wide rather bell-shaped surfaces situated some distance apart clearly sepa rated by a deep valley running between them. Connect the two tops by a narrow sloping ridge without any local extrema. Then the top of the lower hill is no longer a local ma...·dmum. Let this configuration be the input im age. When the heat equation is applied to the geometry the ridge will erode much faster than t he hills. After a while it has eroded so much that the lower hill appears as a local maximum again. Thus, a new local extremum has b een created. The same argument can be carried out in the discrete case. Of course, we have to consider connectedness when we define what we mean by local extrema. But this question is only offorma.l nature. Given an arbitrary non tri vial convolution kernel it is always possible to create a counter-example. The width of the ridge can be set to one pixel. ·w hen the convolution kernel is applied to the image the ridge will erode much faster than the hills provided that the hills are sufficiently wide and that the valley is chosen deep and wide enough. Therefore, it is not clea.r what we should mean with a scale-space property in two space dimensions. In one dimension several formulations arc equivalent. Does there for example exist a natural 41 / 0 / Figure 10: New local extrema can be created by the heat equation in the two-dimensional case formulation su ch that a variation-diminishing property holds in some weaker sense ? Or, can we express an axiom, which in one dimension is equivalent to t he formulation in terms of local extrema, and directly generali zes to higher dimensions. It would be very interesting to get the answers to these questions both in the continuous and the discrete case. Is it true t hat the discrete an alog of the gaussian kernel used as a sep arated kernel is the natural discrete kernel in two dimensions ? If one due to computational consider ations wants to use separable discrete kernels, one could, of course, heuristically argue that the kernel should at least have a good performance in one dimension. Another indication in that direction can b e taken from the t wo-d imensional heat equ ation discretized on a fixe d equidi stant grid in space, such th at the con volu tion kernel , describing one iteration along the scale direction , is separ able in space. In Appendix D.2 we show that as the step length along th e scale-axis goes to zero the solution of the discrete approximation approaches the result we get if we apply the one-dimensional discrete analog of th e gaussian kernel as a separated kernel - first along one coordi nate direction and t he along the ot her. Subsequently, convolution with the presented discrete analog of the gaussian kernel de 42 scribes the solution of the system of equations which appears if the heat equation discretized in space but solved analytically in time. This result hold s both in one and two dimensions. 8 A cknowle dgme nts I would like to thank P rof. Jan-Olof Ek1undh for a lot of valuable advice. Hi s guidance and support and his way to always find time for discussions is very much appreciated. I also wish to thank Dr. Stefan Carlsson for stimulating discussions, which provided a large source of inspiration to thi s work. This paper describes research conducted at the Computer Vision and As sociative Pattern Processing Laboratory (CVAP ), Royal Institute of Tech nology, Stockholm, Sweden. The support from the National Swedish Board for Technical Development is gratefully acknowledged. 43 References [1] Abramowitz M., Stegun LA. (1964) Jfandbook of Mathematical Func tions, Applied Mathematics Series, 55, National Bureau of Standards. [2] Ando T. (1987) "Totally Positive Matrices", Linear Algebra and its Applications, 90, pp165-219. [3] Babaud J. , Witkin A.P., Daudin M. and Duda R.O. (1986) "Uniqueness of the Gaussian Kernel for Scale-Space Filtering", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. PAMI-8 , No. 1, pp26-33. [4] Comtet L. (1974) Advanced Combinator·ics, Reidel , Dordrect/Boston. [5] Dahlquist G., Bjork A., Anderson N. (1974) Numerical Methods, Prentice-Hall, London. [6] Gray R.M. (1972) "On the Asymptotic Eigenvalue Di stribution of Toeplitz Matrices", IEEE Transactions on Information Theory , Vol. IT-18 , No. 6 , pp725-730. (7) Gantmacher F.R. (1959) The Th eor·y of Matrices, Vols. I-II, Chelsea Publishing, New York. [8) Grenander U., Szego G. (1958), Toeplitz For·ms and Their Applications, University of California Press, Los Angeles. [9] Hirschmann I.I. , Widder D.V. (1955) Th e Convolution Tran sform, Princeton University Press, Princeton , New Jersey. [10] Ilummel R.A. (1986) "R epresentations Based on Zero-crossings in Scale Space" , Proceedings of the IEEE Computer Vision and Pattern R ecog nition Conference June 1986, pp204-209. [11] Koenderink J .J ., van Doorn A.J. (1984) "The Structure of Images", Biological Cybernetics, 50, pp363-370. [12] Koenderink J .J ., van Doorn A.J. (1986) " Dynamic Shap e", Biological Cybernetics, 53 , pp383-396. [13] Karlin S. (1968) Total Positivity, Vol.I, Stanford University Press . 44 [14] Lifshitz L.M., Pizer S.M . (1987) "A Multircsolution Hierarchical Ap proach to Image Segmentation Based on Intensity Extrema", Internal report, Departments of Computer Science and Radiology, University of North Carolina, Chapel Hill, N.C., U.S.A. [15] Lindeberg T.P. (1988) "On the Construction of a Scale-Space for Dis crete Images", TRITA-NA-8808, Department of Numerical Analysis and Computing Science, Royal Institute of Technology, Stockholm, Sweden. [16] Miller G.A. (1956) "The Magical Number Seven Plus Minus Two: Some Limits on Our Capacity for Processing Information" Psychological Re view, 63 , pp81-97. [17] Norman E. (1960) "A Discrete Analogue of the vVeierstrass Transform", Proc. Amer. Math. Soc., 11 , pp596-604. [18] W.H. Press, B.P. Flannery, S.A. Teukolsky, W.T.Vetterling, (1986) Nu merical Recipes, Cambridge University Press. [19] Schoenberg I.J. (1930) "Uber Variationsvermindernde Lineare Trans formationen", Mathematische Zeitschrift, 32, pp321-328. (20] Schoenberg I.J. (1946) "Contributions to the Problem of Approxima tion of Equidistant Data by Analytic Functions", Quarterly of Applied Mathematics, 4, pp45-99. [21] Schoenberg I.J. (1948) "Some Analytical Aspects of the Problem of Smoothing", Courant Anniversary Volume, Studies and Essays, New York, pp351-370. [22] Schoenberg I.J. (1953) "On Smoothing Operations and their Generating Functions", Bull. A mer. Math. Soc., 59, pp199-230. [23] Toeplitz 0. (1911) "Zur Theorie der quadratischen und bilinearen F01·men von unendlichvielen Veranderlichen. I. Teil: Theorie der 1 formen", Mathemalisch e Annalen, 70 , pp351-376. [24] Whittaker E.T., Watson G.N. (1952) Mod em Analysis, Cambridge Uni versity Press. [25] Witkin A.P. (1983) "Scale-Space Filtering", Pmc. 7th International Joint Confe7'ence on A1'lificial Intelligence, pp1019-1022. 45 A Numerical Implementation of the Discrete Scale Space (Continued) A.l Tabulated Values of the Discrete Analog of the Gaus sian Kernel For reference purpose we supply some kernel values for t = 1 and t = 10. They have been generated by the algorithm presented in Section 5. The kernel size was determined from ( 47) and ( 48) with Etrun c = 0.25 and M = 255. The estimation yielded N = 5 and N = 13 while the correct values were N = 4 and N = 11. ±n T(n; 1) 0 1 2 3 4 4.657596e-01 2.079104e-01 4.993877e-02 8.155307e-03 1.006930e-03 ±n T(n; 10) 0 1 2 3 4 5 6 7 8 9 10 11 1.278333e-01 1.212626e-01 1.035808e-01 7 .983036e-02 5.568258e-02 3.528429e-02 2.039829e-02 1.080634e- 02 5.269408e-03 2.375292e-03 9.938819e-04 3.875283e-04 A .2 Algorithm Sketch Filter Coefficient Generation Given: t = t =Scale parameter value d = d = Number of significant digits in the filter coefficients epsilon = t:trun c = Maximum truncation error in the ou tput image pixel values M= M = Upper bound for the input image pixels values ihn(x)i:::::; M maxreal Maximum real number allowed in hardware = • Determine an upper bound for N either from the estimation (48)-(52) or a pre-calculated table. Nestimated = estimate_N(t, epsilon/(2•M)); • Determine a start index for the iterations. Nstart = integer(2•(Nestimated + d*sqrt(Nestimated)); 46 • Initial values. T(Nstart) : = 1; T(Nstart+1) := 0; • Calculate iterates from the recursion formula. for n : = Nstart step -1 until 1 do begin T(n-1) := T(n+1) + 2•n•T(n)/t; if T(n-1) > maxreal then for i := n-1 step 1 until Nstart do T(i) /= maxreal; end for; • Renormalize the iterates renorm := T_O(t)/T(O); for n := 0 step 1 until Nstart do T(i) renorm; •= where T_O is a routine which calculates T0 (t) , see [18] p711. real procedure T_O(t); real t; begin real z; if (t < 0) then error; else if (t < 3.75) then begin z := t•t/(3.75•3.75); T_O .- exp(-t)•(1.0 + z•(3.5156229 + z*(3.0899424 + z•(1.2067492 + z•(0.2659732 + z•(0.0360768 + z•0.0045813)))))); end; else begin z := 3 . 75/t; T_O . - 1.0/sqrt(t)•(0.39894228 + z•(0.01328592 + z•(0.00225319 + z•(-0.00157565 + z•(0.00916281 + z•(-0.02057706 + z•(0.02635537 + z•(-0.01647633 + z*0.00392377)))))))); end if-else ; end procedure; • Determine how ma ny coefficients that must be used in the discrete mask in order to obtai n the desired accuracy. 47 N := 0; sum := T(O); while (sum < 1-epsilon) do begin N += 1; sum+= 2*T(N); end while; • Now the filter coefficients are given by Cn = T(n) (n = -N.. O..N). B Numerical Approximations to the Continuous Scale-Space Theory (Continued) B.l Scale-Space Violations for the Sampled Gaussian Ker n el Assume that we construct the "scale-space" for a discrete image by convolu tion with the sampled gaussian kernel, i.e. given a discrete image f : Z -+ R we define the family of functions L : Z x R+ -+ R by L( x; 0) = f( x) (x E Z) and L(x; t) = L g(n ; t)f(x - n) (x E Z,t > 0) (72) n=-oo where (nE Z, t > 0) (73) Earlier we have shown that for all images f the number of local extrema in L(x; t) (t > 0) does not exceed the number of local extrema in f . In this section we will show that this scale-space property does not hold between two arbitrary levels. We will need an expression for the generating function for the discrete kernel corresponding to the sampled gaussian. For simplicity we let qt = 1 e-21. One can show [21] that where ~ Ct = y 2rrt fr (1 - qzn) (75) n=l Let t1 and t2 be two levels (t2 > t 1 > 0) of the representation (72) and let <pin be the generating function of the input image. Then the generating 48 functions of l(x; t 1 ) and l(x; t2) are (76) Let l.?diff descri be t he transformation from l(x; t 1 ) to l(x; t2) . Thus, <pL 2 (z) = 'PdiJJ(z) 'PL 1 (z) (77) Combination of (76), (77) and (75) gives . (z)- <pL2(z )- Ct2 'Pdt f f - r £ (z) t1 IT~=O(l + q~2m+lz)(1 + qz2m+lz-l) (1 + n2n+l z)(1 + q2n+l z-1) n=O "lh (] c . noo (f) - 1 (78) According to the complete characterization of scale-space kernels it follows that the corresponding kernel is a scale-space kernel if and only if (78) can be written on the form (35) . Then, for each factor (1 + fJl1n+l z±1 ) in the denominator there must exist a corresponding factor in th e numerator (1 + fJZ2m+lz±1 ), i .e for each n there must exist an m such that (79) 1 Insertion of requirement fJt; e- '217 and red uction gives the necessary and sufficient 2m = -t2 (2n + 1) tl (80) 1 It is clear that this relation cannot hold for all n E Z if t 1 and t 2 are chosen ar bitrarily. The transformation from l(x; t 1 ) to l(x; t2) (t 2 > t 1) is a is an odd integer. scale-space transformation if and only if the ratio i7 B .2 Discretizing the Two-Dime nsiona l H eat Equation Space 111 We will derive the solution of the two-dimensional heat equation (81) discretized on an equidistant grid in space. The initial condition L(x, y; 0) = f(x, y) is given . Let x; = i, Yi = j and L;,j = L(x;, Yi> t ) (h = 1). There are two common discrete approximations to t he two-dimensional Laplace operator {ir + ~ 49 ( 1 1 -4 1 1) c /2 -2 1/2 ) P~2 ) a -4a- 2{3 a [3 /2 fJ/ 2 ( p2 1/2 1/2 a b..t / 2 Pt.t/4 ab..t/2 1 - (2a + [J) D.t ab..t/2 {JD.t/4 ab..t/2 [J D.t /4 ( pt.t/4 ) Figure 11: Computational molecules approximating the two-dimension al Laplaceoperator (a) h 2 \lg (b) h 2 \l~ (c) h 2 (a\l g+{3 \l~) . (d) Computational symbol for the kernel K x,y corresponding to one iteration in scale. namely the five-point operator \lg and the cross-operator \l~ (see Fig lla and Fig llb). Since we are interested in a separable kernel we choose the linear combination ~ + ~ ~ a\lg + {3 \l~ where a+ [3 = 1, a :2: 0 and [3 :2: 0 (Fig llc) . We then get a system of coupled ordinary differential equations, a a L i,j -1 + -(2a + [J)L;,j + Li,J+1 + 2 2 [3 a [3 4 Li,j-1 + 2Li ,i + 4_Li ,i+I We want to calculate the solution of this system for a given value of t. Vle di scretize in scale using n equidistant points tk = kD.t where D.t = ~ - Let L ;,j,k denote L ;,j(tk) · The derivative 8 ~;-z (tk) is approximated by ~ t (L;,j,k+I- L ;,j,k) according to Eulers explicit method . Then, {JD.t L;,j,k+I = - - L i -l ,j- 1 4 aD.t - - L i,j-1 2 L ;- l,j [J D.t + - 4- L i -l ,J+l ab.t + (1- (2a + [J) D.t)L;,j + - 2- [J D.t L ·. 4 a D.t + - 2- t,J - l + aD.t L ·. 2 t ,J + L i,j+I + (83) + [J D.t L ·. 4 t,J+ l The associated computational molecule is shown in Fig lld. Vve want to investigate if the corresponding kernel J( x,y is separable for some choice of a 50 and {3 . Let Kx,y = K x * Ky where K x and K y correspond to convolutions in the x- and y-directions respectively. Due to symmetry t he filter coefficien ts of ](x and Ky must consist of the same set of values. Denote them by 1, 1 - 21 and 1 (~, > 0) . Then these relations must hold: 2 ~, {JD.t - · - 4 ' (1 - 21) 2 = 1- (2a + {J) t::.t (84) D.t 2 (85) The only non-n egative solution is Q = 1- D.t; {3 = D.t ; 1= - Subsequently, the kernel is se parable if and only if we choose a = 1- t::.t and {3 = D.t. Then the generating function <.?x,y(z,w) = L~ - oo L~ -ooci,jZiwi of the kernel K x,y can be written as IPx,y(z,w) = IPx(z)cpy(w) where <.?x(z) = ~t z- 1 + (1- t::.t ) + ~t z 1 and cpy('w) = ~tw- 1 + (1- t::.t ) + ~tw 1 . The composed kern el K composed,n describing the fi nal solution after n steps has a generating fun ction After calculations similar to t hose performed in Section 6.3 it follow s t hat (87) I.e. the solution of the two-dimensional heat equation discretized in space is obtained by convolut ion wit h the di screte analog of the gaussian kernel applied first in one coordinate direction and then in the other. 51

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement