Calhoun: The NPS Institutional Archive Theses and Dissertations Thesis Collection 1995-09 The discrete, orthogonal wavelet transform, a projective approach Logue, James K. Monterey, California. Naval Postgraduate School http://hdl.handle.net/10945/35162 NAVAL POSTGRADUATE SCHOOL MONTEREY, CALIFORNIA 19960221 045 THE DISCRETE, ORTHOGONAL WAVELET TRANSFORM, A PROJECTIVE APPROACH by James K. Logue September, 1995 Thesis Advisor: Carlos F. Borges Approved for public release, distribution is unlimited. W3C QUALITY INSPECTED 1 REPORT DOCUMENTATION PAGE Form Approved OMB No. 0704-0188 Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instruction, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302, and to the Office of Management and Budget, Paperwork Reduction Project (0704-0188) Washington DC 20503. 1. AGENCY USE ONLY (Leave blank) 2. 3. REPORT DATE September, 1995. 4. TITLE AND SUBTITLE THE DISCRETE, ORTHOGONAL WAVELET TRANSFORM, A PROJECTIVE APPROACH REPORT TYPE AND DATES COVERED Master's Thesis FUNDING NUMBERS 6. AUTHOR(S) James K. Logue 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Naval Postgraduate School Monterey CA 93943-5000 8. PERFORMING ORGANIZATION REPORT NUMBER 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSORING/MONITORING AGENCY REPORT NUMBER 11. SUPPLEMENTARY NOTES The views expressed in this thesis are those of the author and do not reflect the official policy or position of the Department of Defense or the U.S. Government. 12b. DISTRIBUTION CODE 12a. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release; distribution is unlimited. 13. ABSTRACT (maximum 200 words) All integral transforms can be viewed as projections onto collections of functions in a Hubert space. The properties of an integral transform are completely determined by the collection of functions onto which it projects. The wavelet transform projects onto a set of functions which satisfy a simple linear relationship between different levels of dilation. The properties of the wavelet transform are determined by the coefficients of this linear relationship. This thesis examines the connections between the wavelet transform properties and the linear relationship coefficients. 14. SUBJECT TERMS Wavelet, Multiresolution, Dilation 15. NUMBER OF PAGES 59 16. PRICE CODE 17. SECURITY CLASSIFICATION OF REPORT Unclassified NSN 7540-01-280-5500 18. SECURITY CLASSIFICATION OF THIS PAGE Unclassified 19. SECURITY CLASSIFICA- 20. LIMITATION OF ABSTRACT TION OF ABSTRACT Unclassified UL Standard Form 298 (Rev. 2-89) Prescribed by ANSI Std. 239-18 298-102 11 Approved for public release; distribution is unlimited. THE DISCRETE, ORTHOGONAL WAVELET TRANSFORM, A PROJECTIVE APPROACH James K. Logue Lieutenant, United States Navy B.S., Auburn University, 1988 Submitted in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE IN APPLIED MATHEMATICS from the NAVAL POSTGRADUATE SCHOOL September 1995 Author: Approved by: Van Emden Henson, Second Reader Richard Franke, Chairman Department of Mathematics in IV ABSTRACT All integral transforms can be viewed as projections onto collections of functions in a Hilbert space. The properties of an integral transform are completely determined by the collection of functions onto which it projects. The wavelet transform projects onto a set of functions which satisfy a simple linear relationship between different levels of dilation. The properties of the wavelet transform are determined by the coefficients of this linear relationship. This thesis examines the connections between the wavelet transform properties and the linear relationship coefficients. VI TABLE OF CONTENTS I. INTRODUCTION II. EXISTENCE, UNIQUENESS, APPROXIMATION, AND OR- III. IV. 1 THOGONALITY 5 A. EXISTENCE AND UNIQUENESS 5 B. SOME TOOLS FOR DILATION EQUATIONS 6 C. APPROXIMATION 10 D. ORTHOGONALITY 14 THE DISCRETE ORTHOGONAL WAVELET TRANSFORM 21 A. DECOMPOSITION OF THE FUNCTION SPACE L2 21 B. ALGORITHM DEVELOPMENT 32 CONCLUSIONS AND DIRECTIONS FOR FURTHER STUDY 39 REFERENCES 41 APPENDIX A. DERIVATION OF POISSON SUMMATION FORMULA 43 APPENDIX B. PROJECTIONS AT FINEST SCALING LEVEL . . 45 INITIAL DISTRIBUTION LIST 47 Vll Vlll LIST OF FIGURES 1. Dilation of Haar Function 3 2. Dilation of "Hat" Function 7 3. Organization of Basis Function 4. Scaling Functions at Finest Scaling Level DC . 35 45 I. INTRODUCTION A great deal has been written about wavelet analysis in the last several years. Much of this work however is presented in the language of signal processing. This is to be expected since wavelet analysis has been primarily developed in the signal processing field. This paper presents the discrete wavelet transform in a projective framework. The discrete wavelet transform, like most integral transforms, can be viewed as projection onto a new basis determined by a given collection of functions. In light of this the properties of the transform are primarily determined by the properties of the basis functions. The development of a transform is then a question of finding a basis which will produce the properties we desire. What properties do we desire in the wavelet transform? We would like the transform to incorporate the spectral decomposition, or frequency localization, of the Fourier Transform. In addition, we would like to have time-localization of the input function. This allows us to determine when in time a given component of the input function occurs. Frequency localization in the Fourier Transform is accomplished by projecting onto sines and cosines which have non-compact, in fact infinite, support. This precludes the Fourier Transform from having time localization capability. Time localization can be achieved by using the short time Fourier Transform (STFT), which analyzes the input function in small time windows. This time localization is fixed in scale, depending on the time window used. Another desired property of the wavelet transform is arbitrary approximation power. This is useful if we wish to use the transform in a data compression scheme, that is, setting to zero transform coefficients below a certain threshold level and storing the input function in terms of this reduced coefficient set. Finally, and perhaps most importantly, we desire the basis onto which we project to be orthonormal. This will allow easy inversion of the transform. If the basis is orthonormal, inversion is accomplished by weighted superposition of the basis functions using the transform coefficients as weights. This is the case with the Fourier Transform which uses the orthonormal sine and cosine basis. This is contrasted with the Laplace Transform which uses an exponential basis which is not orthogonal over the real line. This is why the Laplace Transform must be inverted by a contour integral in the complex plane. The question of developing a transform with these properties is reduced to that of finding a basis for L2, the space of square integrable functions on the real line, which possesses the desired properties, namely, time and frequency localization, arbitrary approximation power, and orthogonality. If we restrict our attention to compactly supported functions, we can achieve a measure of time localization, however only at a fixed level dependent on the support of the basis function. This severely limits the frequency information available as well, since we can only clearly discern the frequency which corresponds to the basis functions. Compactly supported functions are easier to compute with, hence their use in finite element methods, and we would like to use them here. We need to find a way to use compactly supported basis functions without limiting the time or frequency information available to us. The solution is to use dilations of a compactly supported function. A dilation of / (x) is / (ax) for some a, normally a > 0. Notice that as we dilate a function / the support of / changes and also the frequency of / changes. This allows us to cover much wider time and frequency scales than using basis functions of fixed compact support. To give us the maximum amount of flexibility we will use a basis composed of translations and dilations of a fixed function <j>. Consider the dilation equation, or two scale recursion relation <j) (x) = 2_jCk<t> (2x — k) for some {ck} . (1.1) k The solution is completely determined by the dilation equation coefficients {cfc}. Also notice this dilation equation with coefficients {c0 = l,c1 = l}is satisfied by the Haar function: h(x) = h(x) 1 for x e [0,1] 0 for x £ [0,1] h(2x) h(2x-l) Figure 1. Dilation of Haar Function The Haar Transform which uses as a basis functions derived from h(x) is in fact an early example of a wavelet transform and predates the current interest in wavelet analysis by several decades. In this paper we show that solutions to 1.1 exist which have arbitrary approximation power and are orthogonal to their integer translates. We further show how these solutions can be used to construct a basis for L2, and how the discrete transform which represents projection onto this basis can be computed in order iV operations. II. EXISTENCE, UNIQUENESS, APPROXIMATION, AND ORTHOGONALITY A. EXISTENCE AND UNIQUENESS Before we investigate the relationship between the coefficients of the dilation equation and the properties of the solutions, we need to determine under what conditions a solution in fact exists. To begin our search for solutions to the dilation equation let us consider the iteration <f)n+1 = ]P ckcj)n (2x - k). {Ill) k Solutions to the dilation equation are fixed points of the dynamical system defined by this iteration. The following theorem, which we present without proof, gives conditions under which square summable solutions to the dilation equation exist and is found in [Ref. 1]. THEOREM II.1. If the {ck} satisfy the following conditions 1. {et} is a finite set, 2- Ek c* = 2, 3. \m0 (z) |2 + \m0 (z + TT)|2 = 1 for all zeE, where m0 (z) = £fc cke~ikz then the dynamical system in II. 1 has a nontrivial square summable fixed point with compact support. In the same paper, a similar result is presented for distributional solutions to 1.1 which does not require the third condition. In practice we are looking for a basis for the function space L2, so we restrict our attention to square summable solutions. Uniqueness of this solution, up to normalization, is guaranteed by the same conditions which give us existence. In the square summable case the normalization is / <j)dx = 1. Observe that this normalization requirement is consistent with condition 2 above since / (f> (x) dx = / Y^ ck(f) (2x — k) = -^2,ck j 4>(y)dy k implies T,/_,ck = 1 fc where y = 2x - k •* or y^cfc = 2. k Before proceeding with our discussion of approximation let's look at some solutions to 1.1 which are familiar to us. In the introduction we saw for coefficients {1,1} the solution to 1.1 was the Haar function. Other familiar functions which are solutions to the dilation equation, though we may not recognize them as such, are the Dirac delta function for {2} and the cardinal B-splines for {(™)} normalized to sum to two. The Figure 2 shows the dilation of the "Hat" function which is the solution for the dilation equation for {\, 1, |}, the binomial coefficients for n = 2, normalized to sum to two. The Cardinal B-splines illustrate why the third condition in Theorem II. 1 is a sufficient but not necessary condition for square summability. The Cardinal B-splines are clearly square summable, but it is easily seen that {(£)} do not satisfy the third condition of Theorem II. 1. B. SOME TOOLS FOR DILATION EQUATIONS Before proceeding with our investigation of the relationship between the di- lation equation coefficients {ck} and the corresponding solution (f>(x), we should introduce a very useful tool in the study of dilation equations. Dilation equations are difficult to study because only in special cases do we actually know what the solution function is. We are usually limited to knowing only the recurrence relation between different scalings of the function. Faced with this lack of information we must turn to the Fourier Transform to glean any information we can about the function. Let us see what the recursive nature of 1.1 can tell us about the solution (f>(x). -0.2- Figure 2. Dilation of "Hat" Function We are given 4>{x) = 2_^ Ck<ß(2x — k) for some {ck} (II.2) k Now, consider the Fourier Transform <&(£) of (j){x) /OO <f)(x)e-^xdx (II.3) -oo substituting II.2 into II.3 we get /oo e-^xJ2ckH^-k)dx OO i. or /oo <f>{2x - k)e-iixdx in each integral of this sum make the change of variables y = 2x - k which yields *(6 = ^ECke * / <f>(y)e^dy. (II.4) Now, notice the integral in II.4 is <&(|) ,so we have Applying this process repeatedly to $(£/2), $(£/4), ...$(£/2n) we get Let us now take the limit as n —¥ oo, oo 1 If we require by way of normalization that J (frdx = 1 then we have $(0) = / (f)dx = 1 and II.5 becomes oo 1 Notice we could write this infinite product as oo 1 Now we see the importance of the 7710(2) expression in the third condition of Theorem II.1, the behavior of this trigonometric polynomial determines the convergence of the infinite product representation of the Fourier Transform, and therefore the existence of a square summable solution to 1.1. A fairly simple, yet powerful result of this infinite product representation of the Fourier Transform is the Convolution Theorem for Dilation Equations. THEOREM II.2. Let </> and ip be solutions to the dilation equation for coefficients {cfc} and {di} respectively. The function (ß*ip given by the continuous convolution of <fi with cp is the solution to the dilation equation with coefficients {bj}, where {bj} is one half the discrete convolution of {c^} with {di}. Proof. As we have seen the Fourier Transforms of <f> and cp are given by 00 ^ 00 Cke ®(0 = n=0 Il2^2 ^ k k and 1 die L ^(o = n=0 H^J2 ^ I respectively. The product of these transforms $W is given by oo 1 cke ä oo 1 ^=H2^2 ^ n2^2die^Lra=0 A; m=0 I We can combine the two product operators by associating the factors where n = m to get 00 1 n=0 - 1 k I Observe each factor in this infinite product is a product of two polynomials in e v- , -in namely, | ^fc c^e v- and | Yli die 2" • Let p (£) be the product of these polynomials. Prom polynomial multiplication, we see the coefficients of p (£) are given by A 1 bx= ol-jCidx-^ 1 3=0 that is, the set {b\} is one half the discrete convolution of {ck} with {d{\, and the product polynomial p is P(0 = v^2he>n x Substituting this into the expression for $\£ gives us 00 1 ^ = H^J2bxe^n=0 X Observe the right hand side of this equation is the infinite product representation of the Fourier Transform of the solution to the dilation equation with coefficients {b\}. Thus, $\I> is the Fourier Transform of the solution to the dilation equation for coefficients {bj}. Now, from the convolution theorem for Fourier Transforms we know t&Nt is the transform of (f> * ip, the continuous convolution of <j> with </?. Since Fourier Transforms are unique this completes the proof. D Armed with this result we see why the Cardinal B-splines are solutions to the dilation equation for the binomial coefficients normalized to sum to two. If we develop B-splines from a repeated convolution standpoint as is done in [Ref. 4], the 9 relationship between B-splines and binomial coefficients becomes transparent in light of Theorem II.2. To generate the B-splines we start with the Haar function which we can consider the "zeroth" order B-spline (piecewise constant approximation), we get the next higher order B-spline (piecewise linear approximation) by convolution with the Haar function. By our convolution theorem this says we should take one half the discrete convolution of {1,1} with itself. Convolving {1,1} with itself we get {1, 2,1} the binomial coefficients for n = 2, which we must normalize to sum to two. Similarly, to generate the (n + l)st degree B-spline we convolve the nth degree B-spline with the Haar function, so we convolve the coefficients of the nth degree B-spline with {1,1}. This gives us the binomial coefficients for n+1, which again we must normalize to sum to two. The appearance of the B-splines is of more than passing interest. B-splines and their translates are often used as a basis for approximation. This is very similar to the scheme we are investigating. We are pursuing the goal of representing L2 in its entirety rather than approximating functions in a subspace of L2. Another aspect of spline approximation which will be of great importance in the next section is the use of sliding window filters as a tool to smooth data. Sliding window filters can be viewed as convolution with the Haar function. Thus repeated application of sliding window filters is equivalent to repeated convolution with the Haar function, both of which lead to smoother data, or smoother approximation if one is constructing a basis function, as we shall see in the next section. C. APPROXIMATION Since we will be using solutions to 1.1 as a basis for approximation we would like to know how well an arbitrary function can be approximated by 0 and its translates. Our measure of approximation power will be polynomial precision, or what degree arbitrary polynomial can be exactly represented by <f> and its translates. A key step in finite element methods is to approximate a function by the translates of a compactly supported basis function. To determine the approximation power of solutions to the 10 dilation equation we consider the work done on the finite element method by Strang and Fix. In [Ref. 3] they show that the following statements are equivalent: 1. Any polynomial of degree less than or equal to p— 1 can be exactly represented by a linear combination of <j> and its translates. 2. $(0) ^ 0 and J^$(£) k=2™= 0 for n e Z,n ^ 0 and a = 0,1, ...,p. The first of these statements is the definition of approximation power that we are interested in, the second is related to the Fourier Transform of the function <j>. We will use the second statement to construct a condition on {cjt} that will give the desired approximation power. First notice that $(0) ^ 0 is taken care of by our normalization requirement since $ (0) = f (j>e-Mxdx = f (j)dx To satisfy the rest of this statement we must look at the infinite product representation of <3> (0 oo 1 2n n=l 2 k Since this product converges and $(0) ^ 0 we see that the product vanishes for some value of f if and only if at least one of its factors vanish for that value of £. Observe that the following condition is sufficient to ensure the desired behavior of <& (£): • z = 7T is a root of order p of m0 (z). In the previous section we saw * (0 = 11^0 1 2n (II.6) n=l Now, for all I E Z, I ^ 0, there exists n such that ^± mod 2n = IT. Therefore, one of the factors in II.6 is zero, and <3> (2nl) = 0. Now consider 4 d£ [$ (£)], ml *™l-*L ° | n , (t\ d n^i^ .fc^n 11 (II.7) Since m0 (z) has a root of order p at TT, the right hand side of II.7 is zero. Similarly we can see &£■ [$ (f)] = 0 for j = 0,1, ..p provided m0 (z) has a zero of order p at TT. Unfortunately, this elegant sufficient condition on m0 (z) is not necessary. Observe, if ± [$ (£)] = 0, ^ [m0 of (TT)] need not be zero, since m0 (f), which is a factor Il/fc^n mo ( JF) could be zero, in which case, II.7 will be satisfied regardless of the value of ^ [m0 (TT)]. The sufficient condition for the approximation power of $ can be arrived at in several ways. In [Ref. 2] it is presented as a sum condition, that is: N ^2 (-1)1 lkct = 0 for k = 0,1, ...,p - 1. (II.8) j=o We can see this is equivalent to our condition that TT be a root, of order p, of m0 (z), by first expressing the sum condition as a matrix equation, Kc = 0 where KisapxiV + l matrix with the following structure: 1 -1 1 -1 1 ••■ 0 -1 2 -3 4 ... 0 -1 4 -9 16 ••• 0 -1 8 -27 64 ••• 0 -1 16 -81 256 ••• Calculating the null space of this matrix yields a basis for the null space which is given by {[vf\} where [uj] is the N + 1 long vector whose non-zero entries begin at the kth position and are the binomial coefficients (p) for k = 1, 2,..., N - p. For 12 example, a basis for the null space of 1 -1 1 -1 1 0-12 -3 4 0-14 -9 16 IS ' 0 3 1 3 * > 1 ) 3 1 3 0 1 k > j Since this is a basis for the null space of K, any {ck} which satisfies the sum condition II.8 must be a linear combination of the basis vectors. Since the basis vectors are shifted copies of one vector, the matrix whose columns are the basis vectors is Toeplitz. So, any linear combination of the basis vectors can be represented as the product of this Toeplitz matrix T with a vector whose elements are the weights in the linear combination. Alternatively, we can consider this multiplication by a Toeplitz matrix to be discrete convolution of the vector of weights with the vector of binomial coefficients, i.e. polynomial multiplication if we consider the elements of these vectors to be coefficients of polynomials. So, what does all this do for us? Any {cfc} which satisfies II.8 can be considered to be the coefficients of a polynomial / (x) which in turn can be expressed as the product of two polynomials g (x) and h (x). The coefficients of g (x) are given by the binomial coefficients (?) where p is the required approximation power. Recall that the polynomial with coefficients (?) is g (x) = (x + l)p. That is, the polynomial with coefficients {ck} has a root of order p at x = -1. Now, if we consider the relationship between the trigonometric polynomial m0 (z) = J^k ckeikz and the polynomial / (x) = Ylk ckxk, we see / (eiz) is equivalent to mQ (z). Further, if / (x) has a root at x = -1, then m0 (z) has a root at TT. 13 Yet another way to see the sufficiency of our condition for approximation power is to observe that each higher order of the root of m0 (z) at ■n is equivalent to an additional factor of (x + 1) in the polynomial whose coefficients are {c^}. In turn, each of these factors is equivalent to the discrete convolution of a smaller set of coefficients with {1,1}. Now, by our convolution theorem for dilation equations each of these discrete convolutions is equivalent to the continuous convolution of a function which precedes <f> with the function whose dilation equation coefficients are {1,1}. Of course the function with dilation equation coefficients {1,1} is the Haar function. Recall that the Fourier Transform of the Haar function is -—' ^', which decays like j. Each factor of [x + 1) in the polynomial whose coefficients are {ck} introduces a factor which decays like j in the Fourier Transform of <j). Each factor of j in the Fourier Transform implies one degree of smoothness, or approximation power in the function <f>. D. ORTHOGONALITY As stated in the introduction, an orthogonal basis of functions is advanta- geous. Orthogonality allows us to reconstruct the input function from its transform coefficients by summing the basis functions weighted by the transform coefficients. This is the case in the Fourier Transform. By contrast the Laplace Transform uses the exponential functions {epx} as a basis. These functions are not orthogonal over the real line and as a result the inverse Laplace Transform involves a contour integral in the complex plane. We wish to take advantage of this ease of inversion, so we will restrict our attention to orthogonal basis functions. Again, we are limited by not having an explicit representation of our basis functions. We must show that there exist solutions to 1.1 which are orthogonal to their integer translates. Additionally, we still require these functions to have the desired approximation power. Fortunately, such solutions do exist. In fact, we have already seen one of them. The Haar function is the solution to the dilation equation with coefficients 14 {1,1}. Since the Haar function has support width of one unit, any integer translate of the Haar function has support which does not intersect the support of the original Haar function therefore the Haar function must be orthogonal to any of its integer translates. However, the Haar function has poor approximation power, and is not an ideal candidate for use as a basis for an integral transform. The other members of the Cardinal B-spline family are used for piecewise approximation by higher order polynomials and therefore have increasing approximation power and might be good candidates for transform basis functions. Unfortunately, the higher order Cardinal B-splines are not orthogonal. This is easily seen since each of the B-splines is positive valued over their support and the product of two B-splines with intersecting support would be positive, thus the inner product of these B-splines would be non-zero. In this section we develop both necessary and sufficient conditions for orthogonal solutions to the dilation equation. Let us first look at the consequences of this orthogonality, and develop a necessary condition. Suppose <j) (x) is orthogonal to its integer translates <j>(x — k) for all k E Z, k ± 0. Orthogonality here is with respect to the inner product on L2, that is /oo f(x)Jfr)dx. -oo So far we have only considered real dilation equation coefficients. We will continue to restrict our attention to real coefficients. However, the following development is presented allowing the possibility of complex coefficients to be consistent with the above inner product. Suppose <j> (x) is orthogonal to <j> (x - k) for k G Z, k =£ 0; additionally suppose (j> {2x) is orthogonal to <j> {2x - j) for j G Z, j / 0, then /oo 4> (x) <j>{x- k)dx. ■oo Substituting in LI for <j> (x) and <f>(x — k) we get /OO /.OO <t> (x) <ß(x- k)dx = OO ]T Q</> (2x - I) ^c^<t>{2(x-k)-m)dx. J—OO i 15 Interchanging the order of integration and finite summation /OO /"O0 c (j)(x)(t)(x-k)dx = Y^^ i^ 00 l m <j>{2x-l)<f>(2x-2k-m)dx. (II.9) ^-°° Since <f> (2x) is orthogonal to <f> (2x - j) for j e Z, j'^ 0 the integral on the right hand side of II.9 must be zero unless I = 2k + m. This simplifies II.9 to /oo />oo 0 (x) <f> (x - k)dx = ^2 c*cT2fc / -oo 1 4>(2x -l)</> {2x - l)dx. J—00 So, if <f> (x) is to be orthogonal to (f> (x — k) we require /oo (/)(2x-l)(f){2x-l)dx = 0. And since we assume (j) is not identically zero, the above integral cannot be zero, thus for k €Z, k ^0. ^C/Q7^ = 0 (11.10) 1 Additionally, when k = 0 /oo /-oo (j) (x) (j) {x)dx = Y^ CJQ / -00 f <j)(2x — l)(j) (2x - l)dx. J—00 Let j/ = 2a; — I in the right hand integral and we get /OO 1 /"OO <ß(x)<f)(x)dx = -^2clci •00 ~ 1 (f>(y)<f>{y)dy. J—00 For this equation to be true we must have y^cicj = 2. We can combine these to get a necessary condition for orthogonality ^2 ci^2k = 2Sk for all k E Z (11.11) where <5fc is Kronecker's delta function: 5k = 1 for k = 0, <5fc = 0 for k ^ 0. It appears that 11.10 is yet another condition we are imposing on the dilation equation coefficients. However, with a little work we will see that this necessary 16 condition for orthogonality in 11.10 is equivalent to the third sufficient condition for square summability in Theorem II.1, expressed in terms of the coefficients rather than in terms of the polynomial m0 (z). Let us start with the third condition from Theorem II.l, |m0(z)|2 + |m0(z + 7r)|2 = 1 for all z e R mQ(z) = \^ckeikz. where 2 (11.12) (11.13) k First, observe C KZ+V) m0(z + TT) = 1^2 = \2 E <*<?"***2^ ^ * k (n.14) k Now, if k is even, elkn = 1, and if k is odd, e%k7r — — 1 so m0(z + 7r) = ~^2(-l)k cke k Recall |C|2 = CC which leads us to |m0(z)| + |m0(z + 7r)| = m0{z)m0 (z) + m0 {z + n) m0 (z + IT). Substituting in the expression for mo (z) from 11.13 and the expression for mo (z + ir) from 11.14 we now get |m0(2:)|2 + |m0(z + 7r)|2 = \ E c^kz\ E c^lz + \ E (-1)* c^kz\ E (-1)' c^z- (IL15) k I k I Carrying the conjugation down into the trigonometric polynomials and simplifying we get |m0(2;)|2 + |mo(z + 7r)| = \Ec*eifczE^e^ + iE(-1)*c^kzE(-i)'^-"*- (IL16) k l k 17 I This sum is a Laurent polynomial in e%z. Keeping in mind that this sum must equal one for all z e R, we see that the coefficients of (etz)n for n ^ 0 must be zero since the sum is real-valued. We need an expression for the coefficient of (elz)n; with a little work we can see this expression is 0n = — 2_^ CjCj-n + T {— 1) 2_^ cjcj~n- 0 3 Now notice, if n is odd then bn is zero automatically. If n is even then we can say n = 2m for some m 6 Z, and 71 ~~ o 2_y c-?c-?~ 2m 0 thus y j CjCj_2m = 0 form€Z,m^0 when m = 0, bn — 1 and E c c j j Combining these conditions we get y CjCj^2m = 25m for all me Z j as in condition 11.10. This chain of arguments can be reversed to derive 11.12 from 11.10. So, we have shown the equivalence of the two conditions. If we look for orthogonal solutions to the dilation equation, the coefficients necessarily satisfy the sufficient condition for square summability. This is all well and good, but we still need to find a sufficient condition on the dilation equation coefficients to ensure orthogonality. There is a necessary and sufficient condition for orthogonality of the solutions of the dilation equation. This condition can be found in [Ref. 5], where it is expressed in terms of the location of the roots of the trigonometric polynomial m0 (z) as defined in Theorem II. 1. This 18 condition, while it has the advantage of being both necessary and sufficient, is a little cumbersome. We will instead present a slightly stronger condition on the placement of roots of TOO (Z) which is also sufficient for orthogonality. This condition is found in [Ref. 5] as well and relies on the necessary and sufficient condition mentioned above. We present it here and refer the reader to [Ref. 5] for proof. THEOREM II.3. Let <fi be the solution to the dilation equation for coefficients {c^}. Suppose TOO (Z) = \ J^k c*t%kz satisfies \m0 (z) |2 + |TO0 (Z + ir) |2 = 1 for all z eR , and TOO (0) = 1. // TO0 (Z) has no zeros in [— f, |], then 0 (x) is orthogonal to <f>(x - k) for all k el, k^ 0. Notice since we required J2k c* = 2, we automatically satisfy m0 (0) = 1. Also, the sufficient condition for approximation power was that m0 (z) have roots of a given multiplicity at 7r, it is encouraging that this does not conflict with the sufficient condition for orthogonality. Well known examples of compactly supported, orthogonal solutions to the dilation equation with higher approximation power than the Haar function are the scaling functions associated with the Daubechies' Wavelets [Ref. 2]. We will describe what a wavelet is in the next chapter. 19 20 III. A. THE DISCRETE ORTHOGONAL WAVELET TRANSFORM DECOMPOSITION OF THE FUNCTION SPACE L2 In the previous chapter we saw that we could find solutions to the dilation equation. We also saw that these functions could be made to have a specified approximation power and be orthogonal to their integer translates. In order to develop a transform based on these functions we must show that a basis for L2, the space of square integrable functions defined on the real numbers, can be formed from solutions to the dilation equation. As we have previously seen, the solutions to the dilation equation are square integrable provided the dilation equation coefficients meet some basic conditions. In this section we will show that we can derive from the solution to the dilation equation a basis for L2, and that we need not impose any additional conditions on the dilation equation coefficients to accomplish this. Let us consider the space, call it Vo, of functions which can be expressed as linear combinations of the solution (j) to the dilation equation and its integer translates. Since (j) is square integrable, V0 is a subspace of L2. Let us also consider the space, call it V-i, spanned by <j> (2x — k) where k G Z , this space is composed of functions which are linear combinations of \/2(f) (2x) and its half integer translates. We have scaled the function by \fl so that the norm, in L2, of the prospective basis function will be one, this has no impact on the space spanned by the functions. Again, since (j) is square integrable 0 (2x) is also square integrable and V_x is a subspace of L2. If we continue in this manner, taking spaces spanned by <f>jk = 2~H2(f) {2~*x — k), we get an infinite sequence of subspaces of L2, namely V0,V-i, Vl2,.... If we also allow positive indices for these spaces, which implies negative powers of 2 in 2~H2(j) (2~^x — k) we get a bi-infinite sequence of subspaces of L2 Vj = linear span {(j) (2~jx - k) \ keZ] This sequence of subspaces has the following properties. 21 for j e Z. 1. The Vj form a nested sequence, that is Vj C Vj^\. Suppose f £ Vj then / is a linear combination of <f>{2~^x — k). Since <f> is a solution to the dilation equation, (j) (2~^x — k) is a linear combination of <t> (2-tt~Vx - k), thus Vj C Vj-!. 2. / (x) e ^ if and only if / (2a:) € Vj-i. Suppose f(x) = J2ak<f>(^jx-k), then /(2a:) = £ ak<j> (2^'(2a:) - k) = J2otk(f) (2~ü~^x — k) thus /(2a:) e V}_x. Conversely, suppose /(2y) € T^_j then, /(2y) = ^)M (2~ü"1)y - A;) = 5>^(2-'(2y) - A;). Now let x = 2y, and we get / (x) = Y^ak4> (2"Jx — A;), thus / (x) <E V}. 3-nJez^ = {o}It is sufficient to show that for all / 6 Cg°, the space of infinitely differentiable, compactly supported functions, (Pjf,f) -> 0 as j -> oo. Here Pjf is the orthogonal projection of / onto Vj. As a consequence of 11.11 the <f)jk form an orthogonal set. Thus, {<f>jk} constitutes an orthonormal basis for Vj, and the orthogonal projector P_j can be expressed as Pjf = y ^2(<f>jk,f)^jk- So (Pjf, /> = ( E to*. /> &*> / ) = E I to"*, /)f \ fc Ik and if we assume / has support [—a, a] I to*, /) I2 = f / (*) 2-J'2<j>{2-ix-k)dx J —a — 2 9-J = I / (x) 0 (2-J'x - A;)da: «/ — a As a consequence of Holder's inequality j f{x)<j>{2-Jx-k)dx<\\f\\ ( f (f){2~jx-k)(f)(2^x-k)dx\ so 2 / f{x)(f>{2-ix-k)dx ra 2 j \\f\\ <2^'||/|r / \<ß{2-jx-k)\ dx J —a < IIll/l! fir2 / / \y+k\<2~Ja J\v- 22 \<f>(y)\Zdy. , Observe that as j —> oo the integral on the right hand side goes to zero independent of k. Also, each \((j)jk, f)f is non-negative. Combining these two facts we see 0 < (Pjf,f) < ll/ll2^ / \<t>{y)\2dy -»■ 0 as j -+ oo k J\y+k\<2-ia thus,ni6Z^ = {0}. These properties, while very helpful, leave us somewhat short of our goal of producing a basis for L2. The most obvious shortcoming is that we have not shown that any collection of these functions span L2. Before we address the issue of spanning L2, let us consider a property of <$>, the Fourier Transform of 0. As we shall see, the following limit is of interest to us /oo \<S>(2-juj)\2\g(Lü)\2(Lj ■00 for arbitrary g e 0 CQ . In [Ref. 1] we see that subject to the conditions already imposed on {ck}, <3> is an entire function. Additionally, we have seen $ (0) = 1, and |<fr (z)\ < 1 for z € R. We can now use Lebesgue's bounded convergence theorem [Ref. 6] to show /oo ■oo /-oo \$(2-ju>)\2\g(üj)\2dü>= / \g(co)\2cLu. (ULI) J-oo Now, let us get back to trying to span L2 with some collection of (f>jk- What we wish to show is Üjez K?' = ^■ Since the Vj are nested, it will be sufficient to show that (P-jf, f) —> (/, /) as j —t oo for all / e I? such that /, the Fourier Transform of /, is in CQ°. Let / 6 I? such that / = g 6 C£°. As we have seen before the orthogonal projector P_j can be expressed as fcez Again we see feez 23 Recalling that the Fourier Transform we are using has the normalization constant in the inverse transform, inner products in the function space are related to inner products in the transform space by (<f>, ip) = 2% ($, W). By standard operational rules for Fourier Transforms we can see $-jk, the Fourier Transform of (f>-jk as defined above is given by All of this now leads us to /oo g(w)2-J/*e2->ki»$(2-3üj)du. -oo Simplifying where possible and making the change of variable z = J 2~ UJ we get /oo g {2jz) $ (z) e-kizdz. •oo Recall that g has compact support, so for sufficiently large j, g (2jz) is nonzero only in [—7r,7r] and we can simplify the integral to (/, <f>-jk) = /2 2TT2-> / g (2jz) $ (z) e~klzdz. (III.2) J —IT Now, the right hand side of III.2 is just the kth Fourier coefficient of the function 2-^22Trg(2Jz)^(z). The quantity we are interested in, namely (P-jf,f), is given by the sum over k of the squares of the magnitudes of the individual inner products given in III.2. Since each of these individual inner products is a Fourier coefficient for a specific function, we can use Parseval's Equality to arrive at (P-if, f) = J2 27v2 j/2 ~ f 9 (Vz) <& (z) e~kizdz (III.3) = 2 j/ 2n 2 h I \ ~ * 9 {Vz) $ (z)\ dz. Collecting terms and simplifying where possible we get (P-jf, /) = 2TT2-^ ^ \g (23z) $ (z) |2 dz. J —IT 24 If we now revert to the original variables, that is, let 2^z = u> we get /2J7T /-00 2 \g(u)^(2^u)\ du = 2'K / -2->7T \g (u)\2 |<S> (2"-?a;)|2 dw. ./-OO We have seen in ULI that the right hand integral above converges to /oo |g(ü;)| du. oo Combining all we have done we get /oo \g (u) |2 du = 2*{g,g) = </>/>• •oo Which shows U,ez ^J = ^2 So far, we have shown the existence of a sequence of orthonormal sets of functions the limit of which is dense in L2. It might appear that we are nearing the end of our search for basis functions on which to base our wavelet transform. What we have is a good foundation for an approximation scheme, however this scheme falls short of satisfying all the conditions we need for our basis functions. Namely, projecting onto Vj for fixed j doesn't provide any frequency information. If we attempt to get frequency information by projecting onto other subspaces Vn for various n ^ j, we no longer have a orthonormal system, since <j>jk is not orthogonal to 4>nm for j 7^ n as either Vj <Z Vn ox Vn <Z Vj. This leads us to introduce the detail space Wj associated with Vj, which we will call the scaling space. Formally, Wj is the orthogonal complement of Vj in V^_i. This allows us to express Vj_i as the direct sum of Vj and Wj. We can express this graphically as Vj -> vj+1 -> vj+2 -+vj+3^>--\ \ \ \ wj+l wj+2 wj+3 Consider the sequence of spaces Wj, since each is the orthogonal complement of its companion scaling space Vj and Wj+i is a subspace of Vj, it is clear that Wj 25 is orthogonal to Wj+1. Additionally, we saw earlier that f]jezvj Ujez Wj = ^2- Since tne w s are / = {°K this im PÜes all orthogonal, we have a direct sum decomposition of L2 namely, This certainly is helpful, as all we need to do now is find an orthogonal basis for each Wj. Taking the union of all these basis functions will yield an orthogonal basis for L2. This may seem to be nearly as daunting a task as we initially faced, but we can now take advantage of the highly structured nature of the Vj spaces and the orthogonality of Wj to the Vj. First, observe that since Wj C V}_i, perhaps we can use the orthogonal basis for V}^ to construct an orthogonal basis for Wj. Let us proceed along these lines. Any prospective basis function, </?, for W0, must be an element of W0 and since W0 C V-i we must have the following representation for <p V = Y,M{2x-l) (III.4) Such functions tp are the wavelets which we have alluded to throughout this paper. Since we are interested in compactly supported basis functions we assume {dt} to be finite. We also require cp to be orthogonal to <f> and all its integer translates, since W0 is the orthogonal complement of VJ,. To aid us in our investigation we use the following representation of the Poisson Summation Formula, a derivation of which is found in the appendix, J2 (<P (x) ,<t>(x- I)) e~iajl = J2^(co + i 2/OT) $ (u + 2kir) for k,l£Z. (III.5) k Observe that this formula establishes the equivalence of the following statements: 1. (p(x) is orthogonal to 4>(x - I) for I € Z. 2- Efcez * (w + 2&TT) $ (u + 2kn) = 0 for all wGi 26 We can see this since if 1 is true then the sum on the left hand side of III.5 is clearly zero for all UJ e K. Conversely, if 2 is true then the left hand side of III.5 is a Laurent series in e"" which is zero at all values of a;, and must be identically zero, therefore each of the inner products in the sum must be zero. Now, let us apply this to the problem of finding a basis for Wo- Suppose <p (x) is orthogonal to <p (x - k) for k e Z, this is equivalent to J2 ^ (w + 2k7T) $ (w + 2kn) = 0 for all wel. (III.6) k We can use III.4 to get an expression for *, the Fourier Transform of ip * (o = /~ E w (2x - o e~ixidx = \ E d^^ (|) • As we have seen before, the Fourier Transform, $, of <f> can be expressed Consider the function ne)=;Ei*(£+2A;7r)i2A; We can write this function in the following way, F (0 = ]T $ (f + 2^7r)$(e + 2^7r). fc Now, by III.5 F is equal to £, (<f> (x), <ß (x - I)) = 1 because ^ form an orthonormal basis for V0. So, F (f) = 1 for all £ e R. Define the following trigonometric polynomials D(t) = ±£>2 ce n ceo = ^E » ~* 2 n 27 Substituting into III.6 we get 0 = 2^ * (a; + 2k7r) $ (u + 2/br) + 2&?r\ E"(^)*(^)c(^)*(^2-y. (III.7) Observe when k is even £> (^+|^) = D(fj, and when Jfc is odd D (^fz) = D {l + 7r)- Similarly, when & is even C (^fE) = C(f), and when k is odd C (a,+22fc7r) = C (| + 7r). We can rewrite the sum as E D ÜJ r Mi $ u + 2(2k)TTs + D(i-MiH $ a; + 2 (2k + 1) TT (III.8) Factoring out the terms which are independent of k we get "(!M!)£K!+^) + £ (! + 7r) C (! + 7r) £ |$ (! + TT + 2*7r) |2 . (IH.9) The summations above are simply F (|) and F (f + 7r) so we can write Y^ * (w + 2^TT) $ (w + 2A;TT) = But F (f) = F (f + TT) = 1, so we have (III.ll) To simplify our notation we will use u in place of f. Since 4>ok form an orthonormal set, C (ui) and C (u + n) cannot simultaneously be zero; this is a consequence of the third condition of Theorem II. 1. If III.ll is to be true then when C (u) is zero, C [u + TT) 28 cannot be zero. Therefore, D (u + TT) must be zero. Similarly, C (w + TT) and D (u) must have the same zero structure. Since C (u>) is a trigonometric polynomial, D (a>) must have the following form D(U) = A(Cü)C(U) + TT). Where A (to) is 27r-periodic, and A(u) + -n) = —A(u). Further, since we desire (p to be compactly supported, A (cu) must be a trigonometric polynomial. Now, III.11 becomes 0 = A (u) C (u + 7r)C (U) + A (u + IT) C (LO + TX + TT)C (u + ir) = A(u)C(cü + 7r)C(uj)-A(u))C(cv)C(u> + Tr) since C (u>) is 27r-periodic. Thus we have shown, subject to the above conditions on A(u), functions generated by III.4 lie in the orthogonal complement of Vo in V-\. What remains to be shown is that these functions ip are orthogonal to their integer translates. To show this we will again use the Poisson Summation Formula III.5. We see that orthogonality of cp (x) to tp (x — I) for I e Z is equivalent to 1 = Y2 * (w + 2^7r)^(o; + 2A;7r). k Substituting in the expression for the Fourier Transform of <p and simplifying in a manner similar to that used above we get l = ^2^(u + 2kir) tt (u + 2krr) = D (u) D(co) + D (u + n) D {u + TT). Since D (co) = A (u) C (u + TT), we have l = A(u)C(u + TT)A (U)C (U + TT) + A (u + TT) C (U)A (U + TT)C (W) Reorganizing slightly and recalling that A (co + TT) = -A (u>) we find l = A{u)A(u)C(w + TT)C(U + TT) + A(w) A{u)C (u)C(w). 29 (111.12) Since <j> (x) is orthogonal to cf)(x ~ I), we know C [to + IT) C (u> + n) + C (u) C (ui) = 1. If we additionally require \A (u)\ = 1, then III. 12 will be satisfied, and <p (x) will be orthogonal to <p (x — I). Let us collect the conditions we have imposed on A(u) and see what type function they characterize. 1. A (u) is a trigonometric polynomial, and 27r-periodic. 2. A(u + 7r) = -A{u). Z.\A{u)\ = l. Condition 3 implies the graph of A (u) traces out the unit circle or, in light of condition 1, A(w) = a0e~imw for |a0| = 1 and some m. Condition 2 implies m must be odd. We have a great deal of freedom in our selection of a0, but since we have so far only considered real valued {dt}, we will limit ourselves to a0 = ±1 and for this development we will assume a0 = 1. We will let m be the highest index of the set {ck} 1, that is, if {ck} has four elements we will set m = 3, because {ck} = {Co,Ci,C2,C3}. Having made these choices we see _l ill -y 2f^ D(co) Cke-ik{u+Tt) _ Or, after simplifying 1 m fc=0 which we can express as -ilw D(u 1=0 Condition 11.11 implies the highest index of {ck} must be odd. Further any odd m gives rise to a translated version of this same (p. 30 where dt = {-l)m~lc^Zi. That is {dt} = {-c^,c~i, -c^IJ, • • • , ci, -c{,c^}. Recall this {di} is such that <p (x) = ^ di4> {%x ~ 0 an orthonormal set in an d its integer translates <p(x — k) form WQ. It is not obvious that this set spans W0. We can see that it does by employing an argument similar to that above. Suppose g (x) € Wo, then g (x) G V-\ and g (x) ±V0. We can express g (x) as g(x) = J2ji4>(2x-l). i This, in turn gives us G (£), the Fourier Transform of g (x) C (0 = 5 !>-**(§)• Define the Laurent series in e~^, V (£) by r(0 = ££>-*. z Since g (x) JLVQ, I we can obtain the following relationship between C (£) and V (£) r +r + '- (£M£) (Hni Now, by exploiting the zero structure of C (£) we can see T (£) must have the following form r(£) = ß(e)c(e+7r), where B (£) satisfies the following: 1. B (£) is a Laurent series in e~1^. 2. B (£) is 27T-periodic. 3. B(£ + TT) = -£(£). Conditions 2 and 3 above imply B (£) consists solely of odd powers of e~^. Each of these odd powers in turn corresponds to a translation of the function ip. So, 31 any function g (x) € W0 can be represented by a weighted sum ofip(x-k). Thus we have shown ip (x — k) span the space W0. To summarize, so far we have a sequence of spaces Wj the direct sum of which is L2. We have demonstrated a basis {ip (x - k)} for W0 which is constructed from functions (f) (2x - I) which form a basis for V-X. Now we need to show an orthonormal basis for Wj is given by {<pjk = 2~^2(p (2~^x - k)}. To do this we use III.5. From standard Fourier Transform operations we can see *ifc(f) = 2^V2^W (2J'e) ^(f) = 2^-***$ (2>'f). It is easily verified using arguments similar to those above that ipjk is orthogonal to <f>jk. It is equally easy to verify ((pjk,(pjn) = 6n:k. Thus for each scaling level j we have an orthonormal set {(pjk}, which lies in Wj. All that remains to show is that each set spans the respective space Wj. This is accomplished in a manner completely similar to that used for W0. Finally, we have an orthonormal basis for L2, namely, {<pjk = 2~i/2ip (2-ix - k)} for j, keZ. B. ALGORITHM DEVELOPMENT Armed with this basis for L2 we now set out to implement a discrete transfor- mation which, as promised in the introduction, has good time-frequency localization, suitable power of approximation, and can be computed quickly. Time localization is a result of the compact support of the scaling functions and wavelets, and the fact that the support of these functions shrinks as the index of the subspaces V,- and Wj decrease (toward -co). Frequency localization is accomplished by dividing the spectrum of the input function into "octaves" each of which is represented by the projection of input function onto some Wj. This is not as sharp as the frequency localization of the Fourier Transform, but it is still quite good, and can be computed rapidly. The entire derivation of the basis functions was driven by the desire to take advantage of a recursive relation between basis functions. We do not have a recursive 32 relation between the actual basis functions <pjk, however, we do have a linear relation between the basis functions and a set of auxiliary functions <f)jk, which we call the scaling functions <p{x) = '%2dl<l>(2x-l). (111.13) i This would not be of much use without the associated recursive relation between the scaling functions at subsequent levels of scaling <t> (x) = 53 ck(f> (2x - k). (111.14) k We will exploit these relations to achieve a computational efficiency of order N operations. In computing a transform we are concerned with projections. Since projection operators are linear, we can recast 111.15 and III.16 in terms of the projection of an arbitrary function / (x) on <j>ik and (pjk. Specifically, (/ (:r), <f> (x)> = 53 ck(f (x) ,<f>(2x- k)) (111.15) k and (/ (x), <p (x)) = 53 dx (f (x), 0 (2x - k)) . (111.16) i Put another way, these relationships allow us to compute the projection of / (x) onto Wj given the projection of / (x) onto Vj-\. Before we can develop an algorithm, we need to recognize that the theory we have developed so far defines a transform which operates on functions in L2, square summable functions defined on the real numbers. Further, the transform projects the input function onto a bi-infinite sequence of subspaces of L2. In practice we are given a sequence of uniformly sampled values of the input function instead of the actual function. We will, for the purposes of this development, assume the input function / is zero at its boundaries. By doing this we can "pad" the function with zeros and apply the transform as if the function were defined over the entire real 33 line. There are methods of treating input functions which are not zero at their boundaries, but these methods are beyond the scope of this development. Next we must address the question of where in the bi-infinite sequence of subspaces to start the transform. The sampled values of / are a piecewise constant approximation to the actual input function2. We will define the fine scaling unit to be the "distance" (difference in independent variable) between successive samples. A consequence of the orthogonality of the scaling functions is that they must have approximation power at least one. This is easily seen since \m0 (z)\ + \m0 (z + n)\ =1 for all z 6 R, and mo (0) = 1, m0 (n) = 0. Put another way, the polynomial with coefficients {ck} has a root at 7T, which tells us the scaling function associated with {ck} has approximation power at least one. How do we use this fact? Over the region where / is represented by a constant, the scaling function and its translates can exactly represent /. Thus the projection of / onto the wavelets, (pjk, at this scaling level is zero. Also, at any finer scaling level the projection of / onto the wavelets must be zero. In light of these facts we will start the algorithm at scaling level I, where the scaling function 2~l/2(fr (2lx) has support width of one fine scaling unit. We call this the finest scaling level. We now outline the algorithm to compute the discrete, orthogonal wavelet transform. Figure 3 depicts the arrangement of wavelet and scaling functions. The first step in the algorithm is to calculate the projection of the input function onto the scaling functions at the finest scaling level. There are many ways to accomplish this, one way is presented in the appendix. Once we have these projections, the algorithm is simply a matter of using III.15 and III.16 to reorganize the data into the projections onto Vi+i and Wi+i. This is accomplished by taking weighted sums of the projections onto Vi. To calculate projections onto Vi+i, we use as weights {ck}. To calculate the 2 Other approximations may be used, for instance, if the input function is represented in a Bspline basis of order two, we would have a piecewise quadratic approximation to the input function. Similar results for higher order approximations can be developed in a manner similar to that used for piecewise constant approximation. 34 projections onto Wi+\, we use as weights {dx}. We repeat this process until we have calculated the projections onto all of the subspaces Vj and Wj, for I < j < m. Here m is the index such that the scaling function 2_m/2^> (2mx) has width of support equal to that of /. The discrete, orthogonal wavelet transform of the function / is given by the following collection of projections W (/) = {(/. <Pik) \l<j<m}ö {</, <j>mk)} . The following is a pseudocode version of what we have just described. j-l j-2 j=-3 (finest scaling level) weighted sum of fine scaling functions produces coarse wavelet or scaling function support of scaling functions at finest scaling level support of f(x) Figure 3. Organization of Basis Function 35 input: 1. n + 1 dilation equation coefficients for the wavelet we wish to use 2. sequence of TV values which represent the projection of / onto <f>ik at the finest scaling level. iV = n2p~1 + 1 for some p. begin algorithm: for j — I — m — 1 : 0 for k = 0 : n2^~1 - 3 (/, <Pjk) = \/2 ES/T ci-2k (/, <f>j-lj) (f, Wh) = V2 Yditk c^2/= (/, 0j-i,j> end end output :W(/) = {(/,^fc) | / < j < O}u{(/,0ofc)} This brings up another aspect of implementing the wavelet transform on a computer. The sequence of subspaces we project onto is bi-infinite. We saw earlier that we could pick a starting index based on the resolution with which / was represented. The algorithm to compute the transform must stop at some index. What should this index be, and what are the consequences of stopping the algorithm here? In this development we will stop the transform at the index where the scaling function has support width equal to the width of the input function sample. At each step of the transform we are calculating the projections of / onto succeedingly lower frequency basis functions as the support width of cpjk increases. Terminating the algorithm at any given index establishes a low frequency bound below which we have no frequency localization. This section has presented a very basic outline of the discrete, orthogonal wavelet transform algorithm. There are many choices as to the indices on which the algorithm should begin and terminate. The major concept of the algorithm is the use of the relationships III.15 and III.16 to rapidly compute projections at one level from those at a preceding level. It is precisely these relationships which allow the wavelet transform to be computed in order iV operations as we shall now see. 36 To start our study of the computational efficiency of the algorithm we will assume the input function is given as N sampled values. We further assume that iV = 2pn +1 for some p where n is the highest index of {ck}. We divide the algorithm into the following blocks. 1. Calculation of projections at finest scaling level: Nearly any quadrature rule can be used to calculate these projections in O (N) operations. 2. Calculation of wavelet coefficients: At each level j of the p levels of the algorithm we must calculate n2J — 2 wavelet coefficients, each of these coefficients is calculated by taking a weighted sum of projections onto the scaling coefficients at the preceding level. The computational cost for the weighted sum is n + 1 multiplies and n additions, this makes the total cost for this block v-i J2(n2j -2)(2n + l). j=o 3. Calculation of scaling function coefficients : At each level we must calculate the projections of the function onto the scaling functions. This is accomplished by taking a weighted sum of projections onto the scaling functions at the preceding level. The computational cost is identical to that for calculating the wavelet coefficients above. If we sum the computational cost for each of these blocks we get a total algorithmic cost of p-i O (N) + 2 ]T(n2J' - 2)(2n + 1). 3=0 The above sum is dominated by 2 (2n + 1) 2P which is O (N). So, we have developed a transform which provides good time-frequency localization, can be used as an approximation scheme, and is rapidly computed. Some applications and directions for further studies are discussed in the next chapter. 37 38 IV. CONCLUSIONS AND DIRECTIONS FOR FURTHER STUDY We have shown that we can develop a transform based on solutions to the dilation equation. This transform is useful because we can alter the characteristics of this transform simply by changing the dilation equation coefficients. Also, we can compute this transform quickly. In fact, the order JV operations required for the wavelet transform is considerably better than the N log (JV) operations required for the fast Fourier Transform for large JV. The wavelet transform has the added advantage of time localization. A major advantage of the Fourier Transform is that the basis functions are eigenfunctions for an operator of fundamental importance, namely the Laplacian operator. The wavelet transform, on the other hand, has been restricted predominantly to use in the signal processing field. In this field the wavelets time-frequency localization and rapid computation are great assets, and projection onto a basis of eigenfunctions is not always as desirable as these other properties. To be as useful as the Fourier Transform, wavelets must be "well behaved" under some operator of interest. By well behaved we mean the image of the wavelet under the operator is simply related to the wavelet or its translations and dilations. Eigenfunctions for an operator are the ultimate in well behaved functions. It is clear from our preliminary work in Chapter II that convolution and convolution type operations are the natural operations for wavelets. If we are to find an operator under which wavelets would be well behaved, it is natural to first consider operators based on convolution like processes. A broad field of such operators is the ZygmundCalderon group of operators. A description of these operators is beyond the scope of this work, but this is a burgeoning area for wavelet analysis. An area which has great potential is that of tailoring a wavelet to be well behaved under a particular operator. To see how we might tailor a wavelet we should consider that if we wish our wavelet to have support of width n at the zeroth scaling 39 level, {cfc} must have n + 1 elements, namely CQ, ... , cn. We still desire the wavelets to be orthogonal, so we must meet the necessary conditions of section 1.3. Specifically, YLk ckCk-2m = 25m for m 6 Z. Each of these conditions can be represented as a simple bi-linear form c*D2mc = 25m, where * indicates conjugate transposition and D2m is the 2m down shift matrix. That is, D2m has ones on its 2mth diagonal and zeros elsewhere. If these conditions were linear then we could easily determine the solution space for this system. However, these conditions are not linear and systems of bi-linear forms are not well studied. We can see for m = 0 the condition is J^k \ck? = 2- We can also see for each positive value of m the condition is equivalent to the corresponding condition for —m, since c*D2mc = 0 = 0* = c*D2m(c*)* = c*D_2mc. So, we have reduced the number of conditions from n to n^. To make further progress we need to be able to determine the solution space of a system of bi-linear forms. The use of the term space here refers to the set of all solutions, not necessarily a linear space. The solutions of a system of bi-linear forms need not be linear. Once we have the solutions to this system we would then determine which of these solutions could be written as linear combinations of the vectors {[v£]} as defined in section 1.2 where p is our desired approximation power. This would satisfy the sufficient condition for approximation power. Now we check these solutions against the sufficient condition for orthogonality presented in section 1.3. The result would be a collection of dilation equation coefficients which would yield orthogonal wavelets with the desired approximation power. Perhaps among this collection we would find dilation equation coefficients which would be well behaved under the operator of interest. 40 REFERENCES [1] Volkmer, H., Distributional and Square Summable Solutions of Dilation Equations Preprint [2] Daubechies, I., Ten Lectures on Wavelets, SIAM Philadelphia PA 1992 [3] Strang G. and Fix, G. A Fourier Analysis of the Finite Element Variational Method, Centro Intemazionale Matematico Estivo, Constructive Aspects of Functional Analysis, Edizioni Cremonese, Roma 1973. [4] Schoenberg, I.J., Cardinal Spline Interpolation SIAM Philadelphia PA, 1973 [5] Cohen, A., Ondelettes, Analyses Multiresolutions et Filtres Miroir en Quadrature, Ann. Inst. H. Poincare, Anal. Non Lineaire, 7, pp. 439-459, 1990 [6] Royden, H.L., Real Analysis, Macmillian Publishing Company, New York NY, 1988 41 42 APPENDIX A. DERIVATION OF POISSON SUMMATION FORMULA We start with the Poisson Summation Formula as found in any text on Fourier Analysis ^/(n) = ^F(2fc7r), nez fcez where F (£) is the Fourier Transform of / (x). Consider the function f(cjJ) = (iP(x),<l>(x-l))e-i»1. We see the Fourier Transform, in the variable Z, of / is given by /oo y,(x),<fi(x-i)) e iwl „—i; e *'<fl. -oo Expanding the inner product and simplifying the exponential we get /oo poo i){x)(j){x- l)dx e-il(-w+tidl. / oo J — oo Now, change the order of integration, and we see /oo poo cj){x- I) e-a{u+®dl dx. V> (x) / ■oo J —oo Make the change of variables y = x — I, and we now have /oo pOO ix{ui+ V> (x) e~ ® / -oo <j> (y) eiy{-w+t]dy dx. J —oo We can write the integral in y above as /oo (j){y)eiy^+üdy = ${Lü + Z). -oo This leaves us with /oo </> (x) e-ix{w+Ö<S> -oo 43 (UJ + £)dx. Since $ (u + £) has no x dependence, we can write this as /oo $ (x) e~ix(w+Üdx$(cj + 0-oo The integral in x above is simply the \t (a; + f) which leads us to Now, applying the Poisson Summation Formula we get J2 <V> (x) ,<j>{x- 0) e-*"' = ]T * (u + Jez fcez 2A;TT) $(a; + 2Ä;7r), which is the desired representation of the Poisson Summation Formula. 44 APPENDIX B. PROJECTIONS AT FINEST SCALING LEVEL To see how we might accomplish this, consider the following diagram. f(x) aligned with constant f(x) special treatment } requires aligned with constant f00 support width of scaling functions at finest scaling level Figure 4. Scaling Functions at Finest Scaling Level Each projection at the finest scaling level is the projection of (f>ik onto either a constant or piecewise constant function. Where / is constant over the support of (f>ik the projection is simply a weighted integral of (ßik. Where / is piecewise constant over the support of 4>Lk we can divide the domain of integration so that the projection is given by a sum of integrals over domains where / is constant. Each of these subintegrals is given by a weighted integral of <j>lk over a portion of its support. For instance, if we are computing the projection of / onto the second scaling function in Figure 4, we would have the following / / 0*0 02 {x) dx = a JO <j)2(x)dx + a JO <j)2 (x) dx + b Ji cj>2 (x) dx. J2 Similarly, the projection onto the third scaling function would be / / {x) <f>3 {x) dx = a (f>3(x)dx + b (j>z (x) dx + b (j>3 (x) dx. Jo Jo J\ J2 45 The sub-integrals, fQ (j)3 (x) dx, can be computed once and stored for use since they depend only on the wavelet used. We will assume the projection of the input function onto Wi is zero. This prevents us from incorporating into the transform the discontinuities introduced by the piecewise constant approximation of /. It is worth noting that at this point in the algorithm we have all the information we ever will about the input function. In fact, we do not use the function for the rest of the algorithm. The remainder of the algorithm is devoted to rearranging the projections of / onto the scaling functions at the finest level into projections of / onto the spaces Wj for j > I. 46 INITIAL DISTRIBUTION LIST 1. Defense Technical Information Center Cameron Station Alexandria, VA 22304-6145 2. Library, Code 52 Naval Postgraduate School Monterey, CA 93943-5101 3. Professor Carlos F. Borges, MA/Bc Department of Mathematics Naval Postgraduate School Monterey, CA 93943-5000 4. Professor Richard Franke, MA/Fe Department of Mathematics Naval Postgraduate School Monterey, CA 93943-5000 5. Professor Van Emden Henson, MA/Hv Department of Mathematics Naval Postgraduate School Monterey, CA 93943-5000 6. LT James K. Logue Submarine Officer's Advanced Course Submarine School New London Groton, CT 06340 7. Dr. Lou Cleveland Department of Mathematics 3094 Indian Circle Marianna, FL 32446 8. Professor Jack Rogers Department of Mathematics 228 Parker Hall Auburn University Auburn, AL 36849 47 9. Professor William Transue Department of Mathematics 228 Parker Hall Auburn University Auburn, AL 36849 48

Download PDF

advertisement