The discrete, orthogonal wavelet transform, a projective approach Logue, James K.

The discrete, orthogonal wavelet transform, a projective approach Logue, James K.
Calhoun: The NPS Institutional Archive
Theses and Dissertations
Thesis Collection
1995-09
The discrete, orthogonal wavelet transform, a
projective approach
Logue, James K.
Monterey, California. Naval Postgraduate School
http://hdl.handle.net/10945/35162
NAVAL POSTGRADUATE SCHOOL
MONTEREY, CALIFORNIA
19960221 045
THE DISCRETE, ORTHOGONAL WAVELET
TRANSFORM, A PROJECTIVE APPROACH
by
James K. Logue
September, 1995
Thesis Advisor:
Carlos F. Borges
Approved for public release, distribution is unlimited.
W3C QUALITY INSPECTED 1
REPORT DOCUMENTATION PAGE
Form Approved OMB No. 0704-0188
Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instruction, searching existing data
sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any
other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information
Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302, and to the Office of Management and Budget, Paperwork Reduction
Project (0704-0188) Washington DC 20503.
1.
AGENCY USE ONLY (Leave blank)
2.
3.
REPORT DATE
September, 1995.
4.
TITLE AND SUBTITLE
THE DISCRETE, ORTHOGONAL WAVELET TRANSFORM, A PROJECTIVE
APPROACH
REPORT TYPE AND DATES COVERED
Master's Thesis
FUNDING NUMBERS
6.
AUTHOR(S)
James K. Logue
7.
PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)
Naval Postgraduate School
Monterey CA 93943-5000
8. PERFORMING ORGANIZATION
REPORT NUMBER
9.
SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)
10. SPONSORING/MONITORING
AGENCY REPORT NUMBER
11. SUPPLEMENTARY NOTES The views expressed in this thesis are those of the author and do not reflect the
official policy or position of the Department of Defense or the U.S. Government.
12b. DISTRIBUTION CODE
12a. DISTRIBUTION/AVAILABILITY STATEMENT
Approved for public release; distribution is unlimited.
13.
ABSTRACT (maximum 200 words)
All integral transforms can be viewed as projections onto collections of functions in a Hubert
space. The properties of an integral transform are completely determined by the collection of functions
onto which it projects. The wavelet transform projects onto a set of functions which satisfy a simple
linear relationship between different levels of dilation. The properties of the wavelet transform are
determined by the coefficients of this linear relationship. This thesis examines the connections
between the wavelet transform properties and the linear relationship coefficients.
14.
SUBJECT TERMS Wavelet, Multiresolution, Dilation
15. NUMBER OF PAGES
59
16. PRICE CODE
17. SECURITY CLASSIFICATION OF REPORT
Unclassified
NSN 7540-01-280-5500
18. SECURITY CLASSIFICATION OF THIS PAGE
Unclassified
19. SECURITY CLASSIFICA- 20. LIMITATION OF
ABSTRACT
TION OF ABSTRACT
Unclassified
UL
Standard Form 298 (Rev. 2-89)
Prescribed by ANSI Std. 239-18 298-102
11
Approved for public release; distribution is unlimited.
THE DISCRETE, ORTHOGONAL WAVELET TRANSFORM, A
PROJECTIVE APPROACH
James K. Logue
Lieutenant, United States Navy
B.S., Auburn University, 1988
Submitted in partial fulfillment
of the requirements for the degree of
MASTER OF SCIENCE IN APPLIED MATHEMATICS
from the
NAVAL POSTGRADUATE SCHOOL
September 1995
Author:
Approved by:
Van Emden Henson, Second Reader
Richard Franke, Chairman
Department of Mathematics
in
IV
ABSTRACT
All integral transforms can be viewed as projections onto collections of functions in a Hilbert space. The properties of an integral transform are completely
determined by the collection of functions onto which it projects. The wavelet transform projects onto a set of functions which satisfy a simple linear relationship between
different levels of dilation. The properties of the wavelet transform are determined
by the coefficients of this linear relationship. This thesis examines the connections
between the wavelet transform properties and the linear relationship coefficients.
VI
TABLE OF CONTENTS
I.
INTRODUCTION
II.
EXISTENCE, UNIQUENESS, APPROXIMATION, AND OR-
III.
IV.
1
THOGONALITY
5
A.
EXISTENCE AND UNIQUENESS
5
B.
SOME TOOLS FOR DILATION EQUATIONS
6
C.
APPROXIMATION
10
D.
ORTHOGONALITY
14
THE DISCRETE ORTHOGONAL WAVELET TRANSFORM
21
A.
DECOMPOSITION OF THE FUNCTION SPACE L2
21
B.
ALGORITHM DEVELOPMENT
32
CONCLUSIONS AND DIRECTIONS FOR FURTHER STUDY 39
REFERENCES
41
APPENDIX A. DERIVATION OF POISSON SUMMATION FORMULA
43
APPENDIX B. PROJECTIONS AT FINEST SCALING LEVEL . .
45
INITIAL DISTRIBUTION LIST
47
Vll
Vlll
LIST OF FIGURES
1.
Dilation of Haar Function
3
2.
Dilation of "Hat" Function
7
3.
Organization of Basis Function
4.
Scaling Functions at Finest Scaling Level
DC
.
35
45
I.
INTRODUCTION
A great deal has been written about wavelet analysis in the last several years.
Much of this work however is presented in the language of signal processing. This
is to be expected since wavelet analysis has been primarily developed in the signal
processing field. This paper presents the discrete wavelet transform in a projective
framework. The discrete wavelet transform, like most integral transforms, can be
viewed as projection onto a new basis determined by a given collection of functions. In
light of this the properties of the transform are primarily determined by the properties
of the basis functions. The development of a transform is then a question of finding
a basis which will produce the properties we desire.
What properties do we desire in the wavelet transform? We would like the
transform to incorporate the spectral decomposition, or frequency localization, of the
Fourier Transform. In addition, we would like to have time-localization of the input
function. This allows us to determine when in time a given component of the input
function occurs. Frequency localization in the Fourier Transform is accomplished by
projecting onto sines and cosines which have non-compact, in fact infinite, support.
This precludes the Fourier Transform from having time localization capability. Time
localization can be achieved by using the short time Fourier Transform (STFT),
which analyzes the input function in small time windows. This time localization is
fixed in scale, depending on the time window used. Another desired property of the
wavelet transform is arbitrary approximation power. This is useful if we wish to
use the transform in a data compression scheme, that is, setting to zero transform
coefficients below a certain threshold level and storing the input function in terms
of this reduced coefficient set. Finally, and perhaps most importantly, we desire the
basis onto which we project to be orthonormal. This will allow easy inversion of
the transform. If the basis is orthonormal, inversion is accomplished by weighted
superposition of the basis functions using the transform coefficients as weights. This
is the case with the Fourier Transform which uses the orthonormal sine and cosine
basis. This is contrasted with the Laplace Transform which uses an exponential basis
which is not orthogonal over the real line. This is why the Laplace Transform must
be inverted by a contour integral in the complex plane.
The question of developing a transform with these properties is reduced to that
of finding a basis for L2, the space of square integrable functions on the real line, which
possesses the desired properties, namely, time and frequency localization, arbitrary
approximation power, and orthogonality. If we restrict our attention to compactly
supported functions, we can achieve a measure of time localization, however only at
a fixed level dependent on the support of the basis function. This severely limits the
frequency information available as well, since we can only clearly discern the frequency
which corresponds to the basis functions. Compactly supported functions are easier
to compute with, hence their use in finite element methods, and we would like to
use them here. We need to find a way to use compactly supported basis functions
without limiting the time or frequency information available to us. The solution is to
use dilations of a compactly supported function. A dilation of / (x) is / (ax) for some
a, normally a > 0. Notice that as we dilate a function / the support of / changes
and also the frequency of / changes. This allows us to cover much wider time and
frequency scales than using basis functions of fixed compact support. To give us
the maximum amount of flexibility we will use a basis composed of translations and
dilations of a fixed function <j>.
Consider the dilation equation, or two scale recursion relation
<j) (x) = 2_jCk<t> (2x — k)
for some {ck} .
(1.1)
k
The solution is completely determined by the dilation equation coefficients {cfc}. Also
notice this dilation equation with coefficients {c0 = l,c1 = l}is satisfied by the Haar
function:
h(x) =
h(x)
1
for x e [0,1]
0
for x £ [0,1]
h(2x)
h(2x-l)
Figure 1. Dilation of Haar Function
The Haar Transform which uses as a basis functions derived from h(x) is in
fact an early example of a wavelet transform and predates the current interest in
wavelet analysis by several decades.
In this paper we show that solutions to 1.1 exist which have arbitrary approximation power and are orthogonal to their integer translates. We further show how
these solutions can be used to construct a basis for L2, and how the discrete transform
which represents projection onto this basis can be computed in order iV operations.
II.
EXISTENCE, UNIQUENESS,
APPROXIMATION, AND ORTHOGONALITY
A.
EXISTENCE AND UNIQUENESS
Before we investigate the relationship between the coefficients of the dilation
equation and the properties of the solutions, we need to determine under what conditions a solution in fact exists. To begin our search for solutions to the dilation
equation let us consider the iteration
<f)n+1 = ]P ckcj)n (2x - k).
{Ill)
k
Solutions to the dilation equation are fixed points of the dynamical system defined
by this iteration. The following theorem, which we present without proof, gives
conditions under which square summable solutions to the dilation equation exist and
is found in [Ref. 1].
THEOREM II.1. If the {ck} satisfy the following conditions
1. {et} is a finite set,
2- Ek c* = 2,
3. \m0 (z) |2 + \m0 (z + TT)|2 = 1 for all zeE, where m0 (z) = £fc cke~ikz
then the dynamical system in II. 1 has a nontrivial square summable fixed point
with compact support.
In the same paper, a similar result is presented for distributional solutions to
1.1 which does not require the third condition. In practice we are looking for a basis
for the function space L2, so we restrict our attention to square summable solutions.
Uniqueness of this solution, up to normalization, is guaranteed by the same
conditions which give us existence. In the square summable case the normalization is
/ <j)dx = 1. Observe that this normalization requirement is consistent with condition
2 above since
/ (f> (x) dx =
/ Y^ ck(f) (2x — k)
= -^2,ck j 4>(y)dy
k
implies
T,/_,ck = 1
fc
where y = 2x - k
•*
or
y^cfc = 2.
k
Before proceeding with our discussion of approximation let's look at some
solutions to 1.1 which are familiar to us. In the introduction we saw for coefficients
{1,1} the solution to 1.1 was the Haar function. Other familiar functions which are
solutions to the dilation equation, though we may not recognize them as such, are the
Dirac delta function for {2} and the cardinal B-splines for {(™)} normalized to sum
to two. The Figure 2 shows the dilation of the "Hat" function which is the solution
for the dilation equation for {\, 1, |}, the binomial coefficients for n = 2, normalized
to sum to two.
The Cardinal B-splines illustrate why the third condition in Theorem II. 1 is a
sufficient but not necessary condition for square summability. The Cardinal B-splines
are clearly square summable, but it is easily seen that {(£)} do not satisfy the third
condition of Theorem II. 1.
B.
SOME TOOLS FOR DILATION EQUATIONS
Before proceeding with our investigation of the relationship between the di-
lation equation coefficients {ck} and the corresponding solution (f>(x), we should introduce a very useful tool in the study of dilation equations. Dilation equations are
difficult to study because only in special cases do we actually know what the solution
function is. We are usually limited to knowing only the recurrence relation between
different scalings of the function. Faced with this lack of information we must turn
to the Fourier Transform to glean any information we can about the function. Let us
see what the recursive nature of 1.1 can tell us about the solution (f>(x).
-0.2-
Figure 2. Dilation of "Hat" Function
We are given
4>{x) = 2_^ Ck<ß(2x — k)
for some {ck}
(II.2)
k
Now, consider the Fourier Transform <&(£) of (j){x)
/OO
<f)(x)e-^xdx
(II.3)
-oo
substituting II.2 into II.3 we get
/oo
e-^xJ2ckH^-k)dx
OO
i.
or
/oo
<f>{2x - k)e-iixdx
in each integral of this sum make the change of variables y = 2x - k which yields
*(6 = ^ECke * /
<f>(y)e^dy.
(II.4)
Now, notice the integral in II.4 is <&(|) ,so we have
Applying this process repeatedly to $(£/2), $(£/4), ...$(£/2n) we get
Let us now take the limit as n —¥ oo,
oo
1
If we require by way of normalization that J (frdx = 1 then we have $(0) = / (f)dx = 1
and II.5 becomes
oo
1
Notice we could write this infinite product as
oo
1
Now we see the importance of the 7710(2) expression in the third condition of Theorem
II.1, the behavior of this trigonometric polynomial determines the convergence of the
infinite product representation of the Fourier Transform, and therefore the existence
of a square summable solution to 1.1. A fairly simple, yet powerful result of this
infinite product representation of the Fourier Transform is the Convolution Theorem
for Dilation Equations.
THEOREM II.2. Let </> and ip be solutions to the dilation equation for coefficients
{cfc} and {di} respectively. The function (ß*ip given by the continuous convolution of
<fi with cp is the solution to the dilation equation with coefficients {bj}, where {bj} is
one half the discrete convolution of {c^} with {di}.
Proof. As we have seen the Fourier Transforms of <f> and cp are given by
00
^
00
Cke
®(0 = n=0
Il2^2
^
k
k
and
1
die L
^(o = n=0
H^J2
^
I
respectively. The product of these transforms $W is given by
oo
1
cke ä
oo
1
^=H2^2 ^ n2^2die^Lra=0
A;
m=0
I
We can combine the two product operators by associating the factors where n = m
to get
00
1
n=0
-
1
k
I
Observe each factor in this infinite product is a product of two polynomials in e v- ,
-in
namely, | ^fc c^e v- and | Yli die 2" • Let p (£) be the product of these polynomials.
Prom polynomial multiplication, we see the coefficients of p (£) are given by
A
1
bx= ol-jCidx-^
1
3=0
that is, the set {b\} is one half the discrete convolution of {ck} with {d{\, and the
product polynomial p is
P(0 = v^2he>n
x
Substituting this into the expression for $\£ gives us
00
1
^ = H^J2bxe^n=0
X
Observe the right hand side of this equation is the infinite product representation
of the Fourier Transform of the solution to the dilation equation with coefficients
{b\}. Thus, $\I> is the Fourier Transform of the solution to the dilation equation for
coefficients {bj}. Now, from the convolution theorem for Fourier Transforms we know
t&Nt is the transform of (f> * ip, the continuous convolution of <j> with </?. Since Fourier
Transforms are unique this completes the proof.
D
Armed with this result we see why the Cardinal B-splines are solutions to
the dilation equation for the binomial coefficients normalized to sum to two. If we
develop B-splines from a repeated convolution standpoint as is done in [Ref. 4], the
9
relationship between B-splines and binomial coefficients becomes transparent in light
of Theorem II.2. To generate the B-splines we start with the Haar function which we
can consider the "zeroth" order B-spline (piecewise constant approximation), we get
the next higher order B-spline (piecewise linear approximation) by convolution with
the Haar function. By our convolution theorem this says we should take one half the
discrete convolution of {1,1} with itself. Convolving {1,1} with itself we get {1, 2,1}
the binomial coefficients for n = 2, which we must normalize to sum to two. Similarly,
to generate the (n + l)st degree B-spline we convolve the nth degree B-spline with the
Haar function, so we convolve the coefficients of the nth degree B-spline with {1,1}.
This gives us the binomial coefficients for n+1, which again we must normalize to sum
to two. The appearance of the B-splines is of more than passing interest. B-splines
and their translates are often used as a basis for approximation. This is very similar
to the scheme we are investigating. We are pursuing the goal of representing L2 in
its entirety rather than approximating functions in a subspace of L2. Another aspect
of spline approximation which will be of great importance in the next section is the
use of sliding window filters as a tool to smooth data. Sliding window filters can be
viewed as convolution with the Haar function. Thus repeated application of sliding
window filters is equivalent to repeated convolution with the Haar function, both of
which lead to smoother data, or smoother approximation if one is constructing a basis
function, as we shall see in the next section.
C.
APPROXIMATION
Since we will be using solutions to 1.1 as a basis for approximation we would like
to know how well an arbitrary function can be approximated by 0 and its translates.
Our measure of approximation power will be polynomial precision, or what degree
arbitrary polynomial can be exactly represented by <f> and its translates. A key step in
finite element methods is to approximate a function by the translates of a compactly
supported basis function. To determine the approximation power of solutions to the
10
dilation equation we consider the work done on the finite element method by Strang
and Fix. In [Ref. 3] they show that the following statements are equivalent:
1. Any polynomial of degree less than or equal to p— 1 can be exactly represented
by a linear combination of <j> and its translates.
2. $(0) ^ 0 and J^$(£) k=2™= 0 for n e Z,n ^ 0 and a = 0,1, ...,p.
The first of these statements is the definition of approximation power that
we are interested in, the second is related to the Fourier Transform of the function
<j>. We will use the second statement to construct a condition on {cjt} that will give
the desired approximation power. First notice that $(0) ^ 0 is taken care of by our
normalization requirement since
$
(0) = f (j>e-Mxdx = f (j)dx
To satisfy the rest of this statement we must look at the infinite product representation
of <3> (0
oo
1
2n
n=l
2
k
Since this product converges and $(0) ^ 0 we see that the product vanishes for some
value of f if and only if at least one of its factors vanish for that value of £. Observe
that the following condition is sufficient to ensure the desired behavior of <& (£):
•
z = 7T is a root of order p of m0 (z).
In the previous section we saw
* (0 = 11^0 1
2n
(II.6)
n=l
Now, for all I E Z, I ^ 0, there exists n such that ^± mod 2n =
IT.
Therefore, one
of the factors in II.6 is zero, and <3> (2nl) = 0. Now consider 4
d£ [$ (£)],
ml
*™l-*L
° |
n ,
(t\ d
n^i^
.fc^n
11
(II.7)
Since m0 (z) has a root of order p at
TT,
the right hand side of II.7 is zero. Similarly
we can see &£■ [$ (f)] = 0 for j = 0,1, ..p provided m0 (z) has a zero of order p at
TT.
Unfortunately, this elegant sufficient condition on m0 (z) is not necessary.
Observe, if ± [$ (£)] = 0, ^ [m0
of
(TT)]
need not be zero, since m0 (f), which is a factor
Il/fc^n mo ( JF) could be zero, in which case, II.7 will be satisfied regardless of the
value of ^ [m0
(TT)].
The sufficient condition for the approximation power of $ can be arrived at in
several ways. In [Ref. 2] it is presented as a sum condition, that is:
N
^2 (-1)1 lkct = 0
for k = 0,1, ...,p - 1.
(II.8)
j=o
We can see this is equivalent to our condition that
TT
be a root, of order p, of m0 (z),
by first expressing the sum condition as a matrix equation,
Kc = 0
where
KisapxiV + l matrix with the following structure:
1 -1
1
-1
1
••■
0 -1
2
-3
4
...
0 -1
4
-9
16
•••
0 -1
8
-27
64
•••
0 -1
16 -81 256 •••
Calculating the null space of this matrix yields a basis for the null space which is
given by {[vf\} where [uj] is the N + 1 long vector whose non-zero entries begin
at the kth position and are the binomial coefficients (p) for k = 1, 2,..., N - p. For
12
example, a basis for the null space of
1 -1 1 -1
1
0-12 -3
4
0-14 -9 16
IS
'
0
3
1
3
*
>
1
)
3
1
3
0
1
k
>
j
Since this is a basis for the null space of K, any {ck} which satisfies the sum condition II.8 must be a linear combination of the basis vectors. Since the basis vectors
are shifted copies of one vector, the matrix whose columns are the basis vectors is
Toeplitz. So, any linear combination of the basis vectors can be represented as the
product of this Toeplitz matrix T with a vector whose elements are the weights in the
linear combination. Alternatively, we can consider this multiplication by a Toeplitz
matrix to be discrete convolution of the vector of weights with the vector of binomial
coefficients, i.e. polynomial multiplication if we consider the elements of these vectors
to be coefficients of polynomials.
So, what does all this do for us? Any {cfc} which satisfies II.8 can be considered
to be the coefficients of a polynomial / (x) which in turn can be expressed as the
product of two polynomials g (x) and h (x). The coefficients of g (x) are given by the
binomial coefficients (?) where p is the required approximation power. Recall that
the polynomial with coefficients (?) is g (x) = (x + l)p. That is, the polynomial with
coefficients {ck} has a root of order p at x = -1. Now, if we consider the relationship
between the trigonometric polynomial m0 (z) = J^k ckeikz and the polynomial / (x) =
Ylk ckxk, we see / (eiz) is equivalent to mQ (z). Further, if / (x) has a root at x = -1,
then m0 (z) has a root at
TT.
13
Yet another way to see the sufficiency of our condition for approximation
power is to observe that each higher order of the root of m0 (z) at ■n is equivalent
to an additional factor of (x + 1) in the polynomial whose coefficients are {c^}. In
turn, each of these factors is equivalent to the discrete convolution of a smaller set
of coefficients with {1,1}. Now, by our convolution theorem for dilation equations
each of these discrete convolutions is equivalent to the continuous convolution of a
function which precedes <f> with the function whose dilation equation coefficients are
{1,1}. Of course the function with dilation equation coefficients {1,1} is the Haar
function. Recall that the Fourier Transform of the Haar function is -—' ^', which
decays like j. Each factor of [x + 1) in the polynomial whose coefficients are {ck}
introduces a factor which decays like j in the Fourier Transform of <j). Each factor
of j in the Fourier Transform implies one degree of smoothness, or approximation
power in the function <f>.
D.
ORTHOGONALITY
As stated in the introduction, an orthogonal basis of functions is advanta-
geous. Orthogonality allows us to reconstruct the input function from its transform
coefficients by summing the basis functions weighted by the transform coefficients.
This is the case in the Fourier Transform. By contrast the Laplace Transform uses
the exponential functions {epx} as a basis. These functions are not orthogonal over
the real line and as a result the inverse Laplace Transform involves a contour integral
in the complex plane. We wish to take advantage of this ease of inversion, so we will
restrict our attention to orthogonal basis functions. Again, we are limited by not
having an explicit representation of our basis functions. We must show that there
exist solutions to 1.1 which are orthogonal to their integer translates. Additionally,
we still require these functions to have the desired approximation power.
Fortunately, such solutions do exist. In fact, we have already seen one of
them. The Haar function is the solution to the dilation equation with coefficients
14
{1,1}. Since the Haar function has support width of one unit, any integer translate
of the Haar function has support which does not intersect the support of the original
Haar function therefore the Haar function must be orthogonal to any of its integer
translates. However, the Haar function has poor approximation power, and is not
an ideal candidate for use as a basis for an integral transform. The other members
of the Cardinal B-spline family are used for piecewise approximation by higher order
polynomials and therefore have increasing approximation power and might be good
candidates for transform basis functions. Unfortunately, the higher order Cardinal
B-splines are not orthogonal. This is easily seen since each of the B-splines is positive
valued over their support and the product of two B-splines with intersecting support
would be positive, thus the inner product of these B-splines would be non-zero. In this
section we develop both necessary and sufficient conditions for orthogonal solutions
to the dilation equation.
Let us first look at the consequences of this orthogonality, and develop a
necessary condition. Suppose <j) (x) is orthogonal to its integer translates <j>(x — k)
for all k E Z, k ± 0. Orthogonality here is with respect to the inner product on L2,
that is
/oo
f(x)Jfr)dx.
-oo
So far we have only considered real dilation equation coefficients. We will continue
to restrict our attention to real coefficients. However, the following development is
presented allowing the possibility of complex coefficients to be consistent with the
above inner product.
Suppose <j> (x) is orthogonal to <j> (x - k) for k G Z, k =£ 0; additionally suppose
(j> {2x) is orthogonal to <j> {2x - j) for j G Z, j / 0, then
/oo
4> (x) <j>{x- k)dx.
■oo
Substituting in LI for <j> (x) and <f>(x — k) we get
/OO
/.OO
<t> (x) <ß(x- k)dx =
OO
]T Q</> (2x - I) ^c^<t>{2(x-k)-m)dx.
J—OO
i
15
Interchanging the order of integration and finite summation
/OO
/"O0
c
(j)(x)(t)(x-k)dx = Y^^ i^
00
l
m
<j>{2x-l)<f>(2x-2k-m)dx.
(II.9)
^-°°
Since <f> (2x) is orthogonal to <f> (2x - j) for j e Z, j'^ 0 the integral on the right hand
side of II.9 must be zero unless I = 2k + m. This simplifies II.9 to
/oo
/>oo
0 (x) <f> (x - k)dx = ^2 c*cT2fc /
-oo
1
4>(2x -l)</> {2x - l)dx.
J—00
So, if <f> (x) is to be orthogonal to (f> (x — k) we require
/oo
(/)(2x-l)(f){2x-l)dx = 0.
And since we assume (j) is not identically zero, the above integral cannot be zero, thus
for k €Z, k ^0.
^C/Q7^ = 0
(11.10)
1
Additionally, when k = 0
/oo
/-oo
(j) (x) (j) {x)dx = Y^ CJQ /
-00
f
<j)(2x — l)(j) (2x - l)dx.
J—00
Let j/ = 2a; — I in the right hand integral and we get
/OO
1
/"OO
<ß(x)<f)(x)dx = -^2clci
•00
~
1
(f>(y)<f>{y)dy.
J—00
For this equation to be true we must have
y^cicj = 2.
We can combine these to get a necessary condition for orthogonality
^2 ci^2k = 2Sk
for all k E Z
(11.11)
where <5fc is Kronecker's delta function: 5k = 1 for k = 0, <5fc = 0 for k ^ 0.
It appears that 11.10 is yet another condition we are imposing on the dilation
equation coefficients. However, with a little work we will see that this necessary
16
condition for orthogonality in 11.10 is equivalent to the third sufficient condition for
square summability in Theorem II.1, expressed in terms of the coefficients rather
than in terms of the polynomial m0 (z). Let us start with the third condition from
Theorem II.l,
|m0(z)|2 + |m0(z + 7r)|2 = 1
for all z e R
mQ(z) = \^ckeikz.
where
2
(11.12)
(11.13)
k
First, observe
C
KZ+V)
m0(z + TT) = 1^2
= \2 E <*<?"***2^ ^
*
k
(n.14)
k
Now, if k is even, elkn = 1, and if k is odd, e%k7r — — 1 so
m0(z + 7r) = ~^2(-l)k cke
k
Recall |C|2 = CC which leads us to
|m0(z)| + |m0(z + 7r)| = m0{z)m0 (z) + m0 {z + n) m0 (z + IT).
Substituting in the expression for mo (z) from 11.13 and the expression for mo (z + ir)
from 11.14 we now get
|m0(2:)|2 + |m0(z + 7r)|2 =
\ E c^kz\ E c^lz + \ E (-1)* c^kz\ E (-1)' c^z- (IL15)
k
I
k
I
Carrying the conjugation down into the trigonometric polynomials and simplifying
we get
|m0(2;)|2 + |mo(z + 7r)| =
\Ec*eifczE^e^ + iE(-1)*c^kzE(-i)'^-"*- (IL16)
k
l
k
17
I
This sum is a Laurent polynomial in e%z. Keeping in mind that this sum must equal
one for all z e R, we see that the coefficients of (etz)n for n ^ 0 must be zero since the
sum is real-valued. We need an expression for the coefficient of (elz)n; with a little
work we can see this expression is
0n = — 2_^ CjCj-n +
T
{— 1) 2_^ cjcj~n-
0
3
Now notice, if n is odd then bn is zero automatically. If n is even then we can say
n = 2m for some m 6 Z, and
71
~~ o 2_y c-?c-?~ 2m
0
thus
y j CjCj_2m = 0
form€Z,m^0
when m = 0, bn — 1 and
E
c c
j j
Combining these conditions we get
y CjCj^2m = 25m
for all
me Z
j
as in condition 11.10. This chain of arguments can be reversed to derive 11.12 from
11.10.
So, we have shown the equivalence of the two conditions. If we look for
orthogonal solutions to the dilation equation, the coefficients necessarily satisfy the
sufficient condition for square summability.
This is all well and good, but we still need to find a sufficient condition on
the dilation equation coefficients to ensure orthogonality. There is a necessary and
sufficient condition for orthogonality of the solutions of the dilation equation. This
condition can be found in [Ref. 5], where it is expressed in terms of the location of
the roots of the trigonometric polynomial m0 (z) as defined in Theorem II. 1. This
18
condition, while it has the advantage of being both necessary and sufficient, is a little
cumbersome. We will instead present a slightly stronger condition on the placement
of roots of TOO
(Z)
which is also sufficient for orthogonality. This condition is found in
[Ref. 5] as well and relies on the necessary and sufficient condition mentioned above.
We present it here and refer the reader to [Ref. 5] for proof.
THEOREM II.3. Let <fi be the solution to the dilation equation for coefficients {c^}.
Suppose TOO (Z) = \ J^k c*t%kz satisfies \m0 (z) |2 + |TO0 (Z + ir) |2 = 1 for all z eR , and
TOO (0) = 1. // TO0 (Z) has no zeros in [— f, |], then 0 (x) is orthogonal to <f>(x - k)
for all k el, k^ 0.
Notice since we required J2k c*
=
2, we automatically satisfy m0 (0) = 1.
Also, the sufficient condition for approximation power was that m0 (z) have roots of a
given multiplicity at 7r, it is encouraging that this does not conflict with the sufficient
condition for orthogonality.
Well known examples of compactly supported, orthogonal solutions to the
dilation equation with higher approximation power than the Haar function are the
scaling functions associated with the Daubechies' Wavelets [Ref. 2]. We will describe
what a wavelet is in the next chapter.
19
20
III.
A.
THE DISCRETE ORTHOGONAL
WAVELET TRANSFORM
DECOMPOSITION OF THE FUNCTION SPACE L2
In the previous chapter we saw that we could find solutions to the dilation
equation. We also saw that these functions could be made to have a specified approximation power and be orthogonal to their integer translates. In order to develop
a transform based on these functions we must show that a basis for L2, the space of
square integrable functions defined on the real numbers, can be formed from solutions
to the dilation equation. As we have previously seen, the solutions to the dilation
equation are square integrable provided the dilation equation coefficients meet some
basic conditions. In this section we will show that we can derive from the solution
to the dilation equation a basis for L2, and that we need not impose any additional
conditions on the dilation equation coefficients to accomplish this.
Let us consider the space, call it Vo, of functions which can be expressed as
linear combinations of the solution (j) to the dilation equation and its integer translates.
Since (j) is square integrable, V0 is a subspace of L2. Let us also consider the space,
call it V-i, spanned by <j> (2x — k) where k G Z , this space is composed of functions
which are linear combinations of \/2(f) (2x) and its half integer translates. We have
scaled the function by \fl so that the norm, in L2, of the prospective basis function
will be one, this has no impact on the space spanned by the functions. Again, since (j)
is square integrable 0 (2x) is also square integrable and V_x is a subspace of L2. If we
continue in this manner, taking spaces spanned by <f>jk = 2~H2(f) {2~*x — k), we get an
infinite sequence of subspaces of L2, namely V0,V-i, Vl2,.... If we also allow positive
indices for these spaces, which implies negative powers of 2 in 2~H2(j) (2~^x — k) we
get a bi-infinite sequence of subspaces of L2
Vj = linear span {(j) (2~jx - k) \
keZ]
This sequence of subspaces has the following properties.
21
for j e Z.
1. The Vj form a nested sequence, that is Vj C Vj^\.
Suppose f £ Vj then / is a linear combination of <f>{2~^x — k). Since <f> is
a solution to the dilation equation, (j) (2~^x — k) is a linear combination of
<t> (2-tt~Vx - k), thus Vj C Vj-!.
2. / (x) e ^ if and only if / (2a:) € Vj-i.
Suppose f(x) = J2ak<f>(^jx-k), then /(2a:) = £ ak<j> (2^'(2a:) - k) =
J2otk(f) (2~ü~^x — k) thus /(2a:) e V}_x. Conversely, suppose /(2y) € T^_j
then, /(2y) = ^)M (2~ü"1)y - A;) = 5>^(2-'(2y) - A;). Now let x = 2y,
and we get / (x) = Y^ak4> (2"Jx — A;), thus / (x) <E V}.
3-nJez^ = {o}It is sufficient to show that for all / 6 Cg°, the space of infinitely differentiable,
compactly supported functions, (Pjf,f) -> 0 as j -> oo. Here Pjf is the
orthogonal projection of / onto Vj. As a consequence of 11.11 the <f)jk form an
orthogonal set. Thus, {<f>jk} constitutes an orthonormal basis for Vj, and the
orthogonal projector P_j can be expressed as
Pjf
= y
^2(<f>jk,f)^jk-
So
(Pjf, /> = ( E to*. /> &*> / ) = E I to"*, /)f
\ fc
Ik
and if we assume / has support [—a, a]
I to*, /) I2 =
f / (*) 2-J'2<j>{2-ix-k)dx
J —a
— 2
9-J
=
I / (x) 0 (2-J'x - A;)da:
«/ — a
As a consequence of Holder's inequality
j
f{x)<j>{2-Jx-k)dx<\\f\\ ( f (f){2~jx-k)(f)(2^x-k)dx\
so
2
/ f{x)(f>{2-ix-k)dx
ra
2
j \\f\\
<2^'||/|r
/
\<ß{2-jx-k)\ dx
J —a
< IIll/l!
fir2 /
/
\y+k\<2~Ja
J\v-
22
\<f>(y)\Zdy.
,
Observe that as j —> oo the integral on the right hand side goes to zero
independent of k. Also, each \((j)jk, f)f is non-negative. Combining these two
facts we see
0 < (Pjf,f) < ll/ll2^ /
\<t>{y)\2dy -»■ 0 as j -+ oo
k J\y+k\<2-ia
thus,ni6Z^ = {0}.
These properties, while very helpful, leave us somewhat short of our goal of
producing a basis for L2. The most obvious shortcoming is that we have not shown
that any collection of these functions span L2. Before we address the issue of spanning
L2, let us consider a property of <$>, the Fourier Transform of 0. As we shall see, the
following limit is of interest to us
/oo
\<S>(2-juj)\2\g(Lü)\2(Lj
■00
for arbitrary g e
0
CQ .
In [Ref. 1] we see that subject to the conditions already
imposed on {ck}, <3> is an entire function. Additionally, we have seen $ (0) = 1, and
|<fr (z)\ < 1 for z € R. We can now use Lebesgue's bounded convergence theorem
[Ref. 6] to show
/oo
■oo
/-oo
\$(2-ju>)\2\g(üj)\2dü>= /
\g(co)\2cLu.
(ULI)
J-oo
Now, let us get back to trying to span L2 with some collection of (f>jk- What
we wish to show is Üjez K?'
=
^■ Since the Vj are nested, it will be sufficient to show
that (P-jf, f) —> (/, /) as j —t oo for all / e I? such that /, the Fourier Transform
of /, is in
CQ°.
Let / 6 I? such that / = g 6 C£°. As we have seen before the
orthogonal projector P_j can be expressed as
fcez
Again we see
feez
23
Recalling that the Fourier Transform we are using has the normalization constant
in the inverse transform, inner products in the function space are related to inner
products in the transform space by (<f>, ip) = 2% ($, W). By standard operational rules
for Fourier Transforms we can see $-jk, the Fourier Transform of (f>-jk as defined
above is given by
All of this now leads us to
/oo
g(w)2-J/*e2->ki»$(2-3üj)du.
-oo
Simplifying where possible and making the change of variable z =
J
2~ UJ
we get
/oo
g {2jz) $ (z) e-kizdz.
•oo
Recall that g has compact support, so for sufficiently large j, g (2jz) is nonzero only
in [—7r,7r] and we can simplify the integral to
(/, <f>-jk) =
/2
2TT2->
/ g (2jz) $ (z) e~klzdz.
(III.2)
J —IT
Now, the right hand side of III.2 is just the kth Fourier coefficient of the function
2-^22Trg(2Jz)^(z). The quantity we are interested in, namely (P-jf,f), is given
by the sum over k of the squares of the magnitudes of the individual inner products
given in III.2. Since each of these individual inner products is a Fourier coefficient
for a specific function, we can use Parseval's Equality to arrive at
(P-if, f) = J2
27v2 j/2
~
f 9 (Vz) <& (z) e~kizdz
(III.3)
=
2 j/ 2n
2
h I \ ~ * 9 {Vz) $ (z)\ dz.
Collecting terms and simplifying where possible we get
(P-jf, /) =
2TT2-^
^ \g (23z) $ (z) |2 dz.
J —IT
24
If we now revert to the original variables, that is, let 2^z = u> we get
/2J7T
/-00
2
\g(u)^(2^u)\ du = 2'K /
-2->7T
\g (u)\2 |<S> (2"-?a;)|2 dw.
./-OO
We have seen in ULI that the right hand integral above converges to
/oo
|g(ü;)| du.
oo
Combining all we have done we get
/oo
\g (u) |2 du = 2*{g,g) = </>/>•
•oo
Which shows U,ez ^J
=
^2
So far, we have shown the existence of a sequence of orthonormal sets of
functions the limit of which is dense in L2. It might appear that we are nearing the
end of our search for basis functions on which to base our wavelet transform. What
we have is a good foundation for an approximation scheme, however this scheme
falls short of satisfying all the conditions we need for our basis functions. Namely,
projecting onto Vj for fixed j doesn't provide any frequency information. If we attempt
to get frequency information by projecting onto other subspaces Vn for various n ^ j,
we no longer have a orthonormal system, since <j>jk is not orthogonal to 4>nm for
j 7^ n as either Vj <Z Vn ox Vn <Z Vj. This leads us to introduce the detail space
Wj associated with Vj, which we will call the scaling space. Formally, Wj is the
orthogonal complement of Vj in V^_i. This allows us to express Vj_i as the direct
sum of Vj and Wj. We can express this graphically as
Vj -> vj+1 -> vj+2 -+vj+3^>--\
\
\
\
wj+l wj+2 wj+3
Consider the sequence of spaces Wj, since each is the orthogonal complement of
its companion scaling space Vj and Wj+i is a subspace of Vj, it is clear that Wj
25
is orthogonal to Wj+1. Additionally, we saw earlier that f]jezvj
Ujez Wj = ^2- Since
tne w s are
/
=
{°K
this im
PÜes
all orthogonal, we have a direct sum decomposition
of L2 namely,
This certainly is helpful, as all we need to do now is find an orthogonal basis for each
Wj. Taking the union of all these basis functions will yield an orthogonal basis for
L2.
This may seem to be nearly as daunting a task as we initially faced, but we
can now take advantage of the highly structured nature of the Vj spaces and the
orthogonality of Wj to the Vj. First, observe that since Wj C V}_i, perhaps we can
use the orthogonal basis for V}^ to construct an orthogonal basis for Wj. Let us
proceed along these lines. Any prospective basis function, </?, for W0, must be an
element of W0 and since W0 C V-i we must have the following representation for <p
V
= Y,M{2x-l)
(III.4)
Such functions tp are the wavelets which we have alluded to throughout this paper.
Since we are interested in compactly supported basis functions we assume {dt} to be
finite. We also require cp to be orthogonal to <f> and all its integer translates, since
W0 is the orthogonal complement of VJ,. To aid us in our investigation we use the
following representation of the Poisson Summation Formula, a derivation of which is
found in the appendix,
J2 (<P (x) ,<t>(x- I)) e~iajl = J2^(co +
i
2/OT)
$ (u + 2kir)
for k,l£Z.
(III.5)
k
Observe that this formula establishes the equivalence of the following statements:
1. (p(x) is orthogonal to 4>(x - I) for I € Z.
2- Efcez * (w +
2&TT)
$ (u + 2kn) = 0 for all wGi
26
We can see this since if 1 is true then the sum on the left hand side of III.5 is
clearly zero for all
UJ
e K. Conversely, if 2 is true then the left hand side of III.5 is
a Laurent series in e"" which is zero at all values of a;, and must be identically zero,
therefore each of the inner products in the sum must be zero.
Now, let us apply this to the problem of finding a basis for Wo- Suppose <p (x)
is orthogonal to <p (x - k) for k e Z, this is equivalent to
J2 ^ (w + 2k7T) $ (w + 2kn) = 0
for all wel.
(III.6)
k
We can use III.4 to get an expression for *, the Fourier Transform of ip
* (o = /~ E w (2x - o e~ixidx = \ E d^^ (|) •
As we have seen before, the Fourier Transform, $, of <f> can be expressed
Consider the function
ne)=;Ei*(£+2A;7r)i2A;
We can write this function in the following way,
F (0 = ]T $ (f + 2^7r)$(e + 2^7r).
fc
Now, by III.5 F is equal to £, (<f> (x), <ß (x - I)) = 1 because ^ form an orthonormal
basis for V0. So, F (f) = 1 for all £ e R.
Define the following trigonometric polynomials
D(t) = ±£>2
ce n
ceo = ^E
» ~*
2
n
27
Substituting into III.6 we get
0 = 2^ * (a; + 2k7r) $ (u + 2/br)
+ 2&?r\
E"(^)*(^)c(^)*(^2-y.
(III.7)
Observe when k is even £> (^+|^) = D(fj, and when Jfc is odd D (^fz) =
D
{l + 7r)-
Similarly, when & is even C (^fE) = C(f), and when k is odd
C (a,+22fc7r) = C (| + 7r). We can rewrite the sum as
E
D
ÜJ
r
Mi
$
u + 2(2k)TTs
+ D(i-MiH
$
a; + 2 (2k + 1)
TT
(III.8)
Factoring out the terms which are independent of k we get
"(!M!)£K!+^)
+ £ (! + 7r) C (! + 7r) £ |$ (! + TT + 2*7r) |2 .
(IH.9)
The summations above are simply F (|) and F (f + 7r) so we can write
Y^ * (w + 2^TT) $ (w +
2A;TT)
=
But F (f) = F (f + TT) = 1, so we have
(III.ll)
To simplify our notation we will use u in place of f. Since 4>ok form an orthonormal set,
C (ui) and C (u + n) cannot simultaneously be zero; this is a consequence of the third
condition of Theorem II. 1. If III.ll is to be true then when C (u) is zero, C [u + TT)
28
cannot be zero. Therefore, D (u + TT) must be zero. Similarly, C (w + TT) and D (u)
must have the same zero structure. Since C (u>) is a trigonometric polynomial, D (a>)
must have the following form
D(U) = A(Cü)C(U) + TT).
Where A (to) is 27r-periodic, and A(u) + -n) = —A(u). Further, since we desire (p
to be compactly supported, A (cu) must be a trigonometric polynomial. Now, III.11
becomes
0 = A (u) C (u + 7r)C (U) + A (u + IT) C (LO + TX + TT)C (u + ir)
= A(u)C(cü + 7r)C(uj)-A(u))C(cv)C(u> + Tr)
since C (u>) is 27r-periodic. Thus we have shown, subject to the above conditions on
A(u), functions generated by III.4 lie in the orthogonal complement of Vo in V-\.
What remains to be shown is that these functions ip are orthogonal to their integer
translates.
To show this we will again use the Poisson Summation Formula III.5. We see
that orthogonality of cp (x) to tp (x — I) for I e Z is equivalent to
1 = Y2 * (w + 2^7r)^(o; + 2A;7r).
k
Substituting in the expression for the Fourier Transform of <p and simplifying in a
manner similar to that used above we get
l = ^2^(u + 2kir) tt (u + 2krr) = D (u) D(co) + D (u + n) D {u + TT).
Since D (co) = A (u) C (u + TT), we have
l = A(u)C(u + TT)A (U)C (U + TT) + A (u + TT) C
(U)A (U
+ TT)C (W)
Reorganizing slightly and recalling that A (co + TT) = -A (u>) we find
l = A{u)A(u)C(w + TT)C(U + TT) + A(w) A{u)C (u)C(w).
29
(111.12)
Since <j> (x) is orthogonal to cf)(x ~ I), we know C [to +
IT)
C (u> + n) + C (u) C (ui) = 1.
If we additionally require \A (u)\ = 1, then III. 12 will be satisfied, and <p (x) will be
orthogonal to <p (x — I).
Let us collect the conditions we have imposed on A(u) and see what type
function they characterize.
1. A (u) is a trigonometric polynomial, and 27r-periodic.
2. A(u + 7r) = -A{u).
Z.\A{u)\ = l.
Condition 3 implies the graph of A (u) traces out the unit circle or, in light
of condition 1, A(w) = a0e~imw for |a0| = 1 and some m. Condition 2 implies m
must be odd. We have a great deal of freedom in our selection of a0, but since
we have so far only considered real valued {dt}, we will limit ourselves to a0 = ±1
and for this development we will assume a0 = 1. We will let m be the highest
index of the set {ck} 1, that is, if {ck} has four elements we will set m = 3, because
{ck} = {Co,Ci,C2,C3}.
Having made these choices we see
_l
ill
-y
2f^
D(co)
Cke-ik{u+Tt)
_
Or, after simplifying
1
m
fc=0
which we can express as
-ilw
D(u
1=0
Condition 11.11 implies the highest index of {ck} must be odd. Further any odd m gives rise to
a translated version of this same (p.
30
where dt = {-l)m~lc^Zi. That is {dt} = {-c^,c~i, -c^IJ, • • • , ci, -c{,c^}. Recall
this {di} is such that <p (x) = ^ di4> {%x ~ 0
an orthonormal set in
an
d its integer translates <p(x — k) form
WQ.
It is not obvious that this set spans W0. We can see that it does by employing
an argument similar to that above. Suppose g (x) € Wo, then g (x) G V-\ and
g (x) ±V0. We can express g (x) as
g(x) = J2ji4>(2x-l).
i
This, in turn gives us G (£), the Fourier Transform of g (x)
C (0 = 5 !>-**(§)•
Define the Laurent series in e~^, V (£) by
r(0 = ££>-*.
z
Since g (x)
JLVQ,
I
we can obtain the following relationship between C (£) and V (£)
r
+r
+
'- (£M£) (Hni Now, by exploiting the zero structure of C (£) we can see T (£) must have the following
form
r(£) = ß(e)c(e+7r),
where B (£) satisfies the following:
1. B (£) is a Laurent series in e~1^.
2. B (£) is 27T-periodic.
3.
B(£ + TT)
= -£(£).
Conditions 2 and 3 above imply B (£) consists solely of odd powers of e~^.
Each of these odd powers in turn corresponds to a translation of the function ip. So,
31
any function g (x) € W0 can be represented by a weighted sum ofip(x-k). Thus we
have shown ip (x — k) span the space W0.
To summarize, so far we have a sequence of spaces Wj the direct sum of which
is L2. We have demonstrated a basis {ip (x - k)} for W0 which is constructed from
functions (f) (2x - I) which form a basis for V-X. Now we need to show an orthonormal
basis for Wj is given by {<pjk = 2~^2(p (2~^x - k)}. To do this we use III.5. From
standard Fourier Transform operations we can see
*ifc(f) = 2^V2^W (2J'e)
^(f) = 2^-***$ (2>'f).
It is easily verified using arguments similar to those above that ipjk is orthogonal to
<f>jk. It is equally easy to verify ((pjk,(pjn) = 6n:k. Thus for each scaling level j we
have an orthonormal set {(pjk}, which lies in Wj. All that remains to show is that
each set spans the respective space Wj. This is accomplished in a manner completely
similar to that used for W0. Finally, we have an orthonormal basis for L2, namely,
{<pjk = 2~i/2ip (2-ix - k)} for j, keZ.
B.
ALGORITHM DEVELOPMENT
Armed with this basis for L2 we now set out to implement a discrete transfor-
mation which, as promised in the introduction, has good time-frequency localization,
suitable power of approximation, and can be computed quickly. Time localization is
a result of the compact support of the scaling functions and wavelets, and the fact
that the support of these functions shrinks as the index of the subspaces V,- and
Wj decrease (toward -co). Frequency localization is accomplished by dividing the
spectrum of the input function into "octaves" each of which is represented by the
projection of input function onto some Wj. This is not as sharp as the frequency
localization of the Fourier Transform, but it is still quite good, and can be computed
rapidly. The entire derivation of the basis functions was driven by the desire to take
advantage of a recursive relation between basis functions. We do not have a recursive
32
relation between the actual basis functions <pjk, however, we do have a linear relation
between the basis functions and a set of auxiliary functions <f)jk, which we call the
scaling functions
<p{x) = '%2dl<l>(2x-l).
(111.13)
i
This would not be of much use without the associated recursive relation between the
scaling functions at subsequent levels of scaling
<t> (x) = 53 ck(f> (2x - k).
(111.14)
k
We will exploit these relations to achieve a computational efficiency of order N operations. In computing a transform we are concerned with projections. Since projection
operators are linear, we can recast 111.15 and III.16 in terms of the projection of an
arbitrary function / (x) on <j>ik and (pjk. Specifically,
(/ (:r), <f> (x)> = 53 ck(f (x) ,<f>(2x- k))
(111.15)
k
and
(/ (x), <p (x)) = 53 dx (f (x), 0 (2x - k)) .
(111.16)
i
Put another way, these relationships allow us to compute the projection of / (x) onto
Wj given the projection of / (x) onto Vj-\.
Before we can develop an algorithm, we need to recognize that the theory we
have developed so far defines a transform which operates on functions in L2, square
summable functions defined on the real numbers. Further, the transform projects
the input function onto a bi-infinite sequence of subspaces of L2. In practice we are
given a sequence of uniformly sampled values of the input function instead of the
actual function. We will, for the purposes of this development, assume the input
function / is zero at its boundaries. By doing this we can "pad" the function with
zeros and apply the transform as if the function were defined over the entire real
33
line. There are methods of treating input functions which are not zero at their
boundaries, but these methods are beyond the scope of this development. Next we
must address the question of where in the bi-infinite sequence of subspaces to start
the transform. The sampled values of / are a piecewise constant approximation to
the actual input function2. We will define the fine scaling unit to be the "distance"
(difference in independent variable) between successive samples. A consequence of the
orthogonality of the scaling functions is that they must have approximation power at
least one. This is easily seen since \m0 (z)\ + \m0 (z + n)\ =1 for all z 6 R, and
mo (0) = 1, m0 (n) = 0. Put another way, the polynomial with coefficients {ck} has a
root at 7T, which tells us the scaling function associated with {ck} has approximation
power at least one. How do we use this fact? Over the region where / is represented
by a constant, the scaling function and its translates can exactly represent /. Thus
the projection of / onto the wavelets, (pjk, at this scaling level is zero. Also, at any
finer scaling level the projection of / onto the wavelets must be zero. In light of
these facts we will start the algorithm at scaling level I, where the scaling function
2~l/2(fr (2lx) has support width of one fine scaling unit. We call this the finest scaling
level.
We now outline the algorithm to compute the discrete, orthogonal wavelet
transform. Figure 3 depicts the arrangement of wavelet and scaling functions. The
first step in the algorithm is to calculate the projection of the input function onto the
scaling functions at the finest scaling level. There are many ways to accomplish this,
one way is presented in the appendix. Once we have these projections, the algorithm
is simply a matter of using III.15 and III.16 to reorganize the data into the projections
onto Vi+i and Wi+i. This is accomplished by taking weighted sums of the projections
onto Vi. To calculate projections onto Vi+i, we use as weights {ck}. To calculate the
2
Other approximations may be used, for instance, if the input function is represented in a Bspline basis of order two, we would have a piecewise quadratic approximation to the input function.
Similar results for higher order approximations can be developed in a manner similar to that used
for piecewise constant approximation.
34
projections onto Wi+\, we use as weights {dx}. We repeat this process until we have
calculated the projections onto all of the subspaces Vj and Wj, for I < j < m. Here m
is the index such that the scaling function 2_m/2^> (2mx) has width of support equal
to that of /. The discrete, orthogonal wavelet transform of the function / is given by
the following collection of projections
W (/) = {(/. <Pik) \l<j<m}ö {</, <j>mk)} .
The following is a pseudocode version of what we have just described.
j-l
j-2
j=-3 (finest scaling level)
weighted sum of
fine scaling
functions produces
coarse wavelet or
scaling function
support of scaling functions at finest scaling level
support of f(x)
Figure 3. Organization of Basis Function
35
input:
1. n + 1 dilation equation coefficients for the wavelet we wish to use
2. sequence of TV values which represent the projection of / onto <f>ik at the
finest scaling level. iV = n2p~1 + 1 for some p.
begin algorithm:
for j — I — m — 1 : 0
for k = 0 : n2^~1 - 3
(/, <Pjk) = \/2 ES/T ci-2k (/, <f>j-lj)
(f, Wh) = V2 Yditk c^2/= (/, 0j-i,j>
end
end
output :W(/) = {(/,^fc) | / < j < O}u{(/,0ofc)}
This brings up another aspect of implementing the wavelet transform on a
computer. The sequence of subspaces we project onto is bi-infinite. We saw earlier
that we could pick a starting index based on the resolution with which / was represented. The algorithm to compute the transform must stop at some index. What
should this index be, and what are the consequences of stopping the algorithm here?
In this development we will stop the transform at the index where the scaling function
has support width equal to the width of the input function sample. At each step of the
transform we are calculating the projections of / onto succeedingly lower frequency
basis functions as the support width of cpjk increases. Terminating the algorithm at
any given index establishes a low frequency bound below which we have no frequency
localization.
This section has presented a very basic outline of the discrete, orthogonal
wavelet transform algorithm. There are many choices as to the indices on which the
algorithm should begin and terminate. The major concept of the algorithm is the use
of the relationships III.15 and III.16 to rapidly compute projections at one level from
those at a preceding level. It is precisely these relationships which allow the wavelet
transform to be computed in order iV operations as we shall now see.
36
To start our study of the computational efficiency of the algorithm we will
assume the input function is given as N sampled values. We further assume that
iV = 2pn +1 for some p where n is the highest index of {ck}. We divide the algorithm
into the following blocks.
1. Calculation of projections at finest scaling level: Nearly any quadrature rule
can be used to calculate these projections in O (N) operations.
2. Calculation of wavelet coefficients: At each level j of the p levels of the algorithm we must calculate n2J — 2 wavelet coefficients, each of these coefficients
is calculated by taking a weighted sum of projections onto the scaling coefficients at the preceding level. The computational cost for the weighted sum is
n + 1 multiplies and n additions, this makes the total cost for this block
v-i
J2(n2j -2)(2n + l).
j=o
3. Calculation of scaling function coefficients : At each level we must calculate
the projections of the function onto the scaling functions. This is accomplished
by taking a weighted sum of projections onto the scaling functions at the
preceding level. The computational cost is identical to that for calculating the
wavelet coefficients above.
If we sum the computational cost for each of these blocks we get a total
algorithmic cost of
p-i
O (N) + 2 ]T(n2J' - 2)(2n + 1).
3=0
The above sum is dominated by 2 (2n + 1) 2P which is O (N). So, we have developed
a transform which provides good time-frequency localization, can be used as an approximation scheme, and is rapidly computed. Some applications and directions for
further studies are discussed in the next chapter.
37
38
IV.
CONCLUSIONS AND DIRECTIONS
FOR FURTHER STUDY
We have shown that we can develop a transform based on solutions to the
dilation equation. This transform is useful because we can alter the characteristics
of this transform simply by changing the dilation equation coefficients. Also, we
can compute this transform quickly. In fact, the order JV operations required for
the wavelet transform is considerably better than the N log (JV) operations required
for the fast Fourier Transform for large JV. The wavelet transform has the added
advantage of time localization. A major advantage of the Fourier Transform is that
the basis functions are eigenfunctions for an operator of fundamental importance,
namely the Laplacian operator. The wavelet transform, on the other hand, has been
restricted predominantly to use in the signal processing field. In this field the wavelets
time-frequency localization and rapid computation are great assets, and projection
onto a basis of eigenfunctions is not always as desirable as these other properties.
To be as useful as the Fourier Transform, wavelets must be "well behaved"
under some operator of interest. By well behaved we mean the image of the wavelet
under the operator is simply related to the wavelet or its translations and dilations.
Eigenfunctions for an operator are the ultimate in well behaved functions. It is
clear from our preliminary work in Chapter II that convolution and convolution type
operations are the natural operations for wavelets. If we are to find an operator
under which wavelets would be well behaved, it is natural to first consider operators
based on convolution like processes. A broad field of such operators is the ZygmundCalderon group of operators. A description of these operators is beyond the scope of
this work, but this is a burgeoning area for wavelet analysis.
An area which has great potential is that of tailoring a wavelet to be well
behaved under a particular operator. To see how we might tailor a wavelet we should
consider that if we wish our wavelet to have support of width n at the zeroth scaling
39
level, {cfc} must have n + 1 elements, namely
CQ,
... , cn. We still desire the wavelets to
be orthogonal, so we must meet the necessary conditions of section 1.3. Specifically,
YLk ckCk-2m = 25m for m 6 Z. Each of these conditions can be represented as a simple
bi-linear form
c*D2mc = 25m,
where * indicates conjugate transposition and D2m is the 2m down shift matrix. That
is, D2m has ones on its 2mth diagonal and zeros elsewhere. If these conditions were
linear then we could easily determine the solution space for this system. However,
these conditions are not linear and systems of bi-linear forms are not well studied.
We can see for m = 0 the condition is J^k \ck?
=
2- We can also see for each positive
value of m the condition is equivalent to the corresponding condition for —m, since
c*D2mc = 0 = 0* = c*D2m(c*)* = c*D_2mc. So, we have reduced the number of
conditions from n to n^. To make further progress we need to be able to determine
the solution space of a system of bi-linear forms. The use of the term space here refers
to the set of all solutions, not necessarily a linear space. The solutions of a system of
bi-linear forms need not be linear. Once we have the solutions to this system we would
then determine which of these solutions could be written as linear combinations of the
vectors {[v£]} as defined in section 1.2 where p is our desired approximation power.
This would satisfy the sufficient condition for approximation power. Now we check
these solutions against the sufficient condition for orthogonality presented in section
1.3. The result would be a collection of dilation equation coefficients which would
yield orthogonal wavelets with the desired approximation power. Perhaps among this
collection we would find dilation equation coefficients which would be well behaved
under the operator of interest.
40
REFERENCES
[1] Volkmer, H., Distributional and Square Summable Solutions of Dilation Equations
Preprint
[2] Daubechies, I., Ten Lectures on Wavelets, SIAM Philadelphia PA 1992
[3] Strang G. and Fix, G. A Fourier Analysis of the Finite Element Variational
Method, Centro Intemazionale Matematico Estivo, Constructive Aspects of Functional Analysis, Edizioni Cremonese, Roma 1973.
[4] Schoenberg, I.J., Cardinal Spline Interpolation SIAM Philadelphia PA, 1973
[5] Cohen, A., Ondelettes, Analyses Multiresolutions et Filtres Miroir en Quadrature,
Ann. Inst. H. Poincare, Anal. Non Lineaire, 7, pp. 439-459, 1990
[6] Royden, H.L., Real Analysis, Macmillian Publishing Company, New York NY,
1988
41
42
APPENDIX A. DERIVATION OF POISSON
SUMMATION FORMULA
We start with the Poisson Summation Formula as found in any text on Fourier
Analysis
^/(n) = ^F(2fc7r),
nez
fcez
where F (£) is the Fourier Transform of / (x). Consider the function
f(cjJ) = (iP(x),<l>(x-l))e-i»1.
We see the Fourier Transform, in the variable Z, of / is given by
/oo
y,(x),<fi(x-i)) e
iwl „—i;
e *'<fl.
-oo
Expanding the inner product and simplifying the exponential we get
/oo
poo
i){x)(j){x- l)dx e-il(-w+tidl.
/
oo J — oo
Now, change the order of integration, and we see
/oo
poo
cj){x- I) e-a{u+®dl dx.
V> (x) /
■oo
J —oo
Make the change of variables y = x — I, and we now have
/oo
pOO
ix{ui+
V> (x) e~
® /
-oo
<j> (y) eiy{-w+t]dy dx.
J —oo
We can write the integral in y above as
/oo
(j){y)eiy^+üdy = ${Lü + Z).
-oo
This leaves us with
/oo
</> (x) e-ix{w+Ö<S>
-oo
43
(UJ
+ £)dx.
Since $ (u + £) has no x dependence, we can write this as
/oo
$ (x) e~ix(w+Üdx$(cj + 0-oo
The integral in x above is simply the \t (a; + f) which leads us to
Now, applying the Poisson Summation Formula we get
J2 <V> (x) ,<j>{x- 0) e-*"' = ]T * (u +
Jez
fcez
2A;TT)
$(a; + 2Ä;7r),
which is the desired representation of the Poisson Summation Formula.
44
APPENDIX B. PROJECTIONS AT FINEST
SCALING LEVEL
To see how we might accomplish this, consider the following diagram.
f(x)
aligned with constant f(x)
special treatment
} requires
aligned with constant f00
support width of scaling functions at finest scaling level
Figure 4. Scaling Functions at Finest Scaling Level
Each projection at the finest scaling level is the projection of (f>ik onto either
a constant or piecewise constant function. Where / is constant over the support of
(f>ik the projection is simply a weighted integral of (ßik. Where / is piecewise constant
over the support of 4>Lk we can divide the domain of integration so that the projection
is given by a sum of integrals over domains where / is constant. Each of these subintegrals is given by a weighted integral of <j>lk over a portion of its support. For
instance, if we are computing the projection of / onto the second scaling function in
Figure 4, we would have the following
/ / 0*0 02 {x) dx = a
JO
<j)2(x)dx + a
JO
<j)2 (x) dx + b
Ji
cj>2 (x) dx.
J2
Similarly, the projection onto the third scaling function would be
/ / {x) <f>3 {x) dx = a
(f>3(x)dx + b
(j>z (x) dx + b
(j>3 (x) dx.
Jo
Jo
J\
J2
45
The sub-integrals, fQ (j)3 (x) dx, can be computed once and stored for use since they
depend only on the wavelet used. We will assume the projection of the input function
onto Wi is zero. This prevents us from incorporating into the transform the discontinuities introduced by the piecewise constant approximation of /. It is worth noting
that at this point in the algorithm we have all the information we ever will about the
input function. In fact, we do not use the function for the rest of the algorithm. The
remainder of the algorithm is devoted to rearranging the projections of / onto the
scaling functions at the finest level into projections of / onto the spaces Wj for j > I.
46
INITIAL DISTRIBUTION LIST
1. Defense Technical Information Center
Cameron Station
Alexandria, VA 22304-6145
2. Library, Code 52
Naval Postgraduate School
Monterey, CA 93943-5101
3. Professor Carlos F. Borges, MA/Bc
Department of Mathematics
Naval Postgraduate School
Monterey, CA 93943-5000
4. Professor Richard Franke, MA/Fe
Department of Mathematics
Naval Postgraduate School
Monterey, CA 93943-5000
5. Professor Van Emden Henson, MA/Hv
Department of Mathematics
Naval Postgraduate School
Monterey, CA 93943-5000
6. LT James K. Logue
Submarine Officer's Advanced Course
Submarine School New London
Groton, CT 06340
7. Dr. Lou Cleveland
Department of Mathematics
3094 Indian Circle
Marianna, FL 32446
8. Professor Jack Rogers
Department of Mathematics
228 Parker Hall
Auburn University
Auburn, AL 36849
47
9. Professor William Transue
Department of Mathematics
228 Parker Hall
Auburn University
Auburn, AL 36849
48
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement