INFORMATION TO USERS

INFORMATION TO USERS
INFORMATION TO USERS
This manuscript has been reproduced from the microfilm master. UMI
films the text directly from the original or copy submitted. Thus, some
thesis and dissertation copies are in typewriter face, while others may
be from any type of computer printer.
The quality of this reproduction is dependent upon the quality of the
copy submitted. Broken or indistinct print, colored or poor quality
illustrations and photographs, print bleedtbrough, substandard margins,
and improper alignment can adversely affect reproduction.
In the unlikely event that the author did not send UMI a complete
manuscript and there are missing pages, these will be noted. Also, if
unauthorized copyright material had to be removed, a note will indicate
the deletion.
Oversize materials (e.g., maps, drawings, charts) are reproduced by
sectioning the original, beginning at the upper left-hand comer and
continuing from left to right in equal sections with small overlaps. Each
original is also photographed in one exposure and is included in
reduced form at the back of the book.
Photographs included in the original manuscript have been reproduced
xerographically in this copy. Higher quality 6" x 9" black and white
photographic prints are available for any photographs or illustrations
appearing in this copy for an additional charge. Contact UMI directly
to order.
U·M·I
University Microfilms International
A Bell & Howell Information Company
300 North Zeeb Road. Ann Arbor. M148106-1346 USA
313/761-4700
800:521-0600
Order Number 930'1685
Ne't'-.: methods of non-linear
imag~
restoration and reconstruction
Oh, Choonsuck, Ph.D.
The University of Arizona, 1992
U·M·I
300 N. Zeeb Rd.
Ann Arbor. MI 48106
NEW METHODS OF NON-LINEAR IMAGE
RESTORATION AND RECONSTRUCTION
by
Choonsuck Oh
A Dissertation Submitted to the Faculty of the
DEPARTMENT OF ELECTRICAL AND COMPUTER ENGINEERING
In Partial Fulfillment of the Requirements
For the Degree of
DOCTOR OF PHILOSOPHY
WITH A MAJOR IN ELECTRICAL ENGINEERING
In the Graduate College
THE UNIVERSITY OF ARIZONA
1 992
2
THE UNIVERSITY OF ARIZONA
GRADUATE COLLEGE
As members of the Final Examination Committee. we certify that we have
read the dissertation prepared by____~Ch~o~o~n~s~u~c~k__~O_h_______________________
entitled NEW METHODS OF NON-LINEAR IMAGE RESTORATION AND RECONSTRUCTION
and recommend that it be accepted as fulfilling the dissertation
requirement for the Degree of
Doctor of Philosophy
------------------------------------10/2/92
Date
10/2/92
Date
10/2/92
Dr. Robin N. Strickland
Date
Date
Date
Final approval and acceptance of this dissertation is contingent upon
the candidate's submission of the final copy of the dissertation to the
Graduate College.
I hereby certify that I have read this dissertation prepared under my
direction and recommend that it be accepted as fulfilling the dissertation
requirement.
10(2(92
Dissertation Director
Dr. B. Roy Frieden
Date
10/2/92
Co-Director
Date
3
STATEMENT BY AUTHOR
This dissertation has been submitted in partial fulfillment of requirements for
an advanced degree at The University of Arizona and is deposited in the University
Library to be made available to borrowers under rules of the library.
Brief quotations from this dissertation are allowable without special permission,
provided that accurate acknowledgment of source is made. Requests for permission
for extended quotation from or reproduction of this manuscript in whole or in
part may be granted by the head of the major department or the Dean of the
Graduate College when in his or her judgment the proposed use of the material is
in the interests of scholarship. In all other instances, however, permission must be
obtained from the author.
SIGNED:
~k
4
To my wife and daughters
5
ACKNOWLEDGMENTS
Many individuals have influenced this work. However, in greatest part, the ideas
developed here were sparked by discussions with my dissertation advisor, Professor
B. Roy Frieden. I would like to express my gratitude for his invaluable guidance,
encouragement, and support. I would also like to thank my co-advisor Professor
Robert A. Schowengerdt for his suggestions and his generous permission to use the
computer facilities in Digital Image Analysis Laboratory (DIAL). I wish to thank
Professor Robin N. Strickland for his help in completing my education and my
degree requirements. I appreciate all their time and efforts.
I gratefully thank the present and previous head, and the directors at the Electronics and Telecommunications Research Institute(ETRI) for their financial support and encouragement.
Thanks to my friends Hsien-huang Wu and Dr. Raymond White for invaluable
discussions and computer code development.
My sincere appreciation goes to my dear wife, Youngsook. Without her assistance, patience, and understanding, I would not have been able to pursue my career.
My two daughters, Haejin and Haeyoung, have supported my efforts with forbearance and goodwill. Finally, I would like to express my appreciation to my parents
and my wife's parents for their emotional support.
This research was supported by the Strategic Defense Initiative through the Office of Naval Research, under grant NOaa 14-90-J-404 7, as part of the Unconventional
Imaging Program. I thank F. QueUe and W. Miceli for their encouragement and
support.
6
TABLE OF
COr~TENTS
LIST OF FIGURES .
8
LIST OF TABLES.
10
ABSTRACT . ...
11
1. INTRODUCTION
1.1. Invariance . . . .
1.2. Phase Retrieval.
13
13
17
2. TRANSFORMS FOR GEOMETRICAL INVARIANCE: REVIEW
21
2.1.
2.2.
2.3.
2.4.
2.5.
2.6.
Introduction
Conformal :Mapping
Translation Invariance of Fourier Transform
:\lellin Transform .
Fast Mellin Transform
Fourier-Mellin Transform
2.7. Direct Mellin Transform .
3. INTEGRAL LOGARITHMIC TRANSFORM(ILT)
3.1. Introduction
3.2. Derivation
3.3. Scale-invariant Property
3.4. Rotation-invariant Property .
3.5. Inversion
:3.6. Optical Implementation
:3.1. .-\pplications
21
22
23
2·5
27
28
30
33
33
36
37
39
41
42
44
4. A lV1ULTIPLE-FILTER APPROACH TO PHASE RETRIEVAL
FROM MODULUS DATA. . . . . . . . . . . . . .
58
4.1. Background. . . . . . . . . . . . . . . . . . .
·t2. Wiener Phase-Filter Approaches: Derivation.
·t3. Wiener Phase-Filter Approach: Examples . .
.58
60
65
7
4.4. Discussion
............................. .
68
5. PHASE RECONSTRUCTION OF TURBULENT IMAGES .
79
79
5.1.
5.2.
5.3.
5.4.
5.5.
5.6.
5.7.
Introduction . . . . . . . . .
Image Formation Theory ..
Basic Speckle Interferometry
Shift-and-Add Method ..
Knox-Thompson Method ..
. . . . .
. . . . .
Bispectrum Method .. . . .
Turbulent Image Reconstruction from a Superposition Model
5.7.1. Superposition Process
5.7.2. Theory . . . . . . . .
5.7.3. Experimental Results
5.S. Discussion . . . . . . . . . .
6. CONCLUSIONS AND FUTURE RESEARCH
81
83
85
86
88
90
90
92
95
99
104
APPENDIX A. LEVENBERG-MARQUARDT OPTIMIZATION. 108
REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
8
LIST OF FIGURES
:3.1.
:3.2.
3.3.
3.4.
3.5.
3.6.
3.7.
:3.8.
Optical implementation of the integral logarithmic transform.
Formation of the logarithmic smoothing filter. . . . . . . . . .
Flow of operations in the data smoothing experiment. . . . .
Input images with additive Gaussian noise. (a) 64 x 64 images, (b)
128 x 128 scaled version image with the same noise statistics as
in (a). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ..
Scale invariant filtering study, using LT (secondary) image space.
(a) The output by scale-invariant, \Viener filtering of Fig. 3.4
(a). (b) The output by Wiener filtering of Fig. 3.4 (b) using
the same filter as in (a). . . . . . . . . . . . . . . . . . . . . .
Scale variant filtering study, using primary (direct) image space. (a)
The output by \Viener filtering of Fig. 3.4 (a) using the filter
derived at a different scale, that of Fig. 3.4 (b). (b) The output
by Wiener filtering of Fig. 3.4 (b) using the filter derived at the
same scale, that of Fig. 3.4 (b). . . . . . . . . . . . . . . . ..
Two input images to apply the integral logarithmic transform. . ..
Outputs of the two-dimensional integral logarithmic transform. (a)
Logarithmic transform of Fig. 3. 7( a), (b) Logarithmic transform of Fig. 3.7 (b).
52
53
54
55
55
.56
57
57
4.1. Object training set. . . .
4.2. The nine modulus filters. (The origin is the upper left corner, and
the folding frequency is the center, of each frame.) . . . . . ..
4.3. The nine phase filters. (The origin is the upper left corner, and the
folding frequency is the center, of each frame.)
4.4. ;:\ine-filter outputs (no noise). . .
4.5. Nine-filter outputs (10% noise). . .
4.6. ::,7ine-filter outputs (20% noise). . .
4.7. Output, with 10% noise in filters. .
4.8. Output, from data with 10% obscuration.
4.9. Output, from data with 50% obscuration.
70
72
73
74
75
76
77
78
·5.1. Formation of point spread function s(x) from lens suffering phase
errors b.(f3). . . . . . . . . . . . . . . . . . . . . . . . . . . ..
101
71
9
Short-term images formed (to be processed). (a) Ideal one-point
object; (b) Ideal two-point object; (c) Turbulent image (psf) of
(a); (d) Another image (psf) of (a); (e) Image of (b) via psf (c);
(f) Image of (b) via psf (d). . . . . . . . . . . . . . . . . . . . 102
5.3. Outputs of image modelling algorithm. (a) Output o(x,y) of algorithm based upon data F.T.{Fig. 5.2(c)} JF.T.{Fig. 5.2( d)}.
(b) Output o(x, y) based upon data F.T.{Fig. 5.2(e)} JF.T.{Fig.
5.2(f)}. Use of first starting solution. (c) Output o(x,y) as in
(b), using second starting solution. (d) Output o(x, y) as in (b),
using third starting solution . . . . . . . . . . . . . . . . . . . . 103
10
LIST OF TABLES
.5.1. The three different starting points . . . . . . . . . . . . . . . . "
98
11
ABSTRACT
Integral logarithmic transforms are defined for both one-dimensional and twodimensional input functions. These have the desirable properties of linearity and
invariance to scale change of the input. Two-dimensional integral logarithmic transform is additionally invariant to rotation. The integral logarithmic transforms are
conveniently inverted by simple differentiation. They are amenable to optical analog implementation by using incoherent light and simple collimating lenses. As
an application, the problem of noise suppression of an arbitrarily scaled image, by
using Wiener filtering, is considered. Also, application to a problem of character
recognition and matched filtering is proposed.
A new approach is given for the problem of reconstruction of phase from modulus
data. A set of vViener-filter functions is formed that multiply, in turn, displaced
versions of the modulus data in frequency space such that the sum is a minimum L 2 error norm solution for the object. The modulus data are permitted to contain both
noise and object signal components. The required statistics are power spectra of
the signal and noise, and correlation between modulus data at a given frequencies
and complex object spectral values at adjacent frequencies. Reconstructions are
12
formed in the presence of data noise, data gaps, and filter-construction noise, in
yarying amounts.
Finally, a new technique is proposed to reconstruct a turbulent image from a
superposition model. Imagery through random atmospheric turbulence is modeled
as a stochastic superposition process. By this model, each short-exposure point
spread function is a superposition of randomly weighted and displaced versions of
one intensity profile, e.g. an Airy disc. If we could somehow estimate the weights
and displacements for a given image, then by the superposition model we would
know the spread function, and consequently, could invert the imaging equation for
the object. In principle, this allows an object scene to be reconstructed from but
two short-exposure images, and without the need for a point reference source in
the field. By comparison, all other methods of reconstruction require 20 or more
images for decent quality in the output.
13
CHAPTER 1
INTRODUCTION
1.1
Invariance
There has been a growing interest in the exploitation of invariant pattern recognition. The goal is to describe computational techniques for recognizing patterns,
invariant to the distortions they might have been subject to. The distortions can
be geometric in nature, or they can be caused by changes in illumination and/or
motion. Specifically, obtaining object classification in the face of geometrical distortions [1] in the input object (due to translation, scale, and rotation) is a major
pattern recognition problem that has received extensive attention. They arise in
a variety of situations such as inspection and packaging of manufactured parts [2],
classification of chromosomes [3], target identification [4, 5], and scene analysis [6].
The current approaches to invariant two-dimensional shape recognition include extraction of the global image information using regular moments [5], boundary-based
14
analysis via Fourier descriptors [7, 8, 9], or autoregressive models [10], image representation by circular harmonic expansion [11], syntactic approaches [3], and artificial neural networks [12, 13, 14, 15, 16]. But we will concentrate only on realizing
invariance via mathematical transform in this dissertation.
Translation invariance can be given via the magnitude of the Fourier transform
of the image. If translation between the two object functions need to be known, it
can be done by calculating the cross correlation. Scale-invariant pattern recognition
is one of the basic requirements for general purpose image processing system. Scale
invariance can be realized either via a description of objects by image primitives and
their relation or via mathematical transforms. Recent interest concentrated on the
Mellin transform, which is identical to the Fourier transform of the function with
logarithmically distorted coordinates [17, 18]. The logarithmic distortion converts
scaling to translation; since the absolute value of the subsequent Fourier transform
is translation-invariant, the magnitude of the Mellin transform is scale-invariant.
That two logarithmically distorted functions differ by translation can be detected
..
alternatively via their cross correlation function. If the original images differ by a
scale factor only, the cross correlation has a prominent peak for some relative shift
and the scale factor can be obtained from tile peak location.
This principle of logarithmic mapping seems to playa role in biological sensory
systems. In the simian visual system, as well as in the visual system of the cat, the
mapping of the central 20° - 30° of retinal space onto area 17 of the visual cortex
15
approximates a polar coordinate transformation together with a logarithmic distortion of the radial-axis [19, 20]. In the mammalian auditory system there is also a
logarithmically scaled representation of frequencies along the basilar membrane. A
model of the auditory system, based on the Fourier-Mellin transform, has been proposed by Altes [21, 22]. Casasent and Psaltis [23] describe a transformation that is
invariant to rotation and scale changes in an input image. They present an optical
implementation of the transformation. The optical transformation combines the geometric polar transformation with the conventional optical Fourier transform. They
demonstrate the extension of the transformation to optical pattern recognition. The
transformation sequence can be realized by the following steps [23]:
1. The magnitudes IF1(wI,W2)1 and IF2(Wl,W2)i of the Fourier transforms of the
two input objects are formed;
2. These funct!ons in step (1) are converted to polar coordinates as FI (r, 8) and
F2(r,8);
3. These polar functions are then logarithmically scaled in r to yield FI (e P , 8)
and F2 ( eP, fJ) where p = In r;
4. The magnitudes of the Fourier transforms of the functions in step (3) yield
the translation-, rotation-, and scale-invariant functions 1F,\fl (wp,we)1 and
1 F.\!2
(Wp, we) I·
16
Despite the powerful properties of translation, rotation, and scale invariance,
the combined Fourier and Mellin transform, in its present form, is not suitable for
feature extraction because it obscures much of the discriminatory information contained in the original data. The Fourier and Mellin transform have three operations
that result in attenuation of information.
1. Magnitude of Fourier transform: Translation information contained in the
phase of the Fourier transform is discarded.
2. Magnitude of Mellin transform: Scale information contained in the phase of
the Mellin transform is discarded.
3. Fourier transform followed by exponential sampling: Low frequencies are accentuated.
.,
So as not to distort the data to the extent that valuable information is lost,
integral logarithmic transforms [24], in Chapter three, are defined for both onedimensional and two-dimensional input functions. These have the desirable properties of linearity and invariance to scale change of the input. The two-dimensional
integral logarithmic transform is additionally invariant to rotation. The integral
logarithmic transforms are conveniently inverted by simple differentiation. Also,
they are amenable to optical analog implementation by using incoherent light and
simple collimating lenses. As an application, the problem of noise suppression of
an arbitrarily scaled image, by using Wiener filt.ering, is considered. Use of the
17
integral logarithmic transform of the image data as a preprocessing step permits
the creation of a single \\liener filter optimized for use at all scales of magnification.
Finally, application to a problem of character recognition and matched filtering is
proposed.
1.2
Phase Retrieval
The reconstruction of a signal from the magnitude of its Fourier transform
(Fourier intensity), generally referred to as the phase-retrieval problem, arises in a
variety of different contexts and applications and such diverse fields as astronomy,
x-ray crystallography, electron microscope, optics, wave-front sensing, and signal
processing. One wishes to reconstruct f(x,y), an object function, from !F(wI,w2)1,
the modulus of its Fourier transform
(1.1)
where FT denotes Fourier transform, and B(wI, W2) indicates the phase part of its
Fourier transform. Since the autocorrelation of the object can be computed from the
Fourier modulus by FT- 1 [IF(u..'1,i.I,..'2W]' this problem is equivalent to reconstructing
an object from its autocorrelation.
One must have sufficiently strong a priori information about the object to make
the solution unique. Of course, one has the omnipresent ambiguities that f( x, y),
exp(jBe)f(x - Xo, Y - Yo), and exp(jBe)!*( -x - Xo, -y - Yo), where Be is a constant
phase and j*(x,y) is the conjugate function of f(x,y), all have the same Fourier
18
modulus. If these omnipresent ambiguities (phase constant, translation, and conjugate image) are the only ambiguities, then we consider the phase-retrieval problem
to be unique.
To overcome the difficulties associated with the reconstruction of a signal from
its Fourier intensity, a number of different methods have been proposed for adding
additional information or constraints to the phase-retrieval problem. It has been
shown, for example, that if the boundary values of a finite support two-dimensional
signal are specified along with the Fourier intensity of the signal, then a simple
recursive algorithm [25] can often be used to reconstruct the signal. A number of
researchers have also considered the problem of reconstructing a signal from more
than one intensity function. Gerchberg and Saxton [26, 27], for example, presented
both noniterative and iterative algorithms that use two intensities - one in the
diffraction plane and one in the image plane. A similar problem was considered by
Misell [28, 29], who presented an iterative algorithm to recover phase information
from image intensities measured in two defocused planes. Gonsalves [30], on the
other hand, presented two closed-form solutions to the phase-retrieval problem for
one-dimensional signals, using differential intensity measurements by changing parameters such as the position of the focal plane and transmission of the aperture.
More recently, Nakajima [31, 32] has considered a linear method for phase retrieval
from two intensity measurement that are obtained with and without an exponential
filter in the object plane.
19
In Chapter four a new approach is given for the problem of reconstruction of
phase from modulus data [33]. A set of \Viener-filter functions is formed that multiply, in turn, displaced versions of the modulus data in frequency space such that
the sum is a minimum L 2 -error norm solution for the object. The modulus data are
permitted to contain both noise and object signal components. The required statistics are power spectra of the signal and noise, and correlation between modulus data
at a given frequencies and complex object spectral values at adjacent frequencies.
In a numerical simulation, a 3 x 3 filter array is used to reconstruct any member of
an object class consisting of 16 pictures of space shuttles in various combinations.
The 16 pictures are used as a learning set to form the required power spectra and
correlations mentioned above. Reconstructions are formed in the presence of data
noise. data gaps, and filter-construction noise, in varying amounts.
In Chapter five we formulate a new method for the restoration of atmospherically degraded images [34]. In this problem we must compensate for severe wave
aberrations caused by atmospheric turbulence. First, we review some methods of
obtaining diffraction-limited information through the turbulent atmosphere. These
are stellar interferometers and speckle interferometry. This is followed by some conventional approaches for handling astronomical speckle data. Finally, a new technique is proposed to reconstruct an turbulent image from a superposition model.
Imagery through random atmospheric turbulence is modeled as a stochastic superposition process. By this model, each short-exposure point spread function is a
20
superposition of randomly weighted and displaced versions of one intensity profile,
e.g. an Airy disc. If we could somehow estimate the weights and displacements for
a given image, then by the superposition model we would know the spread function, and consequently, could invert the imaging equation for the object, using any
conventional deconvolution approach such as inverse filtering [3.5]. In principle, this
allows an object scene to be reconstructed from but two short-exposure images,
and without the need for a point reference source in the field. By comparison, all
other methods of reconstruction require 20 or more images for decent quality in the
output. Some computer-simulated demonstrations of the approach are given.
Chapter six will conclude the dissertation by summarizing the results, showing
the main contributions, and suggesting further research related to our work.
21
CHAPTER 2
TRANSFORMS FOR GEOMETRICAL INVARIANCE:
REVIEW
2.1
Introduction
We consider in this chapter conventional mathematical transformations to achieve
invariance to two-dimensional geometrical distortions such as scale, rotation, and
translation. The image is complex-log conform ally mapped so that rotation and
scale changes become translation in the transform domain. The magnitude of the
Fourier transform is invariant to translation of the object in the image. A scaleand translation-invariant description of the image can be obtained via the absolute
value of the Mellin transform of its Fourier amplitude spectrum. Casasent and
Psaltis [23] have shown that rotation invariance, as well as scale and translation
invariance, can be achieved with a rectangular to polar transformation followed by
~'1ellin
transform. Since the absolute value of the ?':mrier transform or Mellin trans-
form, i.e., the amplitude spectrum, contains no information on the relative phases
of the spectral components, valuable structural information is lost. This review
22
illustrates the need for Integral Logarithmic Transform that is examined in detail
in Chapter 3.
2.2
Conformal Mapping
One useful technique used to achieve invariance to rotation and scale changes
is the complex-logarithmic (CL) conformal mapping. Conformal mapping transforms an image from rectangular coordinates to polar exponential coordinates.
This transformation changes rotation and scale into translation. Specifically, assume that Cartesian plane points are given by (x,y)
z
= x + jy.
Thus we can write z
= rexpj(),
where r
=
(Re(z),Im(z)), where
=
Izl
= (x 2 + y2)1/2
and
() = arg(z) = arctan(yfx). Now the CL mapping is simply the conformal mapping
of points z onto points w defined by
= In(z) = In[rexpj()] = lnr + j().
(2.1)
Therefore, points in the target domain are given by (lnr,())
= (Re(w), Im(w)), and
w
logarithmically spaced concentric rings and radials of uniform angular spacing are
mapped into uniformly spaced straight lines. More generally, after CL mapping,
rotation and scaling about the origin in the cartesian domain correspond to simple
linear shifts in the () (mod 2/T) and In r directions, respectively.
There are several problems associated with conformal mapping. First, since
interpolation is necessary at exponentially spaced sample points for the logarithmic
distortion of the image, the image reconstructed from the samples will not carry
23
all the information from the original. In particular, details close to the edge of the
original image will be smeared by sampling and reconstruction. A second problem is
sensiti\·ity to center misalignment of the sampled image. Small shifts from the center
causes dramatic distortions in the codormal mapped image. A third problem that
occurs in the conformal mapping is related to its size invariant aspect. A change in
scale does not appear as a direct translation in practice. When an image is scaled
from smaller to larger a translation occurs in the conformal mapped image but the
points left vacant by the translation are filled with more samples from the center
of the image. If the object in the image has no hole in its center the new samples
which take the place of the translating points will in general be very similar to those
translating points. This has the effect of stretching, not simple translation in the
conformal mapped image.
2.3
Translation Invariance of Fourier Transform
The bread-and-butter approach for dealing with geometrical distortion is based
on the Fourier transform(FT). We consider a two dimensional image characterized
by f(x,y). The corresponding FT is given by F(WI,W2), where
(2.2)
The discrete version of the FT is given by the discrete Fourier transform(DFT),
which takes a discrete spatial image into another discrete and periodic frequency
24
representation. Formally, the DFT is given as
1
F(k, l) = M N
-
N-l M-l
L L
kn
lm
f(n, m) exp[-j27i( N + lV!)]
(2.3)
n=O m=O
where 0 :5 k :5 N - 1, and 0 :5 I :5 A1 - 1.
It is easy to show that the magnitude of both the FT and DFT are shift invariant.
Assume that f(x, y) is shifted by (xo, Yo). Then, the FT of f(x - xo, y - Yo) is given
as
Changing variables, a
=x -
Xo and b = y - Yo, and it follows that
(2.5)
Finally,
In a similar way one can show the same property regarding the DFT.
The magnitude component of the Fourier transform which is invariant to translation, carries much of the contrast information of the image. The phase component
of the Fourier transform carries information about how things are placed in the
image. Translation of f(x,y) corresponds to the addition of a linear phase component. The conformal mapping transforms rotation and scale into translation and
the magnitude of the Fourier transform is invariant to those translation so that it
will not change significantly with rotation and scale of the object in the mage.
25
The phase of the Fourier transform holds the spatial layout of image. Oppenheim and Lim [36] examined the importance of the phase and showed that under
fairly loose conditions the entire image could be reconstructed to within a constant
multiple of the magnitude given only the phase. This implies that most of the
information allowing discrimination between real images lie in the phase. However,
Lane et al. [:37] showed that the intrinsic form of a finite positive image is uniquely
related to the magnitude of its Fourier transform, except under contrived conditions or trivial situations. This suggests that reasonable discrimination can still be
obtained using the magnitude of the Fourier transform of an image.
2.4
Mellin Transform
In this section, the Mellin transform is reviewed, and the scale invariance property is shown. Given a function f(t), t 2:: 0, the continuous Mellin transform in one
dimension is defined as [18]
1\1(8) =
10';0 f(t)tS-1dt.
(2.7)
Introducing an exponential distortion of the independent variable,
= Te x ,
the
:\fellin transform can be implemented by the Fourier transform,
(2.8)
Substituting
8
= -jw, and noting that the magnitude of
T-jw
is unity, the mag-
nitude of J\1( - jw) is the magnitude of the Fourier transform of the exponentially
26
distorted function, as
1111(-jW) I
=
IT-iw111: f(TeX)e-iwXdxl
=
Ii: f(TeX)e-iWXdxl.
(2.9)
Combining the exponential distortion with the shift invariance property of the
magnitude of the Fourier transform, results in the magnitude of the Mellin transform
being scale invariant. For example. letting g(t) = f(mt) and applying (2.7) gives
G(s) = T S1: f(mTeX)eXSdx
=
A change of variable, y = x
(2.10)
T S1: f(Tex+lnm)eXSdx.
+ In m,
(2.11)
and evaluating gives
(2.12)
Taking the magnitude of both side of Eq. (2.12) gives
(2.13)
IG(s)1 = IM(s)l·
The scale factor is reduced to a translation term by the exponential distortion. The
translation term is further transformed into a pure phase component by the Fourier
transform. The magnitude of the resultant Mellin transform, is thus invariant to
the scale factor.
If f(O) is nonzero, then f(Te X ) will be nonzero at x
= -00.
Although the
infinite domain in x is unrealizable, the continuous Mellin transform can still be
27
approximated as follows. Assume f(t) is constant for 0
.M(s) =
iT
j(i)ts-1dt
~
x
~
+ !roo f(t)tS-1dt
=
f(O)T S/ s + T S faoo f(TeX)eXSdx
=
TS[f(O)/s
+
T, then
L:
r(x)eXSdx].
(2.14)
where
J*(x)
=
f(TeX) if x
{
o
~0
otherwise.
Eq. (2.14) will be exact, and scale invariant, if f(t) is constant in 0 ~ t ~ kT
for the largest scale factor k of interest. Thus, for nonzero j(0) the continuous time
correction term
f(O)/w
(2.15)
must be added to the imaginary part of the Fourier transform, at all desired frequencies.
2.5
Fast Mellin Transform
For sampled data, the discrete implementation of the continuous Mellin transform is referred to as the fast Mellin transform(FMT). This terminology reflects the
fact that the Fourier transform operation is performed by an FFT. Suppose that
j( t) is represented by N samples,
f(t),
t
= 0,T,2T,···,(N -l)T.
(2.16)
28
Exponential sampling must both cover the domain of the f(t) samples and not
exceed the sample spacing (assuming f(t) is sampled at the Nyquist rate). This is
accomplished by selecting the uniform spacing in x to be
.6.x
= liN.
(2.17)
The required number M of samples in x has been derived by Casasent and Psaltis [38]
to be
iVf = NlnN.
(2.18)
To complete the FMT the M exponentially sampled data points are then transformed using an FFT.
2.6
Fourier-Mellin Transform
It has been suggested that the Mellin transform be applied to the magnitude of
the Fourier transform data. [39, 23, 40, 22] This compound transform is called the
Fourier-Mellin(FM) transform. The Fourier transform property of shift invariance
eliminates the problem of translation while retaining the scaling problem for the
~-1ellin
transform. Moreover, the intermediate Fourier transform operation has the
advantage of increasing the spectral detail by the artifact of zero filling the data
sequence. [41] If the increase in detail is sufficiently fine, then the interpolation
required by the exponential sampling can be eliminated.
The Fourier-Mellin operation is divided into three processing steps: normalizing,
exponential sampling, FFT. The normalization step is required because a scale
29
change in f( t) translates into a scale change in both the transform variable and the
transform value, i.e.,
f(t)
~
F(s)
f(mt)
~
l/mF(s/m).
(2.19)
The scaling on the transform variable will be counteracted by the Fourier-Mellin
transform, but the scaling on the transform must be removed either by normalizing
f(t) by its area, or normalizing the Fourier spectrum by its dc component.
Despite the powerful properties of both scale and translation invariance, the
Fourier-Mellin transform, in its present form, is not suitable for feature extraction
because it obscures much of the discriminatory information contained in the original
data. The Fourier-Mellin transform has three operations that result in attenuation
of information.
1. FFT Magnitude: Translation information contained in the phase of the FFT
is discarded.
2. FMT :Magnitude: Scale information contained in the phase of the FMT is
discarded.
:3. FFT Followed by Exponential Sampling: Low frequencies are accentuated.
30
2.i
Direct Mellin Transform
An alternative to the Fourier-Mellin transform can be implemented via a direct
expansion of (2.7). The resulting implementation is referred to as the direct Mellin
transform(DMT).
Expanding (2.7) using an integration step size of T gives
F(s) =
i
T
o
](t)tS-ldt + l2T ](t)tS-ldt + ... + /,NT
T
](t)tS-ldt.
(N-l)T
(2.20)
Assuming f( t) is constant in any T interval then the subintegrals are readily evaluated,
Regrouping and defining
](0) = ]1, f(T) = ]2,"', ]((N - l)T) = ]N
(2.22)
and without loss of generality, letting T be unity, (2.21) is expressed as
N-l
sF(s) =
L
P(Jk - fk+l)
+ NS]N.
(2.23)
k=l
The input data is expressed in a more useful manner by defining the incremental
variable
(2.24)
Assuming ] N is zero (the data record length can be adjusted to ensure this) the
DMT becomes
N-l
sF(s)
=L
k=l
k S t1k
(2.25)
31
and using s = -jw
- jwF(w)
= 2) cos(w In k) -
j sin(w In k)).6. k •
(2.26)
k
The magnitude of the DMT is
(2.27)
The DMT is an exact implementation of the Mellin transform when performed on
sampled data. The FMT, on the other hand. requires interpolation between data
points.
The DMT operation is more clearly expressed in matrix notation. Form (2.26),
r
w1G(,,-'d
W2 G(W2)
</>11
</>1 "
</>l,N -1
.6. 1
</>21
</>22
</>2,N -1
.6. 2
(2.28)
=
-]
</>M,N-1
.6. N - 1
.I
where
cPik
and
Wi,
=
COS(Wi
In k) - j sin(wi In k)
(2.29)
i = 1,2,··· ,M are arbitrary spectral components.
The magnitude of the DMT can also be expressed in matrix notation. First,
define
Cik
ans
8ik
to be the real and imaginary components of the DMT matrix ele-
ments; and Ci and Si to be column vectors corresponding to the real and imaginary
row components of the DMT matrix,
32
c'!
, =
[ Cil
Sf
, -
[ Sil Si2
.6. =
Ci2
C;,N-1 ],
Si,N -1
],
[ .6. k ].
(2.30)
The DMT given in (2.28) becomes
-j[W;G(Wi)]
= [cT -jsT.6.],
i
= 1,2,···,l~1.
(2.31)
The magnitude squared of component i is then
(2.32)
The lm element of matrix Mi is given by
(2.33)
33
CHAPTER 3
INTEGRAL LOGARITHMIC TRANSFORM(ILT)
3.1
Introduction
Transform theory [42, 43] has played a key role in optical signal processing for a
number of years. It continues to be a topic of interest in theoretical as well as applied
work in this field [18,39]. We have been seeking a new transform that is potentially
applicable to a wide class of processing problems. It is often desirable to preprocess,
or transform data into an output that is invariant to the scale of magnification of
the data. The scale of magnification is assumed to be unknown. In addition, the
transformation should be linear (for example, the complex modulus operation is
nonlinear) so as not to distort the data to the extent that valuable information,
e.g., phase information, is lost.
Thus, we seek a linear operator L upon given data f(x) (one-dimensional notation used for simplicity) such that
L[J(x)] = L[J(mx)],
(3.1)
34
where m is an arbitrary, unknown magnification factor. Data function f(x) is also
considered arbitrary. An important limitation in the choice of operator L is as
follows.
Theorem 3.1 A simple 1:1 mapping of data f(x) by means of a distortion rule on
coordinate x cannot achieve Eq. (3.1).
Proof
Let each coordinate x be mapped to a new coordinate g(x), function 9 to be
found. Thus a solution of the form
L[j(x)] = f[g(x)]
(3.2)
is sought for some g(.). Then directly
L[j(mx)] = f[g(mx)].
(3.3)
In order for scale invariance (3.1) to be obeyed, g(.) must then obey
f[g(x)] = f[g(mx)].
(3.4)
But since f is arbitrary, the only solution g(.) to Eq. (3.4) is
g(mx)
= g(x).
(3.5)
Since also magnification m is arbitrary, the only solution to Eq. (3.5) is
g(x) = canst.,
(3.6)
35
i.e., all input points map into one point. This is an unacceptable solution, since if
it were used in Eq. (3.2), the resulting transformation would be noninvertible for
the general input f( x ).
A second class of candidates for operator L consists of functionals of f(x), i.e.,
integral transforms. One candidate is the Mellin Transform (MT). It is known [44]
that the magnitude of MT obeys the scale invariance. That is, if
(3.7)
MT[J(x)] = FM(W)
then
MT[J(mx)]
= x-im FM(W),
j
= yCI
(3.8)
has the same magnitude, /FM(w)/. However, taking the magnitude of the MT causes
a loss of important phase information. This is undesirable for purposes of inverting
the MT back into direct (signal) space. In the absence of phase information, such
inversion problems are ill-posed and formidable to solve. [25]
In this chapter an integral logarithmic transform (LT) is defined which has the
required properties of linearity and geometrical scale invariance. Also, the LT is
easily inverted for its input.
Other types of logarithmic operations have been 1:1 point mapping, as follows.
Schwartz [19] has provided evidence that the retinotopic mapping of the visual field
to the surface of the striate cortex may be characterized as a logarithmic conformal
mapping. Weiman and Chaikin [45] describe a logarithmic spiral grid for picture
digitization. Schenker et al. [46] have defined the logarithmic conformal mapping on
36
a polar exponential grid for image understanding applications which have included
image correlation and target boundary estimation.
Unfortunately, these mappings cannot satisfy our requirement (3.1) of invariance
to change of scale. For example, with logarithmic conformal mapping, a change of
scale transforms into a shifted version of the original image. Messner and Szu [47,48]
experimentally confirmed this effect using images of an airplane at various scales. A
scale-dependent shift is not detrimental to scale-invariant Wiener image smoothing.
But it is detrimental to character recognition by matched filtering, because of the
resulting registration problem.
3.2
Derivation
As motivation for the form of the LT, consider the simple 1:1 logarithmic mapping of linear function f(x) = x. Its differential in the presence of magnification
scale m is
dx
d[logf(x)] = d[logmx] = - = d[logx],
x
m = const.,
(3.9)
regardless of m. The derivative operation accomplishes invariance by deleting the
magnification amount m. Such logarithmic mapping has also been proposed as
a model for the brain/cortex function [49]. Property (3.9) suggests the following
transformation of a general function
f (x).
37
Definition 3.1 Given a real, one-dimensional function f( x) with finite support
E ~ X ~
Xo, its logarithmic transform LT[J(x)] is defined as
LT[f(x)]
rco -xf(xy)
dx
= F(y) = Js.
(3.10)
11
E/XO ~
for
Y ~ 1.
This transform is linear in its input f( x). The finite support requirement avoids
a potential pole at x = 0, in particular. In practical problems, an image f(x) will
aiways have finite support. Then the lower support value
E
can be attained by
simply translating the image away from the origin by an amount
E
E.
The choice
= 1 pixel is usually most convenient.
3.3
Scale-invariant Property
One important property of the LT is shown next.
Theorem 3.2 The LT of a scaled version of f(x} is the same as the LT of f(x}.
Proof
Let j(x) be a scaled version of f(x), j(x) = f(mx), m constant. Since f(x) = 0
for x
> Xo or for x <
€,
necessarily j(x) = f(mx)
= 0 for x > xo/m or x < E/m,
and therefore the LT F(y) of j(x) obeys
F(y) =
J~
~
my
j(xy) dx
x
= J~
~
my
f(mxy) dx
X
(3.11)
38
for
f/XO :::;
Y :::; 1. The lower limit on y is still
f/XO,
since in general it is the ratio
of the lower support value to the upper; here, (f/m)/(xo/m). We make a change of
variable t = mx in Eq. (3.11), and obtain
F(y) = (XO f(ty) dt = F(y),
J~!I
t
(3.12)
as was to be shown.
More generally, it can be shown in the same way that a nonlinear, power-law
distortion of scale, to a coordinate t = mxk, k, m real and m > 0, produces a log
transform
1
A
k
Ikl F(y ).
F(y) =
(3.13)
Hence, the transform is distorted 1:1 in the same way as is the input scale. The
power k
=
1 checks with result (3.12) for the linear scale change case. Change
of scale t = mx k also is the most general functional form for causing a mere 1:1
distortion of the original log transform. The utility of result (3.13) is that once
F(y) is known, the log transform for any power-law scale change is trivially known
as well.
It is interesting to consider an alternative definition of the LT:
L(y) =
1 f(x)Y
xO
c
x
dx
(3.14)
for 0 :::; y :::; 1. This definition would also satisfy the property of scale invariance,
as is easily verified. However, because of the variable exponent y, transform L(y) is
39
not linear in I( x), invalidating our linearity requirement. It also appears difficult
to invert Eq. (3.14) for I(x).
Other mathematical properties of F(y) are described following Eq. (3.25) below.
Definition 3.2 Given a real, two-dimensional function /(x,w), its LT can be defined
analogously as
F(y,z) =
f~o
%
f;o f(xy,wz)dxdw
y
xw
f/XQ
:5 y :5 1,
f/WO
:5 z :5 1.
'
(3.15)
Higher-dimensional LT's would have the analogous form. It is easy to show that
F(y, z) is likewise invariant to scale of magnification. The proof is an obvious
generalization of the proof of Theorem 3.2.
The LT was applied to both 64 x 64 image in Fig. 3.7. As shown in Fig. 3.8,
the LT outputs are monotonically increasing. Compared with two LT outputs, it is
found that the increasing slopes are different.
3.4
Rotation-invariant Property
It is interesting to consider the question of whether F(y, z) is invariant to rotation
of the image f(x, w). This cannot be satisfied, since rotation by the particular angle
() = 7r /2 interchanges the roles of x
and w. For this rotated image the LT would
produce an output F(z,y) =f:. F(y, z) generally.
40
However, there is an alternative two-dimensional version of the LT that is indeed
invariant to both magnification and rotation. Given the image J(x, w), first form
the polar coordinate image
jp(r,e) = J(rcose,rsine).
This is a 1:1 remapping of coordinates
ex, w) of j
(3.16)
into polar coordinates (r, e) of
Jp. Define the two-dimensional integral LT of JP as
D
I"p
(
y, Z ) =
r
21r fro
JO
e
y
f p (ry,9z)
r2
r dr de ,
plro
~ y ~
1,
O:5z~l.
Polar image jp is assumed to have a limited support region p :5 r
e ~ 27r.
(3.17)
~
Transforms (3.15) and (3.17) are analogous in form, but with
ro and 0 :5
r2
replacing
coordinate product xw in the denominators.
Consider a polar image
(3.18)
resulting from arbitrary scale stretch Tn and rotation !:le of the initial image Jp(r, B).
Substituting image (3.18) into (3.17) results in a LT:
(3.19)
after change of integration variables to t = mr, and a = () - !:le. In summary, twodimensional transform (3.17) is linear in its input jp(r, e), and invariant to both
change of scale and rotation of the input.
41
3.5
Inversion
Here we derive expressions for the inverses of the one- and two- dimensional
LT's.
Theorem 3.3 If F(y) is the one-dimensional LT of f(x), the inverse logarithmic
transform is given by
f(x) = -=-F'( -=-)
Xo
Xo
for c ::; x ::; xo,
(3.20)
where the prime denotes derivative.
Proof
The logarithmic transform F(y) is defined by Eq. (3.10). If the derivative dJdy
is taken of both sides of Eq. (3.10), it becomes
F'(y) =
r1:
0
J;
By the finite support condition, f(c) =
f'(xy)dx _ f(c).
y
o.
(3.21)
Hence only the integral remains. Making
a change of variable t = xy and integrating give
1 jXOY
1
F'(y) = f'(t)dt = -[J(XOY) - f(c)].
y £
Y
Again use f(c) =
o.
(3.22)
Furthermore, since c/xo ::; y ::; 1, necessarily c ::; xoy ::; Xo,
the required support of f(x). Hence we let x = Xoy in Eq. (3.22). This allows
Eq. (3.22) to be solved directly for f(x) over its entire support region, yielding the
required result (3.20).
42
By analogous steps, inverse to the two-dimensional LT's (3.15) and (3.17) may
be found. The inverse LT of Eq. (3.15) is
J(x, w) =
~~ [EP F(y, z)]
oyoz
Xo Wo
(3.23)
y=:r/:r:o,%=wjwo
Also the inverse LT of Eq. (3.17) is
f (
Jp r,
()) _ _
r
- 27rr
o
[~[ oFp(Y, Z)]]
oz
Z
0
Y
y=rjro,z=()j21r
.
(3.24)
Once Jp(r, (}) is known, J(x, w) can be formed as well by reversing the sense of the
1:1 polar mapping defined Eq. (3.16).
3.6
Optical Implementation
Since the LT operations [Eq. (3.10) or (3.15)] are linear in the inputs J(x),
J(x, w), it should be possible to implement them optically. We suggest one method
of implementation here. The one-dimensional transform is treated first.
It is convenient to change the integration variable to t =
xy
in Eq. (3.10). The
direct result is
F(y) =
l(
XOY
f(t) dt
t
for f.jxo $ y $ l.
(3.25)
This shows that F(y) is a cumulative function, analogous to a cumulative probability
law in statistics, or to an edge response function in optics. Note in particular that
Xo must be finite for F to remain a function of y (and hence be a usable transform).
The LT requires inputs that have truly finite extension. Further regarding limits,
representation (3.25) shows that the limit
f. --lo
0 can be taken. The proviso is
43
that J(t) approach zero with t as fast or faster than denominator t. For example,
J(t) = t or t 2 could be so used.
Figure 3.1 shows an optical arrangement that implements F(y) of Eq. (3.25) as
a spatial image. The pinhole source S radiates white light that is collimated by lens
A. The light is incoherent and passes, in turn, through image energy transparency
profile J(t) in plane I, transmittance make profile lit in plane M, and variable
diaphragm D opening
€ :::;
t :::; xoy, before being refocused by lens A' about point y
in output plane O. The total light energy collected in
°
then obeys F(y) given by
Eq. (3.25). This is for one particular y value.
Parameter y is varied over its domain (€I xo, 1) by moving the upper jaw position
t = xoy of diaphragm D. Synchronous with this motion, output plane 0 moves
vertically (in scanning mode) so that energy F is focused spatially about a new
position y in O. If a piece of film is attached to plane 0, it will produce F(y) as a
spatial image.
This implementation is one-dimensional. An implementation of two-dimensional
transform (3.15) would use, analogously, an image J(t, s) of finite support area
(c:::;t:::;xo; €:::;s:::;wo),
(3.26)
a mask 1I (ts ), and a square diaphragm covering variable area
(3.27)
Motion of the output plane 0 in the directions (t, s) synchronous with jaw positions
(t = xoy, s = woz) will now produce a spatial, two-dimensional image F(y, z) in O.
44
3.1
Applications
The LT may be used as a linear preprocessing step to render data f(x) invariant to its scale of magnification. The resulting secondary data, if now optimally processed by any problem-specific algorithm (e.g., Wiener filtering for a
noise-suppression problem), will be optimized for use by this algorithm independent of scale in the original data. We discuss two such applications below: image
noise suppression and character recognition.
Suppose that an image f(x) suffers from additive noise n(x). It is desired to
estimate the object o( x) which gave rise to the image. This is the familiar problem
of image smoothing or noise suppression.
Let us regard o(x) and n(x) as stochastic processes o(x;).) and n(x; >.'), with
stochastic parameters)., ).'. Let o(x) and n(x) each obey arbitrary probability laws
po(o) and PN(n) at x. Then f(x) is also stochastic:
f( x; )., )") = o( x; ).) + n( x; >.')
(3.28)
It is convenient to Fourier transform both sides of Eq. (3.28), producing
F(w)
= O(w) + N(w)
(3.29)
in terms of corresponding stochastic spectral quantities (parameters).,).' suppressed).
It is well known that the minimum L 2 -norm filter solution Y(w) to this stochastic
problem is the Wiener smoothing filter [35]
<I>o(w)
Y(w) = <I>o(w) + <I>N(W) ,
(3.30)
45
where
(3.31 )
<l>o(w) = (IO(wW)
is the the average power spectrum of the object o(x), and
(3.32)
is the average power spectrum of the additive noise n(x).
Let us call the directly observable image f(x) primary data. We show next that
we cannot optically filter the primary data f(x), which is at one scale of magnification, with the filter Y(w) derived at another scale. (This is also demonstrated
below in Fig. 3.6).
Regard filter Y (w) of Eq. (3.30) as formed at initial scale of magnification m = 1.
The new filter Y(w) for a scaled object, o(x)
= o(mx), m > 0, will of course have
the same form as Eq. (3.30),
Y(w) =
A
~o(w)
<l>o(w) + <I>N(W)
,
(3.33)
where a caret mark denotes that the quantity beneath it is in the scaled space.
There are two components ~o(w) and ~N(W) to the Wiener smoothing filter Y(w).
Since additive noise is independent of the object, it has the same statistics for any
scale of the object. Thus ~N(W) is the same as <I>N(W). However, of course, the
second component ~o(w) is dependent on object scale size. A change of scale in
the x axis by a scale factor m causes
A
<l>o{w) =
1
w
- 2 <I> 0 ( - ) .
m
m
(3.34)
46
Combining results, the Wiener smoothing filter Y(w) for a scaled object is then
given by
~<po(~)
A
Y(w) =
~<Po(~) + iPN{W) =J Y(w).
(3.35)
The last inequality is made by comparison with Eq. (3.30). Hence the filter that is
designed for the initial object scale would not generally work for a new object scale.
[An exception is for the particular object class iPo(w) = aw- 2, as substitution into
Eq. (3.35) verifies.]
However, if we use the LT of the image data f{x) as secondary image data, and
seek Wiener filtering of the secondary data, then we can obtain a scale-invariant
Wiener filter. This is shown next.
According to Theorem 3.1, the LT of a linearly scaled object, o(x) = o(mx),
m
> 0, is the same as the LT of the original object o(x),
LT[o(x)] = LT[o{x)].
(3.36)
Hence their power spectra
<i>LT[o(x)](W)
_
(IFT{LT[o(x)]} 12 ),
iPLT[o(x)](w)
_
(IFT{LT[o(x)]}12),
(3.37)
are also equal,
<i>LT[o(x)](W)
= iPLT[o(x)](w).
(3.38)
The power spectrum of LT[n(x)] must also remain invariant to scale change of the
object, since the noise is additive [see argument below Eq.(3.33)]. The Wiener filter
47
YLT ( w) for the secondary image data is then,
_
}IiLT ( w ) -
~LT[o(x)](w)
~LT[o(x)l(w)
+ ~LT[n(z)l (w) ,
(3.39)
which is explicitly independent of the object change of scale parameter m. Filter
(3.39) will optimally process any linearly scaled version of an object in the object
class (3.31).
We can now use the above results to demonstrate the smoothing of images with
additive noise. The smoothing process was applied to real images with added
computer-generated noise. The scale-invariant filter (3.39) was constructed as in
Fig. 3.2. The particular two-dimensional LT operation (3.15) was used, since invariance to magnification (but not rotation) is the issue here. The actual object
spectrum was used in formation of the power spectrum. Many representative noise
profiles ni(x),i = 1,2, ... ,N, were averaged to form the noise power spectrum, as
indicated.
The data processing procedure is shown in Fig. 3.3. As indicated, it is the
secondary data LT[J(x)] that are actually filtered. The filtering procedure was
applied to the LT's both the 64 x 64 image and the 128 x 128 image in Fig. 3.4.
The image Fig. 3.4(b) is a scaled 2:1 version of image in Fig. 3.4(a). The images
suffer from independent sets of noise values. The filter was constructed by using the
smaller-scale object of Fig. 3.4(a). The LT-filter outputs are shown, respectively,
in Figs. 3.5( a) and 3.5(b). They both show good noise suppression properties, as
48
usual at the expense of resolution. Hence, the same filter works on either image
scale.
For comparison, we also repeated the study using primary image data. As we
saw in Eq. (3.35), results should now not be good when the filter derived at one
scale is applied to data at another scale. Images 3.4(a) and 3.4(b) were used as
primary image data at scales m = 2 and m = 1, respectively. The filter Y(w) was
constructed at the scale of Fig. 3.4(b). The output that is due to filtering the image
in Fig. 3.4(b) is shown in Fig. 3.6(b). The output that is due to using Y(w) upon
the image data in Fig. 3.4(a), i.e., at a different scale, gives the output image in
Fig. 3.6(a). As expected, this output is poor compared with that in Fig. 3.6(b).
It is important to note that the LT preprocessing step is not unique in permitting
Wiener filter outputs that are invariant to scale of magnification. In the Wiener
application, the requirement (3.1) can be replaced by the looser requirement
L[f(mx)] = L[J(x + a)]
(3.40)
for any a. The same Wiener filter operates optimally on shifted or unshifted data.
Shifted data simply produce a shifted output.
For example, a polar-log coordinate 1:1 mapping is a preprocessing step L that
obeys Eq. (3.40) and that allows for scale- (and orientation-) invariant Wiener
filtering (Messner and Szu [47]). This preprocessing would probably be easier to
implement digitally than LT filtering since it is a simple point-to-point mapping.
On the other hand, since polar-log coordinate mapping is a non-linear operation, the
49
linear LT outputs (3.15) might be easier to analyze for noise-propagation and signalto-noise effects. Regarding analog implementation, LT might have an advantage
because of the simplicity of the proposed optical implementation in Fig. 3.1. Further
comparisons of the two approaches would require detailed studies of such factors as
operational complexity, speed, cost, convenience, and accuracy.
Another potential application of the LT is to character recognition. Consider the
basic problem of recognizing individual letters of the alphabet as they are presented,
in turn, to a processing system. Assume that the letters are detected in perfect
registration, but can be at any scale of magnification. One way of recognizing the
letters is to use matched filtering [50]. The working principle of this approach is
that the maximum value of the cross-correlation of two images is largest when the
two images are the same. However, given a continuously variable magnification for
the letter images, it is impossible to have on hand the exactly matching template
image that will optimally identify each. Hence we seek a transformation of the
detected images that is invariant to magnification.
One possibility is conformal logarithmic mapping. However, This approach suffers from magnification-dependent translation (3.40) of the input. Hence, letter
images that are initially in registration will be transformed into randomly misregistered images. This would complicate the matched-filtering steps to follow.
One way around this registration problem would be to work with the modulus
of the Fourier transform of each log-mapped image [50]. The modulus is, of course,
50
invariant to lateral shift. Other methods [51, 52, 53, 54, 55, 56] have been proposed
as well.
It is interesting to consider applications of the integral logarithmic transform
concept to this problem. Two such approaches are described next.
One is to use the two-dimensional LT [Eq. (3.15)] of each primary letter image
as secondary image data to be processed. Secondary templates are also formed as
two-dimensional LT's of the 26 possible letter images. The intensities and registration states of the secondary images and templates are now independent of the scale
of magnification. Hence the matchups between images and templates can be (theoretically) perfect and the identifications made independently of scale. No other
filtering operations (aside from the matched filtering) would be necessary.
This approach lends itself to fairly direct electro-optical implementation. Let a
test character be input J(t) in the optical processor of Fig. 3.1, with jaw setting
Y fixed. The output Ftest (y) is sensed by a fixed detector located in plane 0 on
the OA. A second processor (Fig. 3.1) uses a template character as its input J(t),
simultaneously with formation of Ftest(Y). Call the second output Ftemp(Y). The
product Ftest(y)Ftemp(Y) is formed electronically from the two detector outputs.
The diaphragm D jaws open synchronously in the two processors so that they share
common openings y at any time. The detectors remain fixed in 0 on the OA. The
detector output products are electronically added, forming
(3.41)
51
This is the correlation, at zero lag, between test character and template. This is
a maximum when Ftemp(Y) is proportional to Ftest(Y), as is usual with matched
filtering. However, the match up additionally is independent of the scale of magnification of the test characters and the template characters. An additional virtue of
the approach is that the LT's are made on the fly and do not have to be formed as
two-dimensional hard-copy images, e.g., transparencies, prior to their use.
A final suggested approach to the magnification-registration problem is to use
LT's [Eq. (3.16) and Eq. (3.17)] of each primary letter image and template. The
secondary images are now invariant to magnification and rotation, without suffering
random misregistration. Optical implementation of this approach might not be as
convenient as the previous one, however, because of the necessity for producing
the polar image [Eq. (3.16)]. Computer-generated holograms [52, 54] have been
used for the analogous problem of implementing the log-polar transform. This is
cumbersome, however, compared with the on-the-fly implementation that is possible
by use of the two-dimensional LT [Eq. (3.15)].
52
t= xo
t=xoy
s
y
_ _ _ t=£
A
I
M
D
o
Figure 3.1: Optical implementation of the integral logarithmic transform.
53
IF. T.I
2
--
Form LT[o(x)]
,
I
~LT[o(:z:))(co)
+
Eq. (3.38
1 N
2
N i=l
1
-LIF.T.I .
Form LT[ni (x)]
i =1,2,..•. .N
..
~L T[n(:z:)) (co)
t
---
Figure 3.2: Formation of the logarithmic smoothing filter.
Y (co)
LT
54
Add noise
n(x)
Object data o(x)
.-
Primary data f(x)
~
L.T. Eq.(3 .10)
Fr[LT{f(x)}]
F. T.
Secondary data
LT[f(x)]
~
~
Filter YLT (00)
..
F. T.
Ff[LT{o(x)}]
-1
.-
L. T .
LT[o(x)]
-1
Eq.(3.19)
6'(x)
Figure 3.3: Flow of operations in the data smoothing experiment.
5.5
(a)
(b)
Figure 3.4: Input images with additive Gaussian noise. (a) 64 x 64 images, (b)
128 x 128 scaled version image with the same noise statistics as in (a).
(a)
(b)
Figure 3.5: Scale invariant filtering study, using LT (secondary) image space. (a)
The output by scale-invariant, Wiener filtering of Fig. 3.4 (a). (b) The output by
Wiener filtering of Fig. 3.4 (b) using the same filter as in (a).
56
(a)
(b)
Figure 3.6: Scale variant filtering study, using primary (direct) image space. (a)
The output by Wiener filtering of Fig. 3.4 (a) using the filter derived at a different
scale, that of Fig. 3.4 (b). (b) The output by Wiener filtering of Fig. 3.4 (b) using
the filter derived at the same scale, that of Fig. 3.4 (b).
57
I~
.~
'.
,,"" ">\\
(a)
(b)
Figure 3.7: Two input images to apply the integral logarithmic transform.
(a)
(b)
Figure 3.8: Outputs of the two-dimensional integral logarithmic transform. (a)
Logarithmic transform of Fig. 3.7(a), (b) Logarithmic transform of Fig. 3.7 (b).
58
CHAPTER 4
A MULTIPLE-FILTER APPROACH TO PHASE
RETRIEVAL FROM MODULUS DATA
4.1
Background
All detectors react only to the energy conveyed by light; in other words, they
measure the intensity of the light, and not the complex amplitude, so that phase
information is lost. This can be a big problem in those coherent optical systems
where part of the information is represented by the phase. As an example, consider
the case of obtaining the Fourier transform of an image by use of a lens. If we try
to detect the Fourier transform in the back focal plane of the lens, we will only get
the power spectrum of the image, instead of the spectrum itself.
Estimating the phase part of generally complex spectrum from knowledge of its
modulus is a classical problem in signal processing. This problem is widespread,
occurring in (a) application to atmospheric turbulence speckle reduction, for which
the modulus is known as either the visibility function from optical interferometric
data [57] or as the power spectrum from Labeyrie speckle data [58]; (b) in optical
or electron microscopy [59], for which the modulus data is the image intensity and
59
from this the surface thickness profile of the specimen is to be determined; and (c) in
reconstruction of quantum-dot semiconductor clusters [60] using modulus-squared
structure factor IF(w)/2 as data.
Methods of reconstructing phase from image data generally break into two
classes: iterative or non-iterative. The phase-from-modulus problem, in particular,
seems to be most amenable to iterative approaches. Among these are the inputoutput method of Fienup [25], the exponential filter method of Walker [61], and
the Newton-Raphson approach of Currie and Frieden [62]. Unfortunately, these
approaches sometimes suffer from either slow convergence or nonconvergence. A
second class of phase reconstruction problems arises when many stochastic images
of one object are given, as in astronomy. Multiple complex images convey a good
deal more object information than does a single modulus image. Perhaps not surprisingly, then, the many-images problem has been solved by using noniterative
approaches. Among these are the Knox-Thompson approach [63] and the triplecorrelation approach of Lohmann et al. [64]. However, it is currently uncertain as
to how sensitive the Knox-Thompson approach is to the statistics of the turbulence [58], -nhile the triple-correlation method requires for its use the computation
and storage of a four-dimensional function, the average bispectrum. For a modestsized image of size 256 x 256, for examples, this requires the storage of 108 data
values.
60
We report here on a new approach to phase retrieval from modulus data, called
the multiple Wiener phase-filter approach. This approach reconstructs the phase
by a simple filtering of the given modulus data. It is noniterative, and is described
next.
4.2
Wiener Phase-Filter Approaches: Derivation
For simplicity of notation, the derivation is in one dimension. Two-dimensional
results, in application to pictures, are strictly analogous to these and are explicitly
stated where needed.
The spectrum O(w), w a spatial frequency, of an object o(x), x a pixel coordinate,
is defined as
O(w)
where j =
H.
= (271")-1/21: o(x) exp( -jwx) dx
(4.1)
Both object o(x) and spectrum O(w) are, in general, complex
functions. However, in the applications below, we use real-only objects o(x) arising
from incoherent radiation.
The problem that we are attacking is the reconstruction of O(w) from imperfect
modulus data
D(w) = IQ(w)
+ N(w)l,
Iwl
~
n,
(4.2)
where n is the passband and N(w) is the possible random noise. We take a Wienerfilter approach [65], because of its noise suppression capabilities, and some wellknown practical advantages of implementation. Among these are (a) non-iterative
61
estimation, (b) modest computer memory requirements and (c) possible optical
analog processing.
The Wiener approach is inherently statistical. It models reality, defined by the
data D, as a random sample from stochastic processes [35], here 0 and N. The
unknown complex object 0 that is to be reconstructed is assumed to be a member
of a statistical class, or ensemble, of objects. Likewise, the actual noise N that
contributes to data D is assumed to be a member of a class of noise profiles. All
second-order statistics of the object and noise processes are assumed to be known,
either on the basis of a theoretical model or by direct averaging over a typical set
of object and noise profiles. We take the latter point of view in the applications,
although noise profiles, in particular, are not averaged over.
Ordinarily, Wiener filtering uses a single filter Y (w ). For reasons that will become
clear, we seek a finite set of filter functions instead,
(4.3)
n= 1, ... ,N
such that
N
Q(w) =
E Yn(w)D(w + m~w),
m = n
-1 - (N - 1)/2,
(4.4)
n=l
is a good reconstruction of O(w). Frequency increment ~w is at the user's disposaL
We often use N = 3, corresponding to a 3 x 3 array of filters
}~n(WI,W2)
in two
dimensions. Then Eq.(4.4) becomes a representation:
Q(w) = Yi(w) D(w - ~w)
+ }2(w) D(w) + Ys(w) D(w + ~w).
(4.5)
62
A rationale for the use of multiple-filter representation (4.4) now becomes apparent:
The value of estimate
6 at w is made to depend upon
the data at not only w, but
also at adjacent frequencies. This takes advantage of possible correlation between
the data and the signal at neighboring values in frequency space; these can only
aid in the reconstruction. This means that frequency increment, Dow, should not
be made so large as to lose such correlation. In practice Dow = 1 pixel width in
frequency space, the minimum possible D..w, worked best.
The sense of goodness of fit of representation (4.4) to O(w) is taken to be in the
minimum L2 norm error sense,
e
2
-
I
n
A
-n dw(IO(w) - O(wW} = minimum,
- In-n
dwE.
(4.6)
The indicated average is an ensemble average over both stochastic processes 0, N.
In applications below, the ensemble average is carried through as a sample average
over a discrete learning set of objects. This particular ensemble consists of 16 discrete, typical objects of one class, the class of space shuttles. The statistical nature
of this object class originates in the random three-dimensional position, orientation, and magnification of each shuttle image. (However, the latter information is
assumed to be unavailable to the user. Otherwise, the reconstruction problem could
be parameterized, completely changing its character.) An example of the averaging
process is discussed next.
63
Typical among the average-squared terms in Eq. (4.6) is the term
Y:(w)(IO(w + ~w)
+ N(w + ~w)IO(w)}.
(4.7)
Denote the learning set of object and noise profiles as Om(w), Nm(w), m = 1, ... ,M.
Then the ensemble average in quantity (4.7) is evaluated, at each w, from these
profiles as
1 M
M
L
IOm(w + ~w)
+ Nm(w + b.w)IOm(w).
(4.8)
m=l
We use the case M = 16 in applications below.
The filters (4.3) are solved for by substituting Eq. (4.2) and (4.4) into Eq. (4.6),
explicitly squaring out the integrand, taking the expectation of each term, and then
using the Euler-Lagrange solution
aE
ay* = 0,
n= 1, ... ,N.
(4.9)
n
Whereas Eq.( 4.6) is quadratic in the filters Yn(w), Eqs.( 4.9) is linear in them. The
result is an N x N set of linear equations that may be inverted. In practice, for the
two-dimensional problem with N = 3, we have 9 linear equations in 9 unknowns
Ymn , which are quickly inverted at each w by a suitable linear inversion technique.
It is found empirically (see below) that the resulting filters Ymn yield good results
when used in reconstruction formula (4.4). On the basis of prior information, one
can see why this is so. Quantity (4.7) is the known correlation between the modulus
data at one frequency and the desired complex signal 0 at another frequency. The
inclusion of many terms of the type of expression (4.7) in Eq. (4.6) represents
64
the building of a great deal of prior information; in particular how the complex
object, and noise, relate statistically to the data. The more filters Yn one uses in
reconstruction formula (4.4), the more information terms (4.7) exist, and hence the
better are the reconstructions.
Because of its insights, we show results for a one-dimensional case N = 1. Now
there is only one filter Y(w) to find. [Note: Empirical use of a single filter did not
.
yield good reconstructions. This was, in fact, the immediate motivation for the
multiple-filter approach (4.4).] Relation (4.6) becomes
e
2
Jdw (IY(IO + NI) - 01
Jdw[YY*{IO + N12) + (101
= JdwE.
2
=
=
)
2
) -
Y{IO
+ NIO*} -
Y{IO
+ NIO)]
(4.10)
Then
: : = Y{IO
+ N12) - (IO + NIO)
=
0
(4.11)
according to Eq. (4.9). The result is a filter,
Y(w) = (IO(w) + N(w)IO(w)) .
(IO(w) + N(w)12)
(4.12)
Notice that this allows for the presence of noise in the data and uses knowledge of
the noise statistics to minimize the reconstruction error. This is one of the strengths
of the approach.
In the absence of noise, result (4.12) takes the interesting form
Y( ) = (IO(w)IO(w))
w
(IO(w)l2).
(4.13)
65
The reconstructing filter is then the correlation of the modulus with the complex
spectrum, relative to the power in the spectrum. The all-important phase part of
YeW) comes from the numerator, where the known correlation of the phase with
the modulus is an input. In summary, then, in the single-filter case the filter is
essentially the average correlation between modulus and phase of the unknown
object. This is an intuitively correct dependence.
4.3
Wiener Phase-Filter Approach: Examples
We show in the following eight figures some examples of the use of the twodimensional version of algorithm (4.4). Figure 4.1 shows the object training set,
a set of 16 views om(x,y),m = 1, ... ,16, of space shuttles. To form the filters
Ymn (Wl,W2) by using Eq.(4.9), the object spectra Om(Wt,w2),m = 1, ... ,16 were
formed from the Om(x,y) inputs, and all necessary averages [see Eq.
(4.7) for
example] were taken over this set of 16 objects. In these examples, for simplicity,
we assumed the noise N(Wl,W2) to be zero. However, we add noise to the actual
data (see below). This made for an acid test of the approach. Would knowledge of
the object correlations alone make the procedure robust to data noise? As it turns
out, the answer is yes.
We assumed a two-dimensional N = 3 case, i.e., nine filters Ymn (Wl,W2). This
was the minimum number of filters needed to attain good reconstructions. The
modulus of each of the constructed filters is shown in Fig. 4.2, and the phases are
66
shown in Fig. 4.3 (all phases re:Bected into the fundamental interval). Although
the phases appear to be random, they actually are appropriate signals for the given
training set (Fig. 4.1). The substitution of random noise for the phase filters, in
one experiment, obliterated the output.
In Fig. 4.4, 16 independent reconstructions are shown that use the fixed filters
Ymn of Fig. 4.2 and 4.3. The modulus of the spectrum of each image in Fig.
4.1 was used, in turn, as a new data set for reconstruction formula (4.4).
As
an example, when the modulus data of upper left-hand object of Fig. 4.1 was
presented to the filters, the output was the upper left-hand image in Fig. 4.4. In
this way, each output in Fig. 4.4 is the reconstruction of modulus data from the
corresponding image in Fig. 4.1. This emphasizes the problem: to represent 16
different objects by the use of only 9 different filters. A comparison of Figs. 4.1
and 4.4 shows that the result are good. Each output is a recognizable rendition of
its ideal image. Notice that there is no blurring present in the outputs. Departure
from the ideal images takes the from of distorted intensity values. This is not a
particularly bothersome effect, because the object edge details are reconstructed
crisply and without geometrical distortion. The objects are easily recognized.
In order to test the effect of data noise on the approach, we added 10% additive
Gaussian noise to the 16 data sets and again reconstructed each. Outputs are shown
in Fig. 4.5. Results are still good. There is hardly any difference from the noise-free
outputs in Fig. 4.4.
67
In Fig. 4.6 we show outputs when 20% noise was added to the data. Now the
backgrounds are becoming noisy enough to start affecting recognition of foreground
object details. However, the objects are still recognizable.
Because the overall approach is linear, it can conceivably be performed in analog
fashion using a coherent Fourier plane processor. [Note that physical realizablility
with respect to finite bandwidth is not a problem here because the filters Yn{W) are
in frequency space and, hence, already have a finite bandwidth, that of the data.]
Any number of incoherent processing arrangements could also be used.
With such applications in mind, it becomes crucial to test whether noise in
the filters, which would now have to be physically constructed, would seriously
bother the outputs. Accordingly, we added 10% additive Gaussian noise to both
the modulus and phase parts of the filters. The resulting outputs are shown in Fig.
4.7. Little degradation has taken place. We conclude that analog use of the filtering
approach is a real possibility.
Finally, we anticipated the use of the approach when part of the frequency space
of data is missing. For example, in astronomical use the telescope might have a
significant central obscuration because of the secondary mirror. Then a central,
circular area of frequencies would be missing from the data set. Essentially, all lowfrequency information would be gone. The filter optimization approach was adapted
to the case of missing frequencies by a simple rule: If a frequency (w
+ m.6.w)
in
relation (4.4) lies inside the missing region, it is replaced by the frequency obtained
68
by radial translation away from the missing frequency by a fixed amount, into
the region of given data. It was hoped that the correlation information provided
by these stand-in frequencies would be sufficient to overcome the missing band of
information. This was, in fact, the case.
It was found that data gaps of considerable size may be overcome in this way.
For example, consider two cases of 10% and 50% central obscuration. Figures 4.8
and 4.9 show the outputs when each central hole of diameter 10% and 50% of the
total passband diameter was deleted from each data set. In Fig. 4.8 the foreground
objects still recognizable, but there are much significant distortions as shown in Fig.
4.9.
4.4
Discussion
The utility of this filtering procedure rests upon statistical prior knowledge of
object and noise classes. Such statistical prior knowledge, gleaned from a learning
set of object and noise profiles, has a counterpart in the deterministic object constraint information that many other phase retrieval approaches require [58]. This
statistical knowledge appears to offer an advantage in that it leads to a noniterative
solution to the problem, whereas deterministic constraints require iterative solutions (e.g., Fienup's approach [58]) for their satisfaction. Also, the recognition that
noise can enter the data at the outset, Eq. (4.2), permits the statistical approach
69
to subdue it in the reconstruction optimally (by the L2 norm). No other approach
to the phase problem seems to acknowledge, and attack, noise in its formulation.
The Wiener approach, however, has its limitations. It presupposes the unknown
object to be a member of the given object class. As we saw in the applications
section, this often leads to good reconstructions. Conversely, the attempted reconstruction of modulus data for an object that does not belong to the given object
class is not usually successful.
70
~k.
..
~.:~~o~
,~~
....... :.. .
....... .
,
,
. .
....- - A L
. ."
'
.
.~.'
.....
~.
.
'~"".'
-.'
-.
"
....,,0'.,.'.
Figure 4.1: Object training set.
,~
71
Figure 4.2: The nine modulus filters. (The origin is the upper left corner, and the
folding frequency is the center of each filter.)
Figure 4.3: The nine phase filters. (The origin is the upper left corner, and the
folding frequency is the center of each filter.)
73
Figure 4.4: Nine-filter outputs (no noise).
74
Figure 4.5: Nine-filter outputs (10% noise).
75
Figure 4.6: Nine-filter outputs (20% noise).
76
Figure 4.7: Output, with 10% noise in filters.
77
Figure 4.8: Output, from data with 10% obscuration.
78
Figure 4.9: Output, from data with 50% obscuration.
79
CHAPTERS
PHASE RECONSTRUCTION OF TURBULENT
IMAGES
5.1
Introduction
In optical astronomy the angular resolution of an imaging system is approximately
iJ, where). is the wavelength of the light used and D is the diameter of the
aperture. For the 5 meter (200 inch) telescope, the above relation gives a resolution of 0.02 arc second at a wavelength of 400 nm. However, this resolution is not
even approached by the telescope because of the atmospheric turbulence. The typical resolution obtained is approximately 1 arc second. This resolution is roughly
characteristic of all large earth-based telescopes. This limitation is the result of
atmospheric turbulence, which causes the incoming wavefronts to be distorted in
a random manner so that most light is scattered into a disc much larger than the
Airy disc. One arc second corresponds to the Rayleigh resolution for the 25 cm (10
inch) optical aperture. The primary reason for building telescope larger than 25 cm
has been the collection of more light so that dimmer objects may be detected. But
80
it is difficult to imagine a telescope with a diameter greater than about 50 m in the
foreseeable future. [58]
For a few decades there has been increasing interest in the problem of how to
compensate for the effects of atmospheric turbulence on images. To overcome the
problem of limited resolution, the method of high-resolution imaging, so-called stellar speckle interferometry, has been developed. The basis of these approaches is the
Van Cittern-Zernike theorem, which, in the form of use in astronomy, states that
the spatial coherence function in the far field of a thermal light source is proportional to the Fourier transform of the object intensity. The speckle interferometry
invented by Labeyrie [66] gives only an autocorrelation of the object. Therefore,
the reconstruction of a real image using data obtained by speckle interferometry
has been studied extensively [67, 68, 69, 63, 70, 71].
High angular resolution imaging can be achieved by various interferometric methods in spite of image degradation by the atmospheric turbulence. Methods of reconstructing high angular imaging are generally divided into two main approaches.
One method is phase retrieval using numerical algorithms. [67, 68, 69] A characteristic of methods using numerical algorithms is that the procedure can be easily
performed for a two-dimensional image. However, for the obtained solution, the
uniqueness and convergence properties in the algorithms are not proved under the
given condition. In addition, the effect of various amounts of noise existing in the
process in those methods is not sufficiently known. The second approach, based
81
on phase retrieval using the mathematical properties of the Fourier modulus of
a physical object, [63, 70, 71] mathematically ensures the uniqueness of the solution of a reconstructed image, but it is generally difficult to apply to actual image
reconstruction because of high sensitivity to noises involved in observed images.
We shall review Labeyrie speckle interferometry, shift-and-add method, the KnoxThompson method, and the bispectrum method. Finally, we shall give the development of some new reconstruction methods based on Fourier image division. Before
proceeding with the treatment of restoration methods, it will be useful to present
some theory relevant to the formation of images degraded by turbulence.
5.2
Image Formation Theory
Here we shall now derive some basic deterministic relations governing image
formation. They may be generalized to two dimensions in an obvious way. The
imaging problem we are considering is depicted in Fig. 5.1 [35].
In Fig 5.1 a point source of light, of wavelength >., is located at infinity along the
optical axis of the given lens. Spherical waves emanate from it, but after traveling
the infinite distance to the lens they appear plane, as illustrated at the far left. The
lens has a coordinate y which relates to a reduced coordinate j3 by
j3 = ky/F
(5.1)
82
where k = 27r / >., and F is the distance from the lens to the image plane. At each
coordinate {3 of the pupil there is a phase distance error Ll from a sphere whose
radius is F, which would perfectly focus light in the image plane.
Huygen's principle states that each point of the wavefront just to the right of
the lens acts like a new source of spherical waves. This defines a complex pupil
function
(5.2)
at each point (3 of the pupil.
The focused image has an intensity profile s(x) called the point spread function
which is located in the image plane. This may be represented in terms of a point
amplitude function a(x) as
(5.3)
Quantity a(x) relates to the pupil function Eq. (5.2) by Fraunhofer's integral [72J
a( x) =
l
f30
d{3 ei[k~(f3)+f3xl,
(5.4)
-f3o
which is basically a finite Fourier transform of the pupil function. Eq. (5.4) is
proved by representing a(x) as a superposition integral over the pupil of Huygen's
waves r- 1 eikr , where r is the distance from a general point {3 on the wavefront in
the pupil to a general point x in the image plane. Since
r
~
F
+ Ll by Taylor series where (F + Ll) » Iy - xl.
r2
= (y - x)2
+ (F + Ll)2,
It is assumed that the lens
has a compensating phase k y 2/2F and x is close to the optical axis.
83
Finally, the optical transfer function T{w) is defined as basically the Fourier
transform of s(x),
(5.5)
where
(5.6)
By combining Eq. (5.3), Eq. (5.4), and Eq. (5.5), we obtain T(w), the autocorrelation of the pupil function,
T(w) = (2.80)-11{30
d.8 ejk[~(/3)-~(/3-w)].
(5.7)
w-/3o
5.3
Basic Speckle Interferometry
Speckle interferometric imaging is the a set of techniques which astronomers use
to attempt to overcome the blurring of astronomical images due to atmospheric
turbulence. In optical astronomy, the traditional way to obtain an object map or
object intensity is to use a single large telescope to record conventional long exposure
images. This long exposure imaging might cause much high frequency information
to be lost. In 1970, Labeyrie developed a method for obtaining diffraction limited
information while utilizing the full aperture of a telescope. The method is based
on processing a large number of short exposure images. If a stellar object is bright
enough, it can be imaged by the telescope during a time exposure of 10 msec or less.
This causes atmospheric turbulence, which changes at a slower pace, to be effectively
frozen. The result is a phase function across the telescope pupil that is temporally
84
constant during the exposure but spatially random. Such a pupil function causes,
via Eq.(5.3) and Eq.(5.4), a point spread function s(x,y) that is said to suffer
from short-term turbulence. Typically, such a spread function consists of randomly
scattered blobs or speckles. Suppose that many such short-term exposures of the
object are taken. Since each causes randomly a new pupil phase function to be
recorded, s( x, y) must be considered a stochastic process. Since all the while one
object is being observed, the object o( x, y) is fixed. In mathematical terms, one
estimates the power spectrum < II{w)12 > of the image intensity i(x,y), where
<> denotes the ensemble average, from finite frames. It can be shown that the
image power spectrum is related to that of the object, IO(w)l2, through the speckle
transfer function
< IT{w)12 >
< II{wW >= IQ(wW < IT(w)12 > .
(5.8)
Hence, Eq. (5.8) allows IQ(wW to be found. The modulus IO(w)1 of the unknown
object is then known.
Although the phase of the spectrum IO(w)1 is still unknown, knowledge of the
modulus IO(w)1 by itself often allows the object o(x,y) to be known by using phase
retrieval algorithms.
An advantage of the Labeyrie technique is that it can be implemented by coherent optical processing methods. The collection of images is recorded photographically. The power spectrum < II(w)12
> can then be formed by making a multiple
exposure of the Fraunhofer diffraction patterns of the successive speckle images. As
85
an example of the study of closely-spaced binary stars, consider the case where the
object is a double star of equal intensities B and separation a. Then parameters a
and B are all that are needed to specify the object. Moreover, since for this object
o(x)
= B8(x -
a/2) + B8(x + a/2), we have IQ(w)12
= 4B2 cos2(aw/2).
Or, IO(w)12
will be a set of fringes. Then a can be estimated from the frequency of the fringes,
while B is known from their contrast.
5.4
Shift-and-Add Method
One of the earliest methods proposed to retain high spatial frequency information
in astronomical images is simple shift-and-add. The technique works just as might
be expected
• The brightest point in each of a series of short exposure images is located
• The brightest point is relocated to a reference point
• The shifted image is added to the summed shifted images
This technique is based on the assumption that a speckle image can be modeled
as the convolution of a set of delta functions with the object. The idea is due to
Lynds et al. [73] and has been used with notable success over the years. A similar
technique, called weighted shift-and-add [74], locates a number of maxima in each
image. This technique has met with even greater success than simple shift-and-add.
As might be expected, the shift-and-add techniques have a number of drawbacks.
86
The most serious of these is the fact that shift-and-add seems to produce a object
dependent point spread function. This clearly would make accurate reconstruction
of the object very difficult.
5.5
Knox-Thompson Method
We now examine a method that has considerable potential for overcoming atmospheric turbulence in astronomical imaging. The algorithm by Knox and Thompson
[63] is the first real candidate for overcoming atmospheric turbulence, because it can
handle large phase fluctuation of several waves and also it overcomes the problem of
noise propagation in the phase estimates. The Knox and Thompson (KT) method
is an ad hoc addition to Labeyrie speckle interferometry. In the KT method, the
modulus II(w)1 of the image spectrum is obtained by the Labeyrie method and the
phase of the image spectrum is obtained by averaging the product of closely spaced
frequency components.
Suppose now that N short exposure images in(x,y), n
= 1,··· ,N of one inco-
herent object o( x, y) are given. These are Fourier transformed
(5.9)
The spectra are then auto correlated at 2 or 3 lags b.w, forming associated data
1
D(w; b.w) = N
N
L
n=l
In(w)I:(w
+ b.w)
(5.10)
87
over the passband w
~
n
of the imaging system. This quantity D(w; .6.w), unlike
the power spectrum, is complex and retains phase information. The notation
*
indicates the complex conjugate.
By the transfer theorem each spectrum obeys
In(w) = Tn(w)O(w),
n = 1,···, N
(5.11)
where Tn(w) is the n'th stochastic transfer function of the atmosphere, and O(w)
is the fixed object spectrum. Substituting Eq. (5.11) into Eq. (5.10) gives
D(w; .6.w) = MKT(W; .6.w)O(w)O*(w + .6.w)
(5.12)
where
1
MKT(W; .6.w)
N
= N 2: Tn(w)T;(w + .6.w)
(5.13)
n=l
is the net transfer function linking the associated data function D( w; .6.w) with the
object information O(w)O*(w + .6.w).
It is assumed that transfer function MKT can either be observed by imaging
a nearby point source, (e.g., a star or a glint), or is known from theory. Then
representing each complex spectrum O(w), O(w
+ .6.w)
by its modulus and phase
function
O(w) = /O(w)/ei,p(W)
(5.14)
in Eq. (5.12), allows the all-important object phase function <p(w) to be estimated,
as
<p(w) - <p(w + .6.w)
= Phase[D(w; .6.W)jMKT(W; .6.w)].
(5.15)
88
Now the phase of D(w;bow) may be computed from the data. Also, the phase of
MKT(W; bow) can be known form the image of the an isolated star. This allows the
unknown object phase difference ¢>(w) - ¢>(w + .6.w) to be found in Eq. (5.15), and
is the touchstone of the approach.
The phase ¢> at any particular vector w may be found by noting that ¢>(O, 0) = 0,
since o( x, y) is a real function, and by stepwise using Eq. (5.15) to find ¢>( bow ),
then ¢>(2.6.w), etc., until w is reached. In fact, being a two-dimensional problem,
there are any number of paths in frequency space that can be taken to produce ¢>
at the required w. Each path should give rise to the correct ¢>(w) value. Therefore,
in the case of noisy data, the different path values may be averaged to effect some
gain in accuracy.
The modulus O(w) part of the object may more easily be obtained, by the use
of Labeyrie's approach. Then the modulus and phase functions are combined, via
Eq. (5.14), to form the final spectrum, which is Fourier transformed to get the final
estimated object.
5.6
Bispectrum Method
The bispectrum is defined as the product of the short exposure image transforms
at three spatial frequencies
(5.16)
89
where
WI
and W2 are two-dimensional spatial frequency coordinates and where
*
denotes the complex conjugate. Also, the triple correlation can be represented by
its inverse Fourier transform as
(5.17)
where we use the notation z = (x,y),
Zl
= (x},yt), and
Z2
= (X2,Y2).
Since the image is a convolution of a object with a point spread function, then
in frequency space,
lew) = O(w)T(w)
(5.18)
where O( w) equals the spatial spectrum of the object intensity and T( w) the instantaneous optical transfer function. Using Eq.(5.18)
(5.19)
The average bispectrum of many short exposures is related to that of the object
through a bispectrum transfer function T 3 (wt, W2) as
(5.20)
It is clear that
(5.21 )
where (h(wt, W2) represents the phase of the average image bispectrum, cPT(WI, W2)
represents the phase of the bispectral transfer function and cPo (WI, W2) is the phase
90
of the object spectrum. It has been stated in the literature [70] that h(WJ,W2)
is very small if not equal to zero. Also, if it is assumed that complex amplitude
in the telescope pupil has Gaussian statistics, then it can be shown [75] that the
bispectrum transfer function is real and nonzero over the diffraction-limited portion.
Thus, from the above discussion,
(5.22)
where ¢o(w) represents an estimate of the object phase <l>o(w) at frequency w.
To see how the phase of the object spectrum can be calculated recursively from
that of the object bispectrum, consider Eq. (5.22) with
WI
=
(0,0) and W2 =
(0,1), (0,2),'0 o. In fact, this process can be repeated for the different values of WI
to yield a more accurate estimate of the phase at each spatial frequency. Since
¢o(O,O) equals to zero for a real object, ¢o(O, 1) and ¢o(l, 0) define the phase slope
and are free to be chosen, the recursive scheme can be started. This method is in
some ways similar to, but more general than Knox-Thompson method described
before.
5.7
5.7.1
Turbulent Image Reconstruction from a Superposition Model
Superposition Process
In many physical processes, data are formed as the random superposition [35]
over space, or over time, of one function. The latter is called a disturbance function
h(x,y) and has a known form. For example, we found that short-term atmospheric
91
turbulence causes the image of a star to be a random scatter of Airy-like functions.
Each Airy-like function has about the same form h(x, y), and these are centered
upon random points (xm' Ym). Let there be M fixed disturbances in all. Accordingly,
we postulate the superposition process to obey the following statistical model:
• A known disturbance function h( x, y) exists.
• It is centered upon m random positions (xm' Ym), m ;: 1, ... ,_'l;f.
• The disturbances add, so that the total disturbance with the weights Wm at
any point (x,y) obeys
M
s(x,y);:
L
wmh(x - Xm,Y - Ym).
(5.23)
m=l
• The {(xm,Ym)} are random variables that are independent samples from one,
known probability law PXy(x, y).
From the last model assumption and Eq. (5.23), s(x,y) is a random variable at
each (x,y). In fact,
(5.24)
so that s(x, y) is a stochastic process. We are interested in establishing the statistics
of s at any fixed point (x, y) = (xo, yo). But this problem has been already solved
in detail on Frieden's book [35]. We will consider only the parameter estimation of
the weights Wm and displacements (xm' Ym)
92
5.7.2
Theory
It has often been noted [71] that, in the presence of short-term turbulence, the
image of a star resembles a random superposition of speckles of approximately constant size. This is also theoretical justification for such a model [76]. The model
underlies the previously described shift-and-add algorithm [73] for reconstructing
objects consisting of impulses. In this section, we show that the model, in principIe, permits the efficient reconstruction of general object scenes. Application to a
particular case is also given, with encouraging results.
Model the short-term speckle point spread function s(x,y) as a superposition
process,
M
s(x,y) =
2: wmh(x -
xm,y - Ym).
(5.25)
m=l
In statistical parlance, a disturbance function h( x, y) of known form, e.g., a Gaussian, is assumed present. The weights
Wm
and displacements
(Xm,
Ym) are stochas-
tic, i.e., unpredictable from exposure to exposure. Thus, each short-term spread
function is presumed to be characterized by randomly a different set of
Wm
and
The model (5.25) is, of course, an approximation. The shape of each disturbance
function h(x,y) will vary somewhat with exposure number n. Hence, the function
h(x,y) used in Eq. (5.25) may be regarded as an average entity. In any real case,
the departure of the model (5.25) from the true s(x,y) may be regarded as a noise
term.
93
The model (5.25) is used to form reconstructions as follows. Assume that N
short-exposure images i n ( x, y), n = 1,2,···, N of a constant object scene o( x, y)
are at hand. These obey the linear transform theorem
(5.26)
of the n'th image in(x,y), n'th spread functions sn(x,y), and the object o(x,y).
(For a survey of past methods of reconstructing O(WI,W2) from knowledge of the
can be found, for anyone image n. We accomplish the latter as follow.
Taking the Fourier transform of Eq. (5.25) gives
T(Wll W2)
= H(Wll W2) L
w me-i (wIXm+W2Ym) ,
(5.27)
m
where H(Wl,W2) is the Fourier transform of h(x,y). This is the imaging characteristic of the n-th short-term image, or
Tn(WI, W2) = H(Wb W2)
L wmne-i(wIXmn+W2Ymn).
(5.28)
m
We had to double subscript the unknowns because there is a new set of unknowns
Wm and (xm' Ym) for each image n. Using result Eq. (5.28) gives for the ratio of
two images nand k,
I n(Wl,W2)
Tn(WllW2)O(Wl,W2)
Lm wmne-i(WIXmn+W2Ymn)
Ik(wI, w2) = Tk(WI,W2)O(Wl,W2) = Lm wmke-i(WIXmk+W2Ymk)·
(5.29)
Remarkably, both the object and the disturbance function drop out. This is quite
convenient, since the object is of course unknown, and the disturbance function
94
input would necessarily suffer from error, which conceivably would propagate into
the outputs
Wmn., Xmn.,
and
Ymn..
For a fixed pair (n, k) of images, Eq. (5.29) has 2M unknowns
4M unknowns
Xmn.
Wmn., Wmk,
plus
and Ymn., or a total of 6M unknowns. However, Eq. (5.29) holds
at each frequency (WI,W2) value. This enables many equations in the unknowns
to be formed. If, e.g., the image space is 64-by-64 pixels, by the use of the Fast
Fourier Transform, the frequency space will likewise contain 64-by-64 pixels, for a
total of 4096. Thus, 4096 equations in the unknowns could be formed. If, as well,
there are the order of a few hundred speckles, we see that an overdetermined set of
equations is produced. Seeking a least-square solution then enables the unknowns
to be found. With these known, the turbulence point spread functions of Eq. (5.25)
for the two images nand k are known. Then use of Eq. (5.28) for each image gives
two estimates of the object. These may be averaged to produce the output object.
It may be noted that this procedure holds for a general object intensity profile
o(x, y)j the details of the object spectrum O(Wl,W2) dropped out during the division
in Eq. (5.29). In particular, the approach is not limited to impulse-type object
scenes.
95
5.7.3
Experimental Results
Suppose a binary star is modeled as a pair of point sources separated by some
pixel distance. The object function o( X, y) can be represented by the equation
(5.30)
where a and b are the intensities of the point sources at (Xl, Yl) and (X2, Y2), respectively. The Fourier transform of the object O(Wl,W2) is easily found to be
(5.31)
The image i (x, y) can be formed by telescope during a short-term exposure of
the object o(x, y). The Fourier spectrum J(WI,W2) is obtained by Fourier Transform
of image i(x,y). Thus, it is assumed that estimates are available for the image
spectrum J(Wl,W2) and the transfer function T(WI,W2) in Eq. (5.27).
The object Fourier spectrum O(Wl,W2; 0) then is a function of unknown parameters ()
= (wmn,xmmYmn)
in Eq. (5.31). The problem is to find a parameter set
such that the figure of merit functional S(fJ), a sum over the independent variables
WI and W2 which range over the passband,
(5.32)
is minimized as a function of (). It is clear that the equation for the object Fourier
spectrum is non-linear. Any convenient algorithm for seeking a least-square solution
96
to the problem may then be used. Though many algorithms exist for the leastsquare optimization problem at hand, the Levenberg-Marquardt algorithm for nonlinear least squares is common choice and will be employed here. It has become
one of the most successful and widely used algorithms for nonlinear optimization.
It was briefly described in the Appendix A.
Using Eq. (5.29) takes the ratio of the image spectrum for image number n = 1
to that for image number k = 2,
(5.33)
where M = 4 disturbances to each. For a fixed pair of images, Eq. (5.33) has 8
unknowns
W mn ,
16 unknowns
Xmn
and
Ymn,
or total of 24 unknowns.
For the first test of the algorithm the initial guesses for the parameters are
=
1.0,
W21
= W22 =
0.8,
W31
= W32 =
0.6,
W41
= W42 =
0.4,
Wn =
W12
(xn,Yn)
=
(0.0,0.0),
(X21,Y2d
-
(8.0, -2.0),
(X3b Y31)
=
(-7.0, -7.0),
(X41' Y4d
=
(0.0,8.0),
(X12' Y12)
=
(6.0,0.0),
97
(X22, Y22) -
(0.0, -5.0),
(X32, Y32) -
(-4.0,0.0),
(X42,Y42) =
(-1.0,7.0).
This first starting point was chosen to roughly correspond to what will be seen to
be the answer to the problem. The choice is based on experience with running the
algorithm. The problem of how many iterations to perform is of some importance.
Since the algorithm will iterate for some time after the function value S(O) has
ceased to change, some stopping criterion should be used. The criterion that will
be employed here is based on the function value S(O). Thus if S(Od is the function
value at the
ith
iteration, then if
(5.34)
the iterations are halted. The stopping criterion of 1 x 10- 2 is somewhat arbitrarily
chosen since it is based on experience with running the non-linear least squares
routine. In the absence of a more rigorous method, the criterion appears to be
entirely sufficient.
For othe!" trials, three different starting points were chosen. These starting points
are summarized in Table 5.1. When dealing with non-linear least squares, an important question to ask is how sensitive are the results to the starting point. If more
prior knowledge such as the good starting point are used, the local minimum can be
avoided and convergence can be speeded up.
follows.
A computer-based demonstration
98
Trials
Wn
W21
W31
W41
W12
W22
W32
W42
(xu, Yu)
(X21,Y21)
(X3b Y31)
(X4b Y41)
(X12, Y12)
(X22,Y22)
(X32, Y32)
(X42, Y42)
1
0.9
0.7
0.5
0.3
1.1
0.9
0.7
0.5
(1.0, 1.0)
(7.0,-3.0)
(-6.0,-8.0 )
(1.0,9.0)
(7.0,1.0)
(1.0,-6.0)
(-3.0,1.0)
(-2.0,7.0)
2
0.88
0.83
0.7
0.5
0.92
0.75
0.55
0.46
(-0.7,0.3)
(8.7,-1.8)
(-6.2,-6.5 )
(-0.5,8.9)
(5.3,0.7)
(0.5,-4.2)
(-3.6,0.4)
(-1.5,6.7)
3
0.8
0.6
0.4
0.3
0.85
0.65
0.45
0.35
(2.0,2.0)
(10.0,0.0)
(-5.0,-5.0)
(2.0,10.0)
(8.0,2.0)
(2.0,-3.0)
(-2.0,2.0)
(-3.0,9.0)
Table 5.1: The three different starting points
In Fig. 5.2(a) is shown an ideal single-point object, and Fig. 5.2(b) is a twopoint object with the left-hand impulse at half the intensity of the right-hand one.
The aim will be to reconstruct each object from two short-exposure images. This
constitutes two distinct reconstruction problems. The fields are 64 x 64 pixels in
area. In Fig. 5.2(c) and 5.2(d) are images of the one-point object (hence, these are
the same two spread functions), with M = 4 disturbances to each. Fig. 5.2(e) and
5.2(f) show the images of object Fig. 5.2(b) due to the same two spread functions.
The two images Fig. 5.2(c) and 5.2(d) are input into algorithm Eq. (5.29). Since
M
= 4,
there are 6M
= 24
unknown weights and displacements to recover. We
found it efficient to use the 20 x 20 central frequencies (WI, W2) in Eq. (5.29)., for a
total of 400 equations in the 24 unknowns. This is very overdetermined system of
99
equations. A least-square solution was found using a Levenberg-Marquardt algorithm. The output is the object in Fig. 5.3( a). It is seen to be a good approximation
to the ideal single-point object input Fig. 5.2(a).
Next, the two images Fig. 5.2(e) and 5.2(f) are input into algorithm Eq. (5.29).
Again there are 24 unknown weights and displacements, and we again use the 20 x 20
central frequencies (Wt,W2) in Eq. (5.29). The output is shown in Fig. 5.3(b). Again
it is a good approximation to the ideal two-point object Fig. 5.2(b).
The Levenberg-Marquardt approach requires a first-guess at the unknown weights
and displacements. These were randomly generated. To test the sensitivity of the
approach to starting solution, we repeated the algorithm for the last problem, but
with two randomly different initial solutions. The outputs are shown in Fig. 5.3(c)
and Fig. 5.3( d). They are good approximations to the true object Fig. 5.2(b).
It may be noted that the reconstruction approach avoids the need for observation of a reference point source in the image field. Also, it requires but two
short-exposure images as input. In these regards, the approach offers an advantage over some past approaches to reconstruction, e.g. due to Labeyrie or KnoxThompson [63].
5.8
Discussion
A few conclusions can be drawn from this chapter. The turbulent image reconstruction from a superposition model has some advantage. The previously described
100
approaches in Section 5.3, 5.4, 5.5 and 5.6 require, for implementation, either empirical or theoretical knowledge of the ensemble-average image of a point source.
They also, in practice, require N = 20 or more short-term images for successful resolution of object details. Most importantly, the proposed approach avoids the need
for these inputs. The object is reconstructed without the need for a reference point
source in the image field, and by the observation of but 2 short-exposure images. A
possible way out of the problem of determining a good disturbance function would
be that it could be regarded as an average speckle intensity profile.
It may be noted that the proposed approach holds for a general object intensity
profile o(x, y); the details of the object spectrum O(Wl' W2) dropped out during the
division in Eq. (5.29). In particular, the approach is not limited to impulse-type
object scenes.
101
O.A.
Wavefront
Lens
Image Plane
Figure 5.1: Formation of point spread function s(x) from lens suffering phase errors
1::.((3).
102
(a)
(b)
(d
(d)
(e)
(f)
Figure 5.2: Short-term images formed (to be processed). (a) Ideal one-point object;
(b) Ideal two-point object; (c) Turbulent image (psf) of (a); (d) Another image (psf)
of (a); (e) Image of (b) via psf (c); (f) Image of (b) via psf (d).
103
(a)
(c)
(b)
(d)
Figure 5.3: Outputs of image modelling algorithm. (a) Output o(x,y) of algorithm
based upon data F.T.{Fig. 5.2(c)}/F.T.{Fig. 5.2(d)}. (b) Output o(x,y) based
upon data F.T.{Fig. 5.2(e)}/F.T.{Fig. 5.2(f)}. Use of first starting solution. (c)
Output o(x,y) as in (b), using second starting solution. (d) Output o(x,y) as in
(b), using third starting solution.
104
CHAPTER 6
CONCLUSIONS AND FUTURE RESEARCH
This chapter summarizes the main results presented in this dissertation and
provides recommendations for further research.
In chapter three, one- and two-dimensional integral logarithmic transforms were
defined. These transforms obey the useful properties of linearity and scale invariance. The two-dimensional integral logarithmic transform is additionally invariant
to rotation. The one-dimensional transform is analogous to an optical edge response
function or to a cumulative probability law. The one-dimensional LT of a function
undergoing a nonlinear, power-law change of scale is itself a power-law change of
scale on the unscaled log transform. The inverse one-dimensional LT was found, as
were the inverse two-dimensional transforms. Any scaled version of a given image
has the same LT. This allows scale-invariant Wiener filtering and character recognition to be performed. A computer simulation of scale-invariant Wiener filtering
was carried through, showing good noise suppression st two different scales of magnification. An optical implementation of scale-invariant character recognition was
proposed.
105
We have proposed but a few applications of the concept of the LT. The transform
should find a host of other applications, in optics as well as in other fields of engineering. For example, a polar-log coordinate 1:1 mapping is a preprocessing step L
that obeys Eq. (3.40) and that allows for scale- (and orientation-) invariant Wiener
filtering (Messner and Szu [47]). This preprocessing would probably be easier to
implement digitally than LT filtering since it is a simple point-to-point mapping.
On the other hand, since polar-log coordinate mapping is a non-linear operation,
the linear LT outputs Eq. (3.15) might be easier to analyze for noise-propagation
and signal-to-noise effects. Regarding analog implementation, LT might have an
advantage because of the simplicity of the proposed optical implementation in Fig.
3.1. Further comparisons of the two approaches would require detailed studies of
such factors as operational complexity, speed, cost, convenience, and accuracy.
In chapter four, we showed that the concept of the Wiener filter can be extended
to the phase-retrieval-from-modulus problem. However, in contrast to conventional
Wiener filtering, more than one filter is needed to achieve high-quality outputs. For
example, a 3 x 3 array of Wiener filters is needed to reconstruct successfully one
class of objects - 16 space shuttle images. The approach was found, by computer
simulation, to be relatively insensitive to noise in the modulus data and in the
filters. Also, considerable gaps in the data passband, such as those from a central
obscuration in the imaging telescope, can be overcome by suitable modification of
the rules of filter construction.
106
As further research, consider an object that is a translated, magnified, and rotated version of an object in the learning set. Such an object can probably be
accommodated by a modification of the filtering approach, as follows. The object's
autocorrelation function is invariant to translation, and suffers the same magnification and rotation effects as the object itself. This suggests that we Fourier transform the object's squared modulus data, producing the object's autocorrelation
function, and then take its integral logarithmic transform. The two-dimensional integrallogarithmic transform has the property that is invariant to the magnification
and rotation distortions of its input, except for a translation that depends on the
magnification and rotation. Hence, the transformed data merely suffer from lateral
translation. A subsequent inverse-Fourier transform and then modulus operation
eliminates the translation and places the output back in frequency space. If this
modulus data is now used in place of IQ(w)1 in the development Eqs. (4.2) - (4.13),
the result should be a set of filters Yn (w) optimized for use on this transformed
data. The filters should then work on data arising from any translated, scaled, and
rotated version of a shuttle image as in Fig. 4.l.
In Chapter five, we reconstructed one-point and two-point objects without the
need for a reference point source in the image field, and with the observation of but
2, short-exposure images. The Knox-Thompson approach depends, for their use,
upon the observation of many (20 $ N) short-term images of a point source, in
addition to the N images. The proposed arproach required number N of images. It
107
uses but N = 2 of these, working from their quotient in frequency space, Eq. (5.29).
Each point spread function s(x,y) is modeled [Eq. (5.25)] as a superposition, with
unknown weights and displacements, of a known disturbance function h( x, y). Since
the quotient is known over a wide band of frequencies, this enables the weights and
displacements for two short-term spread functions sn(x,y), Sk(X,y) to be solved
for in a least-squares sense. Once these are known, the object spectrum may be
solved for, by the use of a transfer function in conjunction with either image. Thus,
reconstruction is attained by the use of two images instead of 20 or more, and the
need for observation of a reference point source is replaced by modeling the point
spread function appropriately and working with the division of two image spectra.
A case M = 4 disturbances corresponds to a weak turbulence situation where,
effectively, the optical aperture contains about 4 regions of correlated phase. Work
needs to be continued to extend the application of the algorithm to cases of stronger
turbulence, where M
~
100 disturbances are more common.
108
APPENDIX A
LEVENBERG-MARQUARDT OPTIMIZATION
The following section is a brief summary of Levenberg-Marquardt optimization
algorithm. Some of optimization routines used in this dissertation are from the
Numerical Recipes mathematical subroutine library. Many of the same optimization algorithms are also available through IMSL. The IMSL routines are strictly a
black box, but the Numerical Recipes [77] text contains a very readable overview
of optimization. A more detailed review is found in series of articles edited by
Murray [78].
Optimization is the branch of mathematics which attempts to find the maximum
or minimum of a given function. These extrema may be either local or global
with global extremization being the more difficult of the tasks. The optimization
considered here will be restricted to finding the minima of a function. For the
applications considered here, moreover, the class of functions will be restricted
to ones composed of sums of squares. Specifically, if F( x) is the function to be
minimized then
M
F(x)
= 2]fi(x)]2
;=1
(.1 )
109
where the sum is over the M data points where the function is evaluated. The
function F(x) is dependent on N variables
Xj
which are represented by the N-
dimensional vector x. In addition, all the functions considered will be non-linearly
dependent on the variables x j.
Proofs for the convergence of optimization algorithms are often based on an
assumption that the function to be optimized can be approximated by a quadratic
form. Quadratic form means that the function F( x) can be represented by
(.2)
where b is a N x 1 vector and A is a N x N symmetric matrix. The representation
of F(x) as a Taylor series is an important example of a quadratic form.
The
approximation to F( x) is then written as
(.3)
In this case the elements gi of 9 are the first partial derivatives
aF(x)
aXi
(.4)
•
Likewise, the elements Gij of G are the second partial derivatives
(.5)
The matrix G is often referred to as the Hessian
III
the literature. When the
function to be minimized is the sum of squares then a useful approximation to the
Hessian may be derived. The approximation is found by differentiating the function
110
with respect to each of the
Xi.
The result is a sum composed of first and second
derivatives. The second derivative terms can be neglected because the terms are
small [79] or because the second derivative terms tend to cancel out [77].
The algorithm proposed by Levenberg [80] and Marquardt [81] has become one
of the most successful and widely used algorithms for nonlinear optimization. The
algorithm is actually a clever combination of two other optimization schemes. The
first scheme, steepest descent, is one of the oldest methods in existence and was
first suggested by Cauchy in 1847. Suppose 9 is the N x 1 gradient vector of F(x)
which has as
ith
element
gi =
of(x)
(.6)
OXi
Suppose further that F( x) is well approximated by the quadratic form. Then the
steepest descent iteration is defined as
(.7)
where
xk
represents the values of the variables at the k-th step and where h is
some constant. The second algorithm combined in Levenberg-Marquardt is often
referred to as Newton-Raphson. The method is generated by differentiation of the
N-dimensional Taylor series approximating F(x). The procedure which results is
the iterative step
(.8)
111
where G- 1 is the inverse of the Hessian matrix. Combining these two algorithms,
the Levenberg-Marquardt algorithm [80, 81] is thus defined as the iterative step
(.9)
where I denotes the identity matrix. The form of the algorithm depends on the
constant ..\ at each step. Thus when ..\ is large, the algorithm behaves more like
steepest descent. When). is small, the algorithm resembles Newton-Raphson. The
constant ..\ is initially set to a small value (like 0.001). If during the iteration, the
function value increases, ). is increased by a factor of 10. If the function value
decreases, then). is decreased by a factor of 10.
112
REFERENCES
[1] R. A. Schowengerdt, Techniques for Image Processing and Classification zn
Remote Sensing. New York: Academic press, 1983.
[2] E. Persoon and K. S. Fu, "Shape discrimination using Fourier descriptors,"
IEEE Trans. Syst., Man, and Cybern., vol. SMC-7, pp. 170-179, Mar. 1977.
[3] K. S. Fu, Syntactic Pattern Recognition and Application. Englewood Cliffs,
NJ: Prentice-Hall, 1982.
[4] S. A. Dudani, K. J. Breeding, and R. B. McGhee, "Aircraft identification by
moment invariants," IEEE Trans. Comput., vol. C-26, no. 1, pp. 39-45, Jan.
1985.
[5] A. P. Reeves, R. J. Prokop, S. E. Andrews, and F. Kuhl, "Three-dimensional
shape analysis using moments and Fourier descriptors," IEEE Trans. Pattern
Anal. Machine Intell., vol. PAMI-10, no. 6, pp. 937-943, Nov. 1988.
[6] R. C. Gonzalez and P. Wintz, Digital Image Processing.
Addison-Wesley, 1977.
Reading, MA:
[7] H. Wu and R. A. Schowengerdt, "Shape discrimination using invariant Fourier
representation and a neural network classifier," SPIE, vol. 1569, pp. 147-155,
July, 1991.
[8] A. Krzyzak; S. Y. Leung, and C. Y. Suen, "Reconstruction of two-dimensional
patterns by Fourier descriptors," in Proc. 9th ICPR, Rome, Italy, pp. 555-558,
Nov., 1988.
[9] C. T. Zhan and C. T. Roskies, "Fourier descriptors for plane closed curves,"
IEEE Trans. Comput., vol. C-21, pp. 269-281, Mar. 1972.
113
[10] R. L. Kashyap and R. Chellappa, "Stochastic models for closed boundary
analysis: Representation and reconstruction," IEEE Trans. Inform. Theory,
vol. IT-27, pp. 627-637, Sept. 1981.
[11] Y. N. Hsu, H. H. Arsenault, and G. April, "Rotational invariant digital pattern
recognition using circular harmonic expansion," Appl. Opt., vol. 21, pp. 40124015, 1982.
[12] R. P. Lippmann, "An introduction to computing with neural nets," IEEE
ASSP Magazine, vol. 4, pp. 4-22, April 1987.
[13] T. Kohonen, Self-organization and Associative Memory.
Verlag, 1984.
Berlin: Springer-
[14] G. M. Edelman, Neural Darwinism. NY: Basic Books, 1987.
[15] K. Fukushima, "Neural network model for selective attention in visual pattern
recognition and associative recall," Appl. Opt., vol. 26, pp. 4985-4992, 1987.
[16] D. E. Rumelhart and J. L. McClelland, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Cambridge, MA: MIT Press,
1986.
[17] E. C. Titchmarsh, Introduction to the Theory of Fourier Integrals. Oxford:
Oxford Univ. Press, 1959.
[18] R. Bracewell, The Fourier Transform and its Applications.
McGraw-Hill Book Co., 1965.
New York:
[19] E.1. Schwartz, "Computational anatomy and functional architecture of striate
cortex: A spatial mapping approach to perceptual coding," Vision Res., vol. 20,
pp. 645-669, 1980.
[20] E. 1. Schwartz, "Cortical Mapping and perceptual invariance: a reply to cavanagh," Vision Res., vol. 23, pp. 831-835, 1983.
[21] R. A. Altes, "Mechanism for aural pulse compression in mammals," J. Acoust.
Soc. Amer., vol. 57, pp. 513-515, 1975.
[22] R. A. Altes, "The Fourier-Mellin transform and mammalian hearing," J.
Acoust. Soc. Amer., vol. 63, pp. 174-183, 1978.
114
[23] D. Casasent and D. Psaltis, "New optical transforms for pattern recognition,"
Proc. IEEE, vol. 65, pp. 77-84, 1977.
[24] B. R. Frieden and C. Oh, "Integral logarithmic transform: theory and applications," Appl. Opt., vol. 31, pp. 1138-1145, 1992.
[25] J. R. Fienup, "Phase retrieval algorithms: a comparison," Appl. Opt., vol. 21,
pp. 2758-2769, 1982.
[26] R. W. Gerchberg and W. O. Saxton, "Phase determination from image and
diffraction plane pictures in electron microscopy," Optik, vol. 34, pp. 275-284,
1971.
[27] R. W. Gerchberg and W. O. Saxton, "A practical algorithm for the determination of phase from image and diffraction plane pictures," Optik, vol. 35,
pp. 237-246, 1972.
[28] D. 1. Misell, "An examination of an iterative method for the solution of the
phase problem in optics and electron optics: I. test calculations," J. Phys.,
vol. D 6, pp. 2200-2216, 1973.
[29] D. 1. Misell, "An examination of an iterative method for the solution of the
phase problem in optics and electron optics: II. sources of error," J. Phys.,
vol. D 6, pp. 2217-2225, 1973.
[30] R. A. Gonsalves, "Phase retrieval from differential intensity measurements,"
J. Opt. Soc. Am., vol. A 4, pp. 166-170, 1987.
[31] N. Nakajima, "Phase retrieval from two intensity measurements using the
Fourier series expansion," J. Opt. Soc. Am., vol. A 4, pp. 154-158, 1987.
[32] N. Nakajima, "Phase retrieval using the logarithmic Hilbert transform and
Fourier series expansion," J. Opt. Soc. Am., vol. A 5, pp. 257-262, 1988.
[33] B. R. Frieden and C. Oh, "Multiple-filter approach to phase retrieval from
modulus data," Appl. Opt., vol. 31, pp. 1103-1108, 1992.
[34] B. R. Frieden and C. Oh, "Turbulent image reconstruction from a superposition
model," Opt. Commun., 1993.
115
[35] B. R. Frieden, Probability, Statistical Optics, and Data Testing. Heidelberg:
Springer-Verlag, 2nd ed., 1991.
[36] A. V. Oppenheim and J. S. Lim, "The importance of phase in signals," Proc.
IEEE, vol. 69, no. 5, pp. 529-541, 1981.
[37] R. G. Lane, W. R. Fright, and R. H. Bates, "Direct phase retrieval," IEEE
Trans. Acoust., Speech, Signal Processing, vol. ASSP-35, pp. 520-526, 1987.
[38] D. Casasent and D. Psaltis, "Space-bandwidth product and accuracy of the
optical Mellin transform," Appl. Opt., vol. 16, p. 1472, 1977.
[39] D. Casasent and D. Psaltis, "Position, rotation, and scaling invariant optical
correlation," Appl. Opt., vol. 15, pp. 1795-1799, 1976.
[40] D. Casasent and M. Kraus, "Polar camera for space-variant pattern recognition," Appl. Opt., vol. 17, pp. 1559-1561, 1978.
[41] 1. R. Rabiner and B. Gold, Theory and Application of Digital Signal Processing. Englewood Cliffs: Prentice-Hall, 1975.
[42] 1. Mertz, Transformations in Optics. New York: John Wiley & Sons, 1965.
[43] W. T. Rhodes, J. R. Fienup, and B. E. Saleh, "Transformations in Optical
Signal Processing," SPIE, vol. 373, 1983.
[44] H. Wechsler, Computational Vision. San Diego: Academic Press, 1990.
[45] C. Weiman and G. Chaikin, "Logarithmic spiral grids for the image processing
and display," Computer Graphics and Image Processing, vol. 11, pp. 197-226,
1979.
[46] P. S. Schenker, K. M. Wong, and E. G. Cande, "Fast adaptive algorithms for
low-level scene analysis: Application of polar exponential grid (IPEG) representation to high-speed scale and rotation invariant target segmentation,"
SPIE, vol. 281, pp. 47-57, 1981.
[47] R. A. Messner and H. H. Szu, "Simultaneous image processing and feature
extraction for two-dimensional non-uniform sensors," SPIE, vol. 449, pp. 693710, 1983.
116
[48] R. A. Messner, "Smart visual sensors for real-time image processing and pat-
tern recognition based upon human visual system characteristics," Ph.D. dissertation, Clarkson Univ., Potsdam, N.Y., 1984.
[49] E. L. Schwartz, "Spatial mapping in the primate sensory projection: Analytic
structure and relevance to perception," BioI. Cybern., vol. 25, pp. 181-194,
1977.
[50] A. VanderLugt, F. B. Rotz, and J. A. Kloester, "Character reading by optical spatial filtering," in Optical and Electro-Optical Information Processing,
Cambridge, Mass.: MIT Press, 1965.
[51] T. Szoplik and H. H. Arsenault, "Shift and scale-invariant anamorphic Fourier
correlation using multiple circular harmonic filters," Appl. Opt., vol. 24,
pp. 3179-3183, 1985.
[52] D. Casasent, S. Xi a, A. Lee, and J. Song, "Real-time deformation invariant optical pattern recognition using coordinate transformations," Appl. Opt., vol. 26,
pp. 938-942, 1987.
[53] J. Rosen and J. Shamir, "Scale invariant pattern recognition with logarithmic
radial filters," Appl. Opt., vol. 28, pp. 240-244, 1989.
[54] D. Mendlovic, N. Konforti, and E. Marom, "Scale and projection invariant
pattern recognition," Appl. Opt., vol. 28, pp. 4982-4986, 1989.
[55] Y. Sun, Z. Wang, and G. Mu, "Amplitude compensated matched filters using circular harmonic expansion and a Mellin transform," Appi. Opt., vol. 29,
pp. 4i79-4783, 1990.
[56] D. Mendlovic, N. Konforti, and E. Marom, "Scale and projection invariant pattern recognition using logarithmic harmonics," Appi. Opt., vol. 29, pp. 47844789, 1990.
[57J D. G. Currie, "On a detection scheme for an amplitude interferometer," in
Synthetic-Aperture Optics (J. W. Goodman, ed.), Defense Documentation Cen-
ter, Alexandria, Virginia, p. Appendix II in Volume 2, Woods Hole Summer
Study, 1967.
117
[58] J. C. Dainty and J. R. Fienup, "Phase Retrieval and Image Reconstruction for
Astronomy," in Image Recovery: Theory and Application (fl. Stark, ed.), ch. 7,
Orlando: Academic Press, 1987.
[59] H. A. Ferwerda in Inverse Source Problems (H. P. Baltes, ed.), ch. 2, Berlin:
Springer-Verlag, 1978.
[60] L. C. Liu and S. H. Risbud, "Analysis of TEM image contrast of quantum-dot
semiconductor clusters in glasses," Phil. Mag. Lett., vol. 61, pp. 327-332, 1990.
[61] J. G. Walker, "The phase retrieval problem: A solution based on zero location
by exponential apodization," Opt. Acta, vol. 28, pp. 735-738, 1981.
[62] B. R. Frieden and D. G. Currie, "On unfolding the autocorrelation function,"
J. Opt. Soc. Am., vol. 66, p. UllA, 1976.
[63] K. T. Knox and B. J. Thompson, "Recovery of images from atmospherically
degraded short exposure photographs," Astrophys. J. Lett., vol. 193, pp. L45L48, 1974.
[64] A. W. Lohmann, G. Weigelt, and B. Wirnitzer, "Speckle masking in astronomy:
triple correlation theory and applications," Appl. Opt., vol. 22, pp. 4028-4037,
1983.
[65] N. Wiener, Extrapolation,lnterpolation, and Smoothing of Stationary Time Series. New York: Wiley, 1949.
[66] A. Labeyrie, "Attainment of diffraction limited resolution in large telescopes
by Fourier analyzing speckle patterns in star images," Astron. and Astrophys.,
vol. 6, pp. 85-87, 1970.
[67] J. R. Fienup, "Space object imaging through the turbulent atmosphere," Opt.
Eng., vol. 18, pp. 529-534, 1979.
[68] G. R. Ayers and J. C. Dainty, "Iterative blind deconvolution method and its
applications," Opt. Lett., vol. 13, pp. 547-549, 1988.
[69] R. H. T. Bates, "A stochastic image restoration procedure," Opt. Commun.,
vol. 19, pp. 240-244, 1976.
118
[70] G. Weigelt and B. Wirnitzer, "Image reconstruction by the speckle masking
method," Optics Lett., vol. 8, pp. 389-391, 1983.
[71] J. C. Dainty, "Laser Speckle and Related Phenomena," in Topics in applied
physics (J. C. Dainty, ed.), Heidelberg: Springer-Verlag, 2nd ed., 1984.
[72] J. W. Goodman, Introduction to Fourier Optics. New York: McGraw-Hill,
1968.
[73] C. R. Lynds, S. P. Worden, and J. W. Harvey, "Digital image reconstruction
applied to Alpha Orions," Astrophys. J., vol. 207, pp. 174-180, 1976.
[74] J. C. Christou, E. K. Freeman, and E. Ribak, "A self-calibrating shift-and-add
technique for speckle imaging," J. Opt. Soc. Am., vol. A3, pp. 204-209, 1986.
[75] A. W. Lohmann, G. Weigelt, and B. \Virnitzer, "Speckle making in astronomy:
triple correlation theory and applications," Appl. Opt., vol. 22, pp. 4028-4037,
1983.
[76] F. Roddier, "The effect of atmospheric turbulence in optical astronomy," in
Progress in optics, Amsterdam: North-Holland, 1981.
[77] W. H. Press, B. P. Flannery, S. A. Teukolsky, and W. T. Vetterling, Numerical Recipes in C: The Art of Scientific Computing. Cambridge: Cambridge
university press, 1988.
[78] W. Murray, "Fundamentals," in Numerical Methods in Unconstrained Optimization (W. Murray, ed.), New York: Academic Press, 1972.
[79] D. R. Sadler, Numerical Methods for Nonlinear Regression. St. Lucia, Queensland: University of Queensland Press, 1975.
[80] K. Levenberg, "A method for the solution of certain problems in least squares,"
Quarterly of Applied Mathematics, vol. 2, pp. 164-168, 1944.
[81] D. Marquardt, "An algorithm for least-squares estimation of nonlinear parameters," SIAM Journal on Applied Mathematics, vol. 11, pp. 431-441, 1963.
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement