Enhancement of X-ray Fluoroscopy Image Sequences using Temporal Recursive Filtering and

Enhancement of X-ray Fluoroscopy Image Sequences using Temporal Recursive Filtering and
Enhancement of X-ray Fluoroscopy Image
Sequences using Temporal Recursive Filtering and
Motion Compensation
Anni Forsberg
2006-03-07
LiTH-IMT/MI20-EX--06/427--SE
Enhancement of X-ray Fluoroscopy Image
Sequences using Temporal Recursive Filtering and
Motion Compensation
Examensarbete utfört i medicinsk bildbehandling
vid Tekniska högskolan i Linköping
av
Anni Forsberg
LiTH-IMT/MI20-EX- -06/427- -SE
Handledare:
Mats Andersson
imt, Linköpings universitet
Bo Wärnberg
Swemac Medical Appliances
Examinator:
Hans Knutsson
imt, Linköpings universitet
Linköpings tekniska högskola
Institutionen för medicinsk teknik
Avdelningen för medicinsk informatik
Department of Biomedical Engineering
Division of Medical Informatics
Rapportnr:
LiTH-IMT/MI20-EX--06/427--SE
Datum: 2006-03-07
Svensk
titel
Förbättring av fluoroskopibildsekvenser med användande av temporal rekursiv
filtrering och rörelsekompensering
Engelsk
titel
Enhancement of X-ray Fluoroscopy Image Sequences using Temporal Recursive
Filtering and Motion Compensation
Författare
Anni Forsberg
Uppdragsgivare:
Rapporttyp:
Rapportspråk:
Swemac Medical Appliances/
IMT
Examensarbete
Engelska
Sammanfattning
This thesis consider enhancement of X-ray fluoroscopy image sequences.
The purpose is to investigate the possibilities to improve the image
enhancement in Biplanar 500, a fluoroscopy system developed by
Swemac Medical Appliances, for use in orthopedic surgery.
An algorithm based on recursive filtering, for temporal noise
suppression, and motion compensation, for avoidance of motion
artifacts, is developed and tested on image sequences from the
system. The motion compensation is done both globally, by using
the theory of the shift theorem, and locally, by subtracting
consecutive frames. Also a new type of contrast adjustment is
presented, received with an unlinear mapping function.
The result is a noise reduced image sequence that shows no
blurring effects upon motion. A brief study of the result shows,
that both the image sequences with this algorithm applied and the
contrast adjusted images are preferred by orthopedists compared to
the present images in the system.
Nyckelord
Keywords
Fluoroscopy, image enhancement, recursive filtering, motion compensation.
Bibliotekets anteckningar:
Abstract
This thesis consider enhancement of X-ray uoroscopy image sequences. The
purpose is to investigate the possibilities to improve the image enhancement
in Biplanar 500, a uoroscopy system developed by Swemac Medical Appliances, for use in orthopedic surgery.
An algorithm based on recursive ltering, for temporal noise suppression,
and motion compensation, for avoidance of motion artifacts, is developed
and tested on image sequences from the system. The motion compensation
is done both globally, by using the theory of the shift theorem, and locally,
by subtracting consecutive frames. Also a new type of contrast adjustment
is presented, received with an unlinear mapping function.
The result is a noise reduced image sequence that shows no blurring eects
upon motion. A brief study of the result shows, that both the image sequences with this algorithm applied and the contrast adjusted images are
preferred by orthopedists compared to the present images in the system.
i
Acknowledgements
First of all I want to thank the employees of Swemac Medical Appliances, for
being helpful and friendly and for letting me do my thesis on image sequences
from their Biplanar 500 system. Special thanks are given to Bo Wärnberg
for supporting me with material and explaining the functions of the system
and to Lars Granath, who helped me get in contact with the orthopedists.
Secondly I want to thank my supervisor Mats Andersson for giving me guidance and many ideas. My examiner Hans Knutsson has also been available
for valuable discussions.
Thanks are also given to the orthopedists at the University Hospital in
Linköping, for letting me attend surgeries, and to the orthopedists at the
Hospital in Nyköping, who gave me their opinions on my results.
I also want to thank Soe Åberg for acting as opponent to my work and
Jenny Hammarfors for proofreading the report during development. Thank
you both for fruitful discussions and nice company during this period.
Finally I want to thank David Bäck for being supportive during my ups and
downs and for discussing and proofreading the thesis.
Anni Forsberg
Linköping, March 2006
iii
Contents
1 Introduction
1.1
1.2
1.3
1.4
Background . . . . . . . . . .
Purpose and task description
Resources and limitations . .
Outline . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
2 Overview of X-ray uoroscopy imaging
2.1
2.2
2.3
Introduction . . . . . . . . . . . . . . . . . .
The uoroscopy chain . . . . . . . . . . . .
2.2.1 X-ray generation and the X-ray tube
2.2.2 The image intensier . . . . . . . . .
2.2.3 The CCD-camera . . . . . . . . . . .
2.2.4 Image processing and presentation .
The uoroscopy image . . . . . . . . . . . .
3 The Biplanar 500 system
3.1
3.2
3.3
3.4
System components . . . . . . . . . .
The acquisition into Matlab . . . . .
Input signal properties . . . . . . . .
3.3.1 Interlaced video . . . . . . . .
3.3.2 Spectrum of the input signal
3.3.3 Noise aspect . . . . . . . . . .
Present image processing . . . . . . .
4 Usability aspects
4.1
4.2
4.3
4.4
The human observer
Attending surgery .
The areas of interest
Test image sequences
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
5 Theory of the Fourier transform
5.1
5.2
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
The 2D Discrete Fourier Transform . . . . . . . . . . . . . . .
Convolution and the convolution theorem . . . . . . . . . . .
v
1
1
1
2
2
5
5
6
6
7
8
9
9
11
11
13
14
14
16
17
17
21
21
22
22
24
31
31
32
5.3
5.4
The shift theorem . . . . . . . . . . . . . . . . . . . . . . . . .
The phase correlation method . . . . . . . . . . . . . . . . . .
6 Theory of image enhancement
6.1
6.2
6.3
6.4
6.5
Interpolation methods . . . . . . .
Thresholding . . . . . . . . . . . .
Morphological operations . . . . .
Contrast adjustment . . . . . . . .
6.4.1 Gray scale mapping . . . .
Noise reduction . . . . . . . . . . .
6.5.1 Averaging . . . . . . . . . .
6.5.2 Median ltering . . . . . . .
6.5.3 Temporal averaging . . . .
6.5.4 Temporal recursive ltering
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
7 Implementation and results
7.1
7.2
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Method . . . . . . . . . . . . . . . . . . . . . . .
Preprocessing algorithms . . . . . . . . . . . . . .
7.2.1 RGB to gray scale transformation . . . . .
7.2.2 Cropping . . . . . . . . . . . . . . . . . .
7.2.3 Windowing . . . . . . . . . . . . . . . . .
7.3 Contrast adjustment . . . . . . . . . . . . . . . .
7.4 Deinterlacing . . . . . . . . . . . . . . . . . . . .
7.5 Temporal recursive ltering . . . . . . . . . . . .
7.6 Temporal averaging . . . . . . . . . . . . . . . . .
7.7 Global motion compensation . . . . . . . . . . . .
7.7.1 Calculation of translation . . . . . . . . .
7.7.2 Adapting the temporal impulse response .
7.7.3 Shifting the impulse response . . . . . . .
7.7.4 Width of the impulse response . . . . . .
7.8 Local motion compensation . . . . . . . . . . . .
7.9 An alternative method to detect global motion .
7.9.1 Dividing frames into overlapping sections
7.9.2 Representing shifts in vector plots . . . .
7.9.3 The mean-std method . . . . . . . . . . .
7.10 The combined algorithm . . . . . . . . . . . . . .
8 Evaluation study
8.1
8.2
8.3
8.4
Method . . . . . . . . . . . . . . . . .
Manual contrast adjustment . . . . . .
Comparison between dierent mapping
functions . . . . . . . . . . . . . . . . .
Noise reduction and blurring . . . . . .
vi
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
33
34
37
37
38
38
39
40
41
42
42
43
44
47
47
48
48
48
49
51
52
53
55
56
56
57
58
60
60
63
63
63
64
65
67
67
67
68
70
9 Discussion
9.1
9.2
Contrast adjustment . . . .
The proposed algorithm . .
9.2.1 Global motion . . .
9.2.2 Local motion . . . .
9.2.3 The accuracy of shift
9.2.4 Limitations . . . . .
9.2.5 Time requirements .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
calculation
. . . . . . .
. . . . . . .
10 Conclusion
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
10.1 Short summation . . . . . . . . . . . . . . . . .
10.2 Recommended algorithm for Biplanar 500 . . .
10.3 Future work . . . . . . . . . . . . . . . . . . . .
10.3.1 Further investigation before usage of the
A The combined algorithm
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
. . . . . .
. . . . . .
. . . . . .
algorithm
.
.
.
.
.
.
.
.
75
75
77
77
78
79
80
80
81
81
81
82
82
91
vii
Abbreviations
CCD
DFT
FD
FFT
FIR
FT
HP
IIR
LP
NN
SD
VFM
VCC
Charged Coupled Device
Discrete Fourier Transform
Fourier Domain
Fast Fourier Transform
Finite Impulse Response
Fourier Transform
High Pass
Innite Impulse Response
Low Pass
Nearest Neighbor
Spatial Domain
Vision for Matlab
Video Capture Cable
Glossary
eld
One frame is built up of 2 elds, one consisting of
odd pixel rows and one of even pixel rows.
frame
An image sequence is built up of a number of frames,
i.e. images that correspond to a specic point of time.
In this thesis, the frequency used is 25 frames per second.
interlaced
A video format where the dierent elds of one frame
are shown at dierent times.
progressive
A video format where the dierent frames are shown
consecutively.
pixel
The smallest picture element, with the position (x,y )
in the image matrix. In this thesis, every pixel has a gray value
between 0 and 255, i.e. a depth of 8 bits.
ix
Chapter 1
Introduction
This chapter will introduce the reader to this thesis by giving a background
to and a description of the task. In the end an outline is found to guide the
reader through the thesis.
1.1 Background
This thesis consider enhancement of the images from a Biplanar 500, a digital
X-ray uoroscopy system, used in orthopedic surgery. The system is developed by Swemac Medical Appliances AB, a company which produce and sell
medical equipment. Most of this work has been done at Swemac's oce in
Linköping, Sweden. The thesis is done as a part of the Master of Science in
Applied Physics and Electrical Engineering at Linköping University.
The Biplanar 500 system shows continuous image sequences, so that the
orthopedic surgeon can get images of the area of interest in real-time during operation. Today the image enhancement in the system contains noise
suppression, edge enhancement (on 2D snapshots only) and contrast adjustment. The orthopedists using the system experience the images as noisy and
think that the contrast adjustment is poor.
1.2 Purpose and task description
The purpose of this thesis is to investigate the possibilities to improve the
image enhancement in Biplanar 500. It is interesting for Swemac to know
if better images can be received and in that case, how much it diers from
today's quality.
1
CHAPTER 1. INTRODUCTION
2
The task is to develop an algorithm that increase the contrast, reduces noise
and compensates for motion, with preservation of edges and lines, in a 3D
image sequence.
To do this, the task has been divided into three major areas:
• Testing of the existing system and evaluation of the input signal
• Developing algorithms for continuous video (3D) in Matlab
• Evaluating the results in the meaning of image quality and time requirements
where the main focus is put on development of algorithms. Since Matlab
cannot process in real-time, the image quality is of higher priority than a
fast algorithm.
1.3 Resources and limitations
A Biplanar 500 system, a PC with Windows XP and a VCC (Video Capture
Cable) are provided. The image enhancement algorithms are developed apart
from the current image processing in the system, since the source code is not
known. The algorithms are created in the calculation programme Matlab
with the toolboxes Signal Processing and Image Processing. Implementation
of the algorithms in the Biplanar 500 system is not included. The images
used are hip images, a common application for the Biplanar 500 system. The
algorithms are not tested on other images.
1.4 Outline
In chapter 2 a brief overview of X-ray uoroscopy imaging is given. The
function of dierent components in the uoroscopy chain are explained. In
chapter 3 the Biplanar 500 system is described, properties of the input signal
are investigated and the results from system tests are presented. Chapter 4
consider the human observer and why it is important to get an orthopedists
opinion about the images. The areas of interest in the images are specied
and the test sequences are presented and motivated.
Chapter 5 consider the theory of the Fourier transform and the phase correlation method. In chapter 6 some basic image operations are described and
the image enhancement methods used in the algorithms are presented.
In chapter 7, a description is given of how the algorithms have been developed. The theory from chapter 5 and 6 is here applied to the specic
CHAPTER 1. INTRODUCTION
3
problem. The results are presented as well as solutions to the problems
arising.
Chapter 8 presents an evaluation study of the results. The results are discussed in chapter 9, in the meaning of image quality and time requirement.
Finally, in chapter 10, conclusions are drawn and recommendations are given
to Swemac. Also future work is considered.
Chapter 2
Overview of X-ray uoroscopy
imaging
In this chapter an overview of X-ray uoroscopy is given. Some elds of
application are mentioned and the basic theory of X-ray generation and the
image chain are presented.
2.1 Introduction
X-ray uoroscopy is a diagnostic method used to get X-ray images of a
patient in real-time. It is often used to get a quick overview of the patient
and the X-ray eld so that changes can be made to the position of the patient
and the collimation of the eld. [10]
The real-time aspect also makes it possible to study the motions of organs, for
instance the pulsations of the heart. Another application is in angiography,
where a contrast agent spreads along the blood vessels. A foreign object like a
nail can be seen clearly and the fracture of a bone can easily be reponed. [10]
This is why uoroscopy often is used in orthopedic surgery to get information
about a fractured bone's position when putting it together with nails.
Since the excitations of the X-ray tubes in uoroscopy are longer than in
conventional X-ray investigations, the dose to the patient is much higher.
Therefore it is important to get images of good quality fast to get as low
dose as possible to the patient. Low dose implies noisy images and thus this
is often the main problem to solve in uoroscopy image enhancement. [16]
To reduce the dose to the patient and the surgeon, pulsed uoroscopy can
be used but this method is not further treated in this thesis.
5
CHAPTER 2. OVERVIEW OF X-RAY FLUOROSCOPY IMAGING
6
2.2 The uoroscopy chain
General uoroscopy systems consist of an X-ray tube that generates X-rays,
an image intensier, a CCD-camera that works as a detector and a computer
that processes the image and presents it on a screen to the user. [10]
A schematic overview of the uoroscopy chain is given in gure 2.1. The
subsections below cover the function of these dierent parts.
Figure 2.1: Schematic overview of the uoroscopy chain.
2.2.1 X-ray generation and the X-ray tube
The X-ray generation in a uoroscopy system is made in the X-ray tube. It
is an evacuated tube with a high potential dierence from one side to the
other. [3, 10] An overview of the X-ray tube is given in gure 2.2.
Figure 2.2: The X-ray tube and its components.
On one side there is a positively charged anode with a target media. The
CHAPTER 2. OVERVIEW OF X-RAY FLUOROSCOPY IMAGING
7
target media is characterized by a high atomic number, to maximize the Xray generation, and a high melting point, to manage the heat that will occur
on the anode surface. To further cool the anode, it often rotates so that the
heat get spread over a bigger area. [3, 10]
On the other side is the negatively charged cathode, surrounding a glowing
lament that gets heated up by a tube current. When the tube current is
owing through the lament, electrons are emitted. The free electrons are
then focused in the electric eld to the focusing spot on the anode. [3, 10]
When the electrons hit the target atoms on the anode, two things may
happen. In 99 % of the cases, heat is generated from the kinetic energy
of the electrons. The remaining part of the electrons, about 1 %, generates
X-rays. This is done via bremsstrahlung or characteristic radiation. [5, 3]
For further reading of these processes, see for instance [5].
The X-rays generated are emitted in all directions but only the ones in a
specic angle, see gure 2.2, are emitted out from the tube. There is a thin
window, forming the X-ray beam that can be used in the uoroscopy chain.
[5]
The variable parameters of the X-ray tube are the voltage of the electrical
eld (kV), the time of excitation (s) and the tube current (mA). They are
correlated as
E = mA · s · kV 2
(2.1)
where E is the amount of emitted X-rays from the tube. When increasing
the voltage, the electrons have more energy and therefore hit the anode in
a higher speed. This will cause a higher number of emitted X-rays, since
the higher energy increase the probability of X-ray generation. The X-rays
generated are of an increased energy. Increasing the current generates an
increased number of electrons emitted from the lament. The X-ray generation will then increase in the same amount. The excitation time of course
also have an inuence, the longer excitation, the more X-rays are generated.
[5, 10]
2.2.2 The image intensier
The image intensier is an evacuated tube that transform X-rays into visible light and intensies the image in a few steps. The potential dierence
between the input and output screen is approximately 25-30kV. [5]
In gure 2.3 the components of the image intensier are shown.
CHAPTER 2. OVERVIEW OF X-RAY FLUOROSCOPY IMAGING
8
Figure 2.3: The Image Intensier and its components.
The X-ray beam hits the input uorescent screen, often made of CsI1 , doped
with Na. The uorescent screen absorbs the X-rays photons and visible
light photons are emitted in proportion to the incident X-rays. Next to
the uorescent screen is a photocathode where electrons are pushed out via
photo electrical eect. The photocathode is made of a material that match
well with the spectral output from the CsI screen. [5]
The electrons are then accelerated throughout the tube in the electrical eld
caused by the potential dierence. The electrons are focused with positively
charged electrodes (focusing elements) to the output uorescent screen, often
made of ZnCdS2 , doped with Ag. At the output uorescent screen, electrons
are transformed into photons. [5]
The acceleration of the electrons increases their energy and more light is
then emitted. The focusing makes the output image even more intensied,
since the output screen is much smaller than the input screen. The input
screen and photocathode are convex to make sure that all the electrons has
the same length of transportation to the output screen. The image received
on the output screen is upturned and reversed in relation to the image on
the input screen. [5]
2.2.3 The CCD-camera
A digital detector often used in digital uoroscopy systems is the CCDcamera. The advantages with CCD-cameras are their linear characteristics
and small dimensions. They also have a fast recovering after exposure. [12]
1
Cesium
2
Iodide
Zinc-Cadmium Sulde
CHAPTER 2. OVERVIEW OF X-RAY FLUOROSCOPY IMAGING
9
The CCD is a solid-state chip camera, consisting of an array with photodiodes. The light incident to every photodiode excite valence electrons to the
conduction band. This leads to an electric charge for every diode, proportional to the number of photons. Each diode correspond to one pixel in the
image, but is not read directly. Instead, the charges of the diodes are read,
one row at a time, to an analogous video signal. [12]
The scanning of values can be done in dierent ways, either progressive,
or interlaced. See section 3.3.1 for further explanation of the two dierent
methods. Often the signal intensity is adjusted by a logarithmic step, to
better suit the visual system of the human observer. This is done in the
camera of the Biplanar 500 system.
2.2.4 Image processing and presentation
The image sequence from the CCD-camera is processed in a computer. This
processing can for instance reduce noise, cut the important parts out or
enhance the edges and lines. This is where the algorithms produced in this
thesis t into the uoroscopy chain. The processed images are then presented
to the human observer. The human observer will be discussed further in
section 4.1.
2.3 The uoroscopy image
The uoroscopy image is essentially built up in the same way as an X-ray
image and has the same appearance. The X-ray image is a mapping of
the objects ability to attenuate X-rays. An image occur since the density
diers between dierent tissues in the body. Bones for instance, have a high
density, while soft tissue, e.g. fat or muscles, has a much lower density. The
consequence is that the image gets dark in the bone-areas (low signal) and
lighter in the soft tissue areas (high signal).
The appearance of the image can be changed by adjusting the X-ray tube
parameters to the current object examined. A high voltage gives a low dose
to the patient but will cause a larger amount of scattering. This is often used
when bones are to be represented, sucient contrast is given to the image
anyway. [5, 10]
When representing soft tissue, a much lower voltage is needed to visualize
the contrast between dierent soft tissues, who's density dierence is quite
small. One can use a contrast agent, that have a dierent density, to get
better contrast with a higher voltage. This will decrease the dose, since more
CHAPTER 2. OVERVIEW OF X-RAY FLUOROSCOPY IMAGING
10
photons are getting through the object. This is done in angiography where
the subject of interest is the blood ow. [5, 10]
In uoroscopy, the thickness often vary in dierent parts of the patient examined. This may result in an image that is under- or overexposed in some
areas. [10]
Chapter 3
The Biplanar 500 system
This chapter describes the Biplanar 500 system. The components, the properties of the input signal, the present image processing and test results are
here accounted for.
3.1 System components
The G-stand of Biplanar 500, see gure 3.2, include two sets of X-ray tubes,
image intensiers and CCD-cameras to get images from two dierent views
(frontal and sagittal), without having to move the stand between the series.
According to [10], at least two perpendicular images have to be taken to set
the exact position of a bone fracture and that is the main idea behind the
construction of the G-stand.
During excitation, the output from the CCD-camera leads via a frame grabber, that digitizes the signal, to a computer. Here, the present image processing is done. The result is shown on two big screens, one for each view. There
is also a user interface, where the orthopedist can write information about
the patient, control the excitation of the X-ray tubes, choose which image
enhancement to use and store images to les.
Figure 3.1 gives a schematic overview of the Biplanar 500 system (for one
view). Figure 3.2 shows the Biplanar 500 system and its components. Since
some parts of the Biplanar 500 system, including the image processing, are
not made by Swemac and thus unknown, a number of tests are done to get
a brief view of the prerequisites of the task.
11
CHAPTER 3. THE BIPLANAR 500 SYSTEM
12
Figure 3.1: Schematic overview of the Biplanar 500 system (for one view).
Figure 3.2: The Biplanar 500 system. Left: The G-stand with X-ray
tubes, image intensiers and CCD-cameras marked. Right: The two viewing
screens and the user interface.
CHAPTER 3. THE BIPLANAR 500 SYSTEM
13
3.2 The acquisition into Matlab
To get the video signal into the computer a VCC is used. Grabbing of the
frames into MatLab is done with the free software VFM (Vision For Matlab).
In gure 3.3 the arrangement of acquisition is shown.
Figure 3.3: Schematic overview of the acquisition into Matlab.
This arrangement has been used both when testing the system and for grabbing the test sequences described in section 4.4.
CHAPTER 3. THE BIPLANAR 500 SYSTEM
14
3.3 Input signal properties
The resolution of the video from the CCD-camera is 576x720 pixels and the
frame rate is 25 frames per second. An example of an input signal, an image
of a hip with a nail inserted, is shown in gure 3.4.
Figure 3.4: Example of an input image: A hip with a nail. kV=79, mA=2.5
As one can see, the image has rather similar gray values in dierent parts,
both in the bone tissue and in the soft tissue. This is because the surgeon
needs to see the dierence between the compact and the spongy bone tissue.
Thus the X-ray voltage and current has been chosen so that these borders
are as clear as possible.
3.3.1 Interlaced video
In interlaced video, the odd and even pixel rows in one frame are drawn at
dierent times. Two elds, one consisting of the odd and one of the even
rows, build up one frame, see gure 3.5. [8, 12]
This format is often used when the video is to be shown on a TV monitor.
The advantage with interlaced video, is that you can get the impression
of 50 frames/s with only 25 frames/s. The human eye do not notice the
time dierence, thus experience the video as normal. When displaying video
CHAPTER 3. THE BIPLANAR 500 SYSTEM
15
Figure 3.5: The principle of interlaced video.
on a computer monitor, often progressive video is used. Progressive video
draws the whole frame simultaneously, with the frequency of 25 frames/s for
instance. [8, 12]
In the Biplanar 500 system, the image sequences are in interlaced format but
are presented progressive, i.e. frame-wise, on the two viewing monitors. This
leads to interlacing artifacts upon motion. To investigate this phenomenon,
a test was done in the system with a lead disc as test object. The disc
was moved vertically and horizontally in the X-ray eld during acquisition
into Matlab. A distinct interlacing pattern was seen when the disc was
moved horizontal in the X-ray eld. See gure 3.6 for the interlacing pattern.
This will lead to problems during image enhancement, since the artifacts
introduces high spatial frequencies in the wrong directions (vertical in gure
3.6).
250
255
260
265
270
275
280
285
290
295
380
390
400
410
420
430
440
Figure 3.6: Distinct interlacing pattern when moving the disc from right to
left. A high spatial frequency is introduced in vertical direction.
In the Biplanar 500 system the interlaced video draws the even elds before
CHAPTER 3. THE BIPLANAR 500 SYSTEM
16
the odd ones. Since the frame rate is 25 frames/second, an update of elds
is made every 1/50 second.
3.3.2 Spectrum of the input signal
To analyze the input signal, an image and its corresponding spectrum is
studied (the interesting part of the frame is cut out and windowed as will be
described in section 7.2.3). See gure 3.7.
Figure 3.7: An input image and its corresponding spectrum.
In the plot of the spectrum the intensity is concentrated to lower frequencies,
which is the common case for most images. There is also a distinct white
horizontal line, that corresponds to the high contrast nail in the image, which
has a well dened direction.
The representation of the frequencies is not fully circular and maybe that
depends on the limitation in bandwidth in the CCD-camera. It is more
dicult for the camera to handle high frequencies horizontally than vertically
because of the time and scanning aspect. Two consecutive pixels in the same
row are scanned after each other, while two consecutive pixels in the same
column are drawn with one row in between (or in interlaced video in the
next eld). This result in limited frequency in the x-direction, seen as the
spread in the y-direction of the spectrum.
Clear dark vertical borders are also seen. Probably the VCC is the source of
this error. In the next section we will see that the borders are not present
in the spectrum of the output signals from the system.
CHAPTER 3. THE BIPLANAR 500 SYSTEM
17
3.3.3 Noise aspect
The noise in the image chain can appear in the CCD-camera when the photons are collected, in the image intensier or in the cables. [12] The noise
appearing in the camera is assumed to have Poisson distribution and the
other types of noise are random.
Testing the properties of the input signal from the CCD-camera shows that
there is a lot of noise, even when the X-ray tube is not excited. Since no light
is coming from the image intensier unless there is X-rays incident on the
input screen, this noise is only derived from the camera. To get a picture of
how this noise looks, frames are captured when the X-ray tube is not excited.
The noise image and its corresponding spectrum are shown in gure 3.8.
Figure 3.8: The image without X-ray excitation that describes the noise from
the CCD-camera and its corresponding spectrum (DC-component removed).
The noise is spread in the spectrum and not concentrated to lower frequencies
as images usually are. The white dots might imply that there is some sort
of periodic part in the noise.
3.4 Present image processing
To get a hint of how the image processing is made today, a test was done.
By reading images into Matlab and at the same time save the corresponding
output image in the user interface, two images were received that could be
compared. The test was done using an image of a hip with a nail, similar to
the one presented previously in this chapter.
A cutout of the important part was made in the Matlab image, similar to
CHAPTER 3. THE BIPLANAR 500 SYSTEM
18
the cutout made previously. Then the 1024x1024 image from the system
was downsampled to 512x512 so that the images could be compared. The
dierent images are shown in gure 3.9. The image saved in the user interface
was rotated 90 degrees compared to the input signal and could simply be
rotated back when comparing them. The spectrums are shown in gure 3.10.
Figure 3.9: The input image, the original output image and the edge enhanced
output image.
Figure 3.10: The spectra of the input image, the output image with no enhancements and the output image with edge enhancement.
It is interesting that the spectrum of the output signal does not have the dark
areas on the sides as the input spectrum has. This leads to the conclusion
that the stripes originate from the VCC and not from the camera.
Contrast adjustment Unfortunately, a contrast enhanced image can not
be saved in the system. The quality of the contrast adjusted image on the
screen is not good. The contrast reinforcement is too drastic and the image
gets to dark.
Edge enhancement To the right in gure 3.9, the edge enhancement re-
inforces the noise. This is probably the reason why the surgeons are not
satised with the images. In the corresponding spectrum, gure 3.10, the
higher frequencies are reinforced.
Noise suppression The noise suppression is done by averaging a number
CHAPTER 3. THE BIPLANAR 500 SYSTEM
19
of images in the sequence. When no contrast or edge enhancement is used in
combination with the noise suppression, the images are not very noisy. The
more images used in the averaging, the more blurring occurs when an object
is moving.
Late in this thesis there was a discussion with the creator of the present image
enhancement. It turned out that the contrast adjustment is a linear gray
scale transformation inside a specied window. The limits of the window
was computed in a function, that used the lowest and highest gray values.
See chapter 6 for further reading about contrast adjustment.
The present edge enhancement is done with unsharp masking. It is a method
often used in X-ray imaging. First, the image to enhance is LP ltered. To
get the mask, the blurred image is subtracted from the original. The result
is then computed as the mask added to the original. [15] This is equal to
add he HP ltered image to the original, the high frequent components of
the image will be reinforced. The method of unsharp masking will not be
further treated in this thesis.
There is also a pixel based motion detector available in the Biplanar 500
system. If the intensity value in a pixel change a lot from one frame to the
next, it will be considered as motion. This motion detection is not used in
the system today, since it is too sensitive for noise. When using it, motion
is detected much to often and therefore the system never suppresses noise.
Chapter 4
Usability aspects
Dierent people interpret images in dierent ways, depending on the background of the observer. This chapter describes how this is taken into account
in this thesis, to get the best result possible. The areas of interest in the image
are specied. Finally the chosen test sequences are presented and motivated.
4.1 The human observer
We all se dierent things when we look at an image. An X-ray physician
knows exactly where too look for disorders, which lines and areas that are
of importance. An image analyst has, on the other hand, a dierent relation
to noise and contrast and often think that the most important in the image
are the clear edges and lines. An orthopedic surgeon have yet another relationship to the image, since his or her speciality is to operate and to place
nails and drills in the right position. Thus the areas of interest in the image
depend on the observer.
In uoroscopy, the human observer is of big importance, since the images
are interpreted in real-time. The human observer spatially and temporally
processes the image to reduce the eective noise in his or her visual system.
For instance, when looking at a noisy image sequence, often a low contrast
object can be seen until the display stops. This is because the temporal
ltering of noise then stops and the object disappear in noise. [16]
As mentioned in section 2.2.3, images received with a CCD camera are linear,
meaning that the intensity values are proportional to the incident photons.
The human visual system, on the other hand, is logarithmic, i.e. the eye
responds logarithmical to changes in light intensity. [12, 5]
This fact is used in some image processing methods that will not be imple21
CHAPTER 4. USABILITY ASPECTS
22
mented in this thesis. For further reading of the human visual system, see
for instance [16, 7, 1, 14].
Since the subjective judgement is very important in this application it is of
big importance that the result is evaluated by the users. In the evaluation of
the algorithm later on, the resulting image sequences are shown to a number
of orthopedists, see chapter 8.
4.2 Attending surgery
To get a result that is satisfactory in this thesis it is important to achieve
knowledge about what parts of the images that are of interest in the eye of
an orthopedist. Otherwise perhaps some important structures are removed
from the images during processing. To achieve this knowledge and to see
how the Biplanar 500 system is used in reality, an orthopedic surgery at the
University Hospital in Linköping, was attended.
It showed that both the patient and the Biplanar system are moved a lot
during operation, mostly to get a good overview and to nd the specic angle
from which the images give the best information. When drilling a hole for a
nail the patient is still and only the drill is moving. This is useful information
when constructing the motion compensator.
4.3 The areas of interest
After discussion with Swemac and after attending surgery, the conclusion
is drawn that the most important areas in the Biplanar 500 images are the
bone structures. It is desired to improve the edges and lines that separate
the bone from other tissue and the borders between the compact and the
spongy bone tissue. Also an increased contrast is wanted of the nail and drill
borders to distinguish them from the patient.
In the images the joint area are of great importance, since the nail has to
reach the hard parts of the bone but not as far as to the joint area. An
important line which is wanted clearer is called the caput line. The areas of
interest are shown in gure 4.1.
CHAPTER 4. USABILITY ASPECTS
23
Figure 4.1: The important lines and areas in the input images (frontal view).
CHAPTER 4. USABILITY ASPECTS
24
4.4 Test image sequences
To get a good result that can be of interest for Swemac, it was important to
get test image sequences that as much as possible looked like the ones during
operation.
A phantom of a human, see gure 4.2, was used to get image sequences of
the most important area for which Biplanar 500 is used, the hip. Nails and
drills were stuck into the phantom to get a more realistic image, with dark
objects in the main area.
Figure 4.2: The phantom used in the test sequences.
The phantom gives better images than in reality, since the patients often
are old and their body therefore is not as exemplary as the phantom. The
legs are also missing on the phantom, whereas in a real patient there is a lot
of adjustments of the legs before the images cover the right area. However,
since the phantom can be used many times during testing, regardless of dose,
it was chosen to be the best test object.
Test image sequences were grabbed into Matlab at dierent times during this
work. At rst only still image sequences were grabbed, to see if the signal
was ok. Later on, two dierent kinds of sequences were grabbed, sequences
with local motion and sequences with global motion. This was to see how
the algorithms handled motions. During testing of the algorithms, even more
sequences were grabbed.
The test sequences are in both frontal and sagittal view. In table 4.1 below,
the main test sequences, used during development, testing and evaluation,
CHAPTER 4. USABILITY ASPECTS
Seq.
B
View
Frontal
Frames
65
Time
2.6s
L
Sagittal
55
2.2s
F
Frontal
61
2.44s
C
Sagittal
31
1.24s
25
Description
Alternating between still phantom
and fast global motion.
First still phantom,
then slow global motion.
No motion followed by
slow local motion.
Fast local motion.
Table 4.1: Facts about the test sequences used.
are described. The sequences are cropped to the size 256x256 pixels, to get
the algorithms less computationally demanding during development.
Figure 4.3-4.6 show some chosen frames of the sequences. It is relatively
dicult to render image sequences on paper, but hopefully it is sucient to
clarify the examples later on in this report.
CHAPTER 4. USABILITY ASPECTS
26
Figure 4.3: The B test sequence, frontal view. Alternating between still and
fast moving phantom.
CHAPTER 4. USABILITY ASPECTS
27
Figure 4.4: The L test sequence, sagittal view. No motion followed by slow
global motion.
CHAPTER 4. USABILITY ASPECTS
28
Figure 4.5: The F test sequence, frontal view. No motion followed by slow
local motion.
CHAPTER 4. USABILITY ASPECTS
Figure 4.6: The C test sequence, sagittal view. Fast local motion.
29
Chapter 5
Theory of the Fourier
transform
In this chapter the theory of the Fourier transform is presented. The dierent
useful theorems are discussed and the phase correlation method is explained.
5.1 The 2D Discrete Fourier Transform
The Fourier transform is often used in image processing, since important
features of the image, such as orientation, frequency and phase, are often
easier to describe in the Fourier Domain (FD) than in the Spatial Domain
(SD). An image is a 2D discrete and truncated signal, i.e. the signal is only
dened in a nite number of points. [7] To transform a discrete image into
the FD, the discrete fourier transform (DFT) is used, dened as
F{f (x, y)} = F (u, v) =
N
−1 M
−1
X
X
xu
yv
f (x, y)e−j2π( N + M )
(5.1)
x=0 y=0
where f (x, y) is the image in the SD and F (u, v) is the DFT of the image in
the FD, where u, v are frequencies. In this thesis the fast fourier transform
(FFT) will be used to calculate the DFT. It is done by separating the sum
into smaller sums that are not so demanding to compute. The complexity
goes from O(N 2 ) with DFT to O(N logN ) with FFT for N samples. [6]
The DFT is complex and therefore often the power spectrum, |F (u, v)|2 ,
is used instead. The spectrum shows the frequency contents of the image.
The intensity in the center of the spectrum correspond to low frequencies
and further out, higher frequencies are represented. If, for instance, an
31
CHAPTER 5. THEORY OF THE FOURIER TRANSFORM
32
image of a vertical line is transformed, the corresponding spectrum will show
a horizontal line, due to the higher frequency of the borders of the line
horizontally in the image.
When using DFT, artifacts will occur, called wrap-around-errors. They occur
since, in the FD, the image is treated as a periodic signal, meaning that at the
borders, the image is repeated. If the content of the image dier a lot from
the left to the right side, for instance, this will appear as a sharp horizontal
line in the spectrum. [7]
To avoid these artifacts a window can be used; i.e. a weighting function,
that has a unit value in the middle and softens the image down to zero at
the borders. When using a window like this, the representing of the image
content will also be aected but this is often negligible. This windowed DFT
is often the image meant, when referring to a Fourier transformed image. [7]
5.2 Convolution and the convolution theorem
Convolution is an operation performed through shifting a kernel across an
image. The discrete convolution used on images is dened as
h(x, y) = f (x, y) ∗ g(x, y) =
N
−1 M
−1
X
X
f (x − α, y − β)g(α, β)
(5.2)
α=0 β=0
where f (x, y) is the image and g(x, y) is the kernel in the SD. When the kernel
gets outside of the image at the borders, either zero padding or repetition of
the border pixels is used to ll the empty spaces.
The convolution theorem says that convolution in the SD correspond to
multiplication in the FD. Hence,
f (x, y) ∗ g(x, y) ⇐⇒ F (u, v)G(u, v)
(5.3)
where F (u, v) and G(u, v) are the DFTs of f (x, y) and g(x, y) respectively. So
when ltering an image there is always two ways to do the same operation.
Depending on in what situation the ltering is to be performed, the best
method can be chosen. It is easier to control the locality using convolution
with a small kernel in the SD. When the lter kernel is large, the FFT method
often is to prefer. The operation in the FD is then less computationally
demanding and therefore require less time than the corresponding operation
in the SD.
CHAPTER 5. THEORY OF THE FOURIER TRANSFORM
33
5.3 The shift theorem
The shift theorem says that a shift in the SD corresponds to a shift of phase
in the FD. If the image f (x, y) is shifted the distance a in the x-direction
and b in the y-direction, the shift theorem can be written as below.
F{f (x − a, y − b)} = e−j2π(au+bv) F (u, v)
(5.4)
In equation 5.4 it can be seen that there is no change of magnitude in the
FD, only the phase is aected. All components will be delayed in phase,
proportional to the frequencies u and v . For higher frequencies the shift in
phase angle will be greater. [2, 7] To get a picture of the connection between
the SD and the FD, a graphical example for a 1D signal is presented, see
gure 5.1.
Figure 5.1: An 1D example to illustrate the shift of phase in the FD.
Top: The signal f(x) in the SD, original to the left and the shifted version to
the right. Bottom: The FT, F(u), to the left and the FT to the shifted signal
to the right. Inspired of [2].
If the signal f(x) is shifted the distance a = 12 , the corresponding phase shift
in the FD will be 2πau. This is subject to a uniform twist of π radians per
unit of u.
CHAPTER 5. THEORY OF THE FOURIER TRANSFORM
34
5.4 The phase correlation method
The phase correlation method [9] uses the shift theorem to nd the shift
of an image in the SD, by looking at the corresponding phase shift in the
FD. An example is made out of two uoroscopy images, see gure 5.2, to
illustrate the method.
Figure 5.2: To the left: Original image i1 . To the right: Image i2 , translated
20 pixels upwards and 15 pixels to the right, compared to i1 .
The dierence between the images is that the image i2 has been shifted
compared to the image i1 . To get the distance and direction of the shift,
rst the FT for both images, I1 (u, v) and I2 (u, v), are computed. Then the
phase correlation matrix, Q, is computed with the expression
Q=
I2 (u, v)I1 (u, v)∗
= e−j2π(au+bv)
|I2 (u, v)I1 (u, v)|γ
(5.5)
The pattern seen in Q can be connected to the phase twist that was shown in
gure 5.1. The parameter γ controls the equalization of magnitude. Equalization of magnitude is wanted, since only the phase is of interest. A too
large dierence in magnitude will aect the result. Since the plotting of
phase will be wrapped, a saw-tooth pattern is seen. In this example, every
period means another translated pixel in the SD, since the phase shift is proportional to the spatial shift for every unit of frequency. Hence, the direction
and frequency of the saw-tooth pattern give information about the spatial
shift. To get this information, Q is inverse transformed back to the SD. A
more or less distinct point will then be seen in the SD, since
CHAPTER 5. THEORY OF THE FOURIER TRANSFORM
q = F −1 {Q} = δ(x − a, y − b)
35
(5.6)
for a continuous signal. The discrete nature of the image will make the dirac
spread some to the surrounding pixels, depending on if the shift is an integer
of pixels or not. The vector from the origin to (x − a, y − b) in q equals the
translation, both in direction and distance. [9] Figure 5.3 shows the phase
correlation matrix Q and its inverse FT.
Figure 5.3: To the left is the phase correlation matrix, Q (γ = 0.8), and to
the right, its inverse FT, q .
This method can be used for instance when detecting motions, from one
frame to another in an image sequence. If the exact shift is known, it can
be compensated for.
Chapter 6
Theory of image enhancement
When processing images, basic operations are often used to simplify problems
and to visualize the image properties. Some of these operations will be described in this chapter. The theories of image enhancement, used in dierent
shapes in this thesis, are presented.
6.1 Interpolation methods
Interpolation is used when resampling an image. This is done for instance
when scaling or shifting an image, or when reconstructing an image from a
data set, e.g. in ultrasound. Interpolation is used to get the underlying continuous signal and from that calculate the values in the new sample points.
[6]
First three commonly used interpolation methods applied in the SD are
presented. They all take the original image samples and the corresponding
intensity values as input. Then the corresponding values in the new samples
are calculated in dierent ways. At last, an interpolation method in the FD
will be mentioned.
Nearest Neighbor interpolation
In the nearest neighbor (NN) method, the neighbor that is closest to the new
sample will decide the value. The output image will with this method have a
very edgy appearance. The method is simple and not very computationally
demanding. [6, 12]
Bilinear interpolation
Bilinear interpolation is done using the four surrounding pixel values. First
the linear interpolation is done in one direction. The calculated values from
37
CHAPTER 6. THEORY OF IMAGE ENHANCEMENT
38
the rst direction is then used when interpolating in the second direction.
[12] This will result in a much nicer appearance, the edges are not as edgy
as in the NN-case. This method also preserves locality.
Bicubic interpolation
The bicubic interpolation method uses a cubic spline function to calculate
the new values. It takes into account for instance 4x4 neighbors with a
weighting function. The result will have an even more isotropic appearance
but it is also more computationally demanding. [12]
Adjusting of phase in the FD
According to the shift theorem in chapter 5, a shift in the SD corresponds
to a phase shift in the FD. Hence, to interpolate an image in the SD, the
phase can be adjusted in the FD. In the SD this correspond to interpolating
with a sinc function on a repeated image. This may result in artefacts and
the result will be more smoothed than with bilinear interpolation.
6.2 Thresholding
Thresholding is a simple way to segment an image into dierent objects. A
threshold is set on a specic gray value and all pixels above that value are
set to white (for instance the object pixels) and all pixels with lower gray
value are set to black (for instance the background pixels). The result is a
binary image. This method is often applied when deciding an objects shape
or location, or when counting objects in an image.
6.3 Morphological operations
When thresholding an image, the object segmented often is not perfect for
further treatment. For instance there can be noise or dierent intensities in
the object that is segmented. Then morphological operations can be applied
to shape the object. A binary structure element, a, is applied on the image
and depending on operation, dierent results are received.
Dilation
The center pixel of the structure element a is placed at every 1-set pixel
in the binary image b. In every position, the structure elements pixels are
copied to the resulting image, see gure 6.1. This operation expand the
object, depending on the chosen a. [4]
CHAPTER 6. THEORY OF IMAGE ENHANCEMENT
39
Erosion
The structure element a is now placed at every pixel in b. For the positions
where the elements are entirely covered by 1-set pixels in b, the current pixel
is set to 1 in the resulting image, see gure 6.1.
This operation shrink the object, depending on the form of the structure
elements. A combination of these two operations can open (erosion+dilation)
or close (dilation+erosion) objects. [4]
Figure 6.1: Top: An example of dilation of image b with the structure element
a. The result is shown with the added pixels in gray. Bottom: An example
of erosion of image b with structure element a. The result is shown with the
removed pixels in gray.
6.4 Contrast adjustment
The contrast in an area of an image can be dened as
C=
S1 − S2
S1 + S2
(6.1)
where S1 and S2 is the signal intensity in two close objects in the image.
The purpose of contrast adjustment is to increase C, so that objects can be
clearly separated and to get an increased wealth of details. In the following
section two methods of gray scale mapping are described.
CHAPTER 6. THEORY OF IMAGE ENHANCEMENT
40
6.4.1 Gray scale mapping
Each pixel in an image has a specic gray value. The histogram of an image
shows the total amount of pixels for every gray value in stacks. The stacks
are often divided into 256 dierent gray levels between 0 and 1. See gure
6.2 for an example of an image and its histogram.
Figure 6.2: The original test image Lena and its corresponding histogram.
To change the contrast in an image, usually a gray scale transformation is
done. This is done by mapping the gray levels with a mapping function. The
idea is to represent the image content with more gray levels. Two pixels that
before the transformation are close to each other in gray value, will now be
represented by values more apart. [4]
Histogram stretching
The simplest mapping function is a linear function between the minimum
and maximum gray value in the image. This will stretch the histogram so
that all the 256 levels are represented in the image. In gure 6.3, this method
is applied. Compared with gure 6.2, the contrast is higher.
CHAPTER 6. THEORY OF IMAGE ENHANCEMENT
41
Figure 6.3: The histogram after linear mapping between the lowest and highest pixel values. The histogram shows that the values has been spread so that
all gray levels are used. This result in better contrast in the image.
Histogram windowing
In medical imaging it can be interesting to increase the contrast for some
values and decrease it for others. This is done by creating a window containing the important gray values and set values below or above these values to
0 and 1 respectively. Then the contrast will be even better for the chosen
interval. The other information is lost and therefore knowledge about the
important parts is necessary. This method is used in the Biplanar 500 system today. The window is computed from a function that uses the minimum
and maximum values of the image. A discussion of this method compared
to the proposed method is found in chapter 9.
6.5 Noise reduction
There are dierent kinds of noise that can appear in images. The most common are signal dependent (multiplicative) or signal independent (additive)
noise. To measure noise in images the signal-to-noise ratio (SNR) is often
used. It is expressed in dB and gives the relation between the signal variance
and the noise variance in the image, see equation 6.2.
SN R = 10log
2
σsignal
2
σnoise
(6.2)
The main diculty in noise reduction is to preserve lines and edges. In
the following sections, some dierent noise reduction methods are presented
CHAPTER 6. THEORY OF IMAGE ENHANCEMENT
42
which take care of this in dierent ways.
6.5.1 Averaging
One way to suppress noise in a 2D image is averaging, or Low Pass (LP)
ltering. It is a linear method that can be applied either in the FD or in
the SD. The noise will be eciently smoothed, since it is often high frequent. Unfortunately also detailed, high frequent structures in the image
get smoothed. Hence, this is not a suited method for X-ray images. See
gure 6.4 in the next section for averaging applied to a 1D signal.
6.5.2 Median ltering
This method is often used to reduce so called salt and pepper noise in images.
The principle of median ltering is to look at the neighboring pixels of each
pixel. The median value for every neighborhood is then put in that pixels
position in the output image.
The idea with median ltering is to remove sharp and small tops in the
image, such as noise, but to preserve the edges. [4] It is not easy to show
the meaning of median ltering in 2D, but in 1D the principle can be shown,
see gure 6.4. Median ltering is a non-linear method, there is no equivalent
method to median ltering in the FD.
Figure 6.4: The eect of median ltering on a 1D signal. At the top is the
original signal, in the middle the median ltered signal is shown and at the
bottom is the averaged signal, shown as a reference. Inspired by [4].
The size of the neighborhood used is often the smallest possible, 3x3 pixels.
CHAPTER 6. THEORY OF IMAGE ENHANCEMENT
43
A bigger neighborhood tends to smooth the images. In noise reduction,
iteration of ltering is often applied. This preserves the edges and reduces
noise but thin lines, narrower than half of the kernel, will be lost. After some
iterations also homogenous areas will occur, with the same gray level. [12, 4]
This method is not suitable for X-ray images, since thin lines are to be
preserved and the median ltered images looks constructed because of the
too homogenous areas between the edges. It is also very computationally
demanding.
6.5.3 Temporal averaging
The principle of temporal averaging is to sum a number of consecutive frames
from an image sequence and then divide the result with the number of frames.
It is one example of a FIR-lter, where FIR stands for Finite Impulse Response. [13] The algorithm is shown in equation 6.3,
Iout =
n
1X
f ramek
n k=1
(6.3)
where n is the number of frames. The result is one single, noise reduced
image, Iout . This method preserves the structures that are constant in time.
Other parts that dier from one frame to another, such as noise, will be
suppressed. Thus the more frames used, the more ecient noise reduction.
This method often gives very good result unless there are moving objects in
the image sequence. In case of motion the image gets blurred around the
moving object.
This method can be applied continuously on image sequences. A lot of
data then have to be saved, n frames. When n frames have been collected
though, only one frame has to be replaced with a new one in the next point
of time. The advantage with this method is that when n number of frames
are available, the dierent frames can be weighted dierently in the output
image.
CHAPTER 6. THEORY OF IMAGE ENHANCEMENT
44
6.5.4 Temporal recursive ltering
Recursive ltering is a temporal noise reduction method, often used on image
sequences. [1, 11, 16] The recursive lter is often called IIR-lter, where IIR
stands for Innite Impulse Response. For every new input frame also the
previous output frames are taken into account when creating the new output
frame. The output from a rst order recursive lter can be written as
Iout [t] = αIin [t] + (1 − α)Iout [t − 1]
(6.4)
i.e. it uses one previous output frame to compute the present one. The time
is set in brackets, since the image sequence is discrete in time. Figure 6.5
shows the corresponding block diagram. [11, 13]
Figure 6.5: Block diagram of recursive ltering.
T is a time delay element, giving the previous output image. The parameter
α controls the amount of recursive ltering and can be tuned between 0 and
1. The output is a sum of dierent frames back in time and the composition
of frames at every point of time can be described with the impulse response.
To get the impulse response, rst the Z-transform [13] of equation 6.4 is
computed to
Y [z] − (1 − α)Y [z]z −1 = αX[z]
(6.5)
where X and Y are the Z-transform of Iin and Iout respectively. The transfer
function H is then given as
H=
Y
α
zα
=
=
X
1 − (1 − α)z −1
z − (1 − a)
(6.6)
and the inverse Z-transform gives the impulse response h, according to
h = Z −1 {H} = α(1 − α)t
(6.7)
CHAPTER 6. THEORY OF IMAGE ENHANCEMENT
45
In gure 6.6 the impulse response is shown for dierent α. When α is set to
0 the lter totally ignores the present frame and for α=1 only the present
frame is used.
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
2
4
6
8
10 t
Figure 6.6: Impulse response α(1 − α)t of the temporal IIR-lter for dierent
α; α =0.8: dotted line, α =0.5: solid line and α=0.2: dashed line.
The longer the impulse response of the lter stretches back in time, the
better noise reduction is received. Thus a low α is wanted and that works
very well for image sequences that does not change in time. If an object is
moving from one frame to another though, there will be blurring, motion
artifacts, around the moving object when α is low. [11]
The method of recursive ltering is often used when the time aspect is important. The input frames do not have to be saved, since the impulse response
is always in the last output signal. This can be compared with the temporal
averaging, that needs n frames to build up one output frame.
Chapter 7
Implementation and results
The theory from chapter 5 and 6 is here applied to the specic problem together with some new ideas. First there is a short method description that
explains how the algorithms have been developed. Then the implementation
including preprocessing, contrast adjustment, noise suppression and motion
compensation is described.
7.1 Method
The algorithms have been developed in Matlab. Test image sequences have
been acquired, as described in chapter 4, and saved to disk. The whole
image sequences are thus accessible when starting the operations, saved in 4D
matrices (see next section for an explanation). By testing dierent existing
theories and new ideas separately, the best combination could be chosen
for every subproblem. The algorithms are implemented in dierent Matlab
functions. The implementation is made in a number of steps, see the list
below.
• Preprocessing algorithms
• Deinterlacing
• Contrast adjustment
• Temporal recursive ltering
• Global motion compensation
• Local motion compensation
• A combined algorithm
The following sections describe the implementation and results of these steps.
47
CHAPTER 7. IMPLEMENTATION AND RESULTS
48
7.2 Preprocessing algorithms
The raw data image sequence that is acquired into Matlab is not optimized
for image processing. Thus some preprocessing steps are made to simplify
the image enhancement. These steps are considered in this section.
7.2.1 RGB to gray scale transformation
The VCC used gives an output signal in the image format RGB, consisting
of three channels; one for red, one for green and one for blue contents. This
format is often used when processing color images. The raw data image
sequence is thus a 4D signal (RGB+time).
Since the video from the CCD-camera consists only of gray values, all the
RGB channels are equal. A transformation is made into gray scale format,
that is 3D (x, y, t). This is done by only using one of the three color dimensions. The gray scale image sequence is then ready for further treatment.
7.2.2 Cropping
The original image sequence is built up of frames with the size 576x720
pixels. The uoroscopy image only occupy about 512x512 pixels and the
image is circular. Thus a cutout is made around the circular image, to make
the future processing more specic and less computationally demanding.
The cutout is done by rst thresholding the image to get a binary mask
that tells where the circular image is located. The size of the cutout is
specied depending on the specic operation and it is made from the middle
of the mask to make it symmetric. For instance, in the recursive ltering
and motion compensation algorithms, the size 256x256 pixels is used.
CHAPTER 7. IMPLEMENTATION AND RESULTS
49
7.2.3 Windowing
When transforming an image into the FD, it is important to window the
image, as discussed in section 5.1. Otherwise very distinct artifacts would
be seen in the FD, corresponding to the sharp borders of the image.
The image is circular and thus a circular window is wanted, that softens the
borders of the image. A circular symmetric cos2 is used in this case, that is
multiplied with the same mask as was described in the previous section, see
the left image in gure 7.1 for the window.
A quadratic window is constructed as well, see the image to the right in gure
7.1. It is used in the algorithm for recursive ltering, where the cutout is
256x256 pixels and the image has the shape of a square. This window is
built up of a cos2 in both x- and y-direction this time and is not circular
symmetric.
Figure 7.1: Left: The circular symmetric cos2 window. Right: The quadratic
cos2 window.
The window is then multiplied with the image. The FT of the windowed
image will then be received after transformation, according to
F (u, v) = F{W f (x, y)}
See gure 7.2 for the eect of windowing.
(7.1)
CHAPTER 7. IMPLEMENTATION AND RESULTS
50
Figure 7.2: Top left: The original image. Top right: The windowed image.
Bottom left: FFT of the original image, with distinct artifacts. Bottom right:
FFT of the windowed image. The artifacts are gone.
CHAPTER 7. IMPLEMENTATION AND RESULTS
51
7.3 Contrast adjustment
The idea of contrast adjustment in this thesis is to use a smoother mapping
function than in the present method, which only expands the values within
a certain window. A mapping function that can be adjusted to get dierent slope and position is wanted. A mapping function that comply with
these demands is the h-mapping function, see equation 7.2. In [7], a similar
function is used to map tensors.
Iout =
Iin (1 − a)b
(Iin (1 − a))b + (a(1 − Iin ))b
(7.2)
The h-mapping function is nonlinear and can be adjusted to the desired
shape. The parameter a controls the position of the curve and b controls the
slope. Examples of the h-mapping function for dierent a and b can be seen
in gure 7.3 below. The smoothness is better for the eye to look at and the
lower and higher values are not set to 0 or 1 but is decreasing or increasing
all the way.
1
1
0.9
0.9
0.8
0.8
0.7
0.7
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0
0.1
0
0.2
0.4
0.6
0.8
1
0
0
0.2
0.4
0.6
0.8
1
Figure 7.3: The h-mapping function for dierent a and b. The gray values
on the x-axes in the gures above, are transformed to the values on the yaxes. Left: b=2 and a is changed. a=0.3 (dashed), a=0.5 (solid) and a=0.8
(dotted). Right: a=0.5 and b is changed. b=1 (dashed), b=2 (solid) and b=4
(dotted).
The h-mapping function from equation 7.2 is implemented. In gure 7.4, an
example of mapping result is shown.
CHAPTER 7. IMPLEMENTATION AND RESULTS
52
Figure 7.4: A frame from test sequence F in original (left) and contrast
adjusted with the h-mapping function, a=0.63 and b=2.82 (right).
7.4 Deinterlacing
The original image sequence is in interlaced format but is presented as progressive on the screen, i.e. frame by frame. That makes the image blur and
show interlacing artifacts when an object is moving. See section 3.3.1 for the
theory of interlacing.
To avoid this, an algorithm is developed that compensate for the interlacing
pattern. To separate the elds that the frame is built up of, every second
row is set to zero. This striped image is then convolved with the kernel [0.5
1 0.5]T , that will interpolate the image from the odd/even rows to build up
a new frame, from only one point of time. See gure 7.5 for the original
image, one eld and the resulting image.
Figure 7.5: The method of deinterlacing. Left: The original frame with
interlacing artifacts. Middle: The eld of odd lines. Right: The interpolated
result.
CHAPTER 7. IMPLEMENTATION AND RESULTS
53
When implementing the deinterlacing based on the odd eld, the last pixel
row will get lower pixel values than what is wanted, because of the limit of
the image. To avoid this, the second last row is copied to the last low. The
method of deinterlacing is used both in the global and in the local motion
compensation.
7.5 Temporal recursive ltering
The implementation of pure recursive ltering is quite easy. The output
image is computed according to equation 6.4, using one previous output
frame. Depending on the parameter α, a reduction of noise is received.
Figure 7.6 shows the recursive ltering applied to the B test sequence with
both global motions and constant position, where α is set to 1, 0.6 and 0.2.
As expected the noise is reduced more and more down the left column. In
the right column, it is seen that the small α result in blurred images. To
avoid the blurring that occurs when there is motion, a motion compensator
is developed. The method of recursive ltering will underlie the algorithms
that are described later on.
CHAPTER 7. IMPLEMENTATION AND RESULTS
54
Figure 7.6: Recursive ltering of test sequence B. Top: α = 1 for frame
20 (still phantom) and frame 37 (moving phantom). Middle: α = 0.6 for
frame 20 and 37. Bottom: α = 0.2 for frame 20 and 37. Deinterlacing is
not applied in this gure.
CHAPTER 7. IMPLEMENTATION AND RESULTS
55
7.6 Temporal averaging
In opposite to the recursive ltering, averaging in time does not use previous
output images when computing the present one. Instead a specied number, n, of input frames back in time are used, see section 6.5.3. Since the
present system uses averaging with dierent number of frames, as found out
in section 3.4, this is reconstructed to get something to compare the results
with. Figure 7.7 shows temporal averaging applied to the test sequence C,
compared to the original sequence.
Figure 7.7: Temporal averaging of test sequence C. Top: Original frame 11
(moving drill) to the left and frame 20 (still drill) to the right. Bottom: The
corresponding temporal averaged frames, frame 11 to the left and frame 20
to the right, with n=6.
CHAPTER 7. IMPLEMENTATION AND RESULTS
56
7.7 Global motion compensation
The basic idea of the global motion detection and compensation is the shift
theorem; when a spatial shift is made of an object in the SD, a shift of phase
will occur in the FD. The example from section 5.4 cover the theory and the
implementation is straightforward from that.
7.7.1 Calculation of translation
To detect weather something in the scene has been moved from one frame
to the next, the phase correlation method has been used. Two consecutive
frames are Fourier transformed and the phase correlation matrix, Q, see
equation 5.5, is calculated. Tests showed that a γ equal to 0.82 gives the
best result. After inverse transforming Q back to the SD, an image, q , is
received, with a dirac in the pixel that correspond to the shift. This is only
exact if the shift made is an exact number of pixels. Since that is not often
the case, the dirac will spread out to the surrounding pixels. For an example
of shift calculation, see gure 7.8.
To get the exact translation, rst the maximum value of q is located. Then
the mass center is calculated in an area around that point, according to
P P
xyI
yI
x
y
r= P P
x
(7.3)
where I is the intensity in every coordinate (x, y). The calculated shift will
then be more exact when using it in the motion compensator. In the example in gure 7.8, the calculated shift is 2.57 pixels to the right and 5.59
downwards.
If the size of the shift is equal to or more than 2 pixels, the deinterlacing
algorithm, described in section 7.3, is used, to avoid interlacing artifacts.
CHAPTER 7. IMPLEMENTATION AND RESULTS
57
Figure 7.8: Top left: Frame no 34 of test sequence B. Top right: Frame no
35. Bottom left: The phase correlation matrix, Q, (γ = 0.82). Bottom right:
The inverse FT of Q, q.
7.7.2 Adapting the temporal impulse response
For every input frame, the shift between the current frame and the previous
frame is computed, as described in the previous section. The shift is given in
vector-form, with direction and magnitude. If there is no shift, the recursive
ltering can proceed as usual. If there is a shift, α is set to 1 and the impulse
response is cut. This means that the recursive ltering is only operating when
there is no motion.
When a motion is detected, the input frame is used as output. The bone
structures in the sequence will then be more prominent during movement,
even though there is a lot of noise that has not been reduced. When the
phantom is still, the image will be very clear. In gure 7.9 the result of this
method is shown for a frame with no motion and for a frame with motion.
This gure can be compared to gure 7.6 in the previous section.
CHAPTER 7. IMPLEMENTATION AND RESULTS
58
Figure 7.9: Cutting the impulse response upon motion in the test sequence
B. Left: Frame 20 (still phantom). Right: Frame 37 (moving phantom).
Deinterlacing is here applied.
In [11], Jostschulte et al use the magnitude of the shift to set α according to
a look-up table. For small shifts, α can stay low and the larger shifts, the
higher α is used. Since the exact shift is known in this case, it is chosen to
compensate for the motion instead.
7.7.3 Shifting the impulse response
For every new frame, the impulse response (or the previous output frame) is
shifted to the position of the current frame. The shift is done with bilinear
interpolation in the SD, as described in section 6.1.
When this is done, the recursive ltering can be applied to the compensated
frames. The shifted frames will have dark edges, since the frames have
limited boarders. This leads to that the resulting output frame is built up
of a dierent number of frames in dierent parts. This problem is solved by
normalizing the image sequence with a normalization sequence that follow
in the process. See gure 7.10 for the results. The parts of the image that
are build up of only a few frames become more noisy. This is a small eect,
compared to the enhancement thanks to the global motion compensation
when the phantom is moved.
CHAPTER 7. IMPLEMENTATION AND RESULTS
59
Figure 7.10: Top: Result of the global motion compensation before normalization in test sequence B. Middle: The corresponding normalization sequence. Bottom: Normalized result of the global motion compensation.
CHAPTER 7. IMPLEMENTATION AND RESULTS
60
7.7.4 Width of the impulse response
The global motion compensation combined with the recursive ltering will
result in a LP eect in the frames that originate from distant points of time.
Since the translation is calculated exactly, from the mass center formula in
section 7.6.1, and the images are discrete, the value is split between the
nearest pixels when the translation in pixels is not an integer. In the worst
case, each new sample is in the middle of four pixels. The interpolation
causes a LP eect that will increase back in time, since the whole impulse
response is shifted during compensation. To get a picture of how big this
low-pass eect is, gure 7.11 shows the eect for 4, 8 and 16 frames back in
time in the worst case.
Figure 7.11: The LP eect at 4, 8 and 16 frames back in time, in the worst
case.
Another compensation for global motion was tested, to see how big the
dierence can be. If every frame is only interpolated once, the LP eect will
be much lower. For every new frame, the output frame is instead composed
of the 16 previous frames, all of them compensated the distance that dier
from the present frame.
This algorithm gets much more time demanding, since the shift of frames
is done up to 16 times/frame instead of only one when shifting the impulse
response. This was done with both uniform distribution and a cos4 function.
When this method was compared to the motion compensation described
earlier, no big dierence could be seen. Thus no further work was put into
this idea.
7.8 Local motion compensation
The other type of motion that can occur in the Biplanar 500 image sequences
is a local motion, for instance when the patient is still and only the drill
is moving. To handle this kind of motion, a local motion compensator is
developed.
CHAPTER 7. IMPLEMENTATION AND RESULTS
61
By subtracting the previous frame from the present one, an image is received
that can tell where the motion is. If there is no motion, the dierence image
is all zero, except from the noise. If there is a local motion, the exact position
of this will be shown. See gure 7.12 for an example of two consecutive input
frames that are used to get a dierence image.
Figure 7.12: Frame no 8 (left) and 9 (right) of the C test sequence.
The magnitude of the dierence image is thresholded to get a mask that
shows the area of local motion. After that the mask is opened with a 2x2
kernel to get rid of the small objects referring to noise. This is done via
erosion followed by dilation, as described in section 6.3. LP-ltering is then
applied in the SD with a gaussian lter, to make the mask smooth with no
distinct edges. Figure 7.13 shows the dierent steps of adjusting the mask.
Figure 7.13: From left to right: The thresholded dierence image, the opened
mask and the LP-ltered mask.
The resulting mask, m, is used to cut the moving parts (i.e. the drill in this
case) in the present frame into the recursive ltered frame, according to
Iout = mIin + (1 − m)Irec
(7.4)
CHAPTER 7. IMPLEMENTATION AND RESULTS
62
In gure 7.14 the result of the local compensation is shown, compared to the
result from pure recursive ltering.
Figure 7.14: The result using pure recursive ltering with α = 0.3 (left)
compared to the local motion compensated result (right).
Deinterlacing, described in section 7.3, is also applied to the mask in the
cutting-in step, to avoid interlacing patterns. See gure 7.15 for an example
of the result with and without deinterlacing.
Figure 7.15: Output frame 8, with interlacing artifacts to the left and with
deinterlacing applied to the right.
CHAPTER 7. IMPLEMENTATION AND RESULTS
63
7.9 An alternative method to detect global motion
In the two previous sections, the global and local compensation methods
have been described. Both of the algorithms work well on test sequences with
global and local motions respectively. In the Biplanar 500 system however,
it is wanted to compensate for both global and local motions. Therefore an
algorithm is developed that can distinguish between local and global shifts
and choose between these two methods, see the next section.
The global detection then has to be done dierently than before, since tests
showed that big local motions are treated as global motions. Another approach is then tested, mostly based on dividing the frames into smaller sections.
7.9.1 Dividing frames into overlapping sections
The frames are divided into a number of overlapping sections. See gure
7.16 for the division principle of one frame. The number of sections has been
chosen to 7x7=49 sections, i.e. the side of every section is a quarter of the
whole image side.
Figure 7.16: An image divided into sections. The sections are cut out in this
way. In this case the total number of sections will be 3x3=9.
7.9.2 Representing shifts in vector plots
The shift between two frames is now computed, in the same way as described
earlier, for each section, to detect weather there has been a shift in the specic
CHAPTER 7. IMPLEMENTATION AND RESULTS
64
section or not. The results from this operation are then analyzed in dierent
ways.
In theory, when there is a global motion, the vectors for all the sections will
point in the same direction and have the same magnitude. When there is a
local motion or no motion, most of the vectors will have a zero magnitude,
except for in the sections where the local motion occur.
It showed that the local shifts were not outstanding from the false, higher
magnitude shifts occurring, probably because of noise. The non motion and
local motion cases could therefore not be separated. The global motion on
the other hand could be separated and that was the main idea. In gure
7.17 the resulting vectors for no motion and global motion is shown.
8
8
7
7
6
6
5
5
4
4
3
3
2
2
1
1
0
0
1
2
3
4
5
6
7
8
0
0
1
2
3
4
5
6
7
8
Figure 7.17: The shifts computed in each section at no motion (left) and
global motion (right).
First, the number of sections that registered shifts were used to sort out if
there was a global motion or not. It functioned quite well but to increase
the robustness, also another method was developed.
7.9.3 The mean-std method
The vectors were plotted in a (x-y) -coordinate system, one point representing
one vector. Figure 7.18 shows that a global motion corresponds to points
spread at the global shifts representation in the plot. If no motion is present
or the motion is local, a large amount of points will be tightly clustered to
the origin.
This lead to a classication based on the mean value and the standard deviation of the (x,y) -plot. If the mean value is close to the origin and the standard
deviation is low, probably there is no motion. If the points are concentrated
CHAPTER 7. IMPLEMENTATION AND RESULTS
65
Figure 7.18: A plot of the translations computed, at no motion (left) and at
global motion (right).
to another point instead, and the points are more spread, there is a global
motion. This mean-std method function well and it is more robust than the
method of counting the number of translations. This method is therefore
used in the combined algorithm, described in the next section.
7.10 The combined algorithm
The global and local compensation is now put together in one algorithm.
This algorithm will be referred to as the combined algorithm from now on.
The parameter global is set to true or false, depending on the result from the
global detection, described in the previous section. If it is true, the global
compensation and the recursive ltering is carried through. Otherwise the
recursive ltering goes on as usual and will be followed by the local motion
compensation. Hence, the algorithm now function for both global and local
shifts. The combined algorithm is described in detail in the block schemes
in Appendix A.
Chapter 8
Evaluation study
In this chapter, an evaluation study is presented. First, the method is briey
described. Then the dierent excersices are presented one at a time, with
prerequisites, results and conclusions drawn.
8.1 Method
The study was carried through at the department for orthopedic surgery
at the Nyköping Hospital. This hospital was chosen, since the orthopedists
there often use the Biplanar 500 system during hip operations. The orthopedists are therefore used to work with hip image sequences, matching the
test sequences used during development of the algorithms.
A laptop was used to show the image sequences to the orthopedists, one at a
time. Notes were taken and their opinions and thoughts were written down.
The images and image sequences were rotated so that the orthopedists would
recognize the scenes as they appear in the Biplanar 500 system. I got to talk
to 4 orthopedists and their answers are presented in the following sections.
Each person needed about 5 minutes to look at and discuss the images.
8.2 Manual contrast adjustment
At rst, the orthopedists were to adjust the contrast parameters a and b
in the h-mapping function on a typical hip image, until they were satised. By using the matlab function mumap, the parameters were applied
continuously to the image. The result is shown in table 8.1 below.
67
CHAPTER 8. EVALUATION STUDY
Orthopedist
1
2
3
4
68
a
0.60
0.56
0.53
0.55
b
1.84
1.60
3.96
2.90
Table 8.1: Result of manual contrast adjustment of the h-mapping function
parameters.
This rst contrast adjustment exercise came out with a widespread result,
as expected. Some of the orthopedists liked brighter images and some much
darker and with a higher reinforcement of contrast. They liked the bars and
understood the meaning of adjusting the parameters. One orthopedist asked
if it could be possible to adjust the contrast in a similar way in the X-ray
system.
8.3 Comparison between dierent mapping
functions
In the second contrast exercise, two pairs of images were shown, comparing
the h-mapping function with the linear mapping function used in the system
today. See gure 8.1 and gure 8.2.
Figure 8.1: The images used for comparing the result of the h-mapping function (left) to the result of linear mapping (right). This is for low contrast
reinforcement, a=0.51, b=2.7 in the h-mapping function, corresponding to a
linear mapping of the values between 75-175.
CHAPTER 8. EVALUATION STUDY
69
Figure 8.2: The images used for comparing the result with linear mapping
(left) to the result using the h-mapping function (right). This is for high
contrast reinforcement, a=0.54, b=3.96, corresponding to a linear mapping
of the values between 113-175.
The dierence between those two sets were the gradient and location of the
mapping curves. Between the two sets, the order was shifted, to assure that
the orthopedists choose the best image independent of order. The result of
the second contrast exercise is shown in table 8.2 and 8.3 below.
Orthopedist Choice Comment
1
1A
2
3
1A
1A
4
1A
The other one is too light in the
surroundings.
The white surroundings are disturbing.
The other one takes away the borders
of the trochanter of femur.
This one is better in the caput line.
Table 8.2: The result of comparison between the linear and the h-mapping
function. Low gradient of the mapping functions, see the images in gure
8.2.
CHAPTER 8. EVALUATION STUDY
70
Orthopedist Choice Comment
1
2A
2
2B
3
2B
4
-
This one is better in the main area and
the caput line.
The other one is too white in the outer
parts of the image.
The contours remain even in the outer
parts of the image.
No dierence in the main area.
Table 8.3: The result of comparison between the linear and the h-mapping
function. High gradient of the mapping functions, see the images in gure
8.3
In this second contrast exercise, the comparison of images adjusted with a
linear mapping function versus a h-mapping function, the result was more
unanimously. For the rst set of images, with a low contrast reinforcement,
all of the orthopedists preferred the new h-mapping function. 3 of 4 were
disturbed by the white areas in the outer parts of the linear mapped image,
that made the structures of the trochanter of femur invisible. One thought
that the caput line got more prominent. For the second set of images the
opinion of the too sharp light areas remained but one orthopedist preferred
the linear one, due to better contrast around the caput line, and one could
not see the dierence.
8.4 Noise reduction and blurring
In this exercise the orthopedists were asked to rank dierent image sequences
in the meaning of noise, blurring and sharpness and what would be the
preferred image during operation.
The sequences with global and local motion were shown in dierent sets.
For both of these sets the original sequence, an averaged sequence (n = 6)
and a recursive ltered sequence (α = 0.3) were shown at rst. Then three
dierent motion compensated sequences were shown.
Table 8.4 and 8.5 explain the dierences between the sequences used in the
study. For the global compensation, the test sequence B was used and for
the local compensation, the test sequence C.
The result from the sequences with global and local motion is shown in table
8.6 and 8.7 respectively.
CHAPTER 8. EVALUATION STUDY
71
Sequence Description
3A
3B
3C
3D
3E
Temporal averaging, using n=6.
Recursive ltering, using α=0.3.
Upon motion, the impulse response is cut (α=1/0).
Global motion compensation, α=0.2.
Global motion compensation, α=0.4.
Table 8.4: The global motion compensated test sequences used in the evaluation study.
Sequence Description
4A
4B
4C
4D
4E
Temporal averaging, using n=6.
Recursive ltering, using α=0.3.
Local motion compensation, α=0.4.
Local motion compensation, α=0.2.
Local motion compensation, the dierence
between the present frame and the last
output frame is used as a mask.
Table 8.5: The local motion compensated test sequences used in the evaluation
study.
Orthopedist Comment
1
2
3
4
3B is better than 3A. 3C is better than 3A and 3B since
the contours remain during motion.
3D and 3E are similar and the best sequences.
Maybe 3A is a little bit better than 3B.
3D and 3E are the best sequences. Often I
only use still images and therefore it
is not that relevant.
It is most important to get a sharp image
when there is no motion.
3D and 3E are similar and the best ones.
Maybe the compensation can imply that still
images gets better if the patient is moved
during excitation.
No dierence between 3A and 3B.
3C is a little bit better but 3D and 3E are even better.
Perhaps 3D is a little bit better than 3E.
Table 8.6: The opinions of the dierent versions of test sequence B (global
motion). See table 8.4 for a description of the sequences 3A-E.
CHAPTER 8. EVALUATION STUDY
72
Orthopedist Comment
1
2
3
4
No big dierence between 4A and 4B. 4C-E is
much better! It is important that the image
is not delayed when drilling.
4A is better than the original, the noise is
eciently removed. No big dierence between
4A and 4B. Since I only use still images, the
blurring is no problem. 4C-D are better
sequences but I do not know if it is useful.
The 4C sequence is the best one. It does not
loose in quality but the blurring is gone.
It depends on how experienced the orthopedist
is, how much the continuous excitation is used.
No dierence can be seen between 4A and 4B.
4C-E are better, no dierence can be seen
between those.
Table 8.7: The opinions of the dierent versions of test sequence C (local
motion). See table 8.5 for a description of the sequences 4A-E.
CHAPTER 8. EVALUATION STUDY
73
The result from the global and local sequences show that in almost every
case, no dierence could be seen between the averaged image sequences and
the recursive ltered ones. Two exceptions from this opinion was one who
preferred the averaged sequence and one who preferred the recursive ltered
sequence. The conclusion drawn from this is that averaging of a sequence
with n = 6 corresponds quite well to recursive ltering with α=0.3.
When looking at the results from the compensated sequences, it turned out
to be too small dierences to get a good view of which parameters that
were the best ones. In the global sequence, where the impulse response
was cut in sequence 3C, the main opinion was that it was better than the
non compensated ones. Then, when showing the compensated sequences 3D
and 3E they were even better. A lot of comments on the usefulness of the
compensation were given.
The same thing was said about the local motion compensated sequences, no
big dierence could be seen between 4C-E but the general opinion was that
motion compensation was good. Some questioned the usefulness, while some
where thrilled that the quality was so good and emphasized the importance
of no blurring during drilling.
This study was performed before the algorithm for both local and global
compensation in combination was developed. Hence, that algorithm could
not be evaluated. However, the results from the combined algorithm is similar to the results shown to the orthopedists, since the combined method is
based on the same algorithms. The results of this study is therefore also
valid for the proposed algorithm.
Chapter 9
Discussion
In this chapter the results are discussed in terms of accuracy, image quality
and time requirements.
9.1 Contrast adjustment
When using the h-mapping function, the desired contrast and intensity in
the image can be adjusted, independent of pixel values. The smoothness
of the curve will not result in the black and white areas in the image, that
occur today. An example is given, in gure 9.1, of the present linear mapping function compared to the proposed h-mapping function. The resulting
images are shown in gure 9.2.
Figure 9.1: A comparison between the linear (dashed) and the h-mapping
function (solid).
75
CHAPTER 9. DISCUSSION
76
Figure 9.2: An example of contrast reinforcement with the dierent mapping
functions used. Linear mapping (left) and h-mapping (right).
Contrast will with the h-mapping function be preserved also in the less important areas of the image. See for instance the bone structure in the left
upper corner in gure 9.2.
In the present system, the lowest and highest pixel values in the histogram
are used to compute the window width in the linear mapping of gray values.
That implies that an image will change its contrast, depending on if there is
a dark object present, e.g. a nail, or not. This will not be the case with the
proposed mapping function, see gure 9.3.
Figure 9.3: Mapping of images from the F test sequence, using a=0.63 and
b=2.82. Mapped image without dark objects (left) and the mapped image with
a drill inserted (right).
CHAPTER 9. DISCUSSION
77
9.2 The proposed algorithm
The global detection with the mean-std method is robust and therefore the
combined algorithm functions for both global and local shifts. The proposed
algorithm is therefore the combined algorithm. Below, the algorithm is applied to test sequences with global motions, B and L, and with local motions,
C and F.
9.2.1 Global motion
The proposed algorithm functions well for both the B and the L test sequences. In gure 9.4 and 9.5 comparisons are made between the original
sequence, averaging with n=6 (the present method), and the sequence with
the proposed algorithm applied.
Figure 9.4: A comparison of the proposed method to the present averaging
method on the test sequence B, at global motion. Left: the original sequence.
Middle: the averaged sequence, n = 6. Right: the proposed algorithm, α =
0.3.
Figure 9.5: A comparison of the proposed method to the present averaging
method on the test sequence L, at global motion. Left: the original sequence.
Middle: the averaged sequence, n = 6. Right: the proposed algorithm, α =
0.3.
CHAPTER 9. DISCUSSION
78
9.2.2 Local motion
The proposed algorithm also works for both the C and the F test sequences.
See gure 9.6 and 9.7 below for comparisons to the original and the averaged
sequences.
Figure 9.6: A comparison of the proposed method to the present averaging
method on the test sequence C, at local motion. Left: the original sequence.
Middle: the averaged sequence, n = 6. Right: the proposed algorithm, α =
0.3.
Figure 9.7: A comparison of the proposed method to the present averaging
method on the test sequence F, at local motion. Left: the original sequence.
Middle: the averaged sequence, n = 6. Right: the proposed algorithm, α =
0.3.
The C sequence has a fast local motion and the compensation can handle
this. In the F test sequence, the result is not as good as in the C test
sequence. This is because the drill is moving in its own direction, meaning
that the dierence mask will only cover a small area, just at the end of the
drill. A thin blurring can be seen to the right in gure 9.7. Compared to the
present method, averaging with n=6, the proposed method is still better,
since the position of the drill is more prominent.
CHAPTER 9. DISCUSSION
79
9.2.3 The accuracy of shift calculation
The accuracy of shift calculation can be estimated by adding noise to a test
sequence and see on what noise levels the shift calculated is wrong. This was
done to the B test sequence, by adding random noise to the averaged result
from frames 16-20 (still phantom). To get the shifted image, the cutout was
made dierently. A small shift, (-5,-3) and a large shift (15,-20) was tested.
See table 9.1 and 9.2 below for the results.
SNR[dB]
9.46
6.64
4.77
3.23
1.82
0.42
x
-5.029
-5.082
-5.083
-5.110
-5.181
-5.183
y
-3.047
-3.086
-3.134
-3.168
-3.123
-3.161
Table 9.1: The result of shift calculation for the true shift (-5,-3) when noise
is added.
SNR[dB]
9.46
6.64
4.77
3.23
1.82
0.42
x
14.95
14.94
14.90
14.82
14.70
28.70
y
-19.93
-19.94
-19.95
-19.94
-19.94
-11.82
Table 9.2: The result of shift calculation for the true shift (15,-20) when
noise is added.
It showed that the shift calculation is very robust to noise. Down to an
SNR of 1.82dB, the shift calculated is in the right integer pixel. This is
indicative of that the algorithm can handle even noisier image sequences
than today, but still calculate the true shift. This is interesting, since noisy
images implies a low dose to the patient.
CHAPTER 9. DISCUSSION
80
9.2.4 Limitations
In the proposed algorithm, the mean-std method cannot sort out local motions from no motions. This is probably based on two major things. The
sections have to be quite small to detect a local motion and if the sections
are that small, false shifts are given in the homogenous areas since the noise
is the big dierence between the two compared frames.
The case of global and local shift at the same time has not been included in
the proposed algorithm. If a robust method for local shift detection could
be found also this case could be implemented. On the other hand maybe
the application for this is not that useful, since the patient is hopefully
anaesthetic and still during drilling or nailing.
9.2.5 Time requirements
To estimate the time required for compiling the combined algorithm, the
Matlab function prole is used. When applying the combined function
to the C test sequence, i.e. 31 frames, it takes 45 seconds. Of this time,
32.8 s is used for shift calculations. Other functions that occupy time is
the windowing and FFT calculations. 45 s is not very much, relative to
processing of 31 frames.
To improve the functions included in the combined algorithm, in the meaning
of computational cost, some code parts can be done more eciently. Since
the functions have to be rewritten into C if implemented in the system, not
much time has been put into time eciency.
Chapter 10
Conclusion
The conclusions drawn from this work are summarized in this chapter and a
recommendation to Swemac is done. Future work, including further investigation before implementation, is considered.
10.1 Short summation
In this thesis, an eective algorithm for enhancement of the Biplanar 500
system image sequences is developed. It uses temporal recursive ltering for
noise suppression and compensate for both local and global motions. A new
gray scale mapping function is also presented, that makes the contrast more
smooth in the lower and upper parts of the histogram.
10.2 Recommended algorithm for Biplanar 500
As a rst step it is suggested that a change of the contrast adjustment is
done. A change of mapping function to the proposed one should be quite
easy to carry through. The evaluation of the results show that orthopedists
prefer the proposed mapping function compared to the present one. It is also
suggested that the parameters of the curve shall be adjustable and available
to the user, to further comply with the orthopedists opinions.
The algorithm that is recommended to Swemac is the combined algorithm,
that can handle both global and local motions. The results show that the
motion artifacts, seen as blurring in the averaged image sequences of today,
are essentially reduced with this algorithm applied.
81
CHAPTER 10. CONCLUSION
82
The method of shift computing, with the phase correlation method, is robust
to noise and is very accurate. Compared to pixel-based motion compensation, the proposed compensation is to prefer. The results from the accuracy
test of shift calculation, speaks for that high quality image sequences may
be received even if the dose is lower than today.
10.3 Future work
• The proposed algorithm cannot yet be used on the circular, full-scale
image sequences, because of the global normalization. This could be
solved by instead of a full intensity square use the circular mask of the
image and let it follow the algorithm. For the local motion compensation this is not a problem since the whole image is subtracted.
• Perhaps the image sequences can be improved further by applying high
pass ltering. The resulting sequences are that noiseless and sharp,
that the grainy eect of unsharp masking should not appear.
• The algorithm has to be converted into C-code to t in the Biplanar
500 system. Some calculations are in Matlab easy to carry through,
since some operations are predened in functions. In C, the FFT operation for instance does not have a predetermined function so it has
to be constructed. The time requirement will be reduced a lot when
implemented in C code. Probably the algorithm can be applied in
real-time, as desired.
10.3.1 Further investigation before usage of the algorithm
• A lot more testing is required before using the algorithm in the system.
Also a more extensive study should be done, to more eciently respond
to the orthopedists opinions.
• The possibilities to use a progressive camera instead of an interlaced
should be investigated. There is no reason to use interlaced format
when using computer monitors for presentation. This could improve
the result even more and make the processing less computationally
demanding.
• The possibilities to use a square image, with the size of a second power,
for instance 512x512, should be discussed. That is another way to
reduce time requirements.
List of Figures
2.1
2.2
2.3
Schematic overview of the uoroscopy chain. . . . . . . . . . .
The X-ray tube and its components. . . . . . . . . . . . . . . .
The Image Intensier and its components. . . . . . . . . . . .
Schematic overview of the Biplanar 500 system. . . . . . . . .
The Biplanar 500 system. . . . . . . . . . . . . . . . . . . . .
Schematic overview of the acquisition into Matlab. . . . . . .
Example of an input image. . . . . . . . . . . . . . . . . . . .
The principle of interlaced video. . . . . . . . . . . . . . . . .
Interlacing artefact when moving a lead disc in the X-ray eld.
An input image and its corresponding spectrum. . . . . . . . .
The noise from the CCD-camera and the corresponding spectrum. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.9 A comparison between input and output images in the Biplanar system. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.10 A comparison between the spectrums of the input and output
images in the Biplanar system. . . . . . . . . . . . . . . . . .
3.1
3.2
3.3
3.4
3.5
3.6
3.7
3.8
important lines and areas in the input
phantom used in the test sequences. .
B test sequence. . . . . . . . . . . . .
L test sequence. . . . . . . . . . . . .
F test sequence. . . . . . . . . . . . .
C test sequence. . . . . . . . . . . . .
12
12
13
14
15
15
16
17
18
18
4.1
4.2
4.3
4.4
4.5
4.6
The
The
The
The
The
The
.
.
.
.
.
.
23
24
26
27
28
29
5.1
5.2
5.3
An 1D example to illustrate the shift of phase in the FD. . . .
Original image i1 and the translated image i2 . . . . . . . . . .
The phase correlation matrix Q and its inverse FT, q. . . . .
33
34
35
6.1
6.2
6.3
6.4
6.5
An illustration of dilation and erosion. . . . . . . . . . . . . .
The original test image Lena and its corresponding histogram.
The Lena histogram after linear mapping. . . . . . . . . . . .
The eect of median ltering on a 1D signal. . . . . . . . . .
Block diagram of recursive ltering. . . . . . . . . . . . . . . .
39
40
41
42
44
83
images.
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
6
6
8
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
LIST OF FIGURES
84
6.6
The impulse response of the recursive lter for dierent α. . .
45
7.1
7.2
7.3
7.4
7.5
7.6
7.7
7.8
7.9
7.10
7.11
The quadratic and the circular cos2 windows. . . . . . . . . .
The eect of windowing before transformation. . . . . . . . . .
The h-mapping function for dierent a and b. . . . . . . . . .
Contrast adjusted image using the h-mapping function. . . . .
The method of deinterlacing. . . . . . . . . . . . . . . . . . . .
Recursive ltering of test sequence B. . . . . . . . . . . . . . .
Temporal averaging of test sequence C. . . . . . . . . . . . . .
An example of q and Q, used in the shift calculation. . . . . .
Cutting the impulse response upon motion in test sequence B.
The normalization of a global motion compensated sequence. .
The LP-eect that will occur in the impulse response, in the
worst case. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Frame no 8 and 9 of the C test sequence. . . . . . . . . . . . .
The thresholded dierence image, the opened mask and the
LP-ltered mask. . . . . . . . . . . . . . . . . . . . . . . . . .
Comparison between pure recursive ltering and with the local
motion compensation applied. . . . . . . . . . . . . . . . . . .
The local motion compensation with and without deinterlacing.
The division of a frame into sections. . . . . . . . . . . . . . .
The shifts computed in each section at no motion and global
motion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
An (x,y) plot of the translations at no motion and at global
motion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
49
50
51
52
52
54
55
57
58
59
8.1
8.2
Comparison between linear- and h-mapping. Low contrast. . .
Comparison between linear- and h-mapping. High contrast. . .
68
69
9.1
A comparison between the linear (dashed) and the h-mapping
function (solid). . . . . . . . . . . . . . . . . . . . . . . . . .
An example of contrast reinforcement with the dierent mapping functions used. Linear mapping (left) and h-mapping
(right). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Mapping of images from the F test sequence, using a=0.63
and b=2.82. Mapped image without dark objects (left) and the
mapped image with a drill inserted (right). . . . . . . . . . .
A comparison of the proposed method to the present averaging
method on the test sequence B. . . . . . . . . . . . . . . . . .
A comparison of the proposed method to the present averaging
method on the test sequence L. . . . . . . . . . . . . . . . . .
A comparison of the proposed method to the present averaging
method on the test sequence C. . . . . . . . . . . . . . . . . .
7.12
7.13
7.14
7.15
7.16
7.17
7.18
9.2
9.3
9.4
9.5
9.6
60
61
61
62
62
63
64
65
75
76
76
77
77
78
LIST OF FIGURES
85
A comparison of the proposed method to the present averaging
method on the test sequence F. . . . . . . . . . . . . . . . . .
78
A.1 Block scheme of the detection of global motion. . . . . . . . .
A.2 Block scheme of the local motion compensation. . . . . . . . .
A.3 Block scheme of the global motion compensation. . . . . . . .
91
92
93
9.7
List of Tables
4.1
Facts about the test sequences used. . . . . . . . . . . . . . . .
8.1
8.6
8.7
Result of manual contrast adjustment of the h-mapping function parameters. . . . . . . . . . . . . . . . . . . . . . . . . . .
Result of the comparison between the linear and the h-mapping
function. Low contrast. . . . . . . . . . . . . . . . . . . . . . .
Result of the comparison between the linear and the h-mapping
function. High contrast. . . . . . . . . . . . . . . . . . . . . .
The global motion compensated test sequences used in the evaluation study. . . . . . . . . . . . . . . . . . . . . . . . . . . .
The local motion compensated test sequences used in the evaluation study. . . . . . . . . . . . . . . . . . . . . . . . . . . .
The opinions of the dierent versions of test sequence B. . . .
The opinions of the dierent versions of test sequence C. . . .
9.1
9.2
The result of shift calculation when noise is added. Small shift. 79
The result of shift calculation when noise is added. Large shift. 79
8.2
8.3
8.4
8.5
87
25
68
69
70
71
71
71
72
Bibliography
[1] Aishy Amer and Hartmut Schröder. A new video noise reduction algorithm using spatial subbands. In Electronics, Circuits, and Systems,
1996. ICECS '96.
[2] Ronald N. Bracewell. The Fourier transform and its applications.
McGraw-Hill Book Co., 2 edition, 1986.
[3] Eastman Kodak Company. The Fundamentals of Radiography. Health
Siences Markets Division, Rochester, New York, 12 edition, 1980. Publication No. M1-18.
[4] Per-Erik Danielsson, Olle Seger, Maria Magnusson Seger, and Ingemar
Ragnemalm. Bildanalys, TSBB52, 2004 Kompendium. Department
of Electrical Engineering, Linköping University, 2004.
[5] P P Dendy and B Heaton. Physics for diagnostic radiology. Institute
of physics publishing, 2 edition, 1999. ISBN 0-7503-0590-8.
[6] Per-Erik Danielsson et al. Bilder och grak, 2003 Kompendium. Department of Electrical Engineering, Linköping University, 2003.
[7] Gösta H. Granlund and Hans Knutsson. Signal Processing for Computer
Vision. Kluwer Academic Publisher, 1 edition, 1995. ISBN 0-7923-95301.
[8] Luke's Video Guide. Interlacing, Used on the 1st of March, 2006.
http://neuron2.net/LVG/interlacing.html.
[9] William Scott Hoge. A Subspace Identication Extension to the Phase
Correlation Method. 2003. IEEE.
[10] Bertil Jacobson. Medicin och teknik. Studentlitteratur, 1995. ISBN
91-630-3338-0.
[11] K. Jostschulte, A. Amer, M. Schu, and H. Schröder. Perception adaptive temporal tv-noise reduction using contour preserving prelter techniques. In IEEE Transactions on Consumer Electronics, volume 44,
pages 10911096, Phoenix, AZ, USA, August 1998. IEEE.
89
BIBLIOGRAPHY
90
[12] John C. Russ. Image processing handbook. CRC Press LLC, 4 edition,
2002.
[13] Sune Söderkvist and Lars-Erik Ahnell. Tidsdiskreta signaler och system.
Sune Söderkvist, 2 edition, 1994.
[14] Gerard J. Tortora and Sandra Reynolds Grabowski. Principles of
Anatomy and Physiology. John Wiley & Sons, Inc., 10 edition, 2003.
ISBN 0-471-41501-4.
[15] Keith Wiley. How Unsharp Masking and Laplacian Sharpening Work,
Used on the 23rd of February, 2006.
http://www.unm.edu/ keithw/astroPhotography/imageSharpening.html.
[16] Jabri Kadri N. Wilson, David L. and Aufrichtig Richard. Preception
of temporally ltered x-ray uoroscopy images. IEEE Transactions on
medical imaging, 18(1), January.
Appendix A
The combined algorithm
Figure A.1: Detection of global motion. Depending on the result, the algorithm carry the global or the local compensation through. T is a time delay
element.
91
APPENDIX A. THE COMBINED ALGORITHM
92
Figure A.2: Block scheme of the local motion compensation. T is a time
delay element.
APPENDIX A. THE COMBINED ALGORITHM
93
Figure A.3: Block scheme of the global motion compensation. T is a time
delay element.
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement