Sequential Filter Trees for Ecient 2D 3D and 4D Orientation Estimation

Sequential Filter Trees for Ecient 2D 3D and 4D Orientation Estimation
Sequential Filter Trees for
Ecient 2D 3D and 4D
Orientation Estimation
Mats Andersson Johan Wiklund Hans Knutsson
LiTH-ISY-R-2070
1998
Sequential Filter Trees for Ecient
2D 3D and 4D Orientation Estimation
Mats Andersson Johan Wiklund Hans Knutsson
Computer Vision Laboratory
Linkoping University, Sweden
Abstract
A recursive method to condense general multidimensional FIR-lters
into a sequence of simple kernels with mainly one dimensional extent
has been worked out. Convolver networks adopted for 2, 3 and 4D
signals is presented and the performance is illustrated for spherically
separable quadrature lters. The resulting lter responses are mapped to
a non biased tensor representation where the local tensor constitutes
a robust estimate of both the shape and the orientation (velocity) of the
neighbourhood.
A qualitative evaluation of this General Sequential Filter concept
results in no detectable loss in accuracy when compared to conventional
FIR (Finite Impulse Response) lters but the computational complexity
is reduced several orders in magnitude. For the examples presented in
this paper the attained speed-up is 5, 25 and 300 times for 2D, 3D
and 4D data respectively The magnitude of the attained speed-up implies
that complex spatio-temporal analysis can be performed using standard
hardware, such as a powerful workstation, in close to real time.
Due to the soft implementation of the convolver and the tree structure
of the sequential ltering approach the processing is simple to recongure
for the outer as well as the inner (vector length) dimensionality of the
signal. The implementation was made in AVS (Application Visualization
System) using modules written in C.
Contents
1
2
3
4
5
6
7
8
9
A
B
C
D
Introduction
General Sequential Filters
Optimization of Filter Coecients
Tensor Representation and Error Estimation
A Filtering Structure for 2D Signals
A Filtering Structure for 3D Signals
A Filtering Structure for 4D Signals
Implementation Aspects
Acknowledgment
Sequential lters for 2D
Sequential Filters for 3D
Sequential Filters for 4D
References
1
2
2
4
6
9
12
15
15
16
17
18
20
1 Introduction
In multidimensional signal processing, the main part of the computational
power is usually spent on the initial ltering. The lters do for natural reasons
need to be of the same `dimensionality' as the signal and for 3D and 4D data,
such as time sequences and time sequences of volumes, the number of lter coecients increases dramatically and often limits the practical (interactive) use
of the algorithm. In this paper (see also [9]) a new recursive method to optimize
general multidimensional lters has been worked out which uses only a small
fraction of the number of lter coecients that is required by conventional ltering. The attained speed up implies that multidimensional signal analysis can
be performed using standard hardware, such as a powerful workstation, in close
to real time. An observation that is very important for the proposed approach
is that the performance of a convolver, implemented in standard hardware is directly proportional to the number of kernel coecients, but independent of the
spatial coordinates for these coecients. A software convolver will also be independent of built in limitations in special purpose hardware and can by simple
means be recongured for signals of dierent dimensionality and vector length,
see [10] for more details.
Using N coecients it is always possible to nd a best approximation to a
given lter using all coecients at once for a single lter. In many cases, however, a far more ecient way of attaining essentially the same lter is to use the
same number of coecients but distribute them over a number, (M ), of smaller
lters. These lters are then applied in sequence (sequential convolution) to
obtain the nal lter response.
The proposed method involves the following three steps:
1. Determine the number, M , of sequential lters that is appropriate for the
present application.
2. Determine the distribution of the N coecients, i.e. determine the coordinates of the coecients for each of the M sequential lters.
3. Optimize the values of the N coecients (distributed over the M lters)
so that the combined eect of the lter sequence matches the reference
lter as closely as possible.
To optimize these steps simultaneously is a complex problem indeed and an
overall optimal solution has not been found, (and probably never will be). In
order to reduce the complexity of the problem the number of lter components
and the distribution of coecient coordinates are performed by intuition and by
taking advantage of the symmetry properties of the reference lter. Given the
number of lter M and the spatial coordinates of each lter the last step is still
cumbersome. A recursive method is proposed that in each step optimize each
lter component in relation to the remaining M , 1 components. Convergence of
the algorithm is fast and typically only 4-6 iterations are needed. This sequential
lter optimization method is applied to polar separable quadrature lters for 2D,
3D and 4D signals and a convolver tree structure and performance is presented.
1
2 General Sequential Filters
The objective of introducing sequential ltering is to approximate an ideal
lter, F (u), by M lter components, Fk (u). In the Fourier domain this is
expressed as:
F (u) F (u) =
M
Y
k=1
Fk (u)
(1)
where u is the frequency variable. This operation is meaningful if the lters
Fk (u) can be implemented using a considerable smaller number of kernel coecients in comparison to a direct (M = 1) implementation of F (u). In the
spatial domain eq. (1) corresponds to:
f () = f1 () f2 () : : : fM ()
(2)
where are spatial coordinates. The lter function Fk (u) of a convolution kernel
with Nk coecients is given by its Fourier transform
Fk (u) =
Nk
X
n=1
fk (n ) exp(,i n u)
(3)
where n dene the coordinates where fk () 6= 0.
Note that so far no mention has been made on how these coordinates are
chosen. This decision depend on the number of lter components M as well as
the total number of kernel coecients N . In most cases the symmetry of F (u)
reduce the number of possibilities to a manageable level. The coecients of a
lter component can for example be concentrated to a line, a circle or just be
spread sparsely to cover a large region.
3 Optimization of Filter Coecients
Consider for the moment that the number of lter components M and the
coordinates for the nonzero coecients for each lter are dened. The last
step of the sequential lter optimization procedure is consequently to assign a
value to each coecient such that such that the dierence between the reference
function, F (u), and F (u) is minimized according to some distance measure.
This distance is dened as the weighted square of the dierence
D=
X
W 2 (u)jjF (u) , F (u)jj2
= fu : jui j < g
(4)
The purpose of the weighting function is to make the importance of a close t
proportional to the energy contribution. For most natural images the energy
spectra decays as W (u) = juj, where 1 < < 3. In this paper = 2, for
further discussion about the weighting function see [6].
2
^
F(u)
W(u)
ε
optimizer
module
F(u)
f( ε)
Figure 1: The optimizer module have three inputs, the ideal lter function F (u),
the frequency weight W (u) and the spatial coordinates The Optimizer Module
Consider for a moment the case where M = 1 in eq. (1). The coecients of
the convolution kernel that minimizes eq. (4) is computed by taking the partial
derivative of D with respect to the N1 kernel coecients of f1 (). Setting
the partial derivatives to zero results in a set of coupled equations from which
the coecients of f1 () are computed. The computations are straightforward
but cumbersome and are for that reason left out here. In g. 1 such a single
level lter optimizer is illustrated as a module having three inputs, the ideal
lter function for the kernel F (u), the weighting function W (u) and the spatial
coordinates for the nonzero coecients . This optimization can unfortunately
not be performed simultaneously for two or more lter components which is
essential for the use of the sequential lter concept. It is, however, possible
to use this method to optimize a single lter component with respect to the
present value of the other M , 1 components. A recursive use of the lter
optimization method can consequently be expected to converge rapidly if the
coecient coordinates of the M lter components enable a sucient degree of
freedom.
Sequential Filter Optimization
To extend the kernel optimizer for recursive use require some considerations
as the ideal lter function and weight function for a certain lter component
depends on the reference function F (u) as well as the current value of the
other M , 1 lter components. Consider the optimization of lter component
k 2 [1; M ]. The ideal lter function F (u) is expressed as
Fk (u) = QF (u)
(5)
F
(
u
)
l6=k l
since in the ideal case
F (u) =
M
Y
k=1
Fk (u)
(6)
according to eq. (1). For the weight function W (u) the relation is not as obvious
as for the ideal function but for reasons that will be apparent later on the weight
3
function for F (u) is dened as
Y
Wk (u) = juj,2 Fl (u)
l=
6 k
(7)
where juj,2 relates to the energy spectra of the image. Insertion of eq. (5) and
eq. (7) into the distance measure of eq. (4) results in
D=
X
Q
juj,4 ( l6=k Fl (u))2 [ QF (u)
F (u)
l6=k l
, Fk (u)]2
(8)
where Fk (u) are dened in eq. (3). Simplifying eq. (8) yields:
D=
X
juj,4 [F (u) ,
M
Y
k=1
Fk (u)]2
(9)
which shows that the recursive optimization procedure will tend to minimize
the dierence between the combined eect of the component lters, Fk (u), and
the ideal lter F (u).
A Recursive Algorithm for Sequential Filters
By using the result in eq. (5) and eq. (7) it is obvious how a set the optimizer modules can be connected to perform a recursive sequential lter optimization.The algorithm can be explained in the following steps
1. Set F2 (u); F3 (u); : : : ; FM (u) = 1 8u
2. Dene the nonzero coordinates for f1 (); f2 (); : : : fM ()
3. For k=1 to M(
Q
Fk (u) = F (u)= l6=k Fl (u)
Q
opt[fk ()] :
Wk (u) = juj,2 l6=k Fl (u)
4. Repeat step 3 until convergence (5-6 iterations)
Given enough degree of freedom in the spatial coordinates the algorithm converges rapidly (typically 5-6 iterations). In the following parts of the paper this
algorithm is applied to optimization of polar separable quadrature lters for 2D,
3D and 4D with M ranging from 2 , 4.
4 Tensor Representation and Error Estimation
Before the actual lter implementation is presented it may be appropriate to
discuss how the performance of the resulting lters are estimated. One obvious
4
possibility is to use the distance measure of eq. (4) to estimate the weighted
distortion within the lter. Although such measurements are appropriate it
is more important to estimate the performance of low-level vision tasks such
as estimation of shape and orientation. To enable local orientation estimates
the resulting quadrature lters are mapped to a Local Tensor representation
[7],[6, ch 6]. Let qk be the magnitudes of a set of polar separable quadrature
lters implemented by the proposed sequential method. The local tensor T is
attained by projecting the magnitude of the quadrature lter responses qk onto
a corresponding set of projection tensors Tk
Figure 2: Frame number 5; 14; 21 and 32 of the 64-cube test sequence,
SNR=10dB.
T=
X
k
Tk qk
(10)
where the Tk are symmetrical second order tensors of the same dimensionality
as the signal. The computation of the projection tensor is based on the optimized sequential lter responses and not the ideal lter functions. This process
enables consequently a compensation or whitening of the tensor metric due to
any bias that are present in the lters or the lter orientations. A derivation
of the tensor whitening method is found in [8]. The local tensor T constitute
continuous geometrical representation of the neighbourhood. The distribution
of the eigenvalues of T dene the local shape of the neighbourhood. The most
common case for natural images is the rank 1 case. Such a tensor can be written
T ' 1^e1^eT1
(11)
where ^e1 is the eigenvector of T that corresponds to the largest eigenvalue
1 . In 2D this corresponds to a local neighbourhood consisting of a line or
an edge. In 3D, the environment is interpreted as a planar structure or (for
time sequences) as a moving line. In 4D the rank 1 case corresponds to a
moving plane. For further discussion on interpretation of the eigenvalues see
5
[6, ch 6]. The eigenvector ^e1 corresponding to the largest eigenvalue dene
the orientation of the neighbourhood. To estimate the orientation error in the
tensor representation require a test image containing rank 1 neighbourhoods
and the ideal eigenvectors corresponding to this image. In 3D such a test image
consists by a volume containing concentric spherical shells. Some frames from
this test sequence are displayed in g. 2. Locally all neighbourhoods are planar
and all possible orientations are present. The angular RMS error ' is nally
computed as:
0v
1
u
L
u
X
' = sin,1 @t 21L kx^x^T , e^1 e^1T k2 A
l=1
(12)
where:
x^ is a unit vector in the correct orientation,
e^1 is the eigenvector corresponding to the largest eigenvalue of the estimated tensor T ,
L is the number of points and
' is the angular RMS error.
5 A Filtering Structure for 2D Signals
As mentioned earlier, the examples in this paper are focused on polar separable quadrature lters. The reason for this is the simplicity by which a robust
and continuous tensor representation can be attained from this type of lters.
This does not imply that this ltering approach is limited to this types of lters. In common for 2D, 3D and the 4D case is that
p the radial part of F (u) is
of bandpass type with a center frequency of =(2 2) 1:11 and a bandwidth
of 2 octaves. The orientation of each quadrature lter is dened by a vector n^
and the angular function of F (u) is dened as (u^ n^)2 for (u n^) > 0 and zero
elsewhere. The number of degrees of freedom that are present in a symmetric
second order tensor is
(d + 1)d
(13)
2
where d is the dimensionality of the signal. In two dimensions the minimal
number of lters is consequently three. Using sequential lters where each lter
component only contain few coecients it is of out most importance to make
advantage of all present symmetry aspects. The lter orienting vectors, n^, must
consequently be chosen with care using sequential lters as opposed to conventional FIR-ltering. The most ecient choice is to direct the lters along the
main coordinate axes which takes care of two lters. The second best alternative
is to orient the lters symmetrically between the main coordinate axes which
also gives two alternatives. From earlier experiences on conventional lters we
6
1
0.5
0
25
20
15
10
5
5
5
5
10
15
20
25
1
0.5
0
25
20
15
10
10
15
20
25
Figure 3: Upper part: Resulting Filter function in the FD. Lower part: Deviation from the ideal lter function.
know that using four lters increase the robustness in noisy and ambiguous
neighbourhoods considerably in 2D. The the lter orienting vectors for 2D are
therefore dened as:
n^1 =
( 1; 0 )T
p
n^2 = 1= 2 ( 1; 1 )T
(14)
n^3 =
( 0; 1 )T
p
n^4 = 1= 2 (,1; 1 )T
The following problem is to decide how to separate the quadrature lters in the
above directions into sequential lter set. Since the lters are two dimensional it
seems reasonable that (M=2) lter components should be sucient. The choice
of the coordinates is dependent of the type of lter that is to be implemented
but in this case it is probably most ecient to concentrate the coecients along
a line. Again with the symmetry aspect in mind the nonzero coecient of these
two lters are distributed along with, and orthogonal to the lter orienting
vectors in eq. (14). The sequential ltering structure for sequential lters in 2D
is for this approach expressed as
q1 () = fy () fx()
q2 () = f,x;y() fx;y ()
(15)
q3 () = fx() fy ()
q4 () = fx;y () f,x;y ()
where the indices indicate the distribution of the nonzero coecients of the lter
components. The ltering is performed in two steps, the rst lter component is
7
M=2 (132 coe)
M=1 (648 coe)
SNR Ang. err Density(%) Ang. err Density(%)
0dB
15:99
59.7
15:73
60.7
2dB
9:54
71.7
9:39
72.8
4dB
5:79
81.6
5:71
82.5
6dB
3:66
88.1
3:62
88.9
8dB
2:42
91.9
2:39
92.8
10dB
1:69
93.9
1:66
95.1
15dB
1:01
95.6
0:99
97.3
1
0:90
95.9
0:88
97.7
Table 1: Performance of the sequential 2D lters in comparison to a direct
implementation.
16
Seq
GOP
Angular RMS error (deg)
14
12
10
8
6
4
2
0
0
5
10
15
20
25
SNR dB
Figure 4: Angular RMS error for dierent SNR. The solid line refers to the
sequential (M=2) implementation while the dashed line corresponds to a traditional (M=1) implementation
real valid and is expected to be of low pass type. The second lter component,
which is directed in the main direction of the quadrature lter, is expected to
be complex valued.
The resulting optimized sequential lter set are found below together with
the corresponding projection tensors. The spatial extension of the lter components vary from 5-9 pixels. Note that only four lters need to be computed. The
remaining lters are obtained by mirroring according to the indices in eq. (15).
The reason for f,x;y () and f,x;y () to be tri-diagonal is simply to obtain sufcient sampling density.
The resulting frequency response for q2 () in eq. (15) is displayed in the
8
upper part of g. 3. The lower part of the same gure show the deviation from
the ideal lter F (u). The weighted distance measure of eq. (4) accumulates
to D = 0:22. In comparison to a traditional implementation using a single
complex lter of size 9 9 the distance measure is twice as high. The weighted
distance measure is, however, of limited use when the performance of a lter
set is to be judged. To obtain a more relevant comparison the local tensor T is
computed from these lter responses using the projection tensors below. This
computation where performed for both the sequential lters and the traditional
(M = 1) implementation for dierent SNR. From the tensor data the angular
RMS error was computed according to eq. (12). The result the result is listed in
table (1) and exceeds our most optimistic expectations as there are no detectable
loss in accuracy while the total number of coecients used in the sequential
implementation is 132 as opposed to 648 for a direct implementation. The
computational complexity is consequently reduced 5 times in this example. The
density in table (1) is a graded coverage estimate for the tensor representation.
The density denes to how well the local one dimensional neighbourhoods of the
test image is reected in the tensor representation. In contrast to the angular
RMS error the density only refers to the shape of the neighbourhood and is
computed from the two largest eigen values 1 and 2 of the local tensor T.
P ( 1 ,2 )2 2
P12 1
(16)
Density =
1
The dierence in density between the sequential and the direct method is in
the same magnitude as for the angular RMS error, i.e. insignicant. From the
measurements in table (1) it is concluded that the sequential ltering approach
support an ecient and robust tensor representation in 2D.
6 A Filtering Structure for 3D Signals
Figure 5: The icosahedron, one of 5 platonic polyhedra.
In 3D, the minimal number of lters required to produce a second order tensor representation is six, eq. (13). The apparent choice using conventional lter9
ing is to dene the lter orienting vectors as the vertices of a hemi-icosahedron
[7, 6]. The icosahedron is a regular diametrically symmetric polyhedron (g. 5)
which implies that the lters are equally spread. The six lter orienting vectors
dened by the icosahedron are
n^1 = c ( a
n^3 = c ( b
n^5 = c ( 0
; 0 ; b )T
; a ; 0 )T
; b ; a )T
n^2 = c (,a ; 0 ; b )T
n^4 = c ( b ; ,a ; 0 )T
n^6 = c ( 0 ; b ; ,a )T
(17)
where:
a=2 p
b = (1 + 5)
p
c = (10 + 2 5),1=2
These orientations are unfortunately not suitable for a sequential implementation because of the symmetry aspects discussed earlier. In agreement with
the 2D case the rst choice of lter orienting vectors for sequential ltering in
3D are along the coordinate axes (3 possibilities) and the second symmetrically
between two coordinate axes (6 possibilities). Although 6 lters are sucient
in theory we choose to use nine lters for two reasons. The rst reason is that
the orientation error due to the sequential lters not being polar separable is
reduced by increasing the number of lters. The second reason is that using the
convolver tree structure presented below, the increase in computational complexity by computing 9 lter responses instead of 6 is very small. The 9 lter
orienting vectors for 3D are consequently dened as
p
n^1 = 1=p2 ( 0; ,1;
n^2 = 1= 2 ( 0; 1;
n^3 = p ( 0; 1;
n^4 = 1=p2 (,1; 0;
n^5 = 1= 2 ( 1; 0;
n^6 = p ( 0; 0;
n^7 = 1=p2 ( 1; 1;
n^8 = 1= 2 (,1; 1;
n^9 =
( 1; 0;
1
1
0
1
1
1
0
0
0
)T
)T
)T
)T
)T
)T
)T
)T
)T
These lter orientations are not exactly equally spread as in the lters orientations in eq. (17). To eliminate any orientation bias in the local tensor T the
projection tensors are computed using the tensor whitening method discussed
earlier, [8]. The sequential ltering structure for the 3D case is consequently
10
One dimensional
low−pass filters
fz
fy
Image
sequence
fx
One dimensional
quadrature filters
fy
fx
q9
f x,y
f−x,y
q8
f−x,y
f x,y
q7
fx
fz
q6
f−x,z
f x,z
q5
f x,z
f−x,z
q4
fz
fy
q3
f−y,z
f y,z
q2
f y,z
f−y,z
q1
Figure 6: 21 convolvers organized in 3 levels. Only 3 lters are required on the
rst level.
expressed as
q1 () = fx() fy;z () f,y;z ()
q2 () = fx() f,y;z () fy;z ()
q3 () = fx() fz () fy ()
q4 () = fy () fx;z () f,x;z ()
q5 () = fy () f,x;z () fx;z ()
q6 () = fy () fx() fz ()
q7 () = fz () f,x;y () fx;y ()
q8 () = fz () fx;y () f,x;y ()
q9 () = fz () fy () fx()
(18)
Each quadrature lter response are computed in three steps. The rst two
components are real valued and are both oriented in directions orthogonal to the
lter orienting vectors. The last component is complex valued and directed along
the main direction of the ideal lter. In the sequential lter organization above,
the initial ltering step is identical for several lters. This observation is used
to dene an ecient convolver tree structure of g. 6 consisting of 21 convolvers
on 3 levels. Using only 3 convolvers at the top level. The optimized sequential
lter components are displayed below. In g. 7 the resulting frequency response
for q7 () is illustrated in the u1; u2-plane. The angular RMS error of the local
tensor were computed for dierent SNR and the result is given in table 2. The
11
1
0.5
0
20
15
10
5
0
20
25
10
15
0
5
20
25
10
15
0
5
1
0.5
0
20
15
10
5
0
Figure 7: Upper part: Frequency response for q7 () in the u1 ; u2-plane. Lower
part: Deviation from the ideal lter function.
left column correspond to a lter set consisting of 9 quadrature lters where each
quadrature lter response are computed sequentially by three lter components
according to g. 6 The computation of nine quadrature lter responses requires
345 multiplications. If on the other hand conventional quadrature lters are
used, six complex lters of size 9 9 9 require almost 9000 multiplications.
The computational load is consequently reduced 25 times in this example while
the dierence in accuracy is insignicant, see table 2.
SNR
1
10dB
0dB
M =3
345 coe.
0:76
3:02
9:35
M =1
8748 coe.
0:71
3:03
9:69
Table 2: Angular RMS error in degrees for a sequential lter with 3 components
and the corresponding lter implemented by a single component.
7 A Filtering Structure for 4D Signals
In 4D, the choice of lter orienting vectors is quite simple. From eq. (13) we
conclude the the the minimum number of lters required to produce a tensor
representation in 4D is 10. If the lters are localized symmetrically between
12
two main coordinate axes this results in 12 possibilities, which is sucient. The
lter orienting vectors in 4D are dened as
p
n^1 = 1=p2 ( 0; 0;
n^2 = 1=p2 ( 0; 0;
n^3 = 1=p2 ( 0; 1;
n^4 = 1= 2 ( 0; ,1;
p
n^5 = 1=p2 ( 1; 0;
n^6 = 1=p2 ( 1; 0;
n^7 = 1=p2 ( 1; 0;
n^8 = 1= 2 (,1; 0;
p
n^9 = 1=p2 ( 0; 1;
n^10 = 1=p2 ( 0; 1;
n^11 = 1=p2 ( 1; 1;
n^12 = 1= 2 (,1; 1;
1; 1 )T
1; ,1 )T
1; 0 )T
1; 0 )T
0; 1 )T
0; ,1 )T
1; 0 )T
1; 0 )T
0; 1 )T
0; ,1 )T
0; 0 )T
0; 0 )T
(19)
These 12 lter orientations also correspond to the vertices of the 24-cell, one of
three regular polyhedra in 4D. The above lter orientations are consequently
equally spread in the 4D signal space which simplies to computation of the
projection tensors.
In agreement with the 3D case the sequential ltering structure for 4D signals
is expressed as
q1 () = fx() fy () f,z;w () fz;w ()
q2 () = fx() fy () fz;w () fz;,w ()
q3 () = fx() fw () f,y;z () fy;z ()
q4 () = fx() fw () fy;z () f,y;z ()
q5 () = fy () fz () f,x;w () fx;w ()
q6 () = fy () fz () fx;w () fx;,w ()
q7 () = fy () fw () f,x;z () fx;z ()
q8 () = fy () fw () fx;z () f,x;z ()
q9 () = fz () fx() f,y;w () fy;w ()
q10 () = fz () fx() fy;w () fy;,w ()
q11 () = fz () fw () f,x;y () fx;y ()
q12 () = fz () fw () fx;y () f,x;y ()
The ltering is performed in 4 levels where the three rst are real valued and
the last lter is complex. The distribution of the nonzero coecients within
each lter component is indicated by the indices. Note that the indices of the
13
One dimensional
quadrature filters
One dimensional
low−pass filters
f x,y
f−x,y
q12
f−x,y
f x,y
q11
f y,w
f y,−w
q 10
f−y,w
f y,w
q9
f x,z
f −x,z
q
f −x,z
f x,z
q7
f x,w
f x,−w
q6
f−x,w
f x,w
q5
f y,z
f−y,z
q4
f−y,z
f y,z
q3
f z,w
f z,−w
q2
f−z,w
f z,w
q1
fw
fz
fx
Time sequence
fw
of volumes
8
fy
fz
fw
fx
fy
Figure 8: Convolver organization in 4D, 33 convolvers in 4 levels
components within each lter are orthogonal and that in the two rst ltering
steps the same lter combination occurs several times. I g. 8 a 4D convolver
structure in 4 levels is displayed. By careful combination of the results in the
sequential ltering structure only 3 and 6 convolvers are required at the rst and
second level respectively. The optimized lters with center frequency %0 = 1:11
and bandwidth B = 2 octaves are found below. At this point it starts to get
really interesting to compare the the computational complexity of sequential
convolution to traditional implementations. The spatial extent of the optimized
lter components vary from 5-9 pixels. To compute a single quadrature lter
response require 7+7+5+19+18 = 56 real multiplications. Using the convolver
tree structure of g. 8 reduces this gure further by about 15 % resulting in 567
multiplications for 12 lters. In conventional FIR-ltering a complex lter of
size 9 9 9 9 require almost 160 000 real valued multiplication to compute
12 quadrature lter responses. The sequential approach is consequently almost
280 times more ecient.
14
2D Image
2D:
3D:
Time sequence of images
or a 3D volume
4D:
Time sequence
of volumes
Figure 9: Sequential convolver organization in 2D,3D and 4D
8 Implementation Aspects
The general convolver module is implemented in C with the graphical user
interface in AVS (Application Visualization System), see [10] for more details.
The tree structure of the convolver organization, g. 9, provide an ecient
optimization of the processing performance for a wide variety computer architectures. The soft implementation of the convolver network enable a simple
reconguration for dierent types of input data (dierent outer dimensions) e.g
2D image, 3D spatiotemporal image sequences (3D volumes) and 4D volume
sequences. The vector length (i.e. the inner dimension) of the kernel data is
likewise simple to adapt from e.g scalars, RGB-signals, 2D vector elds, 3D
vector elds, tensor elds etc.
9 Acknowledgment
The support of the National Swedish Board for Technical Development,
NUTEK (project 9305120-1) is greatfully acknowledged.
15
A Sequential lters for 2D
These are the sequential lter components used in the experiments. From
these four lter components all lters in eq. (15) can be produced by mirroring
according to the indices. The coecients are converted to integers to support
an ecient implementation.
fx () = 1256 4195 14278 24981 14278 4195 1256
2
66
fxy () = 6
64
613
8593
24766
8593
613
3
77
77
5
Re [fx ()] = ,287 ,673 ,1507 92 4748 92 ,1507 ,673 ,287
Im [fx ()] = 607 5 ,405 ,3683 0 3683 405 ,5 ,607
2
66
66
932
Re [fxy ] = 66
66
,689 ,1454
4 ,246 ,869 ,689
3
,246 ,187
,689 ,869 ,246 77
77
932 ,1454 ,689
5035
932
77
77
5
932
,187 ,246
2
66
66
2144
,2144
Im [fxy ] = 66
66
,273 ,2101 ,2144
4 219 139 ,273
484
3
,219 ,484
273 ,139 ,219 77
7
2101
2144
273
77
77
75
219
Projection tensors in 2D
T1 =
T3 =
0:75 0:00
0:00 ,0:25
,0:25 0:00
0:00 0:75
T2 =
T4 =
16
0:25 0:50
0:50 0:25
0:25 ,0:50
,0:50 0:25
B Sequential Filters for 3D
fx () = 1277 4060 15481 28833 15481 4060 1277
2
66
fxy () = 6
64
24
492
1565
492
24
3
77
77
5
Re [fx ()] = ,121 ,237 ,612 ,51 2041 ,51 ,612 ,237 ,121
Im [fx ()] = 206 ,28 ,75 ,1569 0 1569 75 28 ,206
2
66
66
6454
Re [fxy ] = 66
66
,4817 ,12369
4 ,1641 ,5819 ,4817
,1641 ,1538
,4817 ,5819 ,1641
6454 ,12369 ,4817
39469
6454
6454
,1538 ,1641
2
66
66
17177
,17177
Im [fxy ] = 66
66
,1495 ,14544 ,17177
4 1297 1106 ,1495
2447
1297
17
3
77
77
77
77
5
3
,1297 ,2447
1495 ,1106 ,1297 77
7
14544
17177
1495
77
77
75
Projection Tensors in 3D
These projection tensors where computed using the tensor whitening method
([8]) to compensate for any orientation bias due to uneven spread of the lters.
0 0:2690
1
0 0:2690
0
0
0
0
0 1:0042 1:5313 A
0 1:0042 ,1:5313
T1 = @
T2 = @
0 1:5313 1:0042
0 ,1:5313 1:0042
0 ,0:1804
0
T3 = @
0
T5 = @
0
1:0042
0
1:5313
0
0
2:2538
0
0 ,0:1804
0
0:2690
0
0 1:0042 ,1:5313
T7 = @ ,1:5313 1:0042
0
T9 = @
0
0
1:5313
0
1:0042
1
A
0
T4 = @
1
A
0 ,0:1804
0
0 ,0:1804
T6 = @
1
0
0A
0:2690
2:2538
0
0
0 ,0:1804
0
0
0 ,0:1804
0
T8 = @
1
A
1:0042
0
,1:5313
0 ,1:5313
0:2690
0
0 1:0042
0
0
0
0
2:2538
1:0042
1:5313
0
1:5313
1:0042
0
0
0
0:2690
C Sequential Filters for 4D
fx() = 201 1891 14815 31267 14815 1891 201
2 ,963
66
fxy () = 6
64
2
66
66
Re [fxy ] = 66
66
4
6837
28702
6837
,963
3
77
77
5
38 345
,2980 ,1804 38
2811 ,11172 ,2980
2811 25786
2811
,2980 ,11172 2811
38 ,1804 ,2980
345
38
18
3
77
77
77
77
5
1
A
1
A
1
A
1
A
2
66
66
Im [fxy ] = 66
66
4 963
785
3
,963 ,785
,1310 ,2221 ,963 77
77
12279 7172 ,1310
77
77
5
,12279
12279
1310 ,7172 ,12279
2221
963
1310
Projection Tensors in 4D
0 ,0:50
0
B
0
,
0
:
50
T1 = B
@ 0
0
1
CC
A
0 ,0:50
0
B
0
,
0
:
50
T2 = B
@ 0
0
0 ,0:50
B 0
T3 = B
@ 0
1
CC
A
0 ,0:50
0
0
B
0
1
:
00
,
1
:
50
T4 = B
@ 0 ,1:50 1:00
1
CC
A
0 1:00
0
0
B
0
,
0
:
50
0
T5 = B
@ 0
0 ,0:50
1
CC
A
0
B
T6 = B
@
1
CC
A
0 1:00
0
B
0
,
0
:
50
T7 = B
@ 1:50
0
1
CC
A
0 1:00
0 ,1:50
B
0
,
0
:
50
0
T8 = B
@ ,1:50
0 1:00
1
CC
A
1
CC
A
0 ,0:50
B 0
T10 = B
@ 0
1
CC
A
1
CC
A
0 1:00 ,1:50
0
B
,
1
:
50
1
:
00
0
T12 = B
@ 0
0 ,0:50
0
0
0
0
1:00 1:50
0 1:50 1:00
0
0
0
0
1:00 1:50
0
1:50 1:00
0
0
0
0 ,0:50
1:50
0
0
0 ,0:50
B 0
T9 = B
@ 0
0 1:00
B 1:50
T11 = B
@ 0
1:50
0
0
0 1:00
1:50
0
0
0
1:00
0
0
0 ,0:50
0
0
0
1:00
0 1:50
0 ,0:50
0
0 1:50
0 1:00
1:50
0
0
1:00
0
0
0 ,0:50
0
0
0
0 ,0:50
19
0
0
0
0
0
1:00 ,1:50
0 ,1:50 1:00
0
0
0
0
0 ,0:50
0
1:00
0
0 ,1:50
0 ,0:50
0
0
0
0 ,0:50
0
,1:50
0
0 1:00
0
0
0
0
0
0 ,0:50
0
0
0
1:00
0 ,1:50
0 ,0:50
0
0 ,1:50
0 1:00
0
0
0
0
0
0 ,0:50
1
CC
A
1
CC
A
References
[1] M. Andersson. Controllable Multidimensional Filters in Low Level Computer Vision. PhD thesis, Linkoping University, Sweden, S{581 83
Linkoping, Sweden, September 1992. Dissertation No 282, ISBN 91{7870{
981{4.
[2] M. T. Andersson and H. Knutsson. Controllable 3-D Filters for Low Level
Computer Vision. In Proceedings of the 8th Scandinavian Conference on
Image Analysis, Troms, May 1993. SCIA.
[3] R. Bracewell. The Fourier Transform and its Applications. McGraw-Hill,
2nd edition, 1986.
[4] D. E. Dudgeon and R. M. Mersereau. Multidimensional Digital Signal Processing. Prentice-Hall signal processing series. Prentice-Hall, 1984. ISBN
0-13-604959-1.
[5] G. H. Granlund. In search of a general picture processing operator. Computer Graphics and Image Processing, 8(2):155{178, 1978.
[6] G. H. Granlund and H. Knutsson. Signal Processing for Computer Vision.
Kluwer Academic Publishers, 1995. ISBN 0-7923-9530-1.
[7] H. Knutsson. Representing local structure using tensors. In The 6th Scandinavian Conference on Image Analysis, pages 244{251, Oulu, Finland, June
1989. Report LiTH{ISY{I{1019, Computer Vision Laboratory, Linkoping
University, Sweden, 1989.
[8] H. Knutsson and M. Andersson. Robust N-Dimensional Orientation Estimation using Quadrature Filters and Tensor Whitening. In Proceedings of
IEEE International Conference on Acoustics, Speech, & Signal Processing,
Adelaide, Australia, April 1994. IEEE. LiTH-ISY-R-1798.
[9] H. Knutsson, M. Andersson, and J Wiklund. Advanced Filter Design. In
Proceedings of the Scandinavian Conference on Image analysis, June 1999.
Submitted.
[10] J. Wiklund and H. Knutsson. A Generalized Convolver. In Proceedings
of the 9th Scandinavian Conference on Image Analysis, Uppsala, Sweden,
June 1995. SCIA.
20
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement