Sequential Filter Trees for Ecient 2D 3D and 4D Orientation Estimation Mats Andersson Johan Wiklund Hans Knutsson LiTH-ISY-R-2070 1998 Sequential Filter Trees for Ecient 2D 3D and 4D Orientation Estimation Mats Andersson Johan Wiklund Hans Knutsson Computer Vision Laboratory Linkoping University, Sweden Abstract A recursive method to condense general multidimensional FIR-lters into a sequence of simple kernels with mainly one dimensional extent has been worked out. Convolver networks adopted for 2, 3 and 4D signals is presented and the performance is illustrated for spherically separable quadrature lters. The resulting lter responses are mapped to a non biased tensor representation where the local tensor constitutes a robust estimate of both the shape and the orientation (velocity) of the neighbourhood. A qualitative evaluation of this General Sequential Filter concept results in no detectable loss in accuracy when compared to conventional FIR (Finite Impulse Response) lters but the computational complexity is reduced several orders in magnitude. For the examples presented in this paper the attained speed-up is 5, 25 and 300 times for 2D, 3D and 4D data respectively The magnitude of the attained speed-up implies that complex spatio-temporal analysis can be performed using standard hardware, such as a powerful workstation, in close to real time. Due to the soft implementation of the convolver and the tree structure of the sequential ltering approach the processing is simple to recongure for the outer as well as the inner (vector length) dimensionality of the signal. The implementation was made in AVS (Application Visualization System) using modules written in C. Contents 1 2 3 4 5 6 7 8 9 A B C D Introduction General Sequential Filters Optimization of Filter Coecients Tensor Representation and Error Estimation A Filtering Structure for 2D Signals A Filtering Structure for 3D Signals A Filtering Structure for 4D Signals Implementation Aspects Acknowledgment Sequential lters for 2D Sequential Filters for 3D Sequential Filters for 4D References 1 2 2 4 6 9 12 15 15 16 17 18 20 1 Introduction In multidimensional signal processing, the main part of the computational power is usually spent on the initial ltering. The lters do for natural reasons need to be of the same `dimensionality' as the signal and for 3D and 4D data, such as time sequences and time sequences of volumes, the number of lter coecients increases dramatically and often limits the practical (interactive) use of the algorithm. In this paper (see also [9]) a new recursive method to optimize general multidimensional lters has been worked out which uses only a small fraction of the number of lter coecients that is required by conventional ltering. The attained speed up implies that multidimensional signal analysis can be performed using standard hardware, such as a powerful workstation, in close to real time. An observation that is very important for the proposed approach is that the performance of a convolver, implemented in standard hardware is directly proportional to the number of kernel coecients, but independent of the spatial coordinates for these coecients. A software convolver will also be independent of built in limitations in special purpose hardware and can by simple means be recongured for signals of dierent dimensionality and vector length, see [10] for more details. Using N coecients it is always possible to nd a best approximation to a given lter using all coecients at once for a single lter. In many cases, however, a far more ecient way of attaining essentially the same lter is to use the same number of coecients but distribute them over a number, (M ), of smaller lters. These lters are then applied in sequence (sequential convolution) to obtain the nal lter response. The proposed method involves the following three steps: 1. Determine the number, M , of sequential lters that is appropriate for the present application. 2. Determine the distribution of the N coecients, i.e. determine the coordinates of the coecients for each of the M sequential lters. 3. Optimize the values of the N coecients (distributed over the M lters) so that the combined eect of the lter sequence matches the reference lter as closely as possible. To optimize these steps simultaneously is a complex problem indeed and an overall optimal solution has not been found, (and probably never will be). In order to reduce the complexity of the problem the number of lter components and the distribution of coecient coordinates are performed by intuition and by taking advantage of the symmetry properties of the reference lter. Given the number of lter M and the spatial coordinates of each lter the last step is still cumbersome. A recursive method is proposed that in each step optimize each lter component in relation to the remaining M , 1 components. Convergence of the algorithm is fast and typically only 4-6 iterations are needed. This sequential lter optimization method is applied to polar separable quadrature lters for 2D, 3D and 4D signals and a convolver tree structure and performance is presented. 1 2 General Sequential Filters The objective of introducing sequential ltering is to approximate an ideal lter, F (u), by M lter components, Fk (u). In the Fourier domain this is expressed as: F (u) F (u) = M Y k=1 Fk (u) (1) where u is the frequency variable. This operation is meaningful if the lters Fk (u) can be implemented using a considerable smaller number of kernel coecients in comparison to a direct (M = 1) implementation of F (u). In the spatial domain eq. (1) corresponds to: f () = f1 () f2 () : : : fM () (2) where are spatial coordinates. The lter function Fk (u) of a convolution kernel with Nk coecients is given by its Fourier transform Fk (u) = Nk X n=1 fk (n ) exp(,i n u) (3) where n dene the coordinates where fk () 6= 0. Note that so far no mention has been made on how these coordinates are chosen. This decision depend on the number of lter components M as well as the total number of kernel coecients N . In most cases the symmetry of F (u) reduce the number of possibilities to a manageable level. The coecients of a lter component can for example be concentrated to a line, a circle or just be spread sparsely to cover a large region. 3 Optimization of Filter Coecients Consider for the moment that the number of lter components M and the coordinates for the nonzero coecients for each lter are dened. The last step of the sequential lter optimization procedure is consequently to assign a value to each coecient such that such that the dierence between the reference function, F (u), and F (u) is minimized according to some distance measure. This distance is dened as the weighted square of the dierence D= X W 2 (u)jjF (u) , F (u)jj2 = fu : jui j < g (4) The purpose of the weighting function is to make the importance of a close t proportional to the energy contribution. For most natural images the energy spectra decays as W (u) = juj, where 1 < < 3. In this paper = 2, for further discussion about the weighting function see [6]. 2 ^ F(u) W(u) ε optimizer module F(u) f( ε) Figure 1: The optimizer module have three inputs, the ideal lter function F (u), the frequency weight W (u) and the spatial coordinates The Optimizer Module Consider for a moment the case where M = 1 in eq. (1). The coecients of the convolution kernel that minimizes eq. (4) is computed by taking the partial derivative of D with respect to the N1 kernel coecients of f1 (). Setting the partial derivatives to zero results in a set of coupled equations from which the coecients of f1 () are computed. The computations are straightforward but cumbersome and are for that reason left out here. In g. 1 such a single level lter optimizer is illustrated as a module having three inputs, the ideal lter function for the kernel F (u), the weighting function W (u) and the spatial coordinates for the nonzero coecients . This optimization can unfortunately not be performed simultaneously for two or more lter components which is essential for the use of the sequential lter concept. It is, however, possible to use this method to optimize a single lter component with respect to the present value of the other M , 1 components. A recursive use of the lter optimization method can consequently be expected to converge rapidly if the coecient coordinates of the M lter components enable a sucient degree of freedom. Sequential Filter Optimization To extend the kernel optimizer for recursive use require some considerations as the ideal lter function and weight function for a certain lter component depends on the reference function F (u) as well as the current value of the other M , 1 lter components. Consider the optimization of lter component k 2 [1; M ]. The ideal lter function F (u) is expressed as Fk (u) = QF (u) (5) F ( u ) l6=k l since in the ideal case F (u) = M Y k=1 Fk (u) (6) according to eq. (1). For the weight function W (u) the relation is not as obvious as for the ideal function but for reasons that will be apparent later on the weight 3 function for F (u) is dened as Y Wk (u) = juj,2 Fl (u) l= 6 k (7) where juj,2 relates to the energy spectra of the image. Insertion of eq. (5) and eq. (7) into the distance measure of eq. (4) results in D= X Q juj,4 ( l6=k Fl (u))2 [ QF (u) F (u) l6=k l , Fk (u)]2 (8) where Fk (u) are dened in eq. (3). Simplifying eq. (8) yields: D= X juj,4 [F (u) , M Y k=1 Fk (u)]2 (9) which shows that the recursive optimization procedure will tend to minimize the dierence between the combined eect of the component lters, Fk (u), and the ideal lter F (u). A Recursive Algorithm for Sequential Filters By using the result in eq. (5) and eq. (7) it is obvious how a set the optimizer modules can be connected to perform a recursive sequential lter optimization.The algorithm can be explained in the following steps 1. Set F2 (u); F3 (u); : : : ; FM (u) = 1 8u 2. Dene the nonzero coordinates for f1 (); f2 (); : : : fM () 3. For k=1 to M( Q Fk (u) = F (u)= l6=k Fl (u) Q opt[fk ()] : Wk (u) = juj,2 l6=k Fl (u) 4. Repeat step 3 until convergence (5-6 iterations) Given enough degree of freedom in the spatial coordinates the algorithm converges rapidly (typically 5-6 iterations). In the following parts of the paper this algorithm is applied to optimization of polar separable quadrature lters for 2D, 3D and 4D with M ranging from 2 , 4. 4 Tensor Representation and Error Estimation Before the actual lter implementation is presented it may be appropriate to discuss how the performance of the resulting lters are estimated. One obvious 4 possibility is to use the distance measure of eq. (4) to estimate the weighted distortion within the lter. Although such measurements are appropriate it is more important to estimate the performance of low-level vision tasks such as estimation of shape and orientation. To enable local orientation estimates the resulting quadrature lters are mapped to a Local Tensor representation [7],[6, ch 6]. Let qk be the magnitudes of a set of polar separable quadrature lters implemented by the proposed sequential method. The local tensor T is attained by projecting the magnitude of the quadrature lter responses qk onto a corresponding set of projection tensors Tk Figure 2: Frame number 5; 14; 21 and 32 of the 64-cube test sequence, SNR=10dB. T= X k Tk qk (10) where the Tk are symmetrical second order tensors of the same dimensionality as the signal. The computation of the projection tensor is based on the optimized sequential lter responses and not the ideal lter functions. This process enables consequently a compensation or whitening of the tensor metric due to any bias that are present in the lters or the lter orientations. A derivation of the tensor whitening method is found in [8]. The local tensor T constitute continuous geometrical representation of the neighbourhood. The distribution of the eigenvalues of T dene the local shape of the neighbourhood. The most common case for natural images is the rank 1 case. Such a tensor can be written T ' 1^e1^eT1 (11) where ^e1 is the eigenvector of T that corresponds to the largest eigenvalue 1 . In 2D this corresponds to a local neighbourhood consisting of a line or an edge. In 3D, the environment is interpreted as a planar structure or (for time sequences) as a moving line. In 4D the rank 1 case corresponds to a moving plane. For further discussion on interpretation of the eigenvalues see 5 [6, ch 6]. The eigenvector ^e1 corresponding to the largest eigenvalue dene the orientation of the neighbourhood. To estimate the orientation error in the tensor representation require a test image containing rank 1 neighbourhoods and the ideal eigenvectors corresponding to this image. In 3D such a test image consists by a volume containing concentric spherical shells. Some frames from this test sequence are displayed in g. 2. Locally all neighbourhoods are planar and all possible orientations are present. The angular RMS error ' is nally computed as: 0v 1 u L u X ' = sin,1 @t 21L kx^x^T , e^1 e^1T k2 A l=1 (12) where: x^ is a unit vector in the correct orientation, e^1 is the eigenvector corresponding to the largest eigenvalue of the estimated tensor T , L is the number of points and ' is the angular RMS error. 5 A Filtering Structure for 2D Signals As mentioned earlier, the examples in this paper are focused on polar separable quadrature lters. The reason for this is the simplicity by which a robust and continuous tensor representation can be attained from this type of lters. This does not imply that this ltering approach is limited to this types of lters. In common for 2D, 3D and the 4D case is that p the radial part of F (u) is of bandpass type with a center frequency of =(2 2) 1:11 and a bandwidth of 2 octaves. The orientation of each quadrature lter is dened by a vector n^ and the angular function of F (u) is dened as (u^ n^)2 for (u n^) > 0 and zero elsewhere. The number of degrees of freedom that are present in a symmetric second order tensor is (d + 1)d (13) 2 where d is the dimensionality of the signal. In two dimensions the minimal number of lters is consequently three. Using sequential lters where each lter component only contain few coecients it is of out most importance to make advantage of all present symmetry aspects. The lter orienting vectors, n^, must consequently be chosen with care using sequential lters as opposed to conventional FIR-ltering. The most ecient choice is to direct the lters along the main coordinate axes which takes care of two lters. The second best alternative is to orient the lters symmetrically between the main coordinate axes which also gives two alternatives. From earlier experiences on conventional lters we 6 1 0.5 0 25 20 15 10 5 5 5 5 10 15 20 25 1 0.5 0 25 20 15 10 10 15 20 25 Figure 3: Upper part: Resulting Filter function in the FD. Lower part: Deviation from the ideal lter function. know that using four lters increase the robustness in noisy and ambiguous neighbourhoods considerably in 2D. The the lter orienting vectors for 2D are therefore dened as: n^1 = ( 1; 0 )T p n^2 = 1= 2 ( 1; 1 )T (14) n^3 = ( 0; 1 )T p n^4 = 1= 2 (,1; 1 )T The following problem is to decide how to separate the quadrature lters in the above directions into sequential lter set. Since the lters are two dimensional it seems reasonable that (M=2) lter components should be sucient. The choice of the coordinates is dependent of the type of lter that is to be implemented but in this case it is probably most ecient to concentrate the coecients along a line. Again with the symmetry aspect in mind the nonzero coecient of these two lters are distributed along with, and orthogonal to the lter orienting vectors in eq. (14). The sequential ltering structure for sequential lters in 2D is for this approach expressed as q1 () = fy () fx() q2 () = f,x;y() fx;y () (15) q3 () = fx() fy () q4 () = fx;y () f,x;y () where the indices indicate the distribution of the nonzero coecients of the lter components. The ltering is performed in two steps, the rst lter component is 7 M=2 (132 coe) M=1 (648 coe) SNR Ang. err Density(%) Ang. err Density(%) 0dB 15:99 59.7 15:73 60.7 2dB 9:54 71.7 9:39 72.8 4dB 5:79 81.6 5:71 82.5 6dB 3:66 88.1 3:62 88.9 8dB 2:42 91.9 2:39 92.8 10dB 1:69 93.9 1:66 95.1 15dB 1:01 95.6 0:99 97.3 1 0:90 95.9 0:88 97.7 Table 1: Performance of the sequential 2D lters in comparison to a direct implementation. 16 Seq GOP Angular RMS error (deg) 14 12 10 8 6 4 2 0 0 5 10 15 20 25 SNR dB Figure 4: Angular RMS error for dierent SNR. The solid line refers to the sequential (M=2) implementation while the dashed line corresponds to a traditional (M=1) implementation real valid and is expected to be of low pass type. The second lter component, which is directed in the main direction of the quadrature lter, is expected to be complex valued. The resulting optimized sequential lter set are found below together with the corresponding projection tensors. The spatial extension of the lter components vary from 5-9 pixels. Note that only four lters need to be computed. The remaining lters are obtained by mirroring according to the indices in eq. (15). The reason for f,x;y () and f,x;y () to be tri-diagonal is simply to obtain sufcient sampling density. The resulting frequency response for q2 () in eq. (15) is displayed in the 8 upper part of g. 3. The lower part of the same gure show the deviation from the ideal lter F (u). The weighted distance measure of eq. (4) accumulates to D = 0:22. In comparison to a traditional implementation using a single complex lter of size 9 9 the distance measure is twice as high. The weighted distance measure is, however, of limited use when the performance of a lter set is to be judged. To obtain a more relevant comparison the local tensor T is computed from these lter responses using the projection tensors below. This computation where performed for both the sequential lters and the traditional (M = 1) implementation for dierent SNR. From the tensor data the angular RMS error was computed according to eq. (12). The result the result is listed in table (1) and exceeds our most optimistic expectations as there are no detectable loss in accuracy while the total number of coecients used in the sequential implementation is 132 as opposed to 648 for a direct implementation. The computational complexity is consequently reduced 5 times in this example. The density in table (1) is a graded coverage estimate for the tensor representation. The density denes to how well the local one dimensional neighbourhoods of the test image is reected in the tensor representation. In contrast to the angular RMS error the density only refers to the shape of the neighbourhood and is computed from the two largest eigen values 1 and 2 of the local tensor T. P ( 1 ,2 )2 2 P12 1 (16) Density = 1 The dierence in density between the sequential and the direct method is in the same magnitude as for the angular RMS error, i.e. insignicant. From the measurements in table (1) it is concluded that the sequential ltering approach support an ecient and robust tensor representation in 2D. 6 A Filtering Structure for 3D Signals Figure 5: The icosahedron, one of 5 platonic polyhedra. In 3D, the minimal number of lters required to produce a second order tensor representation is six, eq. (13). The apparent choice using conventional lter9 ing is to dene the lter orienting vectors as the vertices of a hemi-icosahedron [7, 6]. The icosahedron is a regular diametrically symmetric polyhedron (g. 5) which implies that the lters are equally spread. The six lter orienting vectors dened by the icosahedron are n^1 = c ( a n^3 = c ( b n^5 = c ( 0 ; 0 ; b )T ; a ; 0 )T ; b ; a )T n^2 = c (,a ; 0 ; b )T n^4 = c ( b ; ,a ; 0 )T n^6 = c ( 0 ; b ; ,a )T (17) where: a=2 p b = (1 + 5) p c = (10 + 2 5),1=2 These orientations are unfortunately not suitable for a sequential implementation because of the symmetry aspects discussed earlier. In agreement with the 2D case the rst choice of lter orienting vectors for sequential ltering in 3D are along the coordinate axes (3 possibilities) and the second symmetrically between two coordinate axes (6 possibilities). Although 6 lters are sucient in theory we choose to use nine lters for two reasons. The rst reason is that the orientation error due to the sequential lters not being polar separable is reduced by increasing the number of lters. The second reason is that using the convolver tree structure presented below, the increase in computational complexity by computing 9 lter responses instead of 6 is very small. The 9 lter orienting vectors for 3D are consequently dened as p n^1 = 1=p2 ( 0; ,1; n^2 = 1= 2 ( 0; 1; n^3 = p ( 0; 1; n^4 = 1=p2 (,1; 0; n^5 = 1= 2 ( 1; 0; n^6 = p ( 0; 0; n^7 = 1=p2 ( 1; 1; n^8 = 1= 2 (,1; 1; n^9 = ( 1; 0; 1 1 0 1 1 1 0 0 0 )T )T )T )T )T )T )T )T )T These lter orientations are not exactly equally spread as in the lters orientations in eq. (17). To eliminate any orientation bias in the local tensor T the projection tensors are computed using the tensor whitening method discussed earlier, [8]. The sequential ltering structure for the 3D case is consequently 10 One dimensional low−pass filters fz fy Image sequence fx One dimensional quadrature filters fy fx q9 f x,y f−x,y q8 f−x,y f x,y q7 fx fz q6 f−x,z f x,z q5 f x,z f−x,z q4 fz fy q3 f−y,z f y,z q2 f y,z f−y,z q1 Figure 6: 21 convolvers organized in 3 levels. Only 3 lters are required on the rst level. expressed as q1 () = fx() fy;z () f,y;z () q2 () = fx() f,y;z () fy;z () q3 () = fx() fz () fy () q4 () = fy () fx;z () f,x;z () q5 () = fy () f,x;z () fx;z () q6 () = fy () fx() fz () q7 () = fz () f,x;y () fx;y () q8 () = fz () fx;y () f,x;y () q9 () = fz () fy () fx() (18) Each quadrature lter response are computed in three steps. The rst two components are real valued and are both oriented in directions orthogonal to the lter orienting vectors. The last component is complex valued and directed along the main direction of the ideal lter. In the sequential lter organization above, the initial ltering step is identical for several lters. This observation is used to dene an ecient convolver tree structure of g. 6 consisting of 21 convolvers on 3 levels. Using only 3 convolvers at the top level. The optimized sequential lter components are displayed below. In g. 7 the resulting frequency response for q7 () is illustrated in the u1; u2-plane. The angular RMS error of the local tensor were computed for dierent SNR and the result is given in table 2. The 11 1 0.5 0 20 15 10 5 0 20 25 10 15 0 5 20 25 10 15 0 5 1 0.5 0 20 15 10 5 0 Figure 7: Upper part: Frequency response for q7 () in the u1 ; u2-plane. Lower part: Deviation from the ideal lter function. left column correspond to a lter set consisting of 9 quadrature lters where each quadrature lter response are computed sequentially by three lter components according to g. 6 The computation of nine quadrature lter responses requires 345 multiplications. If on the other hand conventional quadrature lters are used, six complex lters of size 9 9 9 require almost 9000 multiplications. The computational load is consequently reduced 25 times in this example while the dierence in accuracy is insignicant, see table 2. SNR 1 10dB 0dB M =3 345 coe. 0:76 3:02 9:35 M =1 8748 coe. 0:71 3:03 9:69 Table 2: Angular RMS error in degrees for a sequential lter with 3 components and the corresponding lter implemented by a single component. 7 A Filtering Structure for 4D Signals In 4D, the choice of lter orienting vectors is quite simple. From eq. (13) we conclude the the the minimum number of lters required to produce a tensor representation in 4D is 10. If the lters are localized symmetrically between 12 two main coordinate axes this results in 12 possibilities, which is sucient. The lter orienting vectors in 4D are dened as p n^1 = 1=p2 ( 0; 0; n^2 = 1=p2 ( 0; 0; n^3 = 1=p2 ( 0; 1; n^4 = 1= 2 ( 0; ,1; p n^5 = 1=p2 ( 1; 0; n^6 = 1=p2 ( 1; 0; n^7 = 1=p2 ( 1; 0; n^8 = 1= 2 (,1; 0; p n^9 = 1=p2 ( 0; 1; n^10 = 1=p2 ( 0; 1; n^11 = 1=p2 ( 1; 1; n^12 = 1= 2 (,1; 1; 1; 1 )T 1; ,1 )T 1; 0 )T 1; 0 )T 0; 1 )T 0; ,1 )T 1; 0 )T 1; 0 )T 0; 1 )T 0; ,1 )T 0; 0 )T 0; 0 )T (19) These 12 lter orientations also correspond to the vertices of the 24-cell, one of three regular polyhedra in 4D. The above lter orientations are consequently equally spread in the 4D signal space which simplies to computation of the projection tensors. In agreement with the 3D case the sequential ltering structure for 4D signals is expressed as q1 () = fx() fy () f,z;w () fz;w () q2 () = fx() fy () fz;w () fz;,w () q3 () = fx() fw () f,y;z () fy;z () q4 () = fx() fw () fy;z () f,y;z () q5 () = fy () fz () f,x;w () fx;w () q6 () = fy () fz () fx;w () fx;,w () q7 () = fy () fw () f,x;z () fx;z () q8 () = fy () fw () fx;z () f,x;z () q9 () = fz () fx() f,y;w () fy;w () q10 () = fz () fx() fy;w () fy;,w () q11 () = fz () fw () f,x;y () fx;y () q12 () = fz () fw () fx;y () f,x;y () The ltering is performed in 4 levels where the three rst are real valued and the last lter is complex. The distribution of the nonzero coecients within each lter component is indicated by the indices. Note that the indices of the 13 One dimensional quadrature filters One dimensional low−pass filters f x,y f−x,y q12 f−x,y f x,y q11 f y,w f y,−w q 10 f−y,w f y,w q9 f x,z f −x,z q f −x,z f x,z q7 f x,w f x,−w q6 f−x,w f x,w q5 f y,z f−y,z q4 f−y,z f y,z q3 f z,w f z,−w q2 f−z,w f z,w q1 fw fz fx Time sequence fw of volumes 8 fy fz fw fx fy Figure 8: Convolver organization in 4D, 33 convolvers in 4 levels components within each lter are orthogonal and that in the two rst ltering steps the same lter combination occurs several times. I g. 8 a 4D convolver structure in 4 levels is displayed. By careful combination of the results in the sequential ltering structure only 3 and 6 convolvers are required at the rst and second level respectively. The optimized lters with center frequency %0 = 1:11 and bandwidth B = 2 octaves are found below. At this point it starts to get really interesting to compare the the computational complexity of sequential convolution to traditional implementations. The spatial extent of the optimized lter components vary from 5-9 pixels. To compute a single quadrature lter response require 7+7+5+19+18 = 56 real multiplications. Using the convolver tree structure of g. 8 reduces this gure further by about 15 % resulting in 567 multiplications for 12 lters. In conventional FIR-ltering a complex lter of size 9 9 9 9 require almost 160 000 real valued multiplication to compute 12 quadrature lter responses. The sequential approach is consequently almost 280 times more ecient. 14 2D Image 2D: 3D: Time sequence of images or a 3D volume 4D: Time sequence of volumes Figure 9: Sequential convolver organization in 2D,3D and 4D 8 Implementation Aspects The general convolver module is implemented in C with the graphical user interface in AVS (Application Visualization System), see [10] for more details. The tree structure of the convolver organization, g. 9, provide an ecient optimization of the processing performance for a wide variety computer architectures. The soft implementation of the convolver network enable a simple reconguration for dierent types of input data (dierent outer dimensions) e.g 2D image, 3D spatiotemporal image sequences (3D volumes) and 4D volume sequences. The vector length (i.e. the inner dimension) of the kernel data is likewise simple to adapt from e.g scalars, RGB-signals, 2D vector elds, 3D vector elds, tensor elds etc. 9 Acknowledgment The support of the National Swedish Board for Technical Development, NUTEK (project 9305120-1) is greatfully acknowledged. 15 A Sequential lters for 2D These are the sequential lter components used in the experiments. From these four lter components all lters in eq. (15) can be produced by mirroring according to the indices. The coecients are converted to integers to support an ecient implementation. fx () = 1256 4195 14278 24981 14278 4195 1256 2 66 fxy () = 6 64 613 8593 24766 8593 613 3 77 77 5 Re [fx ()] = ,287 ,673 ,1507 92 4748 92 ,1507 ,673 ,287 Im [fx ()] = 607 5 ,405 ,3683 0 3683 405 ,5 ,607 2 66 66 932 Re [fxy ] = 66 66 ,689 ,1454 4 ,246 ,869 ,689 3 ,246 ,187 ,689 ,869 ,246 77 77 932 ,1454 ,689 5035 932 77 77 5 932 ,187 ,246 2 66 66 2144 ,2144 Im [fxy ] = 66 66 ,273 ,2101 ,2144 4 219 139 ,273 484 3 ,219 ,484 273 ,139 ,219 77 7 2101 2144 273 77 77 75 219 Projection tensors in 2D T1 = T3 = 0:75 0:00 0:00 ,0:25 ,0:25 0:00 0:00 0:75 T2 = T4 = 16 0:25 0:50 0:50 0:25 0:25 ,0:50 ,0:50 0:25 B Sequential Filters for 3D fx () = 1277 4060 15481 28833 15481 4060 1277 2 66 fxy () = 6 64 24 492 1565 492 24 3 77 77 5 Re [fx ()] = ,121 ,237 ,612 ,51 2041 ,51 ,612 ,237 ,121 Im [fx ()] = 206 ,28 ,75 ,1569 0 1569 75 28 ,206 2 66 66 6454 Re [fxy ] = 66 66 ,4817 ,12369 4 ,1641 ,5819 ,4817 ,1641 ,1538 ,4817 ,5819 ,1641 6454 ,12369 ,4817 39469 6454 6454 ,1538 ,1641 2 66 66 17177 ,17177 Im [fxy ] = 66 66 ,1495 ,14544 ,17177 4 1297 1106 ,1495 2447 1297 17 3 77 77 77 77 5 3 ,1297 ,2447 1495 ,1106 ,1297 77 7 14544 17177 1495 77 77 75 Projection Tensors in 3D These projection tensors where computed using the tensor whitening method ([8]) to compensate for any orientation bias due to uneven spread of the lters. 0 0:2690 1 0 0:2690 0 0 0 0 0 1:0042 1:5313 A 0 1:0042 ,1:5313 T1 = @ T2 = @ 0 1:5313 1:0042 0 ,1:5313 1:0042 0 ,0:1804 0 T3 = @ 0 T5 = @ 0 1:0042 0 1:5313 0 0 2:2538 0 0 ,0:1804 0 0:2690 0 0 1:0042 ,1:5313 T7 = @ ,1:5313 1:0042 0 T9 = @ 0 0 1:5313 0 1:0042 1 A 0 T4 = @ 1 A 0 ,0:1804 0 0 ,0:1804 T6 = @ 1 0 0A 0:2690 2:2538 0 0 0 ,0:1804 0 0 0 ,0:1804 0 T8 = @ 1 A 1:0042 0 ,1:5313 0 ,1:5313 0:2690 0 0 1:0042 0 0 0 0 2:2538 1:0042 1:5313 0 1:5313 1:0042 0 0 0 0:2690 C Sequential Filters for 4D fx() = 201 1891 14815 31267 14815 1891 201 2 ,963 66 fxy () = 6 64 2 66 66 Re [fxy ] = 66 66 4 6837 28702 6837 ,963 3 77 77 5 38 345 ,2980 ,1804 38 2811 ,11172 ,2980 2811 25786 2811 ,2980 ,11172 2811 38 ,1804 ,2980 345 38 18 3 77 77 77 77 5 1 A 1 A 1 A 1 A 2 66 66 Im [fxy ] = 66 66 4 963 785 3 ,963 ,785 ,1310 ,2221 ,963 77 77 12279 7172 ,1310 77 77 5 ,12279 12279 1310 ,7172 ,12279 2221 963 1310 Projection Tensors in 4D 0 ,0:50 0 B 0 , 0 : 50 T1 = B @ 0 0 1 CC A 0 ,0:50 0 B 0 , 0 : 50 T2 = B @ 0 0 0 ,0:50 B 0 T3 = B @ 0 1 CC A 0 ,0:50 0 0 B 0 1 : 00 , 1 : 50 T4 = B @ 0 ,1:50 1:00 1 CC A 0 1:00 0 0 B 0 , 0 : 50 0 T5 = B @ 0 0 ,0:50 1 CC A 0 B T6 = B @ 1 CC A 0 1:00 0 B 0 , 0 : 50 T7 = B @ 1:50 0 1 CC A 0 1:00 0 ,1:50 B 0 , 0 : 50 0 T8 = B @ ,1:50 0 1:00 1 CC A 1 CC A 0 ,0:50 B 0 T10 = B @ 0 1 CC A 1 CC A 0 1:00 ,1:50 0 B , 1 : 50 1 : 00 0 T12 = B @ 0 0 ,0:50 0 0 0 0 1:00 1:50 0 1:50 1:00 0 0 0 0 1:00 1:50 0 1:50 1:00 0 0 0 0 ,0:50 1:50 0 0 0 ,0:50 B 0 T9 = B @ 0 0 1:00 B 1:50 T11 = B @ 0 1:50 0 0 0 1:00 1:50 0 0 0 1:00 0 0 0 ,0:50 0 0 0 1:00 0 1:50 0 ,0:50 0 0 1:50 0 1:00 1:50 0 0 1:00 0 0 0 ,0:50 0 0 0 0 ,0:50 19 0 0 0 0 0 1:00 ,1:50 0 ,1:50 1:00 0 0 0 0 0 ,0:50 0 1:00 0 0 ,1:50 0 ,0:50 0 0 0 0 ,0:50 0 ,1:50 0 0 1:00 0 0 0 0 0 0 ,0:50 0 0 0 1:00 0 ,1:50 0 ,0:50 0 0 ,1:50 0 1:00 0 0 0 0 0 0 ,0:50 1 CC A 1 CC A References [1] M. Andersson. Controllable Multidimensional Filters in Low Level Computer Vision. PhD thesis, Linkoping University, Sweden, S{581 83 Linkoping, Sweden, September 1992. Dissertation No 282, ISBN 91{7870{ 981{4. [2] M. T. Andersson and H. Knutsson. Controllable 3-D Filters for Low Level Computer Vision. In Proceedings of the 8th Scandinavian Conference on Image Analysis, Troms, May 1993. SCIA. [3] R. Bracewell. The Fourier Transform and its Applications. McGraw-Hill, 2nd edition, 1986. [4] D. E. Dudgeon and R. M. Mersereau. Multidimensional Digital Signal Processing. Prentice-Hall signal processing series. Prentice-Hall, 1984. ISBN 0-13-604959-1. [5] G. H. Granlund. In search of a general picture processing operator. Computer Graphics and Image Processing, 8(2):155{178, 1978. [6] G. H. Granlund and H. Knutsson. Signal Processing for Computer Vision. Kluwer Academic Publishers, 1995. ISBN 0-7923-9530-1. [7] H. Knutsson. Representing local structure using tensors. In The 6th Scandinavian Conference on Image Analysis, pages 244{251, Oulu, Finland, June 1989. Report LiTH{ISY{I{1019, Computer Vision Laboratory, Linkoping University, Sweden, 1989. [8] H. Knutsson and M. Andersson. Robust N-Dimensional Orientation Estimation using Quadrature Filters and Tensor Whitening. In Proceedings of IEEE International Conference on Acoustics, Speech, & Signal Processing, Adelaide, Australia, April 1994. IEEE. LiTH-ISY-R-1798. [9] H. Knutsson, M. Andersson, and J Wiklund. Advanced Filter Design. In Proceedings of the Scandinavian Conference on Image analysis, June 1999. Submitted. [10] J. Wiklund and H. Knutsson. A Generalized Convolver. In Proceedings of the 9th Scandinavian Conference on Image Analysis, Uppsala, Sweden, June 1995. SCIA. 20

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement