Deep Structure, Singularities, and Computer Vision DSSCV IST-2001-35443 Deliverable Deliverable No.: Title: Author(s): Institution: Classification: Date: 21 Report on assessment of MSSTs for 3D Matching Application K. Somchaipeng, J. Sporring, S. Kreiborg, and P. Johansen 3D-Lab, School of Dentistry, Dept. of Pediatric Dentistry, University of Copenhagen Public 20 September 2004 Report on Assessment of MSSTs for 3D Matching Application K. Somchaipeng1,2 , J. Sporring2 , S. Kreiborg1 , and P. Johansen2 1 3D-Lab, School of Dentistry, Dept. of Pediatric Dentistry, University of Copenhagen Nørre Alle 20, DK-2200 Copenhagen N, Denmark 2 Dept. of Computer Science, University of Copenhagen, Universitetsparken 1, DK-2100, Copenhagen N, Denmark Abstract We consider images as manifolds embedded in a hybrid high dimensional space of coordinates and features. Images are partitioned into segments based on the energy functional and mathematical landmarks. The nesting of image segments in scale-space is used to construct image hierarchies called Multi-Scale Singularity Trees (MSSTs). Two kinds of mathematical landmarks are proposed namely extremal points and saddle points. We study the stability of the Extrema-Based MSSTs extracted from a series of smoothly changed generated images. The observed MSST transitions can be categorized into three categories: changes of positions, changes of relations, and changes of connections. The stability comparison between Extrema-Based MSSTs and Saddle-Based MSSTs is also briefly presented. We point out the equivalence between MSSTs and Multi-Scale Singularity Strings (MSSSs) and propose two sets of edit operations on MSSSs, one for Extrema-Based MSSSs and one for Saddle-Based MSSSs. Simple examples of editing MSSSs are also presented. We describe a medical image database of volumetric CT scans of the craniofacial area and manually segmented teeth. The database is aimed at evaluating the image matching algorithm based on MSSTs. 1 Gaussian Scale-Space The N + 1 dimensional Gaussian scale-space, L : RN +1 → R, of an N dimensional image, I : RN → R, is an ordered stack of images, where each image is a blurred version of the former [7, 20, 9]. The blurring is performed according to the diffusion equation, ∂t L = ∇2 L, (1) where ∂t L is the first partial-derivative of the image in the scale direction t, and ∇2 is the Laplacian operator, which in three dimensions reads ∂x2 + ∂y2 + ∂z2 . The Gaussian kernel is the Green’s function of the heat diffusion equation, i.e. L(·; t) = I(·) ⊗ g(·; t), g(x; t) = 1 T e−x x/(4t) , N/2 (4πt) (2) (3) where L(·, t) is the image at scale t, I(·) is the original image, ⊗ is the convolution operator, g(·; t) is the Gaussian kernel at scale t, N is the image dimensionality, and t = σ 2 /2, using σ as the standard 2 deviation of the Gaussian kernel. The Gaussian scale-space is henceforth called the scale-space in this article. The information in scale-space is logarithmically degraded, the scale-parameter is therefore often sampled exponentially using σ = σ0 eT . Since differentiation commutes with convolution and the Gaussian kernel is infinitely differentiable, differentiation of images in scale-spaces is conveniently computed, ∂xn L(·; t) = ∂xn (I(·) ⊗ g(·; t)) = I(·) ⊗ ∂xn g(·; t). (4) Alternative implementations of the scale-space are multiplication in the Fourier Domain, finite differencing schemes for solving the heat diffusion equation, additive operator splitting[19], and recursive implementation[6, 18]. Each method has its advantages and disadvantages, we prefer the spatial convolution, since it is guaranteed not to introduce spurious extrema in homogeneous regions at low scales. Typical border conditions are Dirichlet, Cyclic repetition, and Neumann boundaries. We use Dirichlet boundaries, where the image is extended with zero values in all directions according to the size of the convolution kernels. Although the dimensionality of the constructed scale-space is one higher than the dimensionality of the original image, critical points, in the image at each scale are always points. A critical point is e.g. an extremum, ∂x L = ∂y L = 0. Critical points are classified by the eigenvalues of the Hessian matrix, the matrix of all second derivatives, computed at that point. Critical points with all positive eigenvalues are minima, critical points with all negative eigenvalues are maxima, and critical points with a mixture of both negative and positive eigenvalues are saddles. As we increase the scale parameter, the critical points move smoothly forming critical paths. Along scale, critical points meet and annihilate or are created. Such events are called catastrophic events, and the points where they occur are called catastrophe points. The collection of events at all scales is called the deep structure of the image. The notion of genericity is used to disregard events that are not likely to occur for typical images, i.e. generic events are stable under slight perturbation of the image. There are only two types of generic catastrophe events in scale-space namely pairwise creation events and pairwise annihilation events[5]. It has further been shown that generic catastrophe events only involves pairs of critical points where one and only one eigenvalue of the Hessian matrix changes its sign, e.g. the annihilation of a minimum (+, +, +) and a saddle (+, +, -). A detailed discussion of a robust method for extracting critical paths and catastrophe points from scale-spaces can be found in [15]. 2 Multi-Scale Singularity Trees There are a few methods for constructing hierarchies from the deep structure of two-dimensional images proposed in the literatures so far [11, 10, 13]. To the best of our knowledge, an attempt to construct hierarchies from the deep structure of three-dimensional images is first proposed in [12] followed by [15]. The latter scheme, which will be extended in this Section, produces rooted ordered binary trees called Multi-Scale Singularity Trees (MSSTs) with catastrophe points as nodes. It only considers the linking of the mathematical landmarks which present at the first scale image. Creation catastrophes are ignored 2.1 Energy Functional and Energy Partitions Let Ω ⊂ RN be a compact connected domain and define I : Ω → R+ to be an image, ~e ∈ Ω as a landmark, and ~x ∈ Ω as a point in the domain. Consider a set of continuous functions γ : [0, P ] → Ω Version: August 27, 2014 3 2.2 Building Scale-Space Hierarchies for which γ(0) = ~e and γ(P ) = ~x, γ ∈ Γ~e~x , where Γ~e~x is the set of all possible paths from a landmark ~e to a point ~x, and γ is parameterized using Euclidean arc-length. The energy E~e (~x) with respect to an extremum ~e calculated at ~x is defined as Z Ps 1 ∂γ(p) 2 ∂I(γ(p)) 2 E~e (x) = inf √ | | +α | | dp. (5) γ∈Γ~e~x ∂p ∂p α 0 Considering images as manifolds embedded in a high dimensional space. An N dimensional intensity image becomes an N dimensional manifold embedded in a hybrid N + 1 dimensional space of coordinates and the features1 , the “space-feature”[8]. For two dimensional images, it is commonly known as the height plot of the image. The energy defined at a point associated with a landmark can then be thought of as the minimum of the distances travelling up and down from the landmark to that point. When the coefficient α is set to infinity, the energy is also known as the path variation, a generalization of the total variation[3]. Let E ⊂ Ω be the set of all landmarks in the image. An image segment or an energy partition Si associated with a landmark ~ei ∈ E is defined as the set of all points in the images, where the energy E~ei (~x) is minimal, Si = {~x ∈ Ω|E~ei (~x) < E~ei (~x), ∀~ej ∈ E, i 6= j}. (6) An approximation of the energy map Mi : Ω → R+ , which defines the energy at every point in the image associated with a landmark ~ei , can be efficiently calculated using the Fast Marching Methods[14]. The tessellation of the image segments obtained depends on the selection of the landmarks and the energy functional. Mathematical landmarks like extremal points seem to be a natural selection since they represent physically the image content, significant features in the image contain at least one such points, and they can be easily and automatically detected. The selection of extremal points as landmarks leads to Extrema-Based MSSTs. Another candidate for landmarks are the saddles points, which leads to Saddle-Based MSSTs. We show later in Section 3. that the transition of Saddle-Based MSSTs are simpler than those of Extrema-Based MSSTs. 2.2 Building Scale-Space Hierarchies MSSTs are obtained by linking catastrophe points based on the nesting of image segments in scalespace. Resulting MSSTs are always rooted ordered binary tree. For simplicity, we will begin by describing the building procedures of Extrema-Based MSSTs and then move to Saddle-Based MSSTs at the end of the Section. 2.2.1 Extrema-Based MSSTs Each node of the tree consists of three parts, the leftport, the body, and the rightport. The leftport represents the image segment that immediately covers the image segment represented by the rightport, which disappears after the catastrophe event mentioned in the body. Assuming that critical paths and catastrophe points in the scale-space image are already and correctly extracted, the tree building algorithm is as follows: The algorithm starts by setting the root of the tree as BC∞ Elast , where B denotes border of the image, Ei denotes the last extremum points in scale, and C∞ denotes the virtual catastrophe at scale infinity where the border and the last extremum points virtually annihilate. 1 In this case the only feature is the intensity or the zeroth jet space. The energy functional can be easily defined for higher order jet space images, color images, or locally orderless images with scale-space histograms to handle textures[17], if a metric in the feature space is given. Version: August 27, 2014 4 2.3 Discussion At the highest catastrophe point Ctop in scale, calculate an energy map with respect to the catastrophe Ctop , create a node Ecover Ctop Eann , where Ecover is the extremum which its image segment covers the catastrophe Ctop , and Eann is the extremum that annihilates with a saddle at the catastrophe Ctop . Link the new created node as the left child of the root if Ecover equals B or as the right child of the root if Eann equals Elast . Determining the covering image segment is easy since it is the image segment belonging to the extremum or the border with minimal energy. We simply choose the extremum or the border that has the lowest value in the energy map with respect to the catastrophe. Repeat the same procedure on the next highest catastrophe Cnext to the lowest one but instead of linking the new node as the right child or the left child of the root, link the new node as the left child of a node in the tree, which does not have its left child and its leftport equals the leftport of the new node, or link the new node as the right child of a node in the tree, which does not have its right child and its rightport equals the leftport of the new node. There will be only one possible linking point exists. An example of Extrema-Based MSSTs together with the deep structure it represents are shown in Figure 1(a). 2.2.2 Saddle-Based MSSTs A similar procedure is applied for constructing Saddle-Based MSSTs. The algorithm starts with setting the root of the tree as Ctop Stop , where the leftport is set to null, Ctop denotes the highest catastrophe in scale, and Stop denotes the saddle point that annihilates with an extremum at Ctop . The root will have only the right child which is Stop Cnext Sann , where Cnext denote the next highest catastrophe point, and Sann denotes the saddle point that annihilates with an extremum at the catastrophe Cnext . Go down the scale to the next highest catastrophe and repeat the procedure described in Section 2.2.1. considering saddle points as landmarks instead of extrema. An example of Saddle-Based MSSTs is shown together with the deep structure it represents in Figure 1(b). 2.3 Discussion We prefer our tree building method because unlike all other proposed methods, which produce hardlinked trees: only the best linking point is suggested, our method produces soft-linked trees in the sense that all possible linking points are suggested where linking points with the lowest energy is selected. The added information is very useful for estimating the costs of transitions. If the energies associated with a catastrophe point calculates at two different landmarks are slightly different, it implies that the link could then be easily switched with small perturbation on the image and the cost of this transition should be small accordingly. On the other hand if the energy difference is large, it then can be assumed that the link is rather stable and a high cost should be paid for the transition. The cost of that transition can be estimated proportionally to the difference of the two energies. 3 Stability of MSSTs To empirically study the stability of MSSTs, a series of generated images of three stationary and one moving Gaussian blobs is used in the experiment. All images in the series have the same number of extrema, i.e. a minimum surrounded by four maxima. Smooth changes of the images are forced by smoothly changing the parameters used for generating the images. For each test image a scale-space is computed and the Extrema-Based MSST is constructed. Changes in the constructed MSSTs between neighbouring images are observed and carefully classified into categories. Version: August 27, 2014 5 Border Discussion Scale 3.1 BC∞ Eb Ca Cc BCd Ef Eb Ca Ea Cb null null Eb Cc Ed Cd Ea Sa Eb Ec Sb Ed Sc Sd Ef null null Ea Cb Ec null null Space Border Scale (a) Ca Sa Ca Sa Cc Sc Cc Cb Sa Cb Sb null Cd Ea Sa Eb Ec Sb Ed Sc Sd Ef null Sc Cd Sd null null Space (b) Figure 1: The deep structure with its corresponding Extrema-Based MSST(a) and the deep structure with its corresponding Saddle-Based MSST(b) Three categories of transitions are observed in the experiment, i.e. changes of positions, changes of connections, and changes of relations. Changes of positions are the movements of the extrema and catastrophe points, changes of connections swap the connection between two extremum-catastrophe pairs, and changes of relations are the changes of the nesting of the energy partitions in scales. Interested readers can find more detail on the experiment in [16]. Figure 2. shows the before and after images of the observed three transition categories, with the critical paths and the linkings superimposed on the images. 3.1 Discussion In practice, some of the the observed three transition categories might occur at the same time. In general, changes of positions occur most frequently. Changes of connections and changes of relations usually accompanied by changes of positions. Since the process of constructing Saddle-Based MSSTs is only interested in saddle points and their paths in scale-space, the changes of connections, which are the connections between catastrophe points and extremal points, do not effect their structures at all. There are then only two kinds of transitions for Saddle-Based MSSTs: changes of positions and changes of relations. Therefore we expect Saddle-Based MSSTs to be more stable than Extrema-Based MSSTs. 4 Multi-Scale Singularity Strings Due to redundancies in the tree structure and the data stored in the nodes of the tree, MSSTs can be converted to string structures simply by concatenating nodes of the tree in order according to the Version: August 27, 2014 6 Figure 2: The three observed transitions of Extrema-Based MSSTs shown as images before (left column) and after (right column) the three categories of transitions: changes of positions (top row), changes of connections (middle row) and changes of relations (bottom row). The critical paths and the linkings are superimposed on the images: saddle paths in blue, maximum paths in red, minimum paths in green, and the linkings in yellow. Version: August 27, 2014 7 [BC∞ Eb ][Eb Ca Ea ][Eb Cc Ed ][Ea Cb Ec ][BCd Ef ] (a) [Ca Sa ][Sa Cc Sc ][Sa Cb Sb ][Sc Cd Sd ] (b) Figure 3: The Extrema-Based MSSS(a) and the Saddle-Based MSSS(b) equivalent to their corresponding MSSTs in Figure 1(a) and 1(b). scales of the catastrophes. The resulting strings are called Multi-Scale Singularity Strings (MSSSs). The information encoded in the structure of the tree is completely kept in the string. An MSST can be converted to its equivalent MSSS and vice versa without lose of information. The Extrema-Based MSSS and the Saddle-Based MSSS equivalent to the Extrema-Based MSST and the Saddle-Based MSST in Figure 1(a) and 1(b) are shown in Figure 3 (a) and (b), respectively. 5 Matching MSSTs We would like to solve the problem of image matching by matching their pre-computed MSSTs. The similarity between two images can be estimated as the minimum total cost of a series of edit operations that transforms the MSST of one image to the MSST of the other. This problem is commonly known as the Tree Edit Distance (TED) problem[4]. Because MSSTs can be equivalently represented by their equivalent MSSSs and edit operations on string are simpler and easier to be defined than those on tree, we choose to solve the problem of matching MSSTs by matching their equivalent MSSSs. Due to the nature of causality in scale-space there is a right-to-left dependency between nodes in the MSSSs. Unlike normal character strings, MSSSs are supposed to be edited from right to left. In order to avoid insertion operations, which is hard to define, two MSSSs SA and SB are edited to a common MSSS SC . Given two MSSSs SA and SB , the edit distance edit(SA → SB ) is the sum of the total cost of a series of edit operations {o1 , o2 , . . . , on } that changes SA to SC and the total cost of a series of delete operation {d1 , d2 , . . . , dm } that change SB to SC . edit(SA → SB ) = n X cost(oi ) + i=1 m X cost(dj ) (7) j=1 The cost of the delete operation can be defined proportionally to the stability measures of that catastrophe points. To make the distance symmetric, the distance between SA and SB , distance(SA , SB ), is defined as the sum of edit(SA → SB ) and edit(SB → SA ). distance(SA , SB ) = edit(SA → SB ) + edit(SB → SA ) (8) In order to be able to define costs for all possible transitions full energy matrices are needed, however with only energy maps constructed with respect to catastrophe points, as described in the building process of MSSTs in Section 2.2., we can only obtained values in the lower triangle of the matrix. We propose to calculate energy maps with respect to landmarks at the first-scale image. This way, full energy matrices can be obtained. 5.1 Editing Extrema-Based MSSSs Each node in the Extrema-Based MSSS contains only symbols. The content of each symbol, which is what actually to be matched, must be stored in the following tables and matrices. Version: August 27, 2014 8 5.1 Editing Extrema-Based MSSSs 1. Extremum Table (E-Table) The table containing the positions and features of all extrema in the first-scale image. TE = Eo E1 .. . En : position0 , f eatures0 : position1 , f eatures1 .. .. . . : positionn , f eaturesn 2. Catastrophe Table (C-Table) The table containing the positions, features, and stabilities of all catastrophe points in the scalespace image. TC = C1 C2 .. . : : .. . position1 , scale1 , f eatures1 , stability1 position2 , scale2 , f eatures2 , stability2 .. . Cn : positionn , scale3 , f eaturesn , stabilityn 3. Extremum-Extremum Energy Matrix (E-E Matrix) The matrix containing the energy of the minimal path between all extrema in the first-scale image. MEE = 0 E1 E0 .. . E0 E1 0 .. . En E0 En E1 · · · E0 En · · · E1 En .. .. . . ··· 0 4. Extremum-Catastrophe Energy Matrix (E-C Matrix) The matrix containing the energy of the minimal paths between all catastrophe points in the scale-space image and all extrema in the first-scale image. MCE = C1 E0 C2 E0 .. . C1 E1 C2 E1 .. . Cn E0 Cn E1 · · · C1 En · · · C1 En .. .. . . · · · Cn En These tables and matrices are part of the Extrema-Based MSSS and are given to the matching algorithm. The cost of each edit operation is derived from these tables and matrices. 5.1.1 Edit Operations on Extrema-Based MSSSs Five edit operations make the complete set of edit operations on Extrema-Based MSSSs. The five edit operations are as follows. 1. Extremum-Editing E(Ei → [pos, f eat]) Changing of the position and features of the extrema Ei stored in the E-Table. Version: August 27, 2014 9 5.1 Editing Extrema-Based MSSSs cost(E(Ei → [pos, f eat])) = we | Ei − [pos, f eat] | 2. Catastrophe-Editing C(Ci → [pos, f eat]) Changing of the position and features of the catastrophe Ci stored in the C-Table. If the ordering of catastrophe points in scale changes, MSSS must be changed accordingly. Illegal relations, if exist, must be properly updated. cost(C(Ci → [pos, f eat])) = wc | Ci − [pos, f eat] | 3. Left-Relabeling L(Elef t → Enew ) Changing of the leftport from Elef t to Enew which corresponds to changing of the nesting relations from being nested inside Elef t to Enew . cost(L(Elef t → Enew )) = wl | Eright Enew − Eright Elef t | 4. Right-Relabeling R(Eright → Enew ) Changing of the rightport from Eright to Enew which corresponds to changing of the connection from Ccat being connected to Eright to Enew . Illegal connections and relations, if exist, must be properly updated. cost(R(Eright → Enew )) = wr | Ccat Eright − Ccat Enew | 5. Deletion D(Elef t Ccat Eright → φ)) Deleting a node Elef t Ccat Eright from the MSSS. Illegal relations, if exist, must be properly updated. cost(D(Elef t Eright Ccat )) = wd stability(Ccat ) 5.1.2 Simple Example We consider a simple case of editing two Extrema-Based MSSSs SA and SB extracted from two images IA and IB with two extrema respectively. The extrema in the two images are located at different positions and have different intensity values. The deep structures of SA and SB are shown in Figure 4(a) and Figure 4(b), respectively. Let SA be an Extrema-Based MSSS consisting of two nodes: [BC∞ E0 ][E0 C1 E1 ] with TE = TC = E0 : pos = 4 E1 : pos = 1 , C1 : pos = 2, scale = 2 , and SB be another Extrema-Based MSSS also consisting of two nodes: [BC∞ E0 ][E0 C1 E1 ] with TE = Version: August 27, 2014 E0 : pos = 1 E1 : pos = 4 10 , Scale Saddles-Based MSSSs Scale 5.2 3.0 3.0 C1 2.0 2.0 1.0 0.0 0.0 C1 1.0 1.0 E1 2.0 3.0 S0 Space 4.0 E0 0.0 0.0 1.0 E0 2.0 S0 (a) 3.0 4.0 E1 Space (b) Figure 4: The deep structure of SA and SB TC = C1 : pos = 3, scale = 3 . We would like to find a series of edit operations that change SA to SB through a common MSSS SC . The series of the edit operations that changes SA to SC , where SC is the common MSSS, consists of two edit operations2 C(C1 → [pos = 3, scale = 3])R(E1 → E0 ) . The series of delete operation that changes SB to SC is empty and the common string SC equals SB . The edit cost edit(SA → SB ) is then edit(SA → SB ) = cost(C(C1 → [pos = 3, scale = 3])) + cost(R(E1 → E0 )) . The first operation corrects the position and scale of the catastrophe C1 , the second operation changes the connection of C1 from being connected to E1 to being connected to E0 . Because E0 is no longer present at the scale of C1 , the leftport E0 is then no longer valid and must be updated to E1 . The SA after applying the edit operations is [BC∞ E0 ][E1 C1 E0 ] with TE = TC = E0 : pos = 4 E1 : pos = 1 , C1 : pos = 3, scale = 3 , which is now equivalent to SB . 5.2 Saddles-Based MSSSs Similar to Extrema-Based MSSSs, each node of the Saddle-Based MSSS contains only symbols. The content of each symbol, which is what we actually want to match, are stored in the following tables and matrices. 2 Assuming that this series of edit operation give the lowest total cost. Version: August 27, 2014 11 5.2 Saddles-Based MSSSs 1. Saddle Table (S-Table) The table containing the positions and features of all saddles in the first-scale image. TS = S1 S2 .. . Sn : position0 , f eatures0 : position1 , f eatures1 .. .. . . : positionn , f eaturesn 2. Catastrophe Table (C-Table) The table containing the positions, features, and stabilities of all catastrophe points in the scalespace image. TC = C1 C2 .. . : : .. . position1 , scale1 , f eatures1 , stability1 position2 , scale2 , f eatures2 , stability2 .. . Cn : positionn , scale3 , f eaturesn , stabilityn 3. Saddle-Saddle Energy Matrix (S-S Matrix) The matrix containing the energy of the minimal paths between all saddles in the first-scale image. ESS = 0 S2 S1 .. . S1 S2 0 .. . Sn S1 Sn S2 5.2.1 · · · S1 Sn · · · S2 Sn .. .. . . ··· 0 Edit Operations on Saddles-Based MSSSs Because the connections between catastrophe points and extrema in scale-space can change but the connections between saddles and catastrophes are fixed, edit operations on Saddle-Based MSSSs are simpler. There are only three edit operations needed as follows. 1. Saddle-Catastrophe-Editing SC(Sright → [poss , f eats ], Ccat → [posc , f eatc ]) Changing of the position and features of the saddle Sright stored in the S-Table and the catastrophe Ccat stored in the C-Table. If the ordering of catastrophe points changes, update the MSSS accordingly. Illegal relations, if exist, must be properly updated. cost(SC(Sright → [poss , f eats ], Ccat → [posc , f eatc ])) = wsc | Sright − [poss , f eats ] | +wc | Ccat − [posc , f eatc ] | 2. Left-Relabeling L(Slef t → Snew ) Changing of the leftport from Slef t to Snew which corresponds to changing of the nesting relations from Sright being nested inside Slef t to Snew . cost = wl | Sright Slef t − Sright Snew | Version: August 27, 2014 12 Scale Saddles-Based MSSSs Scale 5.2 3.0 3.0 C1 2.0 2.0 1.0 0.0 0.0 C1 1.0 1.0 E1 2.0 3.0 S0 Space 4.0 E0 0.0 0.0 1.0 E0 2.0 S0 (a) 3.0 4.0 E1 Space (b) Figure 5: The deep structures of SA and SB 3. Deletion D(Slef t Ccat Sright → φ) Deleting a node Slef t Ccat Sright from the MSSS. Illegal relations, if exist, must be properly updated. cost(D(Elef t Eright Ccat )) = wd stability(Ccat ) 5.2.2 Simple Example We consider the image matching of the two images as described in Section 5.1.2 but using SaddleBased MSSSs instead. The deep structures of the Saddle-Based MSSSs SA and SB are shown in Figure 5(a) and Figure 5(b), respectively. Let SA be a Saddle-Based MSSS consisting of one node: [C1 S1 ] with TS = TC = S1 : pos = 3 , C1 : pos = 2, scale = 2 , and SB be another Saddle-Based MSSS consisting of one node: [C1 S1 ] with TS = TC = S1 : pos = 2 , C1 : pos = 3, scale = 3 . The series of the edit operation that transform SA to SB through a common MSSS SC is SC(C1 → [pos = 3, scale = 3], S1 → [pos = 2]) The series of delete operation is empty and the common MSSS SC equals SB . The edit cost edit(SA → SB ) is then edit(SA → SB ) = cost(SC(C1 → [pos = 3, scale = 3], S1 → [pos = 2])) Version: August 27, 2014 13 5.3 Discussion The only operation needed to edit SA to SB is to corrects the position and scale of the catastrophe C1 and the position of the saddle S1 . The SA after applying the edit operations is [C1 S1 ] with TS = TC = S1 : pos = 2 , C1 : pos = 3, scale = 3 . which is now equivalent to SB . 5.3 Discussion We propose to calculate the edit distance as a linear combination of the transition cost, where the coefficients of the linear combination is estimated as a best fit with hand classified images. This estimation has not yet been performed since the edit distance algorithm has not yet been completed. We suggest that the coefficients are all first set to one and then adjusted according to the discrepancy between the matching results and the hand classified images until the best fit is obtained. 6 Medical Image Database This Section describes an image database of the craniofacial area. The database consists of four threedimensional volumetric CT images obtained from four patients with various abnormalities and 113 masks of the teeth segmented manually. The database is aimed at assessing the performance of the image matching algorithm based on the Multi-Scale Singularity Trees (MSSTs). The performance of the matching algorithm can be measured by the rate of correct matching between teeth of the same kind at different poses and locations. 6.1 CT Images and Tooth Masks The image database consists of four three-dimensional volumetric CT images obtained from four patients with various abnormalities and 113 masks of the teeth segmented manually. The teeth are segmented using the ∇V ision[2], the multi-scale interactive segmentation tool, installed at the 3DLab. ∇V ision is a joint development between researchers from the IT University of Copenhagen and the University of Copenhagen. The program has been develop under the project “Computing Natural Shape" and as a project in the now closed company Generic Vision. All images are stored in the ANALYZETM 7.5 file format[1] with big-endian encoding. According to the ANALYZETM 7.5 file format, two separate files with the same name but different endings are used for an image, the header ended with .hdr and the raw data ended with .img. The CT images are stored in different directories namely p3, p4, p5, and p6 together with the segmented tooth masks named with number. The ∇V ision project files ended with .prj can also be found in the corresponding directories. The volumetric craniofacial CT images use a 32-bit real number for a voxel. The segmented tooth masks use an 8-bit integer for a voxel, the value zero for the background and the value 254 for the object. The properties of the images and the number of the masks of segmented teeth for each CT image are listed in Table 1. Figure 6. shows the surfaces of the segmented teeth of the four CT images. Version: August 27, 2014 14 6.1 CT Images and Tooth Masks Name P3 p4 p5 p6 Dimension Size(mm) 399 × 232 × 107 0.37 × 0.37 × 0.80 302 × 196 × 84 0.49 × 0.49 × 1.00 305 × 199 × 160 0.49 × 0.49 × 0.60 288 × 238 × 73 0.47 × 0.47 × 1.00 Total Number of Segmented Teeth Number of Segmented Teeth 44 32 8 29 113 Table 1: Properties of the CT images in the database p4 p3 p6 p5 Figure 6: Surfaces of the segmented teeth of the four craniofacial CT images Version: August 27, 2014 15 6.2 Computational Task 6.2 Computational Task The database is aimed at assessing the performance of the image matching algorithm based on the Multi-Scale Singularity Trees (MSSTs)[15, 16]. The medical task interested is the detection and location of teeth in CT/MR images, which is useful for a number of interesting problems such as studying the statistics of tooth shapes, studying the growth pattern, and analyzing the configuration of teeth for possible corrections. For each segmented tooth, an MSST is computed. The similarity between two segmented teeth is measured as the total cost of edit operations that transform one MSST to the other. This problem is commonly known as the Tree Edit Distance (TED) problem[4]. In order to find the best match, the enquiry tooth have to be matched against all other teeth in the database. Fortunately, the MSSTs of all the segmented teeth can be pre-computed and later reused in the matching process. As a sub-goal, if the matching algorithm permits, MSSTs of teeth will be used to match into the MSSTs computed from the craniofacial CT images of a larger area to evaluate the possibility of sub-object extractions. 6.3 Assessment The set of plastic teeth prepared by human experts can be used as the ground truth. The performance of the matching algorithm can be measured by the rate of the correct matching between teeth of the same kind at different poses and locations. The matching results will include the best match and all other matches ranked according to the distance or similarity. 7 Concluding Remarks The selection of landmarks and the energy functional define the tessellation of energy partitions. The nesting of energy partitions in scale-space creates hierarchies which are captured and presented using tree structure called Multi-Scale Singularity Trees (MSSTs). MSSTs capture the deep structure of images. MSST transitions observed experimentally from a series of generated images are categorized into three categories: changes of positions, changes of relations, and changes of connections. We have not yet clearly proved the completeness of the obtained transitions. MSSTs and introduced MSSSs are equivalent. Both structures topologically represent the deep structure of images. It’s however easier and more intuitive to study and define edit operations on MSSSs since their transitions are simpler[16]. We present the two set of edit operations together with their cost estimates each on Extrema-Based MSSSs and Saddles-Based MSSSs. Since the edit operations on Saddles-Based MSSSs are simpler, we prefer the Saddles-Based MSSSs as a multi-scale representation of images suitable for image matching applications. Creation events and annihilation events between pairs of critical points that cannot be tracked down to the first-scale image are ignored. These catastrophic events, however generic, are not frequently occurred. Including these catastrophic events in MSSTs/MSSSs and defining edit operations that can handle these events might improve the matching results. MSSSs are preferable for image matching applications however one of the advantages of MSSTs over MSSSs is the hierarchical grouping of energy partitions suggested by its sub trees. We believe that exploiting this advantage might be useful for sub-object extraction, e.g. matching MSSS of a sub-object with MSSS equivalent to a sub tree cut from the MSST of the whole image. Version: August 27, 2014 16 REFERENCES References [1] ANALYZE T M 7.5 File Format. Biomedical Imahing Resource, Mayo Foundation. [2] ∇V ision. Generic Vision. [3] P. A. Arbelaez and L. D. Cohen. The Extrema Edges". In Scale Space 2003, LNCS 2695, pages 180–195, 2003. [4] P. Bille. Report on Known Algorithm for Tree Matching. Technical report, Deliverable No.5, DSSCV, IST-2001-35443, 21. March 2003. [5] James Damon. Local Morse theory for Gaussian blurred functions. In Jon Sporring, Mads Nielsen, Luc Florack, and Peter Johansen, editors, Gaussian Scale-Space Theory, chapter 11, pages 147–163. Kluwer Academic Publishers, Dordrecht, The Netherlands, 1997. [6] R. Deriche. Recursively Implementing the Gaussian and its Derivatives. In V. Srinivasan, Ong Sim Heng, and Ang Yew Hock, editors, Proceedings of the 2nd Singapore International Conference on Image Processing, pages 263–267. World Scientific, Singapore, 1992. [7] T. Iijima. Basic theory on normalization of a pattern (in case of typical one-dimensional pattern). Bulletin of Electrotechnical Laboratory, 26:368–388, 1962. (in Japanese). [8] R. Kimmel, N. Sochen, and R. Malladi. From High Energy Physics to Low Level Vision. In Scale-Space Theory in Computer Vision, Utrecht, The Natherlands, 1997. [9] J. J. Koenderink. The Structure of Images. Biological Cybernetics, 50:363–370, 1984. [10] Arjan Kuijper. The deep structure of Gaussian scale space images. PhD thesis, Image Sciences Institute, Institute of Information and Computing Sciences, Faculty of Mathematics and Computer Science, Utrecht University, 2002. [11] L. M. Lifshitz and S. M. Pizer. A multiresolution hierarchical approach to image segmentation based on intensity extrema. IEEE Transaction on Pattern Analysis and Machine Intelligence, 12(6):529–541, 1990. [12] O.F.Olsen and M.Nielsen. Multi-scale gradient magnitude watershed segmentation. In A. Del Bimbo, editor, ICIAP ’97 - 9th Int. Conf. on Image Analysis and Processing, volume 1310 of Lectures Notes in Computer Science, pages 6–13. Springer, September 1997. [13] B. Platel. Multiscale Hierarchical Segmentation. Technical Report BMT-Report no. 2002-04, Department of BioMedical Engineering, Technical Universitet of Eindhoven, 2002. [14] J. A. Sethian. Fast Marching Methods. SIAM Review, 41(2):199–235, 1999. [15] K. Somchaipeng, J Sporring, S. Kreiborg, and P. Johansen. Software for Extracting Multi-Scale Singularity Trees. Technical report, Deliverable No.8, DSSCV, IST-2001-35443, 15. September 2003. [16] K. Somchaipeng, J Sporring, S. Kreiborg, and P. Johansen. Some Transitions of Extrema-Based Multi-Scale Singularity Trees. Technical report, DSAGM’04, Dept. of Computer Science, University of Copenhagen (DIKU), August 2004. Version: August 27, 2014 17 REFERENCES [17] B. van Ginneken and B. M. ter Haar Romeny. Applications of locally orderless images". In Scale Space 1999, 1999. [18] L.J. van Vliet, I.T. Young, and P.W. Verbeek. Recursive Gaussian Derivative Filters. In Proceedings of the 14th International Conference on Pateern Recognition ICPR’98, volume 1, pages 509–514., Brisbane, Australia, 1998. IEEE Computer Society Press. [19] J. Weickert, K. J. Zuiderveld, B. M. ter Haar Romeny, and W. J. Niessen. Parallel implementations of AOS schemes: A fast way of nonlinear diffusion filtering. In Proceedings of the 4th International Conference on Image Processing, volume 3, pages 396–399, Santa Barbara, CA, USA, October26–29 1997. IEEE Computer Society Press. [20] A. P. Witkin. Scale–space filtering. In Proc. 8th Int. Joint Conf. on Artificial Intelligence (IJCAI ’83), volume 2, pages 1019–1022, Karlsruhe, Germany, August 1983. Version: August 27, 2014 18

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement