Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Science and Technology 1137 Distance Functions and Their Use in Adaptive Mathematical Morphology VLADIMIR ĆURIĆ ACTA UNIVERSITATIS UPSALIENSIS UPPSALA 2014 ISSN 1651-6214 ISBN 978-91-554-8923-6 urn:nbn:se:uu:diva-221568 Dissertation presented at Uppsala University to be publicly examined in 2347, Lägerhyddsvägen 2, Hus 2, Uppsala, Friday, 23 May 2014 at 13:15 for the degree of Doctor of Philosophy. The examination will be conducted in English. Faculty examiner: Hugues Talbot (University Paris-Est - ESIEE). Abstract Ćurić, V. 2014. Distance Functions and Their Use in Adaptive Mathematical Morphology. Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Science and Technology 1137. 88 pp. Uppsala: Acta Universitatis Upsaliensis. ISBN 978-91-554-8923-6. One of the main problems in image analysis is a comparison of different shapes in images. It is often desirable to determine the extent to which one shape differs from another. This is usually a difficult task because shapes vary in size, length, contrast, texture, orientation, etc. Shapes can be described using sets of points, crisp of fuzzy. Hence, distance functions between sets have been used for comparing different shapes. Mathematical morphology is a non-linear theory related to the shape or morphology of features in the image, and morphological operators are defined by the interaction between an image and a small set called a structuring element. Although morphological operators have been extensively used to differentiate shapes by their size, it is not an easy task to differentiate shapes with respect to other features such as contrast or orientation. One approach for differentiation on these type of features is to use data-dependent structuring elements. In this thesis, we investigate the usefulness of various distance functions for: (i) shape registration and recognition; and (ii) construction of adaptive structuring elements and functions. We examine existing distance functions between sets, and propose a new one, called the Complement weighted sum of minimal distances, where the contribution of each point to the distance function is determined by the position of the point within the set. The usefulness of the new distance function is shown for different image registration and shape recognition problems. Furthermore, we extend the new distance function to fuzzy sets and show its applicability to classification of fuzzy objects. We propose two different types of adaptive structuring elements from the salience map of the edge strength: (i) the shape of a structuring element is predefined, and its size is determined from the salience map; (ii) the shape and size of a structuring element are dependent on the salience map. Using this salience map, we also define adaptive structuring functions. We also present the applicability of adaptive mathematical morphology to image regularization. The connection between adaptive mathematical morphology and Lasry-Lions regularization of nonsmooth functions provides an elegant tool for image regularization. Keywords: Image analysis, Distance functions, Mathematical morphology, Adaptive mathematical morphology, Image regularization Vladimir Ćurić, Department of Information Technology, Division of Visual Information and Interaction, Box 337, Uppsala University, SE-751 05 Uppsala, Sweden. © Vladimir Ćurić 2014 ISSN 1651-6214 ISBN 978-91-554-8923-6 urn:nbn:se:uu:diva-221568 (http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-221568) To my mother Stana and my father Žarko Mami i tati List of papers This thesis is based on the following papers, which are referred to in the text by their Roman numerals. I II Lindblad, J., Ćurić, V., Sladoje, N. (2009) On set distances and their application to image registration, In Proceedings of the 6th International Symposium on Image and Signal Processing and Analysis (ISPA 2009), IEEE, pp. 449–454. Ćurić, V., Lindblad, J., Sladoje, N., Sarve, H., Borgefors, G. (2014) A new set distance and its application to shape registration, Pattern Analysis and Applications, Volume 17, Issue 1, pp. 141–152. III Ćurić, V., Lindblad, J., Sladoje, N. (2011) Distance Measures between Digital Fuzzy Objects and Their Applicability in Image Processing, In Proceedings of the 14th International Workshop on Combinatorial Image Analysis (IWCIA 2011), LNCS–6636, pp. 385–397. IV V VI VII Ćurić, V., Luengo Hendriks, C.L., Borgefors, G. (2012) Salience Adaptive Structuring Elements, IEEE Journal of Selected Topics in Signal Processing, Special Issue on Filtering and Segmentation in Mathematical Morphology, Volume 6, Issue 7, pp. 809–819. Ćurić, V., Luengo Hendriks, C.L., (2012) Adaptive Structuring Elements Based on Salience Information, In Proceedings of the International Conference on Computer Vision and Graphics (ICCVG 2012), LNCS–7594, pp. 321–328. Ćurić, V., Luengo Hendriks, C.L., (2013) Salience-Based Parabolic Structuring Functions, In Proceedings of the 11th International Symposium on Mathematical Morphology (ISMM 2013), LNCS–7883, pp. 181–192. Ćurić, V., Angulo, J., Morphological Image Regularization Using Adaptive Structuring Functions, Manuscript for journal publication. VIII Ćurić, V., Landström, A., Thurley, M., Luengo Hendriks, C.L. Adaptive Mathematical Morphology – a Survey of the Field, Accepted to Pattern Recognition Letters Reprints were made with permission from the publishers. The author has contributed considerably to method development, implementations and writing of Papers II-VIII. The author also contributed to Paper I, but to a lesser extent. The work presented in Papers I-III were developed under close discussions with Joakim Lindblad and Nataša Sladoje, who also contributed in writing. The code for multi-modal image registration in Paper II was mostly developed by Hamid Sarve, and Gunilla Borgefors contributed in writing. The work in Papers IV-VI was developed in close discussions with Cris L. Luengo Hendriks. The author developed and implemented the methods, and wrote the papers, but with comments and advices from the coauthors. The method in Paper VII was developed in close discussions with Jesús Angulo. The author implemented the method and wrote the paper. Paper VIII was written in a close collaboration with Anders Landström. The implementations and writing was split between the two. The other coauthors contributed in discussions and comments. Related work In addition to the papers included in this thesis, the author has also written or contributed to the following publications. 1. Lindblad, J., Sladoje, N., Ćurić, V., Sarve, H., Johansson, C.B., Borgefors, G. (2009) Improved quantification of bone remodelling by utilizing fuzzy based segmentation, In Proceedings of the 16th Scandinavian Conference on Image Analysis (SCIA 2009), LNCS– 5575, pp. 750–759. 2. Ćurić, V., Heiliö, M., Krejić, N., Nedeljkov, M., (2010) Mathematical Model for Efficient Water Flow Management, Nonlinear Analysis – Real World Applications, Volume 11, Issue 3, pp. 1600–1612. 3. Ćurić, V., Lindblad, J., Sladoje, N. (2010) The Sum of minimal distances as a useful distance measure for image registration, In Proceedings of the Swedish Symposium on Image Analysis (SSBA 2010), pp. 55–58. 4. Allalou, A., Ćurić, V., Pardo Martin, C., Yanik, M.F., Wählby, C. (2011) Approaches for increasing throughput and information content of image-based zebrafish screens, In Proceedings of the Swedish Symposium on Image Analysis (SSBA 2011), pp. 5–8. 5. Ćurić, V., Luengo Hendriks, C.L., Borgefors G. (2012) Adaptive structuring elements based on salience distance transform, In Proceedings of the Swedish Symposium on Image Analysis (SSBA 2012), pp. 127–130. 6. González-Castro, V., Debayle, J., Ćurić, V. (2014) Pixel Classification using General Adaptive Neighbourhood-based Features, To appear in Proceedings of the 22th International Conference on Pattern Recognition (ICPR 2014), IEEE. Contents 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.2 Thesis outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2 Distance Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Brief Introduction to Distance Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Distance Functions Between Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Distance Functions Between Fuzzy Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Distance Transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.1 Distance Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.2 Gray-Weighted Distance Transform . . . . . . . . . . . . . . . . . . . . . . . 2.4.3 Salience Distance Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Applications to Image Registration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 13 15 19 22 22 24 25 27 3 Mathematical Morphology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Basic Morphological Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Adjunction Property . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Opening and Closing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Other morphological operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 On the Selection of Structuring Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 34 36 37 39 40 4 Adaptive Mathematical Morphology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Adaptivity in Mathematical Morphology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Adjunction Property in Adaptive Mathematical Morphology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Methods for Input-Adaptive Structuring Elements and Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Salience-based Adaptive Mathematical Morphology . . . . . . . . . 4.5 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 43 5 Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Concluding Remarks and Perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 63 64 66 6 Brief Summaries of the Included Papers ........................................... 67 .................................................................................................. 71 Appendix 45 49 50 56 Acknowledgements ................................................................................ Summary in Swedish References 73 ............................................................................. 77 ...................................................................................................... 79 1. Introduction “Essentially, all models are wrong, but some are useful.” George Edward Pelham Box 1.1 Motivation Image analysis deals with extracting relevant information from the data represented as digital images. This field started in the 1960’s with the extensive use of computers that evolved in previous few decades. Today, different image analysis methods are used to solve some difficult problems that arise in medicine, biology, astronomy, etc. Those methods benefit from using appropriate mathematical models, and therefore underlying mathematical theories and tools are highly important. Various mathematical models can be used in image analysis. For instance, images can be considered as matrices, and hence, tools developed in linear algebra can be used. Digital images can be represented in a discrete grid, and methods derived in the field of discrete geometry can be a useful tool as well. Similarly, as a discrete structure, an image can be considered as a graph, which is sometimes important for real applications due to a lower computational cost for a method. Furthermore, an image can be considered in a continuous framework by evolving a partial differential equation over time or using Fourier analysis. A comparison of shapes is a problem that often appears in image analysis. This is a difficult task because shapes vary in size, orientation, contrast and other features. Shapes in images can be represented as sets, and distance functions between sets give a measure for how similar the shapes are. These distance functions can be used for various image analysis related tasks, such as shape and image registration, and image retrieval. Hence, it is of interest to study existing distance functions between sets of points, and further develop new ones. Possibly the first well-defined non-linear theory in image analysis is mathematical morphology, which can be used for image filtering, segmentation and measurement and characterization of objects in the image. Mathematical morphology began as a technique for studying random sets with applications in the mining industry. Mathematical morphology was first defined for binary images, and then extended to graylevel and color images. Furthermore, it is defined for lattices and graphs 11 as well as in the continuous domain. Mathematical morphology operators (morphological operators) require a structuring element. Typically this is defined as a small rigid set. The construction of adaptive structuring elements that depend on the content in the image is now a popular and challenging task in mathematical morphology. In this thesis, we study distance functions and their use for the construction of adaptive structuring elements. The main contributions of this thesis are: • We develop new distance functions between set of points and evaluate their applicability to the task of shape and image registration. • We propose new methods for constructing adaptive structuring elements and adaptive structuring functions (adaptive morphological operators, in general) that are based on different distance functions. • We present a unified framework for the proper computation of adaptive morphological operators as well as the applicability of adaptive mathematical morphology to image regularization. 1.2 Thesis outline This thesis consists of six chapters, including the introductory chapter. The next chapter, Distance functions, presents an overview of distance functions between sets of points and their applicability to image registration. The third chapter, Mathematical Morphology, offers the reader a brief overview on the field of mathematical morphology related to this thesis. Methods for adaptive structuring elements as well as theoretical advances and applications of adaptive morphological operators are presented in Adaptive Mathematical Morphology. Conclusions and Future Work, discusses the contributions of the thesis and presents possible extensions and improvements of this work. Finally, the last chapter presents short summaries of the papers included in the thesis. 12 2. Distance Functions Distance functions have been used for many problems in image analysis and are some of the older tools. They are also used for defining different distance transforms, which is an efficient way to calculate distances between pixels in the image. In this chapter, we consider distance functions between sets of points, which can be computed in linear time with respect to the number of pixels in the image. Their extension to distance functions between fuzzy sets are also presented. We explore the applicability of distance functions between sets of points to shape registration and object matching. 2.1 Brief Introduction to Distance Functions Let X be a non-empty set. A function d : X × X → R is called a metric on X, if for all x, y, z ∈ X the following properties hold [38]: (i) Nonnegativity d(x, y) ≥ 0, (ii) Symmetry d(x, y) = d(y, x), (iii) Reflexivity d(x, x) = 0, (iv) Triangle inequality d(x, y) ≤ d(x, z) + d(z, y). The pair (X, d) is usually called a metric space. Some functions do not satisfy all properties of a metric, i.e., properties (i)–(iv). For instance, if a function d satisfies nonegativity (i), symmetry (ii) and triangle inequality (iv) it is called a pseudometric, while when satisfies properties (i)–(iii) it is called a semimetric. A function d is a distance function if it satisfies the nonegativity property (i). Various distance functions are defined in the literature, and used for different applications [38]. Statements in the rest of this section that are not followed by a reference implicitly refer to Deza and Deza [38]. First, we review some well-known distance functions that will be used later in this thesis. Let x = (x1 , ..., xn ) ∈ Rn and y = (y1 , ..., yn ) ∈ Rn , n ∈ N. The Euclidean metric, and possibly the most used distance function, is defined as ! 21 n X 2 . (2.1) d(x, y) = (xi − yi ) i=1 13 (a) (b) (c) Figure 2.1. Open balls (white objects) of the same radius for different distance functions: (a) City block; (b) Euclidean; (c) Chessboard. The Euclidean metric is a special case of Lp , p > 0 metrics defined by ! p1 n X . (2.2) dp (x, y) = (xi − yi )p i=1 For p = 1, this metric is called L1 distance or City block distance. When p → +∞, then dp converges to the Chebyshev distance or L∞ distance, defined by d∞ (x, y) = max{|xi − yi | : i = 1, ..., n}. (2.3) This distance is also known as the Chessboard distance, when one considers x, y ∈ Zn . Given a metric space (X, d), the open ball with the centre x0 and radius r > 0 is defined by B(x0 , r) = {x : d(x0 , x) < r}. (2.4) The open balls for different Lp metrics are shown in Figure 2.1. Let P(a, b) = {a = x1 , ..., xn = b} be a path between points a and b in X, where xi and xi+1 , i = 1, ..., n − 1, are adjacent points in the path. The cost of the path P(a, b) is defined by c(P(a, b)) = n−1 X c(xi , xi+1 ), (2.5) i=1 where the distance function c is the cost to travel between two adjacent points in the path, i.e., the value c(xi , xi+1 ) corresponds to the cost to travel from a point xi to a point xi+1 . The length of a shortest path from a point a to a point b is the minimal cost c of all paths between these two points, i.e., the length is defined by min{c(P(a, b)) : P(a, b) ∈ Π}, (2.6) where Π denotes the set of all paths between points a and b. This type of a distance function is usually refer to as path-based distance. 14 2.2 Distance Functions Between Sets A number of different distance functions are defined between sets of points [38]. Here, we consider distance functions that are computable in linear time with respect to the number of points in the set, and we investigate their applicability to shape registration. A distance between two sets is often based on the point-to-set distance. Let A and B be two non-empty, closed and bounded subsets of X. The distance between a point a ∈ X and a set B ⊂ X is defined as ^ d(a, B) = d(a, b), (2.7) b∈B V W where denotes the infimum. In the rest of the thesis, the symbol denotes the supremum. Note that, the point-to-point distance d(a, b) can be any metric. An element b0 ∈ B is called an element of best approximation to a given point a if ^ d(a, b0 ) = d(a, b), (2.8) b∈B i.e., if the infimum is attained. The earliest distance defined between two sets is possibly the Hausdorff metric, here denoted with dH , [49] ! _ _ d(a, B), d(b, A) . (2.9) dH (A, B) = max a∈A b∈B It is often considered that the Hausdorff metric is introduced in 1914, when this metric was presented in Hausdorff’s famous book [49]. However, the first appearance of a similar distance function, that is nowadays called the Hausforff metric, can be found in the PhD thesis of Pompeiu in 1905 [76]. Therefore, some authors suggest that this metric should be called the Hausdorff–Pompeiu metric or the Pompeiu– Hausdorff metric [9, 11]. The Hausdorff metric is based on an element with the largest distance to the other set (see Figure 2.2 for 2D binary objects). This implies that the Hausdorff metric is highly sensitive to outliers, i.e., this metric depends on a few points that are far from the bulk. We show this property of the Hausdorff metric in Paper I and Paper II. Nonetheless, this metric is very widely used. An extensive list of disciplines where the Hausdorff metric has been used was recently listed by Berinde and Pacurar [9]. This is a really impressive list of disciplines ranging from mathematics to biophysics and economics. Different modifications of the Hausdorff metric have been proposed in the literature to deal with its aforementioned issues with outliers. An 15 Figure 2.2. Illustration of the Hausdorff distance for two sets represented as 2D shapes. The Hausdorff distance is based on two maximal point-to-set distances for sets A and B, i.e., the W Hausdorff distance is the W maximum of the two d(b, A). following values: d(a, b0 ) = d(a, B) and d(b, a0 ) = a∈A b∈B empirical evaluation of different modifications of the Hausdorff metric was presented by Dubuisson and Jain [41]. One modification of the Hausdorff metric, that is the least sensitive to noise in images, is called the Modified Hausdorff distance, dMH , and defined as dMH (A, B) = max ! 1 X 1 X d(a, B), d(b, A) , |A| |B| a∈A (2.10) b∈B where |A| and |B| denote the cardinality of the sets A and B, respectively. dMH is not a metric, since the triangle inequality is not satisfied [41]. This distance function is less sensitive to noise due to its definition as the mean of point-to-set distances for all points from both sets, rather than taking the supremum of those point-to-set distances, which is used in the definition of the Hausdorff metric (2.9). A distance similar to the Modified Hausdorff distance is the Sum of minimal distances, dSMD , defined by Eiter and Mannila [42], 1 dSMD (A, B) = 2 X a∈A d(a, B) + X b∈B ! d(b, A) . (2.11) This distance is not a metric, since it does not satisfy the triangle inequality [42]. As we show in Paper I, this distance function is less sensitive to outliers than the Hausdorff metric. This is especially important for image analysis tasks such as shape matching and image registration, which will be discussed in Section 2.5. As can be seen from the definition of the Hausdorff metric (2.9), this distance function is dependent on the boundaries of objects (see Figure 2.2). The Chamfer matching distance, dCH , was defined by Borge16 fors [16] as dCH (A, B) = X d(a, ∂B), (2.12) a∈∂A where ∂A and ∂B are the boundaries of sets A and B, respectively. This distance function is not a metric, since the symmetry property is not satisfied. Furthermore, this distance function is computationally less expensive than previously introduced distance functions, since it uses only points from the boundaries of the sets. On the other hand, it can be very sensitive to noise that influences the object boundary, as we show in Paper I and Paper II. Apart from the distance functions defined for sets by the point-to-set distance, they can be defined in some other ways too. Probably the easiest way to define a distance between two sets is to take the infimum of all the distances between any two of their respective points, i.e., ^ dINF(A, B) = {d(x, y) : x ∈ A, y ∈ B}. (2.13) This distance function is not suitable for comparing sets. For instance, the distance between two sets with non-empty intersection is zero, being independent of the cardinality of the intersection between these sets. The symmetric difference between sets A and B, defined by A△B = (A \ B) ∪ (B \ A) provides information about these two sets. The metric that corresponds to this set operation, here denoted with dSD, was defined by Klette and Rosenfeld [58] as dSD (A, B) = |A△B|. (2.14) Its main disadvantage is that it does not take into account information about the spatial position of the points in the set, i.e., the position of the shape in the image, which is a specific application that we are looking for in this thesis. For instance, the distance between two disjoint binary objects is the same, independently how far the objects are, as it depends only on cardinality of the objects. Hence, the usefulness of this distance function for shape comparison is limited. The above distance functions between sets (except the one defined by (2.13)) are compared in an empirical study on rigid body registration of binary shapes in Paper I. In that paper, we introduce the notion of the complement distance function d, defined as d(A, B) = d(A, B), where A and B are complements of sets A and B, respectively, with respect to the universe X. The complement distance d adds new information to a distance d if and only if d 6= d. For instance, dSD = dSD and dSMD 6= dSMD . We explore different combinations of dSMD and dSMD , which are the two distance functions with best properties for image registration (among 17 (a) (b) (c) (d) Figure 2.3. (a) Binary set A; (b) Binary set B; (c) Contribution of every point from sets A and B to dSMD (A, B); (d) Contribution of every point from sets A and B to dCW (A, B). the ones compared in Paper I). Our experiments show that it is better to incorporate complements of the sets on a point level, rather than at the whole set level. Following this conclusion, in Paper II, we propose the Complement weighted sum of minimal distances as a weighted version of the Sum of minimal distances, where each point in the set contributes to the distance function according to its spatial position within the set. The underlying idea is to define a distance function in which points that are deeper inside of the object have higher importance and contribution to the distance (see Figure 2.3). This assumption is suitable for problems in image analysis, since points close to the boundary of an object are usually more affected by noise. Therefore, to each point a of a set A is assigned a weight dependent on its position within the set A, i.e., a weight is calculated as d(a, A). The weighted contribution of all points of set A is normalized by the sum of all given weights. The contribution of all points from set B is computed in a similar way. The Complement weighted sum of minimal distances, dCW , between sets A and B is defined as (Paper II) P d(a, B)d(a, A) 1 a∈A + dCW (A, B) = P 2 d(a, A) a∈A 18 d(b, A)d(b, B) b∈B P . d(b, B) P b∈B (2.15) Figure 2.4. Crisp (left) and fuzzy (right) representation of the same object. Crisp object is obtained from the fuzzy one taking the α−cut for α = 0.5. This distance is a semimetric because the triangle inequality is not satisfied. The denominators are non-zero, since for a non-empty set C ⊂ X, it holds that d(c, C) 6= 0, c ∈ C. 2.3 Distance Functions Between Fuzzy Sets Broadly speaking, a fuzzy set is a generalization of a crisp set. An element either belongs to a crisp set, or not, while the belongingness of an element to a fuzzy set can be partial, and it is described by a value between zero and one. The theory of fuzzy sets is a good mathematical model for gray-level images in general, as well as for more specific image analysis problems as in fuzzy segmentation, which produces objects with non-sharp boundaries [75]. For instance, a crisp and a fuzzy representation of the same discrete object are shown in Figure 2.4. Let us briefly introduce fuzzy set theory. A fuzzy set A on a reference set X, is a set of ordered pairs A = {(x, µA (x)) : x ∈ X}, where µA : X → [0, 1] is the membership function of the fuzzy set A, [120]. An α−cut of a fuzzy set A, is the set αA = {x ∈ X : µA (x) ≥ α}, α ∈ (0, 1]. The height of a fuzzy set A is h(A) = max µA (x), while the support of x∈X A is defined as Supp(A) = {x ∈ X : µA (x) > 0}. The complement A of a fuzzy set A, is A = {(x, 1 − µA (x)) : x ∈ X}. There exist several ways to define a distance function between two fuzzy sets, and a good overview on this topic was presented by Bloch [12]. Two different groups of distance functions between fuzzy sets were considered in that paper. The first group is composed of methods that are based solely on comparing membership functions of two fuzzy sets (A, µA ) and (B, µB ), and the best known are Lp , p > 0 distances, defined by 1 X p p . (2.16) |µA (x) − µB (x)| d(A, B) = x∈X Several other distances of this type are listed in a more recent paper [119]. 19 Distance functions that combine membership functions and spatial information of the support of fuzzy sets belong to the second group. Taking the spatial information into account, these distance functions are more suitable for image analysis applications, and such distance functions are explored in Paper III. In that paper, we present extensions of the distance functions between crisp sets of points that have the best performance for image registration (Paper I and Paper II), which are the Sum of minimal distances, dSMD , and the Complement weighted sum of minimal distances, dCW . The most natural way to extend distance functions between crisp sets to distance functions between fuzzy sets is to consider integration over α−cuts [79]. Integration over α−cuts is a general principle for extending properties and relations on crisp sets to corresponding ones on fuzzy sets. Using the integration over α−cuts, a distance function between two fuzzy sets A and B is defined as α d (A, B) = Z 1 d(αA,αB) dα, (2.17) 0 where d is a distance function between crisp sets of points. Following the above discussion and using (2.17), the Sum of minimal distances, dSMD , and the Complement weighted sum of minimal distances, dCW , are defined for fuzzy sets as (Paper III) dα SMD (A, B) = dα CW (A, B) = Z Z 1 dSMD (αA,αB) dα, (2.18) dCW (αA,αB) dα. (2.19) 0 1 0 The main drawback with this approach is that dα (A, B) = ∞ if the heights of the two fuzzy sets are not the same, i.e., h(A) 6= h(B). This problem also occurs when d is the Hausdorff metric in (2.17). Several modifications have been proposed to solve this issue [20, 30], but none of them is widely accepted [21]. Despite that most distance functions between sets rely on the pointto-set distance (Section 2.2), the distances between two fuzzy sets are rarely defined using the point-to-set distance for fuzzy sets. Two definitions of the point-to-set distance for fuzzy sets were proposed by Bloch and Maı̂tre [13]. The first definition is based on integration over α−cuts, where the distance between a point a and a fuzzy set B is defined as Z 1 Z 1 ^ d(a, b) dα. (2.20) d(a, B) = d(a,αB) dα = 0 20 0 b∈αB The second definition is based on weighting the points from the support, Supp(B), of the fuzzy set B with their membership values, ^ d(a, B) = d(a, b) · F (µB (b), (2.21) b∈Supp(B) where F (t) is a decreasing function of t. Note that the point-to-point distance d(a, b) is the spatial distance between two points and is independent on their membership values. A point-to-set distance can also be defined using morphology for fuzzy sets, where the value of such a distance is a fuzzy number [13]. When (2.20) is used, the Sum of minimal distances and the Complement weighted sum of minimal distances for fuzzy sets are, respectively, defined as (Paper III) X X 1 (2.22) d(a, B) + d(b, A) , dps SMD (A, B) = 2 a∈Supp(A) dps CW (A, B) = P b∈Supp(B) d(a, B) · d(a, A) 1 a∈Supp(A) P 2 d(a, A) a∈Supp(A) + P d(b, A) · d(b, B) b∈Supp(B) P b∈Supp(B) d(b, B) . (2.23) In Paper III, we use only the definition (2.20) of the point-to-set distance. We tried different functions including F (t) = 1 − t, F (t) = 1 − t2 and F (t) = e−t for the point-to-set distance defined by (2.21), but we did not observe any good performance. Furthermore, if a ∈ Supp(B), then for b = a d(a, b) = 0 ⇒ d(a, b) · F (µ(b)) = 0 ⇒ d(a, B) = 0, (2.24) which reduces the usefulness of this distance definition for the task of object matching. Therefore, we do not use the point-to-set distance defined by (2.21). When computing the point-to-set distance (2.20), we assumed that µ(a) = 1 for all points a, and in this way the membership value of the observed point in the point-to-set distance is neglected. This membership value was recently considered when defining a point-to-set distance for fuzzy sets [64], by defining Z µ(a) d(a, B) = d(a,αB) dα. (2.25) 0 Another definition of a point-to-set distance is based on the length of the shortest path from a fuzzy point to a fuzzy set. This approach, 21 being based on the shortest paths, becomes more natural since it is consistent with the distance computations in images (Section 2.4). The Sum of minimal distances and the Complement weighted sum of minimal distance were defined using the last two definitions for point-toset distances for fuzzy sets, and their usefulness for the task of template matching and object classification was presented by Lindblad and Sladoje [64]. 2.4 Distance Transforms In general, an image can be represented by a function f : D ⊂ Rn → Rm , n, m ∈ N. A binary image can be seen as a function f defined as follows 1, x ∈ A, f (x) = (2.26) 0, x∈ / A, where A ⊂ D. A gray-level image can be considered as a function f when m = 1, while for color or multivalued images m > 1. The distance transform is an image operator that computes distances between points in the image. A distance transform is usually considered to be defined for binary images, while for gray-level images one can define a gray-weighted distance transform or a geodesic distance transform. Furthermore, a distance transform can be computed for color images, too. In these distance transforms, the underlying point-to-point distance determines the distance transform, as will be seen later in this section. In this section, we first review the distance transform for binary and gray-level images. Then, we present the salience distance transform that is used in Chapter 4 and for the methods that are proposed in Paper IV, Paper V and Paper VI. Note that we present distance transforms for 2D images, but they can be computed for arbitrary dimensions [14]. 2.4.1 Distance Transform To keep presentation simple, we will present distance transforms for two-dimensional digital images. Let (X, d), X ⊂ Z2 be a metric space and A ⊂ X. And let f be a binary image defined by (2.26). Then, the distance transform of an image f is defined as ^ d(x, y), x ∈ X. (2.27) DT [f ](x) = y∈A Note that equation (2.27) corresponds to the point-to-set distance d(x, A) defined by (2.7). A distance transform can be seen as an image, where 22 (a) Mask M1 (b) Mask M2 Figure 2.5. Masks for the computation of a distance transform for 3 × 3 neighbourhood. each pixel in the object has a positive numerical value that corresponds to its distance to the background. The basic idea, utilized for most distance transforms, is to approximate the global distance by propagation of local steps, i.e., distances between neighbouring points in the image. This approach was presented by Rosenfeld and Pfaltz [86]. A distance transform is dependent on the predefined weights (cost) to move from a pixel to the neighbouring one, and it can be computed by propagating local steps using the two-pass chamfering algorithm, where the weights are given by two masks [86]. An example of masks and weights for 3 × 3 neighbourhood is shown in Figure 2.5. Here, we present the case for 2D images, but a similar approach can be used for images of higher dimensions [15]. Prior to running the chamfering algorithm, the elements in the object are set to zero and the elements in the background are set to infinity. Next, the image is scanned from the upper left corner, first right then down (forward scan) using the mask M1 (see Figure 2.5 (a)), followed by a scan from the lower right corner (backward scan) using the mask M2 (see Figure 2.5 (b)). The distance for a pixel a is computed by propagating distance values from the neighbouring pixels in the distance map DT. This process is performed by adding the weight for the local step. The distance in a point a is the minimum value of itself and the values in the neighbourhood increased by respective local weights, i.e., DT(a) = min (DT(a + b) + w(b)), b∈M1 (2.28) where b is a vector from a to pixels in the mask M1 , and w(b) is the corresponding weight given in the mask. The final result is computed using the second (backward) scan by DT(a) = min (DT(a + b) + w(b)). b∈M2 (2.29) The size of the mask and the weights determine the distance transform. For instance, for a mask of size 3×3, the pair w = hw1 , w2 i denotes 23 the weights in the mask, where w1 is the weight for edge neighbours and w2 is the weight for vertex neighbours (see Figure 2.5). Although it seems natural to use the Euclidean distance for the weights, these are not the best weights to use since this distance transform overestimates long distances. Instead, Borgefors [15] recommended to use weights h0.95509, 1.36930i or the integer weights h3, 4i, which have better properties. The use of the weights h1, 2i results in the City block distance, while weights h1, 1i yield to the Chessboard distance. It is possible to utilize the same concept by having larger neighbourhoods [106], as well as for other grids different from Zn [107]. Note that the Euclidean distance transform can be computed in linear time with the respect to the number of pixels in the image [70]. 2.4.2 Gray-Weighted Distance Transform The computational process of a distance transform for gray-level images f : X ⊂ Z2 → R is a bit different than for binary images. A 2D gray-level image can be represented as a surface, denoted here with S, embedded in 3D space, with two spatial coordinates and one coordinate that represents the gray-level value in the image. A geodesic distance between two points (a, f (a)), (b, f (b)) ∈ S, a, b ∈ Z2 is the shortest path between two points along the surface [57]. A distance transform that computes the geodesic distance between points in the image is called a transform on gray-level surfaces [56]. Various different geodesic distances have been considered in the literature [56, 63, 91, 100, 109]. Similarly to the distance transform for binary images, the geodesic distances are based on the local step (cost) to move from one point to the neighbouring one along the surface S. The cost can be dependent on two incommensurate domains: (i) the spatial positions of pixels; (ii) gray-level values for pixels. For instance, in Section 4.4. we use the following cost c for the construction of adaptive structuring elements [109] c(xi , xi+1 ) = d(xi , xi+1 ) + σ|f (xi ) − f (xi+1 )|, (2.30) where d is a spatial distance between two neighbouring points xi and xi+1 in a path that connects points a and b. The parameter σ > 0 makes the two domains commensurate. Then, the geodesic distance between the points a and b is computed with (2.6) by taking the minimal cost of all possible paths between these points. The main difference between the distance transform and the grayweighted distance transform is how they can be computed. While the distance transform can be computed by the two-pass chamfering algorithm, the gray-weighted distance transform requires performing a 24 number of iterations until the stability of the distance is obtained. One way to compute the shortest path between all points in an image is by using Dijkstra’s algorithm [39]. Another method for distance computations is the fast marching algorithm [97]. This method is based on propagating a wave front from a set of pixels by using partial differential equations, in particular the Eikonal equation. In addition, the computational cost to compute the gray-weighted distance transform is O(N log N ), where N is the number of pixels in the image. 2.4.3 Salience Distance Transform The classical distance transform uses as an input a binary image that might be obtained by thresholding. On the other hand, the salience distance transform eliminates the need for binarization, and it incorporates edge attributes such as strength (gradient magnitude), length or curvature. To keep the presentation simple, we focus on the approach where the salience distance transform is computed using the strength of the edges and without other attributes of the edges. The underlying assumption of the salience distance transform is that the edges are weighted by their importance, where stronger edges have higher importance, i.e., salience. Several ways of incorporating salience into the distance transform have been studied by Rosin and West [88]. Three algorithms were proposed in that study, and each one uses the two-pass chamfering algorithm. The first algorithm ([88], Algorithm 1) is the simplest one. The distance transform and the propagated magnitude of the edges are computed separately and stored in two images as follows: b̃ = arg min(DT(a + b) + w(b)), b∈M1 DT(a) = DT(a + b̃) + w(b̃), MP(a) = MP(a + b̃) + w(b̃), where DT(a) and MP(a) are the classical distance transform and the propagated magnitude in a point a, respectively. The initialization for the salience distance transform is similar to the classical distance transform, where the elements of the background (non-edge points) are set to infinity. The edge points are set to zero for the distance transform DT, and to the initial edge strength for the propagated magnitude MP. After the forward scan, the backward scan should be performed in a similar manner. Then, the salience distance transform is computed by simply dividing the classical distance transform with the propagated magnitude, i.e., SDT(a) = DT(a)/MP(a), for each point in the image. Since 25 (a) (b) (c) Figure 2.6. Salience distance transform computed with Algorithm 2. (a) Input image; (b) Edge strength image; (c) Salience distance transform. the propagated edge magnitude map MP is discontinuous, the resulted salience distance transform SDT is also discontinuous. Similarly to the first algorithm, the authors proposed an algorithm ([88], Algorithm 3) that propagates distance as well as magnitude of the edges in the same local step of the two-pass chamfering algorithm. The salience distance transform is computed during the propagation process by dividing the propagated distance with the propagated edge magnitude, repeatedly as b̃ = arg min b∈M1 DT(a + b) + w(b) , MP(a + b) DT(a) = DT(a + b̃) + w(b̃), MP(a) = MP(a + b̃) + w(b̃). This initialization and the computation process is the same as for the above described algorithm, i.e., the resulted salience distance transform is computed as SDT(a) = DT(a)/MP(a). This algorithm is the most computationally expensive one, but it produces the salience distance transform that is continuous. Rosin and West also proposed an algorithm ([88], Algorithm 2) that takes the edge strength image as an input (Figure 2.6 (b)), where the edge pixels are initialized with the negative values of their magnitudes. The non-edge pixels are set to infinity, similarly to as in the distance transform. Then, the salience distance transform is computed with the classical two-pass chamfering algorithm (Figure 2.6 (c)). The salience map produced with this algorithm is continuous. An alternative algorithm to compute the salience distance transform is to use a threshold decomposition of the input edge image [87]. For this algorithm, the salience distance transform is computed as the sum of distance transforms over a threshold decomposition of the edge magnitude image; the result of this algorithm is dependent on the number of 26 (a) (b) Figure 2.7. The salience distance transform with respect to edge strength using Algorithm 2, for 1D function. (a) Edges are far from each other; (b) Edges are close to each other. thresholds used. The resulting salience distance transform is zero at the edge pixels, and the magnitude becomes irrelevant at those points. The same conclusion is valid for Algorithms 1 and 3. However, the salience distance transform obtained by Algorithm 2 has non-zero values for the edge pixels, which makes a distinction between edges based on their salience. Being based on the magnitude of edges in the image, the salience distance transform is dependent on the spatial positions of the edge points. If a weak edge is far from a strong one, i.e., one with higher magnitude, then the resulting salience distance transform might be influenced by the weaker edge (Figure 2.7 (a)). However, if two edges are close to each other, the weaker one might be overshadowed by the stronger one (Figure 2.7 (b)). The salience distance transform is a good alternative to gray-weighted distance transform, taking advantage of easy computations of the classical distance transform by incorporating salience in the chamfering algorithm. The resulting salience map is continuous (for most of the presented algorithms), which is a desirable property when constructing adaptive structuring elements (Section 4.4). For this purpose, we use the second algorithm presented here ([88], Algorithm 2), where the edge strength is considered as salience. 2.5 Applications to Image Registration The process of aligning two images of the same object is far from easy, since images, even those acquired with the same imaging technique, often have large variations in illumination, pose, noise, etc. Moreover, if objects are imaged with different imaging modalities, they have different characteristics. Generally speaking, image registration is the process of finding the best geometric transformation that aligns two images, one 27 called an observed image O, and another a reference image R. The registration is often considered as a process that seeks the transformation that maximizes a similarity measure between two images [103]. Instead, the geometric transformation that minimizes a distance function between the images is good because it simplifies the search. Hence, the registration process can be identified by the following minimization problem arg min d(O, T (R; y)), (2.31) y where d is a distance function, T is a geometric transformation and y belongs to the set of all reasonable transformation parameters. The choice of geometric transformation depends on the application. In the rigid body case, i.e., when the object is not deformed between the different imaging occasions, the transformation typically includes rotation and translation. On the other hand, if that is not the case, the registration is considered to be non-rigid, i.e., the process where the objects might be deformed between imaging occasions. A rigid-body registration is often required as the first step in a non-rigid registration problem. Extensive studies on different image registration methods were presented by Zitova and Flusser [121] and by Hajnal and Hill [47]. One of the key problems in image registration is how to find an appropriate distance function, which is the underlying requirement for a good registration algorithm, since even very sophisticated optimization algorithms do not have good performance with an inadequate distance function. A number of different distance functions have been used for image registration. For instance, the most commonly used intensitybased distance functions are the sum of squared differences and cross correlation; they are useful for registration of images that are acquired in the same modality [8, 90]. The distance function called mutual information is used for multi-modal image registration [65, 118]. Similarly, the distance called normalized gradient fields was proposed for multi-modal image registration problems [46]. The Gromov–Hausdorff distance was recently used for non-rigid shape comparison [24]. Of the distance functions presented in Section 2.2, the Hausdorff metric [54, 55] and the Chamfer matching distance [26, 93] have been used for rigid-body registration. In this thesis, we consider distance functions between sets that can be computed in linear time (Section 2.2), and evaluate their usefulness for rigid-body shape registration. Desirable properties of a distance function used for this task are: (1) The distance between the object and a transformed version of the object monotonically increases with increasing translation and rotation of the object. (2) The distance function has low sensitivity to noise. 28 (a) (b) (c) Figure 2.8. The Chamfer matching distance does not distinguish between points of the object and points of the background. (a): Reference image R, (b): Observed image O, (c): Observed image (blue color) and registered image (orange color) superimposed. The elements that belong to both shapes are shown in gray. The first property enforces that an optimization process (2.31) has only one minimum, which is the global one. The second property is important for real world applications, where images are often corrupted by noise. The best way to evaluate the distance functions presented in Chapter 2.2, especially considering property (1), is to use a greedy search in the registration process. Therefore, we use this simple optimization algorithm since the objective of Paper II is to compare distance functions, and this method emphasizes properties of distances. An ideal situation would be that the distance function has only one global minimum. This distance behaviour is often impossible in practice, and the algorithm might get stuck in a local minimum. Therefore, we also evaluate how far the local minima are from the desirable global one. We perform four different experiments to show the usefulness of the Complement weighted sum of minimal distances to image registration (Paper II). The first evaluation is conducted on a set of binary shapes with different displacements and noise conditions. For most of the experiments, the Complement weighted sum of minimal distances has the best performance, followed by the Sum of Minimal distances (see Paper II, Table 1). The Hausdorff distance has a low rate of perfect registrations, especially when shapes are degraded by noise. The Chamfer matching distance, being based just on object boundaries, is highly influenced by the boundary noise, and the greedy search usually got stuck in a local minimum not far from the global one (see Paper II, Figure 9). Also, the Chamfer matching distance does not differentiate between points of the object and points of the background, which may lead to a local minimum at a position where the registered objects have a large common boundary while the interiors are not well matched (see Figure 2.8). 29 (a) (b) (c) Figure 2.9. Multi-modal image registration. (a) Histological slice; (b) Corresponding slice in 3D SRµCT volume found by the Complement weighted sum of minimal distances; (c) Visualization of this 3D SRµCT volume. We note an interesting property of the studied distance functions: the registration is occasionally better for shapes with noise than for the noise free case, i.e., noise sometimes removes local minima. This property of distance functions shows their imperfections, and that the selection and construction of a “perfect” distance function for a particular task is not an easy problem. In Paper II, we also present three examples of real registration problems in which the Complement sum of minimal distances outperforms other distance functions presented in Chapter 2.2: (i) Recognition of handwritten digits by using the nearest neighbour classification algorithm. A shape is correctly classified if the reference shape with the minimal distance is the same digit as the observation shapes. (ii) Registration of 2D images extracted from synchrotron radiation micro computed tomography (SRµCT) bone implants volume (Figure 2.9 (c)). (iii) Multi-modal registration of the exact location of a 2D histological slice of a bone implant within a 3D SRµCT volume of the same implant (Figure 2.9). In Paper III, we introduce the Complement weighted sum of minimal distances and Sum of minimal distances for fuzzy sets. We perform distance-based object classification (nearest neighbour rule) for crisp and fuzzy discrete object representations of discrete disks and octagons. Note that a fuzzy discrete representation of objects (Figure 2.10 (c) and (f)) often appears visually more similar to the corresponding continuous objects (Figure 2.10 (a) and (d)) than a crisp discrete representation at the same resolution (Figure 2.10 (b) and (e)). Fuzzy representations of disks and octagons were generated using pixel coverage digitization [98] of continuous crisp disks and octagons, respectively. In the pixel coverage representation, the membership of a pixel is equal to the relative area of the pixel that is covered by the object [98]. It has been shown that this representation provides a good framework for higher precision of estimation of different measures [99]. The corre30 (a) (b) (c) (d) (e) (f) Figure 2.10. Crisp and fuzzy discrete representations of continuous objects: (a) Continuous disk; (b) Crisp discrete disk; (c) Fuzzy discrete disk; (d) Continuous octagon; (e) Crisp discrete octagon; (f) Fuzzy discrete octagon. sponding crisp object representations are obtained by taking the α−cut at α = 0.5, of the fuzzy object representations (Figure 2.10). The recognition, i.e., correct classification rate, of the fuzzy discrete objects using distances between fuzzy sets (Section 2.3) is better than for the discrete objects of the same spatial resolution, using the corresponding distances between crisp sets. We also show that the performance of distance functions between fuzzy sets depends on the number of membership levels used for fuzzy set. Obviously, more membership levels provide more information to the distance function, but this will not necessarily lead to the improvement of a distance function for a particular task (Paper III). 31 3. Mathematical Morphology This chapter presents the basics of mathematical morphology, and introduces the notation used in the following chapters. Mathematical morphology was introduced in mid 1960s by two French scientists, Georges Matheron and Jean Serra, who developed the main concepts and tools [69, 95, 96]. Since then, mathematical morphology has been widely used in image analysis, and it is an integral part of almost all basic image analysis courses. A number of excellent books exist on this topic including ones by Soille [102], and Najman and Talbot [74]. Mathematical morphology is based on theories and tools from various branches of mathematics. The field began as a technique for analysing binary images with applications in the mining industry. When mathematical morphology was introduced, it was based on set-theoretical notions. Since then, the field developed to gray-level images using the notion of umbra transform [105], and later using the notion of complete lattices [53, 83]. Nowadays, mathematical morphology is defined on various structures such as multivalued images [6], graphs [50, 115] and manifolds [80], and it can be defined using partial differential equations [23]. In this thesis, we consider mathematical morphology for gray-level images, i.e., for functions f : D ⊂ Rn → R. We review basic morphological operators that use the set S ⊂ Rn to probe the image under study. The set S is referred to as structuring element. Furthermore, such structuring element is often referred to as flat structuring elements because it has two dimensions in the case of 2D images, i.e., when n = 2. The shape and size of a structuring element are chosen depending on the application. Initially, structuring elements were fixed, i.e., one structuring element is used for every point in the image. In the rest of the thesis, we will refer to such structuring elements as rigid. Since the selection of structuring elements is not always an easy task, it recently became an increased interest to define structuring elements that adapt to structures and orientations in the image. Chapter 4 is devoted to this topic. A structuring element is defined with respect to the origin. The origin allows the positioning of the structuring element at a given point. Structuring element at point x ∈ D means that its origin coincides with the point x. By translating a structuring element to a point x, its origin will coincide with x, i.e., Sx = {b + x : b ∈ S}. It should be stressed 33 (a) S1 (b) S2 (c) S3 Figure 3.1. Different structuring elements in Z2 : (a) Square structuring element of side 5; (b) Discrete disk of radius 3; (c) Structuring element that does not contain its origin. The origin of these structuring elements is marked with X. here that a structuring element does not necessarily contain its origin. In Figure 3.1, we show three different structuring elements in Z2 . Since an n dimensional gray-level image f corresponds to an n + 1 dimensional surface, the structures in the image can be investigated with mathematical morphology using functions s : D ⊂ Rn → [−∞, 0]. These functions can be seen as structuring elements with weights assigned to each point. These functions are called non-flat structuring elements, or structuring functions. Similarly to structuring elements, it is important to define the origin of a structuring function. Statements in the rest of this chapter that are not followed by a reference implicitly refer to Soille [102]. 3.1 Basic Morphological Operators Let L be a set of all gray-level images with the domain D ⊂ Rn , i.e., L = {f |f : D → R}. Let (L, ≤) be a complete lattice with a partial order “≤” defined by f ≤ g ⇔ (∀x ∈ D) f (x) ≤ g(x), (3.1) where f, g ∈ L. A lattice is complete if all subsets have both an infimum and a supremum. Each structuring element S has one corresponding structuring element S ∗ , called reflected or transposed structuring element, that is simply the reflection of S through the origin. That means, S ∗ = {−b : b ∈ S}. For instance, S1 = S1∗ , S2 = S2∗ , S3 = S3∗ (see Figure 3.1) The two basic morphological operators are the erosion εS : L → L and the dilation δS : L → L defined as [95] ^ [εS (f )](x) = f (y), x ∈ D, (3.2) y∈Sx 34 (a) Input f (b) Erosion εS2 (f ) (c) Erosion εS3 (f ) (d) Dilation δS1 (f ) (e) Dilation δS2 (f ) (f) Dilation δS3 (f ). Figure 3.2. Morphological operators with different structuring elements that are shown in Figure 3.1. [δS (f )](x) = _ f (y), x ∈ D. (3.3) ∗ y∈Sx We now turn our focus to the non-flat case, i.e. structuring functions. Let sx : D → [−∞, 0] be an arbitrary structuring function with the origin in x. Some typical examples of structuring functions are linear (cone) function sx (y) = −d(x, y) and quadratic (parabolic) function sx (y) = −d(x, y)2 = −kx − yk2 . The corresponding reflected structuring function is defined as [19] sx (y) = sy (x), (3.4) x, y ∈ D. The erosion and the dilation by the structuring function sx are then defined as [95] ^ [εs (f )](x) = (f (y) − sx (y)) , x ∈ D, (3.5) y∈D [δs (f )](x) = _ (f (y) + sy (x)) , x ∈ D. (3.6) y∈D It should be noted that the flat case is given by expressing a structuring element S using a corresponding structuring function s defined as 0, y ∈ Sx , (3.7) sx (y) = −∞, y∈ / Sx , 35 which, inserted into (3.5) and (3.6), gives (3.2) and (3.3), respectively. In Figure 3.2, we present the erosion and the dilation using structuring elements depicted in Figure 3.1. The erosion shrinks light objects in the image, and expands black objects (see Figure 3.2 (b) and (c)). The dilation has the opposite effect (see Figure 3.2 (d), (e) and (f)). These two morphological operators satisfy several important properties. First, the erosion and the dilation are dual operators, i.e., εS (−f ) = −δS (f ), f ∈ L. (3.8) Both operators preserve ordering of the functions, i.e., the property called increasingness is satisfied f ≤g ⇒ εS (f ) ≤ εS (g) and δS (f ) ≤ δS (g) , f, g ∈ L. (3.9) Extensivity of the dilation and antiextensivity of the erosion are satisfied if the structuring element S contains the origin, i.e., εS (f ) ≤ f ≤ δS (f ), f ∈ L. (3.10) If the origin is not included in the structuring element S, the morphological operators can produce unintuitive result (see Figure 3.2 (c) and (f)). 3.2 Adjunction Property Two operators ε : L → L and δ : L → L form an adjunction (ε, δ) when δ(f ) ≤ g ⇔ f ≤ ε(g), f, g ∈ L. (3.11) The adjunction property (3.11) is one of the most fundamental notions in mathematical morphology. This property can be considered as the morphological counterpart of Galois connection [37]. Also, the adjunction property is closely related to the notion of complete lattices. Erosion is any operator ε : L → L that preserves the infimum, i.e., ! ^ ^ ε(Xi), Xi ∈ L, (3.12) Xi = ε i∈I i∈I where I is an index set. Dilation is any operator δ : L → L that preserves the supremum, i.e., ! _ _ δ(Xi ), Xi ∈ L. (3.13) Xi = δ i∈I 36 i∈I (a) Input f (b) Opening γS2 (f ) (c) Closing φS2 (f ) Figure 3.3. Opening and closing with the discrete disk of radius three as structuring element (S2 from Figure 3.1 (b)). It can be proven that any two operators ε and δ that form an adjunction (ε, δ) are an erosion and a dilation, respectively [83]. Moreover, for every dilation δ there exists one and only one erosion ε that forms an adjunction (ε, δ). Similarly, for every erosion ε there exists a unique dilation δ that forms an adjunction (ε, δ) [83]. Two different approaches exist to define an erosion and a dilation. In one approach (presented in Section 3.1), for one operator a structuring element must be replaced with its reflected structuring element (see (3.2) and (3.3)), while for the other approach this is not required, i.e., the same structuring elements is used to compute an erosion and a dilation. These approaches differ in the sense that the basic morphological operators, erosion and dilation, satisfy the adjunction property (3.11) when they are defined by the first one, which is not generally true for the later one. 3.3 Opening and Closing Combinations of the two basic morphological operations, erosion and dilation, with the same structuring element, lead to two other important morphological operators: opening and closing. Opening can be defined by γS = δS ◦ εS , (3.14) and closing by φS = εS ◦ δS . (3.15) Opening removes bright regions in which structuring element S does not fit, while closing removes dark regions in which S does not fit. Figure 3.3 (b) and (c)) depicts an example of opening and closing. Similarly to erosion and dilation, opening and closing are dual operators, i.e., γS (−f ) = −φS (f ), f ∈ L. (3.16) 37 Furthermore, they are idempotent γS ◦ γS = γS , (3.17) φS ◦ φS = φS , increasing, f ≤ g ⇒ γS (f ) ≤ γS (g) and φS (f ) ≤ φS (g) , f, g ∈ L, (3.18) and opening is antiextensive, i.e., γS (f ) ≤ f, f ∈ L, (3.19) f ∈ L. (3.20) and closing is extensive, i.e., f ≤ φS (f ), If the origin belong to the structuring element S then the following ordering relation is valid εS (f ) ≤ γS (f ) ≤ f ≤ φS (f ) ≤ δS (f ). (3.21) Morphological operators defined so far are translation invariant, that is, the two following operations give the same result: (1) first, a morphological operator is applied, and the image is then translated; (2) first, the input image is translated, and the morphological operator (same as in (1)) is applied. Mathematically, ψS ◦ Tt = Tt ◦ ψS , (3.22) where ψ is a morphological operator and Tt (C) = {x + t : x ∈ C}, for a set C. An operator that is increasing, idempotent, anti-extensive and can be defined as an erosion followed by a dilation (3.14) is often referred to as morphological opening. If an operator has the same properties, but cannot be written as a unique erosion followed by dilation, then this operators is called an algebraic opening. An algebraic opening can be written as the supremum of a family of morphological openings as [69] _ Γ(f ) = γSi (f ), (3.23) i∈I where {Si : i ∈ I} is a family of structuring elements. Some of algebraic openings are area opening [116], path opening [52], rank-max opening [101]. For instance, area opening removes all connected components with an area that is smaller than a certain number of points. 38 (a) Input f (b) Gradient GS2 (f ) Figure 3.4. Morphological gradient with the discrete disk of radius three as structuring element S2 . 3.4 Other morphological operators Erosion and dilation, can be combined in various ways to form powerful morphological operators. Apart from basic morphological operators, other morphological tools were used for a number of image analysis tasks. Here, we briefly summarize a few that will be used later in this thesis. One of the useful morphological operators, that will be used in Chapter 4.5, is the morphological gradient, defined by GS (f ) = δS (f ) − εS (f ), f ∈ L. (3.24) Similarly to the gradient magnitudes, the morphological gradient is an operator that indicates the strength of the edges in the image (see Figure 3.4). An operator ψ is a morphological filter if and only if ψ is increasing and idempotent. In other words, a filter preserves the order, and the structures that are preserved by the filter will not be modified by further use of the same filter. It is obvious that morphological closings and openings are morphological filters. The granulometry is an approach to compute a size distributions of grains, and it is obtained using a family of openings {γλ : λ ≥ 0} with increasing size of λ. This family of openings is a granulometry if the semi-group law (absorption property) is satisfied, i.e., γµ ◦ γλ = γmax{µ,λ} , (3.25) or, equivalently, γλ ≤ γµ if 0 ≤ µ ≤ λ. For instance, if we consider the Euclidean space, the family of openings {γλS : λ > 0}, with convex sets as structuring elements {λS : λ > 0} satisfies the semi-group law (3.25) [69]. Alternating sequential filters are morphological filters that are constructed by a series of openings and closings first with a small size of structuring elements and then with an increasing size until a certain 39 size is reached. The white alternating sequential filter that begins with an opening is defined by (φSn ◦ γSn ) ◦ (φSn−1 ◦ γSn−1 ) ◦ ... ◦ (φS1 ◦ γS1 ), (3.26) On the other hand, the black alternating sequential filter, which starts with a closing, is defined as (γSn ◦ φSn ) ◦ (γSn−1 ◦ φSn−1 ) ◦ ... ◦ (γS1 ◦ φS1 ). (3.27) Structuring elements should satisfy S1 ⊂ S2 ⊂ ... ⊂ Sn , for both alternating sequential filters. Morphological operators can be connected with distance functions, presented in Chapter 2. The Hausdorff distance (2.9) between two nonempty sets A and B can be written as dH (A, B) = min{r : A ⊂ δSr (B), B ⊂ δSr (A)}, (3.28) where Sr denotes a disk of radius r. Furthermore, it is shown by Sternberg [104] that the distance transform of a set A can be computed as the erosion of a function f defined as 0, x ∈ A, f (x) = (3.29) ∞, x∈ / A, with the cone structuring function sx (y) = −d(x, y). Moreover, the erosion of f with the parabolic structuring function sx (y) = −d(x, y)2 gives the squared distance transform [111]. 3.5 On the Selection of Structuring Elements Generally speaking, any shape and size can be considered for a structuring element, and a structuring element contains its origin or not (Figure 3.1 (c)). Morphological operators presented in this section are computed using one rigid structuring element for all points in the image. Those morphological operators and mathematical morphology we will refer to as classical in the rest of the thesis. The selection of structuring element depends on the application. For instance, circular structuring elements are well suited for images with circular objects, as well as for any case where isotropic operation is needed; and line segments are useful for images with a certain orientation. Also, morphological operators with larger structuring elements preserve only large features in the image, while those with smaller structuring elements preserve also the finer details in the image. Despite that morphological operators are a very useful tools in image analysis, morphological operators computed with one rigid structuring 40 element cannot differentiate shapes with respect to their contrast or orientation. For instance, classical morphological operators cannot distinguish same-size shapes that have different contrast. Hence, it is beneficial to use structuring elements that adapt their size and shape to the local features in the image such as contrast, luminance, the magnitude of the gradient, orientations in the image, or to the local curvature. Such structuring elements are called adaptive. 41 4. Adaptive Mathematical Morphology 4.1 Adaptivity in Mathematical Morphology In its original form, morphological operators are: (i) translation invariant (3.22); (ii) based on one rigid structuring element that is translated to every point in the image and does not depend on its position. Hence, two possible ways to generalize mathematical morphology were discussed by Roerdink [82]: (i) Translation invariance can be replaced by other types of invariance; this approach leads to group morphology. (ii) Rigid structuring elements can be replaced by adaptive structuring elements; this approach leads to adaptive mathematical morphology, also called adaptive morphology. The basic morphological operators, defined in the classical way (Chapters 3.1 and 3.3), are translation invariant. If other types of invariance are considered, then structuring elements change their size and shape with respect to different geometric transformations that generate an algebraic structure called a group. Therefore, the name of this field of mathematical morphology is called group morphology [81]. A comprehensive overview on group morphology is given by Heijmans [51]. In this thesis, we study adaptive mathematical morphology, which is a challenging topic that attracted a lot of attention in recent years [17, 19, 33, 60, 62, 72, 82]. Most likely, one of the earliest studies of adaptive mathematical morphology was presented by Serra [96]. Other early work was given by Charif-Chefchaouni and Schonfeld [29], who considered a general theory of adaptive morphology for binary images. More recently, comprehensive theoretical results of adaptive mathematical morphology were presented by Bouaynaya et al. [17], and Bouaynaya and Schonfeld [19], which were further explored by Roerdink [82]. Roerdink [82], among the other important aspects of adaptive mathematical morphology, addressed theoretical issues important for the development of the field by noticing often overlooked issues when defining adaptive morphological operators. This paper by Roerdink [82] can be seen as a cornerstone paper for further development of adaptive mathematical morphology, since it presents a way to properly define morphological operators with adaptive structuring elements, i.e., under which conditions the erosion and the dilation computed with adaptive structuring elements will form an adjunction. 43 In the following text, we will refer to morphological operators computed with adaptive structuring elements as adaptive morphological operators. For instance, the erosion computed with adaptive structuring elements will refer to as adaptive erosion. Adaptive structuring elements can be dependent on various image attributes such as spatial position in the image, image content, orientation in the image, etc. To distinguish different types of adaptive structuring elements, we follow the terminology introduced by Roerdink [82], who considered two categories of adaptiveness for structuring elements: location-adaptive structuring elements and input adaptive structuring elements. Location-adaptive structuring elements These structuring elements are dependent only on the position in the image domain D, and not on the input image f . In other words, these adaptive structuring elements are fixed a priori and do not depend on the image content. These adaptive structuring elements are also called extrinsic [33]. Most likely, one of the earliest examples of adaptive mathematical morphology was presented by Beucher et al. [10], who defined adaptive structuring elements that are dependent on the position in the image. Vehicles are analysed, and the ones at the bottom of the image appear larger than the ones at the top, therefore structuring elements vary linearly with their vertical position in the image. Furthermore, Verly and Delanoy [114] and Cuisenaire [32] designed adaptive structuring elements using the similar approach. If structuring elements are not computed from the input image, like the case with location-adaptive structuring elements, but rather following some law regarding the imaging device (e.g. if structuring elements are proportional to the distance to the imaging device), the adaptive morphological operators might not be translation invariant. In other words, the equation (3.22) is not valid in the general case, i.e., ψ ◦ Tt 6= Tt ◦ ψ, (4.1) where ψ is a morphological operator. Input-adaptive structuring elements These structuring elements are dependent on the image content and thereby the structuring elements are also dependent on the location in the image. These adaptive structuring elements are also called intrinsic [33], and they are usually computed from a smoothed version of 44 the input image f , called a pilot image [62]. Several interesting methods for input-adaptive structuring elements have been proposed including morphological amoebas [62], region growing structuring elements [72], general adaptive neighbourhoods [33], elliptical adaptive structuring elements [60], salience adaptive structuring elements (Paper IV), etc. As pointed by Roerdink [82], adaptive morphological operators are translation invariant if adaptive structuring elements are computed from the input (pilot) image. For instance, the results will be the same if we first translate both the input and the pilot image, and then compute the adaptive erosion or if we translate (using the same translation as above) the output of the adaptive erosion. It should be mentioned here that adaptive morphological operators, i.e., operators that do not process all points in the image identically, can also be defined using other tools. For instance, for morphological operators defined by partial differential equations, adaptivity can be incorporated directly into the partial differential equations underlying the basic morphological operators [22, 67]. More details about different approaches in adaptive mathematical morphology can be found in an overview paper presented by Maragos and Vachier [68]. In Paper VIII, we present the recent developments of adaptive mathematical morphology, including latest methods for adaptive structuring elements as well as proper definitions of adaptive morphological operators. We also present an application-oriented study of different methods for adaptive structuring elements, and briefly discuss possible future directions in which this field might further develop (Paper VIII). 4.2 Adjunction Property in Adaptive Mathematical Morphology To obtain an adjunction (ε, δ) in the classical mathematical morphology, given by (3.11), for the erosion (3.2) and the dilation (3.3) a structuring element S used for the erosion is replaced by its reflected structuring element S ∗ for the dilation (Chapter 3.2). On the other hand, this construction can be problematic in the context of adaptive morphology and it is often overlooked in the literature, as noted by Roerdink [82]. Let f : D ⊂ Rn → R be an input image and f 0 : D ⊂ Rn → R be a pilot image, which is obtained by smoothing the input image f . Let {Sf 0 [x] : x ∈ D} be a set of adaptive structuring elements computed from a pilot image f 0 . To each point x ∈ D is associated a unique structuring element Sf 0 [x]. The adaptive structuring elements derived 45 for different points may be independent one from another, i.e., they may contain neighbouring points or not. The adaptive morphological erosion ε : L → L and the corresponding dilation δ : L → L, are defined as [82] ^ f (y), x ∈ D, (4.2) [ε(f )](x) = y∈Sf 0 [x] [δ(f )](x) = _ f (y), x ∈ D, (4.3) y∈S ∗0 [x] f where Sf∗0 [x] is the reflected structuring element of Sf 0 [x] defined as y ∈ Sf 0 [x] ⇔ x ∈ Sf∗0 [y]. (4.4) First of all, note that if Sf 0 [x] = S for all x ∈ D, i.e., if one considers the same rigid structuring element for all points in the image, the erosion (4.2) and the dilation (4.3) lead to the classical erosion (3.2) and the classical dilation (3.3). Hence, it is obvious that the adaptive mathematical morphology is a generalization of the classical mathematical morphology. Adaptive morphological operators ε and δ, defined by (4.2) and (4.3) will form an adjunction (ε, δ), if the structuring elements {Sf 0 [x] : x ∈ D} are computed once from the pilot image f 0 , and then used to compute the adaptive erosion ε and the adaptive dilation δ. Here, we give a proof that ε and δ are adjunct morphological operators [82]: δ(f ) ≤ g ⇔ [δ(f )](x) ≤ g(x), x ∈ D _ (equation (4.3)) ⇔ f (y) ≤ g(x), x ∈ D y∈S ∗0 [x] f ⇔ f (y) ≤ g(x), y ∈ Sf∗0 [x], x ∈ D (equation (4.4)) ⇔ f (y) ≤ g(x), ^ ⇔ f (y) ≤ x ∈ Sf 0 [y], y ∈ D g(x), y∈D x∈Sf 0 [y] (equation (4.2)) ⇔ f (y) ≤ [ε(g)](y), ⇔ f ≤ ε(g). y∈D Since the adaptive erosion ε and the adaptive dilation δ, given by (4.2) and (4.3), respectively, form an adjunction (ε, δ), the products ε◦δ and δ ◦ ε will lead to operators that satisfy the algebraic properties of the closing and opening, respectively. Furthermore, it is now possible to properly define alternative sequential filters, and other morphological filters. 46 Figure 4.1. Adaptive structuring elements derived for points x, y and z. Since x ∈ Sf 0 [y] then y ∈ Sf∗0 [x]; and since x ∈ / Sf 0 [z] then z ∈ / Sf∗0 [x]. In other words, if we first compute the erosion with adaptive structuring elements derived from the pilot image, then derive new adaptive structuring elements from the eroded image and use them to compute the dilation, the resulted operator will not necessarily be an opening, i.e., not necessarily satisfy the algebraic properties of opening (3.17)– (3.19). Once adaptive structuring elements {Sf 0 [x] : x ∈ D} are derived from the pilot image f 0 , it is not necessary to compute their reflected structuring elements {Sf∗0 [x] : x ∈ D} in order to compute the adaptive erosion and its adjoint dilation. For instance, the adaptive erosion ε assigns to the centre x the infimum of f over the structuring element Sf 0 [x]. This process may be called centripetal, as it brings to the centre x all gray-level values of their neighbours in Sf 0 [x]. We observe that a point x is a part of the reflected structuring element Sf∗0 [y] for any point y within the structuring element Sf 0 [x], if x ∈ Sf 0 [y] (see Figure 4.1). Then, the adaptive dilation can be computed as follows: The value f (x) is distributed to all points y ∈ Sf 0 [x] and contributes to their value of the dilation. In this process, the point y receives the contribution of all centres x of all structuring elements Sf 0 [x] which contain y. This process may be called centrifugal, as it distributes the value of the central pixels to their neighbours. Note that / Sf∗0 [x], and f (z) do not contribute to the dilation if x ∈ / Sf 0 [z] then z ∈ in a point x (see Figure 4.1). The adaptive erosion ε and adaptive dilation δ can be computed as follows [62]: Erosion: [ε(f )](x) = +∞, ∀x ∈ D for each point x ∈ D do compute Sf 0 [x] for each point y ∈ Sf 0 [x] do [ε(f )](x) = min{f (y), [ε(f )](x)} 47 end for end for Dilation: [δ(f )](x) = −∞, ∀x ∈ D for each point x ∈ D do for each y ∈ Sf 0 [x] do [δ(f )](y) = max{f (x), [δ(f )](y)} end for end for Let us now consider the case when adaptive structuring functions (non-flat structuring elements) are derived from the pilot image f 0 , here denoted with {sxf0 : x ∈ D}. In the theory provided by Bouaynaya and Schofield [18, 19], non-adaptive morphological operators based on rigid structuring functions (Chapter 3) are directly generalized to the adaptive case. Hence, the erosion and dilation with adaptive structuring functions are defined, respectively, as ^ (4.5) [ε(f )](x) = (f (y) − sxf0 (y)), x ∈ D, y∈D [δ(f )](x) = _ (f (y) + syf 0 (x)), (4.6) x ∈ D. y∈D Similarly to the case with adaptive structuring elements one has to fix adaptive structuring functions once when they are computed from the input (pilot) image. The erosion ε given by (4.5) and the dilation δ given by (4.6) form an adjunction (ε, δ). The proof for this claim is given by Bouaynaya and Schofield [19]: δ(f ) ≤ g ⇔ [δ(f )](x) ≤ g(x), x ∈ D _ (f (y) + syf 0 (x)) ≤ g(x), (equation (4.6)) ⇔ x∈D y∈D ⇔ f (y) + syf 0 (x) ≤ g(x), x, y ∈ D syf 0 (x), sxf0 (y), x, y ∈ D ⇔ f (y) ≤ g(x) − y, x ∈ D (x = y) ⇔ f (x) ≤ g(y) − ^ (equation (4.5)) ⇔ f (x) ≤ (g(y) − sxf0 (y)), x ∈ D y∈D ⇔ f (x) ≤ [ε(g)](x), ⇔ f ≤ ε(g). 48 x∈D 4.3 Methods for Input-Adaptive Structuring Elements and Functions In this section, we briefly present a number of different methods for adaptive structuring elements and adaptive structuring functions that exist in the literature. For more details, see provided reference and Paper VIII. In general, adaptive structuring elements are defined using different features in the image, including gray-level image values, edges, gradients, the Hessian, orientation and connectivity between points. Hence, adaptive structuring elements often rely on a local similarity between neighbouring points or are aligned to edges and contours in the image. Apart from being defined using different attributes in the image, various constraints can be imposed to the shape and size of structuring elements, ranging from being adaptivity to the image content to being constrained by a set of predefined shapes. In this last case, the size and orientation of these structuring elements are adapted to the image. In Paper VIII, we distinguish the two main groups of methods for the construction of adaptive structuring elements by being based on different features in the image: (i) local similarity; (ii) local structure. The methods that belong to the first group are based on a local similarity between the origin of the structuring element and its neighbouring points. A structuring element includes points that are similar to the origin according to some measure of similarity between points. For instance, a structuring element in a point x is defined as a connected component that contains points with similar gray-level values to f (x) [33]. These adaptive structuring elements are called general adaptive neighbourhoods [35]. Structuring elements are also defined as balls in a geodesic metric space (often called geodesic balls), where a geodesic distance can be defined using different image attributes. For example, morphological amoebas are computed using a geodesic distance that takes spatial distance and gradient into account [62], while Grazzini and Soille [45] utilized the same attributes with different geodesic distances. In the same line, we use a geodesic distance to define salience adaptive structuring elements, which are derived from the salience map based on the edge strength (Paper IV). Furthermore, region growing structuring elements include points with a region growing technique that is based on the difference between gray-level values between the adjacent points [72]. Structure-based methods belong to the second group, where adaptive structuring elements are aligned to the structure in the image, such as edges and contours. These methods are characterized by considering orientation in the image, distances to the neighbouring edges and rate of anisotropy. For instance, Tankyevych et al. [108] proposed line structuring elements where their orientations are obtained from a func49 tion that depends on the Hessian. Verdú-Monedero et al. [113] defined adaptive structuring elements (lines and rectangles) that are constrained in width by the distance to the nearby edges, where orientation of structuring elements is obtained using diffusive squared gradient fields. Furthermore, adaptive elliptical structuring elements are derived using the structure tensor, where their shape vary from disks to lines depending on the rate of anisotropy [60]. Adaptive structuring elements can differ in shape and size. For instance, shapes of some adaptive structuring elements are completely adaptive to the content of the image (still dependent on the input parameter that determines the size), such as morphological amoebas [62] and general adaptive neighbourhoods [35], while certain constraints are assigned for other methods [60, 113]. In addition, adaptive structuring elements can be defined to have one shape with adaptive sizes determined by the image content as proposed by Dokládal and Dokládalová [40], which used the distance transform to determine the size of rectangles. Similarly, we use the salience distance transform to determine the size of adaptive structuring elements, where any shape can be used for structuring elements (Paper V). Adaptive structuring functions are not well explored as adaptive structuring elements, and there exists only a few methods for the construction of adaptive structuring functions. Non-flat adaptive structuring elements, i.e., adaptive structuring functions are mostly inspired by the well-known filtering methods, by considering similarity between points or between image patches. In this line, spatially variant bilateral structuring functions [2] are related to bilateral filtering [110], while nonlocal structuring functions [92, 112] are directly related to non-local means filtering method [25]. Recently, Angulo and Velasco-Forero [5], recently proposed adaptive structuring functions that are based on random walks, where the step from one point to the other depends on the distance between points, and different distance functions were considered for this task. In paper VI, we propose parabolic structuring functions based on the salience map of the edge strength. 4.4 Salience-based Adaptive Mathematical Morphology Our methods for constructing adaptive structuring elements (Paper IV and Paper V) as well as for adaptive structuring functions (Paper VI) are based on the salience map, denoted here with SM, which is a result of the salience distance transform (Algorithm 2, Chapter 2.4) applied to the edge strength of the input image. 50 (a) NMS(f ) (b) SDT(f ) (c) SDT+ (f ) (d) SM(f ) Figure 4.2. Computation of the salience map SM(f ) for a one dimensional function f (see text). To preserve most of the edges in the image, we use the gradient estimation and non-maximal suppression from the Canny edge detector [27]. In this process, we use Gaussian derivatives to estimate the gradient in the input image, but exclude the hysteresis thresholding from the Canny edge detector. This approach preserves even the edges with a small response in the gradient image. Formally, NMS(f ) is the image obtained by computing the gradient magnitude and non-maximal suppression of the input image f . The edge pixels are initialized with the negative values of their salience and the non-edge pixels are set to infinity [88]. The salience distance transform SDT(f ) is computed with the classical two-pass chamfering algorithm [15] (Figure 4.2 (b)). After the salience distance transform is propagated from − NMS(f ), the distance image is offset to all positive values and the map SDT+ (f ) is obtained (Figure 4.2 (c)). By inverting the values of SDT+ (f ), we obtain the salience map SM(f ) (Figure 4.2 (d)), which can be formally written as _ [SM(f )](y) = Offset + NMS(f )(x) − d(x, y) , y ∈ D, (4.7) x∈D where Offset = ^ _ y∈D x∈D NMS(f )(x) − d(x, y) , (4.8) and d(x, y) is a spatial distance. For instance, the example of the salience map SM(f ) for “Le fifre” image is shown in Figure 4.3. 51 (a) Input f (b) NMS(f ) (c) SDT+ (f ) (d) SM(f ) Figure 4.3. Steps for computing the salience map SM for the “Le fifre” image. The salience map SM contains the information about the spatial distance between points in the image and preserves the information about the salience of the edges in the image, where the largest values in the salience map SM correspond to the strongest edges in the input image. This salience map can be computed in linear time with respect to the number of pixels in the image N , i.e., O(N ), since the salience distance transform can be computed in linear time. Our main inspiration to use the salience map SM for constructing adaptive structuring elements comes from the approach used for morphological amoebas [62]. Morphological amoebas are geodesic balls in a geodesic (gray-weighted) distance metric space, where the distance between two adjacent points xi and xi+1 is defined as ca (xi , xi+1 ) = 1 + σ|f (xi ) − f (xi+1 )|, (4.9) where σ > 0 is a constant used to scale the two incommensurate domains. For morphological amoebas, the following definition was also used [117] ca (xi , xi+1 ) = d(xi , xi+1 ) + σ|f (xi ) − f (xi+1 )|, (4.10) where d(xi , xi+1 ) is the Euclidean distance or a weighted distance, i.e., h3, 4i distance [15]. Then, a morphological amoeba Ar (x) centred in a 52 (a) (b) Figure 4.4. Salience adaptive structuring elements for one dimensional function f . (a) Salience adaptive structuring elements for points x1 and x2 , with the same radius r; (b) Salience adaptive structuring elements for points x1 and x2 , with with adaptive radii, where r1 < r2 . point x is defined as Ar (x) = {y ∈ D : min ca (P(x, y)) < r}, P(x,y) (4.11) where the cost of the path P(x, y) is computed by using formulas (4.9) or (4.10), and r is the parameter that determines the size. Morphological amoebas adapt well to the content in the image, i.e., they are flexible in shape and strictly align to strong edges in the image (see Paper VIII, Figure 2). Furthermore, morphological amoebas stretch over the points with similar gray-level values rather then points in the neighbourhood, and structuring elements for neighbouring points might have completely different shapes. Our intention is to construct adaptive structuring elements such that structuring elements for two neighbouring points have similar shapes and not completely different ones. For this purpose, we use the salience map SM(f ) that is a continuous map, and SM(f ) gradually differs for neighbouring points. Here, we briefly present our methods for adaptive structuring elements and adaptive structuring functions that are based on the salience map described above. More details about the methods can be found in Paper IV, Paper V and Paper VI of this thesis. In Paper IV, we define adaptive structuring elements, called the salience adaptive structuring elements, that are dependent on the path-based distances computed on the salience map SM. We define the cost of the path between two adjacent points xi and xi+1 as cs (xi , xi+1 ) = [SM(f )](xi ) + [SM(f )](xi+1), (4.12) 53 (a) (b) (c) (d) Figure 4.5. Morphological operators with salience adaptive structuring elements, (a) Input image f ; (b) Dilation for r(x) = 5 · max(SM(f )) − [SM(f )](x), x ∈ D; (c) Erosion for r(x) = 5 · max(SM(f )) − [SM(f )](x), x ∈ D; (d) Erosion for r(x) = 9 · max(SM(f )) − [SM(f )](x), x ∈ D. and the salience adaptive structuring element centered in a point x is defined as Sr (x) = {y ∈ D : min cs (P(x, y)) < r}. (4.13) P(x,y) Our main idea is that structuring elements that are close to salient edges should be smaller in size. Therefore, structuring elements located close to edges in the input image are smaller in size, while structuring elements in more homogeneous areas of the input image are larger (see Figure 4.4 (a)). We also define that the radii of the salience adaptive structuring elements is adaptive and also dependent on the salience map SM. This is opposite to the other methods for adaptive structuring elements that use a fixed radius for each point in the image [45, 62] (or tolerance [33]). For example, we used r(x) = k · max(SM(f )) − [SM(f )](x), x ∈ D, to define adaptive radii, where k > 1 is the input parameter. Two salience adaptive structuring elements with adaptive radii are depicted in Figure 4.4 (b). 54 (a) (b) (c) Figure 4.6. Morphological operators with salience-based rhomboidal (diamond) shapes as adaptive structuring elements. (a) Input image; (b) Erosion for preserving weak edges; (c) Dilation for preserving weak edges. We present a comparison between morphological amoebas and salience adaptive structuring elements in Paper IV. An application-oriented study on different methods for adaptive structuring elements, including morphological amoebas [62], general adaptive neighbourhoods [35], adaptive elliptical structuring elements [60], and salience adaptive structuring elements (Paper IV), is presented in Paper VIII. In Paper VIII, among the other experiments, we examine shapes of adaptive structuring elements of the four aforementioned methods in some typical situations in the image as well as their behaviour when noise is added to the image (see Paper VIII, Figure 2). As opposed to the traditional morphological operators and the ones compared to in Paper VIII, morphological operators with salience adaptive structuring elements process same size objects differently depending on their contrast. For example, the adaptive erosion shrinks dark objects more than light ones, as shown in Figure 4.5 (c) and (d). The computation cost of the salience adaptive structuring elements is relatively high O(N r 2 log r 2 ), where N is the number of pixels in the image and r is the radius of salience adaptive structuring elements. This is due to calculation of path-based distances for every point in the image. Therefore, in Paper V, we propose adaptive structuring elements with a predefined shape, and for which the size is adjusted by the salience map SM. These structuring elements can be computed in linear time with respect to the number of pixels in the image. Our method is based on the geometric relations between the spatial position of the points and their respective values in the salience map. More precisely, we use local minima and maxima in the salience map SM to propose two types of adaptive structuring elements. One type of adaptive structuring elements preserves strong edges better than weak ones, while the other type better preserves weak edges in the image. More details about this method can be found in Paper V, Section 3. For instance, the results of 55 (a) (b) Figure 4.7. Salience-based parabolic structuring functions for 1D function. (a) Two structuring functions centered at edge points. (b) Two structuring functions where one is centered at the edge and one is close to the edge. morphological operators with these structuring elements are depicted in Figure 4.6. In the two methods for adaptive structuring elements that are briefly described above (more details can be found in Paper IV and Paper V), the size of structuring elements decreases as the salience map SM increases and vice versa. In paper VI, we propose salience-based parabolic structuring functions that are larger where the salience map SM is smaller, i.e., structuring functions are larger in points of weak edges than in points with strong edges (Figure 4.7 (a)). Formally, we define a family of salience-based parabolic structuring functions centered at point x ∈ D as sx (y) = − α([SM(f )](x), [SM(f )](y)) + β[SM(f )](x)kx − yk2 , y ∈ D, (4.14) where α([SM(f )](x), [SM(f )](y)) ≥ 0 is a monotonically non-decreasing function, β ≥ 0 is a constant. In addition, these structuring functions are not symmetric and they decrease faster at the side closer to the higher salience (Figure 4.7 (b)). In other words, the parabola sx3 (y) centered in point x3 is skewed away from the edges in the input image f. 4.5 Applications Classical mathematical morphology has been used to solve various problems in image analysis, such as image filtering and image segmentation. A detailed study of various real world applications of mathematical morphology can be found in the book edited by Najman and Talbot [74]. 56 (a) (b) (c) (d) Figure 4.8. Isolation of the text in a historical document. (a) Color image of a historical document; (b) Inverted gray-level image f ; (c) Adaptive erosion ε(f ); (d) Morphological gradient G(f ), i.e., the difference between adaptive dilation δ(f ) and adaptive erosion ε(f ). Adaptive mathematical morphology, similarly to the classical one, found its use in certain tasks, especially for detecting and enhancing elongated structures in the image. For instance, Morard et al. [72] used their region growing structuring elements to detect cracks. For the same application, adaptive elliptical structuring elements were used by Landström and Thurley [60]. Similarly, adaptive dilation with structuring elements derived from distance functions based on the structure tensor were used for closing contours and curves [66]. Furthermore, Tankyevych et al. [108] used their adaptive morphological filters based on the Hessian for enhancing vessels in medical images. The usefulness of the general adaptive neighbourhoods was shown for noise reduction and image segmentation [34]. Moreover, Gonzales et al. [44] used adaptive morphological operators for pixel classification. The descriptor of each pixel is formed by a concatenation of the original image and successive adaptive erosions and dilations of increasing size. In Paper V, we also present the usefulness of salience-based linear adaptive structuring elements for problems of text recognition. Morphological operators with these structuring elements are used to detect 57 the text in a historical document, where the text on the back side of the paper is highly visible on the front side (see Figure 4.8 (a) and (b)). We construct adaptive structuring elements that preserve weak edges (see Paper V, Section 3 for the more details). These structuring elements are larger where the text is, and smaller in the background. Therefore, adaptive morphological operators will affect mostly letters on the front page, (see Figure 4.8 (c) for the adaptive erosion). Note that, for this application, we used a scaled salience map SM(f ) that is obtained by taking NMS(f )/5 of the input image, which reduces the impact of strong edges. Image Regularization A mathematical problem is well-posed (in the sense of Hadamard) if the three following properties are satisfied [94]: (i) The solution exists. (ii) The solution is unique. (iii) The solution is stable, in the sense that small perturbations in the equation only lead to small perturbations in the solution. Otherwise, if one of the requirements (i)-(iii) is not satisfied the problem is ill-posed. Typical examples for ill-posed problems are inverse problems. In general, inverse problems refer to problems used to reconstruct data that are corrupted or processed. The inverse mapping is usually not accessible for real world problems, and inverse problems require regularization, which refers to a process of introducing additional information in order to solve an ill-posed problem. In image analysis, inverse problems can be formulated by f = Ag + b, (4.15) where A ∈ Rn×n is a linear operator, f ∈ Rn is the input image and g ∈ Rn is the desirable (reconstructed) image and b ∈ Rn is noise. One way to find an optimal approximation of g is to consider the following regularization problem 1 2 ∗ kf − Agk + λJ (g) , (4.16) g = arg min 2 g∈Rn where J (g) is a regularization term and λ is a parameter used for balancing between the data and the regularization term. The regularization term J (g) usually contains information about the edges in the image, and it depends on the image derivatives. For instance, the Tikhonov regularization term is defined by L2 norm, i.e., J (g) = kgk, while the 58 total variation regularization is characterized by the gradient [89] J (g) = n X |∇g(xi )|. (4.17) i=1 Several regularization techniques utilize morphological operators in the regularization term, and this regularization is referred to as morphological regularization. For instance, Purkait and Chanda [77] proposed multi-scaled morphological regularization defined as J (g) = N X (φkB (g) − γkB (g)), (4.18) k=1 where φkB and γkB are morphological closing and opening with disks of increasing size k as structuring elements. The same authors recently presented a morphological regularization as [78] J (g) = φ(g) − γ(g), (4.19) where φ and γ are adaptive morphological closing and opening, respectively. In Paper VII, we propose a new framework for morphological image regularization using adaptive structuring functions. Our approach is based on the Lasry–Lions approximations that rely on infimal-supremal convolution formulas, and provide Lipschitz regularization of non-smooth functions [61]. The approximations belong to the class of functions C 1,1 , which means that the approximations are continuously differentiable with Lipschitz continuous gradient (see Appendix for more definitions). Let f : D ⊂ Rn → R ∪ {+∞} be a lower semi-continuous, convex and bounded function from below. Lasry and Lions [61] defined the two following approximations of a function f as _ ^ 1 1 µ 2 2 f λ (x) = f (y) + d(z, y) − d(z, x) , x ∈ D, (4.20) 2λ 2µ z∈D y∈D and µ f λ (x) = ^ _ z∈D y∈D 1 1 2 2 f (y) − d(z, y) + d(z, x) , x ∈ D. 2λ 2µ (4.21) Following definitions (4.20) and (4.21), we have µ f µλ ≤ f ≤ f λ . (4.22) µ Moreover, the approximations f µλ , f λ ∈ C 1,1 , and they converge pointwise to f , when 0 < µ < λ and λ → 0, µ → 0. 59 Note that, if λ = µ then approximations (4.20) and (4.21) correspond to the morphological opening and closing of a function f with the quadratic structuring function sx (y) = −d(x, y)2 , respectively. Since the requirement for the Lipschitz regularization is 0 < µ < λ, then we will refer to the approximations (4.20) and (4.21) as pseudo-opening and pseudo-closing. Therefore, the above discussion implies that the Lasry– Lions regularization of non-smooth functions is a pseudo-morphological operator of the initial function using quadratic structuring elements. This type of regularization is further extended to non-convex functions, and this work is presented by Attouch and Azé [7]. They presented an equivalence of Lasry–Lions approximations without imposing the convexity constraint on a function f nor the boundedness. Instead, they assume that the function is quadratically minorized, i.e., there exists c ≥ 0 such that c f (x) ≥ − (1 + kxk2 ), x ∈ D. 2 (4.23) Then, for all 0 < µ < λ < 1/c the function f µλ ∈ C 1,1 and f µλ → f converges to f , when µ → 0 and λ → 0. Similar statement is valid for µ the approximation f λ [7]. In the above discussion, all approximations are given for the fixed quadratic structuring functions sx (y) = −d(x, y)2 , x, y ∈ D. It is also possible to use structuring functions that are different from the quadratic ones, and structuring functions that depend on the position in the domain D [61]. Furthermore, Cepedello-Boiso [28] developed this theory for functions in Banach spaces and suggested that structuring functions with the following properties can be used to obtain the Lipschitz regularization: (i) sx (x) = 0, (ii) sx (y) = sy (x), (iii) sx (y) is Lipschitz continuous on bounded sets. (iv) sx (y) → −∞, when y → ∞. Note that there is an indefinite number of possible combinations how to define the structuring function and its finite support. In mathematical morphology, parabolic structuring functions have been used for several reasons. For instance, morphology with parabolic structuring functions is the counterpart of the Gaussian convolution kernel, because it is the only structuring element that is rotationally invariant and separable [111]. In Paper VII, we define adaptive structuring functions as sx (y) = −d(x, y)p(x) , x ∈ D, (4.24) where the slope p(x) > 1 depends on the image content. These structuring functions satisfy the above properties (i)-(iv). We define p(x) 60 (a) (b) (c) (d) Figure 4.9. Morphological regularization using a (pseudo openclose + pseudo closeopen)/2 filter using morphological amoeba as the finite support for structuring functions, and regularization parameters λ = 1, µ = 0.5. (a) Original image; (b) Image corrupted by 20% random impulse noise; (c) Regularization using quadratic structuring function for all points in the image. (d) Regularization using adaptive structuring functions. as an increasing function of the salience map SM(f ), i.e., the slope of structuring functions is higher in points with larger salience. In Figure 4.9, we illustrate the described method using the following function to determine the slope of adaptive structuring functions p(x) = 1 + 3 [SM(f )](x) . max(SM(f )) (4.25) For the calculations, we use adaptive finite support for structuring functions, computed by morphological amoebas. The regularization with adaptive structuring functions produces more smoothing than the corresponding regularization using quadratic structuring functions for every point in the image. Moreover, finite support of structuring functions has significant influence on the morphological regularization presented here (Paper VII). 61 5. Conclusions and Future Work This chapter concludes the thesis and presents some directions for future work. 5.1 Contributions In this thesis, we have investigated distance functions and their usefulness for several problems within image analysis. The major contributions presented in this thesis are: • We have proposed a new distance function between sets of points, called the Complement weighted sum of minimal distances, and investigated its properties in Zn . The applicability of this distance function has been presented to image registration. In addition to extensive evaluation with synthetic images, we have shown the usefulness of this distance function for several real world problems (Paper I and Paper II). • We have proposed two different ways of extending the Complement weighted sum of minimal distances to fuzzy sets. We have shown that these distance functions can be a useful tool for classifying digital fuzzy objects (Paper III). • We have proposed adaptive structuring elements based on the salience map computed from the strength of the edges in the image. We have introduced two different types of structuring elements: (i) The shape of a structuring element is predefined, and its size is determined from the salience map (Paper V). (ii) Both the shape and size of a structuring element are dependent on the salience map (Paper IV). • We have proposed parabolic adaptive structuring functions that are based on the salience map (Paper VI). This method is a generalization of the salience adaptive structuring elements proposed in Paper IV. • We have proposed a new framework for morphological image regularization using adaptive mathematical morphology (Paper VII). The methods proposed in Paper IV, Paper V and Paper VI can be used for this type of image regularization. 63 • We presented an overview of adaptive mathematical morphology, which will be published in a journal special issue on mathematical morphology. There, we have presented an application-oriented study of various methods for adaptive structuring elements (Paper VIII). • We have presented a unified framework for proper definition of adaptive morphological operators, such that adaptive morphological erosion and dilation form an adjunction (Paper VIII). In summary, we have proposed tools that can be used for different problems in image analysis. The distance functions, we have proposed, can be used for different tasks related to comparing various shapes in images. In addition, we have proposed salience-based mathematical morphology that can be used to distinguish and modify shapes with respect to their contrast and not just size. 5.2 Future Work The methods presented in this thesis can be further improved. Some of the possible future research topics related to the work presented in this thesis are presented here. Set distances Like other distance functions between sets of points, the Complement weighted sum of minimal distances can be used for various applications. For instance, it can be used for evaluating different segmentation methods, which is a problem that often appears in image analysis. Some initial results clearly show this potential, and this distance function might be a powerful tool for other image analysis problems that include a comparison of shapes. The Gromov–Hausdorff distance has been used for shape comparison [24, 71], and this distance function measures how similar shapes are, where shapes are considered as metric spaces. Similarly to the Hausdorff distance, the Complement weighted sum of minimal distances can be extended to this framework. We have shown that using more membership levels of fuzzy sets does not necessarily lead to improving the performance of distance functions. Hence, it would be valuable to determine how the number of membership levels of fuzzy sets influences a particular distance function. Salience adaptive structuring elements We have only used the magnitude of the edges in the image to compute the salience map SM thus far. It would be interesting to construct 64 similar salience maps using other attributes of the edges such as curvature or length. Moreover, apart from the salience map computed by the salience distance transform, other salience maps could be considered when constructing adaptive structuring elements. The salience map SM can be modified by scaling attributes of the edges in the image. It would be valuable to further study this salience map, especially when considering a specific application. An extension of the salience distance transform has been recently presented by Lagerstrom and Buckley [59]. Their method is refer to as attribute weighted distance transform, and it is dependent on different image attributes and not only edges. It might be valuable to compute adaptive structuring elements from the salience map that is computed using an attribute weighted distance transform. An extension of mathematical morphology to color images is a challenging task, since it depends on ordering vectors [1, 6]. It would be interesting to further extend the salience adaptive structuring elements to color images. Some initial results (not yet published) show that adaptive structuring elements computed from a salience map for color images might have better properties than other similar methods of adaptive structuring elements for color images [36, 62]. There might be a connection between morphological operators with salience adaptive structuring elements and the mean curvature equation. This should be explored. Adaptive mathematical morphology Adaptive morphological operators are not well-known to the wider image analysis community. This might be due to a relatively high computational cost required to compute adaptive structuring elements. Hence, efficient algorithms for computing such structuring elements are required. Some of our initial results show that using GPU implementations, at least for non-local structuring elements, lead to a significant speed-up [48]. The theory of classical morphological operators is very rich. Possibly, most of the theory remains valid when using adaptive structuring elements. For instance, it would be interesting to consider definitions of scale-spaces and a granulometry with adaptive structuring elements as well as other morphological operators. Morphological regularization based on Lasry–Lions approximations is a very elegant method for image regularization. Our initial study presented in Paper VII gives a framework for morphological image regularization using adaptive mathematical morphology. Further studies will focus on finding appropriate adaptive structuring functions for particular applications. 65 Most of the methods for adaptive structuring elements can be considered as special cases of the recently introduced mathematical morphology on Riemannian manifolds [4]. This approach seems to be a prominent future research direction when deriving adaptive structuring elements. In addition, it might be useful to define adaptive mathematical morphology for other image representations, such as orientation scores [43] and Poincaré upper-half plane [3], two frameworks for which classical morphological operators are already defined. The current approach in adaptive mathematical morphology is to define adaptive structuring elements for all points in the image. Even though this seems a natural way to define adaptive structuring elements, not all points in the image are equally important and most of the points in the image could be processed with the same rigid structuring element. Therefore, it seems reasonable to derive adaptive structuring element only for some specific points in the image, possibly the points that are more important according to a predefined criterion. 5.3 Concluding Remarks and Perspectives Because many various distance functions exist in the literature, it is not always easy to select an appropriate distance function for a particular application. Furthermore, using existing distance functions might not always be the best option, when considering a particular problem, and even small modifications of the existing distance functions can significantly improve the performance of the distance. Therefore, it might be better to adapt existing distance functions to a particular problem or propose new distance functions than use the old existing ones. Mathematical morphology found its use in a number of different applications including image filtering and segmentation. Although the need for adaptive mathematical morphology is obvious, its future probably depends on the applicability of adaptive morphological operators to real world problems. 66 6. Brief Summaries of the Included Papers This chapter presents a brief summaries of the papers included in this thesis. Paper I: Distance functions between sets of points are analysed, and their applicability to image registration problems is studied. The notion of the complement distance, as a distance function between complements of two sets, is introduced. An empirical evaluation of monotonicity of distance functions with respect to translations and rotations of 2D binary objects for different noise conditions is given. It is concluded that distance functions based on the contribution of all points from both sets have monotonic behaviour more frequent than Hausdorff-like distance functions, which are based on the maximum of point-to-set distance values. Paper II A novel distance function between sets of points, called the Complement weighted sum of minimal distances, is proposed. For this distance function, the contribution of each point is weighted according to the position within the set that it belongs to. An extensive study on the usefulness of the new distance function for the task of shape registration is performed. Different distance functions between sets of points are compared, and it is concluded that the Complement weighted sum of minimal distances has the best performance overall. In addition, its applicability to two real problems is given: for the recognition of handwritten text characters and for the multi-modal 2D-3D image registration of bone implants in the surrounding tissues. Paper III The Complement weighted sum of minimal distances is extended to the case of fuzzy sets, using two different approaches. One extension is based on the α−cut decomposition of fuzzy sets, and the second approach is based on the point-to-set distance for fuzzy sets. The monotonicity property of the two novel distance functions with respect to 67 increasing translations and rotations of digital fuzzy objects is analysed, as well as the usefulness of those distance functions for the recognition of different crisp and fuzzy digital objects. It is shown that the use of the proposed distance functions between fuzzy sets for classification of fuzzy objects leads to improved performance, compared to a corresponding classification of binary objects at the same resolution using corresponding distance functions between binary objects. Paper IV A method for the construction of salience adaptive structuring elements that locally adjust their shape and size according to the salience of the edges in the image is proposed. Salience adaptive structuring elements adapt differently to different contrasts of same-size objects, and adaptive morphological operators process these objects differently. The comparison with morphological amoebas is given, and it is concluded that the salience adaptive structuring elements have a more compact shape, and are less sensitive to noise than the morphological amoebas. Paper V The computational cost for calculating adaptive structuring elements is relatively high, and often higher than linear with respect to the number of pixels in the image. Therefore, structuring elements with predefined shape and adaptive size are proposed. The size of adaptive structuring elements is determined by the salience of the edges in the image, and they can be computed in linear time. A visual comparison between the new adaptive structuring elements and morphological amoebas is shown. The applicability of the new adaptive morphological operators is demonstrated for isolating text in a historical document. Paper VI Parabolic structuring functions based on the salience of the edges in the image are introduced. They are not necessarily symmetric with respect to the origin, and these adaptive structuring functions can be considered as a generalization of the salience adaptive structuring elements introduced in Paper IV. Furthermore, morphological operators with adaptive flat structuring elements obtained by thresholding the salience-based parabolic structuring functions are also presented. In addition, the proper way of computing adjunct morphological operators is discussed. 68 Paper VII The applicability of adaptive mathematical morphology to the task of image regularization is studied. The underlying theory is based on Lasry–Lions regularization of non-smooth functions. The Lasry–Lions regularization is Lipschitz continuous, and it corresponds to morphological or pseudo-morphological operators with adaptive structuring functions. An empirical evaluation on using different regularization parameters is presented and different adaptive structuring functions are considered. It is concluded that the finite support of structuring functions has significant impact on this regularization process. Paper VIII An overview of adaptive mathematical morphology is presented. Different aspects of adaptivity in mathematical morphology are considered, and the focus is given to different methods for constructing adaptive structuring elements. An application-oriented study of four methods for adaptive structuring elements is presented, where advantages and disadvantages of each method are discussed. The proper definitions of adaptive morphological operators for both structuring elements and structuring functions are given. Furthermore, various possible directions how the field of adaptive mathematical morphology might evolve in future are discussed. 69 Appendix Norm Given a vector space V , a norm on V is a function k · k : V → R that satisfies the following properties, for all x, y ∈ V and α ∈ C: 1. kxk ≥ 0, 2. kxk = 0 if and only if x = 0, 3. kαxk = |α|kxk, 4. kx + yk ≤ kxk + kyk. Lipschitz continuity Let (X, dX ) and (Y, dY ) be two metric spaces. A function f : X → Y is Lipschitz continuous if there exists a constant λ > 0 such that dY (f (x1), f (x2 )) ≤ λdX (x1 , x2 ), for all x1 , x2 ∈ X. A constant λ is referred to as Lipschitz constant. Banach space A metric space (X, d) is a Banach space if every Cauchy sequence has limit in X. Hilbert space An inner product h·, ·i : H × H → C satisfies the following properties: 1. hx, yi ≥ 0, x ∈ H, 2. hx, xi = 0 if an only if x = 0, x ∈ H, 3. hy, xi = hx, yi, x, y ∈ H, 4. hax1 + bx2 , yi = ahx1 , yi + bhx2 , yi, x1 , x2 ∈ H, a, b ∈ C, 5. hx, ay1 + by2 i = āhx, y1 i + b̄hx, y2 i, y1 , y2 ∈ H, a, b ∈ C. A Hilbert space is a vector space H with an inner product hx, yi such that the norm defined by p kx − yk = hx − y, x − yi. A distance d between two points x, y ∈ H can be defined in terms of the norm by p d(x, y) = kx − yk = hx − y, x − yi. 71 Acknowledgements First of all, I am very grateful for these five years as a graduate student at the Uppsala University. My PhD position was funded by the Graduate School in Mathematics and Computing and I am grateful for this financial support. I would like to thank all the people at our centre/division. Special thanks to the following people who contributed to the thesis in one way or the other: • Gunilla Borgefors, my main supervisor, for giving me the opportunity to do research in image analysis, for the support during these years, for having confidence in me and for encouraging me to work on adaptive mathematical morphology. • Cris Luengo, my assistant supervisor, for encouraging me to develop my own ideas and for great discussions that we had on research, mathematical morphology and life in general. It was a great pleasure learning from you! • Joakim Lindblad and Nataša Sladoje, my assistant supervisors, for introducing me to image analysis and for encouraging me to apply for this PhD position, and for their valuable support in the first two years of my PhD studies. • Jesús Angulo, for being such a good host during my visit at the Centre for Mathematical Morphology, Fontainebleau, France. For sharing a lot of interesting ideas on mathematical morphology and research in general, and for our close collaboration on Paper VII. • Anders Landström and Matthew Thurley, for our excellent collaboration during these years. I had the great please working with Anders on Paper VIII. • Researchers in the mathematical morphology community: Sébastien Lefèvre, Johan Debayle, Santiago Velasco-Forero and Victor GonzálesCastro, for our collaborations and for your influence on my work. • Ewert Bengtsson, Ingela Nyström, Lena Nordström, and Olle Eriksson for taking a good care of our division and helping me with all administrative issues. • The more permanent staff: Ingrid Carlbom, Ida-Maria Sintorn, Carolina Wälhby, Robin Strand, Anders Brun and Filip Malmberg, for all ideas and discussions about research and life in general. • My officemates during all these years, Erik Wernersson, Catherine Östlund, Jimmy Azar, Fredrik Wahlberg, Kristina Lidayová and Olle Eriksson for all nice discussions and fun we had. 73 • Hamid Sarve and Amin Allalou, also known as Braintrust or BT, for such a great fun we had all these years, for all discussions we had and for everything. • Milan Gavrilović and Khalid Niazi for late night working hours and for fruitful discussions on image analysis and life in general. • Patrik Malm for being a really good friend and my teacher into the Swedish culture, from surströmming to innebandy. • Gustaf Kylberg, for helping me whenever I had a problem with plots and graphs. • Azadeh Fakhrzadeh, for being a great friend, for all fun we had and for our dreams about the future. • Bettina Selig, for friendship, fun, support, and for always being around. • Lennart Svenson for being a friend and a great lab partner. Also for inviting me to his beautiful home in Västerås. • Pontus Olsson, for interesting discussions about the curve of life and its derivatives. • Elisabeth Linnér, for knitting during the seminars and making me happier. • Alexandra Pacureanu and Christophe Avenel, for bringing France to our division. • The Nysjö brothers, Johan and Fredrik, for being similar and different at the same time. • Andreas Kårsnäs and Fei Liu, for being good friends despite spending only a few days in Uppsala. • More recent PhD students at our division: Omer Ishaq, Kaylan Ram, Sajith Sadanandan Kecheril, for the nice discussions we had. • My closest Uppsala friends: Katja, Markus, Abhi, Majid, Else, Juan, Camille, Vera, Camij, Pavol, Jonathan, Per, Anders, Fredrik and others, for social activities that we had all these years. • My friends from volleyball: Andrea, Eliza, Joanna, Heidi, Gurdip, Marco (el capitano), Justin, Frank, Alex and Dimitris, for having such a great time on the volleyball court. And for being officially the best team in Campus 1477! • Many thanks to my closest friends: Ceca, Sloba, Mića, Mira, Jens, Vuk, Lidia, Djudja, Ćeba, Dejan (VC), Kosta-kumić, and others, for your friendship. • My dear colleagues from the Faculty of Technical Sciences, University of Novi Sad, Serbia, for your support. • Gunilla Borgefors, Cris Luengo, Nataša Sladoje, Robin Strand, Hamid Sarve and Johan Nysjö, for proofreading and commenting on drafts of this thesis. 74 • My extended family for all support I had during these years. For all SMS messages, Skype calls and for awaiting me at the Nikola Tesla International airport in Belgrade and making my life easier. • The end of PhD studies can be seen as the end of an educational journey that in my case started in 1988 in Klek, a small village in Serbia. My mother Stana and my father Žarko have been my constant support during all these years. Mama i tata, hvala vam! Najbolji ste! Volim vas! • My finance, Iva Lučić, for bringing love into my life. Words cannot describe how lucky I am to have you in my life. Thank you for your unconditional love, for your time and support, and all the moments we shared. I look forward to our lifelong journey. I love you! Volim te najviše na svetu, ljubavi moja! ♥ 75 Summary in Swedish Denna avhandling behandlar avståndstransformer och dessas användning inom datoriserad bildanalys. Speciellt behandlas två delområden: avståndstransformer mellan punktmängder och deras tillämpning på bildregistreringsproblem; och användning av avståndstransformer för att definiera adaptiva strukturelement och strukturfunktioner som i sin tur används för att konstruera adaptiva morfologiska operatorer. Ett antal olika avståndsfunktioner har använts för ett antal olika problemlösningar. Trots att den vetenskapliga litteraturen beskriver många olika avståndsfunktioner är det inte lätt att välja den lämpligaste av dem för att lösa ett specifikt problem. Därför är det intressant att studera olika funktioner och beskriva deras egenskaper, liksom att introdusera nya avståndsfunktioner. Här studerar vi avståndsfunktioner mellan mängder och vi utvärderar deras teoretiska egenskaper och deras användbarhet för bildregistreringsproblem. Speciellt undersöker vi monotonitet hos avståndsfunktionerna under translation och rotation samt bruskänslighet. Förutom existerande avståndsfunktioner mellan mängder, där tidsåtgången för beräkningen är en linjär funktion av antalet pixlar i bilden, föreslår vi en ny avståndsfunktion, som vi kallar “komplementet till den viktade summan av minimala avstånd”. Denna avståndsfunktion mellan mängder lägger högre vikt vid punkter som ligger djupare inne i objektet och lägre vikt vid punkter som är närmare objektets kant. Detta överensstämmer väl med situationen i olika tillämpningar, där det oftast är punkter nära objektens kanter som mest påverkas av brus. Vi visar att komplementet till den viktade summan av minimala avstånd är överlägen vid bildregistrering och matchning av 2D-objekt när vi jämför den med andra liknande avståndsfunktioner. Vi visar dess användbarhet för igenkänning av handskrivna symboler och för multimodal registrering från 2D till 3D: det två dimensionella histologiska snittet genom ett benimplantat med kringliggande vävnad registreras till motsvarande läge i en tredimensionell SRµCT-volym av samma implantat. Dessutom utvidgar vi komplementet till den viktade summan av minimala avstånd till ett avstånd mellan oskarpa mängder och vi visar hur användbart detta är vid avståndsbaserad klassifikation. Den andra delen av avhandlingen fokuserar på avståndstransformer för adaptiv matematisk morfologi. Matematisk morfologi utgör ett kraftfullt koncept för icke-linjär bildbehandling, och skapar därmed värde77 fulla verktyg för många uppgifter, såsom bildfiltrering, bildsegmentering, formjämförelse, mm. Det vanligaste sättet att definiera morfologiska operatorer är att använda begreppet strukturelement. Strukturelement är vanligtvis små punktmängder som används för att undersöka bilder. Resultatet av den morfologiska operationen beror på interaktionen mellan bilden och strukturelementet. Adaptiv matematisk morfologi har nyligen blivit ett populärt forskningsområde inom matematisk morfologi. Den underliggande idén är att definiera morfologiska operatorer som inte behandlar alla punkter i bilden på samma sätt, utan tar hänsyn till hur viktig varje punkt är. Därför definieras adaptiva morfologiska operatorer genom adaptiva strukturelement som anpassas till lokala förhållanden i bilden. För varje punkt i bilden kan det adaptiva strukturelementet beräknas med hjälp av olika bildegenskaper, till exempel likhet mellan grånivåerna för punktgrannar, gradienter och orientering. Vi skapar adaptiva strukturelement som baseras på en signifikanskarta som i sin tur baseras på styrkan hos kanterna i bilden, det vill säga på hur viktig varje kantpunkt är. En sådan signifikanskarta skapas genom att beräkna en avståndstransform som tar hänsyn till både avstånd mellan punkter och styrkan hos bildens kanter. Egenskaperna sprids över bilden med hyvlingsalgoritmen. Två olika typer av adaptiva strukturelement har definierats. I det första fallet är strukturelements form förbestämd och endast dess storlek bestäms av signifikanskartan. I det andra fallet beror både strukturelementets form och storlek på signifikanskartan. Dessutom definierar vi adaptiva strukturfunktioner, som är strukturelement där varje punkt har en vikt mellan minus oändligheten och noll. Dessa adaptiva strukturfunktioner baseras också på signifikanskartan och utgör en generalisering av de adaptiva strukturfunktioner vi först introducerade. De nya adaptiva strukturfunktionerna är icke-symmetriska paraboliska funktioner. Den här avhandlingen visar även på användbarheten av adaptiv matematisk morfologi för bildregularisering. Samspelet mellan adaptiv matematisk morfologi och Lipschitz-regularisering av Lasry–Lions-typ för icke-släta funktioner skapar ett elegant verktyg för morfologisk bildregularisering. Med andra ord använder denna typ av bildregularisering adaptiva morfologiska operatorer fr att skapa Lipschitz-regularisering av bilder som innehåller brus. Avhandlingen innehåller även en översikt av adaptiv matematisk morfologi, tillsammans med en jämförelse av olika metoder för att skapa adaptiva strukturelement och avslutas med olika möjliga framtida forskningsinriktningar inom adaptiv matematisk morfologi. 78 References [1] J. Angulo. Morphological colour operators in totally ordered lattices based on distances: Application to image filtering, enhancement and analysis. Computer Vision and Image Understanding, 107(1):56–73, 2007. [2] J. Angulo. Morphological bilateral filtering and spatially-variant adaptive structuring functions. In Proceedings of the International Symposium on Mathematical Morphology, pages 212–223. Springer, 2011. [3] J. Angulo and S. Velasco-Forero. Complete lattice structure of Poincaré upper-half plane and mathematical morphology for hyperbolic-valued images. In Geometric Science of Information, pages 535–542. Springer, 2013. [4] J. Angulo and S. Velasco-Forero. Mathematical morphology for real-valued images on Riemannian manifolds. In Proceedings of the International Symposium on Mathematical Morphology, pages 279–291. Springer, 2013. [5] J. Angulo and S. Velasco-Forero. Stochastic morphological filtering and Bellman-Maslov chains. In Proceedings of the International Symposium on Mathematical Morphology, pages 171–182. Springer, 2013. [6] E. Aptoula and S. Lefèvre. A comparative study on multivariate mathematical morphology. Pattern Recognition, 40(11):2914–2929, 2007. [7] H. Attouch and D. Azé. Approximation and regularization of arbitrary functions in Hilbert spaces by the Lasry–Lions method. In Annales de l’institut Henri Poincaré (C) Analyse non linéaire, volume 10, pages 289–312, 1993. [8] B.B. Avants, C.L. Epstein, M. Grossman, and J.C. Gee. Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brain. Medical image analysis, 12(1):26–41, 2008. [9] V. Berinde and M. Pacurar. The role of the Pompeiu-Hausdorff metric in fixed point theory. Creative Mathematics Informatics, 22(2):35–42, 2013. [10] S. Beucher, J.M. Blosseville, and F. Lenoir. Traffic spatial measurements using video image processing. In Robotics and IECON’87 Conferences, pages 648–655. [11] T. Birsan and D. Tiba. One hundred years since the introduction of the set distance by Dimitrie Pompeiu. In System Modeling and Optimization, pages 35–39. Springer, 2006. [12] I. Bloch. On fuzzy distances and their use in image processing under imprecision. Pattern Recognition, 32(11):1873–1895, 1999. 79 [13] I. Bloch and H. Maı̂tre. Fuzzy distances and image processing. In Proceedings of the ACM Symposium on Applied Computing, pages 570–574, 1995. [14] G. Borgefors. Distance transformations in arbitrary dimensions. Computer Vision, Graphics, and Image Processing, 27(3):321–345, 1984. [15] G. Borgefors. Distance transformations in digital images. Computer Vision, Graphics, and Image Processing, 34(3):344–371, 1986. [16] G. Borgefors. Hierarchical chamfer matching: A parametric edge matching algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10(6):849–865, 1988. [17] N. Bouaynaya, M. Charif-Chefchaouni, and D. Schonfeld. Theoretical foundations of spatially-variant mathematical morphology part I: Binary images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(5):823–836, 2008. [18] N. Bouaynaya and D. Schonfeld. Spatially variant morphological image processing: Theory and applications. In Proceedings of SPIE, volume 6077, pages 673–684, 2006. [19] N. Bouaynaya and D. Schonfeld. Theoretical foundations of spatially-variant mathematical morphology part II: Gray-level images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(5):837–850, 2008. [20] L. Boxer. On Hausdorff-like metrics for fuzzy sets. Pattern Recognition Letters, 18(2):115–118, 1997. [21] P. Brass. On the nonexistence of Hausdorff-like metrics for fuzzy sets. Pattern Recognition Letters, 23(1):39–43, 2002. [22] M. Breuß, B. Burgeth, and J. Weickert. Anisotropic continuous-scale morphology. In Proceedings of the Iberian Conference on Pattern Recognition and Image Analysis, pages 515–522. Springer, 2007. [23] R.W. Brockett and P. Maragos. Evolution equations for continuous-scale morphological filtering. IEEE Transactions on Signal Processing, 42(12):3377–3386, 1994. [24] A. M. Bronstein, M. M. Bronstein, R. Kimmel, M. Mahmoudi, and G. Sapiro. A Gromov–Hausdorff framework with diffusion geometry for topologically-robust non-rigid shape matching. International Journal of Computer Vision, 89(2-3):266–286, 2010. [25] A. Buades, B. Coll, and J.M Morel. A review of image denoising algorithms, with a new one. SIAM Journal on Multiscale Modeling & Simulation, 4(2):490–530, 2005. [26] J. Cai, J. C.H. Chu, D. Recine, M. Sharma, C. Nguyen, R. Rodebaugh, V. A. Saxena, and A. Ali. CT and PET lung image registration and fusion in radiotherapy treatment planning using the chamfer-matching method. International Journal of Radiation Oncology, 43(4):883–891, 1999. [27] J. Canny. A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence,, (6):679–698, 1986. [28] M. Cepedello-Boiso. On regularization in superreflexive Banach spaces 80 [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] by infimal convolution formulas. Studia Mathematica, 129(3):265–284, 1998. M. Charif-Chefchaouni and D. Schonfeld. Spatially-variant mathematical morphology. In Proceedings of the IEEE International Conference on Image Processing, volume 2, pages 555–559, 1994. B. B. Chaudhuri and A. Rosenfeld. A modified Hausdorff distance between fuzzy sets. Information Sciences, 118(1):159–171, 1999. J. Cousty, L. Najman, F. Dias, and J. Serra. Morphological filtering on graphs. Computer Vision and Image Understanding, pages 1–20, 2012. O. Cuisenaire. Locally adaptable mathematical morphology using distance transformations. Pattern Recognition, 39(3):405–416, 2006. J. Debayle and J.C. Pinoli. Spatially adaptive morphological image filtering using intrinsic structuring elements. Image Analysis & Stereology, 24(3):145–158, 2005. J. Debayle and J.C. Pinoli. General adaptive neighborhood image processing - Part II: Practical application examples. Journal of Mathematical Imaging and Vision, 25(2):267–284, 2006. J. Debayle and J.C. Pinoli. General adaptive neighborhood image processing Part I: Introduction and theoretical aspects. Journal of Mathematical Imaging and Vision, 25(2):245–266, 2006. J. Debayle and J.C. Pinoli. Spatially adaptive color image processing. In Advances in Low-Level Color Image Processing, pages 195–222. Springer, 2014. K. Denecke, S. L. Wismath, and M. Erne. Galois connections and applications, volume 565. Springer, 2004. M. M. Deza and E. Deza. Encyclopedia of Distances. Springer, 2009. E. W. Dijkstra. A note on two problems in connexion with graphs. Numerische mathematik, 1(1):269–271, 1959. P. Dokládal and E. Dokládalová. Grey-scale morphology with spatially-variant rectangles in linear time. In Proceedings of the International Conference on Advanced Concepts for Intelligent Vision Systems, pages 674–685. Springer, 2008. M.P. Dubuisson and A. K. Jain. A modified Hausdorff distance for object matching. In Proceedings of the 12th IAPR International Conference on Computer Vision & Image Processing, volume 1, pages 566–568. IEEE, 1994. T. Eiter and H. Mannila. Distance measures for point sets and their computation. Acta Informatica, 34(2):109–133, 1997. E. Franken and R. Duits. Crossing-preserving coherence-enhancing diffusion on invertible orientation scores. International Journal of Computer Vision, 85(3):253–278, 2009. V. Gonzalez-Castro, J. Debayle, and J.C. Pinoli. Color adaptive neighborhood mathematical morphology and its application to pixel-level classification. Pattern Recognition Letters, 2014. accepted paper. J. Grazzini and P. Soille. Edge-preserving smoothing using a similarity measure in adaptive geodesic neighbourhoods. Pattern Recognition, 81 42(10):2306–2316, 2009. [46] E. Haber and J. Modersitzki. Intensity gradient based registration and fusion of multi-modal images. In Proceddings of Medical Image Computing and Computer-Assisted Intervention, pages 726–733. Springer, 2006. [47] J. V. Hajnal and D. L.G. Hill. Medical image registration. CRC press, 2010. [48] L. Hallström Mayrhofer. Development, analysis and implementation of new nonlocal morphological operators. Master’s thesis, Uppsala University, to be published, 2014. [49] F. Hausdorff. Grundzüge der Mengenlehre. Viet. Leipzig, 1914. [50] H.J.A M. Heijmans, P. Nacken, A. Toet, and L. Vincent. Graph morphology. Journal of Visual Communication and Image Representation, 3(1):24–38, 1992. [51] H.J.A.M. Heijmans. Morphological image operators. Advances in Electronics and Electron Physics Suppl., Boston: Academic Press, 1, 1994. [52] H.J.A.M. Heijmans, M. Buckley, and H. Talbot. Path openings and closings. Journal of Mathematical Imaging and Vision, 22(2-3):107–119, 2005. [53] H.J.A.M. Heijmans and C. Ronse. The algebraic basis of mathematical morphology. I Dilations and erosions. Computer Vision, Graphics, and Image Processing, 50(3):245–295, 1990. [54] T. Hrkać, Z. Kalafatić, and J. Krapac. Infrared-visual image registration based on corners and Hausdorff distance. In Proceedings of Scandinavian Conference on Image Analysis, pages 383–392. Springer, 2007. [55] D.P. Huttenlocher, G.A. Klanderman, and W.J. Rucklidge. Comparing images using the Hausdorff distance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(9):850–863, 1993. [56] L. Ikonen and P. Toivanen. Distance and nearest neighbour transforms on gray-level surfaces. Pattern Recognition Letters, 28(5):604–612, 2007. [57] R. Kimmel and J. A. Sethian. Computing geodesic paths on manifolds. Proceedings of the National Academy of Sciences, 95(15):8431–8435, 1998. [58] R. Klette and A. Rosenfeld. Digital Geometry: geometric methods for digital picture analysis. Morgan Kaufmann, 2004. [59] R. Lagerstrom and M. Buckley. An attribute weighted distance transform. Pattern Recognition Letters, 2012. [60] A. Landström and M.J. Thurley. Adaptive morphology using tensor-based elliptical structuring elements. Pattern Recognition Letters, 34(12):1416–1422, 2013. [61] J. M. Lasry and P.L. Lions. A remark on regularization in Hilbert spaces. Israel Journal of Mathematics, 55(3):257–266, 1986. [62] R. Lerallut, É. Decencière, and F. Meyer. Image filtering using morphological amoebas. Image and Vision Computing, 25(4):395–404, 2007. 82 [63] G. Levi and U. Montanari. A grey-weighted skeleton. Information and Control, 17(1):62–91, 1970. [64] J. Lindblad and N. Sladoje. Linear time distances between fuzzy sets with applications to pattern matching and classification. IEEE Transactions on Image Processing, 23(1):126–136, 2014. [65] F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens. Multimodality image registration by maximization of mutual information. IEEE Transactions on Medical Imaging, 16(2):187–198, 1997. [66] P. Malm and A. Brun. Closing curves with Riemannian dilation: Application to segmentation in automated cervical cancer screening. In Proceedings of the International Symposium on Visual Computing, pages 337–346. Springer, 2009. [67] P. Maragos and C. Vachier. A PDE formulation for viscous morphological operators with extensions to intensity-adaptive operators. In Proceedings of the International Conference on Image Processing, 2008, pages 2200–2203. IEEE, 2008. [68] P. Maragos and C. Vachier. Overview of adaptive morphology: Trends and perspectives. In Proceedings of the International Conference on Image Processing, pages 2241–2244. IEEE, 2009. [69] G. Matheron. Random Sets and Integral Geometry. 1975. [70] C. R. Maurer Jr, R. Qi, and V. Raghavan. A linear time algorithm for computing exact Euclidean distance transforms of binary images in arbitrary dimensions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(2):265–270, 2003. [71] F. Memoli. On the use of Gromov–Hausdorff distances for shape comparison. In Proceedings of the Eurographics Symposium on Point-based Graphics, pages 81–90, 2007. [72] V. Morard, E. Decenciere, and P. Dokládal. Region growing structuring elements and new operators based on their shape. In Proceedings of the International Conference on Signal and Image Processing, pages 78–85, 2011. [73] J.J. Moreau. Fonctionele convexes. Lecture Notes, College de France, Paris, 1967. [74] L. Najman and H. Talbot, editors. Mathematical Morphology. John Wiley & Sons, 2013. [75] D. L. Pham and J.L. Prince. Adaptive fuzzy segmentation of magnetic resonance images. IEEE Transactions on Medical Imaging, 18(9):737–752, 1999. [76] D. Pompeiu. Sur la continuité des fonctions de variables complexes (These). PhD thesis, 1905. [77] P. Purkait and B. Chanda. Super resolution image reconstruction through Bregman iteration using morphologic regularization. IEEE Transactions on Image Processing, 21(9):4029–4039, 2012. [78] P. Purkait and B. Chanda. Adaptive morphologic regularizations for inverse problems. In Proceedings of International Symposium on Mathematical Morphology, pages 195–206. Springer, 2013. 83 [79] A. L. Ralescu and D. A. Ralescu. Probability and fuzziness. Information Sciences, 34(2):85–92, 1984. [80] J.B.T.M. Roerdink. Manifold shape: from differential geometry to mathematical morphology. Springer, 1994. [81] J.B.T.M. Roerdink. Group morphology. Pattern Recognition, 33(6):877–895, 2000. [82] J.B.T.M. Roerdink. Adaptivity and group invariance in mathematical morphology. In Proceedings of the International Conference on Image Processing, pages 2253–2256. IEEE, 2009. [83] C. Ronse. Why mathematical morphology needs complete lattices. Signal Processing, 21(2):129–154, 1990. [84] C. Ronse and H.J.A M. Heijmans. The algebraic basis of mathematical morphology. II Openings and closings. Computer Vision, Graphics, and Image Processing, 55(1):74–97, 1991. [85] A. Rosenfeld. Distances between fuzzy sets. Pattern Recognition Letters, 3(4):229–233, 1985. [86] A. Rosenfeld and J. L. Pfaltz. Sequential operations in digital picture processing. Journal of the ACM, 13(4):471–494, 1966. [87] P.L. Rosin. A simple method for detecting salient regions. Pattern Recognition, 42(11):2363–2371, 2009. [88] P.L. Rosin and G.A.W. West. Salience distance transforms. Graphical Models and Image Processing, 57(6):483–521, 1995. [89] L. I. Rudin, S. Osher, and E. Fatemi. Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena, 60(1):259–268, 1992. [90] D. Rueckert, L.I. Sonoda, C. Hayes, D. L.G. Hill, M. O. Leach, and D.J. Hawkes. Nonrigid registration using free-form deformations: Application to breast MR images. IEEE Transactions on Medical Imaging, 18(8):712–721, 1999. [91] D. Rutovitz. Data structures for operations on digital images. Pictorial pattern recognition, pages 105–133, 1968. [92] P. Salembier. Study on nonlocal morphological operators. In Proceedings of the International Conference on Image Processing, pages 2269–2272. IEEE, 2009. [93] H. Sarve, J. Lindblad, and C. B. Johansson. Registration of 2D histological images of bone implants with 3D SRµCT volumes. In Proceedings of the International Symposium on Visual Computing, pages 1071–1080. Springer, 2008. [94] T. Schuster, B. Kaltenbacher, B. Hofmann, and K.S. Kazimierski. Regularization methods in Banach spaces, volume 10. Walter de Gruyter, 2012. [95] J. Serra. Image analysis and mathematical morphology. Academic Press, London, 1982. [96] J. Serra, editor. Image analysis and mathematical morphology. Vol. 2. Academic Press, New York, NY, 1988. [97] J. A. Sethian. Level set methods and fast marching methods: Evolving interfaces in computational geometry, fluid mechanics, computer vision, 84 and materials science, volume 3. Cambridge University Press, 1999. [98] N. Sladoje and J. Lindblad. High-precision boundary length estimation by utilizing gray-level information. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(2):357–363, 2009. [99] N. Sladoje, I. Nyström, and P.K. Saha. Measurements of digitized objects with fuzzy borders in 2D and 3D. Image and Vision Computing, 23(2):123–132, 2005. [100] P. Soille. Generalized geodesy via geodesic time. Pattern Recognition Letters, 15(12):1235–1240, 1994. [101] P. Soille. On morphological operators based on rank filters. Pattern Recognition, 35(2):527–535, 2002. [102] P. Soille. Morphological Image Analysis: Principles and Applications. Springer-Verlag New York, Inc., 2 edition, 2003. [103] M. Sonka and J. M. Fitzpatrick. Handbook of Medical Imaging, Vol. 2: Medical Image Processing and Analysis. SPIE-International Society for Optical Engineering, 2000. [104] S. Sternberg. Language and architecture for parallel image processing. In Proceedings of the Pattern Recognition in Practice. [105] S. R. Sternberg. Grayscale morphology. Computer Vision, Graphics, and Image Processing, 35(3):333–355, 1986. [106] R. Strand. Weighted distances based on neighbourhood sequences. Pattern Recognition Letters, 28(15):2029–2036, 2007. [107] R. Strand. Distance Functions and Image Processing on Point-Lattices: with focus on the 3D face-and body-centered cubic grids. PhD thesis, Uppsala University, 2008. [108] O. Tankyevych, H. Talbot, and P. Dokládal. Curvilinear morpho-Hessian filter. In Proceedings of the International Symposium on Biomedical Imaging, pages 1011–1014. IEEE, 2008. [109] P. J. Toivanen. New geodesic distance transforms for gray-scale images. Pattern Recognition Letters, 17(5):437–450, 1996. [110] C. Tomasi and R. Manduchi. Bilateral filtering for gray and color images. In Proceedings of the International Conference on Computer Vision, pages 839–846. IEEE, 1998. [111] R. van den Boomgaard. Mathematical Morphology: Extensions Towards Computer Vision. PhD thesis, University of Amsterdam, Amsterdam, 1992. [112] S. Velasco-Forero and J. Angulo. On nonlocal mathematical morphology. In Proceedings of the International Symposium on Mathematical Morphology, pages 219–230. Springer, 2013. [113] R. Verdú-Monedero, J. Angulo, and J. Serra. Anisotropic morphological filters with spatially-variant structuring elements based on image-dependent gradient fields. IEEE Transactions on Image Processing, 20(1):200–212, 2011. [114] J.G. Verly and R.L. Delanoy. Adaptive mathematical morphology for range imagery. IEEE Transactions on Image Processing, 2(2):272–275, 1993. [115] L. Vincent. Graphs and mathematical morphology. Signal Processing, 85 16(4):365–388, 1989. [116] L. Vincent. Morphological area openings and closings for grey-scale images. In Shape in Picture, pages 197–208. Springer, 1994. [117] M. Welk, M. Breuß, and O. Vogel. Morphological amoebas are self-snakes. Journal of Mathematical Imaging and Vision, 39(2):87–99, 2011. [118] W.M. Wells, P. Viola, H. Atsumi, S. Nakajima, and R. Kikinis. Multi-modal volume registration by maximization of mutual information. Medical image analysis, 1(1):35–51, 1996. [119] Z. Xu and M. Xia. Distance and similarity measures for hesitant fuzzy sets. Information Sciences, 181(11):2128–2138, 2011. [120] L. A. Zadeh. Fuzzy sets. Information and control, 8(3):338–353, 1965. [121] B. Zitova and J. Flusser. Image registration methods: a survey. Image and Vision Computing, 21(11):977–1000, 2003. 86 Acta Universitatis Upsaliensis Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Science and Technology 1137 Editor: The Dean of the Faculty of Science and Technology A doctoral dissertation from the Faculty of Science and Technology, Uppsala University, is usually a summary of a number of papers. A few copies of the complete dissertation are kept at major Swedish research libraries, while the summary alone is distributed internationally through the series Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Science and Technology. (Prior to January, 2005, the series was published under the title “Comprehensive Summaries of Uppsala Dissertations from the Faculty of Science and Technology”.) Distribution:publications.uu.se urn:nbn:se:uu:diva-221568 ACTA UNIVERSITATIS UPSALIENSIS UPPSALA 2014

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement