Edge Preserving Smoothing Techniques Prof. PhD. Vasile Gui Polytechnic University of Timisoara Content Introduction Brief review of linear operators Linear image smoothing techniques Nonlinear image smoothing techniques Introduction Why do we need image smoothing? What is “image” and what is “noise”? – – Frequency spectrum Statistical properties Brief Review of Linear Operators [Pratt 1991] Generalized 2D linear operator g (m, n) M 1 N 1 O( j, k ; m, n) f ( j, k ) j 0 k 0 Separable linear operator: O( j, k ; m, n) OR ( j; m)OC (k ; n) g (m, n) M 1 O j 0 N 1 R ( j; m) OC (k ; n) f ( j , k ) k 0 Space invariant operator: O( j , k ; m, n) O(m j , n k ) H (m j , n k ) g (m, n) M 1 N 1 h(m j 0 k 0 Convolution sum j, n k ) f ( j, k ) Brief Review of Linear Operators h22 h21h20 h12 h11h10 h02 h01 h00 H L-1 2 G L-1 2 L-1 2 L-1 2 h22 h21h20 h12 h11h10 h02 h01 h00 h22 h21h20 h12 h11h10 h02 h01 h00 NL+1 N F N+L-1 h22 h21h20 h12 h11h10 h02 h01 h00 h22 h21h20 h12 h11h10 h02 h01 h00 Geometrical interpretation of 2D convolution Brief Review of Linear Operators Circular convolution theorem N 1 N 1 g (m, n) h(m j, n k ) f ( j, k ) j 0 k 0 1 t g ( u, v ) t f ( u, v ) t h ( u, v ) N Periodicity of f,h and g with N are assumed Linear Image smoothing techniques Box filters. Arithmetic mean LL operator 1 1 1 h 2 L 1 1 1 1 1 1 1 1 1 1 1 1* 1 L L 1 1 1 Linear Image smoothing techniques Box filters. Arithmetic mean LL operator Separable Can be computed recursively, resulting in roughly 4 operations per pixel m,n m,n+1 + L pixels Linear Image smoothing techniques Box filters. Arithmetic mean LL operator Optimality properties Signal and additive white noise 1 g N 1 z N N 1 ( f k nk ) N k 1 N k 1 1 fk N N n . k 1 k N n k 1 k . Noise varianceN is reducedN NN times N 1 1 E{ z z} 2 E{ nl nk } 2 N N k 1 l 1 1 N N 1 2 2 ( l , k ) = 2 N N k 1 l 1 2 z T E{n n } k 1 l 1 l k Linear Image smoothing techniques Box filters. Arithmetic mean LL operator Unknown constant signal plus noise Minimize MSE of the estimation g: N (g) ( fk g)2 2 k 1 2 ( g ) 0 g 1 gˆ N N f k 1 k Linear Image smoothing techniques Box filters. Arithmetic mean LL operator i.i.d. Gaussian signal with unknown mean. ( f ) 2 p( f | ) ct. exp{ } 2 2 Given the observed samples, maximize N 1 p ( f | ) ct . exp{ k 2 2 k 1 N ( f k 1 k ) 2 } Optimal solution: arithmetic mean Linear Image smoothing techniques Box filters. Arithmetic mean LL operator Frequency response: Non-monotonically decreasing with frequency 1 1 1 1 1 1 1 1 1 1 [ 111 ] 1 hx hy 3 9 3 1 1 1 1 ( N 1)/ 2 t (u) hx (0) n ( N 1)/ 2 hx (n) exp{ 1 2 2 t (u ) cos( u ) 3 3 N i 2 un} N Linear Image smoothing techniques Box filters. Arithmetic mean LL operator An example 8 5 8 8 5 8 7 7 7 7 7 7 8 5 8 5 8 5 6 7 6 7 6 7 1 1 1 1 8 5 8 8 5 8 1 1 1 7 7 7 7 7 7 9 1 1 1 7 7 7 7 7 7 8 5 8 8 5 8 1 1 1 1 8 5 8 5 8 5 1 1 1 6 7 6 7 6 7 9 1 1 1 6 7 6 7 6 7 8 5 8 5 8 5 Linear Image smoothing techniques Box filters. Arithmetic mean LL operator Image smoothed with 33, 55, 99 and 11 11 box filters Linear Image smoothing techniques Box filters. Arithmetic mean LL operator Original Lena image Lena image filtered with 5x5 box filter Linear Image smoothing techniques Binomial filters [Jahne 1995] Computes a weighted average of pixels in the window Less blurring, less noise cleaning for the same size The family of binomial filters can be defined recursively The coefficients can be found from (1+x)n Linear Image smoothing techniques Binomial filters. 1D versions b1 1 1 1 2 1 b 1 2 1 b 1 * b 1 4 1 4 b 1 4 6 4 1 b 1 * b 1 * b 1 * b 1 16 2 1 1 6 15 20 15 6 1 b 1 * b 1 * b 1 * b 1 * b 1 * b 1 b 64 6 As size increases, the shape of the filter is closer to a Gaussian one Linear Image smoothing techniques Binomial filters. 2D versions 1 1 2 1 1 1 1 b 2 1 2 1* 2 2 4 2 4 4 16 1 1 2 1 1 1 4 6 4 1 4 4 16 24 16 4 1 1 1 4 6 24 36 24 6 b 1 4 6 4 1* 6 16 16 256 4 4 16 24 16 4 1 1 4 6 4 1 Linear Image smoothing techniques Binomial filters. Frequency response 1 b 1 2 1 4 2 b4 t (u ) 1 1 4 6 4 1 16 1 1 2 cos( u ) 2 2 N 1 1 2 t (u ) [ cos( u )] 2 2 2 N Monotonically decreasing with frequency Linear Image smoothing techniques Binomial filters. Example Original Lena image Lena image filtered Lena image filtered with binomial 5x5 kernel with box filter 5x5 Linear Image smoothing techniques Binomial and box filters. Edge blurring comparison Linear filters have to compromise smoothing with edge blurring Step edge Result of size L box filter Size L binomial L Nonlinear image smoothing The median filter [Pratt 1991] Block diagram f(1) f1 f(2) f2 . . . fN Order samples . . . select f(N) median f (1) f ( 2 ) f (3) f ( N ) N 1 m 2 N is odd f(m) Nonlinear image smoothing The median filter Numerical example 3 7 8 2 3 7 4 6 7 2, 3, 3, 4, 6, 7, 7, 7, 8 6 Nonlinear image smoothing The median filter Nonlinearity median{ f1 + f2} median{ f1} + median{ f2}. However: median{ c f } = c median{ f }, median{ c + f } = c + median{ f }. • The filter selects a sample from the window, does not average • Edges are better preserved than with liner filters • Best suited for “salt and pepper” noise Nonlinear image smoothing The median filter Noisy image 5x5 median filtered 5x5 box filter Nonlinear image smoothing The median filter Optimality Grey level plateau plus noise. Minimize sum of absolute differences: N ( g ) | f k g | k 1 Result: g median{ f1 , f 2 , f 3 ,, f N } f ( m ) If 51% of samples are correct and 49% outliers, the median still finds the right level! Nonlinear image smoothing The median filter Caution: points, thin lines and corners are erased by the median filter Test images Results of 33 pixel median filter Nonlinear image smoothing The median filter Cross shaped window can correct some of the problems Results of the 9 pixel cross shaped window median filter Nonlinear image smoothing The median filter Implementing the median filter Sorting needs O(N2) comparisons – – – – Bubble sort Quick sort Huang algorithm (based on histogram) VLSI median Nonlinear image smoothing The median filter VLSI median block diagram a b Min(a,b) Max(a,b) x1 x(1) x2 x(2) x3 x(3) x4 x(4) x5 x(5) x6 x(6) x7 x(7) Nonlinear image smoothing The median filter Color median filter – – – – There is no natural ordering in 3D (RGB) color space Separate filtering on R,G and B components does not guarantee that the median selects a true sample from the input window Vector median filter, defined as the sample minimizing the sum of absolute deviations from all the samples Computing the vector median is very time consuming, although several fast algorithms exist Nonlinear image smoothing The median filter Example of color median filtering 5x5 pixels window Up: original image Down: filtered image Nonlinear image smoothing The weighted median filter The basic idea is to give higher weight to some samples, according to their position with respect to the center of the window Each sample is given a weight according to its spatial position in the window. Weights are defined by a weighting mask Weighted samples are ordered as usually The weighted median is the sample in the ordered array such that neither all smaller samples nor all higher samples can cumulate more than 50% of weights. If weights are integers, they specify how many times a sample is replicated in the ordered array Nonlinear image smoothing The weighted median filter Numerical example for the weighted median filter 1 2 1 2 3 2 1 2 1 3 7 8 2 3 7 4 6 7 2, 2, 3, 3, 3, 3, 4, 6, 6, 7, 7, 7, 7, 7, 8 6 Nonlinear image smoothing Rank-order filters (L filters) Block diagram weights f(1) f1 a1 f(2) f2 a2 . . . ordering y . . . sum f(N) fN aN N a k 1 k 1 y = a1f(1) + a2 f(2)+...+ aN f(N) Nonlinear image smoothing Rank-order filters (L filters) Some examples a1=1 . . . Percentile filters (rank selection): Min aN=1 . . . 0%, 50% Max Particular cases of grey level morphological filters am=1 . . . . . . 100% a1=.5 . . . Median aN=.5 Mid range Nonlinear image smoothing Rank-order filters (L filters) Optimality N ( f ) | f ( k ) f |r Minkovski distance k 1 Case r = 1: best estimator is median. Case r = 2, best estimator is arithmetic mean. Case r , best estimator is mid-range. Nonlinear image smoothing Rank-order filters (L filters) Alpha-trimmed mean filter 1 /( 2Q 1), for m Q k m Q ak otherwise 0, = (2Q + 1) / N, defines de degree of averaging = 1 corresponds to arithmetic mean = 1 / N corresponds to the median filter Properties: in between mean and median Nonlinear image smoothing Rank-order filters (L filters) Median of absolute differences trimmed mean Better smoothing than the median filter and good edge preservation M 1 median{ f1 , f 2 , f 3 ,, f N } M 2 median{| f i M 1 |}, i 1,2,..., N output average{ f i : | f i M 1 | M 2 } M2 is a robust estimator of the variance Nonlinear image smoothing Rank-order filters (L filters) k nearest neighbour (kNN) median filter Median of the k grey values nearest by rank to the central pixel Aim: same as above Nonlinear image smoothing Selected area filtering [Nagao 1980] Kuwahara type filtering Form regions Ri Compute mean, mi and variance, Si for each region. Result = mi : mi < mj for all j different from i, i,e. the mean of the most homogeneous region Matsujama & Nagao reg 1 reg 2 reg 4 reg 3 reg 6 reg 5 reg 8 reg 7 Nonlinear image smoothing Conditional mean Pixels in a neighbourhood are averaged only if they differ from the central pixel by less than a given threshold: g (m, n) L L h(k , l ) f (m k , n l ), k L l L 1, if | f (m k , n l ) f (m, n) | th h( k , l ) 0, otherwise L is a space scale parameter and th is a range scale parameter Nonlinear image smoothing Conditional mean Example with L=3, th=32 Nonlinear image smoothing Bilateral filter [Tomasi 1998] Space and range are treated in a similar way Space and range similarity is required for the averaged pixels Tomasi and Manduchi [1998] introduced soft weights to penalize the space and range dissimilarity. h(k , l ) s(k , l )r ( f (m k , n l ) f (m, n)) 1 g ( m, n) h( k , l ) f ( m k , n l ), K k l K h( k , l ) k l s() and r() are space and range similarity functions (Gaussian functions of the Euclidian distance between their arguments). Nonlinear image smoothing Bilateral filter The filter can be seen as weighted averaging in the joint space-range space (3D for monochromatic images and 5D – x,y,R,G,B - for colour images) The vector components are supposed to be properly normalized (divide by variance for example) The weights are given by: || x c x || 2 h(x) exp{ } s h(x) K(d(x c x); s ) Nonlinear image smoothing Bilateral filter Example of Bilateral filtering Low contrast texture has been removed Yet edges are well preserved Nonlinear image smoothing Mean shift filtering [Comaniciu 1999, 2002] Mean shift filtering replaces each pixel’s value with the most probable local value, found by a nonparametric probability density estimation method. xi R d {x i }i 1... n The multivariate kernel density estimate obtained in the point x with the kernel K(x) and window radius r is: 1 fˆ (x) d nr x xi K r i 1 n For the Epanechnikov kernel, the estimated normalized density gradient is proportional to the mean shift: ˆ f ( x) r2 1 M r ( x) d 2 fˆ (x) nx x xi S r ( x ) i x S is a sphere of radius r, centered on x and nx is the number of samples inside the sphere Nonlinear image smoothing Mean shift filtering The mean shift procedure is a gradient ascent method to find local modes (maxima) of the probability density and is guaranteed to converge. Step1: computation of the mean shift vector Mr(x). Step2: translation of the window Sr(x) by Mr(x). Iterations start for each pixel (5D point) and tipically converge in 2-3 steps. Nonlinear image smoothing Mean shift filtering Example1. Nonlinear image smoothing Mean shift filtering Detail of a 24x40 window from the cameraman image a) Original data b) Mean shift paths for some points c) Filtered data d) Segmented data Nonlinear image smoothing Mean shift filtering Example 2 Nonlinear image smoothing Mean shift filtering Comparison to bilateral filtering Both methods based on simultaneous processing of both the spatial and range domains While the bilateral filtering uses a static window, the mean shift window is dynamic, moving in the direction of the maximum increase of the density gradient. REFERENCES D. Comaniciu, P. Meer: Mean shift analysis and applications. 7th International Conference on Computer Vision, Kerkyra, Greece, Sept. 1999, 1197-1203. D. Comaniciu, P. Meer: Mean shift: a robust approach toward feature space analysis. IEEE Trans. on PAMI Vol. 24, No. 5, May 2002, 1-18. M. Elad: On the origin of bilateral filter and ways to improve it. IEEE trans. on Image Processing Vol. 11, No. 10, October 2002, 1141-1151. B. Jahne: Digital image processing. Springer Verlag, Berlin 1995. M. Nagao, T. Matsuiama: A structural analysis of complex aerial photographs. Plenum Press, New York, 1980. W.K. Pratt: Digital image processing. John Wiley and sons, New York 1991 C. Tomasi, R. Manduchi: Bilateral filtering for gray and color images. Proc. Sixth Int’l. Conf. Computer Vision , Bombay, 839-846.

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement