Image Intensity Transformation and Image Enhancement

AJIT RAJWADE

CS 663

Intensity transformations

• Consider an image I(x,y).
• An intensity transformation refers to the transformation applied to individual image intensities to yield a transformed image of the form: J(x,y) = T(I(x,y)) where T(.) is a transformation operator.

​

• For example: J(x,y) = [I(x,y)]² or J(x,y) = I(x,y)-I(x,y+1)

​

• In the latter case, the transformation makes use of other pixel values, usually from the neighborhood of the pixel (x,y).

2

Image Enhancement

• Defined as the process of manipulating image intensities, so that the resulting image is more visually suitable for a specific application.

​

• The quality of an image enhancement algorithm is judged by the perception of the end-user.

3

Popular image transformations

• Image Negatives
• Logarithm and Power Law transformations
• Contrast Stretching
• Bit-plane slicing
• Histogram Equalization
• Histogram Specification

4

Image Negatives

• Let an image have intensity range [0,L-1].
• The negative transformation has the form s(r)=L-1-r where s,r = output, input intensity level.
• This reverses the intensity levels of an image – equivalent of a photographic negative
• This can produce an enhancing effect – to enhance grayish detail on dark background.

5

6

Logarithmic transformation

• Has the form s(r) = c log(1+r) where s,r = output, input intensity level assuming r >= 0 and c is some constant.
• It maps a narrow range of low intensity values to a wider range.
• It maps a range of high intensity values to a narrow range.
• This can be observed from the graphs on the next slide.

7

8

Logarithmic transformations

• A logarithmic transformation compresses the intensity range of images whose initial intensity range is very wide.

​

• For example, the Fourier spectrum of an image (we will study these later in class) often has intensities ranging from close to 0 to 10⁷.

​

• Displaying the original intensities of the Fourier spectrum as is, causes a huge loss of detail.

​

• A log transformation of the intensities allows perception of a great deal of detail (on the same display system). See next slide.

9

10

When we do Fourier transforms later in class, we will make heavy use of log(r+1) type transformations.

Note: the 1 in log(r+1) is used for stabilization since log 0 is undefined. In some applications, if there are intensity values which are very tiny which need to be preserved, the 1 should be replaced by some ε value which is several times smaller than the smallest non-zero value.

Power-Law (Gamma) Transformation

• This is a transformation of the form s(r) = cr^γ.

​

• Here, c > 0 is a constant, r is the input intensity and γ > 0 is a constant for the power law.

​

• For fractional values of γ (between 0 to 1), a narrow range of low intensity values gets mapped to a wider range, and a range of high intensity values gets mapped to a narrower range.

​

• The effect is similar in principle to the log transformation, but we have more flexibility here as we can tune γ per our liking.

11

12

Note: The curves have been scaled to the range [0,L-1] after applying the gamma transformation. This has the effect of enhancing brighter intensities when gamma is more than 1.

Power-Law (Gamma) Transformation

• Many devices for display or printing work as per a power law.

​

• For example, a cathode ray tube (CRT) has an input intensity to output voltage mapping that uses a power law with γ between 1.8 to 2.5.

​

• Since the brightness of the pixel perceived on a CRT monitor depends on the voltage, it ends up producing images that are darker than intended to be (See next slide), i.e. the system displays cr^γ instead of r and hence brighter values dominate.

​

• Hence the image is pre-processed before inputting to the display system of the computer in the following manner: s(r) = (r/c) ^1/γ based on the gamma value (γ) of the display system.

​

• The output voltage values will be cs^γ = c[(r/c) ^1/γ ] ^γ = r as intended to be.

​

​

13

14

Power-Law (Gamma) Transformation

• The aforementioned transformation is called gamma correction.

​

• It is performed internally by the display system (oblivious to the end-user).

​

• Power-law transformations are also used for general “contrast enhancement” apart from gamma correction.

15

16

17

Contrast Stretching

• Low-contrast images occur due to poor lighting or low dynamic range of the camera sensor.

​

• Contrast stretching expands the intensity range in an image so that it spans the full intensity range of the display medium.

​

• For example suppose image I has range from [r_min, r_max] whereas the range of the device is [0,L].

​

• In the most basic form of contrast stretching, we perform: s(r) = (L-1)(r- r_min)/(r_max- r_min) so that r_min is mapped to 0 and r_max is mapped to L-1.

18

19

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

Ignore the yellowed out images

Bit-plane slicing

• Pixel values are usually 8-bit integers in photographic images.

​

• Thus at each pixel location, we have 8 bits defined.

​

• Such an image can be considered as being composed of eight 1-bit planes – each plane being a binary image.

20

21

Bit slicing

• From the previous example, we see that the first 4 most significant bits carry most of the image information.

​

• The 4 least significant bits carry less information and mostly subtle texture, even resembling noise.

​

• One can compress an image (with some loss!) by using the most significant bits from Q’ to Q, as follows:

​

​

22

k-th bit-plane image

23

Parameter selection in image enhancement

• All aforementioned techniques are intended to improve the (subjective) visual appeal of the image.

​

• The choice of various parameters in these techniques (eg: γin power-law) therefore depends on the user.

​

• It can be considered creative user choice.

​

​

24

Parameter selection in image enchancement

• Here image enhancement differs from image restoration (which we will study later in a different chapter).

​

• In the latter, we model a degradation phenomenon (eg: image blurring) and seek to inverts its effect.

​

​

25

Histogram equalization

• A method to improve image contrast

​

26

Low contrast image: histogram is narrow

High contrast image: histogram is more spread out!

Image has higher dynamic range – details more clearly visible

http://en.wikipedia.org/wiki/Histogram_equalization

Histogram equalization

• Seeks to apply an intensity transformation to the pixel intensities of a low-contrast image so as to convert it into one with a (nearly) uniform histogram.

​

• A uniform histogram is a histogram with Q bins where the probability mass in each bin is 1/Q.

​

• Principle: images with uniform histogram (or more spread out histograms) will have better contrast.

​

• Consider S = T(R), where R = random variable standing for intensity in the original image (assume r lies from 0 to L-1).

​

​

27

28

Histogram equalization

• Consider S = T(R), where R = random variable standing for intensity in the original image (assume R lies from 0 to L-1).

​

• S stands for the transformed random variable.

​

• The values that S and R acquire are denoted as s, r respectively.

​

• In particular, we are interested in T which satisfies the following two conditions:
• It should be strictly monotonically increasing
• If 0 <= r <= L-1, then 0 <= T(r) <= L-1 as well.

​

​

29

Segway: Probability refresher

• A random variable is the outcome of a randomized experiment.

​

• Eg: the number of heads appearing in N independent coin tosses, the time at which the first head appears in N independent coin tosses, etc.

​

• A random variable is denoted by upper case alphabets and its values with lower cases.

​

• A random variable may be discrete or continuous valued.

​

• It is characterized by its Probability density function (PDF) in continuous cases or its Probability mass function (PMF) in discrete cases.

​

• Denoted as p_R(r) – acquires values between 0 and 1 and integrates to 1.

​

30

Histogram equalization

• The monotonicity condition prevents artifacts due to intensity reversals.

​

• The second condition prevents change of the intensity range of the image.

​

​

31

Histogram equalization

• If R has a probability density p_R(r), then S has the following probability density:

​

​

​

• Consider the following transformation:

32

Cumulative distribution function (cdf) of R. It satisfies the monotonicity as well as range-based criteria mentioned earlier.

Called “transformation of random variables” (eg: https://www.math.arizona.edu/~jwatkins/f-transform.pdf )

This formula requires T(r) to be one-one and hence monotonically increasing (or decreasing)

Histogram equalization

• Then, we can show that the probability density of S is given as:

​

​

​

​

​

• Thus S has a uniform PDF, independent of the original PDF of R.

33

34

Histogram equalization

• Thus to perform this procedure, replace intensity value r in an image by the cumulative density (cdf) of r, times the maximum intensity level in the image minus 1!

35

http://www.udel.edu/biology/Wags/b617/digital/digital14.gif

36

Histogram equalization

• When you implement this, we change the integrals to a discrete summation.
• Thus we have:

​

​

​

• Thus you go to each pixel with value r_k, replace it by s_k, and repeat this for every intensity value r_k.

37

38

Histogram equalization

• The discrete histogram of an equalized image is more spread out. But it is not exactly uniform.
• Why?
• Quantization (rounding off) of intensity values
• Intensity range missing in the original image

​

39

40

https://towardsdatascience.com/histogram-matching-ee3a67b4cbc1

Histogram specification (also called matching)

• Apply an intensity transformation to an existing image so that the resultant image has some pre-specified histogram (perhaps corresponding to some other image).

​

• Equalization – special case of specification where the pre-specified histogram is the uniform histogram!

41

Histogram specification

• Let p_R(r) = histogram of the original image.
• Let p_Z(z) = pre-specified histogram.
• We apply the following transformation to r:

42

Histogram specification

• The pdf of the transformed image will be p_Z(z).

​

• Problem: the specified histogram needs to be chosen carefully – by trial and error.

43

Proof

44

Chain rule

Inverse function theorem

Recall that G^-1(T(r)) = z from the previous slide. Also G’(z) = (L-1)p_Z(z)

Histogram specification

• Let R be the image which needs to be transformed so that its histogram will now resemble the histogram of image Z.

​

• Compute p_R(r) and p_Z(z).

​

• Compute the functions s = T(r) and G(z). Then compute G^-1(s) .

​

• The histogram of the transformed image containing values G^-1(s) will have PDF p_Z(z).

45

46

Equalization

Histogram specification versus equalization

• Histogram equalization produces an image with a washed out appearance.

​

• Why?

​

• The original image histogram has a number of dark pixels, followed by an empty range, and a moderate number of bright pixels.

​

• HE maps a narrow interval of dark pixels to the brighter side of the grayscale range (see the associated transformation function on the previous slide), causing a washed out appearance.

47

Histogram specification versus equalization

• We want a target histogram which has a smoother transition from dark to light pixels.

​

• Hence the particular histogram chosen on the previous slides.

​

• Notice that this produces an image with improved appearance.

​

• Read example 3.9 of the textbook by Gonzalez.

48

49

https://towardsdatascience.com/histogram-matching-ee3a67b4cbc1

Example

50

Discrete Representation

51

The inverse of G is computed via a lookup table. In practice, one evaluates G at all integer values from 0 to L-1. We use the stored values of G to find the integer z_q such that G(z_q) is closest to s_k for each s_k value.

52

Trial and error

• HE and HS are both image enhancement methods.

​

• The choice of whether to use HE or HS, or which histogram to use for the specification in HS, are left to trial and error.

​

• There is no universal way of specifying a histogram for HS.

​

• In fact, HE or HS can be even be done at a “local” level – see next slide.

53

Local Histogram Equalization

• HE/HS – studied so far – are global methods, i.e. pixel intensities are modified by a transformation function affected by intensity values of the entire image.

​

• This approach sometimes does not enhance details in small areas of an image.

​

• Because the number of pixels in small image areas may have an insufficient influence on the global transformation.

​

• Solution: devise transformations based on local histograms.

54

Local (also called Adaptive) Histogram Equalization

• Define a symmetrical neighborhood N(x,y) around any pixel (x,y).

​

• The neighborhood is typically rectangular and has size (say) K x K.

​

• At each (x,y), compute the histogram of intensity values confined to N(x,y).

​

• Compute a transformation based on this local histogram using either HE or HS.

​

• Replace the intensity value of only (x,y) by the transformed intensity.

​

• Repeat for the entire image.

​

• See example on next slide – where global HE (GHE) is unable to enhance the local details in the black squares, but local HE (LHE) with a window size of 3 x 3 is.

55

56

Adaptive means same as local

https://towardsdatascience.com/histogram-equalization-5d1013626e64

57

Original Image

Global histogram-equalized image (see next slide for the results with local HE)

https://cromwell-intl.com/3d/histogram/

58

Local histogram-equalized image (size 100 x 100)

Local histogram-equalized image (size 50 x 50)

Ignore the fact that these are color images because color image HE is slightly different and we will study it later. Concentrate on the advantages of local over global.

Spatial Filters

Local Spatial Filters

• Change pixel values based on some weighted combination of pixels from a small (often rectangular) neighborhood around the pixel.
• This is represented as follows:

​

​

​

• The formula is applied for each pixel (x,y) in the pixel domain, using the input signal values.
• The output consists of a new “filtered” image g. In most cases, the new values do not replace the values in the original image f.
• The weights in w and the neighborhood size are both characteristic features of the filter.

​

60

Input image

Output image

Weights

Local Spatial Filters

• If the operation performed on the image pixels is linear, we call it a linear filter, otherwise we call it a non-linear filter.

​

• The filtered image is generated as the center of the filter (w(0,0)) visits each pixel (x,y) of the input image f.

61

Local Spatial Filters

• Note: in order to apply a filter to an image, it will usually require zero-padding to accommodate border areas.

62

Image

Filter mask

63

Correlation and convolution

• Local spatial filters involve one of two operations: correlation and convolution

​

• Correlation = moving a filter mask (2D array consisting of filter weights) over the image and computing the sum of products at each location

​

• This is identical to the previous mathematical formula.

64

Correlation and convolution

• Convolution = identical to correlation except that the filter is first rotated by 180 degrees (in both X and Y directions if it is in 2D) before moving over the image and computing sum of products.

65

66

For a filter of size m (in 1D), pad with m-1 zeros on either side

Correlation and convolution

• In the preceding example, we consider a 1D image with all zeros except a single one - called a unit impulse (or a Kronecker delta) function.

​

• The result of correlation with a filter with mask w is a reverse copy of w, centered at the location of the impulse.

​

• The result of convolution with a filter with mask w is a copy of w, centered at the location of the impulse.

​

• We will see the same result on the next slide in 2D.

​

67

68

Correlation and convolution

69

Correlation (denoted by an empty asterisk in the book)

Convolution (denoted by a solid black asterisk in the book)

Notes:

• The negative signs in the formula for the convolution, which represents flipping by 180 degrees. You would get similar formulae in 1D as well.
• Intuitively: correlation between a mask and a unit impulse signal yields a reversed copy of the mask. If we pre-rotate the mask by 180 degrees, we will now get a copy of the mask.
• If w were symmetric, then convolution and correlation yield identical results.

Correlation and convolution

• Convolution is a commutative operation!

​

​

​

​

​

​

​

​

​

• Repeat this exercise for correlation - it is NOT commutative (work out a counter-example).

70

After zero-padding w and f so that they have infinite extent. Note this doesn’t change the summation value.

Correlation and convolution

• Convolution is an associative operation!

​

​

• Correlation is not associative.

​

• The proofs of these results are much easier when using Fourier transforms (which we will study later).

71

Genesis of convolution

• Consider a system which takes in input signal x(t) and produces an output y(t) where y(t) = T[x(t)].

​

• T is the transformation that is executed by the system.

​

• Consider that T satisfies two conditions: linearity and time-invariance.

72

Genesis of convolution

• Consider that T satisfies two conditions: linearity and time-invariance.

​

​

​

• Let the signal x be a unit impulse δ(t) at 0 (Kronecker delta function).

73

Genesis of convolution

• The output of the system for a unit impulse input is called its impulse response h(t).

​

​

​

• It turns out that the response of a linear time-invariant (LTI) system to any input signal x is the convolution of the signal with the impulse response of the LTI system.

74

Genesis of convolution: Proof

75

Note: This specific example talks about 1D systems, but the same theory is applicable for 2D signals (where the shifts will be in 2D) or any higher dimension.

Genesis of correlation

• Correlation gives the similarity between a signal w and (a shifted version) of a signal f.

​

​

​

• Correlation is large for those shifts x for which w and shifted f have similar values, or even similar signs.

​

• It can be used for template matching.

76

77

Search for the template (Messi’s face) inside the larger image in the middle.

​

First row: On the left you have the cross-correlation map

​

Second row: On the left you have the normalized cross-correlation map

​

Why is the latter superior to the former? Notice the false match in the top row and the correct match in the second one.

https://docs.opencv.org/4.5.2/d4/dc6/tutorial_py_template_matching.html

Types of image noise

• Gaussian noise: addition of perturbations to an image, the perturbations are random numbers from the Gaussian distribution.

​

​

​

​

• Examples of Gaussian noise: Film grain noise, thermal noise in a camera

78

Symbol for Gaussian distribution with mean 0, standard deviation σ

Formula for Gaussian distribution with mean μ, standard deviation σ

Types of image noise

• Impulse noise: random large magnitude perturbations at a few pixels – example: flicker artifacts in old movies, or blotches in old photographs.
• Note: Gaussian noise typically affects all pixels but by a smaller magnitude.

79

Low σ, Gaussian

High σ, Gaussian

Impulse

Mean filter

• Replace the central pixel value by arithmetic mean of all surrounding pixel values.
• This acts as a smoothing filter.
• Image noise means random transition in the intensity values – these get attenuated by mean filter.

​

​

• But edges (boundaries between regions with different colors) get blurred as well ☹

80

Mean filter

• Implemented by convolving the image with a mask contains all values equal to 1/(2a+1)².

81

82

Filtered image

Original image

Noisy image

-Amount of smoothing is directly proportional to the width of the filter window.

-Repeated application of a mean filter will lead to a constant intensity image in the limit of infinite iterations

83

Weighted mean filter

• Assign different weights to each pixel: weigh pixels located closer to the center of the window more than those towards the edge of the window.

84

Decay parameter

Gaussian weights

Called the 2D Gaussian function

Weighted mean filter

• Also does not preserve edges.

​

• Note: we want all the weights to sum up to 1, otherwise the dynamic range of the image will not be preserved!

​

• What’s special about the Gaussian function? It gives more weights to image pixels near the center of the mask and lower weights to pixels farther away from the center.

85

Weighted mean filter

• Implemented by convolving the image with a mask containing weights of the form

86

Median Filter

• When there are wild outliers (such as impulse noise, also called salt and pepper noise) in the image, the mean filter gives a poor response.

​

• Common when there are data transmission errors, common in old film recordings.

​

• Median: more robust to large outliers in a given dataset.

​

• Median filters – give better preservation of features under impulse noise than mean filter.

​

• Equation for median filter:

​

​

87

Median versus mean: try out

• Compute mean and median of [1:100], i.e. an array of all integers from 1 to 100.
• Change one of the numbers in the array to a large value (like 10000)
• Re-compute the mean and median.
• The mean changes drastically, the median remains almost the same – as long as these drastic changes affect less than half the number of elements in the array.

​

88

Median Filter

89

Original

With salt and pepper noise

5 x 5 Mean filter

5 x 5 Median filter

Much better feature preservation with median filter

Comparing mean and median filter

• Median filter: MUCH better than mean filter for impulse noise.
• Preserves edges better than mean filter, but creates artifacts in smoother regions.

90

High σ, Gaussian

3 x 3 Median

3 x 3 Mean

http://homepages.inf.ed.ac.uk/rbf/HIPR2/median.htm

91

Works quite well even with strong impulses, provided most neighborhoods have no more than 50 percent of the pixels corrupted.

92

Median filter outputs – given filters of size 3 x 3 and 5 x 5

Original and noisy (impulse corrupted) images

http://what-when-how.com/embedded-image-processing-on-the-tms320c6000-dsp/non-linear-filtering-of-images-image-processing-part-1/

Linear and non-linear filters

• The mean filter is a linear filter:

​

​

• It is also space invariant.

​

• So it can be implemented as a convolution.

​

• The median filter cannot be implemented using a convolution as it is nonlinear.

​

​

93

Sharpening filter

• Smoothing filters – local average (integration)

​

• Sharpening filters – local intensity derivatives (differentiation) to enhance local distinctive features

94

95

sharpening

Sharpening filters

• Mean filter = local averaging filter

​

• Averaging causes blurring, equivalent to integration.

​

• Sharpening – uses intensity differentiation

​

• We will compute local image intensity derivatives.

​

• Greater the derivative magnitude = sharper change in intensity.

​

• Aim: to add back local derivative magnitudes to the low-contrast image!

96

Digital Derivative Operators (1D)

97

1^st derivative: Zero in constant areas, non-zero at the onset of an intensity ramp or intensity step, non-zero along a ramp

2^nd derivative: Zero in constant areas, zero along intensity ramps of constant slope, non-zero at the onset and end of an intensity ramp or step

Assume 1-D image for now, i.e. our image is f(x) instead of f(x,y)

98

From book by Gonzalez and Woods

Intensity discontinuity, also called step edge

From book by Gonzalez and Woods

Digital derivatives: first versus second

• Along a ramp (i.e. intensity change with constant slope), first derivative is a non-zero constant.

​

• Second derivative is zero along the ramp, except at the start and end!

​

• Second derivative is preferable for image sharpening! Many edges in images are ramp-like, in which case first derivative will give thick edges (undesirable), whereas second derivative gives thin edges two pixels wide (desirable).

​

• Second derivative changes sign midway at a step edge (zero crossing property).

​

100

Laplacian of an image

• Images are 2D – what kind of second derivative do we use? Along X or Y direction?

​

• We look at isotropic operators, i.e. operators whose output does not depend upon the direction of image intensity discontinuity.

​

• That is, we are interested in rotationally invariant 2^nd derivative operator for images.

101

Laplacian of an image

• A filter is said to be rotationally invariant if rotating the image and then applying the filter to the rotated image gives the same result as applying the filter to the image and then rotating the result.

102

Laplacian of an image

103

Laplacian operators: second operator is obtained by adding second derivatives along both the diagonals, to the first operator

Rotationally symmetric operator (in the continuous domain)

Laplacian of an image

• The two masks on the previous slide are applied point-wise to the entire image.

​

• For image smoothing as well, we applied point-wise masks.

​

• But here the mask weights sum to 0. In smoothing the weights summed to 1.

104

Laplacian for image sharpening

• A Laplacian de-emphasizes regions with slowly varying intensities.

​

• Highlights intensity discontinuities in an image.

​

• Subtracting the Laplacian from an image yields sharpening:

105

Laplacian for image sharpening

• Subtracting the Laplacian from an image yields sharpening:

​

​

• Why is this the case?
• Observe that the Laplacian changes sign across an edge.
• For an edge with decreasing intensity, it is negative and positive at the beginning and end of the edge respectively.
• Hence deducting it sharpens the edge. See diagram on next slide

106

107

From book by Gonzalez and Woods

Intensity discontinuity, also called step edge

sharpened edge: 6-(-1) = 7 at the beginning and 1-1=0 at the end.

Laplacian for image sharpening

• In some books, the following masks are used for the Laplacian (they are equal to the earlier masks multiplied by -1):

​

​

​

​

• With such masks, the sharpening filter is expressed as:

108

109

From Book by Gonzalez and woods

Sharpening filter: convolution mask

110

111

https://en.wikipedia.org/wiki/Unsharp_masking

Unsharp masking

• Smooth the original image f using a filter g yielding f1 = f*g.
• Compute the difference between original and smoothed image, i.e. f2 = f-f*g.
• Add it back to the original image to yield f3 = f + k f2 for some scalar k.
• You get a sharpened image.
• If k = 1, we call it unsharp masking (i.e. removing blurred components)
• If k > 1, we term it highboost filtering.

112

113

From Book by Gonzalez and woods

114

Derivative filters: dealing with noise

• Derivative filters in general exacerbate noise.

​

• Hence they are applied in conjunction with smoothing filters.

​

• For example, the Sobel filters smooth in the X and Y directions before performing differencing.

​

115

116

117

Grayscale image

Gradient map (after applying Sobel filter)

X-Sobel filter output

Y-Sobel filter output

Sobel filters: outer products

118

• Filter which can be represented in this outer-product form are called separable filters.

​

• Convolution between a M x N image and a K x L mask has a complexity of O(MNKL), which can be expensive.

​

• But with separable filters, this reduces to O( MN [L+K]) since you express the K x L mask as the outer product of K x 1 mask and a 1 X L mask. Applying the 1 X L mask requires O(MNL) operations whereas applying the K x 1 mask requires O(MNK) operations. Here, we are using the associativity of convolution.

Other examples of separable filters

119

Laplacian of Gaussians

• The image Laplacian is susceptible to noise.

​

• Hence the image f is first convolved with a Gaussian of std. dev. σ to smooth some of the noise, and then the Laplacian is applied, i.e. we compute: .

​

• Equivalent to , due to associative and commutative nature of convolution.

​

​

120

Laplacian of Gaussians

• For efficiency, the Laplacian of Gaussians mask is pre-computed.

​

• The formula for this is:

​

​

121

122

Discrete approximation with σ=1

https://homepages.inf.ed.ac.uk/rbf/HIPR2/log.htm

123

https://homepages.inf.ed.ac.uk/rbf/HIPR2/log.htm

LoG filter of size 7 x 7 with σ=1

Bilateral Filters

Bilateral filter versus mean/median filter

• Mean filter: does not preserve image edges

​

• Median filter: too may not always preserve edges

​

• To preserve edges well, we may need a non-linear filter

​

• Bilateral filter is one such

125

126

Gaussian blur

The median filter will assign the median of the values in the red neighborhood to the corner pixel – and will convert the otherwise black pixel to a yellow one (as most of the pixels in the neighborhood are yellow). Thus the median filter does not always preserve edges well!

https://people.csail.mit.edu/sparis/bf_course/course_notes.pdf

Bilateral Filter

127

Intensity-based weights

Space-based weights

Effect of parameter σ_s:

• As it increases, a larger and larger neighborhood of values around p =(x,y) will contribute to the averaging (more noise reduction but possible contribution from dissimilar regions)
• As it shrinks towards 0, fewer and fewer neighbors of the central pixel p will contribute to the averaging

Effect of parameter σ_r:

• For moderate values, only intensities close to I(p) will affect the averaging
• For larger values, the bilateral filter will begin to resemble a Gaussian filter
• Features or edges with intensity difference less than σ_r will be blurred, others will be preserved

128

https://people.csail.mit.edu/sparis/bf_course/course_notes.pdf

129

https://people.csail.mit.edu/sparis/bf_course/course_notes.pdf

130

https://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/MANDUCHI1/Bilateral_Filtering.html

Bilateral filter implementation

• A bilateral filter cannot be implemented as a convolution for two reasons.

​

• The weights themselves depend on the intensity values of the original (noisy) image.

​

• The weights change from pixel to pixel.

​

• Thus the filter is neither linear nor space-invariant

131

Cross-bilateral filter

• Consider a noisy no-flash and a non-noisy flash image of the same scene.

​

• The latter has changes of color in the scene due to the flash

​

• Cross-bilateral filter: apply a bilateral filter to the n0-flash image with intensity/range weights from the flash image

132

133

http://hhoppe.com/flash.pdf

134

http://hhoppe.com/flash.pdf

Staircase effect of the bilateral filter

135

https://people.csail.mit.edu/sparis/bf_course/course_notes.pdf

Staircase effect of the bilateral filter

• The bilateral filter has a tendency to create artificial edges in homogenous regions.

​

• Why? Look at a 1D example – next slide.

​

• The signal is locally concave (line joining two points is above the curve).

​

136

137

https://hal.archives-ouvertes.fr/hal-00271143/document

Staircase effect of the bilateral filter

• We are looking at pixels (y) whose intensities are within a range of u(x)-h to u(x)+h where h is approximately 3σ_r.

​

• But the number of points such that u(x)-h < u(y) <= u(x) is more than the number of points for which u(x) < u(y) <= u(x)+h.

​

• So effectively, the filtered intensity value is less than the original intensity value u(x).

​

• For a locally convex signal (line joining two points is below the curve), the filtered intensity value is more than the original value u(x).

​

• At points where locally convex and locally concave parts meet (called inflection points), you get an artificial increase in contrast – causing the staircase effect.

​

138

Bokeh Filters

Bokeh filters

• Bokeh is an effect in photography where the main object is in focus and the rest of the scene is severely defocussed.

​

• This produces a pleasing aesthetic effect.

140

141

https://en.wikipedia.org/wiki/Bokeh#/media/File:Josefina_with_Bokeh.jpg

142

https://en.wikipedia.org/wiki/Bokeh

Bokeh

• The amount of blur is depth dependent.

​

• Certain areas are in focus, the rest are not and lie in a “circle of confusion”.

​

• The depth of field of a camera is the region where the circle of confusion has a size less than the resolution of the human eye.

​

​

143

144

Shallow depth of field

https://en.wikipedia.org/wiki/Bokeh

Bokeh

• Bokeh characteristics may be quantified by examining the image's circle of confusion. In out-of-focus areas, each point of light becomes an image of the aperture, generally a more or less round disc. 

​

• Bokeh can be simulated by convolving the image with a kernel that corresponds to the image of an out-of-focus point source taken with a real camera.

​

• Unlike conventional convolution, this convolution has a kernel that depends on the distance of each image point and – at least in principle – has to include image points that are occluded by objects in the foreground.^[28] 

​

• Also, bokeh is not just any blur. To a first approximation, defocus blur is convolution by a uniform disk, a more computationally intensive operation than the "standard" Gaussian blur; the former produces sharp circles around highlights whereas the latter is a much softer effect.

​

​

145

146

From left to right: an original photo with no bokeh or blur; the same photo with synthetic bokeh effect applied to its background; the same photo with Gaussian blur applied to its background

https://en.wikipedia.org/wiki/Bokeh

147

Source: https://web.ics.purdue.edu/~tmcgraw/papers/dof_mcgraw_2014.pdf