1 of 97

1

EC 500 A1: �Image Processing – 3

2 of 97

Recap: Fourier transform for frequency profile

Time

DFT: frequency

Spatial

DFT: frequency

Amplitude: A

 

3 of 97

Recap: DFT, IDFT, and “profile manipulation”

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

DFT

Remove high frequencies

IDFT

4 of 97

Recap: Phase

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Amplitude: A

 

5 of 97

Recap: Phase

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Amplitude: A

 

6 of 97

Recap: Convolution and Fourier

  •  

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

7 of 97

Recap: Convolution and Fourier

 

 

 

 

DFT

8 of 97

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Spatial

DFT

Filters (and convolution) can do more!

9 of 97

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Spatial

DFT

10 of 97

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Spatial

DFT

11 of 97

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Spatial

DFT

12 of 97

Today

  • Image filters (Chapters 17 & 18)
  • Image sampling and aliasing (Chapter 20)
  • Downsampling & upsampling (Chapter 21)
  • Image pyramid (Chapter 23)

​

​

12

13 of 97

Image processing by frequency filtering

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

14 of 97

Today

  • Image filters (Chapters 17 & 18)
  • Image sampling and aliasing (Chapter 20)
  • Downsampling & upsampling (Chapter 21)
  • Image pyramid (Chapter 23)

​

​

14

15 of 97

What’s wrong? How to resolve?

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

16 of 97

“Linear” blur filters

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

17 of 97

Box filters

17

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

18 of 97

Box filters

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

19 of 97

Box filter and its frequency response

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

20 of 97

Box filter and its frequency response

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

21 of 97

21

Questions?

22 of 97

Binomial filters

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

23 of 97

2D binomial filters

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

24 of 97

Example

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

25 of 97

Wider binomial; filtering more high frequencies

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

26 of 97

26

Questions?

27 of 97

Human vision systems are filters

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

28 of 97

Fun figures in MIT Museum

29 of 97

Fun figures in MIT Museum

30 of 97

30

Questions?

31 of 97

How about this filter?

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

32 of 97

Image derivatives

  • Goal:

​

​

  • Continuous form:

​

​

  • Discretized form:

​

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

33 of 97

We can achieve this by convolution!

34 of 97

Frequency responses of the filters

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

35 of 97

Image Laplacian

36 of 97

Approximated Laplacian filter

37 of 97

Approximated Laplacian filter

38 of 97

Today

  • Image filters (Chapters 17 & 18)
  • Image sampling and aliasing (Chapter 20)
  • Downsampling & upsampling (Chapter 21)
  • Image pyramid (Chapter 23)

​

​

38

39 of 97

Images are “discrete” representations of the “continuous” world

40 of 97

Discretization requires sampling

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Aliasing:

Confusion about original frequencies

​

Can we recover?

​

41 of 97

Example

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

42 of 97

Example

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

43 of 97

More examples

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

44 of 97

More examples

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

45 of 97

More examples

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

46 of 97

More examples

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

47 of 97

Do we see aliasing in our life?

[https://www.youtube.com/watch?v=eQUPz2vIGgA]

48 of 97

48

Questions?

49 of 97

Sampling theorem

Original signal

Delta train

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Sampled signal

=

50 of 97

Multiplication in time means [What] in frequency?

  • [What] = convolution

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

​

What is the difference between low/high sampling rates?

​

51 of 97

Frequency responses of the sampled signal

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

52 of 97

Frequency responses of the sampled signal

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

53 of 97

Frequency responses of the sampled signal

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Aliasing

54 of 97

54

Questions?

55 of 97

Time

Frequency

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Reconstruction

56 of 97

Time

Frequency

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Reconstruction

57 of 97

Time

Frequency

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Reconstruction

58 of 97

Time

Frequency

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Reconstruction

59 of 97

59

Questions?

60 of 97

Reconstruction

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Time

61 of 97

Reconstruction

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Time

62 of 97

Reconstruction

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Time

Frequency

63 of 97

Reconstruction

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Time

Frequency

64 of 97

Reconstruction

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Time

=

Frequency

65 of 97

What are often used in practice for reconstruction?

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

66 of 97

Today

  • Image filters (Chapters 17 & 18)
  • Image sampling and aliasing (Chapter 20)
  • Downsampling & upsampling (Chapter 21)
  • Image pyramid (Chapter 23)

​

​

66

67 of 97

Decimation

  • Formula:

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

68 of 97

Decimation in the Fourier domain

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

2D delta train

DFT of delta train

69 of 97

Decimation in the Fourier domain

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

2D delta train

DFT of delta train

70 of 97

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

71 of 97

Anti-aliasing filters

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

72 of 97

Anti-aliasing filters

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

73 of 97

Anti-aliasing filters

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Binomial filter

74 of 97

Anti-aliasing filters

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

75 of 97

W/o and W/ anti-aliasing: blurring vs. aliasing

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

w/o

w/

76 of 97

Upsampling

77 of 97

Upsampling

  • Nearest neighbor

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

78 of 97

Upsampling

  • (Bi-)linear interpolation

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

79 of 97

Upsampling

  • Upsampling by convolution

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

80 of 97

Upsampling

  • Upsampling by convolution

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

81 of 97

Upsampling

  • Upsampling by convolution

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

82 of 97

Upsampling

  • Upsampling by convolution

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

83 of 97

Upsampling

  • Upsampling by convolution

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

84 of 97

Today

  • Image filters (Chapters 17 & 18)
  • Image sampling and aliasing (Chapter 20)
  • Downsampling & upsampling (Chapter 21)
  • Image pyramid (Chapter 23)

​

​

84

85 of 97

Besides translation, scale is another factor to be invariant

85

86 of 97

Typically, a single filter can only detect one size

86

87 of 97

However, we can resize the image

87

88 of 97

Image pyramid

88

89 of 97

Image pyramid: down-sampling

Convolution

Subsampling

90 of 97

Image pyramid : Gaussian pyramid

  • Create a sequence of down-sampled images

​

Down-sample

Down-sample

Down-sample

91 of 97

Can we recover?

91

92 of 97

Up-sampling

Convolution

Double-size

93 of 97

Residual image

Down-sample

Up-

sample

-

=

94 of 97

Successful up-sampling needs residual images

Up-

sample

=

+

95 of 97

Gaussian pyramid

95

96 of 97

Laplacian pyramid

96

97 of 97

Image reconstruction

Up-sample

+

=

Up-sample

+

=

Up-sample

……