1 of 56

Find edges (white pixels mean edges)

All edges

Contact edges

Vertical edges

Horizontal

You need to find edge locations!

2 of 56

Recover 3D

Y: 3D height

Z: 3D depth

(i, j)

y: 2D vertical

 

 

3 of 56

Estimating Y[i, j]: cues from vertical edges

 

 

 

4 of 56

CSE 5524: �Image processing – 4

4

5 of 56

Today

  • Review: Fourier analysis (chapter 16)
  • Image filters (chapters 17 & 18)

​

​

​

​

5

6 of 56

What is convolution?

7 of 56

What is convolution?

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

8 of 56

The convolution computation

  •  

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

9 of 56

What is Fourier transform?

10 of 56

Fourier transform

10

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

11 of 56

Fourier transform

11

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

12 of 56

Fourier transform

12

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

A signal can be represented by a linear combination of “periodic” functions!

13 of 56

Periodic functions (sine and cosine) in 2D

13

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

14 of 56

Periodic functions (sine and cosine) in 2D

  • What is the range of u and v?

15 of 56

Periodic functions (sine and cosine) in 2D

  • What is the range of u and v?
    • 0 : N-1, 0 : M-1

16 of 56

Periodic functions (sine and cosine) in 2D

  • What is the range of u and v?
    • 0 : N-1, 0 : M-1
    • -N/2 : N/2 – 1, -M/2 : M/2 – 1
    • -(N – 1)/2 : (N – 1)/2, -(M – 1)/2 : (M – 1)/2

17 of 56

Frequencies and Phases

17

m

n

m

 

 

Fourier-like

transform

Amplitude

Phase

u

u

v

v

u

u

v

v

18 of 56

How about more complicated images?

19 of 56

Discrete Fourier transform (DFT)

Input image

Periodic functions (bases)

Pixel position

Frequency

Frequency response

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

For every (u, v), the computation is an inner product!

20 of 56

Inverse discrete Fourier transform (IDFT)

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Frequency response

Original image

What is “summed” is different from DFT

21 of 56

Example

21

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

22 of 56

For brevity

  • I will use “frequency” responses or domains for things after DFT

​

  • I will use “spatial or time” responses or domains for things before DFT or after IDFT (inverse DFT)

23 of 56

** Complex exponential **

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

j means “imaginary” part in complex values

24 of 56

Dive into DFT

 

 

 

25 of 56

Dive into DFT

 

 

 

 

 

26 of 56

Visualization

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Spatial

DFT: frequency

Amplitude: A

 

27 of 56

Visualization – images at different frequencies

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

28 of 56

Why symmetric?

  •  

 

 

29 of 56

Why symmetric?

  •  

 

 

 

 

 

 

30 of 56

Convolution and Fourier

  • Convolution in time/spatial domains:

​

  • Multiplication in frequency domains:

​

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

31 of 56

Convolution and Fourier

  •  

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

 

32 of 56

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Spatial

DFT

33 of 56

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Spatial

DFT

34 of 56

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Spatial

DFT

35 of 56

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Spatial

DFT

36 of 56

Modulation

  • Convolution in time/space equals multiplication in frequency
  • Multiplication in time/space equals convolution in frequency

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

37 of 56

Modulation

  • Convolution in time/space equals multiplication in frequency
  • Multiplication in time/space equals convolution in frequency

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

38 of 56

Modulation, i.e., multiplication in time/space

  • Convolution in time/space equals multiplication in frequency
  • Multiplication in time/space equals convolution in frequency

​

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

39 of 56

Fun editing by mixing phases

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Amplitude: A

 

40 of 56

Fun editing by mixing phases

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Amplitude: A

 

41 of 56

Today

  • Review: Fourier analysis (chapter 16)
  • Image filters (chapters 17 & 18)

​

​

​

​

41

42 of 56

Recap: Convolution and Fourier

  •  

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

43 of 56

What’s wrong? How to resolve?

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

44 of 56

“Linear” blur filters

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

45 of 56

Box filters

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

46 of 56

Box filters

46

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

47 of 56

Box filter and its frequency response

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

48 of 56

Binomial filters

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

49 of 56

2D binomial filters

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

50 of 56

Example

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

51 of 56

How about this filter?

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

52 of 56

Image derivatives

  • Goal:

​

​

  • Continuous form:

​

​

  • Discretized form:

​

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

53 of 56

We can achieve this by convolution!

54 of 56

Frequency responses of the filters

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

55 of 56

Image Laplacian

56 of 56

Approximated Laplacian filter