1 of 43

Nima Kalantari

CSCE 448/748 – Computational Photography

​

Pyramids

Many slides from Alexei A. Efros, James Hayes, Rob Fergus

2 of 43

Fourier domain

  • Basis functions:

Tells you what is in the image….

​

… but not where it is

………

………

3 of 43

Spatial Domain

  • Basis functions:

…………..

Tells you where things are….

​

… but no concept of what it is

4 of 43

Image Analysis

  • Want representation that combines what and where
    • Image Pyramids

​

5 of 43

Practical uses

  • Searching
  • Image blending
  • Compression
    • Capture important structures with fewer bytes
  • Denoising
    • Model statistics of pyramid sub-bands

​

​

​

6 of 43

Outline

  • Gaussian
  • Laplacian
  • Image blending

7 of 43

Gaussian Pyramid

http://www-bcs.mit.edu/people/adelson/pub_pdfs/pyramid83.pdf

8 of 43

Image half-sizing

  • How to generate a half-sized version?

​

9 of 43

Image sub-sampling

  • Throw away every other row and column to create a 1/2 size image
    • Image sub-sampling
  • Problem
    • Aliasing

​

1/4

1/8

Slide by Steve Seitz

1/2

10 of 43

Extreme example

11 of 43

Aliasing

  • High frequency signal sampled slowly
    • Sample more often
      • Cannot be done; goal is to downsize
    • Get rid of high frequencies
      • Filter the image and then subsample

12 of 43

Gaussian (lowpass) pre-filtering

1/4

1/8

1/2

Slide by Steve Seitz

13 of 43

Direct subsampling

1/4

1/8

1/2

Slide by Steve Seitz

14 of 43

Extreme case

15 of 43

Gaussian pyramid

A Multiresolution Spline with Application to Image Mosaics, Burt and Adelson, SIGGRAPH 83

16 of 43

Gaussian pyramid construction

  • Repeat
    • Filter
    • Subsample
  • Until minimum resolution reached
    • can specify desired number of levels (e.g., 3-level pyramid)

​

filter mask

Slide by Steve Seitz

17 of 43

Gaussian pyramid construction

  • Repeat
    • Filter
    • Subsample
  • Until minimum resolution reached
    • can specify desired number of levels (e.g., 3-level pyramid)

​

filter mask

Slide by Steve Seitz

18 of 43

Gaussian pyramid

  • A bar in the pyramid
    • Big images, a hair on nose
    • Smaller images, a stripe
    • Smallest image, zebra’s nose
  • The whole pyramid is only 4/3 of the original image

​

Figure from David Forsyth

19 of 43

What are they good for?

  • Improve Search
    • Search over translations
      • Classic coarse-to-fine strategy
      • Project 1!
    • Search over scale
      • Template matching
      • E.g., find a face at different scales

20 of 43

Outline

  • Gaussian
  • Laplacian
  • Image blending

21 of 43

Laplacian pyramid algorithm

L1

L2

L3

L4 = G4

22 of 43

23 of 43

24 of 43

Collapsing the pyramid

L1

L2

L3

L4 = G4

+

+

+

25 of 43

Outline

  • Gaussian
  • Laplacian
  • Image blending

26 of 43

Image Blending

27 of 43

Feathering

0

1

0

1

+

=

Iblend = αIleft + (1-α)Iright

​

28 of 43

Affect of Window Size

0

1

left

right

0

1

29 of 43

Affect of Window Size

0

1

0

1

30 of 43

Good Window Size

0

1

“Optimal” Window: smooth but not ghosted

31 of 43

However…

  • One window may not work for every region

Small window

Large window

32 of 43

Pyramid Blending

0

1

0

1

0

1

Left pyramid

Right pyramid

blend

33 of 43

Pyramid Blending

A Multiresolution Spline with Application to Image Mosaics, Burt and Adelson, SIGGRAPH 83

34 of 43

laplacian

level

4

laplacian

level

2

laplacian

level

0

left pyramid

right pyramid

blended pyramid

35 of 43

Pyramid Blending

A Multiresolution Spline with Application to Image Mosaics, Burt and Adelson, SIGGRAPH 83

36 of 43

Blending Regions

  • Input: Images X and Y as well as a mask A
  • Output: blended image

X

Y

A

blended

37 of 43

Laplacian Pyramid/Stack Blending

  • General Approach:
    1. Build Laplacian pyramid LX and LY from images X and Y
    2. Build a Gaussian pyramid GA from the binary alpha mask A
    3. Form a combined pyramid LBlend from LX and LY using the corresponding levels of GA as weights:
      • LBlend(i, j) = GA(i, j) * LX(i, j) + (1 - GA(i, j)) * LY(i,j)
    4. Collapse the LBlend pyramid to get the final blended image

38 of 43

Laplacian Pyramid/Stack Blending

Laplacian LX

Laplacian LY

Gaussian GA

Blended

Collapse

39 of 43

Horror Photo

© david dmartin (Boston College)

40 of 43

Simplification: Two-band Blending

  • Brown & Lowe, 2003
    • Only use two bands: high freq. and low freq.
    • Blends low freq. smoothly
    • Blend high freq. with no smoothing: use binary alpha

​

41 of 43

2-band “Laplacian Stack” Blending

Low frequency (λ > 2 pixels)

High frequency (λ < 2 pixels)

42 of 43

Linear Blending

43 of 43

2-band Blending