Nima Kalantari

CSCE 448/748 – Computational Photography

​

Blending

Many slides from Alexei A. Efros, James Hayes, Rob Fergus

Goal

• Seamless copy of content from one image to another

Gradient manipulation

• Observation:
• Human visual system is sensitive to gradient
• Gradient encodes edges and local contrast
• Idea:
• Do your editing in the gradient domain
• Reconstruct image from gradient
• Based on Perez et al. “Poisson Image Editing” SIGGRAPH 2003

Slide credit: F. Durand

Goal

Gradient domain cloning

Gradient blending (1D) – Example 1

source

target

Gradient blending (1D) – Example 1

source

target

Gradient blending (1D) – Example 2

source

target

Gradient blending (1D) – Example 2

It is impossible to faithfully preserve the gradients

source

target

Membrane interpolation

• Laplace equation (a.k.a. membrane equation)

​

​

Slide credit: F. Durand

1D example: minimization

• Minimize derivatives to interpolate

​

​

​

​

Slide credit: F. Durand

1D example: minimization

• Minimize derivatives to interpolate

​

​

​

​

*

=

Slide credit: F. Durand

1D example

• Minimize derivatives to interpolate
• Pretty much says that second derivative should be zero
• (-1 2 -1) is a second derivative filter

=

*

=

Slide credit: F. Durand

Membrane interpolation

• Laplace equation (a.k.a. membrane equation)

​

​

Slide credit: F. Durand

Seamless Poisson cloning

• Given vector field v (pasted gradient), find the value of f in unknown region that optimize:
• Previously, v was null

Pasted gradient

Slide credit: F. Durand

Discrete 1D example: minimization

​

​

​

​

​

Slide credit: F. Durand

Discrete 1D example: minimization

​

​

​

​

​

*

=

Slide credit: F. Durand

Discrete 1D example: minimization

​

​

​

​

​

*

=

=

Slide credit: F. Durand

Simple 2d example

output, x

What properties do we want x to have?

source, s

target, t

mask

Simple 2d example

1. For unmasked pixels, x_i = t_i
2. For masked pixels, we want the laplacian at x_i to match the laplacian at s_i

output, x

source, s

target, t

mask

Simple 2d example

source, s

target, t

mask

Pixel indexing

output, x

x₁ = t₁

x₂ = t₂

x₃ = t₃

x₄ = t₄

4x₅ – x₄ – x₂ – x₆ – x₈ = 4s₅ – s₄ – s₂ – s₆ – s₈

x₆ = t₆

x₇ = t₇

4x₈ – x₇ – x₅ – x₉ – x₁₁ = 4s₈ – s₇ – s₅ – s₉ – s₁₁

x₉ = t₉

x₁₀ = t₁₀

x₁₁ = t₁₁

x₁₂ = t₁₂

Laplacian

Simple 2d example

x₁ = 0.2

x₂ = 0.7

x₃ = 0.9

x₄ = 0.5

4x₅ – x₄ – x₂ – x₆ – x₈ = 0.2

x₆ = 0.9

x₇ = 0.2

4x₈ – x₇ – x₅ – x₉ – x₁₁ = -1.6

x₉ = 0.8

x₁₀ = 0.2

x₁₁ = 0.7

x₁₂ = 0.9

source, s

target, t

mask

Pixel indexing

output, x

Laplacian

Simple 2d example

4x₅ – 0.5 – 0.7 – 0.9 – x₈ = 0.2

4x₈ – 0.2 – x₅ – 0.8 – 0.7 = -1.6

4x₅– x₈ = 2.3

4x₈ – x₅ = 0.1

x₅ = 0.62

x₈ = 0.18

In this simple case, we could solve for everything by hand.

source, s

target, t

mask

Pixel indexing

output, x

Laplacian

x₁ = 0.2

x₂ = 0.7

x₃ = 0.9

x₄ = 0.5

4x₅ – x₄ – x₂ – x₆ – x₈ = 0.2

x₆ = 0.9

x₇ = 0.2

4x₈ – x₇ – x₅ – x₉ – x₁₁ = -1.6

x₉ = 0.8

x₁₀ = 0.2

x₁₁ = 0.7

x₁₂ = 0.9

Simple 2d example

*

=

A

x

b

source, s

target, t

mask

Pixel indexing

output, x

Laplacian

Simple 2d example

*

=

A

x

b

source, s

target, t

mask

Pixel indexing

output, x

Laplacian

A * x = b

A^-1A * x = A^-1b

x = A^-1b

Simple 2d example

*

=

source, s

target, t

mask

Pixel indexing

output, x

Laplacian

A

x

b

Simple 2d example

output, x

source, s

target, t

mask

Pixel indexing

Laplacian

x

Example

source

target

mask

no blending

gradient domain blending

What’s the difference?

no blending

gradient domain blending

-

=

Perez et al. SIGGRAPH 03

Perez et al. SIGGRAPH 03

Mixing gradients

Source

Average

Mixed