CSE 163

Convolutions�

Suh Young Choi�

🎶 Listening to: Inception soundtrack

💬 Before Class: Buffalo buffalo buffalo buffalo Buffalo buffalo buffalo. Weird how that’s a grammatically correct sentence.

Announcements

Project Report, Code, and Presentation now available

• Full specs can be found on the course website
• Report/Code submitted together in Gradescope
• Presentation submitted as a video link in Google Form
• Hard limits on length of reports/videos!!

​

Resubmission cycle is open, closes Tuesday (8/12)

• Late submission for HW 5 accepted during this cycle
• Note that late submissions will get late feedback!

​

HW5 due on Thursday (last one of the quarter!)

​

2

Remaining Due Dates

To assist with planning:

3

This Time

• Convolutions

Last Time

• Numpy arrays
• Broadcasting
• Images

4

Images as

Matrices

​

​

5

Grey-scale images can be represented as matrices.

Grey-scale: 255

Grey-scale: 0

data = imageio.imread(‘...’)�

data[rows, columns] = #

Color Images

When you overlap each color channel, it creates a picture we are used to seeing.

• Pixels on your monitor let out specified R/G/B light

6

data[rows, columns, channels] = #

Convolution

When wanting to use “local” information, we commonly use a sliding window approach (i.e. a convolution)

​

Move the sliding window across the image, and compute the sum of the element wise product of the window (kernel) and image

7

Image

Kernel

Convolution Example

8

Convolutions!

an unconvoluted example

Image

Kernel

Image

Output

Kernel

Image

Output

2

1

Kernel

2

1

Image

Output

Kernel

2

1

Image

Output

26

Kernel

2

1

Image

Output

26

Kernel

2

1

Image

Output

26

2

1

Kernel

2

1

2

1

Image

Output

26

Kernel

2

1

2

1

Image

Output

26

23

Kernel

2

1

2

1

Image

Output

26

23

Kernel

2

1

2

1

Image

Output

26

23

2

1

17

Kernel

2

1

2

1

Image

Output

26

23

2

1

17

Kernel

2

1

2

1

Image

Output

26

23

2

1

17

14

Kernel

2

1

2

1

Image

Output

26

23

2

1

17

14

Kernel

2

1

2

1

Image

Output

26

23

2

1

17

14

8

Kernel

2

1

2

1

Image

Output

26

23

2

1

17

14

8

Kernel

2

1

2

1

Image

Output

26

23

2

1

17

14

8

5

Kernel

2

1

2

1

Image

Output

26

23

2

1

17

14

8

5

Kernel

How many times can we fit the kernel into the image?

“

How many times can we fit the kernel into the image?

“

How many times can we fit the kernel into the image?

2

“

How many times can we fit the kernel into the image?

2

“

How many times can we fit the kernel into the image?

2

3

“

How many times can we fit the kernel into the image?

2

3 x

“

General formula:

image

(3x3)

kernel

(1x2)

output

(3x2)

(image_height – k_height + 1, image_width – k_width + 1)

“

More Generic Formula for Result Size

35

Image

Kernel

width

height

k_height

k_width

(0, 0)

k[0]

k[1]

(1, 0)

…

(width - k_width, 0)

k[width - k_width]

Total of (width - k_width + 1) kernels because of zero based indexing!

​

So the shape of the result will be (height - k_height + 1, width - k_width + 1)

Common Kernels

What do the numbers in the kernel do?

36

Identity

Edge Detection

Sharpen

Box Blur

Image Classification

For a really long time, image classification was done by painstakingly crafting these features (like edge detectors), by hand.

​

This kind of worked, but we quickly hit our peak using this method.

​

Then came the buzz-word… deep learning

37

Image Classification

• Is this a solved problem?
• We get pretty decent error rates on challenges like ImageNet
• What we can’t do
• Sometimes can’t generalize to other real-world datasets
• Adversarial attacks

38

Before Next Time

• Complete Lesson 20
• Remember not for points, but do go towards Checkpoint Tokens
• HW5 due tomorrow
• Go to section!

Next Time

• Machine learning with images

​

39