1 of 27

1

Lecture 1:

Introduction

​

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

2 of 27

2

Who we are

  • Instructor: Erik Learned-Miller: Office hour(s): Thurs. 3-4.
  • TA: Hang Su
    • Office hours: Mon: 4-5?, Tues: 4-5?, Thurs: 4-5?

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

3 of 27

3

Course web page

  • Go to my web page � (Google “Learned-Miller”)
  • Go to my teaching page
  • Click on top link

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

4 of 27

4

697L: Deep Learning

​

  • What is this class about?
    • Previous related classes
      • Last offering of 697L (Mahadevan)
      • 591NR: Neural Networks: An introduction (Siegelmann)
    • This class:
      • Highly applied: Lots of programming, backpropagation, parameter tuning, etc.
      • Better name would be “Deep neural networks”

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

5 of 27

5

697L: Deep Learning

​

  • Topics: first half of the class.
    • Intro to supervised learning with K-Nearest neighbors
    • Support vector machines
    • Logistic regression for classification
    • Feed forward neural nets
      • Backpropagation
      • Batch normalization
      • Drop-out
      • Speed optimizations
    • Convolutional neural nets

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

6 of 27

6

697L: Deep Learning

  • Balance of theory vs. practice
    • Heavily tilted toward practice.
    • Examples:
      • Regularization will be used, but not much theory of it.
      • No proofs of convergence or optimality
    • Instead:
      • Develop applications “from scratch”
      • Build “layered” architectures from scratch so new models can be easily assembled
      • Implement popular add-ons such as batch normalization
      • Learn techniques for training and setting hyperparameters.

​

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

7 of 27

7

697L: Deep Learning

​

  • Applications
    • Mostly vision: Object recognition in particular.
    • However, can easily be applied to other domains.
      • You will learn what you need to know to apply neural nets broadly.

​

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

8 of 27

8

697L: Deep Learning

​

  • What this course is not:
    • General course on machine learning
    • General course on graphical models
    • Not even a general class on deep learning!!!
      • No Bayes Nets
      • No restricted Boltzmann machines or �deep Boltzmann machines
    • Not a computer vision survey class
      • No tracking, stereo, segmentation, depth estimation, etc., etc.

​

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

9 of 27

9

697L: �

​

Some motivational videos

​

​

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

10 of 27

10

Collaboration

  • This is a hard class, you may need to talk to your fellow students.
    • Things to avoid (be prepared to declare you have not done):
      1. Do not write code together.
      2. Every piece of code you type in should be your own. Do not look at other people’s code while you are typing yours in!
      3. Do not copy other people’s code and modify it!
    • It is OK to:
      • Discuss general methodology. However, first try to do the problem yourself.
      • Discuss types of errors (“can’t assign a tuple to an int”) and how to resolve them.
      • Discuss any aspect of Python or C++ you don’t understand (e.g. broadcasting in Python).
        • However, do this without referring to assignments.

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

11 of 27

11

Course grades

  • 3 Long Assignments
    • Get started as soon as assignments are posted.
    • Some aspects of assignments require only basic knowledge of Python, but some require in-depth understanding of numpy arrays and complicated indexing schemes. They can take a while to work through.
    • If you don’t know Python, work through tutorial now.

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

12 of 27

12

Grading Policy

  • 3 Problem sets: 15%*3 = 45%
  • Midterm exam: 15%
  • Final Course project: 40%
    • Milestone: 5%
    • Final write-up: 35%
    • Bonus points for exceptional poster presentation
  • Late Policy:
    • 7 free late days: use them as you see fit
    • Afterwards: 25% off per day late
    • Not accepted after 3 late days
    • Does not apply to final course project (must be on time)

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

13 of 27

13

Assignments

  • 3 Long Assignments
    • Get started as soon as assignments are posted.
    • Some aspects of assignments require only basic knowledge of Python, but some require in-depth understanding of numpy arrays and complicated indexing schemes. They can take a while to work through.
    • If you don’t know Python, work through tutorial now.

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

14 of 27

14

Getting Started

  • Example: Mac
    • Language: Python
      • Instructions for installing given under first assignment instructions.
      • DEMO
    • Development environment: Jupyter Notebook. Live code environment.
      • DEMO
    • Running a shell on the side: Jupyter QtConsole
      • Good for testing syntax, return values of functions.
      • DEMO�

​

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

15 of 27

15

https://compsci697l.github.io/notes/python-numpy-tutorial/

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

16 of 27

16

Assignment #1

  • Posted on course web site
  • Due in 3 weeks (Sept. 28, 11:59pm).
  • It includes:
  • Write/train/evaluate a kNN classifier
  • Write/train/evaluate a Linear Classifier (SVM and Softmax)
  • Write/train/evaluate a 2-layer Neural Network (backpropagation!)
  • Requires writing numpy/Python code

​

Compute: Use your own laptops

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

17 of 27

Image Classification: a core task in Computer Vision

17

cat

(assume given set of discrete labels)

{dog, cat, truck, plane, ...}

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

18 of 27

The problem:

semantic gap

18

Images are represented as 3D arrays of numbers, with integers between [0, 255].

​

E.g.

300 x 100 x 3

​

(3 for 3 color channels RGB)

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

19 of 27

19

Challenges: Viewpoint Variation

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

20 of 27

20

Challenges: Illumination

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

21 of 27

21

Challenges: Deformation

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

22 of 27

22

Challenges: Occlusion

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

23 of 27

23

Challenges: Background clutter

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

24 of 27

24

Challenges: Intraclass variation

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

25 of 27

An image classifier

25

Unlike e.g. sorting a list of numbers,

no obvious way to hard-code the algorithm for recognizing a cat, or other classes.

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

26 of 27

Data-driven approach:

  1. Collect a dataset of images and labels
  2. Use Machine Learning to train an image classifier
  3. Evaluate the classifier on a withheld set of test images

26

Example training set

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016

27 of 27

First classifier: Nearest Neighbor Classifier

27

Remember all training images and their labels

Predict the label of the most similar training image

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

Lecture 1 -

7 Sep. 2016

Lecture 1 -

Erik Learned-Miller and Hang Su�Some slides kindly provided by Fei-Fei Li, Andrej Karpathy, Justin Johnson

7 Sep. 2016