1 of 48

The Scalable Analytics Institute (ScAI)�Department of Computer Science�University of California, Los Angeles (UCLA)

Instructor: Jeehyun Hwang

LECTURE 3: NEURAL NETWORKS

Data Science for All 2023

Contact: jeehyunhwang@cs.ucla.edu

hwanggh96@gmail.com

Data Science for All 2023

Lecture 1: Classification

2 of 48

Demos

  • Segmentation

2

July 19, 2023

Data Science for All 2023

Lecture 1: Classification

3 of 48

Demos

  • Indoor scene generation

3

July 19, 2023

Data Science for All 2023

Lecture 1: Classification

4 of 48

Demos

  • Music Generation
    • https://google-research.github.io/seanet/musiclm/examples/

4

July 19, 2023

Data Science for All 2023

Lecture 1: Classification

5 of 48

Demos

  • Protein structure prediction
    • https://www.deepmind.com/research/highlighted-research/alphafold

5

July 19, 2023

Data Science for All 2023

Lecture 1: Classification

6 of 48

Demos

  • Image Generation
    • https://labs.openai.com/

6

July 19, 2023

Data Science for All 2023

Lecture 1: Classification

7 of 48

Demos

  • Large-language models
    • https://chat.openai.com/

7

July 19, 2023

Data Science for All 2023

Lecture 1: Classification

8 of 48

Demos

  • https://huggingface.co/spaces

8

July 19, 2023

Data Science for All 2023

Lecture 1: Classification

9 of 48

Examples

9

July 19, 2023

Data Science for All 2023

Lecture 1: Classification

10 of 48

Examples

10

July 19, 2023

Data Science for All 2023

Lecture 1: Classification

11 of 48

Examples

11

July 19, 2023

Data Science for All 2023

Lecture 1: Classification

12 of 48

Historical Trends

12

July 19, 2023

beamandrew.github.io/

Data Science for All 2023

Lecture 1: Classification

13 of 48

Historical Trends

13

July 19, 2023

ArXiv papers on deep learning: 2012-2017

ArXiv papers on deep learning: 2012-2017

Data Science for All 2023

Lecture 1: Classification

14 of 48

Artificial Neural Networks

  •  

14

July 19, 2023

Data Science for All 2023

Lecture 1: Classification

15 of 48

Artificial Neural Networks

  • Architecture – layers and hidden dimensions
  • Activation function – Nonlinear Functions
  • Loss function – MSE? MAE?
  • Optimization – Backpropagation and Global/Local Minimum
  • Regularization & Overfitting

15

July 19, 2023

Data Science for All 2023

Lecture 1: Classification

16 of 48

Artificial Neural Networks

16

July 19, 2023

hidden layer 1

hidden layer 2

output layer

input layer

Data Science for All 2023

Lecture 1: Classification

17 of 48

NN in action

17

July 19, 2023

Data Science for All 2023

Lecture 1: Classification

18 of 48

NN in action

18

July 19, 2023

Data Science for All 2023

Lecture 1: Classification

19 of 48

NN in action

19

July 19, 2023

Data Science for All 2023

Lecture 1: Classification

20 of 48

NN in action

20

July 19, 2023

…

Data Science for All 2023

Lecture 1: Classification

21 of 48

NN in action

21

July 19, 2023

Data Science for All 2023

Lecture 1: Classification

22 of 48

NN in action

22

July 19, 2023

Data Science for All 2023

Lecture 1: Classification

23 of 48

Artificial Neural Networks

  • Neural Network with Hand

23

July 19, 2023

Data Science for All 2023

Lecture 1: Classification

24 of 48

Optimization – Global and Local Minima

24

July 19, 2023

 

 

 

Data Science for All 2023

Lecture 1: Classification

25 of 48

Optimization – Global and Local Minima

25

July 19, 2023

 

L

w

-

+

 

Data Science for All 2023

Lecture 1: Classification

26 of 48

Try it yourself!

26

July 19, 2023

https://playground.tensorflow.org/

Data Science for All 2023

Lecture 1: Classification

27 of 48

Regularization

27

July 19, 2023

 

Biases not penalized

 

 

Data Science for All 2023

Lecture 1: Classification

28 of 48

  • jeehyunhwang@cs.ucla.edu

​

28

July 17, 2023

Data Science for All 2023

Lecture 1: Classification

29 of 48

Towards Deep Learning

29

July 19, 2023

Data Science for All 2023

Lecture 1: Classification

30 of 48

Towards Deep Learning

30

July 19, 2023

Data Science for All 2023

Lecture 1: Classification

31 of 48

Towards Deep Learning

31

July 19, 2023

How big are current large Deep Neural Networks?

WHY? HOW?

Data Science for All 2023

Lecture 1: Classification

32 of 48

Towards Deep Learning

32

July 19, 2023

What data do we have?

Data Science for All 2023

Lecture 1: Classification

33 of 48

Towards Deep Learning

33

July 19, 2023

What data do we have?

- Geology / Image / Biological / Human / Video / Health and pandemic / Prices / Time series / Structured or Table data / Audio data

Data Science for All 2023

Lecture 1: Classification

34 of 48

Towards Deep Learning

34

July 19, 2023

Convolutional Neural Network for Image data

​

Data Science for All 2023

Lecture 1: Classification

35 of 48

Towards Deep Learning

35

July 19, 2023

Representative CNN models for various applications�Ex) Image generation: https://clipdrop.co/stable-diffusion

Image classification: https://huggingface.co/tasks/image-classification

Video generation: https://github.com/nvlabs/long-video-gan

Anything you can think of using image!

Data Science for All 2023

Lecture 1: Classification

36 of 48

Towards Deep Learning

36

July 19, 2023

Recurrent Neural Network for Time series data

​

Data Science for All 2023

Lecture 1: Classification

37 of 48

Towards Deep Learning

37

July 19, 2023

Geostatistical models for Geology

​

​

​

Data Science for All 2023

Lecture 1: Classification

38 of 48

Towards Deep Learning

38

July 19, 2023

Biological models

​

​

​

Data Science for All 2023

Lecture 1: Classification

39 of 48

39

July 17, 2023

Data Science for All 2023

Lecture 1: Classification

40 of 48

Towards Deep Learning

40

July 19, 2023

Social Science Data

​

​

​

Data Science for All 2023

Lecture 1: Classification

41 of 48

Towards Deep Learning

41

July 19, 2023

Autonomous Driving

​

​

Data Science for All 2023

Lecture 1: Classification

42 of 48

Towards Deep Learning

42

July 19, 2023

  • Deep layers and parameter sharing
    • Convolutional Neural Network
      • Applications to: Images, NLP, Graph
    • Recurrent Neural Network
      • Applications to: Sequence data

​

​

​

Data Science for All 2023

Lecture 1: Classification

43 of 48

Convolutional Neural Networks

43

July 19, 2023

The LeNet Architecture (By Yann LeCun et al.)

Data Science for All 2023

Lecture 1: Classification

44 of 48

Convolutional Neural Networks

44

July 19, 2023

Data Science for All 2023

Lecture 1: Classification

45 of 48

Convolutional Neural Networks

45

July 19, 2023

Why not just use the Multi-layer Perceptron, or Deep Neural Network?

Data Science for All 2023

Lecture 1: Classification

46 of 48

Convolutional Neural Networks

46

July 19, 2023

Why not just use the Multi-layer Perceptron, or Deep Neural Network?

Data Science for All 2023

Lecture 1: Classification

47 of 48

What is your interest?

47

July 19, 2023

Data Science for All 2023

Lecture 1: Classification

48 of 48

48

July 17, 2023

Thank You

Data Science for All 2023

Lecture 1: Classification