1 of 65

Modern Artificial Intelligence

S. M. Ali Eslami

​

​

December 2015

2 of 65

3 of 65

Chappie (2015)

Ex Machina (2015)

Elon Musk

Stephen Hawking

4 of 65

Outline

  1. Artificial General Intelligence

​

  • Deep Learning

​

  • Reinforcement Learning

​

  • Model-based Methods

​

  • Reinforced Variational Inference

5 of 65

Output

Input

Algorithm

Programmable

Computer

6 of 65

“Horse”

Search

Output

Input

Algorithm

Programmable

Computer

7 of 65

Human

?

Image Classification

Output

Input

Algorithm

Horse

8 of 65

?

Computer

Output

Input

Algorithm

Horse

Image Classification

9 of 65

Tasks thought to require intelligence

  • Image understanding,
  • Natural language processing,
  • Knowledge acquisition,
  • Text understanding,
  • Planning,
  • Robotics,
  • Forecasting,
  • And many others.

​

Can a general system achieve all these tasks?

10 of 65

Immediate Usefulness

General Applicability

11 of 65

Immediate Usefulness

General Applicability

12 of 65

Immediate Usefulness

General Applicability

13 of 65

Immediate Usefulness

General Applicability

14 of 65

Immediate Usefulness

General Applicability

15 of 65

Immediate Usefulness

General Applicability

?

16 of 65

Immediate Usefulness

General Applicability

?

17 of 65

Deep Learning

18 of 65

?

Computer

Output

Input

Horse

Algorithm

19 of 65

Computer

Output

Input

Horse

Preprocessing

Feature Extraction

Feature Selection

Discrimination

Calibration

Algorithm

20 of 65

Computer

Horse

Cow

Output

Input

Preprocessing

Feature Extraction

Feature Selection

Learned

Discrimination

Calibration

Algorithm

21 of 65

Computer

Horse

Cow

Output

Input

Stage 1

Stage 2

Stage 3

Stage 4

Stage 5

Algorithm

22 of 65

Convolutional neural networks for image classification

Torch (2015)

Deep Learning

23 of 65

Krizhevsky et al. (2012)

24 of 65

Clarifai (2014)

25 of 65

Where do the labels come from?

26 of 65

Reinforcement Learning

27 of 65

Architecture

Agent

Environment

Observations

Actions

28 of 65

ATARI agents

100+ classic 8-bit Atari games

​

  • Observations: Raw video (~30k dimensional)
  • Actions: 18 buttons but not told what they do
  • Goal: Simply to maximize score

​

  • Everything learnt from scratch
  • Zero pre-programmed knowledge
  • One algorithm to play all the different games

29 of 65

Human-level control through deep reinforcement learning

Mnih et al. (Nature, 2015)

30 of 65

Space Invaders agent

31 of 65

Breakout agent

32 of 65

General Atari agent

33 of 65

Human-level control through deep reinforcement learning

Mnih et al. (Nature, 2015)

34 of 65

Human-level control through deep reinforcement learning

Mnih et al. (Nature, 2015)

35 of 65

Deep Reinforcement Learning for Continuous Control

36 of 65

How many experiences do we need?

37 of 65

1 epoch = 50,000 interactions = 30 minutes of experience

Total experience: 10m interactions = 5 days

38 of 65

Model-based Methods

39 of 65

Three learning paradigms

x

z

x

Supervised

Learning

Reinforcement

Learning

y

z

a

horse

left

40 of 65

Three learning paradigms

Model

x

z

x

x

z

x

Supervised

Learning

Reinforcement

Learning

Generative

Modelling

y

z

a

y

horse

left

41 of 65

Three learning paradigms

Model

x

z

x

x

z

x

Supervised

Learning

Reinforcement

Learning

Generative

Modelling

y

z

a

y

(2.3, -1, 0.5, 3)

not blinking

horse

left

42 of 65

Learning to draw shapes

43 of 65

The Shape Boltzmann Machine

44 of 65

Sampling from an SBM

45 of 65

Learning to draw shapes

46 of 65

Learning to draw shapes

The Shape Boltzmann Machine: a Strong Model of Object Shape

S. M. Ali Eslami, Nicolas Heess, Christopher K. I. Williams, John Winn

International Journal of Computer Vision, Springer (IJCV, 2013)

47 of 65

Factored Shapes and Appearances

A Generative Model for Parts-based Object Segmentation

S. M. Ali Eslami, Christopher K. I. Williams

Neural Information Processing Systems (NIPS, 2012)

48 of 65

Learning to segment objects

A Generative Model for Parts-based Object Segmentation

S. M. Ali Eslami, Christopher K. I. Williams

Neural Information Processing Systems (NIPS, 2012)

49 of 65

Learning to segment objects

A Generative Model for Parts-based Object Segmentation

S. M. Ali Eslami, Christopher K. I. Williams

Neural Information Processing Systems (NIPS, 2012)

50 of 65

Recurrent Neural Networks for Image Generation

Gregor et al. (2015)

c

x

z

p(x|c)

Decoding

Generation

Encoding

Inference

51 of 65

Recurrent Neural Networks for Image Generation

Gregor et al. (2015)

c

x

z

p(x|c)

Decoding

Generation

Encoding

Inference

Write

Read

Read

ct

x

ct+1

x

Write

p(x|cT)

z

z

52 of 65

Recurrent Neural Networks for Image Generation

Gregor et al. (2015)

53 of 65

Recurrent Neural Networks for Image Generation

Gregor et al. (2015)

54 of 65

Generative Modelling

Model p(x|z) can be:

  • Fully learned (e.g. autoencoders)
  • Partially specified
  • Fully specified (e.g. renderers)

​

What models should we use?

​

Choice of model p(x|z)

almost always constrained

by our ability to compute p(z|x)

​

Model

x

z

x

y

55 of 65

How should we do inference?

56 of 65

Reinforced Variational Inference

57 of 65

Modern Variational Inference

Model

x

z

x

y

Consensus Message Passing for Layered Graphical Models

Varun Jampani, S. M. Ali Eslami, Daniel Tarlow Pushmeet Kohli, John Winn

Artificial Intelligence and Statistics (AISTATS, 2015)

p(z|x)

58 of 65

Modern Variational Inference

Model

x

z

x

y

Consensus Message Passing for Layered Graphical Models

Varun Jampani, S. M. Ali Eslami, Daniel Tarlow Pushmeet Kohli, John Winn

Artificial Intelligence and Statistics (AISTATS, 2015)

p(z|x)

q0(z)

59 of 65

Modern Variational Inference

Model

x

z

x

y

Consensus Message Passing for Layered Graphical Models

Varun Jampani, S. M. Ali Eslami, Daniel Tarlow Pushmeet Kohli, John Winn

Artificial Intelligence and Statistics (AISTATS, 2015)

p(z|x)

q0(z)

q1(z)

60 of 65

Modern Variational Inference

Model

x

z

x

y

Consensus Message Passing for Layered Graphical Models

Varun Jampani, S. M. Ali Eslami, Daniel Tarlow Pushmeet Kohli, John Winn

Artificial Intelligence and Statistics (AISTATS, 2015)

p(z|x)

q0(z)

q1(z)

q(z|x)

x

61 of 65

Modern Variational Inference

  • Approximate p(z|x) using q(z|x)

​

  • Parameterise q(z|x)

​

  • Minimise KL[ q(z|x) | p(z|x) ]

​

  • Samples from q(z|x) can be used

as codes representing the image x

​

​

Model

x

z

x

y

Consensus Message Passing for Layered Graphical Models

Varun Jampani, S. M. Ali Eslami, Daniel Tarlow Pushmeet Kohli, John Winn

Artificial Intelligence and Statistics (AISTATS, 2015)

62 of 65

Minimising the KL

  • Minimise KL[ q(z|x) | p(z|x) ]

​

  • Maximise L(q) = Eq [ log p(x, z) - log q(z|x) ]

​

    • Potentially high variance
    • Can require knowledge of ∇p(x, z)

​

  • RL objective to maximise J(p) = Ep [ ∑t r(st, at) ]

​

  • Connection between VI and RL hinted at

by many (e.g. VAE, DLGM, NVIL, etc.)

​

​

​

Model

x

z

x

y

63 of 65

Variational Inference as Reinforcement Learning

Reinforced Variational Inference

Theophane Weber, Nicolas Heess, S. M. Ali Eslami, John Schulman, David Wingate, David Silver. Neural Information Processing Systems, Workshop on Advances in Approximate Bayesian Inference (NIPS, 2015)

maximise

64 of 65

Summary

Prediction as a subset of inference

​

Inference as a reinforcement learning problem

​

Reinforcement learning as a deep learning problem

65 of 65

http://arkitus.com

​

ali@arkitus.com