1 of 156

EASI-22

with Navajo Tech

Edge AI Summer �Institute 2022

Yá’át’ééh 👋

Harvard

CC BY-NC-SA 4.0

2 of 156

Hi! I’m Brian!

I’m an Assistant Professor of Computer Science at Barnard College, Columbia University

CC BY-NC-SA 4.0

3 of 156

Our team!

with help from many more

CC BY-NC-SA 4.0

4 of 156

CC BY-NC-SA 4.0

5 of 156

Our website!

home base for all information!

tinyMLedu.org/EASI-22

CC BY-NC-SA 4.0

6 of 156

CC BY-NC-SA 4.0

7 of 156

Make Sure to Pick Up an Arduino Kit!

Question? Contact: Monsuru Ramoni

mramoni@navajotech.edu

CC BY-NC-SA 4.0

8 of 156

Teachers Sign up for Buy2Pay

Question? Contact:

Molly Marshall

mmarshall@seas.harvard.edu

CC BY-NC-SA 4.0

9 of 156

Workshop Agenda

9

Introduction to AI and (Tiny)ML

Cloud ML

Day 1

Keyword Spotting for the Navajo Language

Mobile ML

Day 2

Bringing AI/ML from the Cloud to the Edge

Embedded ML

Day 3

CC BY-NC-SA 4.0

10 of 156

Today’s Agenda

What is Artificial Intelligence?

Hands-on: AutoDraw

What is (Deep) Machine Learning?

Hands-on: ThingTranslator

What is Responsible TinyML?

Summary

10

CC BY-NC-SA 4.0

11 of 156

Today’s Agenda

What is Artificial Intelligence?

Hands-on: AutoDraw

What is (Deep) Machine Learning?

Hands-on: ThingTranslator

What is Responsible TinyML?

Summary

11

CC BY-NC-SA 4.0

12 of 156

Artificial Intelligence (AI) is when a computer can…

Think Like A Human

Think Rationally

Act Rationally

Act Like a Human

CC BY-NC-SA 4.0

13 of 156

The Turing Test

CC BY-NC-SA 4.0

14 of 156

The Turing Test

CC BY-NC-SA 4.0

15 of 156

The Turing Test

Chatbots can be convincing

Q: What is your favorite animal?

A: My favorite animal is a dog.

Q: Why?

A: Because dogs are loyal and friendly.

Q: What are two reasons that a dog might be in a bad mood?

A: Two reasons that a dog might be in a bad mood are if it is hungry or if it is hot.

Q: How many eyes does a giraffe have?

A: A giraffe has two eyes.

Q: How many legs does a frog have?

A: A frog has four legs.

Q: Are there any animals with three legs?

A: No, there are no animals with three legs.

Q: Why don't animals have three legs?

A: Animals don't have three legs because they would fall over.

https://lacker.io/ai/2020/07/06/giving-gpt-3-a-turing-test.html

CC BY-NC-SA 4.0

16 of 156

The Turing Test

Chatbots can be convincing

Q: What is your favorite animal?

A: My favorite animal is a dog.

Q: Why?

A: Because dogs are loyal and friendly.

Q: What are two reasons that a dog might be in a bad mood?

A: Two reasons that a dog might be in a bad mood are if it is hungry or if it is hot.

Q: How many eyes does a giraffe have?

A: A giraffe has two eyes.

Q: How many legs does a frog have?

A: A frog has four legs.

Q: Are there any animals with three legs?

A: No, there are no animals with three legs.

Q: Why don't animals have three legs?

A: Animals don't have three legs because they would fall over.

https://lacker.io/ai/2020/07/06/giving-gpt-3-a-turing-test.html

CC BY-NC-SA 4.0

17 of 156

The Turing Test

But they only know “what they know”

Q: How many eyes does a giraffe have?

A: A giraffe has two eyes.

Q: How many eyes does my foot have?

A: Your foot has two eyes.

Q: How many eyes does a spider have?

A: A spider has eight eyes.

Q: How many eyes does the sun have?

A: The sun has one eye.

Q: How many eyes does a blade of grass have?

A: A blade of grass has one eye.

https://lacker.io/ai/2020/07/06/giving-gpt-3-a-turing-test.html

CC BY-NC-SA 4.0

18 of 156

Traditional AI Programming

Rules

Data

Answers

CC BY-NC-SA 4.0

19 of 156

Traditional AI Programming

Rules

Data

Answers

AI is the study of algorithms that can give computers the rules they need to be “intelligent”!

CC BY-NC-SA 4.0

20 of 156

Traditional AI Programming

Rules

Data

Answers

CC BY-NC-SA 4.0

21 of 156

Traditional AI Programming

Rules

Data

Answers

CC BY-NC-SA 4.0

22 of 156

Traditional AI Programming

Rules

Data

Answers

CC BY-NC-SA 4.0

23 of 156

Deep Blue

CC BY-NC-SA 4.0

24 of 156

Deep Blue

On average in any board configuration there are 35 possible moves in chess.

CC BY-NC-SA 4.0

25 of 156

Deep Blue

On average in any board configuration there are 35 possible moves in chess.

That means that the computer can search for the move that eventually leads to success with a 35^turns guesses

CC BY-NC-SA 4.0

26 of 156

Deep Blue

On average in any board configuration there are 35 possible moves in chess.

That means that the computer can search for the move that eventually leads to success with a 35^turns guesses

That means if a chess game took 10 turns and the computer could check 1 billion guesses a second it would take ?????? to come up with the optimal move!

CC BY-NC-SA 4.0

27 of 156

Deep Blue

On average in any board configuration there are 35 possible moves in chess.

That means that the computer can search for the move that eventually leads to success with a 35^turns guesses

That means if a chess game took 10 turns and the computer could check 1 billion guesses a second it would take 30 days to come up with the optimal move!

CC BY-NC-SA 4.0

28 of 156

Deep Blue

So deep blue searched ~7 turns ahead and relied on a board scoring rule created by the programmers!

Traditional AI Programming

Rules

Data

Answers

CC BY-NC-SA 4.0

29 of 156

Today’s Agenda

What is Artificial Intelligence?

Hands-on: AutoDraw

What is (Deep) Machine Learning?

Hands-on: ThingTranslator

What is Responsible TinyML?

Summary

29

CC BY-NC-SA 4.0

30 of 156

Today’s Agenda

What is Artificial Intelligence?

Hands-on: AutoDraw

What is (Deep) Machine Learning?

Hands-on: ThingTranslator

What is Responsible TinyML?

Summary

30

CC BY-NC-SA 4.0

31 of 156

Experiments with Google

CC BY-NC-SA 4.0

32 of 156

CC BY-NC-SA 4.0

33 of 156

Fill in the blank

CC BY-NC-SA 4.0

34 of 156

Fill in the blank

tomorrow

later

CC BY-NC-SA 4.0

35 of 156

Prediction: autocomplete

CC BY-NC-SA 4.0

36 of 156

AutoDraw

CC BY-NC-SA 4.0

37 of 156

AutoDraw

autodraw.com

CC BY-NC-SA 4.0

38 of 156

Discussion Groups

Do you think the AI did a good job? 👍 / 👎
Why do you think the AI did �(or did not) work well?
How do you think the AI is working �to solve this task? 🤔
What types of things were particularly �hard or easy for the AI?

anything else?

CC BY-NC-SA 4.0

39 of 156

Today’s Agenda

What is Artificial Intelligence?

Hands-on: AutoDraw

What is (Deep) Machine Learning?

Hands-on: ThingTranslator

What is Responsible TinyML?

Summary

39

CC BY-NC-SA 4.0

40 of 156

Today’s Agenda

What is Artificial Intelligence?

Hands-on: AutoDraw

What is (Deep) Machine Learning?

Hands-on: ThingTranslator

What is Responsible TinyML?

Summary

40

CC BY-NC-SA 4.0

41 of 156

Artificial Intelligence

Machine Learning

What’s the difference?

CC BY-NC-SA 4.0

42 of 156

Artificial Intelligence

Machine Learning

CC BY-NC-SA 4.0

43 of 156

Traditional AI Programming

Rules

Data

Answers

CC BY-NC-SA 4.0

44 of 156

Traditional AI Programming

Rules

Data

Answers

CC BY-NC-SA 4.0

45 of 156

Categorize

1

2

3

4

5

6

7

CC BY-NC-SA 4.0

46 of 156

8

CC BY-NC-SA 4.0

47 of 156

The Machine Learning Paradigm

Traditional AI Programming

Rules

Data

Answers

Machine Learning

Answers

BIG Data

Rules

CC BY-NC-SA 4.0

48 of 156

Activity Detection with Machine Learning

CC BY-NC-SA 4.0

49 of 156

Activity Detection with Machine Learning

CC BY-NC-SA 4.0

50 of 156

Activity Detection with Machine Learning

CC BY-NC-SA 4.0

51 of 156

Review what we’ve learned

Machine learning provides a computer with data, rather than explicit instructions. Using these data, the computer learns to recognize patterns and becomes able to execute tasks on its own.

CC BY-NC-SA 4.0

52 of 156

Object Detection

CC BY-NC-SA 4.0

53 of 156

Segmentation

CC BY-NC-SA 4.0

54 of 156

Training the machine

ANSWERS

WE PROVIDE

INPUTS

RULES

CC BY-NC-SA 4.0

55 of 156

Training the machine

For a set of Input Data

�CAT

DOG

Input, Label

CC BY-NC-SA 4.0

56 of 156

Training the machine

Input, Label

Result

�CAT

DOG

Dog

Cat

For a set of Input Data

Guess the Answer�and count mistakes

CC BY-NC-SA 4.0

57 of 156

Training the machine

Loss�function of mistakes

Guess the Answer�and count mistakes

CC BY-NC-SA 4.0

58 of 156

Training the machine

Improve the model to be more correct

For a set of Input Data

Guess the Answer�and count mistakes

CC BY-NC-SA 4.0

59 of 156

Training the machine

Minimum

Loss�number of mistakes

Improve the model to be more correct

CC BY-NC-SA 4.0

60 of 156

Training the machine

Improve the model to be more correct

For a set of Input Data

Guess the Answer�and count mistakes

CC BY-NC-SA 4.0

61 of 156

https://www.researchgate.net/publication/339545333_The_Connectivity_Evaluation_Among_Wells_in_Reservoir_Utilizing_Machine_Learning_Methods/figures?lo=1

CC BY-NC-SA 4.0

62 of 156

Training the machine

ML

RULES

LABELS

INPUTS

�CAT

DOG

CC BY-NC-SA 4.0

63 of 156

After it’s learned:

Model

RULES

NEW

INPUTS

ANSWERS

NEW

CC BY-NC-SA 4.0

64 of 156

After it’s learned:

Model

RULES

NEW

UNLABELED

INPUTS

CC BY-NC-SA 4.0

65 of 156

Making predictions:

Model

RULES

NEW

UNLABELED

INPUTS

NEW

ANSWERS (LABELS)

CAT

DOG

This is often called INFERENCE!

CC BY-NC-SA 4.0

66 of 156

Deep Learning

Machine Learning

Ok so what about Deep Learning?

CC BY-NC-SA 4.0

67 of 156

Artificial Intelligence

Machine Learning

Deep Learning

CC BY-NC-SA 4.0

68 of 156

Artificial Intelligence

Machine Learning

Deep Learning

a lot of�DATA

CC BY-NC-SA 4.0

69 of 156

Training the machine

Make a Guess!

Neural Network

CC BY-NC-SA 4.0

70 of 156

Neural network

CC BY-NC-SA 4.0

71 of 156

Neural network

CC BY-NC-SA 4.0

72 of 156

Neural network

Y

X

threshold

0

1

Activation Function

CC BY-NC-SA 4.0

73 of 156

Neural network

X

Y

artificial

CC BY-NC-SA 4.0

74 of 156

Neural network

X

Y

CC BY-NC-SA 4.0

75 of 156

Neural network

Y

X

threshold

0

1

X

Y

Activation Function

CC BY-NC-SA 4.0

76 of 156

Multi-layer�neural network

X

Y

CC BY-NC-SA 4.0

77 of 156

Deep Learning

with Neural Networks

CC BY-NC-SA 4.0

78 of 156

Training the machine

ML

RULES

LABELS

INPUTS

�CAT

DOG

CC BY-NC-SA 4.0

79 of 156

Case Study:

Handwriting

LABELS

INPUTS

practice�DATA

CC BY-NC-SA 4.0

80 of 156

Case Study:

Handwriting

practice�DATA

ML

RULES

CC BY-NC-SA 4.0

81 of 156

Case Study:

Handwriting

ML

RULES

CC BY-NC-SA 4.0

82 of 156

What number?

00

0

1

0

It’s a 7!

CC BY-NC-SA 4.0

83 of 156

What number?

00

0

1

0

It’s a 7!

28px

CC BY-NC-SA 4.0

84 of 156

What number?

00

0

1

0

It’s a 7!

28px

CC BY-NC-SA 4.0

85 of 156

Transform:�flatten

00

0

1

0

It’s a 7!

784

CC BY-NC-SA 4.0

86 of 156

After Training the Model is VERY good!

https://paperswithcode.com/sota/image-classification-on-mnist

Google Colab

CC BY-NC-SA 4.0

87 of 156

And it can solve problems we couldn’t solve without ML!

DeepBlue

On average in any board configuration there are 35 possible moves in chess.

CC BY-NC-SA 4.0

88 of 156

And it can solve problems we couldn’t solve without ML!

DeepBlue

On average in any board configuration there are 35 possible moves in chess.

AlphaGo

“There are an astonishing 10 to the power of 170 possible board configurations - more than the number of atoms in the known universe. This makes the game of Go a googol times more complex than chess.”

https://www.deepmind.com/research/highlighted-research/alphago

CC BY-NC-SA 4.0

89 of 156

But It Need Lots of Data

6000 Images / Class

10 Classes

This is considered a SMALL and simple dataset (~45MB)

CC BY-NC-SA 4.0

90 of 156

But It Need Lots of Data

6000 Images / Class

10 Classes

GPT-3 Used ~45TB of data that’s ~1,000,000 times more data than MNIST!

CC BY-NC-SA 4.0

91 of 156

Today’s Agenda

What is Artificial Intelligence?

Hands-on: AutoDraw

What is (Deep) Machine Learning?

Hands-on: ThingTranslator

What is Responsible TinyML?

Summary

91

CC BY-NC-SA 4.0

92 of 156

Today’s Agenda

What is Artificial Intelligence?

Hands-on: AutoDraw

What is (Deep) Machine Learning?

Hands-on: ThingTranslator

What is Responsible TinyML?

Summary

92

CC BY-NC-SA 4.0

93 of 156

Thing Translator

CC BY-NC-SA 4.0

94 of 156

Thing Translator

CC BY-NC-SA 4.0

95 of 156

Thing Translator

CC BY-NC-SA 4.0

96 of 156

Thing Translator

thing-translator.appspot.com

CC BY-NC-SA 4.0

97 of 156

Discussion Groups

Do you think the AI did a good job? 👍 / 👎
Why do you think the AI worked well?
How did the AI solve this task? 🤔
What types of things were particularly �hard or easy for the AI?
Was the AI better or worse in this experiment? Why do you think?

anything else?

CC BY-NC-SA 4.0

98 of 156

Today’s Agenda

What is Artificial Intelligence?

Hands-on: AutoDraw

What is (Deep) Machine Learning?

Hands-on: ThingTranslator

What is Responsible TinyML?

Summary

98

CC BY-NC-SA 4.0

99 of 156

Today’s Agenda

What is Artificial Intelligence?

Hands-on: AutoDraw

What is (Deep) Machine Learning?

Hands-on: ThingTranslator

What is Responsible TinyML?

Summary

99

CC BY-NC-SA 4.0

100 of 156

TinyML

ML

What’s the difference?

CC BY-NC-SA 4.0

101 of 156

Cloud / Server

https://tossingbot.cs.princeton.edu/

CC BY-NC-SA 4.0

102 of 156

Mobile

https://plantvillage.psu.edu/

CC BY-NC-SA 4.0

103 of 156

ML on

Embedded Devices

Google Assistant

nest

CC BY-NC-SA 4.0

104 of 156

IoT 2.0: Intelligence on Things

Google Assistant

nest

IoT 1.0:

Internet

of Things

CC BY-NC-SA 4.0

105 of 156

No Good Data Left Behind

5 Quintillion

bytes of data produced every day by IoT

Source: Harvard Business Review, What’s Your Data Strategy?, April 18, 2017

Cisco, Internet of Things (IoT) Data Continues to Explode Exponentially. Who Is Using That Data and How?, Feb 5, 2018

<1%

of unstructured data is analyzed or used at all

CC BY-NC-SA 4.0

106 of 156

IoT 2.0: Intelligence on Things

IoT 1.0:

Internet

of Things

B

L

E

R

P

CC BY-NC-SA 4.0

107 of 156

IoT 2.0: Intelligence on Things

IoT 1.0:

Internet

of Things

Bandwidth

Latency

Energy

Reliability

Privacy

CC BY-NC-SA 4.0

108 of 156

IoT 2.0: Intelligence on Things

IoT 1.0:

Internet

of Things

Bandwidth

Latency

Energy

Reliability

Privacy

Battery Life is only O(months) and only sends GPS signal

CC BY-NC-SA 4.0

109 of 156

IoT 2.0: Intelligence on Things

IoT 1.0:

Internet

of Things

Bandwidth

Latency

Energy

Reliability

Privacy

CC BY-NC-SA 4.0

110 of 156

IoT 2.0: Intelligence on Things

IoT 1.0:

Internet

of Things

Bandwidth

Latency

Energy

Reliability

Privacy

Google Assistant

CC BY-NC-SA 4.0

111 of 156

IoT 2.0: Intelligence on Things

IoT 1.0:

Internet

of Things

Bandwidth

Latency

Energy

Reliability

Privacy

TinyML to the rescue!

CC BY-NC-SA 4.0

112 of 156

What is Tiny Machine Learning (TinyML)?

TinyML

Fastest-growing field of ML

CC BY-NC-SA 4.0

113 of 156

What is Tiny Machine Learning (TinyML)?

TinyML

Fastest-growing field of ML

Algorithms, hardware, software

CC BY-NC-SA 4.0

114 of 156

What is Tiny Machine Learning (TinyML)?

TinyML

Fastest-growing field of ML

Algorithms, hardware, software

On-device sensor analytics

CC BY-NC-SA 4.0

115 of 156

What is Tiny Machine Learning (TinyML)?

TinyML

Fastest-growing field of ML

Algorithms, hardware, software

Low power consumption

On-device sensor analytics

CC BY-NC-SA 4.0

116 of 156

What is Tiny Machine Learning (TinyML)?

TinyML

Fastest-growing field of ML

Algorithms, hardware, software

Low power consumption

On-device sensor analytics

Battery-operated

CC BY-NC-SA 4.0

117 of 156

What is Tiny Machine Learning (TinyML)?

TinyML

Fastest-growing field of ML

Algorithms, hardware, software

Low power consumption

On-device sensor analytics

Always-on ML

Battery-operated

CC BY-NC-SA 4.0

118 of 156

What is Tiny Machine Learning (TinyML)?

TinyML

Fastest-growing field of ML

Algorithms, hardware, software

Low power consumption

On-device sensor analytics

Always-on ML

Battery-operated

Putting ML on embedded devices!

CC BY-NC-SA 4.0

119 of 156

CC BY-NC-SA 4.0

120 of 156

Promising Social Applications of TinyML

Wildlife conservation

Agriculture

May be able to reduce agrichemical use to 0.1% of conventional blanket spraying

CC BY-NC-SA 4.0

121 of 156

TinyRL: Autonomous Navigation on Nano Drone

121

[ICRA’21]

[IROS’21]

CC BY-NC-SA 4.0

122 of 156

CC BY-NC-SA 4.0

123 of 156

The (Tiny) Machine Learning Workflow

Collect & Transform

Data

Design & Train� a Model

Convert

Model

Deploy

Model

Make

Inferences

CC BY-NC-SA 4.0

124 of 156

The (Tiny) Machine Learning Workflow

Collect & Transform

Data

Design & Train� a Model

Convert

Model

Deploy

Model

Make

Inferences

If ML is going to be everywhere we need to consider how to best collect GOOD data RESPONSIBLY

CC BY-NC-SA 4.0

125 of 156

Good Data is Necessary for Accuracy

What problem are you

trying to solve?

Your data must contain useful features
Can a human (expert) distinguish �between examples of each class?
How will you measure performance?

CC BY-NC-SA 4.0

126 of 156

Good Data is Necessary for Accuracy

What problem are you

trying to solve?

Your data must contain useful features
Can a human (expert) distinguish �between examples of each class?
How will you measure performance?

Both quantity and quality

will influence your

model’s performance

Wide variety of training examples
Correct labels (answers)
Good Balance (e.g., dog, cat, random)

CC BY-NC-SA 4.0

127 of 156

Potential Bias in Speech Recognition

Gender

Sources of Bias

Gender

Age

Group

Accent

Dialect

CC BY-NC-SA 4.0

128 of 156

CC BY-NC-SA 4.0

129 of 156

Today’s Agenda

What is Artificial Intelligence?

Hands-on: AutoDraw

What is (Deep) Machine Learning?

Hands-on: ThingTranslator

What is Responsible TinyML?

Summary

129

CC BY-NC-SA 4.0

130 of 156

Today’s Agenda

What is Artificial Intelligence?

Hands-on: AutoDraw

What is (Deep) Machine Learning?

Hands-on: ThingTranslator

What is Responsible TinyML?

Summary

130

CC BY-NC-SA 4.0

131 of 156

Machine Learning Workflow

131

Collect & Transform

Data

Design & Train� a Model

Convert

Model

Deploy

Model

Make

Inferences

CC BY-NC-SA 4.0

132 of 156

Machine Learning Workflow

Collect &�Transform Data

Design & Train� a Model

Evaluate

Optimize

Convert

Model

Deploy

Model

Make

Inferences

LABELS

INPUTS

Dog

CC BY-NC-SA 4.0

133 of 156

Machine Learning Workflow

Collect &�Transform Data

Design & Train� a Model

Evaluate

Optimize

Convert

Model

Deploy

Model

Make

Inferences

Sources of Bias

Gender

Age

Group

Accent

Dialect

CC BY-NC-SA 4.0

134 of 156

Machine Learning Workflow

Y

X

threshold

0

1

Activation Function

Collect &�Transform Data

Design & Train� a Model

Evaluate

Optimize

Convert

Model

Deploy

Model

Make

Inferences

X

Y

CC BY-NC-SA 4.0

135 of 156

Machine Learning Workflow

Collect &�Transform Data

Design & Train� a Model

Evaluate

Optimize

Convert

Model

Deploy

Model

Make

Inferences

practice�DATA

ML

RULES

For a set of Input Data

Guess the Answer�and count mistakes

Improve the model to be more correct

CC BY-NC-SA 4.0

136 of 156

Machine Learning Workflow

Collect &�Transform Data

Design & Train� a Model

Evaluate

Optimize

Convert

Model

Deploy

Model

Make

Inferences

Neural Network

CC BY-NC-SA 4.0

137 of 156

Machine Learning Workflow

Collect &�Transform Data

Design & Train� a Model

Evaluate

Optimize

Convert

Model

Deploy

Model

Make

Inferences

Google Assistant

CC BY-NC-SA 4.0

138 of 156

Machine Learning Workflow

Collect &�Transform Data

Design & Train� a Model

Evaluate

Optimize

Convert

Model

Deploy

Model

Make

Inferences

CC BY-NC-SA 4.0

139 of 156

Bonus Content:

Scaling TinyML

139

CC BY-NC-SA 4.0

140 of 156

140

CC BY-NC-SA 4.0

141 of 156

141

CC BY-NC-SA 4.0

142 of 156

142

?

CC BY-NC-SA 4.0

143 of 156

143

Source: https://blogs.gartner.com/andrew_white/2019/01/03/our-top-data-and-analytics-predicts-for-2019/

CC BY-NC-SA 4.0

144 of 156

144

CC BY-NC-SA 4.0

145 of 156

145

CC BY-NC-SA 4.0

146 of 156

146

CC BY-NC-SA 4.0

147 of 156

MLCode

147

Training formalization

Experimentation & prototyping

Data Engineering

�Dataset & feature repository

Problem Definition

�Model registry

�ML metadata & artifact repository

�Version Control

Model Validation

Model Prototyping

Data Selection

Data Exploration

Feature Engineering

Implement training pipeline

Training operationalization

Code & Config

<\>

Dataset

<\>

Parameters, metrics, annotations

<\>

Trained Model

<\>

Practitioners guide to MLOps: A framework for continuous

delivery and automation of machine learning

Model Evaluation

CC BY-NC-SA 4.0

https://thenounproject.com/search/?q=protractor&i=1413295

This diagram illustrates the process of ML development
As we can note, the primary source of development data is the dataset and feature repository
The experimentation and prototyping of ML model consists of:

Data discovery, selection, exploration, preparation and engineering
Model prototyping and validation
Evaluation: What-if, fairness, bias, explainability

The configurations for each experiment is persisted in the ML metadata repository

There is a pointer to the version of the training code, model architecture, pre-trained modules and hyperparameters
There is Information about training, validation and testing data splits
And also about Model evaluation metrics and the validation procedure

The Formalisation of the ML training procedure for retraining represents the end to end pipeline
The key success here are: experiment tracking, reproducibility and collaboration

Data scientists should be able to find previous experiments
And compare experiments and runs to understand model’s behaviour

148 of 156

148

ML Development

ML Training

Continuous Training

Model Deployment

Prediction Serving

Continuous Monitoring

Data, Model Management

Code & Config

Training Pipeline

Serving Logs

Convert Model

Tensor Flow Lite Model

Serving Package

Registered Model

CC BY-NC-SA 4.0

149 of 156

149

ML Development

ML Training

Continuous Training

Model Deployment

Prediction Serving

Continuous Monitoring

Data, Model Management

Code & Config

Training Pipeline

Serving Logs

Convert Model

Tensor Flow Lite Model

Serving Package

Registered Model

ML Engineer

ML Researcher

Data Scientist

Data Engineer

Software Engineer

DevOps

Business Analyst

CC BY-NC-SA 4.0

150 of 156

150

ML Expertise

BASIC

EXPERT

ML Engineer

ML Researcher

Data Scientist

Data Engineer

Software Engineer

DevOps

Business Analyst

CC BY-NC-SA 4.0

151 of 156

151

Deployment Expertise

BASIC

ML Engineer

ML Researcher

Data Scientist

Data Engineer

Software Engineer

DevOps

Business Analyst

EXPERT

CC BY-NC-SA 4.0

152 of 156

152

ML Operations

TinyML

BREADTH�of experience, knowledge, & sectors

DEPTH�high skills in one discipline

CC BY-NC-SA 4.0

153 of 156

153

CC BY-NC-SA 4.0

154 of 156

Our website!

home base for all information!

tinyMLedu.org/EASI-22

CC BY-NC-SA 4.0

155 of 156

Our team!

with help from many more

CC BY-NC-SA 4.0

156 of 156

see you again at 12pm (Mountain Time)

hágoóneeʼ 👋

CC BY-NC-SA 4.0