1 of 81

AIoT – When IoT meets the AI �Bringing intelligence to sensors

1

Prof. Marcelo José Rovai

rovai@unifei.edu.br

UNIFEI - Universidade Federal de Itajubá, Brazil

1

2 of 81

Marcelo Rovai was born in São Paulo and held a Master’s degree in Data Science from the Universidad del Desarrollo (UDD) in Chile and an MBA from IBMEC (INSPER) in Brazil. He graduated in 1982 as an Engineer from UNIFEI, Federal University of Itajuba, with a specialization from Escola Politécnica de Engenharia of São Paulo University (USP); both institutions are located in Brazil.

Mr. Rovai has experience as a teacher, engineer, and executive in several technology companies such as CDT/ETEP, AVIBRAS Aeroespacial, SID Informática, ATT-GIS, NCR, DELL, COMPAQ (HP), and more recently at IGT as a VP. He now works at IGT as a Senior Advisor for Latin America.

Besides his work at IGT, Marcelo Rovai publishes articles about electronics on websites such as MJRoBot.org, Hackster.io, Instructables.com, and Medium.com. Furthermore, he is a volunteer Professor at the UNIFEI Engineering Institute in Brazil and a lecturer at several Congresses and Universities on the topics of IoT and TinyML. He is an active member and a Co-Chair of the TinyML4D group, an initiative to bring TinyML education to developing countries.

2

3 of 81

Internet of Things (IoT)�

3

4 of 81

Microcontroller

COMM Error Alarm

WAN

Cloud

%

Pump Relay & LED

Lamp Relay & LED

Typical IoT Project

Comm Device

Actuators *

(Output)

Sensors *

(Input)

* “Things”

4

5 of 81

Microcontroller

COMM Error Alarm

WAN

Cloud

%

Pump Relay & LED

Lamp Relay & LED

Typical IoT Project

Comm Device

Actuators

(Output)

Sensors

(Input)

5 Quintillion

bytes of data produced every day by IoT

<1%

of unstructured data is analyzed or used at all

Source: Harvard Business Review, What’s Your Data Strategy?, April 18, 2017

Cisco, Internet of Things (IoT) Data Continues to Explode Exponentially. Who Is Using That Data and How?, Feb 5, 2018

5

6 of 81

Microcontroller

COMM Error Alarm

WAN

Cloud

%

Pump Relay & LED

Lamp Relay & LED

Typical AIoT Project

Comm Device

Actuators

(Output)

Sensors

(Input)

AI

6

7 of 81

Microcontroller

COMM Error Alarm

WAN

Cloud

%

Pump Relay & LED

Lamp Relay & LED

Typical AIoT Project … … Issues

Comm Device

Actuators

(Output)

Sensors

(Input)

Bandwidth

AI

7

8 of 81

Microcontroller

COMM Error Alarm

WAN

Cloud

%

Pump Relay & LED

Lamp Relay & LED

Typical AIoT Project … … Issues

Comm Device

Actuators

(Output)

Sensors

(Input)

Bandwidth

Latency

AI

8

9 of 81

Microcontroller

COMM Error Alarm

WAN

Cloud

%

Pump Relay & LED

Lamp Relay & LED

Typical AIoT Project … … Issues

Comm Device

Actuators

(Output)

Sensors

(Input)

Bandwidth

Latency

Energy

AI

9

10 of 81

Microcontroller

COMM Error Alarm

WAN

Cloud

%

Pump Relay & LED

Lamp Relay & LED

Typical AIoT Project … … Issues

Comm Device

Actuators

(Output)

Sensors

(Input)

Bandwidth

Latency

Energy

Reliability

AI

10

11 of 81

Microcontroller

COMM Error Alarm

WAN

Cloud

%

Pump Relay & LED

Lamp Relay & LED

Typical AIoT Project … … Issues

Comm Device

Actuators

(Output)

Sensors

(Input)

Bandwidth

Latency

Energy

Reliability

Privacy

AI

Google Assistant

11

12 of 81

Microcontroller

COMM Error Alarm

WAN

Cloud

%

Pump Relay & LED

Lamp Relay & LED

IoT 2.0 * – Edge AI/ML (ML goes close to data)

LoRa

Actuators

(Output)

Sensors

(Input)

AI

Few and eventual data to be sent to the cloud

* Intelligence of Things

12

13 of 81

Source: ABI Research: TinyML

13

14 of 81

Market Forecast

Source: ABI Research: TinyML

14

15 of 81

Embedded ML (TinyML)�Introduction�

15

16 of 81

AI: Any technique that enables computers to mimic human behavior

ML: Ability to learn without explicitly being programed

DL: Extract patterns from data using neural networks

16

17 of 81

Edge AI (or Edge ML) is the processing of Artificial Intelligence algorithms on edge, that is, on users’ devices. The concept derives from Edge Computing, which starts from the same premise: data is stored, processed, and managed directly at the Internet of Things (IoT) endpoints.

TinyML is a subset of EdgeML, where sensors are generating data with ultra-low power consumption (batteries), so that we can ultimately deploy machine learning continuously ("always on devices")

EdgeAI/ML

TinyML

17

18 of 81

What is Tiny Machine Learning (TinyML)?

18

TinyML

Fastest-growing field of ML

18

19 of 81

What is Tiny Machine Learning (TinyML)?

19

TinyML

Fastest-growing field of ML

Algorithms, hardware, software

19

20 of 81

What is Tiny Machine Learning (TinyML)?

20

TinyML

Fastest-growing field of ML

Algorithms, hardware, software

On-device sensor analytics

20

21 of 81

What is Tiny Machine Learning (TinyML)?

21

TinyML

Fastest-growing field of ML

Algorithms, hardware, software

On-device sensor analytics

Low power consumption

21

22 of 81

What is Tiny Machine Learning (TinyML)?

22

TinyML

Fastest-growing field of ML

Algorithms, hardware, software

On-device sensor analytics

Low power consumption

Battery-operated

22

23 of 81

What is Tiny Machine Learning (TinyML)?

23

TinyML

Fastest-growing field of ML

Algorithms, hardware, software

On-device sensor analytics

Low power consumption

Battery-operated

Always-on ML

23

24 of 81

Hardware

Software

TinyML ?

TinyML

24

25 of 81

Hardware

Software

TinyML

TinyML ?

25

26 of 81

TinyML Challenges�

26

27 of 81

TF Micro

Software

Hardware

Challenges

CPU

GPU

DSP

NPU

Memory

Heterogeneity

Resource Constraints

malloc

...

Missing Library Features

Limited Operating System Support

Compute

27

28 of 81

TF Micro

Software

Hardware

Challenges

CPU

GPU

DSP

MCU

Memory

Compute

Heterogeneity

Resource Constraints

malloc

...

Missing Library Features

Limited Operating System Support

28

There are a *lot* of different embedded chip architectures in active use, far more than in any other computing domain. Embedded systems have Arm Cortex M microcontrollers, Tensilica DSPs, Qualcomm DSPs, CEVA DSPs, Synopsys microcontrollers, with many others like Texas Instruments dominating in particular markets. These are more than just manufacturers, most of the chips have entirely unique instruction sets that require their own compilers and toolchains. The instruction sets are often incompatible even between different versions of a chip in the same family from a single manufacturer. This variety is because there are so many different tradeoffs that make sense for different situations, so many different solutions exist for all the varying types of problems embedded systems try to solve. A consequence of this is that the framework has to be radically portable, able to compile and run with no hitches across a very wide range of computer architectures. At the same time, compute latency is often an important requirement too, so there has to be an easy path to optimizing the code for the features offered by a platform.

29 of 81

250 Billion

MCUs today

29

30 of 81

Hardware

30

31 of 81

	Raspberry Pico (W)	Arduino Nano Sense	ESP 32	Seeed XIAO BLE Sense	Arduino Pro
32Bits CPU	Dual-core Arm Cortex-M0+	Arm Cortex-M4F	Xtensa LX6 Dual Core	Arm Cortex-M4F	Dual Core Arm Cortex M7/M4
CLOCK	133MHz	64MHz	240MHz	64MHz	480/240MHz
RAM	264KB	256KB	520KB	256KB	1MB
ROM	2MB	1MB	2MB	2MB	2MB
Radio	(Yes for W)	BLE	BLE/WiFi	BLE	BLE/WiFi
Sensors	No	Yes	No	Yes	No (Portenta) Yes (Nicla)
Price	$	$$$	$$	$$	$$$$

Hardware

31

32 of 81

32

TF Micro

Software

Hardware

Challenges

CPU

GPU

DSP

MCU

Memory

Compute

Heterogeneity

Resource Constraints

malloc

...

Missing Library Features

Limited Operating System Support

32

33 of 81

	Raspberry Pico (W)	Arduino Nano Sense	ESP 32	Seeed XIAO BLE Sense	Arduino Pro
32Bits CPU	Dual-core Arm Cortex-M0+	Arm Cortex-M4F	Xtensa LX6 Dual Core	Arm Cortex-M4F	Dual Core Arm Cortex M7/M4
CLOCK	133MHz	64MHz	240MHz	64MHz	480/240MHz
RAM	264KB	256KB	520KB	256KB	1MB
ROM	2MB	1MB	2MB	2MB	2MB
Radio	(Yes for W)	BLE	BLE/WiFi	BLE	BLE/WiFi
Sensors	No	Yes	No	Yes	No (Portenta) Yes (Nicla)
Price	$	$$$	$$	$$	$$$$

Hardware

33

34 of 81

34

https://arxiv.org/pdf/1910.01108.pdf

https://towardsdatascience.com/neural-network-architectures-156e5bad51ba

34

35 of 81

35

https://arxiv.org/pdf/1910.01108.pdf

https://towardsdatascience.com/neural-network-architectures-156e5bad51ba

35

36 of 81

36

TF Micro

Software

Challenges

malloc

...

Missing Library Features

Limited Operating System Support

Hardware

CPU

GPU

DSP

NPU

Memory

Heterogeneity

Resource Constraints

Compute

36

37 of 81

Software

37

38 of 81

38

TF Micro

Software

Hardware

Challenges

CPU

GPU

DSP

MCU

Memory

Compute

Heterogeneity

Resource Constraints

malloc

...

Missing Library Features

Limited Operating System Support

Software

malloc

...

Missing Library Features

Limited Operating System Support

38

39 of 81

Power

ESP32

XIAO

CPU Power / Memory

Application Complexity

Application Complexity vs. HW

39

40 of 81

How to Train a ML Model?

40

41 of 81

Collect

Data

Preprocess

Data

Design a

Model

Train a

Model

Evaluate

Optimize

Convert

Model

Deploy

Model

Make

Inferences

Machine Learning Workflow (“What”)

41

42 of 81

Collect

Data

Preprocess

Data

Design a

Model

Train a

Model

Evaluate

Optimize

Convert

Model

Deploy

Model

Make

Inferences

Machine Learning Workflow (“Where”)

42

43 of 81

Collect

Data

Preprocess

Data

Design a

Model

Train a

Model

Evaluate

Optimize

Convert

Model

Deploy

Model

Make

Inferences

Machine Learning Workflow (“How”)

43

44 of 81

Collect

Data

Preprocess

Data

Design a

Model

Train a

Model

Evaluate

Optimize

Convert

Model

Deploy

Model

Make

Inferences

Machine Learning Workflow (“How”)

44

45 of 81

Collect

Data

Preprocess

Data

Design a

Model

Train a

Model

Evaluate

Optimize

Convert

Model

Deploy

Model

Make

Inferences

Machine Learning Workflow (“How”)

45

46 of 81

From rule-based engineering to…

Rule-based Code

Trial

&

Error

Write More Code

46

47 of 81

Data-driven engineering

47

48 of 81

Collect

Data

Preprocess

Data

Design a

Model

Train a

Model

Evaluate

Optimize

Convert

Model

Make

Inferences

Preprocess

Data

Design a

Model

Train a

Model

Evaluate

Optimize

Convert

Model

Deploy

Model

48

49 of 81

Data-driven engineering

Training samples

Inference update

Camera

Audio

Vibration

49

50 of 81

EI Studio - Embedded ML platform (“AutoML”)

50

51 of 81

TinyML Application�Examples

51

52 of 81

Regression
Classification
Object Detection

Anomaly Detection

Autonomous Navigation

52

53 of 81

53

54 of 81

54

55 of 81

Digital Mic

“UNIFEI”

MFCC Feature

Converter

Obtains an input

Pre-Process

Runs model

postprocesses

Make things happen

16KHz / 16 bits

Sample: [1s]

Output: Image

[13, 50, 1]

Output Dim [1, 3]

Prob ‘iesti’
Prob ‘unifei’
Prob ‘silence’

Check the highest probability

Take actions

KeyWord Spotting (KWS) - Inference

UNIFEI

IESTI

https://youtu.be/XnFYz-RSNe8

55

56 of 81

https://github.com/Mjrovai/wingbeat-mosquito-tinyml

56

57 of 81

57

58 of 81

Mechanical Stresses in Transport

Roll

Yaw

Pitch

Terrestrial

Maritime

Rail

Fork-Lift

Idle

ICTP SciTyniML 21 - Hands on Embedded ML - Motion/Anomaly Detection and Scientific Applications

58

59 of 81

Industrial – Anomaly Detection

IESTI01 2021.2 - Final Group Project: Bearing Failure Detection

59

60 of 81

Predict and classify common Elephant behavior

Aggressive

https://www.hackster.io/dhruvsheth_/eletect-tinyml-and-iot-based-smart-wildlife-tracker-c03e5a

60

61 of 81

61

62 of 81

Computer Vision Main Types

Image Classification

(Multi-Class Classification)

Object Detection

Multi-Label Classification + Object Localization

Cat: 70%

Dog: 80%

62

63 of 81

Computer Vision Main Types

Image Classification

(Multi-Class Classification)

Object Detection

Multi-Label Classification + Object Localization

Cat: 70%

Dog: 80%

63

64 of 81

Forest Fire Detection

IESTI01 - Forest Fire Detection – Proof of Concept

TinyML Aerial Forest Fire Detection

64

65 of 81

Coffee Disease Classification

https://www.hackster.io/Yukio/coffee-disease-classification-with-ml-b0a3fc

João Vitor Yukio Bordin Yamashita

Graduando em Engenharia Eletrônica pela UNIFEI

65

66 of 81

Computer Vision Main Types

Image Classification

(Multi-Class Classification)

Object Detection

Multi-Label Classification + Object Localization

Cat: 70%

Dog: 80%

66

67 of 81

Detecting Objects using TinyML (FOMO)

EdgeAI made simple - Exploring Image Processing (Object Detection) on microcontrollers with Arduino Portenta, Edge Impulse FOMO, and OpenMV

67

68 of 81

Detecting Objects using TinyML (FOMO)

https://youtu.be/MYuc3QISquw

68

69 of 81

Livestock / Wildlife Counting from Drone with FOMO

https://www.hackster.io/jallsonsuryo/livestock-wildlife-counting-from-drone-with-fomo-algorithm-a2f734

69

70 of 81

Other Sensors / MCUs / Models�Examples

70

71 of 81

Atrial Fibrillation Detection on ECG using TinyML

Silva et al. UNIFEI 2021

AD8232 - Single Lead Heart Rate Monitor

Guilherme Silva

Engenheiro - UNIFEI

71

72 of 81

Other TinyML / MCUs Project Examples

[Doc]

Image Classification with ESP32-CAM
Image Classification with Portenta H7

Listening Temperature with Nano 33
COPD Detection with Nano 33

Motion Recognition with RPi Pico
Gesture Recognition with Wio Terminal
Anomaly Detection with XIAO BLE Sense

[Doc]

72

73 of 81

Regression on TinyML

TinyML Made Easy: Exploring Regression - White Wine Quality

Sensor fusion

On-Device IoT-Based Predictive Maintenance Analytics Model: Comparing TinyLSTM and TinyModel from Edge Impulse

73

74 of 81

Reinforcement on TinyML

https://arxiv.org/abs/1909.11236

https://youtu.be/wmVKbX7MOnU

74

75 of 81

Exploring AI at the edge (Computer Vision)

Exploring AI at the Edge!

75

76 of 81

Classifying Images using Smartphones

https://www.hackster.io/mjrobot/app-inventor-edgeml-image-classification-fruit-vs-veggies-b671da

76

77 of 81

The Future of the TinyML (Embedded ML)�

77

78 of 81

78

79 of 81

Vijay Janapa Reddi, Ph. D. | Associate Professor |�John A. Paulson School of Engineering and Applied Sciences | Harvard University |

79

80 of 81

To learn more about Edge AI

80

81 of 81

Thanks

81