Embedded Machine Learning�TinyML – Application Demo
Prof. Marcelo José Rovai
UNIFEI - Federal University of Itajubá, Brazil
TinyML4D Academic Network Co-Chair
1
Summary
2
TinyML Applications�Introduction
3
Hardware
Software
TinyML ?
TinyML
4
Hardware
Software
TinyML
TinyML ?
5
| Raspberry Pico (W) | Arduino Nano Sense | ESP 32 | Seeed XIAO Sense / ESP32S3 | Arduino Pro |
32Bits CPU | Dual-core Arm Cortex-M0+ | Arm Cortex-M4F | Xtensa LX6 Dual Core | Arm Cortex-M4F (BLE) Xtensa LX7 Dual Core | Dual Core Arm Cortex M7/M4 |
CLOCK | 133MHz | 64MHz | 240MHz | 64 / 240MHz | 480/240MHz |
RAM | 264KB | 256KB | 520KB (part available) | 256KB / 8MB | 1MB |
ROM | 2MB | 1MB | 2MB | 2MB / 8MB | 2MB |
Radio | (Yes for W) | BLE | BLE/WiFi | BLE / WiFi (ESP32S3) | BLE/WiFi |
Sensors | No | Yes | No | Yes (Sense) | Yes (Nicla) |
Bat. Power Manag. | No | No | No | Yes | Yes |
Price | $ | $$$ | $ | $$ | $$$$$ |
Hardware
6
Power
CPU Power / Memory
Application Complexity
XIAO
Hardware vs. Apps
7
8
9
10
Personal Assistant
11
Personal Assistant
12
Personal Assistant
KWS
13
“Cascade” Detection: multi-stage model
14
Digital Mic
“Yes”
MFCC Feature
Converter
Obtains an input
Pre-Process
Runs model
Post-Processes
Make things happen
16KHz / 16 bits
Sample: [1s]
Output: Image
[13, 49, 1]
Output Dim [1, 4]
If Probability of YES is greater than 80%
Take actions
KeyWord Spotting (KWS) - Inference
Time-Series Raw data
1D-Image
15
How to Train our ML Model?
16
Collect
Data
Preprocess
Data
Design a
Model
Train a
Model
Evaluate
Optimize
Convert
Model
Deploy
Model
Make
Inferences
Machine Learning Workflow (“What”)
17
Collect
Data
Preprocess
Data
Design a
Model
Train a
Model
Evaluate
Optimize
Convert
Model
Deploy
Model
Make
Inferences
Machine Learning Workflow (“Where”)
18
Collect
Data
Preprocess
Data
Design a
Model
Train a
Model
Evaluate
Optimize
Convert
Model
Deploy
Model
Make
Inferences
Machine Learning Workflow (“How”)
19
Collect
Data
Preprocess
Data
Design a
Model
Train a
Model
Evaluate
Optimize
Convert
Model
Deploy
Model
Make
Inferences
Machine Learning Workflow (“How”)
20
Collect
Data
Preprocess
Data
Design a
Model
Train a
Model
Evaluate
Optimize
Convert
Model
Make
Inferences
Preprocess
Data
Design a
Model
Train a
Model
Evaluate
Optimize
Convert
Model
Deploy
Model
21
EI Studio - Embedded ML platform (“AutoML”)
22
XIAO-ESP32S3-KWS (Edge Impulse)
23
Speech Commands Dataset (reduced set)
24
1 second sample@16KHz raw data -> 16,000 features
Processed features -> 637 features (13 x 49)
49
13
Pre-Processing (MFCC)
25
26
27
28
29
30
31
32
33
34
XIAO-ESP32S3-CAM-Fruits-vs-Veggies-v1-ESP-NN (Edge Impulse)
35
Fruits and Vegetables Image Recognition Dataset (Kaggle)
36
37
38
Serial Monitor
ESP-NN
(96 x 96 x 3) – RGB
39
ESP - CAM Xtensa LX6 240 MHz
171 ms
ARDUINO Pro ARM H7
480 MHz
45 ms
XIAO ESP32S3 Xtensa LX7 240 MHz
135 ms
40
41
D4
D5
GND
3.3V
MPU6050 SCL --> XIAO D5
MPU6050 SDA --> XIAO D4
MPU6050 VCC --> XIAO 3.3V
MPU6050 GND --> XIAO GND
42
43
XIAO-ESP32S3-Motion-Classification
(Edge Impulse)
44
/ WiFi
SciTinyML - 2023 - Marcelo Rovai Presentation (day 2)
45
46
Raw Data from sensor
300 Raw Features
Spectral Analysis
Features
Classes
NN Classifier
Sliding Window: 200 ms
Manual Feature
Extraction
TinyML under the hood: Spectral Analysis
47
48
49
50
51
Preprocess
Data
units = 63
63 Features
Classes
Input Layer
Hidden Layer 1
Hidden Layer 2
Output Layer
(63x20)
52
53
54
55
56
Maritime
Lift
57
To learn more …
58
To learn more …
On the TinyML4D website, You can find lots of educational materials on TinyML. They are all free and open-source for educational uses – we ask that if you use the material, please cite them! TinyML4D is an initiative to make TinyML education available to everyone globally.
59
TinyML4D Show&Tell Presentations
Date August 31st, 2023 May 25th, 2023 | Video Video here when ready Video here when ready | |
April 20th, 2023 | ||
March 30th, 2023 | ||
February 23rd, 2023 | ||
January 26th, 2023 | ||
December 1st, 2022 | ||
October 27th, 2022 |
TinymML4D Academic Network Show and Tell Main Index.
The TinyML4D Academic Network Students should use this form to sign up for the latest presentations.
https://forms.gle/ic52HZMqVv4pBrkP7 2
The Show and Tell are typically held at 2 pm UTC on the last Thursday of each month and will take place in this Zoom room.
https://zoom.us/j/95229860797 1
Meeting ID: 952 2986 0797�Passcode: 141278
60
Q & A
61
Marcelo Rovai was born in São Paulo and holds a Master’s degree in Data Science from the Universidad del Desarrollo (UDD) in Chile and an MBA from IBMEC (INSPER) in Brazil. He graduated in 1982 as an Engineer from UNIFEI, Federal University of Itajubá, with a specialization from Polytechnic School of São Paulo University (USP), both institutions located in Brazil.
Rovai has experience as a teacher, engineer, and executive in several technology companies such as CDT/ETEP, AVIBRAS Aeroespacial, SID Informática, ATT-GIS, NCR, DELL, COMPAQ (HP), and more recently at IGT as a VP and a Senior Advisor for Latin America.
Marcelo Rovai publishes articles about electronics on websites such as MJRoBot.org, Hackster.io, Instructables.com, and Medium.com. Furthermore, he is a volunteer Professor at the UNIFEI in Brazil and a lecturer at several Congresses and Universities on IoT and TinyML. He is an active member and a Co-Chair of the TinyML4D group, an initiative to bring TinyML education to developing countries.
62
TinyML4D Academic Network
Brazil
Morocco
Panama
Ethiopia
Ethiopia
63
Thanks
64