Spring 23: ESE3600 TinyML Course Plan : Student Copy

	A	B	D	E	F	G	H	I	J	K	L	M	N
1	#	Date	Module	Lecture	Topics	Slides & Videos	Tutorial	Readings	Assignment	Quizzes	Assignments Out	Assignments Due	Recitation Topics

2	L1	01/12/2023	Part I	Introduction	What is (tiny) Machine Learning,TinyML Applications, TinyML Challenges, Intro to Software/Hardware used in the course	L1.1 Welcome to TinyML [L1.1 Video], L1.2 What will you learn? [L1.2 Video], L1.3 Why is TinyML important for the future, [L1.3 Video], L1.4 How do we Enable TinyML, [L1.4 Video] L1.5 Challenges for TinyML (A) Hardware, [L1.5 Video] L1.6 Challenges for TinyML (B) Software, [L1.6 Video], L1.7 Challenges for TinyML (C) ML, [L1.7 Video]	Primers
3	L2	01/17/2023		ML Paradigm	Finding Patterns, Fitting Lines & Loss, Gradient Descent, Intro to Neural Networks	L2.1 Challenges for TinyML (D) Model Compression, [L2.1 Video] L2.2 Machine Learning Paradigm, [L2.2 Video] L2.3 Thinking about loss, [L2.3 Video] L2.4 Minimizing Loss, [L2.4 Video] L2.5 First Neural Network, [L2.5 Video]	T1.1_GradientDescent	R1.1_FindingPatterns R1.2_MoreNeuralNetworks R1.3_NeuralNetworksCaseStudies
4	L3	01/19/2023		Building Blocks of Deep Learning	Single-Layer & Multi-Layer Neural Networks, Regression, Classification, Dataset Split (Train,Validation,Test)	L3.1 Understanding Neurons.pdf V3.1 Understanding Neurons L3.2 Introduction to classification.pdf V3.2 Introduction to classification L3.3 Training, val, testing data.pdf V3.3 Training, val, testing data	Tutorial1.2_NeuralNetwork Tutorial1.3_ExploringCategorical	R1.4_Initialization and Learning R1.5_Coding Stepback R1.6_Realities of coding R1.7_Classification
5	L4	01/24/2023	Fundamentals of TinyML	Exploring Machine Learning Scenarios	Convolutions, Convolutional Neural Networks, Mapping Features to Labels	L4.1 Introducing Convolutions.pdf V4.1 Introducing Convolutions L4.2 From DNN to CNN.pdf V4.2 From DNN to CNN L4.3 Working with Images.pdf V4.3 Working with Images	Tutorial1.4_ConvolutionalNeuralNetworks	R1.8_Neural Network Layers R1.9_Convolutions1 R1.10_Convolutions2 R1.11_Mapping Features to Labels	A1_DNN
6	L5	01/26/2023	(~3 weeks)	Building a Computer Vision Model	Image Datasets, Overfitting, Image Augmentation, Dropouts, Explore other Loss Functions & Optimizers, CV Application	L5.1 Avoiding Overfitting.pdf V5.1 Avoiding OverfittingL5.2 What we have Learned so far.pdf V5.2 What we have learned so far L5.3 What you'll learn in Part 2.pdf L5.3 What you will learn in Part 2	Tutorial1.5_ComplexImages	R1.12_Complex Images R1.13_TFDSForImageData R1.14_DropoutRegularization R1.15_ExploreOtherApproaches R1.16_Recap
7	L6	01/31/2023	(~3 weeks)	Responsible AI	Goal, User Requirements, Ethics	L6.1 Preview of TinyML Applications.pdf V6.1 Preview of TinyML Applications L6.2 Responsible AI.pdf V6.2 Responsible AI			A2_CNN	Quiz1 (L1 - L3)
8
9
10	L7	02/02/2023	Part 2	AI Lifecycle & ML workflow, ML on Mobile and Edge Devices (Pt.1)	AI Infrastructure (Data Engineering, Model Engineering, Model Deployment), Model Workflow Pipeline, TensorFlow Lite Converter	L7.1 ML Lifecycle.pdf V7.1 ML Lifecycle L7.2 ML Workflow.pdf V7.2 ML Workflow L7.3 Introduction to TFLite.pdf V7.3 Introduction to TFLite	Tutorial2.1_TensorFlowLite_Converter	R2.1_TheRoleOfSensors R2.2_ML Lifecycle R2.3_MLWorkflowDetails R2.4_TensorFlow
11	L8	02/07/2023		ML on Mobile and Edge Devices (Pt.1) continued.	TensorFlow Lite Optimizations, and Quantization, Quantization Aware Training (QAT)	L8.1 TFLite Optimization and Quantization.pdf V8.1 TFLite Optimization and Quantization L8.2 Post-training Quantization.pdf V8.2 Post training Quantization	Tutorial2.2_TFLiteOptimizations	R2.5_HowToUseTFLiteModels	A3_Quantization
12	L9	02/09/2023		ML on Mobile and Edge Devices (Pt.2)	Post Training Quantization (PTQ), Quantization Aware Training (QAT), Inference using TF vs. TFLite, Model Conversion and Deployment	L9.1 Quantization-aware training - code.pdf V9.1 Quantization-aware training - code L9.2 Quantization-aware Training - Concepts.pdf V9.2 Quantization aware Training - Concepts L9.3 Inference Engine TF vs. TFLite.pdf V9.3 Inference Engine TF vs TFLite	Tutorial2.3_QuantizationAwareTraining	R2.6_WhyAre8BitsEnough? R2.7_ConversionAndDeployment
13	L10	02/14/2023	Applications of TinyML	Keyword Spotting (KWS)	Introduction, Challenges & Constraints, KWS Architecture, Dataset creation (collection/pre-processing), Spectograms and MFCCs, KWS Model Training, Evaluation Metrics, Streaming Audio, Cascade Architectures	L10.1 What is Keyword Spotting.pdf V10.1 What is Keyword Spotting L10.2 Keyword Spotting Challenges and Constraints.pdf V10.2 Keyword Spotting Challenges and Constraints L10.3 Keyword Spotting Datasets.pdf V10.3 Keyword Spotting Datasets L10.4 KWS Data Pre-Processing.pdf V10.4 KWS Data Pre Processing L10.5 Keyword Spotting Model.pdf V10.5 Keyword Spotting Model L10.6 Metrics for KWS.pdf V10.6 Metrics for KWS L10.7 Streaming Audio.pdf V10.7 Streaming Audio L10.8 Cascade Architectures.pdf V10.8 Cascade Architectures		R2.8_KWSArchitecture R2.9_WakeWordsDatasetCreation R2.10_SpectrogramsAndMFCCs R2.11_MonitoringTrainingInColab R2.12_KWSInTheBigPicture		Quiz2 (L4 - L5)
14	L11	02/16/2023	(~4.5 weeks)	Visual Wake Words (VWW)	Introduction, Challenges, VWW Dataset, Neural Network Architectures, MobileNets, Transfer Learning for VWW, Evaluation Metrics	L11.1 What is Visual Wake Words.pdf V11.1 Visual Wake Words (VWW) L11.2 Visual Wake Words Challenges.pdf V11.2 Visual Wake Words Challenges L11.3 VWW Data Collection and Processing.pdf V11.3 VWW Data Collection and Processing L11.4 VWW Model.pdf V11.4 VWW Model L11.5 Training VWW in Colab.pdf V11.5 Training VWW in Colab L11.6 Metrics for VWW.pdf V11.6 Metrics for VWW	Tutorial2.4_KeywordSpotting	R2.13_Introduction to Visual Wake Words R2.14_Data Privacy With Images R2.15_The Math Behind MobileNets Efficient Computation R2.16_Common Myths and Pitfalls about Transfer learning R2.17_VWW Summary
15			(~4.5 weeks)
16
17
18	L12	02/21/2023		Anomaly Detection	Introduction, Applications, Anomaly Detection in Industry, Datasets (Real & Synthetic), Unsupervised Learning (K-means & Autoencoders), Model Training, Evaluation Metrics, Threshold choice	L12.1 What is Anomaly Detection.pdf V12.1 What is Anomaly Detection L12.2 Challenges with Anomaly Detection.pdf V12.2 Challenges with Anomaly Detection L12.3 Anomaly Detection Datasets.pdf V12.3 Anomaly Detection Datasets L12.4 Autoencoders.pdf V12.4 Autoencoders	Tutorial2.5_AnomalyDetection	R2.18_Introduction R2.19_Industry4.0 R2.20_Real and Synthetic Data R2.21_AutoencoderModelArchitecture R2.22_Anomaly Detection Summary MIMII Dataset
19	L13	02/23/2023		Data Engineering	Importance, Speech Commands example, Reusing and Adapting Existing Datasets, Responsible Data Collection	L13.1 What is Data Engineering.pdf V13.1 What is Data Engineering? L13.2 Speech Commands.pdf V13.2 Speech Commands L13.3 Crowdsourcing Data for the Long Tail.pdf V13.3 Crowdsourcing Data for the Long Tail		R2.23_Data Engineering Speech Commands Dataset	A4 due \| A5 out
20	L14	02/28/2023		Responsible AI Development	Data Collection, Bias in ML, Biased Datasets, Fairness, Google's What-if Tool	L14.1 Responsible TinyML - What Data Will Be Collected.pdf V14.1 Responsible TinyML - What Data will be Collected? L14.2 Responsible TinyML - Is the Dataset Biased.pdf V14.2 Responsible TinyML - Is the Dataset Biased? L14.3 Responsible TinyML - Ensuring the model is fair.pdf V14.3 Responsible TinyML - Ensuring the model is fair	Tutorial2.6_Autoencoders			Quiz3 (L7 - L9)
21	L15	03/02/2023	Part 3	Getting Started (C++, SW/HW Setup, Sensors)	Module Overview, Kit introduction, Sensors	L15.1 TinyML Application Deployment Preview.pdf V15.1 TinyML Application Deployment Preview L15.2 TinyML Course Kit Overview.pdf V15.2 TinyML Course Kit Overview	Tutorial3.2_HardwareAssembly, Tutorial3.3_SoftwareAssembly, Tutorial3.4_StepByStep, Libraries ESE3600_Github Tutorial 3.1	Reading 3.1: C++ Primers, Reading 3.2: Recap"
22		03/07/2023		SPRING BREAK
23		03/09/2023		SPRING BREAK
24	L16	03/14/2023		Embedded hardware and Software	Overview of Embedded systems, Comm Protocols, Memory, Software, Frameworks	L16.1 Embedded Systems.pdf V16.1 Embedded Systems L16.2 Embedded Computing Hardware.pdf V16.2 Embedded Computing Hardware L16.3 Embedded IO.pdf V16.3 Embedded IO L16.4 Embedded System Software.pdf V16.4 Embedded System Software L16.5 Embedded ML Software.pdf V16.5 Embedded ML Software	Tutorial3.5_TFLM, Tutorial3.6_TestingSensors Util: Image Viewer	Reading 3.3: Powering your System, Reading 3.4: Embedded Systems	A5 due\| A6 out
25	L17	03/16/2023		TensorFlow Lite Micro	Usage, Interpreter, Model Format, Serialization Libraries, Memory Allocation, NN Operations	L17.1 TFLite Micro - The Big Picture.pdf V17.1 TFLite Micro - The Big Picture L17.2 TFLite Micro - Interpreter.pdf V17.2 TFLite Micro - Interpreter L17.3 TFLite Micro - Model Format.pdf V17.3 TFLite Micro - Model Format L17.4 TFLite Micro - Memory Allocation.pdf V17.4 TFLite Micro - Memory Allocation L17.5 TFLite Micro - NN Operations.pdf V17.5 TFLite Micro - NN Operations	Tutorial3.7_Train_Deploy_KWS_Model, Tutorial3.8_Custom_Dataset_Collection, Tutorial3.8.1_Custom_Model_Deployment	Reading 3.5: TensorFlow Lite Micro, Utility: Model converter
26	L18	03/21/2023	Deploying TinyML	Keyword Spotting(KWS), Custom Dataset engineering	sqrt(pow(x,2) - pow(y,2));	L18.1 KWS Application Architecture.pdf V18.1 KWS Application Architecture L18.2 KWS Initialization.pdf V18.2 KWS Initialization L18.3 KWS Pre-Processing.pdf V18.3 KWS Pre-Processing L18.4 KWS Inference.pdf V18.4 KWS Inference L18.5 KWS Post-processing.pdf V18.5 KWS Post-processing L18.6 KWS Summary.pdf V18.6 KWS Summary		Reading 3.6: MCU memory
27	L19	03/23/2023	(~3.5 weeks)	Visual Wake Words/Person Detection	Component Architecture, Preprocessing, Multimodal, Multi tenancy, TFLite Micro	L19.1 Person Detection - Application Architecture.pdf V19.1 Person Detection - Application Architecture L19.2 Person Detection - Multi-modal.pdf V19.2 Person Detection - Multimodal L19.3 Person Detection - Multi-Tenancy.pdf V19.3 Person Detection - Multi-Tenancy L19.4 Multi Tenancy in TensorFlow Lite Micro.pdf V19.4 Multi-Tenancy in TensorFlow Lite Micro	Tutorial3.9_PersonDetectionDeployment			Quiz4 (L10 - L15)
28	L20	03/28/2023		Gesturing Magic Wand (gesture recognition)	TinyML Ecosystem, Magic Wand Architecture, Preprocessing, Deployment	L20.1 TinyML Sensor Ecosystem.pdf V20.1 TinyML Sensor Ecosystem L20.2 Magic Wand Application.pdf V20.2 Magic Wand Application L20.3 Magic Wand Application Architecture.pdf V20.3 Magic Wand Application Architecture	Tutorial3.10_Train_Deploy_Magic_Wand, Tutorial3.11_Custom_dataset_Magic_Wand, Tutorial3.12_Deploying_MultiTenant	Reading 3.7: Anatomy of an IMU	A6 due \| A7 out
29	L21	03/30/2023		Introduction to the Project		L21 TinyML Final Project Overview			Final Project Details
30	L22	04/04/2023		Project Discussions
31	L23	04/06/2023	Part 4: TinyML Ops (~2 weeks)	Special Topics: MCUNet	Ji Lin, MIT, presents https://mcunet.mit.edu
32	L24	04/11/2023	Part 4: TinyML Ops (~2 weeks)	Special Topics: Safe Autonomous Systems	Wenchao Li, Boston University				A7 due
33	L25	04/13/2023		Final Quiz						Final Quiz (L1 - L21)
34	L26	04/18/2023		Demo Day
35	L27	04/20/2023		Responsible AI Development	Moral Decision Making with ML-enabled Systems	https://tinyurl.com/moral-machines
36	L28	04/25/2023		Final Project Presentations
37		05/04/2023		Project Report & Code Submission	No lecture
38
39				Note: Demos and Final project dates are firm. No extensions.
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100