Team E5:
Sous-Chef
Design Review
Kelly Cha, Michael Crotty, Sebastian Peralta
Recipes
Use Case
Inventory
Scan
Remind
03
04
01
02
Intelligently recommend recipes to use ingredients before they expire
Track consumables and their expiration dates
Recognize produce and smart labels & ability to input custom items
Calendar view and push notifications of expiring produce
Problem: 30-40% of the US food supply is wasted each year.
ECE Areas: Signals, Software
Existing Attempts
S21 Team D3, S22 Team B3, S22 Team B6, Samsung
Design Requirements
5s max latency per scan
50s max startup from door open
98% scan accuracy for smart labels
80% accuracy for 20 common produce
1lb weight limit - held up by 3M command strip
Visual feedback for scans
Low-power when not in use
Recommended recipe generated within 2 minutes
Store expiration date of items
Recipes prioritize likely to expire items
Recipes can be requested by cuisine and dietary restriction filters
Save generated recipes
Why Smart Labels?
Barcodes require hardcoding
Hosted & maintained by the manufacturer
Managed by Consumer Brands Association
Allows for scraping data (Identifier & Allergens)
Increasing adoption by companies
Solution Approach
Self Contained
Expandable
Financially Accessible
Considerations
Public Health, Safety, Welfare
Global, Cultural, Social Factors
Environmental Factors
Economic Factors
System Block Diagram
Scanner #1
Scanner #2
Phone
Laptop
Tablet
OpenCV
Llama2
Flutter App
SQLite
(DB)
Stream captures over WIFI
Web App
Home Server
(Kria KV260)
Hardware
Implementation Plan
Raspberry Pi Zero 2 W
Raspberry Pi Cameras
AMD Kria KV260
3D Printed Cases
Screen for visual feedback and to handle errors
Software
Implementation Plan
App UI Mockup
App UI Mockup
Testing & Validation
Functionality | Measurement | Test Input | Expected Output | Risks |
Scan Smart Label | 98% Accuracy | Open fridge & attempt to scan Pepsi bottle | One Pepsi bottle entered into DB | Glare messing with qr code detection & reading |
Scan Produce | 80% Accuracy | Open fridge & attempt to scan an apple | One apple should be entered into DB (w/ estimate expiration) | Glare (especially on fruits) and misclassification |
Scan Unknown Item | 5s time out for not detecting Smart Label or Produce | Open fridge & attempt to scan unknown ingredient | Item entered & flagged as custom ingredient in DB | Attempting to classify unknown items |
Multiple Scanners | 5s max latency per scan | Have 2 scanners on same network | Both scanners should operate without interfering | Handling 2 scanners sending data at once |
Generate Recipe | 90% Accuracy, Recipes generated within 2 minutes | Fill DB with entries to make a common food like tacos | Generate a recipe for a known dish (like tacos) that exclusively uses ingredients in the DB, within 2 minutes | LLM Hallucination, suggesting to use extra ingredients that are not in the inventory |
Notifications | ⎯ | Create food entry in DB that will expire soon | Push notification of expiring food item 2 days before expiration | Ensuring DB refreshes every day |
Schedule
Work Distribution
Michael: Server & Scanner hardware, Scanner software
Sebastian: Databases, Backend app
Kelly: LLM, Frontend app/ UI design, Help with Backend app