1 of 13

Smart Nutritional Automated Assembly Kiosk

PROGRESS REVIEW PRESENTATION: 2

2/27/2025

TEAM C

Abhinandan Vellanki

Anirudh Shrihari

Parth Gupta

Oliver Berton

Rodrigo Catto�

Mentor: Dr. Oliver Kroemer

2 of 13

PR2 Goals

Re-design of bread detection and pick-up maneuver

End effector with fork structure

Localization of tray and bread in assembly area
ROS actions to follow trajectories, pick and place maneuver
Pneumatic actuation using ROS
Initial version of FSM

3 of 13

Progress - Hardware

Fork End Effector

Idea shelved after discussion with Dr. Kroemer
Focus effort on alternative solution

Alternative - Acrylic plate for bread pickup

Slide bread off plate after pickup

4 of 13

Progress - Pneumatic

Pneumatic End Effector integrated via serial to User PC

Set up udev rules to symlink usb port

3 ROS2 Services Available:

Enable
Disable
Eject

Distributing the vacuum channels

High Suction vs High Flow Studies

Contacted Festo/Local Distributors

5 of 13

Progress - Manipulation

Trajectory Following

Generated trajectories with minimum jerk solver between the bins, assembly area, and home
ROS Action Server to follow these trajectories

Pick up maneuver

ROS Action Server to execute pick up
Integration with pneumatic service

Ingredient Testing

Bought 4 meats and 4 cheeses to test which we could most consistently pick up
30 attempts per ingredient
Bologna and Mozzarella were the top two

6 of 13

Progress - Manipulation

7 of 13

Progress - Vision

Top slice cheese detection and localization in bins

8 of 13

Progress - Vision

Bread and Tray Localization in Assembly Area

9 of 13

Progress - State Machine*

Searched a few libraries available

User Recipe

Initialization

Wait For Recipe

For each ingredient:

Go to Pick Position

Get ingredient X,Y,Z in base_link coordinate

Pick Ingredient

Go to Place Position

Place Ingredient

Go to Home Position

*This is an initial version of the State Machine that will be implemented

10 of 13

Challenges - Issue Log

Number	Initiated	Resolved	Participants	Description	Options	Resolution	Justification
1	2/10/25	N/A	Oliver, Abhinandan	The stop skill function in FrankaPy is not functioning correctly.	1.Have Kevin Help us to Debug 2. Switch to ROS Agnostic Version When it Releases	Pending	Pending
2	2/12/25	N/A	Rodrigo, Oliver	The flow rate of the suction end effector is not high enough to pick up bread, can struggle with other ingredients.	1. Introduce items (such as an acrylic plate) to the environment that can be used to pick up the bread. 2. Design a new tool onto the end effector that can pick up the bread. 3. Purchase a higher flow rate end effector.	Pending	Pending
3	2/24/25	2/27/25	Abhinandan	Frankapy handeye camera calibration does not work with ROS2, hard to find other packages that have the correct dependencies.	1. Measure the transform with a ruler. 2. Create our own calibration package. 3. Search for other packages online.	We found a handeye ROS2 package online, and modified it slightly to meet our needs.	This is more accurate than measuring by hand, and allows for easy re-calibration if the system changes.
4	2/24/25	N/A	Parth, Anirudh	Realsense ROS Node cannot dynamically update camera resolution.	1. Use lower, default camera resolution. 2. Debug Issue	Pending	Pending

11 of 13

Challenges - Risk Management

12 of 13

Future Work

Vision:

Bread and Meat pick up calculation

Manipulation:

Pose trajectory for picking and placing

Currently just going to pose, instead generate trajectory that guarantees straight down movement of arm

Meat and Cheese Place
Debug FrankaPy issue with member of Dr. Kroemer’s lab

Pneumatic System:

Try new suction cup designs
Purchase higher flow rate suction generator

Backend

Preliminary primary FSM design and implementation

Hardware

Integrate weighing scales into manipulation and FSM

13 of 13

Future Work

PR3 Demo

Cheese and ham pick and place
Bread pick-up
Initial state machine implementation

Ability to select amount of cheese and ham and automatically execute actions