1 of 83

Critical Design Review

Date: Monday, May 2, 2022

2 of 83

Table of Contents

Team TouRI
Advisors/ Sponsors
Project Description
Use Case
System-level Requirements
Functional Architecture
Cyber Physical Architecture
Current system status
Project management
Key Fall activities

3 of 83

TouRI Advisors and MRSD Project mentors

Zackory Erikson

Yonatan Bisk

RCHI Lab, RI CMU

CLAW Lab, LTI CMU

Assistant Professor

John Dolan

MRSD Program Director

CMU

Principal Systems Scientist

Dimi Apostolopoulos

Senior Systems Scientist

NREC

CMU

4 of 83

Team H - TouRI

Prakhar Pradeep

Jigar Patel

Shruti Gangopadhyay

Jashkumar Diyora

Shivani Sivakumar

Software and�Interface Lead

Hardware and �Sensors Lead

Autonomous Navigation�System Lead

Perception�Lead

Autonomous Manipulation Lead

5 of 83

Project Description

6 of 83

Project Description

Differential drive base

To facilitate robot locomotion

Manipulator

To facilitate interaction with the environment

API for Subsystems

Quickly deploy new use cases

7 of 83

Project Description

8 of 83

System Requirements

9 of 83

Functional Requirements

FRx	Description
FR 1	Facilitate remote user access
FR 2	Receive inputs: room location and teleoperation
FR 3	Plan path & traverse to the desired location
FR 4	Detect and avoid obstacles
FR 5	Provide traversal feedback to user on map
FR 6	Detect objects for object grasping and placement
FR 7	Estimate grab points and pose of the objects
FR 8	Plan manipulator motion for object grasping and placement
FR 9	Autonomously grasp and place object
FR 10	Provide gimbal control of tablet
FR 11	Provide a video call interface to user
FR 12	Provide traversal feedback

The system shall

10 of 83

Performance Requirements

PRx	Description	Justification / Assumptions
PR1	Traverse on hard, flat indoor floors reliably at 0.4 m/s during teleoperation	Considering the terrain of the defined environment
PR2	Reach the desired location within 30 minutes	Assuming latency in receiving user input, obstacle detection, avoidance and arm manipulation
PR3	Receive user input from interface with a latency less than 5 seconds	Assuming >100mbps broadband connectivity is available to user and robot
PR4	Plan global path to the desired location within 3 minutes	The algorithms of the system will be optimized to use onboard compute capability to achieve PR4
PR5	Detect and avoid obstacles (during autonomous navigation and teleoperation) with mAP of 80%	The mentioned accuracy accounts for dynamic/static obstacles that lie within the FOV of sensing modalities
PR6	Detect objects with a precision of 70% and recall of 60%	For predefined set of objects in the environment and appropriate lighting

The system will

In order to test our system, we had to add metrics to our functional requirements. We will be discussing thse in detail in the rest of this presentation, so I won’t dive much into these at the moment.

Tq Shivani, so we attempted to add metrics to the functional requirements to derive the performance requirements you see on the slide.

The first performance requirement pertains to the traversal of the Touri robot. We assume that Touri is deployed only indoors with hard and flat floors. From our experiments with teleoperating the robot and assuming presence of obstacles, we have concluded that a speed of 0.4 m/s is achievable and reliable.

Next, Touri shall reach the desired location within 30 minutes. This metric assumes the latency in receiving input, detecting, avoiding obtacles and manipulating the arm to open doors.

The third performance requirement states that the robot receives user input from the interface with a latency of 5 seconds considering both the user and robot, have greater than or equal to 200 mbps of internet connectivity.

The fourth requirement assumes that the algorithms of the navigation sub-system are optimized to utilize onboard computing to plan a global path to the desired location within 3 minutes.

Next, Touri observes a minimum of 80% accuracy in detecting and avoiding obstacles during autonomous navigation and teleoperation. We assume that there may exist some obstacles that lie outside the FOV of the sensing modalities. For example, …

The sixth requirement asserts that assuming door knobs are only present in the planned environment and some variation in indoor lighting conditions exist, Touri is able to detect and grab the door-knob with 60% accuracy.

11 of 83

Performance Requirements

PRx	Description	Justification / Assumptions
PR7	Estimate grab points and pose of objects with a precision of 65%	For predefined set of objects in the environment and appropriate lighting
PR8	Plan manipulator motion to grasp object within 3 minutes	The algorithms of the system will be optimized to use onboard compute capability to achieve this
PR9	Grasp or place object within 5 minutes, with 70% success rate (7 successful trails out of 10)	Accounting for slippage of end-effector due to physical properties (torque required, material) of objects
PR10	Provide gimbal motion of 60 degrees in pitch and 120 degree in yaw for the display device	To mimic natural human perceptive FOV
PR11	Provide a 1080*720 resolution video call interface for user to interact with surroundings with a lag less than 2 seconds	The system aims to provide a HD and a real-time experience to the user
PR12	Provide traversal feedback to user every 5 seconds	The system aims to provide a real-time experience to the user

The system will

Next, similar to navigation sub-system, we assume that the manipulation algorithms are optimized to use the onboard compute capability to plan the manipulator motion to open doors within 3 minutes.

The eighth requirement states that the TouRI Robot opens doors with 75% accuracy within 5 minutes. This metric accounts for the various attempts that the manipulator might have to go though due to end-effector slippage caused by the doorknob’s physical properties.

Next, to allow the user to have as close as an in-person experience as possible, we decided to provide a gimbal motion of 60 degrees in pitch and 120 degree in yaw. These numbers are similar to a human’s natural FOV.

Finally, we aim to also make this experience real-time and high definition. These become our last two performance requirements where the video-call resolution will be 1080*720p, have a lag less than 2s and a traversal feedback to user every 5 seconds.

12 of 83

Nonfunctional Requirements

NFx	Description
NF1	Be deployed in a defined environment in the Robotics Institute.
NF2	Provide a user-friendly interface with a small learning curve.
NF3	Be modular with APIs to facilitate further development
NF4	Be situationally aware
NF5	Be friendly to facilitate natural interaction

The system will

13 of 83

Functional Architecture

14 of 83

Functional Architecture

15 of 83

Cyberphysical Architecture

16 of 83

Cyberphysical Architecture

17 of 83

System Description & Current System Status

18 of 83

Stretch RE1 Robot
Manipulator
Mobile Base
Intel RealSense D435i
9DOF IMU
RP-LIDAR A1
Wheel Encoders
Pan Tilt Display

Overall System

RP-Lidar A1

Manipulator

Mobile Base

RealSense D435i

Pan Tilt Display

19 of 83

Current System Status

KEY

✅ Finished ☑️ Proof-of-concept ⏳TODO

✅ Robot can autonomously navigation to room selected by the user

✅ Robot can tele-operate autonomously inside room using the interface

✅ Robot can autonomously grasp an object (cup) as specified by the user

✅ Robot can autonomously place an object (cup) when specified by user

✅ 3D clustering pipeline and 2D detection of objects such as Cup, Sanitizer, Duster, Pringles, CMU Cup

✅ Software control of standalone manufactured gimbal

✅ App controlled user kill switch

⏳ Further integration with interface

⏳ Integration of perception subsystem with manipulation subsystem

⏳ Shared autonomy between teleoperation and autonomous modes

⏳ Integration of interface with gimbal subsystem

⏳ Generalization of autonomous manipulation on custom objects

20 of 83

Interface sub-system

21 of 83

Interface subsystem - Description

Bot commands

Peer-to-peer video telephony

UI

Robot

Cloud

database

The UI is an iOS and Android mobile application through which the user can send commands

to the robot and receive updates from the robot. The UI displays real-time audio and video feed of the robot and provides a human-centric design to send autonomous and teleoperation commands

to the robot.

There are two communication links– the videotelephony link and the commands link. The

videotelephony link is a peer-to-peer communication link that streams the audio and video of the user and the robot. For heavy exchange of data, like with audio and video streams, peer-to-peer architecture is faster and more reliable as compared to a centralized architecture. The command link is a client-server-client link, where the robot and the UI are the clients, and a cloud database is the server. The cloud database (Google’s Firebase Firestore) provides real-time updates to listening clients. This architecture provides a single source of truth to both clients and keeps a track of the commands and the robot state. Lastly, the ease of scalability for this architecture opens many different future potentials for the project like swarm telepresence. While a proof-of-concept for the client-server-client architecture has already been developed, the peer-to-peer

22 of 83

Interface subsystem - Status

KEY TASKS

✅ Tele-op navigation controls�✅ Tele-op manipulation controls

✅ Database-link

✅ Bot-side architecture

✅ UX development

☑️ Tele-op gimbal controls

☑️ Autonomous commands

⏳ Audio-video link

KEY

✅ Finished ☑️ Proof-of-concept ⏳TODO

23 of 83

Interface subsystem - Status

KEY TASKS

✅ Tele-op navigation controls�✅ Tele-op manipulation controls

✅ Database-link

✅ Bot-side architecture

✅ UX development

☑️ Tele-op gimbal controls

☑️ Autonomous commands

⏳ Audio-video link

KEY

✅ Finished ☑️ Proof-of-concept ⏳TODO

24 of 83

Interface subsystem - Status

KEY TASKS

✅ Tele-op navigation controls�✅ Tele-op manipulation controls

✅ Database-link

✅ Bot-side architecture

✅ UX development

☑️ Tele-op gimbal controls

☑️ Autonomous commands

⏳ Audio-video link

KEY

✅ Finished ☑️ Proof-of-concept ⏳TODO

CORE

MANIPULATION

NAVIGATION

PERCEPTION

HARDWARE

25 of 83

Interface subsystem - Status

KEY TASKS

✅ Tele-op navigation controls�✅ Tele-op manipulation controls

✅ Database-link

✅ Bot-side architecture

✅ UX development

☑️ Tele-op gimbal controls

☑️ Autonomous commands

⏳ Audio-video link

KEY

✅ Finished ☑️ Proof-of-concept ⏳TODO

26 of 83

Interface subsystem - Status

KEY TASKS

✅ Tele-op navigation controls�✅ Tele-op manipulation controls

✅ Database-link

✅ Bot-side architecture

✅ UX development

☑️ Tele-op gimbal controls

☑️ Autonomous commands

⏳ Audio-video link

KEY

✅ Finished ☑️ Proof-of-concept ⏳TODO

27 of 83

Interface subsystem - Status

FUTURE WORK

⏳ Audio-video link

☑️ CV Teleop

☑️ Generalized intelligence/reasoning

KEY

✅ Finished ☑️ Proof-of-concept ⏳TODO

SPEED: X2

28 of 83

Interface subsystem - Status

POCs & FUTURE WORK

⏳ Audio-video link

☑️ CV Teleop

☑️ Generalized intelligence/reasoning

KEY

✅ Finished ☑️ Proof-of-concept ⏳TODO

29 of 83

Interface subsystem - Status

30 of 83

Interface subsystem - Evaluation

31 of 83

Navigation sub-system

32 of 83

Navigation subsystem - Description

33 of 83

Navigation subsystem - Description

34 of 83

Navigation subsystem - Status

KEY

✅ Finished ☑️ Proof-of-concept ⏳TODO

✅ Robot setup to interface software

✅ Sensor data integration

✅ Environment map generation

✅ Autonomous navigation stack setup

✅ Robot localization using AMCL with laser scan and fused odometry (IMU and wheel encoders)

✅ Robot localization using EKF with laser scan, IMU and AMCL pose fusion

✅ Autonomous navigation stack tuning

✅ Dynamic obstacle avoidance integration

☑️ Integration of navigation sub-system with interface

⏳ Further integration with interface

⏳ Autonomous dynamic obstacle avoidance in teleoperation mode

35 of 83

Subsystem Validation:

Robot localization within the map.
Path planning to the desired location.
Autonomous navigation with obstacle avoidance (static and dynamic).

Requirements Satisfied:

M.P.2: Robot travels 50 m to reach the desired location within 12 minutes.
M.P.4: Robot plans a path to the desired location within 3 minutes.
M.P.5: Robot detects and avoids obstacles with 80% success rate.

Subsystem Performance Results:

Robot travels 60m to reach the desired location within 4.15 minutes (average of 10 trials).
Robot plans a path to the desired location within 2 seconds (average of 10 trials).
Robot detects and avoids obstacles with 90% success rate (9 successful trials out of 10).

Failure case- Robot is unable to avoid obstacles that are too sudden and too close.

Navigation subsystem - Validation & Performance

36 of 83

60 m autonomous traversal time

Time for path planning

Obstacle avoidance success

Average: 4.15 mins

Average: 2 secs

90% success rate

Navigation subsystem - Performance Results

37 of 83

Navigation subsystem - Demonstration Video

38 of 83

Perception sub-system

39 of 83

Perception subsystem - Description

The perception sub-system detects objects to be grasped and sends the pose information to the manipulation sub-system.

40 of 83

✅ Integrated Mediapipe in the perception package

✅ Implemented ROS nodes for 2D cups pose estimation, hands pose estimation using Mediapipe’s pre-trained models

✅ Developed 3D point-cloud pre-processing pipeline for plane segmentation, oriented bounding box calculation, point cloud cropping, and clustering of objects

✅ Convert 2D Detections to 3D detections using depth-image from intel-real sense and camera intrinsics

✅ Combine 2D detections and 3D clusters to estimate grasp points of objects

✅ Train on custom dataset for other objects

⏳ Full integration perception sub-system with Manipulation subsystem

⏳ Integrate perception sub-system with Interface subsystem

Perception Subsystem – Status

Key

✅ Finished ☑️ On-Going ⏳TODO

41 of 83

Perception Subsystem – Validation and Performance

Perception subsystem validation

3D clustering pipeline - Detection of cluster centroids.
2D detections of objects (Cup, Sanitizer, Duster, Pringles, CMU Cup)

Requirements Satisfied:

M.P.6: Detect objects with a precision of 70% and recall of 60%.

Subsystem Performance Results:

Object detection ( 5 Classes )
Precision - 80.43%
Recall - 97.36%

42 of 83

Perception Subsystem – Validation and Performance

Perception subsystem validation

3D clustering pipeline - Detection of cluster centroids.
2D detections of objects (Cup, Sanitizer, Duster, Pringles, CMU Cup)

Requirements Satisfied:

M.P.6: Detect objects with a precision of 70% and recall of 60%.

Subsystem Performance Results:

Object detection ( 5 Classes )
Precision - 80.43%
Recall - 97.36%

43 of 83

Perception Subsystem – Validation and Performance

Perception subsystem validation

3D clustering pipeline - Detection of cluster centroids.
2D detections of objects (Cup, Sanitizer, Duster, Pringles, CMU Cup)

Requirements Satisfied:

M.P.6: Detect objects with a precision of 70% and recall of 60%.

Subsystem Performance Results:

Object detection ( 5 Classes )
Precision - 80.43%
Recall - 97.36%

44 of 83

Perception Subsystem – Validation and Performance

Perception subsystem validation

3D clustering pipeline - Detection of cluster centroids.
2D detections of objects (Cup, Sanitizer, Duster, Pringles, CMU Cup)

Requirements Satisfied:

M.P.6: Detect objects with a precision of 70% and recall of 60%.

Subsystem Performance Results:

Object detection ( 5 Classes )
Precision - 80.43%
Recall - 97.36%

45 of 83

Manipulation sub-system

46 of 83

Manipulation subsystem - Description

GOALS

User input for object selection/placement
Manipulator and Base Motion Planning
Autonomous Grasping
Autonomous Placement
Teleop Manipulator Controls

47 of 83

AUTONOMOUS OBJECT GRASPING

User input for object selection
Cup detection for grasping
Cup centroid calculation
Extraction of x,y,z position of cup in camera frame
Transformation of x,y,z position to base frame
Path planning for mobile base
Error correction based on visual feedback
Path planning for manipulator
Object Grasping

Manipulation subsystem - Description

48 of 83

Manipulation subsystem - Object Grasping Demonstration Video

49 of 83

AUTONOMOUS OBJECT PLACEMENT

User Input for object placement command
Horizontal plane detection for cup placement
Plane centroid calculation
Extraction of x,y,z position of centroid of plane detected in camera frame
Transformation of x,y,z position to base frame
Path planning for mobile base
Path planning for manipulator
Object Placement

Manipulation subsystem - Description

50 of 83

Manipulation subsystem - Object Placement Demonstration Video

51 of 83

✅ Detection and centroid calculation of cup as selected by user

✅ Base and Manipulator Motion Planning for object grasping

✅ Integration of visual feedback for error correction

✅ Plane segmentation and detection for cup placement

✅ Centroid calculation of horizontal plane to place object

✅ Base and Manipulator Motion Planning for object grasping

⏳ Visual feedback for error correction in Object Placement

⏳ Generalization of objects for manipulation pipeline

⏳ Implementation of obstacle avoidance during motion planning�⏳ Further integration of user interface�

Manipulation subsystem - Status

Key

✅ Finished ☑️ On-Going ⏳TODO

52 of 83

Subsystem Validation:

The manipulator successfully detects and calculates centroid of the cup to grasp
The manipulator autonomously generates trajectory to goal location
The manipulator is successfully able to reach goal pose
The gripper can grasp the object firmly
The gripper holds the object firmly without dropping it

Requirements Satisfied:

M.P.8. Plan manipulator motion to grasp object within 3 minutes
M.P.9. Grasp object within 5 minutes, with 70% success rate (70 successful trials out of 10)

Subsystem Performance Results:

Robot plans and executes autonomous grasping of cup as specified by user within 1.1 minutes (average of 10 trials)
Robot grasps cup with 90% success rate (9 successful trials out of 10).

Failure case- Robot is unable to detect object at a very large angle

Manipulation subsystem - Object Grasping Validation and Performance

53 of 83

Manipulation subsystem - Object Grasping Performance Results

Average: 1.11 mins

90% success rate

54 of 83

Subsystem Validation:

The manipulator successfully detects and calculates centroid of the plane to place cup
The manipulator autonomously generates trajectory to goal location
The manipulator is successfully able to reach goal pose
The gripper can release the object and place the cup stably

Requirements Satisfied:

M.P.8. Plan manipulator motion to place object within 3 minutes
M.P.9. Place object within 5 minutes, with 70% success rate (70 successful trials out of 10)

Subsystem Performance Results:

Robot plans and executes autonomous placement of cup as specified by user within 1.37 minutes (average of 10 trials)
Robot places cup with 70% success rate (7 successful trials out of 10).

Failure cases- Robot is unable to detect plane successfully

Manipulation subsystem - Object Placement Validation and Performance

55 of 83

Manipulation subsystem - Object Placement Performance Results

Average: 1.37 mins

70% success rate

56 of 83

Hardware sub-system

57 of 83

Hardware Subsystem - Description

Subsystem Goal:

Gimbal adds a face to the robot to make the virtual presence of the user more realistic and natural.

Description:

The user operating the robot can control the gimbal to orient the display on the robot through the app interface, for natural interaction with the surrounding. The control inputs are converted into coordinates for the orientation of the display and transferred to the cloud. This data is then received by the task file being managed under the interface module. The ROS package then retrieves this data and passes it to the microcontroller which then converts the coordinate data to pitch and yaw angles for the servos and gimbal moves accordingly.

The UI is an iOS and Android mobile application through which the user can send commands

to the robot and receive updates from the robot. The UI displays real-time audio and video feed of the robot and provides a human-centric design to send autonomous and teleoperation commands

to the robot.

There are two communication links– the videotelephony link and the commands link. The

videotelephony link is a peer-to-peer communication link that streams the audio and video of the user and the robot. For heavy exchange of data, like with audio and video streams, peer-to-peer architecture is faster and more reliable as compared to a centralized architecture. The command link is a client-server-client link, where the robot and the UI are the clients, and a cloud database is the server. The cloud database (Google’s Firebase Firestore) provides real-time updates to listening clients. This architecture provides a single source of truth to both clients and keeps a track of the commands and the robot state. Lastly, the ease of scalability for this architecture opens many different future potentials for the project like swarm telepresence. While a proof-of-concept for the client-server-client architecture has already been developed, the peer-to-peer

58 of 83

Hardware Subsystem - Interconnect

59 of 83

Future work (FVD and beyond):

Smoothen motor velocity profiles
Add face recognition for auto positioning

Hardware Subsystem - Pan Tilt display

PR fulfilled for SVD:

PR 10

60 of 83

Hardware Subsystem - E Kill Switch

61 of 83

Project Teaser

62 of 83

63 of 83

Overall Status- Strengths

Interface - Real-time, reliable, easy of use and seamless integration with other subsystems
Navigation - Robust planning, good static/dynamic obstacle avoidance
Perception - New objects to be added need not to be manually labeled, tracking-based labeling reduces the time and effort by 10 times, real-time detection
Manipulation - Robust planning with accurate error correction based on visual feedback
Gimbal - Mimics head motions like human to facilitate natural conversation, no need to turn robot. Stay localised!

64 of 83

Overall Status- Weaknesses

Hardware - Relatively new platform with frequent firmware issues
Interface - Slow tele-op and jittery motion for manipulation
Navigation - Takes time to plan around obstacles sometimes when map update is slower than required rate
Perception - False positives
Manipulation - No error correction during object placement, no obstacle avoidance during base and manipulator planning

65 of 83

Project Management

66 of 83

Work Breakdown Structure (WBS)

67 of 83

Work Breakdown Structure (WBS) – Management

68 of 83

Work Breakdown Structure (WBS) - Tech

69 of 83

Work Breakdown Structure (WBS) - Tech

70 of 83

Schedule

Second level

1.1 Software management

1.2 Interface

1.3 Perception

1.4 Manipulation

1.5 Navigation

1.6 Gimbal

1.7 Project management

Fall

Software management

Interface

Perception

Manipulation

Navigation

Gimbal

Project management

API Integration

System Integration

TIME

Spring

Fall

71 of 83

Schedule

Second level

1.1 Software management

1.2 Interface

1.3 Perception

1.4 Manipulation

1.5 Navigation

1.6 Gimbal

1.7 Project management

Fall

Software management

Interface

Perception

Manipulation

Navigation

Gimbal

Project management

API Integration

System Integration

TIME

Spring

Fall

☑️

72 of 83

High Level Test Plan – Fall

Month	Milestone	Test/Demo
PR7	Implement peer-to-peer videotelephony. Implement transition between autonomous and teleoperation modes.	Demonstrate real-time audio/video streaming. Test transition between the modes upon reaching room. location.
PR8	Map object detections with 3D centroids. Integrate interface and navigation sub-systems.	Test object centroids with detections. Test autonomous navigation sent through the interface.
PR9	Integrate interface and manipulation sub-systems. Integrate perception and manipulation sub-systems.	Test pose estimates sent from perception module to the manipulation module. Test manipulation skills sent through the interface.
PR10	Integrate interface and gimbal sub-systems. Implement shared autonomy to avoid obstacles during teleoperation.	Test gimbal motion sent through the interface. Demonstrate autonomous obstacle avoidance during teleoperation.
PR11	Implement grasping of custom objects.	Demonstrate grasping of custom objects.
PR12	Implement end-to-end robot functionality.	Demonstrate end-to-end robot functionality.

73 of 83

High Level Test Plan – FVD

Test	Procedure	Validation	Requirements
1	User selects the target room location in the app. Observe the robot autonomously navigate to the room.	Robot is able to receive the goal location from the interface. Robot is able to localize within the map, plan path to the goal (within 3 minutes), and autonomously navigate to the goal(within 12 minutes) with obstacle avoidance (mAP – 80%).	M.F.1, M.F.2, M.F.3, M.F.4, M.F.5, M.P.1, M.P.2, M.P.3, M.P.4
2	User teleoperates the robot inside the room. Observe the robot avoid obstacles autonomously in teleoperation mode.	Robot is able to transition from autonomous to teleoperation. Robot receives and executes the teleoperation commands within 5 seconds. Robot traverses with a speed of 0.4m/s. Robot autonomously avoids obstacles even during teleoperation.	M.F.2, M.P.3
3	User selects the object to be grasped.	Robot is able to detect objects with 60% recall and 70% precision. Robot is able to grasp within 5 minutes, with 70% success rate (7 successful trails out of 10).	M.F.6, M.F.7
4	User controls the gimbal to view the surroundings.	Gimbal is able to move the display by 60 degrees in pitch and 120 degrees in yaw.	M.F.10 M.P.10
5	User commands the robot to place the object.	Robot places the object on flat surface within 5 minutes, with 70% success rate (7 successful trails out of 10).	M.F.8 M.F.9 M.P.8 M.P.9

Location: 3rd floor, AI Maker space (Tepper)

74 of 83

Team Budget

Sr. Number	Part Name	Total Price
1	Wireless Keyboard Mouse	$32.00
2	Dynamixel X430	$147
3	Dynamixel Serial Controller	$22
4	Intel Neural Compute Stick 2	$71.00
5	iPad Mini (Face for robot)	$499.00
6	Type C Hub	$25.00
7	Debug HDMI pin	$8.00
15	iPad Mini Body Protector	$20
16	iPad Mini Screen Protector	$9
17	Clicker	$15
18	Male Headers	$10.00
19	Laser range finder	$44.00
20	Interconnect	$1.40
21	M2.5 Fasteners	$16.99
22	M2.6 Fasteners	$5.48
23	M2.8 Fasteners	$5.07
24	LCD Screen - USB	$146
25	Roller Table	$66
26	3D prints - outsourced	$122
Total		$1264

Current balance: $3200 (74.7%)

* Budget updated on May 02, 2022

Team Budget: $5000

75 of 83

Risk Management Table

ID	Risk	Requirement	Type	Mitigation	L	C	Severity
1	Computational lag	M.F.P.3, M.F.P.1, M.F.P.4, M.F.P.6	Performance	Optimize algorithmic bottlenecks Add compute resources	2	5	Medium
2	Network failure	D.N.1, M.F.P.11, M.F.P.1, M.F.P.10	Technical	Design acknowledgement-based network architecture Add Wi-Fi hotspots if needed	3	3	Medium
3	Accurate training data unavailable	M.F.6, M.P.6	Technical	Create custom dataset Implement non-learning-based algorithm	3	4	Medium
4	Loss of a team member	All	Schedule	Keep understudies for every sub-system Outsource work	2	4	Medium
5	Hardware failure	M.F.P.3, M.F.P.4, M.F.P.7, D.P.1	Performance	Perform hardware tests during development and prior to runtime Implement failsafe algorithm for runtime hardware failure Allocate budget to purchase spare hardware	2	5	Medium

76 of 83

Risk Management Table

5
4
3			RISK 2	RISK 3
2				RISK 4	RISK 1 RISK 5
1
	1	2	3	4	5

RISK

Computational lag
Network failure
Accurate training data unavailable
Loss of a team member
Hardware failure

77 of 83

Lessons Learned

78 of 83

Lessons Learned

THREE BIG LESSONS LEARNED

Hope for the best, prepare for the worst
Focus on the bigger picture
Feedback is very important!
Adding hours to PM software. Back to Jira!

79 of 83

Key fall activities

80 of 83

Key fall activities

Change the TouRing location:

Wean 3108 –> AI Makerspace (Tepper)
AI Makerspace justifies the use case because of its tourable aesthetics
More dynamic environment for robust system testing
Can be used to deploy multiple uses cases, many other robots around for R-R interaction
Can accommodate more people during FVD and FVD-E.

81 of 83

Key fall activities

Proposal: Merge roboceptionist’s capabilities with TouRI and deploy!

*something TeamH might do after all fall requirements are met before FVD and provided all permissions are granted from FMS.

82 of 83

Key fall activities

Fall Activities:

Deliver remaining project requirements.
Make current system more robust by adding closed loop feedback wherever possible.
Conduct a small study on App’s UI/UX to reduce learning curve
Use API’s to create another use case (Let’s say a Med Bot)

83 of 83

Thank You!