Public Presentation

Date: Wednesday, Nov 2 2022

Improved speed with inference on GPU and C++ implementation

The Telepresence Touring Robot

Team: Jashkumar Diyora, Shruti Gangopadhyay , Jigarkumar Patel, Prakhar Pradeep, Shivani Sivakumar

Sponsors : Zackory Erikson, Yonatan Bisk | Advisors: John Dolan, Dimitrios Apostolopoulos

1

User Interface

All teleoperation and autonomous commands are sent to the robot through the user interface.

1

User selects tour location

User teleoperates the robot

User selects desired souvenir

Josh has been accepted to prestigious Carnegie Mellon University.

He wishes to tour CMU and also collect some souvenirs before admission.

Thanks to TouRI he is able to do this from India!

Use Case

Autonomous & Teleoperated Navigation

TouRI autonomously navigates to tour location while avoiding obstacles. User then tele-operates robot within the location.

Dynamic Window for local planner

Global

Plan

TouRI autonomously navigates while avoiding static & dynamic obstacles

3

Results

Pan Tilt Display

TouRI has a pan tilt display mimicking human motions to interact with the surroundings.

Auto face lock feature

2

Autonomous Manipulation & Perception

Josh selects the souvenirs from the interface and TouRI picks the respective souvenirs and places them in the shipping box.

Object & Shipping box Detection

Auto labelling pipeline for training

4

Object Picking & Placing with Cartesian Planner and Visual Correction

Motion: 100^o in pitch and 180^o in yaw

Alert for obstacles

Joystick for teleop control

Souvenir detection

With TouRI, Josh was telepresent at Carnegie Mellon University at the click of a button and was able to successfully take a tour from the comfort of his home. He was also able to pick up souvenirs he liked that were later shipped to him! TouRI, a mobile manipulator, was demonstrated for this with the following metrics-

82% Precision , 98% in object detection

>80% Object pick and place

90% Obstacle Detection & Avoidance

60 m traversed within 5m 30s

100^o pitch, 180^o yaw for display

<20ms latency of user interface

2D and 3D centroid estimation for souvenirs and shipping box

3D centroid estimation

Asynchronous operation

Pan-tilt display Mount

User teleoperates robot while receiving alerts about obstacles

iPad

User receives traversal feedback of the robot on the user interface

Robot Localization

Autonomous picking

Shipping box detection

Robot position feedback on interface

Autonomous placing

Pitch Motor

Yaw

Motor

60 m autonomous traversal time

Average: 5.34 mins

Tour

Location

Joystick control

Detected Souvenir