1 of 1

A Testbed for Studying Human-Robot Collaboration during Disaster Response

Bill Qian�Advisor: Vaibhav Unhelkar

Human-Robot Teamwork and Current Challenges

• To ensure that the robots effectively support human responders during disasters, it is essential to train both humans and robots to collaborate with each other.

​

• Training robots and humans in the real world is resource-intensive. It is also difficult to measure the “ground truth” of human-robot teamwork in real world settings.

​

• To facilitate human-robot training, we have developed a computer-based simulation testbed using Unity in which human-robot teams are tasked with performing time-critical search and rescue operations in a large city.

​

Applications

The Rega drone developed by Swiss Air Rescue

Tethered land robots for clearance and demining operations

Computer-based Simulation Testbed in Unity

Team Training Task

The goal of H-R team is to pick all objects of interest (first aid kits), avoid collecting other objects (crates), while minimizing task completion times and human effort. 

A Grid representation of the domain

The main camera is placed on the human agent to create a first-person perspective. One can use keyboard keys (w, a, s, d) to move the human around.

The player can interact with the robot using a robot control panel. For instance, the player can ask the robot to go to a specific location.

This is a search-and-rescue domain created with Unity. The location of the human (H), robot (R), objects of interest (red dots), and dangerous regions (radioactive signs) are marked on the mini-map of the domain.

Secondary Tasks

Human operators are also required to perform a secondary “system monitoring” task, which prompts them to correct any abnormal behavior on a control panel.

Measuring Human Psychophysiological States

• A Unity program is used to collect game data, such as locations of the human and robot, domain states, and actions of the player.

​

• A wearable heart rate sensor (Zephyr Bioharness) and an eye tracker (Tobii Pro Nano) are used to collect physiological data.

• A software system built using the Lab Streaming Layer (LSL) Is used to synchronize timestamps of data collected from different sensors and enable the viewing of the collected data in real time.

Zephyr Bioharness

Tobii Pro Nano

The view of the training simulation with the secondary task incorporated in the bottom right corner.

​

Contact: Bill Qian (bill.qian@rice.edu), Vaibhav Unhelkar (unhelkar@rice.edu), Department of Computer Science, Rice University

Acknowledgments: This research is partially supported by the Army Research Laboratory and is done in collaboration with Liubove Orlov-Savko.�