1 of 8

ROS-PyBullet Interface: A Framework for Reliable Contact Simulation and Human-Robot Interaction

Christopher E. Mower, Theodoros Stouraitis, João Moura, Christian Rauch, Lei Yan, Nazanin Zamani Behabadi, Michael Gienger, Tom Vercauteren, Christos Bergeles, Sethu Vijayakumar

​

​

https://github.com/ros-pybullet/ros_pybullet_interface

2 of 8

ROS-PyBullet Interface: A Framework for Reliable Contact Simulation and Human-Robot Interaction

Christopher E. Mower, Theodoros Stouraitis, João Moura, Christian Rauch, Lei Yan, Nazanin Zamani Behabadi, Michael Gienger, Tom Vercauteren, Christos Bergeles, Sethu Vijayakumar

​

​

https://github.com/ros-pybullet/ros_pybullet_interface

Online, full-physics simulation using a reliable contact simulator. The framework relies on PyBullet to enable well established contact simulation for rigid/deformable bodies.

​

Integration with the ROS ecosystem. Robot simulation and visualization of real robots/objects (utilizing sensing) are integrated via ROS. Furthermore, this enables (i) ROS packages to be integrated with PyBullet and (ii) a straightforward way to port developed algorithms to real systems.

​

Several interfaces enabling HRI with virtual worlds and telepresence. We provide facilities for human’s to provide examples in a simulated environment via several popular interfaces (including haptic devices).

​

Sensor simulation. Robot joint force-torque sensors and RGB-D cameras (i.e. point clouds) are provided for sensing-based control and can be seamlessly interchanged with real world sensor streams via ROS.

Modular and extensible design. Our framework adopts a modular (i.e. several ROS nodes) and highly extensible design paradigm (i.e. class hierarchy) using the Python programming language. This makes it easy to quickly develop new features for the framework.

​

Data collection with standard ROS tools. Since the framework provides an interface to ROS, we can leverage common tools for data collection such as ROS bags [36] and data processing to common formats in machine learning applications, i.e. rosbag pandas [37].

​

Integration with robot and sensing hardware. Tools are provided to easily remap the virtual system to physical hardware and integrate real sensing apparatus in the PyBullet simulation (e.g. vicon).

Framework features

3 of 8

ROS-PyBullet Interface: A Framework for Reliable Contact Simulation and Human-Robot Interaction

Christopher E. Mower, Theodoros Stouraitis, João Moura, Christian Rauch, Lei Yan, Nazanin Zamani Behabadi, Michael Gienger, Tom Vercauteren, Christos Bergeles, Sethu Vijayakumar

​

​

https://github.com/ros-pybullet/ros_pybullet_interface

Alternatives?

3Possible via additional plugins.

4Isaac Sim was recently integrated with Gazebo.

5Until recently MuJoCo required a license.

6Possible with new Kubric library.

​

ROS

Languages

Deform. Obj.

Hardware

HRI

Photo-realistic

Drake

✗

C++/Python

✔

✗

✗

✗

Gazebo

✔

C++

✗

✔

✗

✗3

Nvidia Issac

✔4

Python

✗

✔

✗

✔

MuJoCo5

✗

C++/Python

✔

✗

✗

✗

ROS-PyBullet

✔

Python

✔

✔

✔

✗6

4 of 8

ROS-PyBullet Interface: A Framework for Reliable Contact Simulation and Human-Robot Interaction

Christopher E. Mower, Theodoros Stouraitis, João Moura, Christian Rauch, Lei Yan, Nazanin Zamani Behabadi, Michael Gienger, Tom Vercauteren, Christos Bergeles, Sethu Vijayakumar

​

​

https://github.com/ros-pybullet/ros_pybullet_interface

Limitations (Dirty Laundry)

  • Python can be limiting in situations where real time systems are required.

​

  • Currently, only ROS1 Noetic is supported. The framework could be interfaced with ROS2 using the ros1_bridge, however some services/messages would not be available.

​

  • Photorealism is not currently supported.

5 of 8

ROS-PyBullet Interface: A Framework for Reliable Contact Simulation and Human-Robot Interaction

Christopher E. Mower, Theodoros Stouraitis, João Moura, Christian Rauch, Lei Yan, Nazanin Zamani Behabadi, Michael Gienger, Tom Vercauteren, Christos Bergeles, Sethu Vijayakumar

​

​

https://github.com/ros-pybullet/ros_pybullet_interface

  • A framework that enables research in contact-rich manipulation scenarios allowing for seamless collection of contact-rich data and human demonstrations within a simulated environment.

​

  • Our system enables transference of robot behaviors (learned or otherwise) from the PyBullet simulation environment (a reliable contact/impact simulator) to hardware using ROS.

​

  • Implementation of several HRI interfaces, e.g. keyboard, mouse, joystick, 6D-mouse, haptic devices, Xsens suit that enable easy interaction with virtual environments for HRI and haptic setups.

​

  • Several use-cases to demonstrate the capabilities and usefulness of the framework, including a variety of robots (e.g. Kawada Nextage humanoid, KUKA LWR robot arm, Kinova arm, and dual-arm KUKA IIWA), and full documentation.

Contributions

6 of 8

ROS-PyBullet Interface: A Framework for Reliable Contact Simulation and Human-Robot Interaction

Christopher E. Mower, Theodoros Stouraitis, João Moura, Christian Rauch, Lei Yan, Nazanin Zamani Behabadi, Michael Gienger, Tom Vercauteren, Christos Bergeles, Sethu Vijayakumar

​

​

https://github.com/ros-pybullet/ros_pybullet_interface

Use case: learning from demonstration and interaction with virtual worlds

7 of 8

ROS-PyBullet Interface: A Framework for Reliable Contact Simulation and Human-Robot Interaction

Christopher E. Mower, Theodoros Stouraitis, João Moura, Christian Rauch, Lei Yan, Nazanin Zamani Behabadi, Michael Gienger, Tom Vercauteren, Christos Bergeles, Sethu Vijayakumar

​

​

https://github.com/ros-pybullet/ros_pybullet_interface

Use case: full-body telepresence

8 of 8

ROS-PyBullet Interface: A Framework for Reliable Contact Simulation and Human-Robot Interaction

Christopher E. Mower, Theodoros Stouraitis, João Moura, Christian Rauch, Lei Yan, Nazanin Zamani Behabadi, Michael Gienger, Tom Vercauteren, Christos Bergeles, Sethu Vijayakumar

​

​

https://github.com/ros-pybullet/ros_pybullet_interface

Use case: hardware integration and MPC