1 of 38

CISC849:

Autonomous Robot Vision

​

http://goo.gl/0Im0yy

Prof. Christopher Rasmussen

​

February 16, 2016

2 of 38

2015 DRC Finals

http://www.theroboticschallenge.org/overview, rules

Largely same tasks as 2013 Trials, but to be

performed serially rather than individually

3 of 38

2015 DRC Finals: Top 5 Robots

1

2

3

4

5

4 of 38

Key Questions about Tasks

  • How much is known about the environment, objects, lighting ahead of time?
  • How much variability could there be?
  • What robot skills will or could be engaged?
    • Recognition (broad categories? specific objects?)
    • Walking (flat ground? stairs? ramps? material?)
    • Manipulation (precise grasping? pushing? hooking?)
  • Bottom line: What do we need to understand about the scene (1) When robot first sees it, and (2) As robot performs the task?

5 of 38

Sensor Selection Process

  • Range. From what min to max distance can the sensor give useful depth information about the scene?
  • Field of view. Within its range, what does the sensor see? Includes HFOV, VFOV, but also mounting pose (on head or elsewhere on the body?)
  • Resolution, capture time...
  • Indoor/outdoor. Where are these tasks happening?

6 of 38

DRC Rules: Communications

  • Rules allowed for 2-way communications between robot and human operators
    • Sensor data -> People
    • Commands -> Robot
  • This allowed for range of strategies from full autonomy to full remote control
  • DARPA intentionally degraded bandwidth so video/ladar streaming was difficult at best...
    • High-res images or point clouds of static scenes sent asynchronously and analyzed
    • + Low-resolution continuous streams

7 of 38

DRC Rules: Communications

8 of 38

2015 DRC Finals: Top 5 Robots' Sensors

KAIST

​

2 nodding Hokuyo ladars

Multiple cameras

IHMC Robotics

​

1 spinning Hokuyo ladar

1 pair of stereo cameras

Tartan Rescue

​

2 continuously spinning Hokuyos

2 pairs of stereo cameras

1 wide-angle camera

Nimbro Rescue

​

1 spinning Hokuyo

8 RGB-D cameras for omnidirectional view

3 HD color cameras + top-down wide-angle camera

Robosimian

​

Velodyne HDL-32E

2 pairs of stereo cameras

9 of 38

2013 DRC Trials: DRC-Hubo sensor head

  • Multi-baseline stereo cameras
  • Tiltable laser range-finder
  • Kinect-like depth camera (facing rear)

10 of 38

Walking: DRC-Hubo ladar point cloud (zigzag detail)

11 of 38

Driving task -- 2013 trials

12 of 38

Driving task -- 2015 Finals

13 of 38

Driving task -- 2015 Finals

14 of 38

Perceptual needs: Driving

  • Detect vehicle in mid-distance
  • Gross ground obstacle detection for walking to vehicle
  • Close-range depth sensing for stepping up, sitting, interfacing with controls
  • Long-range obstacle/road sensing for driving, including intensity color for lane lines/segmentation

​

15 of 38

Dismount/egress task -- 2015 Finals

16 of 38

Walking task -- 2013 Trials

17 of 38

Walking task -- 2015 Finals

18 of 38

Perceptual needs: Walking

  • Gross/small ground obstacle detection for avoidance, footstep planning (either bipedal or quadrupedal)
  • Non-planar ground modeling
  • Ground material inference (hard/ soft/ slippery/etc.)
  • Long-range navigation: where are we headed?

19 of 38

Walking: DRC-Hubo ladar point cloud (ramp, zigzag, steps)

20 of 38

Debris task -- 2013 Trials

21 of 38

Debris task -- 2015 Finals

22 of 38

Perceptual needs: Debris

  • Detect/characterize debris field for obstacle avoidance & walk planning
    • Differentiate between movable and immovable objects
  • Detect and model in detail individual pieces for grasping, lifting

23 of 38

Door task

1st door: push; 2nd: pull; 3rd: push with resistance

24 of 38

Perceptual needs: Door

  • Detect door polygon on wall from a distance and walk to it
  • Localize handle, make detailed model for grasping
  • Track hand/arm/door during opening
  • Step through open doorway

25 of 38

Ladder task -- 2013 Trials

26 of 38

Ladder task -- 2015 Final

27 of 38

Perceptual needs: Ladder

  • Find ladder in this room/area
  • Walk to it
  • Parametrize stringer, rung separation/ cross-section, etc.
  • Monitor hand/foot holds during execution
  • Dismount and walk on catwalk

28 of 38

Wall task -- 2013 Trials

29 of 38

Perceptual needs: Wall breach

  • Where's the wall?
  • Where's the tool the robot will use?
  • Get detailed model of tool for grasping, lifting, triggering
  • Monitor shape-cutting during execution
    • Push to complete hole?

DRC-Hubo at 2013 Trials

30 of 38

Wall & valve tasks -- 2015 Finals

31 of 38

Valve task

32 of 38

Perceptual needs: Valve turning

  • Where are the valves in this room/area?
  • Walk to them
  • Get detailed model of each valve for grasping
  • Monitor hand(s) during turning motion appropriate for current valve

​

33 of 38

Hose task -- 2013 Trials

34 of 38

Perceptual needs: Hose attachment

  • Where's the hose in this area?
  • Detailed modeling for grasp planning
  • Same for attachment spot
  • Monitoring grasp during hold-and-attach motion

​

35 of 38

Hose task: Another view (peg approach)

36 of 38

Hose task: DRC-Hubo distance-filtered ladar point cloud (peg approach)

37 of 38

2015 DRC Finals Overview

5:48

38 of 38

DRC winner KAIST: Overview video

4:30