1 of 21

Viewpoint Tracking and Visual Servoing with

Inverted Robot Arm

IPAB Workshop

March 2018

​

Radim Tylecek

University of Edinburgh

2 of 21

Collaborators

Christos Maniatis Nanbo Li Hanz Cuevas Velasquez Marcelo Saval-Calvo Radim Tylecek Bob Fisher

3 of 21

Workcell Setup

4 of 21

Setup

  • Workcell frame
  • 4x Kinect in corners
  • UR10 arm inverted mount
  • 2x camera on wrist bracket
  • 2x PC

​

5 of 21

Calibration of 4 Kinect extrinsic parameters with April tag grid

6 of 21

Best Viewpoint Tracking

for Camera Mounted

on Robot Arm

with Dynamic Obstacles

Presented at 3DV 2017

7 of 21

8 of 21

Best Viewpoint Tracking - Algorithm Scheme

9 of 21

Prediction of the Next Viewpoint

Choose next state st+1 that maximizes the objective function Ptotal

​

​

Pvis - viewpoint quality Pnocc - occlusion term

Pdist - geodesic distance Pjt - joint space distance

10 of 21

11 of 21

Hybrid Multi-Camera

Visual Servoing

to Moving Target

Presented at IROS 2018

12 of 21

Goal: track object in 3D

and follow it with arm

  • Use stereo pair on the arm wrist when the target is in the view and depth range
  • Switch to nearby Kinects when it gets away
  • Position control of arm endpoint
  • Replanning strategy for continuous movement

13 of 21

Stereo Sensor

  • Camera pair with small baseline
    • 45o FOV
    • WVGA resolution
  • Connected to FPGA controller
    • HW Synchronized @ 30 fps
    • Supports up to 10 cameras
  • Produced by ETHZ startup

14 of 21

Stereo Sensor

  • Real-time depth from stereo
    • Semi-Global Matching (SGM)
  • Limited to 32 disparity values
  • Ranges with minimum 3 cm baseline
    • 60 cm - infinity
    • 20 - 40 cm (used for tracking)
  • Fast but noisy

15 of 21

Stereo Sensor

  • Real-time depth from stereo
    • Semi-Global Matching (SGM)
  • Limited to 32 disparity values
  • Ranges with minimum 3 cm baseline
    • 60 cm - infinity
    • 20 - 40 cm (used for tracking)
  • Fast but noisy

16 of 21

Hybrid Multi-Camera Servoing - Components Interaction

17 of 21

Parallel Planning and Execution Strategy

  • Iterative approach
  • Plan trajectory to reach 80% of the approach vector to current target pose
  • While executing it, use next target pose to plan the next segment
  • In close range plan to 100%

18 of 21

19 of 21

Evaluation - Benefits of Hybrid Servoing

  • Multiple targets and scenarios
  • Quantitative evaluation for static targets in multiple locations
  • Accuracy of stereo camera in the close range was important for success

Tracking mode

Success Rate

Accuracy

Plan Iterations

Hybrid

95 %

15 mm

6

Kinect only

68%

25 mm

8

20 of 21

Future Plans

  • Combine the two approaches: Continuos best view tracking of moving target in presence of dynamic occlusions
  • Alternative motion control in close range - velocity
  • Target motion modelling for prediction

21 of 21

Thanks!

Wondering where the manufacturer of UR series arms got its name from?

K. Čapek: R.U.R. (1920)