Visual Servoing using Industrial Manipulator

Shivam Thukral

[CPSC 515] Computational Robotics

08-12-2020

Outline

• Introduction
• Motivation
• Problem Description
• Related Work
• Proposed Solution
• Approach and Analysis
• Progress
• What’s accomplished?
• What’s remaining?
• Conclusions
• Future Work

2

Introduction

What is Visual Servoing?

• Visual servo (VS) control refers to the use of computer vision data to control the motion of a robot.
• Visual servo control relies on techniques from image processing, computer vision, and control theory.
• Motivation:
• Tracking object of interest
• Difficult to find analytical solution to inverse kinematics problem

Applications:

• Rose Pruning [Velasquez et al ‘20]
• Monitoring Row-Crop Fields [Ahmadi et al ‘20]

3

Problem Description

Visual Servoing in Robotic Manipulators?

• Minimise the error between current and desired image features.
• Input?
• Image features
• Output?
• Joint Velocity
• Robotic Manipulator - UR5
• 6-DOF
• Eye-in-hand configuration

​

4

Related Work

​

• PBVS [Wilson et al ‘96], [Gans et al ‘08]
• Features are designed in cartesian space
• IBVS [Weiss et al ‘85], [Bourquardez et al ’09]
• Features are designed in image space
• Hybrid Approaches [Chaumette et al ’07] [Gans et al ‘03]
• PBVS + IBVS

5

Proposed Solution - Approach

Image Based Visual Servoing (IBVS)

• Image feature
• Blue ball with centroid as the feature
• Tracking
• CAMShift Algorithm for ball tracking
• Kalman filter for occlusions
• Velocity based controller for UR5
• ROS+Gazebo

6

Proposed Solution - Analysis

Experiment Set:

1. Ball tracking w/o kalman filter in image plane
2. Ball tracking w/ kalman filter in image plane
1. Add occlusions randomly.
3. Ball tracking in 3D
4. Application :
• Ping pong : How many times racket is able to hit the ball?

​

[I will be incrementally adding noise to all the experiments, to check the robustness of the proposed approach]

7

Progress - What’s accomplished

Experiment 1: Ball tracking w/o kalman in image plane

8

Progress - What’s accomplished

​

Experiment 2: Ball tracking w/ kalman in image plane with occlusion

​

9

Progress - What’s accomplished

​

Experiment 4: Gazebo Ping - Pong Environment

10

Progress - What’s remaining?

• Incrementally adding noise to experiments
• Experiment 3: Tracking ball in 3D
• Ping Pong Experiment:
• Simulating parabolic path of ping-pong ball
• Tracking and hitting the ping pong ball with manipulator
• Generating results and graphs
• Tracked Path vs Estimated Path
• Effect of noise in object tracking
• Successful hits in ping pong experiment

11

Conclusions

• IBVS using Industrial Manipulator
• Results are demonstrated in Simulation using ROS + Gazebo
• Robotic Arm used to track and hit the ping pong ball

Future Work (Beyond CPSC 515)

• Try out other tracking algorithms
• SIFT, SURF, ORB
• Use particle filter for object tracking
• Play ping pong with two robotic manipulators

12

References

• Cuevas-Velasquez, Hanz, et al. "Real-time Stereo Visual Servoing for Rose Pruning with Robotic Arm." 2020 IEEE International Conference on Robotics and Automation (ICRA) 2020.
• Ahmadi, Alireza, et al. "Visual Servoing-based Navigation for Monitoring Row-Crop Fields." 2020 IEEE International Conference on Robotics and Automation (ICRA) 2020.
• W. J. Wilson, C. C. W. Hulls, and G. S. Bell, “Relative end-effector control using cartesian position based visual servoing” IEEE Transactions on Robotics and Automation, 1996.
• N. R. Gans, A. P. Dani, and W. E. Dixon, “Visual servoing to an arbitrary pose with respect to an object given a single known length” in Proceedings of the American Control Conference, 2008
• L. E. Weiss, A. C. Sanderson, and C. P. Neuman, “Dynamic visual servo control of robots: an adaptive image-based approach” in Proceedings of the IEEE International Conference on Robotics and Automation, 1985.
• O. Bourquardez, R. Mahony, N. Guenard, F. Chaumette, T. Hamel, and L. Eck, “Image-based visual servo control of the translation kinematics of a quadrotor aerial vehicle,” IEEE Transactions on Robotics, 2009.
• F. Chaumette and S. Hutchinson, “Visual servo control. II. Advanced approaches,” IEEE Robotics and Automation Magazine, 2007
• N. R. Gans, S. A. Hutchinson, and P. I. Corke, “Performance tests for visual servo control systems with application to partitioned approaches to visual servo control” International Journal of Robotics Research, 2003

13

Thank you.

14