1 of 16

Mapping and Segmentation For Vine Pruning With RBGD Point Cloud

Ritika Avadhanula, Jinwoo Choi, Josyula Gopala Krishna

​

Introduction To Robotics I

Professor Incharge: Dr Cindy Grimm

1

2 of 16

As seen in the last week’s presentation

Vineyard Setup

​

2

3 of 16

To generate a map and detect branches

  • Sense - RealSense Camera
  • Act - Not Autonomous - Moving along a predefined trajectory to complete mapping
  • Think - Mapping, Segmentation

3

4 of 16

RealSense D435 Camera Placement

The camera is currently place at the end effector’s origin

​

​

​

4

5 of 16

Moving the UR5 Arm

Trajectory consists of 3 loops one along the the length of each branch

​

5

6 of 16

Mapping

6

7 of 16

Mapping - Results

We use RTAB SLAM to generate a dense point cloud map

​

7

8 of 16

Segmentation

8

9 of 16

First unsuccessful Attempt to Segment branch clusters using euclidean clustering in PCL (point cloud library)

9

10 of 16

Second unsuccessful Attempt to Segment branch clusters using Discrete Optical Flow algorithm

​

10

11 of 16

Segmentation - HED

The HED algorithm learns features from multiple hidden layer outputs

11

12 of 16

Successful Segmentation of branches using HED - Lab setup

Branch Edge Detection

​

​

12

13 of 16

Successful Segmentation of branches using HED - Real World Data

13

14 of 16

Lessons Learnt

RTAB mapping is best to generate dense point cloud data to capture the details environment compared to methods like Cartographer SLAM.

Very hard to deal with Cartographer pstream files, they don’t even get processed if the bag has delay.

Always check the correct versions of the packages so you don’t get compiler or symbol not found errors

Learning based methods like HED worked the best for segmentation of the branches by detecting its edges.

Structure Core Camera is awful never use it.

14

15 of 16

Thank You

15

16 of 16

References

[1] S. Xie and Z. Tu, "Holistically-Nested Edge Detection," 2015 IEEE International Conference on Computer Vision (ICCV), 2015, pp. 1395-1403, doi: 10.1109/ICCV.2015.164.

[2] Menze, M., Heipke, C., & Geiger, A. (2015). Discrete Optimization for Optical Flow. German Conference on Pattern Recognition (GCPR), 9358, 16–28. https://doi.org/10.1007/978-3-319-24947-6_2

[3] M. Labbé and F. Michaud, “RTAB-Map as an Open-Source Lidar and Visual SLAM Library for Large-Scale and Long-Term Online Operation,” in Journal of Field Robotics, vol. 36, no. 2, pp. 416–446, 2019.

16