CISC849:

Robot Vision & Learning

​

​

Prof. Christopher Rasmussen

​

September 5, 2019

Ground Segmentation

• A standard task for robot point cloud data is segmentation--e.g., which points belong to the "bicycle", which points to the "car", which to the "tree", etc.
• And what could be more basic than segmenting points into ground vs. obstacles?

​

Ground Segmentation

• A standard task for robot point cloud data is segmentation--e.g., which points belong to the "bicycle", which points to the "car", which to the "tree", etc.
• And what could be more basic than segmenting points into ground vs. obstacles?

​

• With a level ladar we don't really have this problem, because it's not even supposed to "see" the ground
• Based on height, everything detected is assumed an obstacle

​

Ground Segmentation

• Potential obstacles have many possible shapes, but indoors and outdoors in paved areas, we can assume that the ground is planar

​

Ground Segmentation

• Potential obstacles have many possible shapes, but indoors and outdoors in paved areas, we can assume that the ground is planar
• If we can extract the ground plane from the point cloud, then everything left over is an obstacle (positive or negative) or "object"
• "Left over" means computing the distance of each point to the ground plane and thresholding

​

One dominant plane

Important planes

• Ground: obstacle avoidance, debris detection
• Table/shelf: find drill
• Door: find handle
• Wall: find valve, hose reel, "surprise" plug
• Stairs: distinguish treads from railing
• Non-DRC examples???

Important planes

• Ground: obstacle avoidance, debris detection
• Table/shelf: find drill
• Door: find handle
• Wall: find valve, hose reel, "surprise" plug
• Stairs: distinguish treads from railing
• Non-DRC examples???

Walking task

Perceptual needs: Walking

• Gross/small ground obstacle detection for avoidance, footstep planning (either bipedal or quadrupedal)
• Non-planar ground modeling
• Ground material inference (hard/ soft/ slippery/etc.)
• Long-range navigation: where are we headed?

Walking: DRC-Hubo ladar point cloud (zigzag detail)

Walking: DRC-Hubo ladar point cloud (ramp, zigzag, steps)

Debris task

Perceptual needs: Debris

• Detect/characterize debris field for obstacle avoidance & walk planning
• Differentiate between movable and immovable objects
• Detect and model in detail individual pieces for grasping, lifting

Hose task: DRC-Hubo view from one camera -- less planes!

Multiple large planes

Ground Segmentation

• Potential obstacles have many possible shapes, but indoors and outdoors in paved areas, we can assume that the ground is planar
• If we can extract the ground plane from the point cloud, then everything left over is an obstacle (positive or negative) or "object"
• "Left over" means computing the distance of each point to the ground plane and thresholding
• But how to estimate the plane(s) parameters?

Robust Plane Fitting

• The 3-D plane equation is ax + by + cz + d = 0
• We wish to estimate (a, b, c, d)
• 3 non-collinear points are the minimum solution to define a plane
• With noise from our sensor, we would want to use more points for a least-squares fit to get a better result
• But if we include points that don't belong to the plane (i.e., obstacle points), these outliers will bias the results

​

Robust Plane Fitting

• The 3-D plane equation is ax + by + cz + d = 0
• We wish to estimate (a, b, c, d)
• 3 non-collinear points are the minimum solution to define a plane
• With noise from our sensor, we would want to use more points for a least-squares fit to get a better result
• But if we include points that don't belong to the plane (i.e., obstacle points), these outliers will bias the results
• A robust plane-fitting technique will work to identify and reject outliers and only do the least-squares fit on the inliers

RANSAC plane fitting

• RANSAC (RANdomized SAmple Consensus) is a commonly-used approach to robust fitting
• It is iterative, in that multiple hypotheses are considered
• Each hypothesis is a randomly-chosen sample of the points which minimally solves the problem
• For line-fitting, a sample would be 2 points
• For plane-fitting, a sample is 3 points

​

RANSAC plane fitting

• RANSAC (RANdomized SAmple Consensus) is a commonly-used approach to robust fitting
• It is iterative, in that multiple hypotheses are considered
• Each hypothesis is a randomly-chosen sample of the points which minimally solves the problem
• For line-fitting, a sample would be 2 points
• For plane-fitting, a sample is 3 points

​

RANSAC plane fitting

• RANSAC (RANdomized SAmple Consensus) is a commonly-used approach to robust fitting
• It is iterative, in that multiple hypotheses are considered
• Each hypothesis is a randomly-chosen sample of the points which minimally solves the problem
• For line-fitting, a sample would be 2 points
• For plane-fitting, a sample is 3 points

​

RANSAC plane fitting

• RANSAC (RANdomized SAmple Consensus) is a commonly-used approach to robust fitting
• It is iterative, in that multiple hypotheses are considered
• Each hypothesis is a randomly-chosen sample of the points which minimally solves the problem
• For line-fitting, a sample would be 2 points
• For plane-fitting, a sample is 3 points
• The solution for a given sample is scored by counting the number of inliers (points < threshold distance away)

​

RANSAC plane fitting

• RANSAC (RANdomized SAmple Consensus) is a commonly-used approach to robust fitting
• It is iterative, in that multiple hypotheses are considered
• Each hypothesis is a randomly-chosen sample of the points which minimally solves the problem
• For line-fitting, a sample would be 2 points
• For plane-fitting, a sample is 3 points
• The solution for a given sample is scored by counting the number of inliers (points < threshold distance away)

​

RANSAC plane fitting

• RANSAC (RANdomized SAmple Consensus) is a commonly-used approach to robust fitting
• It is iterative, in that multiple hypotheses are considered
• Each hypothesis is a randomly-chosen sample of the points which minimally solves the problem
• For line-fitting, a sample would be 2 points
• For plane-fitting, a sample is 3 points
• The solution for a given sample is scored by counting the number of inliers (points < threshold distance away)

​

RANSAC plane fitting

• RANSAC (RANdomized SAmple Consensus) is a commonly-used approach to robust fitting
• It is iterative, in that multiple hypotheses are considered
• Each hypothesis is a randomly-chosen sample of the points which minimally solves the problem
• For line-fitting, a sample would be 2 points
• For plane-fitting, a sample is 3 points
• The solution for a given sample is scored by counting the number of inliers (points < threshold distance away)
• The sample with the most inliers (after a certain number of tries) is considered the "best", and a least-squares solution is computed from it

RANSAC in

Point Cloud Library (PCL)

• PCL has a sample_consensus module which implements RANSAC and has options for fitting planes as well as other shapes
• Constraints can also be imposed, such as looking for horizontal planes vs. vertical planes
• This tutorial and this one will get you started; see plane_fit example

RANSAC in PCL

#include <iostream>�#include <pcl/ModelCoefficients.h>�#include <pcl/io/pcd_io.h>�#include <pcl/point_types.h>�#include <pcl/sample_consensus/method_types.h>�#include <pcl/sample_consensus/model_types.h>�#include <pcl/segmentation/sac_segmentation.h>��int main (int argc, char** argv)�{� pcl::PointCloud<pcl::PointXYZ>::Ptr cloud(new pcl::PointCloud<pcl::PointXYZ>);�� // Fill in the cloud data…

// ???�� // Try to do robust plane fit� pcl::ModelCoefficients::Ptr coefficients (new pcl::ModelCoefficients);� pcl::PointIndices::Ptr inliers (new pcl::PointIndices);� pcl::SACSegmentation<pcl::PointXYZ> seg; // Create segmentation object�� seg.setOptimizeCoefficients (true); // optional

seg.setModelType (pcl::SACMODEL_PLANE); // mandatory� seg.setMethodType (pcl::SAC_RANSAC);� seg.setDistanceThreshold (0.01); // 1 cm�� seg.setInputCloud (cloud);� seg.segment (*inliers, *coefficients);�

​

� // did it work?

� if (inliers->indices.size () == 0) {� PCL_ERROR ("Could not estimate a planar model for the given dataset.");� return (-1);� }�

// report results

​

std::cerr << "Model coefficients: "

<< coefficients->values[0] << " " � << coefficients->values[1] << " "� << coefficients->values[2] << " " � << coefficients->values[3] << std::endl;�� std::cerr << "Model inliers: " << inliers->indices.size () << std::endl;� for (size_t i = 0; i < inliers->indices.size (); ++i)� std::cerr << inliers->indices[i] << " "

<< cloud->points[inliers->indices[i]].x << " "� << cloud->points[inliers->indices[i]].y << " "� << cloud->points[inliers->indices[i]].z <<

std::endl;

​

return (0);�}

​

​