CSE 481 C - ROBOTICS CAPSTONE

ROBOT PERCEPTION

WEEK 5 | TUESDAY | APR 26, 2022

Instructor: Maya Cakmak

mcakmak@cs.washington.edu

Teaching Assistant: Vinitha Ranganeni

vinitha@cs.washington.edu

ROBOT PERCEPTION

RGBD camera

Laser range sensor

Finger tip

pressure sensors

Joint encoders and torque sensors

IMU

Extracting useful information from sensors

ROBOT PERCEPTION

• What is the goal of perception for manipulation?
• Estimate the state of the environment, to inform manipulation
• How do we represent the state?
• Raw sensor data >> .. (everything in between) .. >> Object 3D mesh + pose

(SUBTLY DIFFERENT)

PERCEPTION PROBLEMS

Detection/Localization

Classification/Recognition

Tracking

Something is here.

It is a human.

It is Tom.

This is human #5 from last frame.

Segmentation

These regions have different things.

PERCEPTION PROBLEMS

Why are they computationally expensive?

IMAGE PROCESSING W/ LOCAL OPERATORS

many-to-one mapping between pixels

The reddish shaded region on the left shows the window W that is the set of pixels used to compute the red output pixel on the right.

RGBD, POINT CLOUDS, VOXEL GRID

RGBD, POINT CLOUDS, VOXEL GRID

RGBD, POINT CLOUDS, VOXEL GRID

ASSIGNMENT #5

• Post #1: Smart cropper that cuts part of the point cloud that corresponds to a particular bin on the shelf in different shelf poses detected based on an AR marker attached to the shelf
• Post #2: Programming by demonstration system for saving poses relative to AR markers (or just the robot base) and replaying those poses in a sequence for a new configuration of the markers. Demonstrate with actions relative to shelf bin: push-in, pull-out, pick-and-place.

PBD EXAMPLES

OBJECT TYPES

​

Rigid, e.g. pill bottle

Articulated, e.g. scissors

Non-rigid, e.g. towel

Fluids, e.g. soup