1 of 15

RoverArm

2 of 15

Robot Design and Action Space

Action Space

​

  • Box(low = [-1] * 6, high = [1] * 6)

​

Rover Actions – Throttle, Steer

Arm Actions – dx, dy, dz, f

​

dx, dy, dz – change in co-ordinates of end effector

f – fingers location

3 of 15

Task1

  • Pick up the object

Reward

  • Once the object is held by bot and raised to some height, � reward = 1
  • Unless the object reaches that state, reward = 0

4 of 15

Observation Space

b - boundary within which the rover can move on the plane.

x,y,z – Coordinates of the end effector of the arm.

f – maximum distance the finger of arm can move.

Box(low = [-b, -b, -x,-y,-z, 0,0], high = [b,b, x,y,z, f,f ] )

Info – contains the location of the objects on tray.

5 of 15

Observation Space

Images from two cameras are passed to model, one for front view of scene, and one for top view of the tray.

6 of 15

Task2

  • Pick up the object (from the tray that is closest to camera) and place it on the second tray (the one that is far away).

7 of 15

Reward Mechanism

​

  • Once the object lies within co-ordinates of second tray, we could say the task is complete and bot gets the reward.

8 of 15

Observation Space Update

All the inputs mentioned for Task1, and additionally, an image of output tray is also passed to model.

​

9 of 15

Task3

  • Pick up the object (from the tray that is closest to camera) and place it on the second tray (the one that is far away) in given position.
  • Input image containing target placement of object in the tray is passed to the model.

10 of 15

Reward Mechanism

​

  • We can segment the object’s location on input image and final placement, once the distance of center pixel of objects are less than a threshold, we could say the task is complete and bot gets the reward.

11 of 15

Task4

  • Task3 with Multiple Objects.
  • Input image containing target arrangement of objects in the tray is passed to the model.

12 of 15

Reward Mechanism

​

  • We can get segment the location for each object on input image and final placement, once the distance of center pixel of every object is less than a threshold, we could say the task is complete and bot gets the reward.

13 of 15

Algorithms

  • DQN with Discrete Actions
  • REINFORCE
  • Actor Critic

​

Probably,

​

  • Model based learning
  • Imitation Learning

14 of 15

Strategies

  • Run the bot using controller / keyboard manually, get training data so that the model would have training data where the end goal is reached.

​

15 of 15

Code

​

​

​