1 of 22

ParaDocs

Team E PR 9 - Shivangi

2 of 22

Progress

1

3 of 22

Our Goals for PR9 included

  • Planning: Initial integration effort for motion compensation
    • Be able to add new goals and track for it in Gazebo
  • Behavior Tree doing overall flow as well as it can
  • Initial tracking results

​

4 of 22

Hybrid-planning

  • Goals
    • Try different planners to have a better idea about the design of planning stack
  • Demo
    • Keep publishing new tracking goals that have large motion
    • If (large motion)
      • Replan using Global planner: ompl/RRTStar
    • Else
      • Not yet implemented

5 of 22

Hybrid-planning

6 of 22

Hybrid-planning

  • Regarding planner selection
    • Global planner: ompl/RRTStar, custom planner (Ideal)
      • Requirements: account for obstacles, keep looking at the bone
    • Local planner: pilz/LIN, OCS2/MPC (current motion compensation module)
      • Requirements: fast motion for small offset

​

7 of 22

Hybrid-planning

  • Not able to test pilz
    • Error: Given frame (link_0) is unequal to model frame(world)
    • https://github.com/moveit/moveit2/issues/2857

8 of 22

SAM Integration : Complete

Has been integrated with the manipulation stack, effectively replacing our old registration pipeline.

​

Also a testament to our improved software development workflow. Was developed entirely offline with Docker + ROS bag files and worked perfectly the first time with the robot-in-loop.

​

9 of 22

Initial Tracking Results

Tested Meta’s SOTA model : Co-Tracker

​

10 of 22

Goal: Autonomous Decision Making w/ Behavior Tree

​

​

​

​

​

11 of 22

Goal: Behavior Tree

​

​

​

​

​

One of the child nodes

Main Node

12 of 22

Goal: Behavior Tree

​

​

​

​

​

13 of 22

Additional Progress

  • Set up external camera for bone tracking + obstacle avoidance
  • Reduced camera calibration error ~11mm → ~2mm.
    • Error lower at close range, might consider dual/active registration.

14 of 22

Challenges

2

15 of 22

Challenges - Behavior Tree

Extremely challenging with steep curve.

Multiple errors during:

  • Installation of library
  • Running library in docker
  • Wrapping ROS 2 node in BT Node
  • Colcon build errors
  • Groot 2 communication

V4 vs V3, lack of documentation

Codebase requires major rewrites

16 of 22

Other Challenges

  • Hybrid-planning namespace remapping
  • Moveit configuration (a lot of components need correct params to work)
  • Need to optimize registration pipeline (lots of slow Python code)

17 of 22

Risk Management

3

18 of 22

Risk: Behavior Tree Not Working

​

​

​

​

​

Mitigation

​

  • Behavior Tree is a non-blocking development, continuing independently
  • Back-up of using a Finite State Machine, or well-designed ROS comms
  • Additional effort - fall break buffer helps a lot

​

19 of 22

Future Work

4

20 of 22

Future Work

  • Hybrid Planning (Local Planner Development + Manager Logic)
  • Calibration for both cameras
  • Tracking integrated with Registration Pipeline
  • D435 Feed -> MoveIt Planning Stack for obstacle avoidance
  • Efforts towards BT resolution

21 of 22

Thank You and Break a Leg!

22 of 22

Please keep this slide for attribution

CREDITS: This presentation template was created by Slidesgo, and includes icons by Flaticon and infographics & images by Freepik