Practical Approaches to �High-Level Behavior in Robotics

Sebastian Castro

Roboticist / Applied Scientist

1

Outline

• Introduction�
• High-level behavior
1. Skill composition and planning
2. Software abstractions for skill execution
3. Examples of practical robot behavior systems�
• Concluding remarks

2

Academic Background

3

B.S. + M.S. Mech. Eng.�Cornell University

High-Level Control of Modular Robots�Castro, Koehler, and Kress-Gazit (2011)

Takeaways: composition + verifiability

• Structured English specification → LTL → verifiable behavior → controllers → robot.
• Test in simulation, validate on hardware.

Professional Background

4

How do we program (autonomous) robots?

5

Deliberation Tools

• Named routines (e.g., “pick”, “place”)
• High-level goals (e.g., “tidy up the kitchen”)
• Natural language or nonverbal communication instead of code
• Learned preferences�(demonstrations, feedback, etc.)
• …

Communicating intent to robots

6

4x

Bring a healthy snack to my desk.

?

What is Deliberation?

Thinking about something and deciding carefully what to do next.

​

Deliberation for autonomous robots: A survey Ingrand & Ghallab (2017)

​

7

I. Skill Composition and Planning

8

Task Planning

Goal: Desired state of the world � “The apple should be on the table”

9

Task plan: List of high-level actions to achieve a goal.

1. Go to shelf
2. Pick up apple
3. Go to table
4. Place apple on table

If we have a model, we can use it for planning.

(Some) Specifications for Task Planning

10

Searching for a solution (PDDL)

11

Goal: (At apple table)

​

Initial state:

(At robot shelf)

(At apple shelf)

(HandEmpty robot)

​

place(r, a, t)

move(r, t, s)

Searching for a solution (PDDL)

12

place(r, a, t)

(At robot shelf)

(Holding robot apple)

(At robot table)

(At apple shelf)

(HandEmpty robot)

pick(r, a, s)

move(r, s, t)

Plan:

pick(robot, apple, shelf)

move(robot, shelf, table)

place(robot, apple, table)

Planning too abstractly has its drawbacks…

13

Goal: (At apple table)

Plan:

pick(robot, apple, desk)

move(robot, desk, table)

place(robot, apple, table)

“The Downward Refinement Property” Bacchus & Yang (1991)

Multi-Modal Motion Planning (MMMP)

Most “hard” robotics problems involve motion planning through mode transitions�(e.g., making / breaking contact, kinematic reconfiguration).

​

14

Kingston et al. (2020)

Task and Motion Planning (TAMP)

Some solutions provide task skeletons and fill them in by sampling mode transitions and feasible motions, e.g., MoveIt Task Constructor (Görner et al. (2019)).

Others perform integrated TAMP by combining search and sampling, e.g.,�LGP (Toussaint (2015)), IDTMP (Dantam et al. (2016)), PDDLStream (Garrett et al. (2018))

15

Classical planning requires domain engineering

This is known as neuro-symbolic planning.

LLMs/VLMs can help us… with many caveats.�Directly planning with LLMs:� Singh et al. (2022), Fabiano et al. (2023), Silver et al. (2023)�Generating domains: Smirnov et al. (2024), Oswald et al. (2024), � Han et al. (2024), Quartey et al. (2024)

Learning can also help discover operators necessary for planning, from task demonstrations and/or self-exploration.�Konidaris et al. (2018), Asai et al. (2021), Silver et al. (2022)

16

Search is expensive for hard problems

Some solutions:

​

​

​

​

​

​

​

* presenter contributed to this work

• Hierarchical task (and motion) planning. �Sacerdoti (1975), Kaelbling & Lozano-Pérez (2011), Toussaint (2015)

• Contracts that (can) guarantee downward refinement. Vasilopoulos et al. (2022)*, Marcucci et al. (2021)

• Search pruning strategies, e.g., Monte Carlo Tree Search. Bradley et al. (2021)*, Ren et al. (2021), Pfeiffer et al. (2023)

• Learning abstract feasibility classifiers as heuristics.�Wells et al. (2019), Bradley et al. (2021)*, Noseworthy et al. (2021)*

17

Summary: Task planning has many design tradeoffs

• Flat vs. hierarchical task planning?�
• Interleaved vs. deferred motion planning?�
• Manually engineered vs. learned transition models?�
• Geometric vs. abstract feasibility?�
• Search vs. sampling? (exploration vs. exploitation)

18

II. Software Abstractions for Skill Execution

19

(Sadly) Planning is not enough

“In preparing for battle I have always found that plans are useless, �but planning is indispensable.” - Dwight D. Eisenhower�

“All models are wrong, but some are useful.” - George E. P. Box�

“Never gonna give you up, never gonna let you down.” - Richard P. Astley

20

Executing task plans requires engineering

• (Even) More up-front deliberation: �TAMP / Contingency planning / belief-space planning.

• Replanning at the task level:�Either on failures or as an anytime planner.

• Robust, reactive skills: Can be manually engineered rules/controllers or learned policies.

• Human-in-the-loop: (aka “task teleop”) Engaging users to addressing uncertainty in plans before execution or at runtime.

21

What does it take to execute task plans in the real world?

22

Execution Layer

Task Planner

?

Modular software systems (like ROS) can enable this:

• Middleware to connect heterogeneous software components.
• Rich ecosystem of domain-specific libraries.

From task plans to executable behavior

23

Task Planner

Skill abstractions

Execution Layer

Implementing task plans as robust, reactive skills is challenging to do with “plain code”.

Tools designed for skill execution can help manage complexity.

Skill Abstractions

24

Images from Castro & Stout (2024)

Some formalisms for skill execution:

• (Hierarchical) Finite-State Machines
• Behavior Trees
• Sequential Composition
• Teleo-reactive programs
• Petri Nets
• …

​

Refer to Colledanchise & Ögren (2017)

Examples: Manually Engineered Reactive Behaviors

25

Tabletop manipulation BT�Robot reacts to object picking failure by moving occluding object elsewhere on the table.

Drone Delivery FSM�Vehicle switches between takeoff, waypoint following, and landing states during operation.�(Video courtesy of José Avendaño @ MathWorks)

Example: Task Plan to Behavior Trees

26

Action type: Pick�Target object: Soda�Target room: ---

Target location: ---

Action type: Move�Target object: ---�Target room: Bedroom

Target location: Table

Action type: Move�Target object: ---�Target room: Living

Target location: Floor

Action type: Place�Target object: Soda�Target room: ---

Target location: Floor

Frameworks for Task Planning and Skill Execution

27

PlanSys2: Martin et al. (2021)�Modular PDDL planners → Skill abstractions → C++ / ROS 2 implementations

​

Frameworks for Task Planning and Skill Execution

28

SkiROS2: Mayr et al. (2023)�World model → Task planner → Skill abstractions → Python / ROS 2 implementations

​

Communicating back to the task planner

29

Need to supply information back to the task planner to keep knowledge updated.

This is true whether a task plan succeeds, fails, or is in progress.

Task Planner

Skill abstractions

Execution Layer

World models

World representations and ontologies are the “glue”

30

Metric-semantic maps�Hydra: Hughes et al. (2022)

Simple surrogate world models�PyRoboSim: Castro (2022)

Ontologies: Data structures that describe entities in the world, task-relevant properties, and how the entities relate to each other.

​

Summary: Planning and execution is a feedback loop

31

Task Planner

Skill abstractions

Execution Layer

World models

III. Examples of Practical �Robot Behavior Systems

32

Home Service Robots (2020)

33

Behavior Composer

Behavior Trees

Knowledge Base

• World model
• Ontology
• Behavior library

Skill �Sequence

Task (and motion) planning with our world model

34

Sampling navigation paths and object placement locations.

PDDLStream �Garrett et al. (2018)

35

~3x

“There should be an apple inside a drawer.”

4x

“The banana should be in a drawer and spam goes on the table.”

Home Service Robots: Summary

• Task and (sort of) Motion Planning.�“Motion” part was minimal; only sampling navigation and placement poses.�Navigation, manipulation planning, and object relocalization deferred to runtime.�
• Heavily reliant on ontologies and world models.�World map, object categories, and initial state were all known.�
• Vision + language models were tightly integrated.�RNNs to translate natural language to valid goal specifications.�Fine-tuned Mask R-CNN to detect objects and modify world models.�Ground complex referring expressions via image + text embeddings (Roy et al. (2019))�
• Gradual sim-to-real verification pipeline.�2D behavioral simulation → Gazebo simulation → Real robot

36

Planning with Learned Policies (2024)

Equivariant Diffusion Policy �Wang et al. (2024)

• Visuomotor policies trained by behavior cloning.
• Equivariant networks generalize �to SO(2) planar rotations.

37

Can we recompose these learned policies to solve different tasks that utilize the same (sub)skills?

Skill Composition with Learned Policies

38

Action type: Pick�Target object: Soda�Target room: ---

Target location: ---

Action type: Move�Target object: ---�Target room: Bedroom

Target location: Table

Action type: Move�Target object: ---�Target room: Living

Target location: Floor

Action type: Place�Target object: Soda�Target room: ---

Target location: Floor

Preliminary Experiments

39

Goal: Bagel in oven + door closed

“Loopback test”: Recreate original demonstration using task planning.

Can we detect when recovery is insufficient because we’re out of the skill’s initiation set / out of distribution?

Preliminary Experiments

40

Goal: Tray stowed + door closed

Subskills rearranged from new task plan that utilizes the same planning domain.

How to make subskill transitions that were not seen during training?

Recap and Looking Forward

• Task planning is purely symbolic… and that’s okay!�Robustness, reactivity, and motion planning is “baked into” learned behaviors.�Intentionally using simple planners and deferring details to the policies.�
• (Near-)Future directions:
• Determining predicates and/or skill termination sets.
• Object- and/or goal-conditioned policies.
• Skill (re-)compositions not seen at training time.�
• Expand robot morphologies and sensor modalities.�Navigation + mobile manipulation, bimanual manipulation, force feedback, etc.

41

Concluding Remarks

42

On Manual Engineering vs. Learning

Compositionality is key:

• End-to-end learning was an interesting exercise, but the community is moving on.
• Isolate sources of failure to specific components (learned or otherwise).
• Modular systems increase research agility.

Don’t discard Good Old Fashioned Engineering:

• We have good mathematical models and algorithms for certain applications �(e.g., robot kinematics, state estimation).
• Apply ML to hard problems, compose/validate with model-based approaches.
• (Good) Visualization and logging tools will save your life.

43

On Robotics Research / Engineering

My career has been a mix of education, software tools, and research.

44

• Good embodied intelligence research requires nontrivial engineering effort. (Woolston (2022))

• Be accessible: Know how to talk the talk, but also make your work palatable to newcomers.

• Fallacy: Robotics is complex, therefore its tools must be difficult to use. �(Holson (2024), Bernhardsson (2024))

Thank You!

Open-source software:

• PyRoboSim: ROS 2 simulator for TAMP https://github.com/sea-bass/pyrobosim
• PyRoboPlan: Motion planning for manipulators https://github.com/sea-bass/pyroboplan
• TurtleBot3 Behavior Tree demos https://github.com/sea-bass/turtlebot3_behavior_demos

Blog posts @ roboticseabass.com:

• Introduction to Behavior Trees (2021)
• Introduction to Task Planning + TAMP (2022)
• How do Robot Manipulators Move? (2024)
• Advice to Young Engineers (2024)

Get in touch! [Twitter] [LinkedIn]

45