Dynamic Handover: Throw and Catch with Bimanual Hands
Overview
Binghao Huang1,*, Yuanpei Chen2,*, Tianyu Wang1, Yuzhe Qin1, Yaodong Yang2, Nikolay Atanasov1, Xiaolong Wang1
1University of California San Diego 2Peking University
Learning Bimanual Dexterous Hands Policy
Experimental Results Highlight
• Multi-Agent Reinforcement Learning (MARL). We propose to tackle this problem as a multi-agent problem, with each hand being one agent.
• Dynamic Dexterity. Using RL with randomization allows the network to learn to handle diverse dynamics.
• Object trajectory prediction. We introduce a model to predict the object’s future trajectory and destination ahead in real time.
Predicted throwing goal
Predefined throwing goal
Manipulated Objects
Our Robot System
We employ two Allegro Hands, each individually mounted on separate XArm-6 robots, arranged in a face-to-face configuration. We incorporate a RealSense D435 camera for real-time object position tracking, which is oriented towards the working space.
• Stage1: Multi-Agent Reinforcement Learning • Stage2: Goal Estimator Learning • Stage3: End2End Joint Learning
Our method outperforms the baseline methods, indicating the effectiveness of multi-agent reinforcement learning(MARL) and goal estimation.
Real World Results
Simulation Results
Perturbation Test in Sim
Our policy could generalize to more unseen objects in the real world!