OPTIMAL ROBOTIC ARM GRIPPER

��������

• Our idea is to develop a robotic arm with the ability to grasp the reformable (rigid) objects using 6d pose estimation, estimated by monocular camera.

​

• Amazon is hosting a yearly robotics “picking” challenge.

​

PROJECT BACKGROUND

The RBO team’s robot placing a pack of Oreo cookies that it retrieved from the warehouse shelf into a tote. Image courtesy of RBO team.

PROJECT BACKGROUND

The final scores APC scores of the 13 teams participated

PROJECT BACKGROUND

TASK DIVISION

​

Task 1: Research and preparation

Task 2: Data collection and preparation

Task 3: Model training and evaluation

Task 4: Model optimization and testing

Task 5: Robotic Arm Assembly and Integration

Task 6: Kinematic Modeling & Inverse Kinematic Modeling

Task 1: Research and preparation

​

​

Task 2: Data collection and preparation

Task 3: Model training and evaluation

Task 4: Model optimization and Quantization

Task 5: Robotic Arm Assembly and Integration

Task 6: Kinematic & Inverse Kinematic Modeling

Assessing the Progress: An Overview of Project Flow

6D Pose Estimation

Research & Preparation

Data Collection & Preparation

Model Training and Evaluation

Model Optimization & Quantization

Robotic Arm Assembly

Kinetic Modeling of Robotic Arm

Raspberry Pi Communication Framework

Grasping Techniques

6D Pose Estimation of the Object

​

Research and Preparation:

​

• Researching and preparing tools for estimating 6D pose from visual data.

​

​

​

Methodology:

We studied and discussed the following most important 4 publications related to 6d pose estimation of the object in detail. Though we also gathered information from more publications and sources too.

​

• https://arxiv.org/pdf/2204.10776.pdf (Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Images )
• https://arxiv.org/pdf/1601.05484.pdf (Analysis and Observations from the First Amazon Picking Challenge)
• https://arxiv.org/pdf/2004.12729.pdf (Single Shot 6D Object Pose Estimation)
• https://arxiv.org/pdf/1711.08848.pdf (Real-Time Seamless Single Shot 6D Object Pose Prediction )

6D Pose Estimation of the Object

Data Collection and preparation :

• Preparing data for 6D pose model training involves collecting images/videos and creating annotations

​

Methodology:

• Collected data from GENMOP and LINEMOD.
• Labelled custom data with FFmpeg for portable mouse.

6D Pose Estimation of the Object

Model Training and Evaluation:

• Train and evaluate the 6D pose model with prepared data, then fine-tune as needed.

​

Methodology:

• Trained and evaluated object detector using Gen6d pose estimator, with results shown below.

​

​

6D Pose Estimation of the Object

Model Optimization and Quantization:

• Optimizing the 6D pose model for performance and testing.

​

Methodology:

• Tuned the detector by hit and trial method and analyzing the validation and train loss.
• Did quantization of the model using pruning and decreasing size of model technique.
• Best hyperparameters:
• reference image resolution: 128
• view number used in selection: 64
• view number used in detection: 32
• refinement iteration number: 3

​

Robotic Arm Assembly and Integration

Components and Hardware Integration

​

Servo Motor Integration:

• Step-by-step attachment and calibration process.

Webcam Integration:

• Secure mounting and visual perspective optimization.

Raspberry Pi Integration:

• Control hub placement and communication interfaces.

Other Component Integration:

• Supporting elements for reliability and functionality.

​

List of Parts Needed

Wiring Diagram

Kinematic Diagram

Servo Motors & Their Angles

Motors Insights

​

Motor 0 :

• 0° to 180°

​

Motor 1 :

• 90° to 180° ↑
• 180° to 90° ↓

​

Motor 2 :

• 90° to 0°

​

Motor 3 :

• 180° to 90°.

​

Motor 4 :

• 0° to 180°

​

Motor 5 :

• 0° to 90°

​

Kinematic Diagram�(With Measuremnets)

Final Picture

Charging Dock Demo

Charging Dock Demo

MILESTONES ACHIEVED

THANKS FOR YOUR TIME ☺