Team Members:
Gitaek Lee
Thomas Xu
Calen Robinson
Jinkun Liu
Yukun Xia
Preliminary Design Review Presentation (3/22)
Project Description
Predictive Avoidance for Industrial Mobile Robots
Context:
Main goals:
Sponsor: OMRON
3
Main Components
Best validated in real-life
4
Best validated in simulation
in Multi-robot-vehicle Interaction
Real-Life System (RLS)
Main Systems
Virtual Robot System (VRS)
5
Use Case
Use Case
The robots are deployed from their storage area and begin moving autonomously toward their commanded goal points to complete tasks. On its way to the next task endpoint, one of the robots detects a moving obstacle in its path. The robot detects both the obstacle’s position and its classification: a forklift. Using this information, the robot predicts its future motion and determines an evasive maneuver. It takes the maneuver to avoid the obstacle and continue on its path.
7
Requirements
Performance Requirements - Virtual System
From sponsor
specs and
environment
Overall performance metric
For real-time
operation
9
The System will:
V1. Autonomously move with max speed of 1.8 m/s
V2. Autonomously move with max rotation speed of 60 deg/s
V3. Autonomously move with min cruise speed of 0.5 m/s
V4. Increase productivity1 by > 5% using avoidance strategies based on object classification, compared to the nominal case2
V5. Operate in real-time with planning time within 100 ms
V6. Receive user-commanded waypoints within 1 s (desirable)
[1] Productivity: The number of finished delivery tasks per unit time (e.g. day)
[2] Nominal case: Avoidance without considering classification or prediction (i.e. considering obstacles as “static” at each instant in time)
Performance Requirements - Real-life System
10
The System will:
R1. Classify obstacles1 of interest with mAP2 of at least 60%
R2. Detect positions of obstacles of interest within 0.1 m accuracy
c. Detect obstacles of interest within a range of 3 2 m
R4. Output results of positioning and classification within 100 ms per
frame
[1] Obstacles: Forklifts, pedestrians, and other OMRON robots (simulation only)
[2] mAP: Mean average precision
Non-functional Requirements
11
(Restriction based on sponsor’s robots)
(Allow comparison against nominal case)
[1] Modular: Able to switch between any combination of individual avoidance algorithms on-the-fly during operation operate with either only local A* (nominal case) or predictive avoidance subsystem
[2] Non-empty: The environment for validation is not only free space, but contains static walls/corridors/objects so that it resembles an actual industrial environment more closely
Architectures
Functional Architecture - Virtual System
13
Cyberphysical Architecture - Virtual System
14
Functional Architecture - Real-life System
15
Cyberphysical Architecture - Real-life System
16
Robot
Data Acquisition Subsystem
Computing Platform
Mobility Subsystem
Visualization Subsystem
Ground Truth Subsystem
Environment
Command Module
YOLOv4
Sensor Fusion
Camera
2D LiDAR
Movement Commands
Marker Tracker
Wi-Fi Module
Motor Controller
Position Error Calculator
Wheel Motor
Wheels
Classification mAP Calculator
Camera
LiDAR Data Processing
Obstacle/robot positions in world frame
Data Visualizer
Obstacle Mock-ups
Obstacle positions in robot frame
Power Supply
Battery/Tether
Power
Subsystem Description/Progress
Factory Environment
18
Full-scale factory environment (92m x 64m)
Occupancy grid map for the factory
Full-scale, 92 x 64 m; Done, with tuning in-progress
Object Models
19
Robot (OMRON LD-60)
- Differential drive
Forklift
- Reverse Ackermann steering
Robot Path Planning and Pure Pursuit
20
Path Planning:
x
x
x
y
y
y
start: (6, -2)
start: (6, 2)
goal: (5, 8)
goal: (5, 8)
Top view of the environment
Costmap with path 1
Costmap with path 2
Forklift Trajectory Recording and Replay
21
2x speed
Obstacle Filtering
22
To be implemented in Fall
Local A*
23
Credit: https://www.jstage.jst.go.jp/article/transinf/E96.D/2/E96.D_314/_article
Trajectory Prediction
24
Kruse, E.; Gutsche, R.; and Wahl, F.: Acquisition of statistical motion patterns in dynamic environments and their application to mobile robot motion planning. In: Intelligent Robots and Systems (IROS). Volume 2, pages 712–717, 1997.
Stochastic Trajectory “Dictionary”
25
[KGW97]
Predictive Avoidance
26
Current Status:
Physical Robot Base
27
Current System Status - Physical Robot Base
Robot base: all sensors, computing platform, mobility subsystem
Untethered operation with Wi-Fi and battery
28
Real-Life Obstacle Perception
29
Credit: Stanford
Real-Life Ground Truth Subsystem
30
Project Management
WBS
32
Spring 2021 Schedule
Fall 2021 Schedule
Milestones
Milestone | Capabilities | Test Plan |
PR3 |
|
|
PR4 |
|
|
PR5-6 (SVD) |
|
|
September |
|
|
October |
|
|
November |
|
|
FVD |
|
|
VRS = Virtual Robotic System
RLS = Real-life Robotic System
Spring Validation Demonstration
36
Fall Validation Demo - Simulation
37
Fall Validation Demo - Real-life Perception
38
Budget
39
Part | Quantity | Unit Cost | Total Cost |
TurtleBot (Burger) | 1 | $549.00 | $549.00 |
Mic/Music stand | 1 | $100.00 | $100.00 |
Lights | 2 | $70.00 | $140.00 |
Doll collection | 1 | $35.00 | $35.00 |
Forklift A | 1 | $38.00 | $38.00 |
Forklift B | 1 | $49.98 | $49.98 |
NVIDIA Jetson Xavier AGX | 1 | $700.00 | $700.00 |
RPLIDAR A2 | 1 | $319.00 | $319.00 |
Firefly S Camera | 1 | $199.00 | $199.00 |
M.2 Wi-Fi card | 1 | $25.00 | $25.00 |
Additional Hardware | 1 | $46.06 | $46.06 |
PDB PCB parts | 1 | $200.00 | $200.00 |
Total | - | - | $2,481.04 |
Risk Table
40
Risk Matrix
Before Mitigation
After Mitigation
41
Questions