1 of 75

Preliminary Design Review

Poorva, Sanil, Sachit, Rohan, Uma

Team J: Align

1

2 of 75

Project Description

Augment Autoware software stack with behaviors for an autonomous vehicle that will transport modular pods in a geofenced environment.
This autonomous vehicle will be able to plan a path to the pods, dock/undock with them, and safely navigate with or without them.
The goal is to make space mobile to save time and human effort.

2

PIXBOT Chassis

Frame representative of Pod

3 of 75

Use Case

3

4 of 75

4

5 of 75

5

6 of 75

6

7 of 75

7

8 of 75

8

9 of 75

9

10 of 75

10

11 of 75

11

12 of 75

12

13 of 75

13

14 of 75

14

15 of 75

15

!

16 of 75

16

17 of 75

17

18 of 75

Mandatory Performance Requirements

The system will:

M.P.1 travel with a maximum speed of 20 kmph with a payload.

M.P.2 plan a path of distance less than 120 km between charges.

M.P.3 travel with a maximum speed of 25 kmph without a payload.

M.P.4 identify the payload handling zone (PHZ) when the chassis is at a distance of 2 m from the periphery of PHZ.

M.P.5 complete docking in less than 120 s. This includes docking initiation, verification and retries.

M.P.6 align with the docking path with an error margin of ± 2 cm.

M.P.7 initiate communication and update sensor and vehicle models with a maximum latency of 60 s.

M.P.8 diagnose failures within 2000 ms from occurrence.

M.P.9 detect object location with an accuracy of ± 5cm.

M.P.10 maintain a minimum distance of 30 cm from obstacles.

18

19 of 75

Mandatory Nonfunctional Requirements

M.N.1 use Autoware as framework for the software stack.

M.N.2 operate in a geofenced environment.

M.N.3 operate in daylight a well-lit environment.

M.N.4 operate on asphalt.

M.N.5 be powered by an on-board battery.

M.N.6 communicate wirelessly in the ISM band.

M.N.7 have a payload capacity of 1000 kg.

M.N.8 be maintainable: code should be modular and well documented.

M.N.9 be unit testable.

M.N.10 be reliable: subsystem failure modes should be well-defined and detectable.

M.N.11 be compliant with relevant standards and regulations.

19

20 of 75

20

21 of 75

21

22 of 75

22

23 of 75

23

24 of 75

24

25 of 75

25

26 of 75

26

27 of 75

27

28 of 75

28

29 of 75

29

30 of 75

30

31 of 75

31

32 of 75

32

33 of 75

33

34 of 75

34

35 of 75

Subsystem Descriptions & Current System Status

35

36 of 75

Sensing Subsystem

36

Sensor Type	Company	Quantity	Purpose
3D LiDAR	Velodyne VLP-16	2	Localization and P2P navigation for general motion
RGB Camera	MindVision	1	Pod identification, Localization wrt pod, Object detection
2D LiDAR	Hokuyo UST 20LX	2	Localization wrt pod, obstacle detection
RADAR	Continental SRR308	1	Obstacle detection (to add redundancy to the 2D LiDAR)
IMU	Xsens MTI-300	1	Localization (augment 3D LiDAR accuracy)
GPS	SwiftNav Piksi	1	Initial pose estimate, Localization (augment 3D LiDAR accuracy)
Encoder	-	~2	Localization (augment 3D LiDAR accuracy)

37 of 75

Sensing Subsystem - Current Status

Have obtained and interfaced with the 3D LiDARs and Mindvision camera

37

VLP-16 puck mounted on a cart to obtain point cloud map of the NSH B-level

MindVision RGB Camera

38 of 75

Navigation Subsystem

Detection

38

Description Process sensor information (LiDAR, Camera, RADAR) Will detect obstacles using sensor data (2D LiDAR, RADAR and Camera) Will detect AprilTags for Pod Identification using Camera data	Current Status Generated point cloud maps using 3D LiDAR data Used ndt_mapping to generate map of NSH B level AprilTag detection implemented in simulation

Generated map of our B Level

39 of 75

Pose estimation using AprilTags

39

40 of 75

Navigation Subsystem

Localization

40

Description Localizes and estimates pose in premapped environment Localizes chassis relative to pod when docking/undocking Generates odometry (using IMU and encoder data)	Current Status Implemented using ndt_matching (using 3D LiDAR data) IRL and in simulation Implemented in simulation, relative to pod legs using 2D LiDAR data and relative to AprilTag on Pod Implemented in simulation

41 of 75

41

42 of 75

Navigation Subsystem

Prediction

42

Description Generates a cost map of system surround- ings using the inputs from the object detector and the precomputed HD vector map.	Current Status Literature review on different methods used in different urban driving scenarios, to draw parallels to our geo-fenced environment needs : Probabilistic graphical models to represent different scenarios Tracking of neighboring objects such as people, cars and other moving objects Review of Autoware’s ‘beyond pixel based tracker’ node, based on Monocular multi-object tracking using simple and complementary 3D and 2D cues (ICRA 2018)

43 of 75

Navigation Subsystem

Align

43

Description This block is activated when the system has to dock or undock. Responsible for switching localization input to AprilTags and 2D LiDAR data.	Current Status Fall Semester

44 of 75

Navigation Subsystem

Mission Planner

44

Description Generates the trajectory waypoints and velocity profile given the cost map and detected landmarks for P2P navigation, and additionally relative pose when planning for approach to pod.	Current Status Implementing in simulation (CARLA) CARLA-Autoware bridge is able to generate waypoints given an initial and final pose for P2P

45 of 75

45

46 of 75

Navigation Subsystem

Motion Planning

46

Description Generates dynamic trajectory and controls the motion of the system on the basis of the current and desired state. Outputs commanded twist.	Current Status Literature survey of control methods using visual servoing done

47 of 75

Navigation Subsystem

Actuation

47

Description Performs twist filtering to ensure that constraints are met, and outputs commanded steering and acceleration.	Current Status Implemented in simulation: sending Ackermann steering and velocity commands to vehicle in CARLA simulation

48 of 75

Docking and Undocking Subsystem

Docking Position Verification

48

Description Once docking position is achieved this node verifies if the correct position has been reached. If yes, the system state is updated to reflect so. If not, then the process of navigating from the PHZ to the pod is repeated until the position is within tolerable limits.	Current Status To be implemented in simulation in Spring.

49 of 75

Docking and Undocking Subsystem

Adapt

49

Description Once successfully docked or undocked, this block updates the system dynamics and sensor model.	Current Status Fall Semester

50 of 75

Safety Subsystem

Health Monitoring System

50

Description Monitors functioning and outputs of critical nodes. If failures are detected, this node updates system state to emergency handling state.	Current Status Exploring the use of the Health Checker Function for Autoware

51 of 75

Safety Subsystem

Emergency Handling

51

Description When the system transitions into the emergency handling state this node ensures: The system is safely brought to a halt Failure log updated Teleop request sent if needed Diagnosis performed	Current Status Fall Semester

52 of 75

Simulation

52

53 of 75

Hardware

Flashed Jetson with all required packages, including Autoware with CUDA.
Currently working with an RC car from inventory and will be controlling it via the Jetson using I2C and servo drivers.
Chassis will be received by the end of Spring semester.

53

54 of 75

Project Management

54

55 of 75

Work Breakdown Structure

55

56 of 75

56

57 of 75

57

58 of 75

Schedule

58

59 of 75

Updated Spring Milestones

59

Date	Old Milestones
1/20	Autoware Environment Setup
2/15	Website Deployed
2/19	Simulation Platform Setup
3/4	Navigation and Safety Design Ready
3/25	Safety Subsystem Ready
4/8	Navigation Subsystem Integrated
4/22	SVD
5/4	CDR

Date	Milestones
1/20	Autoware Environment Setup
2/15	Website Deployed
2/19	Simulation Platform Setup on CARLA-Autoware
3/4	Navigation subsystem design & Simulation setup for docking ready
3/25	Navigation subsystem components tested in Simulation: 1. Planning algorithms for docking, 2. Localization of chassis relative to the pod 3. Control algorithms for docking
4/8	1. Navigation subsystem developed 2. Docking subsystem components tested in in Simulation 3. Safety subsystem components Object Detection and Behaviour Prediction developed
4/15	Health Monitoring Design Ready, Navigation Subsystem and Docking subsystem Software Integration done.
4/22	SVD
5/4	CDR

60 of 75

Fall Milestones

60

Date	Fall Milestone
31-Aug-2020	Hardware testing and setup
15-Sept-2020	Docking subsystem completed
30-Sept-2020	Vehicle models ready for adapt subsystem
15-Oct-2020	Safety subsystem completed, adapt subsystem designed
31-Oct-2020	All subsystems finished and unit tested
30-Nov-2020	Integration testing finished

61 of 75

High Level Test Plan - Spring

61

Date	Functionality	Test Plan
Progress Review 3 3/25	Payload handling zone identification	Add PHZ coordinates to Map In simulation take vehicle near that zone Vehicle should identify Payload Handling Zone and log it
	Planning algorithms for docking in simulation	Place vehicle in docking zone initial pose The chassis should plan a path based on the non-holonomic constraints of an Ackermann steering vehicle
	Point to Point Navigation in mapped custom CARLA world	Provide initial pose and final/desired pose in CARLA map Navigate using Autoware functionality
	Control algorithms for docking in simulation	Place vehicle in docking zone initial pose Execute the planned path in a controlled way , while maintaining alignment.

62 of 75

62

Date	Functionality	Test Plan
Progress Review 4 4/8	Object detection	Use Autoware’s object detection nodes to detect obstacles Perform obstacle avoidance based on detected obstacles
	Behaviour Prediction	Use beyond pixel tracker to predict location of objects between consecutive frames Perform collision avoidance - stop if path is disrupted by obstacle prior hand
	Health Monitoring	The system will monitor all nodes and take a diagnostic feedback. If any node becomes non-functional or provides invalid feedback diagnostics will begin and based on the critical function of the node action will be taken.
	Localization of chassis relative to the pod	The chassis is placed at initial docking pose The chassis should get the relative pose wrt the pod using AprilTags and 2D lidar data. The pose should get more certain with number of measurements being added as the chassis approaches for docking.
	Approach Navigation	The chassis is placed at initial docking pose within the PHZ The final pose of pod is provided (obtained by relative localization) A path to the pod is planned and executed to move the chassis under the pod while considering Ackermann steering constraints
	Position Accuracy Verification	The error between the desired pose and goal pose is measured after approach navigation

63 of 75

Spring Validation Experiments

Equipment: Laptop with Autoware and CARLA, and a precomputed map of the CARLA world.

Test 1: Navigation

Description:

The system will navigate from one end of the designated area in simulation to a distance of 2m from the payload handling zone, without any obstacles in the way.

Procedure:

Use Autoware to ask vehicle to go to PHZ coordinates in the CARLA world.

Success Criteria:

Path to the destination point is planned and executed to the (M.P.2)
After reaching the 2m proximity of the point the vehicle identities the payload handling zone. (M.P.4)

63

64 of 75

Test 2: Alignment and Docking

Description:

Once it has reached the payload handling zone, the system will plan a path for aligning with the payload handling zone and execute it. The system will verify the accuracy of the docking (and relay the output to the user). This will be done in the docking simulation.

Procedure:

Place the chassis in front of the Pod which has 2 April tags and give docking command with pod id in Gazebo Simulation
The chassis will identify the pod, confirm the pod ID and get the relative pose.
The chassis will execute the docking command, align with the pod and reach directly under the pod.

Success Criteria:

The chassis will align with the pod and have an error margin of ±2 cm(M.P.6)

64

65 of 75

Test 3: Safety

Description:

For demonstrating safety during operation, 2 obstacles: one static and one dynamic will be made to obstruct the vehicle. The minimum dimensions of said obstacles will be 30x30 cm 2 (height and width). The system will stop at a distance of more than 30 cm from the detected obstacle.
Additionally, we plan to intentionally make one or more of the sensor nodes crash, so that the system can demonstrate its safety behaviours.

Procedure:

Vehicle will be asked to go from point A to point B in the simulated CARLA world.
Obstacles will be placed in the path while Vehicle is traversing it.
A sensor node will be crashed and response will be noted.

Success Criteria:

Failures will be diagnosed within 2000 ms from occurrence and reported on dashboard (M.P.8)
Vehicle will detect object location with an accuracy of ± 5 cm (M.P.9)
Vehicle will maintain a minimum distance of 30 cm from obstacles (M.P.10)

65

66 of 75

Test 4: Retracing

Description:

We will introduce a slight misalignment between the pod and chassis manually.
The system should recognize this misalignment, and retrace the path and follow the corrected path to align correctly and dock. (There will be no latching or its verification).

Procedure:

Chassis, pod, laptop, and a precomputed map of the location.

Success Criteria:

Docking will be done within 120s ( partially fulfilled). (M.P.5)
The chassis will align with the pod and have an error margin of ±2 cm(M.P.6)

66

67 of 75

High Level Test Plan - Fall

67

Date	Functionality	Test Plan
September	(un)Docking	Give the chassis+pod a command to undock at a specific point The chassis should drop the pos at that specific point.
October	Adapt	Ask the chassis to pick up the pod The chassis should update the vehicle dynamics as per the updated state and log it.
November	Safety and Diagnostics	Diagnostic feedback is given on request and all emergencies are logged
December	Integration Done	Refer to Fall Validation Experiments

68 of 75

Fall Validation Experiments

Equipment: Chassis, pod, laptop with Autoware and CARLA, and a precomputed map of the CARLA world.

Perform Tests 1 to 4 on actual hardware

Test 5: Lock with Pod and Verify

Description:

Having verified the accuracy of the alignment with the pod, the system executes the locking action. A (yet undecided) sensor is used to verify that the mechanism has locked successfully.

Procedure:

Perform Test 4 with locking enabled.

Success Criteria:

Docking will be done within 120s ( complete). (M.P.5)

68

69 of 75

Test 6: Undocking

Description:

Having performed docking and locking, on command the chassis should leave the Pod at desired location

Procedure:

Perform Test 5, and ask chassis to leave pod at another point
The chassis should reach the coordinate and drop the pod as per desired pose.

Success Criteria:

Undocking will be done within 120s ( complete). (M.P.5)
The drop location and its approach coordinated will be updated in the PHZ database.

69

70 of 75

Test 7: Adapt

Description:

The system will update the vehicle model and sensor model after it has docked or undocked.

Procedure:

Perform Test 1 and 5 again, after the docking is complete perform Test 1 again.

Success Criteria:

Vehicle will have shifted the dynamics based upon the updated state and the dynamics involved in Test 1 would be different.
The changed vehicle model will be updated in the published logs, thus verifying that the adapt subsystem works.

70

71 of 75

Budget

As we have hardware and sensors being provided by our sponsor or are able to borrow them from inventory, we have only spent some of our budget shown below.

Remaining budget: $4929.55

71

Item	Reason for purchase	Cost
Adafruit 16-ch 12 bit servo driver (x4)	To control the RC car with a Jetson TX2 while we wait for the chassis from PIX	$70.45

72 of 75

Risk Management Table

72

Risk ID	Reqt	Risk	Type	Likelihood	Consequence	Mitigation
R1	All MPs	Legacy System might have issues	Technical, Schedule	3	4	Develop test plan for chassis Test algorithms in simulation
R2	All MPs	Delay in arrival of chassis	Schedule, Programmatic	5	5	Use RC car or cart as replacement Test subsystems in simulation
R3	MPs 4, 5, 6, 9, 10	Sensors stop working	Technical, Cost	4	3	Sponsor sends sensors early, we perform tests to check usability Use hardware abstraction layer
R4	All MPs	Testing infeasible due to weather conditions	Technical, Schedule	4	3	Plan tests for late spring Test subsystems in NSH B-level

R5	All MPs	Communication gap with sponsor	Schedule	3	3	Document work and provide regular updates

73 of 75

Risk Management Table

73

Risk ID	Reqt	Risk	Type	Likelihood	Consequence	Mitigation
R6	MPs 7, 8, 10	Issues in Subsystem Integration	Technical, Schedule	4	4	Perform testing on individual subsystems Develop integration test plan initially.
R7	MPs 4, 5, 6	Overambitious requirements	Technical, Schedule	4	4	Descope System if Necessary
R8	All MPs	Integration issues with code	Technical, Schedule	2	3	Document work with changes implemented Perform unit testing at each level possible Use a common framework to develop code
R9	All MPs	Global pandemic occurs, causing campus shutdown	Programmatic	1	5	Move everything to simulation Take care of personal health by practicing social distancing and communicating via the internet
R10	All MPs	Simulation does not pan out in real-life	Technical	5	4	Test subsystems individually so that there is no ambiguity in whether the problem is actually due to conversion from sim2real or lies in software

74 of 75

Risk Likelihood-Consequence Table

74

Consequence

Likelihood

Major Risks:

R1: Legacy system issues

R2: Delay in chassis arrival

R3: Sensors stop working

R4: Testing Infeasible

R6: Integration Issues

R7: Overambitious reqs

R8: Code integration issues

R9: COVID 19

R10: Sim2Real doesn’t work

R1

R2

R3

R4

R5

R6

R7

R8

R9

R10

75 of 75

75

Questions?