JavaScript isn't enabled in your browser, so this file can't be opened. Enable and reload.

1 of 92

MRSD Team D: GetAGrip.AI

CoDRP 12/08/2022

1

2 of 92

2

Meet the Team

Manipulation Team

Perception Team

Alec

Sri

Solomon

Jiyoon

Ishu

3 of 92

3

Bill owns a mid-size farm in Iowa and typically hires local workers during the harvest season

4 of 92

4

Labor shortages costed Bill significant amounts of money last year since many of his peppers were not harvested in time and rotted

5 of 92

5

Bill decided to invest in Peter, the autonomous pepper harvesting robot. Peter navigates around the farm and identifies peppers. He moves his robot arm to accurate pepper locations. He grips, cuts, and extracts a pepper without plant damage. Peter places the pepper in a basket and moves on to harvest the next pepper in sight. At the end of the day, Bill collects the buckets of peppers Peter harvested and sends them off to the local market to make a profit.

6 of 92

6

Starting Point

Peduncle: Stem attached to the pepper

7 of 92

7

7

Problem Statement

Our system will demonstrate the identification and extraction of green peppers from pepper crops. To this effort, we will initially go after visible and unobstructed peppers with the following priorities:

Design and build a new end-effector for gripping and cutting the peduncle (50%)
Identification of peppers and peduncles to calculate pose using perception (30%)
Plan path of arm to pepper without causing plant damage (20%)

8 of 92

8

8

System Level Trade Study: FVD Scope

		Value 1 (bad) - 5 (good)
Criteria	Weight (100%)	Very Structured Environment (No occlusion, No navigation, No ripeness check)	Semi Structured Environment 1 (occlusion, No navigation, No ripeness check)	Semi Structured Environment 2 (occlusion, navigation, No ripeness check)	Unstructured Environment (occlusion, navigation, ripeness check)
Team Interest	20%	2	3	5	4
Sponsor Interest	20%	3	3	4	4
Farmer Interest	10%	2	3	4	5
Difficulty	25%	5	4	3	2
Cost	20%	5	4	3	2
Fits into MRSD scope	5%	3	3	4	3
Score		3.6	3.45	3.75	3.15

9 of 92

9

Functional and Non-functional Requirements

	Functional Requirements
FR.1	Distinguish Peppers from Plant
FR.2	Plan Path to Pepper
FR.3	Reach Pepper
FR.4	Grip Pepper
FR.5	Extract Pepper
FR.6	Place Pepper
FR.7	(Stretch) Navigate to New Plant

	Nonfunctional Requirements
NFR.1	Pepper/Plant Damage
NFR.2	Size
NFR.3	Cost
NFR.4	Safe
NFR.5	Speed of Operation

10 of 92

10

Functional and Non-functional Requirements

	Functional Requirements
FR.1	Distinguish Peppers from Plant
FR.2	Plan Path to Pepper
FR.3	Reach Pepper
FR.4	Grip Pepper
FR.5	Extract Pepper
FR.6	Place Pepper
FR.7	(Stretch) Navigate to New Plant

	Nonfunctional Requirements
NFR.1	Pepper/Plant Damage
NFR.2	Size
NFR.3	Cost
NFR.4	Safe
NFR.5	Speed of Operation

11 of 92

11

Functional and Non-functional Requirements

	Functional Requirements
FR.1	Distinguish Peppers from Plant
FR.2	Plan Path to Pepper
FR.3	Reach Pepper
FR.4	Grip Pepper
FR.5	Extract Pepper
FR.6	Place Pepper
FR.7	(Stretch) Navigate to New Plant

	Nonfunctional Requirements
NFR.1	Pepper/Plant Damage
NFR.2	Size
NFR.3	Cost
NFR.4	Safe
NFR.5	Speed of Operation

12 of 92

12

Functional and Non-functional Requirements

	Functional Requirements
FR.1	Distinguish Peppers from Plant
FR.2	Plan Path to Pepper
FR.3	Reach Pepper
FR.4	Grip Pepper
FR.5	Extract Pepper
FR.6	Place Pepper
FR.7	(Stretch) Navigate to New Plant

	Nonfunctional Requirements
NFR.1	Pepper/Plant Damage
NFR.2	Size
NFR.3	Cost
NFR.4	Safe
NFR.5	Speed of Operation

13 of 92

13

Functional and Non-functional Requirements

	Functional Requirements
FR.1	Distinguish Peppers from Plant
FR.2	Plan Path to Pepper
FR.3	Reach Pepper
FR.4	Grip Pepper
FR.5	Extract Pepper
FR.6	Place Pepper
FR.7	(Stretch) Navigate to New Plant

	Nonfunctional Requirements
NFR.1	Pepper/Plant Damage
NFR.2	Size
NFR.3	Cost
NFR.4	Safe
NFR.5	Speed of Operation

14 of 92

14

Functional and Non-functional Requirements

	Functional Requirements
FR.1	Distinguish Peppers from Plant
FR.2	Plan Path to Pepper
FR.3	Reach Pepper
FR.4	Grip Pepper
FR.5	Extract Pepper
FR.6	Place Pepper
FR.7	(Stretch) Navigate to New Plant

	Nonfunctional Requirements
NFR.1	Pepper/Plant Damage
NFR.2	Size
NFR.3	Cost
NFR.4	Safe
NFR.5	Speed of Operation

15 of 92

15

Functional and Non-functional Requirements

	Functional Requirements
FR.1	Distinguish Peppers from Plant
FR.2	Plan Path to Pepper
FR.3	Reach Pepper
FR.4	Grip Pepper
FR.5	Extract Pepper
FR.6	Place Pepper
FR.7	(Stretch) Navigate to New Plant

	Nonfunctional Requirements
NFR.1	Pepper/Plant Damage
NFR.2	Size
NFR.3	Cost
NFR.4	Safe
NFR.5	Speed of Operation

16 of 92

16

Functional and Non-functional Requirements

	Functional Requirements
FR.1	Distinguish Peppers from Plant
FR.2	Plan Path to Pepper
FR.3	Reach Pepper
FR.4	Grip Pepper
FR.5	Extract Pepper
FR.6	Place Pepper
FR.7	(Stretch) Navigate to New Plant

	Nonfunctional Requirements
NFR.1	Pepper/Plant Damage
NFR.2	Size
NFR.3	Cost
NFR.4	Safe
NFR.5	Speed of Operation

17 of 92

17

Functional and Non-functional Requirements

	Functional Requirements
FR.1	Distinguish Peppers from Plant
FR.2	Plan Path to Pepper
FR.3	Reach Pepper
FR.4	Grip Pepper
FR.5	Extract Pepper
FR.6	Place Pepper
FR.7	(Stretch) Navigate to New Plant

	Nonfunctional Requirements
NFR.1	Pepper/Plant Damage
NFR.2	Size
NFR.3	Cost
NFR.4	Safe
NFR.5	Speed of Operation

18 of 92

18

Functional and Non-functional Requirements

	Functional Requirements
FR.1	Distinguish Peppers from Plant
FR.2	Plan Path to Pepper
FR.3	Reach Pepper
FR.4	Grip Pepper
FR.5	Extract Pepper
FR.6	Place Pepper
FR.7	(Stretch) Navigate to New Plant

	Nonfunctional Requirements
NFR.1	Pepper/Plant Damage
NFR.2	Size
NFR.3	Cost
NFR.4	Safe
NFR.5	Speed of Operation

19 of 92

19

Functional and Non-functional Requirements

	Functional Requirements
FR.1	Distinguish Peppers from Plant
FR.2	Plan Path to Pepper
FR.3	Reach Pepper
FR.4	Grip Pepper
FR.5	Extract Pepper
FR.6	Place Pepper
FR.7	(Stretch) Navigate to New Plant

	Nonfunctional Requirements
NFR.1	Pepper/Plant Damage
NFR.2	Size
NFR.3	Cost
NFR.4	Safe
NFR.5	Speed of Operation

20 of 92

20

Functional and Non-functional Requirements

	Functional Requirements
FR.1	Distinguish Peppers from Plant
FR.2	Plan Path to Pepper
FR.3	Reach Pepper
FR.4	Grip Pepper
FR.5	Extract Pepper
FR.6	Place Pepper
FR.7	(Stretch) Navigate to New Plant

	Nonfunctional Requirements
NFR.1	Pepper/Plant Damage
NFR.2	Size
NFR.3	Cost
NFR.4	Safe
NFR.5	Speed of Operation

21 of 92

21

Performance Requirements

FR.1.A	Distinguish at least 75% of visible peppers from plant (count)

FR.1.B	Ensure calculated peduncle centroid does not deviate radially from the actual centroid by more than 3 cm. 1 cm

FR1.C	Ensure the determined cut location is within a threshold of 5 cm above the top of pepper.

FR.1	Distinguish Peppers from Plant

FR.2.A	Avoid 100% of generated plant stem constraints

FR.2	Plan Path to Pepper

FR.3.A	If enforceable, not deviate actual end effector position from planned trajectory more than 1 cm

FR.3	Reach Pepper

F1.A) Distinguish at least 75% of visible peppers from plant (count)

F1.B) Ensure detected peduncle centroid does not deviate radially from the actual peduncle centroid by more than two peduncle diameters

F1.C) Ensure detected peduncle centroid does not deviate longitudinally from the actual peduncle centroid by more than half the peduncle height

F2.A) Plan a trajectory that does not intersect with any generated constraints anywhere outside a radius of 15 cm around the pepper

F3.A) If enforceable, not deviate in actual end effector position from planned trajectory by more than 1cm

F4.A) Ensure 90% success in gripping and retrieving peppers without dropping them

F5.A) Cut the peduncle within a threshold of 5 cm above the top of pepper

F5.B) Have zero false positive actuated cuts

F5.C) Succeed in cutting the peduncle 100% of the time if cutter is properly placed

F6.A) Successfully move the end effector to within 15 cm above a 25 cm radius basket

NF1.A) Avoid visible damage to 90% of peppers (cuts / bruises) and the plant

NF2.A) Navigate within a row of crops 0.6 m wide

NF3.A) Augmentations to the provided hardware platform should not be in excess of $5000

NF4.A) No injuries or harm to our talented engineers or customers

SL1) Pick a pepper in under 5 minutes

22 of 92

22

Performance Requirements

FR.4.A	Ensure 90% success in gripping and retrieving peppers without dropping them 50%

FR.4	Grip Pepper

FR.5	Extract Pepper

FR.5.A	Have 0 false positive actuated cuts

FR.5.B	Succeed in cutting the peduncle 100% of the time if cutter is properly placed 90%

FR.6.A	Successfully move the end effector to within 15 cm above a 25 cm radius basket

FR.6	Place Pepper

F1.A) Distinguish at least 75% of visible peppers from plant (count)

F1.B) Ensure detected peduncle centroid does not deviate radially from the actual peduncle centroid by more than two peduncle diameters

F1.C) Ensure detected peduncle centroid does not deviate longitudinally from the actual peduncle centroid by more than half the peduncle height

F2.A) Plan a trajectory that does not intersect with any generated constraints anywhere outside a radius of 15 cm around the pepper

F3.A) If enforceable, not deviate in actual end effector position from planned trajectory by more than 1cm

F4.A) Ensure 90% success in gripping and retrieving peppers without dropping them

F5.A) Cut the peduncle within a threshold of 5 cm above the top of pepper

F5.B) Have zero false positive actuated cuts

F5.C) Succeed in cutting the peduncle 100% of the time if cutter is properly placed

F6.A) Successfully move the end effector to within 15 cm above a 25 cm radius basket

NF1.A) Avoid visible damage to 90% of peppers (cuts / bruises) and the plant

NF2.A) Navigate within a row of crops 0.6 m wide

NF3.A) Augmentations to the provided hardware platform should not be in excess of $5000

NF4.A) No injuries or harm to our talented engineers or customers

SL1) Pick a pepper in under 5 minutes

23 of 92

23

Performance Requirements

NFR.1.A	Avoid visible damage to 90% of picked peppers (cuts / bruises)

NFR.1	Pepper/Plant Damage

NFR.2.A	Navigate within a row of crops 0.6 m wide

NFR.2	Size

NFR.3.A	Augmentations to the provided hardware platform should not be in excess of $5000

NFR.3	Cost

NFR.4.A	0 injuries or harm to our talented engineers or customers

NFR.4	Safety

NFR.5.A	Pick any reasonable (pre-identified) pepper in under 5 minutes

NFR.5	Speed of Operation

F1.A) Distinguish at least 75% of visible peppers from plant (count)

F1.B) Ensure detected peduncle centroid does not deviate radially from the actual peduncle centroid by more than two peduncle diameters

F1.C) Ensure detected peduncle centroid does not deviate longitudinally from the actual peduncle centroid by more than half the peduncle height

F2.A) Plan a trajectory that does not intersect with any generated constraints anywhere outside a radius of 15 cm around the pepper

F3.A) If enforceable, not deviate in actual end effector position from planned trajectory by more than 1cm

F4.A) Ensure 90% success in gripping and retrieving peppers without dropping them

F5.A) Cut the peduncle within a threshold of 5 cm above the top of pepper

F5.B) Have zero false positive actuated cuts

F5.C) Succeed in cutting the peduncle 100% of the time if cutter is properly placed

F6.A) Successfully move the end effector to within 15 cm above a 25 cm radius basket

NF1.A) Avoid visible damage to 90% of peppers (cuts / bruises) and the plant

NF2.A) Navigate within a row of crops 0.6 m wide

NF3.A) Augmentations to the provided hardware platform should not be in excess of $5000

NF4.A) No injuries or harm to our talented engineers or customers

SL1) Pick a pepper in under 5 minutes

24 of 92

24

Functional Architecture

25 of 92

25

Cyberphysical Architecture

26 of 92

26

Detection

Semantic Segmentation

Instance Segmentation

27 of 92

27

Detection

28 of 92

28

Detection

29 of 92

29

Detection

30 of 92

30

Detection

31 of 92

31

Detection

Depth
Occlusion level
Peduncle orientation
Distance from arm

32 of 92

32

32

Path Planning

33 of 92

33

33

Path Planning

34 of 92

34

Manipulation

35 of 92

35

Manipulation

36 of 92

36

Manipulation

37 of 92

37

Manipulation

38 of 92

38

Manipulation

39 of 92

39

Manipulation

40 of 92

40

Manipulation

41 of 92

41

Manipulation

42 of 92

42

Manipulation

43 of 92

43

Manipulation

44 of 92

44

Navigation

45 of 92

45

45

Navigation

46 of 92

46

46

Power

47 of 92

47

47

Subsystem-Level Trade Study: Cutter/Gripper Design

"The Hook" (Single Motor Actuated)

"Scissor and Clamp" (Double Motor Actuated)

2DOF Complex Design

Integrated Soft Gripper & Cutter

48 of 92

48

		Value (1 (bad) - 5 (good))
Criteria	Weight (100%)	"The Hook" (Single Motor Actuated)	2DOF Complex Design	"Scissor and Clamp" (Double Motor Actuated)	Integrated Soft Gripper/Cutter
Complexity	35%	3	1	4	2
Grip Integrity	20.00%	2	4	4	2
Cut Integrity	20.00%	1	5	4	2
Safety to Plant	10.00%	2	5	2	2
Safety to Pepper	10.00%	4	5	5	4
Safety to People	5.00%	4	5	4	4
Score		2.45	3.4	3.9	2.45

Subsystem-Level Trade Study: Cutter/Gripper Design

49 of 92

49

Initial WBS: Layer 1, 2 and 3

50 of 92

50

Initial WBS: Layer 1, 2, 3, and 4 (some 5)

51 of 92

51

Initial WBS: End Effector

52 of 92

52

Initial WBS: End Effector

53 of 92

53

Initial WBS: End Effector

54 of 92

54

Initial WBS: End Effector

55 of 92

55

Initial WBS: End Effector

56 of 92

56

Initial WBS: End Effector

57 of 92

57

Initial WBS: End Effector

58 of 92

58

Initial WBS: End Effector

59 of 92

59

Initial WBS: Perception

60 of 92

60

Initial WBS: Perception

61 of 92

61

Initial WBS: Perception

62 of 92

62

Initial WBS: Perception

63 of 92

63

Initial WBS: Perception

64 of 92

64

Initial WBS: Perception

65 of 92

65

Initial WBS: Others

66 of 92

66

Initial WBS: Path Planning

67 of 92

67

Initial WBS: Path Planning

68 of 92

68

Initial WBS: System Integration

69 of 92

69

Initial WBS: System Integration

70 of 92

70

Initial WBS: System Integration

71 of 92

71

Initial WBS: Power Distribution

72 of 92

72

Initial WBS: Project Management

73 of 92

73

Initial WBS: Navigation

74 of 92

74

Initial WBS dictionaries

75 of 92

75

Initial Schedule

Perception

End Effector

Integration

Motion Planning

DO NOT TRY TO READ!! :)

76 of 92

76

Initial Schedule - Perception

77 of 92

77

Initial Schedule - End Effector

78 of 92

78

Initial Schedule - Motion Planning

79 of 92

Description:

A team member has a course deadline, interview, or get sicks – interfering with the project deadline.

Consequence:

Inability to properly allocate work resources or adhere to the schedule, causing later development conflicts.

79

Risk: Life Conflicts

R1

After Mitigation

Before Mitigation

80 of 92

Risk Mitigation Plan:

Team Standups
Bi-weekly technical demonstrations
Robust documentation system

80

R1

R1

After Mitigation

Before Mitigation

Risk: Life Conflicts

81 of 92

Description:

Questionable access to a hardware platform, as well as the potential for a change, has not been clarified.

Consequence:

Inability to prototype & perform SVD/FVD. Also, affects design infrastructure.

81

Risk: Mobility Platform Access

After Mitigation

Before Mitigation

R3

82 of 92

Risk Mitigation Plan:

Make navigation a stretch goal
Obtain clarification from Kantor concerning access to base

82

R3

R3

After Mitigation

Before Mitigation

Risk: Mobility Platform Access

83 of 92

83

Risk: Manipulator Access

Description:

Inability to access manipulator

Consequence:

Inability to prototype & perform SVD/FVD. Also, affects design infrastructure.

R7

After Mitigation

Before Mitigation

84 of 92

Risk Mitigation Plan:

Reach out to sponsor & Kantor on a weekly basis until resolved
Find additional backup options (Reach out to Prof. Dimi, Prof. Kroemer, etc.)

84

R7

R7

After Mitigation

Before Mitigation

Risk: Manipulator Access

85 of 92

85

Risk: Peduncle Detection Issues

Description:

Peduncle detection algorithm does not work.

Consequence:

Peduncle detection algorithm can’t distinguish a peduncle from stem, therefore we cannot plan path to the peduncle to cut it.

R9

After Mitigation

Before Mitigation

86 of 92

Risk Mitigation Plan:

Try alternative algorithms identified
Allow user in the loop so that the other subsystems are not affected

86

R9

R9

After Mitigation

Before Mitigation

Risk: Peduncle Detection Issues

87 of 92

87

Risk: End Effector Design

Description:

Difficulties in End Effector Design

Consequence:

End-Effector Takes a Long time to Design, ultimately cannot harvest peppers

R6

After Mitigation

Before Mitigation

88 of 92

Risk Mitigation Plan:

Make design process iterative
Design over winter break

88

R6

R6

After Mitigation

Before Mitigation

Risk: End Effector Design

89 of 92

89

After Mitigation

Risk Number	Risk Name
R1	Missed Internal Milestones due to Life Conflicts
R3	Lack of Clarity of Mobile Platform
R6	Difficulties in End Effector Design
R7	Access to Manipulator
R9	Peduncle detection algorithm does not work

Before Mitigation

R3

R1

R1

R3

R6

R9

R7

R7

R9

R6

Risk Visualization

90 of 92

90

After Mitigation

Before Mitigation

R2

R1

R1

R2

R3

R3

R4

R4

R6

Risk Visualization

R5

R6

91 of 92

91

Thank you!

92 of 92

92

Risk: Mobility Platform Access

Fig. X Previous design architecture with Husky Base

Fig. X Potential Amiga platform