​

​

Date: June 17th, 2020

G-13 UG Project Presentation Examination - 2020

ECE Department, SVNIT

1

2

AUTONOMOUS AGRICULTURE ROBOT

“AGRIBOT”

Presented by: -

Mr. Dhruv Patel (U16EC053)

Mr. Meet Gandhi (U16EC056)

Mr. Shankaranarayanan H (U16EC074)

Guided by: -

Dr. Anand kumar D. Darji.

Associate Professor & HoD,

ECED, SVNIT, Surat-395007.

3

1. Problem Statement Explanation
2. Our Approach and Timeline
3. Modelling & Simulation
4. Algorithms Implementation
5. Results & Conclusion
6. Demo
7. Q&A.

OUTLINE OF THE PRESENTATION

1. PROBLEM STATEMENT EXPLANATION

1. Present scenario in the field of Agriculture.
2. Recent advances in the field of Precision farming.
3. What is AGRIBOT ?

4

5

2020

Aug

Sep

Oct

Nov

Dec

Jan

Feb

Mar

Apr

2019

2020

May

Defining the Problem Statement for AGRIBOT

Oct 3

UNet Model for Crop-Weed Classification + Designing the Mechanical Model using Solidworks

Nov 16

Bonnet Model for Crop-Weed Classification

Feb 3

Prediction on images from Surrounding Farm.

May 17

Surveying the Problems faced in the Agricultural Field.

Aug 15

Studying the recent advancements in the field of Precision Agriculture

Sep 15

Planning on the basis of Problem Statement and Finalising the components required for AGRIBOT

Nov 4

Implementation of different Algorithms for Crop-Weed Classification on various Datasets + Designing the system architecture in ROS

Nov 16

Manufacturing of the Mechanical Model

Dec 13

Modelling and Simulation of AGRIBOT using GAZEBO

Apr 25

2. Our Approach & Timeline

3. Modelling & Simulation

1. Physical model Description
2. World Environment model Description
3. Sensor & hardware Modelling

6

Modelling of AGRIBOT(Robotic Vehicle)

​

​

​

​

​

​

​

​

​

​

​

​

​

Rendered Image of model in Solidworks

​

​

​

​

​

​

​

​

​

​

​

​

Solidworks-2016 Design

7

Continue...

8

Continue slide no. 7

TF Frame/ Coordinate Frame Transfer

Kinetic Frame(Forward Kinematics & Inverse Kinematics)

​

​

​

​

​

​

​

​

​

​

Kinematics model of Skid-Steering System. Courtesy: Wang T, Wu Y, Liang J, Han C, Chen J, Zhao Q. “Analysis and experimental kinematics of a skid-steering wheeled robot based on a laser scanner sensor” in Sensors (Basel). 2015;15(5):9681‐9702. Published 2015 Apr 24. doi:10.3390/s150509681

​

​

​

​

9

Modelling of Environment

• Physical properties like gravity, luminous, wind flow, friction between soil & wheels has been added through GAZEBO API during initialization.

​

​

​

​

​

​

10

Sensor Modelling

Y = Y^’+ B’ + N_Y

B’ = − B/ τ + N_B

​

​

Y is simulated sensor value

Y’ is raw measured value

B is bias/offset Value for sensor

N_Y is additive noise

N_B is characteristics of random drift with time

11

​

​

Ref:Mostafa Sharifi, XiaoQi Chen, Christopher Pretty, Don Clucas, Erwan Carbon-Lunel presented “Modelling and simulation of a non-holonomic omnidirectional mobile robot for offline programming and system performance analysis” in Simulation Modelling Practice and Theory 87 (2018) 155-169 by ELSEVIER

Variation in Sensor Reading at initial stationary position

12

Magnetometer Raw Reading

GPS Reading

Simulation of Camera

Technical specifications:

​

-640 * 480 frame

-Gaussian Noise

-RGB channel

​

​

​

​

13

Control Plugin(Gazebo-ROS-Control)

14

4. Algorithms Implementation

• Filtering and Sensor Fusion
• Control and Navigation Algorithm
• Crop Weed Classification Models

​

15

Filtering Magnetometer data

1. Moving median

2. Kalman Filter

​

16

1

2

Calculation of parameters- Required angle(θ) and distance(d) to Destination

​

​

​

​

​

​

​

​

​

​

​

Calculation of the required angle and distance to destination.

θ = atan2 (Y'₁-Y', X'₁-X') ± ø

d = ((X′₁−𝑋′)²+(𝑌′₁−𝑌′)²)^1/2

17

Control & Navigation Algorithm

​

​

​

​

​

​

​

​

​

​

​

​

Pictorial View of Path Planning Algorithm.

​

​

​

18

19

SEMANTIC SEGMENTATION MODELS

​

• Unet
• Bonnet

Crop Weed Classification

UNet Architecture

20

Ref: Olaf Ronneberger, Phillip Fischer, Thomas Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation”, Medical Image Computing and Computer-Assisted Intervention (MICCAI), Springer, LNCS, Vol.9351: 234--241, 2015.

Bonnet Architecture

21

Ref: Andres Milioto, Philipp Lottes, Cyrill Stachniss. Real-time Semantic Segmentation of Crop and Weed for Precision Agriculture Robots Leveraging Background Knowledge in CNNs, IEEE International Conference on Robotics and Automation 2018.

22

UNET VS BONNET COMPARISON

Bonnet Model is chosen as it generalizes well and has comparatively less parameters.

Bonnet Ref: Andres Milioto, Philipp Lottes, Cyrill Stachniss. Real-time Semantic Segmentation of Crop and Weed for Precision Agriculture Robots Leveraging Background Knowledge in CNNs, IEEE International Conference on Robotics and Automation 2018.

​

​

Performance Comparisons on CWFID Dataset

23

Bonnet’s prediction

​

UNet’s prediction

Metrics Comparison on CWFID Dataset

• Both models were trained and tested on smaller dataset CWFID and their performance were compared. Based on these, final model was selected for further training on larger datasets.

​

• Performance of Bonnet and UNet obtained were similar. Bonnet was selected over Unet due to its better class-wise prediction and 100x lesser parameters than UNet which makes its deployment possible for real-world applications.

• Model selected: Bonnet

​

24

Bonnet’s Performance using different Loss Functions

1. Categorical cross-entropy:

​

Here, y is the ground truth & ŷ is the model prediction. It is summed over all pixels in the image and averaged over the entire batch.

​

​

25

• Model fails to predict pixels with minor class correctly.

​

• Hence, does not address issue of class imbalance.

2.) Dice-coefficient Loss:

​

Here, p is ground truth and ƿ is the prediction. Where p ε {0,1} and 0 ≤ ƿ ≤ 1.

​

• It is a measure of overlap and is a popular loss function in image segmentation tasks.

​

​

​

​

​

​

​

​

​

​

​

​

​

​

• Due to some unknown reason, model fails to predict classes.

​

​

​

​

​

​

26

3.) Weighted Cross-entropy Loss (WCCE):

• Here, the losses from corresponding classes are weighed in inverse proportion to its frequency of occurrence and summed up across all classes.

​

• class_weights = [0.90,0.11,0.1] in order of Weed,Crop and Soil.(i.e R,G and B).

​

• It counters class imbalance which was caused in case of categorical cross-entropy.

​

​

​

​

​

​

​

​

​

• Model trained using WCCE loss performs better than the other losses and its prediction is quite similar to ground truth.

​

• Loss Function used: WCCE

27

5. Results & Conclusion

1. Teleoperation
2. Autonomous Monitoring
3. Bonnet model Performance

​

​

28

1. Teleoperation

29

2. Autonomous Monitoring

30

Testing of Trajectory planner Algorithm(For random points)

Different Points at end of raw in Field

3. Predictions of Classification Model

​

Bonnet’s prediction on CWFID Dataset.

​

​

​

​

​

​

​

Bonnet’s prediction on Bonn Dataset.

31

Metrics

32

• Real-time Latency: Avg 2.5 fps on i7 + NVIDIA 940MX

Prediction on images of Surrounding Farm

33

Continued in the next slide.

34

Conclusion

​

​

• We have modelled and simulated the AGRIBOT and also developed the simulated environment for testing the same. We tested the performance of AGRIBOT with teleoperation and with the algorithms for autonomous navigation. Using Differential GPS and advanced sensor fusion techniques, error can be further reduced to make trajectory more smoother.

​

• Bonnet model: Segments the vegetation mask and has learned to classify less greeny and dry parts of vegetation as weeds. Further improvement can be done by providing data from real-case scenarios/applications where the deployment is to be done.

​

• Real-time latency: avg. 2.5 fps on i7 + NVIDIA 940MX.
• Mean accuracy: 99.47%, Mean iou: 98.03%, Loss: 0.00348 units on Bonn dataset.

​

35

DEMO

36

Q&A

37

THANK YOU.

38