1 of 16

Development of an Algorithm for Autonomous landing of a UAV on a Moving Platform

Lokesh Bansal

MTech. Robotics and Autonomous Systems

Robert Bosch Center for Cyber Physical Systems

Indian Institute of Science, Bangalore

2 of 16

Overview of Objective

3 of 16

Objective- Intelligent Detection of Landing site and Autonomous Navigation and Landing of a Drone on a Moving Platform

Feature Extraction-

  1. Hough Transform
  2. Canny Edge Detection
  3. Kalman and Particle Filter

Trajectory Generation and Landing- Reinforcement Learning and PID Controller

Intelligent Detection and Tracking-

  1. OpenCV
  2. TensorFlow API

4 of 16

Preliminary Image Processing for Land-filed Detection and Tracking

​

  • 1-

​

​

​

  • 2-

​

​

​

​

BW = edge(rgb, 'canny',0.2);

[H, T, R]= hough(BW, 'theta', -50:50);

P = houghpeaks(H,5,'threshold',ceil(0.7*max(H(:))));

x = T(P(:,2)); y = R(P(:,1));

plot(x,y,'s','color','white');

lines = houghlines(BW,T,R,P,'FillGap',600,'MinLength',10);

 

​

BW = edge(rgb, 'canny',0.2);

[H, T, R]= hough(BW, 'theta', -50:50);

P = houghpeaks(H,3,'threshold',ceil(0.4*max(H(:))));

x = T(P(:,2)); y = R(P(:,1));

plot(x,y,'s','color','white');

lines = houghlines(BW,T,R,P,'FillGap',10,'MinLength',80);

 

  • Canny Edge detection for lines with Hough Transform is used to detect the lines
  • The Black and yellow end points of the lines can be used for feature extraction
  • Tuning is required for different images
  • Tuning parameters are ‘canny’, ‘theta’, houghpeaks, ‘FillGap’, ‘MinLength’

5 of 16

Preliminary Image Processing for Horizon Detection

​

  • 1-

​

​

  • 2 -

Horizon Detection view

Specifications in Matlab Code

​

R = imresize(A, [115 300]);

I = rgb2gray(R);

BW = edge(I, 'canny', 0.1);

%hough transform to find the lines

[H, T, R] = hough(BW);

P = houghpeaks(H, 10, 'threshold', ceil(0.75*max(H(:))));

​

R = imresize(A, [115 300]);

I = rgb2gray(R);

BW = edge(I, 'canny', 0.145);

%hough transform to find the lines

[H, T, R] = hough(BW);

P = houghpeaks(H, 10, 'threshold', ceil(0.75*max(H(:))));

 

  • Canny Edge detection for lines with Hough Transform is used to detect the Horizon
  • Using the detected Horizon line (blue line), attitude parameters can be determined
  • Tuning is required for different images
  • Tuning parameters are imresize(), ‘canny’, ‘theta’, houghpeaks(), ‘FillGap’, ‘MinLength’

6 of 16

Preliminary Image Processing for Horizon Detection

​

  • 4-

​

​

  • 5-

​

​

  • 6 -

​

R = imresize(A, [115 300]);

I = rgb2gray(R);

BW = edge(I, 'canny', 0.145);

%hough transform to find the lines

[H, T, R] = hough(BW);

P = houghpeaks(H, 10, 'threshold', ceil(0.75*max(H(:))));

 

​

R = imresize(A, [115 300]);

I = rgb2gray(R);

BW = edge(I, 'canny', 0.145);

%hough transform to find the lines

[H, T, R] = hough(BW);

P = houghpeaks(H, 10, 'threshold', ceil(0.75*max(H(:))));

 

​

R = imresize(A, [115 300]);

I = rgb2gray(R);

BW = edge(I, 'canny', 0.3);

%hough transform to find the lines

[H, T, R] = hough(BW);

P = houghpeaks(H, 10, 'threshold', ceil(0.75*max(H(:))));

 

7 of 16

Detection of Landing Field and Horizon

  • Target - Detect & Track the Horizon and runway track field lines of the landing platform
  • The blue line - Horizon & the Green lines - track field lines
  • Using the Horizon detection line and the track field lines all the features like attitude, velocity can be extracted

8 of 16

Preliminary Image Processing for Land-filed detection and tracking

​

  • 1-

​

​

​

  • 2-

​

​

​

​

I = rgb2gray(A);

I=imcrop(I,[300 325 700 500]);

BW = edge(I,'canny',0.1);

 

[centers, radii] = imfindcircles(I,[50 65],'ObjectPolarity','bright','Sensitivity',0.98,'EdgeThreshold',0.1);

​

​

I = rgb2gray(A);

I=imcrop(I,[400 225 750 400]);

BW = edge(I,'canny',0.1);

 

[centers, radii] = imfindcircles(I,[50 65],'ObjectPolarity','bright','Sensitivity',0.98,'EdgeThreshold',0.12);

  • Canny Edge detection with Hough Transform with a function ‘imfindcircles’ is used to detect the landing point
  • Using this technique, the red circles are obtained which focuses about the landing platform
  • Tuning is required for different images
  • Tuning parameters are imcrop(), ‘canny’, imfindcircles()

9 of 16

Preliminary Image Processing for Land-filed detection and tracking

​

  • 3-

​

​

  • 4-

​

​

  • 5-

Detection View

Specifications of Matlab Code

​

​

I = rgb2gray(A);

I=imcrop(I,[430 275 435 280]);

BW = edge(I,'canny',0.1);

 

[centers, radii] = imfindcircles(I,[100 125],'ObjectPolarity','bright','Sensitivity',0.98,'EdgeThreshold',0.1);

​

​

I = rgb2gray(A);

I=imcrop(I,[300 150 700 500]);

BW = edge(I,'canny',0.1);

 

[centers, radii] = imfindcircles(I,[50 60],'ObjectPolarity','bright','Sensitivity',0.98,'EdgeThreshold',0.1);

​

​

I = rgb2gray(A);

I=imcrop(I,[300 150 700 500]);

BW = edge(I,'canny',0.1);

 

 

[centers, radii] = imfindcircles(I,[100 130],'ObjectPolarity','bright','Sensitivity',0.98,'EdgeThreshold',0.1);

10 of 16

Circular Ellipsoid Detection and Tracking

  • To enhance the accuracy of detection, Ellipsoid detection can be used
  • Using this technique, the yellow and navy blue ellipses are obtained which focuses about the landing platform
  • The accuracy of ellipsoid detection is quite higher than the circular line detection

11 of 16

Detection of Heli-Deck on a Ship

  • Target- Detect and Track the center point of the Landing Platform
  • Hough transform + Canny Circular Edge Detection, used to draw the two red circles
  • The centers of the two circles are joined (blue line)
  • The center point of the line segment is the desired center point of the landing platform
  • The yellow circle is drawn taking the center of the landing platform as circle’s center

12 of 16

Preliminary Video processing for Landfiled detection and tracking

Frame-1

Frame- n

Frame- 167

Frame- l

Frame-k

Frame- p

  • n, p, k, l lies in (1,167) & n < p < k < l
  • Total frames = 167 for 5 seconds video
  • The red circles focuses on the landing platform
  • Through the red circles, the landing platform can be detected and tracked

The frames are from video of moving landing platform

13 of 16

Preliminary Intelligent Object detection using Tensorflow

  • The ship is successfully detected and recognized.

​

  • Using Tensor Flow in Python- algorithm is developed for object detection by image processing.

​

  • Further the algorithm can be implemented with openCV library for real-time video detection.

​

Fig. – Ship detection in marine environment

14 of 16

  • As demonstrated in the video the Reinforcement Learning Algorithm is getting trained in the simulated Environment setup by Unreal Engine and Airsim

15 of 16

Future Works

  • Implement the trained algorithm on a real drone.
  • Ensure that the animated simulated environment training will work in the real world.
  • Modify the Reinforcement Learning code to make autonomous landing (as of now the code is for autonomous navigation).
  • Formulate more sophisticated research problem and implement the developed algorithms and system on a real drone for a real world problem.

16 of 16

References

  • Elisa. Instrumental Landing System. [http://www.electroluminescent-provider.com/elisa-automatic-landing-system/] Accessed July 15, 2019.
  • Kurt. 2019. Object Detection Tutorial in TensorFlow: Real-Time Object Detection. [https://www.edureka.co/blog/tensorflow-object-detection-tutorial/] Accessed September 05, 2019.
  • Raksha Rao K. “Autonomous landing of a quadrotor on stationary and moving platforms using only vision from an on-board monocular camera”. Thesis. Indian Institute of Technology, Madras, India. 2018.  
  • Farid Kendoul. “Survey of Advances in Guidance, Navigation, and Control of Unmanned Rotorcraft Systems”. Australian Research Centre for Aerospace Automation (ARCAA), Australia and CSIRO ICT Centre, Australia. 2011
  • Satish Kumar V., Sudesh K. Kashyap*, and N. Shantha Kumar. “Detection of Runway and Obstacles using Electro-optical and Infrared Sensors before Landing”. National Aerospace Laboratory, Bangalore, India. 2014.
  • Sebastian Scherer ∗, Lyle Chamberlain, Sanjiv Singh, “Autonomous landing at unprepared sites by a full-scale helicopter”. Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA. 2012.
  • Damien Dusha *, Wageeh Boles, Rodney Walker, “Fixed-Wing Attitude Estimation Using Computer Vision Based Horizon Detection”. Australian Research Centre for Aerospace Automation (ARCAA), Queensland University of Technology (QUT), Brisbane, Australia. 2007.
  • Davide Falanga, Alessio Zanchettin, Alessandro Simovic, Jeffrey Delmerico, and Davide Scaramuzza, “Vision-based Autonomous Quadrotor Landing on a Moving Platform”. 2017. University of Zurich, Switzerland.