1 of 24

Visual Tracking and Navigation for UAV Landings on Moving Vessel Decks

Liyang Wang

Mechanical and Aerospace Engineering

Rutgers University

08/08/2019

2 of 24

Project development process

  • Advisor propose project demand
  • Define and narrow down problem, explore potential technologies
  • Find a solution, modify relevant technology when necessary
  • Build up hardware, program to realize algorithms
  • Test, troubleshooting, and improve
  • Publish results

3 of 24

Visual Tracking and Navigation for UAV Landings on Moving Vessel Decks

  • Project demand
    • Autonomously navigate a helicopter to approach and landing on a moving vessel deck from long range
  • Constraints
    • On-board sensors only
    • No GPS signal

4 of 24

Narrow down tasks

Two stages

    • Approaching stage from long range
    • Landing stage when hover above

Challenges

    • Landing target recognition
    • UAV state estimation
    • Navigation
    • UAV controls

Technical solutions

    • Vision algorithms
    • Sensor fusion, coordinates transformations
    • Path planning
    • Controllers design

5 of 24

UAV platform —�Parrot AR.Drone�

  • On-board sensors
    • Inertial measurement unit (IMU)
    • Front camera and bottom camera
    • Ultrasonic altitude sensor
  • 4 control channels
    • Roll angle
    • Pitch angle
    • Angular velocity of yaw
    • Velocity of the altitude direction

6 of 24

Vessel deck movement emulator

  • A platform which can emulate vessel deck movement with sea waves
    • Commercial 6 DOF motion platform?

7 of 24

Self-designed landing deck

Servo distribution

Landing platform base

Side view of the platform

8 of 24

Self-designed landing deck

  • Maximum 10° slop
  • Minimum 5.2m sea wave height for a vessel with 30 meters wide
  • Equivalent to very rough sea state

9 of 24

Approaching stage from long range

  • Landing target tracking

Red LEDs with H landing sign

Red LEDs recognition

10 of 24

UAV front camera layout modification

  • Rotate 90° clockwise and looking down for 75°
    • Before:
      • 60° horizontal FOV
      • 30° vertical FOV
      • Looking forward
    • After:
      • 30° horizontal FOV
      • 60° vertical FOV
      • Ray of central view has 75° downward

Front camera layout after modification

11 of 24

�Approaching stage from long range

​

  • Approaching strategy
    • Move forward when heading to the target
    • Move forward pulse
      • Pitch angle equal to -10° for 0.1s
    • Roll angle maintain 0°
    • Altitude adjust to make sure landing target in the FOV of the front camera

12 of 24

Approaching stage from long range

  •  

Image frame after modification

13 of 24

Approaching stage from long range

  • Horizontal Distance Estimation

​

​

​

​

​

​

​

​

​

Camera configuration

14 of 24

Landing stage when hover above

​

  • H landing sign detection

State machine of the image processing

Contour of a standard H

Original color image

H landing sign detected

15 of 24

Landing stage when hover above

  • State estimation
    • Frame definitions and transformations
    • Calculate heading angle
    • Calculate the landing target position respect to frame L

Image frame

Body frame

Inertial frame and Local reference frame

16 of 24

Landing stage when hover above

  • State estimation
    • Frame transformations

Pin-hole camera model

17 of 24

Landing stage when hover above

Framework of the autonomous landing

18 of 24

Experimental Results

  • Approaching From Long Range

Heading angle estimation and control results

Trajectory of the approaching

19 of 24

Experimental Results

  • Landing stage when hover above

State estimation and control results

Landing altitude control

20 of 24

Experimental Results

  • Landing accuracy
    • 20 independent experiments
    • Average distance error: 9.0 cm
    • Standard distance error deviation: 4.79 cm

Landing accuracy

21 of 24

Experimental Results

  • Videos

View from back

22 of 24

Experimental Results

  • Videos

Camera view

23 of 24

Summary

Strong Passion of Innovation

    • Curious to new trends

Problem Solver

    • Ability to analyze and solve problems
    • Team works

Not Limited to Theories

    • Apply technologies to products

24 of 24

Thank you!