1 of 10

Utilizing computer vision and AI to support pilot fatigue management

Ben Thompson

School of Optometry and Vision Science, University of Waterloo

Centre for Eye and Vision Research, Hong Kong

2 of 10

Background – from Optometry to Aviation

  • Amblyopia

3 of 10

Background – from Optometry to Aviation

Black et al. The Measurement and Treatment of Suppression in Amblyopia. J. Vis. Exp. (70), e3927

4 of 10

Background – from Optometry to Aviation

  • The BRAVO Study
  • Randomized, placebo-controlled clinical trial of at-home digital therapy
  • Five study sites in NZ, Australia, Canada, Hong Kong.
  • 115 patients

​

5 of 10

Background – from Optometry to Aviation

  • The BRAVO Study
  • No effect!
  • Adherence data
    • Short intermittent game play
    • Paused on average every 4 minutes

​

6 of 10

Background – from Optometry to Aviation

  • Development of a monitoring and feedback system

7 of 10

Background – from Optometry to Aviation

  • Six features utilized: gaze direction, yawning, blinking, head pose, emotional state and distance from screen
  • Detection of engagement loss with 97% sensitivity and 93.3% specificity

​

8 of 10

Translation to the cockpit

  • Real time pilot feedback and/or analytics to assess fatigue management systems

9 of 10

Translation to the cockpit

  • Next steps
    • Ongoing proof of concept testing
    • Update system to deal with multiple pilots
    • Consult with potential end-users on feedback modalities
    • Input from pilot groups on privacy and liability issues
    • Infrared system for night flights
  • We are actively seeking partnerships to help shape the technology for optimal use in aviation settings

​

​

10 of 10

Project team

  • Dr. Anoop Thazhathumanackal Radhakrishnan, School of Optometry and Vision Science, University of Waterloo
  • Dr. Amritha Stalin, School of Optometry and Vision Science, University of Waterloo
  • Prof. Otman Basir, Computer and Electrical Engineering, University of Waterloo
  • Prof. Robert Hess, Ophthalmology, McGill University

​

  • Funding: CIHR, WISA, WatCo

​