1 of 20

Women in Data Hackathon - Team 4�Damaged Aircraft Fuselage Dimensioning (Maintenance)

Nov 14 & 15, 2022

2 of 20

Agenda

2

SECTION

PAGE

1. Intro

3

2. Problem Statement

4

3. Current State

5

4. Goal & Scope

6

5. Solution

7

6. Demo

9

7. Future State

11

8. Questions?

14

3 of 20

  1. Intro: Team 4 – ACT App

3

Emma Rousseau

Zohreh Hajabedi

Pat Duong

Laila Nouasri

Kristen Chen

Megan Chang

4 of 20

2. Problem Statement

CURRENT STATE

Technicians manually measure damage to aircraft exterior surfaces using rulers and an optical micrometer/digital microscopes/laser displacement sensor and measure to the closest ⅛”

WHY IS THIS IMPORTANT

The size of the damage determines the amount of work required for the repair, but also if the aircraft is good to continue flying until a permanent repair can be made.

  • How to use Augmented Reality to help optimize the review and measurement of damaged aircraft exteriors.

Problem Statement

5 of 20

3. Current State: What is the current process?

5

6 of 20

4. Goal & Scope

6

App to automatically identify and capture the dimensions of a damaged surface area of an aircraft.

Non-invasive measurement process

Increased accuracy (0.00003”)

REQUEST

Shorter time for damage assessment

7 of 20

5. Solution (Methodology)

7

ARKit for object detection and overlay

Swift UI for User Interface (iOS app)

XCode as our Integrated Development Environment

8 of 20

5. Solution Continued… (Timeline)

8

APP PROTOTYPE (PROOF OF CONCEPT)

​

Develop distance measuring AR app

1

EXPAND PROTOTYPE

​

Add depth measurement, and increase accuracy

2

USER ACCEPTANCE TESTING

​

Get potential end users to test the app

3

INTEGRATE WITH MAINTENANCE SYSTEMS

​

Saving tagged information to a database

4

FUTURE TECHNOLOGIES

​

Machine Learning for object detection or other technologies

5

Future State

9 of 20

6. Demo (Landing Screen)

9

10 of 20

6. Demo Continued… (Measurement)

10

11 of 20

7. Future State - Ideas For Future Related Topics

An AI-based decision support system to decide on flying after damage is detected�According to SMEs, determining whether an aircraft can fly despite of the damage or if it should be left on the ground is one of their most challenging processes. Depending on availability of historical data, a decision support AI-based system can be designed to assist the experts in establishing the best course of action for the damaged aircrafts.

​

11

fly

no fly

CASE SPECIFICATION

​

Damage image:

damage length: x inch

damage width: y inch

damage depth: z inch

damage severity level: L1

​

DECISION SUPPORT SYSTEM

12 of 20

7. Future State -Ideas For Future Related Topics

MediaPipe Objectron

  • Framework for building machine learning pipelines for video, image and audio processing
  • Published by Google
  • Cross-platform framework that works in Desktop/Server, Android, iOS
  • Consists of set of real-time 3D object detection models designed for mobile devices
  • It can predict objects’ 3D bounding boxes

​

​

​

​

12

Leveraging Machine Learning based algorithms and libraries for Object Detection

Objectron - V7 Open Datasets (v7labs.com)

https://google.github.io/mediapipe/solutions/objectron

https://www.v7labs.com/open-datasets/objectron

  • The Objectron dataset is a collection of short, object-centric video clips, which are accompanied by AR session metadata
  • The data also contain manually annotated 3D bounding boxes for each object, which describe the object’s position, orientation, and dimensions

13 of 20

Thank You

  • SME - Mingfang Wu
  • SME - Annie Bellmare
  • Organizer - Ankei Yau
  • Organizer - Raphaelle De Gagne
  • Organizer - Keith Dugas
  • Apple Developer - François Tiffreau

13

14 of 20

Merci�Thank you

aircanada.com

15 of 20

�Appendix A: Code Snapshots

aircanada.com

16 of 20

A. Code

16

17 of 20

17

18 of 20

18

19 of 20

19

20 of 20

20