A CONTINUAL LEARNING-BASED CNN FRAMEWORK FOR STRUCTURAL DAMAGE RECOGNITION

Jiawei Zhang, Jiangpeng Shu, Reachsak Ly, Yiran Ji

College of Civil Engineering and Architecture, Zhejiang University, Hangzhou, China

​

10th INTERNATIONAL CONFERENCE ON STRUCTURAL

HEALTH MONITORING OF INTELLIGENT INFRASTRUCTURE

30 JUNE › 2 JULY 2021

FEUP · PORTO · PORTUGAL

Content

Introduction

Methodology

Dataset

Network Architecture

Experiment and Result

1

2

3

4

5

10th INTERNATIONAL CONFERENCE ON STRUCTURAL

HEALTH MONITORING OF INTELLIGENT INFRASTRUCTURE

30 JUNE › 2 JULY 2021

FEUP · PORTO · PORTUGAL

Introduction

10th INTERNATIONAL CONFERENCE ON STRUCTURAL

HEALTH MONITORING OF INTELLIGENT INFRASTRUCTURE

30 JUNE › 2 JULY 2021

FEUP · PORTO · PORTUGAL

—— The increasing in tasks of structural health inspection (identifying the type of component, the type of material, the degree of damage, the existence of spalling, the cause of cracks, etc.) results urgent need for a efficient model to realize multi-task recognition.

——The manual detection method is highly dependent on the subjective or empirical knowledge of the inspector. Moreover, due to high equipment demand, low efficiency, high omission rate and poor real-time performance, it is difficult to be widely applied.

—— Utilize Convolutional Neural Network (CNN) instead of manual inspection provide higher accuracy, higher efficiency at lower cost, which is gradually becoming mainstream.

—— The ever-expanding data set has driven the research and validation of new technologies, methods, and theoretical models, providing information and evidence for the formulation and optimization of inspection, maintenance, repair, and reinforcement plans for bridge structures.

• Introduction: Background

10th INTERNATIONAL CONFERENCE ON STRUCTURAL

HEALTH MONITORING OF INTELLIGENT INFRASTRUCTURE

30 JUNE › 2 JULY 2021

FEUP · PORTO · PORTUGAL

• Introduction: Current Methods

Current training methods for multi-tasking learning:

• Feature Extraction:

The training speed is fast, but the accuracy is limited.

2.Fine-Tuning:

The training speed is relatively slow and catastrophic forgetting can occur.

3.Duplicate and Fine-Tuning:

The overall accuracy is high, but the prediction speed is slow due to the large number of parameters.

4.Joint Training:

High recognition accuracy, but it requires a large storage space for datasets and is very inefficient when adding new tasks.

​

​

10th INTERNATIONAL CONFERENCE ON STRUCTURAL

HEALTH MONITORING OF INTELLIGENT INFRASTRUCTURE

30 JUNE › 2 JULY 2021

FEUP · PORTO · PORTUGAL

Methodology

10th INTERNATIONAL CONFERENCE ON STRUCTURAL

HEALTH MONITORING OF INTELLIGENT INFRASTRUCTURE

30 JUNE › 2 JULY 2021

FEUP · PORTO · PORTUGAL

• Methodology

10th INTERNATIONAL CONFERENCE ON STRUCTURAL

HEALTH MONITORING OF INTELLIGENT INFRASTRUCTURE

30 JUNE › 2 JULY 2021

FEUP · PORTO · PORTUGAL

Dataset

10th INTERNATIONAL CONFERENCE ON STRUCTURAL

HEALTH MONITORING OF INTELLIGENT INFRASTRUCTURE

30 JUNE › 2 JULY 2021

FEUP · PORTO · PORTUGAL

Hierarchy tree of used dataset for CLDRM

• Dataset

10th INTERNATIONAL CONFERENCE ON STRUCTURAL

HEALTH MONITORING OF INTELLIGENT INFRASTRUCTURE

30 JUNE › 2 JULY 2021

FEUP · PORTO · PORTUGAL

The Number & Ratio of images used in different recognition task in this study

• Dataset

10th INTERNATIONAL CONFERENCE ON STRUCTURAL

HEALTH MONITORING OF INTELLIGENT INFRASTRUCTURE

30 JUNE › 2 JULY 2021

FEUP · PORTO · PORTUGAL

Sample images used for damage level evaluation task: (a) No Damage; (b) Minor Damage; (c) Heavy Damage

Sample images used for spalling condition check: (a) Spalling; (b) No Spalling

• Dataset

10th INTERNATIONAL CONFERENCE ON STRUCTURAL

HEALTH MONITORING OF INTELLIGENT INFRASTRUCTURE

30 JUNE › 2 JULY 2021

FEUP · PORTO · PORTUGAL

Sample images used in component type determination: (a) Column; (b) Wall; (c) Beam

Sample images used in damage type determination:

1. Shear crack (b) Flexural crack (c) Cracks caused by Alkali-Silica Reaction

(d) Cracks caused by corrosion of reinforcing steel.

​

• Dataset

10th INTERNATIONAL CONFERENCE ON STRUCTURAL

HEALTH MONITORING OF INTELLIGENT INFRASTRUCTURE

30 JUNE › 2 JULY 2021

FEUP · PORTO · PORTUGAL

Network Architecture

10th INTERNATIONAL CONFERENCE ON STRUCTURAL

HEALTH MONITORING OF INTELLIGENT INFRASTRUCTURE

30 JUNE › 2 JULY 2021

FEUP · PORTO · PORTUGAL

• Network Architecture

10th INTERNATIONAL CONFERENCE ON STRUCTURAL

HEALTH MONITORING OF INTELLIGENT INFRASTRUCTURE

30 JUNE › 2 JULY 2021

FEUP · PORTO · PORTUGAL

Experiment and Result

10th INTERNATIONAL CONFERENCE ON STRUCTURAL

HEALTH MONITORING OF INTELLIGENT INFRASTRUCTURE

30 JUNE › 2 JULY 2021

FEUP · PORTO · PORTUGAL

Training order:

component type as the first task, spalling identification task as the second task

Purpose :

to find out the optimal value of the distillation temperature by observing the variations of accuracy

of the first task during the training of the second task.

​

• Experiment and Result: Distillation Temperature

​

10th INTERNATIONAL CONFERENCE ON STRUCTURAL

HEALTH MONITORING OF INTELLIGENT INFRASTRUCTURE

30 JUNE › 2 JULY 2021

FEUP · PORTO · PORTUGAL

“Initial (%)” refers to the best test accuracy of a task that was trained for the first time.

“Final (%)” refers to the test accuracy of this specific task when the training of all the tasks have finished.

The difference between Initial(%) and Final(%) of one particular task shows the decline

in accuracy of that task during the entire training process.

• Experiment and Result: Comparison

10th INTERNATIONAL CONFERENCE ON STRUCTURAL

HEALTH MONITORING OF INTELLIGENT INFRASTRUCTURE

30 JUNE › 2 JULY 2021

FEUP · PORTO · PORTUGAL

The test accuracy variations of the model trained with different methods on similar tasks

• Experiment and Result: Similar Task

10th INTERNATIONAL CONFERENCE ON STRUCTURAL

HEALTH MONITORING OF INTELLIGENT INFRASTRUCTURE

30 JUNE › 2 JULY 2021

FEUP · PORTO · PORTUGAL

The test accuracy variations of the model trained with different methods on unsimilar tasks

• Experiment and Result

• Experiment and Result: Unsimilar Task

10th INTERNATIONAL CONFERENCE ON STRUCTURAL

HEALTH MONITORING OF INTELLIGENT INFRASTRUCTURE

30 JUNE › 2 JULY 2021

FEUP · PORTO · PORTUGAL

Different Learning Order and its corresponding recognition accuracy obtained by CLDRM

• Experiment and Result: Learning Order

10th INTERNATIONAL CONFERENCE ON STRUCTURAL

HEALTH MONITORING OF INTELLIGENT INFRASTRUCTURE

30 JUNE › 2 JULY 2021

FEUP · PORTO · PORTUGAL

The test accuracy variations of different tasks on different learning order

• Experiment and Result: Training Process

10th INTERNATIONAL CONFERENCE ON STRUCTURAL

HEALTH MONITORING OF INTELLIGENT INFRASTRUCTURE

30 JUNE › 2 JULY 2021

FEUP · PORTO · PORTUGAL

• Experiment and Result: Model Evaluation

10th INTERNATIONAL CONFERENCE ON STRUCTURAL

HEALTH MONITORING OF INTELLIGENT INFRASTRUCTURE

30 JUNE › 2 JULY 2021

FEUP · PORTO · PORTUGAL

• Conclusion

• Fast, continuous and efficient learning of multi tasks,
• Less data set storage space and fewer parameters
• Faster prediction speed and higher recognition accuracy.
• In practical applications, the appropriate distillation temperature setting is dependent on the characteristics of the trained dataset. Proper increase of distillation temperature can improve the performance of CLDRM; however, it can backfire when the temperature is set too high.
• The CLDRM is more suitable for feature-related multitask learning. The recognition accuracy for old tasks decreases negligibly when learning new tasks.
• The learning order has a certain influence on CLDRM.
• Widely used in mobile phones, drones and other terminals with limited computing power to realize real-time scanning and recognition.

​

10th INTERNATIONAL CONFERENCE ON STRUCTURAL

HEALTH MONITORING OF INTELLIGENT INFRASTRUCTURE

30 JUNE › 2 JULY 2021

FEUP · PORTO · PORTUGAL