Deep Learning-Based Fatigue Cracks Detection in Bridge

Girders using Feature Pyramid Networks

College of Civil Engineering and Architecture

Zhejiang University

Reachsak Ly

Jiawei Zhang *

Jun Li

Yunyi Liu

Adviser: Prof. Shu Jiangpeng

College of Civil Engineering and Architecture

Zhejiang University

Content

Introduction

Literature Review

Experiment and Discussion

Methodology

Conclusion

1

2

3

4

5

Introduction

​

Introduction

​

Cracks inspection in SHM

Post-disaster Structural Health Monitoring

Integrity check of the Civil Infrastructure

Importance of spatial characteristics of cracks

Introduction

​

Traditional Crack Inspection

Expensive

Safety Concerns

Time-consuming

Laborious

Introduction

​

Deep Learning Based Crack inspection

Deep Learning-base crack segmentation’s model training flowchart

Literature Review

​

Introduction

​

Crack detection using DCNN

Sliding Window Technique

For concrete crack segmentation

Y. Cha, W. Choi, O. Büyüköztürk, Deep learning-based crack damage detection using convolutional

neural networks, Comput. Aided Civ. Inf. Eng. 32 (2017) 361–378, https://doi.org/10.1111/mice.12263.

Introduction

​

Concrete crack segmentation using U-net

Computer vision-based concrete crack detection using U-net

Z. Liu, Y. Cao, Y. Wang, and W. Wang, “Computer vision-based concrete crack detection

using U-net fully convolutional networks,” Autom. Constr., 2019.

Introduction

​

Crack segmentation using Fully Convolutional Network

Autonomous concrete crack detection using deep fully convolutional neural network

​

C. Dung, L. Anh, Autonomous concrete crack detection using deep fully convolutional neural

network, Autom. Constr. 99 (2019) 52–58, https://doi.org/10.1016/j.autcon.2018.11.028.

Introduction

​

Skeletonizing operations

Crack segmentation using Fully Convolutional Network

Segmentation and measurement of crack width and length

X. Yang, H. Li, Y. Yu, X. Luo, T. Huang, X. Yang, Automatic pixel-level crack detection and

measurement using fully convolutional network, Comput. Aided Civ. Inf. Eng. 33 (2018) 1090–1109

https://doi.org/10.1111/mice.12412.

Introduction

​

DenseNet

MultiResUnet

Other renown Network Architecture for image segmentation

Ibtehaz, N., & Rahman, M. S. (2020). MultiResUNet: Rethinking the U-Net

architecture for multimodal biomedical image segmentation.

Neural Networks. https://doi.org/10.1016/j.neunet.2019.08.025

​

Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017).

Densely connected convolutional networks. Proceedings - 30th IEEE Conferenc

e on Computer Vision and Pattern Recognition, CVPR 2017. https://doi.org/10.1109/CVPR.2017.243

​

METHODOLOGY

​

OVERVIEW OF THE PROPOSED METHODOLODY

​

Labeled Data

FPN

Model 3

Resize

Split

Train set 4

Train set 3

Train set 2

Train set 1

Model 4

Model 2

Model 1

Test Set

Test Set

Test Set

Test Set

FPN

FPN

FPN

Result 1

Result 2

Result 3

Result 4

Pre-processing

Training

Testing

Framework of the proposed study

NETWORK ARCHITECTURE

​

Feature Pyramid Networks

FPN Structure

Skip connection

W operation

120 Image-Label Pairs

DATASET

​

3264 × 4928 pixels

3864 × 5152 pixels

80 Raw Images

3264 × 4928 pixels

IPC Datasets

​

This study use the 120 image-label pairs for training and testing purposes.

3264 × 4928 pixels

1600 × 2400 pixels

2112 × 3168 pixels

DATASET – Resizing Method

DATASET RESIZING

​

Proposed Resizing Method

120 images

120 labels

TS1

TS2

120 images

120 labels

1923 crack sub-images

1923 crack sub-labels

1923 crack sub-images ,

3500 random background and Corresponding labels

​

DATASET SPLITTING

​

Proposed Splitting Method

TS3

TS4

Experiment & Discussion

​

TRAINING FRAMEWORK

​

120 Raw Images

Train with FPN

Train with FPN

Train with FPN

Train with FPN

Model1

Model2

Model3

Model4

Resizing Method

Splitting Method

Training Framework of the proposed study

MODEL TRAINING

​

Workstation Configuration

Data Augmentation

Intel(R) Xeon(R) E5-2678 v3 @ 2.50GHz with 64.0GB RAM and an NVIDIA RTX2080TI with 11.0GB RAM

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

(i)

(j)

(a) a original sub-image, and the sub-image through augmentation commands

(b) HorizontalFlip

(c) VerticalFlip,

(d) ShiftScaleRotate

(e) Blur(blur_limit=3, p=1)

(f) CLAHE(p=1)

(g) HueSaturationValue(p=1)

(h) IAAPerspective(p=0.5), (i) IAASharpen(p=1),

(j) RandomBrightness(p=1) in albumentations (an OpenCV library)

TESTING WORKFLOW

​

One Test Image （3264 × 4928 pixels）

Model3

Model2

Model4

Model1

Resizing Method

Splitting Method

Output2

Output3

Output4

Resize

Recombine

(98×480×640)

Prediction2

Prediction3

Prediction4

(1600×2400)

(2112×3168)

Testing Framework of the proposed study

PREDICTION RESULTS

​

Model 1 and Model 2

Pred-1 IOU = 0.50

Label

Image

Pred-2 IOU = 0.09

Pred-1 IOU = 0.23

Label

Image

Pred-2 IOU = 0.04

Resizing Method

Comparison between predicted results of Model 1 and Model 2

Pred-3 IOU = 0.64

Label

Image

Pred-4 IOU = 0.80

Pred-3 IOU = 0.16

Label

Image

Pred-4 IOU = 0.68

Pred-3 IOU = 0.57

Label

Image

Pred-4 IOU = 0.75

Comparison between predicted results of Model 3 and Model 4

Splitting Method

PREDICTION RESULTS

​

OVERALL PERFORMANCE

​

The mean IOUs and mean Dice Losses of 120 predictions from four trained models using 120 annotated images as test images.

​

Conclusion

​

Among the two methods, experiment show that by splitting images into sub-images will result improved segmentation capability for high-resolution images.

CONCLUSION

​

The framework of crack segmentation task with high-resolution images using FPN is proposed

​

Even with the disturbance in the background , the model manage to efficiently identified

and annotated the the crack image pixel.

Proposed method show promising accuracy for high-resolution image segmentation and could

Be improved for the future research.

​

The mean IoU reaches 0.78 in maximum.

ability to detect target cracks of different image sizes.

Two methods was proposed for the modification of the dataset:

Resizing high-resolution images with a bilinear interpolation algorithm

Splitting high-resolution images into sub-images are proposed.

​

​

College of Civil Engineering and Architecture

Zhejiang University

Reachsak Ly

Jiawei Zhang *

Jun Li

Yunyi Liu

Adviser: Prof. Shu Jiangpeng

College of Civil Engineering and Architecture

Zhejiang University