Text localization and recognition�on complex street view�

Group 2

陳昱任、康楹婕、洪嘉妤、許哲瑋

Outline

• Introduction
• Dataset
• Methods Comparison
• Final result
• Conclusion
• Reference

Introduction

Introduction

• A task about finding and realizing texts in a image
• It’s difficult to recognize Taiwanese signboard, because…
• Many of signboard’s shape(Rectangle, circle, irregular shape…)
• Multiple languages in a signboard (Chinese, English, Japanese, Korean…)
• Horizontal/Vertical text in the image
• Inconsistent word size
• There are two methods to accomplish the task:
1. Separate: Text detection model + Text recognition model
2. End to end: CRNN model = CNN + RNN
• Evaluation metric is F-score

​

Taiwanese street view

1.Irregular shape/ Different word size

2.Different text direction

Dataset

Dataset

• The organizer provides training set about 15188 images
• There are a lot of competitions about scene text field, the famous one is ICDAR competition
• ICDAR provide some datasets, for example: IC-15, IC-17, LSVT, CTW…
• We expand training set with CTW(Chinese Text in the Wild) dataset

​

Method Comparison

Method Comparison

Remove vertical flip

​

Add MinAreaRect transform

Loss function: cross entropy

Learning Rate: 0.0001

Optimizer: Adam

Epochs: 100

​

Transform

• Using opencv MinAreaRect
• Apply random brightness, rotation, flip, resize, shift on our training set

​

Final result

Final result

Conclusion

Conclusion

• Data processing is the most important part in this task
• Text direction
• Photo filming angle
• End to end model performance is not very well
• What we learned:
• The bigger dataset we have, the bigger progress we done
• Make good use of pre-trained model
• Regularize all text shape in retangular

​

​

​

​