1 of 38

5tasy

Translation Layout Recovery

Mentors: Barb, Jacky

Members (Fantastic 5):�- Phạm Khánh Trình

- Nguyễn Bảo Tín

- Nguyễn Minh Lý

- Nguyễn Hoàng Ngọc Hà

- Yang Tuấn Anh

2 of 38

Table of contents

01

04

02

05

03

06

Introduction

Proposed

Method

Back-end

Design

Front-end

Design

Product demo

Conclusion

2

3 of 38

Introduction

01

3

4 of 38

Hilarious mistranslation

The need for good translation tool

Language statistics:

4

A motivational slogan to respect your teachers and love your friends… oops!

Is that enough?

5 of 38

Let's try with Google Lens in Translation

5

Thành phần/ Nước Thịnh phin (Nude), Ethanol, PE-700, Diethylene Glycol, PEG- Metyl cô

​

Detine, Didecyldmonium Clorua, Moncsodium maleate, Disodium mala Sodun Oran

​

LỜI KHÔNG SỬ DỤC CỦA CHỦ PHỤ GIA GỬI HOA CỦA MÓN VÀ KHÔNG CÓ THÊM & Gamble (Singap ore) Pi L, 11 North Buona Vista Drive, 121-07, The Metropolis Tower 2, Singapor 138509. Tat 800 186 0018 Procter&Gamble (Malaysia) Sch. Brd, 10th Floor, Surian Tower, 1 Jalan PJU 7/3, Mutiara Damansara, 47810 Petaling Jaya, Selangor Danil Ehsan, Malaysia Tat 1-800-883-957. Imported by: Procter & Gamble Philippines, he, 10F SeverNED, 5th Avenue, Crescert Park West, Bonifacio Global City Fort Bonitaci City 1634 Philippines QUESTIONS? COMMENTS? Call 8094-3955 (Metro Manila 1-800-1-589-8008 Outside MM), Distributed in Australia by Procter & Samble Australia Pty Ltd, Level 41 Iroatian Read TO MAXETAL wakin g linh Ting 11 MPlaza Saigon, 39 Lê Duẩn, Quin 1,TRO t 1800 6935. Han sldung 1 thin g acquarie Park, ASW 2113 Australia Questions TN NGÔI CÓ 5 tn mình tốn hao bì Bảo quản và khô thoảng tránh ánh nắng trưanh hai chDem | Arile Việt Nam, Khu Công nghiệp Đồng An, Phường Bình Hòa, Thành phố đi chung làm là biết nắng THE EYE VET BLP For advice, please contact the National Poison Management and f xuất tới 2t Nam bởi Công ty [Note 8 Gale Đông Dương khu Công nghiệp Đông bị Phòng Bình | Thành pin Thiên An tỉnh Bình trong - trách số - 1 tí sản phớt Công ty Nipad 08-524-1078 or a doct mments Cat (A) 1800 201 418, 07088 555 448

​

chate, Dầu thầu dầu hydro hóa PEG- n, Hydroxypropyl Cyclodextrin, Nước hoa (Chill bef

​

K

VẢI

​

TRÁNH

​

ENTA

​

CHS DŨNG

​

THƯA KHÔNG

​

MÓN ĐÌNH

​

NOOT PHÁI SAN

​

4 987176 112682"

​

NG CÁCH LONG Depone sở hữu sau khi sử dụng. Xem ngày sản xuất trên bao bì Xem agysl Thể tin thực: 370ml

LÂU SÁCH

​

P&G www.pg.com

​

90211202 Sản xuất tại Việt Nam

​

Xịt làm mới vải

​

LÀM

​

Cách sử dụng

​

Xịt nước

​

MỘT

​

Curtai

​

Vải C

​

An toàn

Messy because THE LAYOUT IS BROKEN!!

6 of 38

Let's try with onlinedoctranslator.com

6

7 of 38

Product goals

7

Accurate translations from English to Vietnamese/Japanese

Keep the file layout for PDF files

Focus on academic papers

8 of 38

8

Original file in English

Japanese translated file

Vietnamese translated file

9 of 38

Proposal methods

02

9

10 of 38

10

Proposed method

BACK END

File system: PDF, PNG

Database: PostgreSQL

Translation and layout recovering (OCR + NMT)

FRONT END

​

What users see

and interact with

(NextJS and TailwindCSS)

Send PDF file

Receive the translated file

Request

USER

Response

11 of 38

AI Core

11

12 of 38

OCR Model

12

Public dataset: contain various formats

Dataset that we want

13 of 38

OCR Model

13

⇒ Our team created a suitable dataset with 100 data samples to evaluate these models.

More informations:

  • Total 7401 words
  • Manually labeled and cross-validated by our members.

Model

CER (%)

Time Inference (secs/image)

PaddleOCR

4.51

0.3243

EasyOCR

2.05

0.1726

Tesseract

0.95

9.8385

14 of 38

Easy OCR

EasyOCR is an open-source library, performing well in OCR tasks across multiple languages.

The pre-trained model is accurate and highly efficient without the need for resource-intensive fine-tuning.

​

14

15 of 38

Neural Machine Translation (NMT)

enviT5 model from VietAI:

  • Trained on the MTet and PhoMT datasets
  • State-of-the-art model in English-Vietnamese and Vietnamese-English translation.

opus-mt-en-jap from Helsinki-NLP:

  • English-Japanese translation model achieve state-of-the-art results on a variety of benchmarks.
  • In a recent study, Helsinki-NLP's Japanese-English translation model achieved a BLEU score of 42.1 and a chrF score of 0.960 on the Bible-Uedin dataset.

15

16 of 38

NMT Experiments

16

Model

Language

BLEU

envit5-translation

EN -> VI

45.47*

vinai-translate-en2vi

EN -> VI

44.29

Evaluate English-Vietnamese translation model on PhoMT:

Evaluate English-Japanese translation model on OPUS:

Model

Language

BLEU

opus-tatoeba-en-ja

EN -> JA

15.2

opus-mt-en-jap

EN -> JA

42.1*

17 of 38

envit5-translation result

17

18 of 38

envit5-translation fine-tuned result

18

19 of 38

To conclude…

19

20 of 38

Backend Design

03

20

21 of 38

Backend design

21

22 of 38

Database

22

23 of 38

Front-end

Design

23

24 of 38

Wireframe

24

25 of 38

Front-end structure

25

\

GET - authenticated state

POST - uploaded file

DELETE - authenticated state

​

\login

POST - login info

​

\register

POST - register info

​

\history

GET - history info

POST - search query

​

\profile

GET - profile info

PUT - profile info

26 of 38

Front-end frameworks

26

27 of 38

Product

demo

27

28 of 38

28

29 of 38

Deployment

29

Try 5tasy on: https://9a77-35-88-60-70.ngrok-free.app/

30 of 38

Conclusion

30

Successfully deploy the app to the website

Built an end-to-end system

Lightweight, free, and portable

Authenticated & user-center application

31 of 38

References

31

  • MaskRCNN - phamquiluan - PubLayNet
  • EasyOCR - JadedAI - jaded.ai
  • envit5-translation - Chinh Ngo, Trieu H. Trinh, Long Phan, Hieu Tran, Tai Dang, Hieu Nguyen, Minh Nguyen, Minh-Thang Luong - huggingface.co.
  • opus-mt-en-jap - Helsinki-NLP - huggingface.co.
  • AutoPrompt - Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, Sameer Singh - arxiv.org
  • Guides on using Docker for Python application - Docker docs.
  • Install Docker Engine on Ubuntu - Docker docs.

​

32 of 38

Thank you for your attention!

32

33 of 38

Appendix

33

34 of 38

Mask R-CNN

34

Model

mAP (%)

PP Structure V2

93.60

Mask R-CNN

96.00

PPStructure: https://arxiv.org/pdf/2210.05391.pdf

MaskRCNN: https://arxiv.org/pdf/1703.06870.pdf

​

35 of 38

Security?

User password will be hashed before storing into the database…

User must login to view their own files, translation history…

They can not see any files that not belong to them…

User password

Must Login

35

36 of 38

36

  • Limited language availability: The application currently supports only two types of translation.
  • Slow processing time: The execution time of the translation process is still relatively slow.
  • File size constraint: The application is restricted to handling files under 50 MB in size.

​

Limitations

37 of 38

37

Improvement compared with other tools

  • Our tool only translates the needed information instead of translating all files
  • More authenticate
  • Provide privacy for users

​

Advantages

38 of 38

38

  • Work breakdown structure

- Lý, Trình: Model AI

- Hà, Tín: Back end + Database

- Yang, Tín: Front End

Teamwork