1 of 11

Health care misinformation: an artificial intelligence challenge for low-resource languages

Sarah Luger1, Martina Anto-Ocrah2, Allahsera Tapo3, Christopher M. Homan4, Marcos Zampieri4, Michael Leventhal3

1Orange Silicon Valley, 2University of Rochester Medical Center, 3Centre National Collaboratif de l’Education en Robotique et en Intelligence Artificielle (RobotsMali), 4Rochester Institute of Technology,

2 of 11

Overview

​

  • AI for social good in developing nations
  • MT to narrow healthcare information gaps
  • Pilot studies:
    • Human computation
    • Machine translation

​

3 of 11

AI for social good in developing nations

  • International aid reflects priorities of wealthier nations
  • Africans would instead prefer access to the same level of resources as wealthier nations:
    • Information
    • AI expertise (AI leaders stand to gain 20-25% increase in economic benefits)
  • AI technology (machine translation) can help alleviate these disparities

​

4 of 11

Colonialism and its Legacy

British colonies

  • Eg., Ghana
  • Education
    • Decentralized
    • Cheap
  • Higher levels of English understanding

French colonies

  • Eg., Mali
  • Education
    • Centralized
    • Available only to wealthy
  • 20% of Malians mastered French

5 of 11

Impact of colonial language policy on Covid

15.2 Malians fluent in Bambara but not French have much less information about:

  • Viral transmission modes
  • Use of personal protective equipment
  • Movement restrictions
  • Quarantine measures
  • Distancing protocols

6 of 11

Proposed

solution

Machine translator

Human computation

7 of 11

Pilot study: human computation

  • 2 French texts
    • Wikipedia article (195 words, 10 sentences)
    • Malian news transcript (428 words, 22 sentences)
  • 7 participants
    • 4 written-written Bambara
    • 3 written-spoken Bambara
  • 1 evaluator

8 of 11

Results: human computation

Malian news

Wikipedia

Written

Oral

Overall

Written

Oral

Overall

Exact meaning

0.830 ± 0.181

0.770 ± 0.208

0.840 ± 0.197

0.870 ± 0.206

0.530 ± 0.186

0.730 ± 0.220

Literalness

0.730 ± 0.130

0.530 ± 0.298

0.640 ± 0.243

0.830 ± 0.238

0.580 ± 0.211

0.760 ± 0.244

Standard Bambara

0.740 ± 0.234

0.790 ± 0.178

0.760 ± 0.207

0.830 ± 0.171

0.850 ± 0.068

0.830 ± 0.144

Highest BLEU Pair

0.408

0.363

0.408

0.645

0.377

0.645

9 of 11

Pilot study: machine learning

  • Data: SIL Mali French-Bambara-English����
  • Neural machine translator: JoeyNMT (Kreutzer, Bastings, Riezler 2019)
    • Six layers
    • Four attention heads/layer
    • Transformer layer size: 1024
    • Hidden layer & embedding sizes: 256
  • Training
    • 120 epochs
    • 1024 tokens/batch

​

Bambara

French

English

glosses

3,548

4,847

4,855

examples

2,023

2,021

2,021

combined

5,571

6,868

6,876

aligned

2,158

2,146

2,158

10 of 11

Results: machine learning

​

​

bam→fr

fr→bam

bam→en

en→bam

NMT Model

Configuration

BLEU

ChrF

BLEU

ChrF

BLEU

ChrF

BLEU

ChrF

Word

​

18.9

0.3

20.6

0.3

19.1

0.3

17.7

0.3

Char

​

11.7

0.2

10.6

0.2

13.2

0.2

12.5

0.2

BPE

500 subword merges each

19.1

0.3

21.1

0.3

18.1

0.3

21.3

0.3

+ BPE dropout

dropout=0.1

17.8

0.3

16.6

0.2

17

0.2

19.3

0.3

BPE

1000 subword merges each

19.2

0.3

20.4

0.3

19.1

0.3

20

0.3

+ BPE dropout

dropout=0.1

16

0.3

17.7

0.2

15.2

0.2

18.2

0.2

Test results on approx 270 held-out items

20.9

0.3

21.4

0.3

14.8

0.3

20.9

0.3

11 of 11

Conclusion

  • Machine translations of under-resourced languages is fertile ground for AI for social good
  • Human computation is scalable, but more exhaustive guidelines & tools are needed to improve translation consistency
  • First attempt at translating to/from Bambara
  • Additional monolingual language resources, especially those in the medical domain (UMLS, SNOMED, etc.) can be utilized in this work.

​