1 of 47

The Le Gandee Team

1

Markus Zhang

SJI International

Nicole Chay

National Junior College

Jamie Wee

Hwa Chong JC

Lachlan Goh

Raffles Institution

Matthew Han

NUS High

A team of five React, Python, and ML enthusiasts from schools around Singapore.

2 of 47

Securus

Staying Safe 2.0.

3 of 47

Problem & Impact

01

3

4 of 47

$10,000,000,000

COVID-19 is $10 trillion problem. (The Economist)¹

4

5 of 47

And COVID-19 is here to stay.

“New Normal” Fatigue

Popular opinion could turn against wearing masks at any time.

SMEs worldwide can’t survive a long, drawn-out war of attrition against COVID-19.

5

COVID-19 is not going away anytime soon, and the longer it stays, the more damage it inflicts on economies and societies. With easing restrictions and the vaccine rollout, it may seem Singapore has emerged relatively unscathed from the pandemic. But that is to ignore the new COVID-19 mutations, where these epidemics in India and other nations have cost far too many lives. Not to mention, vital domestic industries have had to shoulder the enormous economic costs of the pandemic, in the form of dangerously low sales or foot traffic. Though the Singapore government offers subsidies, economic assistance cannot persist in a long, drawn-out war of attrition against COVID-19. Experts even claim the pandemic will last four or five years.

As Singapore walks the tightrope between economic revitalisation and low community cases, a sociotechnical solution that can optimise for both has never been more important or urgent. Wearing a mask is the preeminent, unparalleled solution deployed today; however, as Singaporeans feel the fatigue of the multi-year “new normal”, the tide of popular opinion may turn against mask-wearing at any time, as it has in some parts of the world already. Singapore’s schools, malls, construction workplaces, and government must build fortifications to ensure Singaporeans continue to support mask-wearing and not just because it’s mandated by law.

6 of 47

Ambassadors

SafeAccess Checkpoints

Can only patrol small limited area
Not enough exist
Very expensive

Errors are frequent
Nothing stops users from removing their masks after the checkpoint

6

Flawed “Solutions”

Currently, no centralised and fully autonomous system exists to detect whether citizens are wearing masks. Various mask-detection systems at SafeEntry checkpoints often need manual intervention to correct its errors, but more importantly, nothing stops citizens from taking off their masks afterwards. Mask-detection systems are of little added benefit when government presence and social pressure are already strong. Instead, mask-detection systems are needed in poorly-monitored, high-traffic local areas where clusters typically arise. Not enough safe-distancing ambassadors exist to tackle a task of this scale. A fleet of virtual ambassadors, driven by a machine learning model, would accurately and proactively identify these rule-breakers to bolster pivotally the core foundations of the COVID-19 effort.

7 of 47

A Social Game

Intrinsic Self-Improvement

Penalise Rule-breakers

7

Our Solution

8 of 47

Today

Tomorrow

Not enough ambassadors
Grudging compliance
Information delay
Can only patrol small limited area

Virtual ambassadors at near-infinite scale
Popular enthusiasm & personal stake
Real-time, non-intrusive data
Near-100% indoor coverage

8

9 of 47

UN SDG Impact Value Chain

9

Metrics to Measure

Results

New COVID-19 Waves despite Ambassadors & TraceTogether

N/A

Game-ified, Unlimited-Scale, Self-Improving Public Safety

# Users
Conversion rate
Retention rate

Enthusiasm to wear masks
Increased detection accuracy

# Rule- breakers Caught
# MAU
ML accuracy / F1 score

COVID-19 protection

Fewer COVID-19 cases than before

V-Shaped Economic Recovery

2021 GDP growth higher than projections
Soaring domestic consumption

10 of 47

Impact on Community

Personal stake in national effort
Blazing-fast response to COVID-19 threats
Repurpose our smart city for public safety.

10

11 of 47

Features

02

11

12 of 47

SafeStop

Our Game-ified Mask-Detection System

12

A CCTV or computer camera at a SafeStop

Lose Streaks

Earn Coins

Compete with Friends

13 of 47

SafeStop

Privacy as a Feature

13

Like Pokemon Go: Users can see the SafeStop cameras on a map

Like ERP: Users know exactly where the “traffic cameras” are

Like TraceTogether: Voluntary opt-in monitoring

User identification inherently impossible so long as they keep their masks on

14 of 47

Leaderboards

Streaks

Virtual Shop

Game-ified Safety

National rankings incentivise users towards health-promoting behaviours

Buying custom avatars and power-ups boosts retention

Longer streaks fetch substantially more coins. Users won’t ever want to lose their streak

14

Securus incentivises users to keep their masks on with several gamification features and a game-like user experience.

Streaks increment after a user passes a SafeStop wearing a mask, and longer streaks fetch substantially more coins, incentivising users to never lose their streak. Users previously unconcerned or apathetic to the COVID-19 effort are now socially incentivised to wear a mask, catalysing the high retention rate and viral growth of Securus.

Points are in-game currency, rewarded when wearing a mask near any SafeStop. For more points, users must pass by more SafeStops, which could significantly drive domestic consumer spending and economic revitalisation in the longer run without the government spending a single cent. Points can be spent in the Securus virtual store on custom avatars and power-ups (to freeze a streak if a user absent-mindedly forgot to put on a mask, for example).

Users compete on a segmented leaderboard for more points. In many games, national rankings incentivise users towards more health-promoting behaviours and move up ranks. A concrete friend rank allows friends to incentivise and challenge each other. Since soaring domestic consumption, a consequence of going outside to earn points, is desirable, Securus uses a rank decay system, where a user’s rank falls over time if they do not continue to earn points. The leaderboard discourages complacency and encourages users to maintain their streaks by wearing masks and going outside.

15 of 47

Social Motivation

Bubble Groups

“Double-or-nothing” for bubble groups of two during a trip outside

Proof: Forest App³

15

Pair Streaks

Earn more points and keep friends accountable

Proof: Snapchat

[3] https://www.citrusbits.com/how-gamification-has-catapulted-duolingo-strava-and-forest-to-the-top-of-their-respective-app-categories/

Group streaks let a group of friends keep a double-or-nothing shared streak, where the friends extend and lose their streaks when any one of them do. On an excursion out, users can also “entangle” their phones (via the phone’s NFC) with friends in the same safe-distancing group by tapping their phones to each other, serving as a temporary Group Streak that deactivates at the day’s end. This social motivation strategy powerfully shapes behaviour and has reaped tremendous rewards with Forest (the number one iOS productivity app). Snapchat’s paired streaks is also pivotal to their user retention strategy, as neither friend gives up because they feel guilty making the other lose their streak. Such social pressure can be applied to do good and incentivise mask-weaking.

16 of 47

MaskCAPTCHA

Our Very Own Free Mechanical Turk

16

maskCAPTCHA

On-Demand

Image Labelling

Crowdsource to

a Billion Users²

maskCAPTCHA

[2] https://www.ted.com/talks/luis_von_ahn_massive_scale_online_collaboration?language=en

However, no deep learning model is completely accurate, and sometimes, users actually wore a mask, but the DeepLens did not detect a mask (a false positive).

Securus has the foresight to account for this inevitable inaccuracy: we design maskCATPCHA, an in-app, community-driven appellate and image-labelling system where wrongly-accused users can submit the “incriminating” image that lost them their points for human verification (low-confidence images are also automatically sent without user intervention). maskCATPCHA is a marriage of the brilliant Amazon Mechanical Turk and CAPTCHA systems, which have both crowdsourced image-labeng on a massive, mammoth scale, ensuring that Securus can effectively scale our accuracy as our data points increase. CAPTCHA, in particular, helped train self-driving cars and digitise millions of books (via the squiggly text prompts) without spending a single cent, testifying to the sheer potential power of a community-driven labelling system. Therefore, it serves as the strategic framework of maskCAPTCHA (hence the name).

Images submitted to maskCAPTCHA appear in a community feed tab of the React Native app, where other users label the images in exchange for game points and street credit (in the form of XP). The system protects against wrong human entries, as it feeds the same image to multiple users and averages the resultant label across their entries, and its effectiveness is proven by its remarkable prior performance with CAPTCHA (see Figure 3). This resultant label is accepted as ground truth, the image is added to the training dataset of our CNN model, and re-training is triggered so that the ML lifecycle (see Figure 1) restarts again.

17 of 47

MaskCAPTCHA

17

Retrain → With new ground-truth label

Inaccurate → Wrongly penalise user

Label → Others earn coins for labelling image

3

2

3

4

2

1

SafeStop

Appeal

User Labelling

Retraining

4

Appeal → Use in-app appellate system

maskCAPTCHA

However, no deep learning model is completely accurate, and sometimes, users actually wore a mask, but the DeepLens did not detect a mask (a false positive).

Securus has the foresight to account for this inevitable inaccuracy: we design maskCATPCHA, an in-app, community-driven appellate and image-labelling system where wrongly-accused users can submit the “incriminating” image that lost them their points for human verification (low-confidence images are also automatically sent without user intervention). maskCATPCHA is a marriage of the brilliant Amazon Mechanical Turk and CAPTCHA systems, which have both crowdsourced image-labeng on a massive, mammoth scale, ensuring that Securus can effectively scale our accuracy as our data points increase. CAPTCHA, in particular, helped train self-driving cars and digitise millions of books (via the squiggly text prompts) without spending a single cent, testifying to the sheer potential power of a community-driven labelling system. Therefore, it serves as the strategic framework of maskCAPTCHA (hence the name).

Images submitted to maskCAPTCHA appear in a community feed tab of the React Native app, where other users label the images in exchange for game points and street credit (in the form of XP). The system protects against wrong human entries, as it feeds the same image to multiple users and averages the resultant label across their entries, and its effectiveness is proven by its remarkable prior performance with CAPTCHA (see Figure 3). This resultant label is accepted as ground truth, the image is added to the training dataset of our CNN model, and re-training is triggered so that the ML lifecycle (see Figure 1) restarts again.

18 of 47

Demo

18

19 of 47

03

Technicals

19

20 of 47

Mobile & API

20

Compliant with industry-standard practices for smooth mobile experience. AWS Amplify connects app to AWS cloud easily.

React Native Mobile App

The first main system in our architecture is the cross-platform React Native client and its serverless API endpoints. First, we create a cross-platform React Native App to interface with the native device APIs like NFC and background geolocation. The app employs the AWS Amplify framework to connect with the AWS cloud easily. Second, most requests made pass through the serverless GraphQL API Gateway with AWS AppSync. We define its queries and mutations with a GraphQL schema, we deploy the API with the Serverless Framework and AWS CloudFormation (omitted from diagram), and we log errors with AWS CloudWatch. Third, we define AWS Lambda resolvers with TypeScript and Jest to handle our requests. Fourth, we leverage the AWS Node.js SDK to call AWS DynamoDB and AWS RDS for strongly relational data. The two databases act as secure storage for our users’ information: coins, achievements, divisions, Group Entanglement states, paired streaks, and position on leaderboard. AWS IAM policies and roles (omitted from diagram) authenticate all reads and writes and selectively grant permission.

21 of 47

Mobile & API

21

Secure, serverless custom API. Scales to 100K+ concurrent requests cheaply. Deployed with AWS CloudFormation with AWS CloudWatch logging. IAM policies and roles selectively grant permission

GraphQL API Gateway

22 of 47

Mobile & API

22

Resolves API endpoints and orchestrates AWS services (S3, DynamoDB, IAM) for hosting and storage. Uses Jest for unit testing.

Lambda Layer

23 of 47

Mobile & API

23

Secure, NoSQL storage with sub-millisecond read/write for storing appeals, leaderboard, pair streaks, etc

AWS DynamoDB

24 of 47

ML

System

24

Massive bucket stores Kaggle datasets and images from maskCAPTHA. Images labelled with metadata tags

AWS S3

Data Lake

25 of 47

ML

System

25

Concurrently transforms unstructured images into ML model input.

AWS Glue Workflow

26 of 47

ML

System

26

Runs PyTorch in Jupyter Notebook to train the state-of-the-art YOLOv3 ResNet50 neural network. Optimises for DeepLens hardware with AWS SageMaker Neo

ML System

27 of 47

ML

System

27

AWS IoT Greengrass and AWS IoT Core syncs the latest SageMaker model. Python Lambda detects mask and uploads image

AWS DeepLens

28 of 47

ML

System

28

DeepLens triggers GraphQL API to identify user by face from S3 Data Lake of all user Face IDs

AWS Rekognition

29 of 47

ML

System

29

If user identified in Rekognition, API deducts virtual in-game points and sends a friendly email reminder to user

AWS SES

Email

30 of 47

Complete Architecture

30

31 of 47

Business

04

31

32 of 47

Why Now?

32

Stop a Flood of COVID-19 Waves

Infrastructure Ready

Only 30%

Through

The Pandemic.³

33 of 47

A No-Brainer Opportunity Cost

33

Global COVID-19 Impact (The Economist)

TAM⁴

Singapore’s COVID-19 Impact (MTI)

SAM⁵

Prevent 1% of Singapore’s COVID-19 Impact

SOM

$27 Billion

$10 Trillion

$273 Million

34 of 47

Competition Matrix

34

Popular Enthusiasm

Real-Life Sentinel

SafeAccess

Securus

SafeEntry

Ambassadors

Self-Improving

Deep Nationwide Coverage

Automated and Cheap

✓

TraceTogether

35 of 47

Case Study: Singapore

35

36 of 47

Same. Cost.

36

800 SafeStops

1 Ambassador

v.

$160.00 each

$0.20 each

37 of 47

99.88%

Cheaper compared to a Safe Ambassador

37

38 of 47

Future

05

38

39 of 47

Fine-Tuned ML

Don’t penalise users for dining-out, exercising, etc
Monitor safe distancing and group sizes
Support future COVID-19 measures: face shields, vaccine passports etc

39

40 of 47

CCTV Support

No infrastructure installation needed
Connect existing CCTV cameras at malls, offices, etc to the IoT cloud
Pivotally reduce upfront cost

40

41 of 47

Partnerships For Our Goals

Business development with corporate campuses, construction sites, hospitals, Changi Airport etc for beta-testing
Integrate with LumiHealth (if permissible) to catalyse user adoption and sticky retention
Real vouchers: the ultimate game reward → collaborate with the F&B industry (Proof: LumiHealth)

41

In addition, our solution has significant untapped potential commercially. The issue of social benefits of wearing masks in the trying times of a pandemic is especially prevalent and a main concern of authorities. Raising social awareness among the public to encourage and incentivise people to wear masks is undoubtedly important, and the means to do so can be fuelled through partnership with retail stores and companies. A successful model inspiring this would be LumiHealth, where users can “turn wins into rewards”. In the same vein, our app can be a collaboration with various outlets (e.g. in food and beverage industry) to enable users to convert their points won into rewards. Similarly, if possible, a collaboration with the respective branch of government such as the Ministry Of Health could be initiated as well, with rewards systems including possibly redeemable vouchers and others.

42 of 47

Securus. For a ludicrously safer world.

42

CREDITS: This presentation template was created by Slidesgo, including icons by Flaticon, and infographics & images by Freepik.

43 of 47

Appendix

43

44 of 47

44

UML Diagrams

Left: MLOps Activity Diagram

Right: Use Case Diagram

45 of 47

45

UML Diagrams

Left: SafeStop Sequence Diagram

Right: maskCAPTCHA Sequence Diagram

46 of 47

Cost Calculation

10 users per minute for 16 hours - 9600 local images - 1% flagged - 1000 cloud images / day / SafeStop

70% of 7M Singaporeans - 5M users

100 Rekognition images - $0.1 / day / SafeStop

100 IoT Lambda Invokations - $0.00001 / day / SafeStop

100 AppSync IoT requests - $0.0004 / day / SafeStop

100 S3 images and maskCAPCTHA - $0.001 / day / SafeStop

100 maskCAPCTHA’ed DynamoDB metadata - $0.0003 / day / SafeStop

100 SES emails - $0.01 / day / SafeStop

10000 AppSync earn coin requests - $0.04 / day / SafeStop

10000 Lambda Invokations - $0.001 / day / SafeStop

1000 game DynamoDB metadata - $0.03 / day / SafeStop

Thus, SafeStop Total - $0.2 / day

Ambassador Total - $20 (substantially below SG median wage) * 8 hours = $160 / day

46

Other costs

1 DeepLens - $250 / SafeStop

5M users’ Rekognition labels - $50 / month

100KB Face ID image for 5M users - $12 / month

5M users’ DynamoDB metadata - $16 / month

Monthly SageMaker retraining for 8 hours on p2.xlarge - $9 / month

47 of 47

47

DynamoDB NoSQL Data Model