Johnson�2025/09/17

Cofacts 跟「字幕

由 Amara.org 社群

提供」說再見

訊息回報機器人與查證協作社群

2

是什麼？

Cofacts 資料庫

EST. 2016

圖 36K

文 182K

影 17K

音 0.1K

QR Code�要消失囉

6

影片訊息

7

影片逐字稿

”36,000以上,再加上扣牌半年,一次你就飽了!

因為現在要設置行人優先區喔!限速20公里而已”

“現在學校跟學區

它會貼設一個告示牌

現在只要看到那個告示牌

它就是限速20”

​

​

“被罰一次36,000你就飽了

現在你只要在路上看到這個標誌「行人優先區」看到你的時速馬上要變成20公里”

​

“十八名法拉比埃特工氣勢洶洶,砰地踹開佩洛西空置半年的豪宅大門。

特工們直奔書房,用掖牙鉗撬開詩坦威鋼琴的暗格,掏出23塊加密硬碟,又撕開飯糕贗品畫框,拽出一本軍火交易賬本。”

​

8

影片逐字稿

“36,000以上,再加上扣牌半年,一次你就飽了!

因為現在要設置行人優先區喔!限速20公里而已”

“現在學校跟學區

它會貼設一個告示牌

現在只要看到那個告示牌

它就是限速20”

“被罰一次36,000你就飽了

現在你只要在路上看到這個標誌「行人優先區」看到你的時速馬上要變成20公里”

內容相同

Search hit

逐字稿不準

10

Cofacts 逐字稿需求

在線等，急！

檢索需求：影音裡有的文字都要有

查證需求：影音裡沒有的文字最好別有

能處理 Cofacts 的守備範圍

11

守備範圍

Cofacts dataset = 使用者所回報的、LINE 上的東西

無人聲有文字

有人聲無文字

無人聲無文字

12

守備範圍

Cofacts dataset = 使用者所回報的、LINE 上的東西

有人聲有翻譯

有人聲亂翻譯

有人聲有文字

13

OpenAI Whisper

「有人聲」均可處理

Avg latency 7.6s / P95 latency 19s

~$17 USD / mo

14

OpenAI Whisper

「有人聲」均可處理

Avg latency 7.6s / P95 latency 19s

~$17 USD / mo

「無人聲」會有幻覺�導致檢索錯誤

15

解決 Whisper 的幻覺？

偵測音量 → 通常有背景音樂

Voice Activity Detection (VAD) models → 要準備跑模型的環境

影音分開用 Video LLM & Whisper 處理 → 需要建立 pipeline

16

17

Google Gemini

「有人聲」或「有文字」即可處理

System: “You are a transcriber that provide precise transcript to video and audio content.”

User: “Your job is to transcribe the given media file accurately.

Please watch the media file as well as listen to the audio from start to end.

Your text will be used for indexing these media files, so please follow these rules carefully:

• Only output the exact text that appears on the media file visually or said verbally.
• Try your best to recognize and include every word said or any piece of text displayed. Don't miss any text.
• Please output transcript only -- no timestamps, no explanation.
• Your output text should follow the language used in the media file.
• When choosing between Traditional Chinese and Simplified Chinese, use the variant that is used in displayed text.
• If there is no displayed text, then prefer Traditional Chinese over the Simplified variant.
• To maximize readability, sentenses should be grouped into paragraphs with appropriate punctuations whenever applicable.”

18

Whisper vs Gemini

19

Gemini Limitations

無人聲無文字 → 其他檢索方式

影片太長、字太小，效果就差

Repetition loop bug → 調參數

20

Conclusion

It’s all about trade-offs

Cost

Latency

守備範圍

Tolerance to hallucination

Langfuse Data

2K sample

2K sample

6.7K sample

3.5K sample

6.7K sample

3.5K sample

Cofacts LINE bot

1. Report msg

ID: @cofacts

In average

10 volunteers per week

20+ new replies per day

2. Volunteer FC

In average

1,000 DAU

100+ new msg per day

Cofacts LINE bot

1. Report msg

ID: @cofacts

2. Volunteer FC

3. bot response

In 30 days

10,000 auto replies

In average

10 volunteers per week

20+ new replies per day

In average

1,000 DAU

100+ new msg per day

2.3M audiences

Collaborative

fact-checking

25

Cofacts LINE bot

1. Report msg

ID: @cofacts

2. Volunteer FC

3. bot response

In last 30 days

10,000 auto replies

In average

10 volunteers per week

20+ new replies per day

In average

1,000 DAU

100+ new msg per day

2.3M audiences

27

Replace human

28

Spam detection

Preliminary analysis

by AI

Transcript

by AI and the crowd

Topic label

by AI and the crowd

Irreplacable

29

AI Transcripts

30

AI Transcripts

”36,000以上,再加上扣牌半年,一次你就飽了!

因為現在要設置行人優先區喔!限速20公里而已”

“現在學校跟學區

它會貼設一個告示牌

現在只要看到那個告示牌

它就是限速20”

​

​

“被罰一次36,000你就飽了

現在你只要在路上看到這個標誌「行人優先區」看到你的時速馬上要變成20公里”

​

“十八名法拉比埃特工氣勢洶洶,砰地踹開佩洛西空置半年的豪宅大門。

特工們直奔書房,用掖牙鉗撬開詩坦威鋼琴的暗格,掏出23塊加密硬碟,又撕開飯糕贗品畫框,拽出一本軍火交易賬本。”

​

31

AI Transcripts

“36,000以上,再加上扣牌半年,一次你就飽了!

因為現在要設置行人優先區喔!限速20公里而已”

“現在學校跟學區

它會貼設一個告示牌

現在只要看到那個告示牌

它就是限速20”

“被罰一次36,000你就飽了

現在你只要在路上看到這個標誌「行人優先區」看到你的時速馬上要變成20公里”

Same topic

32

33

34

Topic Labels

Topics

35

Spam Detection

AI-detected spam posts

Spam takedown announcement by AI

(reviewed by human before in effect)

36

Preliminary Analysis

Automated analysis from AI

Preliminary Analysis

​

Do not assert correctness

Question evidence regarding factual statements

Highlight emotions and specific stances

Provide recommendations

The FBI raided Pelosi’s mansion last night (Feb 14). They found stacks of cash, gold, and even secret documents on military deals ...

This message has been recorded in Cofacts awaiting verification by kind-hearted individuals.

View the reported message

This message is pending verification. Please refrain from believing it for now.

Below are the preliminary analysis results of this message by the bot, hoping to provide you with some insights.

The Prompt for Analysis

It’s YYYY/MM/DD today. You are a helper assisting readers in media literacy. You always speak in Taiwanese Mandarin. A reader has sent you an internet message.

SYSTEM

<The internet message>

USER

USER

As an audience, what aspects of this message should I pay attention to? Please excerpt the parts of the message that require special attention and explain why general audience should be cautious of them. Please just reply the original text and don’t make up new content.

User feedback for Analysis

+1

-1

Average score

0.68 = 16% * -1 + 84% * +1

Proof of concept

Still under development

Takeaway

Generate Answers? Censorship & Hallucination!

Use AI in to speed up processes and lower the entry barrier

Collect human feedback and revise

Use LLM to stimulate the audience's brain

​

voter

Resources

43

Disinformation fighters in Taiwan

Fact-checking Organizations

Media literacy education

Researchers /�Think-tanks

hi@cofacts.tw

Future work

45

USER

AUTO-REPLY

CHATBOT

MESSAGE

PLATFORM

Collaborative

fact-checking

Connect�fact-checks to instant messages

193K messages�Text, video, images, etc

Cofacts Chatbot

Daily PV 40K�Last 365-day PV 12M+

15+ per week

Including TFC、MyGoPen、⋯⋯

​

353K followers

Last 365-day DAU 1K ~ 2K

Statistics from 2016.10 ~ 2024.10.17

Suspicious messages

Public database

Fact-checking Collaborators

Chatbot replies

Fact-check replies

Website

Cofacts Chatbot

Open analytics report

49

🔗 cofacts.tw/analytics

50

Open source and Open data

Source code under MIT license

Open API & Open data under CC BY-SA

51

https://www.readr.tw/project/fake-news

https://medium.com/twreporter/3099f236d00a

https://medium.com/doublethinklab-tw/e3f9ce4e72e

https://iorg.tw/_en/da/25

Cofacts data referred in news and thinktank reports

52

Auntie Meiyu by Gogolook

(583K Followers)

Third party chatbots extending Cofacts’ reach to 2.3M+ audience

Dr.Message by Trend Micro

(1084K Followers)

LINE Fact checker

(854K Followers)

Cofacts 真的假的 🇹🇼 (Old)

Cofact.org 🇹🇭 （Opendream)（Old）

#MilkTeaAlliance

54

Provides open data to scholars, media and researchers

Open data、API

Request

Update

Statistics

Factcheckers

Fact check collaboration

Media literacy lectures

Open data

Analytics

The

Ecosystem

Cofacts Chatbot

Chatbot Users

Researchers

Source code across borders

3rd-Party

Outlets

Who are writing fact checks�on Cofacts?

Editing／

FAQ

Anyone

桌面版：右上角「登入」

cofacts.tw

58

How do you control the quality of volunteer’s work?

Editing／

FAQ

60

Collaborative model without background checks

👿 Bad actors may provide disinformation

😇 Kind hearted people provide ‘corrected’ information

😵 No enough resource for a better governance model

61

...other supporting measures

Allow and display multiple replies

Latest reply

Other replies,

Sorted by usefulness votes

Summary of all replies indicating disagreements among editors

Provide external references

63

❌ Contains misinformation

References

• 2 articles from different fact checking organizations
• 1 blog post analyzing advertising messages from fake line account

​

Displaying one reply

64

Someone on the Internet reacts to your message:

To sum up, the reply author thinks it contains misinformation.

​

The information above is provided by a kind hearted person. Please make your own judgements according to the reasoning and the given references.

​

If you have different thoughts, you are welcomed to input new replies here:

Fact check text

Reference

Usefulness feedback

65

Is the reply above useful?

Y/N

Crowdsourcing without background checks

How do Cofacts deal with subjective opinions?

Mechanisms／

FAQ

68

69

Handling an opinion piece

70

OOOO is definitely going to fail!

OOOO must be really evil to do this!

Replying an opinion piece

71

Handling an opinion piece

72

Different view�OOOO’s design can handle failure...

Different view

OOOO may not be evil if you look at them this way...

73

Profile page, Personal contribution visualization

74

Disinformation fighters in Taiwan

Fact-checking Organizations

Media literacy education

Researchers /�Think-tanks

🔗 bit.ly/cofacts20241018

USER

AUTO-REPLY

CHATBOT

MESSAGE

REPLY

REPLY

WEB APP

FACT

CHECKERS

77

What does Cofacts do？

Cofacts is devoted to creating technology tools for defeating disinformation; the whole circumstance is transparent and liberal; everyone could be the contributor while they do vetting others’replies or fact-checking on this platform.

Open source project makes more collaboration

�

Innovative civic tech community building for democratic society

�

Free and Open, independent from authorities or rules��

78

Collaboration

Cofacts engineers provide all of the source code on GutHUB and toolKit ; let everyone launch your fact-checking chatbot on your laptop.

​

Lesson

Learning

Software developing

AI, Machine learning

Cultivate

Develop

Feedback

Answer

Request

Engineers

Users

Verify Hoax bots

Provides bilingual ToolKit

​

Train

Factcheckers

Rumor Cases

Smearing the integrity of Taiwanese election

​

Outdated information on the previous election process, which has been amended in 2020:

• https://cofacts.tw/article/ks4f1udjczdh
• https://cofacts.tw/article/exj2i2pci359
• https://cofacts.tw/article/1v9n3aaofsjjn

​

Election “fraud” (but there are no fraud)

• https://cofacts.tw/article/_PpmcYQBC7Q3lHuUzSXn
• https://cofacts.tw/article/04ubOIQBv5it-Cx_C6yg
• https://cofacts.tw/article/j4sYZoQBv5it-Cx_3do4

​

Weird “tips” on voting, disincentivize the general public to vote

• https://cofacts.tw/article/4m9gqn1tmm1t
• https://cofacts.tw/article/2itdqim7s5rx0
• https://cofacts.tw/article/y5hfzt46kyna
• https://cofacts.tw/article/cdvw7jruku7r
• https://cofacts.tw/article/ysw9jviejn7e

Opinions around TSMC Arizona Fab

Behind the news that TSMC Arizona Fab hosts the tool-in ceremony, public opinions on Taiwanese engineers flying to the U.S. went viral in closed messaging chat rooms; one of which even contains disinformation, stating that the move is financially backed by the Taiwanese government. Such opinion quickly gained popularity, in fear of Taiwan’s “Silicon Shield” being removed by the U.S.

​

Fact-checking collaborators responded with objective facts including the size of the whole TSMC and TSMC’s future fab building plans in Taiwan.

​

• https://cofacts.tw/article/n2lwozpsf48s
• https://cofacts.tw/article/ufvv02sn7scp
• https://cofacts.tw/article/23z9tfoelebn3 (Disinformation)
• https://cofacts.tw/article/k60yedwgftoy
• https://cofacts.tw/article/24a1t4uh32mhg
• https://cofacts.tw/article/3ly6be27z5d6u
• https://cofacts.tw/article/2_oJjoQBC7Q3lHuUGkLJ

87

Disclaimer

• Cases above are cherry-picked.

​

• Perceived popularity can be affected by subjective importance

​

• Actual stats available at https://cofacts.tw/analytics

88

Good Experiences

AI categorized

Look up Record

Feedback collected

�

LINE Notification

�

RSS Feed

�

Statistics & Analytics

Gamification�

Image checked�

Technical support

Database

89

Pain Points

Developing Process

All Volunteers

Negative

Effects

Regulation changed

burn out issues

Responsibility

Lack of contributors

No Commitment

Priority Option

Civic Tech Issues

Request from large tech companies

​

System Challenges�Free Riders Issues

Motivation Ruin Issues

Collaboration site

Submitted suspicious messages from Cofacts chatbot would be delivered to collaborating website, open and transparent to everyone.

​

�AI classification system helps distribute different topic for each professional volunteers.

Divese replies supported

Voluntary fact-checkers could submit debunking process and references for the susicious messages.

​

​

Chatbot users and website browsers would give up vote or down vote to each fact-checking .

​

Citing or supplement are encouraged. Everyone would be the collaborators in this system.

Notify users new fact-check

Not replied yet messages would reach the users while it has got fact-checked already!��Cofacts Chatbot will inform you if your former request got solved.

​

Let the chatbot users receive the first hand fact-check report immediately.

Fact-checkers submited a new repliy for you!

See new replies

Cofacts toolkit

​

Editor / Chatbot user documentation

Design decisions�

​

🔗 cofacts.g0v.tw/hack

AI Catagorize

1.Filter�2.View the TOPIC�3.Choose what you’re interested in.�4.Catagorization

TinEye

Fact-check Community

g0v slack # cofacts

Show your thankfulness to the collaborators�Communicate with fact-checkers regularily

Watch the contribution list every week

Collaborate with them, not supervise them

Do what you want people do

Trust the contributors who help you

​

USER

AUTO-REPLY

CHATBOT

MESSAGE

REPLY

REPLY

WEB APP

FACT

CHECKERS

# of people used LINE in past 7 days

Source: https://meet.bnext.com.tw/articles/view/38786. Survey conducted by Nielsen Taiwan.

(% Taiwanese population, in 2016)

New to the Internet

Cofacts 真的假的 adventages

Instant disinformation got replied within the chat app

Chatbot automatically respond debunked hoax 24/7

Empowers the citizens to respond to hoax together, in a way that embraces diversity