1 of 22

Fake tweets can be responsible for misinformation/disinformation spread.
Fake tweets are easy to create using online tools.
There are no tools currently available to evaluate the authenticity of attribution of screenshots.

Tarannum Zaki WSDL Research Group Expo 2024 Web Archives for Verifying Attribution in Twitter Screenshots @tarannum_zaki @WebSciDL

8 of 22

Aim

To develop a tool that would automatically provide a probability whether screenshot of a social media post was actually posted by the alleged author using the services of live web and web archives.

Tarannum Zaki WSDL Research Group Expo 2024 Web Archives for Verifying Attribution in Twitter Screenshots @tarannum_zaki @WebSciDL

9 of 22

To search for a tweet in the Wayback Machine, you must first know its URL

https://web.archive.org/web/20220323185843/https://twitter.com/annaturley/status/1506706947239817224

URL of the tweet: https://twitter.com/annaturley/status/1506706947239817224

https://web.archive.org/

Tarannum Zaki WSDL Research Group Expo 2024 Web Archives for Verifying Attribution in Twitter Screenshots @tarannum_zaki @WebSciDL

10 of 22

But, URL of a tweet is not present in most screenshots

Construction of a tweet URL

- Use the Twitter handle and approximate a time window based on the timestamp.

- Construct URL for the tweet.

- Search for the tweet in the Wayback Machine using the URL.

https://twitter.com/AaronBastani/status/1507391218854117377

@annaturley

March 23, 2022

March 25, 2022

https://twitter.com/TWITTER_HANDLE /status /TWEET_ID

https://web.archive.org/web/20220323185843/https://twitter.com/annaturley/status/1506706947239817224

Tweet ID encodes the timestamp of when the tweet was created

Tarannum Zaki WSDL Research Group Expo 2024 Web Archives for Verifying Attribution in Twitter Screenshots @tarannum_zaki @WebSciDL

11 of 22

Verifying if screenshot exists in the Wayback Machine

Tarannum Zaki WSDL Research Group Expo 2024 Web Archives for Verifying Attribution in Twitter Screenshots @tarannum_zaki @WebSciDL

12 of 22

Creating a dataset of screenshots collected from Twitter

Screenshot images shared on Twitter.
200 examples
Examples include both real and fake screenshots

Fields
Shared post’s URL	Original post’s URL
Category	Reason
Content category	Structural features
Post type	Social media
Search strategy	Annotated images
Screenshot	Remarks

https://ws-dl.blogspot.com/2022/12/2022-12-12-disinformation-spread-on.html

https://twitter.com/rvawonk/status/1503227687917305863

https://twitter.com/RealCandaceO/status/1501576352587292673

Category: Real

Reason: Found in the live web

Content category: Politics

Post Type: Tweet

Structural features: Single author, single post

Search strategy: Searched on Twitter interface

Social media: Twitter

Original post’s URL

Shared post’s URL

Screenshot

Tarannum Zaki WSDL Research Group Expo 2024 Web Archives for Verifying Attribution in Twitter Screenshots @tarannum_zaki @WebSciDL

13 of 22

OCRing screenshots: Single tweet images

Optical Character Recognition extracts information as text from digital image.

Example screenshot image

OCR extracted output

OCR

Twitter Handle

Timestamp

Tweet Text

Zaki, T., Nelson, M.L., and Weigle, M.C. (2023, Jun 14). Extracting Information from Twitter Screenshots. Tech Report arXiv:2306.08236. https://doi.org/10.48550/arXiv.2306.08236

Tarannum Zaki WSDL Research Group Expo 2024 Web Archives for Verifying Attribution in Twitter Screenshots @tarannum_zaki @WebSciDL

14 of 22

Computing a time window based on the screenshot timestamp

The maximum difference between two time zones on Earth is 26 hours.

Example screenshot image

OCR extracted output

Twitter handle and computed timestamps

Tarannum Zaki WSDL Research Group Expo 2024 Web Archives for Verifying Attribution in Twitter Screenshots @tarannum_zaki @WebSciDL

15 of 22

Using CDX API to retrieve archived tweets with left hand boundary

CDX API prefix search process

Twitter handle and computed timestamps

Output: Retrieved archived tweets with the left hand boundary(cropped).

request = "http://web.archive.org/cdx/search/cdx?url=" + urir + params

urir = "https://twitter.com/"+randyhillier+"/status"

params = "&matchType=prefix&from="+20220218154100

https://archive.org/help/wayback_api.php

Tarannum Zaki WSDL Research Group Expo 2024 Web Archives for Verifying Attribution in Twitter Screenshots @tarannum_zaki @WebSciDL

16 of 22

Extracting tweet IDs and determining tweet creation timestamp using TweetedAt

An archived tweet’s URL

Mapping between all the tweet IDs and tweet creation timestamps

https://web.archive.org/web/20220222163926/https://twitter.com/randyhillier/status/1006984708109099008

https://ws-dl.blogspot.com/2019/08/2019-08-03-tweetedat-finding-tweet.html

Each tweet ID encodes its creation timestamp

https://oduwsdl.github.io/tweetedat/#1006984708109099008

Tweet ID	Tweet Creation Date
1006984708109099008	20180613194037
…………	…………..

Tarannum Zaki WSDL Research Group Expo 2024 Web Archives for Verifying Attribution in Twitter Screenshots @tarannum_zaki @WebSciDL

17 of 22

Determining the final set of archived tweets by filtering the tweet creation timestamps within the time window

Output: 917 archived tweets with left hand boundary (cropped)

Mapping between tweet ID and tweet creation timestamp

Output: 29 archived tweets within 52 hours time window (cropped)

Tarannum Zaki WSDL Research Group Expo 2024 Web Archives for Verifying Attribution in Twitter Screenshots @tarannum_zaki @WebSciDL

Creation timestamp of tweets which does not fall within the 52 hours time window are filtered out.

449 archived tweets

Multiple mementos are filtered out.

29 archived tweets

18 of 22

Extracting tweet text from archived tweets using BeautifulSoup and Selenium

An archived tweet’s URL

Extracted text from archived tweet

HTML tag containing the tweet text

Selenium automates web scraping and BeautifulSoup parses text from HTML.

https://web.archive.org/web/20220220024223/https://twitter.com/randyhillier/status/1495226962058649603

TweetTextSize TweetTextSize--jumbo js-tweet-text tweet-text

https://www.selenium.dev/

https://pypi.org/project/beautifulsoup4/

Tarannum Zaki WSDL Research Group Expo 2024 Web Archives for Verifying Attribution in Twitter Screenshots @tarannum_zaki @WebSciDL

19 of 22

Computing text similarity score between tweet text from screenshot and archived tweets using Python’s difflib library

Example screenshot image

Extracted text from archived tweet

Extracted tweet text from screenshot

Text similarity score is computed based on longest common subsequence

A match score of 81.40% helps us to prove the existence of the screenshot tweet posted by the alleged author.

https://docs.python.org/3/library/difflib.html

match_score(Archived_Tweet_Text, Screenshot_Tweet_Text)= 81.40%

Archived_Tweet_Text1	Screenshot_Tweet_Text	match _score = 81.40%
Archived_Tweet_Text2	Screenshot_Tweet_Text	match_score = 30.78%
Archived_Tweet_Text3	Screenshot_Tweet_Text	match_score = 5.67%
……………..

Tarannum Zaki WSDL Research Group Expo 2024 Web Archives for Verifying Attribution in Twitter Screenshots @tarannum_zaki @WebSciDL

20 of 22

A threshold of 60% produced the highest F1 (0.69)

Experimented on 108 single tweet images from the collected dataset.

Threshold Value	Precision	Recall	F1 Score
90%	1.00	0.42	0.59
80%	1.00	0.49	0.66
70%	1.00	0.51	0.67
60%	1.00	0.53	0.69

Performance of the overlap between the tweet text from the screenshot and the archived tweets.

Tarannum Zaki WSDL Research Group Expo 2024 Web Archives for Verifying Attribution in Twitter Screenshots @tarannum_zaki @WebSciDL

21 of 22

Limitations & Future Work

Complex screenshot images

Extracted output mostly results in garbage value.

Tarannum Zaki WSDL Research Group Expo 2024 Web Archives for Verifying Attribution in Twitter Screenshots @tarannum_zaki @WebSciDL

OCR

22 of 22

Summary

Screenshots are an easy way to share content on social media.
Since screenshots can be easily faked, it is a critical task to detect a fabricated post.
Services of web archives could be useful to verify attribution of a screenshot by finding an archived version of the screenshot content.
Our research will mitigate misinformation and disinformation spread on social media.

Tarannum Zaki WSDL Research Group Expo 2024 Web Archives for Verifying Attribution in Twitter Screenshots @tarannum_zaki @WebSciDL