1 of 13

Twitter bot detection

ANTHONY RYBKA, ANH DUONG, GARY ELIANO, JOSEPH FRIEDMAN

2 of 13

Twitter has a bot problem

  • “Nearly half of Twitter accounts pushing to reopen America may be bots”
    • – MIT’s Technology Review
  • 25% of climate change denial tweets are posted by bots
    • – The Guardian
  • Social media bots spread misinformation and gain credibility by supporting each others’ posts
    • - Scientific American

3 of 13

What can we do?

  • We’ve designed our own system that can help determine whether an account may be a bot
  • This system uses KNN (K-Nearest Neighbor) analysis to determine what factors bot accounts have in common – and then uses that understanding to label other accounts as possibly being bots
  • Identifying bots is the first step in solving the greater problem

4 of 13

How does it work?

  • We applied for, and were granted, Twitter API access. This allows us to grab Tweet and user data right from the source.
  • That pipeline gives us rich data about Tweets and the people that Tweet them
  • We accumulate that data, make our own determinations on who’s a bot, and then “train” the KNN to recognize them

5 of 13

What makes a bot?

  • We had to be both delicate and firm in our selection criteria for tagging bot accounts
  • Bots have grown more sophisticated over time, and what’s suspicious to us may not be suspicious to a computer
  • There are some common behaviors:
    • Posting the same tweet from multiple accounts, sometimes simultaneously
    • “Too good to be true” – giving away money or promising something valuable
    • Directing users somewhere outside of Twitter – “click here to see”
    • Aggressive advertisements that circumvent Twitter’s rules or systems

6 of 13

What makes a human?

  • Much more difficult
  • People can be popular on Twitter, or not interacted with at all
  • Both people and companies are allowed to have accounts and advertise – doesn’t inherently make them a bot
  • Human accounts can really occupy any “use case” of Twitter – some may not ever set a profile picture, and some may not have any friends

​

7 of 13

What data are we working with?

  • Twitter gave us plenty of data that we’ve narrowed down to a handful of relevant columns
    • Username, tweet text, tweet URL – used for our manual review and not input into KNN
    • Tweet length, URL (b), media included (b)
    • Number of: followers, friends, statuses, and favorites
    • Profile: description (b), site URL (b), personal banner (b)

​

    • (b) means binary – 1 for present, 0 for not present
      • If someone did not include a personal URL in their profile, they would have a 0

8 of 13

Preparation

  • We manually reviewed 3,340 Tweets and coded them ourselves as bots or human
  • We shuffled all the tweets and grabbed 1,002 Tweets to “train” the KNN
  • The KNN knows our bot classifications for the training group, but not the test group
  • We then tasked the KNN with classifying the test group – which contains 2,339 Tweets

9 of 13

What did we search?

  • We combined searches across both normal and “high-risk” searches

​

    • Anh searched for giveaway, cashapp, address, and wallet
    • Gary searched for python and hacked
    • Joseph searched for car
    • Anthony searched for summer

10 of 13

How did we do?

Not bad! The concept works but needs additional tuning.

11 of 13

Examples!

We tagged this

as a bot, but the KNN

didn’t

The KNN tagged this as a bot,

but we believe it’s legitimate

Both us and the KNN

tagged this as a bot

12 of 13

What’s next?

  • Unfortunately, both Twitter’s standard and academic API do not allow us to specify when we want to pull Tweet data from
  • This means all of our Tweets are “most recent” for the given search
  • As a result, data such as likes, comments, and retweets for any given tweet was unreliable
  • This data would improve our system
  • A premium Twitter API starts at $150/month

13 of 13

Works cited

  • Hao, K. (2020, December 10). Nearly half of Twitter accounts pushing to reopen America may be bots. MIT Technology Review. https://www.technologyreview.com/2020/05/21/1002105/covid-bot-twitter-accounts-push-to-reopen-america/.
  • Menczer, F., & Hills, T. (2020, December 1). Information Overload Helps Fake News Spread, and Social Media Knows It. Scientific American. https://www.scientificamerican.com/article/information-overload-helps-fake-news-spread-and-social-media-knows-it/.
  • Milman, O. (2020, February 21). Revealed: quarter of all tweets about climate crisis produced by bots. The Guardian. https://www.theguardian.com/technology/2020/feb/21/climate-tweets-twitter-bots-analysis.

​