R code for Twitter Scrape and Estimation

Scraping Twitter

Obtain codes for scraping

Determine timing and content

Twitter_edge

Twitter_node

Int_edgelist

Int_nodelist

Keywork plot

Tweet count plot

​

Visualizations

Dynamic Visualizations (code)

Influence model (just code)

Latent space estimation of the Influence Model

Selection (just code)

Selection: Latent factor selection models in R

KliqueFinder (code)

basic

advanced

Converting data to integer edgelist

Other data source

Toy data

Toyatt.csv, Toynet.csv

snapshots

​

movies

​

Toy data

ati19x_node.csv

ati19x_tie.csv

Crystalized sociograms

​

Parameter estimates

​

Parameter estimates

​

Introduction to ideas and term of social networks

Centrality, Ethics and Resources

Concerns about causality for influence

Code must be adapted for scraped data

Scraping Twitter in R

• Check out Chapter 11 of General introduction to R
• Get developer access here
• May take 24 hours
• Rmarkdown for scraping twitter

​

​

​

Getting access codes

Copy all these and save them to paste into R code

Setting up tokens: fill in the “” in Scraping Twitter:�Please download and save to a file on your machine�

Conceptualization: Network analysis is about what flows�

• What flows through twitter?
• Changes in beliefs?
• Behaviors?
• Level of activity?
• Use of language?

​

6

Conceptualization: Think about the network model of influence

7

Y_i’t-1

W_{ii’t-1🡪t}

Y_it

t

t-1

B

C

D

Latent space estimation of the Influence Model

Rmarkdown version of latent space influence

Process for Choosing a topic

• Twitter ethnography:
• Look at popular media
• Look at tweets
• Keywords vs hashtags
• Identify key actors (@somebody)
• Advanced searches
• Narrow down:
• Geography: but Twitter is global
• By timing
• By participants

​

​

8

home

reflection

Conceptualization: Think about the process of selection

9

t

t-1

W_{ii’t-1🡪t}

C

B

C

D

Part 3: Selection: Latent factor selection models in R,

Data: Toyatt.csv, Toynet.csv

Rmarkdown of latent selection

R Code for Latent Factor Selection with toy data

Choose your keywords

10

home

reflection

Timing: Think about the network model of influence

11

Y_i’t-1

W_{ii’t-1🡪t}

Y_it

t

t-1

B

C

D

Latent space estimation of the Influence Model

Rmarkdown version of latent space influence

Timing The Formal Model of Influence -- the Network Effect

• w_{ii’t-1🡪t}

Interactions between i and i’ between t-1 and t.

• y_it
• Outcome (dependent variable). An attitude or behavior of actor i at time t.
• ∑_i’ w_{ii’t-1🡪t}y_i’t-1.
• Exposure. Sum of attributes of others with whom the actor interacted between t and t-1.
• y_it = ρ∑_i’w_{ii’t-1🡪t}y_i’t-1 +γ y_it-1 +e_it.
• Model. Errors are assumed iid normal, with mean zero and variance (σ2).

12

Scrape Twitter Timeline for Diffusion:�See Influence Model

Pre interaction

sentiments

T1

T0

Interaction interval

Mention, retweet, reply

T2

Post interaction

sentiments

time_interval1

time_interval2

time_interval3

Influence model:

Ppt

Concerns about causality

Software: spss, R, Stata

With latent space in R

T3

Y_it

W_{ii’t-1🡪t}

Y_i’t-1

Time interval code

Check size of scrape

Keyword plot over time

Reduce size as needed: scrape can take a long time

Execute full tweet, produces edgelist and node data

Scraping Twitter

Twitter_edge

Twitter_node

Int_edgelist

Int_nodelist