Analyzing Polarization in Social Media:�Method and Application to Tweets on 21 Mass Shootings

Dora Demszky, Nikhil Garg, Rob Voigt,

James Zou, Matthew Gentzkow, Jesse Shapiro, Dan Jurafsky

​

​

NAACL-HLT 2019

June 5, 2019

Political Polarization

• public opinion in the US is highly polarized ideologically (Lelkes, 2016; Boxell et al., 2017)

(Brady et al., 2017)

Mass shootings

• gun violence is closely tied to partisan debates in the US

Is there a way to measure mass polarization in language?

@#?!

%*?!

Is there a way to measure mass polarization in language?

Can we measure polarization with respect to...

Is there a way to measure mass polarization in language?

what topic is being discussed?

Can we measure polarization with respect to...

Is there a way to measure mass polarization in language?

what topic is being discussed?

how is something talked about?

Can we measure polarization with respect to...

Is there a way to measure mass polarization in language?

what topic is being discussed?

how is something talked about?

Can we measure polarization with respect to...

labeling

Is there a way to measure mass polarization in language?

what topic is being discussed?

how is something talked about?

Can we measure polarization with respect to...

labeling

grounding

Is there a way to measure mass polarization in language?

what topic is being discussed?

how is something talked about?

Can we measure polarization with respect to...

labeling

grounding

framing(?)

Is there a way to measure mass polarization in language?

what topic is being discussed?

how is something talked about?

Can we measure polarization with respect to...

affect

illocutionary force

labeling

grounding

If time in Q & A

Data

• 4.4M tweets on 21 mass shootings between 2015 and 2018
• collected from archived tweets & Twitter API

Partisanship Assignment

1. compiled a list of Twitter accounts of Dem / Rep politicians
2. users are assigned to a party if they follow more politicians from that party than the other
3. we validated this method by comparing the state-level proportion of Republicans assigned by our method with the state-level Republican vote share in the 2016 elections

Partisanship Assignment

• compiled a list of Twitter accounts of Dem / Rep politicians
• users are assigned to a party if they follow more politicians from that party than the other
• we validated this method by comparing the state-level proportion of Republicans assigned by our method with the state-level Republican vote share in the 2016 elections

high correlation!

(adjusted R² = .82)

Is there a way to measure mass polarization in language?

@#?!

%*?!

Leave-Out Estimate of Polarization

Expected posterior probability that an observer with a neutral prior would assign to a user’s true party after observing a single token by the user.

Leave-Out Estimate of Polarization

Expected posterior probability that an observer with a neutral prior would assign to a user’s true party after observing a single token by the user.

This measure gives an expected value over the distribution of users & the vocabulary.

Leave-Out Estimate of Polarization

✕ ∑ _{p ∈ {D, R}} ✕ ∑ _{u ∈ p} ⋅

c₁

c₂

​

​

c_n

u a

1

|p|

Expected posterior probability that an observer with a neutral prior would assign to a user’s true party after observing a single token by the user.

p₁

p₂

​

​

p_n

f₁

f₂

​

​

f_n

u

token distribution vector of user u

N = vocabulary size

Leave-Out Estimate of Polarization

✕ ∑ _{p ∈ {D, R}} ✕ ∑ _{u ∈ p} ⋅

c₁

c₂

​

​

c_n

u a

1

|p|

Expected posterior probability that an observer with a neutral prior would assign to a user’s true party after observing a single token by the user.

p₁

p₂

​

​

p_n

vector of association scores between a party and each token

N = vocabulary size

Leave-Out Estimate of Polarization

✕ ∑ _{p ∈ {D, R}} ✕ ∑ _{u ∈ p} ⋅

c₁

c₂

​

​

c_n

u a

1

|p|

Expected posterior probability that an observer with a neutral prior would assign to a user’s true party after observing a single token by the user.

p₁

p₂

​

​

p_n

vector of association scores between a party and each token

in our case:�posterior probability

N = vocabulary size

Leave-Out Estimate of Polarization

✕ ∑ _{p ∈ {D, R}} ✕ ∑ _{u ∈ p} ⋅

c₁

c₂

​

​

c_n

u a^P

1

|p|

Expected posterior probability that an observer with a neutral prior would assign to a user’s true party after observing a single token by the user.

p₁

p₂

​

​

p_n

f₁

f₂

​

​

f_n

u

N = vocabulary size

calculate for user’s party P

Leave-Out Estimate of Polarization

✕ ∑ _{p ∈ {D, R}} ✕ ∑ _{u ∈ p} ⋅

c₁

c₂

​

​

c_n

u a^P

1

|p|

Expected posterior probability that an observer with a neutral prior would assign to a user’s true party after observing a single token by the user.

p₁

p₂

​

​

p_n

f₁

f₂

​

​

f_n

u

exclude user u�to eliminate bias

N = vocabulary size

-u

Leave-Out Estimate of Polarization

✕ ∑ _{p ∈ {D, R}} ✕ ∑ _{u ∈ p} ⋅

c₁

c₂

​

​

c_n

u a^P

1

|p|

Expected posterior probability that an observer with a neutral prior would assign to a user’s true party after observing a single token by the user.

p₁

p₂

​

​

p_n

f₁

f₂

​

​

f_n

u

N = vocabulary size

-u

✕ ∑ _{p ∈ {D, R}} ✕ ∑ _{u ∈ p} ⋅

Leave-Out Estimate of Polarization

✕ ∑ _{p ∈ {D, R}} ✕ ∑ _{u ∈ P} ⋅

c₁

c₂

​

​

c_n

u a^P

1

|P|

Expected posterior probability that an observer with a neutral prior would assign to a user’s true party after observing a single token by the user.

p₁

p₂

​

​

p_n

f₁

f₂

​

​

f_n

u

N = vocabulary size

-u

Leave-Out Estimate of Polarization

✕ ∑ _{P ∈ {D, R}} ✕ ∑ _{u ∈ P} ⋅

c₁

c₂

​

​

c_n

u a^P

1

|P|

Expected posterior probability that an observer with a neutral prior would assign to a user’s true party after observing a single token by the user.

p₁

p₂

​

​

p_n

f₁

f₂

​

​

f_n

u

N = vocabulary size

-u

1

2

values above .52 are highly polarized! (US congress in recent years ≈ .53)

Leave-Out Estimate of Polarization

Results

Leave-Out Estimate of Polarization

Results

Leave-Out Estimate of Polarization

Results

polarization of the US congress in recent years

(Gentzkow et al., forthcoming)

​

Leave-Out Estimate of Polarization

Results

polarization of the US congress in recent years

(Gentzkow et al., forthcoming)

​

• discussion of each event is highly polarized!

​

Is there a way to measure mass polarization in language?

what topic is being discussed?

Can we measure polarization with respect to...

Is there a way to measure mass polarization in language?

what topic is being discussed?

Can we measure polarization with respect to...

topic induction method?

LDA-Based Approaches

• LDA (Blei et al, 2003), Biterm Topic Model (BTM) (Yan et al., 2013)

​

LDA-Based Approaches

• LDA (Blei et al, 2003), Biterm Topic Model (BTM) (Yan et al., 2013)
• they capture event-specific topics

​

LDA-Based Approaches

• LDA (Blei et al, 2003), Biterm Topic Model (BTM) (Yan et al., 2013)
• they capture event-specific topics
• high probability words are frequent words

Our Method: Clustering for Nested Data (CND)

• need topics that capture high-level themes across events
• our method:
1. sample an equal # of tweets from all events
2. build a vocabulary V of word stems that occur at least 10 times in at least 3 events
3. train GloVe embeddings for V (Pennington et al., 2014)
4. create tweet embeddings (Arora et al., 2017)
5. cluster embeddings via k-means, using cosine-similarity

Topic Model Evaluation

• Number of topics: 6 – 10

Topic Model Evaluation

• Number of topics: 6 – 10
• Tasks:
• word intrusion (Chang et al., 2009)

honor honour knife vigil memory capitol

Pick the odd one out:

Topic Model Evaluation

• Number of topics: 6 – 10
• Tasks:
• word intrusion (Chang et al., 2009)
• we introduce tweet intrusion

tweet₁ tweet₂ tweet₃ tweet₄

Pick the odd one out:

Topic Model Evaluation

• Number of topics: 6 – 10
• Tasks:
• word intrusion (Chang et al., 2009)
• we introduce tweet intrusion

8 Topics

More Republican vs Democrat topics

Is there a way to measure mass polarization in language?

what topic is being discussed?

how is something talked about?

Can we measure polarization with respect to...

labeling

Labeling

• essentialist noun labels (as opposed to descriptive action verbs) imply strong and stable attitudes (Walton & Banaji, 2004)
• labels that are often used to describe perpetrators of mass shootings:
• shooter
• terrorist
• lone wolf
• crazy person

“Terrorist”

• an ideologically charged label
• found to be used differentially by the news media when describing shooters of different races (Duxbury et al, 2018)

“Terrorist”

• an ideologically charged label
• found to be used differentially by the news media when describing shooters of different races (Duxbury et al, 2018)

“Terrorist”

• an ideologically charged label
• found to be used differentially by the news media when describing shooters of different races (Gade et al., 2018)

Shooter’s race:

“Terrorist”

Shooter’s race:

for all events where the shooter is white, Democrats are more likely to use the label “terrorist”

“Terrorist”

Shooter’s race:

for all events where the shooter is African American or Middle Eastern, Republicans are more likely to use the label “terrorist”

“Terrorist”

Shooter’s race:

“terrorist” is the token which shows the greatest difference in terms of its partisanship based on the shooter’s race

Is there a way to measure mass polarization in language?

what topic is being discussed?

how is something talked about?

Can we measure polarization with respect to...

labeling

grounding

Framing Differences: Grounding

• event contextualization among other events of mass violence

​

America did nothing after BABIES were slaughtered at Sandy Hook so why Orlando? @BarackObama, please. Let #GunControl be your final mic drop

Framing Differences: Grounding

• event contextualization among other events of mass violence

​

America did nothing after BABIES were slaughtered at Sandy Hook so why Orlando? @BarackObama, please. Let #GunControl be your final mic drop

This is second #Muslim attack on #USA after #9/11, #USA needs a permanent solution for Muslims #OrlandoShooting #OrlandoMassacre

Framing Differences: Grounding

• event contextualization among other events of mass violence
• measure: partisan log odds ratio of context event mentions

Framing Differences: Grounding

• event contextualization among other events of mass violence
• measure: partisan log odds ratio of context event mentions

• shootings at schools and places of worship are more likely to be mentioned as context events by Democrats

Framing Differences: Grounding

• event contextualization among other events of mass violence
• measure: partisan log odds ratio of context event mentions

• shootings by people of color are more likely to be mentioned by Republicans

Framing Differences: Grounding

• event contextualization among other events of mass violence
• measure: partisan log odds ratio of context event mentions

partisan context event clusters

Summary

Methods

Findings

Can we measure mass polarization in language?

Methods

Findings

Can we measure mass polarization in language?

Methods

Findings

leave-out estimator of polarization

Can we measure mass polarization in language?

leave-out estimator of polarization

reactions to mass shootings are highly polarized

Methods

Findings

Can we measure polarization in topic choice?

Methods

Findings

leave-out estimator of polarization

reactions to mass shootings are highly polarized

Can we measure polarization in topic choice?

Methods

Findings

Clustering for Nested Data

leave-out estimator of polarization

reactions to mass shootings are highly polarized

Can we measure polarization in topic choice?

Methods

Findings

Clustering for Nested Data

shooter-related and event-specific topics are more Republican, solidarity and policy-related topics are more Democrat

leave-out estimator of polarization

reactions to mass shootings are highly polarized

Can we measure polarization in labeling?

Methods

Findings

Clustering for Nested Data

shooter-related and event-specific topics are more Republican, solidarity and policy-related topics are more Democrat

leave-out estimator of polarization

reactions to mass shootings are highly polarized

Can we measure polarization in labeling?

Methods

Findings

Clustering for Nested Data

shooter-related and event-specific topics are more Republican, solidarity and policy-related topics are more Democrat

identification of salient labels

leave-out estimator of polarization

reactions to mass shootings are highly polarized

Can we measure polarization in labeling?

Methods

Findings

Clustering for Nested Data

shooter-related and event-specific topics are more Republican, solidarity and policy-related topics are more Democrat

identification of salient labels

race of the shooter influences label use

leave-out estimator of polarization

reactions to mass shootings are highly polarized

Can we measure polarization in grounding?

Methods

Findings

Clustering for Nested Data

shooter-related and event-specific topics are more Republican, solidarity and policy-related topics are more Democrat

identification of salient labels

race of the shooter influences label use

leave-out estimator of polarization

reactions to mass shootings are highly polarized

Can we measure polarization in grounding?

Methods

Findings

Clustering for Nested Data

shooter-related and event-specific topics are more Republican, solidarity and policy-related topics are more Democrat

identification of salient labels

race of the shooter influences label use

look at mentions of relevant entities / events

two partisan clusters of event grounding (school shootings vs shootings by POC)

leave-out estimator of polarization

reactions to mass shootings are highly polarized

Can we measure polarization in affect?

Methods

Findings

Clustering for Nested Data

shooter-related and event-specific topics are more Republican, solidarity and policy-related topics are more Democrat

identification of salient labels

race of the shooter influences label use

look at mentions of relevant entities / events

two partisan clusters of event grounding (school shootings vs shootings by POC)

leave-out estimator of polarization

reactions to mass shootings are highly polarized

Can we measure polarization in affect?

Methods

Findings

Clustering for Nested Data

shooter-related and event-specific topics are more Republican, solidarity and policy-related topics are more Democrat

identification of salient labels

race of the shooter influences label use

look at mentions of relevant entities / events

two partisan clusters of event grounding (school shootings vs shootings by POC)

embedding-based affect lexicon filtering

leave-out estimator of polarization

reactions to mass shootings are highly polarized

Can we measure polarization in affect?

Methods

Findings

Clustering for Nested Data

shooter-related and event-specific topics are more Republican, solidarity and policy-related topics are more Democrat

identification of salient labels

race of the shooter influences label use

look at mentions of relevant entities / events

two partisan clusters of event grounding (school shootings vs shootings by POC)

embedding-based affect lexicon filtering

fear and disgust are more likely to be expressed by Republicans and sadness by Democrats

leave-out estimator of polarization

reactions to mass shootings are highly polarized

Can we measure polarization in illocutionary force?

Methods

Findings

Clustering for Nested Data

shooter-related and event-specific topics are more Republican, solidarity and policy-related topics are more Democrat

identification of salient labels

race of the shooter influences label use

look at mentions of relevant entities / events

two partisan clusters of event grounding (school shootings vs shootings by POC)

embedding-based affect lexicon filtering

fear and disgust are more likely to be expressed by Republicans and sadness by Democrats

leave-out estimator of polarization

reactions to mass shootings are highly polarized

Can we measure polarization in illocutionary force?

Methods

Findings

Clustering for Nested Data

shooter-related and event-specific topics are more Republican, solidarity and policy-related topics are more Democrat

identification of salient labels

race of the shooter influences label use

look at mentions of relevant entities / events

two partisan clusters of event grounding (school shootings vs shootings by POC)

embedding-based affect lexicon filtering

fear and disgust are more likely to be expressed by Republicans and sadness by Democrats

necessity modals

leave-out estimator of polarization

reactions to mass shootings are highly polarized

Can we measure polarization in illocutionary force?

Methods

Findings

Clustering for Nested Data

shooter-related and event-specific topics are more Republican, solidarity and policy-related topics are more Democrat

identification of salient labels

race of the shooter influences label use

look at mentions of relevant entities / events

two partisan clusters of event grounding (school shootings vs shootings by POC)

embedding-based affect lexicon filtering

fear and disgust are more likely to be expressed by Republicans and sadness by Democrats

necessity modals

Democrats are more likely to make calls for action and assign blame than Republicans

leave-out estimator of polarization

reactions to mass shootings are highly polarized

Thank you!

Dan Jurafsky

Jesse Shapiro

Matthew Gentzkow

Nikhil Garg

Thanks for feedback: Chris Potts, Cleo Condoravdi, Stanford NLP Group, Zion Mengesha & many others

Rob Voigt

James Zou

References

Boxell, L., Gentzkow, M., & Shapiro, J. M. (2017). Is the internet causing political polarization? Evidence from demographics (No. w23258). National Bureau of Economic Research.

​

Brady, W. J., Wills, J. A., Jost, J. T., Tucker, J. A., & Van Bavel, J. J. (2017). Emotion shapes the diffusion of moralized content in social networks. Proceedings of the National Academy of Sciences, 114(28), 7313–7318.

​

Chang, J., Gerrish, S., Wang, C., Boyd-graber, J. L., & Blei, D. M. (2009). Reading Tea Leaves: How Humans Interpret Topic Models. Advances in Neural Information Processing Systems, 288–296.

​

Druckman, J. N., Peterson, E., & Slothuus, R. (2013). How Elite Partisan Polarization Affects Public Opinion Formation. American Political Science Review, 107(1), 57–79.

​

Duxbury, S. W., Frizzell, L. C., & Lindsay, S. L. (2018). Mental Illness, the Media, and the Moral Politics of Mass Violence: The Role of Race in Mass Shootings Coverage. Journal of Research in Crime and Delinquency, 55(6), 766–797.

​

Gentzkow, M., Shapiro, J., & Taddy, M. (Forthcoming).. Measuring Group Differences in High-Dimensional Choices: Method and Application to Congressional Speech. In Econometrica.

​

Golash-Boza, T. (2016). A Critical and Comprehensive Sociological Theory of Race and Racism. Sociology of Race and Ethnicity, 2(2), 129–141.

​

Kratzer, A. (2008). The Notional Category of Modality. In Formal Semantics (pp. 289–323).

​

Lelkes, Y. (2016). Mass Polarization: Manifestations and Measurements. Public Opinion Quarterly, 80(S1), 392–410.

​

Mohammad, S. M., & Turney, P. D. (2013). NRC Emotion Lexicon. National Research Council, Canada.

​

Walton, G. M., & Banaji, M. R. (2004). Being What You Say: The Effect of Essentialist Linguistic Labels on Preferences. Social Cognition, 22(2), 193–213.

​

​

Questions for Future Work

• How do these linguistic patterns (e.g. frames) arise? How are they taken up and propagate? Is it a bottom-up or top-down phenomenon, or both?
• How does the time of a mass shooting –– in relation to others –– change the discussion of the event?
• How does polarization emerge and manifest in the case of other events (e.g. political debates, sports events)?
• What can researchers do to mitigate this polarization?

“Terrorist”

Shooter’s race:

Critical race theoretical implications

• Democrats are more likely to de-racialize the “terrorist” controlling image
• Republicans are more likely to extend the “terrorist” controlling image

Affect Detection

• filtered the NRC Emotion lexicon (Mohammad and Turney, 2013) based on cosine-similarities of words in our embedding space

Affect

Affect

More Republican

More Democrat

Modality & Illocutionary Force

• modality is concerned with necessity and possibility (Kratzer, 2002)
• in the aftermath of a tragic event, people seek solutions (should) and assign blame (should have)

Necessity Modals

Partisanship of Modals

• all modals are more likely to be used by Democrats
• should is the only modal where the shooter’s race significantly affects partisanship (p < 0.03)
• the partisanship of should have is not different from that of should (all forms), but the effect of race disappears

Partisanship of Modals

Representations of Modals in Topics

Representations of Modals in Topics

• modals are disproportionately overrepresented in laws & policy → call for policy change

Modal collocations

Topic Partisanship

More Republican

More Democrat

Topic Partisanship

• investigation, news, shooter’s identity & ideology and solidarity show significant differences based on the shooter’s race

Other Dimensions

• The differences in Affect and Illocutionary Force are unaffected by the shooter’s race

“Crazi person” (stemmed)

• data sparsity (vocab frequency cutoff = 50 for each event)
• Democrats are more likely to use “crazy person” as a label for a shooter who is a person of color and Republicans for white shooters

Orlando

Parkland

Las Vegas

“Lone Wolf”

• data sparsity (vocab frequency cutoff = 50 for each event)
• Democrats are more likely to use “lone wolf” as a label for white shooters and Republicans for shooters who are Middle Eastern

Dallas �(African American shooter)

“Shooter”

“Shooter”

• more likely to be mentioned by Republicans regardless of the shooter’s race (accords with findings about topic partisanship)

More Republican vs Democrat topics

Post-Event Polarization

• discussions at the event-level are increasingly polarized over time (slope: .002, p < 0.05)

Topic Polarization Over Time (Vegas)

Topic Polarization Over Time (Orlando)

Tokens that are more likely to be used by�Democrats when the shooter is white and by�Republicans when the shooter is a person of color

terrorist, muslim, terror, call, guess, hey, white, black, mention, shooter kill, respons, peac, murder, back

Tokens that are more likely to be used by�Democrats when the shooter is white and by�Republicans when the shooter is a person of color

terrorist, muslim, terror, call, guess, hey,�white, black, mention, shooter kill, respons, peac, murder, back

Tokens that are more likely to be used by�Republicans when the shooter is white and by�Democrats when the shooter is a person of color

crazi, latest, #shoot, fatal, prayer victim, open, lot, detail, #break, open fire, incid, coverag, rt, victim famili, local, area, scene, wrong, peopl dead, offic, affect, situat, cnn, 1, involv, famili victim, leav, year, 8, hour, citi, news, updat, suspect, start, turn, dead, tonight, peopl kill, stori, hurt, bad, victim, famili, watch, injur, person, fire, activ, gunman, twitter, peopl shot, fact, find, pray, post, 3, state, show

Significantly Republican Words Across Events

free, obama, left, fox news, ap, fox, video, blame, #break, detail, motiv, updat, break news, secur, identifi, press, break, report, confirm, media, killer, 6, polic, multipl, god, target, attack, involv, crime, respond, injur, scene, suspect, activ shooter, offici, shooter, activ

Overall

Significantly Republican Words Across Events

free, left, #break, ap, #new, updat, detail, blame, releas, video, wit, fox news, insid, prayer victim, obama, confirm, polit, break, break news, press, prayer famili, sourc, polic, fox, involv, identifi, report, secur, pray investig, scene, target, injur, latest, suspect, #shoot, multipl, custodi, media, open fire,�activ shooter, crazi

free, obama, left, fox news, ap, fox, video, blame, #break, detail, motiv, updat, break news, secur, identifi, press, break, report, confirm, media, killer, 6, polic, multipl, god, target, attack, involv, crime, respond, injur, scene, suspect, activ shooter, offici, shooter, activ

Overall

When the Shooter is White

Significantly Republican Words Across Events

free, left, #break, ap, #new, updat, detail, blame, releas, video, wit, fox news, insid, prayer victim, obama, confirm, polit, break, break news, press, prayer famili, sourc, polic, fox, involv, identifi, report, secur, pray investig, scene, target, injur, latest, suspect, #shoot, multipl, custodi, media, open fire,�crazi, activ shooter

free, obama, left, fox news, ap, fox, video, blame, #break, detail, motiv, updat, break news, secur, identifi, press, break, report, confirm, media, killer, 6, polic, multipl, god, target, attack, involv, crime, respond, injur, scene, suspect, activ shooter, offici, shooter, activ

Overall

When the Shooter is White

little difference!

Significantly Democrat Words Across Events

gun violenc, mass shoot, violenc, fuck, america, mass, heart, send, safe, feel, male, mental, countri, love, thought, shit, american, talk, damn, week, word, read, white, ppl, day, thought prayer, end, trump, tweet, happen, care, gun, stay, hear, act, close, sad, good, hey, today, shoot, friend, home, shot kill, lost, wait, ...

Overall

Significantly Democrat Words Across Events

gun violenc, mass shoot, nra, mental, violenc, #guncontrol, mass, heart, senseless, love, tragic, fuck, communiti, feel, america, thought, end, safe, tragedi, countri, ppl, happen, sad, day, read, week, send, damn, lost, shit, open, shoot, stori, hear, stay, friend, close, year, die, famili, today,

talk, gun, victim,...

Overall

When the Shooter is Non-White

gun violenc, mass shoot, violenc, fuck, america, mass, heart, send, safe, feel, male, mental, countri, love, thought, shit, american, talk, damn, week, word, read, white, ppl, day, thought prayer, end, trump, tweet, happen, care, gun, stay, hear, act, close, sad, good, hey, today, shoot, friend, home, shot kill, lost, wait, ...

Significantly Democrat Words Across Events

gun violenc, mass shoot, nra, mental, violenc, #guncontrol, mass, heart, senseless, love, tragic, fuck, communiti, feel, america, thought, end, safe, tragedi, countri, ppl, happen, sad, day, read, week, send, damn, lost, shit, open, shoot, stori, hear, stay, friend, close, year, die, famili, today,

talk, gun, victim,...

Overall

When the Shooter is Non-White

gun violenc, mass shoot, violenc, fuck, america, mass, heart, send, safe, feel, male, mental, countri, love, thought, shit, american, talk, damn, week, word, read, white, ppl, day, thought prayer, end, trump, tweet, happen, care, gun, stay, hear, act, close, sad, good, hey, today, shoot, friend, home, shot kill, lost, wait, ...

little difference!

Significantly Republican Words Across Events

free, obama, left, fox news, ap, fox, video, blame, #break, detail, motiv, updat, break news, secur, identifi, press, break, report, confirm, media, killer, 6, polic, multipl, god, target, attack, involv, crime, respond, injur, scene, suspect, activ shooter, offici, shooter, activ

Overall

Significantly Republican Words Across Events

obama, terrorist attack, free, terrorist, muslim, fox news, fox, attack, killer, shooter kill, motiv, ap, call, god, cop, respond

free, obama, left, fox news, ap, fox, video, blame, #break, detail, motiv, updat, break news, secur, identifi, press, break, report, confirm, media, killer, 6, polic, multipl, god, target, attack, involv, crime, respond, injur, scene, suspect, activ shooter, offici, shooter, activ

Overall

When the Shooter is Non-White

Significantly Republican Words Across Events

obama, terrorist attack, free, terrorist, muslim, fox news, fox, attack, killer, shooter kill, motiv, ap, call, god, cop, respond

free, obama, left, fox news, ap, fox, video, blame, #break, detail, motiv, updat, break news, secur, identifi, press, break, report, confirm, media, killer, 6, polic, multipl, god, target, attack, involv, crime, respond, injur, scene, suspect, activ shooter, offici, shooter, activ

Overall

When the Shooter is Non-White

TERRORISM

Significantly Democrat Words Across Events

gun violenc, mass shoot, violenc, fuck, america, mass, heart, send, safe, feel, male, mental, countri, love, thought, shit, american, talk, damn, week, word, read, white, ppl, day, thought prayer, end, trump, tweet, happen, care, gun, stay, hear, act, close, sad, good, hey, today, shoot, friend, home, shot kill, lost, wait, ...

Overall

Significantly Democrat Words Across Events

gun violenc, fuck, terrorist, male, white, mass shoot, violenc, nra, america, american, terror, send, #guncontrol, terrorist attack, won, word, safe, hey, talk, countri, shit, feel, trump, folk, thought prayer, damn, thought, mass, tweet, heart, week, read, act, wow, day, care, love,

time

Overall

When the Shooter is White

gun violenc, mass shoot, violenc, fuck, america, mass, heart, send, safe, feel, male, mental, countri, love, thought, shit, american, talk, damn, week, word, read, white, ppl, day, thought prayer, end, trump, tweet, happen, care, gun, stay, hear, act, close, sad, good, hey, today, shoot, friend, home, shot kill, lost, wait, ...

Significantly Democrat Words Across Events

gun violenc, fuck, terrorist, male, white, mass shoot, violenc, nra, america, american, terror, send, #guncontrol, terrorist attack, won, word, safe, hey, talk, countri, shit, feel, trump, folk, thought prayer, damn, thought, mass, tweet, heart, week, read, act, wow, day, care, love,

time

Overall

When the Shooter is White

gun violenc, mass shoot, violenc, fuck, america, mass, heart, send, safe, feel, male, mental, countri, love, thought, shit, american, talk, damn, week, word, read, white, ppl, day, thought prayer, end, trump, tweet, happen, care, gun, stay, hear, act, close, sad, good, hey, today, shoot, friend, home, shot kill, lost, wait, ...

TERRORISM

Which words show systematic partisan differences across events?

free, obama, left, fox news, ap, fox, video, blame, #break, detail, motiv, updat, break news, secur, identifi, press, break, report, confirm, media, killer, 6, polic, multipl, god, target, attack, involv, crime, respond, injur, scene, suspect, activ shooter, offici, shooter, activ

gun violenc, mass shoot, violenc, fuck, america, mass, heart, send, safe, feel, male, mental, countri, love, thought, shit, american, talk, damn, week, word, read, white, ppl, day, thought prayer, end, trump, tweet, happen, care, gun, stay, hear, act, close, sad, good, hey, today, shoot, friend, home, shot kill, lost, wait, ...

→ consider words whose log odds ratio is significantly biased towards one party on average across all events

Too noisy to interpret, but noticeable patterns about...

free, obama, left, fox news, ap, fox, video, blame, #break, detail, motiv, updat, break news, secur, identifi, press, break, report, confirm, media, killer, 6, polic, multipl, god, target, attack, involv, crime, respond, injur, scene, suspect, activ shooter, offici, shooter, activ

gun violenc, mass shoot, violenc, fuck, america, mass, heart, send, safe, feel, male, mental, countri, love, thought, shit, american, talk, damn, week, word, read, white, ppl, day, thought prayer, end, trump, tweet, happen, care, gun, stay, hear, act, close, sad, good, hey, today, shoot, friend, home, shot kill, lost, wait, ...

Too noisy to interpret, but noticeable patterns about…�

TOPICS

free, obama, left, fox news, ap, fox, video, blame, #break, detail, motiv, updat, break, news, secur, identifi, press, break, report, confirm, media, killer, 6, polic, multipl, god, target, attack, involv, crime, respond, injur, scene, suspect, activ shooter, offici, shooter, activ

gun violenc, mass shoot, violenc, fuck, america, mass, heart, send, safe, feel, male, mental, countri, love, thought, shit, american, talk, damn, week, word, read, white, ppl, day, thought prayer, end, trump, tweet, happen, care, gun, stay, hear, act, close, sad, good, hey, today, shoot, friend, home,�shot kill, lost, wait, ...

gun control?

news?

Too noisy to interpret, but noticeable patterns about…�

AFFECT

free, obama, left, fox news, ap, fox, video, blame, #break, detail, motiv, updat, break news, secur, identifi, press, break, report, confirm, media, killer, 6, polic, multipl, god, target, attack, involv, crime, respond, injur, scene, suspect, activ shooter, offici, shooter, activ

gun violenc, mass shoot, violenc, fuck, america, mass, heart, send, safe, feel, male, mental, countri, love, thought, shit, american, talk, damn, week, word, read, white, ppl, day, thought prayer, end, trump, tweet, happen, care, gun, stay, hear, act, close, sad, good, hey, today, shoot, friend, home,�shot kill, lost, wait

more emotional?

more neutral?

Too noisy to interpret, but noticeable patterns about…�

ILLOCUTIONARY FORCE

free, obama, left, fox news, ap, fox, video, blame, #break, detail, motiv, updat, break news, secur, identifi, press, break, report, confirm, media, killer, 6, polic, multipl, god, target, attack, involv, crime, respond, injur, scene, suspect, activ shooter, offici, shooter, activ

gun violenc, mass shoot, violenc, fuck, america, mass, heart, send, safe, feel, male, mental, countri, love, thought, shit, american, talk, damn, week, word, read, white, ppl, day, thought prayer, end, trump, tweet, happen, care, gun, stay, hear, act, close, sad, good, hey, today, shoot, friend, home,�shot kill, lost, wait

calling for action?

asserting?