1 of 28

Revisiting key-key-words:

proposing a method for identifying unique

keywords in a collection of corpora

Mark McGlashan Alexandra Krendel

Mark.McGlashan@bcu.ac.uk a.krendel@lancaster.ac.uk

@Mark_McGlashan @ALexiconArtist

2 of 28

Outline

  1. Background

Keywords

Key-key-words

Identifying unique keywords

  • The study

The ‘manosphere’

  • Conclusions

@Mark_McGlashan | @ALexiconArtist

2

Content Warning

3 of 28

  1. Background�Keywords
  • Keywords being revisited
  • Known issues:
    • Researcher subjectivity
    • Text dispersion not accounted for in ‘traditional’ keyword approaches
    • Keywords may “overplay” differences rather than similarities between target and reference corpora

@Mark_McGlashan | @ALexiconartist

3

4 of 28

  1. Background�Keywords

Criteria for effective keyword analysis:

  • Content-distinctiveness

The strength of the relationship between a keyword and the content of the discourse domain represented by the target corpus, in contrast to all other discourse domains (represented by the reference corpus)

  • Content generalisibility

The degree to which a keyword represents the content of the discourse used across the full range of texts in the target corpus

(Egbert & Biber 2019)

@Mark_McGlashan | @ALexiconartist

4

5 of 28

  1. Background�Keywords

Say we have:

  • corpus y – reference corpus
  • corpus x – target corpus

To find keywords, we will typically:

  • Compare word frequencies in x against word frequencies in y
  • Derive a list of keywords
  • Interpret that list

@Mark_McGlashan | @ALexiconartist

5

y

x

keywords list

x

6 of 28

  1. Background�Keywords

But say x contains 2 subcorpora:

  • Subcorpus a
  • Subcorpus b

@Mark_McGlashan | @ALexiconartist

6

x

a

b

7 of 28

  1. Background�Keywords

To find keywords, we might:

  • Compare the total contents of x (a + b) against y
  • Derive keywords for the whole of x (thus everything that exists in a and b)
  • Interpret that list

@Mark_McGlashan | @ALexiconartist

7

a

b

x

y

x

keywords list

8 of 28

  1. Background�Keywords

…or we might compare a single subcorpus

  • and interpret/compare the resultant lists, which will tell us about keywords in each corpus
  • …but what about this bit?

@Mark_McGlashan | @ALexiconartist

8

x

a

b

y

b

a

a

b

a

keywords list

b

keywords list

9 of 28

  1. Background�Key-key-words

Key-key-words (Scott 1997)

  • How often words were shared across multiple keyword lists – a frequency list of keyword lists
  • Values shared between sets to find keywords shared across subcorpora
  • Intersect

@Mark_McGlashan | @ALexiconartist

9

b

a

10 of 28

  1. Background�Key-key-words

Key-key-words (Scott 1997)

  • Really interesting because it tells us about content-generalisability – how often keywords are key across subcorpora
  • But what about content-distinctiveness…?

@Mark_McGlashan | @ALexiconartist

10

b

a

11 of 28

  1. Background�Identifying unique keywords
  • Just as we can find values that intersect between lists, is it possible to find set differences?
    • i.e. all keywords in one subcorpus that are not found in the other
  • …is it possible to find keywords that are exclusive to a list/subcorpus?

@Mark_McGlashan | @ALexiconartist

11

a

b

12 of 28

  1. Background�Identifying unique keywords
  • Or relative complements
  • In other words, can we identify content-distinctive, unique keywords by focussing specifically on keywords that are not share

@Mark_McGlashan | @ALexiconartist

12

a

b

a

b

13 of 28

  1. Background�Key-key-words

This study explores the potential to extend Scott’s ideas around key-key-words to consider what values shared between keyword lists might tell us about saliency.

Moreover, the method proposed further considers what the exclusive occurrence of a keyword (as opposed to sharedness) might tell us about saliency.

@Mark_McGlashan | @ALexiconartist

13

14 of 28

  1. The studyThe ‘manosphere’

“The manosphere is an aggregate of diverse communities brought together by a common language that orients them in opposition to the discourse and rhetoric of feminism.” (Marwick & Caplan, 2018, p. 553)

Divided broadly into five sub-groups (Ging, 2017; Jane, 2018; Riberio, 2020):

    • Men’s Rights Activists
    • Men-Going-Their-Own-Way
    • Pick-Up Artists
    • Involuntary celibates (incels)
    • “Sub-cultural trolls”

@Mark_McGlashan | @ALexiconArtist

14

15 of 28

  1. The studyThe ‘manosphere’

@Mark_McGlashan | @ALexiconArtist

15

16 of 28

  1. The studyThe ‘manosphere’

Key-key-words

We identified the following main trends in the 528 key-keywords:

    • Social actor terms and pronouns e.g. women, men (93 key-keywords)
    • Sexuality and relationships e.g. attraction, porn (49 key-keywords)
    • Physical e.g. tall (23 key-keywords, 5 negative) and non-physical descriptors e.g. creepy (24 key-keywords, 11 negative) of social actors
    • Thoughts and feelings e.g. think, opinions (41 key-keywords)
    • Intensifying language e.g. really, never (35 key-keywords)

@Mark_McGlashan | @ALexiconArtist

16

17 of 28

  1. The studyThe ‘manosphere’

Key-key-words

@Mark_McGlashan | @ALexiconArtist

17

Male social actors

men

man

guys

guy

niceguys

male

dude

dudes

bro

bruh

boyfriend

bf

cuck

pua

incel

neckbeard

Gender identification

men, man, guys, guy

Relational

boyfriend, bf

Biological sex

male

Derogatory

cuck

Kinship

dude, dudes, bro, bruh

In-group functionalisation and identification

niceguys, pua, incel, neckbeard

18 of 28

  1. The studyThe ‘manosphere’

Key-key-words

@Mark_McGlashan | @ALexiconArtist

18

Female social actors

women

woman

girls

girl

girl’s

girlfriend

girlfriends

gf

bitch

bitches

chick

chicks

female

females

whore

whores

slut

lesbian

Gender identification

women, woman, girls, girls, girl’s

Relational

girlfriend, girlfriends, gf, lesbian

Biological sex

female, females

Derogatory

bitch, bitches, chick, chicks, whore, whores, slut

19 of 28

  1. The studyThe ‘manosphere’

Complement keywords

@Mark_McGlashan | @ALexiconArtist

19

r/TheRedPill

r/MensRights

r/MGTOW

r/seduction

r/braincels

rp

bp

asktrp

plates

rollo

discipline

plate

amoral

bjj

calories

serena

eating

bmi

puerarchy

rpw

machiavellian

quitting

ambition

ltr's

truths

unplugging

ketogenic

diet

elswhere

strategy

bonobos

aziz

gaylubeoil

redpillschool

psychedelics

rights

victims

circumcision

violence

issues

movement

crime

domestic

abuse

foreskin

issue

duluth

prison

shelters

discrimination

innocent

circumcise

evidence

guilty

mutilation

prostate

against

legal

genital

homelessness

punishment

punished

sentencing

both

fgm

bond

gyow

debt

razors

gillette

boomers

honk

feibleman

scouts

b

plantation

eugenics

gtow

simping

dolls

doll

sexbots

tradcuck

tradcucks

razor

elba

terminator

gmow

npcs

ai

minimalist

retire

marvel

idris

cobol

kiss

eye

text

contact

introverts

hi

approached

tips

dress

clubs

dance

askseddit

outer

night

practice

dancing

coffee

routines

kissing

nervous

afc

negs

awkwardness

smiling

busy

zoned

dailymanup

ikxhwq

nwhg

sedditors

foids

cope

jfl

foid

looksmatch

mogs

stacy

ldar

therapy

teehee

blackops

cel

rope

fakecel

lifefuel

brainlet

truecel

stacies

volcels

boyo

dogcel

bullies

betabuxx

blackpills

numales

chadlite

gymcel

mentalcel

currycels

meeks

20 of 28

  1. The studyThe ‘manosphere’

Complement keywords

@Mark_McGlashan | @ALexiconArtist

20

r/TheRedPill

rp

bp

asktrp

plates

rollo

discipline

plate

amoral

bjj

calories

serena

eating

bmi

puerarchy

Rpw

machiavellian

quitting

ambition

ltr's

truths

unplugging

ketogenic

diet

elswhere

strategy

bonobos

aziz

gaylubeoil

redpillschool

psychedelics

Diet & exercise

bjj, calories, eating, bmi, ketogenic, diet

Prominent community members

rollo, puerarchy, gaylubeoil, redpillschool

Withdrawal

quitting, unplugging

Relationships

plates, plate, ltr’s, ambition

Ethics & philosophy

discipline, amoral, machiavellian, truths

21 of 28

  1. The studyThe ‘manosphere’

Complement keywords

@Mark_McGlashan | @ALexiconArtist

21

r/MensRights

rights

victims

circumcision

violence

issues

movement

crime

domestic

abuse

foreskin

issue

duluth

prison

shelters

discrimination

innocent

circumcise

evidence

guilty

mutilation

prostate

against

legal

genital

homelessness

punishment

punished

sentencing

both

fgm

Law

rights, victims, crime, prison, discrimination, innocent, evidence, guilty, legal, punishment, punished, sentencing

Genitalia

circumcision, foreskin, circumcise, prostate, mutilation, genital, fgm

Violence

violence, domestic, abuse, duluth, shelters

Activism

issues, movement, issue

22 of 28

  1. The studyThe ‘manosphere’

Complement keywords

@Mark_McGlashan | @ALexiconArtist

22

r/MGTOW

bond

gyow

debt

razors

gillette

boomers

honk

feibleman

scouts

b

plantation

eugenics

gtow

simping

dolls

doll

sexbots

tradcuck

tradcucks

razor

elba

terminator

gmow

npcs

ai

minimalist

retire

marvel

idris

cobol

Media

bond, razors, gillette, razor, elba, terminator, marvel, idris

Out-group social actors

boomers, honk, simping, tradcuck, tradcucks, npcs

Variants of the term MGTOW

gyow, gtow, gmow

Technology

sexbots, ai, cobol

Discrimination

feibleman, scouts, plantation

MGTOW lifestyle

Dolls, doll, minimalist, retire

23 of 28

  1. The studyThe ‘manosphere’

Complement keywords

@Mark_McGlashan | @ALexiconArtist

23

r/seduction

kiss

eye

text

contact

introverts

hi

approached

tips

dress

clubs

dance

askseddit

outer

night

practice

dancing

coffee

routines

kissing

nervous

afc

negs

awkwardness

smiling

busy

zoned

dailymanup

ikxhwq

nwhg

sedditors

Physical intimacy and body language

kiss, eye, contact, dance, dancing, kissing, smiling

Appearance

dress, outer

Strategy

text, hi, approached, tips, clubs, night, practice, coffee, routines, negs

Nervous and lack of success

introverts, nervous, afc, awkwardness, zoned

External links

askseddit, dailymanup, ikxhwq, nwhg

24 of 28

  1. The studyThe ‘manosphere’

Complement keywords

@Mark_McGlashan | @ALexiconArtist

24

r/braincels

foids

cope

jfl

foid

looksmatch

mogs

stacy

ldar

therapy

teehee

blackops

cel

rope

fakecel

lifefuel

brainlet

truecel

stacies

volcels

boyo

dogcel

bullies

betabuxx

blackpills

numales

chadlite

gymcel

mentalcel

currycels

meeks

Women

foids, foid, stacy, stacies

Mental health

cope, rope, ldar, therapy, lifefuel, bullies, blackpills

Laughter

jfl, teehee

Incel references

blackops, cel, fakecel, truecel, volcels, gymcel, mentalcel, currycels

References to beta males

brainlet, betabuxx, numales

25 of 28

  1. Conclusions

Key-keywords

Complement keywords

  • Social actors and pronouns
  • Sexuality and relationships
  • Physical and non-physical social descriptors
  • Thoughts and feelings
  • Intensifying language
  • /r/TheRedPill as a self-improvement community
  • /r/MensRights as an activist/legal space
  • /r/MGTOW question societal norms
  • /r/seduction as a strategising and emotional space
  • /r/braincels focuses on mental health

@Mark_McGlashan | @ALexiconArtist

25

26 of 28

  1. Conclusions

Further work

Complement keywords

We have employed key-key-words (premised on LL) but what about what about Egbert & Biber’s text-dispersion keyword analysis?

  • Can we compare approaches?
  • Highlight the opposite of key-key-words – content-distinctiveness NOT content-generalisability

  • They tell us what is peculiar/distinctive to corpora

Complement keywords?

Quirk-key-words?

@Mark_McGlashan | @ALexiconArtist

26

27 of 28

References

Egbert, J., & Biber, D. (2019). Incorporating text dispersion into keyword analyses. Corpora, 14(1), 77-104.

Ging, D. (2017). Alphas, betas, and incels: theorizing the masculinities of the manosphere. Men and Masculnities, 22(4), 638-57.

Jane, E. 2018. Systemic misogyny exposed: Translating Rapeglish from the Manosphere with a Random Rape Threat Generator. International Journal of Culture Studies. 21(6), 661-680.

Marwick, A. E., & Caplan, R. (2018). Drinking male tears: language, the manosphere, and networked harassment. Feminist Media Studies, 18(4), 543-559.

Ribeiro, M., Blackburn, J., Bradlyn, B., De Cristofaro, E., Stringhini, G., Long, S., Greenberg, S., and Zannettou, S. 2020. From Pick-Up Artists to Incels: A Data-Driven Sketch of the Manosphere. Retrieved from: https://www.researchgate.net/publication/338737324_From_Pick-Up_Artists_to_Incels_A_Data-Driven_Sketch_of_the_Manosphere

Scott, M. (1997). PC analysis of key words - and key key words. System, 25(2), 233-45.

@Mark_McGlashan | @ALexiconArtist

27

28 of 28

Thank you!

Any questions?

Mark McGlashan Alexandra Krendel

Mark.McGlashan@bcu.ac.uk a.krendel@lancaster.ac.uk

@Mark_McGlashan @ALexiconArtist