1 of 12

A Whirlwind Tour of Linguistics and Linguists in Tech

Hannah Van Brunt

AVL Digital Nomads Tech Connect

May 1, 2024

2 of 12

Outline

1. My background

2. Linguistics in a nutshell

3. A traditional profession vs. recent shifts

4. What do linguists do in tech?

5. Linguist contributions to NLP / AI: Examples

6. Closing and language blog

3 of 12

1. My background

BA in French, MA in Linguistics
Linguist / Researcher on FrameNet lexical database

(International Computer Science Institute in Berkeley, CA)

Short stints as university lecturer of Linguistics, English
Last 6 years, worked on virtual assistants in big tech:

Linguist at Samsung (on Bixby)
Analytical Linguist at Google (on Google Assistant)

4 of 12

2. Linguistics in a nutshell (1)

Linguistics can be subdivided into the following basic branches:�

PHONETICS / PHONOLOGY – The study �of sounds and sound inventory of a language

Ex: The consonants ‘r’ and ‘l’ are discrete sounds �in English, meaning they differentiate between �words – but there’s no clear distinction in �Japanese (‘r’ and ‘l’ are perceived as one sound)

MORPHOLOGY – The study of word formation and �structure

Ex: “Unhappiness” can be broken�into basic units: ‘un’ (not) + �‘happy’ (root word) + �‘ness’ (the state/condition of)

Phonetics/Phonology:

Phonetics includes the physical properties of speech sounds (e.g. their articulation; acoustic aspects).

A “phoneme” is the smallest unit of speech that distinguishes one word or word element from another in a particular language.

Lots of extra nerdy details on the English vs. Japanese ‘r’ & ‘l’ from https://vocaloid.fandom.com/wiki/English_-_Japanese:

The liquid consonants consist of the lateral consonants and the rhotic consonants. The English language features one rhotic (/ɹ/ as in 'r') and one lateral (/l/ as in 'l') consonant. In Japanese, there is no clear distinction between the two. The Japanese 'r' is realized as a post-alveolar liquid consonant. Its sound tends to vary depending on its context, though many native Japanese speakers perceive it as one singular phoneme. The sound is usually intermediate between /ɾ/ (more 'r'-like similar to the unstressed American D/T ) and /ɺ/ (more lateral or 'l'-like), tending to one or another depending the vowel which follows it.^[9] It is for this reason that native English speakers tend to perceive it between their 'l', 'r' and 'd' sounds.

Morphology:

A “morpheme” is the smallest unit of language that has its own meaning (either a word or a part of a word).

5 of 12

2. Linguistics in a nutshell (2)

SYNTAX – The structure of phrases and sentences (i.e. “grammar”)

Ex: “The boy chased the dog” has a subject noun phrase (‘the boy’), �a verb (‘chased’), and an object noun phrase �(‘the dog’). Order of this sentence is: �SVO (Subject-Verb-Object)

SEMANTICS – The study of meaning in language

Ex: We understand that “dog” refers to a domesticated animal with four legs and a tail.

PRAGMATICS – The study of language use in discourse / context

Ex: Two people in a room with an open window, one says “It’s cold in here” – likely an implicit �request for other person to close the window.

Syntax

Images: The white “tree” is a constituency syntax parse tree; the dark grey image is a dependency syntax parse. The sentence parsed in both, if you can’t read it, is “my parakeet drinks mimosas in the morning”. Dependency and constituency parsers have been used in NLP for some time, but are now being replaced (or used in conjunction with) neural net / deep learning-based parsers.

Semantics

Within this domain, there’s lexical semantics (the study of word meaning) and sentential or phrasal semantics (the study of phrase/sentence meaning, including propositional logic).

Pragmatics

Another common example folks like to use for this is if you’re sitting at the dinner table and your friend says “Can you pass the salt?”, you know that she’s making a request, not asking about your physical capability to perform the action.

6 of 12

7 of 12

3. A traditional profession vs. recent shifts

Traditionally

Linguists worked in academia as professors and researchers
…Or as field researchers (documenting endangered /�dying languages around the world), or as lexicographers (dictionary writers)
Related disciplines: Psychology, Cognitive Science, Speech Language Pathology, �Anthropology, Sociology, Neurology

Recent shift

Past 10-15 years, a shift for linguists from academia to industry (and Tech specifically)

Academia becoming more & more unsustainable
Real desire/need for linguists in Tech
NLP, NLU, ASR, TTS, ML, voice assistants (Alexa, Siri, Google), VUI (voice user interface)

8 of 12

4. What do linguists do in tech?

Analytical Linguist / Computational Linguist / NLP Linguist / �Natural Language Analyst

Companies: Google, Meta, Amazon, Samsung, Grammarly
Tasks: Create semantics for intents; Clean, annotate & analyze natural �language data; Train & evaluate ML models; NLP

Conversation (AI) Designer

Companies: Rasa, Amazon, OpenDialog AI, Sensely, LivePerson
Tasks: Design conversational flows for chatbots; Handle E2E dev / product lifecycle

Knowledge (Graph) Engineer / Ontologist / Taxonomist

Companies: LinkedIn, Microsoft, Walmart, Airbnb, data.world
Tasks: Build ontologies, taxonomies, and knowledge representations for businesses and AI

Learning/Curriculum Designer in EdTech

Companies: (Language learning software) Duolingo, Memrise, Babbel, Busuu, Rosetta Stone
Tasks: Plan syllabi; Design digital language courses / app content; Conduct user research

Even more titles similar to “Analytical Linguist”: AI Linguist, Data Specialist, Language Engineer, Language Scientist

Note: Basically, FAANG (Facebook/Meta, Amazon, Apple, Netflix, and Google) + Microsoft and other tech giants employ all of the first 3 types of people, but they’re not listed for every one in order to not be redundant – and to show that other companies outside of big tech also employ linguists.

A few good articles on Knowledge Graphs (KGs) and ontologies, if you’d like to learn more:

9 of 12

5. Linguist contributions to NLP / AI (1)

EXAMPLE 1: Ensuring quality data and annotation

Issue: High-quality natural language data and annotation are crucial for NLP / ML / LLMs / AI (results are only as good as the data – “garbage in, garbage out”)
Linguists understand how language as a system works; are experts in designing annotation pipelines that result in quality training datasets
Ex. (A): Emotions data for sentiment analysis

30% of Google’s Emotions Dataset is Mislabeled

Ex. (B): Verbosity / Style / Tone / Register

Analytical Linguists at Grammarly work on register (“formality”)�and tone (“domain”) for their AI writing assistant

Google’s “GoEmotions” dataset: a human-labeled dataset of 58K Reddit comments categorized according to 27 emotions.

Built for use in chatbots, for task of suggesting emojis in conversational text, improved customer support interactions
Missed idioms, sarcasm, profanity, basic English, US politics & culture, Reddit memes
Problem #1: Comments presented with no extra metadata
Problem #2: “All raters are native English speakers from India” – unfamiliar with US Eng & culture (esp. Reddit culture)
🡪 Underscores the importance of accurate data labeling, and the importance of the data labelers' knowledge of context for the data (see also next slide).

The “Set goals” image presents options from Grammarly’s AI writing assistant for the user.

Another example of verbosity issues in this space: ChatGPTs have a very verbose, super polite but also pedantic register/style that bothers many users (likely due to reinforcement learning). Linguists could help fix this, avoiding the need for users to have to constantly tweak their prompts.

ChatGPT’s verbosity has been a sticking point for me personally.

10 of 12

5. Linguist contributions to NLP / AI (2)

EXAMPLE 2: Dealing with ambiguity - Grounding language in context

Issue: Natural language is inherently ambiguous and needs context to resolve meaning
Ex. (A): Linguists (as Knowledge Engineers and Ontologists) model semantic networks, knowledge graphs, and ontologies that serve as backbones for grounded NLP systems. Linguists also work on incorporating richer user context into NLP pipelines

For voice assistant to disambiguate user command like “Play ‘Message in a Bottle’” �(🡪 could be song by The Police, song by Taylor Swift, 1999 film, or audiobook recording of Nicholas Spark’s novel!), we need extra context:

(i) KG / ontology of music, movies, & other media;
(ii) metadata on user devices (screens for video vs. audio-only);
(iii) metadata (history) on user preferences (like Taylor Swift or The Police?)

11 of 12

5. Linguist contributions to NLP / AI (3)

EXAMPLE 3: Internationalization and Translation

Issue: Many nuances do not translate across languages in straightforward ways

But LLMs use English as an intermediary: AI chatbot models ‘think’ in English even when using other languages

Linguists have rich understanding of cross-linguistic generalities and differences, can bring this knowledge to bear on:

Ex. (A): Lexical-semantic problems. Languages encode concepts differently – strict 1:1 word translations often inaccurate

User commands for smart devices – “Open the dishwasher.” ‘Open’ & ‘turn on’ are same word �in some languages

Ex. (B): ML, LLM, and translation engine problems with lesser-spoken languages

African languages (e.g. Eritrean); even more widely-spoken languages like Arabic (+ dialects)

12 of 12

6. Closing and language blog

Personal project: Language / Linguistics blog since 2017, linguamonium.com

Some recommended articles:

I welcome guest posts! (Or feel free to propose other kinds of collaboration)
Also happy to connect on LinkedIn: linkedin.com/in/hannah-vanbrunt

THANK YOU!