Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
Why is NLP hard?
Lexical Ambiguity
Will Will will Will’s will?
2 / 16
Why is NLP hard?
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
Lexical Ambiguity
Will Will will Will’s will?
Rose rose to put rose roes on her rows of roses.
Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo.
2 / 16
Why is NLP hard?
Lexical Ambiguity
Will Will will Will’s will?
Rose rose to put rose roes on her rows of roses.
Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo.
→ Buffaloes from Buffalo, NY, whom buffaloes from Buffalo bully, bully buffaloes from Buffalo.
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
2 / 16
One possible
Parse tree
Why is NLP hard?
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
Language ambiguity: Structural
The man saw the boy with the binoculars. Flying planes can be dangerous.
Hole found in the room wall; police are looking into it.
Language imprecision and vagueness
3 / 16
Why is NLP hard?
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
Language ambiguity: Structural
The man saw the boy with the binoculars. Flying planes can be dangerous.
Hole found in the room wall; police are looking into it.
Language imprecision and vagueness
It is very warm here.
3 / 16
Why is NLP hard?
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
Language ambiguity: Structural
The man saw the boy with the binoculars. Flying planes can be dangerous.
Hole found in the room wall; police are looking into it.
Language imprecision and vagueness
It is very warm here.
Q: Did your mother call your aunt last night? A: I’m sure she must have.
3 / 16
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
But that’s the fun part of it
Why is the teacher wearing sun-glasses?
...
4 / 16
But that’s the fun part of it
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
Why is the teacher wearing sun-glasses?
...
Because the class is so bright.
4 / 16
Ambiguities
Pawan Goyal (IIT Kharagpur)
5 / 16
Week 1: Lecture 3
News Headlines
Hospitals Are Sued by 7 Foot Doctors
Stolen Painting Found by Tree Teacher Strikes Idle Kids
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
Ambiguity is pervasive
Find at least 5 meanings of this sentence:
) I made her duck
6 / 16
Ambiguity is pervasive
Find at least 5 meanings of this sentence:
) I made her duck
I cooked duck for her
I cooked duck belonging to her
I created the (artificial) duck, she owns
I caused her to quickly lower her head or body
I waved my magic wand and turned her into a duck
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
6 / 16
Ambiguity is pervasive
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
Syntactic Category
‘Duck’ can be a noun or verb
‘her’ can be a possessive (‘of her’) or dative (‘for her’) pronoun
7 / 16
Ambiguity is pervasive
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
Syntactic Category
‘Duck’ can be a noun or verb
‘her’ can be a possessive (‘of her’) or dative (‘for her’) pronoun
Word Meaning
‘make’ can mean ‘create’ or ‘cook’
7 / 16
Ambiguity is pervasive
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
Grammar
make can be
Transitive: (verb with a noun direct object)
Ditransitive: (verb has 2 noun objects)
Action-transitive: (verb has a direct object + verb)
8 / 16
Ambiguity is pervasive
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
Grammar
make can be
Transitive: (verb with a noun direct object)
Ditransitive: (verb has 2 noun objects)
Action-transitive: (verb has a direct object + verb)
Phonetics
I’m eight or duck
I’m aid her duck
8 / 16
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
Ambiguity is Explosive
I saw the man with the telescope. 2 parses
9 / 16
Ambiguity is Explosive
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
I saw the man with the telescope. 2 parses
I saw the man on the hill with the telescope. 5 parses
9 / 16
Ambiguity is Explosive
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
I saw the man with the telescope. 2 parses
I saw the man on the hill with the telescope. 5 parses
I saw the man on the hill in Texas with the telescope. 14 parses
9 / 16
Ambiguity is Explosive
I saw the man with the telescope. 2 parses
I saw the man on the hill with the telescope. 5 parses
I saw the man on the hill in Texas with the telescope. 14 parses
I saw the man on the hill in Texas with the telescope at noon. 42 parses
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
9 / 16
Ambiguity is Explosive
I saw the man with the telescope. 2 parses
I saw the man on the hill with the telescope. 5 parses
I saw the man on the hill in Texas with the telescope. 14 parses
I saw the man on the hill in Texas with the telescope at noon. 42 parses
I saw the man on the hill in Texas with the telescope at noon on Monday.
132 parses
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
9 / 16
Why is Language Ambiguous?
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
The goal in the production and comprehension of natural language is
efficient communication.
10 / 16
Why is Language Ambiguous?
The goal in the production and comprehension of natural language is
efficient communication.
Allowing resolvable ambiguity
) permits shorter linguistic expressions
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
10 / 16
Why is Language Ambiguous?
The goal in the production and comprehension of natural language is
efficient communication.
Allowing resolvable ambiguity
) permits shorter linguistic expressions
) avoids language being overly complex
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
10 / 16
Why is Language Ambiguous?
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
The goal in the production and comprehension of natural language is
efficient communication.
Allowing resolvable ambiguity
) permits shorter linguistic expressions
) avoids language being overly complex
Language relies on people’s ability to use their knowledge and inference abilities to properly resolve ambiguities
10 / 16
Sanskrit vs English
Sanskrit relies on
morphology.
नरः नरौ नराः
English relies on word order.
Courtsey: Amrith Krishna’s blog
Sanskrit vs English
1. Story of Rama, when read forward
2. Story of Krishna, when read backwards.
Courtsey: Amrith Krishna’s blog
Natural Languages vs. Computer Languages
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
Ambiguity is the primary difference between natural and computer languages.
11 / 16
Natural Languages vs. Computer Languages
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
Ambiguity is the primary difference between natural and computer languages.
Formal programming languages are designed to be unambiguous
11 / 16
Natural Languages vs. Computer Languages
Ambiguity is the primary difference between natural and computer languages.
Formal programming languages are designed to be unambiguous
) Formal programming languages can be defined by a grammar that produces a unique parse for each sentence in the language.
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
11 / 16
Natural Languages vs. Computer Languages
Ambiguity is the primary difference between natural and computer languages.
Formal programming languages are designed to be unambiguous
) Formal programming languages can be defined by a grammar that produces a unique parse for each sentence in the language.
Programming languages are also designed for efficient (deterministic) parsing.
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
11 / 16
Pawan Goyal (IIT Kharagpur)
12 / 16
Week 1: Lecture 3
Why else is NLP hard?
Why else is NLP hard?
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
Non-standard English
Great job @justinbieber! Were SOO PROUD of what youve accomplished! U taught us 2 #neversaynever & you yourself should never give up either
13 / 16
Why else is NLP hard?
Non-standard English
Great job @justinbieber! Were SOO PROUD of what youve accomplished! U taught us 2 #neversaynever & you yourself should never give up either
Segmentation Issues
the New York-New Haven Railroad
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
13 / 16
Why else is NLP hard?
Non-standard English
Great job @justinbieber! Were SOO PROUD of what youve accomplished! U taught us 2 #neversaynever & you yourself should never give up either
Segmentation Issues
the New York-New Haven Railroad
the [New] [York-New] [Haven] [Railroad]
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
13 / 16
Why else is NLP hard?
Non-standard English
Great job @justinbieber! Were SOO PROUD of what youve accomplished! U taught us 2 #neversaynever & you yourself should never give up either
Segmentation Issues
the New York-New Haven Railroad
the [New] [York-New] [Haven] [Railroad]
the [New York]-[New Haven] [Railroad]
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
13 / 16
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
Why else is NLP hard?
Idioms
dark horse
Ball in your court Burn the midnight oil
14 / 16
Why else is NLP hard?
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
Idioms
dark horse
Ball in your court Burn the midnight oil
neologisms
unfriend retweet
Google/Skype/photoshop
14 / 16
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
Why is NLP hard?
New Senses of a word That’s sick dude! Giants
15 / 16
Why is NLP hard?
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
New Senses of a word
That’s sick dude!
Giants ... multinationals, conglomerates, manufacturers
15 / 16
Why is NLP hard?
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
New Senses of a word
That’s sick dude!
Giants ... multinationals, conglomerates, manufacturers
Tricky Entity Names
Where is A Bug’s Life playing ...
Let It Be was recorded ...
15 / 16
What we do in NLP?
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
Tools Required
Knowledge about language Knowledge about the world
A way to combine knowledge resources
16 / 16
What we do in NLP?
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
Tools Required
Knowledge about language Knowledge about the world
A way to combine knowledge resources
How is it generally done?
Probabilistic models built from language data
) P(“maison” → “house”) is high
16 / 16
What we do in NLP?
Pawan Goyal (IIT Kharagpur)
Week 1: Lecture 3
Tools Required
Knowledge about language Knowledge about the world
A way to combine knowledge resources
How is it generally done?
Probabilistic models built from language data
) P(“maison” → “house”) is high
) P(I saw a van) > P(eyes awe of an)
Extracting rough text features does half the job.
16 / 16