��MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING �(A Unit of Rajalaxmi Education Trust®, Mangalore)�Autonomous Institute affiliated to VTU, Belagavi, Approved by AICTE, New Delhi�Accredited by NAAC with A+ Grade & ISO 9001:2015 Certified Institution� � � � �

DEPARTMENT OF ARTIFICIAL INTELLIGENCE AND MACHINE LEARNING

Course Materials

for

Subject Title : NATURAL LANGUAGE PROCESSING

Subject Code :BAI601

Semester :6

Scheme : 2022

by

Radha E G

Assistant Professor

Dept of AI and ML

MITE,Moodabidri

​

​

​

Contents

• Syllabus
• Course Outcomes
• Vision & Mission of the Department
• Program Outcomes
• Program Specific Outcomes & Program Education Objectives
• CO-PO Mapping
• Lesson Plan
• Assignment Questions
• IA Marks Distribution & Portion for IA
• Laboratory Mark Rubrics (If IPCC Course)
• Minimum Attendance Requirements
• Module 1 Materials
• Module 1 Question Bank

​

​

​

​

Course Outcomes�

At the end of the course, the student will be able to:

● Apply the fundamental concept of NLP, grammar-based language model and statistical-based language model.

● Model morphological analysis using Finite State Transducers and parsing using context-free grammar and different parsing approaches.

● Develop the Naïve Bayes classifier and sentiment analysis for Natural language problems and text classifications.

● Apply the concepts of information retrieval, lexical semantics, lexical dictionaries such as WordNet, lexical computational semantics, distributional word similarity.

● Identify the Machine Translation applications of NLP using Encode and Decoder.

Vision & Mission of the Department�

Vision

• To create well groomed, technically competent and skilled AIML professionals who can become part of industry and undertake quality research at global level to meet societal needs.

Mission

• Provide state of art infrastructure, tools and facilities to make students competent and achieve excellence in education and research.
• Provide a strong theoretical and practical knowledge across the AIML discipline with an emphasis on AI based research and software development.
• Inculcate strong ethical values, professional behaviour and leadership abilities through various curricular, co-curricular, training and development activities.

​

Program Outcomes�

Engineering Graduates will be able to:

1. Engineering knowledge: Apply the knowledge of mathematics, science, engineering fundamentals, and an engineering specialization to the solution of complex engineering problems.
2. Problem analysis: Identify, formulate, review research literature, and analyze complex engineering problems reaching substantiated conclusions using first principles of mathematics, natural sciences, and engineering sciences.
3. Design/development of solutions: Design solutions for complex engineering problems and design system components or processes that meet the specified needs with appropriate consideration for the public health and safety, and the cultural, societal, and environmental considerations.
4. Conduct investigations of complex problems: Use research-based knowledge and research methods including design of experiments, analysis and interpretation of data, and synthesis of the information to provide valid conclusions.
5. Modern tool usage: Create, select, and apply appropriate techniques, resources, and modern engineering and IT tools including prediction and modeling to complex engineering activities with an understanding of the limitations.

​

6.The engineer and society: Apply reasoning informed by the contextual knowledge to assess societal, health, safety, legal and cultural issues and the consequent responsibilities relevant to the professional engineering practice.

7.Environment and sustainability: Understand the impact of the professional engineering solutions in societal and environmental contexts, and demonstrate the knowledge of, and need for sustainable development.

8.Ethics: Apply ethical principles and commit to professional ethics and responsibilities and norms of the engineering practice.

9.Individual and team work: Function effectively as an individual, and as a member or leader in diverse teams, and in multidisciplinary settings.

10.Communication: Communicate effectively on complex engineering activities with the engineering community and with society at large, such as, being able to comprehend and write effective reports and design documentation, make effective presentations, and give and receive clear instructions.

11.Project management and finance: Demonstrate knowledge and understanding of the engineering and management principles and apply these to one’s own work, as a member and leader in a team, to manage projects and in multidisciplinary environments.

12.Life-long learning: Recognize the need for and have the preparation and ability to engage in independent and life-long learning in the broadest context of technological change

Program Specific Outcomes & Program Education Objectives�

Program Educational Objectives (PEOs)

• Graduates will follow logical, practical and research-oriented approach for solving the real world problems by providing AI based solutions.
• Graduates will work independently as well as in multidisciplinary teams at workplace.
• Graduates will setup start-up and become successful entrepreneurs.
•  

Program Specific Outcomes (PSOs)

• The graduates of AIML department will be able to
• Train machine learning models to address real life challenging problems using acquired AI knowledge.
• Develop applications using ML techniques related to the field of medical, agriculture, defence, education and various scientific explorations.
•  
•  

​

CO-PO Mapping �

Mapping of COs with PO:

Correlation Levels: 1: Low 2: Medium 3: High

Assignment Questions�

IA Marks Distribution & Portion for IA �

��Laboratory Mark Rubrics (If IPCC Course)�

Model 1

Introduction ,Language Modeling

Content

• Introduction: What is Natural Language Processing? Origins of NLP, Language and Knowledge, The Challenges of NLP, Language and Grammar, Processing Indian Languages, NLP Applications.
• Language Modeling: Statistical Language Model - N-gram model (unigram, bigram), Paninion Framework, Karaka theory.
• Textbook 1: Ch. 1, Ch. 2.

Chapter 1-Introduction

​

​

What is Natural Language Processing?

NLP focuses on the interaction between computers and human language, enabling machines to understand, interpret, and generate human language in a way that is both meaningful and useful.

Natural language processing (NLP) is a field of computer science and a subfield of artificial intelligence that aims to make computers understand human language.

• Natural Language Processing (NLP) is a subfield of artificial intelligence (AI) and linguistics that focuses on the interaction between computers and human language. Its primary goal is to enable machines to understand, interpret, generate, and respond to human language in a way that is both meaningful and contextually appropriate.

• NLP involves a wide range of tasks and techniques to process and analyze natural language data, which can include written text or spoken language.

Some of the fundamental tasks in NLP include:

• Tokenization: Breaking down a piece of text into smaller units, such as words or subwords, which are easier to handle and analyze.

• Part-of-speech (POS) tagging: Assigning grammatical categories (e.g., noun, verb, adjective) to each word in a sentence.

• Named Entity Recognition (NER): Identifying and classifying entities (e.g., names of people, organizations, locations) in a text.

• Parsing: Analyzing the grammatical structure of sentences to understand their syntactic relationships.

• Sentiment Analysis: Determining the sentiment or emotion expressed in a piece of text (e.g., positive, negative, neutral).

• Machine Translation: Translating text from one language to another using various techniques, including rule-based, statistical, and neural machine translation.

Tokenization: Splits sentence into words.

• Stemming: Reduces words to root forms (sometimes not valid words).
• Lemmatization: Reduces words to their valid base forms, using context.
• POS Tagging: Identifies the grammatical role of each word.

Applications of NLP

Types of determiners

1. Articles-a,an,the
2. Demonstrative determiners-this, that, these, and those.
3. Distributive determiners-each, every, all, and both 
4. Interrogative determiners-whose,what,which
5. Quantifying determiners and numbers- many, some, few, any, all, and several.
6. Relative determiners-whichever

​

​

​

​

​

​

​

Example : The quick brown fox jumps over the lazy dog

Example 2:The thirsty dog drank all the water in the bowl.

Example 3:The small cat chased the big rat under the table.

Rationalist Approach:

• Core Idea: The rationalist approach is based on the belief that humans have innate knowledge or structures, which can be used to understand and process language. This approach emphasizes theoretical principles, rules, and logic.
• Key Features:
• Emphasis on linguistic theory (e.g., grammar, syntax).
• Often involves rule-based systems where a predefined set of rules is used to analyze or generate language.

​

• Empiricist Approach:
• Core Idea: The empiricist approach is based on the belief that knowledge comes primarily from sensory experience and observation. In NLP, this translates to an approach where the system learns from data rather than predefined rules.
• Key Features:
• Emphasis on data-driven methods and learning from examples.
• Typically uses machine learning, especially statistical methods, to derive patterns from large datasets.
• Language is learned based on observed patterns and statistical correlations rather than innate structures.

​

Computational Linguistics

Computational linguistics is the study of language using computer models and algorithms. It combines linguistics, computer science, and artificial intelligence (AI) to analyze, process, and generate human language.

Key Areas of Computational Linguistics

1. Natural Language Processing (NLP) – Teaching computers to understand and generate human language.

Example: Chatbots, machine translation (Google Translate).

1. Speech Recognition & Synthesis – Converting spoken language into text and vice versa.

Example: Siri, Google Assistant.

• Text Mining & Sentiment Analysis – Extracting useful information and emotions from texts.

Example: Analyzing customer reviews to detect positive or negative opinions.

• Machine Translation – Developing automatic translation systems.

Example: DeepL, Google Translate.

​

4. Computational Phonetics & Phonology – Studying how computers can analyze speech sounds.

Example: Voice recognition in smart assistants.

5. Cognitive Modeling of Language – Simulating how humans process language.

Example: AI models that predict word choice in sentences.

​

How It Relates to Other Linguistic Fields

• Unlike theoretical linguistics, which focuses on rules and structures, computational linguistics builds models to automate language processing.
• Overlaps with psycholinguistics, especially in modeling human language understanding.
• Heavily used in AI, especially for NLP applications like chatbots, search engines, and voice assistants.

​

• Focuses on using computer models and algorithms to process and analyze language.
• Overlaps with Artificial Intelligence (AI) and Natural Language Processing (NLP).
• Core areas include:
• Speech Recognition – Converting spoken language into text (e.g., Siri, Google Assistant).
• Machine Translation – Automatic translation (e.g., Google Translate, DeepL).
• Chatbots & AI – Building conversational agents (e.g., ChatGPT).
• Sentiment Analysis – Understanding emotions in text (e.g., analyzing tweets).
• Example Research:
• Training AI models to understand and generate human-like responses.
• Building algorithms that detect fake news or summarize text.

​

The main difference between theoretical linguistics and psycholinguistics lies in their focus and approach to studying language:

Theoretical Linguistics

• Concerned with the structure and principles of language itself.
• Focuses on topics like phonetics, phonology, morphology, syntax, semantics, and pragmatics.
• Tries to develop abstract models and rules that explain how languages work.
• Examples: Chomsky’s Universal Grammar, formal syntax theories, and phonological rules.

Psycholinguistics

• Studies how the human mind processes language.
• Focuses on how language is acquired, comprehended, produced, and stored in the brain.
• Uses methods from psychology, neuroscience, and cognitive science to understand real-time language processing.
• Examples: Studies on speech perception, sentence processing, memory in language use, and language disorders like aphasia.
• In short, theoretical linguists analyze how language is structured, while psycholinguists study how the brain processes and understands language.

​

1.3 LANGUAGE AND KNOWLEDGE

Language is the medium of expression in which knowledge is deciphered. Language being a medium of expression is the outer form of the content it expresses. The same content can be expressed in different languages.

Lexical analysis:

​

• Lexical analysis is a fundamental step in Natural Language Processing (NLP) that involves breaking down input text into individual units called tokens.
• Example of lexical analysis in NLP.

"The quick brown fox jumps over the lazy dog.“

• Lexical analysis involves breaking down this sentence into individual components called tokens.
• Here’s a step-by-step example:

i.Tokenization: This process splits the text into tokens (words or phrases).

• sentence,look like :
• ["The", "quick", "brown", "fox", "jumps", "over", "the", "lazy", "dog"]

​

ii.Normalization: Converting all tokens to a common format, usually lowercasing all words:

• ["the", "quick", "brown", "fox", "jumps", "over", "the", "lazy", "dog"]

​

iii. Removing Stop Words: Stop words are common words that don't add much meaning and are usually removed in NLP tasks.

Examples of stop words are "the", "over", "and", etc. After removing stop words, the tokens might be:

["quick", "brown", "fox", "jumps", "lazy", "dog"]

iv. Lemmatization: This reduces words to their base or root form.

For example, "jumps" would be reduced to "jump":

["quick", "brown", "fox", "jump", "lazy", "dog"]

​

v. Part-of-Speech Tagging: Identifying the grammatical category of each token. For our sentence, it might look like this:

• [("The", "DT"), ("quick", "JJ"), ("brown", "JJ"), ("fox", "NN"),
• ("jumps", "VBZ"), ("over", "IN"), ("the", "DT"),
• ("lazy", "JJ"), ("dog", "NN")]

​

​

Syntactic analysis, also known as parsing, is the process of analyzing sentences in terms of their grammatical structure. It involves breaking down sentences into their components (words or tokens) and understanding the syntactical relationships between them.

• Example 1:

Consider the sentence: "The quick brown fox jumps over the lazy dog."

• Here's how a syntactic analysis might look:
• Tokenization:
1. [The], [quick], [brown], [fox], [jumps], [over], [the], [lazy], [dog]

​

Part-of-Speech Tagging:

• The (Determiner)
• quick (Adjective)
• brown (Adjective)
• fox (Noun)
• jumps (Verb)
• over (Preposition)
• the (Determiner)
• lazy (Adjective)
• dog (Noun)

​

• Parsing: Using a context-free grammar, the sentence can be broken down into a parse tree that represents its syntactic structure.

​

Common rules for constructing parse tree include:

• Sentence (S) → NP VP�(A sentence consists of a noun phrase and a verb phrase)
• Noun Phrase (NP) → Det N | N�(A noun phrase consists of a determiner and a noun or just a noun)
• Verb Phrase (VP) → V NP | V NP PP�(A verb phrase consists of a verb followed by an optional noun phrase and a prepositional phrase)
• Prepositional Phrase (PP) → P NP�(A prepositional phrase consists of a preposition followed by a noun phrase)

​

• Sentence Breakdown:
• Sentence (S)
• Noun Phrase (NP)
• Determiner (Det): The
• Adjective (Adj): quick
• Adjective (Adj): brown
• Noun (N): fox
• Verb Phrase (VP)
• Verb (V): jumps
• Prepositional Phrase (PP)
• Preposition (P): over
• Noun Phrase (NP)
• Determiner (Det): the
• Adjective (Adj): lazy
• Noun (N): dog

​

"The quick brown fox jumps over the lazy dog."

​

​

• Here, "S" represents the sentence,
• "NP" stands for noun phrase,
• "VP" stands for verb phrase,
• "PP" stands for prepositional phrase,
• "D" stands for determiner,
• "Adj" stands for adjective,
• "N" stands for noun, and
• "V" stands for verb.
• In this example, the syntactic analysis provides a clear structure of how the words in the sentence are related to each other grammatically. This structure can be useful for various NLP tasks like machine translation, information extraction, and sentiment analysis.

​

Example 2: The thirsty dog drank all the water in the bowl

Example 3: The small cat chased the big rat under the table

Here's the parse tree for the sentence "The small cat chased the big rat under the table." in a tree-like structure:

Example 4:She eats an apple

Example 5:the hungry tiger was looking for food.​�

Example 6:The small cat is chasing a mouse."

Example 7:The small dog is barking at the cat

• Examples of Syntax:Consider the following sentences:
• Correct Syntax: "John eats an apple."
• Incorrect Syntax: "Apple eats John an.“
• Assigning POS Tags
• During parsing, each word in the sentence is assigned a POS tag to indicate its grammatical category. Here’s an example breakdown:
• Sentence: "John eats an apple."
• POS Tags:
• John: Proper Noun (NNP)
• eats: Verb (VBZ)
• an: Determiner (DT)
• apple: Noun (NN)

​

Semantic Analysis

• Semantic Analysis is the third phase of Natural Language Processing (NLP), focusing on extracting the meaning from text. Unlike syntactic analysis, which deals with grammatical structure, semantic analysis is concerned with the literal and contextual meaning of words, phrases, and sentences.
• Semantic analysis aims to understand the dictionary definitions of words and their usage in context. It determines whether the arrangement of words in a sentence makes logical sense. This phase helps in finding context and logic by ensuring the semantic coherence of sentences.

​

• Key Tasks in Semantic Analysis
• Named Entity Recognition (NER): NER identifies and classifies entities within the text, such as names of people, places, and organizations. These entities belong to predefined categories and are crucial for understanding the text's content.
• Word Sense Disambiguation (WSD): WSD determines the correct meaning of ambiguous words based on context. For example, the word "bank" can refer to a financial institution or the side of a river. WSD uses contextual clues to assign the appropriate meaning

​

Example:

• Consider the sentence: "The bank will not approve your loan."
• Here's how semantic analysis works on this sentence:
• Word Sense Disambiguation:
1. Bank: In this context, "bank" refers to a financial institution, not the side of a river.
• Semantic Role Labeling:
• Agent: The entity performing the action (the bank).
• Action: The action being performed (will not approve).
• Patient: The entity affected by the action (your loan).
• So, the semantic analysis might identify that:
• The word "bank" is used in the financial sense.
• The "bank" (Agent) is performing the action of "not approving" (Action) the "loan" (Patient).

​

Discourse analysis in Natural Language Processing (NLP) is the study of how texts and spoken language function in communicative contexts. It goes beyond sentence-level analysis to understand how sentences and larger units of text interact to create meaning. Discourse analysis involves various tasks such as identifying discourse markers, anaphora resolution, and analyzing the structure and coherence of texts.

Key Components of Discourse Analysis in NLP

1. Discourse Markers: Words or phrases like "however," "therefore," and "on the other hand" that help to structure discourse.
2. Anaphora Resolution: Resolving references to earlier parts of the text, such as pronouns referring back to nouns.
3. Coherence and Cohesion: Understanding how sentences are connected and how they logically flow from one to another.
4. Topic Segmentation: Identifying boundaries between different topics within a text.
5. Dialogue Act Tagging: Labeling parts of dialogue according to their function, such as questions, statements, or requests.

​

Example: Anaphora Resolution in Discourse Analysis

• Text: "John went to the store. He bought some apples."
• Task: Identify what "He" refers to in the text.
• Steps:
• Text Tokenization:

Split the text into individual tokens:

["John", "went", "to", "the", "store", ".", "He", "bought", "some", "apples", "."]

2. Part-of-Speech Tagging:

Identify the parts of speech for each token:

[("John", "NNP"), ("went", "VBD"), ("to", "TO"), ("the", "DT"), ("store", "NN"), (".", "."), ("He", "PRP"), ("bought", "VBD"), ("some", "DT"), ("apples", "NNS"), (".", ".")]

• Named Entity Recognition (NER):

Identify named entities: [("John", "PERSON")]

• Coreference Resolution:
1. Determine that "He" refers to "John" by matching gender, number, and context.
• Result: "He" refers to "John" in the given text.
• Discourse analysis can be quite complex and involves understanding the intricate relationships between different parts of the text to capture the full meaning.

​

• Pragmatic analysis in Natural Language Processing (NLP) focuses on understanding how context influences the meaning of language. It's about interpreting the intended meaning behind words and sentences based on the situational context1.
• Example of Pragmatic Analysis in NLP:

"The movie was so boring, I fell asleep.“

Literal Meaning: The movie was boring enough to make the reviewer fall asleep.

Pragmatic Meaning: The reviewer is expressing strong dissatisfaction with the movie.

In this case, pragmatic analysis helps us understand that the reviewer didn't literally fall asleep because the movie was boring, but rather, they are using hyperbole to emphasize their negative opinion.

Consider the following examples:

• Contextual Greeting: "Hello! What time is it?"
• "Hello!" is more than just a greeting; it serves to establish contact.
• "What time is it?" might be a straightforward request for the current time, but it could also imply concern about being late.
• Figurative Expression: "I'm falling for you."
• The word "falling" literally means collapsing, but in this context, it means the speaker is expressing love for someone.

​

1.4 The Challenges of NLP

• Ambiguity in computational linguistics is a situation where a word or a sentence may have more than one meaning. That is, a sentence may be interpreted in more than one way. This leads to uncertainty in choosing the right meaning of a sentence especially while processing natural language by computer.
• Ambiguity is a challenging task in natural language understanding (NLU). The process of handling ambiguity is called as disambiguation. Ambiguity is present in almost all the steps of natural language processing. (Steps of NLP – lexical analysis, syntactic analysis, semantic analysis, discourse analysis, and pragmatic analysis)

​

​

​

Consider the following sentence for an example;

“Raj tried to reach his friend on the mobile, but he didn’t attend”

In this sentence, we have the presence of lexical, syntactic, and anaphoric ambiguities.

Lexical ambiguity – The word “tried” means “attempted” not “judged” or “tested”.

Also, the word “reach” means “establish communication” not “gain” or “pass” or“strive”.

Syntactic ambiguity – The phrase “on the mobile” attached to “reach” and thus means“using the mobile”. It is not attached to “friend”.

1. Ambiguity

• Natural language is full of ambiguities. For example, the sentence "I saw the man with the telescope" can mean two different things: either you used a telescope to see the man, or you saw a man who had a telescope. Disambiguating such sentences requires understanding the context and the relationships between words.

​

2. Complexity

• Human languages have complex grammar rules, idioms, and exceptions. For instance, English has irregular verbs like "go" (went) and "come" (came). Modeling these complexities accurately is a significant challenge for NLP systems.

​

​

3. Variability

• People use language differently based on their region, culture, and personal preferences. This variability includes dialects, slang, and individual writing styles. NLP systems need to adapt to these variations to understand and generate natural language accurately.

Example: Different regions use different slang. For instance, "soda" in the US might be "pop" in some regions. NLP systems need to recognize and adapt to these variations.

4. Resource Limitations

• High-quality annotated datasets are crucial for training NLP models. However, creating these datasets can be time-consuming and expensive. Additionally, some languages have fewer resources available, making it harder to develop NLP systems for those languages.
• Example: Training an NLP model for a less common language like Welsh can be challenging due to the lack of large annotated datasets compared to languages like English.

​

5. Context Understanding

• Understanding the context in which a word or sentence is used is essential for accurate interpretation. For example, the word "bank" can refer to a financial institution or the side of a river, depending on the context. NLP systems need to capture this context to understand the intended meaning.

​

6. Misspellings

• Detecting and correcting misspellings is a challenge for NLP systems. Unlike humans, who can quickly recognize and correct misspellings, NLP systems need sophisticated algorithms to handle this task accurately.
• Example: If someone types "teh" instead of "the," an NLP system must recognize and correct this misspelling to understand the sentence correctly.

​

​

7. Language Differences

• Supporting multiple languages with different vocabularies, grammar rules, and cultural norms is a major challenge. Each language has its unique characteristics, and NLP systems need to be tailored to handle these differences effectively.
• Example: Supporting multiple languages involves understanding different vocabularies and grammar rules. For instance, translating "I am going to the store" into Spanish requires knowledge of both languages' structures: "Voy a la tienda."

​

1.5 Language and Grammar

Types of grammar in NLP

Transformational grammar

• In linguistics, a transformational grammar or transformational-generative grammar (TGG) is a generative grammar, especially of a natural language, that has been developed in the syntactic structures of phrase structure grammars (as opposed to dependency grammars).
• Transformational grammar is the tradition of specific transformational grammars. Much current research in transformational grammar is inspired by Chomsky’s Minimalist Program.

​

• In 1957, Noam Chomsky published Syntactic Structures, in which he developed the idea that each sentence in a language has two levels of representation — a deep structure and a surface structure. 
• The deep structure represented the core semantic relations of a sentence, and was mapped on to the surface structure via transformations. 
• Chomsky believed there are considerable similarities between languages’ deep structures, and that these structures reveal properties, common to all languages that surface structures conceal. 

​

The mapping from deep structure to surface structure is carried out by transformation.

​

The passive transformation rules will convert the sentence into:

The +snatcher+will +be+en+catch+by+police (figure 1.3)

Example:

1. Original Sentence (Deep Structure):

"The boy is reading the book."

• Transformation Rules:

Apply the transformation to create a passive voice sentence:

"The book is being read by the boy."

• Transformed Sentence (Surface Structure):

"The book is being read by the boy."

• In this example, the deep structure of the sentence "The boy is reading the book" is transformed into the surface structure "The book is being read by the boy" using the rules of transformational grammar.

​

• Phrase structure grammar, also known as constituency grammar, breaks down sentences into their constituent parts, or phrases, which are then analyzed hierarchically. Let’s use the original sentence and its transformed version from our previous example:
• Original Sentence: "The boy is reading the book."
• Sentence (S):

Noun Phrase (NP): "The boy"

• Determiner (Det): "The"
• Noun (N): "boy"

Verb Phrase (VP): "is reading the book"

• Auxiliary Verb (Aux): "is"
• Verb (V): "reading"
• Noun Phrase (NP): "the book"
1. Determiner (Det): "the"
2. Noun (N): "book"

​

In this structure:

• The sentence (S) is broken down into a noun phrase (NP) and a verb phrase (VP).
• The NP "The boy" consists of a determiner (Det) "The" and a noun (N) "boy".
• The VP "is reading the book" consists of an auxiliary verb (Aux) "is" and another verb phrase (VP).
• The second VP "reading the book" consists of a verb (V) "reading" and a noun phrase (NP).
• The NP "the book" consists of a determiner (Det) "the" and a noun (N) "book".

In this structure:

• The sentence (S) is divided into a noun phrase (NP) and a verb phrase (VP).
• The NP "The book" consists of a determiner (Det) "The" and a noun (N) "book".
• The VP "is being read by the boy" consists of an auxiliary verb (Aux) "is" and another verb phrase (VP).
• The second VP "being read by the boy" consists of a verb (V) "being read" and a prepositional phrase (PP).
• The PP "by the boy" consists of a preposition (P) "by" and a noun phrase (NP).
• The NP "the boy" consists of a determiner (Det) "the" and a noun (N) "boy".

• Transformed Sentence: "The book is being read by the boy."
• Sentence (S):
1. Noun Phrase (NP): "The book"
1. Determiner (Det): "The"
2. Noun (N): "book"
2. Verb Phrase (VP): "is being read by the boy"
• Auxiliary Verb (Aux): "is"
• Verb Phrase (VP): "being read by the boy"
1. Verb (V): "being read"
2. Prepositional Phrase (PP): "by the boy"
1. Preposition (P): "by"
2. Noun Phrase (NP): "the boy"
1. Determiner (Det): "the"
2. Noun (N): "boy"

​

1.6 Processing Indian Languages,

1.7 NLP Applications

Machine Translation

This refers to automatic translation of text from one language to another. In order to carry out this translation, it is necessary to have an understanding of words and phrases, grammar of the two languages involved, semantics of the languages, and world knowledge.

​

​

​

Speech Recognition

Speech recognition is used for converting spoken words into text. It is used in applications, such as mobile, home automation, video recovery, dictating to Microsoft word, voice biometrics.

This is the process of mapping acoustic speech signals to asset of words. The difficulties arise due to wide variations in the pronunciation of words,homonym(e.g. dear and deer ) and acoustic ambiguities (e.g.in the rest and interest).

​

Speech Synthesis

Speech Synthesis referes to automatic production of speech(utterance of natural language sentences). Such system can read out your email on telephone or even read out a storybook for you. In order to generate utterances, text has to be processed. So NLP remains an important component of any speech synthesis system

Natural Languages Interfaces to Databases��Natural language interfaces allow querying a structured database using natural language sentences.

From the above diagram, it is clear that a user who needs information will have to formulate a request in the form of a query in natural language. After that, the IR system will return output by retrieving the relevant output, in the form of documents, about the required information.

The step by step procedure of these systems are as follows:

• Indexing the collection of documents.
• Transforming the query in the same way as the document content is represented.
• Comparing the description of each document with that of the query.
• Listing the results in order of relevancy.

Chapter 2: statistical language model

• A statistical language model in natural language processing (NLP) is a type of probabilistic model designed to predict the likelihood of sequences of words. These models leverage statistical methods to capture patterns in large text corpora and are essential for a variety of NLP tasks like text generation, speech recognition, and machine translation.
• Key Components of Statistical Language Models:
• N-gram Models:
1. Unigram Model: Considers each word independently.
2. Bigram Model: Considers pairs of consecutive words.
3. Trigram Model: Considers triplets of consecutive words.
4. Higher-order N-gram models can be constructed by considering larger contexts.

​

Example:

• Consider the training set:

The Arabian knights

These are the fairy tales of the east

The stories of the Arabian knights are translated in many languages .

Compute using the bigram model the probability of the sentence. Include start and end symbol in your calculations.

Example Calculation

• Let's use a simple sentence:
• Sentence: "I love natural language processing and natural language understanding."
• Bigrams
• Breaking it down into bigrams, we get:
• ("I love")
• ("love natural")
• ("natural language")
• ("language processing")
• ("processing and")
• ("and natural")
• ("natural language")
• ("language understanding")

​

• Frequency Counts
• Count the occurrences:
• Count("I love") = 1
• Count("love natural") = 1
• Count("natural language") = 2
• Count("language processing") = 1
• Count("processing and") = 1
• Count("and natural") = 1
• Count("language understanding") = 1
• For the individual word counts:
• Count("I") = 1
• Count("love") = 1
• Count("natural") = 2
• Count("language") = 2
• Count("processing") = 1
• Count("and") = 1
• Count("understanding") = 1

​

• Calculating Probabilities
• Using the formula, we calculate:
• P("love" | "I") = Count("I love") / Count("I") = 1/1 = 1.0
• P("natural" | "love") = Count("love natural") / Count("love") = 1/1 = 1.0
• P("language" | "natural") = Count("natural language") / Count("natural") = 2/2 = 1.0
• P("processing" | "language") = Count("language processing") / Count("language") = 1/2 = 0.5
• P("and" | "processing") = Count("processing and") / Count("processing") = 1/1 = 1.0
• P("natural" | "and") = Count("and natural") / Count("and") = 1/1 = 1.0
• P("language" | "natural") = Count("natural language") / Count("natural") = 2/2 = 1.0
• P("understanding" | "language") = Count("language understanding") / Count("language") = 1/2 = 0.5

​

2.2.5 Paninion Framework

Karaka Theory

Model questions

Section 1: Introduction to NLP

1. Define Natural Language Processing (NLP).
2. Explain the origins of NLP and its evolution over time.
3. List some key challenges of NLP and briefly describe each.
4. Differentiate between language and knowledge in the context of NLP.
5. Describe the role of grammar in NLP.
6. What are the major challenges in processing Indian languages?
7. Identify at least five real-world applications of NLP.
8. Explain how linguistic rules contribute to NLP systems.

​

Section 2: Language Modeling

9. What is a statistical language model?
10. Define and explain the concept of an N-gram model with examples.
11. Differentiate between unigram, bigram, and trigram models with examples.
12. Describe the Paninian framework and its relevance in NLP.
13. What is Karaka theory? How is it applied in NLP?
14. Explain the limitations of the N-gram model in language modeling.
15. How does a probabilistic language model differ from a rule-based model?

​