Natural Language Processing 101

Vaibhav Srivastav | Deloitte Consulting LLP | @reach_vb |

Who am I?

I build smart things for the web, and make sense out of data (atleast I try)

The Audience (You!)

  • Know some A/IM already?
  • Know some NLP already?
  • Both / None of the above?

To get a sense of the kind of audience we are dealing with?

What is Natural Language Processing?

  • A field of Artificial Intelligence which enables computers to analyze and understand the human language.
  • NLP was formulated to build software that generates and understand natural languages so that a user can have natural conversations with his computer.

Computers are really powerful, how can we leverage their power to do our day to day jobs without always coding something or the other

For example how can I get to know about the weather in Seattle without explicitly going to my browser and going to a website entering Seattle and then finding about it

Or, how can I write a message to someone without explicitly opening the message app or the keyboard

How can I translate a text from English to Japanese

And so on and so forth


To find restaurants serving Nasi Lemak in TTDI:

SELECT restaurant_name, restaurant_address FROM restaurants WHERE area = ‘Kuala Lumpur’ AND food_type = ‘Nasi Lemak’

Natural way:

Where can I find some Nasi Lemak in TTDI?

In a typical SQL setup this how things work

But this involves coding knowledge and not everyone is a coder :P

Natural Language Applications

  • Text Classification
  • Text Summarisation
  • Machine Translation
  • Search
  • Information Extraction (Sentiment, LDA, LSI)
  • Question Answering

Explain each point with examples

Text Classification

Assigning categories to documents, which can be a web page, library book, media articles, gallery etc. has many applications like e.g. spam filtering, email routing, sentiment analysis etc

Text Summarisation

Create a summary with the major points of the original document. Technologies that can make a coherent summary take into account variables such as length, writing style and syntax.

Machine Translation

3 Common Approaches to NLP

  • Rule-Based Approach
  • Statistical Analysis
  • Machine Learning

Demo in python

Why is NLP so Hard?

  • No perfect solution exists
  • Language itself is a moving target
  • Computationally complex

Way ahead?



Demystifying Natural Language Processing - Google Slides