1 of 38

CS 162: Natural Language Processing

Lecture 1

Saadia Gabriel

2 of 38

3 of 38

First, what brought you here?

4 of 38

What is Natural Language Processing?

  • We develop NLP algorithms to make sense of human language…

The intersection of computer science, artificial intelligence and computational linguistics

  • And to produce language that mimics human language

5 of 38

Key Aspects of NLP

  • Identify the structure and meaning of words, then sentences, texts and full conversations
  • Use this to power many real-world applications!
  • Use this knowledge to design computational algorithms

6 of 38

Real-world Applications of NLP

7 of 38

A Few Recent Breakthroughs

8 of 38

A Few Recent Breakthroughs

9 of 38

A Few Recent Breakthroughs

November 2014

10 of 38

A Few Recent Breakthroughs

11 of 38

GPT2 in 2019

​

A Few Recent Breakthroughs

12 of 38

Now most people use NLP in their daily lives…

Claude,

Copilot: Launched 2023

​

Gemini: Launched 2024

​

13 of 38

NLP

NLP ≠ LLM Science

LLMs

But we will focus heavily on modern NLP and large-scale language modeling in the second half

14 of 38

15 of 38

How do we get to LLMs?

This is not alchemy! It starts with understanding the mechanics of human language.

16 of 38

Levels of Language

17 of 38

Levels of Language

18 of 38

Levels of Language

19 of 38

Major Challenges in NLP

Ambiguity

Commonsense tells us this probably means the painting was next to the tree, not that a tree found the painting…

Probably the kids are making the snacks, not that they are the snacks, unless this is satire…

20 of 38

Administration:�The Annotated Syllabus

21 of 38

We usually won’t use the entire class time, but there will be exams and in-class demos

The Bruin Learn page is coming soon with zoom links

Your awesome TAs

https://saadiagabriel.com/cs162_s26.html

22 of 38

Starting from week 2, submit a question after each lecture

23 of 38

There will be 3 homework assignments with MCQs and coding

24 of 38

One of 2 exams

First TA lecture

You will form your own final project groups and indicate what kind of paper you’d like to work on

25 of 38

Submit a short report on your progress and sign up for a paper presentation slot

26 of 38

27 of 38

How is this all graded?

28 of 38

29 of 38

30 of 38

31 of 38

32 of 38

33 of 38

34 of 38

Other Software Use Rules

You may use existing software packages (with acknowledgement of the packages used), unless an assignment explicitly prohibits this.

You may not use AI coding assistants for homework assignments, but you may use them for the final project (with acknowledgement).

35 of 38

36 of 38

NOTE: Extra credit will be given for discussion section attendance.

37 of 38

38 of 38

Waitlist and PTEs

The class will be capped at 120 students. I will add students as spots open up. Students with dropped enrollment will be prioritized.

In-class participation

It’s very important to show up in-person as much as possible, though there will be a zoom link after today. In the next few weeks, learning assistants will be present to help you with in-class exercises.