1 of 23

Lecture 12 : Learning Word Representation - Part 1

2 of 23

  • Learning Word Vectors - Overview
  • Objective Function
  • Example Problem, deriving the gradients

​

3 of 23

Learning Word Vectors: Overview

4 of 23

Two Variations: CBOW and Skip-grams

5 of 23

Word2Vec (Skip-gram) Overview

Example windows and process for computing P(wt+j |wt )

6 of 23

Word2Vec (Skip-gram) Overview

Example windows and process for computing P(wt+j |wt )

7 of 23

Word2Vec: objective function

8 of 23

Understanding P(o|c) further

9 of 23

More Explanation

10 of 23

More Explanation

11 of 23

Try this problem

12 of 23

Try this problem

13 of 23

Gradient Descent for Parameter Updates

14 of 23

Training the model

15 of 23

Training the model

Source: https://web.stanford.edu/class/cs224n/slides/cs224n-spr2024-lecture01-wordvecs1.pdf

16 of 23

Source: https://web.stanford.edu/class/cs224n/slides/cs224n-spr2024-lecture01-wordvecs1.pdf

17 of 23

Source: https://web.stanford.edu/class/cs224n/slides/cs224n-spr2024-lecture01-wordvecs1.pdf

18 of 23

Source: https://web.stanford.edu/class/cs224n/slides/cs224n-spr2024-lecture01-wordvecs1.pdf

19 of 23

Source: https://web.stanford.edu/class/cs224n/slides/cs224n-spr2024-lecture01-wordvecs1.pdf

20 of 23

Visualization

21 of 23

Issues with word2Vec

22 of 23

Daniel Jurafsky and James H. Martin. 2024. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition with Language Models, 3rd edition. Online manuscript released August 20, 2024. [Chapter 6]

​

​

​

​

23 of 23