1 of 10

ME 5990: Introduction to Machine Learning

�

Transformer Model Discussion

Slide Based on:

Lena Voita: https://lena-voita.github.io/nlp_course/seq2seq_and_attention.html

​

​

2 of 10

Outline

  • Transformer

3 of 10

Review: Seq-2-seq Encoder/Decoder

  • RNN with two languages

4 of 10

Review: Attention

  • The concept

5 of 10

Transformer

  • Transformer: attention is all you need
    • Cited 142609 as of December 2, 2024

6 of 10

Transformer

https://research.google/blog/transformer-a-novel-neural-network-architecture-for-language-understanding/

7 of 10

Transformer

8 of 10

Transformer

9 of 10

Transformer

10 of 10

Transformer

​

​