1 of 10

2 of 10

3 of 10

4 of 10

Review 01-2

CSE 10124

5 of 10

Submitted Questions

  • for the exam can you go over the scoring structure and how you will grade open ended questions (with examples). How lenient will you be with definitions etc. Thank you
  • help going over q9 in the text as data chapter in the practice packet

6 of 10

7 of 10

9.) Consider the following code that builds a word-level graph. Explain what the function does and what the output graph would look like for the input ["the", "cat", "sat"].

8 of 10

An Aside: Markov Models

9 of 10

irishQwen

Qwen3 (0.6B)

Linear output layer

Final RMSNorm

Feed forward

RMSNorm 2

Masked grouped-query

attention

RMSNorm 1

×28

Token embedding layer

Tokenized text

Sample input text

QK-Norm

RoPE

Linear layer

⊙

SiLU activation

Linear

layer

Linear layer

Vocabulary size of 151k

Hidden layer dimension of 3,072

16 query heads, 8 KV groups

Embedding dimension of 1,024

Supported context length of 32k tokens

+

+

10 of 10

irishQwen

Qwen3 (0.6B)

×28

Sample input text

⊙

Vocabulary size of 151k

Hidden layer dimension of 3,072

16 query heads, 8 KV groups

Embedding dimension of 1,024

Supported context length of 32k tokens

+

+