ABCDEFGHIJKLMNOPQRSTUVWXYZAAABACADAE
1
[Description] Gated Recurrent Unit (GRU) - simplified variant of LSTM with only 2 gates, fewer parameters but similar performance.
2
GRU (Gated Recurrent Unit)
3
4
═══ GRU: Simplified Version of LSTM ═══
5
6
• Reduced LSTM's 4 gates to 2 (Reset, Update)
7
• Use only Hidden State without Cell State
8
• Similar performance with fewer parameters
9
• Faster learning and more computationally efficient
10
11
12
═══ GRU Architecture ═══
13
14
Reset Gate (r_t)
How much to forget past information
σ(W_r·[h_{t-1}, x_t] + b_r)
15
Update Gate (z_t)
Ratio of past and new information
σ(W_z·[h_{t-1}, x_t] + b_z)
16
Candidate (h̃_t)
New candidate hidden state
tanh(W_h·[r_t⊙h_{t-1}, x_t] + b_h)
17
18
19
═══ Hidden State Update ═══
20
21
h_t = (1 - z_t) ⊙ h_{t-1} + z_t ⊙ h̃_t
22
23
• (1 - z_t): Ratio of preserving past information
24
• z_t: Ratio of new information
25
• z_t ≈ 0: Keep past info (h_t ≈ h_{t-1})
26
• z_t ≈ 1: Replace with new info (h_t ≈ h̃_t)
27
28
29
═══ GRU Cell Diagram ═══
30
31
h_{t-1}────→
×(1-z_t)
───────→
+──→h_t
32
│↑
33
×r_t──→h̃_t──→×z_t──↗
34
35
x_t──→
[concat]
──→
r_t, z_t
36
37
38
Input: 'this movie is good' (y_true = 1)
39
Embedding dim: 4, Hidden dim: 3
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100