ABCDEFGHIJKLMNOPQRSTUVWXYZ
1
[Description] Convolutional Neural Network (CNN) - specialized architecture for image processing using local connectivity and weight sharing.
2
CNN Forward Propagation with Zero Padding
3
4
RGB(0-255) → Normalize → Standardize → ZeroPad → Conv2D → ReLU → MaxPool → FC → Sigmoid
5
 6×6×3 → 6×6×3 → 6×6×3 → 8×8×3 → 6×6 → 6×6 → 3×3 → 1 → 1
6
7
Zero Padding (p=1): Input preserved! 6×6 → 6×6
8
9
10
Kernels
11
12
K_RK_GK_B
bias = 0.0
13
10-11210-10
14
20-2000-14-1
15
10-1-1-2-10-10
16
17
18
19
20
21
22
Standardization (Z-score normalization) is critical for neural network training:
23
24
1. Improves Training Stability:
25
 • Ensures all input features have similar scales (mean=0, std=1)
26
 • Prevents features with large values from dominating the learning process
27
28
2. Prevents Gradient Vanishing/Exploding:
29
 • Large input values can cause very large or very small gradients
30
 • Standardization keeps gradients in a reasonable range
31
32
3. Faster Convergence:
33
 • Optimizers (like SGD, Adam) work better with standardized inputs
34
 • Learning rate can be set more easily and consistently
35
36
4. Better Weight Initialization:
37
 • Standard initialization techniques (Xavier, He) assume normalized inputs
38
 • Helps weights start in an optimal range
39
40
Formula: z = (x - mean) / std
41
Result: Transformed data has mean ≈ 0 and standard deviation ≈ 1
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100