GPT OOMs

	A	B	C	D	E	F	G	H	I	J
1	Model	Published	Month gap	Training FLOP	Training FLOP factor increase	Training FLOP OOM increase	Alg improvements factor increase	Alg improvements OOM increase	Effective compute factor increase	Effective compute OOM increase
2	GPT-1	Jun-2018	-	1.75781E+19	-	-	-	-	-	-
3	GPT-2	Feb-2019	8	1.92E+21	109	2.03832868	2.0	0.3	219.9	2.3
4	GPT-3	May-2020	15	3.14E+23	164	2.213628419	3.7	0.5	611.6	2.7
5	GPT-4	Mar-2023	34	2.10E+25	67	1.825289647	8.5	1.1	568.4	3.0
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100