llm_deployment_pricing_eval

	A	B	C	D	E	F	G	H	I	J	K
1	GPU Type	Modal	Beam	HF Endpoints	Replicate	BentoML	Baseten	RunPod	EC2	SageMaker
2	NVIDIA T4	$0.59	$0.54	$0.50	$0.81	$0.51	$0.63		$0.53	$0.78	- on-demand comes packaged
3	NVIDIA L4	$0.80	N/A	$0.80	N/A	$1.01	$0.85	$0.69
4	NVIDIA A10G	$1.10	$1.05	$1.00	N/A	$1.21	$1.21		$1.01	$1.49	- on-demand comes packaged
5	NVIDIA A100 40GB	$2.78	$2.75	N/A	N/A	$3.74	N/A
6	NVIDIA A100 80GB	$3.40	N/A	$4.00	$5.04	N/A	$6.14	$2.72
7	NVIDIA H100	$4.56	$4.40	$4.00	N/A	N/A	$9.98
8
9										vCPU for T4 and L4 and A10G is usually 4 for on-demand (except for BentoML that has 8 vCPU for L4)
10	Platform	vCPUs	Memory (GB)	Price per Hour ($)	Price per vCPU per Hour ($)					13-16GB for T4
11	Modal	1	Custom	$0.0675 (CPU) + RAM	$0.07
12	Beam	1	Custom	$0.095 (CPU) + RAM	$0.10
13	HF Endpoints	1	2	$0.03	$0.03
14	BentoML	1	4	$0.05	$0.05
15	Baseten	1	2	$0.03	$0.03
16	AWS Lambda	~1	1.792	$0.11	$0.11
17
18	Platform	vCPUs	Memory (GB)	Price per Hour ($)	Price per vCPU per Hour ($)
19	Modal	2	3	$0.135 (CPU) + $0.072 (Memory)	$0.07
20	Beam	2	3	$0.190 (CPU) + $0.060 (Memory)	$0.10
21	HF Endpoints	2	4	$0.07	$0.04
22	BentoML	2	8	$0.10	$0.05
23	Baseten	2	8	$0.10	$0.05
24	AWS Lambda	~2	3.584	$0.22	$0.11
25
26	Platform	vCPUs	Memory (GB)	Price per Hour ($)	Price per vCPU per Hour ($)
27	Modal	4	Custom	$0.27 (CPU) + RAM	$0.07
28	Beam	4	Custom	$0.38 (CPU) + RAM	$0.10
29	HF Endpoints	4	8	$0.13	$0.03
30	BentoML	4	16	$0.20	$0.05
31	Baseten	4	16	$0.21	$0.05
32	Replicate	4	8	$0.36	$0.09	- no control over CPU
33	AWS Lambda	~4	7.168	$0.43	$0.11
34
35	Observations
36	AWS Lambda has higher per vCPU costs compared to other platforms due to its pricing model and the way CPU power scales with memory allocation.
37	HF Endpoints offer the lowest per vCPU pricing, especially for larger instances.
38	Baseten and BentoML provide competitive pricing with bundled RAM, which can be advantageous for memory-intensive applications.
39	Modal and Beam allow granular control over CPU and RAM but may result in higher costs when combining resources.
40	AWS Lambda is cost-effective for event-driven, short-duration tasks rather than continuous workloads.
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100