Apache Beam Portability Support Matrix
 Share
The version of the browser you are using is no longer supported. Please upgrade to a supported browser.Dismiss

Comment only
 
 
ABCDEFGHIJKLMNOPQRSTUVWXYZAAABACADAEAFAGAH
1
Apache Beam Portability Support Matrix - https://s.apache.org/apache-beam-portability-support-table
2
3
This spreadsheet tracks the status of portability support across SDKs and runners. The goal is to identify the gaps and implementation irregularies.
4
The table represents the 4D status of "Feature|Example x SDK x Runner x Batch|Streaming" flattened as "Feature|Example x (SDK x Runner x Batch|Streaming)".
5
6
Legend
7
Works and has pre/post-commit regression test. Yay!
8
Works, based on manual verification. Test desirable.
9
"Should work". To be verified.
10
BEAM-xxx
Partially works. Cell should contain JIRA. Comment may contain elaboration. Consider splitting the row into "works" and "doesn't work" instead.
11
BEAM-xxx
Does not work. Cell should contain JIRA.
12
No information. To be evaluated.
13
N/A
14
15
16
"What works?"
RUNNER x SDK x Batch|Streaming
17
18
Direct (specific to each SDK and not necessarily portable)
Universal Local Runner
Flink (master)
instructionsDataflow
Spark (master)
instructionsSamza
19
JavaPythonGoJavaPythonGoJavaPythonGoJavaPythonGoJavaPythonGoJavaPythonGo
20
FEATURE
Batch|Streaming
BatchStreamingBatch only
Batch|Streaming
Batch|Streaming
Batch|Streaming
BatchStreamingBatchStreamingBatchStreamingBatchStreamingBatchStreamingBatchStreamingBatchStreamingBatchStreamingBatchStreamingBatchStreamingBatchStreamingBatchStreaming
21
Impulse
22
ParDo
23
w/ side inputBEAM-2928BEAM-2928BEAM-3286BEAM-3286BEAM-3286BEAM-3286BEAM-3286BEAM-3286BEAM-3286BEAM-7564BEAM-7564BEAM-7564BEAM-7564
24
w/ multiple output
25
w/ user stateBEAM-3298
BEAM-2917/BEAM-2915
BEAM-2917/BEAM-2916
BEAM-2917/BEAM-3298
BEAM-2918/BEAM-3298
BEAM-2918/BEAM-3298
BEAM-2902BEAM-2902BEAM-2902BEAM-2902
BEAM-2902/BEAM-3298
BEAM-2902/BEAM-3298
BEAM-2902/BEAM-3298
BEAM-2902/BEAM-3298
BEAM-7564BEAM-7564BEAM-7564BEAM-7564
26
w/ user timersBEAM-4680BEAM-4680BEAM-4680BEAM-7564BEAM-7564BEAM-7564BEAM-7564
27
w/ user metricsBEAM-5452BEAM-5452BEAM-5452BEAM-7564BEAM-7564BEAM-7564BEAM-7564
28
Flatten
29
w/ explicit flattenBEAM-3300BEAM-3300BEAM-3300BEAM-3300BEAM-3300BEAM-3300BEAM-3300
30
Combine
31
w/ first-class repBEAM-3708BEAM-5453BEAM-5453BEAM-4276BEAM-4276BEAM-4276BEAM-3513BEAM-3513BEAM-4276BEAM-4276BEAM-4276BEAM-4276
32
w/ liftingBEAM-3708BEAM-4679BEAM-4679BEAM-4276BEAM-4276BEAM-4276BEAM-3711BEAM-3711BEAM-4276BEAM-4276BEAM-4276BEAM-4276
33
SDFBEAM-3743BEAM-3743BEAM-3301BEAM-3301BEAM-3301BEAM-3301BEAM-3301BEAM-3301BEAM-3301
34
w/ liquid shardingBEAM-4287
35
GBK
36
CoGBK
37
WindowInto
38
w/ sessionsBEAM-4152BEAM-4152BEAM-4152BEAM-4152BEAM-4152BEAM-4152BEAM-4152BEAM-4152
39
w/ custom windowfn
40
EXAMPLE
Batch|Streaming
BatchStreamingBatch onlyBatchBatchBatchBatchStreamingBatchStreamingBatchStreamingBatchStreamingBatchStreamingBatchStreamingBatchStreamingBatchStreamingBatchStreamingBatchStreamingBatchStreamingBatchStreaming
41
WordCapBEAM-7566BEAM-7566
42
WordCountBEAM-7566BEAM-7566
43
w/ write to Sink
44
w/ write to GCS
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
Loading...