DSTC 2 Results - Featured Metrics (public)

	A	B	C	D	E	F	G	H	I	J	K	L	M	N	O	P
1			Bold denotes best result in column, (over all systems irrespective of what inputs they take). L2 metric is lower the better. ROC metric is only comparable between systems of similar accuracy, in which case the higher is better.
2
3					Tracker Inputs			Joint Goals			Method			Requested Slots
4			Team	Entry	Live ASR	Live SLU	Batch ASR	Acc	L2	ROC	Acc	L2	ROC	Acc	L2	ROC
5	Baselines	baseline	0	0		✓		0.619	0.738	0.000	0.879	0.209	0.000	0.884	0.196	0.000
6		focus baseline		1		✓		0.719	0.464	0.000	0.867	0.210	0.349	0.879	0.206	0.000
7		HWU baseline		2		✓		0.711	0.466	0.000	0.897	0.158	0.000	0.884	0.201	0.000
8		HWU baseline orig.		3		✓		0.666	0.498	0.210	0.885	0.168	0.000	0.883	0.199	0.000
9			1	0		✓		0.601	0.649	0.064	0.904	0.155	0.187	0.960	0.073	0.000
10				1		✓		0.596	0.671	0.036	0.877	0.204	0.397	0.957	0.081	0.000
11			2	0	✓	✓		0.775	0.758	0.063	0.944	0.092	0.306	0.954	0.073	0.383
12				1	✓	✓	✓	0.784	0.735	0.065	0.947	0.087	0.355	0.957	0.068	0.446
13				2	✓			0.668	0.505	0.249	0.944	0.095	0.499	0.972	0.043	0.300
14				3	✓	✓	✓	0.771	0.354	0.313	0.947	0.093	0.294	0.941	0.090	0.262
15				4	✓	✓	✓	0.773	0.467	0.140	0.950	0.082	0.351	0.968	0.050	0.497
16			3	0		✓		0.729	0.452	0.000	0.878	0.210	0.000	0.889	0.188	0.000
17			4	0	✓			0.768	0.346	0.365	0.940	0.095	0.452	0.978	0.035	0.525
18				1	✓			0.746	0.381	0.383	0.939	0.097	0.423	0.977	0.038	0.490
19				2		✓		0.742	0.387	0.345	0.922	0.124	0.447	0.957	0.069	0.340
20				3		✓		0.737	0.406	0.321	0.922	0.125	0.406	0.957	0.073	0.385
21			5	0	✓	✓		0.686	0.628	0.000	0.889	0.221	0.000	0.868	0.264	0.000
22				1	✓	✓		0.609	0.782	0.000	0.927	0.147	0.000	0.974	0.053	0.000
23				2	✓	✓		0.637	0.726	0.000	0.927	0.147	0.000	0.974	0.053	0.000
24				3	✓	✓		0.609	0.782	0.000	0.927	0.147	0.000	0.974	0.053	0.000
25				4	✓	✓		0.695	0.610	0.000	0.927	0.147	0.000	0.974	0.053	0.000
26			6	0		✓		0.713	0.461	0.100	0.865	0.228	0.199	0.932	0.118	0.057
27				1		✓		0.707	0.447	0.223	0.871	0.211	0.290	0.947	0.093	0.218
28				2		✓		0.718	0.437	0.207	0.871	0.210	0.287	0.951	0.085	0.225
29			7	0	✓			0.750	0.416	0.081	0.936	0.105	0.237	0.970	0.056	0.000
30				1	✓			0.739	0.428	0.159	0.921	0.161	0.554	0.970	0.056	0.000
31				2	✓			0.750	0.416	0.081	0.929	0.117	0.379	0.971	0.054	0.000
32				3	✓			0.725	0.432	0.105	0.936	0.105	0.237	0.972	0.047	0.000
33				4		✓		0.735	0.433	0.086	0.910	0.140	0.280	0.946	0.089	0.190
34			8	0		✓		0.692	0.505	0.071	0.899	0.153	0.000	0.935	0.106	0.000
35				1		✓		0.699	0.498	0.067	0.899	0.153	0.000	0.939	0.101	0.000
36				2		✓		0.698	0.504	0.067	0.899	0.153	0.000	0.939	0.101	0.000
37				3		✓		0.697	0.501	0.068	0.899	0.153	0.000	0.939	0.101	0.000
38				4		✓		0.697	0.508	0.068	0.899	0.153	0.000	0.939	0.101	0.000
39			9	0		✓		0.499	0.760	0.000	0.857	0.229	0.000	0.905	0.149	0.000
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100