BMVC 19 - A Less Biased Evaluation of Out-of-distribution Sample Detectors -- Supplementary Material

	A	B	C	D	E	G	H	I
1	Sub Results					Average Results
2	Method	D_s	D_v	D_t	Accuracy	Method	Mean Test Accuracy	# Experiments
3	1-NNSVM	MNIST	UniformNoise	NormalNoise	1	ODIN/VGG	0.7866428571	308
4	1-NNSVM	MNIST	UniformNoise	FashionMNIST	0.858	ScoreSVM/VGG	0.7716623377	308
5	1-NNSVM	MNIST	UniformNoise	NotMNIST	0.988	ODIN/Res	0.7698733766	308
6	1-NNSVM	MNIST	UniformNoise	CIFAR10	0.985	Log.SVM/VGG	0.7467954545	308
7	1-NNSVM	MNIST	UniformNoise	CIFAR100	0.979	AEThre./BCE	0.7443019481	308
8	1-NNSVM	MNIST	UniformNoise	STL10	0.983	MC-Dropout	0.7347435065	308
9	1-NNSVM	MNIST	UniformNoise	TinyImagenet	0.976	1-NNSVM	0.7337402597	308
10	1-NNSVM	MNIST	NormalNoise	UniformNoise	1	2-NNSVM	0.7294383117	308
11	1-NNSVM	MNIST	NormalNoise	FashionMNIST	0.767	DeepEns./VGG	0.7284253247	308
12	1-NNSVM	MNIST	NormalNoise	NotMNIST	0.977	4-NNSVM	0.7263733766	308
13	1-NNSVM	MNIST	NormalNoise	CIFAR10	0.966	PbThresh/VGG	0.7263409091	308
14	1-NNSVM	MNIST	NormalNoise	CIFAR100	0.958	8-NNSVM	0.7262337662	308
15	1-NNSVM	MNIST	NormalNoise	STL10	0.96	4-BNNSVM	0.7209188312	308
16	1-NNSVM	MNIST	NormalNoise	TinyImagenet	0.952	8-BNNSVM	0.7188506494	308
17	1-NNSVM	MNIST	FashionMNIST	UniformNoise	0.986	2-BNNSVM	0.7181915584	308
18	1-NNSVM	MNIST	FashionMNIST	NormalNoise	0.986	1-BNNSVM	0.7178506494	308
19	1-NNSVM	MNIST	FashionMNIST	NotMNIST	0.982	1-MNNSVM	0.7142597403	308
20	1-NNSVM	MNIST	FashionMNIST	CIFAR10	0.983	8-MNNSVM	0.7130681818	308
21	1-NNSVM	MNIST	FashionMNIST	CIFAR100	0.979	4-MNNSVM	0.7125746753	308
22	1-NNSVM	MNIST	FashionMNIST	STL10	0.982	2-MNNSVM	0.711461039	308
23	1-NNSVM	MNIST	FashionMNIST	TinyImagenet	0.979	AEThre./MSE	0.706275974	308
24	1-NNSVM	MNIST	NotMNIST	UniformNoise	0.998	PbThresh/Res	0.7048149351	308
25	1-NNSVM	MNIST	NotMNIST	NormalNoise	0.998	ScoreSVM/Res	0.6987597403	308
26	1-NNSVM	MNIST	NotMNIST	FashionMNIST	0.916	BinClass/Res	0.6906493506	308
27	1-NNSVM	MNIST	NotMNIST	CIFAR10	0.992	Log.SVM/Res	0.6897597403	308
28	1-NNSVM	MNIST	NotMNIST	CIFAR100	0.986	BinClass/VGG	0.6881201299	308
29	1-NNSVM	MNIST	NotMNIST	STL10	0.99	4-VNNSVM	0.6653863636	308
30	1-NNSVM	MNIST	NotMNIST	TinyImagenet	0.985	2-VNNSVM	0.6599902597	308
31	1-NNSVM	MNIST	CIFAR10	UniformNoise	0.999	8-VNNSVM	0.6577792208	308
32	1-NNSVM	MNIST	CIFAR10	NormalNoise	0.999	DeepEns./Res	0.6568279221	308
33	1-NNSVM	MNIST	CIFAR10	FashionMNIST	0.896	1-VNNSVM	0.6515584416	308
34	1-NNSVM	MNIST	CIFAR10	NotMNIST	0.99	OpenMax/VGG	0.6230551948	308
35	1-NNSVM	MNIST	CIFAR10	CIFAR100	0.985	PixelCNN++	0.6074025974	308
36	1-NNSVM	MNIST	CIFAR10	STL10	0.988	OpenMax/Res	0.5844512987	308
37	1-NNSVM	MNIST	CIFAR10	TinyImagenet	0.983
38	1-NNSVM	MNIST	CIFAR100	UniformNoise	0.994
39	1-NNSVM	MNIST	CIFAR100	NormalNoise	0.994
40	1-NNSVM	MNIST	CIFAR100	FashionMNIST	0.934
41	1-NNSVM	MNIST	CIFAR100	NotMNIST	0.989
42	1-NNSVM	MNIST	CIFAR100	CIFAR10	0.991
43	1-NNSVM	MNIST	CIFAR100	STL10	0.989
44	1-NNSVM	MNIST	CIFAR100	TinyImagenet	0.985
45	1-NNSVM	MNIST	STL10	UniformNoise	0.998
46	1-NNSVM	MNIST	STL10	NormalNoise	0.998
47	1-NNSVM	MNIST	STL10	FashionMNIST	0.908
48	1-NNSVM	MNIST	STL10	NotMNIST	0.991
49	1-NNSVM	MNIST	STL10	CIFAR10	0.991
50	1-NNSVM	MNIST	STL10	CIFAR100	0.986
51	1-NNSVM	MNIST	STL10	TinyImagenet	0.984
52	1-NNSVM	MNIST	TinyImagenet	UniformNoise	0.995
53	1-NNSVM	MNIST	TinyImagenet	NormalNoise	0.995
54	1-NNSVM	MNIST	TinyImagenet	FashionMNIST	0.933
55	1-NNSVM	MNIST	TinyImagenet	NotMNIST	0.99
56	1-NNSVM	MNIST	TinyImagenet	CIFAR10	0.991
57	1-NNSVM	MNIST	TinyImagenet	CIFAR100	0.986
58	1-NNSVM	MNIST	TinyImagenet	STL10	0.99
59	2-NNSVM	MNIST	UniformNoise	NormalNoise	1
60	2-NNSVM	MNIST	UniformNoise	FashionMNIST	0.822
61	2-NNSVM	MNIST	UniformNoise	NotMNIST	0.984
62	2-NNSVM	MNIST	UniformNoise	CIFAR10	0.978
63	2-NNSVM	MNIST	UniformNoise	CIFAR100	0.971
64	2-NNSVM	MNIST	UniformNoise	STL10	0.975
65	2-NNSVM	MNIST	UniformNoise	TinyImagenet	0.966
66	2-NNSVM	MNIST	NormalNoise	UniformNoise	1
67	2-NNSVM	MNIST	NormalNoise	FashionMNIST	0.863
68	2-NNSVM	MNIST	NormalNoise	NotMNIST	0.988
69	2-NNSVM	MNIST	NormalNoise	CIFAR10	0.986
70	2-NNSVM	MNIST	NormalNoise	CIFAR100	0.979
71	2-NNSVM	MNIST	NormalNoise	STL10	0.983
72	2-NNSVM	MNIST	NormalNoise	TinyImagenet	0.977
73	2-NNSVM	MNIST	FashionMNIST	UniformNoise	0.975
74	2-NNSVM	MNIST	FashionMNIST	NormalNoise	0.975
75	2-NNSVM	MNIST	FashionMNIST	NotMNIST	0.972
76	2-NNSVM	MNIST	FashionMNIST	CIFAR10	0.973
77	2-NNSVM	MNIST	FashionMNIST	CIFAR100	0.97
78	2-NNSVM	MNIST	FashionMNIST	STL10	0.972
79	2-NNSVM	MNIST	FashionMNIST	TinyImagenet	0.969
80	2-NNSVM	MNIST	NotMNIST	UniformNoise	0.998
81	2-NNSVM	MNIST	NotMNIST	NormalNoise	0.998
82	2-NNSVM	MNIST	NotMNIST	FashionMNIST	0.913
83	2-NNSVM	MNIST	NotMNIST	CIFAR10	0.991
84	2-NNSVM	MNIST	NotMNIST	CIFAR100	0.986
85	2-NNSVM	MNIST	NotMNIST	STL10	0.99
86	2-NNSVM	MNIST	NotMNIST	TinyImagenet	0.984
87	2-NNSVM	MNIST	CIFAR10	UniformNoise	0.998
88	2-NNSVM	MNIST	CIFAR10	NormalNoise	0.998
89	2-NNSVM	MNIST	CIFAR10	FashionMNIST	0.909
90	2-NNSVM	MNIST	CIFAR10	NotMNIST	0.991
91	2-NNSVM	MNIST	CIFAR10	CIFAR100	0.985
92	2-NNSVM	MNIST	CIFAR10	STL10	0.989
93	2-NNSVM	MNIST	CIFAR10	TinyImagenet	0.984
94	2-NNSVM	MNIST	CIFAR100	UniformNoise	0.997
95	2-NNSVM	MNIST	CIFAR100	NormalNoise	0.997
96	2-NNSVM	MNIST	CIFAR100	FashionMNIST	0.922
97	2-NNSVM	MNIST	CIFAR100	NotMNIST	0.991
98	2-NNSVM	MNIST	CIFAR100	CIFAR10	0.992
99	2-NNSVM	MNIST	CIFAR100	STL10	0.99
100	2-NNSVM	MNIST	CIFAR100	TinyImagenet	0.985