DSTC9 Track 3 Interactive Evaluation of Dialog

	A	B	C	D	E	F	G	H	I	J	K	L	M	N	O	P	Q
1			Human Annotation (100 utterances x 3 annotators/utterance)									Automatic Evaluation
2	Model Name	Rank	Overall Human Rating	Interesting	Engaging	Specific	Relevant	Correct	Semantically Appropriate	Understandable	Fluent	USR	USR-MLM	USR-DRc	USR-DRf	METEOR	BERTScore
3	diverse-cap-model_v0	1	4.281355932	2.874576271	2.854237288	2.562711864	2.881355932	2.66779661	2.877966102	0.993220339	2.911864407	4.257584587	-0.819869629	0.956314202	0.587511715	0.090553425	0.849129942
4	more-diverse-response_v3	1	4.280405405	2.814189189	2.847972973	2.543918919	2.868243243	2.652027027	2.868243243	0.996610169	2.888513514	4.586287669	-0.848447149	0.976160915	0.830832588	0.131123267	0.861703992
5	final-results-test-plato-best-1004_v0	1	4.279863481	2.839590444	2.870307167	2.563139932	2.897610922	2.686006826	2.914675768	0.996587031	2.914675768	3.863344795	-0.943333307	0.945558127	0.583119988	0.068322597	0.84358041
6	nofact-data-finetune_v0	4	4.260273973	2.818493151	2.808219178	2.606164384	2.849315068	2.684931507	2.804794521	1	2.907534247	4.25778592	-0.794782852	0.944482947	0.579268161	0.089637963	0.851451581
7	dialogpt-fact-dialogrpt-v0_v0	5	4.252559727	2.860068259	2.877133106	2.576791809	2.870307167	2.716723549	2.90443686	0.996587031	2.907849829	4.832009598	-0.688407393	0.952151433	0.773343711	0.123706551	0.862131507
8	gpt2-ed-baseline_v4		4.231034483	2.820689655	2.810344828	2.620689655	2.851724138	2.710344828	2.848275862	0.986206897	2.862068966	4.730429174	-0.756914645	0.949381651	0.850683485	0.123127742	0.863243632
9	finetune-with-large-model_v4		4.229452055	2.808219178	2.797945205	2.630136986	2.808219178	2.640410959	2.897260274	1	2.869863014	4.477763749	-0.8351279	0.939294491	0.858129178	0.139566775	0.868364188
10	filter-data-finetune_v0		4.228668942	2.846416382	2.825938567	2.610921502	2.870307167	2.699658703	2.873720137	0.989761092	2.880546075	4.451502643	-0.887475596	0.974101114	0.816339492	0.125130885	0.85913778
11	diverse-ensemble-model_v0		4.215753425	2.821917808	2.842465753	2.619863014	2.825342466	2.691780822	2.876712329	0.989726027	2.886986301	4.464064719	-0.830850407	0.963787764	0.744547939	0.121360467	0.859138791
12	cruzcontrol-baseline-bert-knowledge_v1		4.209621993	2.848797251	2.869415808	2.594501718	2.821305842	2.656357388	2.865979381	0.993127148	2.890034364	4.534744193	-0.814119522	0.950266473	0.815980143	0.108715873	0.856524662
13	final-results-test-1004-m-w_v0		4.205574913	2.81184669	2.857142857	2.613240418	2.832752613	2.679442509	2.891986063	0.993031359	2.898954704	4.509827172	-0.802372132	0.948060807	0.779726891	0.159973491	0.873781577
14	power-dialog-chat-k_v0		4.178947368	2.810526316	2.866666667	2.484210526	2.835087719	2.596491228	2.866666667	0.992982456	2.915789474	2.595068664	-0.904790748	0.69897031	0.447888921	0.073984414	0.843421203
15	pretrained-diverse-v2_v10		4.177474403	2.829351536	2.805460751	2.539249147	2.825938567	2.624573379	2.815699659	0.996587031	2.880546075	4.977801334	-0.767958896	0.99027404	0.912152596	0.134986256	0.86494546
16	tx-zp-nofact-ensemble-rerank_v0		4.177474403	2.778156997	2.784982935	2.576791809	2.846416382	2.671232877	2.846416382	0.993174061	2.880546075	4.618743837	-0.697689326	0.945455908	0.651337798	0.109542189	0.856891338
17	final-results-test-online-w_v0		4.171821306	2.810996564	2.841924399	2.593103448	2.797250859	2.686206897	2.862542955	0.996563574	2.924398625	3.867634857	-0.966773073	0.938687382	0.662895183	0.071852437	0.844197832
18	final-results-test-rpt-all-w_v0		4.167235495	2.815699659	2.805460751	2.594501718	2.791808874	2.62116041	2.825938567	0.986348123	2.928327645	3.958319994	-0.947533496	0.944281999	0.671561213	0.082669727	0.847489246
19	cruzcontrol-swbd-pd-nrg-topline_v1		4.156996587	2.778156997	2.764505119	2.556313993	2.767918089	2.57337884	2.832764505	0.972696246	2.907849829	3.404301374	-0.861552287	0.744478666	0.816806922	0.113086054	0.857690959
20	tx-zp-fact-ensemble-rerank-rerank_v0		4.152249135	2.778546713	2.771626298	2.55017301	2.820069204	2.698961938	2.84083045	0.993079585	2.903114187	4.863434695	-0.686207111	0.9597205	0.764365811	0.122839438	0.860801831
21	transformer-gpt2-like_v1		4.151724138	2.789655172	2.786206897	2.527586207	2.782758621	2.617241379	2.85862069	0.993079585	2.917241379	2.467925677	-0.910408693	0.675769168	0.448800786	0.073211601	0.842835034
22	tx-zp-fact-ensemble-rerank_v0		4.147766323	2.810996564	2.817869416	2.508591065	2.821305842	2.687285223	2.865979381	0.993127148	2.886597938	4.833800817	-0.692552803	0.957103338	0.764676786	0.121513894	0.861417566
23	dialogpt-large-finetune-fact-head-V1_v1		4.139931741	2.744027304	2.723549488	2.566552901	2.839590444	2.626712329	2.852739726	1	2.880546075	4.551654628	-0.728021912	0.92630894	0.736755869	0.110690084	0.859520007
24	dialogpt-small-fintune-fact-head_v2		4.130136986	2.797945205	2.801369863	2.54109589	2.79109589	2.657534247	2.835616438	0.993150685	2.897260274	4.125210576	-0.737907456	0.881631952	0.593908442	0.089912217	0.85323514
25	final-results-test-1004-sum-w_v0		4.129692833	2.805460751	2.795221843	2.535836177	2.812286689	2.679180887	2.853242321	0.989761092	2.907849829	4.34279219	-0.830549966	0.930817671	0.774774374	0.147137404	0.875802402
26	finetune-with-mid-model_v0		4.129692833	2.822525597	2.825938567	2.593856655	2.788395904	2.62116041	2.853242321	0.979522184	2.853242321	4.909336877	-0.724955454	0.958853171	0.886595113	0.156175097	0.868694613
27	middle-diverse-response_v0		4.128027682	2.837370242	2.816608997	2.619377163	2.820069204	2.6816609	2.844290657	0.989619377	2.896193772	4.409917531	-0.902945131	0.974797237	0.81410648	0.119996832	0.858372706
28	cruzcontrol-bert-sentence-knowledge_v2		4.117241379	2.75862069	2.8	2.493103448	2.827586207	2.593103448	2.824137931	0.982758621	2.9	3.96149749	-0.843945813	0.876582584	0.71144272	0.11901896	0.859797703
29	final-results-test-rpt-w_v0		4.111888112	2.783216783	2.765734266	2.51048951	2.793706294	2.608391608	2.84965035	0.98951049	2.877622378	4.46641587	-0.811850137	0.944561967	0.779290214	0.154020814	0.875045863
30	sk-attn-300-test_v0		4.089347079	2.783505155	2.793814433	2.484536082	2.766323024	2.606896552	2.862542955	0.989690722	2.890034364	1.713615542	-0.944806902	0.560547914	0.37072612	0.0549213	0.83887378
31	sk-attn-100-test_v0		4.086206897	2.751724138	2.779310345	2.486206897	2.775862069	2.565517241	2.84137931	0.986206897	2.903448276	1.400817344	-0.9665904	0.511743687	0.358347692	0.048778295	0.836409242
32	GPT-medium-fact-plus-history_v0		4.078767123	2.787671233	2.801369863	2.534246575	2.794520548	2.616438356	2.825342466	0.976027397	2.832191781	4.686322492	-0.775205865	0.94813252	0.859944735	0.12771603	0.859424687
33	cruzcontrol-kd-pd-nrg-topline_v1		4.031358885	2.770034843	2.801393728	2.540069686	2.780487805	2.571428571	2.770034843	0.989547038	2.923344948	3.318520033	-0.936077676	0.779692223	0.774153157	0.089545329	0.848289946
34	fine-tune-on-DialoGPT-M_v3		3.925423729	2.66779661	2.742372881	2.481355932	2.701694915	2.542372881	2.745762712	0.962585034	2.837288136	1.674030843	-1.101308976	0.630596211	0.406436807	0.046321122	0.831202384
35	mdb-ret-20M_v0		3.883161512	2.639175258	2.704467354	2.474048443	2.738831615	2.563573883	2.766323024	0.951890034	2.81443299	1.351498884	-1.344883991	0.695062469	0.427532156	0.032711137	0.822683892
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100