topRuns-rnd2 - Google Drive


1	tag	team	type	judged?	NDCG@10	P@5	RBP(p=.5)	bpref	MAP	score-report
2	mpiid5_run3	mpiid5	manual	JUDGED	0.6893	0.8514	0.7547	0.5679	0.338	report	TREC-COVID: Effectiveness of Top-scoring Round 2 Runs
3	mpiid5_run2	mpiid5	manual	UNJUDGED	0.6864	0.8057	0.7472	0.4943	0.3185	report
4	SparseDenseSciBert	CMT	feedback	JUDGED	0.6772	0.76	0.752	0.5096	0.3115	report	Notes:
5	mpiid5_run1	mpiid5	feedback	UNJUDGED	0.6677	0.7771	0.7305	0.4609	0.2946	report	1. This consists of all runs that rank in the top 50 runs for at least one of the measures.
6	UIowaS_Run3	UIowaS	feedback	UNJUDGED	0.6382	0.7657	0.7037	0.4867	0.2845	report	Runs that did not make it into the top 50 for any measure may mave better
7	UIowaS_Run1	UIowaS	feedback	JUDGED	0.6365	0.76	0.7042	0.4853	0.283	report	scores for some measures than the runs at the bottom of this list for
8	UIowaS_Run2	UIowaS	feedback	UNJUDGED	0.6349	0.7714	0.699	0.4887	0.2855	report	those measures.
9	UPrrf16lgbertd50-r2	unique_ptr	feedback	JUDGED	0.632	0.7086	0.7153	0.4414	0.3	report	2. A total of 136 runs from 51 teams was submitted, counting two baseline runs
10	GUIR_S2_run2	GUIR_S2	feedback	JUDGED	0.6286	0.7771	0.6805	0.4067	0.2531	report	in those totals.
11	ReInfoSelect	CMT	feedback	UNJUDGED	0.6259	0.6971	0.667	0.4435	0.285	report	3. Scores are means over the 35 topics in the Round 2 test set.
12	GUIR_S2_run1	GUIR_S2	automatic	UNJUDGED	0.6251	0.7486	0.6787	0.4569	0.2842	report	4. Judgment sets were created by pooling over the top priority run from each team
13	covidex.t5	covidex	automatic	JUDGED	0.625	0.7314	0.6634	0.4876	0.288	report	except both baseline runs were judged. For topics 1-30 the pool depth was 7.
14	sab20.2.dfo.metadocs	sabir	feedback	JUDGED	0.6161	0.7086	0.6746	0.4704	0.2269	report	For topics 31-35 (the new topics in this round) the pool depth was 15.
15	ContrastNLGSciBert	CMT	feedback	UNJUDGED	0.6138	0.7314	0.6933	0.4302	0.2775	report	5. Round 2 implemented residual collection scoring, meaning all documents appearing
16	udel_fang_L2R	udel_fang	feedback	UNJUDGED	0.6132	0.6857	0.6426	0.3904	0.2434	report	in the Round 1 qrels were removed from the submitted runs and are not part of the
17	cogir-ibm-qQ-combs	CogIR	automatic	UNJUDGED	0.6131	0.7086	0.6456	0.4222	0.259	report	Round 2 qrels.
18	cogir-ibm-qQ-PolRnk	CogIR	automatic	JUDGED	0.6104	0.7314	0.648	0.425	0.2628	report
19	UPrrf16bert20-r2	unique_ptr	feedback	UNJUDGED	0.6099	0.6743	0.6758	0.4391	0.2958	report	Full Round 2 archive:	https://ir.nist.gov/covidSubmit/archive/archive-round2.html
20	UPrrf16lgbert50-r2	unique_ptr	feedback	UNJUDGED	0.6077	0.6971	0.6507	0.4414	0.2964	report
21	combined_bm25_dfr	risklick	feedback	UNJUDGED	0.5998	0.7143	0.6773	0.4203	0.2312	report
22	FullTxt_R2_Time	OHSU	manual	JUDGED	0.5969	0.7029	0.6193	0.4525	0.268	report
23	elhuyar_rRnk_cbert1	Elhuyar_NLP_team	automatic	JUDGED	0.5912	0.72	0.6313	0.4793	0.2941	report
24	xj4wang_run3	xj4wang	manual	UNJUDGED	0.5907	0.7314	0.6546	0.4823	0.221	report
25	elhuyar_rRnk_cbert3	Elhuyar_NLP_team	automatic	UNJUDGED	0.5905	0.7314	0.6227	0.4791	0.2923	report
26	covidex.fuse	covidex	feedback	UNJUDGED	0.5904	0.6743	0.6544	0.5345	0.291	report
27	elhuyar_rRnk_cbert2	Elhuyar_NLP_team	automatic	UNJUDGED	0.59	0.7314	0.6203	0.4772	0.2851	report
28	cogir-ibm-q-PolRnk	CogIR	automatic	UNJUDGED	0.589	0.6971	0.6174	0.4082	0.238	report
29	combined	risklick	feedback	JUDGED	0.5878	0.6971	0.6743	0.4237	0.2243	report
30	BBGhelani3	BBGhelani	manual	UNJUDGED	0.5868	0.7543	0.6376	0.5027	0.2386	report
31	xj4wang_run1	xj4wang	manual	JUDGED	0.586	0.7314	0.6543	0.481	0.2146	report
32	BBGhelani1	BBGhelani	manual	JUDGED	0.5846	0.7543	0.6374	0.5045	0.2412	report
33	BBGhelani2	BBGhelani	manual	UNJUDGED	0.5846	0.7543	0.6374	0.502	0.2402	report
34	Emory_IRLab_Run1	Emory_IRLab	automatic	JUDGED	0.5832	0.6971	0.6338	0.3539	0.2247	report
35	CSIROmed_RF_RR	CSIROmed	feedback	UNJUDGED	0.5804	0.68	0.6481	0.4358	0.2176	report
36	CSIROmedNIR	CSIROmed	automatic	JUDGED	0.578	0.72	0.6899	0.3755	0.192	report
37	Emory_IRLab_Run2	Emory_IRLab	automatic	UNJUDGED	0.5766	0.68	0.627	0.3528	0.2211	report
38	CSIROmedNIRR	CSIROmed	automatic	UNJUDGED	0.568	0.7029	0.6737	0.3714	0.1857	report
39	bm25_syn_0.8_2.6	risklick	feedback	UNJUDGED	0.5655	0.6914	0.6635	0.4054	0.2132	report
40	udel_fang_FB	udel_fang	feedback	JUDGED	0.5625	0.6686	0.6237	0.4185	0.2436	report
41	Technion-RRF	Technion	feedback	JUDGED	0.5566	0.68	0.6236	0.429	0.2465	report
42	cu_dbmi_bm25	columbia_university_dbmi	manual	UNJUDGED	0.5564	0.6171	0.6355	0.4132	0.2083	report
43	r2.fusion2	anserini	automatic	JUDGED	0.5553	0.68	0.5737	0.4284	0.2725	report
44	cord19.vespa.ai-bm25	cord19.vespa.ai	automatic	UNJUDGED	0.5549	0.64	0.6359	0.3607	0.1914	report
45	Technion-LTR	Technion	feedback	UNJUDGED	0.5491	0.64	0.5919	0.4252	0.2497	report
46	DD_prf	DY_XD	automatic	JUDGED	0.5482	0.6629	0.6018	0.3802	0.2064	report
47	sab20.2.meta.docs	sabir	automatic	UNJUDGED	0.543	0.6457	0.5871	0.4211	0.2452	report
48	cord19.vespa.ai-gbdt	cord19.vespa.ai	feedback	JUDGED	0.5422	0.6686	0.6324	0.3887	0.1884	report
49	IRIT_markers_all	IRIT_markers	automatic	JUDGED	0.5366	0.6629	0.6089	0.4348	0.2174	report
50	IRIT_markers_base_mu	IRIT_markers	automatic	UNJUDGED	0.5292	0.6514	0.6025	0.4327	0.214	report
51	IRIT_markers_base_un	IRIT_markers	automatic	UNJUDGED	0.529	0.6629	0.6083	0.4343	0.2169	report
52	Technion-JPDs	Technion	feedback	UNJUDGED	0.5221	0.5943	0.6007	0.4268	0.2356	report
53	TF_IDF_Bo1QE	UB_BW	automatic	UNJUDGED	0.5149	0.6171	0.547	0.3829	0.2209	report
54	CincyMedIR-11	CincyMedIR	feedback	JUDGED	0.5122	0.6286	0.6134	0.3544	0.1749	report
55	ES_Defaults	CovidSearch	automatic	UNJUDGED	0.5112	0.6686	0.5628	0.3991	0.2117	report
56	FullTxt_R2_Orig	OHSU	manual	UNJUDGED	0.5103	0.6114	0.5079	0.4479	0.2373	report
57	run1	cuni	automatic	JUDGED	0.5092	0.6343	0.5951	0.3919	0.2037	report
58	COMBSUM	UB_BW	automatic	JUDGED	0.5028	0.6114	0.5471	0.3784	0.2135	report
59	sab20.2.dfo.meta	sabir	feedback	UNJUDGED	0.4927	0.5657	0.553	0.4226	0.1677	report
60	BITEM_BL	BITEM	automatic	JUDGED	0.4875	0.6	0.5411	0.4358	0.218	report
61	r2.fusion1	anserini	automatic	JUDGED	0.4827	0.6114	0.5428	0.4003	0.2418	report
62	irc_logreg_tfidf	IRC	feedback	UNJUDGED	0.417	0.5371	0.4841	0.4025	0.1666	report
63	xj4wang_run2	xj4wang	manual	UNJUDGED	0.3474	0.3543	0.2464	0.4862	0.1742	report
64	covidex.sim	covidex	feedback	UNJUDGED	0.3254	0.3829	0.3511	0.4546	0.1567	report