FB3_high_score_solutions_r1.xlsx

	A	B	C	D	E	F	G	H	I	J	K	L	M	N	O
1	Private LB Rank	Team / Account	Privete LB	CV	LLRD	LLRI	Pseudo Labeling	Knowledge Disttilation	AWP	Ensemble weight	Num_ens _models	Loss func	Other	Not work	URL

2	1	Rohit Singh	0.4334	0.441		〇	〇		〇	Optuna	19		・various pooling ・various LR ・Model Used for Embeddings and SVR ・freezing top n layers ・re_init top n layers. ・Training with Differential learning rate	・Augmentations ・Post processing	https://www.kaggle.com/competitions/feedback-prize-english-language-learning/discussion/369457
3	2	gezi	0.4336	0.4449			〇				23	rank loss pearson loss	・Let bert deal with the target relation ・Add pearson loss ・Use back translation for pretrain ・Use feedback2 data for pretrain ・Tune for each target ・add SVR method ・T4x2		https://www.kaggle.com/competitions/feedback-prize-english-language-learning/discussion/369369
4	3	Team: Now You See Me	0.4337	0.442		〇				Hill climbing	24		・max_len: 2048 ・RAPIDS SVR using embeddings ・Different loss rates per target ・train batch size 1 ・different max length ・replace "\n\n" with "\|"		https://www.kaggle.com/competitions/feedback-prize-english-language-learning/discussion/369609
5	4	hakubishin3	0.4339	0.4423	〇		2 patterns				22		・using pre-trained model embeddings ・Ridge and LGB stacking ・Add special token ・various max length for train and inference Full precision training	・AWP ・LLRI ・MLM ・Train SVR using fine-tuned model embeddings	https://www.kaggle.com/competitions/feedback-prize-english-language-learning/discussion/369621
6	5	Psi	0.4341				〇			Nelder-Mead			・weight bounds between 1 and 3 on the weights ・various max length ・pooling : cls token , GeM pooling	・Augmentetions ・Different losses ・TF-IDF ・Other backbones outside of deberta ・2nd stage models	https://www.kaggle.com/competitions/feedback-prize-english-language-learning/discussion/369578
7	6	Tom	0.4346	0.4422						class-wise weighted avg (gp_minimise)	30		・created models for each class ・ensemble separate model and single model	・AWP ・pre-training with fb1 data ・detector pre-training with fb1 data ・pseudo-label with fb1 data ・random masking augmentation ・lgb/catboost stacking	https://www.kaggle.com/competitions/feedback-prize-english-language-learning/discussion/369567
8	7	tk	0.4341		〇				〇	Nelder-Mead	16	BCE l1_loss	・freeze layers	・fgm ・LLRI ・lgbm stacking	https://www.kaggle.com/competitions/feedback-prize-english-language-learning/discussion/369736
9	8	Team: finally we did it	0.4342		〇	〇	〇	〇	〇	Ridge	27	Custom Loss	・various pooling ・MaskAddedAttentionHead ・various max length ・replacing “\n\n” with [PARAGRAPH] ・deberta v3 x small model with knowledge distillation ・Post Processing with Nelder-Mead	・MLM ・focal loss ・ensemble with weight tuning	https://www.kaggle.com/competitions/feedback-prize-english-language-learning/discussion/369524
10	13	kaerururu	0.4345	0.4436						Nelder-Mead	12, 6		・various seed ・various max length ・Add special token ・rapids SVR ・freeze embedding layer and first 2 layers ・replaced svr with xgboost ・longer max length (1536) and use head-and-tail tokens	・finetuned embeddings ・Universal Sentence Encoder Embedding ・Use text features as meta feature when I finetuneing hugging face models and my 2nd stage stacking models	https://www.kaggle.com/competitions/feedback-prize-english-language-learning/discussion/369440
11	14	Takoi	0.4346	0.4435		〇	〇			Nelder-Mead	21		・Add lstm after bert. And freeze the bert part for the first few epochs of training ・Add special token ・train model for each target ・train model for two targets		https://www.kaggle.com/competitions/feedback-prize-english-language-learning/discussion/369564
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100