Hydra ML Datasets

	A	B	C	D	E	F	G	H	I	J	K	L	M	N	O
1		HydraLM HF	Which GPT Model(if applicable)	Size (rows)	Domains	Status	Model	Who's working	Reviewer	Review Notes	Message Types	Notes	Basic Data Cleaning: DeDupe & Whitespaces	Data Clean,Who's working

2	NobodyExistsOnTheInternet/ConvoEvolLIMAuncensored	https://huggingface.co/datasets/nigh8w0lf/hydra_moe_ConvoEvolLIMAuncensored			uncensored, code, computer science,physics,	Pushed to HF		night_w0lf	vikp		instruction,output	multi turn conversation, no instructions or system message, defaulted all inputs to instruction	Pending
3	toolLLM: https://drive.google.com/file/d/1lTelETDJ1TeAYiXmi485brsPucagpTnk/view?usp=share_link	https://huggingface.co/datasets/nigh8w0lf/Hydra_moe_toolllama_dataset	3.5/ChatGPT		Tool/API Use	Won't use		night_w0lf	vikp	Formatting is a bit odd on this, it has multiple instructions in a row. Also has a lot of "request invalid" errors in the data, and instructions with "all previous trails failed"	system, instruction, input, output	Use of the phrase "You are AutoGPT" in System messages will need clean up, I have left it un modified for now.	Pending
4	PygmalionAI/PIPPA	pippa_rp_std		815,507	roleplay	Under Review		nion	vikp	Look at conv id 16825 and decide for yourself how good this dataset is			Pending
5	camel-ai/physics	HydraLM/physics_dataset_standardized		40000	physics	Pushed to HF	Both	lskywalker			instruction, output		Pending
6	https://huggingface.co/datasets/wenhu/TheoremQA	https://huggingface.co/datasets/HydraLM/TheoremQA_standardized			Notes: Removed additional explanations from the dataset similar to Open-Platypus. (Original TheoremQA dataset has image data as well.)	Under Review		moonlightgarden					Pending
7	camel-ai/math	HydraLM/math_dataset_standardized			math	Pushed to HF	Both						Pending
8	https://huggingface.co/datasets/garage-bAInd/Open-Platypus	https://huggingface.co/datasets/HydraLM/Open_Platypus_standardized			logical reasoning	Under Review		moonlightgarden					Pending
9	knowrohit07/know_logic	https://huggingface.co/datasets/nigh8w0lf/hydra_moe_know_logic			logic,reasoning,code	Pushed to HF		night_w0lf	vikp		instruction, output		Completed	night_w0lf	Completed basic data cleaning,filterd out all model name data 'airoboros'
10	OpenOrca/blob/main/1M-GPT4-Augmented.parquet	HydraLM/OpenOrca-GPT4-standardized			instruct/orca	Pushed to HF			vikp	Reccomend sampling down (high row count relative to other data)	[system, instruction, output]		Pending
11	https://raw.githubusercontent.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM/main/data/unnatural_instruction_gpt4_data.json	https://huggingface.co/datasets/YoungPhlo/GPT4LLM-unnatural_instruction_standardized	4	27,000	instruct, logic, reasoning	Won't use		youngphlo	vikp	Raw data is weird, see conversation ids 8971 and 8972	instruction, input, output	seems the correct answer bounces between "output" and "label"	Pending
12	openchat/openchat_sharegpt4_dataset	https://huggingface.co/datasets/YoungPhlo/openchat-sharegpt_gpt4_standardized	4	90,231	instruct	Under Review		youngphlo	vikp	Instruction quality not good, see https://huggingface.co/datasets/YoungPhlo/openchat-sharegpt_gpt4_standardized/viewer/default/train?p=4	instruction, output	Multiple languages	Pending
13	WizardLM/WizardLM_evol_instruct_V2_196k	HydraLM/WizardLM_evol_instruct_V2_196k_standardized			instruct	Pushed to HF	Both		vikp	Recommend sampling down			Pending
14	teknium/GPT4-LLM-Cleaned	HydraLM/GPT4-LLM-Cleaned_standardized			instruct	Pushed to HF	Both						Pending
15	totally-not-an-llm/EverythingLM-data-V2	https://huggingface.co/datasets/HydraLM/EverythingLM-data-V2-standardized		3,000	instruct	Under Review		thennal	vikp		system, instruction, output		Pending
16	andreaskoepf/megacode2-min100	https://huggingface.co/datasets/HydraLM/megacode2-min100-standardized		1,026,386	instruct	Won't use		thennal	vikp	See conversation id 103. It's possibly missing a system prompt that should be in there. Most outputs are too verbose for just the instructions (see conversation 0-10).	instruction, output		Pending
17	mrm8488/unnatural-instructions	HydraLM/unnatural-instructions_standardized			instruct	Won't use	Both		vikp	Dataset has incorrect examples, like in conversation 66050. Also has a lot of duplication.			Pending
18	rombodawg/LosslessMegaCodeTrainingV2_1m_Evol_Uncensored	https://huggingface.co/datasets/HydraLM/LosslessMegaCodeTrainingV2-1m-Evol-Uncensored-standardized		1,884,414	code, instruct	Pushed to HF		thennal	vikp	This dataset looks fine, but I would recommend sampling it down (very large).	instruction, output		Pending
19	https://github.com/teknium1/GPTeacher	HydraLM/GPTeacher_codegen_standardized			code	Pushed to HF	Both						Pending
20	camel-ai/chemistry	HydraLM/chemistry_dataset_standardized			chemistry	Pushed to HF	Both						Pending
21	biology	HydraLM/biology_dataset_standardized			biology	Pushed to HF	Both						Pending
22	GAIR/lima	https://huggingface.co/datasets/HydraLM/lima_standardized			(done) OG "less is more"	Pushed to HF		yam peleg					Pending
23	neulab/conala	https://huggingface.co/datasets/HydraLM/conala_standardized			(done) instruct, code gen	Pushed to HF		vikp	vikp	Outputs duplicated			Pending
24	Airoboros 2.2	https://huggingface.co/datasets/khalidalt/airoboros-2.2-standardized				Pushed to HF		Khalid	vikp		[system, instruction, output]
25	evol-codealpaca-v1	https://huggingface.co/datasets/khalidalt/evol-codealpaca-v1-standardized				Pushed to HF		Khalid
26	https://huggingface.co/datasets/LDJnr/Puffin	https://huggingface.co/datasets/HydraLM/puffin_standardized				Under Review		yam peleg					Pending
27	OpenAssistant/oasst_top1_2023-08-25	https://huggingface.co/datasets/HydraLM/oasst_top1_standardized				Pushed to HF		vikp	vikp			Multilingual	Pending
28	OpenAssistant/oasst1					In progress		il_vitorio	vikp	Look through the first 20 rows of the oasst data			Pending
29	TokenBender/unnatural_code_instructions_20M (will get unformatted version from Token Bender)	https://huggingface.co/datasets/ChallengerSpaceShuttle/HydraLM_TokenBender_Datasets				Won't use		challenger	vikp	Dataset has formatting issues, see conversation ids 200, 201, etc. Output is marked as system	instruction, system		Pending
30
31
32
33
34
35
36
37	ehartford/wizard_vicuna_70k_unfiltered	(for later)
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100