Sorted Evaluations

	A	B	C	D	E	F	G	H	I	J	K	L	M
1	Category	Instance	What Does It Measure?	Use Case	Did/Does It Work?	What's Used To Measure If It Worked?	Cost To Develop	Marginal cost	Accuracy	Human Judgement In Evaluation	Predictive vs Evaluative	Comments	Sources

2	IQ Tests		Intelligence	School placement, work placement,				$20	predict 0.5 of variation in school grades				https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4557354/#:~:text=Predictive%20Validity%20of%20IQ,those%20criteria%20have%20been%20reported.&text=It%20is%20widely%20accepted%20that,0.5%20(Mackintosh%2C%202011), https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5346574/
3		Wechsler Adult Intelligence Scale (WAIS)								0	Predictive
4		Wechsler Intelligence Scale for Children (WISC)								0	Predictive
5		Stanford-Binet Intelligence Scales								0	Predictive
6		Woodcock-Johnson Tests of Cognitive Abilities								0	Predictive
7		Kaufman Assessment Battery for Children								0	Predictive
8		Cognitive Assessment System								0	Predictive
9		Differential Ability Scales								0	Predictive
10		Raven's Progressive Matrices								0	Predictive
11		Cattell Culture Fair III								0	Predictive
12		Reynolds Intellectual Assessment Scales								0	Predictive
13		Thurstone's Primary Mental Abilities[67][68]								0	Predictive
14		Kaufman Brief Intelligence Test[69]								0	Predictive
15		Multidimensional Aptitude Battery II								0	Predictive
16		Das–Naglieri cognitive assessment system								0	Predictive
17		Naglieri Nonverbal Ability Test								0	Predictive
18	Animal Behavior
19		Pavlov's Saliva Measurements	How much did a dog salivate?	Classical Conditioning Research	I hope so, a lot of science is based on it					0	Evaluative		https://sites.psu.edu/dps16/2016/02/18/pavlovs-dogs/ , https://www.youtube.com/watch?v=NzBDScsHL44
20		Inebriometer	How drunks is your fruit fly?	Enable evaluation of intoxication, mostly for examining genes' effects on alcohol processing						0	Evaluative
21	Market Value		"How much can I sell X for?"	Insurance, sales, taxes
22		Art Appraisal								High	Depends on use
23	Subject Matter Tests
24		AP Exams									Evaluative
25		SAT IIs						$22			Evaluative
26		NYS Regent Exams
27	Academic Admissions Exams
28
29		SATs
30		ACTs
31	Job Performance Evaluations- Teachers		"How useful is this teacher (possibly relative to other teachers)?"	Hiring, firing, incentive-based pay							Both
32		Specific types of evaluation would go here
33	Job Performance- Software Engineer
34
35	Job or Task Performance- Generic/Other
36		Stack Ranking	How is a worker performing, relative to their peers.	Firing, promotions, raises	Depends on situation- it can work initially in a bloated company, but degrades as the deadweight is removed	Corporate profits, morale		Lowered morale, lower employee cooperation, best employees leave	Self-validating	Very high	Evaluative		https://www.perdoo.com/resources/stack-ranking/#:~:text=Stack%20ranking%20is%20a%20practice,General%20Electric%20in%20the%201980s.
37		Peer Review on a Paper	Is this paper empirically correct? Interpreted correctly? Impactful?	Journals deciding whether to publish, decline, or as for rewrites of a submission	Depends on the field	Paper retractions		$1,000	Self-validating	Very high	Evaluative
38		Code Review	What changes does this code need before being checked in?	Coding projects with more than one contributor		Performance, future defects, necessity of refactoring		$500		High	Evaluative		https://en.wikipedia.org/wiki/Code_review
39	Animal Evaluation
40		Westminster Dog Show	Confirmation to breed standards set by Westminster	Give dog lovers a hobby	Dogs conform to breed standards more but are becoming less healthy	Longevity, health, and behavior of dogs under their jurisdiction
41		Thoroughbred Racing	Which horse can run the fastest?	Leisure		Self-justifying				0	Both
42	Athletics
43		Rhythmic Gymnastics		Leisure	Self-validating
44		100 meter dash	Which human can run the fastest over 100 meteres	Leisure
45		Moneyball	Which athletes bring in the most wins per dollar?	Financial gain	Temporarily	Game wins	$200,000/year	$0		Medium	Predictive	Cost to Develop is a WAG based on salary of practitioner	https://grantland.com/features/the-economics-moneyball/
46		Rock Climbing Difficulty Grades	How difficult is this climb?	Allow climbers to choose the right difficulty level for themselves	People commonly call them unreliable, but in practice the ratings are remarably stable					High	Evaluative		https://forum.effectivealtruism.org/posts/oTN5t79mXRpafHDsL/prize-interesting-examples-of-evaluations?commentId=Tptkju6rKq7kbBNmm
47	Food
48		USDA Egg Grading	How good is this egg?	Decide what is fit for human consumption		Self-validating		$0.01	Self-validating	High	Evaluative	Marginal cost is a guess based on inspector salaries	https://www.ams.usda.gov/grades-standards/egg/grade-shields , https://www.ams.usda.gov/services/grading/fees#egg
49	Medical
50		Mamogram	Density and regularity of density of breast tissue	Early diagnosis of breast cancer	Yes, although with many false negatives	False negatives. We care about false positives, but htose are significantly harder to measure		$100		Medium	Predictive		https://en.wikipedia.org/wiki/Mammography , https://cityhospital.co/cost-of-a-mammogram/#:~:text=How%20Much%20Is%20a%20Mammogram,depends%20on%20where%20it's%20done.
51		Apgar	Overall health of a newborn baby	Should resuscitation be continued? Are interventions necesary	Yes, as part of a larger set of cultural changes	Decrease in child mortality/stillbirths (babies would previously be recorded as stillborn when they were alive and revivable)	$1,000,000	$1	Unknown, no control	Medium	Predictive	Cost to develop is a WAG for developer's career earnings, since the test was basically operationalizing her intuitions	https://healthmatters.nyp.org/apgar-score/ , https://en.wikipedia.org/wiki/Apgar_score
52		Checklist Manifesto
53
54	Physics
55		Digital Thermometer	Temperature	Many	yes	Other thermometers		$0
56		Ruler
57		Scale
58		Air Quality Meter						0
59	Professional Admittance Exams
60		Chinese Imperial Exam (Sui Dynasty, 581-618)	Fitness for wor as a Chinese bureaucrat	Merit based hiring by ancient Chinese bureaucracy	No(t in this time period). In practice appointments were still made by recommendation					Yes	Predictive		https://en.wikipedia.org/wiki/Imperial_examination#Sui_dynasty_(581%E2%80%93618)
61		Chinese Imperial Exam (Qing dynasty, 1636–1912)	Fitness for wor as a Chinese bureaucrat	Merit based hiring by ancient Chinese bureaucracy							Predictive		https://en.wikipedia.org/wiki/Imperial_examination#Qing_dynasty_(1636%E2%80%931912)
62		Armed Forces Qualification Test 1965	Usefulness as a solider to the US military	Determine admission and placement to/in the US military	Yes. Troops admitted as part of a standards-lowering initiative in 1966 performed noticably worse, suggesting the overall evaluation was predictive. They had 3x the casualty rate (overall, not controlling for role; they were 2x as likely to see combat), were reassigned 11x more often, 7-9x more likely to need remedial training	Death rate, reassignment rate		$20		No	Predictive	Test is free to takers; I guessed at marginal cost from the SATs	https://en.wikipedia.org/wiki/Project_100,000 , https://bigthink.com/politics-current-affairs/story-behind-mcnamaras-morons?rebelltitem=1#rebelltitem1 , https://medium.com/@LivingHistory/project-100-000-the-mentally-disabled-men-who-fought-in-vietnam-1cbe145cc126
63		California General Electrician Certification Exam	Knowledge of electrical repair	Admission to become a certified electrician in CA				$100		No?	Evaluative		https://www.dir.ca.gov/dlse/ecu/electricaltrade.html
64	Institutions
65		Netflix Chaos Monkey	How robust is Netflix's system against random server failures	Find bugs in Netflix's system while they are easy and cheap to fix	Yes, according to Netflix (actual numbers not released)	Customer downtime	$1m/year	$10,000	NA	No	Evaluative		https://www.gremlin.com/chaos-monkey/
66		Dodd-Frank Act Stress Testing	How robus is a financial institution against various potential problems	Discovering failure points ahead of time so they can be remedied before a financial crisis	No, according to Tim Harford	Actual robustness to financial crises (which haven't repeated since 2008)		$10b/year	Low (according to Tim Harford)	Yes	Predictive	Marginal cost is a WAG based on https://www.americanbanker.com/slideshow/the-real-world-impact-of-dodd-frank-stress-tests-and-other-regs	https://www.econtalk.org/tim-harford-on-the-virtues-of-disorder-and-messy/ , https://en.wikipedia.org/wiki/Stress_test_(financial)
67
68
69	Internet
70		PageRank
71		HN Rankings
72	Interprsonal
73		Dueling
74
75	Other
76		Polygraph
77
78
79
80	Justice
81		20th Century Trial
82		Trial by Combat
83		Trial by Ordeal
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100