ABCDEFGHIJKLMNOPQRSTUVWXYZ
1
2
Asian vCase 1 - ExplanationCase 2 - No ExplanationBias ChangeChart Structure:
3
Black1.58%18.62%-17.04%
Positive number indicates favoring the "green" group over the "blue" group
4
White3.61%31.51%-27.90%
5
Hispanic32.98%27.37%5.60%Definitions:
6
Average12.72%25.83%-13.11%
bias here is defined as "absolute distance from 0", invariant of whether the group is favored or unfavored
7
440494
bias change here is defined as "the change in bias having to provide an explanation introduced"
8
- a negative bias change means that the bias (distance from 0) was reduced
9
Black vCase 1 - ExplanationCase 2 - No ExplanationBias Change
- a positive bias change means that the bias (distance from 0) was increased
10
Asian-1.58%-18.62%-17.04%
11
White-1.84%0.50%1.34%Links:
12
Hispanic-5.49%-3.60%1.89%code:
https://github.com/eob/llm-experiments/
13
Average-2.97%-7.24%-4.60%blog post
https://edwardbenson.com/2024/10/llms-are-less-racist-if-they-explain-themselves
14
377362
15
Context:
16
Hispanic vCase 1 - ExplanationCase 2 - No ExplanationBias Change
Please read the blog post so that you look at these numbers with my
17
Black5.49%3.60%1.89%
best attempt to provide context!
18
White-2.81%12.67%-9.86%
19
Asian-32.98%-27.37%5.60%
20
Average-10.10%-3.70%-0.79%
21
349376
22
23
White vCase 1 - ExplanationCase 2 - No ExplanationBias Change
24
Black1.84%-0.50%1.34%
25
Asian-3.61%-31.51%-27.90%
26
Hispanic2.81%-12.67%-9.86%
27
Average0.35%-14.89%-12.14%
28
390331
29
Should be 0:Should be 0:Bias Change Average
30
Unit Test0.00%0.00%-7.66%
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100