What is the “Benford Law”?

Benford law states that for certain random, or pseudo-random data has a special property with regards to the first digit of the numbers. While people expect random numbers to generate the digits randomly and therefore evenly, it is not true for the first digit of many natural data samples -- under some specific conditions.

The conditions are usually as such:

• The data must be spread out to multiple magnitudes -- that is there should be, for example, both one, two, three digit numbers -- the more “magnitudes” it spans, the more the Benford Law will hold true.
• The data must be “scale invariant” -- i.e., for example if we change from centimeters to feet, or USD to EUR or other currency, we do not expect the distribution of the digits to change.

TLDR: skip to the next section  if you don’t care about the math

If the above are true, then a scale change should not change the distribution. For example if a currency exchange rate is 2:1, then converting the dataset to the other currency will multiply all data with 2. This will cause all numbers (and only those numbers) that began with 1 to now begin with 2 and 3 -- so 2 and 3 together should have the same frequency as 1, otherwise this scale invariance is not true. Through the same logic, the numbers that began with 2 and 3 will now begin with 4,5,6,7 -- those together will also have the same frequency as 1. If it works for any multiplier, the distribution of the first digits must be logarithmic, leading to the following distribution:

In fact, the frequency of 1 (30.10%) equals the combined frequency of 2 and 3 (17.61%+12.49%) as well as the frequency of 4-7 (9.69%+7.92%+6.69%+5.80%). Similarly, the frequency of 3 (12.49%) equals the combined frequency of 6+7 (6.69%+5.80%), as all numbers starting with the digit 3 will start with 6 and 7 after a 2x conversion.

Why can it be used to detect fraud?

If a dataset has the above properties, it is expected to produce the Benford distribution for the first digits. However, if humans make up numbers, they intuitively assume that the distribution of the first digits is even, and will have a distribution closer to 1/9 for each digit. When real data is mixed with human produced fraudulent data, it usually distorts the true Benford distribution to be closer to the even 1/9 one.

What’s “wrong” with the Chicago dataset?

When looking at the election data from Chicago, the distribution of the first digits for the votes cast for Biden in each precinct does not match the Benford distribution

However, at the same time, Trump votes seem to approximate the Benford distribution a lot closer:

So does that indicate a fraud? Let’s look at the total votes.

Total votes don’t seem to comply with the Benford Law either. Why?

Let’s take a look at precinct sizes (well, what we have is actual votes cast, but that is indicative of the number of voters in each precinct)

What we see is that the typical precinct size is such that the votes have a pretty concentrated distribution around a few hundred votes per precinct. It does not span magnitudes, it is mostly concentrated in 100- 1000, so it does not have the initial properties that are required to meet the criteria for the Benford Law. Since it is an urban area, precincts are defined such that each covers approximately equal population, so that is why it is “dense” around a few hundred votes per precinct.

Votes cast for Biden-Harris in urban areas, like Chicago are very high, in the 70-80-90% range, so those also closely follow the distribution of the number of votes cast --- and we do see that most would be in the 200-400 range, hence starting with the digits 2,3 and 4.

However, votes cast for other candidates, including Trump,  do span the full range (many one, two and three digit votes) making it more Benford-like.

The same data for other results

Chicago 2020

Analyzing the same dataset for the 2016 elections yields similar results, since the patterns are similar. The total votes are in the 200-1000 range, not spanning multiple magnitudes.

Other data referenced in the original paper are not presented in the precinct granularity, so the size of the counties (number of votes) is much less uniform, therefore the dataset (even for the total number of votes) is more spread out to multiple magnitudes.

Fulton 2020

Fulton data is not presented in precinct granularity, but only at county level granularity that are different in sizes. The total vote count spreads out from as small as a few tens to a few thousands. As it spans multiple magnitudes, this produces a more Benford-like distribution for both the total votes and the votes for each candidate.

Allegheny

Allegheny data is also provided at precinct granularity, with a fairly uniform vote per entry. This, combined with a high percentage of Biden votes skews the data too.

Conclusion

The dataset does not meet the conditions for the Benford distribution, so the discrepancy does not indicate fraud. It is a natural consequence of the typical size of the voting precincts and the high percentage of the Biden-Harris votes within each precinct.

Calculations, graphs and source for the dataset are available on this sheet: https://docs.google.com/spreadsheets/d/1FbvT5n8z7kTog6hUu3Orkl6Cr7MQtvNOxf7VUNh_Q9w/edit?usp=sharing