1

Counterintuitive Challenges in Measurement

Prof. Ricardo Valerdi

Systems & Industrial Engineering

rvalerdi@arizona.edu

​

Measurement Challenges

2

3

If everything instinct you’ve had is wrong then the opposite would have to be right.

Good Decision Quality Increases the �Likelihood of Good Outcome Quality

The Decision Analysis Creed:

“We work diligently to help you make good decisions and we pray you get good outcomes.”

Bad Decision

Good Decision

Good Outcome

Bad Outcome

Expected

Expected

“Lucky”

“Unlucky”

Decision Quality

Outcome Quality

Abbas, A. E., & Howard, R. A. (2015). Foundations of decision analysis. Pearson Higher Ed.

Good Decision Quality Increases the �Likelihood of Good Outcome Quality

The Decision Analysis Creed:

“We work diligently to help you make good decisions and we pray you get good outcomes.”

Bad Decision

Good Decision

Good Outcome

Bad Outcome

Expected

Expected

“Lucky”

“Unlucky”

Decision Quality

Outcome Quality

Abbas, A. E., & Howard, R. A. (2015). Foundations of decision analysis. Pearson Higher Ed.

6

Measurement Challenge #1:

​

Moving an element from one set to another set may raise the average values of both sets

​

(Will Rogers Paradox)

7

8

https://www.espn.com/mens-college-basketball/team/stats/_/id/12/season/2019

https://www.espn.com/mens-college-basketball/team/stats/_/id/12/season/2020 

https://www.espn.com/mens-college-basketball/team/stats/_/id/252/season/2019

https://www.espn.com/mens-college-basketball/team/stats/_/id/252/season/2020

9

Field Goal % Differences

10

Necessary Conditions for

Will Rogers Paradox

• Two or more groups of data exist and one group has a higher average than another (e.g., one team is better than the other)
• There is movement of data points (e.g., players) between the two groups
• The sample size (e.g., size of the teams) is relatively small compared to the number of data points being moved. In the basketball team example, one player out of fourteen can make a noticeable difference, especially if they get significant playing time

11

Counterintuitive example #2:

​

A paradox in which a statistical trend appears to be present when data are considered separately but disappears or reverses when data are considered as a whole

(Simpson’s Paradox)

Premise

If a hypothesis is supported by two independent trials, then it will be supported when the data from those trials are combined

​

​

​

Day, S. M., Simpson’s Paradox and Major League Baseball’s Hall of Fame, AMATYC, 15(2), 26-35, 1994.

12

Batting average

13

Babe Ruth

(active 1914-1935)

Lou Gehrig

(active 1923-1939)

Batting Average Comparison

14

http://www.baseball-reference.com/players/r/ruthba01.shtml

http://www.baseball-reference.com/players/g/gehrilo01.shtml

Batting Average Comparison

15

Necessary Conditions for Simpson’s Paradox

​

Identify two data sets with different sample sizes

16

17

Counterintuitive example #3:

​

The length of the measurement tool impacts the measured length (Coastline Paradox)

����Lewis Fry Richardson�Statistics of Deadly �Quarrels (1960)���P(war) = # of grievances between two countries, length of common border

18

Length of border between Portugal and Spain [987 km – 1,214 km]

19

1789

20

Unit = 100 km, length = 2,800 km Unit = 50 km, length = 3,400 km

Necessary Conditions for Coastline Paradox

​

High fidelity measurements

Fractals

21

Measurement Challenges

22