Sources of Variation

WORKSHOP SEPT 20, 2024

1

2

3

Sources of Variation

• Activity 1: Explaining variation in a quantitative response
• 2020 US Census data on income – observational data

​

• Activity 2: Does random assignment ‘work’?
• Memory study: data from a designed experiment

4

Activity 1: Explaining variation

• DATA: random sample of 10,000 cases from 2020 US Census
• Response variable: yearly income ($)
• Data cleaning: removed people who were not employed and/or income was reported as 0
• n = 4,365

​

Student background: understanding of univariate summary statistics and graphs

Activity:

5

https://docs.google.com/document/d/1sVkkAdSS2YrBgm78P3e1HOq4u5oILKgdY3EpR0oK-gQ/edit?usp=sharing

​

6

1. What is a ‘typical’ yearly income for these 4,365 people?

2. Do the yearly incomes of these 4,365 people vary? How do you know?

7

3. What would the values of the standard deviation and IQR be if there was no variability in yearly income for these 4,365 people?

4. What would the values of the mean, median, Q1, Q3, minimum and maximum yearly incomes be if there was no variability?

8

5. What would the histogram of yearly incomes look like if there was no variability in yearly income for these 4,365 people?

9

6. What do you think are some possible causes or reasons or potential sources of the variability in the yearly incomes of these 4,365 people? (type into chat)

10

Here are the same 4,365 yearly incomes categorized by education level

7. Is there a relationship (or association) between yearly income and education level? Provide an explanation of your thinking with numerical support.

11

8. Does it appear that education level is responsible for or explains at least some of the variability we saw in the 4,365 yearly incomes? Explain.

Left-over variation in yearly income is generally less than the overall standard deviation

12

9. If education level was not responsible for any variability (or very little) in the yearly incomes what would be true of the 5 boxplots relative to each other?

13

10. Does education level explain all of the variability in the 4,365 yearly incomes? Explain your thinking.

14

11. What are some potential sources or reasons for the left-over variability in yearly incomes?

Left-over variation in yearly income is generally less than the overall standard deviation

15

Here are the same 4,365 yearly incomes categorized by region of the U.S.:

12. Does region of the U.S. explain at least some of the variability in the 4,365 yearly incomes? Explain.

16

Here are the same 4,365 yearly incomes categorized by region of the U.S.

13. Does region of the U.S. explain all of the variability in the 4,365 yearly incomes? Explain.

17

14. Which variable, education level or region of the U.S. do you think explains more of the variation in the yearly incomes of these 4,365 people? Explain.

18

Left-over

Proportion explained

Activity 2: Memory Study

• DATA: collect data from students
• Each student is randomly assigned to a sequence of 30 letters and is given 20 seconds to memorize as many as they can in order from left to right
• Example: FGHO-ERBK-…..
• Record each student’s memory score, the number of letters they remembered correctly (in order)
• Ask each student whether they’ve had any caffeine today (Y,N) and how much sleep they got the previous night (0, 0.5, 1, 1.5…. Hours)

Motivational Video (4 minutes): https://youtu.be/4caEuNmlaWA

19

Data Collection

• Follow a link to a data collection webpage. (in chat)

20

2 follow-up questions

Memory Scores

• 1. For what variables did each student provide information? (Hint: There are 4 variables.)
• The main goal of this study is to investigate how the type of sequence (i.e., the way the letters are chunked or grouped) affects memory score.

​

​

21

Memory Score (# of letters correct)

Type of Sequence

Goal: Cause-and-effect?

• Sequence A: FBI-OMG-CIA etc.
• Sequence B: FBIO-MGC-IAU etc.

https://docs.google.com/document/d/1HafLC062-9glks0dSRs5tbEOE45t9K_ZbvxmNJKhM_I/edit?usp=sharing

22

Memory Score (# of letters correct)

Sources of variation in memory score

​

​

​

​

​

​

​

​

​

2. Will each student have the exact same memory score?

Before we look at any data:

If not, what are some potential sources of variation in memory scores? (type into the chat)

​

23

Memory Score (# of letters correct)

Other Sources of variation in memory score:

Caffeine,

amount of sleep,

how good a person is at memorizing, whether the person was distracted

……. unknown

​

How would caffeine consumption (yes, no) be a source of variability in memory scores?

How would the amount of sleep be a source of variability in memory scores?

24

Memory Score (# of letters correct)

Other Sources of variation in memory score:

Caffeine,

amount of sleep,

how good a person is at memorizing, whether the person was distracted

……. unknown

​

What about student age? Is age likely to be responsible for much variation in the memory scores?

What about the conditions of the experiment? Are the conditions under which the data were collected likely to be causing much variation in the memory scores?

Sources of Variation Diagram

25

Sources of Variation Diagram

26

Sources of Variation Diagram

27

Sources of Variation Diagram

28

Sources of Variation Diagram

29

Sources of Variation Diagram

30

Scope of inference

Limitations of study

Memory Scores

• Load the data into the Multiple Variables applet.
• Drag the variable ‘score’ into the response box.

​

4. Describe the distribution of memory scores. What is a ‘typical’ memory score? How much variability is there in the memory scores? Any unusual memory scores?

31

Memory Scores

• Drag the variable ‘sequence’ into the Subset By box.
• Make sure the variable ‘score’ is still in the response box.

​

5. What differences/similarities are there in the distributions of memory score for the two sequences? Which sequence tended to have higher memory scores? Support your statements with numerical evidence.

32

Memory Scores

• 6. Can you imagine another reason or explanation for the difference in performance between the two sequences, besides the explanation that one sequence may be easier to memorize than the other?

33

• 7. Explain how whether or not a student had caffeine today (yes, no) could provide an alternative explanation for the difference in mean memory score between the two sequences.

34

Memory Score (# of letters correct)

Type of Sequence

Other Sources of variation in memory score:

Caffeine

What would need to be true for caffeine to provide an alternative explanation for the observed difference in memory score between the two sequence groups?

1.

2.

What would it look like for caffeine to be related to the type of sequence?

A confounding variable is a variable that provides an alternative explanation for the study outcome. To act as a confounder, the variable must have a relationship with both the response and explanatory variable.

Is caffeine a confounding variable in our study?

• Drag the variable caffeine into the Response box and drag the variable sequence into the Subset By box. Note the results.

35

• 8. Use these results to explain whether caffeine is acting as a confounding variable in this study.

36

Memory Score (# of letters correct)

Type of Sequence

Other Sources of variation in memory score:

Caffeine

The purpose of random assignment is to reduce the possibility of confounding variables. How?

1.

2.

Purpose of Random Assignment

• Reduce the possibility or potential for confounding variables
• How?
• Ensures (hopefully) that the distribution of each unexplained source of variation is similar for the two explanatory variable groups.

​

​

• Well designed studies using random assignment allow for cause-and-effect conclusions.

37

Back to Sequence and Memory Scores

38

Memory Score (# of letters correct)

Type of Sequence

Goal: Cause-and-effect?

Other Sources of variation in memory score:

Caffeine,

amount of sleep,

how good a person is at memorizing, whether the person was distracted

……. unknown

Random Assignment, so hopefully no confounders

Could the results we’ve observed have happened by random chance?

Random Sampling vs. Random Assignment

• Random Sampling
• Goal: To obtain a representative sample from the population
• 🡺results from the study can be generalized to this population

• Random Assignment
• Goal: Create groups (e.g., treatment groups, explanatory variable groups) that are similar with respect to extraneous sources of variation
• 🡺 cause-and-effect conclusions may be drawn if the data reveal a significant difference in the mean response between the groups

39

40

To generate a class specific URL for the Memory Study:

https://www.rossmanchance.com/applets/getMemTestURL.html

​

https://www.isi-stats.com/

​

Applets: