Unit 5 Effect Size �

1

2

To better understand what we mean by “effect size,” let’s start off by reading the following news article…

3

Example

4

Example

5

Do you think you would do worse on an exam if you were in the cellphone group in this study?

​

If so, how much worse? If not, why not?

Well, there’s probably a few more things you’d like to know.

6

Example

First, how do the test scores compare between students who did and did not use their cellphones during class?

​

Is there a difference?

7

Example

The article reports that students using their cellphones during class on average scored ‘half a letter grade’ lower than students who did not use their cellphones.

​

By ‘half a letter grade’ this means that students in the cell phone group on average scored 5% less than those who didn’t use cell phones (a letter grade is 10%).

​

8

Example

Does this mean that everyone scored lower in the cell phone group?

9

This result doesn’t mean that everyone scored lower, it just means that overall, this group scored lower on average.

​

Below is an example distribution of exam scores for each group. The dashed lines show each group’s average score. But notice how there is still a lot of overlap between the groups’ distributions? That means that not every student in the No Cell Phone Group scored higher than students in the Cell Phone Group.

10

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Number of Students with the Exam Score

Exam Score

Graph of Exam Scores by Group

No Cell Phone Group

Cell Phone Group

85% Average

80% Average

Example

In scientific studies, whenever researchers compare the outcomes of two groups (like test performance in the cell phone and no cell phone groups), they first want to know if there is a reliable difference between the groups.

​

11

But people also want to know how big of a difference there is between the two groups. The magnitude of a result is generally known as an effect size.

​

12

Knowing the effect size of a result is important, because it tells you whether a scientific finding is practically meaningful.

​

In the case of the cell phone study, you may not want to use your phone during class if it means losing half a letter grade on your exam score!

​

13

A+

A

A-

B+

B

B-

C+

C

C-

D+

D

D-

F

Example

It can be tricky to figure out whether a scientific result is practically meaningful.

​

To show why, let’s practice looking at some examples of large and small effect sizes in experiments.

14

15

16

17

18

19

Now that you’ve had a chance to see examples of small and large effects in different contexts, let’s practice thinking about effect size information in news articles.

20

21

Example

22

Example

23

1. What is the journalist’s claim in this article?
2. Which of the following graphs shows the change in students’ average grades from before the time switch to after the time switch, as it is described in the article?

​

​

​

​

​

​

​

• Do you think students’ academic performance improved in a meaningful way after the start time change? If not, what difference in grades would you consider to be a meaningful improvement for these students?
• Based on the evidence, do you think school start times should be shifted later to improve student learning? Why or why not?

A)

B)

C) There is not enough information in the article

Let’s try one more article!

24

25

Example

26

Example

27

• What is the journalist’s claim in this article?
• Which of the following graphs shows the strengths of the relationships between fruit intake and well being and between savory snack intake and well being, as they are described in the article?

​

​

​

​

​

​

​

​

​

• What additional information would you need to know in order to determine whether well being increased in a meaningful way with increased fruit intake, and whether well being decreased in a meaningful way with increased savory snack intake?
• Based on the evidence in the article, would you try eating more fruit and fewer savory snacks in order to improve your well being? Explain your answer.

A)

B)

C) The article does not provide enough detail about the strengths of these relationships

When you read articles about science studies, remember to think about whether the finding is practically meaningful based on the magnitude (or effect size) of the main result. If the author doesn’t mention the effect size, you may need to wait to act on those findings until you can figure out for yourself whether the results are practically meaningful.

28