Activity 7: Memory Study

1

GAISE

1. Teach statistics and data science as iterative processes of gleaning insights from data to inform evidence-based decisions.
2. Emphasize effective written and oral communication of results from data, with attention to the scope and limitations of conclusions.
3. Focus on conceptual understanding rather than algebraic manipulation and formulas.
4. Integrate real data with a context and purpose throughout the course.  Select data that are meaningful and engaging to the students. 
5. Encourage multivariable thinking.
6. Incorporate software/apps to explore concepts and work with data.
7. Emphasize responsible and ethical conduct in the collection and use of data and in their analysis.
8. Employ evidence-based pedagogies that actively engage students in the learning process.
9. Use a variety of formative and summative assessments to improve teaching and learning.
10. Implement a course design that uses inclusive strategies to foster a sense of belonging. 

​

2

Memory Study

3

2 follow-up questions

Motivational Video (4 minutes): https://youtu.be/4caEuNmlaWA

Memory Test

• https://www.rossmanchance.com/memorytest.html?instructor=Chandler2025

​

4

Leave the webpage open after you complete the questions.

STEP 1: Ask a research question

• 1. What sorts of things do you think affect a person’s memory?
• RQ: Does the type of chunking make a difference in the number of letters from a sequence that people can remember correctly after memorizing for 20-seconds?

​

​

5

Memory Score (# of letters correct)

Type of Sequence

Goal: Cause-and-effect?

• Sequence A: FBI-OMG-CIA etc.
• Sequence B: FBIO-MGC-IAU etc.

STEP 2: Design a Study & Collect Data

DATA: collect data from students

• Each student was randomly assigned to a sequence of 30 letters and was given 20 seconds to memorize as many as they could in order from left to right
• Recorded each student’s memory score, the number of letters they remembered correctly (in order)
• Recorded the sequence: A: FBI-O or B: FBIO-
• Recorded whether they’ve had any caffeine today (Y,N) and how much sleep they got the previous night (0, 0.5, 1, 1.5…. Hours)

6

STEP 2: Design a Study & Collect Data

• 2. What is the observational (or experimental) unit in this study and how many are there?
• 3. Identify the explanatory variable in this study and classify it as categorical or quantitative.
• 4. Identify the response variable in this study and classify it as categorical or quantitative.
• 5. Was this a designed experiment or an observational study? How are you deciding?

7

STEP 2: Design a Study & Collect Data

• 6. Will each person have the exact same memory score? If not, what are some possible sources/reasons for the variability in memory scores?

8

Memory Score (# of letters correct)

Type of Sequence

Goal:

Sources of Variation Diagram

9

Sources of Variation Diagram

10

Sources of Variation Diagram

11

Sources of Variation Diagram

12

Sources of Variation Diagram

13

Sources of Variation Diagram

14

Scope of inference

Limitations of study

Memory Scores

• Return to your open Memory Score page
• Load the results into the Comparing Groups applet.
• Make sure memory score is the response variable and sequence is the explanatory variable

​

7a. Based on the graphs alone, which sequence tends to a have higher memory score? How much variability is there in the memory scores? Any unusual memory scores?

7b. Record the sample sizes, means and standard deviations for each sequence.

7c. Based on the numerical summaries, which sequence had a higher mean memory score?

15

STEP 4: Draw inferences beyond the data

• 8. What are two possible explanations for Sequence A: FBI-O having had a higher mean memory score than Sequence B: FBIO-?

16

• 9. Which explanation do you think is responsible for the difference in the mean memory scores between the two sequences?

Ho: Sequence does not have an effect on mean memory score in the long-run

Ha: Sequence does have an effect on mean memory score in the long-run

STEP 4: Draw Inferences beyond the data

• We cannot prove directly that the sequence matters (explanation 2, Ha)
• We can try to rule out explanation 1 (Ho)
• The thinking goes: If we rule out explanation 1, and the study is well-designed, then explanation 2 is the ‘only’ other reasonable explanation for our study result.
• How do we rule out explanation 1?

17

STEP 4: Draw inferences beyond the data

• Because explanation 1 represents “random chance” we can use a simulation to investigate the types of results we can expect just by random chance alone.

​

• Use 3S strategy: Statistic, Simulate, Strength of Evidence

18

Statistic

• 10. A natural statistic for measuring how different the mean memory scores for the 2 sequences are is the difference in means. Report the value of this statistic (Seq A minus Seq B).

19

Simulate

• For a moment, let’s assume that explanation 1 is true, i.e., Ho is true.
• 🡺 The sequence has no effect on mean memory score
• 🡺 For our study data, this means that the scores are not a result of the sequence and could just as easily come from either sequence
• 🡺 Shuffle the observed scores and re-assign them (at random) to the two sequences
• 🡺 Allows us to see what could have happened just from the random assignment process alone (for these particular data)

20

Simulate

21

11.

​

1. Obtain a set of _____ cards
2. What will be written on each card?

Mean A – Mean B

Graph the shuffled difference in means

Simulate

• If you have time in class, have pairs of students simulate one outcome using cards
• 🡺 Plot these outcomes on the board

​

• 11e. At what value is the dotplot centered?
• 11f. Where is the observed difference in means from the original study located in this dotplot? How often did this value or something more extreme happen by random chance alone?

22

Strength of Evidence

• From the Comparing Groups applet:
• 13. Based on the p-value, evaluate the strength of evidence against the null hypothesis: not much evidence, moderate evidence, strong evidence, very strong evidence.

23

STEP 5: Formulate Conclusions

• 14. Generalization Were the participants in the study randomly selected from a larger population? Describe the population to which you would fee comfortable generalizing the results of this study.

​

• 15. Causation: Were the participants randomly assigned to a sequence? What type of conclusion can we draw?

​

24

STEP 5: Formulate Conclusions

• 16. Conclusion: Write a conclusion which answers the research question with regard to the strength of evidence in the context of the study.

​

• Based on the _____ p-value = _____ we do/do not have ______ evidence to conclude the type of sequence has an effect on mean memory score. This conclusion applies to all people of the similar age and major who are otherwise similar to those in the study, under similar conditions as the study conditions (see constant by design criteria).

25

STEP 5: Formulate Conclusions

• 17. Estimation: 95% CI

​

​

26

STEP 6: Look back and ahead

• 18. Looking back: Did anything about the design and/or study conclusions concern you?
• Possible examples:
• Any mismatch between the RQ and the study design
• How observational units were obtained
• How treatments were assigned to the observational units
• How the measurement of the RV was done
• The number of observational units in the study
• Whether what was observed is of practical value; how will humans in the wild respond?

27

STEP 6: Look back and ahead

• 19. Looking ahead: What should the researchers’ next steps be to address the limitations/issues identified in #18?

28

Theory-based test and CI

29

30

To generate a class specific URL for the Memory Study:

https://www.rossmanchance.com/applets/getMemTestURL.html

​

https://www.isi-stats.com/

​

Applets: