1 of 21

A Critical Statistical Literacy Framework for Reading Data Visualizations

Travis Weiland �Laura Shelton

University of Houston

2 of 21

Acknowledgement

3 of 21

Why a critical statistical literacy framework for reading data visualizations?

Graphs have long been a part of math curricula.

Beyond graphs and tables to data visualizations
Media is changing and the curriculum needs to change with it

Critical perspectives

Reading the word and the world (Freire, 1972)
Reading the word and the world with statistics (Weiland, 2017)

Reading graphs of data is a particular type of reading often associated with statistical literacy (Gal, 2002) that has increasingly become a part of U.S. schools’ mathematics curriculum in the past several decades (Scheaffer & Jacobbe, 2014). Past scholarship (i.e., Curcio, 1987; Friel et al., 2001; Shaughnessy, 2007) has provided guidance on the different ways that people engage in reading graphs, shaped curriculum, and guided scholarship on supporting people’s understanding of reading graphs. However, the visualizations of data that we commonly see in the media today go far beyond the traditional graphs considered in earlier scholarship. For example, Wilkerson and Laina (2018) describe data visualizations as those that “use context rich and interactive methods to create narratives and allow users to explore data for themselves” (p. 1), which expands beyond graphs to include new forms of data visualization such as interactive spatial displays of data or scrolly-telling where data visualizations change as a person scrolls down an article on a device. Therefore, we see a need to revisit old frameworks with new data and lenses to consider the realities of how people encounter data in the world today.

Our work draws upon the theoretical framing of critical statistical literacy (Author, 2017), which theorizes the intersection of critical literacies drawing heavily from Freire’s (1970) literacy work and the idea of reading and writing the word and the world and statistical literacy drawing heavily from Gal’s (2002) descriptions of reading in statistics and Wild and Pfannkuch’s (1999) work on statistical enquiry.

4 of 21

Background Literature on Reading Data Visualizations

Reading Level	Description
Reading the data (Friel et al., 2001)	Lifting information from the graph to answer explicit questions for which the obvious answer is in the graph
Reading between data (Friel et al., 2001)	Interpretation and integration of information that is presented in a graph – the reader completes at least one step of logical or pragmatic inferring to get from the question to the answer
Reading beyond the data (Friel et al., 2001)	Extending, predicting, or inferring from the representation to answer questions – the reader gives an answer that requires prior knowledge about a question that is related to the graph
Reading behind the data (Shaughnessy, 2007, as cited in Rubel et al., 2016)	Interpretations of why particular patterns exist in the data as well as questioning the sources of the data, the sampling used to generate it, and other factors

5 of 21

Updating the Framework

The reading graph levels framework described by Friel et al. (2001) and Shaughnessy (2007) was helpful but not sufficient
Was missing a critical literacy lens (Freire, 1970; Gutstein, 2006; Gutierrez, 2013).
Drawing from more recent literature (Bailey & McCulloch, 2023; da Silva et al., 2021; Rubel et al., 2016; Rubel et al., 2021)

6 of 21

Reading the Word and the World

Reading the Word: drawing from disciplinary or dominant discourses, technical language, etc.

Reading the World: Making sense of one’s day to day lived experiences, the communities they are a part of, interrogating disciplinary or dominant discourses and what they produce, sociopolitical consciousness.

7 of 21

	Description	Reading the Word Practices	Reading the World Practices
Reading the data	Lifting information from the graph to answer explicit questions for which the obvious answer is in the graph Extract information from the data (Friel et al., 2001)	Recognizing the components of graphs and the relationship among the components Speaking the language of specific graphs when discussing information displayed in graphical form	Looking for oneself in the data
Reading between the data	Interpretation and integration of information that is presented in a graph the reader completes at least one step of logical or pragmatic inferring to get from the question to the answer Find relationships in the data (Friel et al., 2001)	To understand the relationships among multiple representations of data Finding relationships or trends in the data visualized beyond simply lifting information from the graph Making sense of a graph beyond simply lifting information from the graph Recognizing the types of relationships (correlational or causal) that could be claimed based on the data collection methods Identifying how the author has highlighted particular relationships in the graph	Making sense of the data visualization in relation to one’s own personal experiences Questioning how and why the author has highlighted particular relationships in the graph Reimagining other ways that relationships could be highlighted or visualized

8 of 21

	Description	Reading the Word Practices	Reading the World Practices
Reading beyond the data	Extending, predicting, or inferring from the representation to answer questions – the reader gives an answer that requires prior knowledge about a question that is related to the graph Move beyond the data (Friel et al., 2001)	Interpreting information in a graph and answering questions about it beyond initial sense making Predicting or extrapolating based on claims made from the graph Making claims/inferences based on patterns and trends in the data to a population beyond what is represented in the data	Making predictions/claims/inferences from the data visualization by drawing upon one’s own personal experiences Acknowledging possible alternate explanations of the data represented Recognizing the story, the author is trying to tell with a data visualization Questioning the author’s motives for telling the story they are telling in the data Identifying possible inequities while interpreting the data visualization Recognizing one’s bias and its impact on interpreting the data
Reading behind the data	Making connections between the context and the data (Shaughnessy, 2007), including questioning how the data was collected and how the data visualization represents the context	Looking for possible causes of variation in the data, based on the context being measured and the way the data was collected Recognizing appropriate graphs for a given data set and its context Questioning methods (e.g. sample size, collection methods, etc.) and their impacts on inferences (e.g. practical significance vs. statistical significant; effect vs. no effect) Recognizing when common sources of bias are present in the data collection Questioning the methods used to represent/display the data Recognizing and questioning the source of the data including what is quantified and how it was measured	Using your knowledge of the context of the data to interpret why particular patterns exist in the data and data generation process Using knowledge of one’s community to interpret why particular patterns exist in the data to question aspects of the data generation process Questioning the investigative process undertaken based on personal experiences/identity Recognizing that one needs more contextual knowledge to interpret the statistical message in the data visualization

9 of 21

Methods: Context

Part of a larger design-based research project
Context: University is HSI and AAPISI institution. Took place during a master’s level course on teaching statistics and probability
Sampling: convenience sampling from a course
Participants: 8 practicing teachers. All women – 4 white, 2 biracial white and Latina, 1 biracial Black and white, 1 Black
Wanted to use our framework with practicing teachers because they are more likely to be used to teaching and reading data visualizations in a variety of ways.

10 of 21

Methods: Data Collection & Analysis

Data: Participants’ responses to 3 discussion board assignments using data visualizations from the NYT “What’s Going on in this Graph?”
Responses were coded based on the framework we have shared
Both authors coded all the data independently and reconciled for 100% agreement

11 of 21

Results

Question	WORD				WORLD
Question	Reading the Data	Reading Between the Data	Reading Beyond the Data	Reading Behind the Data	Reading the Data	Reading Between the Data	Reading Beyond the Data	Reading Behind the Data
What do you notice?	10	11	3	0	0	0	3	0
What do you wonder?	0	1	0	10	1	5	4	6
Create a catchy headline that captures the graph’s main idea.	1	3	13	0	0	0	7	0
What relationship(s) is the author highlighting?	3	16	7	0	0	0	2	0
What is a claim you could make from this data visualization?	2	7	14	0	0	0	2	0
What evidence are you using to make this claim?	13	14	2	0	0	0	2	0
How does this relate to you and/or your community?	0	0	2	0	4	4	9	1

Overall, we saw evidence of all of the ways of reading data visualizations in our sample (see Table 3). Additionally, we were able to code all participants’ responses to the reading data visualization questions. This provides evidence that our framework was capturing the variation in ways of reading data visualizations in the sample we collected. We also found evidence of the need for a reading the world in line with the growing body of literature in critical data literacies described earlier. For example, here is a statement from one participant that is an example of reading the data (world) as they are seeing their own personal story played out in the data:

This data is representative of the trends that I have experienced in higher education myself. I completed my undergraduate education as a traditional-aged undergraduate student. Most of my course work was in person; very seldomly did I have the chance to take an online class or opted into taking an online class. Whereas my graduate education has primarily had some education component in each class.

Connecting one’s personal experiences to the broader trends in a visualization is important for readers when interpreting data. This practice is an aspect of what experts do in moving between views of data including individual, case and aggregate (Konold et al., 2015). The participant showed evidence of this in an earlier statement in her response to the question of, what do you notice?

This graph is being used to represent how chief online officers expect students to experience their classes. The data is being represented in a horizontal bar graph. A trend that I observed in the graph is that by 2025 less than 5% of classes are expected to be experienced in a fully on campus or little to no online matter. Most classes are expected to be experienced in a balance of online and on campus format.

In this statement, we see the participant reading the data by stating elements of the data visualization she observes in stating the type of graph is being used. She then goes on to read between by stating a trend in the data she is observing of few experiences for any group to experience little to no online educational components. She then ends with reading beyond the data by making a claim/inference that most students will experience a balance of online and on-campus educational experiences.

From looking at the frequency at which our participants are reading graphs in different ways, it is clear that overall participants are reading the graphs in typical, statistical kinds of ways, which makes sense given the participants are mathematics teachers and have had many experiences in developing quantitative reasoning over their educational experiences. Furthermore, much of the focus is on the original three ways of reading describe by Friel et al. (2001) in their framework. Reading behind the data occurred but was not very prevalent as compared to the other types of reading, which is important to note as this type of reading encapsulates making connections between the context and the data (Shaughnessy, 2007). Shuttling between context and data is a crucial aspect of statistical inquiry (Wild & Pfannkuch, 1999), so it is troubling that it is the least common type of reading occurring.

12 of 21

Results

	WORD				WORLD
Participant	Reading the Data	Reading Between the Data	Reading Beyond the Data	Reading Behind the Data	Reading the Data	Reading Between the Data	Reading Beyond the Data	Reading Behind the Data
Haya	5	8	6	2	0	1	2	1
Alice	3	9	6	3	1	0	1	0
Julie	4	4	6	2	1	0	5	1
Danielle	4	7	2	1	0	1	6	1
Jennifer	6	8	2	0	0	2	3	1
Michelle	5	6	6	0	1	1	5	2
Nicole	1	5	11	1	1	3	1	1
Valerie	7	5	3	1	1	1	3	0
TOTAL	35	52	42	10	5	9	26	7

13 of 21

Discussion

Takeaways:

The type of questions asked lead to different types of reading – useful for curriculum development and teacher educators
Probing and follow up questions can also help support learners in reading data visualizations

Where we are now:

Framing writing the world needed to be expanded.
Reflecting on using Freire’s reading & writing the world
Reflecting on reading types vs. levels.

This study presents findings from the third iteration of a design-based research project that build upon a foundational framework. The framework we present here has scholarly significance in several ways. One way is that this framework can be used to create learning experiences for people in reading data visualizations and can also be used to assess people’s learning of reading data visualizations. In this direction, we have been creating activities to support learners in using different types of reading of data visualization and tools for teachers that include questions to try to assess and advance student’s reading of data visualizations. Our analysis based on the questions posed in Table 2 shows that this is a powerful avenue to pursue for future work. Another example of the scholarly significance of this work is its potential for supporting future research around people’s reading of data visualizations. This framework still needs to be refined with a variety of populations to make sure it captures the variation of reading types and refined to help differentiate between types of reading in ways that help inform research and practice.

14 of 21

References

Bailey, N. G., & McCulloch, A. W. (2023). Describing critical statistical literacy habits of mind. Journal of Mathematical Behavior, 70. https://doi.org/10.1016/j.jmathb.2023.101063

da Silva, A. S., Barbosa, M. T. S., de Souza Velasque, L., da Silveira Barroso Alves, D., & Magalhães, M. N. (2021). The COVID-19 epidemic in Brazil: How statistics education may contribute to unravel the reality behind the charts. Educational Studies in Mathematics, 108(1–2), 269–289. https://doi.org/10.1007/s10649-021-10112-6

Freire, P. (1970). Pedagogy of the oppressed. Continuum.

Friel, S. N., Curcio, F. R., & Bright, G. W. (2001). Making sense of graphs: Critical factors influencing comprehension and instructional implications. Journal for Research in Mathematics Education, 32(2), 124–158. https://doi.org/10.2307/749671

Rubel, L. H., Lim, V. Y., Hall-Wieckert, M., & Sullivan, M. (2016). Teaching mathematics for spatial justice: An investigation of the lottery. Cognition and Instruction, 34(1), 1–26. https://doi.org/10.1080/07370008.2015.1118691

Weiland, T. (2017). Problematizing statistical literacy: An intersection of critical and statistical literacies. Educational Studies in Mathematics, 96(1), 33-47.

15 of 21

Data Visualization 1

https://www.nytimes.com/2023/01/12/learning/whats-going-on-in-this-graph-jan-18-2023.html

16 of 21

Data Visualization 2

https://www.nytimes.com/2022/10/27/learning/whats-going-on-in-this-graph-nov-9-2022.html

17 of 21

Data Visualization 3

https://www.nytimes.com/2022/05/05/learning/whats-going-on-in-this-graph-may-11-2022.html

18 of 21

Current Iteration of Framework

Reading Types	Reading the Word Practices	Reading the World Personal/Community Practices	Reading the World Sociopolitical Practices
Reading the data Extract information from the data (Friel et al., 2001)	To recognize the components of graphs, the interrelationships among these components, and the effect of these components on the presentation of information in graphs (Friel et al., 2001) To speak the language of specific graphs when reasoning about information displayed in graphical form (Friel et al., 2001)	Looking for oneself in the data (Rubel et al., 2016)	Look for source of data Look for author/affiliation of visualization Questioning why an author has highlighted particular aspects of a graph or left them absent

19 of 21

Current Iteration of Framework

Reading Types	Reading the Word Practices	Reading the World Personal/Community Practices	Reading the World Sociopolitical Practices
Reading between the data Find relationships in the data (Friel et al., 2001)	To understand the relationships among a table, a graph, and the data being analyzed (Friel et al., 2001). Finding relationships or trends in the data visualized Recognizing the types of relationships (correlational or causal) can be claimed based on the data collection methods (Utts, 2003) Identifying the relationships highlighted in the graph (Rubel et al. 2021)	Making sense of the data visualization in relation to one’s own personal experiences	Questioning how and why the author has highlighted particular relationships in the graph (Rubel et al. 2021)

20 of 21

Current Iteration of Framework

Reading Types	Reading the Word Practices	Reading the World Personal/Community Practices	Reading the World Sociopolitical Practices
Reading beyond the data Move beyond the data (Friel et al., 2001)	Interpreting information in a graph and answering questions about it (Shaughnessy, 2007) Predicting outcomes based on reasonable claims made from the graph (Shaughnessy, 2007) Making claims/inferences based on patterns and trends in the data to a population beyond what is represented in the data.	Making predictions/ claims/ inferences from the data visualizations by drawing upon one’s own personal experiences Recognition of one’s bias and its impact on interpreting data (modified from Bailey & McCulloch, 2023; Weiland, 2017) Acknowledging possible Alternate Explanations (Bailey & McCulloch, 2023) Drawing upon personal experiences facing inequities in the interpretation of the data visualization (Bailey & McCulloch, 2023) Connecting to one’s feelings/emotions related to the data visualization (Kahn et al. 2022)	Making connections to alternate explanations of others (Bailey & McCulloch, 2023) Recognizing the story, the author is trying to tell with this data (Rubel, 2021) Questioning the author’s motives for telling this story (Rubel et al., 2021) Identifying structural inequities at play in the interpretation of the data visualization (Bailey & McCulloch, 2023)

21 of 21

Current Iteration of Framework

Reading Types	Reading the Word Practices	Reading the World Personal/Community Practices	Reading the World Sociopolitical Practices
Reading behind the data Making connections between the context and the data (Shaughnessy, 2007)	Looking for possible causes of variation (Shaughnessy, 2007), based on the context being measured and the way the data was collected Looking for relationships between variables based on the context Recognizing appropriate graphs for a given data set and its context (Shaughnessy, 2007) Recognizing Appropriate Statistics & Appropriate Representations (Bailey & McCulloch, 2023)	Using your knowledge of the context of the data to interpret why particular patterns exist in data as well as data generation process Using knowledge of one’s community to interpret why particular patterns exist in the data to question aspects of the data generations process Questioning the investigative process undertaken based on personal experiences/identity Recognition of the gaps in one’s knowledge [of the context] needed to interpret the statistical message. (Bailey & McCulloch, 2023)	Questioning sample size and methods (Bailey & McCulloch, 2023) and their impacts on inferences (i.e. practical significance vs. statistical significant; effect vs. no effect) (Utts, 2003) Recognizing when common sources of bias are present in the data collection (Utts, 2003) Recognizing and questioning the source of the data including what is quantified and how it was measured (Rubel et al., 2021; Weiland, 2017)