Engaging with Data Argumentation in Teacher Education

Travis Weiland & Constant Segbefia

University of North Carolina Charlotte

Acknowledgement

This material is based upon work supported by the National Science Foundation under DRK-12 Grant #2143816 and #2517085. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the view of the National Science Foundation.

Overview

• Overview of Design of Project
• Specific Intervention around Argumentation
• Themes from initial Results

Design

Project Goal

Investigating how to support secondary mathematics teachers in developing their own critical statistical literacy through data investigations of sociopolitical issues and translating that literacy into classroom practice

​

https://bit.ly/AMTE26_DataArgue

Critical Statistical Literacy Practices

Theory of Change

Generative Themes to “spark” the learning process

Learning through authentic practice in a community of practice

Cycles of reflection and action (i.e. Praxis) to become enculturated to practices

Based on a sociopolitical perspective on learning drawing heavily from Communities of Practice (Lave & Wenger, 1996) and Critical Literacy (Freire, 1970).

Data Argumentation Intervention

Authentic Practice

Zoom into Data investigations and how the argumentation process is connected to that to support teachers in teaching students about how to argue with data

(Lee et al., 2022)

Argumentation Process

​

Data argumentation is part of the data investigative process, but is a part of the process that has not been commonly focused on in teaching and is the part of the process that other people generally interact with and learn from making it especially important.

To highlight the argumentation process we have chosen to depict it as an embedded process that merits individual attention.

Argumentation

Process

Argumentation Process

​

The argumentation process is a part of the entire investigative process.

However there are two components of the data investigation process where the argumentation process is particularly important and often under emphasized in teaching. Those are during framing the problem and communicate and proposed action.

Data Investigative Process

​

We are not leaving the argumentation process we are just shifting to focus predominantly on the investigative process.

In particular, by focusing on what we have spent a lot of time on in the past which is considering the data we have, exploring and visualizing it in CODAP, and modeling it to create evidence for our argument and to develop claims (more on this later today)

Methods

Frame the Problem

Data investigations are driven by the desire to explore an issue/problem. Therefore a first step is often to try and frame the problem you want to investigate. This starts informally as just trying to put into words an issue you have noticed.

• You could start by stating a conjecture.
• A conjecture is an opinion or conviction formed on the basis of anecdotes, guesswork, or intuition.
• In other words, it is a speculation, or an educated guess that is believed to be true based on lived experience, but lacks formal proof or substantial evidence.

Conjecture Examples

It appears that when students use flashcards for vocabulary, their test scores tend to be higher.

My observation suggests that students who actively participate in school clubs are less likely to experience academic stress.

Based on the patterns I've seen in our class, I'd conjecture that the average time spent on homework per night is around 1.5 to 2 hours for most students freshman year and increases each year and as classes get more difficult.

It seems plausible that if we start school later in the morning, then students would be happier and better able to pay attention.

I'm inclined to believe that students who teach a concept to someone else retain that information better themselves.

Frame the Problem

• From a conjecture you then need to frame the problem into a question that you are trying to answer with your data investigation.
• Research questions often start very broad and might just be your conjecture rephrased as a question.
• From a broad research question you need to narrow it into an investigative question that is easily answerable with data that we can collect or find and analyze in a relatively short period of time (i.e. 2-5 class periods).
• A final key element is a hypothesis. A hypothesis is a possible explanation or educated guess about a phenomenon that can be tested through investigation.

Hypotheses

A hypothesis is a possible explanation or educated guess about a phenomenon that can be tested through investigation. In other words it is your prediction about what might happen, based on your understanding.

Key characteristics of hypotheses

Tentative: It's a proposed explanation that hasn't been proven yet.

Testable: It can be put to the test through experiments, observations, or other research methods.

Specific: It clearly states what the researcher expects to find or observe. This should include some detail on the measures that will be used and statistics that will be relied on in the analysis.

Based on existing knowledge: It should be informed by previous research and observations, not just a random guess.

Addresses a research question: It provides a clear answer or prediction to the question being investigated.

Example: Putting it all together

Conjecture

Research Question

Investigative Question

Hypothesis

It seems plausible that if we start school later in the morning, then students would be happier and better able to pay attention.

I hypothesize that mean self-reported attentiveness and mood of high school students will increase significantly from a normal start time to starting school one hour later.

How would student mood and attentiveness change if school started later?

How does mean self-reported attentiveness and mood of high school students change from a normal start time to starting school one hour later?

Problem/Issue: Many high school students struggle with sleep deprivation, which is believed to negatively impact their attentiveness and mood during early morning classes. However, the direct impact of later school start times on these specific student outcomes in a high school setting has not been sufficiently quantified to inform policy decisions.

Framing the Problem Worksheet

Framing the Problem Guide

​

Frame the Problem

KLEWS Chart �Problem/Issue: School Funding

What do you notice? What do you wonder?

What do you notice? What do you wonder?

Frame the Problem: School Funding

School Funding is complicated in the U.S. as funding for schools comes from multiple sources including Federal, State, and local government funds.

The pie charts to the right so how funding from different sources breaks down for NC in the 2021-2022 school year (Source: Public School Forum of NC).

​

​

Frame the Problem: School Funding In NC Over Time

School funding changes year by year based on what the annual appropriated for the budget to education at Federal, State, and local levels.

• What do you notice in the graph?

​

• What are you wondering about what is pictured in the graph?

Frame the Problem: How does NC Compare?

• For some context let’s first check out this video on public School Funding in NC compared to other states in the U.S.
• Here is further information about the issue of school funding in NC from WUNC.
• We can see the report card for NC from the report both of these media sources referenced here.
• The detailed report can be found here.

Revisiting KLEWS and Considering your Contextual Knowledge

What do you know about this problem?

What additional knowledge do you need to seek out?

What are your experiences with this issue/problem?

What biases might you have in relation to this problem/issue?

Who is an expert in this problem/issue I could consult?

Frame the Problem

Research Question

Conjecture

On Your Own

On Your Own

A research question is focused on a problem/issue that is framed for open-ended inquiry that guides a research study. In other words, it's a question that the research aims to answer, providing the starting point and guiding the entire research process. This is generally a large and broad question that would require multiple analyzes to begin to answer. It serves as a starting point to then refine to specific investigative questions

A conjecture is an opinion or conviction formed on the basis of anecdotes, guesswork, or intuition. In other words it is a speculation, or an educated guess that is believed to be true based on lived experience, but lacks formal proof or substantial evidence. Creating conjectures helps to write out what you think you will find based on your own lived experiences.

​

Write a conjecture(s) you have related to the problem/issue you identified?

Write a research question you have related to the problem/issue you identified?

Idea comparison

Framing the Problem-OYO

To help guide your process we have created a template. On the slide that follows you will find your name and when you click on it it will link you to a google doc you can use to record all this information.

Warm-Up

1. What do you notice?

​

1. What do you wonder?

​

1. How does this relate to our work yesterday?

​

Consider Data

Process Data

In your Group

In your Group

Let’s take a look at the data on high schools for the county. Here is the link. You have considered the data broadly now let’s consider the data more specifically for your investigative question

• What attributes/variables do you need to consider for your question?
• Do you have all the data you need to answer your question or do you need to modify it to fit the data you have? Or perhaps you want to collect some data?
• What is the structure of your data?
• Do you need to create, modify, or filter any of the attributes/variables?

If you answered yes to the last question in the previous section now you need to consider what data moves you may need to make to the data.

• Filtering—removing some cases based on the value of an attribute
• Grouping—combing cases or clustering them into distinct groups
• Calculating and recording—creating a new variable based on a calculation applied to existing attributes or by combining outcomes in an attribute to create new outcomes
• Joining— Joining together multiple datasets that measure the same cases but contain different attributes

​

Do you need to process your data? If not continue on, if yes take time to do so and record what processing you did to the data such that you could follow your directions in the future

Consider and respond to each of the questions above.

Visualize Data

Explore Data

In your Group

In your Group

A picture says a thousand words. Start with a picture of your data. Play around with different types of data visualizations and see what new aspects of the data you are able to observe. COnsider processing the data in different ways in conjunction with visualizing it. Try overlaying different types of visuals. COnsider what visualizations of the data create the best evidence for your question.

Exploring the data can take many forms. In general it is important to collect summary statistics for the variables you are considering for your investigative question. This may also involving going back and forth between exploring the dating and processing the data in conjunction with visualizing the data.

​

FInd the descriptive statistics relative to the variables you are investigating and the question you have posed

Play with different visualizations of the variables you are exploring for your investigative question

1-Categorical Variable

• Question: What proportion of high schools fall into each title I status category?
• Question Type: Descriptive
• Data Visualization: Bar graph
• Explore/Relevant Statistics:
• Describe the frequency (i.e. count) of distribution of the outcomes for the variable in a table
• Describe the relative frequency (i.e. proportion) of distribution of the outcomes for the variable in a table

​

1-Quantitative Variable

• Question: What is the typical proportion of students receiving free or reduced lunch at high schools?
• Question Type: Descriptive
• Data Visualization: Dot plot, histogram, or boxplots
• Explore/Relevant Statistics:
• Measures of center: mean or median
• Measures of spread: standard deviation, interquartile range(IQR)
• Shape: symmetric, skewed right, skewed left, bimodal, etc.
• Unusual values/outliers

​

1-Quantitative Variable & 1-Categorical Variable

• Question: How does the typically proportion of student receiving free or reduced lunches in schools compare between majority White and majority not-white schools?
• Question Type: Comparison between groups
• Data Visualization: stacked dot plots or bar graphs or histograms

• Explore/Relevant statistics:
• Same as for 1-quantitative variable except now you are comparing between two or more groups
• When comparing measures of center between groups you need to do so relative to measures of spread. For example, how far apart are the means of two groups measured in standard deviations?

2-Categorical Variables

• Question: is there an association between the racial majority in a school and it’s title I status?
• Question Type: Association/Relationship
• Data Visualization: Two-way tables, stacked bar graphs, side by side bar graphs

• Explore/Relevant statistics:
• Frequency/Counts: cell, row total, column totals
• Relative Frequencies/Percents: cell percents, row percents, column percents

Stacked bar graphs

Simple Stacked Bar Graphs place each value for the segment after the previous one. The total value of the bar is all the segment values added together. Ideal for comparing the total amounts across each segmented bar.

100% Stacked Bar Graphs show the percentage-of-the-whole by plotting the percentage of each value to the total amount in each group. This makes it easier to see the relative differences between quantities in each group.

Two-Way Table

Side-by-side bar graph

2-Quantitative Variables

• Question: What is the relationship between the total per pupil spending and the diversity index of students of high schools?
• Question Type: Relationship/Association
• Data Visualization: Scatterplot

• Explore/Relevant statistics:
• Start with the same exploration as you would have done for 1-quantitative variable
• Additionally look for patterns in the graph
• If the pattern appears at all linear describe the direction and strength of the linear relationship
• Correlation coefficient (i.e. r)
• If appropriate consider creating a linear regression model

Consider Models

​

What type of model you need to create depends on what type of question you are asking. In particular are you asking a descriptive question or an inferential one.

For most descriptive questions your model is a description of the distribution including what outcomes are possible and how often they occur.

Consider Models

​

For inferential questions your model need to take into account probability to estimate population parameters using confidence intervals or to measure the likelihood of a sample statistics given a population parameter or benchmark. Each type of variable has several types of models though in high school we typically only consider a few

Consider Models

Consider Models

Consider Models

​

Use the standard flow chart and investigation briefs to consider what model would be most appropriate for your data and question.

​

Create Your Model

Supporting the Data Investigative Process

Tools to Help Guide the Process

Worksheet for Consider, Process, Explore, Visualize and Model Data

Data Investigations Pathways

Data Investigation Briefs

• 1 Cat
• 1 Quant
• 1 Cat/1 Quant
• 2 Cat
• 2 Quant

Data Moves

• Overview
• Calculating/Recording
• Filtering
• Grouping
• Joining
• Summarizing

We have developed resources and tools to support teachers in learning about data investigations and the argumentation process as well as to help them do it with their students. We designed many of these tools to be used with teacher or student audiences.

Models and Hypotheses

How does your model compare to your hypothesis?

​

If your model confirms your hypothesis look over it again and consider if there may be other models that might also be appropriate and perhap beter

​

If your model refutes your hypothesis consider why and what that means in the context.

​

​

Communicate and Propose Action

After carrying out a data investigation it is important to communicate what you have learned from this process and relate it back to your original framing of the problem you are investigating. Some call this a data story as data does not speak for itself; people create stories with data. Others refer to this as a principled argument.

We choose to focus on principled arguments as the “principled” part of that name implies an argument that follows certain agreed upon norms or chains of reasoning.

A chain of reasoning is a multi-step explanation where each step logically leads to the next, such that someone else can follow them to the same final conclusion.

Making a Claim

After exploring, visualizing, and modeling with data, our next step is often to make a claim(s) related to the investigative question you have been investigating. A claim should be a concise and specific statement that is debatable and can be supported with data.

Concise: It should be 1-2 sentence at most

Specific: It should narrow down to a specific point, rather than being overly broad.

Debatable: Your claim should not merely be a statement of fact or summary – you need to take a position based on your analysis

Evidence

Claims cannot stand on their own in a principled argument, they must come with evidence and reasoning. In statistics there are many types of evidence to provide including:

• descriptive statistics
• data visualizations
• measurement considerations
• descriptions of sample and sampling
• a point of comparison
• and depending on the situation, inferential statistics.

To help make sense of how to use evidence we have created a rubric in this document

Reasoning

You cannot simply provide evidence though. To create an argument you must also provide reasons for why that evidence justifies the claim that you have made. The reasoning is like the glue that holds it all together. In the end you combine claims and evidence through a chain a reasoning such that someone else can follow your think to come to the same conclusions you have.

We have also created a rubric for considering the reasoning in an argument in this document.

​

Connecting the Pieces

Let’s start with some initial argument development to try identify and connect the basic pieces of the argument. For evidence in your final argument you will want to provide select data visualizations and statistics but for now you can just take notes as to what those things will be and focus on connecting the piece of evidence to the claim with reasoning.

​

This document may serve as a helpful reference for this activity

​

Elements of a Principled Argument

• A statement of the problem/issue you are discussing. This may include things like:
• Research questions
• Conjectures
• Hypotheses
• Important contextual information or relevant literature
• Overview of Methods
• What is your data source?
• What is your sample and sample size?
• How did you process the data?
• How did you explore and visualize the data?
• How did you model the data?
• Claim(s)
• Evidence
• Reasoning
• Action/Decisions.
• What do we do based on this information?
• What actions do you recommend?

Communicate and Propose Action

Propose Action

Generally an argument does not end with merely supporting a claim it usually goes into what we often call the “So What.” So what do we do know based on what we have learned? How does this help us make decisions? What actions are recommended based on these results? This is different for every question and issue and will draw upon your know of the issue you are investigating not just the results of your investigation.

​

Initial Results

Data

• End of day reflections
• Documents produced during activity
• Fieldnotes
• Transcripts from presentations

​

Themes

• Community is important for developing understanding
• Teachers struggle with what counts as good evidence to develop strong data based arguments
• Teachers see data argumentation as useful for teaching
• Teachers struggle to incorporate data argumentation into their planned instruction

Community

Building confidence and feeling supported

“...I was stressed about the hypothesis, claim, evidence and response. But! My group encouraged me to take that on and using the documents provided helped me feel more confident.”

Developing understanding

“It help me simplify investigative framework and narrow down targeted topics. It helped me understand common errors on disaggregate vs aggregate phenomenon.”

“Seeing how others interpret or work with data helps me clarify my own understandings.”

Usefulness for Students

Help support the investigative process and students interpreting data

“I think that learning how to design an investigative plan helps translate to our classes for students to do a project for our class”

“I am really excited about the potential conversations my students would have around this data and I think with some further chunking that my students could use this as a way to learn writing a research question, making data visualizations, analyzing data and forming a conclusion.”

“The statistical argument organizer and the presentation”

Challenges in Creating Arguments

Challenges

• What counts as good statistical evidence
• Questions exceeded technical background
• How to clearly present evidence of reasoning in a data visualization

Possible Improvements in Future Design

• There needs to be more direct feedback and refinement loops built in for feedback on the content
• More time.
• More examples

​

High schools in the county are grouped together high-poverty areas into the same zone.

Academy

Early College

Evidence and Reasoning

57.69% of HS State PPEs were below the mean funds.

72.73% above the mean funds achieved higher than 67.5% (mean achievement score) and 27.27% of HS State PPEs above the mean scored below 67.5%.

​

73.33% of schools above -0.39 (mean) SGI received more than $7376 per pupil (mean).

Challenges in Translating to Lesson Planning

Challenge

• Data argumentation did not end up in their lesson plans that they created at the end of our summer session

Possible Improvements in Future Design

• Provide explicit examples of lesson plans with data augmentation incorporated
• Provide examples of how data argumentation supports their current teaching
• Provide more time for planning

​

Thank You

Please email us anytime if you have questions or are looking for resources

​

Also check out our website for more resources

www.criticalstatisticalliteracy.org

Data Moves Resources

​

​

Other Resources

• Awash in Data Open Book
• Research/Practitioner Article
• Blog connections

​