Supporting Children’s Learning�Through AI-Driven Dialogues

Ying Xu, PhD

Marsal Family School of Education

University of Michigan

Title Slide 2

AI as Conversation Partners?

Converse to Learn Project

• Integrate AI-driven dialogue in educational media to support personalized learning

Converse to Learn Project

• Integrate AI-driven dialogue in educational media to support personalized learning

AI-Driven Dialogue Improves Science Learning

Conversational video

Non-interactive video

Children’s understanding of science concepts

Xu et al., CHI 2022

Xu et al., Journal of Applied Developmental Psychology, 2022

Science Assessment Scores

How We Developed AI-Driven Dialogue

Narrative

…

Child

response

…

Question 2

Question 9

…

Feedback to Color

Generic Feedback

…

Pre-determined questions

Pre-determined response categories

Pre-determined feedback

Question 1

Feedback to Engine

Feedback to Pattern

Feedback to Size

Feedback to Stripe

Trade-Offs of Pre-Scripted AI

• Unable to provide tailored feedback for unpredicted responses

​

• Labor- and resource- intensive�

What new possibilities can Generative AI offer？

• Proven benefits to learning outcomes

​

• All content validated by human experts

Experiment with Off-the-Shelf LLMs

• Using GPT-4 to co-create math-focused stories with children in real time

Starting point of the story

Question

GPT-4

Child

response

GPT-4

Question

GPT-4

Question

Feedback/Story

continuation

Child

response

How We Developed AI-Driven Dialogue

Question generation

Question generation

Prompting

Feedback/story continuation

Feedback/story continuation

Prompting

AI as Classifier

AI as Content Creator

Prompt engineering

Using natural language to prompt GPT-4 to behave as an engaging storyteller

You're a storytelling robot. Let's play a joint storytelling game where we build a story together with the main character being a [character] named [name] in the [setting]. This story should revolve around the main character and a little kid named [the child's name] reaching a certain location, attaining a certain object, or fulfilling a certain goal while conquering many obstacles along the way. To start, please craft a 2-sentence introduction to the story. From there, we will alternate turns, with each person adding more to the story. When it is your turn, only write the story content while using as many of the following words as possible: equal, sum, half, add, subtract, and estimate. Please keep your responses simple and appropriate for a young child. Please be creative and have fun with this storytelling adventure!

First, acknowledge my addition and praise me in a short sentence. Next, continue the story by correctly using [mathematical term one] and [mathematical term two] or their variants in two short sentences. Lastly, explain their meanings in one short sentence in the context of the story.

Quality of the generated text

Children’s engagement

Math learning

Challenges with Prompt Engineering

Question generation

Prompting

Feedback/story

continuation

Prompting

Blackbox:

We don’t always get exactly what we want.

AI as Content Creator

The Human-in-the-Loop Approach

Question generation

Prompting

Feedback/story continuation

Prompting

AI as Collaborator for Content Creation

Verification &

Modification

The Human-in-the-Loop Approach

Open Questions:

1. What are the desired levels of automation that teachers find helpful when it comes to AI?
2. What level of revisions and modifications do teachers consider necessary?
3. How does this integration align with teachers' instructional workflow to provide support and enhance their teaching practices?

Ying Xu

yxying@umich.edu

Ying-xu.com