1 of 46

Generative Artificial Intelligence for K-12 Education

MIT CSAIL x MIT MEDIA LAB

Prof Hal Abelson, Prof Cynthia Breazeal, Prof Randall Davis, Prerna Ravi, Safinah Ali and Kate Moore

2 of 46

3 of 46

4 of 46

5 of 46

Agenda

10 Community Announcements + Intro to Final Project

5 Post-it placement for today’s project mixer :D

10 Homework Highlights Partner Share

10 Intro to Examples of Generative AI for Education

20 Activity 1: Multi-dimensional Post-it Clustering

30 Activity 2: Jigsaw the Learning Sciences

30 Activity 3: Project Brainstorming + Mixer

10 Closing

5

6 of 46

Community Announcement

MIT Ignite: Generative AI Entrepreneurship Competition Taking place Oct. 30; interest forms due Sept. 22nd

6

Scan for more info!

7 of 46

Class website

mit-cml.github.io/gen-ai-fall-2023.github.io/

Class links are also here:

tinyurl.com/

GAIK12Class2

8 of 46

Final Project

Class 3 September 26th

Class 6

October 24th

Class 10 November 21st

Class 13 December 12th

1st Check-In

Have a team & a plan

2nd Check-In

Have a prototype to demo

3rd Check-In

Have results from playtest

Report + Pre-Recorded Presentation Due

Timeline

Action Items Due

9 of 46

What brought you here? Here’s what you said! :D

Image: Your introductions (controlling for “generative AI” and “interested”

Source: Free Word Cloud Generator

10 of 46

Post-It Placement for Today’s Project Mixer

EdTech /

Media Design

Computer Science

Education

What’s your background?

10

11 of 46

Homework Highlights

Partner Share

How much experience do you have with coding?

1 = none 2 = a little 3 = not sure 4 = some 5 = lots

12 of 46

Intro to Generative AI in Education

13 of 46

First a caveat,

We are riding a wave of generative AI development.

The technology is changing very rapidly.
It is disruptive technology.
It is currently driven by hype and the orgs out front.

14 of 46

Historic Disruptive Technologies

Some examples…

Cameras
Personal computers
iPhones

Some examples specific to education…

Video Players
Calculators
The Internet
Wikipedia

15 of 46

Positive Effects of Disruptive Ed. Technology

Accessibility.
Access to knowledge for various SES backgrounds.
personalization
taps into different/new modalities of learning and styles
Good to educate the next generation regarding the new tech out there: instead of complicating it

Negative Effects of Disruptive Ed. Technology

misuse/ambiguous purposes

ChatGPT: how to use it and “ways” to use it
creates moments of misuse

Perplexity.ai: doesn’t leave room for things that are less represented in the data → creates bias
short-term: available only to few people → only some enjoy the benefits of it
methods of learning:
tech is considered solution to all problems: when some are better solved without tech

16 of 46

Surge of Generative AI for Education

Source: MOOC AI & Education, figure by Sergio Martin Associate Professor at UNED

17 of 46

Summary of Research on Generative AI in Education

Good summary. Top search result as of this morning.

UNESCO offers another up to date,

less rosey, overview.

18 of 46

Hype on Generative AI in Education

19 of 46

Companies currently “out front”

in development of generative AI tools for education

Google

Bard

Microsoft

Bing

OpenAI

chatGPT

DALL-E

Runway

Gen-2

Stable Diffusion

Dream Studio

20 of 46

What is chatGPT?

Generative. Pre-trained. Transformer.

21 of 46

What does chatGPT do?

It’s a generative AI that predicts the next word in a sentence.

GPT-4

Input

Output

If

the

shoe

fits

...

22 of 46

How does chatGPT work?

Model is trained.

Reward model

is trained.

Model and reward model train each other

Source: OpenAI, November 2022: https://openai.com/blog/chatgpt

Human example responses to the prompt.

23 of 46

How does chatGPT work?

Model is trained.

Reward model

is trained.

Model and reward model train each other

Source: OpenAI, November 2022: https://openai.com/blog/chatgpt

Humans rank the output.

24 of 46

How does chatGPT work?

Model is trained.

Reward model

is trained.

Model and reward model train each other

Source: OpenAI, November 2022: https://openai.com/blog/chatgpt

Here is the reward model.

25 of 46

How does chatGPT work?

Model is trained.

Reward model

is trained.

Model and reward model train each other

Source: OpenAI, November 2022: https://openai.com/blog/chatgpt

There is an initial random seed…

26 of 46

How does chatGPT work?

Model is trained.

Reward model

is trained.

Model and reward model train each other

Source: OpenAI, November 2022: https://openai.com/blog/chatgpt

The reward model corrects and updates the policy.

27 of 46

How does chatGPT work?

Model is trained.

Reward model

is trained.

Model and reward model train each other

Source: OpenAI, November 2022: https://openai.com/blog/chatgpt

and the updated model generates new output.

The reward model corrects and updates the policy.

28 of 46

How does chatGPT work? Why do we care?

Model is trained.

Reward model

is trained.

Model and reward model train each other

Source: OpenAI, November 2022: https://openai.com/blog/chatgpt

This data is scraped from the Internet.

Humans are evaluating the response.

The output is based on a policy for predicting which word is likely.

So, output sounds plausible, but it is not based on Truth.

29 of 46

What can chatGPT do?

A note on the Uniform Bar Exam → GPT-4 reached the 90th percentile according to OpenAI, independent research has shown that the score was more likely in 68th percentile.

Sources: OpenAI (2023)

It can pass standardized tests. Is this evidence of intelligence?

No, it’s evidence that the data it was trained with prepares it with enough information to answer the questions on these tests. It also suggests that the tests may checking for recall of information rather than an ability to reason with or about it. This high performance could be interpreted as a critique of the tests; it suggests they can’t distinguish between advanced memorization and understanding / reasoning.

Which means that if new tests were made that asked for information that wasn’t readily available online before 2021, chatGPT might not perform as well.

Importantly, these scores are reported by OpenAI. There has been little time for independent research to validate these scores.

Although, those who have been able to do so have found that the scores are not as high as OpenAI claims, which suggests that chatGPT may produce medicure, not stellar responses.

References

Additional Notes from Martinez (2023)

“GPT-4's UBE score nears the 90th percentile when examining approximate conversions from February administrations of the Illinois Bar Exam, these estimates are heavily skewed towards repeat test-takers who failed the July administration and score significantly lower than the general test-taking population.”
“data from a recent July administration of the same exam suggests GPT-4's overall UBE percentile was ~68th percentile, and ~48th percentile on essays.”
“examining official NCBE data and using several conservative statistical assumptions, GPT-4's performance against first-time test takers is estimated to be ~63rd percentile, including ~42nd percentile on essays.”
“when examining only those who passed the exam (i.e. licensed or license-pending attorneys), GPT-4's performance is estimated to drop to ~48th percentile overall, and ~15th percentile on essays.”

30 of 46

What can chatGPT do?

It can…

Provide factual information that is in its training dataset
Apply a style of writing
Summarize and paraphrase text
Write code

It can not (currently)...

Provide up-to-date information or news
Provide references (maybe?)

31 of 46

What can chatGPT do in education?

For teachers:

Generate lesson ideas, simple activities, problem sets
Give feedback on assessments
Formative assessment
Scale difficulty of assignments
Teach students to use generative AI to enhance their skills

For students:

Summative informational resource
Get ubiquitous feedback on assignments
Simplify or translate language
Generate seed ideas
Debugging code
Ask for step-by-step instructions for complex tasks.

32 of 46

Multi-dimensional Post-it Clustering

Activity 1

33 of 46

Post-it Clustering in 2-dimensions

Step 1. Draw a coordinate plane.

Step 2. Label X

Step 3. Label Y

Step 4. Create as many post-its as you can answering the question:

What can generative AI be used to do in education?

Step 5. Place your post-its on the coordinate plane.

use it for work

use it for learning

interact with it

apply its output

34 of 46

What needs / opportunities are there in education?

What can chatGPT do?

35 of 46

Needs / Opportunities

in Education

Teaching styles that fit learning styles and needs
Students learn in 1:1 settings
Culturally responsive pedagogy (CRP)
Mental health needs
Availability
Better evaluation, better than test
We have a teacher shortage
Foster creativity, curiosity and playful learning

Differentiation
Dynamic Grouping
Access to Teacher
Social Interaction for Learning
Love, trust, sense of belonging
Emotional safety and well-being
Affirmation of full identity
Sense of agency and power
Curiosity
Opportunities for mastery
Timely feedback
Personalized learning
Self-Paced learning
Bite-sized learning
Mindfulness
Brain break
Experiential learning
STEAM Curriculum

36 of 46

Jigsaw the Learning Sciences

Activity 2

37 of 46

Collaborative Learning

Reciprocal Peer Tutoring

Group Diversity

Ubiquitous Feedback

4 Theories of Learning

Jigsaw

Mutual engagement of participants in a coordinated effort to solve the problem

- Dillenbourg et al. (1996)

Diversity catalyzes benefits of working with others like sharing ideas, and discussing differences.

- Gillies (2003), Gillies & Ashman (1998)

Feedback should be of high quality and adapted to the student's needs

- Hattie (2012)

A type of cooperative learning in which students coach one another as they develop specific skills.

- Choudhury (2002), Sanders (2001)

Study Group
Switch groups
Presentation Group

38 of 46

4 Theories of Learning

Collaborative Learning

Under what conditions does collaboration positively impact learning?
How might an intelligent agent interact collaboratively with a learner to facilitate learning?

39 of 46

4 Theories of Learning

Reciprocal Peer Tutoring

Does tutoring offer greater benefits to learning than being the tutee? Does these effects differ among subpopulations of learners (i.e., low vs high performing students)?

40 of 46

4 Theories of Learning

Group Diversity

Group diversity introduces affordances and challenges to learning from group work. How can the affordances be enhanced while challenges be minimized or removed?
Does diversity offer desirable difficulties?

41 of 46

4 Theories of Learning

Ubiquitous Feedback

What are the characteristics of feedback that have the most positive impact on student learning?
In what contexts / applications has ubiquitous feedback been found to enhance learning experiences?

42 of 46

Collaborative Learning

Reciprocal Peer Tutoring

Group Diversity

Ubiquitous Feedback

4 Theories of Learning

Jigsaw

Mutual engagement of participants in a coordinated effort to solve the problem

- Dillenbourg et al. (1996)

Diversity catalyzes benefits of working with others like sharing ideas, and discussing differences.

- Gillies (2003), Gillies & Ashman (1998)

Feedback should be of high quality and adapted to the student's needs

- Hattie (2012)

A type of cooperative learning in which students coach one another as they develop specific skills.

- Choudhury (2002), Sanders (2001)

Study Group
Switch groups
Presentation Group

43 of 46

What needs / opportunities are there in education?

What can chatGPT do?

What opportunities are there for generative AI in education?

How do we connect these pipes?

44 of 46

Project Brainstorming + Mixer

Activity 3

45 of 46

chalkboard headlines

Topics of interest
Problems to solve
Opportunities to explore
Project ideas

… grab some chalk and write your idea(s) for how to, essentially, connect the pipes. You can write a headline describing your idea or any of the following things bulleted below on this slide. Then write your name by your idea. When the brainstorming time is over, you can circulate to look at others’ ideas or you can stand by your idea(s) and chat about them with people as they circulate. If you don’t have an idea you’re super excited about by the end of the brainstorming moment, feel free to circulate the room and chat with people about their ideas. If anyone sees an idea you’re interested in working on go ahead and add your name to the space around the idea. If you like, you can even add your contact information. We’ll let you guys sort out your teams for the final project on your own, but we’re hoping this is a good way to get started. Any questions before we begin?

[return to the previous slide with the guiding question, and start the brainstorming session. The session can be as long as necessary, but leave at least 10 minutes for the mixer and 10 minutes for the closing].

46 of 46

Next Class

First Final Project Check-In

Know your team members
Topic of your project and student age.
What activities will students be doing?
How does it fulfill a need / opportunity?
What role does generative AI play?
What should students learn from it?
How will you playtest your tool?
What are the key steps for your team?

Readings: 3 Readings on accessible, inclusive, and equitable design for Educational Technologies

Coding Task: Re-design your code from last week to achieve a learning goal and apply a teaching approach discussed in today’s class.