1 of 23

Supporting Peer Engagement Utilizing the Peer Instruction Xblock

Derek White, Senior Manager, UBC

Pan Luo, Developer, UBC

OpenEdX 2016, Stanford University

2 of 23

Agenda

  • The Peer Instruction process
  • Underlying educational theory
  • Live demo
  • The key features of the Peer Instruction tool
  • Development process
  • General experiences and feedback from tool users
  • Methodology being used to evaluate the effectiveness of the tool
  • Roadmap plans

3 of 23

How it works

Step 2

Read some other students’ answers and explanations

Student #1:

Option B

“Reasons for option B”

Student #2:

Option C

“Reasons for option C”

Student #3:

Option A

“Reasons for option A”

Step 4

See the correct answer/explanation & class breakdown

Correct Answer:

Option A

“Correct reasons for

option B”

Initial Final

Answers Answers

A B C A B C

Step 1

Choose an answer option & explain why it is correct

What is your answer

to this question?

Option A

Option B

Option C

Explain why:

Here is why I think

this option is best.

Step 3

Reconsider your original answer & optionally revise it

What is your answer

to this question?

Option A

Option B

Option C

Explain why:

Here is why I think

this option is better.

4 of 23

Underlying Theory and Practice

  • Based on Eric Mazur’s Peer Instruction methodology
  • Interactive engagement strategy to enhance classroom teaching
  • Students asked questions designed to expose common difficulties in their understanding material
  • Questions interspersed with small group discussion
  • Well established efficacy of process
  • Works because it engages higher-order cognitive skills : articulation, evaluation, synthesis etc.

Smith, M. K., Wood, W. B., Adams, W. K., Wieman, C., Knight, J. K., Guild, N., & Su, T. T. (2009). Why Peer Discussion Improves Student Performance on In-Class Concept Questions. Science, 323(5910), 122-124. doi:10.1126/science.1165919

http://mazur.harvard.edu/research/detailspage.php?rowid=8

5 of 23

6 of 23

Why Peer Instruction?

UBC Context: Carl Wieman Science Education Initiative

The goal of the Carl Wieman Science Education Initiative is to achieve highly effective, evidence-based science education for all post-secondary students by applying the latest advances in pedagogical and organizational excellence.

→ 170 transformed courses

The MOOC challenge:

  • Difficulty of supporting engagement at scale with limited instructor presence
  • Difficulty of supporting synchronous interaction

7 of 23

Why Peer Instruction?

edX Global Forum 2014 - Instructional Design and Pedagogy Group

“Can we take an example pedagogy that we know works *really* well in the f-2-f environment and recreate it in a fully online environment?”

→ Online Peer Instruction utilizing student answer

rationales as proxy for student discussion.

8 of 23

Live Demo

https://goo.gl/bBmZS2

Be sure to use HTTPS.

9 of 23

Key Features

  • Asynchronous two stage MCQ interaction
  • Multiple algorithms for distributing student answers
  • Ability to set the number of student answers displayed
  • Ability to seed “student” answers for each option
  • Grading - currently only completion grade
  • Images and HTML input / display
  • Graphical presentation of activity statistics
  • Internationalization support
  • Event tracking
  • Available on edx.org and edge.edx.org as XBlock

10 of 23

11 of 23

12 of 23

Events

13 of 23

{

...

"username": "USERNAME", -- common field

"event_source": "server/browser", -- common field

"name": "ubc.peer_instruction.original_submitted",

"context": {

"user_id": ####, -- common field

"org_id": "", -- common field

"course_id": "", -- common field

"path": "", -- common field,

"module": {

"display_name": "Peer Instruction Question 1",

}

},

"referer": "", -- common field

"event": {

"answer": "1", -- An integer

"rationale": "This is the rationale", -- string field

"truncated": False, -- boolean field,

},

"event_type": "ubc.peer_instruction.original_submitted",

"page": null

}

14 of 23

Development Process

  • Initial Proposal and Feedback
  • Options Exploration and Challenges
  • Coding
  • Code Review
  • Merging and Deployment
  • Documentation

15 of 23

Experiences and Feedbacks

16 of 23

Experiences and Feedbacks

17 of 23

Main reasons answers were helpful

Legitimized Alternatives:

"[Answers] provided other options that you might have overlooked", "it made me think would [another answer] work?"

Provided Guidance:

“[Answers] guided you in a direction", "now I see what I'm supposed to look at", "it helped me target my attention to specific clues".

Second Thoughts:

"It made me stop and be more careful, to question myself"

Increased Confidence:

“[Other answers] helped me feel like I was assured with my choice", "confirmed to me my answer was correct"

18 of 23

Reasons students gave for liking the exercise

"Other students' insight could provide a different perspective"

"When we're not sure about our choice, we can see other answers" these can help you "get to the correct way to approach the question"

There’s value when you can "see the reasoning in the [other] answers instead of just the correct answer . [It] gives me an idea how to formulate my answers"

"Just to have other student opinions is good" in any assignment.

19 of 23

Evaluation Process

Evaluation of two PI tools: UBCPI and Dalite

  • Piotr Mitros - edX
  • Nina Huntemann - edX
  • Melanie McFarlane
  • Colin Fredericks - Harvard
  • Yigal Rosen - Harvard
  • Saif Rayyan - MIT
  • Ido Roll - UBC
  • Pan Luo - UBC
  • Derek White - UBC

20 of 23

Evaluation Process

  • Measure impact on engagement
    • Time on pages, how often switch answer, positive switching, negative switching, quality of answers
    • Impact on persistence in course, time on task...
  • Measure impact on learning
    • Correlate to performance in course, grades...
    • Perceptions of value and competence (survey)
  • Design
    • What questions work better?
    • How does this interact with learner characteristics?

21 of 23

Development Roadmap

  • Grade for correct answer
  • Additional answer selection algorithms
  • Ability for student to ask for additional student answers to view before submitting final answer
  • Flagging inappropriate answers
  • Enhance event emitting to include answers shown to students
  • Explore option of synchronous discussion groups

22 of 23

Development Team

Derek White

Pan Luo

Letitia Englund

Richard Tape

Bill Pichard

William Ono

John Hsu

Michael Tang

Special thanks to Open Edx team for the help, code review and contribution!

23 of 23

We’d like to hear from you

  • Interested in using the tool?
  • Want to participate in evaluation project?
  • Want to contribute or suggest enhancements?

Derek White: derek.white@ubc.ca

Pan Luo: pan.luo@ubc.ca