Workshop| Ian McCanless | February 7, 2024
1
Question Authoring for Trainers
Logo
Meet Your Designer
2
Logo
Ian McCanless
Senior Learning Designer
Salt Lake City, UT
3
Logo
Agenda
01
Session Introduction
02
Consider the Learning Objectives
03
Levels of Questions
04
Gamification of Learning
05
Formulating Accessible Content
06
Pulling it All Together
4
Logo
Session Guidelines
Be Present
Demonstrate
Respect
Ask All
Questions
Participate
Session Objectives
5
By the end of this session, you will be able to:
Logo
Parking Lot
6
Logo
7
Logo
Consider the Learning Objectives
8
Benjamin Samuel Bloom (February 21, 1913 – September 13, 1999) was an American educational psychologist who made contributions to the classification of educational objectives and to the theory of mastery learning. He is particularly noted for leading educational psychologists to develop the comprehensive system of describing and assessing educational outcomes in the mid-1950s. He has influenced the practices and philosophies of educators around the world from the latter part of the twentieth century.
Dr. Benjamin Bloom
The Taxonomy of Educational Objectives: The Classification of Educational Goals | Benjamin S. Bloom
Logo
Bloom’s Taxonomy
9
There are six levels of cognitive learning according to Bloom's Taxonomy. These levels can be helpful in developing learning outcomes and objectives because certain verbs are particularly appropriate at each level and not appropriate at other levels. A learner might list our funding models or recite our misrepresentation policy to demonstrate that they remember something they learned, but generating a list does not demonstrate (for example) that the learner can explain the funding model to a member or apply the misrepresentation policy to the verbiage they use with a member.
Logo
Remember
Understand
Apply
Analyze
Evaluate
Create
Remember
10
Recall facts and basic concepts
Logo
Remember
Understand
Apply
Analyze
Evaluate
Create
Understand
11
Explain ideas or concepts
Logo
Remember
Understand
Apply
Analyze
Evaluate
Create
Apply
12
Use information in new situations
Logo
Remember
Understand
Apply
Analyze
Evaluate
Create
Analyze
13
Draw connections among ideas
Logo
Remember
Understand
Apply
Analyze
Evaluate
Create
Evaluate
14
Justify a stand or decision
Logo
Remember
Understand
Apply
Analyze
Evaluate
Create
Create
15
Produce new or original work
Logo
Remember
Understand
Apply
Analyze
Evaluate
Create
Learning Objectives Example
16
Logo
17
Logo
Pause for Practice
18
Logo
Which taxonomy level is used in the learning objective below?
Explain how the Quick Text, Help Center, and File Request chat features function.
Create
Evaluate
Analyze
Apply
Understand
Remember
19
Logo
Which taxonomy level is used in the learning objective below?
Demonstrate using Salesforce to authenticate a member chat.
Create
Evaluate
Analyze
Apply
Understand
Remember
20
Logo
Which taxonomy level is used in the learning objective below?
Define the scope of support responsibilities Individual Contributors in Tier 1 and Tier 2 have for each funding model.
Create
Evaluate
Analyze
Apply
Understand
Remember
Bloom’s Taxonomy Revisited
21
Which three taxonomy levels are most commonly used in our new hire training courses?
Logo
Remember
Understand
Apply
Analyze
Evaluate
Create
Why do you think that is?
How does understanding the Bloom’s Taxonomy and its application to learning objectives impact the questions you craft in our gamified learning solution?
22
Logo
Resources
The resources to the right provide more information about Bloom’s Taxonomy.
23
Logo
Levels of Questions
24
Arthur L. Costa, EdD, is an emeritus professor of education at California State University, Sacramento. He is cofounder of the Institute for Habits of Mind and cofounder, with Bob Garmston, of the Center for Cognitive Coaching. He served as a classroom teacher, a curriculum consultant, an assistant superintendent for instruction, and as the director of educational programs for the National Aeronautics and Space Administration. He has made presentations and conducted workshops in all 50 states as well as on six of the seven continents.
Dr. Arthur Costa
Learning and Leading with Habits of Mind: 16 Essential Characteristics for Success | Arthur L. Costa & Bena Kallick
Logo
Costa’s Three Levels of Thinking and Questioning
25
Logo
Level 1
Level 2
Level 3
Focus
Gathering and recalling information
Making sense of gathered information
Applying and evaluating
information
Definition
These questions can be answered with yes, no, or specific information found in online resources, VILT presentations, eLearning courses, etc. A person can point to the information, read it, or physically see it.
These questions require learners to expand what they already know by using facts, details, or clues.
These questions require learners to reflect on their thinking and respond with a personal opinion that is supported by facts. The learner makes a value judgment or wonders about something. There is no right or wrong answer.
The House Metaphor
26
Logo
Level 1: Gathering
Level 2: Processing
Level 3: Applying
Complete
Identify
Recite
Define
List
Select
Describe
Observe
Compare
Sort
Infer
Contrast
Distinguish
Analyze
Classify
Explain
What four funding models exist across our Employer Partner benefits policies?
How would you explain the Tuition Assistance funding model to a member?
Evaluate
Judge
If/Then
Generalize
Predict
Hypothesize
Imagine
Speculate
Forecast
Provide two pros and two cons of the Tuition Reimbursement funding model.
27
Logo
Pause for Practice
28
Logo
What level question is represented below?
True or False: It is acceptable to inform a student how much money they will or may earn who have graduated from, or have completed courses at, an institution.
Level 1: Gathering
Level 2: Processing
Level 3: Applying
29
Logo
What level question is represented below?
If you were planning on starting a degree program, which funding model would you prefer, and why?
Level 1: Gathering
Level 2: Processing
Level 3: Applying
30
Logo
What level question is represented below?
A member who works for an Employer Partner that uses SSO is unable to sign on. You have confirmed that they are attempting to access their account through their HR portal. What action should you take next?
Level 1: Gathering
Level 2: Processing
Level 3: Applying
Costa’s Levels Revisited
31
Logo
Level 1: Gathering
Level 2: Processing
Level 3: Applying
Complete
Identify
Recite
Define
List
Select
Describe
Observe
Compare
Sort
Infer
Contrast
Distinguish
Analyze
Classify
Explain
Why do you think that is?
Which of Costa’s Levels do you think are least suited to a gamified learning solution?
Evaluate
Judge
If/Then
Generalize
Predict
Hypothesize
Imagine
Speculate
Forecast
Which of Costa’s Levels do you think are best suited to a gamified learning solution?
32
Logo
Resources
The resources to the right provide more information about Costa’s Levels.
33
Logo
Gamification of Learning
34
Karl M. Kapp, Ed.D., is an international speaker, scholar, writer, and expert on the convergence of learning, technology and business with a focus on game-thinking and gamification for learning. He serves as a professor of Instructional Technology at Bloomsburg University in Bloomsburg, PA. where he teaches several graduate courses and serves as the Director of the university’s Institute for Interactive Technologies. The institute works with businesses, non-profits and other organizations to help them create interactive and meaningful instruction.
Karl M. Kapp
The Gamification of Learning and Instruction: Game-based Methods and Strategies for Training and Education | Karl M. Kapp
Logo
35
Logo
Defining “Game”
36
A game is a system in which players engage in an abstract challenge, defined by rules, interactivity, and feedback, that results in a quantifiable outcome often eliciting an emotional reaction.
Logo
Defining “Game”
37
A game is a system in which players engage in an abstract challenge, defined by rules, interactivity, and feedback, that results in a quantifiable outcome often eliciting an emotional reaction.
Logo
A set of interconnected elements occur within the “space” of the game. A score is related to behaviors and activities that, in turn, are related to a strategy or movement of pieces. The system aspect is the idea that each part of a game impacts and is integrated with other parts of the game. Scores are linked to actions, and actions are limited by rules.
Defining “Game”
38
A game is a system in which players engage in an abstract challenge, defined by rules, interactivity, and feedback, that results in a quantifiable outcome often eliciting an emotional reaction.
Logo
Games involve a person interacting with game content or other players. This happens in first-person shooters, board games, and games like Tetris. The person playing the game is the player, or learner. The act of playing the game often results in learning, and learners are our target audience for gamification of instruction.
Defining “Game”
39
A game is a system in which players engage in an abstract challenge, defined by rules, interactivity, and feedback, that results in a quantifiable outcome often eliciting an emotional reaction.
Logo
Games typically involve an abstraction of reality and typically take place in a narrowly defined “game space.” This means that a game contains elements of a realistic situation or the essence of the situation but is not an exact replica. This is true of the game Monopoly, which mimics some of the essence of real estate transactions and business dealings, but is not an accurate portrayal of those transactions.
Defining “Game”
40
A game is a system in which players engage in an abstract challenge, defined by rules, interactivity, and feedback, that results in a quantifiable outcome often eliciting an emotional reaction.
Logo
Games challenge learners to achieve goals and outcomes that are not simple or straightforward. For example, even a simple game like Tic Tac Toe is a challenge when you play against another person who has equal knowledge of the game. A game becomes boring when the challenge no longer exists. But even the challenge involved with the card game of Solitaire provides enough challenge that the player continues to try to achieve the winning state within the game.
Defining “Game”
41
A game is a system in which players engage in an abstract challenge, defined by rules, interactivity, and feedback, that results in a quantifiable outcome often eliciting an emotional reaction.
Logo
The rules of the game define the game. They are the structure that allows the artificial construct to occur. They define the sequence of play, the winning state, and what is “fair” and what is “not fair” within the confines of the game environment.
Defining “Game”
42
A game is a system in which players engage in an abstract challenge, defined by rules, interactivity, and feedback, that results in a quantifiable outcome often eliciting an emotional reaction.
Logo
Games involve interaction. Learners interact with one another, with the game system, and with the content presented during the game. Interactivity is a large part of games.
Defining “Game”
43
A game is a system in which players engage in an abstract challenge, defined by rules, interactivity, and feedback, that results in a quantifiable outcome often eliciting an emotional reaction.
Logo
A hallmark of games is the feedback they provide learners. Feedback within a game is typically instant, direct, and clear. Learners are able to take in the feedback and attempt corrections or changes based on both the positive feedback they receive as well as the negative feedback.
Defining “Game”
44
A game is a system in which players engage in an abstract challenge, defined by rules, interactivity, and feedback, that results in a quantifiable outcome often eliciting an emotional reaction.
Logo
Games are designed so that the winning state is concrete. The result of a well-designed game is that the learner clearly knows when he or she has won or lost. There is no ambiguity. There is a score, level, or winning state (checkmate) that defines a clear outcome. This is one element that distinguishes games from a state of “play,” which has no defined end state or quantifiable outcome. This is also one of the traits that make games ideal for instructional design.
Defining “Game”
45
A game is a system in which players engage in an abstract challenge, defined by rules, interactivity, and feedback, that results in a quantifiable outcome often eliciting an emotional reaction.
Logo
Games typically involve emotion. From the “thrill of victory” to the “agony of defeat,” a wide range of emotions enter into games. The feeling of completing a game in many cases is as exhilarating as the actual playing of the game. But at times frustration, anger, and sadness can be part of the game as well. Games, more than most human interactions, evoke strong emotions on many levels.
Defining “Game” Revisited
46
A game is a system in which players engage in an abstract challenge, defined by rules, interactivity, and feedback, that results in a quantifiable outcome often eliciting an emotional reaction.
Logo
What are two key learnings you can take from Kapp’s definition of game?
How can you apply your learning to building games in our gamified learning solution?
47
Logo
Resources
The resources to the right provide more information about Kapp’s theories of gamification in learning.
48
Logo
Formulating Accessible Content
Scored Questions
49
Logo
50
Logo
Quiz Questions
Example 1
51
Logo
Question length exceeds 120 character limit. How could you address this?
Answer length exceeds 75 character limit. How could you address this?
Example 2
52
Logo
Question length exceeds 120 character limit. How could you address this?
53
Logo
True or False Questions
54
Logo
Type Answer Questions
55
Logo
Slider Questions
56
Logo
Puzzle Questions
57
Logo
Quiz + Audio Questions
Non-Scored Questions
58
Logo
Maintaining Multiple Versions
59
Logo
Extending the Question Time Limit
60
Logo
61
Logo
Pulling it all together
62
Logo
Activity: Practice Crafting Kahoot Questions
Purpose: The purpose of this activity is to provide participants with an opportunity to practice crafting questions for Kahoot.
Directions:
63
Logo
Debrief
64
Logo
Closing Summary
Session Objectives
65
You should now be able to:
Logo
Parking Lot
66
Logo
67
Logo
Thank you!