Motivating Learning with Digital Badges

Katerina Schenke and Cathy Tran

Abstract: This chapter on motivating learning presents one of four categories of principles uncovered by the Design Principles Documentation Project on recognizing, assessing, motivating, and studying learning in digital badge systems. The Design Principles Documentation Project has followed the development of thirty digital badging projects as they adjusted their intended practices from the initial designs outlined in their proposals to accommodate the realities of their audience and the badging platform. The DPD project aimed to capture the knowledge that emerged in this shift from intended and formalized practices concerning badge system development and transform it into resources that could be used to design or study future badge systems. This chapter also describes how the broader ecosystem, recognition practices, and assessment practices impact motivation, and the tensions between competing conceptualizations of motivation as they relate to digital badges.

Motivating learning with digital badges focuses on how the design of badge systems affects learners’ participation within the badge ecosystem. Researchers and developers appear divided on the role of digital badges in motivating learners. Skeptics of badges “worry that students will focus on accumulating badges rather than making connections with the ideas and material associated with the badges – the same way that students too often focus on grades in a class rather than the material in the class, or the points in an educational game rather than the ideas in the game” (Resnick, 2012). Badge evangelists find promise in having a new way to assess learners apart from the “current multiple-choice form of testing doesn’t measure all that is being learned and de-motivates true curiosity” (Davidson, 2012).

A crucial step in reconciling these two views on badges is the consideration of the ways that their design and function differentially affect learner motivation. Learner motivation in badges projects can mean a variety of things such as motivation to continue in the project (for example, learners attending an after school program), motivation to use the badge system implemented in the project (learners valuing the badges they receive), and motivation to learn (encouraging learners to develop new skills and knowledge forms). In addition, the context under which those motivation practices operate, such as the corresponding assessment and recognition practices within the badge ecosystem, also matter (Goodenow, 1992; Hickey, 1997, 2003). We derived a set of motivation principles from practices that badge developers intended to implement as well as those that were enacted in programs and analyzed how those principles may impact learner motivation. In order to do this, we drew on well-known psychological theories of motivation and consider the effect that context has on the implications that can be drawn using current theories. These principles aren't meant to be prescriptivethe process of designing a learning environment is not an exact science. Additionally, the process of categorizing and labeling the motivation principles is one way to conceptualize the hypothesized effects of using badges on learner motivation. Our goal is to provide perspectives and resources for educators and badge system developers to consider as they design badge ecosystems and figure out which badge design elements work best in within their context to motivate learners.

Assembling Design Principles for Motivating Learning

To identify the motivational practices in badge ecosystems, we analyzed the 30 projects funded by the MacArthur Foundation to develop digital badge systems. Our team asked each project’s staff, through phone and in-person interviews, about design decisions they made to motivate learners to generate a list of intended practices. In addition, based on their grant proposals, we flagged other practices that may have unintended motivational consequences based on motivation research, and that, too, constituted the list of practices.

Below, we present a series of practices (what projects award badges for and how these badges are awarded) categorized under specific principles (clusters of practices related to a similar idea), which were then categorized under general principles (a collection of specific principles that draw on the same motivation construct) for motivating learning with digital badges. For example, specific principles of “provide privileges” are: “tangible prizes unrelated to the subject domain” and “peer mentorship positions”. The enactment of the specific principle is done differently by each project.

In order to come up with our list of principles, we first identified the intended practices that each project had planned to implement, categorized these practices into design specific principles by dynamically sorting and re-sorting the practices into different groups of specific principles. We subsequently grouped specific principles into more general principles that encompass an overarching theme such as “recognizing identities” or “providing privileges” from which we could draw on extant literature and motivation research.

It is important to note that practices were grouped into specific principles and principles based on our reading of motivation in the research literature. We draw on prominent theories (e.g. Expectancy-value theory; Wigfield & Eccles, 2000) and meta theories (Self Determination Theory; Ryan & Deci, 2000), of motivation as well as on different traditions (e.g. sociocultural approaches; Hickey, 2003) in order to derive the complete set of general principles. Drawing from different theories allows us to gain a more comprehensive view on how motivation may play out in badge ecosystems since deep and continuous learning does not depend only on the level of motivation (i.e., amount of engagement) but also on its type (i.e., reason for engagement). Principles and specific principles were based on both the actual practices of the thirty projects, as well as areas in motivation that we felt were important to consider based on current discussions in the field. The eleven general principles that emerged were vetted for feedback and revised both by representatives from the badge projects as well as attendees at several conferences and workshops this past year.

An important note is that practices were concrete activities that were extracted from resources and interviews provided by the badge projects, whereas the organization of these practices into principles is more interpretative; these principles are one way to categorize the practices.

Principles for Motivating Learning with Digital Badges

The following descriptions of each motivation principle draw on selected examples from the DML competition.

Recognize Identities

The development of identity in educational contexts plays a vital role in the choices that learners make. The concept of identity can take a perspective that is focused on the individual and the collective. As such, we present our specific principles considering the two perspectives. Badges can recognize a learner’s role (1) within the badging system and the real-world ecosystems it may represent, such as recognizing their specialization in journalism, engineering, or peer mentoring and (2) a learner’s collective identity by being incorporated into badge projects that themselves target specific groups. Ten projects have developed badges that focus on identity of the individual whereas only four have focused on recognizing roles within the collective (such is the case in Girl Scouts and EarthWorks where the projects themselves target specific groups such as girls and students of Native American background). Tapping into learner’s identity is one way that badge projects can motivate learning and participation.

Example. Supporter To Reporter (S2R) Medals intends that badges will motivate learners by recognizing key roles played in a community. S2R Medals recognize the three main S2R roles - journalist, live reporter and peer mentor. Students can pursue their strengths and interest in one of these roles or develop skills in an unfamiliar or new role. For example, a student may achieve a Gold in journalism while remaining at Bronze in coaching. Alternatively, they may aim for Silver in each set.

Engage with the Community

Some learners are able to earn badges for their involvement in their communities both at the (1) physical and (2) digital level signifying two specific principles under this general principle. For example, earners can be awarded badges by engaging with members of their immediate community while other projects have designed badges that value interactions with members of the digital community via blog posts and discussion forums. Strengthening relationships with the community can help learners feel more connected and therefore persist within that learning environment (for an example in undergraduate education see Summers, Svinicki, Gorin, & Sullivan, 2002). Nine projects in total have designed badges for engaging with the community.

Example. Planet Stewards awards learners badges for engaging with their online community and acting as science communicator and collaborator.

Display Badges to the Public

Badges are web-enable tokens of accomplished that contain specific claims and detailed evidence about learning. Allowing for the capability to display badges to the public is a feature that badge systems may consider enabling in order for others to recognize that learning. Some projects give learners the option of displaying badges themselves, while other projects automatically display badges for learners suggesting two specific principles of the general principle. Who chooses to display badges has implications for learner motivation and it is thought that giving badge earners a choice of whether or not to display the badges they’ve earned is related to motivation (for readings on choice see Malone & Lepper, 1987). Few projects (nine in total) have practices aligned with this general principle.

Example. MOUSE Wins! automatically displays learner's badges on their website so that users of the website can see which badges their participants have earned.

Build Outside Value of Badges

Some projects integrate practices to give badges value outside of the badge system. These include: (1) having badges count as academic or course credit, (2) showing badges to outside agencies thereby giving badge opportunities outside of the project, and/or (3) documenting the link between the badges and real life applications of knowledge, causing badge earners to value the activities that lead to the badge more (for review see Wigfield & Eccles, 2000). Nine projects in total have made an effort to link badges and their badge ecosystem to outside forms of knowledge.

Example. Earners of 4H badges have the opportunity to earn internships with partner institutions such as NASA because NASA values the badges that 4H participants have earned.

Set Goals

Badges allow for learners to set goals and visualize the previous goals that they’ve accomplished through the accumulation of badges. Specific principles include: (1) displaying the trajectory of badges earned for the learner, (2) allowing the learner to determine this trajectory, and (3) allowing for user-created badges in which learners decide for what and how a badge is awarded (4) allow the provider to determine the learning trajectory and (5) allow the provider to give a personalized recommendation of subsequent goals to the learner. Encouraging learners to set goals and visualize those goals is an important strategy for self-regulated learning in which learners plan and  monitor their learning (Zimmerman, 2000). This specific principle is similar to the “Provide routes or pathways” specific principle under the “Use badges to map learning trajectories” general principle in the recognition strand and a total of eleven projects have implemented badges around setting goals.

Example. BuzzMath provides learners with clear learning pathways of the badges they have earned. For example, badges are awarded in a progressive manner and displayed in a clear manner to guide learners to the next mathematics activities.

Promote Collaboration

Though several projects allow for collaborative efforts, some make a concerted effort to encourage this through awarding group badges for (1) group accomplishments and (2) personal badges for having a role in a group collaboration. Only three projects in total have practices around recognizing collaboration. Research has shown that individuals can learn more in a group than individually, therefore designing badges to encourage collaboration can have an effect on motivation (Yager, Johnson, & Johnson, 1985).

Example. Robotics and STEM Badges using NASA Content awards badges to groups of learners such that each individual in the group receives the badge for a group accomplishment.

Stimulate Competition

Some badge projects have created competition by (1) making badges scarce and therefore more difficult to earn, or (2) use a point system to award badges to earners. The use of competition in badge systems has not been thoroughly investigated within the context of badges. In classroom research, the outcomes related to competition are mixed: when competition is paired with collaboration, we would expect to see beneficial outcomes for learners in those environments allowing all learners to participate in competition (see Hickey, 2003).

Example. Supporter to Reporter Medals uses limited access to special opportunities as a motivator. The program provides prestigious live reporting opportunities for students who have completed training and performed well. This is reflective of the limited number of live reporting opportunities that are available to students and is a result of a real world constraint. Badges give program coordinators a clear system for allocating these opportunities.

Recognize Different Outcomes

The type of learning that projects recognize as worth earning a badge for has implications for motivation. For example, projects can award badges for (1) the effort that learners make or (2) for the learner’s performance. Learners who think they can grow their intelligence and are rewarded for their effort or improvement instead of their performance are associated with being more persistent on tasks and more orientated towards learning and improving (Dweck & Leggett, 1988; Henderson & Dweck, 1990). Therefore, how learners interpret the feedback they receive, whether for their improvement or for their performance, has implications for motivation. Two projects have designed badges that are awarded based on performance and none have designed badges around learner improvement. Even though we did not see the enactment of the improvement specific principle, motivation research suggests that recognizing improvement can be beneficial for learners.

Example. MOUSE Wins! has designed a badge that is awarded depending on the number of blog posts learners write. This is indicative of how learners perform. None of the projects in our study have designed badges based on the amount of effort learners put in, but one can imagine awarding learners badges based on improvement or effort.

Utilize Different Types of Assessments

Projects utilize different types of assessments for learning such as (1) computer, (2) peer, (3) expert, or (4) self assessment. While computer assessment may benefit from being more efficient and free from social judgment, peer or expert assessment may be more meaningful and therefore increase the quality of work put into earning the badge. This general principle is an example of assessment having implications for motivation. Interestingly, most projects who utilized different types of assessment used peers (seven projects). Learners who are asked to assess themselves on a task may reflect more deeply on their own learning than learners who are assessed using a computer.

Example: Sweet Water AQUAPONS intends to implement a variety of assessments. The program structure includes staff approving requests for badges and peer assessment. Students are also required to self-assess their accomplishments, as they upload evidence to apply for badges. Badges are also earned through formal scoring of written assessments, photo and video projects, and in-person demonstrations of proficiency.

Provide Privileges

Learners are awarded a variety of privileges in response to earning a badge such as (1) prizes, (2) the opportunity to take part in new activities, (3) access to internships, and (4) becoming a peer mentor. The prize or privilege that is awarded to learners as a result of earning a badge has implications for whether or not earners choose to earn the badge or not. Giving learners access to internships or to new activities within the badge project are thought to be more valued by learners than simply awarding physical prizes to learners (see Malone & Lepper, 1987). Additionally if becoming a peer mentor is of value to the learner, designing a badge around peer mentorship can be very effect for learner motivation. A total of fourteen projects have designed badges under the principle providing privileges.

Example. Sweet Water AQUAPONS aims to motivate students to learn about aquaponics by helping ensure the credential will have value to external employers. The program reaches out to partner organizations in education and business and publicizing the portability of students' acquired skills to related disciplines.

Constraints and Tensions when Motivating Learning with Badges

Motivation plays an integral part in the discussion around implementing badge systems. Whether motivation is purposefully integrated into the design of a badge or occurs as a by product of designing a badge for another purpose, understanding the implications of using a particular badge on learner motivation is vital for the research and interest on badges to continue. Badges can be designed to promote motivation that is adaptive in learners. In order to understand these implications more thoroughly, we discuss tensions that occur within the category of motivation, as well as between recognition and assessment.

Tensions Within this Category

It is difficult to make assumptions about the outcomes of the motivation principles that we have derived without understanding the context in which these principles are used. For the principle of “Providing privileges,” for example, there is not a clear yes or no answer to whether that principle should be enacted in badge systems and the exact effect that the practice will have on learner outcomes. Using established research from the motivation literature can provide badge issuers with ideas of the effects of the design of a particular badge system on learner motivation, however, it is important to note that this literature was developed within very specific contexts such as lab-based studies or in the context of classrooms. Incentives in themselves do not necessarily have positive or negative motivational influences. Rather, it is studying them in the contexts in which they operate that provide valuable insight about their impact.

There are many ways to provide privileges to badge earners. The privilege of allowing learners to be peer mentors may be motivating for learners because students feel in charge and are eager to help their tutees improve and as a byproduct put forth more effort and learn more themselves (Chase, Chin, Oppezzo, & Schwartz, 2009). Providing prizes unrelated to the subject domain can undermine motivation and learning for complex activities such as problem solving but have no effect on rote memorization (Dweck & Leggett, 1988). Giving learners access to new activities may be appealing to some students but others may be motivated to stay with easier tasks rather than choose harder tasks (Ames & Archer, 1988). As such, badges for privileges will be differentially pursued depending on individual differences and the context at hand.

Tensions Between Categories of Practice

Though this project focuses on principles for motivating learning, it is crucial to acknowledge the interactivity of our strands and how assessment and recognition principles, together, influence learner motivation. This is illustrated in the program Supporter to Reporter (S2R) Medals. S2R Medals gives youths a glimpse at what it is like to be a journalist in the sports reporting world. More than 2,000 individuals have developed their reporting skills through S2R and reported at more than 1,000 events including the 2012 Olympic Games. In this program, badges are intended to guide students along the path from novice to mentor, enabling advanced students to become a source of peer assessment for newer students.

In considering how recognition and assessment practices work together to influence learner motivation, what learning is recognized has implications for how that learning is assessed and how that learning can be motivating. For example in S2R Medals, recognition practices include having badges credentialled by the community by allowing experts to issue badges, having those badges be valued by an external community, and using badges as a means of external communication of learning. Participants of the S2R Medals program are recognized by the journalism and sports reporting community and consequences of earning particular badges give learners the privilege to report at sports events that are meaningful to the learner thereby influencing the learner’s motivation by increasing their value for the task. Assessment practices used by S2R Medals include leveled badges from bronze to gold that are indirectly aligned to standards that teachers are using in school and are awarded by computer scoring systems, peers, and experts (different types of assessment). These various assessment practices have implications for motivating students. For example, using leveled badges allows learners to set goals for themselves and being assessed by peers has different implications for motivation than being assessed by a computer scoring system.

Considering how motivation works in the context of recognition and assessment principles is vital to understanding the implications of these practices on learner motivation. The current literature around motivation was not developed with this context in mind, and before conclusions can be drawn about how to motivate learners in these environments, a complete understanding of the entire context (not just recognition and assessment, but also the context of the project such as after school or digital) is necessary.

General Design Principle

Description

Specific Design Principles

Recognize identities

Motivate achievement by creating badges named for the careers or roles that learners wish to fill or by acknowledging the groups that learners are a part of.

Roles within a System (Youth Filmmaker, Who Built America, Design Exchange , Design for America, Pixar, EarthWorks, MOUSE Wins, Girl Scouts, NASA Robotics, S2R Medals)

Targets a Specific Group (4H, EarthWorks, Girl Scouts, Manufacturing Badges)

Engage with the community

Badges awarded for interacting with communities can increase motivation through building social relationships and shared goals.

Involvement in Local Community (Pixar, EarthWorks, Manufacturing Badges, S2R Medals)

Involvement in Digital Community (Global Pathways, Design for America, Intel, MOUSE Wins, Planet Stewards)

Display badges to the public

Showing off learners' achievements to the public as part of regular operation could motivate them to achieve.

Learners can choose to share their badges with others (Global Pathways, Girl Scouts, Nature Badges, SA&FS)

Learners do not choose which badges to share with others (Youth Filmmaker, MOUSE Wins)

Build outside value of badges

Through partnerships or simply recognizing necessary skills, ensuring that stakeholders in public value a set of badges will motivate learners to earn them.

Badges as Academic Credit (Global Pathways, 4H)

Evidence for Outside Opportunities (News Hour, Manufacturing Badges)

Real-life Application of Knowledge (CSSN, YALSA, S2R Medals, SA&FS, AQUAPONS)

Set goals

Badges allow for learners to set goals and visualize the previous goals that they’ve accomplished.

User-created Badges (Design Exchange , Design for America, Pixar, PASA, SA&FS)

Display of Goal Trajectory (BuzzMath, CSSN, Pixar, PASA, NASA Robotics)

User-determined Learning Trajectory (LevelUp)

Recognize collaboration

Specifically recognizing collaborative achievements can help motivate students, because they see the tasks differently.

Badge for Group Accomplishment (LevelUp)

Personal Badge that is Earned Through Collaboration (News Hour, Digital On-Ramps)

Stimulate competition

Scarcity, point systems, and leaderboards help create competition between learners, which can motivate some to try harder.

Use of Points System (Youth Filmmaker, Who Built America)

Scarcity of Badges (LevelUp, S2R Medals)

Recognize different outcomes

Recognizing both performance-based and improvement-based achievements can have complex effects on learner motivation.

Performance-based (Intel, MOUSE Wins)

Improvement-based

Utilize different types of assessment

Encountering different assessment methods has complex consequences for motivation.

Peer (Youth Filmmaker, Who Built America, Design for America, EarthWorks, MOUSE Wins, PASA, SA&FS)

Expert (Digital On-Ramps, NASA Robotics)

Self (SA&FS, AQUAPONS)

Computer (CSSN)

Provide privileges

Encourage earning badges by granting privileges to those who do.

Prizes (Who Built America, NASA Robotics)

Peer Mentorship (Who Built America, Pixar, Nature Badges, S2R Medals)

Peer Mentorship (LevelUp, Who Built America, News Hour, BuzzMath, Design Exchange, Pixar, Nature Badges, S2R Medals)

New Activities (Youth Filmmaker, News Hour, BuzzMath, Design Exchange, Planet Stewards)

Internships (PASA, S2R Medals, SA&FS)

Table 1. Summary of design principles for motivating learning in digital badge systems

References

Ames, C., & Archer, J. (1988). Achievement goals in the classroom: Students' learning strategies and motivation processes. Journal of Educational Psychology, 80(3), 260.

Chase, C. C., Chin, D. B., Oppezzo, M. A., & Schwartz, D. L. (2009). Teachable agents and the protégé effect: Increasing the effort towards learning. Journal of Science Education and Technology, 18(4), 334-352.

Davidson, C. (2012, February 21). Can badging be the zipcar of testing and assessment? [Web log post]. Retrieved from http://dmlcentral.net/blog/cathy-davidson/can-badging-be-zipcar-testing-and-assessment

Dweck, C. S., & Leggett, E. L. (1988). A social-cognitive approach to motivation and personality. Psychological Review, 95(2), 256-273.

Goodenow, C. (1992). Strengthening the links between educational psychology and the study of   social contexts. Educational Psychologist, 27,177–196.

Henderson, V. L., & Dweck, C. S. (1990). Motivation and achievement. In S. S. Feldman & G. R. Elliott (Eds.), At the threshold: The developing adolescent (pp. 308-329). Cambridge, MA: Harvard University Press.

Hickey, D. T. (2003). Engaged participation versus marginal nonparticipation: A stridently

sociocultural approach to achievement motivation. The Elementary School Journal, 401-429.

Hickey, D. T. (1997). Motivation and contemporary socio-constructivist instructional perspectives. Educational Psychologist, 32(3), 175-193.

Malone, T. W., & Lepper, M. R. (1987). Making learning fun: A taxonomy of intrinsic motivations for learning. Aptitude, Learning, and Instruction, 3, 223-253.

Orvis, K. A., Horn, D. B., & Belanich, J. (2008). The roles of task difficulty and prior videogame experience on performance and motivation in instructional videogames. Computers in Human behavior, 24(5), 2415-2433.

Resnick, M. (2012, February 27). Still a badge skeptic [Web log post]. Retrieved from   http://hastac.org/blogs/mres/2012/02/27/still-badge-skeptic

Summers, J. J., Svinicki, M. D., Gorin, J. S., & Sullivan, T. (2002). Student feelings of connection to the campus and openness to diversity and challenge at a large research university: Evidence of progress? Innovative Higher Education, 27, 53-64.

Wigfield, A., & Eccles, J. S. (2000). Expectancy–value theory of achievement motivation. Contemporary Educational Psychology, 25(1), 68-81.

Yager, S., Johnson, D. W., & Johnson, R. T. (1985). Oral discussion, group-to-individual transfer, and achievement in cooperative learning groups. Journal of Educational Psychology, 77(1), 60.

Zimmerman, B. J. (1990). Self-regulated learning and academic achievement: An overview. Educational Psychologist, 25(1), 3-17.