Your Source for Learning
Technology, Strategy, and News
ARTICLES       RSS feed RSS feed


email print

Report Card: Gamification in Learning (What Works?)

by Carol Leaman

June 23, 2014


by Carol Leaman

June 23, 2014

“By adding gamification to your learning, then refining it to meet learner preferences, you will more fully engage your employees to deliver the results you need: better knowledge retention, improved job performance, and a workforce aligned with your vision.”

Gamification applied to learning is receiving a lot of attention these days, and corporate learning teams are being tasked with identifying how best to incorporate it into their learning strategies.

At its simplest, gamification is the process of making non-game activities more fun and engaging. Karl Kapp, a respected expert on the convergence of learning and technology, defines gamification as “a careful and considered application of game thinking to solving problems and encouraging learning, using all the elements of games that are appropriate.”

While there’s a great deal of information available on how to add gaming elements to learning, there hasn’t been much intelligence published about how people react, act, or perform within gamified learning environments, until now.

At Axonify, we believe we’ve accumulated the largest body of data related to which game mechanics actually work in the real world of corporate learning. Every day, hundreds of thousands of employees from world-class organizations like Walmart, Toys R Us, and J&J interact with our gamification elements, resulting in more than 50 million discrete interactions per year! From these interactions, we are able to extract important information about gamification and its impact on learning.

What we’ve discovered is pretty amazing.

Lesson one: Game popularity is a formula

The Axonify platform embeds learning in short games that range from arcade-style play to word puzzles to brain games. In developing, testing, and releasing hundreds of games to learners, we’ve concluded that it’s not the style of game that makes a game popular, but rather the presence of three key components (Figure 1).

  • Multi-level action: Learners must be able to progress through levels of difficulty, which gives them a continuing sense of achievement.
  • Challenging: Games must continually challenge the learners, encouraging them to stretch beyond their comfort zones. Sometimes this will result in success, and sometimes it will result in failure. But with their comfort level from recreational gaming, most of today’s learners take failure in stride and, within a game environment, find it motivational.
  • Fun: learning games must be fun, which helps keep learners excited, motivated, and engaged in the learning process.

Figure 1:
Game popularity = multi-level + challenge + fun 

When you combine multi-level action with challenges and fun, participation goes through the roof. Pep Boys has been using Axonify for several years to help reduce retail shrink by a whopping 55 percent, a result almost unheard of in retail. Bryan Hoppe, VP Operations, says, “The most surprising benefit of Axonify was how well it was received and continues to be received by our employees. We get well over 95 percent participation in our Axonify learning, and the game approach is a big part of why employees are sticking with it.”

Lesson two: Game-play variety is critical

Offering a large selection of games and game types is essential to driving participation in learning.

A key reason is that learners become partial to specific types of games and will participate more enthusiastically if the games meet their preference. For instance, a Millennial retail associate may prefer “Angry Birds”-style games, while a Baby Boomer may be more drawn to “Color Fill,” a game that combines luck and skill.

One of the things we’ve found is that of the total number of learning games played on our platform, there’s no clear winner in terms of user preference. Learners are creating their own variety by choosing different games to keep the experience fresh and interesting. This helps avoid the disconnect that could occur if learners master a particular game or get bored.

Lesson three: Leaderboards drive competition and community

It’s important to have both personal and team leaderboards. In fact, nearly 100 percent of Axonify users check their leaderboard weekly, and 37 percent check every single day! Clearly they care about how they and their teams stack up against the competition (Figure 2). Team leaderboards drive social connectedness, creating a strong sense of community.

Figure 2:
Team leaderboards drive competitiveness and social connectedness, creating a strong sense of community

Capital Blue Cross is a US health-insurance company with over 2,000 employees in Pennsylvania. They were looking for a solution that would allow them to centralize training, reinforce classroom-based instruction for improved effectiveness, plus easily and swiftly communicate procedure changes.

What surprised them was how engaged their employees would become in their own learning. According to Mike Keeler, their VP of operations, “It’s the ongoing friendly competition that has our people looking forward to their daily training session: that, and the fact that they can immediately see how they are stacking up against their colleagues. Our employees check their leaderboards 64 percent of the time that they log on for their training sessions. I’m sure that’s a major contributor to the results we’re seeing, which include knowledge lifts of up to about 85 percent on most of our critical topics. That’s huge for our business, where we have complicated procedures, and growing complexity driven by healthcare reform.”

Lesson four: Tangible rewards are a key to participation

While intangible rewards such as recognition, interesting challenges, and badges motivate many people to learn, we’ve found that by adding tangible rewards employers can dramatically drive participation and enhance the overall learning experience. These rewards vary from company to company, but can include everything from gift cards to company swag to vacation days.

Clients that take advantage of the Axonify-reward auction feature with tangible prizes achieve at least 15 percent more participation than those who don’t.

We’ve seen the most dramatic impact of adding rewards with the Kaplan Higher Education Group, who utilize the Axonify platform to reinforce learning with their student support and internal training teams.

J.D. Dillon of the Kaplan Higher Education Group describes the impact of rewards in their environment. “In our Axonify implementation, we’re experiencing over 90 percent voluntary participation for those pockets of learners with achievable, tangible rewards. That contrasts to 40 percent voluntary participation when learners do not yet have tangible rewards in place.”

Lesson five: Learners appreciate having a coach

In our learning environment, learners have the option of choosing a coach. These virtual coaches motivate learners by celebrating when they do well, and looking disappointed when they don’t.

At first glance, it wouldn’t seem very critical to the learning process, but we were surprised to find that of the hundreds of thousands of people using Axonify, over 80 percent of them take the time to select a Coach that appeals to them. It was also interesting that people preferred the term Coach over the term Avatar. In the video-gaming world, an Avatar is a virtual representation of the person playing the game. This suggests to us that people feel more comfortable with someone at their side offering support, rather than looking at a virtual representation of themselves.

We’ve also found that it’s important to offer the ability for learners to select from a variety of Coaches, so they can choose one that they feel most comfortable with. Some select very realistic looking coaches, while others prefer a fun, “cartoony” character.

Lesson six: Employees like to see their progress

While tangible rewards motivate many learners, there are also intrinsic motivators that drive many people to succeed, such as the feeling of achievement or desire for self-improvement.

These learners want to see their own progress so they can analyze their performance and make the adjustments necessary to improve (Figure 3). It’s extremely important to provide a progress report that:

  • Is easy to access and understand
  • Provides a graphical interface
  • Allows for drilling down to deeper detail

Figure 3:
Access and illustrate an individual’s progress in a simple, digestible format

We have found that 9 percent of Axonify users check their report card daily, and 7 percent take extra training so they can improve their scores. This can be quite significant when you consider that these employees likely represent your population of self-directed learners, often on the promotion track.

Lesson seven: Social connectivity creates learning engagement

One of the most powerful engagement tools in next-generation online-learning environments relates to social connectivity—connecting employees to each other, to learning content, and to the organization. Providing a forum for employees to engage socially and contribute content is not only beneficial, but in the age of Facebook and Twitter, it’s expected.

We’ve found that the ability for learners to contribute content is quickly becoming one of the most popular features of our system: over 15 percent of Axonify users engage socially every day, and that number continues to grow.

This is exciting news; it’s an area where organizations can dramatically improve employee engagement. We believe there are significant benefits to adding social connectivity to the learning experience:

  • Social connectivity relies on active participation, increasing engagement in the overall learning experience.
  • Employees are able to contribute as collaborators and team players, increasing their connectedness to coworkers.
  • Employees feel they are contributing to the learning experience of their coworkers.
  • As contributors to the knowledge base, employees feel more connected to the success of their organization. 

Get A+ on your learning report card

These powerful—and surprising—lessons prove that gamification more completely engages learners. But you also need to pay attention to detail. Provide variety, challenge, and community. Offer rewards. Allow employees to contribute to your knowledge base.

By adding gamification to your learning, then refining it to meet learner preferences, you will more fully engage your employees to deliver the results you need: better knowledge retention, improved job performance, and a workforce aligned with your vision.

Topics Covered

Appreciate this!

Login or subscribe to comment

"Karl Kapp, a respected expert on the convergence of learning and technology" LMAO
How about that:

Bryan Hoppe, VP Operations, says, “The most surprising benefit of Axonify was how well it was received and continues to be received by our employees. We get well over 95 percent participation in our Axonify learning, and the game approach is a big part of why employees are sticking with it.”

You are using your company's name at various points in the article, and expect us to believe the lessons you gained? Which independent third-party company has done the analysis?

You can use big numbers in your ad campaigns, but not in "an article which has a title as report card" without any reference. Or maybe I should stop reading articles in this site.
Yes I agree that this sounds a bit like an ad for Axonify. I am a bit skeptical of these "findings" and disappointed because I would love to see more real research and critical evaluation on this data.
I am reminded of the movie "A Christmas Story," where Ralphie uses his special decoder to ultimately decode an ovaltine commercial. This article feels like an ad.
There's a large amount of research to suggest that extrinsic rewards, like vacation days, are damaging when pursuing something creative, like learning.

You say you increase participation with these rewards, but what effect did that have on outcomes, business or otherwise? We've seen lots of research where increases in participation actually negatively correlate with the quality of that participation. A lot of effort, but not productive.

You've got some interesting data potentially, but like others say, you are presenting it like an advert with no real substance as to how increases in participation lead to better performance.
While it is natural to be skeptical of data from a vendor, it is also important not to miss the point of what is being said. The article offers the author's conclusions about which gamification features appear to work, in terms of learner participation and engagement, and game popularity. These conclusions would seem to be good candidates for further research and testing. They are not definitive, but they may be suggestive. None of them depend on using the author's product. None of them make claims as to the effectiveness of the author's product per se. We would be glad to publish findings from research that looks into exactly what the effects of gamification are. We do indeed need to understand more about those effects on outcomes. I would like to suggest that it would be very helpful to discussion if commenters would check the box marked "Include username in post". Anonymous comments are as subject to skepticism on the part of other readers as vendor-provided data. I would also suggest that using the name of someone mentioned in the article and appending "LMAO" to it verges on personal attack, particularly when the comment is left as anonymous. Personal attacks are not acceptable.
How about that: ...
comment belongs to me, I live in Turkey and I believe you will have some difficulties pronouncing my name. Also, probably, I am neither in their market nor their competitor. So I am not attacking them. But we live in a global world. People will read these articles and talk with me referring to these “lessons”. And I think I will have some difficulties explaining whether these are true or not.

Ok I get that these are findings of that company and subjective, but presented like conclusions and I want some reference.

How about that again:
These powerful—and surprising—lessons prove that gamification more completely engages learners.

It would have been great if they have concluded like “these are our lessons and would like to discuss bla bla…”
P.S I also would like to add that I prefer pure game design over gamification. So I don't like to see poorly designed gamification sold on those ideas. No offense Benbetts I know you are doing heavy research on making gamification better (or maybe just to support your products with data :) ).

And I meant "presented like conclusions as definitive"
gamification: games designed by people who don't play games aimed at people who don't play games, about topics that don't lend themselves to games. enjoy! (now reviewed by people who don't play games.)
Great comments everyone. To the question of how engaging learning experiences impact learning results, we actually do correlate participation and individual learning success with behaviour in the workplace that drives financial outcomes. I'd be happy to share more specific data involving several applications like growth in sales and reduction of OSHA reportable incidents if you'd like to connect with me directly. Unfortunately, most of our clients are reluctant to share specific data publicly, although the WalMart article references a very significant and specific measurable outcome and a result of direct knowledge acquisition.
I hear a lot about encouraging intrinsic learning because this is kind of learning that is retained. However the use of tangible rewards and leader boards are definitely extrinsic rewards. have any studies been done on retention of learning when learning occurs through gaming. Do the behaviors decline when the games stop reinforcing them?
It seems to me that generalizations can be dangerous. The article looked only at which gamification features are effective. Just incorporating those features, and doing nothing else to support successful job performance, is not the same thing as having an effective support/channel for learning. In considering whether to gamify/use games, as I look at the various studies (which are available online, easily found with your favorite search engine), it seems necessary to consider variables such as the amount and type of cognitive content, the character of the learners, whether the game/gamification was in addition to classroom training or other instruction, the amount of practice built into the game, etc. Since the conclusions in the article are similar to ideas endorsed by developers of commercial games (anything from Candy Crush Saga to Halo and beyond), I don't have any big issues with the notion that designs that incorporate the "formula" the author suggests will probably increase learner engagement. However, it also seems to me that an educational game design that is instructionally unsound or weak, or that is otherwise unsupported by practice/other instruction/organizational policy and practice/coaching/culture, probably won't succeed no matter how engaging it may be. Mario Brothers games are certainly engaging, but apart from teaching how to play the game, they are not of much value teaching cognitive skills. You have to look at the total package. What the article does is suggest game features that actually work to generate engagement, which is exactly what the author promised in the opening paragraphs.
Apparently I'm not as respected as Carol thought;(

I feel like Bob Uecker. LMAO.
Submitted by Karl Kapp

First, of all, I am not going to address the childish, hurtful and anonymous swipe at my credibility. I can’t image the thought process or intended goal of that activity.

However, I do think the comments submitted have flaws in their logic in terms of the article.

First of all, the vendor is 100% transparent in stating that they are a vendor and they have a gamification platform. There is no attempt to mislead the reader under the guise of a “research” whitepaper or article. They are 100% up front about who they are. So the expectations are set early.

Second, participation numbers are easy to verify. You either have 95% of people participating or you don’t. There is no research needed to verify participation numbers. You don’t need a null hypothesis or a statistical chi-square formula. It is purely descriptive data. Same with the fact that 37% of people check the leaderboard on a daily basis. You can research effectiveness or questions the way some one measures “learning” but most of the numbers in the article are very straightforward. So you either believe those number of you believe the vendor is just “lying” about those numbers because they are a vendor—that thought process being, because it is a vendor it is a lie. But the article also have customer quotes, why would a customer allow the vendor to lie about this information and, on what grounds, do you have to state that you think they are lying? The argument that because it is a vendor, the number must be false is a cynical and unproductive argument. Be skeptical but also keep an open mind, the number might just be true in the situation reported.

Third, it is so easy for people to say “show me the peer-reviewed research” before I make a decision. Otherwise I won’t believe the results. The reality is that doing peer-reviewed research within corporations and academic institutions is a painstaking, time consuming process which requires Internal Review Board (IRB) authorization, informed consent and dozens of other restrictions. It takes too long and many companies won’t go through that process (meaning the customers of the vendor). Having vendor contributed data moves the field along quickly. We do need to read with an understanding of the source but the results can still be valuable. And, please remember, most peer-reviewed research has some type of bias as well…it’s just not as clearly stated or transparent as this article.

Fourth, there is a false dichotomy between intrinsic and extrinsic rewards. It turns out that they are not polar opposites but actually co-exist. And the study of intrinsic vs extrinsic motivation is itself flawed in some cases. One widely used scale to measure the intrinsic vs. extrinsic motivation was created by Harter, who design the scale with 3 subscales of intrinsic motivation and one scale of extrinsic motivation and the scales are specially “designed in such a way that it is not possible for children (or others) to report themselves as simultaneously intrinsically and extrinsically motivated. [In fact] the negative correlation between the two scales is built into the scale.” The bias is built right in—a person cannot, on that scale, report being both intrinsically and extrinsically motivated. The reality is that humans are both internally and externally motivated at the same time. Intrinsic and extrinsic motivation should be viewed as two largely mutually independent constructs rather than opposite ends of a single dimension. (see citations below.)

Fifth, I find it interesting about the notion of “pure game design” over gamification. The individual from Turkey is not the first person to make this argument. The argument seems to be that because a certain group of people create “serious games” that no one else has the right, privilege or understanding to use elements of games in any other context. These people seem to believe they are the high priests of game design and the tools of game design are too dangerous or unwieldy to be used by anyone else. I reject this view as elitist and arrogant. Educators, instructional designers, elearning designers and teachers have been using game elements in non-game settings for decades and longer. Schools use points, levels, leaderboard all the time to motivate kids. Good trainers use challenge, story and feedback loops in their classrooms and online with great success. To assert that because you build “serious games” no one else has the right to use parts of games in any other context and if they do, it is wrong or unproductive is silly and uninformed. No group of people has an exclusive right to use game elements.

Harter, S. (1981). A new self-report scale of intrinsic versus extrinsic orientation in the classroom: Motivational and informational components. Developmental Psychology 17, 300-312.
Lepper, M. R., Iyengar, S.S., & Corpus, J.H. (2005) Intrinsic and extrinsic motivational orientations in the classroom: Age differences and academic correlates. Journal of Educational Psychology 97(2), 184-196.
The usage metrics and client feedback are clear. The indication is that games increase engagement and interest. When correctly implemented and aligned with the instructional goal and objectives gamification can be a powerful tool to produce successful training outcomes. Anyone trying to poke holes in the information provided in Carol's article is sadly missing the point.
Nadia Laubach
LJL Consulting Inc.
Related Articles