Your Source for Learning
Technology, Strategy, and News
ARTICLES       RSS feed RSS feed

Why Games Don't Teach

by Ruth Clark

February 19, 2013

Feature

by Ruth Clark

February 19, 2013

“Although many games don’t teach, that’s not to say that they can’t teach. Our challenge is to empirically identify and catalog game features that align to learning goals and build a repertoire of principles for game design.”

My claim is that games don’t teach. Not that games can’t teach, but that advocating games as a main or even frequent instructional strategy is misleading. Here’s why.

What is a game?

If you ask a group of trainers to define what they mean by the term “game,” you will likely hear a rather eclectic mix of features or examples. Kapp defines games as “a system in which players engage in an abstract challenge, defined by rules, interactivity, and feedback, that results in a quantifiable outcome often eliciting an emotional reaction.” Mayer identifies the following four features that characterize all games: 1) rule-based 2) responsive 3) challenging and 4) cumulative. I have no argument with these definitions. However, they encompass such a broad range of possibilities that I’m sure we can come up with a lot of items that we could call a game but which would not necessarily lead to learning.

So what about a taxonomy of games? There are several of these around, based primarily on the major types of entertainment games you’ve likely seen or played. Common categories include action, action-adventure, adventure, roleplay, strategy, simulations, puzzle, sports, board, and card games. My guess is that we could classify many specific games into two or more of these categories. And I’m not sure that any of these categories relate to the instructional effectiveness of a game.

We need a taxonomy of games or game features that link to desired instructional outcomes. Let me give an example from a different common instructional strategy: graphics. Chopeta Lyons and I defined graphics from three perspectives in order to organize their potentially relevant features. First, we defined graphics in terms of surface features such as still graphics versus animated graphics. Second, we identified communication functions such as decorative graphics or explanatory graphics. Third, we listed psychological functions of graphics or graphic features such as promoting encoding, focusing attention, or minimizing extraneous mental load. I believe game categories such as adventure or strategy tell us a bit about the surface features of games but little about their instructional potential. My challenge to game advocates is to develop a meaningful taxonomy that connects game features to learning outcomes.

There is little evidence for learning value of games

A couple of recent technical reviews have carefully evaluated documents on games, looking for credible evidence of what works. The consistent conclusion is that there is insufficient well-designed experimental research on which to base many conclusions. For example, Hays initially identified 274 documents on the design, use, and evaluation of games. Of these, he discarded 62 percent because they were opinion-based rather than data-based. His final review included 105 documents, of which 48 reported empirical evidence of game effectiveness. Based on the 48 studies, he concluded that there is no evidence to indicate that games are the preferred instructional method in all situations, allowing, however, that some games can provide effective learning for a variety of learners for several different tasks such as math, electronics, and economics.

Sitzmann also reviewed research studies that compared the learning effectiveness of computer-based simulation games with a comparison group. Although she weighed in favor of simulation games, her analysis shows that when the comparison group involved some form of active learning (versus listening to a lecture or reading), learning was less effective among the simulation-game learners. Specifically, she concluded, “computerized tutorials were much more effective than simulation games.” Overall I read her analysis as support for active learning, be it in a simulation game or a computer tutorial. Take a look at her report and see what you think.

Moving beyond game hyperbole

Because games come in so many varieties and are useful for so many purposes, I hope that we will move beyond hyperbole and start to consider what specific features of games will promote learning of a particular type. Mayer and his colleagues have conducted a series of experiments in which they create two or more versions of the same game and identify how specific features promote or depress learning. From these experiments he is building a repository of guidelines for game design.

For example, using a circuit game which he describes as a puzzle game intended to teach how electrical circuits work, he has found that encouraging learners to engage in self-explanations by clicking on a rationale for a game move resulted in better learning than the same game without the self-explanation additive. In another experiment with the circuit game, Fiorella and Mayer report that adding an interactive paper-based game principles sheet improved learning from the game among those who correctly identified the principles. Furthermore, learners using the paper-based aids enjoyed the games more than those who played the games without the aids. As these types of experiments accumulate, we will have a number of prescriptions for how games can be made more effective for learning as well as what to avoid.

So, can games teach?

Certainly games that align to the instructional goals, and that offer the right balance of challenge and guidance, can support learning. I’m a language learner, and one game used to stimulate vocabulary learning is a digital version of Concentration. You see 16 facedown cards on the screen and you can click on any two. Half of the cards have pictures and the others have vocabulary words. If the two that you select match, the game eliminates them. You continue clicking on pairs of cards until you have eliminated all of them. What’s the problem with this game design for learning vocabulary?

Here’s my analysis as a player. Since the goal is to learn vocabulary, it’s a good idea to have some drill and practice activity that involves matching pictures and words. Learning vocabulary is a rather tedious process, so putting a game face on a drill-and-practice exercise is a good idea. The problem, however, is that the task of having to recall the location of a particular picture or word among the cards adds irrelevant mental load to learning vocabulary. The game requires me to match words and pictures (good), but at the same time to recall the location of specific word and picture cards (irrelevant to learning a language).

How would I design a more effective game? A better version involves displaying 16 cards with pictures facing the player. Then by audio (or print) the game presents a word and the learner must click on the correct picture. As the words become more automatic, response time would be faster. Therefore the basis for the score could be the time to identify all of the words. Displaying a record of scores over game trials would allow the player to see learning progress. These two game versions illustrate how the instructional goal of accurately and quickly matching a word to a meaning could be slowed or enhanced based on the design features of a game.

Conclusion

Although many games don’t teach, that’s not to say that they can’t teach. Our challenge is to empirically identify and catalog game features that align to learning goals and build a repertoire of principles for game design. Meanwhile, we should work to implement games that will encourage the mental processes required by the learning objectives and add features to these games such as self-explanation questions known to improve learning.

References

Clark, R.C., Lyons, C. (2011). Graphics for Learning – 2nd Edition. San Francisco: Pfeiffer.

Fiorella, L, & Mayer, R.E. (2012). Paper-based aids for learning with a computer-based game. Journal of Educational Psychology, 104, 1074-1082.

Hays, R.T. (2005). The effectiveness of instructional games: a literature review and discussion. Technical Report 2005-004. Naval Air Warfare Center Training Systems Division. http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA441935

Kapp, K.M. (2012). The Gamification of Learning and Instruction. San Francisco: Pfeiffer.

Mayer, R.E. (2011). Multimedia learning and games. In S. Tobias & D Fletcher (Eds.), Can Computer
Games be Used for Instruction?
Greenwich, CT: Information Age Publisher.

Sitzmann, T., (2011). A meta-analytic examination of the instructional effectiveness of computer-based simulation games. Personnel Psychology, 64, 489-528.


Topics Covered

(51)
Appreciate this!
Google Plusone Twitter LinkedIn Facebook Email Print
Comments

Login or subscribe to comment

Ruth, again you have cut through the hype.
For years every new development, from online learning, to simulations and games, to mobile have sometimes been presented as the answer.
Having developed numerous games, simulations and mobile solutions, I believe that these programs can be a very effective answer if the right question is asked. The problem determines the solution, not vice versa. Otherwise you are a carpenter looking for a nail where a screw, glue, or other solution might be better.
Games are most effective when they use imagination, graphic design and game design principles to transform a boring test into a competitive activity in which a high score gives the learner a sense of accomplishment and mastery.
The strategy for success in the game or simulation must match the strategy for success on the job. For example, if the job task requires thoroughness and accuracy, you don’t create a game in which speed is rewarded more than accuracy.
Thanks again for your wisdom.
Ruth,

I think that the most important piece of an effective learning game is the inclusion of generalization strategies. You allude to this in your article when you describe giving the player feedback for learning progress.

But I think it can go beyond feedback and even start with defining the game goals in terms of real world learning. Perhaps more importantly, many of todays games (that do not have transfer or generalization strategies built-in) can use parents or educators to connect the game to real world activity. We are working and researching these strategies at LearningWorks for Kids and have found that many popular games and apps can become great venues for learning when generalization tools are utilized.
Ruth,
Interesting article and overall I do think we need some additional research but I do take issue with several elements of your article and in spirit of a friendly, collegial academic debate here goes...

First, your title "Why Games Don't Teach" is frustrating and mis-leading and leads to people dismissing games as an educational option for learning within organizations when it can be a very viable and useful solution. Games do help people learn and are motivational across genders and age groups (Ke, 2009).

Also, I'd love to see the research that supports the conclusion that bullet-ridden PowerPoint slides presented one after another with an occasional multiple choice question provides any type of transferable learning or even the slightest bit of retention beyond the online or classroom experience.

There have been numerous meta-analysis studies and individual studies that clearly indicate that games teach and teach more effectively than lectures or even class discussions. Sitzmann looked at 65 studies and found interactive simulation/game learning 17% more effective than traditional lectures and 5% more effective than discussions.

Hays, in addition to what you indicated writes that "games can provide effective learning for a variety of learners for several different tasks (e.g., math, attitudes, electronics, and economics)…(p.6) [and]… The second "claim," that games enhance cognitive learning, continues to be supported. The research shows that people can learn from games."

A meta-analysis by Ke (2009) found the effects of learning within games was positive in 52% of the studies. (examined 256 studies 89 empirical studies were used in analysis)

Studies indicate that pro-social games encourage pro-social behavior outside of the game environment (transferability) (Greitemeyer & Osswald, 2010)

The preponderance of evidence from the research leads to the conclusion that games facilitate learning.

Do they always 100% of the time facilitate learning? NO, Absolutely NOT. But neither do other forms of instruction.

I agree a taxonomy is needed, don't agree with the type. I have created a taxonomy that maps type of learning (i.e. declarative, conceptual, procedural, etc.) with type of game activity (e.g. when teaching declarative knowledge use a matching game.) We both agree that what practitioners need are tools to decide when to use games and game elements.

For example the use of avatars or game characters is heavily supported by the literature, the use of a challenge (which all games have) is heavily supported by research, as is the use of continual corrective feedback which games have over traditional lectures and instruction in buckets.

So, let's not say "Games Don't Teach" (because that is clearly not the case), let's say "Games Teach the following..."

References:

Greitemeyer, T. & Osswald, S. (February, 2010) Effects of Prosocial Video Games on Prosocial Behavior. Journal of Personality and Social Psychology, Vol 98(2), 211-221.

Hays, R.T. (2005). The effectiveness of instructional games: a literature review and discussion. Technical Report 2005-004. Naval Air Warfare Center Training Systems Division. http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA441935

Ke, F. (2009). A qualitative meta-analysis of computer games as learning tools. In R. E. Ferdig (Ed.),Effective Electronic Gaming in Education (Vol. 1, pp. 1-32). Hershey: Information Science Reference.

Sitzmann, T., (2011). A meta-analytic examination of the instructional effectiveness of computer-based simulation games. Personnel Psychology, 64, 489-528.
First, thanks to Ruth for highlighting some of these studies, and also thanks to Karl for his response -- I particualarly think the taxonomy question is a good one, and I'm also really interested in breaking down the criteria we use to match instructional challenges to instructional interventions (games or otherwise).

Specifically, I think that there are many aspects that need to be looked at before we can make good judgments about the efficacy of a game approach. Individual games can vary wildly in terms of the quality of the game, the appropriateness for audience, and the degree to which the game-play matches the real world desired behavior.

That aside, I guess I'm still puzzled about the title -- I don't really know what to do with an article title that is so obviously false (and falsifiable - as evidenced by the article content itself). I assume it's an attempt to stir the pot -- using hyperbole as a way of provoking discussion. But given that one of the stated objectives is "moving beyond game hyperbole", maybe a more balanced title would be more conducive to a constructive (and necessary) conversation around the effectiveness of instructional games?
Because games add substantial overhead, except in the most trivial instances, to the learning process, they have a large burden. A "learning game" must not just provide learning but better learning than alternatives in the same time span (and cost framework).

As Ruth Clark points out, they frequently (probably usually) don't. She neatly illustrates the thinking that must go into creating games can do help learning. Mostly these involve memory. Occasionally, higher-level thinking skills can be learned, but these examples generally involve a learning scaffold that extends beyond the game itself to facilitators and group members.

A single-person game done alone does not produce higher-level learning commensurate with the effort involved. Someday, someone may figure out how to get past that barrier.

As a scientist and science education specialist, I have not seen a non-memory science learning game that's worth the effort to create it. Instead, I've seen tons of money from government, foundations, and schools spent in a pursuit that just wasted that money.

Read carefully what Ruth Clark wrote. You'll understand why and also why a little more thinking should be done before spending on something just because it's the latest thing.
I agree that the article should have been titled differently.

It's almost as if it was meant to be provocative, rather than informative.

Thanks to KKapp and JDirksen for your responses. I can't think of anything to add.
Adding my thanks to kkapp and jdirksen. Brilliant, well thought out and spot on responses!
Excellent article. I agree that we are too often prone to hyperbole and using game strategy when it doesn't fit the situation. When as designers, we push for "best fit" opportunities (such as the language learning example), I think the learning outcomes for game-based would be more realistic, showing when they truly benefit a learning situation.
Games don't teach (say) mathematics in the same way pianos don't teach music. But just as a piano provides an excellent tool with which to learn music, so too can a good game, well designed, provide an excellent tool with which to learn mathematics. I use math as an example since that is the domain I am familiar with, and the focus of my recent article in American Scientist http://www.americanscientist.org/issues/pub/the-music-of-math-games on the design of video games that facilitate good math learning.
I agree with the major points of the article, though not the deliberately provocative title. It could be said just as easily that textbooks and lectures and OJT don't teach. That's too broad a brush.

As it happens, this is exactly the area I'm devoting my sabbatical to this semester. I think the difference between games that teach and those that don't depends on something smaller than the existing taxonomies of games. It has to do with the relationship between game mechanics, the things that make a game a game, and the cognitive processes they provoke. To take a simple example, consider what is on a card you pick up in a board game. What's visible on the card? What's hidden that forces you to fill in the blanks by retrieving it from memory? Small differences in the design of the cards could make a large difference in how effective the game is for learning. Similarly small differences in rules, scoring schemes, screen design might lead to large differences in learning.

I believe that having a fine-grained taxonomy of the building blocks of games as they relate to the processes of learning would help us design better learning experiences. In a few months, I'll know if that really pans out.
Hi All - Delighted to hear all of the different perspectives - a good debate is useful for all of us. Let's continue the discussion!
Games don't teach. That's a very strange assertion. Learning, for externally measurable purposes, is an observable change in behavior after a learning experience has been lived through. How well does the ID control what was learned...? Well that's a question that applies to any contrived learning experience. What I hear is an objection that the learner is learning things the ID didn't intend and may not be acquiring the objectives. Hello! Of course. There is no learning experience that is free of this gap.
Any game, even one not at all concerned with any academic task, teaches the player to play the game. If the ID is too far from gaming, or doesn't understand game design, doesn't play games, then that ID is unlikely to produce a game, which when played, will train the player. For such ID's it IS best to stick to traditional command-control top-down design scenarios.
Ruth,
Great commentary, as always!

I suspect you and Rich will both conclude that the limited effectiveness of both games and simulations comes down to the integrity of the underlying cognitive task analysis. In either, if the designer has correctly understood the cognitive structure of the task, and there is high cognitive fidelity to the transfer task, then effectiveness can show. Otherwise, whatever the simulation or game teachers won't transfer.
Thanks to Karl and Julie for their thoughtful and research-based comments. I'm disappointed, however, that those of us with expertise and experience on this subject have to come out to counter-balance inflammatory and misleading article titles, as well as provide data to disprove an over-simplification of a complex design process. Having just completed my book on Immersive Learning that examines how immersive experiential games (and other immersive environments) focus on behavior change and skill improvement, another major complaint with this article is that it's focused on learning.

Are we really so wrapped up in what people know, not what they can do? Isn't our goal performance improvement and behavior change? Shouldn't we be focused on what games can help you DO better? Let's stop limiting our design thinking around games to flash card activities and actually leverage the rich, complex technologies available to us to allow practice with continuous feedback (something at which games are EXCEPTIONAL).

Also, who says games need to be technology-centric? Alternate reality games (ARGs) have been used extensively for learning and practice; early data suggests these types of games are effective in improving performance in context.

If we continue to frame conversations around learning, we will continue to miss the point, and the real opportunity for games.

Just like with any learning environment, good design is the differentiator between an effective experience and not-so-effective one. I'm not JUST talking about instructional design: game design, UX design, storytelling, character development, graphic design and others all are part of the process.

I hope, in the future, that articles are more informed and less inflammatory. It does learning professionals a disservice to promote opinions based on one or two research studies based on a limited definition of "game" when there are a wealth of studies that have found quite the opposite results.
Karl Kapp has offered to provide a counterpoint article. My plan is to publish that on March 4 (unfortunately, I don't expect to receive it in time to publish this coming Monday, February 25). -- Bill Brandon, Editor
I should add that many of the comments here are excellent outlines of potential articles on the fine points of instructional design, game design, learning, and the relationship between the three. It would be a wonderful thing to be able to publish those articles. In my opinion, we (the community represented by the commenters and the readers) really need this kind of discussion at this point in the evolution of our increasingly technology-mediated/technology-assisted learning and performance. Ruth, Karl, Julie, Koreen, Bernie, Rob, Tahiya, and others whose monikers I do not recognize, are experienced and well-respected practitioners whose ideas, approaches and contributions are important to the advancement of our science, craft, and art. We would be proud and honored to publish those ideas. -- Bill Brandon
Part of the problem seems to be with the word "game" which means too many differrent things. Some read "games" and think "concentration," "wheel of fortune," "jeopardy," and other similar games that embed no real-world context. While these games can enliven drill-and-kill style training, I hope these aren't the games that instructional designers are most excited about.They strike me as straw men in the argument against the instructional use of games.

When I think of designing games for instructional purposes, I am almost always thinking of a scored simulation in which the learner has one or more defined goals. I'm usually trying to simulate some aspect of the learner's job, and the goals are to successfully demonstrate new behaviors that the course is trying to teach. The simulation needs to present a realistic challenge and embed as much real-world context as necessary for the learner to recognize and act on the relevant aspects of the challenge. This basically boils down to giving the leaner a chance to practice in the course the skills you are trying to teach him or her to perform outside of the course. If the simulation captures the essential nature of the challenge, then the experience that the learner gains from mastering it should transfer pretty easily to the real world outside of the course.

If this learning activity remains undirected and unscored, then I think of it as a simulation that the learner can use to safely explore possibilities. By contrast, if the learner earns points for doing things well and loses points for doing things poorly, or if the learner has a realistic goal (say, a good performance evaluation at the end of the simulated "year", or maybe a raise or a promotion--you get the idea), then the activity becomes a game.

To obtain meaningful data on how effective this approach is, we need studies that study this kind of game. If you mix this kind of game with games that have no realistic business context, and try to study the effectiveness of all kinds of games together in one big lump, I have to question whether such studies can really draw meaningful conclusions. Which, I guess gets us back to the need for a clear taxonomy.
First, thank you Karl - you just saved me time from looking up all those references. Second, this is some kind of nightmare scenario where people who have never designed a game or studied game design hold forth on the capabilities of games. Third, games - unlike instructional design - focus on keeping the player in a state in which they are open to learning and actively engaged in the content. Fourth, and this is where I'll really get going - where is the demand that we exam Bloom, Gagne, Kirkpatrick, et al with the same rigor and skepticism demanded of games? I began my life in learning and training coming directly out of academia and a doctoral program in history - I will tell you this, what passes for "rigor" in this field would get you severe talking to if you handed it in during a grad seminar in history. I'm not going to debate the efficacy of games to teach when they have been incredibly effective learning environments since the invention of Go - to argue otherwise is to describe an ignorance of the history of games and game design. I will argue that what would serve this industry well is an in-depth and ongoing examination and critical analysis of its entire canon.
As a proponent of games in schools, I thought this article missed potentially the most important idea. Games and game designers build games to ENGAGE all players. Schools struggle with this. I would suggest that games allow teachers to have common anchor points to build off of for their classroom. And teaching students to see learning within the games is powerful. This article reminds me that tools that are available in today's classroom don't fit in yesterday's classroom very well.
http://youtu.be/tsQ6n1j75UA

Gamifying the classroom.
Much as I respect Ruth, this article as other comments suggest, misses the mark. I've summed up my thoughts in a blog post: http://blog.learnlets.com/?p=3124
I appreciate the well-thought out article, but disagree with the main message of your article. I think there is far too much research on the topic that clearly shows the educational games do teach and the learner benefits from using them.
Hi Ruth,

Appreciate your thoughts.

But, in my opinion games are a fine medium to provide real-time feedback and are therefore a good mechanism to include a feedback loop into a story. Gamification also inevitably is one of the best methods to reinforce learning since it immerses learners into a visual experience; hence enabling them memorize facts faster. However, I do second Koreen’s thoughts about a good design. Indeed, to use this mechanism to achieve the set goals/objectives of a course, it is important to have a good design. A well designed course involving games certainly proves to be effective.
Here are few points that help me while involving games into the learning environment:
1. Consider your learning outcomes and support them by using suitable games; for e.g. board games, strategy games, puzzles, role based games, etc.
2. Channelize learners in such a way that they ultimately achieve the set goals; explain the goal of each level of the game and what the reward is in return on completion. This keeps them motivated.

Thanks,
Sanobar
www.raptivity.com
Related Articles