Can Digital Games Boost Students’ Test Scores?

| June 17, 2013 | 22 Comments
  • Email Post
ipadgame

In the past few years, educators have been closely watching the evolution of digital games used for learning. With a huge influx of products — whether they’re individual apps for tablets or an entire suite of software — the market is already big and continues to grow, with entire game-based schools cropping up across the country.

There’s no question students are interested in digital games – 97 percent of kids play them — but what educators and industry watchers want to know is whether playing those games can actually improve student achievement.

A new SRI study released today suggests they do — at least in the subjects of science, math, engineering, and technology. According to the report, which is an analysis of 77 peer-reviewed journal articles of students K-16 studying STEM subjects, “when digital games were compared to other instruction conditions without digital games, there was a moderate to strong effect in favor of digital games in terms of broad cognitive competencies.”

More specifically, “students at the median in the control group (no games) could have been raised 12 percent in cognitive learning outcomes if they had received the digital game.”

Another way to explain it: “For a student sitting in the median who doesn’t have a game, his or her learning achievement would have increased by 12 percent if he or she had that game,” said Ed Dieterle,  Senior Program Officer for Research, Measurement, and Evaluation for the Bill and Melinda Gates Foundation, which funded the SRI report.

“The games and learning space is still in an exploratory, R&D phase. We shouldn’t frame games, or any other instructional support, as ‘the answer.’”

Simulations have an even bigger impact, according to this analysis. When considering simulations — taking a phenomena, process, or behavior and coding it into something that can be manipulated and studied — improvement index jumped to 25 percent, meaning students who used simulations could have increased their learning outcomes by that amount.

Which begs the question, how do we define learning outcomes? According to Stacey Childress, deputy director of education at the Gates Foundation, learning outcomes can be defined in a few ways: progress toward mastery of a particular set of content and skill objectives in areas such as math and literacy; demonstration of complex skills like collaboration and critical thinking; and improvement in what researchers call “non-cognitive” skills such as persistence and grit.

“With learning games, it’s important to understand which kinds of outcomes they are designed to improve and whether or not students are actually making progress on those dimensions,” Childress said.

[RELATED: Teachers, Students, Digital Games: What's the Right Mix?]

The Gates Foundation has made huge investments in the educational gaming world. Last year, the foundation launched the Games Learning and Assessment Lab (GlassLab), which was tasked with prototyping and developing games and formative assessments. The work is being conducted by the Institute of Play, the Educational Testing Service (ETS), Pearson, Inc., Electronic Arts (EA), and the Entertainment Software Association (ESA). GlassLab recently released, SimCityEdu, which integrates assessments aligned with Common Core State Standards. The educational version uses the same code as the commercial game, but with the addition of using students’ choices during challenges as a method of assessment, though not all education experts agree that assessment should be built into games.

The foundation has also invested in The Center for Game Science and the Radix Endeavor at MIT with the intent to develop games that embed valid assessment measures.

For this analysis, SRI considered reports from a gamut of sources in those 77 studies — going as far back as a 1992 study from The Journal of Educational Research looking at the effects of computer simulations and problem-solving approaches on high school students, to a 2006 study in the journal Interactive Learning Environments using just-in-time information to support scientific discovery learning in a computer-based simulation.

Other studies examined include one from 2011 that compares different versions of a game in terms of the degree to which the learning mechanics and goals are integrated directly into the central game mechanics (intrinsic design) versus separating the learning mechanics and goals from the central game mechanic (extrinsic design); and another from 2012 that compares different approaches to socially organizing players within a game in terms of collaboration and competition to maximize learning. The games within each study were developed specifically for research purposes, and thus are not as elaborate as some commercial titles like SimCity, but are solid examples of learning games, according to Dieterle.*

“This is the first big study to hit the pause button for a second and reach back in time and extract everything we could from what previous researchers have done with the intent of using that information to inform us about the field going forward,” Dieterle said.

FUTURE OF GAMES IN CLASSROOMS IS NOW

If digital games were rare in the past, that’s no longer the case. According to a recent teacher survey conducted by PBS, 43 percent of classroom computing goes to playing educational digital games. And in one study undertaken last year by the Joan Ganz Cooney Center, surveying 505 teachers, the majority of teachers reported that games increase motivation and make it easier to personalize learning.

But experiences and perceptions around games are still very much a mixed bag, depending on whom you ask. Some educators are skeptical that digital games are the answer. They question whether games provide enough context and depth that come from hands-on experiences.

“Imagine the difference between a student who’s playing an online math video game and a student who’s sitting in a small group with a teacher, working out problems and receiving immediate, individualized feedback and guidance,” said St. Louis-based fifth-grade teacher Jenny Kavanaugh in a recent interview. “There is no comparison.”

But Childress points out that the issue is more nuanced.

“The games and learning space is still in an exploratory, R&D phase. We shouldn’t frame games, or any other instructional support, as ‘the answer,’” she said. “All of us working in education should be skeptical about any innovation that doesn’t aim to produce evidence of its effectiveness. The SRI results are a strong start in the direction of solid evidence.”

And Childress does not see the use of digital games as an either/or scenario — either teachers or digital games.

“We should be careful not to view learning technologies as a replacement for deep teacher and student interactions. We see effective technology supports as enabling the opposite,” she said.

Digital games can be a part of a holistic plan that challenges students with things like “quests” and “missions,” when paired with tactics like spending targeted time with students in small groups or individually to help them address areas where they need help, she said.

[RELATED: Money, Time and Tactics: Can Games Be Effective in School?]

For educators who aren’t sure where to start, or how to find ways to integrate digital games into the current model, Childress said teachers’ own network can be a great resource.  For instance, one of the Gates-funded organizations, Playful Learning, focuses on creating a national network that offers teachers workshops on using games in the classroom.

“All of us working in education should be skeptical about any innovation that doesn’t aim to produce evidence of its effectiveness.”

For their part, game developers should incorporate ways to help educators do their jobs better, as is the case with other industries that have embraced technologies. “The best product developers deeply understand who they are designing for and the use case they are targeting, and offer the kind of implementation supports professionals need to integrate new tools into their daily work,” Childress said.

If Quest to Learn, the entirely game-based schools in New York and Chicago, are any indication of whether games can be successful learning tools, the potential seems bright. According to CNN, the school’s New York test scores, “an admittedly conventional metric, show the Quest kids have outperformed peers in the New York City school system in each of the last three years, in both English Language Arts and Math, according to data provided by the school,” with the only exception being the 2010 math scores.

But not every school can be a Quest to Learn, with dedicated funding for games. Finding the funds to finance digital games is one of the main obstacles, in fact. In the Cooney Center survey, 51 percent of teachers said that cost of digital games was the primary obstacle to integrating them into class, and only 17 percent of those surveyed said the school spent $100 or more on games.

To that end, Childress said there are a number of free resources available on the web, and that the foundation has funded 17 game development projects over the last three years, a number of which are free or available at reduced cost to districts serving students in low-income neighborhoods.

*The updated version of the article includes information about the studies from 2011 and 2012.

Related

Explore: , ,

  • Email Post
  • deserteacher

    One activity does not an effective student make. We need to address the needs and functions of the whole student. But a dose of fun attached to academic endeavor invigorates the school day as well as reinforces important response skills.

  • Ellen Cavanaugh

    Kids, too, are asking these questions. Baden Academy 3rd grade research fellow, Owen R-K, ran an experiment this year. He went into 2nd grade classrooms, one with a game he programmed in Scratch to include superheroes, soccer balls, and choices made by the class. The control group, playing games not customized to them, slightly improved from pretest to post-test scores. The classroom that played the customized game had significant gains. Implications that he summarized at the recent Imagination Celebration include the need for teachers to learn some basic coding to modify games for their classrooms. http://www.ellencavanaugh.com/2013/06/15/the-fellowship-of-the-media-lab-the-dawn-of-imagination/

  • Geri Caruso

    We can leave the question of research about technology done by the Gates Foundation…! out but.. the question here is the whole child. It is evident that some skills can certainly be learned using games… Surprise… not a new concept, teachers have always used some kind of game to teach specific skills. The issue is the difficulty kids have leaving the video games and coming back to the real world. The games are addictive and hypnotic in a way that “Scrabble” is not. It can be somewhat controlled in school by time, but this is the new one eyed babysitter for tired parents at home. It would be the rare parent that hasn’t had to struggle with getting their children to put the games down, no matter how wonderful they are, and interact with people.

  • Alex Sarlin

    This is exciting and wonderful news for teachers and students alike, not to mention those of us who have been championing the use of games in educational settings. It’s not every day that a medium that happens to be engaging and enjoyable turns out to also be effective and educational- it’s a win-win.

    For those concerned that games will make kids antisocial or “hypnotized,” I’d like to point out that these concerns have been raised for every new communications medium in the last 100 years- teens talk on the phone too much instead of being with their families, watch too much TV, text too much, read too much instead of playing with friends, spend too much time on social networks, listen to too much music, watch too much Youtube, etc., etc.

    Every parent is free to set limits, and moderation is always best, but knee-jerk reactions can shut down the conversation about the incredible power of all of these media to benefit children. I mean, we’re talking about STEM games and simulations here- how can those be bad!? Moreover, true media literacy actually prepares children for their adult lives, and for all the even newer media opportunities that will inevitably emerge. Let’s be future-oriented, shall we?

    • seejay james

      “these concerns have been raised for every new communications medium in the last 100 years- teens talk on the phone too much instead of being with their families, watch too much TV, text too much, read too much instead of playing with friends, spend too much time on social networks, listen to too much music, watch too much Youtube, etc., etc.”

      erm…I would say that all these are true (except reading too much), and a lot of these activities ARE in fact antisocial and hypnotizing. How much is certainly a question, but I think the amounts done today are far too much, and increasing all the time. See how long a typical teen or college student can last without their smartphone…maybe an hour? So there is legitimate concern here. I agree, some of these activities can be beneficial (though not always for “learning” per se), so they shouldn’t be discounted outright.

      That said, I’m a huge believer in the potential of games for education, an avid player of all sorts of them, and a developer as well. How they’re used is key. No game will suit everyone. Lots of really really fun games have little to no educational content, and take many hours away from kids who get hooked and don’t have restrictions. Other games might be rich with learning content but kids don’t like them.

      One comment in the story which bugged me: comparing the online math game to sitting in a circle with a teacher, and claiming “There is no comparison”, as though the game was far inferior in every way. That is simply not true. They each have benefits. What about kids that don’t like working in groups all the time? Who’d rather try things on their own, at their own pace? Can make mistakes and not feel embarrassed? Maybe the math game has better explanations and real-time feedback than the teacher does. Maybe it moves faster when needed, and slower when needed…not always possible with a teacher, and certainly not in a group. The game is most likely also FREE, and teachers cost money. So there are plenty of reasons why BOTH ways of learning are perfectly valid…learning has a lot of facets, and each of these scenarios covers some very well and others very poorly.

      Anyway, great stuff in general. I hope we get to some sort of happy medium eventually… ;)

  • Pingback: Technapex Exciting EdTech Startups, K-12 Gamification: Today’s EdTech News | Technapex

  • Pingback: Mr G's Idle Musings » Blog Archive » My Diigo 06/20/2013

  • Courtorr

    I think that technology is a good way to get and keep kids attention while they are learning. the best part is, they do not even know that are learning. I am glad that there is research to back up the idea of educational gaming. Games are great way to excite students and help guide learning. Parents can also get involved with learning through educational games. I hope to hear more!

  • Pingback: Update: Diigo in Education group (weekly) | ChalkTech

  • Danny Fain

    This article gives a pretty good “overview from 10,000 feet up”, but the title is a bit misleading, and the SRI report itself is much more nuanced and complex. If you have the time to dig a little deeper, I recommend perusing the executive summary, http://www.sri.com/sites/default/files/brochures/digital-games-for-learning-exec-summ.pdf, especially the “Meta-analysis Comparisons” on pages 10-11 (pages 14-15 of the pdf document).

    Here’s the crux:

    Testing hinges on what you measure, how you measure it, and how those relate to desired outcomes. When an article like this one refers to “test scores”, readers usually think of traditional standardized school tests, such as the SAT and high-stakes (graduation-requirement) tests. The limited usefulness (and myriad problems) of those kinds of tests has been acknowledged by a growing number of employers and higher-ed institutions. As far as I can tell from the exec summary, only some of the studies examined in the SRI report relied on those sorts of tests, and that’s a good thing: there are many desirable learning outcomes which cannot be measured by those sorts of tests.

    Unfortunately, the jury is still out (according to the SRI report) on whether game-based learning is effective in promoting these other kinds of learning outcomes:

    “Findings from these studies indicated that digital games were associated with significantly better cognitive competency outcomes among students relative to the other instruction comparison conditions. These beneficial effects were primarily based on knowledge outcome measures rather than cognitive processes/strategies outcome measures, of which there were fewer, or creativity outcome measures, of which there were none.”

    Furthermore, the analysis shows substantial differences in effects between various categories of game-based (or game-like) interventions, which really shouldn’t be that surprising. If someone were researching whether “books can help raise students’ test scores”, wouldn’t you naturally expect such complexity?

    As a veteran classroom teacher and outspoken advocate for game-based learning, I have high hopes that more research and better measurement tools will reveal the benefits of good GBL in fostering a gamut of learning. But let’s resist the urge to oversimplify the subject; and let’s try to focus more on outcomes (and expressions) of value in the 21st century.

  • Danny Fain

    Regarding the “hypnotism” and “anti-social” concerns expressed by some commenters: watching an Oscar-caliber film, or listening to a symphony performance, can be entrancing and solitary experiences, but I’ve yet to hear anyone criticize them as too hypnotic or anti-social for students (particularly teens). I think that’s because people understand these experiences can be springboards for learning and social interaction. Well-designed game experiences are even more so, as the player may interact with others both within and around the game (part of what James Paul Gee calls the “Game with a big G”).

    Geri’s concern about parents struggling to get a kid to “put the game down … and interact with people” is valid (I’ve run into that problem with my 10 year-old son), but it speaks more to the concept of enjoyment in moderation and setting appropriate boundaries. Besides, engaging and educational games don’t all have to be digital; some of my (and my kids’) favorite games are multiplayer card & board games, for which we’re often willing to put down the single-player computer game. Maybe teens just need the right incentives …

  • Pingback: » Pograj sobie. Najlepiej w szkole Czytajnik.pl

  • Pingback: Can Digital Games Boost Student STEM scores? | The EdLib Report

  • Pingback: Can Digital Games Boost Students’ Test Scores? | Fluency21 – Committed Sardine Blog

  • Oksana Deinak

    Three things:
    1- Many of the school districts that may need technology the most, are also those that can’t afford it. Thus, the achievement gap perseveres.
    2- I would like to see digital games for foreign languages that expand beyond the usual memorization of vocabulary or the fill in the blank grammar practice. How about some simulations with the possibility for students to have conversations.

    3- Would like to see more research and testing from organizations that are not financially involved in proving the unbound efficacy of technology for education.

  • Maya Winchester

    I’ve found that my students scores have improved since I’ve let them learn through online and other educational games. I did a baseline assessment to see the level at which they were at and then introduced them to this DreamWorks game – http://www.schoolofdragons.com . I read about it somewhere online and was curious to see if they picked up the science that is embedded in the game. Not only had the kids concentration improved, they scored much higher and retained information longer. But this is not true of all students so I wonder where you can strike a balance between technology and learning in the classroom.

  • Pingback: Blogging and Education | MDST 485 Class Blog

  • Pingback: How Digital Learning is Boosting Achievement - Getting Smart by Tom Vander Ark - blended learning, Online Learning | Getting Smart

  • Pingback: Can Students Learn Better with Digital Games? | InKleined-To-Teach

  • Pingback: Game Developers Experiment With More Open-Ended Apps | MindShift

  • Pingback: Games In The Classroom: What the Research Says | MindShift

  • Pingback: 6 Tips for Finding Great Educational Apps for Kids | Your high education waits for you!