Could Video Games Measure Skills That Tests Can’t Capture?

| August 11, 2014
  • Email Post
A screenshot from the Posterlet game: choosing negative or positive feedback.

A screenshot from the Posterlet game: choosing negative or positive feedback.

Imagine you’re playing a computer game that asks you to design a poster for the school fair. You’re fiddling with fonts, changing background colors and deciding what activity to feature: Will a basketball toss appeal to more people than a pie bake-off?

Then, animal characters — maybe a panda or an ostrich — offer feedback on your design. You can choose whether to hear a compliment or a complaint: “The words are overlapping too much,” or, “I like that you put in the dates.”

You can use their critiques as guides to help you revise your poster. Finally, you get to see how many tickets your poster sold.

This little Web-based game isn’t just a game. It’s a test, too.

“In our assessments we make little fun games, and to do well at the games you need to learn something,” says Dan Schwartz, the director of the AAA Lab at Stanford University. “So they’re not just measures of what the student already knows, but attempts to measure whether they are prepared to continue learning when they’re no longer told exactly what to do.”

Schwartz is among a new breed of researchers who are applying the mechanics of games to the science of psychometrics — the measurement of the mind.

Right now, he’s working on a series of video games called Choicelets. They’re designed to evaluate students on factors that traditional tests can’t assess. He wants to measure how students learn, how they make decisions and how they respond to feedback.

A Test Or A Learning Encounter?

Most kids like video games — a lot more than they like taking tests.

But the purpose of making tests more like games isn’t just to add a spoonful of sugar to the medicine.

Scholars like James Paul Gee believe video games actually come much closer to capturing the learning process in action than traditional fill-in-the-bubble tests.

Gee, a professor of education at Arizona State University, is considered the godfather of game-based assessment.

“Is a video game a test or a learning encounter? It’s both,” he said. In fact, in a video game, “you’re always being tested — you can’t get out of a level until you finish it.”

Gee and other experts explain that classic video games, like Mario Brothers or Tetris, are designed to adapt as the player gets better. Each level moves faster, or presents harder obstacles.

The game tugs you along, held in a state of “flow” between anxiety (when things get too hard) and boredom (when they’re too easy).

And, the researchers point out, at the same time you’re playing a game, the game can record your actions. When it’s over, the software can create a report: not just a record of right and wrong answers, but all the steps you took to get there.

Schwartz’s theory of assessment focuses on choice. He argues that the ultimate goal of education is to create independent thinkers who make good decisions. And so we need assessments that test how students think, not what they happen to know at a given moment.

For example, the real point of the school-fair game, called Posterlet, is not to test how good students are at graphic design.

Instead, the crux of the game comes when students choose to hear comments on their work. Seeking negative feedback, it turns out, is the best way to improve the design of your posters quickly. It’s also a healthy strategy for doing well in school and in life.

In Photolet, another game from Schwartz’s lab, players pretend to take photos of animals. Then they edit, crop and filter the shots.

On the surface, the game is about photography skills like focus and composition. Underneath, it emphasizes another concept: making several versions of your work and then self-editing to choose the best.

One of Schwartz’s experiments took place recently at Hillview Middle School in Menlo Park, Calif. “We took over their sixth grade and taught design thinking for five weeks,” he says. Design thinking includes using empathy and fact-finding to understand people’s needs, generate prototype solutions to problems and test them out.

Schwartz’s study showed that students were able to apply the concepts they learned in social studies or science class to improve their performance on Posterlet or Photolet.

Larra Olson, who teaches humanities at Hillview, says her students loved playing the games. “They were completely engaged,” she said.

Olson said the project left her “totally open” to the potential of games as assessments, especially as an alternative to the traditional emphasis on grades and multiple-choice tests.

“I think you have to figure out the structure of the game so you’re not ‘leading the witness,’ ” she said. “But I think they’re very powerful.”

Copyright 2014 NPR. To see more, visit http://www.npr.org/.
Related

Explore: , ,

  • Email Post
  • Pingback: Could Video Games Measure Skills That Tests Can’t Capture? | MindShift | SCOE News Reader

  • Pingback: Could Video Games Measure Skills That Tests Can’t Capture? | Teachers Blog

  • Pingback: Could Video Games Measure Skills That Tests Can’t Capture? | MindShift : Adhoc Language

  • Pingback: Could Video Games Measure Skills That Tests Can’t Capture? | MindShift | The Sharing Tree

  • http://www.magicalmaths.org/ Magicalmaths.org

    Thinking about doing something similar in the UK. Taking some year 9 students or off lesson at the end of the year and teaching them design thinking for a couple of weeks. Any extra information would be great!

  • Pingback: All about education 08/13/2014 | Musings on education, teaching, lifelong learning and world events

  • Jay Powell

    Games can do much more than the list given here.

    Mind-shifts involves
    adding options to be considered. When insight closes the gap separating two or
    more options to form a synthesis, the person having the insight is creating
    their own new knowledge. The effect is to rewire his/her brain into a new
    configuration that did not exist before for that person and sometimes unique to
    humanity.

    Insight produces a rush of endorphins from the amygdala, producing a high that
    can become addicting, creating life-long learners and increasing human
    diversity

  • Vincent Mysliwiec

    Our project, Storyverse, uses gamification to help motivate learning and reading for young children by playing games expressed thru a 3D popup book. Kids feel like they are having fun and engaged while being the hero at the same time reading and immersing themselves further thru words. Check out our Kickstarter: kck.st/1q6rC0n

  • Pingback: How To Choose A Learning Game | MindShift

  • Pingback: Could Video Games Measure Skills That Tests Can’t Capture?

  • Pingback: Using Video Games for Assessments | Literacy Teaching and Teacher Education

  • Rene Grimes

    I am an elementary teacher; and I am a skeptic. Some might say cynic. I also have a MEd in Mind, Brain and Education. So when I attended a workshop for early number sense and heard about a new math app, I wasn’t about to incorporate it in my class until I could “test” it out. By “test” I mean research. Two years later, I am still researching this app! I can say, do say, unequivocally, YES digital games can measure at a much more fine-grained level than paper and pencil tests!!! I’ve seen this in well over 100 students. We should be researching this rather than “engagement.” But, that doesn’t mean any game can or does. These are highly complicated computer adaptive software designs. When the adaptive features are married with cognitive models of learning (yes, like Flow; and Perceptual Control Theory, Skill Theory…others) you get an instructional tool with embedded assessment;; what Valerie Shute from FSU calls “stealth assessment.” Carnegie Mellon has done quite a bit of design and research on this; as well as other labs. As for me, the math app I have used and research is what I consider my plumb-line for instructional apps AND for assessment. It may only be for early number sense…for now…but it’s hard to look at other apps now and settle for less. Don’t for a minute take my word for this; that kind of thinking causes all kinds of trouble when adopting instructional materials in education. Try it yourself, it’s free…and if you want to give me feedback, I’d love to hear it!!! (I am NOT in any way associated with the developers; so this is purely one teacher shouting out to the world to take notice of what can happen in technology for education!) http://www.nativebrain.com and Native Numbers at iTunes app store!

    BTW…my latest thinking is that we can validate these types of game/assessments using Bayesian Knowledge Tracing…but I am NO statistician…I HATE statistics! Yet, it is going to take a different method to validate these digital tools because the only way we have to get convergent validity is by using…paper and pencil assessments…which brings us back full circle. Someone get on this will you, and let me know what you find! Feel free to contact me @MrsG2nd

  • Pingback: Could Video Games Measure Skills That Tests Can’t Capture? | Profil primaire