What Are the Risks in Using Data to Predict Student Outcome?
“Is it good to tell a first-grader, ‘You might be a dropout?’”
The obvious answer would seem to be: Uh, no. But when Thomas C. West posed this question recently to Education Week reporter Sarah D. Sparks, he had a genuine dilemma in mind. West, who is an evaluation specialist at Montgomery County Public Schools in Maryland, has devised a tracking formula that can predict, with startling accuracy, which students will drop out of high school—as early as their second semester of first grade.
The predictive factors themselves—behavior problems, frequent absences from school, reading skills that are below grade level—are not so surprising. What is significant about West’s formula is the larger trend it represents, and the practical and ethical issues raised by that trend. Thanks to widespread automation and digitization, we now have access to more information, gathered at ever-earlier stages, about individuals’ performance at school and at work. While once it took many months or even years to compile a track record that could support predictions about the future, today we can glean hints of how people are doing much sooner. Clever tools can even allow us to measure and monitor our own progress. Newly awash in data, the question becomes: What do we do with this information?
There is a danger, of course, that people who struggle early on will be written off too soon, before they’ve had a chance to prove themselves. But ignoring these super-early warning signs also carries risks. That’s because small initial differences have a way of snowballing into bigger ones over time. Here’s how one common scenario plays out:
Some third-grade students are reading a little less well than the rest of their classmates. The grade is important here, because third grade is the year that students move from learning to read—decoding words using their knowledge of the alphabet—to “reading to learn.” The books children are expected to master are no longer simple primers but fact-filled texts on the solar system, Native Americans, the Civil War. Kids who haven’t made the leap to fast, fluent reading begin at this moment to fall further behind.
Difficulties in third grade lead to the “fourth-grade slump,” as the reading-to-learn model comes to dominate instruction. While their more skilled classmates are amassing knowledge and learning new words from context, the less-adept readers begin to avoid reading out of frustration. A vicious cycle sets in: school assignments increasingly require background knowledge and familiarity with “book words” (literary, abstract, and technical terms)—competencies that are themselves acquired through reading. Meanwhile, classes in science, social studies, history and even math come to rely more and more on textual analysis, so that the struggling readers begin to lag in these subjects as well. What began as a small gap has widened into a chasm.
In operation here is what researchers call the “Matthew effect,” after the Bible verse found in the Gospel of Matthew: “For whosoever hath, to him shall be given, and he shall have more abundance: but whosoever hath not, from him shall be taken away even that he hath.” In other words, the academically- (and professionally-) rich get even richer, and the poor get poorer, as minor differences in ability grow into major ones. But the Matthew effect has an important upside: well-timed interventions can reverse its direction, turning a vicious cycle into a virtuous one.
The availability of very early indicators of performance puts a whole new spin on the Matthew effect: teachers and employers can use these indicators to address trouble spots before the student or employee ever has a chance to fall seriously behind. This principle applies not only to intervening at early points, but also at subsequent “pivot points,” to borrow the phrase of Donald J. Hernandez, a professor of sociology at CUNY-Hunter College who has studied predictors of academic success and failure.
For students, these crucial junctures include the transition from elementary school to middle school and the one from middle school to high school, as well as the period covering senior year of high school and freshman year of college (and don’t forget the summer in between, which I wrote about here). For workers, key pivot points are the first months of starting a new job, and of assuming new responsibilities following a promotion or reorganization.
No, we shouldn’t tell first-graders—or older students, or employees—that they might be failures one day. But we also shouldn’t wait to help them avoid that fate.