More Test-Based Accountability? It's Complicated

One popular explanation for declining test scores has been, basically, “because Covid.” But that analysis has been undercut by recent data underlining the fact that the decline has been going on since at least 2015, well before the pandemic hit.

That evidence has fueled another narrative, based on something that changed in 2015: the federal No Child Left Behind Act, which held schools accountable for raising reading and math scores—sometimes in draconian ways—was replaced by the Every Student Succeeds Act, or ESSA, which backed off from that approach.

Democratic presidential hopeful Rahm Emanuel has called for a return to test-based accountability, arguing that the retreat from it is at least partly responsible for the decline in test scores. Some education commentators have also pointed to the end of NCLB as a factor.

But there are also problems with the “because ESSA” explanation. It’s true that test scores rose under NCLB, but most of that increase occurred during the early years of the regime. Progress basically stalled after 2008, the year of the Great Recession—and seven years before NCLB was replaced by ESSA. So another narrative is that changing socioeconomic conditions had more to do with changing test scores than NCLB did.

But when it comes to reading scores, even those narratives appear overly optimistic. Yes, math scores did improve under NCLB, for a while. But in 2018, a panel of reading experts convened by the governing board of the NAEP—the national tests in reading and math that are considered the most reliable barometers of progress—argued that reading scores had been essentially stagnant since 1998. Since 2018, they’ve only gotten worse.

True, ethnic and racial test-score gaps narrowed to some extent under NCLB, but the gap between students at the upper and lower ends of the socioeconomic spectrum didn’t budge. And gains in elementary school scores failed to carry over to middle and high school.

None of this is surprising. It’s almost always easier to raise scores at lower grade levels, when the concepts and knowledge being tested are less complex. And it’s generally easier to raise math scores than reading scores. The usual explanation is that math is something kids learn at school, whereas reading skills are more dependent on what happens outside of school. That’s why kids from wealthier, more highly educated families generally do better on reading tests.

That’s true, but it doesn’t mean schools are powerless to boost reading outcomes in the face of societal factors. In fact, there’s much more that schools could do to improve students’ reading ability and to narrow those large and stubborn socioeconomic test-score gaps. But accountability for reading test scores did not and will not nudge schools in the right direction. In fact, that kind of accountability has been a major obstacle to improvement—and doubling down on it will only make things worse.

There’s a fundamental difference between test-based accountability for math and for reading, and it’s one that accountability hawks overlook at their peril: Math tests, at least theoretically, test students on concepts and skills they’ve actually been taught.

That’s not the case for reading tests, which purport to test abstract reading comprehension skills rather than any particular content. As Robert Pondiscio has observed, standardized reading tests have led to “mindless test-prep and instruction in ‘reading strategies’”—strategies that don’t work in the absence of relevant knowledge.

Under that instructional regime, the kids who are better able to pick up knowledge outside of school will continue to do better on reading tests—not because they’re better at deploying “strategies” but primarily because they have a greater store of academic knowledge and vocabulary to draw on. Standardized reading tests, as cognitive psychologist Daniel Willingham has said, are really “knowledge tests in disguise.”

The focus on skills and strategies also explains why it’s so much harder to raise reading scores at higher grade levels. At lower levels, the passages on the tests are fairly simple, and students’ ability to just decipher the words plays a large role in their score. But as grade levels go up, the passages—which are designed to avoid topics kids might have learned about in school—assume that students are acquiring increasingly sophisticated knowledge and vocabulary.

If students don’t possess a minimum threshold of that knowledge, they never get a chance to demonstrate their ability to “make an inference,” or whatever the test purports to be measuring. And because knowledge acquisition is a gradual, cumulative process, by the time kids get to high school or even middle school, it may be too late to make up for all they’ve missed during an elementary school experience that was largely devoid of content.

Even more dismaying, it’s becoming clear that high scores on reading tests don’t necessarily equate to deep reading comprehension. For the last several years, college professors, including many at colleges that admit only applicants with top scores, have been reporting a marked decline in the ability of their students to navigate lengthy and/or complex text.

Don’t get me wrong: I’m not against standardized tests per se. When test scores are broken down by student subgroup, including by race and income level, they reveal inequities in the system. That’s true even for standardized reading tests—although it’s important to recognize that they primarily reveal differences in the distribution of knowledge, not abstract skills.

Without standardized tests, these inequities are likely to be hidden. States, districts, and some schools can tout overall average scores that mask large internal gaps. And students at low-performing schools, along with their families, can be lulled into a false sense of achievement. An A grade at one school doesn’t necessarily mean the same as an A at another.

Standardized tests also guard against the often unconscious bias that is almost inevitable when teachers evaluate their own students. In his book Reforming Lessons, England’s former Schools Minister Nick Gibb describes what happened there when the pandemic forced the cancellation of externally graded exams given to high school students. Because the exams determine graduation and college admission, they had to be given—and teachers had to do the grading. The average grade rose significantly, especially for the college admissions test, where the percentage of students receiving the highest grades almost doubled in the space of two years. When teachers personally know the students they’re evaluating, it’s understandable that they tend to be generous.

Gibb also cites a study that compared grades given by teachers to their own students in the equivalent of fifth grade against grades from an external evaluator. It found that teachers gave lower grades to Black students and higher grades to Asian students. They also tended to underrate boys in reading and girls in math.

Share

So we do need test-based accountability—just not accountability for things that students haven’t actually been taught. Educators might believe they’re teaching the skills measured by standardized reading comprehension tests, but it’s just not possible to teach—or measure—those skills in the abstract.

Intentionally or not, we’ve been using tests to pressure schools to waste time on “comprehension” instruction and marginalize subjects that could build the knowledge and vocabulary that actually fuel reading comprehension. To correct that course, we need to attach accountability to standardized tests covering curriculum content, not just in math but in subjects like literature, history, and science.1

The problem is that in almost all states, schools are using a variety of curricula—or, in the case of social studies, often no curriculum at all. One possible solution is to ground passages on reading tests in the topics specified in state social studies and science standards. Or states could just require schools to follow a specific curriculum.

Until recently, that idea seemed beyond the pale, given the strong tradition of local control over curriculum in the U.S. Some states have incentivized the use of certain curricula, but none has required schools to use a specific curriculum. As of Friday, however, the state of Texas appears to be on the road to bucking that tradition.

As I’ve noted before, Texas education officials have come up with an extensive required “reading list” in English language arts for grades K through twelve—a list that the state board of education has now endorsed, with a few modifications. The plan is for the list to become a statewide mandatory literacy curriculum, to which standardized reading tests will then be aligned.

Coverage of this move in the media has focused on controversy over the inclusion of Bible passages and a lack of cultural diversity in the texts on the list. But that overlooks an important part of the impetus for the initiative, which is to get away from the typical skills-focused approach to reading comprehension instruction and evaluation and towards one that focuses on building and assessing actual knowledge.

I have extremely mixed feelings about this development. I’ll be delighted, of course, if Texas—the state with the second-largest number of public school students in the country—abandons the flawed approach to comprehension that has held so many students back. And I agree with state officials when they say that Bible passages and works of classic literature are an important component of the cultural literacy students need in order to understand complex texts in a Western-dominated society. Texas is doing what I and other advocates for knowledge-building have urged: specifying content to be covered and attaching accountability to teaching that content.

But, as with its existing (non-mandatory) state-created literacy curriculum, Bluebonnet Learning, Texas officials appear to be crossing the line between building knowledge and advocating for a specific cultural and political perspective. In Bluebonnet, for example, some references to Bible stories treat them as received truth rather than literature.

And while there isn’t yet a curriculum attached to the state’s mandatory reading list, the inclusion or exclusion of certain supporting materials for anchor texts seems to reflect a particular political orientation. One supplemental reading for Shakespeare’s Julius Caesar is Margaret Thatcher’s eulogy for Ronald Reagan. Meanwhile, there are no supplemental texts on McCarthyism in connection with Arthur Miller’s play The Crucible, which is generally seen as an allegory about the anti-communism of that era.2

Those kinds of choices, along with resistance to any mandated curriculum, could well lead to pushback from school districts, especially those that would prefer not to get embroiled in controversies. One sign of potential trouble is that, at least as of a year ago, many Texas school districts were opting not to use Bluebonnet, despite financial incentives to do so. More broadly, Texas’s reading list could reinforce the mistaken perception that knowledge-building is an inherently conservative endeavor.

I can only hope there’s another state out there where officials will recognize the need to change standardized reading tests so they no longer warp instruction in ways that prevent students from reaching their full potential. Yes, choices have to be made when deciding which content should be taught and assessed. But they don’t have to be made in ways that only add fuel to distracting and unnecessary culture wars.

Wary as I am about what is going on in Texas, I’m even warier about a call for more accountability when it comes to reading test scores. Frankly, I doubt the relaxed accountability of ESSA has made much difference to reading comprehension instruction. The idea that comprehension should be taught as just a set of abstract skills and strategies is now so deeply ingrained that simply easing up on testing requirements isn’t enough to change that approach.

But attaching more accountability to reading test scores will only make it even harder to overcome entrenched practices, to the lasting detriment of millions of students. If we want to make real progress in improving education outcomes and narrowing test-score gaps, it’s crucial to add some much-needed nuance to the “accountability” conversation.