Pamela Snow: The Dog Ate My Homework: Education’s Evidence-Excuses Echo Chamber

Editor’s Note: Since launching The Next 30 Years, I’ve opened its pages to outside voices whose work sharpens, challenges, and clarifies our thinking about education. Few have done so more consistently or forcefully than Pamela Snow, a professor of cognitive psychology at La Trobe University whose research focuses on language, literacy, oral language development, and the science of reading. She is one of Australia's most prominent advocates for evidence-based education practice.

Last year, I republished her essay on the corrosive habit of blaming parents for educational shortcomings. This time, she turns her attention to another vexing and enduring feature of the education landscape: our remarkable capacity to generate excuses for ignoring evidence. Drawing on decades of examples, Snow examines the familiar refrains—"every child is different," "experience matters more than research," "evidence threatens teacher autonomy"—that allow ineffective practices to persist long after the evidence against them has become clear. Pam’s argument is not that professional judgment doesn't matter. Rather, she asks why education so often grants itself exemptions from the standards of evidence we readily expect in medicine, aviation, and other serious professions. It's a timely and provocative essay, particularly as debates over reading instruction, curriculum, and classroom practice continue to expose the gap between what we know and what we do – RP

I am not the first to write about the fact that education has a fraught and not very pretty history when it comes to generating, critiquing, and applying rigorous evidence in order to maximize student academic and well-being outcomes. Examples of this commentary can be found here, here and here. Way back in 1996, as part of the Teacher Training Agency Annual Lecture series in London, David Hargreaves observed that “Teaching is not at present a research-based profession. I have no doubt that if it were, teaching would be more effective and more satisfying.”

In 2000, Dr. Louisa Moats commented that “Unfortunately, lack of rigor and respect for evidence in reading education are reinforced by the passivity of education leaders who feel that any idea that can muster a vigorous advocate is legitimate and deserves to be aired.”

Dr. Moats was discussing reading instruction, but her words are no less applicable to other curriculum areas, most notably mathematics instruction.

Of course, you can see my bias right there in my opening paragraph: I believe that education exists to maximize student outcomes, academically and psychosocially. That is, for me, a strong ethical and civic responsibility for those who work in education, across the university, policy, research, and classroom practice levels. And yes, I am of the view that transmission of knowledge, as well teaching students how to apply said knowledge, is the core business of schools. For some, that immediately casts me as “neoliberal” in my political views. I’m not now, nor have I ever been, a member of any political party, but if my position that accountability matters as much in education as it does in other professional fields makes me neoliberal, I’ll take that over being what I’ve termed previously: paleoprogressive.

On social media platforms, I see some astonishing approaches to evidence. Sadly, these approaches often come from other education academics, but they also come from education consultants whose steady-as-she-goes operating models cleverly create illusions of change while simultaneously minimizing it (often with a great deal of fee-for-service “busy work” in the middle).

Meanwhile, classroom teachers are making it their business to learn the language of evidence, and often do so in their own time via teacher-led platforms, such as ResearchED, around the globe. Others engage in further study or join closed communities of practice on platforms such as Facebook. They are not waiting for those in the academy to take the lead on prioritizing evidence-based practice, and sadly, that’s probably just as well. I have blogged previously about the fact that much change is occurring on the metaphorical forest floor (classrooms) while the tall trees, whose branches thrive in rarefied air (education academics) are somewhere between oblivious and hostile to these efforts.

“Evidence-based medicine” is a term that entered clinical parlance as recently as the 1990s, though it had antecedents that go back much further than that, such as the US Flexner Report in the early 20th century. Critically, it was a call to medical practitioners (and rapidly, by extension, all health professionals) to move beyond blind and often uncritical faith in practitioner judgement, beliefs, and established practice. The report argued that independent research might offer different, more effective, and sometimes more efficient approaches to patient care.

Evidence-based practice in health is not about:

• So-called “cookbook” practice (aka “one-size fits all” approaches)

• Practitioners blindly following research findings

• Replacing or overriding professional judgement

• Ignoring the context of the individual

• Pretending that endless resources are available

Medicine, like education, is an interface of science, craft, and the complex vagaries of human nature and human behavior. Claims by some in education that “evidence-based practice is easy in medicine; you just do a clinical trial and find the answer” betray a phenomenal ignorance of the psychological, sociocultural, cognitive, linguistic, and economic factors that introduce enormous variability into clinical trials and have to be accounted for in research analysis.

Education is not special. Like health, it deals with human beings in all their complex and unpredictable glory. Like health, too, its practitioners have ethical obligations to a) do no harm and b) deliver effective services that improve the human condition. The difference, however, lies in the misconceptions and misrepresentations of research that are perpetuated in university lecture theaters and by extension in classrooms around the globe.

Education has enabled excessive permeability on ideas and practices that have little to no empirical evidence to support them (e.g., learning styles, left-brain-right-brain learners, Brain Gym, whole language, colored lenses for struggling readers, and more; see here) while resisting approaches that do have substantial evidence to support them (explicit reading instruction being a case in point).

I’ve compiled a list below of the arguments that I see most commonly on social media and hear from teachers who are grappling with change at their schools and across the sector.

These arguments all amount to poorly conceptualized excuses as to why education should be given an evidence leave-pass. They should not go unchecked.

Argument 1: Teaching is too complex to be studied scientifically.

Complexity does not take systematic investigation off the table.

Researchers in medicine, allied health, economics, psychology, nursing, criminology, and public health all study highly complex humans and human systems. They do so using research methodologies and tools that are appropriate for the questions being asked. They then triangulate findings across research methodologies and findings to determine the best direction of travel, based on current evidence, knowing that their approach will need to change as the evidence evolves. This is not the same as “choose your own adventure.”

Research in education is not about achieving “perfect certainty” —that’s a straw man argument. Rather, it should seek to identify practices that are more likely than alternatives to improve outcomes for the majority of students.

Education research, like health research, should allow practitioners to play the probabilities in measured, careful ways.

Argument 2: Every child is different, so findings based on aggregated data are meaningless.

Children are individuals, yes, as are patients. Nevertheless, there are more similarities than there are differences between humans with respect to how they learn. (See for reference, the work of Dan Willingham and Stanislas Dehaene.)

Human variation exists in every field where evidence is used; this is not unique to education.

Research identifies tendencies and probabilities within and across populations. In so doing, it is drawing on amounts of data that extend way beyond the bounds of individual practitioner experience.

The late Daniel Kahneman, author of Thinking, Fast and Slow reminded us that “The confidence that individuals have in their beliefs depends mostly on the quality of the story they can tell about what they see, even if they see little.”

Argument 3: Experience is more important than research evidence.

Experience is of course valuable, but it can also reinforce misconceptions and create blind spots when it comes to new evidence, especially if it challenges existing beliefs and practices.

People are often poor judges of cause and effect, partly because they do not have a clean line of sight of all of the factors in play. This is one reason that we are all prone to various forms of cognitive bias (see here and here): Mental heuristics have a place in everyday thought processes, but they need to be kept in check to allow us to verify assumptions and revise beliefs and practices in response to new evidence, even when doing so creates cognitive dissonance or ideological discomfort.

Many educational practices that “felt” effective to teachers for a number of decades were later shown to have little or inadequate impact on student learning. Emily Hanford’s award-winning Sold a Story podcast laid this bare in relation to the pernicious harm done to tens of thousands of children, around the globe, by balanced literacy and its antecedent, whole language. Teachers used these approaches in good faith; they had not been presented with alternatives. In many cases, when alternatives are made available, teachers seize them with enthusiasm.

Argument 4: Evidence-based practice threatens teacher professionalism and autonomy.

Professions that we hold in high esteem (e.g., medicine, aviation) actually have very low levels of autonomy, and high levels of accountability. I have blogged about this aspect of professionalism previously.

There is a double-standard in the fact that folks in education expect evidence-based decision-making when they are the patient, client, or passenger in a transaction but don’t see that the rest of the community expects the same rights as “consumers” of education.

What threatens teacher professionalism (and workforce longevity) is teachers’ creeping realization that they have not been adequately equipped for classroom practice—that they have not been given effective approaches for positive behavior support and instructional success.

Evidence-based practice does not inherently require pre-prepared lesson plans, but it’s difficult to understand opposition to practices that reduce variability and promote consistency within and between classrooms. High variability and high quality cannot coexist. Schools and sectors need to choose one.

In most professional fields, expertise involves integrating evidence with professional judgement. This can occur when teachers are well-equipped by their initial teacher preparation to critique and use research evidence.

The application of evidence does not remove the need for teacher decision-making. It enhances and refines teacher judgement and allows teachers to apply their pattern-recognition skills in decisions across the day.

Classroom practices should be dictated by the best interests of the students, not the personal preferences or ideologies of the teacher. Think about how this principle plays out in medicine, nursing, and aviation.

When teacher unions (mis)play the autonomy card, they are working against the best interests of their members.

Argument 5: Randomized controlled trials are not applicable or ethical in education.

Randomized controlled trials (RCTs) are sometimes described as the “gold standard” for establishing cause and effect with respect to whether an intervention works. I won’t provide a tutorial on RCTs here since readers can easily learn more from online sources. Suffice to say that when they are well-designed, RCTs give us the greatest clarity about the efficacy of a teaching or intervention approach, because they control for the interference of many variables that can sit between cause (the teaching or intervention) and effect (student outcomes).

It is important to distinguish between efficacy (how well an approach works under well-controlled experimental conditions) and effectiveness (how well it performs “in the wild” of the real world). If an RCT supports efficacy, then the next step is an effectiveness trial.

A common misconception is that when RCTs are done in schools, some students receive “something” while those in the control arm receive “nothing.” This is incorrect. Those in the control arm may receive business-as-usual teaching, or they may receive another intervention that is considered worthy of testing. In some cases, there is a cross-over design, where students are shifted from one arm of the study to the other, and so are exposed to more than one approach.

RCTs, like all research carried out in schools, are subject to rigorous review by human research ethics committees.

For a fascinating account of how RCTs have transformed our world across health, criminology, education, and economics, have a read of Andrew Leigh’s Randomistas.

Schools are no more or less complex as research environments than hospitals, community health settings, prisons, or rehabilitation facilities. All must grapple with the vagaries of human nature and variation in all its guises.

Argument 6: Research changes all the time, so teachers can’t rely on it.

Revision in light of new evidence is a strength of science, not a weakness. Science is by definition self-correcting. The story of stomach ulcers is a good example of this. While once thought to be due to stress, groundbreaking and paradigm-shifting research by two Australian investigators, Barry J. Marshall and J. Robin Warren, showed that bacterial infection, not stress, was the culprit. As a result, management of ulcers was overhauled at scale via revised clinical care pathways that all practitioners were expected to adopt. The practitioners did so because that was their ethical responsibility.

The fact that some findings are modified or evolve over time does not mean that all findings are equally uncertain. It means that we need to be open to calibration and, in some cases, de-implementation of current practices.

Argument 7: It’s insulting to talk about a “hierarchy of evidence.” All methodologies are equally valid.

This is an unsophisticated play for egalitarianism in the face of both common sense and Research Methods 101 principles.

Research methodologies should be primarily selected for their goodness-of-fit with the question(s) under investigation. In many cases, researchers are faced with choices. For example, if they are asking “What is the lived experience of teachers who have been abused and bullied by dissatisfied parents?”, they may elect to carry out qualitative, in-depth interviews, or they may elect to construct a survey that is framed in such a way that responses can be quantified (e.g., via Likert-type scales); or they may do a mix of both. In all cases, their study scope will be constrained by resource issues, and we cannot escape the reality that a large, representative survey sample will have higher validity and generalizability than will a small, qualitative study of say, six to eight teachers’ experiences. This does not “dismiss” the smaller group of teachers; it just means that one is going to add more weight to the evidence scales than the other.

Argument 8: Medicine is a so-called evidence-based discipline, and errors still occur in that field.

This argument is reminiscent of the school-yard defense that “other people were throwing stones too, so why am I getting into trouble?”

Yes, errors, malpractice, and snake-oil interventions do occur in medicine (and other health fields), but that is no reason for education or any other discipline not to apply the highest standards of research rigor and accountability.

Disrespect for the scientific method (by which I mean all methodologies, applied rigorously in ways that are appropriate for the questions under investigation) ultimately leads away from improvements in the human condition.

Argument 9: There’s conflicting evidence about X/Y/Z, so in the meantime, teachers should just make up their own minds.

When we think about evidence, it’s most helpful to think of a set of scales and ask where the current weight of evidence sits. We should not overlook the fact that some studies seem to be outliers, but we also can’t wait for every question about every practice to be answered. Staying curious and open-minded while getting on with it in the meantime is what happens in schools that roll up their sleeves and go on change journeys towards improved practice. See examples here and here.

Argument 10: Quantitative research reduces complex phenomena in ways that make them overly simplistic. Just because you can count something doesn’t mean it counts.

Like some other arguments in this debate, there is some truth to this claim, but it is not the whole story. Yes, quantitative tools (like surveys that use numerical rating scales) by necessity represent something complex (like anxiety level) as a number. This does not mean, however, that the data is somehow meaningless. If, on a five-point scale, I tell the researchers that my anxiety level is “5,” that is a meaningful representation of my subjective experience. I may also find it easier to circle a 5 on an anonymous survey than to take part in an in-depth research interview. Large-scale quantitative studies often provide persuasive arguments to funders and policymakers because they enable researchers to say things like “55 percent of Year 7 students experience anxiety associated with poor reading skills.”

Argument 11: Education has a history of fads and fashions, and evidence-based practice is just another one. It too shall pass.

Sadly, it is true that education’s history is one of fads and fashions, so teachers who have been around for a few decades do see many approaches come, go, and sometimes come again, slightly dressed up but not heavily enough disguised that they are not recognizable. It’s one thing to experience change fatigue when it leads to improvement, but change fatigue when the only outcome is change is understandably frustrating and disheartening for teachers.

This frustration is no tribute to education as a discipline and speaks to the fact that while other disciplines have moved forward over the last 50 years, education has changed, but it has not necessarily progressed. This cycling and recycling approach to practice in education was summed up by Hewitt and Sachdeva in 2025 this way:

Education…lacks this cycle of progressive improvement. In spite of decades of reform and billions spent on improvement initiatives, it’s not clear that teachers are any more effective today than they were fifty years ago. While the curriculum has evolved over that period, it would be difficult to claim that the quality of education has significantly improved or that the gap in student outcomes has narrowed substantially.

Argument 12: Evidence-based practice is just a political ideology.

This is essentially an ad hominem argument and flags a reluctance to engage with substance.

While stakeholders may have ideological or political alignments (as is their right), the central claim—that educational decisions should be informed by the best available evidence—should not be inherently ideological (or political). We’re in big trouble if it is.

Criticizing those who argue in favor of education’s better engagement in the generation, critiquing and application of evidence does not address the quality of the evidence itself. It’s just an attempt at deflection.

Pleasingly, as noted above, we’re seeing schools generate practice-based evidence that complements the body of scientific evidence about how children learn and optimal ways to teach them. See here and here. This is powerful evidence in its own right and is gaining a seat at the policy decision-making table. It also needs to start re-focusing debates among education faculties.

Argument 13: If there’s a paper to support an approach in a peer-reviewed journal, it must be evidence-based.

This is lazy reasoning and betrays ignorance of the academic publication food-chain in practice. Some journals only publish papers that adhere to rigorous methodological standards, and as a consequence, reject 90 percent or more of the submissions they receive. Others are less fussy about methodological rigor and accept a wider array of manuscripts, inevitably of variable quality.

So no, being able to locate a peer-reviewed paper that says the earth is flat does not, in fact, mean that if you walk out your back door and keep walking, you will eventually fall off the edge.

Argument 14: Facts are malleable. It’s a cultural imposition to say that “2+2=4.”

I know, I know, I know, but I saw an example of this claim on social media just recently. I’ll just make the observation here that I’m pretty sure the academic who made this claim would not accept this position from their payroll department if a previously agreed-upon salary was then reduced by their employer.

Argument 15: Approach XYZ was tried and failed.

Implementation science is a complex field, and claims that something was tried or implemented in a school or system often fall well short of what is required to create replicable and testable conditions that can shed light on whether something is:

• Likely to have benefits, and if so, for whom

• Likely to maintain the status quo, and if so, for whom

• Likely to worsen outcomes, and if so, for whom.

Fidelity of implementation is no trivial matter, but it is often neglected in policy shifts at the sector level.

One of my greatest concerns, as Australia transitions to more explicit teaching, is that implementation will naturally be uneven, and we will have poor perspective and patience. This will result in all sorts of unsubstantiated claims about what did and did not work, and for whom.

Argument 16: Children’s academic achievement reflects their parents’ socioeconomic status, and no research will change that.

This argument is a particular heart-sink for me because it betrays an underlying belief that schools and the teachers who work in them are impotent; that teachers’ everyday instructional practices don’t matter, because a child’s socioeconomic status at birth is their destiny.

Yes, family socioeconomic status is a strong predictor of student academic achievement, but shouldn’t we be disrupting that trajectory through high-quality, impactful teaching and support? Fortunately, in many cases we are. My colleagues and I have addressed this argument here as well.

***

I could go on, but I hope that this list gives readers a sense of the unproductive and often unsophisticated arguments that plague education. These arguments revolve around the idea that education does not need to hold itself to the same standards of evidence as other disciplines.

In health, we have the Therapeutic Goods Administration in Australia (and cognate bodies in other industrialized nations) to systematically test the safety and efficacy of medicines, devices, and other therapeutic goods before they can be prescribed or purchased. Imagine the chaos, harm, and exploitation that would rain down on us if we were not the beneficiaries of such protection.

Where is this product and practice protection in education?

Teachers cannot be gatekeepers in

• The adoption of practices that are consistent with current scientific evidence

• The de-implementation of superseded approaches

• The identification and avoidance of pseudoscientific approaches

Unfortunately, leaders (academics and policymakers) in the education profession do not subscribe to the scientific method as it’s used in other fields of human endeavor.

I repeat, education is not special. But education is, ironically, a slow learner. It would seem of the maxim that when you stand for nothing, you fall for everything.