Why We Jump to Conclusions About What Works in Education

The most visible explanation is not always the most important one.

Have you ever noticed how quickly we create explanations for things we observe?

A school introduces a new teaching approach. A year later, student achievement improves. A company adopts a new workplace policy and profits increase. Someone starts taking a vitamin and their headaches disappear. In each case, our minds instinctively connect the two events. We naturally assume that because one thing happened before the other, it probably caused it.

It’s a deeply human tendency. We are pattern-seeking creatures, constantly looking for explanations that make sense of a complicated world. Most of the time, it serves us well. But it can also lead us astray. When many factors are changing at once—as they usually are in education—it becomes surprisingly difficult to know which ones are actually responsible for a particular outcome.

Educational debates are especially vulnerable to this kind of reasoning. Schools adopt new programs, instructional approaches, technologies, or policies, and when student outcomes improve, it is tempting to attribute the success to the most visible change. Sometimes that conclusion is correct. But often it is not. Before concluding that one factor caused an improvement, we need to ask what else may have contributed, and whether the available evidence actually supports the claim.

This is precisely why educational researchers rely on carefully designed studies rather than intuition alone. The goal of the Science of Learning is to identify the instructional practices that genuinely improve learning while avoiding conclusions based on compelling but potentially misleading observations.

A recent Globe and Mail article about mathematics achievement in the Halton District School Board provides a useful illustration of how easily people can jump to causal explanations, even when the evidence is more complicated.

On June 22, 2026, The Globe and Mail, a Canadian newspaper, ran a story headlined “Using a different approach, Halton District School Board succeeds in raising math scores.” The Halton District School Board is a large public board serving several communities just west of Toronto, Ontario — among them Oakville, Burlington, Milton, and Halton Hills — and it consistently ranks among the higher-performing boards in the province.

The article profiled a Grade 9 classroom at Milton District High School, where students cluster around whiteboards solving problems in small groups while the teacher circulates as a “guide on the side” rather than lecturing from the front. It ties this collaborative, active-learning approach to a striking result: 74 per cent of Halton’s Grade 9 students met the provincial Education Quality and Accountability Office (EQAO) math standard — well above the provincial average, and among the highest in Ontario. It credits a decade-long investment in teacher professional development and classroom design, and frames Halton as a possible model for the province as Ontario reviews its approach to standardized testing.

It’s a feel-good, appealing story, and the results are real. The Halton students deserve genuine congratulations, as do their teachers and school board. But it’s also an excellent example of how people can move quickly to causal explanations. Linking Halton’s success to a particular teaching method is not as clear-cut as the article suggests.

The central problem in the Globe’s article is the leap from correlation to causation. Student achievement is the product of many factors. Whenever we encounter a story linking educational success to a particular teaching method, there are several questions worth asking:

• Could other factors explain the results? Halton serves one of Ontario’s most affluent, well-educated populations. Parental education, household income, and access to private tutoring are well-known as some of the strongest predictors of mathematics achievement. Given these advantages, Halton’s strong mathematics performance is exactly what many researchers would predict (with or without the teaching method described in the article). Whether the teaching method also played a role is unclear.

• Has Halton been improving more than comparable school boards? The available evidence does not support the notion that Halton is improving at a faster rate (Education Quality and Accountability Office, 2025). In recent years, Halton’s mathematics scores have risen in lock-step with provincial averages. In Grade 9 mathematics, for example, Ontario’s percentage of students meeting the standard rose from 53 per cent in 2022-23 to 54 per cent in 2023-24, while Halton rose from 69 per cent to 70 per cent. The following year, Ontario increased from 54 per cent to 58 per cent while Halton increased from 70 per cent to 74 per cent. The Grade 3 and Grade 6 mathematics scores show similar patterns. If Halton’s professional development strategy were driving improvement, one would expect the board to pull away from the provincial trend. Instead, Halton’s gains closely mirror those of the province.

• Who actually wrote the assessment? Another factor we need to consider is which students actually sit to write the test. EQAO results are based on the students who participate, yet participation and exemption rates vary across school boards. If one board assesses a higher proportion of struggling students than another, comparisons of percentages become less straightforward. Participation rates are just one more reminder that a school board’s EQAO results reflect many influences beyond what happens in the classroom.

• Does the same approach produce similar outcomes elsewhere? One way to judge an instructional approach is to ask whether it produces similar results in other schools and school boards. The “guide-on-the-side” model described in the article is not unique to Halton. It closely resembles Building Thinking Classrooms (BTC), a framework adopted by many schools and school boards across Ontario. If this approach were responsible for Halton’s high scores, we might expect superior outcomes in all boards where it has been implemented. However, that’s not happening. Results vary widely, with scores rising in some boards and falling in others. That variation does not mean that the approach is ineffective. Rather, it illustrates why we shouldn’t jump to conclusions about the causes of success. Demonstrating that the approach itself causes higher mathematics achievement requires stronger evidence — for example, showing that otherwise similar students taught this way consistently outperform those taught using other approaches. Without evidence of this kind, we should not conclude that the approach led to Halton’s gains.

• Are we comparing realistic alternatives? The article also illustrates another common reasoning trap: treating complex educational questions as though they involve a choice between two competing philosophies. It contrasts an engaging, collaborative classroom led by a "guide on the side" with traditional "rote" instruction, implying that educators must choose between these alternatives. But effective mathematics instruction is not an either-or proposition. As the cognitive scientist Barak Rosenshine documents in his widely cited American Educator article, successful lessons typically combine multiple instructional approaches, including whole-class instruction, modelling, guided practice, questioning, and independent practice. Rather than asking whether teachers should lecture or facilitate, a more useful question is when and how different instructional strategies are most effective. Framing mathematics instruction as a contest between two opposing philosophies oversimplifies both classroom practice and the research evidence.

None of these points proves that the Globe's explanation is wrong. Rather, they illustrate why it is premature to conclude that the instructional approach alone caused Halton's success. Doing so ignores many other factors that are likely to have played a role.

This brings us back to our earlier point: sometimes we are deceived by our own intuitions. It is tempting to visit a successful school, observe an interesting practice, and assume it explains the board’s relatively high EQAO scores. Humans are naturally drawn to visible explanations. A classroom lesson is easy to observe; the cumulative effects of family background and prior knowledge are not. Yet those less visible influences often matter just as much, or more. Before concluding that one “different” teaching method explains a school board’s success, we should demand evidence capable of separating instructional effects from the many other influences on student achievement. Inferring causation from correlation is a particularly risky habit in education reporting, because the policy stakes are simply too high.

Educational success almost always has multiple causes. Student achievement is the product of a complex combination of prior knowledge, teacher expertise, curriculum, family and community factors, school resources, assessment practices, and countless other influences. Identifying which of these factors mattered—and by how much—requires careful research rather than intuition.

That is why evidence matters. Compelling stories are valuable because they engage our attention. They are not, by themselves, evidence of cause and effect.

The next time we read that a particular teaching method, technology, or policy transformed a school’s performance, we should treat such claims with cautious skepticism. Humans have a natural tendency to invent cause-and-effect stories to explain what they see. Moreover, we are prone to attribute academic success to highly visible things, such as a classroom lesson, while overlooking the multitude of factors we cannot see. Only careful research can distinguish the effects of a particular teaching method from the many other influences on student achievement.

Education needs more than inspiring stories. It needs careful thinking about evidence.

Share

Education Quality and Accountability Office. (2025, October). Highlights of provincial results: Grade 9 assessment of mathematics. https://www.eqao.com/wp-content/uploads/2025/10/highlights-provincial-results-2025-g9.pdf

McGinn, D. (2026, June 22). Using a different approach, Halton District School Board succeeds in raising math scores. Globe and Mail.

Rosenshine, B. (2012). Principles of instruction: Research-based strategies that all teachers should know. American Educator, 36(1), 12–19.