Retconning the Curriculum: Why The Science of Learning Has a Serious Design Problem

When I was a kid, there were several media events which ironically brought society together in a way you almost never see today with social media. The release of certain albums or TV shows were a kind of simultaneous human experience which we have seemingly lost. In the 80s and pre-Internet 90s, my generation would go to record stores to buy albums on the day of release and read them cover to cover on the bus home before ever hearing them, or come into school the next day excitedly talking about a TV episode the night before, recreating scenes and acting out various lines to prove our having borne witness to the event, thus cementing our social affiliation with the group.

I can vividly remember my parents and their friends talking about one particular episode of show called Dallas and a cliffhanger where a key character, Bobby Ewing had been killed and properly killed, funeral and all, then in the first episode of the next season, stepped out of the shower and the shocking death attributed to his wife’s dream. All 30 odd episodes of the last season, the time and investment from the audience, written off by a retrospective design completely at odds with the previous season.

In the industry this is know as ‘retconning’, or retroactive-continuity; the often clumsy placement of a narrative element onto a story which was never originally planned for it. In the case of Dallas, the audience felt especially aggrieved because it was very obvious that the previous season was never written with this sleight of hand in mind and the show-runners had taken a fairly patronising option to resolve it.

Retrieval practice is a curriculum property, not an activity

In the science of learning, the practice of retrieving previously learned material from memory is about the closest thing we have to penicillin in learning terms. If you want to strengthen memory, the dominant finding over 100 years is the counterintuitive one that you should stop inputting material into memory and instead focus on extracting it. The crucial factor however is that there has to be something to retrieve in the first place.

Over the last 10-15 years, retrieval practice (and its symbiotic twin, spacing) have become an article of faith but increasingly I feel they are being used in ways which are not supported by the evidence. They have mutated from properties which exist to serve a curriculum into a range of techniques which give the impression of effective teaching but in reality, often grafted onto curricula never designed for them. In other words, they have become retconned narrative elements bolted onto a story with no actual narrative continuity.

In 2020, Doug Rohrer and colleagues at the University of South Florida took several of the most used Maths textbooks in US schools and coded every problem (over 13,000 of them) against an ingenious question: does this practice problem require the student to retrieve anything from a previous chapter or does it just live inside the existing chapter? In other words, are the practice problems blocked practice at the end of each chapter or are they designed with the science of learning in mind and designed with return paths to previous learning? The results were that around one in 10 problems involved any return path at all.

Then in another paper, the same team ran a randomised trial in over 50 classrooms and had two groups work on the same practice problems with the only difference being how they were arranged. In the first group, the problems were arranged in the way in the vast majority of textbooks, within the same chapter, usually at the end. In the second group, previously learned material was systematically interleaved using return paths as new topics came up. The results were that the second group outperformed the first in of the largest effects this kind of study has ever produced.

For me, all this confirms the fact that the science of learning has far more to do with curriculum design than teaching activities. Christine Counsell puts it far more brilliantly that I ever could; “you cannot Rosenshine your way into a curriculum”.

I think a lot of this accounts for the fact that teaching is based on a strong managerial model with a premium on the visible layer of teacher performance rather than the invisible architecture of learning underneath it. A five minute starter quiz fits well into a school culture of ‘learning walks’ and the optics of learning in two minutes whereas the return path of a curriculum needs someone to actually work through a highly complex domain of knowledge and codes it, the way Roher and his team did. A classroom full of kids raising mini-whiteboards like a Mexican wave might give the impression of learning but unless we examine closely what was being attended to then we risk the whole enterprise being pedagogical kayfabe.

An effective curriculum depends on well designed return paths

Let me be clear: retrieval practice and spacing are not so much teaching techniques, but rather tools for ensuring learning continuity within a broader landscape of knowledge. This whole process only works when old material is deliberately brought back into contact with to-be-learned material. And not just any material, but hinge concepts, threshold ideas and high leverage knowledge which genuinely moves the needle on student understanding.

A key question to be asked of any retrieval activity is this: what is this knowledge in service of? What is the downstream benefit of this thing I am doing upstream?

If students of Macbeth are retrieving dates and quotes but there isn’t a clear return path to the bigger thematic ideas they are meant to be building, such as Macbeth’s moral ambiguity or the corrupting nature of ambition, then students are merely cosplaying building knowledge. A student may be able to recall ‘Is this a dagger which I see before me?’ but still not be able to actually do anything with it. Because the quotation is not in service of itself, it’s in service of the bigger idea of a mind under siege from its own unbridled ambition. It is the broader pattern (planted in the opening scene, then paid off with Banquo’s ghost) of Macbeth seeing things that are not there but acting as if they are. To borrow again from Christine Counsell, retrieval practice without a destination or return path is an intransitive pedagogy, a pedagogy without an object.

Likewise the quick ‘Do now’ activity of recalling what they did last lesson. Now that might be effective, but the Rosenshine dictum of ‘retrieving material from last lesson’ is often manifested as an administrative ritual rather than something which builds deep understanding. If students are ‘retrieving’ something they did yesterday then this will often be easy retrieval and frictionless retrieval does almost nothing. The strength of the effect scales with the effort required to retrieve it. A retrieved memory still fresh is not modified much, whereas the same memory dragged back just as it teeters over the abyss of ‘catastrophic’ forgetting is transformed.

The evidence never showed that quizzing strengthens memory per se; it showed that retrieval at a delay, timed against forgetting, strengthens memory. The delay is the active ingredient here and the delay is not something a teacher can improvise on a Monday morning. It is a property of how the story was written in the first place, in other words curriculum design.

The science of learning was established in a lab not a classroom

The deeper and more uncomfortable truth behind all this is that the science of learning (specifically the findings around memory), was developed in conditions that usually look nothing like a classroom, in fact the classroom is in many ways hostile to the science of learning and memory.

Whenever I talk about this topic, I always find myself returning to Tom Perry’s review of cognitive science which showed that the lab evidence is strong but the classroom evidence is incredibly thin with most studies being carried out by researchers, with researcher built materials with conditions which are a world away from the reality of a classroom teacher. Controlled studies in a lab produce replicable results but studies done in a classroom are often mixed. In medicine, this is referred to as a gap between efficacy and effectiveness, whether the drug worked in a trial versus whether it worked when prescribed to a patient.

None of this means we should abandon the science and practice of retrieval or spacing. It means we need to be honest about what kind of thing it is. I think it’s highly unlikely that the way in which human cognitive architecture interacts with knowledge works one way in a lab and a completely different way in the classroom, but I feel we need to sound caution against the whole enterprise becoming pedagogical theatre at the expense of learning.

Retconning the curriculum by bolting on the science of learning to materials that were never designed with them in mind might look evidence based but we will find out later if any learning actually took place, and probably too late.