Expertise Research was supposed to transform education; what went wrong?

Studying experts was supposed to teach us how all students could be more successful. That was the promise Benjamin Bloom made in his 1985 book Talent Development in Young People. In the 40 years since, educational outcomes have improved very little. In many parents’ minds, outcomes have gotten worse. The promise that “virtually all people have enormous potential” and that we would redesign education to help them achieve it has not played out. Why is that?

Starting in the 1980s, expertise research split. On one side of that split were the academics who became known as cognitivists. They were largely education researchers, most famously Benjamin Bloom and, later, K. Anders Ericsson. Their work became famous through Malcolm Gladwell, who popularized the 10,000-hour rule. It’s now the most popular model of expertise. It’s also the model we’ve brought into education.

On the other side of the split were the post-cognitivists, specifically Naturalist Decision Making (NDM) practitioners1. The field started with Gary Klein. They were practitioners and consultants, working for organizations who needed to understand their own expertise to win. Klein has had no popular science books written about his work. He has no randomized controlled trials studying the effects of his view of expertise research. Few outside certain cultures of expertise have heard of Naturalistic Decision Making — the field Klein created. But his work is trusted in high-risk environments as varied as the U.S. military, trauma centers, firefighters, and nuclear power response teams. The White House even trusted it, asking him to help redesign the Situation Room. Despite its following, Klein’s research has never been incorporated into primary education.

The cognitivists noticed that experts almost always did their best work after 10 years of “deliberate practice.” Bloom looked at experts in four domains: athletic, artistic, cognitive, and interpersonal. He quickly dropped interpersonal domains from his study (think teachers, business leaders, etc.), because he couldn’t reliably identify expertise there. Ericsson picked up the thread, looking at expert violinists and pianists and comparing them to less accomplished players. He found that the difference between the two was the amount of deliberate practice they had accumulated (that is, solo practice guided by an expert teacher that focuses on the student’s weaknesses).

They studied well-defined, known, structured domains. They generalized to everywhere else.

In the cognitivist view, expertise is a matter of having more knowledge that can be easily taught. Students who apply themselves more (or are pushed by a parent/teacher) become experts. Expert chess players know more strategy, have more mental models, and can analyze a position better than a lesser player.

In the cognitivist view, expertise is rare because that effort is rare. Expertise can’t occur early because children need those 10+ years to build that knowledge.

Post-cognitivists, meanwhile, believe expertise is common. The core insight of NDM is that expertise is so common that you can speedrun it. It all began with firefighters and the military. In 1984, the US Army Research Institute for the Behavioral and Social Sciences asked for research proposals to study how experts make decisions under time pressure. To quote the RFP:

“Commanders, intelligence analysts, and others are often required to make decisions under conditions of uncertainty and severe time stress. Research is needed to better understand the cognitive processes of the decision maker and to suggest approaches for supporting the cognitive processes so that overall quality of timeliness of decisions made under uncertainty and time stress are enhanced.“

The Army had spent the prior 10 years studying the research on decision making and building expensive decision aid software for battlefield commanders. The problem was that no one on the battlefield would use anything they built. So they went looking for a new understanding.

Klein started his work in that context.

Gary wanted to study the decision making of fire commanders — the people in charge of fire crews. The common view was that commanders weigh options rationally and choose the best outcome. Barring that, they must choose using analogies or mental models, another popular idea in decision-making theory even today. Unfortunately for Gary (but fortunately for us today), the commanders weren’t cooperating. One fire commander announced confidently in an early interview, “I don’t remember when I’ve ever made a decision.” Another fire commander told Gary in a hushed voice, “The reason I don’t make decisions is because I have ESP. I know what’s going to happen before it does.”

These experts were operating in unstructured domains. That is, they were working in domains where quick feedback isn’t always possible, and where a teacher can’t guide your practice. They didn’t think they were doing anything special because they couldn’t communicate or measure it. But they had real expertise.

Klein discovered that experts don’t decide between options; they naturally discover the best option the first time. They know what’s important to notice, and they train themselves to notice the most important points. They learn from stories of others as much as their own practice. They are able to rapidly make sense of a situation, even when the situation is rapidly changing. He came to call this sensemaking. He discovered that these skills can be trained explicitly.

In the post-cognitivist, NDM view, expertise is a matter of knowing what to pay attention to and rapidly making sense of a situation. Experts don’t have better mental models loaded with more information. They pay attention to less information, but they quickly identify the important information. They have intuition about what matters. They are able to play forward an idea in their mind, seeing how it will play out. Expert chess players can intuit the right moves, without mental models, and don’t choose between anything.

In this view, expertise is common, because attention is common. Expertise can and does occur early, and it can be accelerated.

I know how quickly expertise can be developed. 22 years after Gary Klein discovered NDM, the Navy taught me to fly carrier airplanes based on his work. In two years I went from never having been at the controls of an airplane to landing a jet on an aircraft carrier 100 miles off the coast of Florida.

A Navy aircraft landing on an aircraft carrier is 46,500 lbs of steel crashing into a moving ship the size of a football field, and it must hit the same point within five vertical feet. Every time. The pilot flies at 130 knots, trying to hold centerline, position, and angle of attack, for 18 seconds, all while the ship is moving away and sideways. It’s a dance on the easiest days, and on the worst days the ship can move 40 feet in a giant corkscrew motion while you’re trying to land in that same five-foot window. It’s dangerous. Three feet off centerline, the airplane will hit another plane or person. Five feet too low, the plane kills hundreds of people. Five feet too high and the pilot misses completely. And by the way, it’s all done without talking.

If we trained through deliberate practice, we’d break down every possibility that could occur into specific tasks. Practice being a little low. Practice being really high. Through lots of well defined scenarios, we’d start to develop a mental model of the perfect landing. We’d focus exclusively on the parts we weren’t very good at, moving through those scenarios so that we could quickly analyze every situation we could find ourselves in.

I can tell you from experience, that’s not how I learned to fly. There’s no way it could have, the complexity is too high.

Instead, we train attention and understanding. It starts by being introduced to the Fresnel lens, affectionately termed the meatball. This device tells us where we are on a three degree glide slope. We are taught the mechanics of how it works. More importantly, we see it on every flight, far before we need it to help us land on a ship. We train our attention to notice movement in the meatball. We train our bodies to know how to make it move.

The final portion of flight training is landing on the ship. We start in the simulator. Simulations are big in NDM. A high fidelity simulation can teach decision making without the danger of real flight operations.

Once we’re safe in the simulator, we go to the runway. First in very constrained conditions with no wind, only focused on the last 15 seconds before landing. We gradually move out to the whole landing pattern. There is naturally lots of variation in training (because we suck at it to start)2.

A safety officer grades every pass, telling us not only what score we got, but what he saw. An example early pass might look like “On at the start, too much power in the middle, high the rest of the way. You wouldn’t have caught a wire.”3 This is the expert telling us what he noticed. It’s not just any expert, but the exact safety officer who will be on the ship when we first land. They’re learning to read us, as individual pilots, just as much as we’re learning what they want from us. And we’re all learning a shared language that will make sure we don’t confuse each other

It takes months of dedicated training before we’re trusted enough to go to the ship. And in the training command, they still won’t let us land at the ship unless we’re in a very tight window.

It takes on average two years to train someone to be a carrier pilot. At the beginning, most of us have only seen airplanes. By the end, we’ve learned how to fly a T-45 jet safely onto a carrier. A year after that I took the E-2C to the airplane. Before long, landing became routine, and I moved on to other aspects of flying I had to master. And today, I bring that learning home, with my children.

From pilot training, I have incorporated “chair flying”, imagining the future outcomes as they might occur so we can train our attention before a learning opportunity. I’ve started doing a pre-day brief and a debrief every week. We try to imagine what will go well and what might not, then look back over the week and see how to do it again. I hope they bring a sense of awareness to our time. From NDM, I’m trying to incorporate more stories as teaching structures. From The Golden Thread to Plutarch’s Lives to Maxwell, Faraday, and the Search for the Electromagnetic Field, I’m trying to read more studies of great lives with them. We’re trying to stop more, to talk about the texts more, to predict and notice and compare our learnings.

I want to figure out how to bring more and deeper expertise into our homeschool. I want to understand what expertise as a scientist looks like, so they can learn what scientists really pay attention to, how they see the right experiment, how they make sense of what they’re learning. I want to know what tacit knowledge in art, science, writing, and other areas looks like, so I can figure out how to train them in it. I want to train them how to see it for themselves. I want to understand when I could incorporate tactical decision games and simulations into our lessons. From my experience, these accelerated learning tools are a lot of fun and people learn a lot quickly. But I don’t yet know how to apply them to school. I’m working on it.

If I can figure this out, I imagine my kids being able to quickly make sense of any situation. I think of it a bit like seeing the matrix. I imagine them having the confidence that they can contribute now, and do great things soon. My kids could easily contribute to the fields they care about before they graduate high school. I imagine what it would feel like to know they have 40 years ahead of them to use the expertise they have already developed. I imagine the confidence that comes from knowing how to develop expertise, and feeling how regular it is. I see them developing new insights, new discoveries, and succeeding in situations none of us ever imagined. I see a future where expertise is as mundane as a degree is now, and I want that world for my kids.

There is a lot I want from NDM. There’s even more I hope it will give my children. I think it’s the key to fulfilling that promise from 40 years ago, that expertise research could make all students more successful.