How to Learn When AI Knows Everything

Recent research suggests that when experimental subjects are asked to perform a coding task using an AI-assisted editor, they find it much more difficult to subsequently go in and fix and extend what they have built. When AI was used without restrictions subjects failed 77 percent of the time, while a second group, which used a development environment that included an “explanation gate” requiring users to explain the logic of the artifacts generated, only failed 39 percent of the time. This suggests that cognitive outsourcing to AI when building artifacts hurts cognitive understanding quite dramatically. The author refers to this as “epistemic debt” generated when a user doesn’t “cognitively own” the output they have generated using AI.

For most of the industrial era, education was about knowing stuff. Over the past few decades, it shifted toward doing stuff: hands-on projects in largely simulated environments. We are now moving toward building stuff. When an agent can retrieve any purely cognitive answer in seconds, knowing is a commodity. Increasingly, agents can also build complex technical artifacts. However, we are not just building “artifacts,” we are also building “socio-technical systems” consisting of both humans and machines, and machines still cannot build such systems without human intervention. Therefore, building the capacity to manage the process of constructing such human-machine systems is a critical human responsibility.

As a case in point, other recent research shows the importance of building development environments that integrate human cognition and machine computation. Researchers conducted a quasi-experiment with students split into three groups: no AI, AI, and AI plus scaffolding built on self-determination theory that walked students through planning, monitoring, and reflecting on their own learning. The AI group improved on intrinsic motivation, self-efficacy, creativity, and higher-order thinking skills. The scaffolded group improved further on computational thinking and self-efficacy specifically. This suggests that custom-made software environments that strengthen human self-determination produces better results in terms of learning compared to off-the-shelf AI products.

So, what does a designed learning process look like when the goal is building? I have been converging on six steps, each with a defined role for the human and a defined role for the AI.

1. Define the learning context

Human-embedded agentic design means that design should still be fundamentally human-centered: you decide what you want to build, and you let that decision generate an initial outline of competencies that you think you will need to build that thing. To get to this point you will need to develop a sense of how a particular context works through doing some context engineering. This runs on two levels at once. You are engineering your own cognitive context while you engineer the context of the agents working alongside you. The key here is to establish your goals centered in human needs and wants and then work with AI tools to establish what human competencies are needed to accomplish these goals.

2. Build the learning spine

Next, you need to build out an overall spine of concepts that you think you will need to understand to master a domain and develop the competencies needed to move freely within that domain. These are words and concepts you know exist but cannot yet define, practices you can name but not perform. AI is good at surfacing what these concepts might be and how they are related to each other. Your job is to validate the set of suggested concepts and decide which gaps matter for what you’re building.

3. Fill in the blanks

Once you have established a learning spine you have a set of blank boxes: concepts and practices that you need to learn. I usually approach this through establishing a dictionary that I can then fill out using AI. Often it is helpful to me to work in an interactive, conversational mode with Claude to accomplish this. I also find it helpful to develop custom NotebookLM podcasts just for me, my situation, and the things I am trying to learn.

4. Construct workflows

Of course, we are concerned with building, so we don’t want to just understand the concepts, we want to be able to take action. The next step is therefore to build a set of workflows consisting of interlinked actions that will help you build what you want to build. AI agents are useful for building such workflows for you, and if you have completed steps 1-3 conscientiously you should be able to engineer a rich context for an agent to design such workflows that are entirely customized to what you want to achieve.

5. Build rules of thumb

Workflows should not be static steps of action but should rather be dynamic flows of activities that can adjust to various situations. Last week I wrote about Herbert Simon, bounded rationality, and how we can use rules of thumb to navigate complexity. The goal here is thus to generate a set of rules of thumb (or heuristics) that will help to indicate how to handle various situations and edge cases. AI tools are good at generating a set of variations of a pattern specified by a human.

6. Design dynamic artifacts

Previously I’ve made the point that in Build Culture, a key principle is to ask “Who can I bring along?” To bring others along in your learning journey, and to make sure you have truly understood what you are trying to learn, I’ve found it really helpful to generate dynamic artifacts that a) encode the knowledge that you have arrived at, and b) enables the workflows that you specified in steps 4-5. Tools like Claude Code and Claude Design are quite good at generating these artifacts if they are provided with a good enough context, which steps 1-5 should provide.

The most discernible danger here is to rely too much on AI and thereby participating in a “performance of learning” without real human cognitive learning outcomes. I think the solution is that we need to bootstrap our learning through building impromptu learning environments using AI that increase human self-determination through introducing cognitive friction across the learning process. The six steps above is a first prototype of how we can do just that.