Human-Embedded Agentic Design (HEAD)

Back in 2023 the term “prompt engineering” was everywhere. The idea was that if you could write the right prompt that was specific enough, structured enough, with the right instructions in

the right order, etc., the AI would do what you needed it to do. This spurred on new courses, job titles, but perhaps not careers. Because, increasingly, people are talking less about prompt engineering. For the last two years the conversation has been increasingly dominated by the term “context engineering” and since OpenClaw launched, “harness engineering” has become the term du jour.

A prompt is a single input to a single conversation. It has no memory of what you built yesterday, no awareness of the project’s history, no connection to the standards and patterns you’ve developed over months of work. Every time you start a new conversation, you start from scratch. The AI doesn’t know who you are, what you’ve already decided, or what you’re trying to build. Context engineering is the answer to that problem. Not a better prompt but rather a better world for the AI to operate in. Hence, the new role for the future is the context engineer (at least for the next few months, until something new comes around…).

This of course has occurred at the same time as the idea of agents, i.e., sequences of prompts, infused with context retrieved from documents or the web, that can accomplish more complex tasks, has completely overtaken our conversation about AI and LLMs. The initial narrative around such agents was that they would be able to do work autonomously. Increasingly, however, I think it seems obvious that they will need to mesh with humans and collaborate with us at scale. The hilarious and brilliant podcast Shell Game makes this point vividly through telling the story of an agent-only startup that fails miserably. The fact that the big AI labs are now starting consulting firms staffed by humans helping enterprises to implement AI solutions also makes this point. It is somewhat ironic after Anthropic and OpenAI having mused about how all jobs will disappear shortly for years now. Anyhow.

I think it is time to start re-imagining our workflows as designers and creators. Human-Centered Design and its cousins Lean Startup, Agile, participatory design, etc., has served us well for decades. LLM-powered agents, however, are changing the way that knowledge work is being done at a fundamental level. Therefore, I am suggesting we need a new set of design processes that are not only human-centered, but that focus explicitly on human-agent interaction and how that changes design, innovation, knowledge discovery, and engineering processes in general. I call this the HEAD (Human-Embedded Agentic Design) method to work with multi-agent systems to design artifacts.

The HEAD approach is an approach to multi-agent systems that relies on multiple, adversarial agents, embedded within the overall design gestalt (or vision) provided by a human designer. Here, I want to avoid the term “human-in-the-loop.” I think this term largely has been applied in a shallow fashion for systems where humans have been arbitrarily inserted to sign off on agent-produced outputs. I think this approach largely fails. Therefore, I argue for “AI in the loop” of a human designer, or expressed differently, embedding the agentic design system within desiderata and guardrails defined by humans.

Core Components of a System for Human-Embedded Agentic Design (HEAD)

The figure above shows what a system for HEAD might look like. The core components are, of course, the human designer and their design vision or gestalt. This is the core driver of the model—the human is still in charge, and their design vision or gestalt is what provides the initial outlines for the context architecture and all the subsequent work done by the agents. The context architecture is the set of data, texts, documents, skills, scripts, etc., that can be used by the agent in different combinations across the design process. The two types of agents: design and crit agents work in a collaborative, yet antagonistic stance to each other. The design agents generate and update the artifact, while crit agents provide critiques, feedback, and evaluation. This process tends to create enormous amounts of computation, which expresses itself in the generation of large amounts of code, prose, perhaps also images, audio, and video. The eval artifacts serve as abstraction scaffolds that help to take the work that has been done previously and summarize it in a way that makes sense to a human designer with bounded cognition.

Over the next few posts, I will discuss each component of the HEAD system, along with the overall process of designing artifacts using this system. More to come!