A Media Ecology of AI Agents

Brief definitions:

• Media: the tools and technologies by and through which we perceive and communicate.

• Media ecology: the study of media as environments that shape our thoughts, actions, and existence.

• AI: generative artificial intelligence, particularly large language models, often multimodal (can handle image and audio data, not just text)

• AI Agent: an LLM (often multimodal) connected to a framework that enables it to “take action” by using framework-defined tool calls. Typically, these frameworks are programming-related, allowing agents to organize and execute written code and interact with local or remote environments and applications (Claude Code, Codex, Cursor). Recently, many agent frameworks have been developed for use outside programming (e.g., Claude Cowork, OpenClaw).

A media ecological analysis is not a process of creation. Nothing new is created; things are only observed. In assessing the ecology of media, one analyzes how it has already impacted the human environment, understands how it has extended the human sensorium, and notes the pressure the medium exerts on its content. Media ecology is not invention but rather discovery (which does not preclude prediction). Having been a longtime student of media ecology, my aggressive foray into the cutting edge of AI Agents since the paradigm shifted in December has had a media ecology sub-process running in the background. I have been paying attention. How does this impact the human environment? How is this extending me? How is this shaping me? What biases is this introducing? But not only me. How is this/will this shape society, the human macro-environment? What happens to human beings when the Noosphere-given-voice grows metaversal arms and legs?

There is more than one way to skin a cat, so the saying goes. This analogy breaks down pretty quickly because I do not know any of the ways, but I imagine some are better than others. Prompting is like this. Ask someone to skin a cat, and you may have a serious cleanup job on your hands. Prompting is itself a skill, and the more I do it, the more I realize that fact.

Consider the scenario: You have a data-processing application that ingests data from a web source, transforms it, and then sends it to an external process. It has an error. You do not know what the error is, just that the process has stopped running. With agentic programming, you have two avenues:

There was a problem, fix it.

This is the first and easiest thought to write into the prompt. It is also likely the wrong prompt if the problem was severe or the software in question is important. A stack trace might show the full issue, but it may also omit important details that the AI and console logs do not capture. Or perhaps the problem did not show up in the logs at all; perhaps there is some issue with the processed data that would seem fine to an AI agent without domain knowledge. Defining the problem clearly is the first step, and then it is also necessary to define what the solution looks like, and occasionally what steps need to be taken to get there, to ensure that it is solved effectively.

Fellow Substack author Duncan Reyburn states in ‘Against Sloppification’, subtitled “Resisting the temptation to automate thought”:1

AI offers a merely functional orientation fixated on problem-solving. Problem-solving, I need hardly mention, is certainly an aspect of human intelligence. And yet, it is also by no means all there is to human intelligence. Most of the best stuff in life is not a problem to be solved but a mystery to be basked in.

Reyburn’s assessment of the difference between AI and human intelligence is accurate and is often lost by materialists. Later in the essay, he says:

Now, let’s look at adding AI to the digital mix. What do we get?... AI encourages what is called “cognitive offloading.” Think of any skill you have. Now, think of getting a machine to use that skill. Then, come back to it yourself. The effect of not using the skill turns out to be exactly like muscle atrophy. What you don’t use, you lose.

(Go read his full essay, an excellent McLuhanite analysis of generative AI from a different angle.)

For these reasons, I do not let AI anywhere near my writing process. I enjoy thinking, figuring things out, and writing. I do not want the cognitive rot of the easy button to ruin that for me. I make an exception for bulk research, but I often have to sift through a lot of chaff before I get to the wheat. But it is still faster. (If you have a problem with this, consider that Google search is simply a dumber (sloppier) AI). I do not have the same reservations about letting it near work/hobby software and computer projects. I take it to the extreme in software, and what I have noticed is the exact opposite of what happens when people use AI for writing.

Because in the situation I just described, the AI has not replaced the core thinking. It has handled a lot of the more granular thinking about code syntax and algorithms and the like, but the abstracted thinking, the programming decisions, are done by me. After a few months of this, my thinking has become much clearer, and I think through problems at a more abstract level, applying that thinking both at work and in life. Coming up with a good prompt is a matter of properly articulating what you want, how it is measured, and then imagining every single counterfactual interpretation and projecting multiple future paths to protect against the ones you do not want. Many prompt AI without telling it what they think; they are telling it. They leave their statements ambiguous, giving the agent room to make assumptions they will pay for later. They are not treading it like the breakfast question. What would your agent have built if you hadn’t written make no mistakes.

Sure, my ability to write Python by hand will likely never recover, but why would it need to? Unlike writing, artisanal, handcrafted code quickly loses utility if it can be done just as well with a prompt. In an email or Substack article, the recipients must read the words to consume the output. With good code, no one should need to read it. The software should just work. Then again, how much AI-written software is out there right now that just works?

Agents can spew out code faster than anything before. The world is abuzz with people talking about all the stuff their agents are building. The world is not abuzz with agent-built stuff. Strange disconnect here.

AI agents have a heavy bias toward tactical programming. They will find the quickest solution to patch over whatever the prompt asks for, with little regard for the overall architecture. John Ousterhout describes in his book A Philosophy of Software Design a distinction between tactical and strategic programming. Tactical programming chooses the shortest path. Strategic programming chooses the path that makes all future paths shorter. The design decisions that go into the latter option require some foresight and reasoning about uncertainty that agents seem to fail at.

This bias goes hand in hand with another bias that agents introduce. They spew out code so fast, the diffs whiz by at breakneck speed. It is like a slot machine. Tokens in, always a lot more tokens out, but are the tokens any good? At a certain point, it becomes hard to tell. Fire up three Claude Code sessions in parallel on three distinct projects and start working on them. 15 prompts later, you have no idea what they have done or where they have gone, no way to know whether you have what you wanted, because you do not even know if the feedback loop was correct. These are all solvable problems, of course, but the bias is against them. Especially when the cursor in the prompt box blinks away, telling you more activity, max out this usage session, wouldn’t want to leave money on the table, would you?

Korean-German media theorist Byung-Chul Han distinguishes between the active and contemplative lives. Contemplation is often lost or forgotten in the loop described above. One must contemplate before the prompt. One must contemplate after the prompt. One must contemplate a lot after the prompt (unless it is web design, in which case you just see it). At a low threshold of complexity, it quickly becomes necessary for the human user to activate the electric meat of the mind and figure out what is going on. Sorry, agent, your usefulness has rapidly diminished.

The only limiting factor for an agent is what the user told it to do. For the user to tell it what to do, it needs to have a better idea of what is going on than the agent does: not necessarily about each and every bit of code, but at the abstract layers of the software as a whole. Agents can quickly lose sight of the goals and broader context, instead optimizing for some arbitrary metric they extract from the prompt's ambiguity.

Therefore, the optimal agent workflow includes revolving periods of activity and contemplation, the contemplative parts being more boring than the rapid slop-machine prompting. The bias is towards maximal activity, of course, but the optimal way to use it is the former, making it a second-order bias: those who choose to overcome the first bias will be the most effective with this medium. Unless, of course, someone can strike the balance and effectively process the rapid activity and translate it into actual outcomes, like the synthesists in Blindsight, who speak to AI and mediate the post-human symbolic system of the AI through information topologies for the sake of useful insight and verification. Such an activity or ability would be of great value and would involve identifying and tightening the most important feedback loop so that the outputs meet requirements and steer agents in the right direction without intensive human oversight.

Leave a comment

There are still doubters out there, still some people who believe AI is a bubble. Meanwhile, there is more demand than supply for GPUs used to run Claude. Enterprise and individual customers compete against one another and the researchers to get a piece of the finite pie. When there is not enough to go around, it is not a too-much-VC-money situation. It is quite the opposite. Pets.com never had to consider how soon they should start using the trillion-word in infrastructure discussions.

Almost every day I am speaking with friends and family members new and old as they talk about or ask me what the AI can do. A good place to start is the boring stuff. This is how I got hooked on it in the first place, doing tasks that are little more than the digital analog of loading boxes into a truck. If the scale is right and the outcomes are measurable and verifiable, AI agents are a very good fit. There are many little annoying things like that across many information work jobs, and AI can shave anywhere from 30-95% off them. This impact has yet to be fully realized, as most AI users who have prompted ChatGPT a couple of times have not yet discovered the extent of the available optimization.2

In the short term, this has allowed me and many others to work faster on our individual jobs, creating a bit of breathing room and asymmetry. But this is rapidly snapping back in the other direction as institutional adoption steams ahead at breakneck speed. Shareholders and managers expect higher output. While this expectation is justified when one takes a simple view of agents, it feeds directly into the activity bias. This runs the risk of sloppification of the bureaucracy (aka hell).

This will likely resolve itself in the coming years, though, as agents are used more effectively across more domains and automated agentic systems are put in place. They will make bureaucracy effectively frictionless, being able to process, handle, and parse near-endless swaths of documentation and data. This being effective has some serious dependencies, though. Currently, agents can handle large code bases and document stores relatively effectively, and certainly quickly, but they primarily rely on tool use. They do not read everything from top to bottom; instead, they use bash commands to list files and find specific words in code and documents.

This can occasionally lead to prompt ambiguity, causing the agent to provide an incorrect answer. Even if they do not give an incorrect answer, they may give an overly confident answer. Users may assume that agents are competent in areas where they are not and trust their answers anyway. Both of these scenarios could be resolved in the next few years (or days3), but until they are, widespread agent adoption will put a question mark on everything. Even if one assumes that agents make such mistakes 10% of the time (it certainly varies by domain and prompting competence), that still leaves the rest of the information in question, creating a liminal work environment of quasi-truth and possibly useful information flows. Nobody will be certain of any of it.

I will forgo any discussion of “alignment” or related topics because I find the subject uninteresting and do not wish to make predictions about it. AI Agents lack the (ironically) agency to make any of the end-of-the-world predictions come true, and on its own, an LLM is simply space-age auto-complete and pretty dumb without the advanced agentic harnesses that are in use today. That being said, the threat model of AI Agents, both with and without accompanying world models in their multimodal capacity, certainly falls within an assessment of the human environment, but it is a more specialized and predictive domain, whereas this essay focuses on my direct observations of the impact of agentic AI.

Thank you for giving your time and attention to this essay. Much of what I have described I have discovered through practice, so translating it into the written word has been a bit of a challenge, and I am still unsure whether I have been accurate in everything I have written. But this is a rapidly evolving media landscape, and the environment and impact are certain to change.