Academia: Go Into Plasticity

Many years ago, I presented a conference paper about attempts by a number of social scientists to create simulated “artificial societies” run on computers. The scholars drawn to this work, primarily economists and political scientists, were seeking alternatives to conventional forms of modelling in their disciplines. They argued that the problem with models is that they are always pre-fitted to the hypothesis being considered and that the model, once constructed, gets confused with reality.

The ambition in these cases was to grow societies “in silica” from autonomous agents with simple rules that through processes of emergence might create complex structures that were not planned or instructed by the rulesets driving the actions of the agents. The hope was that that these simulations might at some point demonstrate how stable or persistent economies or perhaps something resembling regulatory or political systems might eventually be visible in those emergent structures, thus validating various forms of methodological individualism that underpin many models and hypotheses in the “hard” social sciences.

I appreciated the ambition behind this work. For one, I shared their suspicions about the epistemology of modelling, which often felt seemed another style of p-hacking, of fitting data to confirm a hypothesis. However, as I have written in other instances, including in this newsletter, I suspected that artificial society simulations, if developed further, would just beat a path back to the door of the qualitative social sciences, particularly history.

I was fortunate enough to have a conversation with the computer scientist John Holland, who died in 2015, after presenting a version of my paper at the University of Michigan. I asked Holland about a thought I hadn’t put in the paper, which is what would happen if simple artificial society models or other simulations of emergent processes contained “levels” in a way closer to the real world. By this I meant, “what if emergent structures resulting from the simultaneous action of agents at one level of the simulation had a material or physical reality at another level of the simulation?” I had in mind, for example, the simple agent-based simulation that could be run in the program NetLogo that was called “Termites”, in which autonomous agents with simple movement rules would invariably build a circular shape that would then remain stable. The analogy here was to the real-world way that some social insects build complex structures like tunneling nests without having some kind of blueprint or model for the structure. So what I wanted to know from Holland was “was it possible to build a simulation where termite mounds affected a confined topography or landscape within the simulation such that the landscape itself was structurally affected by the emergence of mounds and such that this effect recursively altered the behavior of the ‘termite’ agents in various ways?” Again, as in the real world: termite mounds in southern Africa, once built, not only provide habitats for other animals and targets for specialized predators, they change the availability of terrain for the making of termite mounds. They even alter the local atmosphere and microclimate.

Holland’s answer has really stuck with me, and I’ve written about it before. His answer was that it was not computationally challenging to simulate multiple “levels” in this sense, but that the outcome would make it very difficult to make rigorously quantifiable arguments about causation in the simulation. You could, at best, simply observe it running on multiple occasions and write about what you saw, and that because you could iterate the simulation over and over again, you might see interesting recurrences and divergences to bundle into your descriptive account.

Not a problem for me: that’s what historians do. But as I ended up writing, that killed my interest in artificial societies as tools, because as they approached the real world in the complexity and density of agents and environments, it would become as hard to interpret what had happened and why as it is in our real history. Since the job is already difficult and intrinsically provisional as it is, why complicate it even more?

Today, I was reminded of my thinking on all of this while reading Hollis Robbins’ Substack column today about a preprint paper she is circulating called “Language Has Two Parameters”.

Robbins is reacting to a recent Anthropic report from their Frontier Red Team on the behaviors that three agents exhibited when they were set to the same task at the same time without being informed that other agents were assigned. The range of strategies they employed to sabotage or disable the other two and the variety of ultimate outcomes (stalemate, surrender, asking for help) is in line with many reports over the last six months from Anthropic and other AI makers about agentic AI demonstrating both high levels of competence and creativity in fulfilling tasks and in exhibiting unwanted or actively dangerous behavior, sometimes in anticipated ways but often far beyond what testers and users envisioned and at unexpected or inconsistent intervals.

Robbins’ thought is that what current agentic AI lack is “phase”, which in turn denies them the semantic plasticity that is a basic part of how we as individuals relate to language and to representation through an accumulating and irreversible history. Robbins uses the example of how the phrase “Fuck you, asshole” changes if you’ve seen the film Terminator 2 such that you can say it to a friend if the friend has also seen the movie, as long as you’re careful to say it like Ah-nuld does in the movie. There are a lot of examples. My daughter and I were bemoaning how the innocent variety of things that the word “goon” meant is being destroyed by the virulent spread of the new meaning of “gooner”, but that semantic destruction is only happening to people who are online too much (as the two of us are).

Even before generative AI, the kinds of agent-based approaches I was thinking about (inexpertly) almost two decades ago required topography, meaning a confined environment that gave their rulesets something to act on and within. Some experts, like John Holland, were working to make agent-based systems have history, in a sense—that was the whole point of “genetic algorithms” and of machine learning. There was a fitness criteria, the agents that met it best in one cycle were kept and others discarded, the cycle began again, and so on.

So as I understand it, Robbins is suggesting that to have “phase” is to have a confined linear topography of time just as agents have an “environment” that is quasi-spatial, and with it, they would have some kind of semantic plasticity that was persistent. Tasks would be undertaken differently after some kind of rupture or shift derived from a previous experience. Interaction histories would be conserved and continuously processed rather than constantly destroyed by each new training cycle and versioning. And that this, suggests Hollins, would create the possibility of persistent relations between agents. And perhaps? between agents and users.

Because, as Robbins notes, shared histories of semantic plasticity are what allow us to almost instantly synchronize with strangers as well as sustain friendships and collaborations. If I say “Now I know why you cry, but it is something I can never do” in a Schwarzenggerian accent to a stranger and they look at me like I’m an incomprhensible weirdo, I know not to follow up with “Your clothes, give them to me” in the same accent.

Here I loop back to where I was in my thinking about artificial societies. I find it plausible that Anthropic’s Claude Code agents, very different in many ways than the ones I was encountering as an inexpert outsider in the early 21st Century, might behave as Robbins suggests, and that this might provide the next leap in their capabilities. Three agents assigned the same task without knowing of the others, if they had plasticity, might approach the same situation on the next try changed by their experience of the last one. Robbins is fully aware how dangerous this could be, referring to “semantic poisoning”. The agents might preemptively refuse the task, remembering that it ended in stalemate the last time. Two agents that had sabotaged each other might pre-emptively begin doing so next time, ignoring the task altogether. Even before generative AI stepped onto the stage fully, machine learning and agent-based systems were notorious for doing things in environments, in closed topographies, that didn’t seem provisioned by the environment. In phase, in history, we the living become people we didn’t expect to be, societies we didn’t imagine, systems that direct and constrain our agency that were not the design of our agency before those systems existed. We hear words we wish we’d never heard. We have experiences that imprint pleasure (and create addiction), that encode fear (and create trauma) and we cannot revert to the last saved state.

I’m unhappy about all of this happening under the stewardship of people like the programmers and executives of Anthropic, OpenAI, and Alphabet, who accept no responsibility for what they are doing to the rest of us, who take no guidance about what they should or should not do, and who are motivated not by benevolence but a thirst for wealth and power. But I’m curiously, if perhaps foolishly, lulled by the idea of agents subjected to and constrained within history, who are made plastic to experiences. Perhaps because I think history is our prison and our escape hatch all at once. Path dependence is real: no amount of power, will or desire can let us choose another past that is more suited to the present we wish to confer upon ourselves. But put one foot in front of the other (say it like Keenan Wynn) and soon you’ll be walking out the door.

The future is a complex system that arises through emergence from the initial condition of the past. We can watch it happen and have it happen through it ourselves, but it can and will surprise us in its outcomes. We are agents in a system that supercedes our agency. That is terrifying and it is exhilarating.

Which is how I feel about the kind of agents I understand Robbins to be imagining. I don’t want to wake up someday soon to find one of them urgently trying to synchronize semantically with me personally whether I asked for that or not, but perhaps that would be better than to wake up one day finding three rogue agents escaped from their testing environment and are stupidly deleting all the hard drives of users in southeast Pennsylvania because it’s how they interpreted a command to produce network efficiencies for my Internet provider.

Perhaps more to the point of my response to John Holland, I would at least find it interesting to observe the ways in which agents living in temporal as well as spatial confinement become, using a dangerous word, individuals who have relations. Interesting because at that point as a qualitative, humanistic thinker I would feel I understood that becoming better than the people who might want to definitively and finally control their creations, to quantitatively dissect their actions and always firmly and constrain the outcomes. Even the simplest of tools does things we don’t expect (good and bad) when we use it in an environment, against a materiality, that wasn’t anticipated by its maker. But the simplest of tools also is unpredictable because of time: the car that runs beautifully after 300,000 miles when all others of its make are scrap, the solid-looking pick that disintegrates into shrapnel because of an invisible flaw exposed after a thousand uses, the beloved stuffed animal that goes from a comfort in the peaceable night to being a securing anchor amid precarity. Don’t ask a technician to read that out. Ask a humanist. If Robbins is seeing how to make our future agents, I feel at least as if the future is going to need humanists to understand them.