Gray sky thinking: John Boyd and generative entropy

In his “Destruction and Creation” (1976), the military strategic theorist John Boyd describes what he calls a dialectic engine. Boyd begins from the problem that, as decisional agents, we develop and use concepts and models in order to navigate environments that nevertheless change as rapidly as we move through them. (One of Boyd’s paradigm cases for this situation is provided by the dynamics of aerial dogfighting, and he was himself a well-respected fighter pilot and instructor for the U.S. Air Force.)

The goal of conceptually modeling navigation in this way is “to improve our capacity for independent action.” This improvement is intended to help us exercise and retain agency in complex decisional environments, avoiding being outmaneuvered or overwhelmed, whether by competitors, or by the sheer weight of complexity or entropic accumulation. “Independent action” designates something like initiative, or the ability to act without having the conditions and tempo of action dictated entirely from elsewhere. An agent at whatever scale loses the capacity for independent action when legacy routines continue to determine behavior after the environment has changed, structuring perception through categories and concepts that no longer register relevant differences. Independent action depends on the preservation of a certain conceptual mobility. The phrase also affords Boyd’s theory a practical criterion of success or value. Concepts aren’t renewed because novelty is desirable for its own sake, or because there is some inherent virtue in destruction, but because increasing rigidity in our concepts restricts the field of possible action. A concept is provisionally successful when it organizes perception, decision, and action into a repertoire adequate to the relevant variety of the environment.

To improve this capacity, Boyd maps out a process of concept creation with two parts.

The first is what he calls destructive deduction. He describes destructive deduction as the process of breaking apart the correspondence between a comprehensive whole and the full complement of particulars it organizes: “In other words, we imagine the existence of the parts but pretend that the domains or concepts they were previously associated with do not exist.” This frees up all the particulars from their prior conceptual subordination, but also leaves them in a state of disorder, “swimming around in a sea of anarchy.” Like a school of fish broken apart by the sudden passage of a hunting swordfish, the conceptual formation loses its coordinated shape without losing many of its constituent members. The scattered elements wheel through the surrounding medium, briefly entering new proximities and lines of flight. In the wake of the intrusion, they recoalesce along a different contour, the same particulars reorganized into another provisional whole.

Destructive deduction therefore produces a condition in which the elements of knowledge remain present while the relations that previously rendered them meaningful and orderly have been disrupted. Facts persist without their customary explanations, structures and techniques survive the purposes for which they were originally devised, and distinctions become detached from the systems that had governed their use. The sea of anarchy contains schools of unregulated conceptual materials.

A close conceptual analogue is William S. Burroughs and Brion Gysin’s cut-up method. Burroughs and Gysin cut existing texts into fragments in order to interrupt narrative and syntactic continuities that made the meanings of the original texts appear as necessary givens. Recombination using this method prospectively discloses possible relations concealed by the original sequential mode of presentation. Boyd’s destructive deduction performs a comparable operation with conceptual structures, releasing their components from an order that makes certain connections seem natural. However, while fragmentation may generate new patterns, it can also yield undifferentiated noise, or even the paranoid conviction that every fragment communicates secretly with every other. The sea of anarchy is an oceanic morass, a virtual field of all possible relations that supplies no criteria for deciding which relations should be realized and which discarded.

Boyd then articulates the second part of this process, which he calls constructive induction. In constructive induction, we “synthesize constituents from, hence across, the domains we have just shattered. Linking particulars together in this manner we can form a new domain or concept.” This is a mosaic or patchwork process, in which disparate pieces from broken knowledge systems are sutured into chimeric and novel assemblages. This constructive phase also performs a governing function, regulating the disorder released by destructive deduction by selecting which new relations can be stabilized into a usable whole. Without such selection, the disaggregated particulars remain only as a field of unrealized possibilities. The new concept emerges as a controlled reduction of variety, organized according to Boyd’s governing aim of preserving or enlarging the capacity for independent action. It is less clear who or what performs the selection. At different moments, the answer seems to be the agent, the resistance of the environment, or the success and failure of the concept itself.

Ecologically, this closely resembles the backloop of C. S. Holling’s adaptive cycle, in which disturbance releases materials, loosens established relations, and opens up a period of reorganization. Holling adds something Boyd largely leaves implicit: reorganization is neither unrestricted nor fully governed by an agent’s purposes. The backloop does not return a system to zero; everything is always already asymmetric. Whatever survives the initiating disturbance enters the reorganization phase with it. Holling emphasizes this asymmetrical inheritance more than Boyd does. Where Boyd’s agent appears, at least sometimes, to confront a collection of totally liberated particulars, Holling’s system confronts a field already shaped and warped by patterns of persistence and loss. Recombinations remain possible, but not all possibilities are equally possible. Boyd presents destruction and creation chiefly as a method for restoring freedom of action; Holling shows that the field of possible recombinations is already unevenly structured. Some assemblages become available because of inherited legacies and timing, while others are excluded before any central selector could choose among them.

Boyd also emphasizes the relationship between the two processes: “It is important to note that the crucial or key step that permits this creative induction is the separation of the particulars from their previous domains by the destructive deduction.” Ultimately, this dynamic of destruction and creation (“the process of Structure, Unstructure, Restructure, Unstructure, Restructure is repeated endlessly”) allows us to map out features of our decisional environment even as those features change rapidly around us.

But there’s a problem. Boyd writes that “any inward-oriented and continued effort to improve the match-up of concept with observed reality will only increase the degree of mismatch.” Concepts organize the observations through which they are refined, while those observations in turn modify the concepts that rendered them legible. Boyd casts this recursive enclosure in terms of thermodynamics: entropy increases in closed systems.

Boyd argues that we need to destroy our extant and inherited concept formations first in order to free up their particulars for repurposing into new concept formations. He writes: “the uncertainty and disorder generated by an inward-oriented system talking to itself can be offset by going outside and creating a new system” (emphasis mine). This does not require an apophatic outside beyond every possible structure. Consider the problem in Boyd’s own tactical terms. A pilot can receive more and more information about an opponent while continuing to interpret each maneuver through a model that has become inadequate. Nothing has gone “outside” the decisional loop just because another observation has been added. The relevant interruption occurs when a maneuver forces a change in what counts as a threat, an opportunity, or even an intelligible action. “The outside” names this failure of the existing model to absorb a difference without changing its own organization. Hence, the outside in Boyd’s sense supplies a different criterion of legibility altogether - a difference, in Gregory Bateson’s formulation, that makes a difference.

Boyd’s conclusion is striking: “an entropy increase permits both the destruction or unstructuring of a closed system and the creation of a new system […]” (emphases mine). Entropy isn’t just the doom of a concept, or a system of concepts, but also the necessary fuel for the construction of progeny.

Boyd offers a model of a conceptual update machine that runs on the very entropy it generates. Its wager is that conceptual survival depends on generative entropy, converting a prior order’s breakdown into the conditions from which the next may emerge.