Reading The Water

This week I’ve been in Colorado with my family for fly fishing, which is one of my favorite things to do. It’s not so much the catching of fish per se, but rather the exploratory, experimental nature of fly fishing, that I enjoy. When you fly fish, you cast a line with a tiny “fly” imitating something like a mayfly nymph (a bug that lives in the river) tied to the end of it. The fly consists of feathers and hair tied on to a hook a few millimeters long. The fly, in turn, is tied to the end of a thin nylon line. The goal is to make the trout (both brown and rainbow) think the artificial fly is a real bug, attempt to eat it, and thereby allow me to hook the fish and reel it in.

What makes this exploratory is that you can’t see the fish. The surface of a trout river is a set of swirls, eddies, and runs moving across an uneven bottom, with rocks jutting out of the water here and there shaping the course of the currents. What you can do is to “read the water,” that is, look at patterns in how the water moves on the surface, to estimate what the structure of the river looks like at the bottom. This can lead to some educated guesses about where trout might like to hang out (usually somewhere where they have access to food, the current doesn’t require too much energy to swim against, and where they are protected from predators like me). Any such guess, of course, is highly uncertain, and therefore fly fishing mostly consists of an experimental search process by which you try to read the water and test whether a trout is present in a given spot using your instrument (the fly, the line, and the rod with which you cast the whole arrangement in a, hopefully, graceful set of ellipses).

I want to make the case that this loop: cast, observe, revise, is exactly the relationship we should have with AI when we are trying to build something interesting. Not because rivers and language models are poetically similar, but because they put us in the same epistemic situation: you are trying to act intelligently on a system whose interior you cannot see.

Herbert Simon captured this idea and gave it the shape that has been used in social science in the past few decades: “bounded rationality.” Humans face at least two hard limits when trying to act in the world: they lack access to complete knowledge about their environment, and they lack the processing capacity to compute optimal answers even when the information exists. To navigate in such an environment, Simon argued that humans work with a set of simple heuristics that are fit to the structure of the environment. This means that we use “rules of thumb” for navigating various situations in a way that satisfice various goals and desires we have. We don’t optimize, because being boundedly rational, we can’t. Rather, we try to find “good enough” solutions using knowledge encoded in “if this, do that” rules—much like me trying to catch trout in the Rockies.

The lesson for those of us trying to build interesting artifacts (and other, more important things, like communities of humans) is that we need to be aware of our lack of knowledge, along with our insufficient capacity to process what little we know. Trying to do this working together with AI tools, however, can be difficult. Recent research suggests that working with AI may produce “artificial certainty:” the illusion that inherently uncertain futures are definitively knowable. The fluid and often somewhat absolutist prose produced by LLMs tend to encourage this illusion, while offering accounts that pull you towards the statistical averages in the underlying training data, rather than exposing knowledge or truth in an unvarnished form.

This begs the question: if you need to use heuristics to manage your own bounded rationality, how do you learn what those heuristics are? The answer is experimentation, which is not only a discipline needed when we are trying to conduct science, but rather a fundamental way of engaging the world, even in creative endeavors. Karl Weick (one of my favorite academic writers) described this as “disciplined imagination.” Good theorizing, he argued, works like evolution, generating variations, selecting against feedback, retaining what survives. Hence, to build interesting stuff with machines, we need to possess some type of creative engine that can generate variations, and then you need some set of criteria with regards to what you are trying to build or achieve so that you can evaluate which variations should be selected and retained.

I’m writing this on my flight back to New Jersey from Colorado, so the fishing trip is over. But I am going back to building out the HEAD process. Much like fly fishing, this work is experimental, and this Substack seems to be the place where I generate variations so that I can start to evaluate them. So, in life, as in fly fishing, you can’t see everything that goes on under the surface of the river, the model, or reality. Cast anyway. Watch what happens. Cast again.