Artificial Infinitude #4: AGI's Victories Would Not Be Ours

Artificial Infinitude #4: AGI’s Victories and Our Victory

As Eric Schmidt says, a key distinction between Narrow and General AI lies in who dictates the direction of its learning:

The eventual state of AI is called General Intelligence, where you get a human kind of reasoning…We describe that as: the computer can set its own objective. And today, the good news is the computer cannot choose its own objective. At some point that will not be true.

Here, Schmidt articulates why it is unclear that the development of General AI will truly represent the conquering of our finitude, even though that is the goal under which many AI developers labor. The desire to overcome human limits drives us to build ever-stronger AI, but if we manage to create an immortal and omnipotent being, will we actually have defeated our own finitude, or will we have released upon the world something separate from ourselves, something entirely inhuman?

If its objectives are not necessarily our objectives, then General AI’s achievements are not necessarily our achievements.1 If we create something unbound from finitude, it does not follow that we’ve become unbound from our own. Yet for some reason the Altmen all make the assumption that it does.

The idea that all technological development can be viewed as a product of the same yearning to exceed our natural limits—the most concrete and obvious of which is death—has been a central premise of the Artificial Infinitude series. Unease with our own finitude expresses itself in our discomfort with machines that think. Our horror at the numerous apocalyptic scenarios imagined by novelists, filmmakers, and philosophers goes deeper than simple shock; it is more complex than discomfort with the level of human suffering that would come along with the end of the world. It’s not just what those apocalyptic scenarios entail, but how they come about, that reconfigures and amplifies the imagined suffering. Contemplating General AI creates an envisionable scenario that throws our finitude into sharp relief. Existential dread arises from the tension between humans’ inexorable need to expand beyond their current limits and the fact that doing so often results in psychological pain at both the population and individual level.

In stories like I, Robot and 2001: A Space Odyssey, AI systems that were meant to serve humans end up trying to dominate them. Nick Bostrom has posed the “paperclip problem,” which points out that an all-powerful AI need not have malice toward humans to do them significant damage: if an incredibly powerful AI was tasked with making as many paperclips as possible, and it was easier to do that without humans around, then the command “Kill all humans” might be executed in service of the higher command “Make more paperclips.” Others have likened humanity’s role in the future of AI to a hill of ants in the path of a freeway being built. Certainly, ant status is enough to remind us of our own limitations. It would be no consolation to reflect that the ants commissioned the highway—or highwAI—in the first place.

The idea that Narrow AI is an extension of this development, but General AI would represent a departure, was introduced in the last post, “Artificial Infinitude #3: Finitude and the Two Categories of AI.” Our paradoxical emotions about General AI resemble—and indeed are rooted in—our paradoxical emotions about our own finitude. In many ways, the creation of AGI would represent a final overcoming of our constraints. Yet these constraints would remain with us and render us unable to defend ourselves against AGI, making it an existential threat.

The instant AI reaches General status, it effectively becomes an other, officially completing the transition from one of our tools to an entity that exists outside of our bounds. In doing so, it makes those bounds clear. Whereas narrow AIs are tools in the fight against finitude, General AI can only provide a painful reminder that our finitude still holds. We teach a bird to flap its wings and it flies away, reminding us that we cannot fly. The moment our creation ceases to be finite is the moment it ceases to be ours.

Thus, the being created at the culmination of technological development will, in theory, leave us with our human limits and move on to its own objectives, which may or may not include killing us all (we may not have a say). It would be, as a few observers have pointed out, a spectacular own goal, a hoisting by our own petard. Part of the disappointment here—aside from the end of our species—would be that in the end we did not manage to shed what we sought to: our finitude. But this thought experiment also provides confirmation of what makes us human.

We can cope with the central challenge AI poses to finitude by acknowledging that an apparent contradiction exists. It is a recasting of the eternal contradiction of existence. Accepting that the role artificial intelligence plays in our present and future is and will be full of contradictions can become a life- and humanity-affirming exercise. Computers cannot replicate the feeling we get when we contemplate our own finitude.

Our discomfort with artificial intelligence springs from our discomfort with our own mortality and lack of omnipotence. Perhaps the resolution to both shades of discomfort can be found in the same place: a middle ground where we accept the reality of our own finitude but recognize the value in the fundamentally human act of pushing upward against it. Just as Hägglund and Kass argue that our mortality provides our lives with an essential ingredient of meaning, we can steady the connection to our own humanity by positioning ourselves within a theoretical ecosystem in which AGI is present.

Though AGI reminds us that we will never defeat finitude, its presence can also anchor each of us in the comfort of knowing what we are. Artificial General Intelligence, once it is born, may live forever. But it cannot live on.