AGI and Lock-in

The long-term future of intelligent life is currently unpredictable and undetermined. We argue that the invention of artificial general intelligence (AGI) could change this by making extreme types of lock-in technologically feasible. In particular, we argue that AGI would make it technologically feasible to (i) perfectly preserve nuanced specifications of a wide variety of values or goals far into the future, and (ii) develop AGI-based institutions that would (with high probability) competently pursue any such values for at least millions, and plausibly trillions, of years.

This is a piece of research that I wrote together with Jess Riedel and Carl Shulman while I was at the Future of Humanity Institute. It’s available on Forethought’s website at forethought.org/research/agi-and-lock-in.