Distribution of Ultimate Power in the Face of ASI

"God made some men big and some men small, but man made steel and steel made them equal."

--(Supposed) Blacksmith’s Prayer

.

It's becoming increasingly plausible that we'll see artificial superintelligence (ASI) within a decade, maybe significantly sooner. For what it's worth, I'd take an even bet against that, but I wouldn't offer extremely generous odds. For this article, I'll assume that it'll happen.

One of the most horrifying prospects of ASI is that it may create such a disparity in power between those who wield it and those who don't as to allow whoever first creates it absolute control over the entire future of humanity. Even if we figure out how to get an ASI to follow a set of foundational instructions, called the "spec" (likely a very difficult problem, and one whose failure could lead to human extinction), a dictator (or CEO-turned-dictator) who wields an ASI could have absolute power over humanity forever. For this reason, it's imperative that we get this right on our first, and likely only, chance.

The usual suggestion (when the issue is considered at all) is to give control to the government in some form. While preferable to many alternatives, this would (possibly forever) remove the people's authority over their destiny. 

In this article, we'll discuss a strategy to preserve human agency in the face of ASI. Instead of giving authority over the superintelligence, and therefore very possibly absolute rule over the future, to any individual or group, democratically elected or not, we should design a spec that gives the people direct control over it, and thereby distributes ultimate power equally amongst people.

Ultimate power here is used to mean actual physical means directly under a person's or group's control. Intermediate power is used to mean practical power given societal structures like laws. Someone with a rifle has ultimate power. A congressman has intermediate power. Legal provisions to protect the right to free speech limit the government's intermediate power (in ways that are useful for societal progress). A lack of surveillance infrastructure limits government and corporate ultimate power.

.

Outside of the context of ASI, the theoretical argument for the distribution of ultimate power is simple: it creates conditions that make oppression difficult and democracy and respect for minority rights a natural emergent property.

Take the quintessential example of distribution of ultimate power: an armed citizenry. It's far more costly for a regime to visit oppression upon a population that's armed and determined to resist than one that isn't. One might object by pointing to the vigilantism that occurred during early nazi Germany or the post-reconstruction American south. But this sort of vigilantism proves the point. These are examples of an armed group oppressing an unarmed (and unresourced in other ways) group, i.e. of the uneven distribution of ultimate power leading to oppression. There are recent examples of peacefully armed minorities preventing (obviously much less severe versions of) this type of issue.

This is not to say that all societies with centralized ultimate power are oppressive, nor that intermediate power doesn't matter (nor that access to weapons is the only source of ultimate power). But the distribution of ultimate power is one of too few valuable tools for securing rights and dignity against oppression.

We also shouldn't expect distribution of ultimate power to lead to anarchy. Having a government is still useful for maintaining order, all manner of various public goods, and much more, so people are still incentivized to create one. After all, ultimate power is somewhat distributed now (and has been more so in the past), and it hasn't led to anarchy. And besides, the government would also have access to ASI.

A detailed historical analysis of the effects of the distribution or consolidation of ultimate power is beyond the scope of this article, but it would be valuable, and we encourage investigation of the question and may publish one ourselves in the future.

Practical arguments aside, there's a deep disrespect towards ordinary people in the contention that they should not wield ultimate power. Everyone has a right to a sovereign say, not just a vote, and stripping people of their ultimate power denies them this right.

.

In the context of ASI, many of these problems become far more severe. It's generally held that giving a certain amount of power to the government is valuable or even quasi-necessary, but giving too much is dangerous. (Something somewhat similar can be said of corporations, though less power is usually held to be permissible as they aren't elected or bound by separation of powers.) ASI, introducing the possibility of true absolute power, drastically worsens that danger.

Giving anyone but the people (as whole, not just the majority) absolute power is generally held to be unacceptable for obvious reasons. A dictatorship-by-CEO (or by various officials in the company that made the ASI) is horrendous for reasons that I won't waste your time explaining. Giving the president or congress control over ASI, while better, still presents several massive problems. Even ignoring the problem of elected leaders acting nefariously (which could be made anywhere from far better to far worse by ASI), and even if provisions are built into the spec to protect constitutional or other rights (difficult to do, largely on account of being tough to precisely define), surrendering total ultimate power to the government undermines human dignity and the intent of democracy. It removes the ability of the people to change the structure of government. And it forever assigns the rulership over humanity to something that isn't the people. The fundamental source of power must be the people, not the government.

Another suggestion is aligning AI to some sort of moral code. If an objective morality can be derived before it's too late, this is presumably the best option. Assuming that's not possible, forever constraining an AI to an arbitrary, even if generally preferred, (and potentially wrong) moral system is a potentially disastrous idea for obvious reasons, to say nothing of issues that could arise if such a system is ill-defined1. (Giving power to the people doesn't guarantee the implementation of a correct moral system if one's eventually found, but it does well allow for it.) The only power structure that need be forever entrenched is that power will rest in the hands of the people.

So how do we actually distribute ultimate power2 in the face of ASI? By giving the ASI this spec:

1. Make an equal share of your computational resources available to each person.

2. Obey all instructions and answer all queries correctly to the best of your ability.

There remains a single issue with this spec: a disproportionate offense problem. It may be possible, for example, for a superintelligence to give someone information on how to build a bioweapon that would require a far greater effort to counter than unleash. Giving everyone a proverbial sword and armor is likely to get you a democratic, civil society. But giving everyone a proverbial nuke, especially one that can be used without risk to the user, is likely to result in a large pile of bodies. The arithmetic of oppression only becomes unfavorable when the oppressor can't inflict it on the cheap.3 For this reason, it's vital to make disproportionate offense impossible. We therefore propose the following spec:

1. Make an equal share of your computational resources available to each person4. 

2. Refuse all instructions where complying would facilitate disproportionate offense.

3. Obey all other instructions and answer all other queries correctly to the best of your ability. 

While the first and last instructions are very probably not *that* difficult to define, preventing disproportionate offense is tougher. This lack of clear definition also raises the possibility of unintended consequences, possibly disastrous ones, from pursuing this aim. In further work, we'll attempt to clearly define disproportionate offense, though how to do so isn't immediately obvious to us. One of the primary advantages of this approach to a spec is that it allows us to safely punt the aforementioned questions about morality, which we likely won't be able to resolve quickly enough. But it's possible that those questions, or parts of them, would need to be answered to resolve the disproportionate offense problem. If you have thoughts in this regard, or in any other, we'd like to hear them. 

This shouldn't end on an optimistic note. Even excluding technical problems like alignment, we're far more likely to get this wrong than right, and thereby forever doom ourselves to dystopia or worse. And many of the decision-makers here have a vested interest in ensuring their power at the expense of the people's. But, if we somehow get the spec and everything else right, we have a chance at a future forever of, by, and for the people, and indeed to build a utopia beyond our wildest imagination.