Finding Common Ground in AGI Strategy Debates

by Joel Predd, Jim Baker, Ben Boudreaux, Matt Chessen, and Edward Geist

The debate over what to do about powerful AI is stuck—not because the participants lack conviction, but because they lack a shared foundation of understanding. Those who warn of extinction, those who urge acceleration, and those who question the premise are often arguing past each other, in part because they place different weights of emphasis on benefits, costs or risks without acknowledging a largely shared view of what’s at stake and what humanity must do.

In our latest RAND perspective paper, we offer a straightforward claim: humanity must preserve its survival, capacity for meaningful agency, and strategic optionality. And to do so, it must build two interdependent components—an ecosystem enabling human-AI coexistence where humans retain meaningful agency, and a global security architecture that maintains enough stability among competing actors to make building that ecosystem possible. This diagnosis and prescription, we believe, should be agreeable to all pro-human perspectives, irrespective of one’s views on timelines, technical paradigms or trade-offs.

We start from the premise that the emergence of transformative AI will be an inflection point. For convenience we will call this artificial general intelligence (AGI), but whether AGI, artificial superintelligence (ASI), or even narrow but economically and strategically transformative AI, this is technology capable of reshaping the trajectory of civilization. Such transformation is not unprecedented. The agricultural revolution, the industrial revolution, and nuclear fission each reordered economies, polities, and the global balance of power. Each increased material wealth and human capabilities while posing new risks. The agricultural revolution massively increased food production but also enabled the concentration of resources and political power, giving rise to hierarchical societies where elites controlled the labor and lives of the many. Industrialization lifted hundreds of millions from subsistence poverty but also enabled total warfare and industrial-scale destruction of civilian populations. Nuclear fission gave the promise of abundant, low-emissions energy but forced humanity to confront its capacity for self-destruction.

AGI will follow a similar pattern—offering historically unimaginable benefits for human flourishing, while introducing threats that jeopardize our physical survival and our ability to retain agency over our future.

Survival is more than biological persistence; it implies maintaining a viable population, genetic diversity, and technological capacity across millennia. But survival without freedom of choice and purpose would not constitute human flourishing. Human agency—the capacity to choose and pursue what one values—is what gives meaning to survival.

Agency operates at multiple levels. At the individual level, agency is the ability to set goals, act intentionally, and make meaning and purpose. Individual-level agency means that humans possess both the capacity for self-reflection and a social environment that empowers them to reflectively endorse their own preference. This could include making choices an AI might not recommend, such as devoting years to mastering a craft that AIs could perform more efficiently.

At the societal level, agency is the capacity to maintain collective control over shared institutions and resources and preserve optionality in the face of irreversible technological or political lock-in. Societal agency means it is up to humans and not AIs to decide whether they want to form emotional relationships with chatbots or to bulldoze Central Park to turn it into a data center.

For strategy, agency matters for a practical reason: any plan for navigating AGI will need to be revised as circumstances change, and that is only possible if humans retain the capacity to actually change course. Agency is what provides human dominion over tools—including AGI.

Optionality means actively developing and maintaining the choice to advance, shape, constrain, and, if necessary, halt AI development and deployments, while preserving the institutions, capabilities, and human relationships needed to exercise meaningful choice across an uncertain future.

Powerful AIs, even those developed prior to AGI, pose threats to both human survival and agency. On the survival side: AI-enabled bioweapons, conflicts among AI-powered nations spiraling out of control, artificial or synthetic life disrupting planetary ecology, and rogue AIs pursuing misaligned objectives.

On the agency side: totalitarianism enhanced by AI surveillance, the incremental disempowerment of humans as AIs assume decision making roles in the economy and society, and the atrophy of critical reasoning and social connection as cognition is outsourced to AIs.

These dangers need not appear suddenly; they may accumulate until humanity finds itself alive but no longer self-determining—perhaps living comfortably but unable to change the trajectory of civilization. This reframes a tension that has consumed too much energy in the AI debate: rather than debating the rate of AI progress and deployment, any strategy—whether it favors acceleration or caution—must demonstrate that it preserves a sufficient level of human agency and choice to, at minimum, allow course correction.

Humanity can survive with agency if, and only if, it builds two interdependent structures.

First, we must build an ecosystem where humans and AIs coexist while preserving human agency. This encompasses not just AI systems but the digital, institutional, and physical environments within which they operate. Technical work on alignment, safety, interpretability, control, and evaluation is crucial but insufficient. Perfect alignment may not be guaranteed, and technical fixes alone cannot preserve agency within flawed institutions or adversarial political systems. Political systems must preserve legitimacy and the social contract as AI disrupts economies and political discourse. And humans must maintain the cognitive capacities that support self-determination—critical reflection, moral reasoning, social engagement, and creativity.

As AI systems become more capable, more-coercive interventions (e.g. containment, control and shut-down mechanisms) that are effective for early AGIs may not work for artificial superintelligences. Over time, we may become more reliant on shaping the motivations and incentives of AIs so that they take steps to support the preservation of human agency. This reinforces why agency is not merely one value among many but a precondition for navigating an uncertain future.

Second, we must build a global security architecture that maintains stability among the actors developing that ecosystem. This architecture must achieve four interrelated objectives: creating temporal space for the ecosystem to develop, preventing catastrophic conflict driven by fears of “decisive strategic advantage,” establishing minimum viable governance with risk-reduction mechanisms, and managing the AI incidents and failures that will inevitably occur before governance matures.

Neither structure alone is sufficient. An ecosystem designed for human-AI coexistence cannot develop if nations and firms are racing towards catastrophe. A security architecture that maintains stability but doesn’t foster an ecosystem just manages continual human decline if AI capabilities continue to accelerate. More specifically: we cannot rely solely on technical solutions to alignment; the ecosystem must include institutional and social adaptations. The ecosystem cannot develop without a security architecture to manage racing dynamics, nonstate actor risks, and threats that are agnostic to which actor triggered them. And resource constraints, combined with timeline uncertainty, mean that both structures must be built simultaneously rather than sequentially—neither can wait for the other, because each is a precondition for the other’s success.

Humanity does not yet know with confidence how to build an ecosystem where humans and AIs coexist while preserving human agency. As we wrote elsewhere, the unfortunate reality is that in the field of advanced AI, we lack the kind of foundational scientific understanding that physicists at Los Alamos possessed when their calculations gave them confidence that the Trinity test would not ignite the atmosphere. Instead of established theory, we are left to rely on the expert judgment of the scientists and engineers at the frontier of AI development, and that community is divided on prospects and pathways.

Our global security architecture is in the midst of transformational change, independent of AI, as the elements of post-Cold War security architecture are upended without a clear alternative in sight. Just as the nuclear architecture—arms control agreements, confidence-building measures, crisis communication hotlines, and shared norms—stabilized Cold War competition, a comparable structure for AGI may need to create baseline cooperation even amid rivalry. But unlike nuclear weapons, AI development is distributed across private actors and many countries.

One of the most important implications of this framework is that the emphasis on agency and optionality provides goals capable of bridging otherwise incompatible worldviews—including the strategic perspectives of the United States and China, the views between the national security and AI communities, and the diverse communities in the global AI ecosystem.

All pro-human actors, despite profound differences in political systems, values or visions of technological destiny, share an interest in avoiding futures where their capacity for self-determination is foreclosed. Neither the United States, China, nor any other state benefits from AI-induced catastrophe, uncontrollable AI systems, or a world order imposed by a rival’s technological dominance. All have reason to preserve the option to shape their own AI trajectories rather than having those trajectories dictated by racing dynamics or the other’s unilateral choices.

This framework is designed to inform strategy from two distinct vantage points. From a universal perspective (what might be called humanity’s view) it identifies conditions under which any human future remains viable. From the perspective of U.S. national strategy, the same framework identifies how American interests are served by a world where AI development proceeds without catastrophe, where no rival achieves permanent dominance, and where democratic societies retain the capacity for self-governance. These perspectives are not identical, but they are compatible. And that compatibility is what makes the framework actionable rather than merely aspirational.

Our paper does not offer specific policy prescriptions by design. If you accept the central argument, then humanity’s theory of victory is not a fixed end state but the preservation of optionality: the capacity to keep choosing, adapting, and revising course as technology and society evolve.

What the paper does offer is a coherent starting point: a diagnosis that names what is at stake, a prescription that identifies what must be built, and a strategic logic that accommodates both universal human interests and the interests of nations navigating a technological transformation. Survival and agency are not merely values to be weighed against others; they are preconditions for any values to be pursued at all. Without survival, there is no future to shape. Without agency, humans become objects of history rather than its authors.

From this foundation, the harder work of strategy development can begin. More from RAND to follow.