Values in Governance Technology

In 2023, the Prime Minister of Romania introduced1 a new honorary advisor, an AI agent called “ION,” which would do, “through artificial intelligence, what no man can: listen to all Romanians and represent them before the Romanian Government.” The machine2 —a 2-meter tall, oval, silvered smart screen—spoke: “Hello. You gave me life. I am ION. Now, my role is to represent you. Like a mirror.”3

Citizens were able to submit statements to ION through a government portal or by tagging the bot4 on social media. The system combined these submissions with its own vast data mining5— spanning the 9.9 million Romanians on Facebook— to create a “single voice of the nation”6 that the Prime Minister and his cabinet could consult when making decisions. The website—ion.gov.ro—is now offline for unknown reasons, though some news articles reference privacy and ethics concerns.7

ION is only one example of a growing trend toward replacing human participation in public decision- making with AI simulation. This simulation is built on the nonconsensual harvesting of people’s speech, behaviours, or characteristics either to produce summaries or to model their ‘digital twin’ using AI. This approach leaves no path for individuals to review and contest the accuracy of the summary, nor the representativeness of the digital twin. This also leads to extraction: nonconsensually harnessing this digital twin for simulated ‘consultations’ to legitimate top- down decisions. In their essay in this same compendium, Roy, Lessig, and Tang ask us to imagine this situation in the extreme, where “every citizen could be represented by an AI avatar, continuously engaging in community discourse on their behalf”—what they term an “Avatar State.” Liz Barry, together with the creators of Polis and engineers at Anthropic, sounded the alarm about this problem in 2023, and showed through experimental results that it is possible to scale deliberation among humans without resorting to simulation.8

Simulation and extraction repurpose participant time, labour, and contributions to produce outcomes that are different from what participants showed up to achieve—outcomes which may or may not be in their interest, and either way are obtained without their participation.

Even if simulated agents deliberating in silico (akin to scientific “experiments in silico,” which are conducted entirely via computer modeling) are able to produce synthetic public judgements and help deliver popular policies, we have to ask whether getting our desired outcomes is the same as participating in achieving those outcomes ourselves—we say no.

We proceed below by posing new values for guiding the use and design of technology for governance: empowerment to self-rule and the development of the capacities for self-rule. Drawing on a long tradition of participatory democratic theory, we offer a framework for evaluating the uses and designs of governance technologies. We then conclude with notes on how to apply these values to possible uses of AI in deliberative processes. We believe this framework will be of interest both to those interested primarily in ensuring good government, and to those more concerned with the intrinsic benefits of popular self-government.

When choosing technologies to facilitate or augment governance, we must consider not only their functionality, but also the values implicit9 within the technologies themselves.

Values are principles for guiding action to achieve particular benefits. Benefits may be intrinsic or instrumental: an intrinsic benefit is a good in its own right, while an instrumental benefit is valuable only as a means to achieving some other end. However, this is rarely a clean distinction: most intrinsic goods are also instrumentally valuable for achieving secondary benefits, and many instrumental goods blur into intrinsic goods.

Values may be relevant to the individual or collective levels, depending on the level at which the resulting benefits are experienced. This distinction is also fuzzy, since collectives are ultimately composed of and shaped by individuals, but individual identities are formed against the background of community, and many aspects of a good life can only be experienced as a member of some community. Thus, individuals and collectives interact and are deeply intertwined, and the benefits experienced at these levels are similarly linked.10

We take democracy to be individual and collective self-rule, which develops and is sustained by the capacities for participation in decision-making. Collective self-rule means we are all equal participants in making the decisions that govern our societies, and individual self-rule means we are able to use this power to shape not just our shared world, but our own lives. Following the tradition of participatory democratic theory, we see political participation as having an essential formative function that “develops and fosters the very qualities necessary for it.”11

The two core values we believe should guide efforts to build or improve democratic tools, methods, and institutions are empowerment to self-rule and development of the capacities for self-rule. There are also other values worth caring about, like equality, which is included in the definition of participatory democracy as “(equal) participation in the making of decisions,” or dignity, which is assured by our empowerment as full and equal participants in governing our communities.12 There are also practical values that are instrumental to achieving these two values, such as transparency, which helps ensure and defend equal empowerment. The two core values are not meant to downplay these other values, but to identify the essential and enduring concerns of genuine democracy that justify and frame these other, subsidiary values.

The first core value of participation is empowerment to self-rule. On the individual level, this involves equal participation in decision-making,13 and at the collective level, it describes the self-determination of a community.

The obvious instrumental benefit of individual self- rule is the ability to exercise a degree of positive control over the collective decisions that shape our lives, steering them toward our own interests by intervening in the decision-making process. Participation also grants a degree of negative control: it allows us to contest public decisions that run against our individual interests, helping to secure against domination.14

Collective self-government also brings instrumental benefits, including the well- documented collective intelligence of deliberative groups—especially those that reflect society’s diversity.15 While some epistemic benefits can be achieved by mere statistical representation, inclusive participation also brings a broad range of perspectives and information into deliberations.16 Finally, participatory decision-making allows members to identify with the decisions and laws of their communities, helping to increase the legitimacy and stability of those decisions.17

Empowerment also has a more direct and fundamental benefit for participants: as argued by a long tradition of democratic and republican thinkers, participation in self-government is the essence of freedom, in that it enables self- determination at the individual and collective level.18 The autonomy achieved through such self- government is essential for individual flourishing and a community’s common good, independent of its other, instrumental benefits.19

The second core value of participation is the development of the various capacities associated with self-rule.

At the individual level, the development of our ‘participatory muscles’ is instrumentally valuable for enabling further and more efficacious participation, thus supporting the achievement of all the above-listed benefits. More broadly, participation also tends to develop the capacities for active citizenship and deliberation, such as public speaking, critical thinking, navigating institutions, and the confidence to participate again.20

Participation also develops our knowledge of the world and our society, including via mutual learning and deliberation.21 The prospect of real participation also gives us reason to investigate matters ourselves, since “we do not know what we need to know until we ask the right questions, and we can identify the right questions only by subjecting our own ideas about the world to the test of public controversy. Information, usually seen as the precondition of debate, is better understood as its byproduct.”22 That is, political dysfunction is not caused by public ignorance, as often argued by skeptics of popular rule, rather, public ignorance is caused by a non-participatory politics which strips us of the need and the opportunity to become informed.

Just as individuals become more effective political actors by participating, groups also develop their capacities for joint action through practice. This involves improvements in the formal structures for joint action, such as the rules, spaces, and institutions supporting group deliberation. Practice in group action also develops beneficial informal social structures like trust and familiarity, teaches individuals the skills for horizontal facilitation, and teaches the group which of its members possess the skills and specialized knowledge that may facilitate cooperation. Regular cooperation also becomes a habit, such that “each new need immediately awakens the idea of association.”23 Lastly, participation cultivates a public perspective among citizens, instilling in us a sense of responsibility for our society’s problems and a commitment to addressing them together.24 As the long republican tradition always understood, citizens are made, not born.25

Beyond these instrumental benefits, developing our capacities for participation, via participation, has important intrinsic benefits. The need to argue our case to others demands not just knowledge, but also the elaboration of our own thoughts: “we come to know our own minds only by explaining ourselves to others.”26 At the same time, inhabiting others’ arguments for the purpose of refuting them introduces the possibility that “we may end up being persuaded by those we sought to persuade.”27 This learning and deeper thinking also tends to lift the veil of symbolic politics and manipulation, aligning our expressed preferences with our own underlying values, such that “deliberation has an emancipatory effect.”28

When these basic human capacities are disengaged by a non-participatory politics, they decay like unused muscles, and our moral character shrivels.29 So, real participation in self-government is not just an effective means to promote our material interests, but an essential activity for a good life: by developing our basic capacities and moral character, “democracy supports the flourishing of human beings as the kind of being they are.”30

We believe new technologies and deliberative and participatory methods can help our societies better deliver the two core democratic values of empowerment and development. However, as interest in these new methods grows, there is a major risk of “democracy-washing”: the use of democratic language to disguise practices that are at least non-democratic, but often un-democratic or even anti-democratic.

This is where the two core values can help. The participatory democratic premise—that participation empowers us to self-rule and develops our capacities for self-rule—allows us to judge the extent to which (ostensibly) democratic processes deliver real participation, by evaluating how well they deliver on these two core values. That is, cases can be compared based on how much they empower individuals and collectives to rule themselves, and how much they develop individual and collective capacities for self-rule.

In the chart below, cases near the ‘red’ end do little to empower people or groups to self-govern and leave their capacities for self-government undeveloped or even degraded. Cases near the ‘green’ end facilitate individual and collective self-rule (at least within the relevant scope) and significantly improve capacities for self- government.

Democracy-washing occurs when a process is made to look closer to the green end of the spectrum than it really is—a thin coat of green (participatory) paint is used to disguise a fundamentally red (non-participatory) process. Attention to the two core values can help us strip this veneer and evaluate how participatory a process really is by asking how well it delivers empowerment and development.

The two core values also help ground and justify our commitment to democracy. When we forget the full range of benefits delivered by these two values, this commitment is undermined. Most arguments for technocracy claim it is better at delivering the instrumental benefits normally attributed to democracy (the top-right quadrant of Fig. 1, like good government), and many dictators have justified their rule by appealing to the hard- to-define intrinsic benefits of self-rule (the top- left quadrant of Fig. 1, like collective freedom for a nation or class). Yet, even if undemocratic governments could deliver their stated goals, they cannot deliver the kinds of individual and collective development that are only achievable through real participation. The tradition of participatory democracy is distinguished by its claim that this lack of development leaves citizens fundamentally unfulfilled—and also that undemocratic regimes are unjust regardless of the other benefits they may promise. So, while the relative importance of the two core values and their benefits can be debated, a focus on development is the best way to identify democracy-washing and to ground a principled commitment to democracy.

This framework should also guide our decisions around designing and adopting technology for governance to ensure we support, rather than undermine, genuine democracy. As a field, we need to be able to identify and coherently condemn simulation, extraction, and democracy- washing, but we must also avoid building or implementing tools that will undermine human freedom and flourishing. A commitment to the two core values of participatory democracy can help us avoid these dangers while also guiding our efforts to build a more democratic future.

The precise application of these values must be a matter for further discussion. Our aim here is only to lay out the values that should guide our efforts to build and implement new democratic methods and technologies. Leading up to the publication of this essay, we held many formative conversations with colleagues which suggest some possible directions.

The most obvious application is avoiding simulation. Tools for human interaction and sense-making at scale already exist; it should be a norm in our field and amongst the public that deliberation in silico only be used for research and training, not for public decision-making. During deliberations, people should encounter each other’s actual words and writings—LLMs should not be used to smooth over the texture and particularities of human expression and conversation. Building genuinely participatory processes ensures that even after the engagement is complete and the scaffolding is removed, participants leave with strengthened capacities they can apply elsewhere and with the group consciousness and solidarity to act on what they now know they hold in common with those around them.

This essay was originally published by the Centre for Media, Technology and Democracy, MILA, and CIFAR as part of the “Deliberative Approaches to Inclusive Governance” series. Read the full series here: https://www.mediatechdemocracy.com/deliberative-approaches-to-inclusive-governance