Earning a Social License for Transformative AI

Claude’s constitution and equivalent documents at leading AI labs inform the behavior of the most powerful technological tools deployed by humanity. These documents specify the values and guiding directives that steer how frontier AI models act in the real world. To the surprise of some, these constitutions are very effective. By way of example, Anthropic’s recent models are arguably the most aligned they’ve released. This is a big deal.A People’s AI Constitution Council might represent the novel approach necessary to meet the unique democratic demands of transformative AI. While the precise contours of this proposal deserve deeper analysis from a wide range of stakeholders, a brief sketch of the Council suffices to get the conversation started about how this or a related approach could function in practice.

The proposal is straightforward: a randomly selected body of citizens—the Council—chooses a small set of model constitutions, and access to federal contracts is conditioned on adopting one of them. In short, it would become a procurement standard that a model adheres to one of the Council’s constitutions. This mandate would address the current concern that the public has little to no oversight over the characteristics of the AI models presently being adopted by the government.

To avoid the odds of the Council favoring constitutions with a narrow or biased set of interest, the Council would include a representative group of 100 individuals from across the country; they would serve two-year terms. Prior to the start of their term, Council members would undergo a three-month briefing on the fundamentals of AI and, once those basics are down, the latest thinking on constitutional AI. Training materials would be compiled by AI experts in government and industry. The Center for AI Standards and Innovation (CAISI) would likely play a central role in compiling and vetting that material.

The Council would then be tasked with selecting—not drafting—three constitutions. The pool of potential constitutions would be drawn from frontier AI labs, a handful of AI research organizations--both nonprofits and university programs, and submissions from the general public with at least 100,000 electronically gathered signatures (electronically gathered). CAISI would provide objective analysis of submissions to ensure none of them lack key components or otherwise pose undue risks. CAISI would not select constitutions. To be clear, its role would be to evaluate how models trained under each constitution perform, identify emerging risks, and recommend review when capabilities change.

Selected constitutions would remain in place for two years, subject to CAISI review. If, for example, frontier AI capabilities progressed in a way that rendered any one of the existing constitutions inadequate, then CAISI could convene the Council and suggest sample amendments. Subsequent Councils would have the ability to retain the constitutions selected by the prior Council, amend them, or adopt new constitutions to replace one or more of the existing constitutions.

Frontier AI labs would not be under any legal obligation to train their models pursuant to the Council’s constitutions. They could make the choice to forgo federal contracts that require as a basis of procurement that models be trained to those constitutions. Of course, it seems likely that few labs would opt out of such a lucrative business line. The incentive to adopt one of the Council’s constitutions is strengthened by likely spillovers. Large firms, nonprofits, and other users will have reason to prefer models that meet the same standard, effectively extending the Council’s influence beyond federal procurement. Note, however, there would still be plenty of room for model developers to offer models trained to different standards.

Principles to guide public involvement in transformative AI governance

This proposal is subject to myriad questions and is not perfect. The devil as well as fatal flaws are in the details. Regardless of whether this specific proposal warrants popular endorsement, a few principles can guide analysis of the Council and related ideas:

First, as technology eliminates the barriers that once prevented more regular and substantive public engagement, new democratic processes will need to emerge. Imagine being gifted a new car but told you could never use it. You’d rightfully be pretty upset if your parents insisted you stick with the bus--it’ll still get you to where you need to go but it denies you the autonomy offered by your new. TheOur republican system was formed around the idea that large scale deliberation was not possible. Representative governance was a practical necessity.

Those days may be coming to a close thanks to new deliberative democracy techniques. Researchers at Google DeepMind, scholars at Stanford’s Deliberative Democracy Lab, and organizers around the world are leaning into technological advances and popular demand to address the logistical barriers that once foreclosed more participatory governance mechanisms. “The Habermas Machine” developed by Google DeepMind, for instance, leverages AI to assist large groups with finding consensus statements on complex topics. It’s foreseeable that as AI improves and its integration into public affairs becomes more common, the public may insist on these new techniques becoming a regular part of their democracy. The odds of such public pressure are heightened by the fact that distrust in elected representatives and legislative bodies will persist and, likely, worsen in the near future. The lag between AI development and diffusion and government policy creation is growing. Take, for instance, the fact that Congress has yet to pass legislation related to the Mythos Moment, such as bolstering the Cybersecurity Infrastructure Security Agency and related cybersecurity institutions. Members of the public will cite those missed opportunities as justification for substantive changes. They will have viable alternatives to point to.

Second, with the institutional constraints of lawmakers in mind, it’s important that AI governance efforts focus on regulating outcomes more so than the specifics of today’s AI models. The well-intentioned, yet misguided efforts of contemporary officials to legislate AI as if it were a static technology will be the subject of ridicule by future generations. Future democratic human-based governance must be tailored to the parts of the AI stack and development process that are still amenable to deliberations and judgements that cannot transpire in seconds. For example, legislators should focus less on designing policy around models with specific attributes, such as the number of parameters in a model, and more on shaping policy to achieve or prevent certain outcomes. The latter strategy will result in the creation of different regulatory ecosystem in which investment is made in evaluation techniques that surface if models of various designs do or do not satisfy the legislature’s mandates.

Third, the threat of a mass opposition to AI development and diffusion will grow so long as AI developers (and their defenders) ignore the question of social license. A mass movement against AI is not outside the question—especially so long as a handful of individuals generally perceived as out of touch and unsympathetic to the plight of the commoner are left at the steering wheel of the most important technology.

Aligning highly capable models to the specifications of their developers has been the subject of much debate and even more technical investigation by some of the smart minds on the planet. Though AI experts at frontier labs would stop far short of concluding that they’ve cracked the alignment nut, their continual improvements in baking certain values, norms, and rules into their models is noteworthy. All signs suggest that they will not only develop more sophisticated models but also do so with a greater ability to influence how the model performs in myriad circumstances.

The upshot? She who shapes a model’s constitution shapes the model; he who shapes the model shapes the world. Today, that power sits with a small number of private actors.

Rather than John or Jane holding the pen during that drafting process, you and I—the general public that will live with the consequences of certain values being selected or prioritized over others—ought to have some role. This essay details a possible means to achieve popular oversight of and participation in AI constitution drafting and revising.

“The People’s AI Constitution Council” is one approach to fulfilling that ambitious goal.

A randomly-selected group of citizens would serve stints as councilors. They’d be tasked with selecting three constitutions from which frontier AI models can be trained to qualify for use by the federal government. Labs would be free to forgo seeking federal contracts or to train models pursuant to other constitutions. However, this process would ensure a meaningful opportunity for we the people to ensure the AI systems that increasingly shape governmental operations and decisions align with our values.

The limits of private alignment and public legitimacy

No formal democratic oversight shaped Claude’s constitution nor the equivalent training guardrails developed and imposed by other frontier AI labs. This gulf in public participation is unsustainable.

The social license for frontier labs is already under strain. Public opposition to data centers is growing. Skepticism about whether AI will deliver broad-based gains is widespread. As models become more capable while remaining shaped by a narrow set of internal decisions, that tension will intensify.

That’s precisely why the creation of transformative AI will also demand the creation of a transformative democratic oversight mechanism. Dario Amodei, CEO of Anthropic, has arguably recognized as much. He floated the possibility of existing democratic institutions shaping AI constitutions. However, he doubted whether current legislative bodies had the institutional capacity to take on such a technically nuanced and weighty task.

Many Americans share those concerns with respect to the US Congress. A low estimation of Congress across the general public also renders it ill-suited for this task. Any decision by lawmakers in D.C. (assuming they were able to reach a majority decision) would be reflexively challenged by a significant share of the population. Such an outcome would fail to remedy the aforementioned democratic deficit associated with determining the values of the most consequential models.

Why existing models of oversight fall short

New democratic institutions have commonly emerged in response to novel democratic challenges. The Second Industrial Revolution coincided with the introduction and spread of the initiative and referendum, the democratization of electing US senators, and expansion of the franchise to women. The post-World War II boom—economic, demographic, and technological—also sparked a wave of super-statutes that altered the nation’s governance. The modern technological moment has arguably yet to see a responsive update. Attempts by the private sector to trial new processes have fallen short of expectations.

Meta, then-Facebook, previously subjected some platform policies to a user plebiscite. Lower turnout rates doomed the effort. The Oversight Board, later created by Meta to serve as an arbiter of content moderation decisions, has likewise struggled to find its footing. Composed of elite lawyers, former policymakers, and representatives of major civil society organizations, the Board lacks a direct connection to the platform’s users. A lack of institutional capacity to handle the deluge of appealed content moderation decisions has also saddled the Board. Observers have questioned its case selection approach, which often seems responsive to headlines more so than the interests of users. Yet, few would prefer an alternative in which Congress or any other formal representative body usurped the private judgment of Meta or any other platform provider to exercise broad discretion over the design and character of their offerings.

These experiments related to Meta show the limits of both internal governance and elite-led oversight bodies. Neither provides a durable foundation for decisions that shape general-purpose intelligence systems. Yet, there’s also a real risk to participation washing—”turning [public engagement processes] into a tokenistic practice.” What follows is one potential throughline: meaningfully soliciting public input and ensuring it shapes private sector actions.

Designing the ‘People’s AI Constitution Council’

A People’s AI Constitution Council might represent the novel approach necessary to meet the unique democratic demands of transformative AI. While the precise contours of this proposal deserve deeper analysis from a wide range of stakeholders, a brief sketch of the Council suffices to get the conversation started about how this or a related approach could function in practice.

The proposal is straightforward: a randomly selected body of citizens—the Council—chooses a small set of model constitutions, and access to federal contracts is conditioned on adopting one of them. In short, it would become a procurement standard that a model adheres to one of the Council’s constitutions. This mandate would address the current concern that the public has little to no oversight over the characteristics of the AI models presently being adopted by the government.

To avoid the odds of the Council favoring constitutions with a narrow or biased set of interest, the Council would include a representative group of 100 individuals from across the country; they would serve two-year terms. Prior to the start of their term, Council members would undergo a three-month briefing on the fundamentals of AI and, once those basics are down, the latest thinking on constitutional AI. Training materials would be compiled by AI experts in government and industry. The Center for AI Standards and Innovation (CAISI) would likely play a central role in compiling and vetting that material.

The Council would then be tasked with selecting—not drafting—three constitutions. The pool of potential constitutions would be drawn from frontier AI labs, a handful of AI research organizations--both nonprofits and university programs, and submissions from the general public with at least 100,000 electronically gathered signatures. CAISI would provide objective analysis of submissions to ensure none of them lack key components or otherwise pose undue risks. CAISI would not select constitutions. To be clear, its role would be to evaluate how models trained under each constitution perform, identify emerging risks, and recommend review when capabilities change.

Selected constitutions would remain in place for two years, subject to CAISI review. If, for example, frontier AI capabilities progressed in a way that rendered any one of the existing constitutions inadequate, then CAISI could convene the Council and suggest sample amendments. Subsequent Councils would have the ability to retain the constitutions selected by the prior Council, amend them, or adopt new constitutions to replace one or more of the existing constitutions.

Frontier AI labs would not be under any legal obligation to train their models pursuant to the Council’s constitutions. They could make the choice to forgo federal contracts that require as a basis of procurement that models be trained to those constitutions. Of course, it seems likely that few labs would opt out of such a lucrative business line. The incentive to adopt one of the Council’s constitutions is strengthened by likely spillovers. Large firms, nonprofits, and other users will have reason to prefer models that meet the same standard, effectively extending the Council’s influence beyond federal procurement. Note, however, there would still be plenty of room for model developers to offer models trained to different standards.

Principles to guide public involvement in transformative AI governance

This proposal is subject to myriad questions and is not perfect. The devil as well as fatal flaws are in the details. Regardless of whether this specific proposal warrants popular endorsement, a few principles can guide analysis of the Council and related ideas:

First, as technology eliminates the barriers that once prevented more regular and substantive public engagement, new democratic processes will need to emerge. Imagine being gifted a new car but told you could never use it. You’d rightfully be pretty upset if your parents insisted you stick with the bus--it’ll still get you to where you need to go but it denies you the autonomy offered by your new. The republican system was formed around the idea that large scale deliberation was not possible. Representative governance was a practical necessity.

Those days may be coming to a close thanks to new deliberative democracy techniques. Researchers at Google DeepMind, scholars at Stanford’s Deliberative Democracy Lab, and organizers around the world are leaning into technological advances and popular demand to address the logistical barriers that once foreclosed more participatory governance mechanisms. “The Habermas Machine” developed by Google DeepMind, for instance, leverages AI to assist large groups with finding consensus statements on complex topics. It’s foreseeable that as AI improves and its integration into public affairs becomes more common, the public may insist on these new techniques becoming a regular part of their democracy. The odds of such public pressure are heightened by the fact that distrust in elected representatives and legislative bodies will persist and, likely, worsen in the near future. The lag between AI development and diffusion and government policy creation is growing. Take, for instance, the fact that Congress has yet to pass legislation related to the Mythos Moment, such as bolstering the Cybersecurity Infrastructure Security Agency and related cybersecurity institutions. Members of the public will cite those missed opportunities as justification for substantive changes. They will have viable alternatives to point to.

Second, with the institutional constraints of lawmakers in mind, it’s important that AI governance efforts focus on regulating outcomes more so than the specifics of today’s AI models. The well-intentioned, yet misguided efforts of contemporary officials to legislate AI as if it were a static technology will be the subject of ridicule by future generations. Future democratic governance must be tailored to the parts of the AI stack and development process that are still amenable to deliberations and judgements that cannot transpire in seconds. For example, legislators should focus less on designing policy around models with specific attributes, such as the number of parameters in a model, and more on shaping policy to achieve or prevent certain outcomes. The latter strategy will result in the creation of different regulatory ecosystem in which investment is made in evaluation techniques that surface if models of various designs do or do not satisfy the legislature’s mandates.

Third, the threat of a mass opposition to AI development and diffusion will grow so long as AI developers (and their defenders) ignore the question of social license. A mass movement against AI is not outside the question—especially so long as a handful of individuals generally perceived as out of touch and unsympathetic to the plight of the commoner are left at the steering wheel of the most important technology.