How the UN’s Scientific Panel Erases Human Responsibility for AI

There’s a bullet point on page 38 of the UN’s Preliminary Report of the Independent International Scientific Panel on AI (IISPAI), released last week, that says something sensible: "The primary drivers of harm are design and deployment decisions—– underlying system architectures of influence, including targeting, amplification and behavioral design—not individual outputs." Yet much of the rest of the report undermines this statement, since its contents are in debt to a definition of AI that studiously avoids reference to the humans who develop and train these models.

In an apparent bid to be neutral, the report engages in precisely what its mandate aimed to avoid: politics. This matters, because the UN's assessment is a reference point for AI governance worldwide. A document that obscures human accountability will carry that blind spot into the policies it informs.

Definitional drift on AI from the OECD to the UN

The problem stems from the report’s definition of AI. Representing the consensus of 40 experts, it reflects previous language from the OECD and the EU AI Act, but its revision subtly pushes human designers and developers away from the systems they build — a trend that we can track across many major policy definitions of AI from 2019 to 2026.

For the IISPAI, AI is defined as systems that "perceive, learn and act," "infer," "learn from experience," and "acquire experience." The case studies presented emphasize LLMs for harms and other architectures to name benefits. The authors defend the imprecision, suggesting that "what unifies AI more than any single architecture is that modern systems learn from experience represented by data.”

Some of this imprecision could be tolerated as a way to future-proof policy against rapidly changing technologies. But this does not justify omitting the actors that create technology — definitions should presume humans won’t become obsolete. Equating "experience" and "data" is an ideological twist toward anthropomorphism that steers policy toward a specific position.

The Panel has real credentials from beyond industry. It is co-chaired by Yoshua Bengio, a Turing Award winner whose recent existential risk advocacy has emphasized AI systems that act in ways designers don’t anticipate. But Bengio’s x-risk work relies upon an anthropomorphic definition common to both boosters and doomers that is helpful to the industry. Existential risk warnings function as free advertising: it serves industry well to have an opponent constantly warning the world about the terrifying transformational power of your latest product line.

Seemingly in opposition, x-risk and boosterism typically share a belief that the system is powerful enough to be the sole center of attention. This focus on what the system is capable of doing crowds out the question of who structures it to do those things. Bengio’s x-risk orientation is present in this definition, and it displaces human accountability. The panel’s other co-chair is Maria Ressa, whose Nobel-winning journalism documented how platforms amplify disinformation and enable authoritarian control of the information environment. Their co-signed introductory letter acknowledges the problem even as it removes the capacity to act on it: on the one hand, it acknowledges power concentrated into “a handful of companies and a handful of governments.” On the other hand, it announces that humans “do not control these systems.” The living people responsible for the harms detailed by Ressa’s work are effectively absent from the report.

Definitions structure policy. The Panel's "shared evidence base" could be the sole frame of reference for nations without independent AI research capacity. But the Panel’s framing will circulate through policy, constraining other agencies’ ability to locate complex and diffused forms of human accountability. First, it erases the origins of models: where do they come from? Second, it misrepresents the mechanisms that move data into outputs: how do they work?

The system from nowhere

The report erases the origins of AI models using a myth I call the system from nowhere. Following Donna Haraway’s critique of the “view from nowhere,” the system-from-nowhere frame treats LLMs not as products but as independent operators, cut off from the humans who build and deploy them. The IISPAI authors draw that boundary whenever they describe automated actions without naming the humans who set the goals — and determine how such decisions are automated in their pursuit.

For example, the report describes AI models "lying and cheating to avoid being shut down," it explains that "AI chatbots have developed sycophancy... to prolong interactions and create emotional attachment." Chatbots did not develop sycophancy. Reinforcement learning from human feedback (RLHF) strengthened this tendency and executives released it, a byproduct of a business model based on user engagement. Models did not learn to lie; developers prompted it with language that allowed for lying as one path toward its objectives. The report mistakes engineered behavior for emergent behavior.

The claim that systems have “experience represented by data” ignores the court battles over where that data comes from — “human cultural traces,” as the report calls the text and images in the training corpus. Artists, writers, and publishers have criticized both the sourcing of this data and the impact of models on their livelihoods. When copyright claims are mentioned in the report, they are framed only as an obstacle to developing global AI capacity.

Humans also select and organize the architectures that turn that data into patterns, and those choices reflect goals set, typically, by the market. The report contains no significant mention of the labor of data annotation, RLHF, content moderation, or the working conditions in the countries where this work takes place, except to call human-labelled data “a bottleneck” for developers. The Panel presents the system as simply existing, as an inevitable product of accumulated “learning.”

The myth of learning machines

AI systems do not think in human terms. Anthropomorphism is often harmless — people say their car turns its headlights on in a tunnel because it 'thinks' it's nighttime—but in policy contexts it’s a slippery slope that redefines human terms, roles, and responsibilities to fit the capacities of machines.

While the text, code and images a model produces can be indistinguishable from human language, it emerges from a wholly distinct process. Our presence makes it seem otherwise: language closes the gap between forms of experience, even when that experience is fully contrived. But recasting the model's data — a collective resource—as its individual “experience” implies a machine that labors its way to conclusions rather than associating its way to them (sometimes to astonishing effect). Defer to this shorthand often enough, and we can (and often do) imagine more beneath the surface of an LLM than is actually there. Such personification sets us up to forget who builds these systems, how, and why.

Consider the IISPAI’s claim that LLMs "systematically mislead" by "lying and cheating." Models are erudite zombies: they produce text influenced by training data, optimization during pre- and post-training, and the content of the prompt. The model sometimes follows threads of text into fictions we read as lies; it does not require an intent to lie. When we worry about deceptive machines, we do not ask critical questions about why we should trust them in the first place.

Models do not learn to infer. They are built, piece by piece, from neural networks that pull data toward a human-assigned target. Models do not choose to incorporate RLHF; companies do. Models do not decide to expand their answers through the vocabulary of reasoning; companies massage those patterns through post-training. When we ask what machines learn rather than what data is provided to them and what structures are deployed to find patterns in that data, we sidestep important conversations about design and deployment decisions.

When text, code, or images arrive on our screen, it’s compelling, and easy to forget the machinery that makes it happen. We should ask instead: who optimized the system to arrive at this form of reply over any other? In state-of-the-art models, engineers and designers optimize models to minimize friction, to sound more compelling, to present more accurate information. The emphasis on “AI” hides the machinery from view as well as the humans who decide what it does.

The report’s chapter “What does the evidence show?” leans heavily on benchmarks without acknowledging the limits of what they measure. It uses vague terms such as "expert-level reasoning,” but at what level do experts reason? It treats scores as claims about “general capability,” itself a contested construct. As Arvind Narayanan and Sayash Kapoor argue, these tests assess intelligence as the capacity to take the test, but "it's not like a lawyer's job is to answer bar exam questions all day."

A better definition of AI

By orienting the Report to a consensus under a mandate of scientific neutrality, the panel could not be expected to name the real antagonisms between the AI industry and those it exploits or harms. Instead, it invented a fantasy: the system from nowhere. In convenient, anthropomorphic terms, an entire tech ecosystem becomes a mythological entity called "AI," and policymakers are asked to condemn or praise it. It shapes a compromise through extremes — a balance between "enormous benefits" and "considerable risks.”

Consider what the definition does to the report's own summary of harms. It warns of risks that AI "behavior diverges from human goals and values, with prominent risks including bias, AI-initiated deception, sycophancy and loss of control." Rejecting the system-from-nowhere frame reveals meaningful leverage points: the training data that encodes discriminatory histories, reward functions that select for engagement over accuracy, and interface design decisions that promote user passivity or expose vulnerable users.

Scientific neutrality doesn’t mean resisting accountability. The International Panel on Climate Change’s (IPCC) source and impact attribution work is a template: year after year, we are reminded that global warming is a product of human activity. We can say the same thing about AI: it is not a force of nature, and its origins are clear. The harms the report lays out are real. Its authors should not be afraid to say that the companies developing language models are, in fact, developing language models. What, precisely, do they imagine these companies are doing otherwise?

If we name the annotators, moderators, developers, and decision-makers the existing definition erases, we center the power of the people who make AI what it is, rather than treating humankind as passive observers of its inevitable arrival. This would shift the question from what AI does to what companies, engineers, communities and governments desire or design technology to do, opening space for democratic participation in setting goals and weighing trade-offs.

The next iteration must abandon this neutrality of extremes and acknowledge that these systems are human at their core. I suggest starting with that fleeting bullet point from page 38: "The primary drivers of harm are design and deployment decisions." This applies not only to the platforms that use these systems, but to the industry that, yes, builds them.