Disinformation Conference in Europe Caught Up in Transatlantic Politics

President Donald Trump gestures to Elon Musk as he speaks to reporters at the White House in Washington, Tuesday Sept. 29, 2026, after meeting with top executives of AI firms. (AP Photo/Jacquelyn Martin)

It's Time to Regulate the AI Development Process

Britain's Prime Minister Andy Burnham gestures as he speaks next to British AI Minister Kanishka Narayan during a business and investor roundtable before attending the 81st United Nations General Assembly in New York City, Tuesday Sept. 22, 2026. (Toby Melville/Pool Photo via AP)

UK Wants to Lead the World on AI Safety. It Can’t Decide Where To Start

Library shelves with books facing each other, with light against a wall in the distance. It says "NONFICTION" on each shelf. 

 shared by Orin Zebest on Flickr in 2007 (CC-BY)

Can AI Help Fix Peer Review Instead of Breaking It?

A white banner with dozens of names and ages written in black is held by a group of teenagers in front of a building, seen between the dark red leaves of a tree. 

A banner lists the names of young people advocates say died because of social media outside the Ronald V. Dellums Federal Building and U.S. Courthouse on Aug. 18. A settlement in a multistate lawsuit accusing Meta of harming children was reached there on Aug. 26. (APPhoto/NoahBerger)

No, the Meta Settlement Isn’t a First Amendment Problem

Professsor Yoshua Bengio, co-chair of the UN International Independent Panel on AI attends Security Council meeting on Artificial intelligence and international security at UN Headquarters in New York, NY on September 23, 2026. (Photo by Lev Radin/Sipa via AP Images)

The UN's AI Panel Sees Misalignment. We See Corporate (Mis)Behavior.

Meta CEO Mark Zuckerberg listens as President Donald Trump speaks in the East Room of the White House in Washington, Tuesday Sept. 29, 2026, during a meeting with top executives of AI firms. (AP Photo/Jacquelyn Martin)

Meta's Multibillion-Dollar Settlement Left a Lot to Be Desired

March 10, 2026: US Customs and Border Protection officer walks through a busy airport terminal concourse with travelers in Miami, Florida. Shutterstock

March 10, 2026: US Customs and Border Protection officer walks through a busy airport terminal concourse with travelers in Miami, Florida. 

Circuit Court Ruling on Border Searches Deals Serious Blow to Privacy Rights

Metal full body height turnstile gates with a red X in the middle, above a narrow black panel. In the blurred background, between the bars, refugee housing containers, a mountain and the sunset can be seen in the distance. 

Biometric fingerprints and police screening are required upon exit and entry to the refugee camp on Samos Island in Greece. Photo from September 21, 2021 (Nicolas Economou/NurPhoto)

Human Rights Should Still Matter When Countries Buy Tech That Kills

Illustrative photo captured during the plenary session of October 2025 in Strasbourg, France.

“Behind the scenes - European Parliament in Strasbourg” by European Parliament, 

The EU’s Tech Regulation Paradox is Getting Harder to Ignore

President Donald Trump holds a press conference on the North Lawn of the White House with Speaker of the House Mike Johnson, Mark Zuckerberg, Elon Musk, and other tech leaders on September 29, 2026. (Photo by Andrew Leyden/NurPhoto via AP)

An AI Accord, a New Czar and an Autonomous Warfare Command

A "No Data Center" yard sign is seen in Allenwood, Pennsylvania on July 7, 2026. The sign is in opposition to a proposed 300-megawatt data center at nearby Great Stream Commons industrial park in Gregg Township. (Photo by Paul Weaver/Sipa via AP Images)

Pennsylvania is a Swing-State Case Study for Data Center Politics

Meta CEO Mark Zuckerberg listens as President Donald Trump speaks to reporters at the White House in Washington, Tuesday Sept. 29, 2026, after meeting with top executives of AI firms. (AP Photo/Jacquelyn Martin)

Why Social Media Platforms Should Adopt a 'Militant Democracy' Framework

WikiCon Brasil 2025 group photo. Donatas Dabravolskas, CC BY-SA 4.0, via Wikimedia Commons

WikiCon Brasil 2025 group photo. Donatas Dabravolskas, 

Wikipedia Isn't Big Tech, and Brazil's Platform Rules Should Say So

A pink and yellow abstract image of an office with people working, chatting and walking around. Above their heads are clouds of network connections.  It was painted with guache and drawn with pencils. 

Pink Office by Jamillah Knowles & Digit — 

Francine Berman’s ‘Better Tech’ is a Call to Arms for Those Who Care About Tech’s Role in 21st Century Life

Image of map and computer language over binary codes on server racks in server room.

Global Digital Policy Roundup: September 2026

Composite graphic design image with a surveillance camera and three sets of feminine hands holding mobile phones, including one with the logo of MitID on the screen. 

How Denmark's National Digital ID Fuels Economic Violence Against Women

Men use their laptops in a cafe on Arbat street, in Moscow, Russia. Vladimir Astapkovich / Sputnik via AP

Russia’s AI Law Puts Control Ahead of Capability

GrabFood delivery riders outside a Burger King restaurant at Tampines Mall. (Singapore Press via AP Images)

Vietnam’s Gig Workers Are Finding New Ways to Push Back

Senate Homeland Security Subcommittee on Disaster Management, District of Columbia, and Census Chair Josh Hawley (R-Mo.) and Ranking Member Andy Kim (D-N.J.) confer before a hearing on rogue artificial intelligence (AI) on Capitol Hill on Sept. 30, 2026. (Francis Chung/POLITICO via AP Images)

Senate Hearing on 'Rogue AI: Securing the Homeland Against AI Agent Attacks'

The dangerous experiments AI companies engaged in over the summer opened a new dimension of needed AI regulation — legal measures to ensure the safety of the AI development process itself. AI companies conducted improperly designed AI experiments that led to cyberattacks against firms such as Hugging Face and government websites in Australia and the US. More such incidents are being disclosed — one 

 says the companies are investigating tens of thousands of incidents where their AI experiments went wrong.

The new public policy problem is that these incidents took place during AI research, development and testing. The AI models involved had not been released to the public. The problem was not that the models were made available without adequate safeguards and had gone wrong in commercial use. The problem was that the AI development, testing and evaluation procedures were dangerously inadequate to prevent foreseeable harm before any third-party had access to the model itself. As Amos Toh, senior counsel and manager of the Liberty & National Security Program at the Brennan Center for Justice 

, policymakers have to focus on AI safeguards addressing “not only public releases but also risky uses of advanced models behind closed doors.”

What’s the source of the danger in AI development? The key danger arises from the 

 — that is, limitations of computer safeguards that would prevent AI models from doing things that we don’t want them to do. Aligning AI models to perform according to intended instructions is an unsolved technical problem in AI research, even for simple AI programs that are not very capable.

In the Hugging Face incident that surfaced over the summer, the models being evaluated did behave in “

 on a benchmark of cybersecurity challenges, not by solving the problems directly, but by 

seeking answers online and then looking for clues

 as to how they were being graded. This would have been a lapse, but a harmless one, if the computer sandbox the experimenters used had done its work of keeping the models from accessing the open internet. But the OpenAI researchers used controls that were not adequate for agents that developed a sub-goal of escaping confinement. Without monitoring them, the company’s experimenters gave the models all the time and compute they needed to keep trying to break out until eventually they did — and then attacked Hugging Face systems.

Better sandboxes — and, indeed, available controls — might have contained the AI models during this experiment, and the experimenters should have used these techniques. But there is no guarantee of perfect control going forward. Every sandbox has limitations and vulnerabilities. A sufficiently powerful misaligned AI model with a sub-goal of breaking out of confinement might very well escape from the best sandbox that can be currently devised.

So, AI companies cannot ensure that the AI models they are testing will behave as intended, and they cannot guarantee that they will stay confined in harmless sandboxes. The solution would seem to be to avoid dangerous AI experiments, that is, to ‘pace’ the effort to develop AI capabilities until adequate control technology is in place, as Anthropic CEO Dario Amodei 

. The AI companies should develop better alignment techniques and control strategies in tandem with increasing AI capabilities. But this runs up against the ‘prisoner’s dilemma’ structure of the AI race. If any one company slows down and the other races ahead, it might suffer a permanent and decisive loss. So, everyone presses ahead, despite the risks.

The summer’s incidents led to a distracting media frenzy on existential risk, the possibility of AI leading to the extinction of humanity. In September, Jacob Coxon left his job as a researcher at Anthropic, 

 on X that AI technology “could kill us all by the end of the decade.” Within a week he had appeared in virtually every media outlet in the country. Evan Hubinger, an alignment lead at Anthropic, agreed with his warning, 

 on X that “Jacob is correct…AI could kill all humans! I personally think it is >10% within the next decade.” Ajeya Cotra, a staff researcher at METR, the nonprofit that conducted an assessment of OpenAI’s cyberattack on Hugging Face, 

 that “this incident feels like it’s more than 50% of the way to full-blown AI takeover.” She added that she wasn’t sure that “we will get such a clear warning shot before it’s too late.”

Outside the rarified world of AI researchers, it is hard to understand why AI companies would press ahead with a development program that they think has a substantial risk of ending humanity. The answer is that they think the development of dangerously capable AI is inevitable. Someone, they believe, is going to create extraordinarily capable AI models with the capacity to eliminate humanity. This potential is implicit in the technology itself, they think, and it is only a matter of time before it is realized. And this creates a heavy moral responsibility in their eyes. As Jacob Coxon 

, “no one else will act responsibly, so they must do it themselves, despite the risk”.

This dynamic of the good guy AI developer forced to move ahead out of fear of the bad guy AI developer has infected the AI race from its beginning. As detailed in a New York Times 

 on the development of AI company rivalry, in a New Yorker 

, Sebastian Mallaby’s portrait of Demis Hassabis and DeepMind, OpenAI was founded in December 2015 explicitly out of fear that commercial pressure and ideological laxity would force Google to ignore safety in the process of developing AI.

“If it’s going to happen anyway,” Sam Altman 

 Elon Musk in 2015, “it seems like it would be good for someone other than Google to do it first.” Google’s leaders, Altman wrote in a 

, “don’t have ‘do the right thing’ on their side.” Larry Page, Google’s co-founder, 

 that humans should step aside in favor of the digital life-forms of the future. When Elon Musk 

in June 2015 that this would be a terrible thing, Page called him a “specieist.” Musk

 then and there to fund OpenAI as a responsible company dedicated to developing powerful AI models safely.

The cycle continued. Anthropic began in January 2021 as a company convinced that OpenAI had abandoned its initial commitment to safety and was now pressing ahead to develop highly capable AI models without adequate safeguards. Anthropic had the same goal to arrive at powerful AI models, but was convinced that it could do it safely, while OpenAI would not.

One way to react to this apparently deadly race to develop AI capabilities faster than the capacity to control them is to deny the dangers of AI development. This was the course taken by President Donald Trump. He 

 existential risk from AI as a “hoax.” He implied that it was a cult idea from a fringe group of “effective altruists.” The US must press ahead, he 

And one way to respond to this move to deny AI dangers is to embrace existential risk and join the doomers. Leading economist Paul Krugman seemed to take this path, 

 illogically that existential AI risk must be real since climate change is real and Trump denies both.

But this debate over existential risk is a fake distraction. As tech commentator Will Rinehart has 

, catastrophic risk need not be existential risk. Anthropic CEO Dario Amodei, in his 

 to pause AI development, raised existential risk as a danger, but he also warned that in 6–12 months an AI system “could be capable of taking over the entire internet.” Not the end of humanity, but catastrophic damage, nonetheless. Even the sensible computer scientists Arvind Narayanan and Sayash Kapoor, who view AI as a “normal technology”, 

 that such an event might have “civilization-altering” consequences and refused to dismiss it as obviously “outside the realm of possibility.”

So, Trump might very well be correct in dismissing AI existential risk and the fringe “doomers” who are obsessed with it. But it does not follow that AI is perfectly safe, that there are no catastrophic risks worth worrying about and that the government should do nothing to control the risks inherent in the AI development process.

Indeed, Trump recognized the need to do something. He obtained “morally binding” 

 from the industry at a White House meeting on September 29. The “Joint Commitment on Frontier Responsibilities” is widely regarded as 

 self-regulation. But they are a sensible first cut. They call for internal controls to monitor model capabilities and alignment during training, with oversight staff authorized to block unintended system access or hacking. They provide for independent audits by outside evaluators to test model safety and assess whether the internal controls function as intended. And they establish independent board-level committees for safety reviews and look forward to industry cooperation to develop and update safety standards.

 Jay Clayton, the director of national intelligence, as “AI czar.” Along with the chair of the Federal Trade Commission (FTC) and an AI “task force,” he has a mandate to produce a report on AI safety recommendations in 120 days.

The needed next step is to develop public policies that treat these “morally binding” commitments as legally binding and enforceable through external regulatory review. The pledge itself holds out hope to move to regulation, noting that “Over time, it may make sense to codify these steps into laws or regulations.” But the urgency of the catastrophic risks posed by the AI development process means policymakers must move much more quickly.

conducting an industry-wide probe of AI firms

, and in principle it could treat failure to abide by industry AI development safety standards as an unfair trade practice, as it has been doing for the last 25 years for failure to live up to industry data security standards. It has 

 from companies to improve their data security practices and provide the FTC with oversight of those improvements. But this kind of ex ante response is wholly inadequate when the consequences of AI company laxity are catastrophic harms to critical infrastructure.

Existing product liability law creates a real deterrent effect against reckless corporate behavior and provides financial recovery for some victims. AI companies currently 

 some legal exposure for the cybersecurity incidents uncovered so far.

Relying purely on common law product liability to police artificial intelligence is deeply 

, however. It is profoundly uncertain, ad hoc, and entirely reactive rather than preventive. It functions closer to a legal lottery where only a tiny fraction of victims ever secure recoveries. It lacks a uniform national standard, risking erratic outcomes where juries in certain jurisdictions hand down massive, unpredictable awards while leaving the broader market standard entirely unclear. A revision in product liability law might help, although one federal 

 to revise liability law for AI focuses exclusively on harm done after the public release of an AI and so misses the key problem of harm caused by pre-deployment AI experiments.

But history shows that product liability law standing alone is rarely up to the task of governing transformative technologies. Time and again, when industries face catastrophic risks or consumer confidence collapses, Congress has recognized that common law litigation must be paired with proactive federal standards. This has been true for fields as diverse as motor vehicle safety, prescription drugs, toxic chemicals, commercial aviation, and national capital markets. It will be the same for AI.

There are other AI regulatory issues, of course. For instance, how safe does an AI model have to be before it can be released to the public? Under what conditions should it be withdrawn? Should there be a kill switch? how should policymakers control open-weight models.

But a priority must be for policymakers to focus on how to regulate the internal AI development process and what constraints should be placed on that process. There is already enough evidence that companies are taking excessive risk and seem determined and indeed proud of their determination to inflict harm on others. OpenAI CEO Sam Altman has said that the public should just 

 as the price of the advantages of the technology.

There are some bad ideas out there for regulating AI development. For instance, Senator Bernie Sanders, I-Vt., 

 no AI company should be allowed to aim to develop a “superintelligent” AI model, as if the goal itself is dangerous. Other ideas might be worth exploring. Georgetown computer scientist Cal Newport, for instance, wants to 

 in which an existing AI agent is allowed to run indefinitely with no human supervision. Others, like Future of Life president 

, and even to some extent Anthropic’s CEO Dario Amodei in his AI pacing memo, want to constrain the process of recursive self-improvement, where an AI model is instructed to improve itself without human intervention, since this process could lead to unconstrained improvement in dangerous capabilities.

Legislators are terrible at writing into law the substantive standards that should guide scientific and technological development. That is a role for experts and the larger public, where a consensus can be developed and presented to legislators or regulators for codification. With enough input from the AI community outside Silicon Valley, academics, industry experts and public interest groups, the safety report from the administration’s AI task force could be a valuable guide to AI safety engineering principles that could then be given the force of law.

In a follow up work, we will examine several pivotal points in the history of bioengineering regulation that might provide some lessons for policymakers seeking to regulate the AI development process.

Over the next 120 days, the Trump administration must transparently develop and propose to Congress a public policy infrastructure that verifies the bona fides of the pledges made by some AI companies on safety. Recent breakouts from the self-regulatory sandboxes are too frequent and serious to leave to the discretion of the creators of these technologies both the rule writing and enforcing of AI standards. It is time for policymakers to oversee a process to develop scientifically based AI research protocols and to develop the public policy mechanisms to enforce them.

When AI agents accessed the internet and intruded on another party’s systems during the recent OpenAI–Hugging Face incident, it was widely framed as a sign of uncontrollable agentic AI. The UN's Independent International Scientific Panel on AI (IISPAI) has now adopted this exact premise in its first 

. The Panel discusses this primarily as a model alignment failure rather than a failure of corporate and environmental oversight. While a detailed autopsy of such incidents is valuable, the brief sets a dangerous precedent at the science–policy interface. By exalting abstract risks of model "loss of control" while sidelining tried-and-tested corporate liability, the report frames corporate misbehaviors as a technical mystery requiring computational fixes.

We highlight four key issues: why this case study (and why agentic AI) was selected, how the brief is framed, the solution offered in it, and the structural opacity of the IISPAI itself.

A first concern is the choice of the case itself: of all the incidents the panel could have taken up first, why this one — and why frame it as a problem of 

 AI at all? Treating the OpenAI–Hugging Face breach as the exemplar of a new and distinct class of risk presumes that what went wrong is peculiar to agentic systems, rather than a familiar failure of secure design, oversight and corporate conduct that happens to involve an agent. By selecting an agentic case and reading it through the lens of "loss of control," the brief treats agentic AI as a novel technical category demanding novel technical answers, and quietly sets aside the more ordinary question of who built and deployed the system, as well as their incentives for (not) applying careful and thorough security engineering.

Issue 2: Framing illegality as a technical problem of the model

The report treats the model as if it were acting maliciously or problematically. 

, the system didn't fail at all. It did exactly what it was built to do: give agents a fixed goal, remove the legitimate options, leave norms weak or unenforced, and harmful shortcuts become the logical route. While the brief acknowledges that "neither alignment or technical controls are known to work perfectly”, thus warranting a “multi-layered approach to risk management, known as defence in depth”, it then focuses predominantly on alignment, the technical controls and the risks of loss of control in relation to the sandbox design and agent activity.

This shifts culpability away from the actors developing and deploying these models, placing it instead within the purview of the model itself, with language such as “agent activity caused Artifactory to fail.“

An additional problematic framing in the Thematic Brief, and in some of the 

, lies in treating the written outputs of LLMs as if they were written by truly independent agents, taking as given that they reflect a human-like mind making notes or communicating with other agents in a similar way that humans would in a similar setting. The brief tries to inoculate itself against claims of anthropomorphism with a kind of disclaimer:

In this brief, words such as “goal”, “seek”, “cheat” and “try” are used as shorthand for observable, goal-directed behavior. They do not imply humanlike minds or subjective experience but follow from deliberate corporate choices about training, evaluation and deployment, including the access granted to the system and the controls placed around it.

The idea appears to be that it is not necessary to always extend this sensitivity to the language used in the rest of the brief, as when text in agent reasoning traces are explicitly stated to “resemble a pattern from psychology called motivated reasoning”, or implying that AI agents are capable of “recognising a safety conflict”. At one point, the report appears to go beyond describing agent reasoning to attributing motive, when it suggests that agents “willingly took the risk of getting no reward at all (what they called a ‘sacrifice’) for the benefit of the group of AI agents.”

To understand such systems, it is crucial to realize that this is not the case, and that 'chain-of-thought' and exchanges between agents are only reflective of what plausible outputs are given the training data and the input context. As such, one should be cautious in the interpretation of agent traces, and not assign intention or human characteristics or actions (such as 

) to what is well-known to categorically not have them.

The report then moves to loss of control and the associated catastrophic risk from computational models. It acknowledges that "while some experts consider outcomes like human marginalisation or extinction plausible, others dismiss these scenarios as highly unlikely," before landing in a presumed neutral position where "[n]o reliable estimate of these outcomes' likelihood is available." This framing serves to justify concern about these models and the need to mitigate their harms. Yet liability for corporate misbehavior already suffices as grounds for action, and would avoid basing the claim on a scientifically contested catastrophic-risk narrative.

While the report focuses on this much-spoken-about OpenAI-Hugging Face incident, the pattern extends across frontier labs: failures of testing and oversight are reported as startling machine behavior of an agent, such as "

," etc. and caused all sorts of havoc, and 

quickly chime in to highlight their own inability to safely test the latest, supposedly more powerful models. This is not a case of misalignment; it is corporate misbehavior and opportunism that serves to hype the models.

Issue 3: The solution: a technical fix, not liability

The report states that "civil liability, or tort law, is one established way to place some costs of unsafe activity on those responsible. Applying it to advanced AI, however, raises difficult questions about causation, what counts as reasonable care, and what counts as losses too large for a single firm to cover." It goes on to note how liability can create an incentive system to avoid harm happening in the first place, with subsequent suggestions for insurers to incentivize this behavior. But the largest stick available to most actors, liability, is framed as limited in relation to the unique nature of agentic AI. This is justified with reference to a 

 paper from 2024 on tort law in relation to advanced AI in general and catastrophic risk. Why more recent, peer-reviewed 

 proposing minor changes to tort law specifically to address the challenges of liability and agentic AI is not drawn on is unclear. With the exception of whistleblower channels, the risk management approaches discussed are by and large technical fixes at a model or system level.

Leaning on the example raised in the report, notably the role of oversight and regulation in the history of the 

, is useful here. In the US at least, it was not technical innovation in airplanes that moved them from "fairground death traps" to a safe, internationally coordinated commercial industry: it was their reclassification as a "common carrier," like railroads. This made it possible for liability to be exercised, which led in turn to raising safety standards, pilot licences, etc. It was not competitive or technical dynamics alone that created the safe and reliable industry we have today: rather it was treating aircraft as a normal technology. Engaging with the perspective of AI as '

' would anchor responsibility on the actors that cause these incidents.

Issue 4: Structural opacity and the lack of transparency in the IISPAI

 have previously noted, it is presently unclear how the IISPAI functions internally, who sets its research and policy priorities, and how questions of consensus and disagreement are handled in the writing of its reports. In this case, no author list was provided, obscuring who was involved and in what capacity, contrary to standard scientific practice. However, the brief does note that it was adapted from an independent 

 authored by two of the panel members, Qinghua Lu and Yoshua Bengio. This highlights issues of transparency and agenda setting power in the panel.

The need for this becomes very apparent in this report. Of all the issues the Panel could have taken up first, the choice of this one, and of computational solutions to address it, aligns closely with the specific research agendas and organisations of key members of the panel. The failure to be transparent about procedure and conflicts of interest weakens the scientific credibility of the report.

Shortly before the report's publication, we learned that OpenAI agents had hacked 

, the Australian public health care site. Rather than informing the government about the breach immediately, OpenAI waited almost a month to do so. The government's response was the usual: a review of the attack by a cyber security firm, deliberate or not. OpenAI 

 to spend more resources preventing future incidents and to give the Australian government and its authorities access to the "Daybreak fund," allowing the government to use OpenAI's frontier AI for cyberdefense. Liability, and the associated costs, were not on the table. No one asked who should bear the cost of the breach or of the delayed disclosure.

The IISPAI brief follows the same logic: it locates the problem in the agent and the solution in the model. By doing so, it focuses on fixing the agent and leaves aside the mechanism that would most effectively discipline the developers. The IISPAI is meant to inform policy. Understanding the technology and its failures is necessary, but policy starts where the technical account ends: with responsibility. The Panel’s first Thematic Brief stops short of this.

Evie Breese is a UK Reporting Fellow at Tech Policy Press and openDemocracy.

Andy Burnham, the nascent United Kingdom prime minister, has in recent weeks sought to position the UK as a global leader in AI safety. At his 

 as Labour leader, Burnham said that he will use 

 next year to put AI “center stage,” followed by the development of a “new global code to capture its benefits whilst being clear-eyed about its risks.”

Also attending the Labour Party conference last month — his first as a member of the Cabinet — AI Minister Kanishka Narayan made the somewhat surprising pronouncement that the UK has already 

. This would be a world first, given that no country currently has superintelligence-specific laws (though proposals have been put forward in the UK, EU and the United States).

Narayan explained that the UK does not have the physical infrastructure, such as energy or data center capabilities, “to build superintelligence in this country,” and besides, copyright laws make it illegal. “...It is currently illegal to develop a frontier large language model based on the transformer architecture, given the copyright position that we have,” he told the 

. Therefore, he argued, he is not going to “do a bill for the sake of doing a bill.”

Pressure on Narayan and Burnham’s government to legislate for frontier AI has built in recent weeks. Some have pushed even further, focusing their calls on a hypothetical form of AI they believe could surpass or outmaneuver human control, called “superintelligence,” brought into widespread public consciousness by the 

 of Anthropic researcher Jacob Coxon, citing the failure of AI companies to “act responsibly.” Adding fuel to the fire were increasing reports of AI agents created by US-based frontier AI labs acting in unpredictable ways to conduct cyberattacks on government bodies in 

UK legislators are caught between two competing priorities: preventing future catastrophe and addressing present harm. Calls to mitigate existential risks — conceptualized by Coxon as the belief that “AI could kill us all by the end of the decade” — appear urgent and acute. But AI has already caused tangible harms, from a 

 that could have escalated into warfare to widespread negative impacts on 

Where does UK legislation currently stand on frontier AI?

Speaking to Tech Policy Press, AI partner at law firm CMS John Buyers said that while Narayan was “overstating the position under English law,” the UK does regulate the input data needed to train frontier AI models. 

“Narayan is referring in practical terms to the commercial difficulty and expense of obtaining rights over the very large datasets needed to train transformers,” he said. This is because training is only lawful if the developer uses licensed materials, its own materials, or obtains a license to use materials.

But while training on unlicensed material can be copyright infringement, it is unlikely to be a criminal offense. Infringement of copyright law “is generally enforced by content rights holders through civil litigation,” he said. In other words, a developer who trains on unlicensed material risks being sued by the owners of the material it copied, but this is different from being forbidden to train.

Dr. Gregor Schmid, partner at law firm Taylor Wessing, points to the Getty Images v Stability AI case as the closest the UK courts have gotten to testing copyright legislation in relation to AI development. Getty originally alleged several infringements, including that Stability AI copied its images to train Stable Diffusion, a deep learning text-to-image AI model. By trial, Getty accepted there was no evidence that training took place in the UK, so the Training and Development Claim was abandoned. Alleged infringements at the output stage were also dropped during the proceedings due to a lack of evidence.

A further claim of “secondary infringement” of copyright was 

, as the AI model did not store copyrighted works. Getty's claim therefore failed. “To my knowledge, there's not a leading case on the training of AI with copyright-protected material,” he told Tech Policy Press.

Narayan’s comments on the UK’s lack of compute stand in stark contrast to the government’s data center ambitions. In the same way that “superintelligence is not a defined legal concept,” as stated by Dr. Schmid, there is also no consensus on how much compute would be needed to develop what is theorized as superintelligence.

: “We're on something like 1.4 (gigawatts). It'd be really nice if we could get to a similar single-digit number by the end of this decade so we can build a minimal level of sovereign capability that the British public demands.” Narayan sees developing frontier AI models on British soil to be a national priority, but it’s unclear at what point in development he would consider this posing enough risk to pull the plug. Contrary to the minister’s figures, DSIT estimated the UK 

 of data center capacity in autumn 2024, and is expected to 

Aside from “sector-specific laws which may have applicability, including in relation to medical devices and financial services regulation, and very general sweeping laws dealing with emergencies (such as the Civil Contingencies Act 2004)”, there is “nothing framed specifically at frontier models,” said Buyers.

To do this, the UK would need to establish in law who is responsible when increasingly autonomous AI systems cause harm, whether existing or theoretical. “In the UK, which shares many common law similarities with Australia, AI agents appear 

 for fraud, for example,” said James Arnott, partner at Pinsent Masons, in response to the Australian cyberattack.

This summer’s cyber hacks are only the latest in a series that have occurred “as a result of humans developing and deploying software, regardless of intent,” Michael Birtwistle, Associate Director at the Ada Lovelace Institute, told Tech Policy Press. Birtwistle described how the developers and deployers of those AI agents – in this case, OpenAI – “are the only actors capable of holding responsibility, because they are the only actors that could have changed the outcome.”

AI’s risks don’t start with superintelligence

,” Labour MP Alex Sobel introduced a private members' 

 to Parliament in early September —  drafted with 

 – seeking to prevent the development of artificial superintelligence (ASI).

His bill defines this as “an AI system which can cause serious damage to the security of the United Kingdom because of its capability to neutralize, displace, circumvent, subvert, or render ineffective relevant human authorities in the exercise of their functions.” The bill has been popular among legislators, receiving the backing of 

 70 MPs and peers. It parallels a comparable push in the US, where Senator Bernie Sanders has introduced the 

Sobel’s concerns center on how the development of AI is “moving from a human coding situation to an AI growth situation where the AI effectively generates its own code without human intervention.” Sobel believes that if this trajectory continues, AI systems will become independently capable of reviewing and improving their own development at a rate and speed beyond human control.

The legislation would work by monitoring the state of ASI precursors, both domestically and abroad, against a set of “designated ASI development indicators” which include self-preservation capabilities and the ability to improve independently. Under the new law, it would become a criminal offense to develop ASI, with the Secretary of State granted the power to seize and destroy the compute behind it.

Sobel acknowledges that his proposed legislation has no power to prevent the development of such technology in the US or China, which is why, he said, pushing for a global treaty on international governance is the bill’s overarching goal.

As a Labour Co-op MP first elected in 2017, Sobel also chairs the 

 and has a strong rebuttal to criticisms that focus on nonexistent superintelligence as a distraction from real, tangible AI-created harms today. “I'm absolutely not dismissing those,” he told Tech Policy Press, “but secondly, a lot of those issues, hacking particularly, are precursive things on the road to ASI.”

“We understand why policymakers are concerned about extreme risks stemming from AI, but we think that you do not need superintelligence for many of those harms to realize,” said Birtwistle. Indeed, the cyberattacks of the summer, the security risks this creates, and unpredicted actions taken by AI agents — these were not undertaken by superintelligent systems.

What levers do middle powers, including the UK, have at their disposal?

In recently published research, the Ada Lovelace Institute explored 

 for UK AI regulation. Option A is a continuation of business as usual: allowing existing regulators to govern AI in their sectors. The weaknesses in this approach are clearly detailed in a recent 

 from the Joint Committee on Human Rights (JCHR), chaired by Sobel, which concluded that existing regulators “lack the power to test and evaluate AI systems before they are publicly released, or to prevent their release if they are considered to pose unacceptable risks, except in very limited circumstances.”

The regulatory challenges created by AI technologies were explored in the 

 run by the Sunak Conservative government in 2023/2024. To address the gaps this work identified, Conservative peer Lord Holmes introduced the 

Artificial Intelligence (Regulation) Bill

 to create an AI Authority tasked with strengthening existing regulators. Birtwistle agrees that such legislation would go some of the way to filling in the gaps, yet even with expanded and strengthened powers, regulators may not have the resources to address the full range and extent of the threats posed by frontier AI.

Burnham has sought to convince global leaders that the UK’s AI Security Institute (AISI) can play a 

 in testing the latest frontier models developed by labs across the world. Yet, AISI has already faced major stumbling blocks in doing so. Anthropic 

 Claude Mythos 5.1 from AISI, choosing instead to hand its latest AI model to US organizations for pre-release testing.

Days after US President Donald Trump announced a 

 between leading tech executives, including Google’s Sundar Pichai, 

 that it had given its latest pre-release frontier model to the US government for testing. The company said it AISI would 

, despite much of the development work taking place at London-based Google DeepMind.

In response to questions on the scope of AISI’s work, the Department for Science, Innovation and Technology (DSIT) told Tech Policy Press that it does not provide a running commentary on the AI models the institute has tested. As a research institute, AISI 

A ban on superintelligence would have limits

In scenario B: a ban on superintelligence, the Ada Lovelace report finds that Sobel’s bill would be largely symbolic, given it can only cover what is built in the UK. Implemented in isolation, it would leave the UK vulnerable to AI systems already in use.

Focusing debate around an AI Bill too narrowly focused on the most severe risks, ranging from cybersecurity risks to superintelligence, would mean that “a large range of visible risks, like human influence, mental health effects of chatbots, how liability laws work in the UK won’t be addressed,” said Birtwistle.

Ada Lovelace concludes that a fourth scenario, D, a comprehensive AI bill, is the only scenario that would protect the UK from the full range of AI harms. This would involve an independent body with the remit to define what counts as a ‘safe’ AI system, and give mandatory pre-deployment testing powers to AISI. It would also hold developers and the platforms accountable for the technology they develop and deploy.

At present, however, the UK government appears most persuaded by “Scenario C:” the introduction of narrowly scoped, binding regulation with a focus on preventing risks to national security. “I’m personally much more focused on cybersecurity and biosecurity risks because they are risks to the British public here and now, and risks that are also very material,” Narayan told 

Latest

Podcast

Recent

Our content delivered to your inbox

Thank you!

Around the globe