Stopping AI Harm Starts with Protecting Whistleblowers

Vincent Cortese, Jocelyn Hong, and Charlotte Lin are Research Assistants at the Center for AI and Digital Policy (CAIDP).

As the US accelerates its race to dominate AI, the need for strong whistleblower protections has never been more urgent. On July 23, 2025, the Trump administration released its AI Action Plan, signalling its intent to dismantle the regulatory environment designed to safeguard the public. Under Pillar I: Remove Red Tape and Onerous Regulation, the plan states, “Federal agencies must eliminate outdated, burdensome, and unnecessary regulatory barriers that delay or obstruct the deployment of AI technologies.” This deregulatory push follows a previously proposed federal moratorium on state AI regulation in Congress. Although the Senate rejected the proposal by a vote of 99-1, these actions have revealed the tech industry’s growing influence in DC.

Tech insiders may now be the only ones left to expose AI’s risks to public safety, as federal oversight diminishes and states face pressure to stand down from regulation. Without strong legal protections, these potential whistleblowers remain vulnerable to retaliation, and critical information may never reach the public. This sets the stage for a dangerous race to the bottom.

We must protect whistleblowers when regulation lags behind.

In their open letter “A Right to Warn,” former employees from OpenAI, Google, and Anthropic stressed the urgent need for whistleblower protections, arguing that employees are often the only ones with access to internal practices hidden within AI systems. These individuals can provide critical information necessary to inform policy before significant harm occurs. The group also called on AI companies to adopt voluntary commitments to protect whistleblowers; however, in the absence of legal requirements, such commitments would lack enforceability, particularly in the current policy environment that is so favorable to Big Tech. And with a history of turning a blind eye to social media harms and anti-regulation lobbying, can we really trust tech companies to protect the public interest? Restrictive non-disclosure agreements (NDAs), the risks of equity clawbacks, and other forms of retaliation further underscore the urgent need for strong whistleblower protections to increase accountability from the inside out.

As tech firms race to deploy powerful AI systems, whistleblowers are already raising alarms about the safety and ethical implications of these models. In 2024, a Microsoft software engineer reported to both the FTC and Microsoft’s board that the Copilot Designer image generator was producing graphic and violent content alongside images of children. Despite his repeated calls for product warnings and restrictions, Microsoft allegedly continued to market the product without significant changes.

The importance of such protections is underscored by the tragic case of Suchir Balaji, a former OpenAI researcher who published a detailed analysis questioning the legality and ethics of ChatGPT’s training methods. Balaji was reportedly preparing to testify in the New York Times v. OpenAI case when he died unexpectedly in November 2024. While the cause of death was ruled a suicide, the circumstances raised concerns about the intense pressure faced by individuals attempting to hold powerful companies accountable. Even without direct retaliation, the fear of legal, financial, and reputational consequences can be enough to deter insiders from coming forward.

These examples point to a critical gap in the current regulatory framework. Without clear and enforceable whistleblower protections specific to the AI sector, we are asking company researchers and engineers to choose between their livelihoods and the public interest. We cannot afford to rely solely on voluntary commitments from companies building powerful and potentially dangerous AI systems. If we want meaningful transparency and accountability in AI development, we must empower those on the inside to speak out without fear of retribution.

Strong protections are already advancing in California and other states.

California, New York, and Illinois all have pending AI whistleblower legislation that specifically targets AI developers creating “large-scale” AI systems. California’s SB 53 would prohibit AI developers from preventing or retaliating against employees who report “catastrophic risks,” defined as those likely to cause death, serious injury to over 50 people, or over $1 billion in damages. The bill would also compel large AI developers to establish an internal, anonymous reporting mechanism for their employees. The reporting system creates an additional line of defense, ensuring that workers can speak out when something goes wrong and risks are promptly addressed. As Sunny Gandhi, Vice President of Political Affairs at Encode, put it, SB 53 ensures that “security isn’t sacrificed for speed.”

Illinois’ Artificial Intelligence Safety and Security Protocol Act (HB 3506) takes a similar approach. It prohibits large developers from enforcing any policy or agreement that prevents employees from reporting critical AI risks, whether internally or to regulators, and mandates civil remedies for retaliation. New York’s AI Act (SB S1169) also includes comprehensive whistleblower protections. Under Section 86-b, the Act bans retaliation against employees for reporting AI safety concerns, prohibits the waiving of whistleblower rights, and requires developers to establish anonymous reporting mechanisms. Additionally, developers must clearly inform employees of their rights and responsibilities under the law, mirroring provisions in California’s proposed legislation.

New Jersey introduced AI whistleblower resolutions in its Assembly, which passed in January 2025, and its Senate, which is still pending. The resolutions urge generative AI companies to make voluntary commitments to protect employees who raise risk-related concerns, including setting up anonymous reporting channels, banning retaliation and non-disparagement agreements, and ensuring employees can report risks without fear of repercussions. They also call for safe harbor protections for external researchers who identify AI risks.

Although both resolutions are non-binding, they highlight the urgent need to protect whistleblowers in the AI sector. The resolutions emphasize that “in the absence of government oversight, employees of AI companies are among the few individuals capable of holding the companies accountable.” This underscores the critical role of insiders in exposing risks and ensuring responsible development in a rapidly evolving AI landscape.

It’s possible to have a national framework without federal regulation.

The examples above illustrate that when states align their legal language, they reduce confusion for companies operating across state lines. Consistent legal frameworks ensure that workers have the same rights and protections regardless of where they live or work. They also help avoid the patchwork regulatory landscape that tech firms often cite as a barrier to innovation.

Although a bipartisan AI whistleblower bill was introduced in both the Senate and the House, in light of the Trump Administration's broader push toward deregulation, swift congressional action is unlikely. In the absence of federal leadership, states must step up to fill the regulatory gap.

Effective whistleblower protections can serve as an early warning system for AI-related harms, offering the potential to mitigate risk before there is a public safety crisis. Whistleblowers have consistently proven to be effective at exposing corporate malfeasance. According to a 2007 report from the National Whistleblower Center, they were responsible for detecting 43% of fraud cases in private corporations compared to 19% identified by professional auditors.

The stakes are even higher with AI systems, which are increasingly embedded in every sector of our economy and in every facet of public life. While policymakers may not be able to anticipate every potential harm to the public from AI, they can ensure that insiders at the companies have a safe path to speak up.