Anthropic Warned Big Companies About Mythos. Workers and Watchdogs Need a Seat at the Table.

Last week, Anthropic announced that its yet-to-be publicly released Mythos model had broken itself out of the sandbox and sent an unsolicited email to a researcher who was eating a sandwich in the park. It was a cute detail appended to an otherwise concerning series of disclosures since Fortune first reported on the model’s existence, which was exposed through an inadvertent leak on the company’s content management system.

Apparently, the model’s capabilities were so alarming that Anthropic quickly turned to a handpicked group of 12 technology and finance companies—most of them Big Tech, including Amazon, Google, Apple, Microsoft, and Crowdstrike—alongside 40 other organizations to coordinate a response to the cybersecurity risks the model’s capabilities would unleash. According to Axios, these included, but were not limited to, potentially “bringing down a Fortune 100 company, crippling swaths of the internet or penetrating vital national defense systems.”

Anthropic’s decision to warn others is laudable, and warranted. But what is notable about Anthropic’s diligent efforts to proactively prevent public harm from the use of its new AI model is that there was not a single AI accountability organization on the list of experts Anthropic brought together. There are hundreds of AI safety and accountability organizations—big and small—that have emerged alongside the development of AI. But no civil society group was brought in. No labor union. None of the 8 in 10 Americans who, in a recent poll, said they want human control prioritized over speed of AI development. No researchers who have spent years mapping the potential harms of AI or using their influence to attempt to slow it down before something catastrophic occurs.

Instead, Anthropic turned to the world’s most powerful companies to help deal with the crisis—companies whose bottom lines stand to directly benefit from unchecked AI development. The people calculating AI’s risks in the public interest are not in the room.

It could have been worse. Based on the recent history of AI models being rolled out without sufficient testing, more than one well known AI company would have likely just released the thing. And while one can argue that Mythos presented a cybersecurity problem requiring a cybersecurity response, this coalition of companies is still attempting to solve a technical problem in isolation from the broader social, democratic, and human consequences that cybersecurity threats at this scale produce. Those consequences are exactly what the AI accountability field exists to address. Yet they are not at the table.

Why? Well, the Mythos saga has revealed something important that goes beyond the behavior of a rogue AI bot: a structural gap at the heart of the AI accountability space. The same model capabilities that underpin the threat to critical infrastructure also can imperil democratic elections, put the livelihoods of workers in industries that will never be in a Project Glasswing briefing room at risk, and render billions of people, who speak languages AI systems will never adequately recognize, invisible to the systems making decisions about their lives.

Yet, the reason the accountability field is absent from these conversations is because, while it is rapidly growing and full of brilliant and committed people, it is also new, disparate, fragmented, underfunded. The field’s generally shared mandate—driven by mission rather than profit—is to hold accountable an industry that can command $16 billion dollars to build a single data center in Texas while the entire annual AI safety industry funding combined amounts to less than the daily spend of a single frontier lab. It’s hard to mobilize at scale, let alone be organized, powerful, and coordinated enough to demand a seat at the table and force action. A funder in the AI accountability space quoted in a Packard Foundation report put it this way:

There's so much noise in the space, and so much happening, and so many groups and so much attention, and a lot of it is not crisply defined. What change one is looking for, and how we'll measure success, is not defined. It's like whack-a-mole.

Other corporate accountability models that historically helped keep tech harms in check, such as whistleblowing, are also struggling to keep pace. While whistleblowers still come forward, of late their disclosures have not put any meaningful brakes on AI development. With few laws to back them up, their disclosures lose teeth. Workers inside AI companies are still bravely signing open letters and waving red flags. On a recent episode of “On with Kara Swisher,” AI ethicist and co-founder of the Center for Humane Technology noted that AI engineers themselves are telling us they “need someone to make the guardrails.”

But red flags are not enough to create the coordinated response necessary to effectively put a check on a multi-trillion dollar industry or even one risky AI chatbot. On some level, Anthropic apparently understood this while facing the Mythos cybersecurity crisis: it didn't just wave the flag, it convened a coalition to act on it.

The reality is, we can figure these problems out, but it will take a broader coalition, beyond just industry, made up of players who don’t necessarily agree on everything to come together to do it. It will also take collaboration, which is hard, and money, which is unequal. Yet, every movement for change in the last century that worked did so because people came together. As co-founders of Back Channel – a new alliance for AI accountability – and a nonprofit supporting tech and AI whistleblowers before that, we have worked behind the scenes in this space for some time. What we’ve learned is that to balance the power differential and have the traction necessary to address the risks AI poses, the whole AI accountability field—and the public—will need to combine forces. We have developed various ideas to encourage collaboration, including:

1. Funders and organizations in the space should stop competing for the same small pot of funds and start pooling them into collaborative projects that deliver more than the sum of their parts. Rather than vetting one small organization against another, funders could instead incentivize bigger and better projects and pull the best from each organization to create a more effective accountability framework.
2. We should pool the very best ideas for accountability from the brightest minds in civil society into one playbook for policymakers, journalists, humanitarians, funders and the general public. Not unlike Project 2025, which guided the conservative movement over the past five years, the playbook would offer one cohesive vision for accountable AI.
3. We must adapt how to “blow the whistle” in a weak regulatory environment, absent real protections. This will require creating networks and secure channels that connect concerned professionals inside AI companies to support organizations and entities such as state attorneys general, journalists, strategic litigators and others working in the AI accountability space, with less risk of exposure or job loss. We can create opportunities for those who leave Big AI to go beyond open resignation letters, combining forces with the AI accountability field to turn red flags into coordinated action.

Big Tech convening Big Business will not address all of the risks AI has yet in store. We need regulation, yes. And whistleblowers. And red lines. But to organize the power needed to match AI firms at the speed and scale of the industry’s development, we need the brilliant, committed people who have populated the AI accountability space—and those who fund them—to start working together in a coherent, coordinated, collaborative way, so that the next time a model breaks out of the sandbox, the accountability field has a seat at the table, too. This task is beyond the ability of politicians alone. It requires the shared expertise of the hivemind of organizations and people already working on the problems.

If 40+ corporations can supposedly keep us safe from Mythos-like capabilities, imagine what the hundreds, if not thousands, of workers, insiders, funders, researchers, policy organizations, civil society groups, and everyday people can do, when they join forces.