Australia Wants an Off Switch for 'The Algorithm.' That's the Wrong Fix

On September 8, the Australian government released an exposure draft of its proposed Digital Duty of Care, a welcome and necessary reform requiring online services to proactively identify foreseeable risks of harm and take reasonable steps to address them. It is a positive step towards protecting children, in particular, from the worst forms of harm originating online.

The draft bill also requires platforms to provide “user empowerment tools,” which are yet to be specified. But across press conferences, interviews and media coverage, encouraged by the government’s messaging, media outlets and commentators seized on an idea far simpler than the full proposed reforms: Australians would be able, for the first time, to turn off ‘the algorithm’.

Branded “My Feed, My Way,” the government sold the duty of care as a proposal to provide safety for vulnerable users — especially children — and to give users aged 16+ greater control over whether their primary social media feed maintains the status quo of algorithmically recommended content or reverts to only displaying material from accounts they choose to follow, in chronological or reverse chronological order. More extensive, accessible and meaningful user control over the online experience is certainly needed, and the draft bill offers significant opportunities to push platforms beyond the status quo. But why has the simplistic idea of an on-off switch for algorithmic recommendations become such a focus of the government’s public case for reform?

Part of the answer is that, in government rhetoric and public debate, ‘the algorithm’ has become a powerful shorthand for the much larger machinery of platforms’ power and how they curate and serve content to users. Harmful content recommendations, predatory advertising, addictive design, political polarization and creator visibility are all now framed as problems of ‘the algorithm’. The concerns are real, but collapsing them obscures the fact that algorithms are not ‘bad guys’ in and of themselves. What matters is how algorithms are optimized: What signals they maximize, whose interests they serve, and what effects they produce at scale.

Algorithms can’t be ‘switched off’

There is no single ‘algorithm’ that can be switched on or off. Data-driven machine learning systems incorporating algorithms are the building blocks of many computational systems.

Even if algorithms could be switched off, it’s typically not advisable. Recommender systems help solve an abundance problem: there is vastly more content available than any person can navigate, including from those that we may directly and actively follow. Recommender systems help curate and locate music, films, people, products and information.

Digital platforms also use algorithms to moderate and recommend content, rank search results, target advertising, and detect spam or other inauthentic activity. They underpin safety systems and are critical to delivering a diversity of content to users. Increasingly, these processes involve machine learning models drawing on large volumes of behavioral data captured by these same platforms, which also means that user practices already partly shape the content of algorithmic feeds. In short, platforms must always decide which content is eligible to appear, what is removed or demoted, how spam is detected, where advertising goes and how other content is presented — and this requires algorithms.

The problem, then, is not the existence of algorithmic recommendation, but what these systems are designed to optimize for, what signals are valued, whose interests those choices serve and what consequences follow. Recommendation systems need not prioritize advertising dollars and eyes on screens.

For example, in a long-term experiment to optimize for kids’ and teens’ wellbeing, YouTube has worked routinely with a group of international experts to channel evidence-based guidance into detailed tweaks of their recommender systems for minors’ accounts. Algorithms already weed out egregious content that violates platform guidelines but this leaves the more delicate question of how to deal with the corpus of content which is non-violative but potentially harmful for children at volume (think, endless makeup videos for girls). YouTube has developed and applied a dual system of safeguards and quality principles to ensure that low quality content that might be harmful at volume is systematically deprioritized and dispersed through kids’ and teens’ feeds to limit how often they encounter it. Unfortunately, since the implementation of the social media age restrictions, Australian under-16s can no longer benefit from this wellbeing-oriented optimization of teen accounts, including teens parent-supervised accounts.

“My Feed, My Way” contrasts the recommended feed with a chronological and/or “following” feed. Giving users choice is sensible and many platforms already offer tools to modify, disable or clear the algorithms shaping their feeds, although they are often difficult to find, use or understand. But chronology is not the absence of algorithmic mediation. A chronological or “following” feed still applies rules about what content is included, excluded, and ordered from a potentially vast pool of material. More importantly, presenting users with a choice between two platform-defined modes offers only a limited form of agency. Our research on advertising transparency has shown the limits of the enhanced-controls approach: providing users with information and controls does not necessarily render platform systems transparent or enhance users’ agency over them.

Nor does user choice necessarily need to mean a binary choice between complex, platform-driven algorithmic recommendation (which usually includes chronology and/or followed accounts) and linear chronology or followed accounts alone. Feeds could prioritize users’ most loved accounts, allow users to nominate sources they particularly value, or provide more direct transparency and control over ranking objectives. Alternative social media platforms like Bluesky take this a step further with its concept of a “marketplace of feeds” where users can design and share curated feeds for the benefit of the wider community.

The algorithm doesn't operate on an empty world

Treating recommendation as the source of harmful content can also obscure a more basic problem: the content must exist before it can be recommended.

This is particularly relevant in relation to Australia's social media minimum-age restrictions. Young people without accounts can still access designated platforms in a logged-out state. For a recommender, this creates a classic “cold start” problem: if the system knows almost nothing about a user, what should it show them?

When examining logged-out or new account experiences, many report on how quickly poor-quality material is surfaced. This raises questions about what platforms recommend to an unknown user, but also about the pool of content from which those recommendations are drawn. If misogynistic or violent material is readily available before a recommender knows much about a user, removing personalization does not resolve the underlying problem.

Importantly, the Digital Duty of Care itself recognizes this broader problem. Particularly for children, it reaches beyond algorithmic recommendation systems to the specific material made available by a service, the design features through which users encounter it, and the wider systems and processes that give rise to risks of harm. In this respect, the duty of care is considerably more sophisticated than the public debate about on-off switches for algorithmic feeds suggests.

The Digital Duty of Care exposes the contradiction

There is a curious tension in the government's position: presenting ‘the algorithm’ as something from which users may want protection, while requiring platforms to become better at anticipating and mitigating harms, which logically requires more sophisticated algorithmic approaches, not fewer.

At platform scale, identifying spam, abuse, coordinated harmful activity or patterns of risky negative exposure cannot be done manually. If we want platforms to identify repeated exposure to, for example, eating-disorder content and intervene, we are asking them to classify content, detect patterns and change how material is ranked or recommended for some or all users.

This is why the detail of the Digital Duty of Care is more interesting than the rhetoric surrounding it. Its central idea is that providers should identify foreseeable risks of harm and take reasonable steps to mitigate them. The bill explicitly asks providers to assess potential harms arising not only from content, but from design features and other systems and processes. It assigns to platforms an overarching responsibility to consider, monitor and address the outcomes of algorithmic features like recommender systems, requiring a ‘safety by design’ approach.

The bill’s proposed rules requiring the provision of user empowerment tools also offers and requires a kind of flexibility that is very far from an on-off switch for algorithmic feeds. Platform specificity will be important: the algorithmic features of YouTube work differently from those of TikTok, Facebook or Instagram; TikTok may have a dominant ‘For You’ feed, while Instagram distributes recommendation across its main feed, Explore and Reels. Even identifying which ‘algorithm’ a user is supposedly switching off is less straightforward than the slogan suggests. Consequently, the controls users need to gain more agency over their experience will differ across platforms.

We can't govern what we can't observe

There is another important but lesser discussed detail of this bill. Politicians can demand that platforms ‘fix the algorithm’, but governments, researchers and the public have limited capacity to determine what these systems actually do.

Source-code transparency is not enough. Socially consequential outcomes emerge when recommendation systems interact with different users, content, contexts and behaviors.

What we need is observability: the capacity to examine these systems in operation. What are different users encountering? Are some groups systematically exposed to different content? What specific changes to recommender systems are effective in reducing harm? Can independent researchers test a platform's claims about what its interventions have achieved?

This matters particularly under a duty of care. If platforms must identify risks of harm, mitigate them and assess whether their interventions work, independent actors require the capacity to verify those claims. Otherwise platforms remain responsible for both operating the systems and telling us whether they are safe.

Encouragingly, the bill begins to recognize this problem. Providers would be required to conduct and retain risk assessments, while separate provisions create a basis for researcher data access and protect approved researchers using ‘sock puppet’ identities to study platforms. The details of those access arrangements will matter enormously, but the principle is important: accountability cannot depend solely on what platforms choose to tell us about their own systems.

Perhaps this is ultimately the problem with ‘the algorithm’ as our regulatory target. It directs attention towards a mechanic when the more important questions concern systems: what they optimize, how they are designed, what choices they afford, what harms they produce, and whether anyone outside the companies operating them can independently see the results.