The case for banning under-13s, the rise of “doom trolling” and slice of Comte

Hello and welcome to Everything in Moderation's Week in Review, your need-to-know news and analysis about platform policy, content moderation and internet regulation. It's written by me, Ben Whitelaw and supported by paid members like you.

Politicians around the world are acutely facing the same big question: do they follow the research on social media bans or ride the wave of parental concern? Today's edition looks at the fallout from last week's UK social media ban announcement, not to mention Germany getting in on the act.

In addition to today's round-up, allow me to recommend Alice Hunsberger's T&S Insider counterargument to the idea that AI is only going to perpetuate harm among LGBTQ+ users. Becoming a member gets you access to the full archive of T&S Insider and Week in Review — and it's easy to expense too.

Welcome to new subscribers from Mozilla Data Collective, eMap, Ofcom, Miudos Seguros Na, the Technical University of Munich, the Competition and Markets Authority, the UK Home Office and elsewhere.

From a sweltering London and in solidarity with all extreme sweaters, this is your Week in Review — BW

Policies

New and emerging internet policy and online speech regulation

In Germany, an independent expert commission tasked with better protecting children online this week presented its recommendations (in German) to the federal government for further review. The so-called Prien Commission — after Karin Prien, Germany’s Federal Minister of Education and Family Affairs — called for a social media ban for under-13s with tiered restrictions for 13-16 and 16-18 year olds. An alternative approach of restricting access to features or services — much like Canada has announced (EiM #339) — was also suggested, setting the major European power apart from the approaches taken by Australia and the UK. Reuters has a high-level story in English.

What do EU think?: While Germany’s approach seems sensible, we don’t know how it will chime with the European Commission’s independent panel on online safety, which delivers its findings in July. A full-scale, bloc-wide under-16 ban has been previously mooted (EiM #333), setting up a potential divergence between Germany law and EU-wide legislation.

On the topic of social media bans, the giant machinery of tech trade associations and industry groups whirred into action to shape the UK's implementation following the government's announcement last week (EiM #340): 

• CCIA, the longstanding US non-profit that represents major platforms on competition policy, say the measures announced last week “need a lot of work” to meet the bar of enabling children to grow up safer online “in a way which is workable and achieves its core goals in practice.”
• The Electronic Frontier Foundation wrote that the ban “will cause more harm than it will prevent” and said “young people deserve better than a policy built on panic”.
• Free market think tank Cato Institute highlighted the UK’s plans to let 16-year-olds vote and said the confused thinking showed “remarkably little consistency regarding when it thinks teenagers should be trusted to exercise their judgment”.

Meanwhile, new findings from an observational study by the University of Newcastle in Australia has found 4 in 5 teens still had access to social media three months after the ban come into play (EiM #318). The Guardian has the write-up, which will no doubt be shared by anti-banners and those who want to further restrict teen access alike. Over to you, Andy Burnham...

Also in this section...

• Lawmakers demand answers on the administration’s Anthropic restrictions (The Washington Post)
• Reported AI-Generated sexualised video (Oversight Board) 
• Separating fact from fiction on the technology used to detect child sexual abuse (IWF)
• CCIA Concerns with Brazil’s Amendments to the Marco Civil da Internet Implementing Regulation (CCIA)

Will we fix AI bias against LGBTQ+ users?

A new report shows how AI systems are already failing LGBTQ+ users. The problem is: it may also the best way to fix moderation issues that traditional systems never managed to address.

Everything in Moderation*Ben Whitelaw

Products

Features, functionality and technology shaping online speech

Meta has already replaced half of human review requests with AI, according to story published by the Financial Times this week. While not explicit, it suggests that more appeals, including post and account takedowns, are being reviewed by LLMs — which is notable in the context of recent reports by several out-of-court dispute settlement (ODS) bodies that show Instagram and Facebook account for a significant number of user complaints.

Meta has also been called out for its lack of co-operation with ODS bodies; I wonder if that’s because providing information on the moderation rationale of its AI systems is hard/impossible.

The piece also buries a few important details that EiM readers may find interesting: 

• A version of Google’s Gemini was used for moderation and customer support until Meta’s new Muse Spark model became the default. That’s the same Muse Spark model that the company has yet share with the US federal government for safety reviews. Hmmm.
• Sources say the AI moderation is happening faster than expected with multiple people claiming is “still makes errors and lacks sufficient oversight”.
• The expansion of AI moderation had led to the company cancelling contracts with third-party moderation groups.

One thing Meta can’t be described as doing — because it gives little public credence to the dangers of AI — is “doom trolling”, the new term given to the communications strategy of major AI Labs. Cal Newport, who many will know as the author of Deep Work, coined the term in a New York Times op-ed that went as far as calling Anthropic and OpenAI “morally indefensible” for their handwashing.

Also in this section...

• Why do AI models struggle with online hate speech detection? (Al Jazeera) 
• Blocking Illegal Content Before It’s Online? French Moderation Startup Turns To AI To Make The Internet Safer (Le Figaro)

Is this what assessing risk *actually* looks like?

Regulators have spent years trying to get platforms to anticipate harm before it happens. Anthropic’s Mythos release suggests some AI labs may already be adopting similar principles.

Everything in Moderation*Ben Whitelaw

Platforms

Social networks and the application of content guidelines

Confusion has reigned in Turkey this week, after reports incorrectly suggested that Roblox’s ban had been lifted. Tech outlet WebTekno reported that users were able to access the world-building site without a VPN and government minister posted on X/Twitter about Roblox’s parental controls. But that is reportedly not the case, meaning the 2024 ban remains in place. Separately, Change.org has been blocked in the country after a legal decision, potentially for its role in organising opposition and funding to anti-government causes.

For EiM readers who remember the tragic death of Ladi Olubunmi (EiM #285), OpenDemocracy has published its year-long investigation into the situation surrounding her death. The findings highlight how Teleperformance, the third-party company who employed on behalf of TikTok, exploited workers — in the words of one non-profit — similar to the “European colonialisation of African nations in the 19th and 20th centuries”. 

Also in this section...

• Israel asked Facebook to censor Iran war content, internal documents show  (The Intercept)
• Australian AI porn empire could be generating child abuse material (ABC News)
• ICE does in-person Instagram moderation now (Boing Boing)

People

Those impacting the future of online safety and moderation

If you’re a regular Ctrl-Alt-Speech listener, you’ll know Mike and I made a passing mentioned of Patreon CEO Jack Comte a few weeks back (Generous to a Default). We gave him his due for his reaction to the Bricks & Minifigs scandal but I’ve really taken to him since listening to this week’s Decoded podcast with Nilay Patel.

In no particular order, he talks about: 

• Content policy as a platform differentiator and claims “one of the biggest differences between Substack and Patreon is our content policy”. 
• Co-iterating Patreon’s policies with payment processors to develop “parallel content policies” that mean creators don’t violate their guidelines — but still needing to send begging emails to CEOs at 3am. 
• How Apple has a “stronghold on the system” as a result of the 30% fee creators must pay for subs via in-app purchase — which may ring alarm bells if you've been following its growing role in the user safety stack.

Patreon isn’t so different to other platforms that it prevents users easily transferring their support for creators. That makes some of what Comte says a little rich. But you should go and listen nonetheless. It’s a great conversation with a likeable, thoughtful CEO — which are frankly few and far between.

Posts of note

Handpicked posts that caught my eye this week

• “What I am saying is this: the policy solutions he advocates for — age verification and social media bans — do not protect children. They build surveillance infrastructure.” - Siobhán MacDermott points out the — how do we say? — unfortunate links between Jonathan Haidt and Peter Thiel.
• “Incredibly grateful to workers, lawmakers, activists, and scholars, whose ideas and words eventually shaped our own understanding of where AI is heading and how we can create a different world.” - as someone who hates badly-used stock imagery, I love the illustrations created by Mohammad Amir Anwar in collaboration with Techworker Community Africa.
• “I spoke to a lot of folks about social media when I was trying to create a social platform that was the reverse of the ones we have today, and he had some of the most creative ideas I heard.” - Nicholas Thompson speaks to New_Public’s Eli Pariser on the latest episode of his podcast. Very much recommended listening.