Spotlight on TikTok child safety, Paris AI call falls flat and the regulator revolving door
Hello and welcome to Everything in Moderation's Week in Review.
A warm welcome to new subscribers from Carnegie Mellon University, Bumble, Nara Communications, SmugMug, Columbia University, Viafoura, Ofcom and elsewhere.
Earlier this week, I wrote about what the T&S industry loses when the focus is on compliance. Many of you got in touch to say it struck a nerve and, with the strong regulatory impetus in today's edition, I don't expect that nerve to stop throbbing any time soon.
Here's what I've been reading and thinking about this past week — BW
Regulators no longer ask platforms to promise how safe they are; they ask them to prove it. As DSA and other regulatory enforcement matures, "we have a policy" is no longer a defensible position. Every removal now needs a reason, and every reason needs a receipt.
George Billinge, CEO of Illuminate Tech - and a former Ofcom regulator - joined Checkstep’s webinar on what safety-case readiness actually means in practice: what regulators look for, where platforms fall short, and how to turn day-to-day moderation decisions into transparent, audit-ready records.
You can watch the webinar on demand for free.
Policies
New and emerging internet policy and online speech regulation
On Monday, French President Emmanuel Macron convinced twelve countries and ten companies, including OpenAI, Anthropic, Microsoft and Mistral, to sign the latest set of voluntary commitments to roll out safer AI models. The so-called Paris Call for AI Safety and Security lays out the need to improve independent, public assessment of the most advanced models and better sharing of alerts after serious incidents but sets no deadline for doing so and doesn't go a great deal further than the 2024 Seoul agreement or even the voluntary commitments signed under during the presidency of Joe Biden (remember him?) in 2023.
Missing persons: As always, the story is more about who isn't on the list, rather than who is. Google and Meta did not sign, and the US, China and middle powers like South Korea, Japan and Brazil are conspicuously absent. Australia seemingly took a pass because it has announced that it is pursuing its own AI regulator, following the breach of government systems by OpenAI earlier this year (EiM #305).
It all adds to the idea that the EU still can't get it together on regulating AI. According to Politico, 16 of the blocs 27 countries has failed to appoint a national authority to oversee the much-heralded AI Act, replicating the issues we've seen with countries appointing Digital Service Co-ordinators under the Digital Services Act (EiM #361). Dex Hunter-Torricke, my guest co-host on Ctrl-Alt-Speech this week called the Call a "large bucket of warm spit".
Over in the US, however, things are moving: New York followed California and Illinois by introducing obligations for frontier labs to share details of have to share safety frameworks on their websites, report critical safety incidents within 72 hours, and issue public model transparency reports. TechCrunch has more about the RAISE Act and how it came about.
Also in this section...
- Europe's missing watchdogs: How both the DSA and the AI Act are failing on enforcement (Disinfo.eu)
- Equality Now Report Links Online Misogyny to Violence Against Women (Africa Brief)
- Targeted consultation on the draft revised election guidelines under the Digital Services Act (European Commission)
- Gov't weighs tighter social media restrictions for minors, targeting addictive features (Korea Times)
Ctrl-Alt-Speech Patreon exclusive: When The Chips Are Down
In this week’s episode, I'm joined by Dex Hunter-Torricke, former Meta, Deepmind and SpaceX communications exec and founder of the Center for Tomorrow, to talk about:
- What the midterms means for Big Tech
- Regulating the frontier
- How serious should we be taking AI safety fears?
Products
Features, functionality and technology shaping online speech
With the spectacular rise of bot traffic changing the way the internet works, Cloudflare and Tools for Humanity have put forward a framework for working out who is behind an AI agent. The Trusted Agent Delegation (TAD) would allow sites to ask for a range of information from an agent, from no user identifier at all up to a government ID for riskier actions. The push towards providing identity documentation has echoes of the current age assurance debate (EiM #360 and others) and will concern privacy experts.
One platform product story from last week: the 'Find Friends' feature that is synonymous with Snapchat is being disabled for under 18 users after being pushed by the UK regulator Ofcom to do more to protect children from grooming.
Also in this section...
- Reddit pivots to a login-only experience (Marketplace)
- Jess Phillips calls for phone tech to block live-streamed child sex abuse (BBC News)
Platforms
Social networks and the application of content guidelines
[warning: mentions of child sexual abuse material] A must-read, wide-ranging investigation by Spanish non-profit outlet Maldita into TikTok's child safety processes found more than 20,000 profiles of suspected paedophiles over the course of a month, with many using TikTok Live to pay children. The details are difficult to read, particularly the analysis of 'Team 94', a group associated with convicted child abuser Matthew Estes (EiM #358). Spain's Youth and Children Minister has already met with TikTok representatives on Tuesday and asked the country's data protection agency to suspend the platform while an investigation takes place.
Tit-for-tat: The non-profit has also claimed that TikTok got friendly media organisations to publish articles suggesting Maldita was hampering violative content from being removed by not sharing its investigation findings. I don't support this view: news media isn't ever obliged to share source material, especially not when its as sensitive as CSAM and definitely not when the journalists have already reported the content via the platform. If reporters are working as “unpaid content moderators”, maybe the answer is to hire more journalists? Or at least some humans who know what they're doing?
Still on TikTok, and linked to the investigation, the platform has been found preliminarily in breach of the Digital Services Act for the design of its "For You" feed and recommender systems, which caused a "rabbit hole effect" that exposed users to isolating and body image content.
Also in this section...
- A Close Read of Big TikTok and Meta Settlements Raises Important Questions (Tech Policy Press)
- Exclusive: Former Meta and OpenAI Contractor Sama Liquidates, Citing Lawsuits (TIME)
- Meta Returns to Kenya's High Court Over Hate Speech in Ethiopia (Tech Policy Press)
People
Those impacting the future of online safety and moderation
The revolving door between regulators and platforms is back on the agenda. Politico has reported that UN experts have raised concerns about the independence of Ireland’s Data Protection Commission. The statement doesn’t name anyone, but it comes a year after former Meta lobbyist Niamh Sweeney joined the regulator.
I wrote about the issue back in 2020 when a senior Ofcom official joined Meta. Since then, former platform staff have turned up at the UK regulator and others around the world.
Now, this isn’t a bad thing per se: regulators need people who know how platforms work. But the DPC is lead regulator for much of big tech in Europe, so there’s a strong case for more transparent conflicts rules and clear recusal.
Check out the full list of people in one place, with a link back to the edition they appeared in > EiM People Directory
Posts of note
Handpicked posts that caught my eye this week
Posts
- "I was pleased to speak to Sky News about the role technology can play in helping platforms meet these requirements. AI can flag content at scale, with trained human moderators making the final call on borderline cases." - VerifyMy's Andy Lulham on the under-the-radar UK law with online safety implications.
- "We know this work will never be finished — new risks emerge, technology evolves, and platforms must continually learn and improve to stay ahead. Today we shared more about our approach to safeguarding our platform and sharing information with law enforcement partners." - Roblox's Matt Kaufman with a new blogpost on how the platform keeps its users safe.
- "We're getting together in person. Join the T&S community for a holiday happy hour on Thursday, December 10 at 5pm at the Sixtyfour office in San Francisco." - SixtyFour's Julian Wong shares details of a SF T&S get-together.
Member discussion