EU teen ban lands (with a twist), AI labs' safety PR blitz and a T&S story worth telling
Hello and welcome to Everything in Moderation's Week in Review, your need-to-know news and analysis about platform policy, content moderation and internet regulation. It's written by me, Ben Whitelaw and supported by paying members like you.
Everyone's briefing this week. Whether that's government teeing up major new kids safety regulation (See: Policies) or AI labs sharing crumbs about their new kumbaya moment (See: Platforms), there's a lot of noise about who's doing safety well right — and crucially, who should be leading it in future. I've waded through most of it so you don't have to.
I'm excited to be launching something new: EiM monthly Community Calls. A dozen readers, 60 minutes, discussing a topic that matters to EiM readers. Paying members get priority but I'm opening up to all newsletter subscribers as of today. Register your spot for the first one on Monday 28th September, which will be pegged to my recent piece on why more people need to know about T&S.
Let's jump into this week's major talking points — BW
Powered by data from trusted sources and Thorn’s issue expertise, Safer helps trust and safety teams proactively detect and prioritize CSAM and child sexual exploitation conversations with:
- Proprietary hashing and matching for known CSAM
- CSAM Classifier for finding new or previously unreported content in images or videos
- Context Labels for classified CSAM that can be used to filter, sort, and prioritize flagged content.
- Text Classifier provides risk scores for conversations that contain sextortion, grooming or requests for self-generated content from a minor.
Online harms continue to evolve. Get purpose-built solutions backed by experts in child safety technology.
Policies
New and emerging internet policy and online speech regulation
The much-trailed EU KIDS Act was finally announced this week with some of what we expected — a graduated approach to access, strong focus on product design (EiM #344) — and a surprising twist: the social media ban law also includes AI chatbots and online video games that pose “specific design risks for minors”. If you’re catching up on the details, here’s the gist:
- Under 13s cannot have personal accounts but can access with parents and are allowed to use “kids-safe” apps — which explains OpenAI’s recent GPT for Teens move (EiM #350).
- 13-14-year-olds get limited access to features and time limits of an hour a day
- Underpinned by "effective" age assurance technology or the EU’s own open-source but maligned Age Verification solution (EiM #333).
- A ban on design features like infinite scrolling and certain types of notifications plus tighter guidance on recommender systems.
Da Do Macron Ron: von der Leyen’s announcement came just a few days after France submitted a reworked social media ban, after having its previous version struck down by the country's Constitutional Council for being disproportionate. It allowed President Macron to crow on X/Twitter that he had seen through his commitment to "protect our children online", despite said law not yet coming into effect yet or any evidence showing that it would do what it set out to do. Talk about trying to control how history remembers you, Emmanuel.
Across the Channel, UK politicians have reacted to misinformation and harassment of lifeboat volunteers (a big no no by any Brit’s standards) by making doxxing illegal under the Online Safety Act. As Politico reports, UK culture secretary Lisa Nandy met platforms this week — save X/Twitter which didn’t send a representative — to implore them to do more in the short term.
The prevalence of doxxing is difficult to track since it can fall under existing UK laws covering stalking, fraud or computer misuse. However, the recent case of a 17-year-old boy convicted for online offences has put it higher on the agenda. Much to the annoyance, I'm sure, of women's and trans rights campaigners who have called for stronger protections against doxxing for years.
Also in this section...
- EU faces revolt over social media ban for children (France 24)
- Digital sovereignty and Europe’s democracy stack (Atlantic Council)
In this week's Ctrl-Alt-Speech Podcast....
Mike and I are back with our usual weekly round-up, talking about everything from Europe's social media ban to some knotty data about the effect of AI on education.
Get it now and extended if you're a Patreon supporter — from just a few dollars a week — or get a slimmer version this evening wherever you get your podcasts.
Products
Features, functionality and technology shaping online speech
A startup that audits AI models for companies like ElevenLabs and Cursor this week raised $55 million to build out its safety standard for agents — right at a time when independent AI assessment is top of the agenda. The Artificial Intelligence Underwriting Company — bit of a mouthful but in a refreshing way — is built on some impressive pedigree; its founders are ex-Anthropic and ex-METR and its standard is based on feedback from 250+ AI leaders.
And the funny thing? Its whole schtick involves taking already well-understood cybersecurity standards, building a fresh version for agents and using that to create a 100-page report of where a model works or doesn't. TechCrunch has more, but my thought: maybe the future isn't all sci-fi death notices; it's more well-paid auditors and consultants providing fallbacks for companies who should know better.
Also in this section...
- UN High Commissioner for Human Rights presses for urgent AI governance (Dig Watch)
- Beyond Age-Gating: Regulating Platform Design for Child Safety (Observer Research Foundation)
- Google rolls out age inference in Canada (Biometric Update)
Ben Whitelaw
Platforms
Social networks and the application of content guidelines
After a weekend in which every major global power — and some intergovernmental ones too — has raised concerns about the pace of AI development, a story conveniently emerged on Tuesday saying that OpenAI has been working together with Anthropic and Alphabet — Google’s parent company — “to prioritize safety” and had been doing so “for several weeks”. The briefing, by chief global affairs officer Chris Lehane, comes just a week after OpenAI published a blog post outlining its desire to work on “industry-led standards” — although the majority of his 2,000 words were pushing for US Congress to step in and decide for them. But, you know, it pays to look concerned right now.
Tale of two IPOs: Aside from the questions about how frontier models are audited and regulated, the safety concerns of elected officials and the general public also appear to have a material effect on the much-discussed IPO plans: Anthropic say the transparency of being a public company is better for users — which might make UK officials chortle — while OpenAI said doing so would be “ill-advised” to proceed and looks like it will now wait until 2027.
That’s not stopped OpenAI CEO Sam Altman claiming that the public should “trust that we are going to do the right thing”. Seriously, what planet is this guy on?
Also in this section...
- Watchdog bans ‘nudify’ ads on Meta over sex abuse concerns (The Independent)
- The TSJC judges the mass layoffs in the content moderator companies of Meta in Barcelona (ARA)
People
Those impacting the future of online safety and moderation
I’ve been writing and talking recently about how Trust & Safety professionals can explain the work they do and inform the general public and then, lo and behold, an example appeared in the wild.
Writing for BuiltIn, Bella Blue, head of trust and safety and support at Stack Overflow, outlines the challenge that comes from a glut of AI-generated content allied with limited — and in some cases declining — human resources needed to police it.
She goes further by explaining that, while AI can triage scaled harms like spam and low-quality content, it can't replace the context, ethics and pattern-recognition that T&S professionals bring to novel harms. That’s because every model is trained on "yesterday's knowledge" but new abuse patterns crop up all the time.
Her call-to-action: companies must invest in well-staffed moderation teams and plug them into scalable AI-powered systems that can manage known harms, but spot new ones too. Seems sensible to me.
Posts of note
Handpicked posts that caught my eye this week
- "I made my 13 year old son sit through a 30-minute WELCOME TO THE INTERNET presentation before I gave him access to WhatsApp. Like, an actual slide deck, created by me, to talk about our rules for digital safety." - Saving this fun presentation from Brandy Fleming for when my son is (quite a lot) older.
- "I’m happy to share that I’m starting a new position as Interim Director, Trust & Safety at Depop!" - friend of EiM Emma Beddall celebrates her new job at one of my favourite online marketplaces.
- "We published how we rebuilt moderation to run synchronously: scanning frames as they generate, in under half a second, and halting the stream when something's wrong." - Conner McDowell shares more details on how AI-powered creative platform Runway does real-time video moderation.
Member discussion