A second Online Safety Act?, OpenAI enters CSAM detection and Meta’s tween push
Hello and welcome to Everything in Moderation's Week in Review, especially new subscribers from Mod.io, GoFundMe, Sustainable Media, Like Goblins, Falcons.ai and elsewhere.
For the next week, I'm giving new members 20% off the annual membership. As well as full access to the EiM archive (500+ posts), you'll get:
- Alice's latest T&S Insider piece on the questions we need to be asking the next time there's a Meta/TikTok settlement on the cards.
- Priority access to our upcoming EiM Community Call and the chance to connect with other EiM readers.
- Access to ten in-depth practical guides on everything from how to write user-facing policies to getting buy-in for T&S work.
Without further ado, here's your Week in Review — BW
How do you train a CSAM classifier on data nobody on the team is allowed to see?
The Roke team behind Vigil AI explain how they built it in a secure, air-gapped lab using the UK’s Child Abuse Image Database, how they protect victims’ privacy, and why they won’t open-source it: released weights would help criminals find material and evade detection.
Resolver’s Athena brings the classifier to platforms, with no imagery retained.
Policies
New and emerging internet policy and online speech regulation
The UK's Online Safety Act has been criticised for leaving gaps after Google rejected a government request to take down a YouTube video in which convicted rapist Russell Brand’s wife read out a message from him while he was in prison. Labour MP Jess Phillips admitted the government has no power to force the video’s removal, calling Ofcom’s voluntary codes on women’s safety “the triumph of hope over experience” and saying a second online safety act is “very likely”. An already large and complex piece of legislation is seemingly about to get even larger and more complex.
A UK parliamentary committee has called for the creation of an AI regulator that would have the power to stop frontier lab training, block deployments or suspend AI services entirely if safety checks are not in place. Guardrails for Growth, Making AI Safe highlights many of the same issues as the Paris Call for AI Safety and Security (EiM #362) but, unlike the voluntary call, is clear that it can't depend on voluntary cooperation from the likes of OpenAI and Anthropic, who the committee chair said acted like "modern-day Dr Frankensteins".
Not that the UK government can take the high ground this week: 18 months after a leak, a tribunal confirmed that the Home Office had ordered Apple to give it access to encrypted customer data. Privacy campaigners said the "government’s actions undermine [the public's] privacy and safety" and celebrated the decision. The Financial Times has the story.
It comes in the same week that California brought a case against OpenAI alleging that cyberattacks, including the Hugging Face incident in July (EiM #348), created a "public nuisance" and amounted to unfair or fraudulent business practices.

Also in this section...
- European Commission Plan to Give Frontex Trusted Flagger Status Is A Threat to Free Speech (Electronic Frontier Foundation)
- Press START to prove your age! How the EU KIDS Act could age-gate online games (KU Leuven)
- Can EU Action Against TikTok Rabbit Holes Lead to Real Social Media Design Change? (Tech Policy Press)
Ctrl-Alt-Speech Patreon exclusive: Make America Break Again
Mike is back in the co-host chair and we get into the outages that took down banks, air travel, the NSA and Apple. Plus stories relating to Anthropic, Snap and the UK social media ban and, for Patreon supporters, OpenAI and Ofcom's push to detect novel child sexual abuse material (CSAM).
Products
Features, functionality and technology shaping online speech
OpenAI has released a free classifier that detects child sexual abuse material (CSAM), including novel, previously unseen material. It gives each image two scores: one for likely CSAM, and a separate one for whether the material may be AI-generated. OpenAI says it’s the first broadly available CSAM classifier with that kind of AI-generated signal. It also has a first-of-its-kind model card, which means it can be compared with other classifiers and shows the areas where it is weak — namely detecting images featuring Black children and boys.
Timing is everything: In an amazing coincidence, the classifier was announced the same day that Ofcom announced new measures for certain platforms to detect previously unknown CSAM by February 2027. Respondents to the UK regulator's consultation in June last year had noted the high costs of CSAM detection at scale, so the fact that OpenAI is entering the market with a free tool is a win — unless you're a detection vendor that charges.
Also in this section...
- Attackers are using AI agents to automate cybercrime at 'industrial scale,' FBI and AWS warn (The Register)
- American A.I. Models Have Potential to Be Turned Against U.S., Study Finds (New York Times)
- As Meta bets big on Muse AI agent, early safety mishaps surface (Reuters)
Platforms
Social networks and the application of content guidelines
Meta’s efforts to explicitly go after users below its minimum age requirements are laid out in previously unreported documents obtained through discovery in a major multidistrict lawsuit and reported by NPR.
In one particularly creepy presentation titled “The Young Ones are the Best Ones and Other Learnings”, they show how 10- to 12-year-olds have the highest long-term retention and that the company should “whole should consider investing more heavily in bringing in larger volumes of tweens”. It backs up accusations made during the Los Angeles trial earlier this year (EiM #327) that the company sought to onboard tweens “to win big with teens”.

A new report from one of the out-of-court dispute settlement bodies under the Digital Services Act hints at the growing issue of platform-enabled scams and fraud. Appeals Centre Europe (ACE) noted that it was the leading driver of cases over the last six months and overturned more than 80% of eligible cases, showing that platform moderation of fraud and scams leaves a lot to be desired.
In demand?: The question has always been whether there is public demand for ODS bodies' work. Two stats in the ACE report suggest there is: firstly, disputes trebled over the course of the year (October 2025 to September 2026) and secondly, almost 43,000 disputes were received, which is two thirds of its total review since it became operational in 2024. Eligibility is still a problem but the trend is upwards.

Also in this section...
- An Open Letter from Members of Teen Online Safety, Wellbeing, and Youth Councils Around the World (Snap)
- Instagram's Algorithm Is Promoting Antisemitic Content, Report Finds (Futurism)
- Elon Musk's AI Chatbot Grok Sexualized These Teens' Childhood Photos (Bloomberg)
People
Those impacting the future of online safety and moderation
When the discussion turns to potential misuse of open AI models, a lot is said about cyberattacks, CSAM, disinformation and even biological harm. But it feels that there is comparatively little focus on radicalisation and terrorism.
Adam Hadley, the executive director of Tech Against Terrorism, believes that's a mistake. A recent report from his organisation, in which it tested 162 open-weight models, found that modified open-weight systems provided useful weapons and attack-planning information, and guardrails could be stripped within days of release.
Hadley doesn't mince his words: he calls it "a major public health and national security crisis" and says:
"we've only got a few months to focus the minds of our political classes for them to do something … I certainly don't think hard regulation is the solution to this in most cases".
So, who should be made to do something? Regulators like the eSafety Commissioner believe repositories like Hugging Face have a greater role to play in excluding unsafe models from their platforms. But the focus, as Australia and the UK has made clear, should be about models being released in the first place.
Posts of note
Handpicked posts that caught my eye this week
- "We surveyed 438 LGBTQ+ young people aged 13–19 across the UK and Channel Islands to understand their experiences of online blackmail, its impacts and the support they need." - University of Birmingham's Willem Stander on new NSPCC research about the impact of online blackmail.
- "It looks like a policy in search of a problem that didn’t exist and unable to address even basic issues such as time spent online for kids." - Chris Ferguson of Stetson University on the flaws of a social media ban and what we should be doing instead.
- "We will use data from the Wellcome-funded IMAGINE study as well as GP and hospital data to look at changes in depression and anxiety among young people before and after the ban." - UCL's Francesca Solmi on being part of a huge new Wellcome Trust-funded study looking at the effect of the UK social media ban.




Member discussion