Indian apologies, beware the ‘takedown extortionists’ and Schneider on becoming CEO
Hello and welcome to Everything in Moderation's Week in Review, your need-to-know news and analysis about platform policy, content moderation and internet regulation. It's written by me, Ben Whitelaw, and supported by paid members like you.
There's no time for pre-amble this week — too many stories to get to — but I must welcome new subscribers from eSafety Commissioner, GetStream, Ofcom, Vinted, Christchurch Call, MLex, Spotify and the excellently named heikadog. If you enjoyed today's newsletter, forward it to a friend or colleague.
Becoming a paid EiM member supports independent coverage of the T&S industry at a time when it's most needed and gets you unfettered access to EiM's archive of 450+ editions, including this week's debate on whether digital safety regulators should attend T&S conferences.
Here's the big stories from the past week — BW
The next trust breaking points do not look like the last ones.
AI is changing the shape of trust and safety risk. The next challenges won’t always start with a single piece of content or a clear policy violation. They will more likely originate from autonomous agents acting across systems, synthetic identities passing verification checks, physical AI creating real-world consequences, or coordinated manipulation moving faster than human oversight can respond.
PwC’s Trust and Safety Outlook 2026 explores seven themes shaping this shift, and why leaders need to rethink how trust is designed, governed, measured, and operated before pressure points become failures.
Policies
New and emerging internet policy and online speech regulation
The Indian government received an apology from Meta executives this week after the company briefly took down a video by Prime Minister Narendra Modi after what it called an “error in its AI-powered automated content filtering system”. Chief global affairs officer Joel Kaplan (aka the new Nick Clegg) was in New Delhi to explain the takedown and promise that “new protocols” had been established to prevent similar events happening again. Separately, police in the southern city of Hyderabad brought a case against the head of Meta’s India business for “abusive” videos of Modi.
Strained relations: India has pushed its luck with platforms in the past (EiM #110) and has had a strained relationship with Meta over the years. This particular episode has been building since May, when two million students were forced to retake an exam after the papers leaked. That led to a week-long ban of Telegram (EiM #346), followed by a flood of online dissatisfaction towards Modi and his ministers, the resignation of the education minister and large protests in New Delhi on 20th July. With the government under so much pressure, Meta — for all its failures — is a familiar, and easy, target.
Ben Whitelaw
A breaking story overnight: a New Mexico judge has ordered Meta to pay another $567m for being a “public nuisance” and failing to warn the public about the dangers of using its platforms. On top of the earlier fine (EiM #330), it makes the total damage to Meta almost $1 billion. The company has said it will appeal, although the amount is 1/60th of the revenue it made in Q2 this year.
Also in this section...
- ChatGPT, Roblox to Fall Under Strictest EU Digital Rules (Bloomberg)
- Europe’s Billion-Dollar Google Fine Is a Shot at American Sovereignty (Heritage Foundation)
Ben Whitelaw
Products
Features, functionality and technology shaping online speech
Mistral, the French AI company, has released Shieldstral, an open-weights multimodal safety classifier that, despite its weeny three-billion-parameter size, can reportedly outperform an OpenAI safety model seven times larger.
Like other startups on the market, the model allows T&S professionals to define what’s safe by writing plain-text policies, rather than relying on an in-built harm taxonomy. That means it doesn’t need retraining when new harms emerge. And its major innovation is supposedly providing a yes/no response without a chain of reasoning, thus making it cheaper and faster for teams to run.
Also in this section...
- Reddit is introducing a new moderator: AI (The Verge)
💡 Become an individual member and get access to the whole EiM archive, including the full back catalogue of Alice Hunsberger's T&S Insider.
💸 Send a tip whenever you particularly enjoyed an edition or shared a link you read in EiM with a colleague or friend.
📎 Urge your employer to take out organisational access so your whole team can benefit from ongoing access to all parts of EiM!
Platforms
Social networks and the application of content guidelines
Telegram founder Pavel Durov has pointed the finger at “takedown extortionists” who he says found a way to “manipulate Apple into overreacting” and removing the messaging app from the App Store. In an X/Twitter post, he also claimed that Telegram uses “all kinds of moderation tools” and that “illegal pornographic content in Telegram’s public groups is not a systemic problem”. The fact Durov uses that term instead of child sexual abuse material (CSAM) is, I think, very telling.
A technical trick?: You may be wondering what was the sophisticated means by which the attacker inserted CSAM into groups. The answer is: editing old messages, which meant they were “effectively hidden from the group’s members”. He fails to say is that Telegram made a conscious product decision to allow users to edit a message within 48 hours, compared with 15 minutes for WhatsApp and 24 hours for Signal. If you make it easy for harm to be perpetrated, it will be used as such.
A jaw-dropping report from Bloomberg alleges that the death of a 16-year-old in 2022 was impacted by a TikTok A/B test that meant he didn’t get the safer algorithm that other users received. Chase Nasca was part of a control group of around 15 million users and subsequently received content recommendations that promoted and normalised self-harm as well as non-violative but upsetting videos of suicide and mental health struggles. It’s bad timing for the video platform after news about the closure of its Nashville office (see: Posts).
Mike and I talked about various angles of this story in the extended episode of this week’s Ctrl-Alt-Speech.
Also in this section...
- Boy George hits out at “bunch of c***s” Bandcamp for removing pro-Israel song (NME)
- Snapchat is the latest platform to turn against AI slop (Mashable)
- Nvidia is quietly staffing a new AI safety team as it doubles down on open models (Business Insider)
- Apple launches legal challenge to UK attempt to access encrypted user data (Financial Times)
People
Those impacting the future of online safety and moderation
Almost a month after being appointed full-time CEO of Bluesky, Toni Schneider has spoken to Nilay Patel in a lengthy interview for The Verge’s Decoder podcast.
Schneider, for folks who don’t know, was the founding CEO of Automattic, the company behind WordPress.com, before becoming a venture capitalist and an early Bluesky adviser. He stepped into the top job on a temporary basis after then-chief exec Jay Graber opted to move into an innovation role and had the “interim” removed soon after.
The interview is worth listening to in full, but I paid particular attention to Schneider’s answers when Patel asks how content moderation scales. He describes it as “a lot of work, but it’s manageable” — perhaps understating just how much flak Bluesky’s team gets for how it moderates.
More interestingly, he suggests the problem will evolve as the company starts handling more private data through AT Protocol-based products and introduces monetisation later this year. In other words, Bluesky’s moderation challenge is about to get a whole lot more complex.
Posts of note (TikTok workers edition)
Handpicked posts that caught my eye this week
After the news about the closure of the TikTok Nashville office (see: Platforms), I wanted to feature some of the workers now looking for jobs in the hope that EiM readers might be able to help. Reach out if you can.
- “I was hopeful for, and looking forward to, having the opportunity to learn more, and grow further in my role as a Critical Operations Content Moderator, and in a newfound career of Trust & Safety.” - Former 911 dispatcher Sean Murphy hopes to stay in T&S after his career transition.
- “Leaving a place where so much great work was accomplished and so many incredible memories were created is truly bittersweet.” - You can't have enough T&S training specialists like Bart Dement.
- “Yesterday, while enjoying PTO with my family, I received the news that I would not be returning to my role at USDS TikTok as part of the Nashville layoffs.” - Quality analyst Cheyenne Broyles plans to finish her vacation before deciding what’s next.
Member discussion