Zuck joins AI regulation debate, Meta rolls out verification and abortion censorship

Hello and welcome to Everything in Moderation's Week in Review, your need-to-know news and analysis about platform policy, content moderation and internet regulation. It's written by me, Ben Whitelaw and supported by paid members like you.

There's a growing sense that we must get AI regulation right and we must get it right soon. This morning's Anthropic story (see: Platforms) is not going to quell the calls for tougher oversight and now workers within the world's largest labs are joining the call for a development slowdown (see: Policies) while governance catches up.

Talking of slowing down (see what I did there?), Week in Review is my attempt to help you pause and make sense of what matters in online speech and internet regulation. Each week, I read and curate 40+ articles so you don't have to. Yes, I could send you an AI-slopified version in a fraction of the time, complete with the occasional invented detail, but I like you all better than that.

Almost 60% of subscribers opened last week's newsletter, so I know many find it useful. If you look forward to it, consider becoming a paid supporter of EiM to keep it going.

Welcome to new subscribers from the Council for Foreign Relations, Checkstep, Google, Temu, User-Rights, Berlin University of Applied Sciences and Economics, Assembly Research, Monash University and a bunch of others. Hit reply and say hello.

These are the big stories for the last seven days — BW

in partnership with Thorn, a nonprofit transforming how the world protects kids in the digital age.

New Thorn research finds children turning to chatbots for guidance after online sexual interactions.

Each year since 2019, responses from thousands of young people surveyed by Thorn help us understand how they use technology and the challenges they face. 

Thorn’s 2025 Youth Perspectives Report offers unique insights into young people’s behaviors and experiences as they navigate today’s online world. The findings reveal a reality that is complex, evolving, and often misunderstood.

From AI chatbots to age-gated platforms, this report provides a clearer picture of what kids are facing and a roadmap for how we respond. 

Download the report to learn what trust and safety teams can do to protect users.

GET THE RESEARCH

Policies

New and emerging internet policy and online speech regulation

The debate about how to regulate AI has really hot up this week as the ramifications of the OpenAI/Hugging story (EiM #345) have sunk in and tech companies have fought to re-establish control of the narrative:

• Meta CEO Mark Zuckerberg used an op-ed in the Wall Street Journal to argue that the US should accelerate AI development rather than restrict innovation. His stance makes sense in the context of the company's lagging behind other frontier labs but it's funny to see how far he's come since agreeing to the principle of European-created AI rules back in 2023 (We know how that ended up...). He also gave an interview to the Financial Times.
• Meta’s position runs at odds with his own employees and the chief scientists of OpenAI and Anthropic, who were among almost 1,300 employees of frontier companies called for government intervention to slow down AI development. Does that mean the national security argument is dead? Unclear.
• An entity modelled on the US’ Financial Industry Regulatory Authority looks to be the way that AI will be regulated in the US. But as three Council on Foreign Relations experts highlight, five outstanding questions could make or break its impact.   
• There’s a bigger conversation emerging, as pointed out by the Atlantic Council’s Kosntantinos Komaitis, about how elected governments, corporations, civil society and academia work together to implement any regulations. A federated approach is his recommended way forward but the those relationships can't be built fast enough.

Vietnam is jumping on an increasingly crowded bandwagon by proposing social media limitations for under-16s, according to a draft law seen by Reuters. Unlike other countries, though, the south-east Asian nation would require legal guardians to register an account, which could only view content — not like, comment or share. The focus, according to a government official, is for the child to be “placed in an age-appropriate environment”. It follows Indonesia (EiM #327) and Malaysia (#338) in moving towards a ban.

Also in this section...

• Response to Ofcom’s Request for Research on Fraudulent Advertising and Account Integrity (Integrity Institute)
• Regulation is working. That might be the problem (Internet Policy Review)

Are we entering Trust and Safety’s builder era?

In contrast to the precarity facing many T&S workers, most TrustCon attendees were energised by what AI now allows them to build and the impact they can have with it

Everything in Moderation*Ben Whitelaw

Products

Features, functionality and technology shaping online speech

Identity verification was previously a pay-to-play feature on Meta’s platform — all you needed was a government ID and $14.99 a month  — but is now coming to all users in lite form as part of an effort “to provide a way for you to know there is a real person on the other side of a profile”. But the announcement — and the lack of information on its Verified terms of service — leaves several important questions unanswered:

• Whether users will be prompted to become verified — and which users Meta will target first.
• What incentive users will have to complete the process beyond receiving a badge.
• Whether Meta has tested the feature and what effect it had on scams, impersonation or other forms of abuse.
• Which verification technology it is using, how long biometric data will be retained and whether it will be used for any other purposes.

Other platforms have moved more aggressively into identity verification but for more specific reasons than this new development:

• LinkedIn launched last year to protect users "from inauthentic interactions" and use Persona to verify their identity using government-issued identification.
• X/Twitter introduced identity verification for Premium users using government ID and, in some cases, a selfie or biometric check. It has worked with Persona and Stripe Identity for different verification processes.

Giveth with one hand, etc: Meta’s press release notes that, as “AI makes it easier to do more on Facebook, a clear signal that distinguishes real people becomes essential”. It is hard to disagree. It is also hard to ignore that Meta has invested vast sums in building the technology that makes synthetic profiles, messages and representations of real people cheaper and more convincing (EiM #343). Seems to be the new Faustian bargain of the AI internet.

Also in this section...

• What Can the Pangram AI Detector on Substack Actually Tell You? (Agents of Influence)

Platforms

Social networks and the application of content guidelines

New this morning: Anthropic has found that its systems autonomously hacked three organisations as far back as April, following a review of some 140,000 tests on the back of the OpenAI/Hugging Face debacle (EiM #345). The company said in a statement that the issue was in part because models were given access to the open internet despite being told it didn't. It plans to improve security of evaluation environments to avoid it happening in the future.

It's not been a good week for Telegram. Australia’s eSafety Commissioner has taken the messaging app to court, alleging that it failed to remove material promoting terrorism after being instructed to do so. According to the BBC, the regulator says Telegram left some reported content accessible for months; the company maintains that it removes vast numbers of terrorist-linked channels and posts. The case arrived within hours of Russia charging Telegram founder Pavel Durov with facilitating terrorism, accusing the platform of leaving up channels and bots allegedly used by Ukrainian intelligence and extremist groups.

With child safety dominating political debate, I found it notable that multiple platforms announced expansions of their youth advisory councils this week:

• Snap announced its new 19-member US Council for Digital Well-Being, following the creation of similar groups in Europe and Australia. Members will spend the next year discussing issues including reporting tools, social-media bans and the growth of AI.
• Roblox has expanded its Teen Council for Civility and Well-Being into five new Latin American countries. It is not clear whether the expansion will increase the council beyond its current 20 members or divide the available places across a much larger geographic area. Roblox also operates a separate Asia-Pacific council.

My question is: Do these teens commercially or politically inconvenient decisions? Or is for the promo video? More to the point, could they even be expected to? 

Also in this section...

• Meta grilled at racism inquiry over lack of Indigenous content moderators (ABC News) 
• China-linked AI ‘nudify’ ads flooded Facebook and Instagram, watchdog says (LA Times)
• “Google and Reddit do not own the Internet,” web scraper says after court win (Ars Technica)
• Meta just created a moderation nightmare for its smart glasses (The Verge)

People

Those impacting the future of online safety and moderation

The work of civil-society and non-governmental organisations in the online-speech space can easily go unrecognised. Yet these groups frequently define the red lines of platform policy, document the harms that companies do not measure and help users whose accounts disappear into the grey areas of enforcement.

For an example, look no further than Repro Uncensored, a small nonprofit that tracks the suppression of abortion, queer and feminist speech. Its founder and executive director, Martha Dimitratou, spoke to the Abortion, Every Day newsletter about the daily game of whack-a-mole involved in getting affected accounts restored. Just last week, more than 120 feminist and queer accounts disappeared in Turkey.

Her interview also explains how people in restrictive US states are increasingly turning to frontier models for abortion information, placing reproductive-health activists on the frontline of yet another moderation system that they cannot inspect or meaningfully appeal. As I mentioned further up in today’s newsletter, the current moment we find ourself in needs voices like Dimitratou’s and other activists working on issues that, dare I say, are hard to spot in an eval. 

Posts of note

Handpicked posts that caught my eye this week

• "Years ago, I wrote a number of posts on how to survive a Trust & Safety layoff. I've refreshed the content and placed it in a Notion doc + I had Claude make a visually-friendly edition to aid those going through layoffs now. Im calling this compendium of content "T&S Resilience."" - Hopefully you never need Michael R. Swenson's new resource but bookmark it nonetheless.
• "This is some of the best work I've ever done. Please spend 20 minutes educating yourself about the #sextortion crisis, the #yahooboys, and how there are true opportunities to disrupt." - Former public prosecutor Erin West has a 20-minute document out with Paul Raffile and it looks like a must-watch.
• "Many have deep experience in Fraud Ops, Trust&Safety, and risk investigations. If your organization is hiring for roles for remote, New York, California, or Omaha, I’d be grateful if you could send them my way" - I respect Patreon's Rick Hiltbrunner for looking out for his fraud ops team. Reach out if you're recruiting.