'The moralization of machines', biometric checks and Board gets boost
Hello and welcome to Everything in Moderation's Week in Review, your need-to-know news and analysis about platform policy, content moderation and internet regulation. It's written by me, Ben Whitelaw and supported by paid members like you.
It’s not every week that you get global religious figures turning their attention to topics that EiM covers week in, week out. But that was the case this week as Pope Leo XIV’s inaugural enycylical placed online safety, internet regulation and AI governance on the global agenda. (see: People section).
It was there that Mike and I spent a lot of time on this week’s Ctrl-Alt-Speech, where I was back in the co-hosting chair after three weeks off. Become a Patreon supporter to get instant access and extended analysis or get the free episode later today wherever you listen.
Good news for anyone who’s been considering becoming a paid EiM member: You can now download an invoice/receipt directly via your Account on the EiM website. If you have a personal development budget or scope to expense EiM, now’s a good chance to do so.
Here's everything in moderation from the last seven days — BW
Following our recent webinar with Ofcom on Year 2 risk assessments, Resolver is hosting the next session in our Regulatory Readiness series on June 18th.
"Paper compliance" is giving way to active supervision. Regulators across the UK and Europe are now calling out easily bypassed systems and demanding "highly effective" child safety controls.
Join senior leaders from Ofcom, Coimisiún na Meán, the ICO and Internet Matters to unpack real-world enforcement, move past the safety vs. privacy binary, and explore technical insights from market red-teaming. Designed for GCs and Trust & Safety leaders.
Policies
New and emerging internet policy and online speech regulation
After three months that have felt like double that, the UK’s under-16 social media ban consultation closed on Tuesday with more than 80,000 responses. UK prime minister Sir Keir Starmer took the chance to meet with various groups calling for change, including parents of children who reportedly died as a result of social media; the Smartphone Free Childhood campaign group — which reportedly mobilised a significant numbers of the 42,000+ parent who filled in the consultation — and campaigners of all stripes.
What happens now: Starmer reportedly told parents that the government would be announced in “weeks, not months” but the bigger question is what will that look like. The Guardian asked experts what they believe should happen next and, unsurprisingly, everyone has different ideas. Notably, one expert, who was part of the Australia age assurance trial last year, said: “[A ban] is not the golden recipe, this is not the template you should follow”. Hmmm.
Also in this section...
- Malaysia issues statutory demand to TikTok over alleged failure to moderate 'offensive' content on royals (Reuters)
- Ofcom fines porn company £600,000 (Ofcom)
Products
Features, functionality and technology shaping online speech
Another week, another app rolling out age assurance technology with flaky justification. This week it was online therapy platform Headway in an effort to remain a “safe and reliable place to get care”. 404 Media reported that the company will use biometric scanning from Persona — also utilised by Uber, Doordash and Roblox — as part of an effort to stem what it calls “real documented fraud”.
Think of the users: Unlike other age verification rollouts, regulation doesn’t seem to be the primary driver here. Nonetheless, Headway users are being asked to hand over a bunch of personal data on the basis that it is stored securely or face the prospect of finding a new provider, navigating insurance requirements and rebuilding a therapeutic relationship elsewhere. The question is whether they or indeed prescribers or therapists will kick off in the same way that Discord users did about the integration of Persona in February this year.
Alice Hunsberger
Also in this section...
💡 Become an individual member and get access to the whole EiM archive, including the full back catalogue of Alice Hunsberger's T&S Insider.
💸 Send a tip whenever you particularly enjoyed an edition or shared a link you read in EiM with a colleague or friend.
📎 Urge your employer to take out organisational access so your whole team can benefit from ongoing access to all parts of EiM!
Platforms
Social networks and the application of content guidelines
Spotify seems to be continuing to figure out its position on AI-generated music with co-CEO Alex Norström this week defending new subscriber functionality that allows its users to remix “one song to become 10,000 songs” (is it just me or does that sound quite awful?).
Music to who's ears?: The justification is interesting and perhaps counterintuitive: Norström says that “legal” and “controlled” AI-enabled creation is better than AI slop and that any issues will be mitgated by its recommendation systems and verification tools, some of which were launched in September last year (EiM #307) in an effort to placate concerned artists. Not sure this latest announcement will help.
The Oversight Board — the independent-but-Meta-funded group of digital rights experts tasked with reviewing the platform’s most consequentual moderation decisions — has received a boost after news of “top-up” funding to the tune of $13m and a renewed commitment from Meta to co-operate with the board. The board is one of the genuinely innovative online speech innovations of the last decade but, as I mention on this week’s Ctrl-Alt-Speech Plus episode, I’m worried that this announcement isn’t the good news it should be.
One story that caught my eye this week came from Thailand, where authorities arrested six Nigerian nationals accused of operating a romance scam network after overstaying their visas. We hear plenty about scam compounds in Southeast Asia and plenty about online fraud originating from West Africa. It's comparatively rare to see reporting that directly connects the two. This case suggests the global fraud ecosystem may be more interconnected than many of us assume.
Also in this section...
- Exclusive: OpenAI readies cyber, misinformation defenses ahead of elections (Axios)
- X is cracking down on content thieves (Business Insider)
- We analysed the TikTok history of 142 men. Here’s what it taught us about the manosphere (The Conversation)
- The Big Ban Theory: A Pre- and Post-Intervention Dataset of Online Content Moderation Actions (ARXIV)
People
Those impacting the future of online safety and moderation
I’ve been fascinated this week by the commentary around Pope Leo XIV’s encyclical, the 42,000 word open letter that — among many things — explores the moral argument for responsible AI and digital regulation.
Mike and I discuss different aspects of Magnifica Humanitas on this week’s Ctrl-Alt Speech, including his pushback against AI alignment with human values — a "moralization of machines" which says does not go far enough — and its “disarming” to allow the slow cogs governance and ethics to catch up. Hit reply and let me know what you thought about his framing and our assessment.
There were other elements of the story that caught my eye, too. The apparent close ties and straight-up lobbying by the tech industry, as reported by Politico. The appearance of an Anthropic co-founder that wasn't Dario Amodei. The coded Tolkien references that speak heavily to my past life as Literature undergraduate. The predictable reaction of converted Catholic and AI booster, JD Vance.
Will it change the trajectory of AI adoption or humanity's relationship with the technology? Probably not. But it was refreshing to spend a week talking about someone willing to challenge the assumptions of the major AI labs rather than simply swallow them wholesale.
Posts of note (book release edition)
Handpicked posts that caught my eye this week
- "We just received our author copies of 'Platforms: A Critical Introduction' this morning. It is exciting to finally hold the book in print!" - Co-author Anne Helmond on the timely release of her book on platformisation.
- "I am thrilled to share the release of "Trust, Safety, and the Internet We Share: Multistakeholder Insights" (published by Routledge). This is a monumental milestone for our field" - Sujata Mukherjee is rightly excited about a new volume edited by the Trust & Safety Foundation (and which I have a chapter in too!)
- "The paperback publication week of Logging Off is bittersweet for multiple reasons." - Adele Zeynep Walton's important story continues to raise awareness about online harms and what can be done.
Member discussion