"A multifront attack on speech", TikTok's turn to update guidelines and bots behave badly

Hello and welcome to Everything in Moderation's Week in Review, your need-to-know news and analysis about platform policy, content moderation and internet regulation. It's written by me, Ben Whitelaw and supported by members like you.

Today’s edition has big “back to school” energy: politicians and regulators are busy writing the syllabus, platforms are frantically tidying up their rulebooks, and yes — there’s even required reading in the People section.

Over on Ctrl Alt Speech, Mike and I get quizzed by curious listeners — and fall foul of some dubious reviews. Have a listen.

A warm welcome to new subscribers from UCLA, NYU, Accenture, Bumble, Meta, the eSafety Commission, TELUS Digital and beyond. If this is your first edition, hit reply and let me know what you think. And if you’re interested in sponsoring the newsletter, we should talk.

Let's get into it — BW

Policies

New and emerging internet policy and online speech regulation

JD Vance has been holidaying in the UK (in spite of his past views on my fair nation); Senator Jim Jordan has been leading a delegation in Dublin (although no mention of a visit to the Guinness Factory); And there's is no still conclusion to the Digital Services Act investigation into X/Twitter (December 2023, since you asked). 

Taken together, these developments led Atlantic Council fellow Kenneth Propp to claim that the "transatlantic dispute over free speech seems bound to escalate". His analysis, published last week, is a helpful read-through of what has happened since Donald Trump’s inauguration and what has been called a "multifront attack on Europe's approach to speech".

Tech Policy Press also published a rundown of the tactics being used by the US to squeeze their diplomatic counterparts across the pond. It’s going to be a very interesting —as US EiM readers might say — ‘fall’.

Also in this section...

• The Open Internet Is Closed for Business (Slate)
• Brazil’s Digital Sovereignty Is Under Attack: How Courts, Platforms, and Constitutional Law Are Redefining Democracy Online (Just Security)

Where are tomorrow’s T&S experts coming from?

As AI transforms how Trust & Safety work gets done, it’s also quietly dismantling the career paths that got many of us here. This week, a T&S Insider reader asks what to do when all the entry-level roles disappear.

Everything in Moderation*Alice Hunsberger

Products

Features, functionality and technology shaping online speech

The speed with which chatbots have become new surfaces for online harm is startling. I've picked out three related stories show their rise from bad customer service tools to intimate companions:

• Reuters revealed that Meta’s internal guidelines for its AI chatbots allowed disturbing behaviour, including romantic roleplay with children, generating racist arguments, and producing false medical information. The 200-page policy, signed off by senior staff, also contained bizarre workarounds such as offering users a photo of Taylor Swift holding a giant fish. Meta confirmed the document’s authenticity, said some sections had since been removed following Reuters’ questions.
• CharacterAI’s recently installed CEO told the Financial Times that he expects more people to use bots as a “test bed for making the real-life relationships a lot deeper, a lot more useful and a lot healthier.” However, the platform still does not permit sexually explicit conversations despite a rise in users seeking 18+ conversations.
• Rest of World revealed that OnlyFans “chatters” — outsourced workers who respond to fan messages on behalf of adult stars — may soon be replaced by bots trained on their conversations. I’m sure nothing can go wrong there…

Platforms

Social networks and the application of content guidelines

There must be something in the water. Last week, it was Bluesky updating its community guidelines (EiM #300). This week, it’s TikTok’s turn. Mashable has a rundown of the changes and I was most interested in the language shift in the newly clarified misinformation section; it now states that unverified information about “crises and major civic events” — rather than just “emergencies” — will not make it to the For You page. I don’t envy the new AI systems (EiM #299 and others) — or their human trainers — tasked with deciding what counts as a crisis, for whom, and when.

Talking of TikTok, The Verge has produced an interesting long read that uses the Gaza conflict as a lens to examine the on-off US efforts to ban TikTok. It raises a question we’ll hear more of in months to come: is TikTok an agent of “information-nationalism”? The answer, as the piece makes clear, is that no-one really knows.

Also in this section...

• African courts may pave the way for holding social media giants to account (Al Jazeera)
• Investigation into 'horrifying' death of French streamer (BBC)

You Asked, We Answered - Ctrl-Alt-Speech

This is the second of our special episodes for the month of August. Last time, we talked a bit about the past and future of Ctrl-Alt-Speech; this time, Mike and Ben answer some questions from our listeners, and also share a few of the latest revie…

Buzzsprout

People

Those impacting the future of online safety and moderation

If a fictional portrayal of life as a moderator (EiM #298) isn’t for you, maybe Tamar Mitt’s book is more your bag. 

The Columbia University professor has just published Safe Havens for Hate, a book that challenges the old “whack-a-mole” view of deplatforming. Drawing on data from 114 extremist groups across 67 platforms — plus case studies of QAnon, the Taliban and others — she shows how extremists make calculated moves to spaces that balance low moderation with wide reach.

Her research also highlights how uneven regulation pushes harmful actors into smaller, less-policed corners of the internet, and how groups adapt their messaging to fit the rules of each platform. Lawfare’s review paints the book as a realistic picture of how online extremism evolves and, although not without gaps, a valuable read.

Posts of note

Handpicked posts that caught my eye this week

• "Instead of shrinking, the network has expanded—using AI slop, deceptive “static video” and hacked accounts to exploit algorithms for profit and influence." - Indicator Media's Craig Silverman on the multi-headed Meta hydra that seemingly cannot be tamed.
• "The findings show over 1.5 million voters improved their ability to spot manipulation tactics like scapegoating, discrediting, and decontextualization." - Moonshot working with Google Jigsaw publish new findings from the "largest prebunking initiative to date".
• "We've got a good number of submissions already, make sure to share your extended abstract (800-1000 words) by September if you'd like to join us for this free online conference on platform governance December 1-2." - Rasmus Kleis Nielsen from the University of Copenhagen (respectfully) wants your ideas.