The 'cost' of slow moderation, #SkinnyTok finally banned and Musk gets check mated

Hello and welcome to Everything in Moderation's Week in Review, your need-to-know news and analysis about platform policy, content moderation and internet regulation. It's written by me, Ben Whitelaw and supported by members like you.

Week in Review is back after a two-week hiatus while I became a dad. My brilliant wife is doing well, and so is our little boy. I have a lot of thoughts about the intensity of the last few weeks but I'll save them for a more suitable medium — and when I've had a bit more sleep. For now, I'll just say thanks to everyone who messaged with well wishes and who subscribed to EiM in the meantime.

T&S Insider, written by the brilliant Alice Hunsberger, continued in my absence and I urge you to have a read her thoughts on moderator wellbeing and what AI is doing to the T&S job market. Likewise, Mike has had two stellar guests on Ctrl-Alt-Speech while I've been learning to change nappies/diapers. Have a listen.

I'm not yet fully back up to speed but here's my best, bleary-eyed attempt at summing up everything in moderation from the last seven days — BW

Policies

New and emerging internet policy and online speech regulation

We start with a story from last week now but a notable one: the Digital Services Act has launched a formal investigation into Pornhub, Stripchat, XNXX and XVideos for tailing to prevent underage access to explicit content. It comes months after Pornhub and Stripchat sought to challenge their status as a Very Large Online Platform (although I can’t find the outcome of that appeal?).

Easy target?: Due to broad public support for their regulation, porn sites have become a focus for nascent regulatory regimes seeking to build momentum. Earlier this year, Ofcom announced it had opened an investigation into two sites that failed to lay out age assurance plans and on Wednesday, new French rules led to Pornhub suspending access to users in its second largest market.

With the US State Department announcing visa restrictions for “foreign nationals who are responsible for censorship of protected expression in the United States”, I wonder how long it will be before an Online Safety Act-supporting MP or an elected official with pro-Digital Services Act views is prevented from entering the US? Politico suggested that Marco Rubio had one particular Brazilian in mind when he came up with the new rules. Either way, it’s yet another reason for non US citizens to not travel stateside for the foreseeable.  

Also in this section...

• A new Texas law mandates age checks on phones. It may be just the start (Washington Post)
• Why the UN’s Internet Governance Forum still matters twenty years on (Atlantic Council)
• Internet Fragmentation’s Outward Turn (Sciences Po)

Products

Features, functionality and technology shaping online speech

Elon Musk’s X/Twitter is attempting to appease EU regulators by adding disclaimers to its blue check marks, clarifying that the badges signify payment rather identity verification. The EU has had the blue check in its sights since bringing an investigation against the platform in December 2023; penalties are suspected to be announced in “the summer”, which might explain why this has happened now. 

Also in this section...

• ‘Godfather’ of AI Yoshua Bengio says latest models lie to users (The Financial Times) 

Platforms

Social networks and the application of content guidelines

TikTok has finally banned the #SkinnyTok hashtag, meaning that users are blocked from seeing search results. It's been a long time coming but, interestingly, there’s some disagreement about who is responsible for the safety win: Euractiv reported that EU officials were not part of the delegation to Dublin at which the ban was agreed last month and that French representatives drove the discussion. However, Irish broadcaster RTE said EU Commissioner Michael McGrath held an online meeting with Shou Chew, TikTok’s CEO, in which it was part of the agenda. 

A new report from OpenAI has shared details about how China and other countries are employing a “growing range of covert operations”, including social engineering, task scams and fashioned comment spanning. Few tactics got significant engagement, the report notes, but it continues a trend seen in OpenAI's February report, in which Chinese users used ChatGPT to write and translate op-eds that were critical of the US administration and published in reputable Latin American newspapers. Should we read too much into the Sino-focus of Sam Altman’s safety reports at a time when massive trade wars are the talk of Washington? Who knows. 

Also in this section...

• Match Group Expands Global Online Dating Safety Efforts with Brazilian NGO Partnership (Dating News)
• More than half of top 100 mental health TikToks contain misinformation, study finds (The Guardian)

People

Those impacting the future of online safety and moderation

Content moderation metrics usually focus on actions taken; for example, posts removed, reports submitted, decisions reversed. But Laura Edelson, assistant professor of computer sciences at Northwestern University, and six other authors have come up with another. 

“Prevented dissemination”, as the name suggests, attempts to measure the impact of a moderation decision. In effect, it looks at the expected engagement of a post without moderation intervention and the reduced engagement having been downranked or removed and calculates the delta (the researchers dataset did not include reach but they believe the methodology works just the same).

The research concludes that most removals happen after the posts have accrued the majority of its 48 hour engagement (which is not hugely surprising) and that takedowns prevent around 25-30% engagement in that period (which feels new). That’s deemed to be not very much by the researchers but feels like a significant number to me. Imagine if a third of people were prevented from seeing hate speech?

The solution, according to Edelman, is to bring moderation and dissemination systems closer together in terms of the speed they operate. Either “moderation systems can operate faster, or algorithmic feeds can slow down”. Both have trade offs but it’s an interesting idea.