The proliferation of bad ads, Kaley Glenn speaks and EU revives Chat Control 1.0

Hello and welcome to Everything in Moderation's Week in Review, your need-to-know news and analysis about platform policy, content moderation and internet regulation. It's written by me, Ben Whitelaw and supported by paid members like you.

Breaking news as I was putting the final touches to this week’s newsletter: the European Commission has found Meta in breach of the Digital Services Act for the addictive design of Instagram and Facebook. Following on from the EU’s previous ruling against TikTok and the landmarked California trial in March (EiM #330) and it really does feel like we’re really in the addictive design era of platform regulation.

A big welcome to recent subscribers from Spotify, Bending Spoons, Go Fund Me, Mojang, Milltown Partners, Ofcom, Linklaters and elsewhere. Week in Review (every Friday) and T&S Insider (every Monday) are here to keep you up-to-speed and in-the-know about the way that the internet is shifting. But, as I wrote this week, there are a handful of other newsletters that I’d highly recommend for keeping track of what’s going on in online safety, internet regulation and AI governance.

Thanks for reading Week in Review, here's everything in moderation from the past seven days — BW

IN PARTNERSHIP WITH MOONBOUNCE; CREATING Systems that behave exactly as designed — at any scale.

Make your platform safe on your terms.

From traditional trust and safety challenges to emerging risks in AI, Moonbounce helps you define, find and address risks on your platform before they spread.

Sign up for free, and take control of your platform — in as little as 15 minutes.

GET STARTED

Policies

New and emerging internet policy and online speech regulation

In what must be one of the most ding-dong legislative battles in recent EU history, the European Parliament yesterday reintroduced an interim rule that allows platforms to voluntarily scan chats for evidence of child sexual abuse material (CSAM) after the previous version was allowed to expire in April. In a special vote on what’s known as “Chat Control 1.0”, MEPs failed to reach the majority needed to scrap the rules, meaning it was reintroduced by default.

Former MEP and digital rights campaigner Patrick Breyer condemned the move as an undemocratic return to "mass surveillance", while T&S professionals I’ve spoken to say the extension gives them clarity until 2028, when it is hoped a permanent EU framework will be agreed. The Register has the best account of what happened. 

Also in this section...

• YouTube review results in smear campaign remaining online (The Australian) 
• Supreme Court won’t pause app store age checks for Texas teens (Courthouse News)

Products

Features, functionality and technology shaping online speech

This one is less about safety product features and more about the lack thereof: a new AI-powered Instagram feature released in the US this week allows anyone to generate an image using someone else’s photos simply by tagging that person’s username. There is no notification system — so you wouldn’t know if someone had remixed your photos — and users must actively opt out through Instagram’s vast array of settings.

Critics argue the feature could lead to issues similar to those seen during X/Twitter’s botched rollout of Grok, which created thousands of NCII images in a matter of days. Wired has the full, sorry story. 

Weeks after the US government placed restrictions on Anthropic’s Fable 5 model (EiM #342), China has decided that it fancies a bit of the same. Reuters reported that Alibaba and ByteDance have attended meetings with Chinese government officials about locking down both closed and open-source, models, reinforcing the idea that AI models are now matters of national security.   

Also in this section...

• Major Updates to In-Game Reporting Tools (Roblox) 

Platforms

Social networks and the application of content guidelines

Eight months on from the relevations that up to 10% of its annual revenue comes from scam ads, a Meta exec has declared that “it’s not like we knowingly have bad ads” on the platform. Speaking to The Media Leader, EMEA Vice-President Derya Matras also justified the platform’s approach to ads T&S by saying it has spent $30bn fighting fraud, including on “AI systems tgo catch fraudulent ads and scams”. The interview took place at Cannes back in May so the fact that it has been published a few days after the BBC’s investigation into CSAM being promoted via Instagram ads (EiM #342) is spectacular timing.

Try harder: Meta doesn’t seem to be going all out to stop the scourge of scam and fraud ads. Former ads execs have previously gone public about the failure to invest in teams and tech to fight exploitative ads and, despite proudly trumpeting it’s AI capabilities for moderation of organic content and handling user appeals, it doesn’t seem like that those models are being deployed to moderate ads.

In this week’s Ctrl-Alt-Speech, Mike put it down to capitalism and pressure from Wall Street. I don’t buy that — and I was glad to see Ofcom announce a consultation on scam ads just this morning.

Reddit — a platform we don’t always hear much about — has outlined how it is using AI to strengthen automated moderation. In a blog post, the company say it has beefed up its systems to stop suspicious accounts before they post, detect coordinated spam that older tools missed, and cut user exposure to spam by 20%. Not only that but it has retrospectively revoked nearly 2 million fake upvotes each day for the last three months, and reduced enforcement times for hateful and violent content from hours to under five seconds.

Alice wrote recently in T&S Insider about how AI can be used as a shield against bad actors who also wield the technology, and this announcement is a similar signal that T&S is increasingly about the use of AI to fight, you guessed it, AI. 

Also in this section...

• LinkedIn and X Are Flooded With AI Spam, Browsing Data Suggests (404 Media)
• How criminals turn Telegram into a ‘dark mall’ for child sex abuse material (Malaysiakini)
• Wikipedia Is Battling for the Soul of the Internet (New York Times)

People

Those impacting the future of online safety and moderation

It’s a fairly new idea that the design of a platform, rather than just its content, contributes to harm experienced by users. A large part of the reason that idea has taken hold is K.G.M. v. Meta et al., the California case that sent shockwaves through the technology and online-safety sectors (EiM #330).

Three months on, the young woman behind the case has decided to tell her story. Kaley Glenn Mills, now 20 years old, has given an interview to Bloomberg in which she talks about her 10-hour deposition, wanting to run away before it began and feeling as though Mark Zuckerberg was “inconvenienced” when asked to take the stand.

The profile shows a young woman left vulnerable by her parents’ split and whose mother lacked the time or capacity to police her screen use. Even after the trial, she admits that she still can’t stop scrolling. It’s a reminder that smartphones and social media do not affect everyone equally — and that bans are meant to set a protective floor for families that are less equipped to set boundaries.

Google and Meta say they plan to appeal the decision, although no timeline has been given. There will be more social media addiction trials before that happens — and more young people like Kaley Glenn Mills. But she will always be known as the first.

Posts of note

Handpicked posts that caught my eye this week

• "Good for The Atlantic for publishing this given all they have done to aid the phone panic by publishing Haidt's articles without as much of the counter evidence.” - Editor-in-chief of Science Holden Thorp weighs in on the smartphone debate via an interesting long read.
• "Did I mention that we've been writing a book? Fundamentally, security, privacy, trust&safety, AI safety – all the fields aimed at dealing with dealing with how technical systems hurt people and how to avoid it – " LinkedIn's Lea Kissner has a book you just might want to check out.
• "If you think you are covered on the impending social media ban due to having school bans or phone pouches, think again...." Times Educational Supplement's Jon Severs gets into back-to-school mode ahead of what will be a busy September.