Sam Altman is speedrunning the Content Moderation Learning Curve
I'm Alice Hunsberger. Trust & Safety Insider is my weekly rundown on the topics, industry trends and workplace strategies that Trust & Safety professionals need to know about to do their job.
I'm finally back home after what seems like endless weeks of travel! In today's edition, I write about lessons that Sam Altman can learn from Trust & Safety professionals. Plus, a brief section on climate change disinformation.
Get in touch if you'd like your questions answered or just want to share your feedback. Here we go!
— Alice
P.S. You'll find me next (virtually) at Safer eCommerce Day on November 5th, where I'll be talking about Human-in-the-Loop systems and what skills fraud and T&S professionals need in the age of AI. Hope to see you there!
In 2005, the biggest concerns for online platforms looked very different from today. Over time, regulations have sharpened, expectations have risen, and what once sat in a gray area is now a clear threat.
Our second blog in the “20 Years in Online Safety” series reflects how definitions of harm and responsibility have evolved, and how those changes have shaped our own journey at Resolver.
From reactive moderation to proactive protection, each milestone has pushed us to rethink not just what safety means, but how it’s achieved.
It’s a look back at how far we’ve come, and a reminder of how much further our industry still has to go.
Elon did it and now Sam is doing it too
In 2022, Techdirt's Mike Masnick (also of Ctrl-Alt-Speech fame) wrote, "Hey Elon: Let Me Help You Speed Run The Content Moderation Learning Curve".
In it, he mapped how every platform stumbles through the same stages of discovery — from "We're the free speech platform!" to "We're just a freaking website. Can't you people behave?".
Along the way, platforms will often encounter — in no particular order — unique child safety needs, copyright requirements, the difficult balance between privacy and safety, a deluge of spam, FBI reporting obligations, international legal conflicts, and the reality that different users' needs are often incompatible.
Once you've come face-to-face with all that, you typically end up with — in no particular order — a set of complex policies, dedicated T&S teams, age-gating technology, some form of rate limiting, and the exhausting realisation that "humanity is messy."
Learning the hard way
As has been reported recently, Sam Altman, CEO of OpenAI, is now speedrunning the same curve. By my count, he’s at Level Thirteen — based on his recent tweets:
As we have said earlier, we are making a decision to prioritize safety over privacy and freedom for teenagers. And we are not loosening any policies related to mental health. This is a new and powerful technology, and we believe minors need significant protection.
We also care very much about the principle of treating adult users like adults. As AI becomes more important in people's lives, allowing a lot of freedom for people to use AI in the ways that they want is an important part of our mission.
It doesn't apply across the board of course: for example, we will still not allow things that cause harm to others, and we will treat users who are having mental health crises very different from users who are not. Without being paternalistic we will attempt to help users achieve their long-term goals.
But we are not the elected moral police of the world. In the same way that society differentiates other appropriate boundaries (R-rated movies, for example) we want to do a similar thing here.”
This is textbook early-stage T&S:
- Age-gating teenagers (some users need different protections)
- Mental health crisis differentiation (context-dependent handling)
- "We're not the moral police but we have boundaries"
- Balancing youth safety against adult freedom (the eternal platform tension)
- Awkward comparison to existing frameworks that don’t quite fit
He's articulating policies every social platform eventually realises they need, in roughly the same order, for the same reasons. It's exactly what the speedrun is all about. And that before we really get into the speedrun for Sora.
It begs the question: what would it look like if AI platforms didn't fall into the "content moderation speedrun" before making changes?
Old lessons, new systems
The Economist recently argued that AI systems should be “overbuilt,” likening them to Victorian bridges that were constructed with redundancies to guard against material failure. The logic goes that since we can’t always predict how AI models will behave, we should design with generous safety margins.
This problem that I have with this metaphor is that doesn’t quite fit the reality of online safety. Platforms — including ChatGPT and Sora — aren’t static systems but constantly evolving adversarial environments. The challenge isn’t just ensuring the bridge holds — it’s making sure that people crossing it aren't also trying to blow it up.
Still, the focus on how systems are built is useful. It echoes the growing emphasis within Trust & Safety on Safety by Design; that is, building structural safeguards and ethical constraints into systems from the start, rather than patching them after failure. We know harm can’t be eliminated entirely, but neither can it be ignored until catastrophe strikes.
Designing safety defaults
In fairness to OpenAI, the company already does a lot in this area: red teaming (which I've been part of), safety guardrails, and its own dedicated T&S teams. And AI safety does have genuinely novel challenges, including models demonstrating strategic behavior during evaluation, that traditional T&S systems doesn't often have to solve for.
However, that doesn't stop me thinking that Safety by Design principles have a greater role to play. Imagine what kind of products ChatGPT and Sora would be if they were grounded in the following ideals:
- Build systems that promote safety and well-being proactively.
- Reduce and prevent harm, where and how you can, knowing it will definitely happen.
- Remediate harm by providing thoughtful recourse when it occurs.
- Start broad with safety, then get narrow. Default to systems that promote privacy and safety first.
- Think about user experience first and speed to feature deployment second.
- Decide on your values at the beginning.
- And most importantly: center the most vulnerable and marginalised in every decision.
Every platform that launched without full safety features eventually had to retrofit them — usually after a crisis that damaged user trust, attracted regulatory attention, or created legal liability. And being reactive and deploying features piecemeal (after they're already urgently needed!) means that platforms look underprepared and uncaring.
Those of us who worked in T&S in the early days learned this the hard way. I hope Sam Altman does too.
You ask, I answer
Send me your questions — or things you need help to think through — and I'll answer them in an upcoming edition of T&S Insider, only with Everything in Moderation*
Get in touchHitting up Harvard

Last week, I participated in a really great collaborative seminar at the Harvard Radcliffe Institute last week on Online Civic Trust in Climate Change Science, with 14 academics, AI, and T&S experts concerned about the same issues.
It's a tricky topic, as climate change disinformation isn't as immediately urgent and harmful as other online harms. And yet, climate change is an existential risk that we all should be doing our best to combat.
I came away with a lot of insights that I hope to share more of soon but I thought I'd share a couple links now that can be immediately helpful:
- Platform's Policies on Climate Change Disinformation (EU Disinfo Lab) - a helpful look at where the major platforms currently are on their policies, and where room for improvement is.
- What makes us (unwittingly) share climate misinformation online? (Harvard School of Public Health) - a really great summary of why and how disinfo posts go viral
Also worth reading
The Top Challenges of Using LLMs for Content Moderation (and How to Overcome Them) (Musubi, written by me!)
Why? I wrote a giant resource on challenges with using LLMs for content moderation– not because I don't believe in it– but because designing systems for T&S is often harder than it seems. I've spent the last 6 months thinking about and working on these systems and want to share what I've learned.
The Future of Being Trans on the Internet (The Verge)
Why? "Trans rights and the internet itself are in a moment of crisis. What happens next?" A thoughtful, comprehensive series of articles coming at just the right time.
Scarcity Mindset in Trust & Safety (Bri Riggio)
Why? An amazing resource for every T&S person who ever felt like they were doing too much, too urgently, with not enough resources (maybe all of us?)
Member discussion