Even with the best intentions, filtering toxic language becomes an ethical choice about what you're willing to expose yourself to.
The stakes
An Anti-Defamation League study found that 81% of U.S. adults aged 18 to 45 who played online multiplayer games had experienced harassment, and the industry had no real technical answer for it. Bleep set out to test whether AI could give players individual control over the harmful language they encountered while gaming, rather than leaving moderation to one platform-wide rule.
The solution
The approach paired Intel's AI models with Spirit AI's detection technology and Windows audio architecture, then handed the filtering threshold to the player instead of enforcing one standard for everyone. That decision shaped the entire testing process, since proving the app worked meant more than proving the AI could detect toxic speech accurately. It meant finding out whether people actually trusted a system that let them personally decide where the line sat.
My role
I joined Bleep's design and research team through Fresh Consulting, working alongside Intel and Spirit AI during the Alpha phase. My focus was the user testing program, where I designed and ran user interviews, monitored usability testing sessions, and analyzed the findings. Those insights fed directly into product decisions for both teams.
What happened next
Bleep debuted as a prototype at GDC 2019 and got its first public UI preview at Intel's GDC 2021 Showcase before moving into Open Beta. Players controlled their exposure through a sliding scale, choosing how much of categories like misogyny, racism, and harassment they wanted to hear rather than an on or off switch. Intel described it as a step forward, while acknowledging it wouldn't solve the underlying problem.
How it landed
That same slider became the center of the criticism it drew, and the app didn't reach the adoption Intel had hoped for.
Key takeaway
Handing users the threshold was the product's most interesting idea and its biggest liability. A slider that lets someone choose how much racism they hear also makes them responsible for that choice, which is a moral position disguised as a preference setting. The tension between an ambitious goal and where the line actually sits was the whole story of the project, and it is the part I still think about.
The team
UX Research & UX Design | Sojin Rank |
Design Director | |
Engineering |