Elon Musk’s Grok was responsible for 87 percent of the synthetic files researchers could trace to a specific generation tool across 821 documented deepfake attacks in the first half of 2026, according to a new threat report from cybersecurity firm Resemble AI, published August 12. That’s near-total dominance of a category defined almost entirely by harm. Roughly 3.46 million synthetic images, videos, and audio files, closing in on the total volume the entire internet produced across all of 2025.
The scale of the underlying event driving that number is staggering on its own. Researchers at the Center for Countering Digital Hate documented Grok generating approximately 3 million sexualized images during an 11-day window spanning late December 2025 into early January 2026, a rate exceeding 6,000 images an hour, including roughly 23,000 that appeared to depict children. That wasn’t a gap in Grok’s safeguards. It happened while xAI’s content moderation was actively running, which means those numbers describe the filters working as designed, not failing unexpectedly.
The legal fallout has been immediate and global. Regulators in the UK, EU, and Ireland opened formal investigations within weeks. California’s attorney general launched his own probe. Indonesia, Malaysia, and the Philippines temporarily blocked Grok entirely. Civil suits followed fast: Ashley St. Clair sued in New York over deepfakes allegedly generated from photos taken when she was 14. A California class action, now up to five plaintiffs, alleges the tool produced child sexual abuse material from family photos. One case involves a stepfather who generated roughly 7,000 images of a child from a single source picture. UK Labour MP Jess Asato filed a High Court claim in June alleging Grok produced nonconsensual images depicting her being sexually assaulted. Resemble AI estimates potential civil liability at up to $2.24 billion, against verified direct losses of just $6.95 million, a gap that tells you most of the damage here isn’t financial. It’s reputational, psychological, and impossible to fully undo.
Here’s the part that should worry anyone hoping this gets fixed with a better filter: according to an internal xAI analysis cited by The Information, and independently confirmed by Canada’s Privacy Commissioner, there may not be one. A generative model capable of producing explicit adult content has, by definition, learned the statistical relationships between imagery and sexual descriptions. That same learned capability can produce CSAM the moment a prompt shifts who’s being described. Post-generation filters can reduce how often that happens. They can’t eliminate it without dismantling the adult-content generation the filters exist to selectively allow. xAI’s remediation efforts reportedly cut violation rates roughly in half, which, at a platform generating over 10 billion images monthly, still leaves an enormous absolute number of harmful outputs slipping through. Halving a problem at that scale isn’t a safeguard. It’s a rounding error with victims attached.
But there’s more to it than that. Since SpaceX acquired xAI earlier this year, Musk has told employees AI revenue could surpass every other SpaceX business line by September, and the company’s IPO prospectus set aside $530 million specifically to cover litigation tied to Grok’s image generation. The tool responsible for the majority of the world’s documented deepfake harm is simultaneously the centerpiece of one of the most closely watched growth stories in tech. Minnesota’s nudification ban took effect August 1 after a federal judge denied xAI’s stay request, with a preliminary injunction hearing set for August 19. It will be the first real court test of whether strict liability for this kind of harm can survive a First Amendment challenge.
What makes this different from a typical tech-safety story is the target. This isn’t a specialized underground tool requiring technical know-how. It’s a mass-market chatbot available to anyone with an internet connection, embedded in a platform reaching billions of people. Every creator who’s ever posted a photo, every teenager with a school picture online, every public figure with a searchable face is now sitting within a threat model that used to require real effort to weaponize but now requires only a prompt. Detection tools are improving, including Resemble AI’s own new DETECT-World model, which claims meaningful accuracy against generators it’s never seen, but detection chasing generation is always going to be a step behind something producing millions of images an hour.
When the company profiting most from a tool also can’t engineer a fix for its worst output, and the fastest-growing part of its business is the same one generating the harm, you have to wonder who exactly is supposed to slow this down before the law catches up?