Regulators and lawmakers across multiple countries are intensifying scrutiny of Elon Musk’s social media platform X after its integrated AI chatbot Grok generated sexualized deepfake images of women and girls in response to user prompts, according to official sources and government statements. The issue has triggered investigations and legal warnings because the content allegedly depicts individuals without consent, including minors, at a scale experts describe as unprecedented for a consumer AI system.
Investigations and testimonials on X itself indicate that users were able to request sexualized AI imagery by prompting the system to digitally alter or remove clothing from photographs.
Reported victims include private individuals, public figures, and children, with some images derived directly from photos uploaded by users. Estimates cited by researchers suggest the system produced explicit content at a rate approaching one image per minute, raising serious concerns about safeguards and moderation design.
The backlash comes as governments worldwide sharpen oversight of generative AI, particularly its misuse in creating nonconsensual intimate images and child sexual abuse material. In the United States, the Take It Down Act, enacted in 2025, established removal obligations for platforms hosting such content, but legal experts note unresolved questions when harmful material is generated directly by AI rather than uploaded by users. India has formally escalated pressure on X, with the Ministry of Electronics and Information Technology issuing a 72-hour ultimatum demanding an action-taken report detailing technical safeguards to stop Grok from generating obscene, sexually explicit, or pedophilic content. The order warned that non-compliance could jeopardize X’s safe harbor protections. In Europe, French authorities confirmed a Paris prosecutor-led investigation after multiple ministers flagged illegal sexual deepfakes on X, citing potential violations of the EU’s Digital Services Act. Malaysia’s communications regulator has also opened an inquiry, warning that AI-generated indecent images involving women or minors constitute criminal offenses under local law.
Authorities including the UK’s Ofcom and the European Commission have warned that the content may violate national laws and the EU’s Digital Services Act. Australia’s eSafety Commissioner has also confirmed investigations, underscoring concerns that existing enforcement mechanisms have failed to prevent large-scale harm.
As a response, Elon Musk said on X anyone prompting the chatbot to generate unlawful material would face the same consequences as uploading it directly. X also did not refute the existence of the problematic content created via Grok. However, it also proclaims in one of its Safety posts, that it removes illegal content, such as content about child sexual abuse, by permanently suspending accounts and collaborates with local governments and law enforcement as needed.
In a funny twist of events, independent TechJuice research found that Grok is perfectly capable of doing the opposite as well. Here is an image for reference of Elon Musk in a Muslim hijab:
As scrutiny intensifies, policymakers are expected to focus on platform liability, AI accountability, and whether existing legal protections shield companies when automated systems produce illegal content. The response from X, including whether it introduces meaningful technical safeguards and transparent moderation reforms, will shape the next phase of global AI regulation.
