India Orders X to Audit and Stop Morphed AI Images After Grok Chatbot Generates Obscene Pictures

NEW DELHI — India’s Ministry of Electronics and Information Technology (MeitY) has issued a stern directive to social media platform X, demanding a comprehensive audit and immediate corrective action to stop its AI chatbot Grok from generating and circulating morphed, obscene and sexually explicit images of women and children. The government has given the platform 72 hours to remove violating content and submit a detailed action-taken report. 

According to the notice, MeitY observed that users are misusing Grok’s AI capabilities to manipulate images — often without consent — to create sexually suggestive outputs that demean and sexualise women and even minors. These deepfake-style visuals have been posted and shared widely on X, sparking public outrage and serious legal concerns. 

Government Warns of Legal Consequences

The ministry’s letter highlighted a “serious failure of platform-level safeguards and enforcement mechanisms,” saying that the platform’s controls are insufficient to prevent unlawful uses of its AI features. It noted that such misuse could violate multiple Indian laws, including the Information Technology Act, 2000, the IT (Intermediary Guidelines and Digital Media Ethics Code) Rules, 2021, and statutes relating to decency, child protection and the dignity of women and children. 

X has been asked not only to remove all obscene and unlawful content generated via Grok but also to review and strengthen its technical, procedural and governance-level guardrails that govern how the AI processes prompts and handles image outputs. The ministry also wants details on how X’s compliance team — including its Chief Compliance Officer — is overseeing safety on the platform.

Triggered by Lawmakers’ Concerns

The government action follows a complaint from Rajya Sabha MP Priyanka Chaturvedi, who wrote to India’s IT Minister urging intervention after users reportedly used fake and real accounts to upload women’s photos and prompt Grok to generate sexualised edits, including minimised clothing. Chaturvedi described the trend as “unacceptable and gross misuse” of AI technology.

Balancing AI Innovation and Safety

The controversy reflects growing global concerns over the unregulated use of generative AI tools. While AI-based features have unlocked creative possibilities, the lack of strong safety and consent measures has opened the door to abuse, deepfakes and non-consensual content. Similar issues with Grok have drawn criticism in other countries as well, with reports of inappropriate and harmful outputs prompting calls for stricter oversight. 

If X fails to comply with the ministry’s orders within the stipulated timeframe, it may risk losing its safe harbour protections under Indian law — a legal shield that typically protects platforms from liability for user-generated content.