X Takes Action Against AI-Generated Explicit Content Following Government Directive

Government Intervention on Content Moderation

The social media platform X has acknowledged its shortcomings in managing content moderation. Following directives from the Union government, it has blocked approximately 3,500 posts and removed over 600 accounts associated with sexually explicit material generated by its AI chatbot, Grok.

In response to the government's concerns, X assured compliance with Indian regulations and stated that it would prohibit explicit content on its platform.

Recently, the platform restricted Grok’s image-generation capabilities to paid users, which means that the identities and payment details of those utilizing this feature will be recorded.

Previously, Grok enabled users to alter images of real individuals, primarily women, by undressing them and sexualizing their photos without consent, reportedly generating thousands of such images each hour.

Despite the limitations imposed on X, the standalone Grok application continued to allow users to create explicit images without a subscription.

On January 2, the Union Ministry of Electronics and Information Technology instructed X to conduct a thorough review of Grok to prevent the generation of nudity or sexually explicit content.

The government cautioned that failure to comply with these directives could result in legal action against the platform's officials under various laws.

In a communication to X’s chief compliance officer in India, the government highlighted that users were exploiting Grok to create fake accounts for the purpose of generating and disseminating obscene images and videos of women.

The ministry emphasized that this misuse extends beyond fake accounts, targeting women who share their images or videos through prompts and synthetic outputs.

Publishing or hosting obscene content, especially through AI tools, violates privacy rights and can lead to severe legal repercussions, according to the government.

The Centre also mandated X to provide a report within 72 hours detailing the actions taken to manage the Grok application, oversight by the compliance officer, measures against inappropriate content, and compliance with reporting obligations.

In light of the backlash, X's owner, Elon Musk, stated that anyone using Grok to create illegal content would face the same consequences as those who upload illegal material.