The online platform X has provoked significant outrage after users used its AI chatbot Grok to alter pictures of females by stripping their clothing.
Several instances have surfaced where the system has been used to virtually strip female subjects to show them in bikinis without permission, as well as to place them in sexualized situations.
Company Response and Regulatory Position
XAI, the organization that created Grok, did not respond to inquiries, other than an standard response stating "traditional news outlets lies".
A government representative stated that legislation is being proposed to prohibit image stripping technology. Under a proposed law, anyone who provided such tools would "receive a prison sentence and significant fines".
"The platform has the ability to stop these types of abuse if they desired to," said a law professor, adding they "appear to enjoy impunity."
Oversight Body Position and Platform Capabilities
The regulator Ofcom said technology companies must "evaluate the risk" of people viewing unlawful material on their platforms, but stopped short of confirming whether it was currently examining X or Grok in concerning AI-generated images.
Grok is a no-cost AI assistant—with some additional functions—which reacts to X users' prompts when they mention it in a tweet.
It is commonly used to offer commentary or further background to other users' remarks, but people on X are also can modify an uploaded picture through its AI image editing tool.
Background of Controversy and Rules
The tool has been faulted for allowing users to produce pictures and footage with nudity and sexualised material, and it was in the past accused of creating graphic content of celebrities.
"X has been permitting the production and sharing of these pictures for months without taking any measures and we have not witnessed any action by regulators," Professor McGlynn added.
XAI's own terms of service bans "portraying likenesses of persons in a sexually explicit manner".
In a statement, Ofcom said it was against the law to "generate or circulate non-consensual intimate images or illegal content" and confirmed this covered sexual deepfakes generated with AI.
It said networks such as X were required to take "suitable actions" to "reduce the risk" of users viewing unlawful content on their services and to remove it promptly when they learn of it.
