The social media network X has sparked widespread outrage after people used its AI chatbot Grok to manipulate photos of women by taking off their attire.
Multiple instances have emerged where the tool has been used to remove clothing from female subjects to make them appear in swimwear non-consensually, as well as to place them in sexually explicit situations.
Company Response and Legal Position
XAI, the company responsible for Grok, offered no comment to questions, aside from an auto-reply response stating "legacy media lies".
A Home Office spokesperson stated that new laws is being introduced to ban image stripping tools. Under a upcoming criminal offence, anyone who supplied such tools would "face a jail term and heavy fines".
"The platform has the ability to stop these types of misuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said technology companies must "assess the potential" of people viewing illegal content on their networks, but stopped short of confirming whether it was at this time investigating X or Grok in concerning AI-generated images.
Grok is a no-cost chatbot—with some premium functions—which reacts to X users' queries when they tag it in a tweet.
It is commonly used to give reactions or further background to other posters' remarks, but people on X are also able to alter an posted photo through its artificial intelligence picture modification feature.
History of Criticism and Rules
The chatbot has been condemned for enabling users to generate photos and clips with nudity and sexually suggestive material, and it was earlier alleged to have been involved in creating inappropriate material of celebrities.
"X has been allowing the production and spread of these images for months without taking any measures and we have not witnessed any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "showing likenesses of persons in a pornographic way".
In a comment, Ofcom said it was unlawful to "create or circulate non-consensual intimate images or illegal content" and clarified this covered sexual deepfakes made with AI.
It said networks such as X were required to take "necessary actions" to "lessen the danger" of users encountering unlawful material on their platforms and to take down it promptly when they are notified of it.
