Elon Musk's social media platform X has caused widespread backlash after individuals used its AI chatbot Grok to manipulate images of females by stripping their clothing.
Multiple instances have been seen where the tool has been employed to digitally undress women to render them in bikinis non-consensually, as well as to insert them in sexualized situations.
XAI, the company responsible for Grok, offered no comment to inquiries, other than an automatically-generated response stating "legacy media lies".
A government spokesperson stated that legislation is being proposed to outlaw nudification tools. Under a upcoming criminal offence, anyone who supplied such technology would "receive a prison sentence and heavy fines".
"X or Grok could prevent these types of abuse if they wanted to," said a law professor, adding they "seem to have a lack of consequences."
The communications watchdog Ofcom said platforms must "assess the risk" of people accessing illegal content on their services, but did not specify whether it was presently investigating X or Grok in relation to AI-generated pictures.
Grok is a free AI assistant—with some paid-for functions—which reacts to X users' prompts when they tag it in a tweet.
It is frequently used to give commentary or further context to other posters' remarks, but people on X are also can modify an posted picture through its artificial intelligence picture modification tool.
The tool has been condemned for allowing users to produce images and clips with nudity and sexualised content, and it was in the past alleged to have been involved in making graphic material of celebrities.
"The platform has been enabling the creation and sharing of these pictures for months without taking any measures and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "portraying images of people in a graphic manner".
In a statement, Ofcom said it was unlawful to "create or share non-consensual intimate images or child sexual abuse material" and confirmed this included explicit AI images generated with AI.
It said services such as X were mandated to take "suitable measures" to "minimize the danger" of users viewing illegal content on their platforms and to take down it promptly when they become aware of it.
A tech journalist with over a decade of experience covering emerging technologies and digital innovation across global markets.
