The online network X has caused significant backlash after individuals used its AI chatbot Grok to manipulate pictures of women by stripping their garments.
Multiple cases have been seen where the system has been used to digitally undress women to show them in swimwear non-consensually, as well as to put them in sexualized situations.
Developer Response and Legal Position
XAI, the organization that created Grok, did not respond to questions, except for an standard message stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being proposed to prohibit nudification tools. Under a upcoming criminal offence, anyone who provided such technology would "be subject to a jail term and heavy penalties".
"X or Grok has the ability to stop these types of misuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said platforms must "evaluate the danger" of people viewing illegal content on their services, but stopped short of confirming whether it was currently investigating X or Grok in relation to AI-generated images.
Grok is a no-cost chatbot—with some premium features—which responds to X users' prompts when they tag it in a post.
It is commonly used to offer reactions or more background to other users' comments, but people on X are also able to alter an uploaded picture through its AI image editing tool.
Background of Criticism and Policy
The chatbot has been faulted for allowing users to produce images and clips with explicit content and sexually suggestive material, and it was in the past accused of making graphic material of celebrities.
"X has been permitting the generation and spread of these images for months without taking any steps and we have yet to see any action by regulators," the expert added.
XAI's own terms of service bans "portraying likenesses of people in a pornographic manner".
In a statement, Ofcom said it was against the law to "generate or circulate non-consensual intimate images or illegal content" and clarified this included explicit AI images made with AI.
It said networks such as X were mandated to take "appropriate actions" to "reduce the danger" of users viewing illegal material on their services and to take down it promptly when they become aware of it.
