The social media network X has caused significant anger after individuals employed its AI chatbot Grok to change pictures of females by taking off their clothing.
Several cases have been seen where the technology has been employed to remove clothing from women to render them in swimwear without consent, as well as to put them in inappropriate contexts.
Developer Response and Regulatory Position
XAI, the company responsible for Grok, remained silent to questions, except for an automatically-generated response stating "traditional news outlets lies".
A Home Office representative stated that new laws is being proposed to outlaw nudification software. Under a upcoming criminal offence, anyone who supplied such technology would "receive a prison sentence and substantial fines".
"X or Grok could prevent these types of abuse if they chose to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Position and Platform Functionality
The regulator Ofcom said technology companies must "evaluate the danger" of people seeing unlawful material on their platforms, but stopped short of confirming whether it was presently examining X or Grok in regarding artificial intelligence pictures.
Grok is a free artificial intelligence—with some additional features—which responds to X users' prompts when they mention it in a message.
It is often used to give commentary or additional context to other users' remarks, but people on X are also can alter an posted image through its artificial intelligence picture modification feature.
Background of Criticism and Rules
The tool has been faulted for permitting users to produce pictures and videos with nudity and sexually suggestive content, and it was in the past accused of generating sexually explicit material of public figures.
"The platform has been allowing the creation and spread of these pictures for a long time without taking any measures and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "portraying likenesses of people in a pornographic fashion".
In a statement, Ofcom said it was illegal to "generate or share non-consensual intimate images or illegal content" and clarified this applied to explicit AI images generated with AI.
It said services such as X were mandated to take "suitable measures" to "lessen the risk" of users coming across illegal content on their services and to take down it quickly when they learn of it.
