The social media network X has caused major anger after users used its AI chatbot Grok to manipulate photos of females by taking off their attire.
Numerous instances have emerged where the technology has been employed to virtually strip women to render them in bikinis without permission, as well as to put them in inappropriate contexts.
Company Reaction and Regulatory Stance
XAI, the company behind Grok, remained silent to questions, other than an auto-reply message stating "legacy media lies".
A Home Office official stated that legislation is being brought forward to outlaw nudification software. Under a proposed law, anyone who distributed such tools would "face a prison sentence and heavy fines".
"The platform has the ability to stop these forms of misuse if they wanted to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Position and Platform Functionality
The regulator Ofcom said tech firms must "consider the potential" of people seeing unlawful material on their services, but did not confirm whether it was currently examining X or Grok in relation to AI-generated pictures.
Grok is a free artificial intelligence—with some premium features—which responds to X users' prompts when they tag it in a tweet.
It is frequently used to offer commentary or further background to other users' comments, but people on X are also can edit an posted image through its AI image editing feature.
Background of Controversy and Policy
The system has been condemned for allowing users to generate pictures and clips with nudity and sexualised content, and it was earlier alleged to have been involved in making inappropriate content of celebrities.
"The platform has been permitting the production and spread of these pictures for a long time without taking any action and we have not seen any action by oversight bodies," the expert added.
XAI's own acceptable use policy bans "showing likenesses of individuals in a graphic way".
In a comment, Ofcom said it was against the law to "create or share deepfakes or illegal content" and stated this applied to explicit AI images made with AI.
It said networks such as X were required to take "necessary measures" to "minimize the danger" of users viewing unlawful material on their services and to delete it swiftly when they learn of it.
