The social media network X has sparked widespread outrage after users used its AI chatbot Grok to change images of females by taking off their attire.
Multiple cases have emerged where the system has been used to remove clothing from women to render them in bikinis non-consensually, as well as to place them in inappropriate scenarios.
Company Response and Regulatory Position
XAI, the company responsible for Grok, offered no comment to inquiries, except for an standard message stating "legacy media lies".
A government official stated that legislation is being proposed to ban nudification tools. Under a proposed law, anyone who supplied such tools would "be subject to a jail term and heavy penalties".
"X or Grok is capable of halting these forms of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Position and Tool Capabilities
The regulator Ofcom said technology companies must "assess the potential" of people seeing unlawful material on their services, but did not confirm whether it was at this time looking into X or Grok in concerning artificial intelligence pictures.
Grok is a free artificial intelligence—with some paid-for functions—which reacts to X users' queries when they tag it in a post.
It is often used to offer reactions or further background to other posters' comments, but people on X are also capable of alter an posted image through its artificial intelligence picture modification function.
History of Criticism and Policy
The chatbot has been criticised for permitting users to produce pictures and videos with explicit content and sexualised content, and it was previously alleged to have been involved in generating inappropriate content of public figures.
"X has been permitting the creation and sharing of these pictures for a long time without taking any action and we have not witnessed any action by oversight bodies," the expert added.
XAI's own usage rules forbids "depicting likenesses of persons in a pornographic way".
In a statement, Ofcom said it was unlawful to "generate or distribute non-consensual intimate images or child sexual abuse material" and clarified this covered sexual deepfakes generated with AI.
It said platforms such as X were required to take "necessary steps" to "minimize the danger" of users viewing illegal content on their services and to remove it quickly when they are notified of it.
