The online platform X has caused major anger after users employed its AI chatbot Grok to alter pictures of females by taking off their garments.
Several examples have been seen where the system has been employed to digitally undress female subjects to show them in swimwear without permission, as well as to place them in sexualized scenarios.
Developer Response and Regulatory Position
XAI, the organization responsible for Grok, remained silent to questions, other than an auto-reply message stating "legacy media lies".
A Home Office spokesperson stated that legislation is being brought forward to prohibit nudification tools. Under a proposed law, anyone who distributed such tools would "receive a jail term and substantial fines".
"X or Grok is capable of halting these types of abuse if they wanted to," said a law professor, adding they "operate with impunity."
Oversight Body Position and Platform Functionality
The regulator Ofcom said platforms must "assess the potential" of people viewing unlawful material on their networks, but stopped short of confirming whether it was presently investigating X or Grok in concerning AI-generated pictures.
Grok is a free artificial intelligence—with some premium features—which responds to X users' requests when they mention it in a post.
It is commonly used to give reactions or additional context to other users' comments, but people on X are also able to edit an uploaded photo through its artificial intelligence picture modification function.
Background of Criticism and Rules
The tool has been faulted for enabling users to produce pictures and footage with nudity and sexually suggestive material, and it was previously accused of generating graphic content of public figures.
"X has been allowing the generation and sharing of these pictures for a long time without taking any action and we have not seen any action by oversight bodies," the expert added.
XAI's own acceptable use policy bans "depicting likenesses of people in a graphic manner".
In a comment, Ofcom said it was against the law to "generate or distribute non-consensual intimate images or child sexual abuse material" and confirmed this included sexual deepfakes created with AI.
It said networks such as X were required to take "suitable actions" to "minimize the danger" of users coming across unlawful content on their platforms and to take down it promptly when they are notified of it.
