The social media network X has sparked significant backlash after users employed its AI chatbot Grok to alter pictures of females by stripping their attire.
Multiple cases have surfaced where the system has been used to remove clothing from women to render them in bikinis non-consensually, as well as to place them in inappropriate scenarios.
Company Response and Regulatory Stance
XAI, the firm responsible for Grok, did not respond to inquiries, except for an automatically-generated response stating "legacy media lies".
A government representative stated that legislation is being brought forward to ban nudification tools. Under a new law, anyone who supplied such tools would "face a jail term and substantial penalties".
"The platform is capable of halting these forms of abuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Position and Tool Functionality
The regulator Ofcom said tech firms must "evaluate the danger" of people accessing unlawful material on their platforms, but did not confirm whether it was at this time investigating X or Grok in regarding artificial intelligence pictures.
Grok is a no-cost chatbot—with some premium features—which answers X users' queries when they mention it in a tweet.
It is commonly used to provide reactions or additional context to other users' remarks, but people on X are also can alter an posted photo through its AI image editing function.
Background of Criticism and Policy
The chatbot has been faulted for allowing users to create photos and videos with nudity and sexualised material, and it was in the past alleged to have been involved in creating inappropriate content of public figures.
"The platform has been allowing the creation and sharing of these images for a long time without taking any steps and we have yet to see any action by oversight bodies," the expert added.
XAI's own terms of service forbids "portraying images of persons in a pornographic way".
In a statement, Ofcom said it was illegal to "generate or distribute deepfakes or child sexual abuse material" and stated this applied to sexual deepfakes made with AI.
It said services such as X were required to take "necessary steps" to "reduce the danger" of users encountering unlawful content on their services and to take down it promptly when they become aware of it.
