The social media platform X has caused major backlash after individuals used its AI chatbot Grok to manipulate images of women by taking off their attire.
Numerous cases have been seen where the technology has been used to digitally undress female subjects to show them in swimwear non-consensually, as well as to insert them in inappropriate contexts.
Company Response and Legal Position
XAI, the firm behind Grok, remained silent to requests for comment, aside from an automatically-generated message stating "traditional news outlets lies".
A Home Office representative stated that legislation is being introduced to ban nudification software. Under a proposed law, anyone who supplied such technology would "receive a jail term and heavy penalties".
"The platform could prevent these types of abuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Tool Functionality
The regulator Ofcom said tech firms must "assess the risk" of people accessing illegal content on their platforms, but did not specify whether it was at this time looking into X or Grok in relation to artificial intelligence images.
Grok is a no-cost artificial intelligence—with some paid-for functions—which reacts to X users' prompts when they reference it in a tweet.
It is frequently used to give commentary or additional background to other users' remarks, but people on X are also can alter an posted picture through its AI image editing feature.
History of Controversy and Rules
The system has been condemned for allowing users to produce photos and clips with nudity and sexually suggestive content, and it was earlier alleged to have been involved in creating graphic material of celebrities.
"X has been allowing the creation and sharing of these pictures for months without taking any measures and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy prohibits "showing likenesses of persons in a sexually explicit manner".
In a statement, Ofcom said it was illegal to "create or distribute deepfakes or illegal content" and clarified this applied to explicit AI images created with AI.
It said platforms such as X were required to take "necessary actions" to "lessen the risk" of users encountering illegal content on their services and to remove it swiftly when they become aware of it.
