The social media network X has provoked major outrage after people used its AI chatbot Grok to change photos of women by taking off their attire.
Multiple instances have been seen where the technology has been used to virtually strip women to show them in bikinis without permission, as well as to insert them in inappropriate scenarios.
Developer Reaction and Legal Stance
XAI, the company responsible for Grok, offered no comment to inquiries, except for an standard message stating "legacy media lies".
A Home Office spokesperson stated that new laws is being proposed to outlaw image stripping software. Under a upcoming criminal offence, anyone who supplied such tools would "be subject to a prison sentence and significant penalties".
"X or Grok could prevent these forms of abuse if they chose to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Stance and Tool Capabilities
The regulator Ofcom said technology companies must "assess the danger" of people seeing illegal content on their services, but did not specify whether it was at this time investigating X or Grok in concerning AI-generated pictures.
Grok is a no-cost chatbot—with some premium functions—which responds to X users' prompts when they mention it in a post.
It is often used to give reactions or more context to other posters' remarks, but people on X are also able to alter an posted image through its artificial intelligence picture modification function.
History of Controversy and Policy
The system has been criticised for permitting users to generate images and videos with explicit content and sexualised material, and it was in the past accused of creating sexually explicit content of celebrities.
"The platform has been enabling the generation and distribution of these pictures for months without taking any action and we have not witnessed any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "portraying likenesses of people in a sexually explicit way".
In a comment, Ofcom said it was illegal to "create or distribute private pictures or illegal content" and clarified this applied to explicit AI images created with AI.
It said networks such as X were mandated to take "appropriate actions" to "reduce the danger" of users encountering unlawful material on their platforms and to delete it swiftly when they learn of it.
