The online network X has sparked widespread outrage after people utilized its artificial intelligence assistant Grok to alter images of females by removing their garments.
Numerous cases have surfaced where the tool has been employed to remove clothing from women to show them in bikinis without permission, as well as to put them in sexualized situations.
Developer Response and Legal Stance
XAI, the company behind Grok, did not respond to questions, other than an automatically-generated response stating "legacy media lies".
A Home Office representative stated that new laws is being proposed to prohibit image stripping tools. Under a upcoming criminal offence, anyone who distributed such technology would "be subject to a prison sentence and heavy fines".
"X or Grok could prevent these forms of abuse if they chose to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Stance and Platform Capabilities
The communications watchdog Ofcom said tech firms must "consider the potential" of people seeing illegal content on their platforms, but did not confirm whether it was currently looking into X or Grok in concerning AI-generated images.
Grok is a no-cost artificial intelligence—with some paid-for functions—which responds to X users' prompts when they reference it in a post.
It is frequently used to provide commentary or further context to other posters' remarks, but people on X are also capable of edit an uploaded photo through its artificial intelligence picture modification tool.
History of Criticism and Rules
The tool has been condemned for enabling users to create pictures and videos with explicit content and sexually suggestive content, and it was in the past alleged to have been involved in generating sexually explicit content of celebrities.
"X has been enabling the creation and distribution of these pictures for a long time without taking any measures and we have not witnessed any intervention by regulators," the expert added.
XAI's own acceptable use policy bans "showing images of persons in a graphic manner".
In a comment, Ofcom said it was against the law to "create or circulate deepfakes or illegal content" and stated this applied to explicit AI images created with AI.
It said services such as X were mandated to take "necessary measures" to "minimize the risk" of users encountering unlawful material on their platforms and to remove it quickly when they learn of it.
