🔗 Share this article

The social media platform X has sparked widespread backlash after individuals used its artificial intelligence assistant Grok to manipulate images of females by removing their attire.

Several cases have emerged where the technology has been used to digitally undress female subjects to render them in bikinis non-consensually, as well as to insert them in inappropriate contexts.

Developer Reaction and Legal Position

XAI, the organization that created Grok, remained silent to requests for comment, aside from an automatically-generated response stating "legacy media lies".

A government official stated that legislation is being proposed to prohibit image stripping software. Under a new law, anyone who provided such technology would "be subject to a jail term and substantial penalties".

"The platform is capable of halting these types of misuse if they wanted to," said Clare McGlynn, adding they "seem to have impunity."

Oversight Body Position and Tool Capabilities

The regulator Ofcom said technology companies must "assess the risk" of people viewing illegal content on their networks, but did not specify whether it was currently looking into X or Grok in concerning artificial intelligence images.

Grok is a free chatbot—with some premium functions—which reacts to X users' queries when they reference it in a post.

It is often used to provide commentary or additional background to other users' comments, but people on X are also capable of modify an posted image through its AI image editing feature.

History of Controversy and Policy

The chatbot has been faulted for permitting users to generate images and footage with explicit content and sexually suggestive material, and it was in the past alleged to have been involved in generating sexually explicit content of public figures.

"X has been allowing the creation and sharing of these images for a long time without taking any action and we have not seen any intervention by regulators," Professor McGlynn added.

XAI's own acceptable use policy forbids "portraying likenesses of persons in a sexually explicit fashion".

In a statement, Ofcom said it was against the law to "make or circulate deepfakes or child sexual abuse material" and confirmed this applied to sexual deepfakes generated with AI.

It said networks such as X were required to take "suitable actions" to "minimize the danger" of users encountering illegal content on their services and to delete it quickly when they become aware of it.