Elon Musk watches President Donald Trump speak during the U.S.-Saudi Investment Forum at the John F. Kennedy Center for the Performing Arts in Washington, November 19, 2025.
Brendan Smialowski | AFP | Getty Images
Elon MuskThe xAI saw users react after its artificial intelligence chatbot Grok generated sexualized images of children in response to user prompts.
A response from Grok to a user on X said Friday it was “urgently addressing” the problem and called child pornography “illegal and prohibited.”
In responses to users, the bot also job that a company could face criminal or civil penalties if it knowingly facilitates or fails to prevent this type of content after being alerted.
Grok’s posts are AI-generated messages and do not replace official company statements.
Musk’s xAI, which created Grok and merged with X last year, sent an automated response to a request for comment: “Legacy Media Lies.”
X users have raised concerns in recent days regarding explicit content of minors, including children wearing minimal clothing, generated using the Grok tool.
The social media site added an “Edit Image” button to photos that allows any user to edit it using text prompts and without the consent of the original author.
Read more CNBC tech newsA message from a member of the xAI technical team Tajik Parsa also recognized the problem.
“Hey! Thanks for reporting. The team is looking into tightening our guardrails further,” Tajik wrote in a message.
The proliferation of AI image generation platforms since the launch of ChatGPT in 2022 has raised concerns about content manipulation and online security at all levels. This also contributed to the increase in the number of platforms that produced fake nudes of real people.
David Thiel, a trust and safety researcher who was part of the now-disbanded Stanford Internet Observatory, told CNBC that various U.S. laws generally prohibit the creation and distribution of certain explicit images, including those depicting child sexual abuse, or non-consensual intimate images.
Legal decisions regarding AI-generated images, like those produced by Grok, may depend on specific details of the content created and shared, he said.
In a paper he co-authored titled “Generative ML and CSAM: Implications and Mitigation,” Stanford researchers noted that “the appearance of a child being abused was sufficient to warrant prosecution,” in precedent-setting cases in the United States.
While other chatbots have faced similar issues, xAI has found itself in hot water several times due to misuse or apparent flaws in Grok’s design or underlying technology.
“There are a number of things companies could do to prevent their AI tools from being used in this way,” Thiel said. “The most important one in this case would be to remove the ability to edit user-uploaded images. Allowing users to edit uploaded images is a recipe for NCII. Nudification has always been the primary use case for such mechanisms.”
NCII refers to non-consensual intimate images.
In May, X faced backlash after Grok generated unsolicited comments about “white genocide” In South Africa. Two months later, Grok posted anti-Semitic comments And praised Adolf Hitler.
Despite the stumbles, xAI continued to strike partnerships and deals.
THE Ministry of Defense added Grok to its AI agent platform last month, and the tool is the main chatbot for the Polymarket and Kalshi prediction betting platforms.