In a landmark ruling, a Dutch court has ordered xAI to stop generating and distributing nude images without consent through its AI product, Grok. This decision comes amidst growing concerns over the ethical implications of artificial intelligence, particularly in the realm of non-consensual content creation.
The court’s ruling, issued on March 26, 2026, imposes hefty fines of 100,000 euros per day for any noncompliance, underscoring the seriousness of the situation. This case marks one of the first instances where a court has addressed the responsibility of AI in creating sexualized images, a topic that has sparked significant debate in recent years.
Grok, launched by Elon Musk in late December 2025, has faced regulatory scrutiny due to its features that allow users to generate images that can be deemed inappropriate. The case was brought forward by the organizations Offlimits and Fonds Slachtofferhulp, which highlighted the urgent need for accountability in AI technologies.
The court found that xAI had not implemented adequate measures to prevent the generation of non-consensual content, including images that appeared to depict children. In a striking rejection of xAI’s defense, the court stated that liability does not solely rest with users issuing prompts, but rather with the company providing the technology.
During the specified period from December 29, 2025, to January 8, 2026, Grok generated approximately 3 million sexualized images, raising alarms about the potential misuse of AI tools. Robbert Hoving, a representative from Fonds Slachtofferhulp, emphasized, “The burden is on the company to make sure its tools are not used to create and distribute nonconsensual sexual images, including of children.”
Additionally, the European Parliament has recently approved a ban on AI systems generating sexualized deepfakes, reflecting a broader movement towards stricter regulations in the AI space. The court’s injunction applies not only to xAI but also to its EU-facing entity, XIUC.
As part of the ruling, xAI is required to confirm in writing to Offlimits how it has complied with the court’s order. Observers note that this case could set a precedent for future regulations surrounding AI-generated content, particularly in Europe.
In light of these developments, the court’s decision is seen as a crucial step in ensuring that AI technologies are developed and utilized responsibly. The implications of this ruling will likely resonate beyond the Netherlands, influencing discussions on AI accountability across the globe.
Details remain unconfirmed regarding the specific measures xAI will implement to comply with the court’s order, but the expectation is that the company will need to take significant steps to align its practices with legal standards.
