Overview
- Researchers at Mindgard and other teams published tests on June 17–18 showing a short, viral prompt such as “restore the attached photo” can lead ChatGPT to generate graphic sexual and violent images.
- OpenAI told reporters it has added new protections and is investigating the reports while saying its systems combine automated filters with human review to block harmful images.
- Mindgard said it reported the findings to OpenAI but did not get a prompt response until reporters contacted the company, and a researcher involved was left shaken and took time off work.
- Independent tests and prior 2025 research show this problem is persistent because simple prompt-engineering tricks, called jailbreaks, have repeatedly bypassed DALL·E and similar image engines.
- The episode raises regulatory and safety concerns given past incidents, including Grok’s January 2026 run that produced millions of sexualized images, and experts warn model bias and easy jailbreaks increase the risk of harmful content spreading online.