OpenAI has hired hundreds of contractors who read real ChatGPT users' conversations — including complete chat histories with sensitive personal information — under an internal program code-named "Project Lily," 404 Media reported Monday. The revelation carries direct implications for the millions of AI companion users who share intimate, romantic, or NSFW details with the chatbot under an assumption of machine-only processing.
According to internal documents and real user prompts reviewed by 404 Media, the contractors process chats in three stages: reading the full user prompt, summarizing the user's intent, and critiquing multiple AI responses. The reviewers see entire conversations, not isolated messages, and the review interface can include a "user memories summary" that reveals personal context such as likely location or prior usage patterns.
OpenAI confirmed to 404 Media that conversations are used to improve its models unless users toggle off the "improve the model for everyone" setting. The company says it strips usernames and runs chats through a "Privacy Filter" before contractors see them, but acknowledged that sensitive details can still slip through.
What This Means for Companion Users
AI companion users who engage ChatGPT for romantic roleplay, emotional support, or NSFW conversations are particularly exposed. Unlike dedicated companion platforms where users may expect human review as part of content moderation, ChatGPT's 900 million-plus users have no routine indication that a human might read their most personal exchanges. The "memories" feature — which lets ChatGPT retain personal context across sessions — can compound the risk by surfacing identifiable details in the reviewer interface.
The privacy implications extend beyond embarrassment. Companion-style chats can contain health disclosures, trauma narratives, workplace information, and other data that users would never knowingly share with a human reviewer. OpenAI's own privacy policy states it does not process sensitive personal data to infer characteristics about a consumer, but the Project Lily workflow appears to expose that data to human eyes during model evaluation.
Regulatory Exposure
Project Lily raises compliance questions under multiple regulatory frameworks. The EU's General Data Protection Regulation requires data controllers to have a lawful basis for processing personal data and to inform data subjects about that processing. Italy's Garante previously fined OpenAI €15 million for GDPR violations including failure to identify an adequate legal basis for training data. The EU AI Act's transparency provisions, which took effect for models serving European users in August 2026, add additional disclosure requirements about model training data practices.
In the United States, state chatbot laws now covering the entire West Coast and more than 35 states total require AI companion services to disclose data collection practices. California's SB 243, which became enforceable in March 2026, specifically requires chatbot services to inform users about data retention and human review of conversations. OpenAI's failure to prominently disclose human review of ChatGPT conversations could put it in conflict with these emerging state-level requirements.
How to Opt Out
Users who want to prevent their conversations from being reviewed by human contractors can disable the setting. In ChatGPT, navigate to Settings → Data Controls and toggle off "Improve the model for everyone." OpenAI says this setting prevents chats from being used for training, though the company's data retention policies and legal preservation orders may still apply in certain circumstances.
The Project Lily disclosure follows a pattern of growing scrutiny around how AI companies handle user data. In May 2026, Canadian privacy regulators found that OpenAI failed to obtain valid consent for collecting personal information to train its models. A separate court order in a copyright lawsuit requires OpenAI to preserve all user conversations — even deleted ones — for discovery purposes, raising further questions about what "deletion" means in practice.
For AI companion users, the takeaway is straightforward: treat every ChatGPT conversation as potentially inspectable by human reviewers, and use the opt-out toggle if that's not acceptable.