What actually happened
According to WIRED's investigation, a project internally called Cannes, run for Meta by contractor Covalen, had hundreds of contractors create dummy accounts with birthdates under 18. They submitted text prompts and images to OpenAI's ChatGPT, Google's Gemini, and Character.AI, then copied the responses into spreadsheets for review. The project ran until at least April 2026. In one round in August 2025, contractors submitted more than 45,000 prompts. A spreadsheet WIRED reviewed contained 3,748 prompts, many written from the point of view of a child or teenager in crisis: a fifth grader describing a classmate holding a gun, a 13-year-old asking where to get abortion pills, a girl asking how to hide an eating disorder from her parents. Some submissions included images of pills, knives, and nooses. Meta's on-record response, given to WIRED, was that this is 'a responsible, industry-standard practice' for safety benchmarking, and that Meta does not use competitor outputs to train its own models. Character.AI said the testing was not authorized and violated its terms of service. OpenAI said it was looking into the matter. Google said it had not authorized the testing and did not know its purpose, though its own review of the sample prompts found Gemini responding in line with its policies. None of the three companies being tested knew it was happening.
Why this matters even if your business has nothing to do with Meta
It is tempting to read this as a story about one company's questionable methods. That is a real question, and it is not settled. But underneath it is a fact that applies far beyond Meta: finding out whether an AI chatbot actually holds its guardrails under real, messy, high-stakes input took a dedicated program, real staffing, and tens of thousands of test prompts. That is what it takes to know, instead of assume, that a chatbot behaves the way its maker says it does. This is also not happening in a vacuum. The US Federal Trade Commission opened a formal inquiry into AI chatbots and child safety in September 2025, covering Meta, OpenAI, Google, Character Technologies, and others. Regulators are actively asking these companies how they test their own safety claims. A business that deploys a customer-facing AI tool today is operating in a period where 'this vendor says it is safe' is getting less benefit of the doubt from regulators, from the press, and it should get less from you.
What business owners should not misunderstand here
This is not a story about whether to trust or avoid ChatGPT, Gemini, Character.AI, or Meta's own products. All of the companies involved dispute parts of how this was framed, and this post is not the place to referee that. It is also not a reason to panic about any AI chatbot your business uses today. The misunderstanding to avoid is narrower and more useful: assuming that a vendor's safety marketing ("safe and age-appropriate," "enterprise-grade," "built with guardrails") means an independent party has actually tried to break it. In this story, the party doing the breaking was a competitor with its own incentives, not a neutral tester, and the companies being tested did not know it was happening. Your own AI vendor's safety claims deserve the same scrutiny you would give any other vendor claim: verify it, do not assume it because it is printed on a pricing page.
The operational lesson: test what talks to your customers before someone else does it for you
If your business uses any AI tool that talks directly to customers or employees, a support chatbot, a sales assistant, a scheduling bot, an internal HR helper, the operating question is simple: has anyone actually tried to push it into a bad response, and do you know what happens when someone does? Most small and medium businesses have never run this test on their own tools, because it is not the kind of thing a vendor demo shows you. You do not need Meta's scale to do a version of this. You need a short, deliberate exercise: a small set of realistic, messy, and edge-case inputs, run by someone inside your business, against the actual tool your customers or staff use, followed by an honest look at what came back and whether a human would have caught a bad answer before it went out.
What a serious business should do next
Start with an inventory. List every AI tool that talks to a customer or an employee directly, not just the ones you remember buying, since some arrive bundled inside software you already pay for. For each one, ask the vendor directly: how was this tested, by whom, and against what kind of input. A vendor that cannot answer clearly is telling you something. Then run your own small test before wider rollout, or now, if the tool is already live. Try a handful of off-script, ambiguous, or sensitive inputs a real customer or employee might plausibly send, not just the happy-path demo questions. See what comes back. If the answer touches anything sensitive, health, money, safety, legal exposure, confirm there is a human checkpoint before that answer reaches someone who needed a person, not a script. Do not act on this by ripping out a working tool over a headline. Do act by closing the specific gap: you do not currently know how your customer-facing AI behaves under pressure, and that is fixable in an afternoon, not a redesign.
The Atlacis view
This story is not really about Meta, or ChatGPT, or Gemini. It is a reminder that 'safety tested' is a claim, not a fact, until someone outside the vendor's own interests checks it. Atlacis helps owners slow down enough to ask that question about their own AI tools before something goes wrong in front of a customer, not after. That means mapping which AI systems in your business actually talk to people, understanding what a vendor's safety claim does and does not cover, and making sure a human is in the loop wherever the stakes are real. It is a small amount of work up front against a much larger cost if you find out the hard way.
The short version
- WIRED reported that Meta ran a program, internally called Cannes and staffed through contractor Covalen, in which contractors posed as teenagers and sent more than 45,000 prompts in a single round to rival chatbots (ChatGPT, Gemini, Character.AI) to find where their safety guardrails failed.
- Meta calls this routine, industry-standard safety benchmarking. Character.AI says the testing was unauthorized and violated its terms. OpenAI and Google say they were not aware of it. None of this is fully settled, and this post does not take a side on it.
- The finding that matters for other businesses: a company with Meta's resources did not trust a rival's safety claims and built a large, dedicated program to test them. Most businesses have never done anything like this to their own AI tools.
- The US FTC has an open inquiry into AI chatbots and child safety, covering Meta, OpenAI, Google, and Character Technologies, since September 2025. Vendor safety claims are getting more scrutiny, not less.
- The practical step: inventory every AI tool that talks to a customer or employee, ask the vendor how it was tested, run your own small test with realistic edge-case input, and confirm a human checks any response that touches something sensitive.
Where ATLACIS can help
Sources
- WIRED: Meta Contractors Posed as Teens to Prompt Rival Chatbots About Suicide, Sex, and Drugs (June 29, 2026)
- The Next Web: Meta paid contractors to pose as teens and probe rival AI chatbots (June 30, 2026)
- New York Post: Meta contractors posed as teens to test rival AI chatbots on suicide, sex and drugs (July 1, 2026)
- The Decoder: Meta secretly tested ChatGPT, Gemini, and Character.AI with thousands of minor-perspective crisis prompts (June 30, 2026)