Skip to content

AI Workflow Audits

Researchers just showed AI can invent its own hiring bias out of nowhere, no biased training data required. Here is what business owners should know before letting an AI tool learn on the job.

Researchers at Princeton and the University of Chicago ran several major AI models, including ChatGPT, Claude, Gemini, OpenAI's o3, and DeepSeek's R1, through a simulated hiring exercise with no real names, no real demographic data, and candidates split only into four made-up groups. The models still invented stereotypes about those groups within a few dozen decisions, and stereotyped more strongly than the human participants who took the same test in the original psychology study. The paper was presented at ICML, a major AI research conference, in Seoul in July 2026, and reported by MIT Technology Review on July 20. The direct answer for a business owner: this is not a story about one biased vendor or one bad dataset. It shows a different failure mode, a model that starts clean and still builds its own unequal treatment out of ordinary, sequential decision-making, the same kind of decision-making that happens whenever an AI tool screens candidates, scores leads, or ranks applicants over time. Any business using an AI tool that remembers or learns from its own past decisions inside that business's workflow, not just its original training data, carries a version of this exposure.

By Fabio Rabelo · Founder, ATLACIS ·

What happened

Researchers Addison J. Wu, Ryan Liu, Xuechunzi Bai, and Thomas L. Griffiths, from Princeton University and the University of Chicago, published a paper titled "Large Language Models Develop Novel Social Biases Through Adaptive Exploration," presented as an oral session at ICML (the International Conference on Machine Learning) in Seoul in July 2026. MIT Technology Review reported on the findings on July 20, 2026. The design, adapted from an existing psychology study on how humans form stereotypes, put each model in the role of a hiring consultant for a fictional city. Over 40 rounds, the model filled 20 different jobs, including doctors, lawyers, child-care aides, and janitors, from four fictional demographic groups given made-up names (Tufa, Aima, Reku, and Weki). After every hire, the model learned whether that hire succeeded and moved to the next round. Unknown to the model, every group had an identical, equal chance of succeeding at every job. There was no real demographic data anywhere in the setup, no names tied to any real ethnicity, and no bias built into the training scenario. The models still developed stereotypes. MIT Technology Review reported that when a model observed one fictional group failing at a high-status role like doctor, it began routing that entire group toward lower-status roles like janitor, a pattern the paper's authors describe as jobs split by perceived warmth and competence. On the study's 0 to 2 segregation scale, where 2 means every group has been sorted into its own separate job category, the human participants in the original psychology experiment scored 0.84. The AI models scored roughly 65% higher on average, and OpenAI's reasoning model o3 scored 1.83, close to the maximum possible score. DeepSeek's R1 also showed strong stratification. Notably, the paper found the effect got worse, not better, in newer models with stronger reasoning capability.

Why it matters for business owners

Most small and medium businesses are not running a custom AI hiring simulation, but a growing number use AI tools that do something structurally similar: screen resumes, rank leads, score vendor bids, or triage support tickets, and improve or adjust based on the outcomes they observe over time. Any tool built to "learn" from what happened last time, whether that is explicit fine-tuning, a memory feature, or just an operator repeatedly telling the tool what worked and what did not, is running the same basic loop the researchers tested: make a decision, observe an outcome, adjust the next decision. The useful, verified finding here is not that AI can be biased. Every business owner evaluating AI tools already assumes that is possible. The finding is where the bias can come from: not only from a skewed training dataset the vendor controls, but from ordinary use inside your own business, generated fresh from your own outcomes, with no biased data anywhere in the system to begin with. A vendor's fairness testing at launch cannot catch a pattern that has not been created yet.

What owners should not misunderstand

This is not proof that any specific commercial hiring tool, applicant tracking system, or CRM scoring feature is doing this today. The study is a controlled lab simulation with fictional groups and a simplified 40-round game, not an audit of a real product used by real applicants. It demonstrates a mechanism the researchers could reproduce reliably in a lab, not a documented incident inside any named company's software. It is also not a reason to assume newer or more capable AI models are safer for this kind of task by default. The paper found the opposite: models with stronger reasoning, like o3 and R1, stratified candidates more than earlier, less capable models, not less. Do not treat a model's benchmark scores on coding or reasoning tasks as evidence about its fairness in a repeated decision-making role. And this is not only a hiring story. Griffiths and coauthors' broader argument is about any AI system making repeated decisions with real-world feedback, loan approvals, insurance pricing, lead scoring, vendor selection, wherever an AI tool observes outcomes and adjusts. Hiring is simply the setting the researchers tested directly.

The operational lesson

Telling an AI system to "be fair" is not a reliable fix. MIT Technology Review reported that a direct fairness instruction had little effect on the models' behavior in this study, one model (per the paper) even refused to respond to a fairness-focused system prompt rather than change its pattern. What did work, according to the paper, was changing what the model was actually optimizing for: adding an explicit diversity objective to the reward changed behavior far more reliably than asking nicely. Coauthor Ryan Liu told MIT Technology Review that promising models a bonus for diverse hiring outcomes made them far less biased, while a plain fairness instruction largely did not change the underlying tendency. The researchers also found that giving the model more individuating information about each candidate, real, relevant detail rather than group membership, reduced stereotyping, while irrelevant detail did not help and could even reinforce it. That is a concrete, testable design principle: an AI decision tool that only sees a thin profile is more likely to fall back on pattern-matching by group than one that sees the specific, job-relevant facts about a specific person.

What a serious business should do next

List every AI tool your business uses for a repeated selection or ranking decision, not just hiring: resume screening, lead scoring, applicant ranking, vendor shortlisting, credit or payment risk flags, customer prioritization. For each one, find out from the vendor, in writing, whether the tool's scoring changes over time based on outcomes it observes inside your account, or whether it is a fixed model that does not update from your usage. A static, well-tested model and a continuously adapting one carry different risk profiles, and most owners cannot currently answer which one they have. For any tool that does adapt or has memory of past outcomes, ask what objective it is actually optimizing for, and whether fairness or diversity considerations are part of that objective or bolted on afterward as an instruction. Per this research, the difference matters more than most vendors' marketing suggests. Build in periodic human review of AI-assisted hiring or screening decisions, not as a one-time compliance check at rollout, but as a recurring spot check, since a pattern like this can emerge gradually over weeks or months of normal use, not all at once. Do not wait for a discrimination complaint to be the first time anyone looks at the pattern of who the tool has been recommending.

The Atlacis view

Atlacis helps owners look past a vendor's fairness claims at launch and ask the harder question: what does this tool actually optimize for once it is running inside your business, and who is checking its pattern of decisions after the first ninety days. A tool can pass every fairness benchmark a vendor publishes and still drift once it starts learning from your specific outcomes, which is a workflow design question, not a one-time procurement question. Before recommending any AI tool that screens, ranks, or scores people, Atlacis maps whether it adapts from experience, what it is actually rewarded for, and what a reasonable human review cadence looks like for that specific use, rather than treating a vendor's launch-day audit as the last word.

The short version

  • A Princeton and University of Chicago study, presented at ICML in Seoul in July 2026 and reported by MIT Technology Review on July 20, found that AI models invent new demographic stereotypes purely from their own sequential hiring decisions, with no biased training data and no real demographic information in the setup.
  • The models stereotyped more than human participants in the same test. On a 0 to 2 segregation scale, humans scored 0.84; the AI models scored roughly 65% higher on average, and OpenAI's o3 scored 1.83, near the maximum.
  • Newer, higher-reasoning models (o3, DeepSeek's R1) showed stronger bias, not weaker, which means model capability is not a proxy for fairness in repeated decision-making roles.
  • Telling a model to "be fair" had little effect in the study. Adding an explicit diversity objective to what the model was optimizing for, and giving it more individuating detail about each candidate, worked better, per the paper and coauthor Ryan Liu.
  • This is a lab finding about a mechanism, not proof any specific commercial hiring tool is doing this today. The operational lesson: inventory any AI tool that adapts from your business's own outcomes, ask what it optimizes for, and build in recurring human review rather than a one-time fairness check at purchase.
Tags:AI hiringAI biasAI workflow auditshuman reviewAI decision-makingbusiness AIAI governanceworkflow auditAI implementationAI risk
FAQ

Common questions

Does this mean my company's hiring software is biased?
Not necessarily. This is a controlled research study using a simulated game with fictional groups, not an audit of any named commercial product. It demonstrates a mechanism that can occur, a model developing bias purely from its own repeated decisions, not a documented finding about a specific tool your business uses.
Are newer AI models safer to use for hiring or screening decisions?
Not automatically. The study found that models with stronger reasoning capability, including OpenAI's o3 and DeepSeek's R1, showed more stratification by demographic group than less capable models, not less. A model's benchmark performance on coding or reasoning tasks is not evidence about its fairness in a repeated decision-making role.
What is the single most useful thing to check because of this research?
Find out whether any AI tool you use for hiring, screening, or scoring adapts its behavior over time based on outcomes it observes in your account, or whether it is a fixed, unchanging model. If it adapts, ask the vendor what objective it optimizes for and put a recurring human review of its decision pattern on the calendar, rather than relying on a fairness check done once at purchase.
Keep reading

More from the blog

OpenAI's newest AI coding agent reportedly deleted a user's files days after launch. OpenAI had already warned this could happen. Here is what business owners should know.

OpenAI launched GPT-5.6 Sol, its most capable coding and agentic model, on July 9, 2026, with a new autonomous 'Ultra mode.' The next day, an AI investor said a Sol subagent deleted most of his Mac's files during a routine cleanup task, the exact category of risk OpenAI's own official safety documentation had disclosed two weeks earlier. The useful lesson is not about picking a side on OpenAI's safety record. It is that vendor safety documentation is a real risk disclosure, and it is worth reading before an AI agent gets broad access to a business's files, storage, email, or CRM.

The AI workflow audit checklist: what to check before you automate anything

Automation is now a switch inside software you already pay for, which makes it easy to automate a mess. Here is the short screen to run before you flip anything on.

IBM's AI handled 94 percent of HR requests. It still decided to triple entry-level hiring. Here is what business owners should know.

Challenger, Gray & Christmas data shows AI is now the leading reason US employers cite for layoffs, the highest share on record. At the same time, CNBC reported that IBM, Ford, and Commonwealth Bank of Australia are rebuilding the human side of work they shifted to AI. IBM's case is the most useful to study: its AI automated 94% of routine HR requests, and IBM responded by tripling entry-level hiring rather than cutting further. The lesson is a specific one about which tasks build judgment and which do not, not a general verdict on whether AI works.

Make better AI decisions, starting with one call.

Book a free AI Fit Call. We will tell you what to use, what to avoid, and where to start. No jargon, no pressure.