Ask an organisation how it manages bias risk in AI-assisted hiring and the answer is almost always the same. A human reviews the output. Lead author Kyra Wilson has noted that in one survey, 80% of organisations using AI hiring tools said they do not reject applicants without human review.
Researchers at the University of Washington tested whether that review works. It largely does not.
What this is and why it matters now
The University of Washington study put 528 participants through 1,526 resume-screening scenarios across 16 job categories. Without AI recommendations, participants’ choices showed little racial bias. When the AI recommendations were severely biased, participants followed them around 90% of the time. The findings were presented on 22 October 2025 at the AAAI/ACM Conference on Artificial Intelligence, Ethics and Society.
The significance is not that AI models carry bias. That is established, and vendors compete on mitigating it. The significance is that the control organisations rely on to catch it, a person looking at the output, does not function by default. Reviewers were not negligent. They were shown a score without the reasoning behind it, and they deferred to it.
A control that fails this quietly is one an organisation will keep describing as working, and the commercial stake in that gap has moved from theoretical to live. Mobley v. Workday, Inc., in the Northern District of California, is proceeding on a specific theory. A screening vendor may be an agent of its client-employers, and therefore within the statutory definition of employer. On 22 June 2026 the court declined to dismiss the California anti-discrimination claims and allowed a federal disability claim to proceed, while dismissing a claim brought on behalf of Asian American applicants. On 2 July 2026 it refused to certify an interlocutory appeal, and discovery continues. Earlier, in July 2025, the court had ordered Workday to produce a list of customers that enabled its AI screening features, including the HiredScore products. These remain unproven allegations, but employers using those features have been drawn into a matter they did not choose to join.
This piece is for talent acquisition and HR leaders who have deployed or are evaluating AI screening. It is equally for the legal and compliance partners who carry the exposure when a decision is challenged.
Readiness is a different variable from accuracy
The University of Washington finding reframes what to procure for. If a reviewer defers to whatever score appears, then improving the score improves outcomes and changes nothing about the control. The organisation remains one vendor change away from a problem it cannot detect.
Readiness describes whether the organisation can govern the tool. Do reviewers see reasoning rather than only a rank? Is anyone accountable for auditing outcomes? Does the workflow make disagreement with the tool practical? Are decisions recorded well enough to reconstruct? None of that is a property of the model. All of it determines whether the model is safe in this organisation.
Every one of those questions is answered by the organisation rather than the vendor, and the study found something useful on exactly that point. Bias dropped 13% when participants took an implicit association test before screening. The intervention that helped was applied to the reviewer and the workflow, not to the model.
What does readiness mean in practice for AI hiring?
Four things must be true before the tool ranks a candidate. The reviewer can see why the tool scored as it did. A named person owns adverse-impact testing. The process allows and records disagreement with the recommendation. The decision record would let someone else reach the same conclusion. Where any is missing, the human review is nominal.
Bias arrives through routes procurement does not examine
Readiness assumes the organisation knows what it is auditing for. Bias in HR systems is rarely a single defect. Aicadium, a Singapore-based AI solutions company, teaches its People Operations team five recurring forms of it.
Historical or sample bias comes from skewed training data, such as a model that ranks men higher for technical roles because the history it learned from did.
Proxy bias uses stand-ins for identity: postal code, school name, hobbies, language cues.
Measurement bias misreads styles, for example scoring assertive feedback as negative.
Algorithmic bias favours formats, rewarding polished, fluent English regardless of substance.
Deployment or feedback bias is self-reinforcing, where teams labelled low potential receive fewer opportunities and the label confirms itself.
Proxy and feedback bias are the hardest to see, because neither requires a protected attribute to be present in the data. A model can be demonstrably blind to race and still rank on a proxy for it.
Documentation is becoming the contested asset
Knowing those five failure modes is necessary. Showing what you did about them is the harder requirement, and the litigation record has made that concrete in an unexpected way.
The clearest sign of that is what the two sides in Mobley have been fighting over. In May 2026, a magistrate judge denied the plaintiffs’ motion to compel production of Workday’s bias-testing data. The court found that attorney-client privilege applied. Workday’s attorneys had curated the data and used the results in providing legal advice. Whatever one makes of that outcome, it establishes that bias-testing artefacts are now contested material in employment litigation, and that how testing is commissioned affects its legal status.
Bias testing is not the only artefact now being litigated. A separate class action filed in January 2026 against Eightfold AI makes a different allegation. It claims the company operated as a consumer reporting agency under the Fair Credit Reporting Act, collecting and scoring applicant data from third-party sources without consent. Also unproven. The pattern across the two is worth noting: one attacks outcomes, the other attacks process.
Regulation is converging on the same requirement. The European Union Artificial Intelligence Act classifies recruitment and selection systems as high-risk, with obligations including risk management, logging, technical documentation and human oversight. Those obligations were deferred to 2 December 2027 by the Digital Omnibus on AI, which entered into force in July 2026. The Act’s transparency duties still applied from 2 August 2026. In California, SB 947, the No Robo Bosses Act, was signed on 30 September 2026 and applies from 1 July 2027. It bars reliance on an automated decision system alone to discipline or dismiss a worker, and where such a system is the primary basis, a human must corroborate the decision against other evidence. SB 951 was signed the same day and lands earlier, on 1 January 2027. It amends California’s WARN Act so that where a mass layoff, relocation or termination is caused in whole or substantial part by AI or other automated technology, the notice must identify the technology responsible and the job functions being automated. In Singapore, the Workplace Fairness Act is slated to take effect at the end of 2027. It will cover recruitment, appraisal, promotion and dismissal for employers with 25 or more employees.
Does a deferred deadline reduce the urgency?
No, for two reasons. The evidence a defence relies on is contemporaneous. A bias audit run in 2027 says nothing about screening decisions made in 2026, and those decisions are already being made. The deferral also did not move everything. California’s SB 951 applies from 1 January 2027, ahead of every other date in this section.
What this changes about procurement
Documentation and readiness are internal work. Both change what the organisation should be asking of the vendors it buys from. Accuracy and fairness metrics remain relevant and are no longer sufficient on their own. Four questions are more diagnostic. Does the tool show a reviewer the reasoning behind a score? Does it record disagreement? What logging does it produce? Can the deploying organisation run its own adverse-impact testing, rather than relying on the vendor’s?
Asking those four questions is uncomfortable precisely because of what the University of Washington result implies about the alternative. An organisation that buys a better-scoring model and changes nothing about the review workflow has improved its inputs and left its control exactly as weak.
Where to start
Run a small internal check before the next procurement cycle. Take a set of past screening decisions where a tool made a recommendation, and ask a reviewer to reconstruct why the recommendation was made and why they agreed or disagreed. If the reconstruction is not possible, the human-in-the-loop control is nominal, and that is a readiness finding rather than a model finding. For a more complete guide, check out our handbook compliance kit.
