Skip to content
AI in hiring7 min read

Human Review of AI Hiring Scores

Why the law expects real human oversight of AI hiring scores, what automation bias research shows, and a review protocol you can measure.

By the HireRabbit.AI team · Published · Last updated

A recruiter at a desk holds a magnifying glass over a resume while a laptop beside her shows a column of candidate scores and a short checklist
On this page
  1. Why human oversight of AI hiring scores is expected
  2. What automation bias research shows
  3. What rubber-stamping looks like
  4. How to review AI scores well
  5. How to tell whether review is real
  6. Worked example: a review protocol for one role
  7. How this maps to HireRabbit.AI

Most hiring teams that use AI scores say a person makes the final call. Often that person sees a ranked list, opens the top five, and clicks "advance" on the AI's picks. Nobody below the line gets opened.

That is a human in the loop on paper only. This guide covers why regulators expect more, why reviewers drift into rubber-stamping, how to review AI scores well, and how to check that review is real.

Why human oversight of AI hiring scores is expected

This section is general information, not legal advice. Rules vary by jurisdiction and are still changing, so check your process with employment counsel.

The EU AI Act

The EU AI Act lists AI used in employment, including tools that filter applications and evaluate candidates, as high-risk under Annex III. Article 14 requires high-risk systems to be designed so they "can be effectively overseen by natural persons" while in use. The people overseeing them must be able to stay aware of the tendency to over-rely on the output, which the Act names "automation bias", and must be able to disregard, override or reverse that output in any particular case.

Article 26 then puts duties on the employer. Deployers must assign human oversight to people "who have the necessary competence, training and authority".

The timing has moved. The Digital Omnibus on AI was published in the EU's Official Journal on July 24, 2026, as Regulation (EU) 2026/1744. As the Future of Privacy Forum's summary of the final text explains, the high-risk requirements for Annex III systems, employment included, now apply from December 2, 2027 instead of August 2, 2026.

GDPR Article 22

GDPR Article 22 gives people the right not to be subject to a decision "based solely on automated processing" that significantly affects them. A rejection can count. Where such decisions are allowed, the controller must offer at least the right to obtain human intervention.

The catch is what counts as human. Spain's data protection authority, the AEPD, set out the European guidance in a March 2024 post: oversight must be "meaningful, rather than just a token gesture", the reviewer needs the authority and competence to change the decision, and an employer cannot dodge Article 22 by "fabricating human involvement". A recruiter who clicks approve on every AI rejection is the textbook token gesture.

US rules

The US has no single federal rule on human review. Colorado is the clearest state example. SB 26-189, signed on May 14, 2026, takes effect on January 1, 2027. As Snell & Wilmer summarizes it, deployers must explain an adverse decision within 30 days of a request and offer human review where commercially reasonable, with the Attorney General's rulemaking expected to clarify that standard. Courts are also watching how much AI shapes screening, as the Mobley v. Workday lawsuit shows.

What automation bias research shows

Automation bias was studied in cockpits and hospitals long before AI hiring.

In a 2012 systematic review in the Journal of the American Medical Informatics Association, Kate Goddard, Abdul Roudsari and Jeremy Wyatt screened 13,821 papers and kept 74 studies on clinical decision support. Heavy workload, task complexity and time pressure made automation bias worse. Mitigators included training, emphasizing users' accountability, and presenting information rather than a recommendation.

Raja Parasuraman and Dietrich Manzey's 2010 review in Human Factors found that automation bias produces two kinds of error when the aid is imperfect: missing a problem the system didn't flag, and following advice that is wrong. It affected both novices and experts, appeared in teams as well as individuals, and was not prevented by training or instructions alone.

Hiring now has its own evidence. In a study presented at the 2025 AAAI/ACM Conference on AI, Ethics, and Society, University of Washington researchers asked 528 people to pick candidates for 16 jobs with help from simulated AI recommendations. Without AI, or with a neutral AI, participants chose white and non-white candidates at similar rates. With a biased AI, they followed its lean. When the bias was severe, their choices followed the AI's picks about 90% of the time. Taking an implicit association test before screening reduced bias by 13%.

"Unless bias is obvious, people were perfectly willing to accept the AI's biases," lead author Kyra Wilson said in the university's November 2025 release. The AI was simulated, but the lesson holds: reviewers tend to adopt the pick they see.

What rubber-stamping looks like

Rubber-stamping shows up in the activity log, not the policy:

  • Reviewers agree with the AI on nearly every candidate, week after week.
  • Reviews take seconds, too short to open the evidence behind a score.
  • Only the top of the ranked list gets opened. Low scorers are never seen.
  • Overrides happen, but the reason field says "agree" or is blank.
  • Rejection emails go out before anyone has looked at the batch.

AEPD's post gives a useful test for time. A reviewer expected to read 100 reports of 100 pages a day would need to get through about 21 pages a minute, which it says would generally not be feasible. If your workload implies a review speed nobody could meet, the review is not real.

How to review AI scores well

Read the evidence before the number

A score is a summary of judgments. Open the part of the resume each judgment rests on and ask whether you would reach the same view. Our guide to explainable AI resume screening covers what a reviewable score should show.

Where you can, form a view from the evidence before looking at the AI's suggestion.

Review a sample of rejects, not only top scores

Automation bias shows up as misses as well as wrong picks. Each week, pull a random sample of low-scored or auto-rejected candidates and read them in full. Look for strong people the tool missed and for patterns, such as career gaps or unusual job titles.

Decide in advance when to override

Write down, before review starts, what justifies overriding a score. Examples: a must-have skill the tool missed, a scoring error against your resume scoring rubric, or evidence the rubric did not anticipate. Pre-agreed grounds keep overrides consistent across reviewers.

Record a reason for every override

Accountability is one of the mitigators in Goddard's review, and a reason gives you a record if a candidate or regulator asks later. The guide to documenting hiring decisions covers what that record should hold.

Give the borderline band a second look

Scores just above and below your cut-off are where small scoring errors change outcomes. Set a band, such as five points either side of the line, and have a person read every candidate in it before anyone is rejected.

Budget the time

Decide how many minutes a careful review takes, then staff for it. If volume exceeds that, narrow the band or sample rather than shrinking each review to a glance.

How to tell whether review is real

  • Override rate. Track how often reviewers change the AI's outcome, in both directions, by reviewer and by role. A rate near zero for months suggests rubber-stamping. A very high one suggests the tool or rubric needs fixing.
  • Agreement on a blind sample. Have a second reviewer score a small sample without seeing the AI's score or the first reviewer's view. Low agreement between people means your criteria are unclear.
  • Adverse impact checks. Compare selection rates by group at the AI stage and again after human review. US federal guidelines treat a group's selection rate below four-fifths (80%) of the highest group's rate as general evidence of adverse impact. Check whether review narrows gaps or adds new ones. The adverse impact analysis guide walks through the math.
  • Time per review. Compare actual review time with the time you budgeted.

Worked example: a review protocol for one role

An illustration for a role with a few hundred applicants and two recruiters. Adjust the numbers to your volume.

StepWhoWhat they doWhat gets recorded
1. Set the rulesRecruiter and hiring managerAgree the cut-off, a borderline band of 5 points each side, and written grounds for overrideGrounds and band saved with the job
2. Review the topRecruiterRead the evidence for each candidate above the band before advancingMove reason for each candidate advanced
3. Review the bandSecond recruiterRead every candidate in the band in fullDecision and reason for each
4. Sample the rejectsEither recruiterRead a random 10% of candidates below the band before any rejection emailAny candidate pulled back, with reason
5. Blind checkHiring managerScore 10 random candidates without seeing AI scoresAgreement with the AI and the recruiters
6. Weekly metricsRecruiting leadCheck override rate, time per review and selection rates by groupShort note on what changed and why

Two rules make this work. Nobody is rejected until steps 3 and 4 are done, and an override without a reason does not count.

How this maps to HireRabbit.AI

HireRabbit.AI scores each resume on skill match, work experience, projects and education, and builds the overall score from those four with weights shown on the job. Reviewers can open any area to read the reasoning, linked back to the resume. Rabbit's Strong hire, Advance, Review or Reject call is always labeled as a suggestion. See how scoring works for the details.

Auto-reject only files resumes below a cut-off you set, in a batch you confirm, sends no email, and you can move any of them back. Moving a single candidate needs a written reason. For AI interviews, the interview score comes only from the candidate's answers. A separate overall AI score weighs answers and integrity flags together, and flags only mark moments for a person to review.

Put this into practice.

Free during invite-only early access. All you need is a work email, and a person reviews every request.