How to Choose an AI Interviewer
How hiring teams should choose an AI interviewer: eight questions for any AI interview platform, how to run a pilot, and legal dates to plan around.
By the HireRabbit.AI team · Published · Last updated

On this page
In 2026, economists Brian Jabarian and Luca Henkel released a working paper on a field experiment with about 70,000 job applicants in the Philippines. Each was randomly assigned to a first-round interview with either a human recruiter or an AI voice interviewer, and people made every hiring decision afterwards. Applicants interviewed by the AI were 12% more likely to receive an offer, and the AI interviews were more structured and consistent. It covers one company and is not peer reviewed, but it suggests a well-built AI first round need not cost candidates offers.
Every vendor of AI interview software demos well. This guide covers what an AI interviewer should do, eight questions to ask, how to run a pilot, and the legal dates to plan around.
What an AI interviewer is
An AI interviewer is software that runs an interview round with a candidate on its own. It asks questions, listens to the answers, asks follow-ups, and hands your team a record and an assessment. Newer tools hold a real-time spoken conversation in the browser. Older one-way video tools only record answers to fixed prompts for someone to watch later.
It is not resume screening, which scores applications before anyone is interviewed, and it should not make the final decision. Its usual job is the first-round screen a recruiter would otherwise repeat dozens of times for one role.
For the candidate's side, including what they are owed and how to prepare, see our guide to AI interviews for candidates and employers. This guide is for the team choosing the software.
Start with the round, not the tool
Decide what the round is for before you look at vendors. Write down the three or four job-related things it must establish, the knock-out requirements, and what a strong answer sounds like.
Structure is a large part of what makes interviews predictive. A 2022 meta-analysis by Paul Sackett and colleagues in the Journal of Applied Psychology found structured interviews had the highest average validity of the selection methods they studied, ahead of job knowledge tests, work samples and cognitive ability tests. Those figures come from human interviewers, so an AI interviewer only borrows the advantage if its interview is genuinely structured. If your questions are vague, it will run a vague interview very consistently. Our structured interview scorecard guide shows how to define criteria and anchors first.
Eight questions to ask any AI interview platform
The UK government's Responsible AI in Recruitment guide, published by the Department for Science, Innovation and Technology in March 2024, organizes buying into pre-procurement, procurement, deployment and live operation. The questions below follow the same idea: ask for evidence, not adjectives.
1. Does it run an interview you control?
You should set the questions, their order and their weights, the knock-out requirements, and how hard the follow-ups push. Ask to see the setup screen, not a slide. If the vendor writes the questions and you can't change them, you are buying their interview, not yours.
2. What exactly does it score, and can you see why?
The score should come from what the candidate said, judged against criteria you set. For every question, you should see the score, the reasoning, and the moment in the recording it refers to. If the report shows more than one score, ask what goes into each.
Walk away from any tool that rates tone of voice, facial expression or "emotion". Since February 2, 2025, the EU AI Act has prohibited AI that infers emotions in the workplace, and the European Commission's guidelines on prohibited practices apply that to recruitment.
3. Who makes the decision?
A person should. Pew Research Center found in 2023 that 71% of US adults opposed AI making the final hiring decision. Ask what the tool does when a candidate fails a knock-out or triggers a review flag. The safe answer is that it marks the candidate for review and never rejects anyone on its own. Our guide to reviewing AI hiring scores explains what good human review looks like.
4. How does it tell candidates it is AI and get consent?
Look at the candidate's first screen. It should say plainly that the interview is run by AI, that it is recorded, what is shared, and that a person decides. In the EU, Article 50 of the AI Act has required providers to make AI interaction clear to people since August 2, 2026. For Illinois-based positions, the AI Video Interview Act requires notice, an explanation of how the AI works and the general types of characteristics it evaluates, and consent before AI evaluates a recorded video interview. The Act is written for video, so whether it reaches audio-only interviews is untested.
5. What bias evidence can the vendor show?
Ask for the most recent bias audit, who ran it, and which groups it covers. If you hire for roles in New York City, Local Law 144 requires an independent bias audit within the past year before you use an automated employment decision tool, plus a published summary and notice to candidates. A vendor can commission that audit, but the employer stays responsible for making sure it exists. The UK guide adds asks worth copying anywhere: bias audits that cover groups including disabled people, accuracy measured across groups, and a commitment to repeat the testing.
6. How does it handle accommodations and accessibility?
Ask what a candidate who can't use the format should do, and how quickly your team can switch them to a human interview. Ask which accessibility standard the candidate screens meet. WCAG 2.2 at level AA is a reasonable baseline. In the US, the EEOC removed its AI technical guidance in January 2025, but the Americans with Disabilities Act still requires reasonable accommodation.
7. Where does candidate data go, and for how long?
Ask where they are stored, who on your team can see them, whether the vendor's AI provider trains on them, and how deletion works. Illinois requires deletion within 30 days of a candidate's request for covered video interviews. In India, the core duties under the Digital Personal Data Protection Rules, including notice, security and retention, are scheduled to apply from May 2027. MeitY proposed in January 2026 to cut that lead time to 12 months, which would bring them forward to November 2026, so confirm the start date before you sign. Our guide to candidate data retention covers the rest.
8. What happens when things break?
Candidates join from home Wi-Fi on whatever computer they have. Ask which browsers and devices are supported, what happens if the connection drops mid-answer, whether a reload resumes the interview, and how your team books another attempt.
Run a pilot before you buy AI interview software
Pick one role with steady volume. Send a random half of its candidates through the AI interviewer and the rest through your current first round for a few weeks, then check four things:
- Agreement. Have two recruiters score a sample of the AI interviews without seeing the tool's scores, and compare. Large, unexplained gaps mean the criteria or the scoring need work.
- Completion. Track how many invited candidates finish, and where the rest drop off.
- Candidate feedback. Add one or two questions after the interview about clarity and fairness.
- Downstream results. When pilot hires have been in the role for a few months, compare their interview scores with how they are doing. One role over a few weeks gives only a rough signal.
As the UK guide recommends, include candidates who use screen readers or need extra time.
Legal dates to plan around
| Date | Rule | What it means for buyers |
|---|---|---|
| Jan 1, 2020 | Illinois AI Video Interview Act | Notice, explanation and consent for AI-analyzed video; deletion within 30 days of a request |
| Jul 5, 2023 | NYC Local Law 144 enforcement | Bias audit and notice for NYC roles |
| Feb 2, 2025 | EU AI Act ban on emotion inference at work | Avoid tools that read feelings |
| Aug 2, 2026 | EU AI Act Article 50 | Vendors must make clear to candidates that they are talking to AI |
| May 2027 (scheduled) | India DPDP Rules core duties | Notice, security and retention for candidate data |
| Dec 2, 2027 | EU AI Act high-risk duties for hiring AI | Human oversight, logs and monitoring for deployers |
The EU's AI omnibus law moved the high-risk date from August 2, 2026. This table is a planning aid, not legal advice. Our AI hiring laws guide has more detail.
How HireRabbit.AI answers these questions
Rabbit, the AI interviewer in HireRabbit.AI, runs a spoken first-round interview from a one-time link. Here is how it answers the eight questions:
- Control. Your team sets the difficulty, the knock-out requirements Rabbit asks first, and weighted questions in your order. Rabbit suggests questions from the job description, and you decide which to use.
- Scoring. The interview score comes only from the candidate's answers. Each answer gets a score, Rabbit's evaluation, and a link to that moment in the recording. The report also shows a separate overall AI score that weighs answers and integrity together, and it is never higher than the interview score.
- Decisions. Rabbit gives a verdict from Strong hire to No hire, and a person makes the call. A failed knock-out ends the interview politely and flags the candidate for review. Flags never reject anyone on their own.
- Consent. Before any device is touched, a consent screen says it is an AI interview, that it is recorded and transcribed, that the screen is shared, and that a person makes every decision.
- Bias testing. Ask us for current testing details before you buy. For NYC roles, plan the independent audit Local Law 144 requires.
- Accommodations. The consent screen tells candidates to contact you first, and your team can move them to a human interview. Candidate screens are built to WCAG 2.2 AA.
- Data. Our AI provider doesn't train on interviews, zero data retention is on, and admins can delete candidates from inside the product. See security.
- Reliability. Candidates join in the latest Chrome or Edge on a computer. A dropped connection reconnects on its own, a reload resumes without losing answers, and your team can book another attempt.
Integrity checks run in the candidate's browser, and frames never leave the device for those checks. Candidates can read exactly what to expect in our guide for candidates.