The Structured Interview Scorecard
A practical guide to structured interviews: define what good looks like, write anchored rating scales, split questions by interviewer, run fair debriefs.
By the HireRabbit.AI team · Published · Last updated

On this page
- Why structure changes the outcome
- Start with an intake meeting that defines good
- Turn must-haves into four to six rated competencies
- Write behavior-anchored rating scales
- A worked example: customer success manager
- Assign questions per interviewer so rounds don't overlap
- Collect independent written feedback before the debrief
- Run a debrief that doesn't anchor on the loudest voice
- Train interviewers before they rate anyone
- Putting it into your hiring tool
Most HR managers have sat through a debrief like this one. Four people interviewed the same candidate for the same job. One loved her, one "wasn't sure about the energy", one asked mostly about her last company's tech stack, and one ran out of time.
The hiring manager picks someone, and three months later nobody can say why that person won. A structured interview with a written scorecard fixes this. This guide walks through building one, with a worked example for a customer success manager.
Why structure changes the outcome
In 1998, Frank Schmidt and John Hunter's summary of 85 years of selection research put the validity of structured interviews for predicting job performance at .51 and unstructured interviews at .38. In 2022, Paul Sackett and colleagues found that earlier corrections had inflated many of those numbers.
In their revised estimates, structured interviews came out on top at .42, ahead of job knowledge tests (.40), empirically keyed biodata (.38) and general cognitive ability tests (.31). Unstructured interviews dropped to .19. The 80% credibility interval for structured interviews runs from .18 to .66, so a careless build can land near the bottom.
Google's re:Work guide, updated in March 2026, reports that pre-made questions, guides and rubrics saved an average of 40 minutes per interview. Rejected candidates who had a structured interview were 35% happier than those who did not.
Start with an intake meeting that defines good
The US Office of Personnel Management's structured interview guide starts with a job analysis. For most private employers, that means a 45-minute intake meeting between the recruiter and the hiring manager, ideally with one person who does the job today. Bring the job description, but spend the time on the work itself:
- What will this person own in their first 90 days?
- What did the best person in this role do that others didn't?
- Which skills can someone learn here in a few weeks, and which do they need on arrival?
- What would make you reject a candidate who looked great on paper?
Write the answers in plain sentences. "Keeps renewals on track" is too vague. "Spots when a customer's usage drops, finds out why and has a plan in front of the account owner within a week" is something an interviewer can listen for.
Turn must-haves into four to six rated competencies
OPM's guide says a structured interview typically assesses between four and six competencies unless the job is unusual or very senior. To get there, group related items and drop anything another step already checks. Years of experience or a required tool belong in the resume screen, scored with a resume scoring rubric.
Each competency needs a one-sentence definition that everyone signs off on. "Escalation handling: stays calm with an unhappy customer, gets to the real problem and coordinates a fix without overpromising" gives interviewers a shared target. "Communication" on its own does not.
Write behavior-anchored rating scales
A rating scale needs each number defined. On a five-point scale, is a 3 acceptable, average or something else? Different interviewers will answer differently unless the scorecard answers for them.
OPM recommends one range for all competencies, at least three proficiency levels and ideally five to seven, with at least three of them labeled. Make the 3 anchor describe someone you would be happy to hire, so the scale does not drift into "5 means hire".
A 1 to 5 scale with written anchors at 1, 3 and 5 works well for most teams. Interviewers can still give a 2 or a 4 when an answer falls between two anchors.
Good anchors describe what the candidate said or did in their example, without adjectives like "impressive" or traits nobody can observe in an hour. Without anchors, a SHRM article warns, interviewers drift toward rating charisma or perceived culture fit.
A worked example: customer success manager
Say you're hiring a customer success manager for a B2B software company who will own about 40 mid-market accounts and carry a renewal target. The intake produced five competencies. Here are three of its five rows.
| Competency | What a 1 looks like | What a 3 looks like | What a 5 looks like |
|---|---|---|---|
| Escalation handling | Blames the customer or another team. Promised a fix they couldn't deliver. | Calmed the customer, found the actual problem, brought in support or product and kept the customer updated until it closed. | Did all of that, set honest timelines, then changed a process so the same escalation didn't recur. |
| Cross-team follow-through | Passed feedback to product and never checked back. | Logged a customer request with context, followed up and told the customer the outcome, including a no. | Grouped requests across accounts into a case the product team acted on and closed the loop with every affected customer. |
| Prioritizing a book of accounts | Works whichever account is loudest that day. | Explains a sensible way they split time between high-value, at-risk and healthy accounts. | Explains their system, gives an example of changing it when the numbers showed it wasn't working and names what they deliberately let slide. |
None of the anchors mention where the candidate worked or whether the interviewer liked them. Each describes something the candidate reports having done, which rests on the premise OPM describes: the best predictor of future behavior on the job is past behavior under similar circumstances.
Assign questions per interviewer so rounds don't overlap
The most common failure in a multi-round loop is repetition: three interviewers ask a version of the same conflict question, and nobody covers prioritization. Give each interviewer one or two competencies to own, and write their questions for them:
- Recruiter screen (30 minutes): logistics, salary range and motivation, not scored.
- Hiring manager (45 minutes): retention ownership and prioritizing a book of accounts.
- Peer customer success manager (45 minutes): escalation handling, with a behavioral question and a short role play.
- Product or support partner (30 minutes): cross-team follow-through.
- Head of customer success (45 minutes): discovery and value conversations.
Each guide should hold the competency definition, two or three main questions, follow-up probes and the anchors. Use very similar probes for every candidate, as OPM recommends, and keep the questions, their order and the time the same.
Collect independent written feedback before the debrief
Each interviewer writes their ratings and evidence before talking to anyone else about the candidate. OPM's guide has each interviewer rate the candidate right after the interview, without discussion with other panel members, supported by examples of what the candidate actually said.
Set a rule that written feedback is due within a few hours and before anyone sees the rest of the panel's scores. The same idea applies to who sees candidate data in your ATS: an interviewer who hasn't read earlier notes can't be anchored by them.
Ask for a rating and two or three sentences of evidence per competency, plus an overall recommendation, and hold hallway conversations until feedback is in.
Run a debrief that doesn't anchor on the loudest voice
In OPM's process, panels compare notes and ratings, explore the reasons for differences and then reach a consensus. Interviewers should explain the evidence behind their ratings rather than trade scores. A format that works for most teams:
- The recruiter facilitates, not the hiring manager.
- Everyone reads the submitted scorecards first, in silence, for five minutes.
- Discussion goes competency by competency, not interviewer by interviewer.
- Where two ratings differ by two points or more, both interviewers share their evidence.
- The most senior person speaks last on each competency, because the first strong opinion tends to pull the others toward it.
- The decision and its reasons are written down before the meeting ends.
Judge each candidate against the scorecard, not against the last person you interviewed. Comparing candidates with each other can leave you "hiring the best of the worst" when all of them fall below the standard.
Train interviewers before they rate anyone
OPM's guide says interviewer training increases the accuracy of the interview and should cover note-taking and common rating errors: the halo effect, rating everything a 3, rating everyone high or low, and favoring people who seem like you.
Awareness alone is not enough, a SHRM article notes, so plan one afternoon of practice. Play three or four recorded or role-played answers, have everyone rate each independently and discuss the gaps. New interviewers then shadow two interviews and run one with an experienced interviewer watching.
Putting it into your hiring tool
In HireRabbit.AI, you can limit an interviewer to one hiring stage, such as the escalation round your peer interviewer owns. Moving a single candidate to a new stage requires a written reason, and the AI only suggests reasons for a person to use or change.
The AI Interviewer can also run a spoken first-round interview from a link, with a consent screen first, and give the team a recording, transcript and scored report to review alongside the human scorecards. See how these fit together on the product page.