Skip to content
TailoredMinds

Future of hiringPerspectiveUpdated 5 min read

Human-in-the-loop hiring: where AI helps and where people decide

AI can widen a search and read skills better than a keyword filter. The decisions that change someone's chances should stay with accountable people.

Written by

Lucas Salvaia, Co-Founder & COO

Reviewed by

Tercio Lima, Growth Lead

The short answer

AI is useful in hiring for widening the search, reading skills rather than keywords and summarising long profiles. People should make the decisions that change someone's chances, such as shortlisting, rejecting and making an offer, because they are accountable, they can weigh context the data misses, and regulations such as the EU AI Act require human oversight of AI used in recruitment. Human-in-the-loop hiring means designing the process so that split is explicit.

Key takeaways

  1. Human-in-the-loop hiring means AI assists specific tasks while named people make and own every decision about a candidate.
  2. AI earns its place in search breadth, skills extraction, matching and summarising, where it saves time without deciding anyone's outcome.
  3. Shortlisting, rejection and offers should stay with people: they carry accountability, context and legal responsibility.
  4. The main failure modes are biased rankings, confident but wrong summaries, and people rubber-stamping what the tool suggests.
  5. Developers themselves are cautious: more distrust the accuracy of AI tools than trust it, and most still turn to a person when they doubt an AI answer.
  6. Tell candidates where AI is used and how to reach a person. In several jurisdictions, that is already the law.

What "human in the loop" means in hiring

Human-in-the-loop hiring is a process in which AI tools assist with defined tasks, and a named, trained person makes and is accountable for every decision that affects a candidate. The person must be able to understand the tool's output, disagree with it and override it. If the person only clicks "approve" on whatever the tool ranks first, the loop exists on paper only.

That definition matches where regulation has landed. The EU AI Act classifies AI used to filter applications and evaluate candidates as high-risk, and requires that such systems can be overseen by people who understand their limits, avoid over-relying on them and can override or stop them (Article 14). Employers using them must assign that oversight to people with the necessary competence, training and authority (Article 26) [1]. Those obligations apply from 2 December 2027. Our explainer on the EU AI Act and hiring has the details.

Where AI genuinely helps

AI is most useful in hiring where it widens what people can see, without deciding what happens to anyone.

  • Search breadth. Semantic search finds people whose experience matches the role even when they describe it in different words. A keyword filter misses the engineer who wrote "retrieval pipeline" when you searched "RAG".
  • Skills extraction. Reading a long profile or CV and pulling out skills, tools and the kind of systems someone built, so a person can compare candidates on the same terms.
  • Matching and sourcing. Surfacing candidates from a large talent pool for a reviewer to look at, including people who didn't apply.
  • Summaries and preparation. Drafting a summary of a profile or an interview's notes for a reviewer to check against the source.
  • Admin. Scheduling, reminders and status updates, so candidates aren't left waiting.

In each case, the output is an input to a person's judgement, and the person can check it against the source.

Where people must decide

Any step that changes a candidate's chances should be decided by a person. Three reasons:

  1. Accountability. A hiring decision has to be explainable to the candidate, the team and, sometimes, a regulator. "The model ranked them low" is not an explanation.
  2. Context. People can weigh what the data misses: a career break, an unusual path into engineering, a project described modestly, a reference that changes the picture.
  3. Regulation. In the EU, the GDPR gives people the right not to be subject to a decision based solely on automated processing that significantly affects them, with limited exceptions (Article 22) [2], and the AI Act adds human oversight duties for recruitment AI [1]. In New York City, automated employment decision tools need a bias audit and candidate notice before use [6].

A responsibility matrix

The simplest way to make the split explicit is to write it down, task by task. This is a starting point; adjust it to your process.

TaskAIPersonNotes
Writing the role briefDrafts and suggestsDecidesThe hiring manager owns what "good" means
Searching and sourcingWidens the search, suggests matchesReviews who is worth a lookAI should read skills, not just keywords
Extracting skills from profilesExtracts and structuresSpot-checks against the sourceErrors here propagate everywhere else
Summarising a profileDraftsReads the source before relying on itSummaries can be confidently wrong
ShortlistingCan suggestDecidesThe decision that most shapes outcomes
Rejecting a candidateNever aloneDecides and can explain whyFully automated rejection raises legal questions in the EU
Assessing interviewsCan transcribe or organise notesScores against the rubricNo emotion or personality inference from faces or voices
Making an offerNo roleDecidesIncluding pay, start date and terms
Scheduling and updatesAutomatesOverseesSpeed here helps candidates

Failure modes to design against

Biased rankings

AI ranking tools learn from data, and hiring data carries history. In 2018 Reuters reported that Amazon had scrapped an experimental recruiting tool after finding it penalised CVs that included the word "women's", because it had learned from a decade of mostly male applicants [5]. In 2024 University of Washington researchers tested three open-source language-model-based retrieval systems on CV screening and found they favoured CVs with white-associated names 85% of the time and female-associated names only 11% of the time [4]. These are studies of specific systems, not every tool, but they show why rankings need checking.

Design against it: audit outcomes by group where the law allows, keep people reviewing the shortlist, and never let a ranking silently drop candidates.

Confident but wrong summaries

Language models can produce fluent summaries that are partly invented: a skill the candidate doesn't have, a job title merged from two roles. Design against it: treat summaries as drafts, link every summary to its source and train reviewers to check before relying on it.

Over-trust and rubber-stamping

The more polished an AI output looks, the easier it is to accept without reading. Developers know this well. In Stack Overflow's 2025 survey, more developers distrusted the accuracy of AI tools (46%) than trusted it (33%), and 75% said they would still ask another person for help when they don't trust an AI answer [3]. Design against it: give reviewers time and authority to disagree, and look at how often they do. A reviewer who never overrides the tool may not be reviewing.

What candidates should be told

Candidates should know where AI is used, what it does and how to reach a person. A short, plain notice covers it:

  1. Which steps use AI, for example "to help us search and summarise profiles".
  2. That a person reviews every shortlist and makes every decision.
  3. How to ask for a human review, or a different way to be assessed if they need one.
  4. What data is used and how long it's kept.

In New York City, candidate notice is already required before using an automated employment decision tool [6]. In the EU, the AI Act requires deployers of high-risk recruitment AI to inform the people affected, from 2 December 2027 [1].

How Tailored Minds splits the work

On Tailored Minds, AI assists and people decide. AI helps our team search the talent network more widely and read skills rather than keywords, AI sourcing agents can suggest candidates for a role, and AI-generated talent reports are advisory. A person reviews and curates every shortlist before a company sees it. Companies see only the candidates we've deliberately shared for their role, never a browsable database, and the hiring decision stays with them. Approved independent recruiters can submit candidates too, and those candidates go through the same review.

Talent badges work the same way: Profile complete, then Video verified, then Highly Qualified, which comes from human review and never from an algorithm alone. You can read how we keep quality high, how talent profiles are reviewed on our page for engineers, and how recruiters work on the platform on our page for recruiters. For how hiring models compare more broadly, see in-house, agency, recruiter network or freelance marketplace.

Frequently asked questions

Should AI make hiring decisions?
No. AI can help a hiring team search more widely, extract skills and summarise profiles, but decisions that change a candidate's chances, such as shortlisting, rejection and offers, should be made by accountable people. They can weigh context the data misses and explain the outcome. In the EU, the GDPR limits solely automated decisions with significant effects, and the AI Act requires human oversight of recruitment AI.
What parts of recruiting should be automated?
Tasks that widen what people can see or save them admin time, without deciding anyone's outcome: semantic search and sourcing, skills extraction, drafting summaries for a reviewer to check, and scheduling and status updates. Keep a person on shortlisting, rejections, interview scoring and offers, and make sure they can override any tool's suggestion.
What are the risks of AI screening candidates?
The main risks are biased rankings learned from historical data, fluent summaries that contain errors, and reviewers who accept the tool's output without checking it. Research and reporting have documented ranking systems that disadvantaged women and candidates with names associated with particular racial groups. Regular outcome checks, human review of every shortlist and clear candidate notice all reduce the risk.
What does human oversight of AI in hiring look like in practice?
Named people, trained on what the tool does and where it fails, review its output before any decision and have the authority to disagree. They read the source rather than only the summary, write down the reason for each shortlist or rejection, and can stop using the tool if it misbehaves. If reviewers never override the tool, check whether they are really reviewing.
Do I have to tell candidates that I use AI?
In several places, yes. New York City requires notice before using an automated employment decision tool, Illinois requires notice when AI is used in employment decisions, and the EU AI Act requires deployers of high-risk recruitment AI to inform candidates from 2 December 2027. Even where it isn't required, a short plain notice builds trust. This is general information, not legal advice.

Sources

  1. Regulation (EU) 2024/1689 (Artificial Intelligence Act), Articles 14 and 26 and Annex III, EUR-Lex
  2. Regulation (EU) 2016/679 (General Data Protection Regulation), Article 22, EUR-Lex
  3. 2025 Developer Survey: AI, Stack Overflow
  4. Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval (Wilson and Caliskan, 2024), AAAI/ACM Conference on AI, Ethics, and Society
  5. Amazon scraps secret AI recruiting tool that showed bias against women, Reuters
  6. Automated Employment Decision Tools (Local Law 144), NYC Department of Consumer and Worker Protection

About the author

Portrait of Lucas Salvaia

Lucas Salvaia

Co-Founder & COO at Tailored Minds

Lucas Salvaia is a co-founder and the COO of Tailored Minds. He works closely with engineers through the whole hiring process, from the first conversation and interview preparation to the offer.

More from LucasLinkedIn

Share this article

Put a whole network on your next hire.

Tell us who you're looking for. Our talent network and recruiters take it from there.

Talent? Join freeRecruiter? Join the workforce