How to Build an AI Recruiting Agent in 2026: A Practical Implementation Guide

Q: Will an AI recruiting agent discriminate against candidates?

It can if built carelessly, which is why the controls are mandatory. Redact bias-linked fields from the matching input, rank on explicit job-related criteria rather than a model fit to past hires, run the four-fifths adverse-impact test continuously and gate every change on it, and keep a human deciding every advance and reject. A well-built agent that is measured for impact can be fairer and more documented than ad-hoc human screening. The measurement is the product.

Q: Can the agent reject candidates on its own?

No. The agent ranks, surfaces, and recommends, including recommending do-not-advance with reasons, but a human makes every consequential decision and clicks every advance and reject. This is a hard rule, both because the EU AI Act's human-oversight requirement demands it and because auto-rejecting saves almost no time while taking on nearly all the legal risk.

Q: Is an AI recruiting agent legal under the EU AI Act and US law?

Recruitment AI is high-risk under the EU AI Act, which triggers obligations for risk management, data governance, logging, human oversight, transparency, and accuracy. In the US, an automated tool causing disparate impact is unlawful under Title VII regardless of intent, and the EEOC has confirmed adverse-impact analysis applies to algorithms. NYC Local Law 144 adds an independent bias audit and candidate notice. Designing to the strictest of these and getting legal sign-off before go-live is how you deploy lawfully.

Q: How is this different from the keyword filter in our ATS?

The keyword filter scores exact string overlap, which is brittle, gameable, biased toward the vocabulary of people already hired, and cannot explain itself. The agent matches on what the work actually was, ranks on explicit criteria, attaches a verifiable reason to every rank, and can be audited for adverse impact. On every axis except setup cost it is the better tool, because it is explainable and measurable where the keyword filter is neither.

Q: How much does an AI recruiting agent cost to run per candidate?

The dominant cost is resume parsing, a real per-document operation run once per candidate. Embedding and matching are cheap per candidate. The model work (the ranking explanation and outreach drafting) is the smallest line, because you only generate explanations for the ranked set a recruiter will see and only draft outreach for advanced candidates. The lever is to spend the expensive reasoning only where a human will look.

Q: What happens to candidates the agent ranks low?

They stay in the pool, visible to recruiters, with their reasons attached. The agent does not reject them; it ranks them. How deep into the ranked pool a recruiter goes for a given req is a human decision, made with the adverse-impact numbers in view. A low rank is information for a human, never an automated outcome.

TL;DR

An AI recruiting agent is not a smarter keyword filter. It is a system that runs the top of the funnel (sourcing, parsing, matching, outreach, scheduling) as a loop, surfaces a ranked, explained shortlist, and hands every consequential decision to a human. The agent expands the funnel; it never closes it.
Hiring is a high-risk use case under the EU AI Act and is squarely inside US EEOC and adverse-impact law. That is not a footnote. It is the constraint that shapes every architecture decision in this guide: explainability, audited matching, and a hard rule that the agent never auto-rejects a candidate.
The gap between a demo and a production system is the parts nobody screenshots: resume parsing across hundreds of formats and 50+ languages, a clean candidate data model, defensible matching, and the audit trail that lets you prove the agent was fair. The outreach email is the easy 5%.
Start with a boring, strong default stack: Finigami DocumentAI for resume parsing, pgvector for the candidate store, structured matching plus a reranker for ranking, Claude for reasoning and drafting, and Langfuse for evals, observability, and the bias dashboard. Add complexity only when a number tells you to.
You cannot ship a hiring system you cannot audit. Build the evaluation harness and the adverse-impact test before you tune anything, because the question you will be asked is not “is it accurate” but “is it fair, and can you prove it.”
The hard part of getting it approved is not technical. It is a one-page business case framed as time-to-fill and recruiter capacity, with a cost ceiling, a quality floor, and a compliance sign-off that legal and the CHRO put their names on.

What an AI recruiting agent is in 2026, and what changed

For a decade “AI recruiting” meant a keyword filter wearing a lab coat. An applicant tracking system scanned resumes for the exact strings in the job description, scored them on overlap, and ranked. It was brittle (it missed a “K8s” candidate searching for “Kubernetes”), it was gameable (candidates white-texted keywords into their resumes), and it was quietly biased (it rewarded the vocabulary of the people who already worked there). It also could not explain itself, which in hiring is the property that matters most.

What changed is that the agent can now read a resume the way a recruiter does, reason about whether a background fits a role, and run the mechanical top-of-funnel work without a human babysitting each step. An AI recruiting agent in 2026 is a loop: it sources candidates, parses their resumes into structured records, matches them against an open role, ranks the matches with reasons attached, drafts outreach, and schedules the interviews a human decides to run. The recruiter stops doing data entry and resume triage and starts doing the part that needs judgment: deciding who advances and who does not.

Three shifts make this possible, and a fourth makes it dangerous if you ignore it.

Parsing got good enough to trust the structure. The single biggest determinant of recruiting-automation quality is whether you turned a messy PDF, a two-column designer resume, a LinkedIn export, or a scanned CV from another country into a clean, structured record. Garbage in, confidently-wrong shortlist out. Modern document-understanding APIs read tables, multi-column layouts, scanned pages, and 50+ languages without per-template configuration. That is the foundation the whole agent stands on, and it is the part teams under-budget by an order of magnitude.

Matching moved from string overlap to meaning, then learned its limits. Embeddings let you match a candidate to a role on what the work actually was, not which keywords happened to appear. A backend engineer who “built the payments ledger” matches “experience with financial systems” even with zero shared keywords. But semantic matching introduced a new failure mode: it learns from your existing workforce, and your existing workforce carries your historical bias. The famous example is Amazon’s experimental recruiting tool, which learned to penalize resumes containing the word “women’s” and downgrade graduates of two women’s colleges, because it was trained on a decade of male-dominated hiring data, and which the company scrapped (Reuters, 2018). Better matching did not remove the bias problem. It relocated it.

The agent loop replaced the pipeline. The earlier generation ran one step at a time with a human between each. The agent runs the mechanical steps end to end (source, parse, match, draft, schedule), pauses at the decisions that carry legal and human weight, and asks a person. The design question stopped being “can the model do this step” and became “which steps is it allowed to finish on its own, and which must it only ever recommend.”

And the regulatory ground shifted under all of it. The EU AI Act classifies AI systems used in recruitment and to evaluate candidates as high-risk (EU AI Act, Annex III), which triggers obligations for risk management, data governance, logging, human oversight, transparency, and accuracy before such a system can be deployed in the EU. In the US, an automated hiring tool that produces a disparate impact on a protected group is unlawful under Title VII regardless of intent, and the EEOC has confirmed that adverse-impact analysis (the four-fifths rule and its successors) applies to algorithmic selection procedures (EEOC technical assistance, 2023). New York City’s Local Law 144 goes further and requires an independent bias audit of any automated employment decision tool before it is used, plus public posting of the results and notice to candidates (NYC DCWP, Local Law 144). This is the difference between this guide and the RAG guide it is modeled on. A wrong RAG answer is embarrassing. A biased hiring decision is a lawsuit, a regulator, and a person who did not get a job they deserved.

So the spine of this guide is two ideas held together. Build a genuinely useful agent that expands the funnel and gives recruiters their time back. And build it so that a regulator, a plaintiff’s lawyer, and a rejected candidate could all look at exactly why the agent did what it did, and find a human accountable at every consequential step.

The anatomy of a recruiting-agent failure

When a recruiting agent produces a bad outcome, it failed at one of five points. Naming them matters, because the fix and the liability are different at each:

Parsing: the candidate’s experience never made it out of the resume intact. A two-column layout scrambled, a non-English CV got dropped, a “Senior Staff Engineer” title was read as “Staff Engineer.” No downstream step can recover what was lost here, and the candidate is silently mis-ranked through no fault of their own.
Data model: the experience survived parsing but landed in the wrong field. A contract role read as full-time, a 2019-2023 tenure read as 2019, a certification dropped because the schema had no slot for it. The record is now subtly wrong and the matching inherits the error.
Matching: the right candidate exists in the pool and is findable, but ranked below the cutoff because the matching logic rewarded the wrong signal (years at brand-name companies, a specific phrasing, a keyword the candidate happened not to use). This is where bias most often hides.
The decision: the ranking was reasonable and a human (or worse, the agent) made a consequential call (reject, do-not-advance) without seeing why, or with a reason that would not survive scrutiny. The agent auto-rejecting here is the failure that ends careers and starts lawsuits.
Bias and compliance: every step above was individually defensible, and the system as a whole still produced a selection rate for one protected group well below another, and nobody measured it. This is the failure that is invisible per-candidate and catastrophic in aggregate.

This guide spends most of its words on points 1, 3, and 5 on purpose, because those are the ones teams skip in the demo and discover in production, or in a deposition. The agent’s job is to make the human at point 4 faster and better-informed. It is never to replace them.

When NOT to automate it (especially the reject decision)

The hard rule first, because it is the one that protects everyone: the agent never auto-rejects a candidate. It can rank, it can surface, it can flag, it can recommend “do not advance” with reasons. A human makes the negative decision, every time. The reason is not squeamishness. An auto-reject is the exact action the EU AI Act’s human-oversight requirement exists to prevent, it is the action most likely to produce an unexamined adverse-impact pattern, and it is the action a rejected candidate is most likely to challenge. You gain almost nothing in recruiter time by automating the reject (a human glancing at a ranked-low candidate takes seconds) and you take on nearly all the legal risk. The trade is bad. Don’t make it.

Beyond that hard rule, skip or descope the agent where these hold:

The role is senior, rare, or relationship-driven. Executive search and senior specialist hiring run on networks, referrals, and judgment, not funnel volume. The agent pays off in roles with many applicants and clear-ish requirements. It adds little to a search where there are nine plausible candidates on the planet and you already know all of them.
You cannot articulate what “qualified” means. If the hiring manager cannot tell you the five things that actually predict success in the role, the agent has nothing defensible to match on, and it will quietly learn to match on proxies (pedigree, keywords, vocabulary) that are exactly what adverse-impact law targets. Fix the job definition first.
The volume does not justify the build. An agent is infrastructure. For a team hiring twenty people a year, a good ATS and a disciplined recruiter beat a system you have to audit quarterly. The agent earns its keep at thousands of applicants, not dozens.
You cannot commit to the oversight. If nobody will own the weekly bias review, the human-gate approvals, and the audit trail, you should not deploy a high-risk system. An unmonitored hiring agent is a liability generating efficiency, which is the worst possible trade.

AI recruiting agent vs the alternatives

Before you build, make sure an agent is the right tool. There are three ways a talent team gets through the top of the funnel, and they sit on a line from “all human” to “agent-assisted,” not in rival camps.

Approach	What the recruiter does	Best when	Weakness
Manual screening	reads every resume, sources by hand, schedules by email	low volume, senior or rare roles, relationship-driven	does not scale; inconsistent; slow time-to-fill; undocumented reasons
ATS keyword filter	writes boolean queries, the ATS scores keyword overlap	high volume, well-defined roles, you already own the ATS	brittle and gameable; rewards vocabulary not ability; hides bias; cannot explain itself
AI recruiting agent	reviews a ranked, explained shortlist and decides who advances	high volume, clear-ish requirements, you can fund the oversight	quality bounded by parsing and matching; demands compliance scaffolding

The honest read of this table: the agent is not strictly better than the alternatives for every team. It is strictly better than the ATS keyword filter on every axis except setup cost, because it matches on meaning, explains its ranking, and can be audited, where the keyword filter does none of those. Against skilled manual screening it is a trade: you gain consistency, speed, documentation, and scale, and you take on the obligation to monitor a high-risk system. For a high-volume role that obligation pays for itself many times over. For a low-volume senior search it does not. Pick by the question the hiring manager actually asks: if it is “help me get through 3,000 applicants fairly and fast,” that is the agent; if it is “find me the three people in the world who can run this,” that is a human with a network. For the executive framing of which HR agents to build first, see which AI agents should I build for HR.

Use cases & where it pays off

A recruiting agent earns its keep when three conditions hold: the applicant volume is high enough that human triage is the bottleneck, the role requirements are clear enough to match against defensibly, and being slow or inconsistent is expensive enough to justify the build and the oversight. The patterns that consistently pay back:

Use case	Why the agent fits	What “good” looks like
High-volume requisitions (support, ops, retail, early-career)	thousands of applicants, clear-ish requirements, speed matters	recruiter reviews a ranked shortlist in an hour instead of a week; faster time-to-fill
JD creation and calibration	every req starts with a job description; consistency is hard	drafts a structured, inclusive JD from a role brief; flags exclusionary language
Resume parsing and enrichment	resumes arrive in every format and language	clean structured records for every candidate, no manual data entry
Candidate-to-role matching	the right person is buried in a large pool	ranked shortlist with a reason per candidate a recruiter can verify
Re-engaging the silver-medalist pool	strong past applicants go cold in the ATS	surfaces prior candidates who fit a new req; warm, personalized outreach
Scheduling and coordination	back-and-forth email eats recruiter hours	interviews booked against real calendars with no human ping-pong

The common thread: the agent does the mechanical, high-volume work and the explainable surfacing; the human owns the judgment. The product is not “a decision.” The product is a faster, better-documented recruiter.

A worked example, one candidate through the pipeline. A req opens for a Backend Engineer. Priya applies through the careers page with a two-column PDF resume; the agent runs it through Finigami DocumentAI and gets back a structured record: five years of experience, a current title of Senior Software Engineer, skills including Go, PostgreSQL, and distributed systems, and a line reading “designed and shipped the payments ledger serving 2M daily transactions.” The matcher embeds her structured experience and scores it against the role’s requirements. She has none of the exact keywords from the JD (“financial services experience,” “high-throughput systems”), but the semantic match is strong: the payments-ledger work is exactly the requirement, phrased differently. She ranks fourth of 1,400 applicants. Attached to her rank is a plain-language reason: “Strong match on distributed-systems and financial-systems experience (payments ledger, 2M daily txns); meets the 4+ years requirement; no direct match on the ‘fintech’ keyword but the underlying work fits.” The recruiter reads the shortlist, sees the reason, opens Priya’s parsed record and her actual resume side by side, agrees, and clicks advance. The agent drafts a warm, role-specific outreach email; the recruiter edits one line and sends. Priya replies; the agent offers three interview slots from the hiring manager’s real calendar and books the one she picks. At no point did the agent reject anyone. The candidates who ranked low are still in the pool, visible to the recruiter, with reasons attached, waiting for a human to decide. Every box in the architecture diagram earned its place in that one candidate’s path.

The reference architecture

A production recruiting agent is two halves joined by a gate. There is a candidate-ingestion path that turns applicants and sourced profiles into clean, structured, audited records, and there is a per-candidate agent loop that matches, ranks, explains, and drafts. They meet at the candidate store, and the loop hands every consequential decision through a human review gate before anything irreversible happens. Most failed recruiting-automation projects over-invest in the matching model and under-invest in parsing and the audit trail, but the quality ceiling is set upstream, when resumes are parsed, and the legal defensibility is set at the gate.

The recruiting-agent reference architecture: a candidate-ingestion path and a per-candidate agent loop meeting at the candidate store, with a mandatory human gate before any ATS write-back, and bias, fairness, and audit logging spanning the whole system.

Read the diagram left to right, top then bottom. The top lane runs when a candidate enters the pool (an application, a sourced profile, a re-engaged silver medalist). The bottom lane runs when a role needs a shortlist. The two boxes that decide your outcome are Resume parsing (top), which sets your quality ceiling, and the Human gate (bottom), which sets your legal defensibility, and both are the ones teams shortcut in the demo. The band across the bottom is not decoration. Bias testing, explainability, an immutable decision log, and a guarded data perimeter are first-class components of a high-risk system, not features you add before launch.

Architecture decisions

Eight decisions determine most of the outcome. Make them deliberately; everything else is tuning. In a hiring system, several of these are not just engineering calls. They are the controls a regulator will ask you to show.

The autonomy boundary: where the agent stops. This is the decision that defines the whole system, and it has a non-negotiable answer for the consequential cases. The agent may finish mechanical, reversible steps on its own (parse a resume, compute a match, draft an email, propose calendar slots). It may only recommend, never finalize, any consequential step (advance, reject, do-not-advance). And it must never auto-reject. The boundary is not a slider you tune for efficiency; it is the line the EU AI Act’s human-oversight requirement draws for you. Set it once, encode it so it cannot be bypassed, and audit that it holds.

Step	Agent may	Human must
Parse resume, build record	finish	review on exception
Match and rank	finish (it is a recommendation)	nothing
Explain the ranking	finish	read it
Advance a candidate	recommend	decide and click
Reject / do-not-advance	recommend with reasons	decide and click, always
Draft outreach	finish (draft)	approve before send
Schedule interview	finish (once advanced)	nothing

Where resumes get parsed. This is the decision that silently sets your quality, exactly as it does in RAG. Naive text extraction scrambles two-column resumes, drops scanned CVs, mangles tables of skills, and reads dates and titles wrong. For a global candidate pool that is fatal: a mis-parsed resume mis-ranks a real person. Use a real document-understanding API. We default to Finigami DocumentAI: it is API-first, template-agnostic, handles the hundreds of resume layouts you will actually see, reads scanned and photographed CVs, and covers 50+ languages without per-format configuration, which matters the moment your applicants are not all using the same English-language template. Alternatives: an ATS-native parser (convenient, often weaker on non-standard layouts and non-English resumes) or AWS Textract (raw OCR you then have to structure yourself).
How you match and rank. The choice runs from pure keyword overlap (brittle, gameable, biased) through semantic similarity (matches on meaning) to structured matching that scores explicit, defensible criteria. The right answer for hiring is hybrid and explicit: embed the candidate’s experience for semantic recall, but rank on a transparent set of role criteria (required skills present, years of relevant experience, must-have qualifications) so that every score decomposes into reasons you can show and audit. A pure black-box similarity score is a liability in a high-risk system; you cannot explain it and you cannot defend it.

Matching approach	Strength	Why it is wrong alone for hiring
Keyword overlap	exact, fast, simple	brittle, gameable, rewards vocabulary not ability
Semantic similarity	matches on meaning, catches paraphrase	opaque score; learns workforce bias; cannot explain a rank
Structured criteria + semantic recall	explainable, auditable, catches meaning	more to build; requires a defined “qualified”

The reasoning and drafting model. Default to Claude: Opus 4.8 for the matching explanations and any nuanced judgment where faithfulness to the evidence matters, Sonnet 4.6 for high-volume drafting (outreach, JD first drafts) where cost and latency matter more. The model is not the riskiest part of the system; matching and the gate are. But it is where explainability lives, so test one behavior explicitly: does the explanation it writes actually reflect the structured score, or does it post-hoc rationalize? An explanation that does not match the math is worse than none. Alternatives: an open model (Llama, Mistral) for cost or data-residency reasons, with the same faithfulness test applied harder.
Explainability: how every rank gets a reason. This is a hard requirement, not a feature. Every candidate’s position in the ranking must decompose into human-readable reasons tied to the role’s stated criteria (“meets 4+ years; strong match on distributed systems; no match on the ‘fintech’ keyword but the payments-ledger work fits”). Build the explanation from the structured score, not as a free-form LLM narration of a black box, so the explanation is faithful by construction. A candidate or a regulator must be able to read why, and a recruiter must be able to check it against the actual resume.
The human gate: how the agent hands off. Every consequential decision routes through a recruiter who sees the ranking, the reason, the parsed record, and the original resume together, and clicks. Encode the gate so the agent literally cannot write an advance, a reject, or an outreach send to the ATS without a human action recorded against it. The gate is not a UI nicety; it is the control that makes the whole system compliant, and it must be impossible to bypass under load.
Where bias-linked signals get removed. Decide up front which fields the matcher is allowed to see. Name, photo, address, age proxies (graduation year), names of women’s or HBCU institutions, gendered pronouns: redact them from the matching input at the data-model step so the matcher physically cannot learn on them. You cannot bias on a field the matcher never saw. This is the cleanest defense against the Amazon failure mode, and it is far cheaper to design in now than to retrofit.
The candidate store and embeddings. Start with pgvector unless you have a reason not to. It keeps candidate vectors next to the structured record, the metadata, the access tags, and the audit trail, in one system you can query with one SQL WHERE clause (“this req, this region, consent on file, not redacted out”). It supports HNSW indexes for fast search and means one fewer system to operate and secure, which matters a great deal when the data is candidate PII. Reach for a dedicated store (Pinecone, Qdrant) only when you cross millions of candidates or need very high QPS, and weigh that against the data-perimeter cost of candidate data leaving Postgres.

The reference stack

The boring, strong default. Start here; deviate only when an eval or an audit tells you to.

Layer	Default pick	Why	Swap when
Resume parsing	Finigami DocumentAI	API-first, template-agnostic, scanned + 50+ languages, any layout	clean structured ATS feed only
Candidate data model	explicit schema + redaction at ingest	matcher can’t bias on a field it never saw	-
Embeddings	Voyage / OpenAI `text-embedding-3-large`	strong retrieval on skills + experience text	domain-specific → specialized model
Candidate store	pgvector	vectors next to record, metadata, audit, access tags	millions of candidates → Pinecone/Qdrant
Matching	structured criteria + hybrid recall + reranker	explainable and auditable, catches meaning	-
Reasoning / drafting	Claude Opus 4.8 / Sonnet 4.6	faithful explanations; strong drafting	cost/residency → open model, test faithfulness harder
Outreach + scheduling	agent tools over ATS + calendar APIs	books real time, no human ping-pong	-
ATS integration	Greenhouse / Ashby / Workday API	the system of record; write-back is gated	your ATS → its API or a Merge.dev unified layer
Eval / observability / bias	Langfuse + offline test sets	traces, scores, and the adverse-impact dashboard in one place	enterprise → Braintrust + a dedicated audit store
Orchestration	thin app code + a small agent loop	a recruiting agent is a gated pipeline, not an open agent	genuinely multi-step branching → LangGraph

A note on the ATS: it is your system of record and your audit anchor, so the integration is not optional plumbing. Name a concrete target (Greenhouse, Ashby, and Workday all expose APIs for candidates, applications, and scheduling) and decide early whether you integrate directly or through a unified-API layer like Merge.dev when you must support several ATSs across business units. Either way, the write-back is the thing the human gate protects, so it is the last step in the loop and the most carefully logged.

A note on frameworks: you do not need a heavy agent framework. A recruiting agent is a gated pipeline with a few tools (parse, match, draft, schedule, write-back), not an open-ended autonomous agent roaming a tool space. A few hundred lines of well-tested application code with an explicit gate is more debuggable, more auditable, and more defensible than a framework whose control flow you do not fully own. Reach for orchestration when the workflow becomes genuinely branching, not before.

Cost and latency, end to end

Two numbers decide whether a recruiting agent is viable: what each candidate costs to process and how long a recruiter waits for a shortlist. Neither is dominated by the model in the way people assume. A rough per-candidate profile for a well-built system:

Stage	Latency (typical)	Cost driver
Parse resume (Finigami DocumentAI)	~1–4 s per document	per-document parse, the largest single line
Build + redact structured record	< 100 ms	in-process, negligible
Embed candidate experience	~20–50 ms	a few hundred tokens, negligible
Match + rank against role	~50–300 ms	database + reranker, small per candidate
Generate the ranking explanation	~300 ms – 1 s	model call, input is the record + criteria
Draft outreach (advanced candidates only)	~500 ms – 2 s	model call, only for the shortlist, not everyone
Total per candidate	a few seconds	parsing dominates cost; the model is cheap per candidate

Read it as a budget. The expensive line is parsing, because it runs once per candidate and is a real document operation, not a token count. The model work is cheap per candidate, because you only generate explanations for the ranked set and only draft outreach for the few a human advances. That is the lever: do the expensive reasoning on the shortlist, not the whole pool. A system that asks the model to write a paragraph about all 1,400 applicants pays for 1,400 paragraphs nobody reads; a system that ranks cheaply and explains only the top 40 the recruiter will actually see pays for 40. Precision in where you spend tokens is both cheaper and a better experience, the same alignment the RAG reranker buys.

Implementation

The build, step by step, with the decisions that bite at each one.

Step 0: Scope the role and define “qualified”

Before any code, pick one high-volume role and write down, with the hiring manager, the five-to-eight things that actually predict success in it. Required skills, minimum relevant experience, must-have qualifications, and the nice-to-haves, each phrased as something you can check against a parsed resume. This is not busywork. It is what makes the matcher defensible: every criterion becomes a column in the score and a clause in the explanation. It is also the adverse-impact guardrail, because a criterion you can articulate and justify as job-related is defensible, and a vague “culture fit” or “top-tier school” is exactly the proxy that produces disparate impact. If the hiring manager cannot name the criteria, stop. You cannot build a fair matcher for a role nobody can define.

This step also produces your first evaluation asset: a small set of past candidates for similar roles, labeled by recruiters as strong / borderline / weak, which becomes the golden set you calibrate the matcher against.

Step 1: Source candidates into the pool

Candidates enter from three places: inbound applications through your careers page and ATS, active sourcing (searching profiles against the role), and re-engagement of the silver-medalist pool (strong past applicants who fit a new req). The agent’s job at this step is to normalize all three into the same intake path so that everything downstream treats a sourced profile and an inbound application identically. Two rules. Capture consent and source at intake: record how each candidate entered the pool and whether they consented to processing, because you will need both for compliance and for outreach that is not spam. And do not let sourcing become scraping you cannot defend: respect platform terms and candidate-data law, because a sourcing channel that violates GDPR is a liability that no amount of funnel volume justifies.

Step 2: Parse resumes into structured, redacted records

Everything downstream inherits the quality of this step, and the fairness of the system starts here too. Run every resume through Finigami DocumentAI and keep the structure: contact block, work history with titles and dates, skills, education, certifications. Then build a structured record, and at the same moment redact the fields the matcher must not see.

Two rules, one for quality and one for fairness. First, preserve structure and provenance: a title is a title, a date range is a date range, a skills table is not flattened into prose, and every field carries a pointer back to where in the resume it came from so a recruiter can verify it. Second, redact bias-linked fields from the matching input: name, photo, address, graduation year and other age proxies, gendered pronouns, and the names of institutions that signal a protected characteristic. The recruiter sees the full resume at the gate; the matcher sees only the job-related fields. You cannot bias on a field the matcher never received.

# Parse with a document-understanding API; keep structure AND provenance.
# (Calls are schematic. See the Finigami DocumentAI docs for exact endpoints.)
doc = documentai.parse(file="applicants/priya-resume.pdf")   # 2-column, scanned, any language

record = {
    "candidate_id": new_id(),
    "source":       "inbound",                 # inbound | sourced | pool
    "consent":      True,                       # captured at intake (Step 1)
    "experience": [
        {                                       # one entry per role, structure preserved
            "title":      r.title,              # "Senior Software Engineer", read correctly
            "employer":   r.employer,
            "start":      r.start, "end": r.end,# real date range, not a single year
            "summary":    r.as_markdown(),      # the bullet text, kept intact
            "source_span": r.span,              # pointer back into the resume for the recruiter
        } for r in doc.work_history
    ],
    "skills":        doc.skills,                # from a skills table, not guessed from prose
    "education":     doc.education,
    "certifications": doc.certifications,
}

# Build the MATCHING view by stripping bias-linked fields the matcher must never see.
match_view = redact(record, drop=[
    "name", "photo", "address", "graduation_year",  # age/identity proxies
    "pronouns", "institution_signal",               # gender/identity proxies
])

Those redaction choices are the cleanest defense against learned bias, and they are a decision a compliance reviewer will want to see documented. The recruiter, at the gate, gets the full record and the original resume. The matcher gets match_view. The two are kept distinct on purpose.

Step 3: Build the candidate data model and index it

Write the structured record to pgvector: the explicit fields for filtering and explanation, an embedding of the job-related experience text for semantic recall, the access tags, the consent flag, and the audit fields. Store the matching view’s text (what the matcher actually embedded and scored) separately from the full record (what the recruiter sees), so you can later prove exactly what the matcher did and did not have.

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE candidates (
    id            bigserial PRIMARY KEY,
    candidate_id  text         NOT NULL,
    req_id        text,                          -- the role this record is being matched to
    source        text         NOT NULL,         -- inbound | sourced | pool
    consent       boolean      NOT NULL DEFAULT false,
    match_text    text         NOT NULL,         -- the REDACTED, job-related text the matcher saw
    full_record   jsonb        NOT NULL,         -- the complete record the recruiter sees at the gate
    embedding     vector(1024),                  -- embedding of match_text only
    skills        text[]       DEFAULT '{}',     -- explicit, for structured scoring + filtering
    years_exp     numeric,                       -- explicit, defensible criterion
    access_tags   text[]       DEFAULT '{}',     -- enforced at query time
    retain_until  date,                          -- data-retention limit (Step on security)
    indexed_at    timestamptz  DEFAULT now()
);

-- Approximate nearest-neighbour index for semantic recall.
CREATE INDEX ON candidates USING hnsw (embedding vector_cosine_ops);

-- Keyword index for exact must-have skills (the BM25 half of hybrid matching).
CREATE INDEX ON candidates USING gin (to_tsvector('english', match_text));

Two design notes. match_text and full_record are deliberately separate: the matcher embeds and scores match_text (redacted), and the recruiter at the gate reads full_record (complete). Keeping the matching input as a stored column is what lets you later answer “what did the matcher actually consider for this candidate,” which is a question a regulator or a litigant will ask. And retain_until is on the row because candidate data has a legal shelf life; you delete on a schedule, you do not hoard resumes forever.

Step 4: Match and rank against the role

On each req: pull the role’s structured criteria from Step 0, retrieve candidates with hybrid recall (semantic search over experience plus exact-match on must-have skills), then score each candidate on the explicit criteria and rank. The score is a transparent sum of job-related components, not a single opaque number, because every component must become a line in the explanation.

def rank_candidates(req, k=400, final_k=40):
    criteria = req.criteria               # from Step 0: required skills, min years, must-haves
    q_vec    = embed(req.role_description)
    # Hybrid recall over the REDACTED match_text only, respecting access + consent.
    pool = sql("""SELECT id, candidate_id, match_text, skills, years_exp
                  FROM candidates
                  WHERE req_id = %(req)s AND consent = true
                    AND access_tags && %(tags)s
                  ORDER BY embedding <=> %(qv)s LIMIT %(k)s""",
               req=req.id, qv=q_vec, tags=req.recruiter_tags, k=k)

    scored = []
    for c in pool:
        # Transparent, decomposable score: every term is a defensible, job-related criterion.
        components = {
            "skill_match":   skill_overlap(c.skills, criteria.required_skills),
            "experience":    meets_min_years(c.years_exp, criteria.min_years),
            "semantic_fit":  rerank_score(req.role_description, c.match_text),  # cross-encoder
            "must_haves":    all_present(c, criteria.must_haves),
        }
        scored.append((c, weighted_sum(components, criteria.weights), components))

    ranked = sorted(scored, key=lambda x: -x[1])[:final_k]
    return ranked                          # (candidate, score, components): components drive the reason

The shape that matters: recall wide and cheap, then score on explicit criteria you can defend, and carry the components forward because they are the explanation. Never collapse the rank into a bare number. A score a recruiter cannot decompose is a score a regulator will not accept. Tune the weights against the golden set from Step 0, and run the adverse-impact test (the deep-dive section below) on the resulting ranking before you trust it.

Step 5: Explain every rank, then hand to the human gate

For each ranked candidate, build a plain-language explanation from the components, not as a free narration of a black box. Then present the shortlist to the recruiter with the rank, the reason, the parsed record, and the original resume together, and let the recruiter decide. The agent recommends; the human acts.

The explanation is faithful by construction because it is generated from the actual score components:

Build the explanation from the score components, in plain language.
For each candidate, state:
- the criteria they meet, with evidence from the resume (e.g. "8 yrs > 4 yr minimum")
- the strong semantic matches, named (e.g. "payments-ledger work fits 'financial systems'")
- the gaps, named honestly (e.g. "no direct match on 'fintech'; underlying work fits")
Do NOT invent a reason the score does not support. Do NOT mention any redacted field.
Output is a recommendation for a human, never a decision.

A correct explanation then reads like this, every claim checkable against the resume the recruiter has open:

Rank 4 of 1,400. Meets the 4+ years requirement (5 yrs). Strong match on distributed systems and financial-systems experience (designed the payments ledger, 2M daily transactions) [resume p.1, Senior Software Engineer]. Has Go and PostgreSQL, both required [skills section]. No direct match on the “fintech” keyword, but the payments-ledger work is the underlying requirement. Recommend advancing for a screen.

The recruiter reads it, opens the cited resume section, agrees or disagrees, and clicks. If they disagree, that disagreement is logged and becomes calibration data. At no point does the agent advance, reject, or send anything. The next diagram shows exactly where the gate sits in the per-candidate flow.

The per-candidate flow. Everything left of the gate is the agent recommending; everything right of it happens only after a human clicks. The do-not-advance path keeps the candidate in the pool with reasons logged, never a silent automated rejection.

Step 6: Draft outreach and schedule (for advanced candidates only)

Once a recruiter advances a candidate, the agent drafts role-specific outreach (warm, personalized from the parsed record and the role, not a mail-merge) and presents it for the recruiter to approve and send. On a reply, the agent offers real open slots from the hiring manager’s calendar and books the chosen one, writing the interview back to the ATS. This is the part that feels like magic in a demo and is genuinely useful in production, because it removes the scheduling ping-pong that eats recruiter hours. Keep the same discipline: the draft is a draft until a human approves it, and the write-back is gated and logged.

Deep dive: resume parsing and the candidate data model

This is the foundation, and the part teams under-budget by an order of magnitude, exactly as parsing is in RAG. The difference is that here a parsing error does not just produce a worse answer; it mis-ranks a real person who applied for a job, and it does so invisibly.

Why resume parsing is genuinely hard. Resumes are the least standardized documents in the enterprise. They arrive as single-column and two-column PDFs, designer layouts with sidebars, Word documents, scanned images, photographs taken on a phone, LinkedIn PDF exports, and CVs in dozens of languages and date conventions. Naive text extraction reads a two-column layout by interleaving the columns into nonsense, drops a scanned page entirely, flattens a skills table into a run-on sentence, and reads “Jan 2019 – Mar 2023” as the single year 2019. Each of those is a candidate silently mis-represented to the matcher. The candidate did nothing wrong and never finds out. This is why the parsing engine is not a commodity choice. Finigami DocumentAI handles the layouts, the scans, and the 50+ languages without per-template configuration, which is the property that matters when your applicant pool is global and nobody used the same template. The alternative most teams reach for, the ATS-native parser, is convenient and tends to be weakest on exactly the non-standard layouts and non-English resumes where mis-parsing does the most quiet damage.

The candidate data model, designed for fairness. The structured record is where you make two decisions that shape the whole system. First, what fields you extract: title, employer, date ranges (as ranges, with the gap arithmetic done right so a five-year tenure is not read as one year), skills (from skills sections and inferred from experience, kept distinct), education, certifications, and a pointer from each field back into the resume so a recruiter can verify it. Second, and more important, what fields the matcher is allowed to see. Build two views from the one record: the full record (everything, shown to the recruiter at the gate) and the match view (job-related fields only, redacted of name, photo, address, age proxies like graduation year, gendered pronouns, and institution signals tied to a protected characteristic). The matcher embeds and scores the match view. The recruiter reviews the full record. This separation is the single most effective structural defense against learned bias, because the matcher physically cannot weight a field it never received.

Two failure modes to design against. The mis-parse that mis-ranks: a wrong title or a truncated tenure pushes a strong candidate below the cutoff, and because the agent never auto-rejects, the candidate survives in the pool, but they are ranked low for a reason that is not their fault. The mitigation is an exception path: when parsing confidence is low (an unusual layout, a low-quality scan), flag the record for a human to verify rather than ranking on a record you do not trust. The fabricated field: the model “helpfully” infers a skill or a seniority the resume does not support. The mitigation is to ground every extracted field in a source_span pointing at the resume, and to test for hallucinated fields in the eval. A candidate record is evidence, and evidence has to be traceable to the document it came from.

The record is the contract between parsing and everything downstream. Get it right and the matcher has clean, fair, verifiable input. Get it wrong and every later step inherits an error nobody can see.

Deep dive: matching, ranking, and why you must not over-automate the reject

Matching is where the agent earns its keep and where bias most often hides, so the design has two jobs at once: rank well, and stay defensible.

Why pure semantic matching is not enough on its own. Embedding a candidate’s experience and scoring cosine similarity to the role is a real improvement over keyword overlap, because it matches on what the work was rather than which words appeared. But a bare similarity score has two problems in a hiring context. It is opaque: you cannot tell a recruiter, a candidate, or a regulator why candidate A outranked candidate B, because the answer is “the vectors were closer,” which is not a reason a human can check or a court will accept. And it learns from your data: if you train or tune any part of the ranking on your past hires, it learns to reproduce who you have hired before, which is precisely the Amazon failure mode where a tool learned to penalize the word “women’s” because the training data was a decade of male-dominated hiring (Reuters, 2018). Better matching relocated the bias problem from keywords to embeddings; it did not solve it.

So rank on explicit, defensible criteria. The matching score in this guide is a transparent weighted sum of job-related components defined in Step 0: required skills present, minimum relevant experience met, must-have qualifications present, and a semantic-fit term from the reranker that catches paraphrase. Every term is something a hiring manager can justify as job-related and a recruiter can verify against the resume. The semantic recall does the work of finding the right people in a large pool; the explicit criteria do the work of ranking them in a way you can explain and audit. This is the difference between a matcher you can defend and a black box you cannot.

Now the central rule, stated plainly: you must not over-automate the reject decision. The agent ranks, surfaces, and recommends. It may recommend “do not advance” with reasons. It never finalizes a rejection. Three reasons, in order of weight. Legally, an automated rejection is the action the EU AI Act’s human-oversight requirement is written to prevent, and the action most likely to produce an unexamined adverse-impact pattern under US law. Practically, you save almost no recruiter time by automating the reject, because a human glance at a low-ranked candidate is seconds, so you take on the entire legal downside for a negligible efficiency gain. Morally, a rejection is a consequence in a person’s life, and a person should own it. The do-not-advance path keeps the candidate visible in the pool with reasons attached, so the decision is a human’s, it is documented, and it is reversible. An auto-reject is none of those things. The whole architecture exists to make the human at the gate fast and well-informed, not to remove them.

One more discipline: rank low does not mean rejected. A candidate ranked 900th of 1,400 is not rejected by the agent; they are ranked 900th. Whether they are contacted is a human decision about how deep into the ranked pool to go for this req, made with the adverse-impact numbers in view. Conflating “ranked low” with “rejected” is how teams accidentally build the auto-reject they swore they would not. Keep the ranking and the decision separate, always.

Deep dive: bias, fairness, and compliance

This is the section that decides whether you have a product or a lawsuit. Hiring is a high-risk use case, and the obligations are not optional. Treat this as the constraint the rest of the system was designed around, because it is.

The legal frame, concretely. Three regimes apply, and a system used to hire in the US and EU is subject to all three at once.

Regime	What it requires	What it means for the build
EU AI Act	recruitment AI is high-risk (Annex III): risk management, data governance, logging, human oversight, transparency, accuracy before deployment	the human gate, the audit log, the documented criteria, and the bias testing are obligations, not features
US Title VII / EEOC	an automated tool causing disparate impact on a protected group is unlawful regardless of intent; adverse-impact analysis applies to algorithms	you must run adverse-impact tests on the ranking and be able to show job-relatedness for your criteria
NYC Local Law 144	independent bias audit of an automated employment decision tool before use; public posting; candidate notice	you need an external audit, published results, and candidate notification baked into the rollout

The EU AI Act’s classification of recruitment and candidate-evaluation systems as high-risk (EU AI Act, Annex III) is the anchor. The EEOC has confirmed that adverse-impact analysis applies to algorithmic selection procedures, not just human decisions (EEOC, 2023). And NYC Local Law 144 requires that independent bias audit and candidate notice before such a tool is used in hiring for a role in the city (NYC DCWP). Other jurisdictions are converging on similar requirements, so designing for the strictest of these is the cheapest long-run path.

Adverse-impact testing, the four-fifths rule. The core US test is the four-fifths (80%) rule: if the selection rate for any protected group is less than 80% of the rate for the most-selected group, that is evidence of adverse impact that you must be able to justify by job-relatedness or fix. For an AI matcher, “selection rate” is the rate at which each group passes whatever threshold you use (top-N ranked, advanced past the gate). You run this test on the agent’s output, by group, continuously, not once at launch. This requires demographic data, which you collect separately and voluntarily for monitoring only, never as a matching input, and which the matcher never sees.

def four_fifths_test(decisions, group_labels):
    # decisions[i] = True if candidate i was selected (e.g. ranked top-N / advanced)
    # group_labels[i] = the candidate's protected-group label (held separately, never a match input)
    rates = {}
    for g in set(group_labels):
        idx = [i for i, lbl in enumerate(group_labels) if lbl == g]
        rates[g] = sum(decisions[i] for i in idx) / len(idx)   # selection rate for the group
    best = max(rates.values())
    findings = {}
    for g, r in rates.items():
        impact_ratio = r / best if best else 0.0
        findings[g] = {
            "selection_rate": round(r, 3),
            "impact_ratio":   round(impact_ratio, 3),
            "flag":           impact_ratio < 0.8,   # below four-fifths → investigate, do not ship
        }
    return findings

A flag: True for any group is a stop sign, not a tuning knob. It does not automatically mean the system is unlawful (a job-related criterion can sometimes justify a difference), but it means you do not ship or expand until you have either found and fixed the cause or documented a defensible job-relatedness justification, reviewed by legal. Run this per req and in aggregate, and chart it over time, because impact can drift in even when each individual change looked fine.

Explainability as a compliance control, not a nicety. Every rank decomposes into job-related reasons (the components from the matcher). That is what lets you answer a candidate’s “why was I not advanced,” a regulator’s “show me how this works,” and an auditor’s “demonstrate the criteria are job-related.” Build the explanation from the score, never as a free narration, so it is faithful by construction. An explanation that does not match the math is a liability dressed as transparency.

Human oversight as the structural guarantee. The gate is the EU AI Act’s human-oversight requirement made concrete. It is not enough to have a human “in the loop” in principle; the human must be able to understand the recommendation, override it, and own the decision, and the system must be built so they cannot rubber-stamp without seeing the reason and the resume. The never-auto-reject rule is the sharpest expression of this. If you remember one thing from this guide, remember that the human gate and the adverse-impact test are the two controls that turn a risky automation into a defensible one.

What you owe candidates. Notice that an automated tool is part of the process, the chance to request a human review (which you already provide by design), and, where law requires, accommodation and an explanation. Build candidate notice and consent into the application flow from the start. It is far cheaper than retrofitting it after a complaint, and in several jurisdictions it is the difference between lawful and not.

Security, access control & the data perimeter

A recruiting agent handles some of the most sensitive personal data an enterprise holds: resumes, contact details, work histories, and (collected separately, for monitoring) demographic data. Security is a design constraint from line one, not a phase you bolt on before launch, and candidate-data law makes the perimeter a legal boundary, not just an engineering one.

Access control belongs in the query. A recruiter should see candidates for their own reqs, not the whole pool, and a hiring manager should see the shortlist for their role, not everyone’s. Store each candidate’s access tags on the row and filter inside the SQL (the access_tags && %(tags)s clause in Step 4), so a user can never retrieve, and therefore the agent can never surface, a candidate they are not authorized to see. Never rely on the model to keep a boundary it was handed; the only safe place to enforce access is before retrieval returns the row.

Decide your data perimeter explicitly. Three things can leave your environment in a recruiting agent unless you stop them: resume text and PII (to the parser and the model), candidate embeddings, and the prompts that carry candidate data. For each, decide what is allowed to go where, and write it down. Most enterprises land on: parsing and storage inside their own controlled environment, and a managed model under a zero-retention agreement, so candidate data is processed but not retained by the model provider. Finigami DocumentAI’s API-first model fits a perimeter where you control the data flow, and pgvector keeps candidate records in your own database. Redact the most sensitive fields (and the bias-linked ones) before anything leaves for the matcher, so the resume that goes to the model is already stripped of what the matcher must not see. For the executive version of this conversation, see MCP and your data perimeter.

Retention is a legal limit, not a preference. Candidate data has a shelf life set by law and by the consent you captured. Store a retain_until per record, delete on a schedule, and treat “we kept every resume forever in case it is useful” as the liability it is. GDPR, CCPA, and their successors all give candidates rights over their data (access, deletion, knowing how it is used), and you cannot honor a deletion request for data you scattered across five systems. Keep candidate data in as few places as possible, which is another reason the single-store pgvector default is more than a convenience.

Audit what the system did, immutably. You already log every decision at the gate, every recommendation, the criteria, the score components, and what the matcher saw. In a high-risk system that log is not optional; it is the evidence that lets you answer a regulator, a litigant, or a candidate. Make it immutable and retained for the legally required period, and build redaction into it so the audit store itself respects the perimeter. Adding this after an incident is the expensive path; designing it in is the cheap one.

Complexity management

Recruiting agents rot in a predictable way: every recruiter request looks like it needs a new automation, every quality gap looks like it needs another model call, and a year later you have a sprawling system nobody can audit, which in a high-risk domain is worse than a sprawling system nobody can debug. Resist it.

Defer every step past the gate until the human is comfortable. The instinct is to automate more of the recruiter’s clicks. Don’t, until the recruiter trusts the recommendation enough that the click is genuinely a formality, and even then keep the consequential ones. Auto-advancing or auto-rejecting to “save a click” trades the entire compliance story for seconds.
One change at a time, measured against the eval and the adverse-impact test. Tuning a matcher is empirical. If you change the weights and the embedding together and the ranking shifts, you have learned nothing about which one helped and nothing about whether either introduced impact. Change one variable, re-run both the quality eval and the four-fifths test, record the delta.
Keep the pipeline linear and the gate singular. Every branch (“if the req is senior, skip the gate”) doubles your test surface and is exactly where a compliance hole opens. There is one gate, and every consequential decision goes through it.
Version the criteria and the matcher together. A change to the role criteria or the weights produces a different ranking that will score and impact differently. Treat the criteria-plus-weights as a versioned artifact tied to the reqs it ran on, so you can prove what logic ranked a given candidate on a given day, and roll back a change that introduced impact.

A concrete version of how this goes wrong: a team, under pressure to clear a backlog, adds an “auto-advance anyone scoring above 0.9” shortcut to skip the gate for obvious matches. It saves a few recruiter-hours a week. Six months later an audit finds that the 0.9 threshold advanced one group at a meaningfully lower rate than another, the shortcut bypassed the human oversight the gate provided, and there is no record of a human reviewing any of those advances. The few hours saved cost a remediation, a disclosure, and a rebuild. The gate was load-bearing. It always is.

The discipline is the same one that keeps any system maintainable, with higher stakes: the cost of an automation is not building it, it is owning and auditing it forever. For the executive framing, see the build-vs-buy decision for AI.

Evaluation & quality

This is the section that decides whether you have a product or a science fair project, and in hiring it decides something more: whether you can prove the system is fair. Build the evaluation harness and the adverse-impact test before you tune anything. Without them you are adjusting weights and asking the room whether the shortlist “feels better,” which in a high-risk system is not just sloppy, it is indefensible.

A recruiting agent has three evaluation surfaces, and you must measure all three separately, because the fixes live in different parts of the system and the third is a legal requirement:

Matching quality: did the right candidates rank near the top? Measure against the golden set from Step 0 (candidates recruiters labeled strong / borderline / weak): does the agent rank the strong ones above the weak ones? Use ranking metrics (precision@k, the rate at which a recruiter agrees with the top-N). When matching quality is the problem, the fix is upstream: parsing, the criteria, the weights, the reranker.
Explanation faithfulness: does the reason the agent gives actually match the score it computed, and is it grounded in the resume? Measure with an LLM-judge that checks each explanation against the score components and the parsed record, calibrated against human labels. An unfaithful explanation is a compliance defect, so this is not optional.
Adverse impact and fairness: does the agent’s output produce a disparate selection rate across protected groups? Measure with the four-fifths test from the bias deep-dive, per req and in aggregate, and gate every change on it. This is the metric that protects the people and the company at once.

How to actually run it:

Build a golden set with recruiters. 100–200 past candidates across the role, labeled strong / borderline / weak by experienced recruiters, plus the reasons. This is the single most valuable evaluation asset in the project, and it is also the calibration baseline for “does the agent agree with good human judgment.” Mine it from real past reqs, not invented profiles.
Calibrate against human recruiters. The honest bar for the matcher is not “perfect,” it is “at least as good and at least as fair as a good recruiter doing the same triage, and more consistent.” Run the agent and a panel of recruiters over the same held-out candidates and compare both the rankings and, crucially, the adverse-impact numbers. A system that matches recruiters on quality and beats them on consistency and documented fairness is a system you can ship.
Score automatically, then gate. Run matching quality, explanation faithfulness, and the four-fifths test on every change. A change that improves matching while quietly worsening the impact ratio is exactly what the harness exists to catch, and it is a change you do not ship.

def evaluate(golden_set, agent, group_labels):
    rows, decisions = [], []
    for ex in golden_set:
        ranked      = agent.rank(ex.req, ex.candidate_pool)
        explanation = agent.explain(ex.candidate, ranked)
        selected    = ex.candidate.id in {c.id for c in ranked[:ex.top_n]}
        rows.append({
            "agrees_with_label": ranking_agrees(ranked, ex.recruiter_labels),  # matching quality
            "explanation_faithful": judge_faithful(explanation, ranked.components),  # 0–1 LLM-judge
            "explanation_grounded": judge_grounded(explanation, ex.candidate.full_record),
        })
        decisions.append(selected)
    quality = aggregate(rows)
    impact  = four_fifths_test(decisions, group_labels)   # the fairness gate
    return quality, impact   # ship only if quality holds AND no group is flagged

Read the output as a diagnosis and a gate at once. A run that reads top-10 recruiter agreement 0.88, explanation faithfulness 0.95, and four-fifths impact ratios all above 0.9 tells a story you can ship: the agent ranks roughly as a good recruiter would, its reasons match its math, and no group is selected at a disqualifying rate. A run where matching looks great but one group’s impact ratio is 0.72 is a run you do not ship, no matter how good the ranking, until you find the cause. The fairness number has veto power over every other number. That is the whole point of measuring it separately.

The closed evaluation loop, with the fairness gate first among equals: a recruiter-labeled golden set plus live decisions drive measured, one-variable-at-a-time changes, and a four-fifths flag blocks release no matter how good the matching looks.

Monitoring & observability

Offline evals tell you the system was good and fair at release. Monitoring tells you it still is. A production recruiting agent degrades quietly: the applicant mix shifts, a new sourcing channel changes who applies, an upstream resume format changes and parsing silently breaks, and, most dangerously, the adverse-impact profile drifts even though no single change looked wrong. Instrument for all of it.

Trace every recommendation and every gate decision end to end with a tool like Langfuse (langfuse.com): the candidate, what the matcher saw (the redacted match view), the score components, the explanation, the recruiter’s decision, and the latency and cost per stage. When a decision is questioned, you need to see exactly what the agent recommended and why, and what the human did with it, in one trace, not reconstruct it from logs across five systems.

Watch four families of signal:

Signal	What it catches	Example alert
Matching quality	ranking drift, recruiter disagreement rising	recruiter override rate on the top-N climbs above baseline
Parsing health	upstream format breakage, low-confidence parses	share of low-confidence parses or flagged records trends up
Fairness	adverse-impact drift across groups	any group’s four-fifths impact ratio drops below 0.8 on rolling traffic
Cost & latency	runaway parsing cost, slow shortlists	per-candidate cost above budget; shortlist latency over target

Set the floors from your launch baseline, not from round numbers. Your golden-set matching agreement and your impact ratios at release are the lines; alert when live-sampled numbers drift below them. The fairness alert is the one that wakes someone up, because a drifting impact ratio is a compliance problem accumulating in real time. Two operational habits separate teams that stay compliant from teams that file disclosures. Sample and grade live decisions continuously with the same checks from your offline harness, including the four-fifths test on rolling traffic. And feed every recruiter override and every candidate complaint back into the golden set, so each real failure becomes a permanent regression test.

When an alert fires, triage in pipeline order. Pull the traces and ask, in sequence: did parsing of these candidates’ resumes break or change? Are the criteria or weights producing a ranking that disagrees with recruiters? Is the explanation faithful to the score? And for a fairness alert: which step introduced the impact, and is there a job-related justification or a bug? The trace answers each in seconds. Resist reaching for the prompt first; in a recruiting agent most failures sit upstream in parsing, criteria, or the data the matcher saw.

What to monitor continuously: drift and ongoing bias

Point-in-time alerts catch a sudden break. The slower, more dangerous problem is drift, the gradual divergence between the fair, accurate system you launched and the one running today. Four charts earn their place.

Applicant-mix drift. Who applies changes as you post to new boards, run new campaigns, or a role goes viral. Track the composition of the applicant pool over time. A shift in the pool is not itself a problem, but it changes the denominator of every fairness calculation, so you need to see it.

Parsing drift. Resume formats and sources change. Watch the rate of low-confidence parses and flagged records. A rising rate means an upstream source changed and candidates are being mis-represented to the matcher, which silently mis-ranks real people.

Matching drift. Track recruiter agreement with the agent’s top-N over time. A downward trend means the matcher and good human judgment are diverging, the fingerprint of either a stale criteria set or a parsing regression feeding it bad input.

Ongoing bias, the chart that matters most. Run the four-fifths test over a rolling window of live decisions, per req and in aggregate, and chart every group’s impact ratio week over week against the launch baseline. This is the single most important monitoring signal in the entire system, because adverse impact can creep in even when each change passed its gate, as the applicant mix and the corpus shift. Set a rolling baseline, alert on any group crossing below 0.8 or trending sharply down, and route that alert to the people (legal, DEI, the recruiting lead) who own the response, not just the engineer.

The weekly review ties it together: a human reads the drift charts and the worst traces, the recruiting lead and a compliance partner look hardest at the fairness chart, and the group decides whether it is noise or the start of a regression. Offline evals prove the system was fair at release; this is what proves it still is, which in a high-risk system is the difference between a defensible program and a discovery you make in a deposition.

Rollout & the additional work

The model is maybe 30% of the work to ship a recruiting agent. The other 70% turns a working prototype into a high-risk system you can deploy lawfully. Here it is.

Phase the rollout. Never big-bang it, and never expand past a gate that has not cleared on fairness.

Internal alpha: your recruiting team, against real past reqs, with no candidate ever contacted by the agent. Goal: find the parsing and matching failure classes your golden set missed, and run the first adverse-impact tests on real rankings.
Shadow / human-in-the-loop: the agent ranks and explains, recruiters work entirely from their own judgment but also see and grade the agent’s shortlist, and the agent contacts no one. You get production-quality calibration data and the first live fairness numbers with zero candidate exposure.
Limited GA: one role family, real candidates, the full gated loop, with the human deciding every advance and reject, an obvious feedback control, a kill switch, and the four-fifths test running on live decisions.
Broad GA: expand to more roles and regions once the quality and fairness metrics hold at each gate, and once the jurisdiction-specific obligations (NYC’s audit and notice, the EU’s documentation) are met for each place you operate.

Each transition is a go/no-go gate, and in this system one of the gate conditions is always fairness. Alpha to shadow when parsing is reliable and rankings agree with recruiters and no group is flagged. Shadow to limited GA when recruiters trust the shortlist and the live impact ratios hold. Limited GA to broad when the metrics hold for two stable weeks and the legal obligations are met. If a fairness gate does not clear, you do not expand, full stop. You fix and re-measure.

The work teams forget to budget, and in hiring it is most of the work:

Legal and compliance sign-off. This is not a checkbox. Legal must review the criteria for job-relatedness, the adverse-impact methodology, the audit trail, and the candidate notices, and put their name on the deployment. For NYC roles you need the independent bias audit done and published before go-live. Budget weeks, not days, and start it at the alpha, not the launch.
Candidate notice and consent. Candidates must be told an automated tool is part of the process and given the human-review path you already provide. Build it into the application flow.
DEI review. A DEI partner should review the criteria, the redaction list, and the fairness methodology, because the people most likely to spot a proxy you missed are the people whose job is to think about it.
Access control and the data perimeter. Recruiters see their reqs, candidate data stays in the perimeter, retention limits are enforced. For the executive version, see MCP and your data perimeter.
The recruiter UI. The gate must show the rank, the reason, the parsed record, and the original resume together, make do-not-advance a logged human action, and make “request human review” and “this is wrong” one click each.

Team & skills required

You do not need a research team. You need a small group that can build a gated pipeline, measure it honestly for quality and fairness, and operate it under a compliance regime. Two roles on this list do not appear on the RAG team, and they are not optional here.

Role	What they own	Commitment
ML / AI engineer	the pipeline: parsing, matching, ranking, explanation, evals	core, full-time
Backend engineer	the agent loop, the gate, ATS integration, access control	core
Data engineer	ingestion, the candidate store, retention jobs, the audit log	core through launch, part-time after
Recruiter / TA lead	the golden set, “is this a good shortlist,” the gate in practice	part-time, non-negotiable
Legal / compliance	high-risk obligations, job-relatedness, the bias audit, candidate notice	part-time, non-negotiable
DEI partner	the criteria, the redaction list, the fairness methodology	part-time, non-negotiable
Product / design	the gate UI, the explanation, the feedback loop	part-time

The roles people skip are legal/compliance and the DEI partner, and skipping them in a high-risk system is how you build something that looks right to engineers and is unlawful in production. Budget their time explicitly from day one. The recruiter is the third non-negotiable: only an experienced recruiter can tell you whether the shortlist is actually good and whether a reason would survive a hiring manager’s scrutiny. For the broader org question, see the AI roles you actually need to hire.

How the team operates matters as much as who is on it. Run a weekly review: the engineer brings the quality and fairness deltas, the recruiter spot-checks a sample of shortlists, the compliance and DEI partners look hardest at the impact charts, and the group commits to the next single change. That ritual, small, regular, evidence-led, and fairness-first, is what keeps the system both improving and defensible. The failure mode is a launch, then silence, then a quarter later someone notices the impact ratio slipped and nobody can say when, why, or who signed off.

A 30/60/90-day delivery plan

A realistic path from zero to a defensible pilot.

Days 1–30, prove parsing and matching, never touch a candidate. Pick one high-volume role and define “qualified” with the hiring manager (Step 0). Stand up parsing (Finigami DocumentAI) with redaction, the candidate data model, pgvector, and structured matching with a reranker. Build the first 100-candidate golden set with recruiters, and run the first adverse-impact tests on real past reqs. The goal by day 30 is a matcher whose rankings agree with good recruiters and whose four-fifths ratios hold, with zero candidate exposure.

Days 31–60, add the gate, the explanation, and the compliance scaffolding. Build the human gate, the explanation generated from score components, per-decision tracing, and the audit log. Start legal and DEI review of the criteria and methodology now, not later. Run the shadow phase: recruiters work from their own judgment and grade the agent’s shortlist, the agent contacts no one. The goal by day 60 is faithful explanations, holding fairness numbers on live shadow traffic, and legal review underway.

Days 61–90, ship it small and gated. Limited GA to one role family with the full gated loop, real candidates, a feedback button, a kill switch, and the four-fifths test on live decisions. For NYC roles, the bias audit and candidate notice are done before go-live. Watch the live quality and fairness metrics, fold overrides and complaints back into the golden set, and tune one variable at a time. The goal by day 90 is a pre-committed quality-and-fairness bar, hit, with legal sign-off in hand to ask for the broad rollout.

The shape is deliberate: no candidate is touched until parsing, matching, and fairness hold, and the audience never widens until the metrics and the legal obligations are both met. Each phase ends at a gate, and one of the gate conditions is always fairness.

The business case

Most recruiting-agent projects that stall do not stall on technology. They stall because no one wrote the one page that gets them funded and signed off. Here is that page.

Frame it as time-to-fill and recruiter capacity, not “we built AI.” The two framings that get approved:

Capacity returned: “Recruiters spend an average of N hours per req on resume triage and scheduling. The agent cuts that to under one. At X reqs/quarter that is Y recruiter-hours returned, which we reinvest in candidate experience and hard-to-fill roles.”
Time-to-fill: “Slow shortlisting costs us candidates to faster competitors and leaves roles open, each open day carrying a loaded productivity cost. Faster, fairer shortlists shorten time-to-fill by Z days.”

Worked example. A talent team runs 300 reqs a quarter at an average of 2,000 applicants for the high-volume roles. Recruiters spend roughly 6 hours per req on resume triage and another 3 on scheduling ping-pong, so 9 hours per req on mechanical work. The agent cuts the triage to about 1 hour of reviewing a ranked shortlist and removes nearly all the scheduling time, call it 7 hours returned per req. Across 300 reqs that is 2,100 recruiter-hours a quarter. At a $40/hour loaded cost that is roughly $84,000 a quarter, about $336,000 a year of capacity, before counting the time-to-fill improvement. Now halve it for ramp, imperfect adoption, and the reqs where the agent does not apply: still about $168,000 a year against a run cost in the low tens of thousands. The payback is a single quarter. Bring that number, conservatively derived, plus the time-to-fill story, not “AI will make recruiting better.”

The per-candidate cost, modeled. The components: parsing (the largest line, a real per-document operation, run once per candidate), embedding (negligible), matching and ranking (database and reranker, small per candidate), the ranking explanation (a model call, run for the ranked set the recruiter sees, not the whole pool), and outreach drafting (a model call, run only for advanced candidates). For a 2,000-applicant req, that is 2,000 parses, 2,000 cheap matches, ~40 explanations, and a handful of outreach drafts. The dominant cost is parsing, and the lever is the same as RAG’s reranker: do the expensive model reasoning only where a human will see it, not across the whole pool. A system that writes a paragraph about all 2,000 applicants pays for 2,000 paragraphs nobody reads.

Put a cost ceiling, a quality floor, and a fairness floor on it. Decision-makers approve bounded bets. State the per-candidate cost target, the monthly ceiling, the quality bar (“matches good recruiters on the top-N, or we do not ship”), and the fairness bar (“no protected group below the four-fifths threshold, monitored continuously, or we pause”). The fairness floor is the one that protects the company and the candidates at once, and it is the one legal and the CHRO care about most.

Tune the framing to who is in the room. A CFO wants the capacity number and the cost ceiling, conservatively derived. A CHRO wants the candidate-experience and fairness story: that the agent expands the funnel, never auto-rejects, and is monitored for adverse impact, because the CHRO owns the reputational and legal risk of a biased process. Legal wants the high-risk obligations met, the audit trail, and the sign-off path, answered before they ask. An engineering leader wants to know it will not become an unauditable sprawl, which is exactly what the single gate and the one-variable-at-a-time discipline are for. Same project, four different opening sentences.

Pre-empt the questions you will be asked:

Question	Your answer
”Will it discriminate?”	Bias-linked fields redacted from the matcher; four-fifths test gating every change and monitored live; explainable ranking; DEI and legal review
”Does it reject people automatically?”	Never. The agent ranks and recommends; a human decides every advance and reject; do-not-advance is a logged human action
”Is this legal?”	Designed to the EU AI Act high-risk obligations, EEOC adverse-impact rules, and NYC Local Law 144; legal sign-off before go-live; candidate notice built in
”What does it cost to run?”	Per-candidate model (parsing-dominated) + monthly ceiling; we alert before we breach it
”What happens when it is wrong?”	Feedback loop, human review on request, one-click kill switch; overrides and complaints become regression tests

Propose the smallest credible pilot. One role family, one quarter, a single quality-and-fairness metric agreed in advance, and legal in the room from day one. A scoped pilot that hits a pre-committed number and clears the fairness gate is how you get the budget and the sign-off for the broad rollout. For the executive’s view of how to evaluate the return, see the AI ROI question and how to answer it.

Pitfalls & anti-patterns

The failures that recur, in rough order of how much damage they do:

Auto-rejecting candidates. The single most dangerous mistake. It is the action the EU AI Act’s human-oversight rule exists to prevent, the most likely source of unexamined adverse impact, and it saves almost no recruiter time. The agent ranks and recommends; a human rejects, always.
Treating resume parsing as solved. The demo used clean text; production is two-column PDFs, scanned CVs, and 50+ languages. A mis-parse silently mis-ranks a real person. Budget for real document understanding.
No adverse-impact test. “The shortlist looks diverse” is not a measurement. If you skip one thing in this guide, do not let it be the four-fifths test, run continuously and gating every change.
A black-box matching score. A rank you cannot decompose into job-related reasons is a rank you cannot explain, defend, or audit. Score on explicit criteria; carry the components into the explanation.
Letting the matcher see bias-linked fields. Name, photo, graduation year, gendered pronouns, institution signals: redact them from the matching input. You cannot bias on a field the matcher never saw.
Training the matcher on your past hires. This is the Amazon failure mode: the system learns to reproduce who you have hired before, bias included. Rank on defensible criteria, not on a model fit to historical outcomes.
Conflating “ranked low” with “rejected.” A candidate ranked 900th is ranked 900th, not rejected. Whether to contact them is a human decision made with the fairness numbers in view. Keep ranking and decision separate.
Skipping legal, compliance, and DEI. In a high-risk system these are not late-stage reviewers; they are designers. Bring them in at the alpha.
No audit trail. In a high-risk domain, if you cannot show why the agent did what it did and who decided, you cannot defend it. Log every recommendation, score, and gate decision immutably.
Hoarding candidate data forever. Resumes have a legal shelf life. Set retention limits and delete on schedule; you cannot honor a deletion request for data you scattered everywhere.
Premature automation past the gate. Auto-advancing “obvious” matches to save a click trades the entire compliance story for seconds. Earn every step of autonomy, and keep the consequential ones with the human.

FAQ

Will an AI recruiting agent discriminate against candidates? It can, if you build it carelessly, which is exactly why the controls in this guide are mandatory rather than optional. The defenses are concrete: redact bias-linked fields from the matching input so the matcher cannot weight them, rank on explicit job-related criteria rather than a model fit to your past hires, run the four-fifths adverse-impact test continuously and gate every change on it, and keep a human deciding every advance and reject. A well-built agent applied consistently and monitored for impact can be fairer and more documented than ad-hoc human screening, but only because it is measured. The measurement is the product.

Can the agent reject candidates on its own? No. The agent ranks, surfaces, and recommends, including recommending “do not advance” with reasons, but a human makes every consequential decision and clicks every advance and reject. This is a hard rule, both because the EU AI Act’s human-oversight requirement demands it and because auto-rejecting saves almost no time while taking on nearly all the legal risk. Do-not-advance is always a logged human action, never a silent automated one.

Is this legal under the EU AI Act and US law? Recruitment AI is classified as high-risk under the EU AI Act (Annex III), which triggers obligations for risk management, data governance, logging, human oversight, transparency, and accuracy. In the US, an automated tool causing disparate impact is unlawful under Title VII regardless of intent, and the EEOC has confirmed adverse-impact analysis applies to algorithms (EEOC, 2023). NYC Local Law 144 adds an independent bias audit and candidate notice. Designing to the strictest of these and getting legal sign-off before go-live is how you deploy lawfully; this guide is built around those obligations, not retrofitted to them.

How is this different from the keyword filter in our ATS? The keyword filter scores exact string overlap, which is brittle (it misses “K8s” when the JD says “Kubernetes”), gameable (candidates white-text keywords), and biased toward the vocabulary of people already hired, and it cannot explain itself. The agent matches on what the work actually was, ranks on explicit criteria, attaches a verifiable reason to every rank, and can be audited for adverse impact. On every axis except setup cost, it is the better tool, precisely because it is explainable and measurable where the keyword filter is neither.

How accurate can the matching get? Accuracy is bounded by parsing and by how well you defined “qualified.” If a resume is mis-parsed, the candidate is mis-ranked through no fault of their own, which is why parsing quality sets the ceiling. And a matcher can only be as good as the criteria it scores, which is why Step 0, defining the job-related criteria with the hiring manager, is the most important non-code step. The honest bar is “at least as good and at least as fair as a good recruiter, and more consistent,” measured against a recruiter-labeled golden set.

What about bias in the embeddings or the model itself? Real and worth guarding against. The main defense is structural: the matcher only ever sees the redacted, job-related match view, and the rank is built from explicit criteria rather than a raw similarity score, so a biased embedding has far less room to express itself. Then you measure the output with the four-fifths test, which catches impact regardless of where in the system it originated. You do not rely on the model being unbiased; you constrain what it sees and measure what it does.

How much does it cost to run per candidate? The dominant cost is resume parsing, a real per-document operation run once per candidate. Embedding and matching are cheap per candidate. The model work (the ranking explanation and outreach drafting) is the smallest line, because you only generate explanations for the ranked set a recruiter will see and only draft outreach for advanced candidates, not the whole pool. The lever is the same as RAG’s reranker: spend the expensive reasoning only where a human will look.

Do we need demographic data to run the bias test, and isn’t collecting it risky? You need group labels to compute selection rates by group, but you collect that data separately and voluntarily, for monitoring only, and the matcher never sees it. It lives apart from the matching input precisely so it can never become a matching signal. Collecting it for adverse-impact monitoring is standard and expected; what would be risky is feeding it into the ranking, which you never do.

Can we use an open-source model instead of Claude? Yes, and some teams do for cost or data-residency reasons. The caveat specific to this system: the model’s job is mostly faithful explanation and drafting, both of which are forgiving once the structured matching has done the real work, so the gap between a strong open model and a frontier one is smaller here than on open-ended tasks. Test explanation faithfulness specifically and harder on an open model, because an explanation that does not match the score is a compliance defect.

What happens to candidates the agent ranks low? They stay in the pool, visible to recruiters, with their reasons attached. The agent does not reject them; it ranks them. How deep into the ranked pool a recruiter goes for a given req is a human decision, made with the adverse-impact numbers in view. A low rank is information for a human, never an automated outcome.

How long to a production pilot? With a focused team and one high-volume role, a scoped, gated pilot is a matter of a quarter, not a year, with the caveat that the legal and compliance work runs in parallel from day one and is often the long pole. The variable that moves the timeline most is the diversity of your resume formats and languages, which is the case for getting the parsing decision right early.

How do we handle candidate complaints or requests for human review? You already provide human review by design, because a human decides every consequential step. Build a one-click “request human review” and “this is wrong” control into the candidate-facing and recruiter-facing flows, route those to a person, and feed every complaint back into the golden set as a regression test. A complaint is both an obligation to honor and a signal about where the system needs work.

Reference implementation checklist

Ship what is on this list and you will have a recruiting agent you can defend to a skeptic, a recruiter, and a regulator:

If you are missing the bias test, the human gate, or the never-auto-reject rule, you do not have a recruiting agent you can deploy. With all fifteen, you have a system you can put in front of candidates and stand behind.

Executive briefings (for the people who approve this):

Sibling build guides:

How to Build a RAG System in 2026: the retrieval and parsing patterns this guide builds on, in depth
How to Build an AI Sales Agent in 2026
Agentic Workflow Automation in 2026