Intelligent Document Processing in 2026: A Practical Implementation Guide

Q: What's the difference between OCR and IDP?

OCR turns an image into text; that's one step. IDP is the whole pipeline that turns a document into trustworthy structured data your systems can act on: capture (which includes OCR), classification, extraction with confidence, validation, the human-in-the-loop decision, and the downstream post. OCR is a component of IDP, not a substitute for it.

Q: Do I still need templates in 2026?

No, and avoiding them is the point. Template-agnostic extraction reads layout and labels to find fields across formats it has never seen, which lets one system handle the hundreds of vendor layouts a templated approach never could. If you find yourself building per-vendor templates, the real fix is usually a single normalization or extraction improvement that helps every format at once.

Q: What straight-through rate is realistic?

It depends on the document type and input quality, and it climbs over time. A structured, high-volume type like standard invoices commonly starts around 50-60% straight-through in the first weeks with conservative thresholds, then climbs past 85% as calibration improves. Messier types start lower. Commit to whatever your shadow-mode data shows at your error bar, not an industry headline.

Q: How do I set the confidence threshold?

Per field, with data. Run a labeled batch and for each field find the lowest confidence threshold at which the auto-accepted error rate stays under your tolerance for that field. Strict fields where errors cost money get high thresholds; cosmetic fields get lower ones. Then verify calibration: confirm the model's reported confidence actually matches empirical accuracy in each bucket, because an uncalibrated threshold lies to you.

Q: Will it post wrong data into my system?

Only at a rate you choose and measure. A field auto-posts only when its confidence beats its threshold and the validation rules pass; everything else goes to a human. You set the threshold to hold the auto-cleared error rate under a committed bar, measure that bar continuously, and the human-in-the-loop tier means the worst case for an uncertain document is a person reviewing it, not an error reaching your ledger.

Q: How much does it cost to process a document?

For a document that clears straight through: the extraction API (cents per page) plus negligible compute. For a flagged document: that plus a minute or two of a reviewer's time, an order of magnitude more. Per-document cost is dominated by the straight-through rate, so the lever that controls your unit economics is how many documents you can safely clear without a human, which is why STP is the metric on the business case.

Q: Can't I just paste the PDF text into an LLM and ask for the fields?

For a quick prototype on clean digital documents, yes. For production, no. It has no calibrated confidence to threshold on, so you can't tell which fields are safe to auto-post, which is the entire IDP value. And it's weak on the scanned, skewed, table-heavy documents that make up much of a real corpus, because a raw text paste loses the layout a document-understanding model uses.

Q: How is IDP different from RAG?

Different jobs that often connect. IDP turns documents into structured data and posts it to a system of record. RAG retrieves passages from a corpus to answer questions. They chain naturally: IDP is frequently the layer that produces the clean, structured, well-provenanced text that a RAG system then indexes and retrieves over. If you're building both, build IDP first, because its output is RAG's input.

Q: Do I need a human in the loop forever?

Yes, and that's a feature. The human-in-the-loop tier bounds the downside: it catches the documents the system isn't sure about and the genuinely novel ones, and its corrections keep the system calibrated as the document mix drifts. The goal is to shrink the share of documents that need a human by raising the straight-through rate, not to eliminate the human tier, which would mean accepting errors or having no safety net when a new format arrives.

TL;DR

Intelligent document processing turns the documents your business runs on (invoices, claims, onboarding packets, contracts, forms) into structured data your systems can act on, with a confidence number attached to every field so you know which results are safe to trust and which need a human.
The job is not “read the document.” It’s “decide which fields you can post automatically and which you can’t, and be right about that decision.” The one metric that captures whether the system is working is the straight-through-processing rate: the share of documents that clear with zero human touch at your required accuracy.
A production IDP system is a pipeline of decisions, not a single model: ingest, classify and route, extract, validate against confidence thresholds and business rules, send the uncertain cases to a human, then post the clean results downstream. Each stage has its own failure mode and its own metric.
Start with a boring, strong default stack: Finigami DocumentAI for OCR, classification, and extraction; pgvector/Postgres for the extracted data and an audit trail; business rules for validation; Claude for the reasoning that rules can’t express; and Langfuse for evals and observability. Add complexity only when a metric tells you to.
You cannot tune what you cannot measure. Build the field-level eval harness and label a golden set before you touch a threshold. Field accuracy, extraction precision and recall, and the STP rate are the numbers that decide whether this ships.
The hard part of getting it approved isn’t technical. It’s a one-page business case that frames IDP as labor deflected per document, with a cost-per-document model, a confidence-calibrated quality floor, and a human-in-the-loop safety net that bounds the downside.

What IDP is in 2026, and what changed

Intelligent document processing is the discipline of getting structured, trustworthy data out of unstructured or semi-structured documents, at volume, with enough confidence to act on it without a person reading every page. An invoice arrives as a PDF; IDP turns it into {vendor, invoice_number, line_items, total, tax, due_date} with a confidence score on each field, checks that the math adds up and the vendor exists, and either posts it to the ERP or hands the two uncertain fields to a clerk. Multiply that by tens of thousands of documents a month and you have the problem.

The pattern is old. What changed is that the technology underneath it crossed a line where most of the document, most of the time, no longer needs a human. Three shifts matter for how you build in 2026.

Extraction stopped requiring templates. The previous generation of “OCR plus rules” worked by teaching the system the exact coordinates of each field on each known form. It broke the moment a vendor moved their invoice total two centimeters or a new claim form appeared. Modern document-understanding models read layout the way a person does: they find the total because it’s labeled “Total” and sits at the bottom of a column of numbers, not because it’s at pixel (840, 1120). Template-agnostic extraction is the single biggest change, because it’s what lets one system handle the long tail of formats that templated systems never could. Most enterprises have hundreds of invoice layouts; nobody was ever going to template all of them.

Classification and extraction merged into one understanding pass. You used to classify a document (is this an invoice or a remittance?), then route it to a template built for that class. Now a single model can tell you what the document is and pull the fields in the same pass, which collapses two brittle stages into one and removes a whole class of routing errors where a misclassified document hit the wrong extractor and produced confident garbage.

Confidence got good enough to automate on. The reason IDP is a 2026 story and not a 2016 one is that the confidence scores coming out of extraction are now calibrated well enough to act on. A calibrated score means that fields the model reports at 0.98 confidence really are right about 98% of the time, so you can set a threshold, auto-post everything above it, and route everything below it to a human, and the math actually holds. That single property, trustworthy confidence, is what turns extraction from a demo into straight-through processing. Without it, you either trust everything (and post errors) or trust nothing (and you’ve automated nothing). For the whole field, the analyst framing of “automating document-heavy work pays back faster than almost any other AI investment” lines up with what teams see in practice: document workflows are repetitive, measurable, and expensive in exactly the way that makes automation worth it (McKinsey’s work on automation potential in operations puts document-heavy back-office tasks among the highest-automatable, see mckinsey.com).

What did not change: garbage in, confidently-wrong out. A scanned, skewed, coffee-stained fax still has to be read correctly before any of the clever downstream logic matters, and the single biggest determinant of IDP quality in real enterprises is still how well you turned a messy physical document into clean text and structure. This is the part teams under-budget, because the demo ran on clean digital PDFs and production is scans.

The anatomy of an IDP failure

When an IDP result is wrong, it failed at one of five points. Naming them matters, because the fix is different at each, and because the most expensive failures are the ones that pass silently:

Capture / OCR: the characters never came out of the document correctly. A 5 read as an S, a blurry scan, a table that became gibberish. No downstream step can recover a digit that was never read.
Classification: the document was sent to the wrong handler. A credit note processed as an invoice, a passport routed to the W-9 extractor. The extraction that follows is confident and wrong.
Extraction: the right text was read and classified correctly, but the wrong span was pulled for a field. The “ship to” address landed in the “bill to” field; a line-item amount became the total.
Validation: the field was extracted correctly, but a business rule that should have caught a problem (totals don’t reconcile, the PO doesn’t exist, the date is in the future) wasn’t there or wasn’t enforced.
Confidence calibration: the worst failure. The field is wrong and the model reported high confidence, so it cleared straight-through and posted an error into your ledger. A miscalibrated 0.99 is more dangerous than a wrong answer flagged at 0.4, because nobody looked.

This guide spends most of its words on classification, confidence, validation, and the human-in-the-loop decision on purpose: extraction itself is increasingly a solved-enough problem, and the value is in deciding, correctly and per field, what you can trust. Your eval’s entire job is to tell you which of the five it is, so you fix the right box.

When NOT to build IDP

Skip IDP, or descope it hard, if any of these is true:

The documents are already structured data. If your “documents” are EDI feeds, API payloads, or clean CSVs from a partner, you have a data-integration problem, not a document problem. Parse them directly. Running OCR over something that was never an image is an anti-pattern.
Volume is too low to amortize. IDP is infrastructure. Building a confidence-calibrated, human-in-the-loop pipeline to process forty documents a month is more expensive than the clerk who does it today. The break-even is usually in the thousands of documents a month, not the dozens. Below that, a person with a good template and a spreadsheet wins.
One document type, one fixed layout, forever. If you receive exactly one form, it never changes, and it’s already digital, a narrow templated extractor (or even regex over the text layer) is cheaper and more predictable than a full IDP stack. IDP earns its complexity on variety and volume.
The downstream system can’t consume structured output. If the data has nowhere to go (no ERP integration, no API, a human re-keys it anyway), you’ve automated the reading but not the work. Fix the downstream integration first, or you’ve built a more expensive way to produce a number someone retypes.
You can’t define “correct” for a field. If two reasonable people disagree on what the “effective date” of a contract is, you can’t build an eval set, which means you can’t measure accuracy, which means you can’t set a confidence threshold. Resolve the definition before you build the extractor.

IDP vs the alternatives

Before you build, make sure IDP is the right tool. There are five common ways to get data off a document and into a system, and they sit on a spectrum from cheap-and-brittle to flexible-and-involved:

Approach	Gets you…	Best when	Weakness
Manual data entry	a human reads and re-keys	volume is tiny, formats wild, stakes high	doesn’t scale; error rate is real; expensive per document
Templated OCR / zonal	fields at fixed coordinates	one layout, never changes, already digital	breaks on any layout drift; per-form setup cost
OCR + regex/rules	text, then pattern-matched fields	semi-structured, predictable text, few formats	brittle on the long tail; no confidence signal
IDP (document AI)	classified docs + extracted fields + per-field confidence	many formats, high volume, needs straight-through	needs eval, thresholds, and a human-in-the-loop tier
LLM-only (paste the PDF text in)	a model’s best guess at the fields	quick prototype, low stakes, clean digital docs	no calibrated confidence; weak on scans/tables; hard to audit

The common mistake is reaching for the LLM-only approach because it demos well on a clean PDF, then discovering it has no calibrated confidence to threshold on and falls apart on the scanned documents that make up half the real corpus. A raw model can read a clean invoice; it cannot tell you, reliably, how sure it is, and the confidence number is the whole product in IDP. The other mistake is staying on templated OCR because it’s familiar, and paying the per-form configuration tax forever as formats multiply. Pick IDP when you have variety and volume and you need a number you can act on. For the executive framing of where document work pays off across functions, see AI by industry.

Use cases & where it pays off

IDP earns its keep when three conditions hold: the documents arrive in volume, the same handful of fields gets pulled from each one over and over, and a wrong field is expensive enough to justify confidence scoring and a human safety net. The patterns that consistently pay back are these.

Use case	Why IDP fits	What “good” looks like
Accounts payable / invoices	High volume, many vendor layouts, fields are stable (vendor, total, line items, PO)	Most invoices post to the ERP untouched; only exceptions reach a clerk
Insurance claims	Mixed-format packets (forms, photos, bills), high stakes, regulated	Structured claim record assembled from the packet; adjuster reviews only flagged fields
KYC / customer onboarding	IDs, proofs of address, applications; compliance-critical	ID fields extracted and cross-checked; clean cases onboard in minutes, not days
Contracts	Long, dense, parsing-heavy; key terms drive obligations	Effective dates, parties, renewal and termination clauses extracted with a citation a lawyer can verify
Forms & applications	Structured-ish, high volume, repetitive	Fields mapped to the system of record; the form never gets re-keyed by hand

The common thread: the value is the field plus its confidence, not the field alone. A vendor name with no confidence score is a guess you have to check anyway; a vendor name at 0.99 calibrated confidence is a number you can post. If you take one design principle from this guide, take that one: every extracted field carries a confidence, and the threshold on that confidence is what you actually tune.

A worked example, accounts payable. A 4,000-invoice-a-month AP team receives a scanned PDF from a vendor it has never billed before. IDP captures the page, classifies it as an invoice (not a statement or a remittance), and extracts the fields: vendor, invoice number, date, line items, subtotal, tax, total, PO reference. Validation runs: do the line items sum to the subtotal? Does subtotal plus tax equal the total? Does the PO exist in the ERP and is there budget against it? Eleven of the twelve fields come back above the auto-post threshold and reconcile cleanly. One field, a line-item description on a smudged row, comes back at 0.71, below threshold. The system posts nothing yet. It surfaces that single field to a clerk, pre-filled with its best guess and the cropped image of that exact row, and the clerk confirms it in eight seconds. The invoice posts. The clerk touched one field on one invoice, not twelve fields on four thousand. That ratio, and not the model’s raw accuracy, is what the business bought.

The same pattern carries to the other four, with one twist each. Insurance claims arrive as packets, not single documents: a claim form, photographs, repair estimates, and medical or police reports stapled together as one PDF. So the first job is splitting the packet into its component documents, classifying each, and extracting per component, then assembling one structured claim record from the pieces. The confidence gate and the human-in-the-loop work identically; what’s different is the up-front document-splitting step, which is its own classification problem and the place packet workflows most often break. KYC and onboarding raise the stakes and the cross-referencing: an extracted name and date of birth off a passport mean little until they’re checked against the application form and a sanctions list, so validation here leans heavily on cross-reference rules, and a mismatch (the passport says one date, the form another) is exactly the kind of flag that has to reach a human, because it’s often the signal of fraud rather than a reading error. Contracts invert the volume-versus-depth trade: there are fewer of them, but each is long and dense, the fields that matter (effective date, parties, renewal and termination terms, liability caps) are buried in pages of boilerplate, and a wrong extraction has legal weight. So contract IDP cares less about straight-through volume and more about extraction with a verifiable citation: the system shows the lawyer the exact clause it pulled the renewal date from, the way a RAG citation points to a source page, so a human can confirm the high-stakes terms quickly rather than re-reading the whole agreement. Forms and applications are the easy end of the spectrum (structured-ish, repetitive, high volume) and the place straight-through rates climb highest fastest, which makes them a good first target if invoices aren’t your pain point. Across all five, the architecture is the same; what changes is the packet-splitting at the front, the validation rules in the middle, and how much you lean on citation versus volume at the end.

The reference architecture

A production IDP system is a pipeline of decisions. Documents flow in from many channels, get captured and read, get classified and routed, get their fields extracted with confidence scores, get validated against business rules, and then split: the confident, clean ones go straight through to the downstream system, and the uncertain ones go to a human, whose corrections flow back to make the system better. Evaluation and observability span the whole thing.

The canonical IDP reference architecture: capture and understand, validate and decide, then split into straight-through processing and human review, with corrections feeding back and evaluation spanning the whole pipeline.

Read the diagram top to bottom. The top lane reads and understands the document. The middle lane decides, per field, whether you can trust it, combining the model’s confidence with whether the business rules passed. The split is the heart of the system: confident-and-valid goes straight through, anything else goes to a human, and the human’s corrections write back to make tomorrow’s system better. The two boxes that decide your economics are Capture & OCR (set your quality ceiling) and the Confidence gate (set your straight-through rate), and both are the ones teams skip in the demo.

Architecture decisions

Seven decisions determine 90% of the outcome. Make them deliberately; everything else is tuning.

Build vs. buy the whole thing. Managed “IDP platform” suites will get you a demo and a contract in an afternoon. They will also own your accuracy ceiling, your data perimeter, and your per-document cost curve, and most price per page in a way that punishes you exactly when volume makes the project worth doing. The rule: buy the components (capture, classification, extraction, the reasoning model), build the pipeline (routing, validation, the confidence gate, the review queue, the feedback loop). A hosted black box is fine for a prototype and a liability for a system whose thresholds you’ll tune for a year and whose audit trail you’ll be asked to produce.
Where the documents get read. This is the decision that silently sets your quality ceiling. Naive OCR (a raw open-source engine over a skewed scan) drops digits, scrambles tables, and ignores handwriting and low-contrast text entirely, and no downstream cleverness recovers a character that was never read. Use a real document-understanding API. We default to Finigami DocumentAI: it’s API-first, template-agnostic, handles scanned and photographed documents, and reads 50+ languages without per-form configuration, which is exactly the failure mode (mixed vendors, multi-language invoices, photographed IDs) where naive OCR falls apart. Alternatives: AWS Textract, Google Document AI, or Azure Document Intelligence if you’re committed to one of those clouds and willing to wire their output into your own pipeline.
Classification and routing strategy. Decide whether you classify documents as a separate stage or fold it into extraction. For a handful of clean, distinct document types, a separate classifier is simple and debuggable. As types multiply and blur (an invoice versus a statement versus a remittance can look near-identical), fold classification into the same understanding pass as extraction so the model uses the same layout signal for both. Either way, route on the classification result to a type-specific extraction schema, and treat a low-confidence classification as its own exception that goes to a human, because a misclassified document produces confident, well-formed, wrong output, which is the most dangerous kind.

Option	Use when	Watch out for
Separate classifier	few, visually distinct doc types	a new type silently routes to the closest existing class
Joint classify + extract (default)	many types, blurry boundaries	one model to evaluate for both jobs; watch per-type accuracy
Rules / sender-based routing	the channel already tells you the type	breaks when senders mix document types in one email

Extraction schema and ground truth. Define the exact fields you need per document type, their types, and the rule for what “correct” means, before you build the extractor. This sounds like paperwork and it’s the decision that makes everything downstream measurable. “Extract the date” is ambiguous; “extract invoice_date as ISO-8601, defined as the date printed next to the label Invoice Date, not the due date or the ship date” is a thing you can grade. The schema is also what the confidence threshold attaches to: you set thresholds per field, because the address tolerates more error than the total does.
Confidence threshold strategy. This is the lever that sets your straight-through rate, and it’s per field, not global. A wrong total posts money incorrectly; a wrong line-item description is a cosmetic annoyance. So the total gets a high threshold (only auto-post above 0.97, say) and the description a lower one. Start conservative (high thresholds, low STP, few errors), measure the error rate that escapes, then loosen field by field as the data tells you each field is safe. The mistake is one global threshold, which forces you to choose between auto-posting risky fields and reviewing safe ones. More on calibrating these below.
Validation: rules, model, or both. Most validation is deterministic and belongs in code: totals must reconcile, dates must be plausible, the PO must exist, the IBAN must checksum. Write those as explicit rules; they’re cheap, fast, auditable, and a failing rule is a precise reason to flag a document. Reserve the reasoning model for the checks rules can’t express: “does this remittance advice actually correspond to these three invoices,” “is this contract clause a standard indemnity or an unusual one.” Default to Claude for those judgment calls: Sonnet 4.6 for high-volume per-document checks where latency and cost matter, Opus 4.8 for the genuinely hard reasoning over a long contract. Rules first, model only where rules run out.
Where the extracted data and audit trail live. The structured output, the per-field confidence, the validation results, the human corrections, and a pointer to the source document all need a home, and in a regulated setting that home is also your audit trail. Default to pgvector/Postgres: it keeps the extracted records next to relational business data (the vendor master, the PO table), supports the embeddings you’ll want if this data later feeds a RAG system, and gives you one transactional store to query and back up. Reach for a dedicated document store only if your documents are enormous and your access patterns are pure blob retrieval.
Index freshness and reprocessing. Decide up front how you handle a document that needs reprocessing: a vendor format changes, a classifier improves, a clerk finds a systematic error. You need to be able to re-run extraction on a batch without losing the original capture or the human corrections, and you need to version the model and schema that produced each record so you can tell whether a result came from the old extractor or the new one. Store the model version and schema version on every extracted record; it’s near-free at write time and the only way to reason about a regression later.

The reference stack

The boring, strong default. Start here; deviate only when a metric tells you to.

Layer	Default pick	Why	Swap when
Capture / OCR	Finigami DocumentAI	API-first, template-agnostic, scanned + 50+ languages	You receive only clean digital PDFs with a text layer
Classification	Finigami DocumentAI (joint with extraction)	One understanding pass for type + fields	Few, distinct types → a small separate classifier is fine
Extraction	Finigami DocumentAI	Template-agnostic, per-field confidence, handles scans	Cloud-committed → Textract / Google / Azure
Validation (rules)	explicit business rules in app code	Cheap, fast, auditable, precise flags	-
Validation (judgment)	Claude Sonnet 4.6 / Opus 4.8	Reasoning rules can’t express; faithful, good at citing	High volume, simple checks → rules only
Data + audit store	pgvector / Postgres	Co-located with business data; one store to audit	Blob-only at huge scale → object store + index
Human-in-the-loop	a review queue with field-level UI + the source crop	Where the STP rate is actually earned	-
Eval / observability	Langfuse + offline test sets	Traces + field scores + cost in one place	Enterprise → Braintrust
Orchestration	thin app code + a queue	IDP is a pipeline with a human branch, not an agent	Genuinely multi-step/branching → a workflow engine

A note on platforms: you do not need an end-to-end IDP suite to build this. It’s a pipeline with a human branch and a feedback loop, all of which you want to own because all of them encode your business rules and your risk tolerance. A few thousand lines of well-tested application code around strong capture and extraction APIs is more debuggable and far cheaper at volume than a per-page suite whose abstractions leak exactly where you need to tune the confidence gate. Buy the hard ML (reading documents), build the pipeline (deciding what to do with the results).

Cost and latency, per document

Two numbers decide whether an IDP system is viable: what each document costs to process and how fast it clears. Unlike a chat system, the dominant cost is usually not the model: it’s the capture/extraction API call, priced per page, plus the human minutes spent on the documents that don’t clear straight through. A rough per-document profile for a well-built system on a typical multi-page invoice:

Stage	Latency (typical)	Cost driver
Ingest & queue	< 100 ms	infra, negligible
Capture, OCR, classify, extract	~1–5 s per document	the main machine cost: per-page extraction API
Validation rules	< 50 ms	in-process, free
LLM judgment check (when used)	~300 ms – 2 s	only on the fields/docs that need it
Confidence gate & post (straight-through)	< 100 ms	a database write
Human review (only the flagged share)	seconds to minutes per flagged doc	the real variable cost, labor
Total (auto-cleared doc)	a few seconds	extraction API dominates
Total (flagged doc)	seconds + human time	human minutes dominate

Read it as a budget with two regimes. For documents that clear straight through, cost is the extraction API call (cents per page) and latency is a few seconds; you optimize by not running the expensive LLM check on documents the rules already cleared. For documents that get flagged, the cost is human time, which dwarfs the machine cost. This is why the straight-through rate is the metric that matters: every point of STP you win moves a document out of the expensive human regime into the cheap machine regime. A system at 60% STP and one at 85% STP can have identical extraction accuracy and wildly different unit economics, because the difference is entirely in how many documents a person had to touch.

Implementation

The build, step by step, with the decisions that bite at each one.

Step 0: Scope the document type and define correctness

Before any code: pick one document type (start with the highest-volume, most-painful one, usually invoices) and write down the exact fields you need, their formats, and the rule for what “correct” means for each. This isn’t busywork. It’s what makes every later step measurable. The field list becomes your extraction schema; the correctness rules become your eval; and the pair tells you whether IDP is even the right tool. If the “document” turns out to be a structured feed, stop and parse it directly. If two people can’t agree on what a field’s correct value is, fix that definition before you build the extractor, because an un-gradeable field can’t be thresholded.

Write the schema down explicitly, with the validation intent visible from the start:

# The extraction schema IS the contract. Define it before building anything.
INVOICE_SCHEMA = {
    "vendor_name":    {"type": "string",  "required": True,  "threshold": 0.95},
    "invoice_number": {"type": "string",  "required": True,  "threshold": 0.97},
    "invoice_date":   {"type": "date",    "required": True,  "threshold": 0.95},  # ISO-8601
    "po_number":      {"type": "string",  "required": False, "threshold": 0.90},
    "subtotal":       {"type": "money",   "required": True,  "threshold": 0.98},
    "tax":            {"type": "money",   "required": True,  "threshold": 0.97},
    "total":          {"type": "money",   "required": True,  "threshold": 0.99},  # money in: strict
    "line_items":     {"type": "array",   "required": True,  "threshold": 0.92},
}
# Per-field thresholds, not one global number. The total is strict; the PO is lenient.

Those per-field thresholds aren’t guesses to leave at these values forever. They’re a conservative starting point you’ll move with data in Step 5. But writing them into the schema now forces the conversation that matters: which fields, if wrong, cost real money, and which are cosmetic.

Step 1: Ingest from every channel into one queue

Documents arrive by email attachment, scanner, upload portal, and API, often the same document type through several channels. Normalize all of them into one intake queue with a consistent envelope: the raw file, its source channel, a received timestamp, and a unique document ID that follows the document through every later stage. This is unglamorous plumbing and it’s where two real problems hide. First, deduplication: the same invoice arrives twice (emailed and scanned), and posting it twice is a double payment, so hash the file and the key fields and catch duplicates at intake. Second, the envelope: capturing the source channel and timestamp here is what lets you later answer “why did this document get processed at 2am from an unknown sender,” which is a question audit will ask.

def ingest(raw_file, channel):
    file_hash = sha256(raw_file)
    if db.exists("documents", file_hash=file_hash):
        return flag_duplicate(file_hash, channel)        # never post the same doc twice
    doc_id = new_doc_id()
    db.insert("documents", {
        "doc_id":     doc_id,
        "file_hash":  file_hash,
        "channel":    channel,                            # email | scan | upload | api
        "received_at": now(),
        "status":     "queued",
        "raw_ref":    store_blob(raw_file),               # keep the original, always
    })
    queue.publish("capture", doc_id)
    return doc_id

Keep the original file forever, or for as long as your retention policy demands. Every later stage, and every audit, refers back to it. You cannot re-extract from a document you didn’t keep.

Step 2: Capture and read with a document-understanding API

Everything downstream inherits the quality of this step. Run each document through Finigami DocumentAI and keep the structure, not just a flat string of text: word-level text with bounding boxes, detected tables as tables, page layout, and (critically) a confidence score per recognized element. You need the bounding boxes so the human-review UI can show a reviewer the exact crop of the document a low-confidence field came from, and you need tables-as-tables so line items don’t collapse into prose.

Two rules. First, preserve provenance: every recognized element carries its page, its position, and its OCR confidence. That confidence is the raw material the gate later combines with extraction confidence. Second, don’t pre-clean destructively: deskew and denoise the image if it helps the engine, but keep the original, because a reviewer occasionally needs to see what the machine saw.

# Read the document; keep text, layout, tables, AND per-element confidence.
# (Calls are schematic. See the Finigami DocumentAI docs for exact endpoints.)
doc = documentai.read(file=raw, features=["text", "layout", "tables"])  # scans, 50+ langs

elements = []
for page in doc.pages:
    for el in page.elements:
        elements.append({
            "text":       el.text,
            "bbox":       el.bounding_box,        # for the review-UI crop later
            "page":       page.number,
            "ocr_conf":   el.confidence,          # raw read confidence, per element
            "is_table":   el.type == "table",     # a table stays a table, not flattened prose
        })

The bbox and ocr_conf fields aren’t optional bookkeeping. The bounding box is what becomes the highlighted crop a reviewer sees; the OCR confidence is one of the two confidence signals the gate fuses. Discard them here and you cannot add them back: you’d be re-running capture.

Step 3: Classify the document and route to its schema

Determine what the document is, then route it to the extraction schema built for that type. With Finigami DocumentAI you can fold this into the same understanding pass as extraction; either way, get a document-type label and a classification confidence. Route on the label to a type-specific schema (invoice, credit note, statement, claim form, ID document, contract). And treat a low-confidence classification as a first-class exception: an invoice mistakenly run through the statement schema produces confident, well-formed, wrong output, so when the model isn’t sure what the document is, that uncertainty must reach a human before extraction commits to the wrong schema.

def classify_and_route(doc):
    result = documentai.classify(doc)                    # → label + confidence
    if result.confidence < CLASSIFY_THRESHOLD:           # unsure what it even is
        return route_to_human(doc, reason="low_classification_confidence",
                              suggested_type=result.label)
    schema = SCHEMAS[result.label]                       # the right fields for THIS type
    return extract(doc, schema)                          # never extract against the wrong schema

The cheap insurance here is the classification threshold. It costs you a small number of documents going to a human who didn’t strictly need to, and it prevents the expensive failure: a misrouted document producing a clean-looking record that posts wrong data downstream and is hard to catch because nothing about it looks broken.

Step 4: Extract fields with per-field confidence

Pull the fields defined in the type’s schema, each with its own confidence score. Template-agnostic extraction means you’re not maintaining coordinates per vendor; the model finds total because it’s labeled and positioned like a total, across layouts it’s never seen. Map every extracted value back to where it came from on the page (the bounding box from Step 2), so the value is traceable and a reviewer can see it in context. The output of this step is not just values: it’s {value, confidence, source_bbox} per field, which is the unit the entire rest of the pipeline operates on.

The confidence gate, per field: a field clears automatically only if it beats its own threshold and passes the rules. A document posts straight through only when every required field clears. Everything else lands in the human queue with its source crop, and the correction recalibrates the threshold.

The shape that matters: confidence is per field, and the document-level decision is the AND of its fields. One uncertain field holds the whole document, but a reviewer only ever sees the uncertain field, not the eleven that were fine. That’s the difference between “review this invoice” (slow) and “confirm this one smudged amount” (fast), and it’s where the straight-through economics actually live.

Step 5: Validate against business rules and the confidence gate

Now combine three signals into one decision per field, then per document. The signals: the model’s extraction confidence, the OCR confidence from capture, and whether the business rules passed. A field is accepted automatically only when its confidence beats its threshold and every rule that touches it passed. Rules are deterministic and belong in code: line items must sum to subtotal, subtotal plus tax must equal total, the PO must exist in the ERP, the date can’t be in the future, the IBAN must checksum. A failing rule is a precise, auditable reason to flag, and it catches the dangerous case the confidence score misses: a field the model was confident about that’s nonetheless wrong, because the totals don’t reconcile.

def gate(doc, schema):
    decisions = {}
    for field, spec in schema.items():
        ext = doc.extracted[field]                       # value, confidence, bbox
        rules_ok = run_rules(field, doc.extracted)       # deterministic checks
        confident = ext.confidence >= spec["threshold"]  # per-field threshold
        decisions[field] = {
            "value":    ext.value,
            "accepted": confident and rules_ok,          # BOTH must hold
            "reason":   None if (confident and rules_ok)
                        else ("low_confidence" if not confident else "rule_failed"),
        }
    # Document posts straight through only if every REQUIRED field is accepted.
    auto = all(decisions[f]["accepted"]
               for f, s in schema.items() if s["required"])
    return ("straight_through" if auto else "human_review"), decisions

Reserve the reasoning model for what rules can’t express. “Do these three invoices reconcile to this remittance total” is a rule. “Is this an unusual indemnity clause for a contract of this type” is a judgment call, and that’s where you call Claude (Sonnet 4.6 for the per-document volume, Opus 4.8 for the genuinely hard contract reasoning) and treat its output as another signal feeding the gate, never as the final unchecked word on a high-stakes field. Rules first, model only where rules run out.

The thresholds in the schema started conservative on purpose. Now you move them with data. Process a labeled batch, and for each field plot the accuracy of values above each candidate threshold. The right threshold for a field is the lowest one at which the auto-accepted error rate stays under your tolerance for that field. The total might need 0.99 to hit a near-zero error rate; the line-item description might be safe at 0.85. This is the single most important tuning loop in the system, because it directly trades straight-through rate against error rate, and it’s the calibration the deep-dive section below covers in full.

Step 6: Route the uncertain cases to human review

Everything the gate didn’t auto-accept goes to a review queue. The design principle that earns the straight-through economics: show the reviewer the smallest decision, not the whole document. A reviewer looking at a flagged invoice should see the one field that’s uncertain, pre-filled with the model’s best guess, next to the cropped image of exactly where on the page that value came from (the bounding box from Step 2). They confirm or correct it in seconds. They should never have to scroll the whole document hunting for a field, and they should never re-key the fields the model got right. The review UI is where most of the value of the whole project is realized or thrown away, because a slow review queue erases the labor savings the straight-through rate was supposed to deliver.

Every correction does double duty: it fixes this document, and it’s a labeled example that feeds back to recalibrate thresholds and improve extraction. The write-back loop is what makes the system get better with use instead of staying static. More on this in the human-in-the-loop deep dive below.

Step 7: Post the structured result downstream

A document that cleared (auto, or after review) gets written to its system of record: the ERP for an invoice, the claims system for a claim, the customer record for an onboarding packet. Two production details close out the step. First, make the post idempotent and tied to the document ID, so a retry after a network blip doesn’t double-post. Second, store the full result, not just the values: the per-field confidence, the validation outcomes, whether it was auto or human-reviewed, who reviewed it, the model and schema versions, and the pointer to the source document. That record is your audit trail, and in a regulated workflow you will be asked to produce it.

def post(doc, decisions, route):
    record = {
        "doc_id":        doc.id,
        "fields":        {f: d["value"] for f, d in decisions.items()},
        "confidence":    {f: doc.extracted[f].confidence for f in decisions},
        "route":         route,                          # straight_through | human_review
        "reviewed_by":   doc.reviewer or None,           # null if auto
        "model_version": doc.model_version,              # for regression analysis later
        "schema_version": doc.schema_version,
        "source_ref":    doc.raw_ref,                    # the original document
        "posted_at":     now(),
    }
    erp.upsert(key=doc.id, payload=to_erp_format(record))  # idempotent on doc_id
    db.insert("processed_documents", record)               # the audit trail

Document classification & routing, in depth

Classification is the decision that most quietly determines whether the rest of the pipeline operates on truth or on confident fiction, and it’s the one teams treat as trivially solved. It isn’t, for a specific reason: the document types that matter most in a real corpus look almost identical to each other. An invoice, a statement of account, a credit note, and a remittance advice share a vendor header, a table of amounts, and a total. A passport, a national ID card, and a driver’s license share a photo, a name, a date of birth, and a document number. The hard part of classification is never the easy types; it’s the near-twins, and getting a near-twin wrong sends a real document to the wrong extractor.

The field, worst to best for typical enterprise documents:

Strategy	How it decides	Best for	The catch
Sender / channel rules	”anything from this inbox is a claim”	a channel dedicated to one type	breaks when a sender mixes types in one email
Keyword / regex on text	looks for “Invoice” or “Statement”	distinct types with reliable headers	the near-twins share the same words
Filename / metadata	trusts the uploader’s label	cooperative internal users	external senders mislabel constantly
Layout classifier	reads the visual structure	visually distinct document classes	extra model to evaluate and maintain
Joint classify + extract (+)	one understanding pass labels and extracts	many types, blurry boundaries, scale	one model to evaluate for both jobs

The decision rule is short. If your channel genuinely segregates types (a dedicated claims intake address that only ever receives claim forms), trust the channel and skip a classifier, but verify the assumption holds, because the day a sender forwards an invoice to the claims address, channel-trust posts it as a claim. For everything else, default to a layout-aware classifier, and as the type count grows and the boundaries blur, fold classification into the same understanding pass as extraction so the model uses one consistent layout signal for both jobs.

Two things make classification reliable in production. The first is the low-confidence exception. A classifier that’s forced to pick a label even when it’s unsure will pick the closest known class, which is precisely how a brand-new document type silently routes to whatever existing type it most resembles and produces confident garbage. Give the classifier permission to say “I’m not sure,” route those to a human, and you both catch the near-twins and discover new document types as they appear, because a rising rate of low-confidence classifications is the fingerprint of a format you’ve never seen. The second is per-type accuracy in your eval. A global classification accuracy of 97% can hide a 60% accuracy on credit notes if credit notes are 3% of volume, and credit notes misclassified as invoices are exactly the errors that post a negative as a positive. Measure classification accuracy per type, with a confusion matrix, so you can see which types bleed into which, and route the bleeding pairs (invoice/credit-note, statement/remittance) through tighter thresholds or an explicit disambiguation rule.

Routing is the cheap half of this section and the easy place to be sloppy. Once you have a confident label, route to the type’s extraction schema and nothing else. Resist the temptation to run one universal “extract everything” schema across all types, because a schema that tries to pull invoice fields and claim fields and ID fields from every document invites the model to hallucinate a field that isn’t there. The schema is a constraint, and a tight, type-specific schema is what tells the extractor “these eight fields exist on this document, find them” rather than “find anything that looks like a field,” which is the prompt for confident invention.

Extraction, confidence scoring & validation, in depth

This is the section the whole system turns on, because the output here, a value and a calibrated confidence per field, is what every later decision consumes. Three things have to be true for the system to work: the extraction has to find the right value, the confidence has to mean what it says, and the validation has to catch the errors confidence misses. Get any one wrong and the straight-through rate is either dangerously high (errors posting automatically) or uselessly low (everything going to a human).

Extraction itself, template-agnostic, is the mature part. The model reads layout and labels to find a field across formats it has never seen, which is what frees you from per-vendor templates. The two extraction failure modes to watch for are span errors and structure errors. A span error pulls the wrong text for the right field: the “ship to” address into the “bill to” field, a line-item amount into the total. A structure error mishandles repeating or nested data: a multi-page table whose rows split across pages, line items that wrap, a claim with multiple claimants. Both show up in your eval as field-level accuracy below your bar on specific fields, which tells you exactly where to focus rather than letting you conclude vaguely that “extraction is bad.”

Confidence is the part that makes IDP different from “an LLM reads a PDF,” and it has to be calibrated, not just present. A calibrated confidence means the score matches the empirical accuracy: fields reported at 0.9 are right about 90% of the time, fields at 0.99 about 99% of the time. You verify this with a reliability check: take a labeled batch, bucket every field’s prediction by its reported confidence (0.5–0.6, 0.6–0.7, and so on), and compare the bucket’s reported confidence to the actual accuracy of the predictions in it. If the 0.9 bucket is actually right 75% of the time, your confidence is overconfident and every threshold you set on it is a lie, so you either recalibrate (a monotonic mapping from reported score to empirical accuracy, fit on the labeled batch) or you raise your thresholds to compensate. This calibration step is the one teams skip, and skipping it is why some IDP deployments quietly post errors: they set a threshold of 0.9 believing it means 90% accuracy, when on their documents and their model it meant 75%.

def calibration_report(labeled_batch, field):
    # Bucket predictions by reported confidence; compare to actual accuracy.
    buckets = defaultdict(lambda: {"n": 0, "correct": 0, "conf_sum": 0.0})
    for ex in labeled_batch:
        pred = ex.extracted[field]
        b = round(pred.confidence, 1)                    # 0.1-wide buckets
        buckets[b]["n"] += 1
        buckets[b]["conf_sum"] += pred.confidence
        buckets[b]["correct"] += (pred.value == ex.gold[field])
    # A well-calibrated field: reported ≈ actual in every bucket.
    return {b: {"reported": v["conf_sum"] / v["n"],
                "actual":   v["correct"] / v["n"],
                "n":        v["n"]}
            for b, v in sorted(buckets.items())}

def choose_threshold(labeled_batch, field, max_error_rate):
    # Lowest threshold whose auto-accepted error rate stays under tolerance.
    for t in [round(x, 2) for x in arange(0.99, 0.50, -0.01)]:
        accepted = [ex for ex in labeled_batch if ex.extracted[field].confidence >= t]
        if not accepted:
            continue
        errors = sum(ex.extracted[field].value != ex.gold[field] for ex in accepted)
        if errors / len(accepted) <= max_error_rate:
            return t                                      # this field is safe here
    return 0.99                                           # nothing safe → strictest, low STP

One more property of confidence deserves naming, because it changes how you set thresholds: the two ways a confidence decision can be wrong have very different costs. A false accept is a wrong field that cleared above threshold and posted automatically; it lands silently in your system of record and someone finds it later, expensively, often in a reconciliation or an audit. A false reject is a correct field that fell below threshold and went to a human who confirmed it was right all along; it cost you a few seconds of review and nothing else. These are not symmetric, and the asymmetry is the whole reason to set thresholds conservatively at first. A false reject is cheap and self-correcting (the reviewer’s confirmation becomes a label that tells you the threshold was too strict for that field). A false accept is expensive and invisible until it surfaces. So you tune toward the side of more false rejects early, accept a lower straight-through rate, and let the accumulating corrections show you exactly how far you can safely lower each field’s threshold. The fields where false accepts are catastrophic (a payment amount, a bank account number for a new payee) stay strict the longest; the fields where a false accept is merely annoying loosen first. This is the same logic a fraud team uses on a transaction-blocking threshold, applied to a field-posting one.

Validation is the third leg, and it exists because confidence and accuracy are not the same thing. A model can be perfectly confident about a perfectly-read value that is nonetheless wrong in context: it read the total correctly off the page, but the page’s total is itself a typo and doesn’t match the line items. Confidence can’t catch that; a reconciliation rule can. So validation is where you encode the relationships the document should satisfy, independent of how confident the reading was:

Internal consistency. Line items sum to subtotal; subtotal plus tax equals total; quantities times unit prices match line amounts. A document that fails its own arithmetic is flagged regardless of confidence.
Cross-reference. The PO exists in the ERP and has open budget; the vendor exists in the vendor master; the claimant’s policy is active. These catch valid-looking values that don’t correspond to reality.
Format and plausibility. Dates parse and fall in a sane range (an invoice dated three years in the future is wrong even at high confidence); IBANs and tax IDs checksum; amounts are positive where they must be.
Duplicate detection. This invoice number from this vendor hasn’t been posted before. The most expensive AP error is paying the same invoice twice, and no extraction confidence prevents it; a uniqueness rule does.

The relationship between the three is the design insight of the whole section: confidence tells you how sure the model is about what it read; validation tells you whether what it read makes sense; and you need both, fused, because each catches what the other misses. A field auto-accepts only when it’s both confident and consistent. That AND is what keeps the dangerous case, confident-but-wrong, out of your straight-through stream.

Human-in-the-loop and the straight-through-processing rate

The straight-through-processing rate is the number this entire system exists to move. It’s the share of documents that clear from intake to downstream post with zero human touch, at your required accuracy. Every other metric (field accuracy, extraction precision, calibration) matters because of how it moves the STP rate, and the STP rate matters because it’s the direct translation of the system into money: a document that clears straight through cost you cents of machine time, and a document that needs review cost you a human’s minutes, which is one to two orders of magnitude more.

So the design goal is not “maximize accuracy.” It’s “maximize the share of documents safely automated, while keeping the error rate on the automated share under a bar you’ve committed to.” Those two are in tension, and the confidence threshold is the dial between them. Loosen thresholds and more documents clear automatically (higher STP) but more errors slip through; tighten them and fewer errors slip through but more documents go to humans (lower STP). The whole art is finding, per field, the threshold that maximizes STP subject to the error bar, which is exactly what choose_threshold above does.

Here’s the part teams get backwards. They chase a higher STP rate by loosening thresholds, and they post more errors. The right way to raise STP is to make the extraction and calibration better so that more fields are genuinely confident, not to lower the bar for what counts as confident. A system at 70% STP with a 0.5% error rate is healthier than one at 90% STP with a 4% error rate, because the second one is shipping errors into your ledger that someone will find later, expensively. STP without an error bar attached is a vanity metric.

The human side of human-in-the-loop is where the savings are realized or squandered. Three principles make the review tier earn its keep:

Field-level review, not document-level. Surface the one uncertain field with its source crop and the model’s best guess, not the whole document. A reviewer confirming one smudged amount in eight seconds is the economics working; a reviewer re-reading a whole invoice because one field was flagged is the economics broken.
Route to the right reviewer. Not every flag needs a senior person. A low-confidence address can go to a clerk; an unusual contract clause goes to legal; a flagged claim over a threshold goes to a senior adjuster. Matching the flag’s difficulty to the reviewer’s level keeps your expensive people on the expensive cases.
Every correction is training data. The reviewer’s correction fixes the document and becomes a labeled example. Feed it back: it recalibrates the threshold for that field, it’s a new row in your eval set, and over time it’s what lets you safely loosen thresholds because you now have evidence about where the model is and isn’t reliable. A human-in-the-loop system that doesn’t capture corrections as data is leaving its main compounding advantage on the table.

There’s a maturity curve worth naming. Early on, your STP rate is deliberately low: you’ve set conservative thresholds, lots of documents go to humans, and that’s correct, because you’re gathering the labeled corrections that tell you where it’s safe to automate. As corrections accumulate and you recalibrate, the safe-to-automate set grows and the STP rate climbs, not by lowering the bar but by earning the confidence to clear more documents. A realistic invoice deployment might start at 50–60% STP and climb past 85% over a few months as the feedback loop does its work. The teams that succeed treat that climb as the actual project, with the human reviewers as the engine of improvement rather than a cost to minimize on day one.

Security, access control & the data perimeter

IDP touches your most sensitive documents (financials, IDs, medical records on claims, contracts), so security is a design constraint from line one, not a phase you bolt on before launch.

Access control belongs on the data, enforced in the query. The extracted records and the source documents carry the same permissions as the originals: an invoice a user couldn’t open in the ERP, they shouldn’t be able to retrieve from your processed-documents store either. Store each record’s access tags on the row and filter inside the query, so a reviewer or a downstream consumer can never pull a document they couldn’t see directly. The review queue needs the same discipline: a claims reviewer should see the claims in their book of business, not every claim in the system. Never rely on the application layer alone; enforce at the data layer where it can’t be bypassed.

Decide your data perimeter explicitly. Three things leave your environment in an IDP system unless you stop them: the document images (to the capture/extraction API), the extracted text (to any reasoning model you call for validation), and the structured results (wherever they’re stored). For each, decide what’s allowed to go where and write it down. The options run from fully managed APIs (fastest to ship, data leaves your VPC) to self-hosted models and stores (slowest to stand up, nothing leaves). Most enterprises land in between: a managed extraction API under a zero-retention agreement, a managed reasoning model under the same, and the extracted data and audit trail in their own database. Finigami DocumentAI’s API-first model fits that middle path, and running pgvector in your own database keeps the structured results and the audit trail inside your perimeter.

PII and redaction. Documents in an IDP pipeline are dense with regulated data: account numbers, national IDs, dates of birth, medical details on claims. Decide field by field what you actually need to retain. Often the right move is to extract a field, use it for validation or routing, and then store only a masked or hashed version (the last four digits of an account number, a hash of the national ID for duplicate-checking), so a breach of your processed-documents store doesn’t expose the full identifiers the source documents contained. You cannot leak what you never stored in the clear.

Audit what the system did. In regulated workflows (financial controls, insurance, KYC) you will be asked why a document was processed the way it was: why it auto-posted, who reviewed it if it didn’t, what the confidence was, which model and schema version produced it. Because you store the full result per document (Step 7), you can answer that and retain the evidence. Build retention and redaction into the audit store from the start; adding it after an examiner asks is the expensive path.

Complexity management

IDP systems rot in a predictable way: every accuracy problem looks like it needs a new model, a new rule, or a new document-type branch, and a year later you have a pipeline of special cases nobody can reason about. Resist it.

Defer everything until a metric demands it. A second extraction model for hard cases, a custom-trained classifier, an LLM ensemble: all real techniques, all premature until your field-level eval shows the simple pipeline failing on a specific, named field or document type. Add the component that fixes that failure, measure the delta in field accuracy and STP, keep it only if it earns its place.
One change at a time. Tuning IDP is empirical. If you change the extraction model and three thresholds together and the STP rate moves, you’ve learned nothing about which change did it. Change one variable, re-run the eval, record the delta.
Keep the routing shallow. Every document-type branch and every special-case rule doubles a slice of your test surface. Sometimes a new branch is necessary; it’s never free, and a thicket of “if this vendor, do that” rules is how an IDP pipeline becomes unmaintainable. Prefer a general extractor with good confidence over a forest of vendor-specific patches.
Version the model and schema on every record. An extraction-model upgrade or a schema change produces records that behave differently. Store the versions that produced each record so you can tell, when the error rate moves, whether it’s a regression from a change you made or drift in the incoming documents.

A concrete version of how this goes wrong: a team sees invoices from one large vendor extracting poorly, concludes it needs a vendor-specific template for that vendor, and builds one. It works, so they build another for the next problem vendor, and a year later they’re maintaining forty templates, which is exactly the templated-OCR maintenance burden that template-agnostic extraction was supposed to eliminate. The eval would have shown that the real problem was a single field (the vendor used a non-standard date format) fixable with one normalization rule that helped every vendor at once. The discipline is the same one that keeps any system maintainable: the cost of a component is not building it, it’s owning it forever. For the executive framing of where to draw the build-versus-buy line on document work, see which AI agents to build for accounting.

Evaluation & quality

This is the section that decides whether you have a product or a science fair project. Build the field-level eval harness before you tune anything. Without it, you’re adjusting thresholds and asking the room whether the extractions “feel better.” With it, every change has a number attached, per field.

IDP has three quality surfaces, and you measure each separately because the fixes live in different parts of the pipeline:

Extraction quality, per field. Did each field come out right? Measure field-level accuracy (exact match against the gold value, per field), and where a field can be partially right (a multi-line address), a normalized similarity. Break it down by field and by document type, because “94% accurate” hides that the total is at 99.5% and one line-item field is at 78%, and you only care about fixing the second.
Extraction precision and recall, for fields that can be present or absent. For optional and repeating fields (line items, multiple claimants), precision is “of the values you extracted, how many were correct” and recall is “of the values that were there, how many you found.” A system that invents line items has a precision problem; one that misses them on multi-page tables has a recall problem, and the fixes differ.
The straight-through rate at the error bar. The system-level metric: at your chosen thresholds, what share of documents clear automatically, and what’s the error rate on that auto-cleared share? These two move together as you tune thresholds, and reporting one without the other is meaningless. A high STP with an unmeasured error rate is the failure dressed as success.

How to actually run it:

Build a golden test set. 200–500 real documents per type, each with every field labeled with its correct value by a domain expert. This is the single most valuable asset in the project. Mine it from real production documents (with the messy scans and odd formats included, not a cherry-picked clean set), because a golden set of clean documents tells you you’re winning right up until production’s scans prove you weren’t.
Score automatically, per field. Run the pipeline over the set and compute field-level accuracy, precision/recall for present-or-absent fields, the calibration report per field, and the STP rate and auto-cleared error rate at your current thresholds. Use an eval framework to track it over time; Langfuse holds the per-document traces and the field scores together.
Gate releases on it. Every change to the pipeline (a model upgrade, a new rule, a threshold move) runs the eval. A change that raises the STP rate while quietly raising the auto-cleared error rate is exactly what the harness exists to catch, because that trade is the one that posts errors into production.

The harness is small. Each row knows the document, its type, and the gold value of every field:

def evaluate(test_set, pipeline):
    rows = []
    for ex in test_set:
        result = pipeline.process(ex.document)           # classify → extract → gate
        for field, gold in ex.gold.items():
            pred = result.fields.get(field)
            rows.append({
                "doc_type":   ex.doc_type,
                "field":      field,
                "correct":    pred is not None and pred.value == gold,   # field accuracy
                "extracted":  pred is not None,            # for precision/recall
                "present":    gold is not None,
                "confidence": pred.confidence if pred else 0.0,
                "auto":       result.route == "straight_through",        # for STP
                "rule_flag":  result.rule_failed,
            })
    return aggregate(rows)   # → per-field accuracy, precision/recall, calibration, STP + error

Read the output as a diagnosis, not a grade. A run that reads field accuracy 0.96 overall, with total at 0.995 and line_item_desc at 0.81, calibration tight on money fields but overconfident on descriptions, STP 0.78 at a 0.4% auto-error rate, tells a specific story: money fields are production-ready and clearing reliably; the description field is both less accurate and miscalibrated, so it’s dragging the STP rate down by flagging documents and, worse, the times it doesn’t flag, it’s overconfident. The fix is targeted at that one field (better extraction or recalibration for descriptions), not a global model swap. Each metric points at a different box. That’s the whole reason you measure them separately.

The closed evaluation loop: an offline golden set plus a stream of human corrections drive measured, one-variable-at-a-time changes, gated on field accuracy, calibration, and the straight-through rate at its error bar.

Monitoring & observability

Offline evals tell you the system was good at release. Monitoring tells you it still is. Production IDP degrades quietly: vendors change invoice formats, a new document type starts arriving, scan quality drops when a branch buys a cheaper scanner, an upstream system starts sending PDFs without a text layer. Instrument for it.

Trace every document end to end with a tool like Langfuse (langfuse.com): the source channel, the capture output and OCR confidence, the classification and its confidence, every extracted field with its confidence and bounding box, the validation results, the gate decision, the human correction if any, and the cost and latency per stage. When a document is processed wrong, you need to see which stage failed (capture, classification, extraction, validation) in one trace, not reconstruct it from logs.

Watch four families of signal:

Signal	What it catches	Example alert
Quality	field-accuracy drift, miscalibration creeping in	sampled field accuracy on a type drops below its floor
Throughput	STP rate falling, review queue backing up	STP rate down week over week; queue age past SLA
Capture / classification	format changes, scan-quality drops, new doc types	OCR confidence trending down; low-classification-confidence rate rising
Cost	per-document cost creeping, human minutes climbing	cost-per-document above budget; review time per doc rising

Set the floors from your launch baseline, not from round numbers. Your golden-set field accuracy and your STP rate at release are the lines; alert when live-sampled accuracy or the live STP rate drifts below them by more than a small margin. The single most useful operational signal is the STP rate itself, because it integrates everything: when capture quality drops or a new format arrives, more documents fail the confidence gate and the STP rate falls before anyone files a complaint. A falling STP rate is the smoke; the trace tells you the fire.

Two operational habits separate teams that hold quality from teams that file incidents. First, sample auto-cleared documents and re-check them against a human or a stronger model on a schedule; that’s your early warning that the auto-stream is still accurate, and it catches the dangerous silent case where confident extractions started drifting wrong. Second, feed every human correction back into the golden set and the calibration, so the system can’t break the same way twice and so your thresholds stay honest as the document mix changes.

When a quality or throughput alert fires, triage in pipeline order. Pull the offending documents and ask, in sequence: did capture/OCR degrade (scan quality, a format without a text layer)? Was the document misclassified, sending it to the wrong schema? Was a field extracted wrong despite good capture? Or did a validation rule miss something it should have caught? The trace (channel, capture output, classification, fields with confidence, rule results, gate decision) answers each in seconds. Resist the reflex to retune thresholds first; a threshold change papers over a capture or classification regression instead of fixing it, and you’ll pay for it when the next format arrives.

What to monitor continuously: drift

Point-in-time alerts catch a sudden break. The slower danger is drift, the gradual divergence between the document stream you launched on and the one flowing today. Four kinds are worth a chart each.

Format drift. Vendors redesign invoices, agencies revise claim forms, a government updates an ID layout. Watch per-vendor or per-source extraction accuracy and the rate of new layouts. A specific vendor’s accuracy quietly falling is the fingerprint of a format change, and it’s invisible in the global number until that vendor is a big enough share to drag it down.

Document-mix drift. The proportions of document types shift as the business changes: more credit notes after a product recall, more onboarding packets after a marketing push. Track the type distribution and the rate of low-confidence classifications. A growing cluster of “unsure” classifications is often a new type you don’t yet have a schema for.

Calibration drift. The relationship between confidence and accuracy can move as the document mix changes, which silently breaks your thresholds. Re-run the calibration report on sampled live traffic on a schedule; if the 0.95 bucket’s actual accuracy has slipped, your thresholds are now too loose and the auto-stream is posting more errors than you signed up for.

STP and queue drift. Chart the STP rate, the review-queue depth, and the average review time per document, week over week. A falling STP rate raises labor cost; a rising review time means reviewers are working harder per document, often because the flagged cases are getting genuinely harder (a sign of format drift upstream) or because the review UI degraded.

Set rolling baselines rather than fixed thresholds for these. “STP fell four points below the trailing four-week mean” is a better alert than a hard floor, because it catches the gradual erosion that a static threshold sits above until it becomes a crisis. The weekly review from the team section is what ties it together: a human reads the drift charts and the worst traces and decides whether it’s noise or the start of a regression.

Rollout & the additional work

The extraction model is maybe 30% of the work to ship. Here’s the other 70% that turns a working prototype into something you can put in front of real document volume.

Phase the rollout. Never big-bang it.

Shadow mode. The system processes real documents in parallel with the existing manual process, and posts nothing. You compare its output to what the humans did. Goal: measure real-world field accuracy and STP on production documents, find the failure classes your eval set missed, and build trust before a single automated post.
Human-in-the-loop on everything. The system processes and proposes, a human approves every document before it posts. Slower than the end state, but every approval is labeled data, the error rate of automated posting is zero (a human checked each one), and you’re gathering the corrections that let you raise thresholds safely.
Straight-through for the safe slice. Turn on auto-posting only for the fields and document types where your eval and your shadow data show the error rate under the bar. The total on standard invoices might auto-post first; the line items on complex multi-page invoices stay human-reviewed longer. STP climbs as evidence accumulates.
Broad STP. Expand the auto-posted set type by type and field by field as the metrics hold, always with the human queue catching the exceptions.

Each transition is a go/no-go gate, not a calendar date. Shadow → human-in-the-loop when the measured field accuracy clears your bar on production documents. Human-in-the-loop → partial STP when the auto-accept error rate on the safe slice is under tolerance and stable. Partial → broad when the live STP rate and error rate hold at target for two stable weeks. If a gate doesn’t clear, you don’t expand; you fix and re-measure. Turning on straight-through processing on a date instead of a number is how a miscalibrated system starts posting errors into your ledger at scale.

The work teams forget to budget:

The human-review tooling. The field-level review UI with source crops is not a nice-to-have; it’s where the STP economics are realized. A clunky queue erases the labor savings. Budget real engineering for it.
Downstream integration. Posting to the ERP, the claims system, or the customer record is its own project: idempotency, error handling, the mapping from your schema to theirs, and what happens when the downstream system rejects a post. The “last mile” is where a lot of IDP projects stall.
Access control and the data perimeter. The corpus is sensitive. Filter the review queue and the stored records by the user’s permissions, and decide what leaves your environment, before launch, not after.
Exception handling beyond confidence. Some documents will be genuinely unprocessable (a blank page, a wrong document type with no schema, a corrupt file). Design the dead-letter path for these so they reach a human cleanly instead of silently failing.
Audit and compliance. In regulated workflows you’ll need to show why each document was processed as it was. That’s another reason to store the full per-document result and version everything.

Team & skills required

You do not need a research team. You need a small group that can build a pipeline, measure it honestly, operate it, and a domain expert who can say whether a result is right.

Role	What they own	Commitment
ML / AI engineer	The pipeline: capture, classification, extraction, the gate, evals	Core, full-time
Backend engineer	Ingestion, the data store, downstream integration, access control	Core
Domain expert / SME	The golden set, field correctness, the validation rules	Part-time, non-negotiable
Operations / review lead	The review queue, reviewer staffing, the correction loop	Core through launch, ongoing
Product / design	The field-level review UI, the feedback loop	Part-time, high-impact
DevOps / platform	Deployment, monitoring, cost controls	Shared

The role people skip is the SME, and it’s the one that decides success. Engineers can build extraction; only a domain expert can tell you whether “the effective date of this contract” is the signing date or the date the obligations begin, and only they can write the validation rules that encode how the business actually checks these documents today. Budget their time explicitly, a few hours a week building the golden set and curating the rules, or you’ll ship something that looks right to engineers and wrong to the people who process these documents for a living. The second role teams under-resource is the operations lead who owns the review queue, because the human-in-the-loop tier isn’t a temporary scaffold to remove; it’s a permanent part of the system that needs staffing, routing logic, and someone accountable for keeping review time low. For the broader org question of which document workflows to automate first, see which AI agents to build for accounting.

How the team operates matters as much as who’s on it. Run a weekly review: the engineer brings the week’s field-accuracy and STP deltas, the operations lead brings the review-time and queue trends, the SME spot-checks a sample of auto-cleared documents (to confirm the auto-stream is still accurate) and a sample of flagged ones (to confirm the flags are catching real problems), and the group commits to the next single change to try. That ritual, small, regular, and evidence-led, is what keeps the STP rate climbing instead of the system drifting. The failure mode is the opposite: a launch, then silence, then a quarter later someone notices the error rate crept up or the queue backed up, and nobody can say when or why.

A 30/60/90-day delivery plan

A realistic path from zero to a defensible pilot on one document type.

Days 1–30, prove extraction. Pick the highest-volume, most-painful document type (usually invoices). Define the schema and the field-level correctness rules with an SME. Stand up ingestion, capture and extraction (Finigami DocumentAI), classification, and the confidence gate. Build the first 200-document golden set from real production documents, scans and odd formats included. The goal by day 30 is field-level accuracy you’d bet on, per field, because the gate and the human-in-the-loop are only as good as the extraction underneath them.

Days 31–60, make it decide and measure it. Add the validation rules, calibrate the per-field thresholds against the golden set, and build the field-level human-review UI with source crops. Wire up per-document tracing and the eval harness. Run shadow mode against the live document stream, comparing the system’s output to the manual process, and start the human-in-the-loop phase where the system proposes and a human approves. The goal by day 60 is a measured STP rate at a known error bar on production documents, plus a growing stream of corrections feeding the calibration.

Days 61–90, ship straight-through for the safe slice. Turn on auto-posting for the fields and document types where the data shows the error rate under the bar, with the human queue catching everything else and a kill switch on the auto-stream. Watch the live STP rate and auto-error rate, fold corrections back into the calibration, and tune one variable at a time. The goal by day 90 is a pre-committed success metric, an STP rate at a held error bar, hit, with the evidence in hand to ask for the broad rollout and the next document type.

The shape is deliberate: you don’t turn on straight-through processing until the extraction and calibration are good, and you don’t widen the auto-stream until the metrics hold. Each phase ends at a gate, and the gate is a number, not a vibe.

The business case

Most IDP projects that stall don’t stall on technology. They stall because no one wrote the one page that gets them funded. Here’s that page.

Frame it as labor deflected per document, not “we built AI.” The two framings that get approved:

Cost deflection: “Processing one document manually takes a clerk N minutes and costs roughly $C in loaded labor. At V documents a month, automating the S% that clear straight through saves S% × V × $C/month.” Make the rate conservative and defensible, and use your shadow-mode STP rate, not a hoped-for one.
Risk and speed: “Manual processing has an error rate of E% that costs us in rework, late-payment penalties, and overpayments; it also takes D days, which costs us in early-payment discounts missed and onboarding abandoned. Faster, more accurate processing protects $X.”

Worked example. An AP team processes 4,000 invoices a month. Each invoice takes a clerk about 6 minutes to key and check, at a $30/hour loaded cost, so manual processing costs about $3/invoice, or $12,000/month. The system reaches an 80% straight-through rate at a 0.5% auto-error rate. The 80% that clear automatically (3,200 invoices) cost a few cents each in extraction API plus negligible compute, call it $0.20/invoice all-in, or $640/month. The 20% that go to review (800 invoices) still need a clerk, but only on the one or two flagged fields, so review takes about 1 minute each instead of 6, costing about $0.50/invoice, or $400/month. Total run cost: roughly $1,040/month against $12,000/month of manual labor, deflecting about $11,000/month, or $130K/year, on this one document type. Now halve it for ramp and imperfect adoption: still ~$65K/year against a run cost in the low thousands. The payback is weeks. Bring that number, conservatively derived from your own shadow-mode STP rate, not “AI will make AP faster.”

The per-document cost, modeled. Three components: the extraction API (priced per page, the dominant machine cost, a few cents per page), the optional reasoning-model check (only on the documents and fields that need it, often pennies and often zero), and the human review time on the flagged share (the real variable cost, and the one the STP rate controls). For a document that clears straight through, you pay only the extraction API: cents. For a flagged document, you pay extraction plus a minute or two of a clerk’s time: an order of magnitude more. So the unit cost of your pipeline is dominated by the STP rate, and every point of STP you earn moves documents from the expensive regime to the cheap one. This is why STP, not raw accuracy, is the number on the business case.

Put a cost ceiling and a quality floor on it. Decision-makers approve bounded bets, not open-ended ones. State the target cost-per-document, the monthly ceiling, the STP rate you expect, and the error bar you’ll hold on the auto-cleared stream (e.g. “we auto-post only at a measured field-error rate under 0.5%, or it goes to a human”). The quality floor is what protects the ledger and the brand; the cost ceiling is what protects the budget; and the human-in-the-loop tier is what bounds the downside, because the worst case is a document going to a person, not an error going to the ledger.

Pre-empt the questions you’ll be asked:

Question	Your answer
”Will it post wrong data?”	Per-field confidence thresholds + validation rules + a measured auto-error bar; below it, a human reviews
”What does it cost per document?”	Extraction API + occasional model check + human time on the flagged share; dominated by the STP rate, with a monthly ceiling we alert before breaching
”What happens when it’s unsure?”	It flags the field, not the document, and routes it to a reviewer with the source crop; the correction makes the system better
”How do we prove what it did?”	Every document stores its fields, confidence, validation results, reviewer, and model/schema version: a full audit trail
”What if a vendor changes their format?”	Template-agnostic extraction handles most format change with no work; format drift shows up as a per-vendor accuracy alert, and worst case it just flags more documents to humans

Tune the framing to who’s in the room. A CFO wants the deflected-labor number, the cost-per-document model, and the auto-error bar, conservatively derived. A CISO wants the data-perimeter decision and the access-control story; answer it before you’re asked. A controller or compliance lead wants the audit trail and the assurance that nothing posts above the error bar without a human. An operations leader wants to know the review queue won’t become a new bottleneck. Same project, four different opening sentences. For the executive’s view of where document automation pays off across the business, see AI by industry.

Propose the smallest credible pilot. One document type, one team, an eight-week window, a single success metric agreed in advance (an STP rate at a held error bar). A scoped pilot that hits a pre-committed number is how you get the budget for the next document type and the broad rollout.

Pitfalls & anti-patterns

The failures that recur, in rough order of how much damage they do:

Trusting confidence without calibrating it. Setting a 0.9 threshold and assuming it means 90% accuracy, when on your documents it meant 75%. This silently posts errors and it’s the number-one killer. Calibrate before you threshold.
Treating capture as solved. The demo used clean digital PDFs; production is scans, photos, and faxes. Naive OCR drops digits no downstream step recovers. Budget for real document understanding.
One global confidence threshold. Forces you to choose between auto-posting risky fields and reviewing safe ones. Thresholds are per field: the total is strict, the description is lenient.
No validation rules. Confidence tells you how sure the model is, not whether the value makes sense. A confident total that doesn’t reconcile still needs to be caught. Rules catch what confidence can’t.
Chasing STP by loosening thresholds. The wrong way to raise the straight-through rate. Raise it by improving extraction and calibration, not by lowering the bar for what counts as confident.
Document-level review instead of field-level. Flagging a whole invoice because one field is uncertain, and making a reviewer re-read all of it. Surface the one field with its source crop; that’s where the labor savings live.
Throwing away corrections. Every human correction is labeled data that should recalibrate thresholds and grow the eval set. A human-in-the-loop system that doesn’t capture corrections leaves its main compounding advantage unused.
No audit trail. In a regulated workflow you’ll be asked why a document posted as it did. If you didn’t store the confidence, the validation results, the reviewer, and the versions, you can’t answer.
A universal “extract everything” schema. Running one schema across all document types invites the model to invent fields that aren’t there. Classify first, then extract against a tight, type-specific schema.
Forgetting the downstream integration. Extracting perfectly and then having a human re-key the result into the ERP automates the reading but not the work. The last mile is the point.
Building vendor-specific templates. The slow slide back into the templated-OCR maintenance burden that template-agnostic extraction exists to avoid. Fix the underlying field problem once, for all vendors.

FAQ

What’s the difference between OCR and IDP? OCR turns an image into text; that’s one step. IDP is the whole pipeline that turns a document into trustworthy structured data your systems can act on: capture (which includes OCR), classification, extraction with confidence, validation, the human-in-the-loop decision, and the downstream post. OCR is a component of IDP, not a substitute for it. A system that OCRs a document and hands you a wall of text has done the easy 10%.

Do I still need templates in 2026? No, and avoiding them is the point. Template-agnostic extraction reads layout and labels to find fields across formats it has never seen, which is what lets one system handle the hundreds of vendor layouts a templated approach never could. If you find yourself building per-vendor templates, treat it as a smell: the real fix is usually a single normalization or extraction improvement that helps every format at once.

What straight-through rate is realistic? It depends on the document type and how clean the inputs are, and it climbs over time. A structured, high-volume type like standard invoices commonly starts around 50–60% STP in the first weeks (conservative thresholds, gathering corrections) and climbs past 85% as calibration improves. Messier types (mixed-format claims, photographed IDs) start lower. The number to commit to is whatever your shadow-mode data shows at your error bar, not an industry headline.

How do I set the confidence threshold? Per field, with data. Run a labeled batch, and for each field find the lowest confidence threshold at which the auto-accepted error rate stays under your tolerance for that field. Strict fields where errors cost money (the total) get high thresholds; cosmetic fields get lower ones. Then verify calibration: confirm that the model’s reported confidence actually matches empirical accuracy in each bucket, because an uncalibrated threshold is a number that lies to you.

Will it post wrong data into my system? Only at a rate you choose and measure. A field auto-posts only when its confidence beats its threshold and the validation rules pass; everything else goes to a human. You set the threshold to hold the auto-cleared error rate under a committed bar, you measure that bar continuously, and the human-in-the-loop tier means the worst case for an uncertain document is a person reviewing it, not an error reaching your ledger.

How much does it cost to process a document? For a document that clears straight through: the extraction API (cents per page) plus negligible compute. For a flagged document: that plus a minute or two of a reviewer’s time, which is an order of magnitude more. So your per-document cost is dominated by the straight-through rate, and the lever that controls your unit economics is how many documents you can safely clear without a human, which is exactly why STP is the metric on the business case.

Can’t I just paste the PDF text into an LLM and ask for the fields? For a quick prototype on clean digital documents, yes. For production, no, for two reasons. It has no calibrated confidence to threshold on, so you can’t tell which fields are safe to auto-post, which is the entire IDP value. And it’s weak on the scanned, skewed, table-heavy documents that make up much of a real corpus, because a raw text paste loses the layout a document-understanding model uses. The LLM-only approach demos well and doesn’t survive the second week of real documents.

How do I handle documents in many languages? A capable capture-and-extraction layer reads many languages out of the box; Finigami DocumentAI handles 50+. The reading is rarely the hard part. The downstream normalization is: dates, number formats, and addresses differ by locale, and your validation rules and schema have to normalize them (an ISO-8601 date, a canonical amount) so a German invoice and a US invoice produce the same shape of record.

How do I deal with handwriting? Handwriting is the hardest capture case and the one where confidence and the human-in-the-loop tier earn their keep most. A good capture layer reads printed text near-perfectly and handwriting unevenly, so handwritten fields tend to come back with lower confidence, which is correct: they flag to a human more often. Don’t fight to auto-clear handwriting; let the confidence gate route it to review, and accept a lower STP rate on heavily handwritten documents.

How is this different from RAG? Different jobs that often connect. IDP turns documents into structured data and posts it to a system of record. RAG retrieves passages from a corpus to answer questions. They chain naturally: IDP is frequently the layer that produces the clean, structured, well-provenanced text that a RAG system then indexes and retrieves over. If you’re building both, build IDP first, because its output is RAG’s input. See the RAG system guide for the retrieval side.

How long to a production pilot? With a focused team and one document type, a scoped pilot is a matter of weeks, not quarters. The variable that moves the timeline most is input quality: a corpus of clean digital PDFs is fast; a corpus of scanned, multi-language, handwritten-annotated forms takes longer to reach a high straight-through rate, which is the case for picking your first document type carefully and getting the capture decision right early.

Do I need a human in the loop forever? Yes, and that’s a feature, not a failure. The human-in-the-loop tier is what bounds the downside: it catches the documents the system isn’t sure about and the genuinely novel ones, and its corrections are what keep the system calibrated as the document mix drifts. The goal is to shrink the share of documents that need a human (raise the STP rate), not to eliminate the human tier, which would mean either accepting errors or never having a safety net when a new format arrives.

Reference implementation checklist

Ship what’s on this list and you’ll have a system you can defend to a skeptic:

If you’re missing calibration, per-field thresholds, validation, and the field-level review UI, you have a demo. With all fifteen, you have a system.

Executive briefings (for the people who approve this):

AI by industry: where document-heavy work pays off across functions
Which AI agents to build for accounting: the AP and finance document workflows to automate first

Sibling build guides:

How to Build a RAG System in 2026: IDP’s structured output is frequently RAG’s input; the retrieval side, in depth
Agentic Workflow Automation in 2026: when the document is one step in a longer automated workflow