All guides Agents

Agentic Workflow Automation in 2026: A Practical Implementation Guide

A working blueprint for automating multi-step business workflows with AI agents: the deterministic-vs-agentic boundary, orchestration and state, human checkpoints, evals, monitoring, rollout, the team, and the business case to get it approved.

64 min read Advanced

TL;DR

  • Agentic workflow automation is not “RPA with a brain.” It’s a deterministic workflow with a few steps handed to a model because the input is too varied for a fixed rule. The design philosophy that keeps it shippable: deterministic by default, agentic by exception.
  • The single most expensive mistake is making a step agentic when a rule would do. A model deciding what a for loop should decide is slower, costlier, non-reproducible, and harder to audit. Most steps in most business workflows are deterministic; the agentic steps are the minority you can name on one hand.
  • A production system is an orchestrator that runs a durable, retryable workflow, calls deterministic steps and agentic steps through the same tool interface, persists state at every boundary, and stops at human checkpoints where the blast radius of a wrong action is too high to automate.
  • Start with a boring, strong default stack: a durable-workflow engine (LangGraph, or Temporal/Inngest for the execution layer), Postgres for state, Claude for the agentic steps, and Langfuse for traces and evals. Add multi-agent structure only when an eval tells you a single agent can’t hold the task.
  • You cannot improve what you cannot measure. The four numbers that decide whether this ships are task success rate, cost per run, human-intervention rate, and end-to-end correctness. Build the eval harness before you tune anything.
  • The hard part of getting it approved isn’t technical. It’s a one-page business case that frames the workflow as deflected cost, with a cost ceiling, an autonomy ceiling, and a quality floor, plus an honest answer to “what happens when the agent does the wrong thing.”

What agentic workflow automation is in 2026, and what changed

A business workflow is a sequence of steps that turns an input into an outcome. An invoice arrives and ends up posted to the ledger. A new hire is named and ends up with accounts, equipment, and a calendar full of onboarding. A support ticket arrives and ends up in the right queue with the right priority. For thirty years we automated these with rules: if the field says X, route to Y. That works until the input stops being uniform, and real business inputs are never uniform for long.

Agentic workflow automation is what you reach for when some of those steps can no longer be expressed as a rule. Instead of a fixed branch, you hand the step to a model that can read a messy input, decide what to do, and call tools to do it. The rest of the workflow stays exactly as deterministic as it was. The word that matters in the phrase is workflow, not agent: you are automating a multi-step process, and the agent is one kind of step inside it, used where deciding requires judgment a rule can’t encode.

Three shifts made this practical in 2026, and one of them is a correction.

Tool use got reliable enough to trust inside a process. Models now call tools, read structured results, and chain calls with far fewer of the malformed-arguments and hallucinated-function failures that made 2023-era “agents” a demo-only category. A step that says “look up this vendor in the ERP, and if the address doesn’t match, flag it” is now something a model does dependably, not something it does four times out of five. That reliability is what lets an agentic step sit inside a workflow other steps depend on, rather than in a sandbox where a human catches its mistakes.

Durable execution went mainstream, so workflows survive reality. The hard part of automating a real process was never the logic; it was the failures. A step calls an external API that times out. The process crashes halfway through onboarding, after the email account was created but before payroll was set up. Durable-workflow engines (Temporal, Inngest, LangGraph’s checkpointed graphs) made it normal to write a long-running workflow that persists its state at every step, retries the failed step without re-running the successful ones, and resumes exactly where it stopped. This is the unglamorous infrastructure that turns an agent script into a system you can run on Monday’s invoices.

The correction: “give the agent the whole job” mostly failed, and the field moved to constrained autonomy. The 2024 dream was a single autonomous agent you hand a goal and walk away from. In production that pattern is expensive, non-reproducible, and hard to audit, and it fails in ways nobody can explain after the fact. What actually ships in 2026 is the opposite: a tightly scoped agent doing the one or two steps that genuinely need judgment, inside a deterministic workflow that controls everything else. The autonomy is rationed on purpose. The executive version of this argument is in autonomous AI agents for business; this guide is how you build the rationing in.

The anatomy of a failure

When an agentic workflow does the wrong thing, it failed at one of four points. Naming them matters, because the fix is different at each, and because teams reflexively blame the model when three of the four sit elsewhere.

  1. Scoping: a step was made agentic that should have been deterministic. The model “decided” something a rule should have decided, and now the outcome varies run to run for no good reason. No prompt tuning fixes this; the fix is to delete the agent and write the rule.
  2. State and durability: the workflow crashed mid-run and either re-did work it had already done (a duplicate payment) or lost work it had finished (an onboarding that’s half-complete and invisible). The logic was fine; the execution layer wasn’t durable or idempotent.
  3. Tool boundary: the agent had a tool it shouldn’t have, or a tool that did more than the step needed, and it took an action with a blast radius nobody intended. The model behaved reasonably given its tools; the tools were the problem.
  4. Judgment: the step genuinely needed a decision, the agent made it, and the decision was wrong. The input was ambiguous, the context was thin, or the task was past the model’s reliable range. This is the only failure that’s actually about the agent.

Most postmortems of failed automation land on points 1 through 3. The team that built it assumed the agent was the risky part and watched it closely, while the duplicate payment came from a missing idempotency key and the over-broad action came from a tool with more scope than the step required. Your eval and your traces exist to tell you which of the four it was, so you fix the right box instead of tuning a prompt for a durability bug.

When NOT to make a step agentic

Keep a step deterministic, or descope the agent out of it, if any of these is true:

  • The input is uniform and the rule is writable. If every invoice from a vendor has the total in the same place, you don’t need a model to find it; you need a field mapping. A model that re-derives a constant on every run is paying tokens and latency to be less reliable than an assignment statement.
  • The step must be reproducible to the byte. Tax calculations, financial postings, anything an auditor will replay: these must give the same output for the same input, every time. A model is the wrong tool for a step whose whole value is determinism.
  • The decision is high-stakes and the volume is low. If you process three of these a week and getting one wrong is a six-figure event, the economics of automation don’t clear. A human is cheaper and safer than the eval harness you’d need to trust the agent.
  • You can’t articulate what “correct” means for the step. If nobody can write down how to tell a good output from a bad one, you can’t build an eval, which means you can’t tell if the agent works. Fix the definition first, or keep the human.

Agentic vs the alternatives

Before you build, make sure an agent is the right tool for the steps you’re worried about. There are three ways to automate a multi-step process, and the right system usually uses all three as layers rather than picking one as a religion.

ApproachWhat it isBest whenWeakness
RPA / screen automationscripted clicks and keystrokes over existing UIsthe system has no API and the UI is stablebrittle; a layout change breaks it silently
Deterministic workflowcoded steps, fixed branches, explicit rulesinputs are uniform and the logic is writablecan’t handle variation it wasn’t coded for
Agentic stepa model reads the input, decides, calls toolsthe input is varied and the step needs judgmentnon-reproducible; costs tokens; needs evals
Fully autonomous agenthand the whole goal to one agent, walk awayrarely the right answer in business workflowsexpensive, unauditable, fails inexplicably

The decision rule is per step, not per workflow. Walk the process step by step and ask the same question at each one: can I write the rule? If the rule is writable and the input is uniform, the step is deterministic, full stop. If the input is too varied to express as a rule and the step needs to read something and decide, that step is a candidate for an agent. Most steps in most workflows answer “yes, the rule is writable.” The agentic steps are the exceptions, and you should be able to count them on one hand for any given workflow. For the executive framing of how much autonomy to grant and when, see the autonomy spectrum for AI agents.

The common error is adopting one layer as ideology. RPA shops try to script everything and end up with a thousand brittle bots. Agent enthusiasts make every step a model call and end up with a workflow that costs dollars per run, can’t be reproduced, and can’t be audited. The boring answer is a deterministic workflow with two or three agentic steps in it, and that boring answer is what ships and survives.


Use cases & where it pays off

Agentic workflow automation earns its keep when three conditions hold: the workflow is multi-step and runs at volume, at least one step receives input too varied to encode as a rule, and a wrong outcome is cheap enough to tolerate occasionally but expensive enough in aggregate to be worth automating. The patterns that consistently pay back are these.

Use caseWhy it fitsThe agentic step(s)What “good” looks like
Invoice processinghigh volume, messy formats, mostly rulesextract fields from a varied document; match to POstraight-through posting on clean invoices; humans see only exceptions
Employee onboardingmany systems, long-running, brittle to failuresinterpret the role into the right access bundlenew hire fully provisioned day one; nothing half-done
Ticket triagehigh volume, free-text input, clear routing rulesread the ticket, classify intent and priorityright queue, right priority, in seconds, with a confidence escape hatch
Data reconciliationtwo sources that mostly agree, exceptions that don’tjudge whether a near-match is the same entityclean matches auto-resolved; ambiguous ones queued with the evidence
Procurementstructured request, policy-heavy, approval-gatedcheck a request against policy and flag the edge casescompliant requests flow; exceptions escalate with the reason attached

The common thread: the volume is in the boring, rule-following cases, and the value of the agent is that it handles the messy minority without dragging a human into the clean majority. A good agentic workflow does not make a human review everything; it makes a human review only what the workflow couldn’t confidently resolve. If you take one design principle from this guide, take that one.

A worked example, invoice processing, traced end to end. An invoice PDF lands in an inbox. The workflow triggers. Step one is deterministic: detect the attachment, log the run, write an invoices/{run_id} record in the state store. Step two is agentic in the narrow sense that the document is varied: Finigami DocumentAI extracts the vendor, invoice number, line items, and total from a layout the system has never seen before, and returns structured fields with per-field confidence. Step three is deterministic again: look up the vendor in the ERP, pull the matching purchase order, and compare totals. If the extracted total matches the PO within tolerance and the vendor is known, step four posts the invoice to the ledger and the run completes with zero human touch. If the total is off by more than tolerance, or the vendor is new, or a field came back with low confidence, the workflow does not guess. It stops at a human checkpoint, presents the extracted fields next to the PO and the source document, and waits for an approve-or-correct. The human’s correction is written back as training and eval data. Notice how little of that is the agent: one step reads a messy document, and everything around it is rules, lookups, and a checkpoint. That ratio is the whole design.

The same shape recurs across every use case in the table, and it’s worth tracing two more to see the pattern hold. Take employee onboarding, the case where durability matters most because the workflow is long and touches many systems. A new hire’s name and role arrive. The deterministic steps dominate: validate the role against the org’s role catalog, create the identity record, provision the standard account bundle for that role (email, SSO group, the default SaaS seats), enroll in payroll, order the standard equipment, and schedule the onboarding calendar. The one place judgment enters is interpreting a role that isn’t a clean catalog match: a title like “Senior Platform Engineer, Payments” that maps to a known bundle plus two payments-specific systems the catalog doesn’t enumerate. That interpretation is the agentic step, and it’s narrow, mapping a messy title to an access bundle, with everything else a coded provisioning rule. The reason durability is non-negotiable here is the failure mode: this workflow makes a dozen side effects across a dozen systems over several minutes, and a crash after the email account exists but before payroll is enrolled must resume at payroll, not start over and create a second account. Idempotency on every provisioning call is what makes that safe, and a human checkpoint sits before exactly one action: granting access to a system the role doesn’t normally get, where a wrong grant is a security event.

Now data reconciliation, the case that looks like pure rules until you hit the exceptions. Two sources (say a bank feed and the ledger) mostly agree, and the deterministic step matches the clean cases: same amount, same date, same reference, auto-reconciled, no model involved. The agentic step is the residue, the near-matches that a rule can’t settle: a $4,512.10 debit on the bank feed and a $4,512.00 entry in the ledger dated a day apart with slightly different payee strings. Is that the same transaction with a rounding and timing difference, or two different ones? That judgment over an ambiguous, unbounded space of payee-string variations is exactly what a rule can’t enumerate and a model can, and it’s the only agentic step in the workflow. The clean matches (the overwhelming majority) never touch the model; the genuine ambiguities get the agent, and the ones the agent itself isn’t confident about get a human with both records side by side. Three layers, and the volume is in the free deterministic one.


The reference architecture

A production agentic workflow has five parts that never change, whatever the use case: a trigger that starts a run, an orchestrator that runs the steps durably, a set of tools the steps call, the steps themselves (most deterministic, a few agentic), human checkpoints where the workflow pauses for approval, and a state store that persists everything so a crash is a resume rather than a restart. Read the architecture as a process the orchestrator drives, not as an agent that drives the process.

Agentic workflow automation reference architecture A trigger starts a run. A durable orchestrator (LangGraph or a durable engine such as Temporal) drives the workflow, persisting state to Postgres at every step. The workflow runs a mix of deterministic steps and agentic steps; agentic steps call tools and a model (Claude). A human checkpoint pauses the workflow for approval before high-blast-radius actions. Evaluation and observability span the whole run. Highlighted = suggested tool pick 1 · TRIGGER & ORCHESTRATION Trigger event · webhook · cron Durable orchestrator LangGraph / Temporal retries · resume · idempotent State store Postgres persist 2 · THE WORKFLOW: STEPS RUN IN ORDER Deterministic log · validate a coded rule Agentic step read · decide · act Claude + tools the exception, not the rule Deterministic look up · compare a coded rule Human checkpoint approve · correct high blast radius only Outcome posted · logged orchestrator drives each step 3 · TOOLS: THE ONLY WAY A STEP TOUCHES THE WORLD Doc extraction Finigami DocumentAI Systems of record ERP · HRIS · ticketing Scoped, audited tools least privilege · per call Notify email · Slack · queue tool calls Evaluation & observability: spans the whole run task success rate · cost per run · intervention rate · end-to-end correctness · per-step latency Langfuse traces + offline workflow test set
The canonical agentic-workflow architecture: a durable orchestrator drives a mostly-deterministic workflow with a few agentic steps, tools are the only way a step touches the world, and a human checkpoint gates the high-blast-radius action.

Read the diagram top to bottom. The orchestrator and state store are the spine: they make a run durable, so a crash resumes rather than restarts. The workflow lane is the process itself, and the ratio in it is the point: three deterministic steps, one agentic step, one human checkpoint. The tools lane is the only surface where a step touches the world, which is exactly why it’s where access control lives. The two boxes that decide whether you can sleep at night are the state store (does a crash lose or duplicate work?) and the human checkpoint (does a wrong agentic decision reach production unattended?).


Architecture decisions

Seven decisions determine most of the outcome. Make them deliberately; everything else is tuning.

  1. Which steps are agentic. This is the decision that sets everything else, and it’s the one teams get backwards. Walk the workflow step by step and default each step to deterministic. A step earns “agentic” only when its input is too varied to express as a rule and the step requires reading-then-deciding. Document extraction from arbitrary layouts is agentic; comparing two numbers is not. Classifying free-text intent is agentic; routing a known category is not. The rule of thumb: if you can imagine writing the if/else, write it. Every step you wrongly make agentic costs tokens, costs latency, loses reproducibility, and adds a thing to evaluate.
Step looks like…Make it…Why
Read a varied document, free text, an ambiguous matchAgenticThe input space is too large to enumerate as rules
Compare values, apply a known threshold, route a known categoryDeterministicA rule is faster, cheaper, reproducible, auditable
Call an API, write a record, send a notificationDeterministicThese are actions, not decisions; code them
Decide between actions when the input is messyAgenticThe judgment is the value; that’s what the model is for
  1. Single agent vs multi-agent. Default to a single agent doing the agentic steps, with deterministic glue between them. Multi-agent (a planner that delegates to specialist sub-agents) is a real pattern, but it multiplies cost, latency, and the surface where things go wrong, and most workflows don’t need it. Reach for multiple agents only when the task has genuinely distinct skill domains that don’t fit one prompt, or when one agent’s context window can’t hold the whole job. Even then, prefer a deterministic orchestrator calling specialized single-purpose agents over a swarm of agents negotiating with each other, because the former is debuggable and the latter is not.
PatternUse whenWatch out for
Single agent + deterministic glue (default)One coherent judgment domain per stepKeep each agentic step narrow
Orchestrator + specialist agentsDistinct skill domains, large contextCost and latency multiply per agent
Agent swarm (peer negotiation)Rarely the right answerUnauditable, non-reproducible, hard to debug
  1. Orchestration engine. The workflow must be durable: it persists state at every step, retries failures without re-running successes, and resumes after a crash. Default to LangGraph for agent-shaped workflows where the graph of steps and the model calls live together (langchain-ai.github.io/langgraph); reach for a dedicated durable-execution engine (Temporal, temporal.io; or Inngest, inngest.com) when the workflow is long-running, spans many services, and durability is the dominant concern. The anti-pattern is a plain script with a try/except: it cannot survive a process crash, and business workflows crash. Whatever you pick, the non-negotiable property is that a run is a durable, resumable object, not an ephemeral function call.
OptionUse whenWatch out for
LangGraph (default for agent-shaped)Graph of steps + model calls in one placeYou still need a durable backend under it
Temporal / InngestLong-running, multi-service, durability-firstMore infrastructure to operate
Plain script + try/exceptNever for productionNo crash recovery; loses or duplicates work
  1. State and durability. Persist the workflow’s state at every step boundary to Postgres, keyed by a stable run_id, so a crash resumes from the last completed step. Two properties are non-negotiable. Retries: a transient failure (an API timeout) must retry the failed step, not the whole workflow. Idempotency: every step that touches the world (posts a payment, creates an account) must carry an idempotency key, so that retrying it does the action once even if it ran partway before the crash. The classic failure is a workflow that crashes after sending a payment but before recording it, retries, and pays twice. An idempotency key on the payment call makes the second attempt a no-op. This is the difference between an automation you can run unattended and one you can’t.

  2. Human checkpoints and the autonomy ceiling. Decide, per workflow, which actions are too high-blast-radius to take unattended, and put a human checkpoint before exactly those. A checkpoint pauses the durable workflow, presents the proposed action with its evidence, and waits for approve, correct, or reject. The autonomy ceiling is a deliberate setting, not a default: start with humans approving more than they need to, and remove checkpoints as the intervention rate proves the agent is right at that step. The rule for where a checkpoint goes: place it before any action that is expensive to reverse or impossible to reverse, and before any action a regulator or an auditor would expect a human to have seen.

Action typeCheckpoint?Why
Reversible, low value (route a ticket, draft a reply)No (or sample)Cheap to be wrong; auto-resolve and monitor
Irreversible or high value (post a payment, grant access)YesBlast radius too high to take unattended
Anything below a confidence thresholdYesThe agent itself flagged it; route to a human
Regulated or audited decisionsYesA human must be in the record
  1. Model. Default to Claude for the agentic steps: Opus 4.8 for the hardest judgment and multi-tool reasoning, Sonnet 4.6 where the step is well-scoped and latency and cost matter more. The model is among the least important decisions once the workflow is shaped well, because a narrow, well-defined agentic step is forgiving: most capable models do it. Spend your design budget on scoping the step tightly and giving it good tools, not on the model leaderboard. The one behavior worth testing explicitly is tool-use discipline: does the model call the right tool with valid arguments and stop when the step is done, rather than looping or inventing a tool. Alternatives where data residency or cost forces it: a strong open model self-hosted, with the caveat that you re-validate tool-use reliability, which varies more between models than raw answer quality does.

  2. The exception path. Decide up front what happens when a step can’t be completed: low extraction confidence, an ambiguous match, a tool error, a policy edge case. The answer is never “guess and continue.” It’s “stop and route to a human, with the evidence attached.” Design the exception path as a first-class branch of the workflow, not an afterthought, because in a well-built system the exceptions are where all the human attention goes and the happy path is unattended. A workflow with no explicit exception path will improvise one at the worst possible moment.


The reference stack

The boring, strong default. Start here; deviate only when an eval or an operational reality tells you to.

LayerDefault pickWhySwap when
OrchestrationLangGraph (or Temporal/Inngest)Durable graph of steps + model callsLong-running multi-service → Temporal
State storePostgresDurable, transactional, you already run itExtreme scale → a dedicated event store
Agentic stepsClaude Opus 4.8 / Sonnet 4.6Reliable tool use, good judgmentData residency → self-hosted open model
Document extractionFinigami DocumentAIAPI-first, template-agnostic, scanned + 50+ languagesYou only ever see one clean format
Tool layertyped tool interface (MCP or in-process)One audited surface for every action-
Deterministic stepsplain application codeFast, reproducible, auditable-
Eval / observabilityLangfuse + offline workflow test setTraces + scores in one placeEnterprise → Braintrust
Human checkpointsa task queue + approval UIPauses the durable run for a humanOff-the-shelf inbox works to start

A note on frameworks: the framework’s job is durability and the graph of steps, not magic. A heavyweight agent framework that hides the control flow is a liability in a workflow you’ll operate for a year, because the abstractions leak exactly where you need to tune retries, idempotency, and checkpoints. Pick the lightest engine that gives you durable, resumable runs, and keep the deterministic steps as plain, readable code you fully control.


Cost and latency, end to end

Two numbers decide whether an agentic workflow is viable: what each run costs and how long it takes. Both are dominated by one lever, how many agentic (model) steps a run touches and how much context each one carries, which is why “deterministic by default” is a cost control before it is a design principle. A rough per-run profile for a well-built invoice workflow with one agentic extraction step:

StageLatency (typical)Cost driver
Trigger + state write~10–50 msdatabase time, negligible
Document extraction (agentic)~1–4 sthe driver: model + extraction tokens
ERP/PO lookup + compare (deterministic)~50–300 msAPI time, no model cost
Post to ledger (deterministic)~100–500 msAPI time, no model cost
Human checkpoint (exceptions only)seconds to hourshuman time, not machine time
Total (straight-through)a few secondsthe one agentic step dominates

Read it as a budget. Over on cost? The fix is almost never a cheaper model; it’s fewer agentic steps. Every step you made agentic that a rule could have handled is model tokens billed on every single run, forever, for a worse and less reproducible result. A workflow that “uses the agent to” look up a vendor, compare two totals, and decide a known route is paying three model calls to do what three lines of code do faster and for free. Over on latency? Same lever, plus parallelize the independent deterministic steps and reserve the human checkpoint for the exception path so the happy path never waits on a person. Precision in scoping is cheaper and faster at once, one of the rare places those align.


Implementation

The build, step by step, with the decisions that bite at each one.

Step 0: Map the workflow and label every step

Before any code, write the workflow down as an ordered list of steps, and label each one deterministic or agentic using the rule from decision 1. This is the most important hour of the project. The labeling tells you how many agentic steps you actually have (usually one or two, often zero in workflows teams assumed were “AI projects”), where the human checkpoints go (before the irreversible actions), and what your eval has to measure (the agentic steps and the end-to-end outcome). Teams that skip this build an agent that does the whole job, then spend a quarter discovering that nine of its eleven “decisions” were rules in disguise.

The output of Step 0 is a table, not code. For invoice processing it reads: trigger (det), extract fields (agentic), look up vendor (det), match to PO (det), check tolerance (det), post or checkpoint (det branch), record (det). One agentic step out of seven. That ratio is your design.

Step 1: Stand up the durable orchestrator and state

Everything downstream depends on runs being durable. Model the workflow as a graph of steps in LangGraph (or a durable engine), keyed by a run_id, persisting state to Postgres at every boundary. The property you’re buying is that a crash mid-run resumes from the last completed step instead of restarting, and a retry of a failed step doesn’t re-run the successful ones.

The state row is deliberately boring: the run, its current step, its persisted state, and a status the orchestrator drives.

CREATE TABLE workflow_runs (
    run_id        uuid PRIMARY KEY,
    workflow      text        NOT NULL,        -- 'invoice_processing'
    status        text        NOT NULL,        -- running | awaiting_human | done | failed
    current_step  text        NOT NULL,        -- resume point after a crash
    state         jsonb       NOT NULL DEFAULT '{}',   -- accumulated step outputs
    created_at    timestamptz NOT NULL DEFAULT now(),
    updated_at    timestamptz NOT NULL DEFAULT now()
);

-- Idempotency: a side-effecting step records its key here BEFORE acting,
-- so a retry after a crash is a no-op instead of a double action.
CREATE TABLE step_effects (
    run_id        uuid        NOT NULL REFERENCES workflow_runs(run_id),
    idempotency_key text      NOT NULL,        -- e.g. 'post_ledger:{invoice_id}'
    result        jsonb,
    created_at    timestamptz NOT NULL DEFAULT now(),
    PRIMARY KEY (run_id, idempotency_key)
);

The step_effects table is the unglamorous core of “run it unattended.” Before a side-effecting step acts, it checks for its idempotency key; if the key exists, the action already happened and the step returns the recorded result. That single pattern is what stops a crash-and-retry from paying an invoice twice.

Step 2: Define tools as a typed, scoped, audited interface

A step touches the world only through tools, and a tool is the unit of access control. Define each tool with a typed input and output, the narrowest scope that does the job, and an audit log entry on every call. The agentic step gets only the tools it needs for that step, never a general-purpose “do anything” tool. A tool that posts to the ledger takes a specific invoice and amount and is permitted to post exactly that; it is not a tool that “runs ERP commands.”

# Each tool is narrow, typed, audited, and idempotent where it has effects.
@tool(scopes=["ledger:post"])           # least privilege, enforced at the boundary
def post_invoice(run_id: str, invoice_id: str, amount_cents: int, gl_code: str) -> PostResult:
    key = f"post_ledger:{invoice_id}"
    if existing := lookup_effect(run_id, key):    # idempotency: already done?
        return existing                            # retry is a no-op
    audit.log(run_id, "post_invoice", invoice_id=invoice_id, amount_cents=amount_cents)
    result = erp.post(invoice_id, amount_cents, gl_code)
    record_effect(run_id, key, result)             # record BEFORE returning
    return result

Two rules. First, scope the tool to the step, not the system: the blast radius of the agentic step is exactly the set of tools you handed it, so a narrow tool is a narrow blast radius. Second, audit every call with the run_id, so that when something goes wrong you can replay precisely what the workflow did and why. The audit log is also your compliance story and half your debugging story.

Step 3: Build the agentic step as a narrow, tool-using sub-task

Now the model. The agentic step takes a tightly bounded input, has a clear definition of done, and calls only its scoped tools. Keep the prompt narrow: it describes one job, names the tools, and specifies what to do when it can’t complete the step (which is always “stop and signal an exception,” never “guess”). The narrower the step, the more reliable and the cheaper it is.

def extract_invoice_step(state):
    # The agentic step: a varied document in, structured fields + confidence out.
    doc = state["attachment"]
    extracted = documentai.extract(doc, schema=INVOICE_SCHEMA)   # Finigami DocumentAI
    # The model reconciles ambiguous fields and judges confidence; tools are scoped.
    result = claude_step(
        system=EXTRACT_SYSTEM,          # "Return fields; if a field is unclear, mark low confidence."
        tools=[lookup_vendor],          # only the tools this step needs
        input=extracted,
    )
    if result.min_confidence < CONF_THRESHOLD or result.vendor_unknown:
        return route_to_human(state, reason=result.reason)   # explicit exception path
    return {"invoice": result.fields}

The agentic step does one thing: turn a messy document into structured fields it’s confident about, or hand off to a human with the reason attached. It does not look up the PO, compare totals, or post the invoice. Those are deterministic steps, written in plain code, because they’re rules. Resist the pull to let the agent “just handle the rest”; every deterministic step you fold into the agent is a step you can no longer reproduce or audit.

Step 4: Wire the deterministic steps and the branch

The steps around the agent are ordinary code: look up the vendor in the ERP, pull the matching PO, compare the extracted total against it within tolerance, and branch. The branch is deterministic too: if everything matches and confidence is high, proceed to post; otherwise route to the human checkpoint. The decision of whether to involve a human is a rule, even though the thing that produced the uncertain input was an agent.

def match_and_branch(state):
    inv = state["invoice"]
    po  = erp.find_po(inv.vendor_id, inv.po_number)        # deterministic lookup
    if po is None:
        return route_to_human(state, reason="no matching PO")
    if abs(inv.total_cents - po.total_cents) > TOLERANCE_CENTS:
        return route_to_human(state, reason="total mismatch")
    return "post"     # clean: proceed to the deterministic posting step

This is where most of the workflow’s logic lives, and none of it is a model. The agent supplied a confident extraction; rules do the rest. That division is what keeps the run cheap, fast, reproducible, and auditable.

Step 5: Build the human checkpoint as a durable pause

A human checkpoint is not an input() call. It’s a durable pause: the workflow writes its state, sets status to awaiting_human, emits a task to an approval queue with the evidence, and stops. When the human approves or corrects, the workflow resumes from exactly that step. Because the run is durable, the pause can last seconds or hours without holding a process open, and a crash during the pause loses nothing.

A workflow run with a human checkpoint An invoice triggers a run. The agentic extraction step runs. A deterministic match-and-branch step decides: a clean match proceeds straight through to posting and completion; an exception (low confidence, no PO, or total mismatch) routes to a durable human checkpoint where a person approves or corrects, after which the run resumes and posts. Both paths end at a recorded outcome. Trigger invoice in Extract (agentic) fields + confidence Match & branch deterministic rule Human checkpoint durable pause · approve evidence attached Post to ledger idempotent Outcome posted · recorded exception clean match resume after approve
One run, two paths: a clean match flows straight through unattended; an exception pauses durably at a human checkpoint and resumes after approval. The happy path never waits on a person, and the human only ever sees what the workflow couldn't confidently resolve.

The implementation is a durable interrupt, not a blocking call:

def route_to_human(state, reason):
    # Persist, emit a task with evidence, and STOP. The run survives the wait.
    save_run(state.run_id, status="awaiting_human", current_step="human_checkpoint")
    approval_queue.create(
        run_id=state.run_id,
        reason=reason,                       # "total mismatch", the human sees why
        evidence={"extracted": state["invoice"], "source_doc": state["attachment"]},
    )
    raise WorkflowInterrupt()                 # durable engine parks the run here

def on_human_decision(run_id, decision):
    run = load_run(run_id)
    if decision.action == "approve":
        run.state["invoice"] = decision.corrected_fields or run.state["invoice"]
        resume(run, from_step="post")         # continue exactly where it paused
    elif decision.action == "reject":
        save_run(run_id, status="done", outcome="rejected_by_human")
    record_training_example(run_id, decision)  # the correction becomes eval/training data

Two production details close out the step. The human always sees why the checkpoint fired (the reason) and the evidence (extracted fields next to the source document), so approval takes seconds, not a re-investigation. And every correction is captured: it’s your highest-quality eval data and the signal that tells you when a checkpoint has earned the right to be removed.

Step N: Close the loop with the exception path and recording

The last step records the outcome of every run, clean or exception, with its full trace. This is what feeds evaluation, monitoring, and the slow tightening of the autonomy ceiling. A run that completed straight-through is a data point that the agentic step and the rules held; a run that hit a checkpoint, and what the human did there, is a data point about where the agent still needs supervision. Both go to the trace store. The workflow isn’t done when it posts the invoice; it’s done when it has recorded what happened well enough that you can grade it.


Deterministic by default, agentic by exception

This is the section the whole guide is built around, so it’s worth slowing down. The central design philosophy of agentic workflow automation is a default and an exception. The default is deterministic: every step is a rule until proven otherwise. The exception is agentic: a step becomes a model call only when its input is too varied to express as a rule and completing it requires reading-then-deciding. Get the default right and the rest of the system is straightforward. Get it wrong and you’ve built something expensive, slow, non-reproducible, and unauditable, dressed up as innovation.

The deciding question for each step is one sentence: can I write the rule? Run it honestly and most steps fail the “needs an agent” test, which is the correct outcome. “Compare the invoice total to the PO total within a tolerance” is a rule. “Route a ticket tagged billing to the billing queue” is a rule. “Provision the standard access bundle for a known role” is a rule. None of these need a model, and putting one there makes them worse. The steps that genuinely pass the test share a shape: a large, open input space (an arbitrary document, free-text intent, an ambiguous entity match) and a judgment that resists enumeration. Those, and only those, are agentic.

A useful way to size a step is by its input space. If you can list the inputs the step will see, even a long list, it’s deterministic: write the cases. If the inputs are effectively unbounded (every invoice layout that has ever existed and every one that will), you can’t enumerate them, and that unbounded variety is precisely what a model handles and a rule can’t. The boundary is not “is this hard” but “is the input space enumerable.” A hard calculation over enumerable inputs is still deterministic; an easy-sounding judgment over unbounded inputs (“is this the same company?”) is agentic.

Why the default has to be deterministic, in four concrete failure costs of getting it wrong. Reproducibility: a deterministic step gives the same output for the same input forever, which an auditor can replay and a test can pin; an agentic step doesn’t, so every step you make agentic is a step you can no longer guarantee. Cost: a deterministic step is free to run; an agentic step costs tokens on every execution, multiplied by your volume, forever. Latency: a rule returns in microseconds; a model call returns in seconds, and a workflow of needless model calls is slow for no reason. Debuggability: when a rule is wrong you read the rule and fix it; when an agent is wrong you’re debugging a probability distribution. Four taxes, paid on every run, for every step you made agentic that didn’t need to be.

There’s a subtler version of the mistake worth naming: making a step agentic because it feels like AI work, not because the input demands it. “Use the agent to look up the customer” feels modern; it’s a database query. “Have the agent decide the priority” feels smart; if priority is a function of two known fields, it’s a lookup table. The tell is that you can describe the logic in a sentence with “if.” When you can, you’ve found a rule wearing an agent costume, and taking the costume off makes the system better on all four axes at once. The discipline is to keep asking “can I write the rule?” at every step and to be a little disappointed each time the answer is yes, because a yes is a step that just got cheaper, faster, and more reliable.

A practical heuristic for the decision, applied per step: enumerable input and writable logic means deterministic; unbounded input and judgment that resists rules means agentic; and when you’re genuinely unsure, start deterministic and let a real failure (a class of inputs the rules can’t handle, surfacing in your eval) promote the step to agentic. Promotion on evidence is cheap and reversible. The opposite, building it agentic and discovering it should have been a rule, costs you a quarter and a maintenance burden. Default down, promote on evidence.


Orchestration, state and checkpoints

If “deterministic by default” is the design philosophy, durable execution is the engineering that makes it survive contact with production. Business workflows are long-running and they fail in the middle: an external API times out, a process gets redeployed mid-run, a downstream system is briefly down. An automation you can run unattended has to treat all of that as normal, which means three properties that a plain script does not have.

Durable execution. The workflow persists its state at every step boundary, so the unit of work is a durable, resumable object rather than an ephemeral function call. When the process running it dies, the run isn’t lost; it resumes from the last completed step on another worker. This is the property that distinguishes a durable-workflow engine (Temporal, Inngest, LangGraph with a persistent checkpointer) from a script in a loop. The script’s state lives in process memory and dies with the process. The durable run’s state lives in Postgres and outlives any single worker. For a workflow that takes an hour and touches six systems, this is the difference between “a crash is a non-event” and “a crash leaves a half-onboarded employee nobody can see.”

Retries that don’t redo finished work. A transient failure should retry the failed step, not the whole workflow. If onboarding crashes after creating the email account but before setting up payroll, the retry must not create a second email account. This falls out of durable execution: because each completed step’s result is persisted, the retry resumes at payroll and skips the email step entirely. The anti-pattern is wrapping the whole workflow in one try/except that re-runs everything on any failure, which turns one flaky API into a cascade of duplicate side effects.

Idempotency on every side effect. Retries and durability are necessary but not sufficient, because a step can crash after its side effect lands but before its result is recorded. The payment went through; the crash happened before the workflow wrote “paid”; the retry tries to pay again. The fix is an idempotency key on every world-touching action, recorded before the action and checked before each attempt, so the second attempt finds the key and becomes a no-op. The step_effects table from Step 1 is exactly this. Every step that posts, creates, grants, or sends needs one. A workflow without idempotency is a workflow you cannot safely retry, which means a workflow you cannot run unattended, which means not really an automation at all.

Checkpoints as durable pauses, not blocking waits. A human checkpoint is a special kind of resume point: the workflow parks itself with status awaiting_human and waits for an external event (an approval) instead of a step result. The same durability that survives a crash is what lets a run wait hours for a human without holding any resources, then resume cleanly. Build checkpoints on the engine’s interrupt mechanism, not on a thread blocked on a queue, and a checkpoint becomes just another point a durable run can sit at safely.

A concrete walkthrough makes the three properties click. An onboarding run is provisioning a new hire: it has created the identity record and the email account, and it’s mid-way through enrolling in payroll when the worker process is redeployed and the run’s process dies. Without durability, the run is gone; nobody knows the hire is half-provisioned until the person can’t log into payroll on day one. With durability, the run’s state is in Postgres at current_step = enroll_payroll, and a surviving worker picks it up and resumes there. It does not re-create the email account, because that step’s result is persisted and the orchestrator skips completed steps. The payroll enrollment retries, and because the enrollment call carries an idempotency key, even if the original call had actually reached the payroll system before the crash, the retry finds the key and returns the existing enrollment instead of creating a duplicate. The hire ends up provisioned exactly once, and the crash was a non-event. That is the entire promise of durable execution, and it is the difference between a workflow you trust on Monday morning and one you babysit.

The throughline: state lives in the store, not in the process, and every side effect is idempotent. Hold those two and the workflow tolerates crashes, redeploys, retries, and human-length pauses as routine events rather than incidents. Skip them and the system works in the demo and corrupts data in week two, when the first API times out at the wrong moment.


Human-in-the-loop and the autonomy spectrum

Autonomy is not binary. Between “a human does every step” and “the agent does everything unattended” is a spectrum, and the engineering question is where on it each action sits and how you move it rightward over time as evidence accumulates. The executive framing of this is in the autonomy spectrum for AI agents; here is how to implement it.

The spectrum, from least to most autonomous, as four operating modes you can set per action:

ModeWhat the human doesUse when
Human does itThe whole step; the agent suggestsThe step is new, high-stakes, or unproven
Approve every actionReviews and approves before each effectIrreversible actions; early in rollout
Approve exceptionsReviews only what the agent flagsThe agent is reliable on the clean cases
Monitor after the factSpot-checks a sample post-hocReversible, low-value, high-confidence actions

The decision of where an action starts is a function of two variables: how reversible the action is and how confident you are in the agent at that step. An irreversible action (a payment, a permission grant, an external message to a customer) starts at “approve every action” no matter how good the agent looks, because the cost of a wrong one is too high to discover later. A reversible action (routing a ticket, drafting an internal note) can start at “approve exceptions” or even “monitor after the fact,” because a wrong one is cheap to catch and fix. The two variables, reversibility and proven confidence, place every action on the spectrum.

The two mechanisms that make human-in-the-loop work in practice are confidence-gated escalation and the durable checkpoint. Confidence-gated escalation means the agentic step emits a confidence signal, and the workflow routes anything below a threshold to a human automatically. This is how you get the volume benefit (the confident majority flows through) without the risk (the uncertain minority gets a human). The confidence signal can be the model’s own calibrated uncertainty, a downstream consistency check (does the extracted total reconcile?), or a business rule (is this above a dollar threshold that always needs sign-off?). In practice you use all three: the agent flags what it’s unsure of, the rules flag what’s structurally risky, and either one routes to the checkpoint.

Escalation paths need to be designed, not improvised. When a step escalates, three things must be true: the right human gets it (routing by the kind of exception, not a single overflowing inbox), they get the evidence to decide fast (the extracted fields, the source, the reason), and their decision flows back into the run and into your training data. The last part is how the autonomy ceiling rises over time. You start an action at “approve every action,” watch the human approve the agent’s proposal unchanged 98 times out of 100, and that record is your evidence to move the action to “approve exceptions.” Autonomy is earned per action, by measured intervention rate, not granted by optimism. And it moves in both directions: if an action’s intervention rate climbs (the agent is getting it wrong more often, maybe because the input distribution shifted), you move it back leftward and investigate. The autonomy ceiling is a dial you turn on evidence, continuously, not a switch you flip at launch.

One discipline keeps this honest: never let the agent take an action a human couldn’t undo or audit just because the agent was confident. Confidence is not authority. The checkpoint placement is a business decision about blast radius, and a confident wrong action with a high blast radius is exactly the event the whole architecture exists to prevent. Confidence routes the easy cases away from humans; it does not earn the agent the right to take irreversible actions unattended. Those two rules, confidence-gated escalation for volume and reversibility-gated checkpoints for safety, are the entire human-in-the-loop design.


Security, access control & the data perimeter

An agentic workflow takes actions in your systems of record, which makes security a design constraint from line one, not a phase you bolt on before launch. The mental model is simple and unforgiving: the blast radius of an agentic step is exactly the set of tools you handed it, and nothing more. Design the tools and you’ve designed the blast radius.

Tool permissions are the access-control surface. A step can only do what its tools let it do, so the tools are where least privilege lives. Give the extraction step a tool that reads a document and looks up a vendor, and that is the entire set of things it can do; it cannot post to the ledger because it doesn’t have that tool. Give the posting step a tool that posts a specific invoice for a specific amount, and it cannot “run arbitrary ERP commands” because that tool doesn’t exist. The anti-pattern is a single broad tool (“execute SQL,” “call any API”) handed to an agent, which makes the blast radius the entire system and makes the agent’s mistakes unbounded. Narrow, typed, single-purpose tools are not just cleaner; they are the security model.

The blast radius of an autonomous action is the thing you’re rationing. Before you let any step take an action unattended, ask what the worst version of that action does. A wrongly-routed ticket is a minor annoyance, fixed in seconds. A wrongly-posted payment is money out the door and a reconciliation problem. A wrongly-granted permission is a security incident. The size of that worst case is what decides whether the action gets a human checkpoint, and it’s why reversibility and value, not the agent’s confidence, set the autonomy ceiling. A confident agent with a high-blast-radius tool and no checkpoint is the single most dangerous configuration in this whole field, and it’s the one a team ships when they’re impressed by a demo.

Approvals are enforced before the effect, never after. A human checkpoint has to sit before the irreversible action, so that the action doesn’t happen until a human has signed off. An “approval” that fires after the payment is sent is a notification, not a control. This is the same principle as enforcing access control before retrieval rather than trusting a model not to leak: the only safe place to gate an action is upstream of it, in code the model can’t talk its way around.

Decide your data perimeter explicitly. Three things tend to leave your environment in an agentic workflow unless you stop them: document and record contents (to the model and to extraction tools), the prompts and tool arguments themselves, and any data the agent writes outward. For each, decide what’s allowed to go where and write it down. The options run from fully managed APIs (fastest to ship, data leaves your VPC) to self-hosted models and tools (slowest to stand up, nothing leaves). Most enterprises land in between: a managed model under a zero-retention agreement, with the systems of record and the state store inside their own perimeter. Finigami DocumentAI’s API-first model fits that middle path for the extraction step; so does running your orchestrator and Postgres in your own environment. For the executive version of this conversation, see autonomous AI agents for business.

Audit everything the workflow did. Because every tool call is logged with the run_id and every run persists its full trace, you can answer “why did the system do that” precisely, which you will be asked in any regulated setting and after any incident. Build retention and redaction into the trace and audit store from the start; adding it after an incident is the expensive path. And decide whether sensitive fields (account numbers, personal data) belong in prompts and traces at all. Often the right move is to redact before the model sees them and before they hit the trace store, because you cannot leak what you never sent.


Complexity management

Agentic workflows rot in a predictable way: every problem looks like it needs another agent, another tool, or another branch, and a year later you have a system nobody can reason about. Resist it.

  • Default down, promote on evidence. Every step starts deterministic. A step becomes agentic only when a real, named class of inputs proves the rules can’t handle it, shown in your eval. The reflex to “let the agent handle it” is how a clean three-rule workflow becomes a dollar-per-run model loop that’s worse at the job.
  • One agentic step, not a swarm. Multi-agent architectures multiply cost, latency, and the surface where things go wrong. Reach for them only when a single agent genuinely can’t hold the task, and even then prefer a deterministic orchestrator calling narrow agents over agents negotiating with each other. A swarm is the most expensive way to make a system you can’t debug.
  • One change at a time. Tuning a workflow is empirical. If you change the prompt and add a tool together and the success rate moves, you’ve learned nothing. Change one variable, re-run the eval, record the delta.
  • Keep the graph as linear as you can. Every branch you add doubles your test surface. Branches are sometimes necessary (the exception path is one) and never free. A workflow that’s mostly a straight line with one exception branch is far easier to operate than a graph with a dozen conditional edges.
  • Version the workflow. A change to the steps, the prompts, or the tools produces a different workflow that will behave differently. Treat the workflow definition as a versioned artifact tied to the config that built it, so you can roll back a bad change and so a run records which version produced it.

A concrete version of how this goes wrong: a team sees the workflow stumble on a hard invoice, concludes it needs a multi-agent “validation crew,” and spends two months building a planner, a checker, and a reconciler agent that pass messages around. The success rate barely moves, because the real failure was a tolerance threshold set too tight, fixable in an afternoon. Now the agent crew is a permanent cost-and-maintenance burden, bought to solve a problem it never had. An eval run on day one would have named the actual cause. The cost of a component is not building it; it’s owning it forever. For the executive framing of that trade-off, see autonomous AI agents for business.


Evaluation & quality

This is the section that decides whether you have a product or a science fair project. Build the eval harness before you tune anything. Without it, you are adjusting prompts and thresholds and asking the room whether the runs “feel better.” With it, every change has a number attached.

An agentic workflow has four failure surfaces, and you measure them separately because the fixes live in different parts of the system:

  • Task success rate: did the run reach the right outcome? For a clean invoice, did it post the right amount to the right account? This is the top-line number, but on its own it hides why a run failed, which is why you also measure the other three.
  • End-to-end correctness: of the runs that completed, were the outcomes actually correct, judged against ground truth? A run can “succeed” (complete without error) and still be wrong (posted the wrong total). Correctness is the honest metric; success rate is the optimistic one.
  • Human-intervention rate: what fraction of runs needed a human, and was each intervention warranted? A high rate where the agent was actually right means your thresholds are too conservative (you’re dragging humans into clean cases). A low rate where the agent was wrong means the opposite and is far more dangerous. This metric is how you tune the autonomy ceiling.
  • Cost per run: tokens and tool calls per completed run. This is where “deterministic by default” shows up as a number: a run with one agentic step costs a fraction of a run that made everything a model call, for a better result.

How to actually run it:

  1. Build a golden workflow test set. 100–200 real cases with known-correct outcomes: invoices with their correct postings, tickets with their correct queues, onboarding requests with their correct access bundles. Mine them from real history, including the messy exceptions, not just clean cases. This is the single most valuable asset in the project, and it’s your regression suite.
  2. Score automatically. Run each case through the workflow and grade four things: did it reach the right outcome (correctness, often checkable against ground truth deterministically), did it succeed without error (success rate), did it escalate appropriately (intervention rate, comparing what it escalated against what should have been escalated), and what did it cost (cost per run). Use an LLM-as-judge only where the outcome isn’t deterministically checkable; much of a workflow’s correctness is checkable, which is an advantage agentic workflows have over open-ended generation.
  3. Gate releases on it. Every change to the workflow runs the eval. A change that improves success rate while quietly raising the rate of confidently-wrong unescalated runs is exactly what the harness exists to catch, and it’s the most dangerous regression there is.

The harness is small. Each row knows the input, the correct outcome, and whether it should have escalated:

def evaluate(test_set, workflow):
    rows = []
    for ex in test_set:
        run = workflow.run(ex.input)                       # full run, sandboxed effects
        rows.append({
            "succeeded":   run.status == "done",            # completed without error
            "correct":     run.outcome == ex.gold_outcome,  # right answer vs ground truth
            "escalated":   run.hit_checkpoint,              # did it route to a human?
            "should_esc":  ex.should_escalate,              # was escalation warranted?
            "cost_cents":  run.token_cost + run.tool_cost,  # cost per run
        })
    return aggregate(rows)   # → success rate, correctness, intervention accuracy, mean cost

Read the output as a diagnosis, not a grade. A run that reads success 0.98, correctness 0.94, intervention 0.12, cost 1.4¢ tells a specific story: nearly all runs complete (0.98), but correctness at 0.94 means some completed runs are wrong (chase those, they’re the dangerous ones); 12% escalate to humans; and the cost is dominated by the one agentic step. The number to fear is a correct-but-unescalated failure: a run that completed, was wrong, and didn’t ask for help. Track that cell specifically, because it’s the one that erodes trust and the one a naive success-rate metric hides.

The agentic-workflow evaluation loop A golden workflow test set and live production traces both feed an evaluation step measuring task success rate, end-to-end correctness, human-intervention rate, and cost per run. Results drive targeted changes to step scoping, prompts, tools, or thresholds, which are re-run against the test set before release, forming a closed loop. Golden workflow set 100–200 cases + outcomes Production traces real runs · interventions Evaluate success · correctness intervention · cost / run Langfuse + judge Targeted change scope · prompt · tool threshold · checkpoint one variable at a time re-run before release
The closed evaluation loop: an offline workflow test set plus live traces drive measured, one-variable-at-a-time changes. Never ship a change that hasn't cleared the gate, and watch the correct-but-unescalated cell hardest.

Building the golden set is the unglamorous work that decides everything. Mine real cases (from historical runs, including the exceptions and the disputes) rather than invented ones, which cluster around the clean happy path and miss exactly the messy inputs that break the agent. For each, record the correct outcome and whether a human should have been involved. 150 cases that cover your real distribution, exceptions included, beat 1,000 synthetic clean ones. Then treat it as a living asset: every production failure and every human correction at a checkpoint becomes a new row, so the suite grows toward your actual failure modes instead of your imagined ones.


Monitoring & observability

Offline evals tell you the workflow was good at release. Monitoring tells you it still is. A production agentic workflow degrades quietly: the input distribution drifts, an upstream system changes a format, a vendor starts sending a layout the extraction step has never seen. Instrument for it.

Trace every run end to end with a tool like Langfuse (langfuse.com): the trigger, each step and its inputs and outputs, every tool call and its result, the agentic step’s reasoning and confidence, every checkpoint and the human’s decision, latency per step, and token and tool cost. When a run does the wrong thing, you need to see which step failed in one trace, not reconstruct it from scattered logs.

Watch four families of signal:

SignalWhat it catchesExample alert
Outcome qualityrising incorrect-but-completed runs, correctness driftsampled-run correctness drops below the launch floor
Interventionthe agent silently getting worse or the thresholds driftingintervention rate spikes, or drops while errors rise
Cost & latencyrunaway model steps, slow toolscost per run above ceiling; p95 run latency over budget
Operationalcrashes, stuck runs, retry stormsruns stuck in awaiting_human or running past an SLA

Set the floors from your launch baseline, not from round numbers. Your golden-set correctness at release is the line; alert when sampled-run correctness drifts below it by more than a small margin. Watch the intervention rate in both directions: a spike means the agent is hitting inputs it can’t handle (investigate the new input class), and a drop accompanied by rising downstream errors means the agent stopped escalating things it should (the more dangerous case). For cost, set a per-run ceiling and alert before you breach it, not after the invoice. For operations, alert on runs stuck in any non-terminal state past an SLA, because a durable workflow that’s “awaiting_human” for three days is a queue nobody is working.

Two operational habits separate teams that maintain quality from teams that file incidents. First, sample and grade live runs continuously against ground truth where you have it (and an LLM-judge where you don’t); that’s your early-warning system. Second, feed every checkpoint correction and every flagged-wrong run back into the golden set, so each real failure becomes a permanent regression test and the workflow can’t break the same way twice.

When an alert fires, triage in pipeline order, not by reaching for the agent first. Pull the offending traces and ask, in sequence: did an upstream input change (a new document format, a renamed field)? Did the durability layer misbehave (a retry that double-acted, a run that resumed wrong)? Did a tool fail or return something unexpected? Or did the agentic step genuinely make a bad judgment? The trace answers each in seconds. Three of the four likely causes sit outside the model, and prompt-tuning a format change or a durability bug just buries it.

What to monitor continuously: drift

Point-in-time alerts catch a sudden break. The slower danger is drift, the gradual divergence between the workflow you launched and the one running today. Four kinds are worth a chart each.

Input drift. The distribution of what the workflow receives moves: new vendors, new ticket types, new document layouts, seasonal mix shifts. Track the rate of low-confidence agentic steps and cluster the recent escalations; a growing cluster of a new kind of input is both your next golden-set additions and, often, a signal that a step’s scope needs revisiting.

Intervention drift. Chart the human-intervention rate over time against the launch baseline. A slow rise means the agent is meeting more inputs it can’t handle (the input distribution moved). A slow fall is good only if correctness held; a fall with rising errors means escalation thresholds have drifted loose. This is the single most informative drift signal for an agentic workflow.

Cost drift. Track cost per run over time. A creep upward usually means runs are touching the agentic step more often (more exceptions, or scope creep that quietly made a step agentic) or carrying more context. Cost drift is often the first visible symptom of the “everything is becoming an agent” rot.

Outcome drift. Run ground-truth checks and the LLM-judge over a sample of live runs on a schedule, and chart correctness week over week against the launch baseline. Offline evals prove the workflow was good at release; this proves it still is.

Set rolling baselines rather than fixed thresholds for these. “Correctness fell three points below the trailing four-week mean” is a better alert than a hard floor, because it catches the gradual erosion a static threshold sits above until it becomes a crisis. The weekly review from the team section ties it together: a human reads the drift charts and the worst traces and decides whether it’s noise or the start of a regression.


Rollout & the additional work

The agentic steps are maybe 30% of the work to ship. Here’s the other 70% that turns a working prototype into something you can run unattended on real volume.

Phase the rollout. Never big-bang it.

  1. Internal alpha: your team runs the workflow against real historical inputs, with all side effects sandboxed, hammering it for two weeks. Goal: find the failure classes your golden set missed, especially the exceptions.
  2. Shadow / human approves everything: the workflow runs on live inputs but every action is proposed to a human who approves before it takes effect. You get production-quality data with a full safety net, and the approvals become training and eval data and your evidence for raising the autonomy ceiling.
  3. Limited GA, confidence-gated: the confident clean cases run straight through; everything below threshold or above a value limit routes to a human. One workflow, one input source, real volume, with an obvious feedback path and a fast kill switch.
  4. Broad GA: widen the input sources and raise the autonomy ceiling per action as the intervention-rate evidence supports it.

Each transition is a go/no-go gate, not a calendar date. Alpha to shadow when the team stops finding new failure classes. Shadow to limited GA when humans approve the agent’s proposals unchanged on the great majority of clean cases. Limited to broad when live correctness holds at golden-set levels and the intervention rate is stable for two weeks. If a gate doesn’t clear, you don’t expand; you fix and re-measure. Shipping on a date instead of a number is how a quietly-wrong workflow starts taking unattended actions at scale.

The work teams forget to budget:

  • Idempotency and durability everywhere there’s a side effect. Every action that posts, creates, grants, or sends needs an idempotency key and lives inside a durable run. This is the difference between an automation you can retry and a source of duplicate payments.
  • The exception path and the approval UX. The happy path is the easy 30%; the exception path (routing, evidence, the approval inbox, the resume) is where the operational work is, and it’s what makes the human-in-the-loop phases survivable.
  • Access control and the data perimeter. Scope every tool to least privilege, decide what leaves your environment, and write it down. For the executive version, see autonomous AI agents for business and the autonomy spectrum for AI agents.
  • The kill switch and the rollback. A one-action way to pause the workflow and a way to roll back a bad workflow version. You will need both, probably in week one.
  • Compliance and audit. In regulated settings you’ll be asked to show why the workflow took an action, which is another reason every tool call and every run is traced and retained.

Team & skills required

You do not need a research team. You need a small group that can build a durable workflow, scope the agentic steps tightly, measure the system honestly, and operate it.

RoleWhat they ownCommitment
Workflow / AI engineerStep scoping, the agentic steps, evalsCore, full-time
Backend engineerThe orchestrator, durability, idempotency, toolsCore
Integration engineerConnectors to the systems of record (ERP, HRIS, ticketing)Core through launch, part-time after
Domain expert / SMEThe golden set, judging “is this outcome right,” checkpoint rulesPart-time, non-negotiable
Process ownerThe workflow being automated; sign-off on the autonomy ceilingPart-time, decisive
DevOps / platformDeployment, monitoring, cost controls, the kill switchShared

The role people skip is the SME, and it’s the one that decides success. Engineers can build a durable workflow; only a domain expert can tell you whether posting this invoice to that account was correct, and where a human checkpoint genuinely belongs. The role people underweight is the process owner: they own the real-world process being automated, and they’re the one who signs off on how much autonomy the agent gets, which is a business decision about blast radius, not an engineering one. Budget both explicitly. For the broader org question, see the autonomy spectrum for AI agents.

How the team operates matters as much as who’s on it. Run a weekly review: the engineer brings the week’s metric deltas (success, correctness, intervention rate, cost per run), the SME spot-checks a sample of runs the system completed confidently and a sample it escalated, and the group decides the next single change to try and whether any action has earned a move up the autonomy spectrum. That ritual, small, regular, and evidence-led, is what raises the autonomy ceiling safely instead of by optimism. The failure mode is the opposite: a launch, then silence, then a quarter later someone notices the workflow has been quietly mis-posting and nobody can say since when.


A 30/60/90-day delivery plan

A realistic path from zero to a defensible pilot.

Days 1–30, map and make it durable. Pick one workflow and write Step 0’s table: every step labeled deterministic or agentic, the human checkpoints marked, the exception path drawn. Stand up the durable orchestrator (LangGraph or a durable engine), Postgres state, idempotent tools, and the one or two agentic steps with Finigami DocumentAI where a document is involved. Build the first 100-case golden set with the SME, exceptions included. The goal by day 30 is a workflow that runs end to end on historical inputs with sandboxed side effects and a success rate you’d bet on.

Days 31–60, measure it and run it shadowed. Wire up the eval harness (success, correctness, intervention rate, cost per run) and full per-run tracing. Run the shadow phase: the workflow proposes every action on live inputs, a human approves before anything takes effect, and the approvals become eval data. The goal by day 60 is correctness above your floor on the golden set, an intervention rate you understand, and a growing backlog of real exceptions converted into new test cases.

Days 61–90, ship it small and confidence-gated. Limited GA on one input source: confident clean cases run straight through, everything else routes to a human, with the feedback path and the kill switch live. Watch the live metrics, fold every correction back into the set, and tune one variable at a time. The goal by day 90 is a pre-committed success metric, hit, with the intervention-rate evidence in hand to ask for the broad rollout and the next notch up the autonomy ceiling.

The shape is deliberate: you make the workflow durable before you make it autonomous, you measure before you remove humans, and you raise autonomy per action on evidence, not on a date. Each phase ends at a gate, and the gate is a number, not a vibe.


The business case

Most agentic-workflow projects that stall don’t stall on technology. They stall because no one wrote the one page that gets them funded. Here’s that page.

Frame it as deflected cost, not “we built an agent.” The framing that gets approved is straight-through processing rate against a manual baseline: “Today, N people spend M minutes each manually processing these. A workflow that handles the clean cases unattended and routes only the exceptions to a human deflects X hours/month, ≈ $Y/year, while a human still sees everything risky.” Make the number conservative and defensible, and lead with the fact that humans still see the exceptions, because that’s what makes the bet safe.

Worked example. Accounts payable processes 30,000 invoices/month. Today each takes a clerk ~6 minutes of data entry and matching; ~80% are clean and could run straight through, ~20% are genuine exceptions a human should see anyway. Automate the clean 80%: that’s 24,000 invoices × 6 minutes = 2,400 hours/month no longer touched by a person. At a $30/hour loaded cost, ≈ $72,000/month, roughly $864K/year of capacity returned. Now halve it for ramp, imperfect straight-through rate, and the exceptions you’ll still review: still ~$430K/year against a run cost in the low thousands. The payback is weeks. Bring that number, conservatively derived, not “agents will make AP better.”

The per-run cost, modeled. A run’s cost is dominated by its agentic steps: each is model tokens (input context plus output) plus any extraction or tool cost, while every deterministic step is effectively free. For an invoice workflow with one agentic extraction step over a few thousand tokens, that’s low single-digit cents per run at frontier-model rates, less at Sonnet rates, and zero marginal model cost on the deterministic majority of steps. The lever is the number of agentic steps: every step you made agentic that a rule could handle is model tokens billed on every run, forever, for a worse and non-reproducible result. This is why “deterministic by default” is a cost control before it’s a design principle, and why the reranker-of-this-architecture is the Step 0 labeling exercise.

Put a cost ceiling, an autonomy ceiling, and a quality floor on it. Decision-makers approve bounded bets, not open-ended ones. State the per-run cost target (you can model it: agentic steps × tokens), the monthly ceiling, the autonomy ceiling you’ll start at (e.g., “humans approve every posting in phase one; we raise it only on measured intervention data”), and the quality bar you’ll hold (e.g., ”≥ 95% end-to-end correctness on the test set, with every irreversible action gated, or we don’t ship”). The quality floor protects the brand; the cost ceiling protects the budget; the autonomy ceiling protects you from the confident-wrong action.

Pre-empt the questions you’ll be asked:

QuestionYour answer
”What happens when the agent does the wrong thing?”Irreversible actions are gated by a human checkpoint; confidence-gated escalation routes the uncertain cases; failures become regression tests
”Will it take actions we can’t undo?”Not unattended. Reversibility and value set the autonomy ceiling, not the agent’s confidence; we raise autonomy only on measured intervention data
”What does it cost to run?”Per-run model: agentic-step tokens + tool cost, with a monthly ceiling we alert before breaching; deterministic steps are free
”How is this different from the RPA bots that keep breaking?”Durable execution, idempotent side effects, and an eval-gated rollout; an agentic step for varied inputs instead of a brittle script
”Can you prove what it did?”Every tool call and every run is traced and retained; we can replay any decision

Tune the framing to who’s in the room. A CFO wants the deflected-cost number and the cost ceiling, conservatively derived. A CISO wants the tool-scoping and data-perimeter story and the answer to “what’s the blast radius of a wrong action”; answer it before you’re asked. A process owner wants to know the agent won’t take an irreversible action their team can’t see, which is exactly what the checkpoints and the autonomy ceiling are for. An engineering leader wants to know it won’t become an unmaintainable swarm, which is what “deterministic by default” and one-variable-at-a-time tuning are for. Same project, four different opening sentences.

Propose the smallest credible pilot. One workflow, one input source, an eight-week window, a single success metric agreed in advance, and humans approving everything irreversible until the data earns more autonomy. A scoped pilot that hits a pre-committed number, safely, is how you get the budget for the broad rollout. For the executive’s view of the autonomy decision behind all of this, see autonomous AI agents for business.


Pitfalls & anti-patterns

The failures that recur, in rough order of how much damage they do:

  • Making steps agentic that should be rules. The number-one silent killer. A model deciding what a for loop should decide is slower, costlier, non-reproducible, and harder to audit. Default every step to deterministic; promote on evidence.
  • No idempotency on side effects. A crash-and-retry pays the invoice twice. Every world-touching action needs an idempotency key recorded before it acts. Skip this and you cannot safely run the workflow unattended.
  • A plain script instead of durable execution. A try/except around the whole workflow cannot survive a process crash, and business workflows crash. Use a durable engine so a crash is a resume, not a restart or a corruption.
  • No eval set. “It feels better” is not a metric. If you skip one thing in this guide, don’t let it be the golden workflow set.
  • Over-broad tools. Handing an agent an “execute SQL” or “call any API” tool makes the blast radius the whole system. Scope every tool to the narrowest action the step needs.
  • Confidence mistaken for authority. Letting a confident agent take an irreversible action unattended because it “seemed sure.” Reversibility and value set the autonomy ceiling, never the agent’s confidence.
  • Approvals after the effect. An approval that fires after the payment is sent is a notification, not a control. Gate irreversible actions before they happen.
  • The agent swarm reflex. Reaching for multi-agent architectures before a single agent has been measured and found wanting. A swarm is the most expensive way to build a system you can’t debug.
  • No exception path. A workflow that improvises what to do when it can’t complete a step will improvise at the worst possible moment. Design the “stop and route to a human with the reason” branch as a first-class path.
  • Shipping on a date, not a gate. Expanding the audience because the calendar said to, not because the metrics held. Each rollout phase is a go/no-go gate; a number, not a vibe.
  • Evaluating only on clean, happy-path cases. A test set of invoices that all match their POs hides exactly the exceptions the agent will fumble. Mine the set from real history, exceptions included, or it will tell you you’re winning while production drifts.

FAQ

Is this just RPA with an LLM bolted on? No. RPA scripts fixed clicks over a UI and breaks when the layout changes. An agentic workflow runs deterministic steps for the parts that are rules and uses a model only for the steps whose input is too varied to encode as a rule, on a durable execution layer that survives crashes and retries. The model handles variation RPA can’t, and the durability handles failures a script can’t.

How do I decide which steps should be agentic? Ask one question per step: can I write the rule? If the input is enumerable and the logic is writable, the step is deterministic, full stop. A step earns “agentic” only when its input space is effectively unbounded (an arbitrary document, free-text intent, an ambiguous match) and completing it needs reading-then-deciding. When in doubt, start deterministic and let a real, named failure in your eval promote the step. Default down, promote on evidence.

Single agent or multi-agent? Default to a single agent doing the agentic steps with deterministic glue between them. Multi-agent multiplies cost, latency, and the surface where things go wrong, and most workflows don’t need it. Reach for multiple agents only when the task has genuinely distinct skill domains or exceeds one agent’s context, and even then prefer a deterministic orchestrator calling narrow specialist agents over a swarm negotiating with each other, because the former is debuggable and the latter isn’t.

What orchestration engine should I use? LangGraph is a strong default for agent-shaped workflows where the graph of steps and the model calls live together. For long-running workflows that span many services and where durability is the dominant concern, use a dedicated durable-execution engine like Temporal or Inngest. The non-negotiable property, whatever you pick, is that a run is a durable, resumable object, not an ephemeral function call. A plain script with a try/except is not an option for production.

How do I stop it from paying an invoice twice? Idempotency. Every step that takes a side effect (post a payment, create an account) carries an idempotency key, recorded before the action and checked before each attempt. If a crash happens after the payment lands but before the result is recorded, the retry finds the key and becomes a no-op. Without idempotency you cannot safely retry, which means you cannot run the workflow unattended.

Where do human checkpoints go? Before any action that’s expensive or impossible to reverse, and before any decision a regulator or auditor expects a human to have seen. Reversibility and value set the placement, not the agent’s confidence. Reversible, low-value actions (routing a ticket) can run unattended and be sampled after the fact; irreversible, high-value actions (posting a payment, granting access) get a checkpoint that gates them until a human approves.

How much autonomy should I give the agent at launch? Less than you think, and you raise it on evidence. Start irreversible actions at “human approves every one,” watch the human approve the agent’s proposal unchanged, and use that measured intervention rate to move the action up the autonomy spectrum. Autonomy is earned per action by data, not granted by optimism, and it moves both ways: if intervention rate climbs, you move the action back and investigate.

What does a run cost? A run’s cost is dominated by its agentic steps; deterministic steps are effectively free. A workflow with one agentic extraction step costs low single-digit cents per run at frontier-model rates, less at Sonnet rates. The lever is how many steps you made agentic: every step that’s a model call instead of a rule is tokens billed on every run, forever. This is why “deterministic by default” is a cost control.

Can I use an open-source model for the agentic steps? Yes, and many do for data residency or cost reasons. The caveat specific to workflows: test tool-use reliability explicitly, because the ability to call the right tool with valid arguments and stop when done varies more between models than raw answer quality, and a narrow, well-scoped agentic step is forgiving on quality but unforgiving on tool discipline.

How long to a production pilot? With a focused team and one well-chosen workflow, a scoped pilot is a matter of weeks, not quarters. The variable that moves the timeline most is integration: connecting to the systems of record (ERP, HRIS, ticketing) and getting durability and idempotency right takes longer than the agentic step itself, which is usually the easy part once the step is scoped narrowly.

How do I know it’s still working a month after launch? Monitoring and drift charts. Trace every run, sample live runs against ground truth, and chart correctness, intervention rate, and cost per run against your launch baseline. The signal to fear is intervention rate falling while errors rise, which means the agent stopped escalating things it should. Feed every checkpoint correction back into the golden set so the workflow can’t break the same way twice.

What’s the difference between this and a fully autonomous agent? Scope and rationing. A fully autonomous agent is handed a goal and left to do everything, which in production is expensive, non-reproducible, and unauditable. An agentic workflow rations autonomy deliberately: most steps are deterministic, one or two are agentic, and the irreversible actions are gated by human checkpoints. The autonomy is a dial you turn up on evidence, not a switch you flip at launch.


Reference implementation checklist

Ship what’s on this list and you’ll have a system you can defend to a skeptic:

  • Every step labeled deterministic or agentic, with the agentic steps a small minority you can name
  • A durable-execution engine so a crash resumes from the last completed step, not a restart
  • Idempotency keys on every side-effecting step, recorded before the action
  • State persisted to Postgres at every step boundary, keyed by a stable run ID
  • Tools scoped to least privilege, typed, and audited on every call
  • Human checkpoints before every irreversible or high-value action, gating it until approval
  • Confidence-gated escalation routing uncertain cases to a human automatically
  • An explicit exception path that stops and routes to a human with the reason and evidence attached
  • A golden workflow test set of 100–200 real cases, exceptions included, version-controlled
  • Automated eval (success rate, correctness, intervention rate, cost per run) gating every release
  • Full per-run tracing in production; live sampling graded against ground truth
  • Checkpoint corrections and flagged-wrong runs feeding back into the golden set
  • A versioned workflow definition, a kill switch, and a rollback path
  • A per-run cost ceiling, an autonomy ceiling, and a quality floor, all alerted on

If you’re missing the first eight, you have a demo. With all fourteen, you have a system you can run unattended.

Executive briefings (for the people who approve this):

Sibling build guides:

Building this for real?

30 minutes, no slides. We'll work the specific implementation call your team is facing.