AI in Healthcare: The Liability Question Most Vendors Won't Answer

Q: Should we avoid clinical AI until the liability picture clears up?

Not avoid, sequence. Capture the administrative wins where exposure is manageable, build the contracts, controls, and audit trails on those, then move toward clinical use deliberately. Leading with clinical AI means absorbing the duty-of-care risk before you've built the muscle to defend it.

TL;DR

When healthcare AI is wrong, the exposure rarely lands where the buyer assumes:

Failure scenario	Who is exposed	Mitigation
Clinical decision-support tool surfaces a wrong recommendation	Your clinician and institution (duty of care doesn’t transfer)	Keep the clinician’s independent judgment documented; don’t let the tool become the de facto decision
Administrative tool (coding, prior auth) makes a costly error	You — usually as ordinary business/contract loss, not the vendor	Negotiate indemnity and accuracy SLAs; reconcile outputs
Vendor model causes patient harm	You first; vendor only if you can prove it under a narrow contract	Read the indemnity and limitation-of-liability clauses before signing
”Human in the loop” rubber-stamps a bad output	The human — i.e., your staff	Design review that’s real, not theatre; log the reviewer’s reasoning

Vendors sell capability and go quiet on liability. The contract, not the demo, decides who pays — and by default that is you. This is advisory content, not legal advice; involve counsel before you sign anything.

Healthcare AI vendors sell you capability and stay silent on liability — and unless your contract says otherwise, you are the one holding the duty of care when the model is wrong.

Ask a healthcare AI vendor “what happens when your model is wrong and a patient is harmed?” and watch the conversation change. You will hear about validation studies, accuracy benchmarks, and “clinical oversight.” What you will rarely hear is a clean answer to the actual question: who is financially and legally on the hook. That silence is not an accident. It is the business model.

This piece is the liability counterpart to AI in Healthcare: Administrative Wins First. That article argued you should sequence administrative use cases ahead of clinical ones because the ROI is faster and the regulatory frame is workable. The liability lens reaches the same conclusion from a different direction: the administrative wedge is also where your exposure is most manageable. Lead clinical and you are taking on the duty-of-care risk before you have the contracts, controls, and documentation to survive it.

The liability gradient runs from administrative to clinical

Not all healthcare AI carries the same exposure. Picture a gradient. At the low end sit purely administrative tools — scheduling, claims scrubbing, coding assistance, prior-auth drafting. When these are wrong, the harm is usually money and rework: a denied claim, a miscoded encounter, a delayed appointment. That is ordinary business loss. It is unpleasant, it is recoverable through reconciliation and contract terms, and it rarely touches a patient’s body.

At the high end sits anything that informs or makes a clinical decision — diagnostic support, triage, dosing suggestions, imaging flags. When these are wrong, the harm can be a misdiagnosis or a missed condition. That is patient harm, and patient harm is a different legal universe: malpractice, negligence, and the duty of care a licensed provider owes a patient.

The trap is the middle of the gradient, where a tool is marketed as “administrative” but quietly shapes clinical judgment. A prior-auth tool that nudges what gets ordered. A “documentation assistant” that drafts the assessment. A patient-messaging bot that answers symptom questions. These look administrative on the org chart and behave clinically in practice. Map every tool to where it actually sits on this gradient — not where procurement filed it.

The duty of care does not transfer to the vendor

Here is the load-bearing point most buyers miss. In a clinical setting, the duty of care is owed by the licensed provider and the institution to the patient. An AI vendor is not a licensed clinician and does not assume that duty by selling you software. So when an AI-informed clinical decision goes wrong, the first parties exposed are your clinician and your organization — not the company whose model produced the suggestion.

This is the reverse of what the sales narrative implies. The pitch frames the AI as a safety net that reduces your risk. Legally, in most arrangements, the AI is a tool you chose to use, and you remain responsible for how it is used. The model can be a contributing cause; it rarely becomes the sole defendant. You are the one in the room with the patient.

Indemnity gaps are where the contract quietly shifts everything to you

If liability is going to move from you to the vendor at all, it moves through the contract — specifically the indemnification, limitation-of-liability, and warranty clauses. This is where vendor silence becomes vendor strategy.

Read those clauses and you typically find three things. First, indemnification that covers intellectual-property claims and data breaches but says nothing about clinical error or model output. Second, a limitation-of-liability cap set at something like twelve months of fees — trivially small against a malpractice claim. Third, warranties that disclaim fitness for any particular purpose and explicitly state the software is not a substitute for professional judgment. Stack those together and the contract has quietly assigned the clinical risk to you while the vendor keeps the upside.

None of this is hidden, exactly. It is in the document you are about to sign. The gap exists because most buyers evaluate the demo and let counsel skim the paper, when the paper is the product’s actual risk profile.

”Human in the loop” often just relocates the liability to your clinician

“There’s always a human in the loop” is the reassurance vendors reach for, and executives accept it because it sounds like a control. Sometimes it is. Often it is liability relocation dressed as safety.

If the human in the loop is your clinician, then designating a human reviewer does not reduce your exposure — it confirms it. You have now formally placed your own staff member at the decision point. If that reviewer is shown a confident AI recommendation, under time pressure, with no easy way to see the model’s uncertainty, the “review” becomes a rubber stamp. Automation bias is well documented: people defer to a system that is usually right, especially when overriding it is slow and second-guessing it is uncomfortable. When the rare wrong answer arrives, the human approved it, and the human is yours.

A human in the loop only reduces risk if the review is real: the reviewer has the information and the time to disagree, dissent is easy and logged, and the workflow does not punish the override. Otherwise you have built a liability sink and labeled it a safeguard. (See Human-in-the-Loop Is an Org-Design Choice for how to design review that actually functions.)

Documentation and the audit trail are your primary defense

Because the duty of care stays with you, your strongest protection is not the vendor’s accuracy claim — it is your ability to show, after the fact, that the decision was reasonable. That is a documentation problem.

The defensible record captures what the AI recommended, what the clinician decided, and why — particularly when the human agreed with or overrode the system. It captures which model version produced the output, what inputs it saw, and what the workflow showed the reviewer. In a dispute, “a reasonable clinician, given this information, made this call, and here is the contemporaneous reasoning” is a far stronger position than “the AI said so.” Build the audit trail before you deploy, not after the first incident, because you cannot reconstruct reasoning that was never recorded. Audit Trails for AI Decisions covers what to log and how to keep it admissible.

The counter-argument

A reasonable objection: “We hold liability for every tool, drug, and device we already use — the lab analyzer, the infusion pump, the EHR. AI is just one more. Why treat it as special?”

It is partly right, and that is the useful part. You already have a framework for tool-related risk, and AI should plug into it rather than spawn a parallel governance circus. But two things make AI different in degree. First, traditional devices fail in legible, bounded ways; a generative model can be confidently wrong in novel ways across a vast input space, which makes validation and monitoring harder. Second, device makers operate inside a mature product-liability regime with established standards; many AI vendors are startups with thin balance sheets, narrow contracts, and no clear product-liability footing. So the objection holds on principle and breaks on practice: treat AI inside your existing risk framework, but do not assume the vendor accountability you take for granted with a regulated device exists here. Verify it in the contract.

What to do this quarter

Map every healthcare AI tool to the liability gradient. Sort each by whether an error produces business loss or patient harm, and flag the “administrative” tools that actually shape clinical decisions.
Have counsel read the indemnity, limitation-of-liability, and warranty clauses of your top three AI contracts and write a one-page exposure summary for each.
Pressure-test “human in the loop.” For each clinical-adjacent tool, confirm the reviewer has the time, information, and authority to disagree — and that overrides are logged.
Stand up the audit trail before the next clinical-adjacent deployment. Capture recommendation, decision, reasoning, and model version.
Sequence administrative wins ahead of clinical ones as the risk-managed path while the controls mature.

FAQ

If the AI vendor is at fault, can’t we just sue them? You can try, but the contract usually limits it. Standard healthcare AI agreements cap liability at a fraction of patient-harm damages and disclaim fitness for clinical use. You may recover something; you will rarely recover enough, and you remain the first party the patient’s claim targets.

Is this malpractice liability or product liability? Often both, against different parties. A patient-harm claim typically runs as malpractice against the clinician and institution who held the duty of care, and may separately pursue the vendor on product-liability or warranty theories — which are harder to win and constrained by the contract. Plan for the malpractice exposure to land on you first.

Does keeping a human in the loop protect us? Only if the review is real. A reviewer who can’t see the model’s uncertainty, has no time to disagree, and defers by default doesn’t reduce your exposure — they formalize it. Design the workflow so dissent is easy and documented, or the “loop” is just a liability sink.

What single contract term matters most? The limitation-of-liability cap, read alongside the indemnification clause. A cap set at twelve months of fees tells you exactly how much risk the vendor will actually absorb — usually far less than one patient-harm event costs. If those clauses are silent on model error, assume the risk is yours.

Should we avoid clinical AI until the liability picture clears up? Not avoid — sequence. Capture the administrative wins where exposure is manageable, build the contracts, controls, and audit trails on those, then move toward clinical use deliberately. Leading with clinical AI means absorbing the duty-of-care risk before you’ve built the muscle to defend it.

Working with JAIN on healthcare AI risk? We help health system, payer, and provider executives map their liability exposure, fix vendor contract gaps, and build the audit trails that hold up. Book a 30-minute call.

Related reading: