All resources AI Security for the C-Suite

AI Vendor Security: 12 Questions to Ask Before You Sign

Specific questions for AI vendor security review, what good and bad answers look like, and the disqualifying answers.

TL;DR

The 12 questions:

  1. What’s your data retention default?
  2. Is our data used for training?
  3. Where does our data physically reside?
  4. What’s your incident-notification timeline?
  5. What SOC 2 / ISO 27001 / sectoral certifications do you hold?
  6. What’s your indemnification posture for IP claims?
  7. What’s your behavior-change notification policy?
  8. What audit rights do we have?
  9. What’s your sub-processor list?
  10. What’s your data-deletion guarantee on contract end?
  11. What’s your model-deprecation timeline and migration support?
  12. Have you had a security incident in the last 24 months?

The disqualifying answers: vague on data retention, default-yes on training, ambiguous incident timelines, no SOC 2, refusal of audit rights, no indemnification, “no” to question 12 without supporting detail.

Specific questions, why each matters, and what answers should disqualify the vendor.

The AI vendor security review is often the difference between an AI investment that compounds and one that creates exposure. Most procurement processes ask the wrong questions or ask them in ways that produce non-answers. This piece is the 12 questions that produce real signal, what good and bad answers look like, and the red flags that should disqualify a vendor.

The 12 questions

1. What’s your data retention default?

Why it matters: data retained by the vendor is data that can leak (via breach, subpoena, employee misuse, or accidental exposure).

Good answer: “Zero retention by default for enterprise customers, with audit logs retained per your retention policy.” Bad answer: “We retain prompts for model improvement.” (Disqualifying for sensitive workloads.)

2. Is our data used for training?

Why it matters: data used for training can be reproduced in outputs to other users.

Good answer: “No, customer data is not used for training. This is contractually committed.” Bad answer: “By default we may use it; you can opt out.” (Acceptable if you opt out in writing; weak otherwise.)

3. Where does our data physically reside?

Why it matters: data residency obligations (GDPR, sectoral) depend on where data is processed and stored.

Good answer: specific regional residency commitments aligned with your jurisdiction needs. Bad answer: “We operate globally” without specifics. (Disqualifying for regulated data.)

4. What’s your incident-notification timeline?

Why it matters: regulators and your incident-response process require timely notification.

Good answer: “72 hours for confirmed incidents; sooner if customer data is implicated; with material technical detail.” Bad answer: “We’ll notify when we have all the facts.” (Sounds reasonable; in practice means delay.)

5. What SOC 2 / ISO 27001 / sectoral certifications do you hold?

Why it matters: third-party assurance reduces your due-diligence burden.

Good answer: “SOC 2 Type II annually, with the most recent report available under NDA. Plus [sectoral certs as relevant — HITRUST, FedRAMP, PCI].” Bad answer: “SOC 2 Type I in progress.” (For enterprise: usually disqualifying. Type II is the bar.)

6. What’s your indemnification posture for IP claims?

Why it matters: AI output can infringe copyright; you want vendor coverage.

Good answer: “We indemnify for IP claims arising from model output, with reasonable limits.” Bad answer: “We disclaim liability for output.” (Negotiate or pass.)

7. What’s your behavior-change notification policy?

Why it matters: model updates can break your production agents in subtle ways.

Good answer: “30-day notice for material model changes; rollback path available; legacy version maintained for 6 months.” Bad answer: “We update continuously without notice.” (Operational risk.)

8. What audit rights do we have?

Why it matters: when something goes wrong, you may need to verify the vendor’s claims.

Good answer: “Annual audit rights at enterprise tier; access to control documentation and incident reports.” Bad answer: “We don’t permit customer audits.” (Disqualifying for sensitive workloads.)

9. What’s your sub-processor list?

Why it matters: your data flows through your vendor’s vendors. The chain matters.

Good answer: published sub-processor list, updated quarterly, with notification of additions. Bad answer: “We use cloud providers.” (Vague; insist on the list.)

10. What’s your data-deletion guarantee on contract end?

Why it matters: when you leave, your data should leave with you.

Good answer: “30-day deletion across all systems including backups; certification of deletion provided.” Bad answer: “Data is deleted on a best-effort basis.” (Insufficient.)

11. What’s your model-deprecation timeline and migration support?

Why it matters: models get deprecated; you need runway and help.

Good answer: “12-month deprecation notice for production models; backward-compatible migration path; engineering support during transition.” Bad answer: “We deprecate as needed.” (Operational risk.)

12. Have you had a security incident in the last 24 months?

Why it matters: forces disclosure. The answer is rarely “no.”

Good answer: “Yes, [specific incidents]. Here’s what happened, how we responded, what we changed.” Bad answer: “No incidents.” (Almost always wrong; either lying or not detecting.)

How to use the questions

Send the 12 questions in writing during procurement. Get written answers. Score them.

Pass: 10+ good answers, no disqualifying ones. Conditional: 7–9 good answers, no disqualifying ones; negotiate the gaps. Fail: any disqualifying answer or fewer than 7 good answers.

The scoring isn’t perfect, but it’s better than the typical procurement review that asks none of these questions and signs the standard contract.

What to do this quarter

  1. Add the 12 questions to your AI vendor procurement template.
  2. Audit your existing AI vendors against the 12 questions retroactively. Identify gaps.
  3. Negotiate the gaps at renewal. Most are negotiable for enterprise customers.
  4. Build a “vendor security scorecard” reviewed annually. Track each vendor’s posture; replace those that don’t improve.

FAQ

What if a vendor refuses to answer some questions? Refusal is a signal. Vague answers are also a signal. Document both and weight them in the procurement decision.

Should we negotiate or accept the vendor’s standard MSA? Negotiate, especially for the data-handling, indemnification, and audit-rights clauses. Standard MSAs are the vendor’s preference; negotiated terms are yours.

How long does a thorough AI vendor security review take? For a major procurement: 4–8 weeks including question round-trips, document review, and red-line negotiation. For a low-risk procurement: 1–2 weeks. Plan accordingly.

Should we use a third party to evaluate AI vendors? For high-stakes procurements, yes. The specialized expertise on AI-specific risks is worth the cost. For commodity procurements, internal review is fine.

Will vendors actually answer the harder questions? Mostly yes for enterprise customers. The negotiating leverage of a six- or seven-figure contract gets specific answers that smaller customers don’t get.

Working with JAIN on AI vendor procurement? We help executive teams run the 12-question review and negotiate the gaps. Book a 30-minute call.

Related reading:

Want to talk through this for your team?

30 minutes, no slides. We'll work the specific call your company is facing.