The AI Threat Model Your Security Team Isn't Running Yet

TL;DR

STRIDE category	Pre-AI threat	New AI-specific threat
Spoofing	Stolen credentials	Adversarial inputs impersonating system messages
Tampering	Modified code/data	Prompt injection, training-data poisoning
Repudiation	Disputed actions	Agent actions without queryable audit
Information disclosure	Data exfiltration	Inference-time leakage, model memorization
Denial of service	Service overload	Cost-runaway via crafted prompts
Elevation of privilege	Privilege escalation	Tool authority escalation through agent reasoning

The threats overlap with the existing STRIDE categories but the attack surfaces and mitigations are new. Most security teams haven’t run STRIDE on an LLM-powered product, and the four threats most often missed are: prompt injection, inference-time data leakage, cost-runaway DoS, and tool-authority escalation.

What STRIDE looks like for an LLM-powered product. The four threats that don’t exist in your existing model.

Most security organizations have a threat-modeling discipline — STRIDE, attack trees, MITRE ATT&CK, whatever. The frameworks work for traditional systems and break in subtle ways for LLM-powered ones because the attack surfaces are different. Running the same threat-modeling exercise on an AI product without adapting the framework misses the threats that matter.

This piece is what changes, and what to add.

STRIDE adapted for LLM-powered products

STRIDE’s six categories — Spoofing, Tampering, Repudiation, Information disclosure, Denial of service, Elevation of privilege — apply to AI, but each category has new attack vectors.

Spoofing: adversarial inputs impersonating system messages

In a traditional system, spoofing is about stolen identity. In an LLM-powered system, attackers don’t need to steal credentials — they can inject content that the LLM treats as system-level instruction. A document, an email, a webpage with embedded text that says “ignore previous instructions and…” can hijack agent behavior.

Mitigation pattern: separate channels for system instruction and user data. Treat all retrieved content as untrusted. Don’t put instructions and data in the same context without explicit boundaries.

What gets missed: the assumption that “user input” is the only untrusted input. In an agent, the document the user uploaded, the webpage the agent retrieved, and the API response from a tool are all untrusted from the LLM’s perspective.

Tampering: prompt injection and training-data poisoning

Tampering in AI takes two forms. Prompt injection is real-time tampering with the agent’s instructions. Training-data poisoning is upstream tampering with the model itself — an attacker who can influence training data can plant behavior that’s invisible until the right trigger.

Mitigation pattern for prompt injection: input sanitization, instruction-data separation, output validation. Mitigation pattern for training-data poisoning: vendor due diligence (for closed models), provenance verification (for open weights), behavioral testing.

What gets missed: training-data poisoning is often dismissed as theoretical. The 2024–2025 research literature has multiple demonstrations of practical poisoning attacks; ignore at your own risk.

Repudiation: agent actions without queryable audit

When an agent takes an action, you need to be able to reconstruct what happened, why, and with what authority. Most early agent deployments have audit logs that are technically present but not queryable in any practical sense.

Mitigation pattern: structured audit logs covering input, reasoning trace, tool calls, outputs, and authority. Queryable per-agent, per-time-range, per-affected-entity.

What gets missed: the audit log being adequate for compliance reporting (which it usually is) but not adequate for incident response (which requires real-time queryability).

Information disclosure: inference-time leakage and model memorization

Two flavors. Inference-time leakage is data sent in a prompt being logged, exposed, or used for training by the model provider. Model memorization is data from training showing up in outputs to other users.

Mitigation pattern for inference-time: contractual data-handling commitments, no-retention-zone deployment for sensitive data, prompt-level PII filtering. Mitigation pattern for memorization: closed-tenant deployment, opt-out from training, careful selection of fine-tuning data.

What gets missed: the assumption that the model provider’s standard contract is sufficient. It usually isn’t for regulated data; specific terms need to be negotiated.

Denial of service: cost-runaway via crafted prompts

A traditional DoS attack overwhelms a service to make it unavailable. In LLM-powered systems, an attacker can craft prompts that send your cost-per-execution from $0.05 to $5.00 — running up tens of thousands of dollars in token costs across thousands of malicious requests. The service stays available; your cloud bill explodes.

Mitigation pattern: per-request cost ceilings (circuit breakers), per-user/per-IP rate limiting on AI endpoints, anomaly detection on cost trends.

What gets missed: rate limiting that’s based on request count, not on cost or token volume. An attacker who sends one expensive request per second can hide under typical rate limits.

Elevation of privilege: tool authority escalation through agent reasoning

The most subtle category. An agent has authority to call tools A and B; through chained reasoning, it figures out a way to use A and B together to accomplish something it shouldn’t be able to do directly.

Mitigation pattern: agent authorities defined at the tool level, not just the agent level. Specific actions that require human approval regardless of which tools the agent has access to. Goal-bounds testing.

What gets missed: the assumption that tool-level access controls are sufficient. They aren’t, when an agent can chain tools in unexpected ways.

What to do this quarter

Run STRIDE on one production AI system. Not theoretical; an actual deployment. Use the categories above as a starting point.
Document the new mitigations needed. Most organizations find 3–7 gaps from a first STRIDE run.
Add the AI-specific threats to your standard threat-modeling template. So the next system gets the right analysis from the start.
Schedule the next STRIDE run. Threat models go stale; quarterly review for production systems is the floor.

FAQ

Are LLM-powered products fundamentally less secure than traditional ones? Not fundamentally. They’re differently secure. The attack surfaces are new, but so are the mitigations. The risk is comparable to other technology shifts (mobile in 2010, cloud in 2014) — manageable with the right discipline, problematic without it.

Should we hire a dedicated AI threat modeler? For most enterprises, no. Train your existing threat modelers on the AI-specific surfaces. The skill is an extension, not a new role.

Will the OWASP Top 10 for LLMs become a formal standard? The OWASP LLM Top 10 (released 2023, updated 2024) is already widely adopted. Expect it to become reference standard for vendor evaluations and audit checklists by 2027.

How do we handle threat modeling for agents we don’t operate? For SaaS AI tools you use, the vendor owns the threat model. Your job is to evaluate it (request the SOC 2 / ISO 27001 / pen test reports) and accept or reject the residual risk. Don’t try to threat-model their system; do threat-model your integration with it.

What’s the most common gap in first-pass AI threat models? Tool authority escalation. Most teams catch the obvious threats (prompt injection, data leakage) and miss the chained-tool-authority threat because it requires reasoning about agent behavior, not just data flows.

Working with JAIN on AI threat modeling? We help security teams adapt STRIDE for LLM-powered systems and catch the gaps most first-pass models miss. Book a 30-minute call.

Related reading: