LLM security in production

LLM security protects the full data and action flow around a model. Inspect untrusted input and retrieval, validate output, keep tool authority narrow, minimise sensitive telemetry and prepare for failures. No single detector covers identities, providers, data sources, plugins and tools.

Main LLM security risks

The OWASP GenAI LLM Top 10 2026 provides a current risk catalogue. The NIST Generative AI Profile covers lifecycle governance, testing, monitoring and incident response.

• Prompt injection: user or retrieved content tries to override trusted instructions.
• Sensitive information exposure: prompts or responses reveal credentials, personal information or private context.
• Unsafe output handling: generated code, HTML or commands reach another component without validation.
• Excessive agency: an agent can read, write, send or delete beyond the user's authority.
• Retrieval and supply-chain compromise: poisoned content, models or dependencies influence the application.
• Unbounded consumption: crafted requests exhaust tokens, tools, time or spend.
• Confabulation: plausible false output drives a consequential decision.

Production control stack

1. Inventory every AI path: owner, purpose, model, provider, identity, data, tools and environment. Include Shadow AI.
2. Classify and minimise data: define what may enter each provider and remove unnecessary fields.
3. Separate instruction from data: preserve provenance for uploads, retrieval and tool results; keep policy outside user-editable context.
4. Inspect both directions: detect injection and sensitive input before forwarding; validate and redact output before delivery or execution.
5. Authorise tools outside the model: check identity, operation, resource, arguments and approval in deterministic code.
6. Bound use: limit payloads, tokens, time, retries, calls, concurrency and spend.
7. Retain content-minimised evidence: record findings, decisions and delivery state; make raw content a reviewed exception.
8. Test after material change: re-evaluate models, prompts, retrieval, tools and policy.

Normal authentication, encryption, secrets management, patching, segmentation and supply-chain controls still apply. The AI security boundary guide, application protection guide, prompt injection guide and 15 prompt injection examples with layered defences cover these paths in more detail.

Secure retrieval, output and tools

Retrieved internal content remains untrusted data. Keep its source and access scope attached, prevent it from redefining policy and verify citations against underlying sources for high-risk decisions.

Validate output for its destination: strict schemas for structured data, context-specific encoding, secret and personal-information checks, and human review where residual risk is unacceptable. Treat generated URLs, code and attachments as untrusted.

The ASD Australian Cyber Security Centre's AI data security guidance recommends minimisation, access controls, encryption, provenance, integrity checks, continuous assessment and secure deletion.

Give agents narrow typed operations. Scope reads to existing access; validate exact write targets; preview external recipients; bind approvals to privileged actions, arguments, identity and expiry; isolate code execution with filesystem, network, time and resource limits. See the AI agent security guide.

Log decisions, then test them

Routine events can include identity and application context, detector and policy versions, reason codes, tool and risk class, decision, payload size, content-part hash, transformation state and delivery outcome. Conversation logs create a concentrated sensitive-data store. Apply explicit access, encryption, region, retention and deletion rules to any raw-content exception.

Test direct and indirect injection, sensitive-data handling, cross-tenant retrieval, malformed output, tool misuse, exhaustion and control outages. Verify high-risk denials and approvals with deterministic oracles, then conduct impact-appropriate independent review or red teaming. Monitor false positives, missed detections, denials, latency, cost and drift.

The NIST Generative AI Profile calls for pre-deployment testing, regular adversarial evaluation, post-deployment monitoring and incident response.

Prepare incident response

Runbooks should cover injection, data exposure, compromised credentials, poisoned retrieval, abusive consumption and unauthorised tools. Teams need authority to disable models or tools, revoke credentials, quarantine sources and preserve content-minimised evidence.

Detect and triage, contain, assess exposure without copying sensitive content into unrestricted systems, recover and add a controlled regression case.

The joint Guidelines for secure AI system development, co-sealed by ASD's ACSC, covers secure design, development, deployment, operation and maintenance.

Production checklist

• Every path has an owner, identity, provider, data classification and tool inventory.
• Input, retrieval, output and tool results retain provenance.
• Sensitive input is minimised before provider delivery.
• Output is validated before delivery or execution.
• Tool authority is deterministic, least-privileged and model-independent.
• Resource limits and exact-action approvals are enforced.
• Routine telemetry excludes raw conversations.
• Degraded and outage behaviour is explicit per path.
• Tests run before launch and after material changes.
• Claims distinguish protected, observed and unsupported paths.

Promptective's product overview explains this scope for supported AI traffic.

Authoritative LLM security sources

• OWASP GenAI LLM Top 10 2026, published 3 August 2026.
• NIST AI 600-1: Generative AI Profile, July 2024.
• ASD's ACSC: AI data security, published 23 May 2025.
• Guidelines for secure AI system development, version 1.0 published 27 November 2023 and co-sealed by ASD's ACSC.

This guide was reviewed on 21 August 2026. Recheck source versions and legal obligations before relying on a checklist.

Promptective applies policy and records content-minimised evidence on supported AI paths. Explore the protection model across browser, desktop and agent workflows.