How LLM jailbreaks bend the rules

Jailbreak attacks target an LLM's safety guardrails, often to make it generate content it was trained to refuse. They can create user harm, reputational damage, model abuse and compliance exposure.

Jailbreaks and prompt injection

Common jailbreak techniques

Models balance safety with helpfulness and instruction-following. Attackers probe that tension through:

1. Role-play: asking the model to act as an unrestricted character.
2. Hypothetical framing: presenting a harmful request as fiction, education or research.
3. Multi-turn escalation: beginning with benign questions and gradually increasing risk.
4. Token smuggling: disguising intent with encoding, spacing, substitutions or another language.
5. Instruction overrides: presenting user text as a higher-priority system directive.
6. Format manipulation: requesting restricted material through unusual structured output.

These patterns change quickly. A phrase can also appear in legitimate security analysis, so a match provides one input to the decision.

Build layered defences

Inspect input and context

Normalise text before inspection and detect known role-play, override, encoding and escalation patterns. Apply session-wide limits alongside per-message checks. Avoid automatically blocking broad phrases such as “academic research”, which can create predictable false positives.

Keep policy outside the model

State the assistant's purpose and constraints clearly, but do not rely on the system prompt as an enforcement boundary. Identity, data access, tool scope and action approval belong in deterministic application policy. Treat retrieved documents and tool results as untrusted context on every turn.

Validate output

Inspect generated content before users or downstream systems receive it. Use destination-specific checks, such as strict schemas for structured output, encoding for rendered content and review for consequential advice. Generated code, links and tool arguments remain untrusted until validated.

Monitor behaviour

Correlate repeated attempts, changing encodings and escalating requests across the session. Record finding categories, policy versions, outcomes and delivery facts while keeping prompt and response text outside routine telemetry. Rate limits and cost budgets can constrain automated probing.

No layer is sufficient alone. Detection can miss a novel prompt, while output validation and narrow permissions can still limit its effect.

Test the assembled application

Maintain versioned cases for role-play, hypothetical framing, multi-turn escalation, encoded requests and format manipulation. Include benign examples that resemble attacks to measure false positives.

Automated tests should verify the expected refusal or policy outcome against the exact model, prompt, retrieval sources and tools in production. Human red teams should then explore novel variations and document both successful and unsuccessful attempts. Repeat testing whenever a model, system prompt, classifier, retrieval source or tool changes.

A useful test record includes the case version, model and policy version, expected outcome, observed outcome and whether content reached the user or another system. Re-test every fix and monitor production for drift.

Key controls

• Distinguish safety bypass from attempts to gain application authority.
• Inspect prompts, retrieved context and output.
• Authorise tools independently of model text.
• Apply least privilege, schemas, approvals and rate limits.
• Test known attacks and benign lookalikes.
• Monitor sessions for escalation without retaining unrestricted transcripts.
• Define failure behaviour when a detector or policy service is unavailable.

Promptective checks named jailbreak and prompt injection patterns on supported AI paths, then records the finding, policy version, outcome and delivery facts for review.

Continue with the LLM security boundary-controls guide to map monitoring and enforcement around your AI deployment.