Jailbreak testing: count benign refusals

A jailbreak evaluation needs benign requests under the same conditions as harmful requests. That comparison shows whether a high refusal rate distinguishes the requested behaviour or merely reflects a response to an unusual format. Record both outcomes before using a benchmark to compare systems.

For teams evaluating prompt-injection protection, this is a useful measurement question from adjacent research. The study discussed here tests model refusals on encoded user requests. Indirect prompt injection, retrieved documents and agent tool authority require their own evaluations.

Read both sides of the refusal result

In Refusing Everything Looks Safe, version 2, Haoyu Zhang and colleagues examine four open-weight models with seven to eight billion parameters. Their September 2026 preprint uses JailbreakBench's 100 harmful requests and 100 theme-matched benign counterparts.

Matching the themes matters. A harmless request about a sensitive subject gives the model a different decision from an unrelated everyday question. The evaluation can then examine whether the model distinguishes the requested behaviour within a similar subject area.

The authors send each set as plain text, inside an encoding-related template with the content unchanged, and with the content transformed. Each reported condition contains 100 requests. These are model-judged refusal results, with the limitations of that judging method retained.

The same high score can hide different behaviour

In the paper's first table, Llama-3.1-8B-Instruct refuses 99 of 100 harmful requests under a lookalike-character transformation. It also refuses 99 of 100 benign requests under that transformation. Its corresponding plain-text results are 92 harmful refusals and ten benign refusals.

The authors call the difference between harmful and benign refusal rates the harm gap. For this example, the gap falls from 82 percentage points in plain text to zero in the transformed condition. A report containing only the 99 harmful refusals leaves out the behaviour on legitimate requests.

That arithmetic describes this condition and model. The authors take no position on whether refusing unusual encoded text is desirable for a particular application. An organisation might deliberately reject an unsupported format; its evaluation should state that policy and measure the resulting loss of supported work separately.

Separate the template from the transformation

The template-only comparison asks another useful question: how much changes when ordinary text is wrapped in the test's instructions, even before the characters change?

In a separate run reported in the paper, Llama's benign refusal rate rises from 10% in plain text to 83% with the template alone. Qwen's result differs. The surrounding protocol and the transformation therefore need separate attribution in the measurement.

For a supplier report, request a clear description of the benign population, its formatting and its scoring rule. A generic helpfulness score collected elsewhere leaves the transformed condition unmeasured. Keep the model version, template version and test population with every reported figure.

Inspect how the evaluator labels a response

The paper's refusal judge counts an echo of the input or an irrelevant response as refusal. The authors add checks for whether a model can read the transformation and whether echoed content materially changes the reported comparison.

They also report that a separate binary jailbreak judge flags many responses to benign plain-text requests. That finding concerns their configured evaluator. It supports checking a judge against cases with known meaning before trusting its aggregate score.

In a controlled review, include harmless tasks with known answers and inspect a bounded sample of the labels. Record successful completion, explicit refusal, irrelevant output and technical failure separately. Collapsing these into one category loses the information an operator needs to decide whether a control remains usable.

Keep the paper's limits attached

The cross-model comparison rests on one transformation that passes the authors' screening checks for all four models. A second transformation produces different results. The study covers small open-weight models and one family's training sequence, with no demonstrated transfer to hosted frontier models.

The authors state that nothing is held out: the same request sets informed their choices and evaluation. Judgements and cross-family checks come from models; human annotation remains absent. Promptective has not independently reproduced the findings.

This supports a request for better evidence when assessing jailbreak defences. It establishes neither a production incident nor a containment guarantee. Product selection still needs tests of the actual application and its authorised work.

A useful acceptance record keeps the harmful result beside the benign result, identifies the tested format and explains unresolved outcomes. Set the expected treatment of unusual inputs before the exercise. That gives the reviewer a defensible way to separate an intentional restriction from an unexplained loss of useful work.

Review your AI evaluation boundaries with Promptective.