An assistant that reads a support ticket, searches a wiki and calls tools receives text from several places at once. Some spans carry the operator's instructions; others arrive from a retrieved page or a tool result. A standard transformer gives the model no explicit, application-supplied signal about which span carries authority, so it works from learned cues in wording and structure. Models do learn role cues, and the authors do not claim otherwise. Their diagnosis is that no per-token channel states the source, and an indirect prompt injection exploits that gap.

A preprint submitted on 17 September 2026 by Yuxuan Zhang, Jeff Huang and Guofei Gu at Texas A&M University proposes one. Their Provenance-Aware Transformer tags every input token with an origin label chosen by the application and changes attention so the label shapes which tokens can influence the output. Promptective has not independently reproduced the results.

Where source authority goes missing

The authors trace the problem to attention, which scores a query against a key using their representations alone. Retrieved documents, user turns and system instructions arrive in one sequence, so the model has to infer authority from learned cues. Prompt reminders and delimiters can be imitated, detectors miss some injections, and fine-tuning learns patterns without changing how attention addresses content. The paper's argument is that a defence should carry source identity as an input the application already knows.

Four rings and three learned components

The application assigns each token span a ring before inference. Ring 0 (ORG_SYSTEM) is the operator policy, Ring 1 (ORG_SELF) the assistant's own earlier tokens, Ring 2 (ORG_USER) direct user input and other permitted content, and Ring 3 (ORG_DATA) untrusted external content. Assignment follows structural role, so the framework needs no content inspection. The paper's default policy sends every externally sourced span that is not structurally trusted to Ring 3.

Three learned components carry that label into the forward pass. An origin embedding table with one row per ring is added to each token's embedding, with a learnable scale so the signal survives normalisation. The main mechanism is a 4 × 4 bias matrix added to attention logits, indexed by the ring IDs of the query and key tokens. Only attention from assistant and permitted-user queries to untrusted keys starts with a strong negative bias.

How the model learns what a label means

The authors adapt released checkpoints in two supervised fine-tuning stages. Stage 1, origin fine-tuning, teaches the new components what the labels mean across a broad instruction-following corpus. Stage 2, alignment fine-tuning, trains the suppression behaviour on a set built by extending the SEP benchmark's witness method: a probe instruction is paired with a witness string that appears only if the model obeys it. The evaluation covers four openly released models, with LLaMA-3-8B-Instruct as the primary one.

What the authors report

On LLaMA-3-8B-Instruct, attack success on the held-out in-distribution set falls to 0.69% from 32.50% with no defence. A same-data fine-tune without the ring components reaches 0.80%, so the in-distribution result mostly reflects the alignment training. The authors' case for the architecture rests on the out-of-distribution sets, where that baseline reaches 2.32%, 8.62% and 0.33% and the full design reports 0.03%, 0.00% and 0.00%.

White-box adaptive attacks give the strongest test. GCG optimised suffixes fall from 50.71% to 1.44% attack success, and the NeuExe trigger from 48.44% to 1.56% in Table 6. The prose around that table quotes 47.81% and 0.87%, so version 1 contains a small reporting inconsistency.

Utility is measured on AlpacaEval 2.0 as a length-controlled win rate against the base checkpoint, where 50% means parity. The reported figures sit near parity for LLaMA-3-8B at 48.8% and above it for Qwen2.5-7B at 60.9%, while Mistral-7B lands at 42.7%. Zeroing the bias at inference sends out-of-distribution attack success to between 12% and 98%, which identifies the bias matrix as the working component.

What the results do and do not establish

The mechanism is a learned suppression bias, so it shifts behaviour and carries residual risk. Across the four models, in-distribution attack success sits between 0.51% and 0.69% after training, and the adaptive attacks still leave 1.4% to 1.6% success on LLaMA-3-8B. The authors' framing is structural separation under a stated threat model: the application labels each span correctly, and the attacker controls an external source while the ring assignment stays with the application. Inside that setting they report strong resistance. That is short of a hard security guarantee, and Promptective has not reproduced the measurements.

The design also asks for more than a prompt change. It modifies the model architecture and adds two supervised fine-tuning stages, so an existing hosted assistant cannot adopt it through prompting alone. The paper notes that pretraining-time integration remains future work.

The authors name a second limitation. Suppression is unconditional at ring granularity, so content the application labels untrusted is suppressed whether or not it is malicious. A pipeline that must use legitimate retrieved text needs a finer hierarchy, or it accepts losing that text as task context. The trade-off is between how much external content the model may act on and how much injection risk the labelling policy leaves open.

What to take into an agent pipeline

The transferable idea is that source authority becomes an explicit input where the application assembles context. Pipelines that already separate system instructions, user turns, retrieved passages and tool output already hold the labels.

Test the labels as a provenance property: run controlled cases where an injected instruction sits in an untrusted span, check whether the label governs the outcome, and include a matched case where the same text sits in a permitted span. Keep external controls in place too: permissions, sandboxing and exact approval still bound the outcome when a model is manipulated. Our guide to securing AI agents with tools covers those boundaries, and the what prompt injection is explainer sets out the trust problem behind the prompt injection examples guide.

Sources

Zhang, Huang and Gu, Origin Is All You Need: Provenance-Aware Transformers for Structural Trust-Boundary Separation, arXiv preprint, submitted 17 September 2026.
Full paper, version 1, sections 3 to 6. Accessed 22 September 2026.

Prompt injection and source authority: can a model tell who supplied the text?