R—03Prompt injection1 min read

Designing for untrusted context

A threat model for retrieval systems that assumes every document is adversarial.

Retrieval-augmented systems turn documents into instructions unless the surrounding architecture prevents it. The safe default is simple: retrieved content is data from an untrusted principal.

Treat provenance as a control

Preserve source identity as content moves through chunking, embedding, retrieval, and generation. If provenance disappears before the model sees a passage, downstream controls cannot make meaningful decisions about it.

Separate content used to answer a question from content allowed to authorize an action. Retrieval can supply evidence. It should not grant permission.

Design for containment

Limit the number of documents and tools available in one step. Strip active content where possible. Gate privileged tool calls behind structured policy checks. Show citations to users so they can inspect the evidence behind consequential answers.

The goal is not to make injection text impossible. The goal is to ensure that hostile text has nowhere dangerous to go.