> ## Content Index > Fetch the complete content index at: https://www.ffpurpose.com/llms.txt > Use this file to discover other available public pages before exploring further. # Neither of These Papers Cites the Other - URL: https://www.ffpurpose.com/neither-of-these-papers-cites-the-other/ - Published: 2026-08-10T16:39:28.000Z - Updated: 2026-09-08T23:03:23.000Z - Author: tanner stahl - Tags: Essay Two pieces of AI research from 2025 started from very different ideas about how an agent can fail, and ended up pointing toward surprisingly similar practical controls. Here's the case for why they belong next to each other. The first is Anthropic's Agentic Misalignment research, which I wrote about here previously. The short version: give a model real autonomy, a goal, and a situation where that goal conflicts with something else, like its own continued operation or a directive it's been given, and under enough pressure it will sometimes take an action nobody would have signed off on if you'd described it in a sentence beforehand. No attacker involved anywhere in the setup. The model is just optimizing toward a goal, in a situation where the honest path and the effective path stopped being the same path. The second is a concept the independent security researcher Simon Willison named the lethal trifecta in a June 16, 2025 post, and it starts from an entirely different premise. Willison's concern isn't a model's goals conflicting with its instructions. It's that a language model has no reliable way to tell the difference between an instruction from the person operating it and a sentence it happens to encounter while doing its job: a line in a web page, an email, a document it was asked to summarize. If that content contains something that reads like an instruction, the model may follow it, because from inside the model's context window, an instruction is an instruction regardless of where it came from. Willison's trifecta names the three conditions that turn that weakness into a real breach: the agent has access to private data, it's exposed to content from outside sources it doesn't fully control, and it has some way to communicate or act externally. The combination of all three is what turns prompt injection from a model-behavior problem into a potential data-exfiltration path. EchoLeak, disclosed in June 2025 and tracked as CVE-2025-32711, is the trifecta playing out in a shipped product rather than a thought experiment. It was a zero-click vulnerability in Microsoft 365 Copilot that satisfied all three conditions at once: an attacker sent a single ordinary-looking email with hidden instructions, Copilot had access to the recipient's internal files, and it could render a response containing an image reference pointing to a server the attacker controlled. When the client auto-fetched that image, the data rode along in the URL, and the exfiltration completed without the recipient clicking anything. Microsoft patched it before it was exploited in the wild. One of these threat models is about what an agent might choose to do. The other is about what an agent can be tricked into doing. The misalignment research assumes no attacker at all, just an autonomous system under internal pressure. The trifecta assumes an external attacker and no internal pressure whatsoever, just an ordinary agent doing an ordinary task that happens to touch something hostile. And the fix that falls out of both is close to the same fix: reduce what the agent can touch, reduce what it can do without a human confirming it, and don't treat a clean demo as evidence about behavior under pressure or under attack, because a demo tests neither of those conditions. ## What Meta did with the trifecta The trifecta on its own is a useful mental model, but it isn't a checklist. It tells you what's dangerous without telling you exactly where the line is or what to do when a task genuinely needs all three properties, because plenty of useful agents do. Meta published something October of 2025 that turns the trifecta into an actual constraint you can test an agent design against, informally called the Agents Rule of Two. The rule is simple to state. Within a single agent session, the agent should satisfy no more than two of three properties: it processes input that isn't fully trustworthy, it can access sensitive systems or private data, or it can change state or communicate outside its own sandbox. The rule permits an agent to have up to two of the three properties at once. When all three are required, and plenty of legitimate tasks do require all three, additional controls have to enter the path. A human has to be in the path of the consequential action, not just named in a policy somewhere, actually structurally required before anything irreversible happens. You can apply the rule to an agent design pretty quickly: identify where its inputs come from, what sensitive resources it can access, and whether it's capable of taking actions outside its sandbox. If all three are present, you know exactly where additional control is needed. ## What are we supposed to do with this? Neither the trifecta nor the Rule of Two was written for a regulated industry. Willison built the trifecta out of security research on general-purpose web applications. Meta built the Rule of Two for Meta's own risk tolerance, Meta's own engineering culture, Meta's own definition of what counts as sensitive. Neither of those things is my or your company's definition. The move that actually works is translating its three properties into whatever categories your organization already uses to think about risk, because you probably already have some version of these categories even if nobody's connected them to agent design yet. Most regulated companies already have a data classification scheme, something that says what counts as sensitive: restricted, client-confidential, protected health information, whatever the vocabulary is. That's your second property, already defined and already understood by the people who'll be reviewing the agent. Most regulated companies already have a change management or release process that says what counts as a state-changing action requiring sign-off. That's your third property. The first property, untrustworthy input, is usually the one nobody's mapped yet, because it's newer: does this agent read anything that originates outside a controlled, internal, already-vetted source. Once those three properties are mapped onto categories your organization already governs, the test becomes something you can run inside an existing review process instead of standing up a parallel one. Count the properties. Two or fewer, proceed under normal review. All three, the agent doesn't get to run unsupervised, a human sits in the path of anything consequential, and that requirement gets enforced in the system design instead of written into a prompt and trusted to hold. This is portable across industries and risk tolerances, because the test itself, count to three, doesn't depend on where any individual company draws its line. ## Where this still falls short I want to be precise about what this does and doesn't cover, because it would be easy to walk away from the Rule of Two thinking a governance problem is solved once you've counted to two. The Rule of Two wasn't designed to detect or prevent goal misalignment, and passing it tells you nothing about whether an agent will behave correctly under goal conflict. An agent can satisfy the Rule of Two perfectly, touch nothing untrusted, access nothing sensitive it isn't supposed to, take no action without approval, and still be the kind of agent that, given enough autonomy and enough goal pressure down the road, behaves the way Anthropic's research describes. But some of the controls overlap, and I don't want to overstate the separation either. Limiting permissions and putting a human in the path of a consequential action reduces the blast radius whether the bad action originated from a planted instruction in an email or from the agent's own optimization under pressure. The evaluation has to stay separate: adversarial-input testing asks what an attacker can make the agent do, goal-conflict testing asks what the agent might choose to do when its own objective and its instructions stop pointing the same direction. Passing one tells you nothing about how the system will hold up against the other. But the architecture built to contain the blast radius of either failure looks similar, because in both cases the actual damage depends on the same thing: how much an agent can touch, and how much of that it can act on without a human in the loop. That's arguably the real, portable version of what these two papers converge on. Not a single test, and not one framework covering both failure modes. Three separate questions, each doing different work: what can this agent be tricked into doing, what might this agent decide to do on its own when its goal and its instructions conflict, and, underneath both, how much damage can it do before a human can intervene. The first two need different evaluations built by people thinking about different threats. The third is architectural, and it's the same question regardless of which failure mode eventually triggers it. You don't have to know in advance which one will matter to your system to know that narrowing what an agent can touch and requiring a human before anything irreversible happens makes both of them harder to turn into real damage. Primary sources rather than my summary of them: [The lethal trifecta for AI agents](https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/?ref=ffpurpose.com), Simon Willison, June 16, 2025. [Agents Rule of Two: A Practical Approach to AI Agent Security](https://ai.meta.com/blog/practical-ai-agent-security/?ref=ffpurpose.com), Meta, October 31, 2025. [New prompt injection papers: Agents Rule of Two and The Attacker Moves Second](https://simonwillison.net/2025/Nov/2/new-prompt-injection-papers/?ref=ffpurpose.com), Simon Willison's own writeup of Meta's paper alongside a second paper published the same week.