When External Threats Become User Context: Tricking Agents into Accepting Malicious Information

Abstract

Autonomous LLM agents receive task goals through user requests, but these requests may also relay untrusted external information. We study source relocation, where an externally available artifact is additionally included in a benign user's initial request while the task environment remains unchanged. Through controlled experiments across eight models and four AgentDojo domains, we find that user-context placement increases adoption of both malicious and benefit-matched benign artifacts, including without explicit artifact-use instructions. For GPT-4o in Travel, malicious-artifact adoption rises from 27.8\% to 95.6\%. Matched-checkpoint analyses of open-weight models reveal model-dependent differences in conditional artifact support, and activation patching identifies components contributing to this controlled score. The evaluated defense configurations leave substantial malicious execution under user-context placement. Explicit verification guidance raises GPT-4o's safe-completion rate from 2.2\% to 15.0\%, but gains vary across models and substantial residual risk remains. A complementary study of deployed agents finds sensitivity to safety guidance and instances of unsupported verification claims. Together, these results expose a challenge in agent delegation: the authority of a user's task does not establish the reliability of externally sourced material embedded in the same request. Our findings motivate separating task authorization from artifact verification and evaluating agents by safe task completion rather than task success alone.