Can LLMs Separate Instructions From Data? And What Do We Even Mean By That? · HackerPapers discussions

Can LLMs Separate Instructions From Data? And What Do We Even Mean By That? — mentioned by dijksterhuis

Sat, 20 Jun 2026 14:30:20 GMT

if you want to avoid my massive post (sorry), there's a paper here positing how instruction-data separation is likely a major cause of prompt injection specifically. https://arxiv.org/pdf/2403.06833 then another paper where they change the architecture of a model to deal with the problem and it doesn't eliminate prompt injection. changing the architecture doesn't make this problem go away. the approximate function still gets tricked. > On average, ASIDE lowers attack success rate by 8.6 and 9.4 percentage points https://arxiv.org/pdf/2503.10566 the real over-arching cause of all these…

Paper and discussion history. A mention is not an endorsement.

Can LLMs Separate Instructions From Data? And What Do We Even Mean By That? — mentioned by egozverev

Wed, 09 Jul 2025 09:43:35 GMT

Academic researcher here working on this exact issue. Prompt engineering methods are no sufficient to address the challenge. People in Academy and Industry labs are aware of the issue and actively working on it, see for instance: [1] Camel: work by google deepmind on how to (provably!) prevent agent planner from being prompt-injected: https://github.com/google-research/camel-prompt-injection [2] FIDES: similar idea by Microsoft, formal guarantees: https://github.com/microsoft/fides [3] ASIDE: marking non-executable parts of input and rotating their embedding by 90 degrees to defend against…

Paper and discussion history. A mention is not an endorsement.

Can LLMs Separate Instructions From Data? And What Do We Even Mean By That? — HN submission

Fri, 29 Mar 2024 16:55:49 GMT

Can LLMs Separate Instructions from Data? and What Do We Even Mean by That?

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.