The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performance · HackerPapers discussions

The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performance — mentioned by HaZeust

Thu, 24 Oct 2024 16:59:54 GMT

Every single prompt hack I listed are ones with studies that show it positively increases performance. Since the most contested one in this thread is the "tipping" prompt hack: https://arxiv.org/pdf/2401.03729

Paper and discussion history. A mention is not an endorsement.

The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performance — HN submission

Wed, 10 Apr 2024 15:16:08 GMT

Small Changes and Jailbreaks Affect Large Language Model Performance

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performance — mentioned by absal

Sat, 24 Feb 2024 23:37:52 GMT

I have some related work where we looked at how tipping (and other variations) affect predictions and accuracy in classification tasks. We experimented with ChatGPT and the different versions of Llama 2. TLDR: We found similar results where tipping performs better in some tasks and worse in others, but it doesn't make a big difference overall. The one exception was Llama 7B where tipping beat all the other prompt variations we tested by several percentage points. This suggests that the impact of tipping might diminish with model size. https://arxiv.org/pdf/2401.03729.pdf

Paper and discussion history. A mention is not an endorsement.