Prefix-Tuning: Optimizing Continuous Prompts for Generation — mentioned by gwern

Mon, 22 Feb 2021 18:18:18 GMT

https://arxiv.org/abs/2102.07350 already calls it "metaprompt" :) I gave it a quick stab a while ago but I think prompt programming is too new, and you can't easily cram demonstrations into an existing prompt, for it to really work well. It's more promising to train models on examples of tasks from instructions, or work on directly optimizing prompts for a goal (https://arxiv.org/abs/2101.00190) - it's a differentiable model and a whitebox, so use that power!

Paper and discussion history. A mention is not an endorsement.

Prefix-Tuning: Optimizing Continuous Prompts for Generation · HackerPapers discussions

Prefix-Tuning: Optimizing Continuous Prompts for Generation — mentioned by gwern