How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources — mentioned by airgapstopgap

Thu, 06 Jul 2023 21:07:39 GMT

I believe it's more like they used Anthropic human preference data [1] or similar, and accordingly Anthropic/progressive American notion of honest-helpful-harmless behavior. Thus I've seen models misgeneralize towards prudish finger-wagging. For example they parse badwords like "beat", "abuse", "steal" in morally neutral contexts ("beat a benchmark" or something) as signifiers of substantial transgression and spiral into telling me how, as language models, they insist it's never okay to etc. etc. This attitude was strikingly reminiscent of American models, even though other failure modes –…

Paper and discussion history. A mention is not an endorsement.

How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources · HackerPapers discussions

How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources — mentioned by airgapstopgap