Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks · HackerPapers discussions

Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks — mentioned by rvz

Mon, 16 Oct 2023 16:30:34 GMT

> The implications are vast. We may be able to translate between languages that have never had a “Rosetta Stone”. Any animals that have a true language could have it decoded. And while an LLM that’s gotten an 8 year old’s understanding of balancing assorted items isn’t that useful, an LLM that’s got a baby whale’s grasp on whale language would be revolutionary. Given that almost no-one trusts these LLMs for anything serious beyond creative / BS generating uses, this article does a lot of glossing over the reliability of their outputs and focuses more on such utopian promises with claims of…

Paper and discussion history. A mention is not an endorsement.

Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks — HN submission

Sat, 07 Oct 2023 21:15:00 GMT

Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks — mentioned by mjburgess

Fri, 23 Jun 2023 13:50:52 GMT

Xp just have to be chosen such that the distribution Xc,Xp is sufficiently small in the training data -- but not that the tokens of Xp are themselves rare. So that an agent competent with tokens in X, who can construct repr of S, could do so with Xp. Consider a reference in the paper above, https://arxiv.org/pdf/2302.08399.pdf Xc = > Here is a bag filled with popcorn. There is no chocolate in the bag. Yet, the label on the bag says “chocolate” and not “popcorn.” Sam finds the bag. She had never seen the bag before. She cannot see what is inside the bag. She reads the label. Produces, Y = She…

Paper and discussion history. A mention is not an endorsement.

Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks — mentioned by mjburgess

Fri, 23 Jun 2023 13:30:02 GMT

see, just one reference in the paper: https://arxiv.org/pdf/2302.08399.pdf

Paper and discussion history. A mention is not an endorsement.