Chain of Thoughtlessness? An Analysis of CoT in Planning · HackerPapers discussions

Chain of Thoughtlessness? An Analysis of CoT in Planning — mentioned by YeGoblynQueenne

Sat, 19 Oct 2024 15:15:58 GMT

My thoughts on CoT and the extent to which it "elicits reasoning" align almost perfectly with the criticism of Rao Kambhampati and his students: https://arxiv.org/abs/2405.04776 Their argument is that CoT can only improve performance of LLMs in reasoning tasks when the prompter already knows the answer and can somehow embed it in their prompt. The paper I link above supports this intuition with empirical results, summarised in the abstract as follows: While our problems are very simple, we only find meaningful performance improvements from chain of thought prompts when those prompts are…

Paper and discussion history. A mention is not an endorsement.

Chain of Thoughtlessness? An Analysis of CoT in Planning — mentioned by subbarao2z

Sun, 12 May 2024 14:12:26 GMT

[I am the one who wrote that tweet thread]. fwiw, a lot of the questions that are being raised in this thread have been answered in this talk I gave for Google/Deepmind LLM reasoning seminar a few days back: https://www.youtube.com/watch?v=hGXhFa3gzBs (Also, the preprint paper on the CoT study with all the prompts etc is available at https://arxiv.org/abs/2405.04776 )

Paper and discussion history. A mention is not an endorsement.

Chain of Thoughtlessness? An Analysis of CoT in Planning — HN submission

Thu, 09 May 2024 19:47:54 GMT

Chain of Thoughtlessness: An Analysis of Cot in Planning

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.