Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models · HackerPapers discussions

Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models — mentioned by tkellogg

Tue, 21 Jan 2025 20:57:26 GMT

I think their thought process is unconvincing, although I think they're probably correct. A much better paper is, "Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models". They took it much further and trained an LLM such that for every inference, they could see exactly which document from the training dataset it referenced to answer a particular question. ngl This paper AGI-pilled me. https://arxiv.org/abs/2411.12580

Paper and discussion history. A mention is not an endorsement.

Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models — mentioned by sgt101

Sun, 12 Jan 2025 17:13:25 GMT

Proof? :) where has proof ever been found in truth? It's not proof, but there is some good evidence out there. Here are two interesting papers I found informative. https://arxiv.org/html/2403.04121v2 https://arxiv.org/html/2411.12580v1

Paper and discussion history. A mention is not an endorsement.

Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models — HN submission

Sun, 01 Dec 2024 16:54:26 GMT

Procedural knowledge in pretraining drives reasoning in large language models

248 points; 101 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models — HN submission

Wed, 20 Nov 2024 21:34:24 GMT

Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.