Can LLM Graph Reasoning Generalize beyond Pattern Memorization? — mentioned by jokethrowaway

Wed, 01 Jan 2025 13:08:50 GMT

I don't think llm generalise much, that's why they're not creative and can't solve novel problems. It's pattern matching with a huge amount of data. Study on the topic: https://arxiv.org/html/2406.15992v1 This would explain o1 poor performance with problems with variations. o3 seems to be expensive brute forcing in latent space followed by verification which should yield better results - but I don't think we can call it generalisation. I think we need to go back to the drawing board.

Paper and discussion history. A mention is not an endorsement.

Can LLM Graph Reasoning Generalize beyond Pattern Memorization? · HackerPapers discussions

Can LLM Graph Reasoning Generalize beyond Pattern Memorization? — mentioned by jokethrowaway