System 2 Attention (is something you might need too) · HackerPapers discussions

System 2 Attention (is something you might need too) — mentioned by activatedgeek

Mon, 03 Feb 2025 18:09:15 GMT

That has been my understanding too. More generally, a verifier at the end certainly helps. In our paper [1], we find that asking a follow up question like "Is the answer correct?" and taking the normalized probability of "Yes" or "No" token (or more generally any such token trained for) seems to be best bet so far to get well-calibrated probabilities out of the model. In general, the log-probability of tokens is not a good indicator of anything other than satisfying the pre-training loss function of predicting the "next token." (it likely is very well-calibrated on that task though) Semantics…

Paper and discussion history. A mention is not an endorsement.

System 2 Attention (is something you might need too) — mentioned by astrange

Sat, 23 Mar 2024 00:05:48 GMT

Facebook had a paper about "system 2" LLM attention, where they identified which parts of the input would be distracting for the LLM and just deleted them. https://arxiv.org/abs/2311.11829

Paper and discussion history. A mention is not an endorsement.

System 2 Attention (is something you might need too) — mentioned by armcat

Fri, 08 Mar 2024 16:33:51 GMT

RAG approaches should work quite well for the examples you mentioned. It's a matter of how you approach the retrieval part - you can opt for a larger recall on retrieval, and leverage the large context window for the LLM to figure out the answer. Even if it's not "as-is", semantically if it's in there, it should be able to find it. Other things to try out is how you approach the question "expansion" part, for example using Hypothetical Document Embeddings (HyDE); or how you approach the filtering-out part, e.g. using "System 2 Attention", https://arxiv.org/abs/2311.11829.

Paper and discussion history. A mention is not an endorsement.

System 2 Attention (is something you might need too) — HN submission

Sun, 26 Nov 2023 07:12:57 GMT

System 2 Attention (is something you might need too)

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

System 2 Attention (is something you might need too) — HN submission

Wed, 22 Nov 2023 17:29:05 GMT

System 2 Attention (is something you might need too)

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

System 2 Attention (is something you might need too) — HN submission

Tue, 21 Nov 2023 14:11:06 GMT

System 2 Attention (is something you might need too)

1 point; 2 discussion replies on HN. Paper and source history. A submission is not an endorsement.