Approximating CKY with Transformers — mentioned by thomasahle

Sun, 25 Jun 2023 07:24:42 GMT

> LLM's don't actually understand anything (as greater concepts), but rather operate as complex probability machines? What does it mean to "operate as a probability machine"? And what does it mean to understand anything? One recent example of understanding is that llms/transformers learn to parse context free grammars via dynamic programming (https://arxiv.org/abs/2305.02386). Basically they've understood what's going on well enough to mold their neurons I to the optimal algorthm for parsing this kind of text. I think they understand lots of things like this. Of course there's other things…

Paper and discussion history. A mention is not an endorsement.

Approximating CKY with Transformers · HackerPapers discussions

Approximating CKY with Transformers — mentioned by thomasahle