Schema-learning and rebinding as mechanisms of in-context learning and emergence — mentioned by ssivark

Thu, 20 Nov 2025 19:46:19 GMT

The Markov property means that the next token is determined purely by the current token. Well, if it were a Hidden Markov Model, the next state would actually be determined by the current state, and the respective tokens would be a lossy representation of the states. The problem with HMMs is that the sequence model (Markov transition matrix) accounts for much less context than even Tiny LLMs. One natural way to improve this is to allow the model to have more hidden states, representing more context -- called "clones" because these different hidden states would all be producing the same token…

Paper and discussion history. A mention is not an endorsement.

Schema-learning and rebinding as mechanisms of in-context learning and emergence — HN submission

Mon, 18 Sep 2023 20:15:55 GMT

Schema-learning and rebinding as mechanisms of in-context learning and emergence

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Schema-learning and rebinding as mechanisms of in-context learning and emergence · HackerPapers discussions

Schema-learning and rebinding as mechanisms of in-context learning and emergence — mentioned by ssivark

Schema-learning and rebinding as mechanisms of in-context learning and emergence — HN submission