Mirostat: A Neural Text Decoding Algorithm that Directly Controls Perplexity · HackerPapers discussions

Mirostat: A Neural Text Decoding Algorithm that Directly Controls Perplexity — mentioned by tibbar

Wed, 25 Feb 2026 07:05:07 GMT

Determinism in agents is a complex topic because there are several different layers of abstraction, each of which may introduce its own non-determinism. But yeah, it is going to be difficult to induce determinism in a commercial coding agent, for reasons discussed below. However, we can start by claiming that non-determinism is not necessarily a bad thing - non-greedy token sampling helps prevent certain degenerate/repetitive states and tends to produce overall higher quality responses [0]. I would also observe that part of the yin-yang of working with the agents is letting go of the idea…

Paper and discussion history. A mention is not an endorsement.

Mirostat: A Neural Text Decoding Algorithm that Directly Controls Perplexity — mentioned by numeri

Mon, 08 Jan 2024 00:16:43 GMT

Almost all LLM inference these days includes a repetition penalty, to help prevent models from falling into this pattern, the so-called "boredom trap" [1]. Once the model repeats itself once, that greatly increases the chances that the next best token is another repetition (for a raw model like this, at least, that hasn't been chat/instruct/RLHF fine-tuned). I ran the same prompt with a repetition penalty of 1.1 and got this: The mountain in front of me was a little higher than the other two, and I could see the top of the mountain. I'm going to climb it," I said. "It's a good place for a…

Paper and discussion history. A mention is not an endorsement.