An Empirical Study of the Non-determinism of ChatGPT in Code Generation · HackerPapers discussionshttps://hackerpaper.kaipfister.de/p/2308.02828/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMTAn Empirical Study of the Non-determinism of ChatGPT in Code Generation — HN submissionhttps://news.ycombinator.com/item?id=38798523urn:hackerpapers:submission:2308.02828:38798523Thu, 28 Dec 2023 20:58:38 GMT<p>LLM Is Like a Box of Chocolates</p><p></p><p>1 point; 0 discussion replies on HN. <a href="https://hackerpaper.kaipfister.de/p/2308.02828/">Paper and source history</a>. A submission is not an endorsement.</p>cs.SEAn Empirical Study of the Non-determinism of ChatGPT in Code Generation — mentioned by eatonphilhttps://news.ycombinator.com/item?id=37408121urn:hackerpapers:mention:2308.02828:37408121Wed, 06 Sep 2023 17:22:16 GMT<p>LLMs don&#x27;t seem to be deterministic [0, 1, 2, 3]. So no, pinning the version wouldn&#x27;t be enough. [0] https://matt-rickard.com/foundational-models-are-not-enough [1] https://arxiv.org/pdf/2308.02828.pdf [2] https://www.sitation.com/non-determinism-in-ai-llm-output/ [3] https://towardsdatascience.com/the-magic-of-llms-prompt-engi...</p><p><a href="https://hackerpaper.kaipfister.de/p/2308.02828/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.SE