Large Language Models of Code Fail at Completing Code with Potential Bugs — mentioned by sigtstp

Mon, 05 May 2025 08:17:15 GMT

I feel this makes some fundamental conceptual mistakes and is just riding the LLM wave. "Semantics" is literally behavior under execution. This is syntactical analysis by a stochastic language model. I know the NLP literature uses "semantics" to talk about representations but that is an assertion which is contested [1]. Coming back to testing, this implicitly relies on the strong assumption of the LLM correctly associating the code (syntax) with assertions of properties under execution (semantic properties). This is a very risky assumption considering, once again, these things are stochastic…

Paper and discussion history. A mention is not an endorsement.

Large Language Models of Code Fail at Completing Code with Potential Bugs — HN submission

Wed, 07 Jun 2023 01:24:47 GMT

Large Language Models of Code Fail at Completing Code with Potential Bugs

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Large Language Models of Code Fail at Completing Code with Potential Bugs · HackerPapers discussions

Large Language Models of Code Fail at Completing Code with Potential Bugs — mentioned by sigtstp

Large Language Models of Code Fail at Completing Code with Potential Bugs — HN submission