Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions · HackerPapers discussions

Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions — mentioned by sigtstp

Mon, 05 May 2025 08:17:15 GMT

I feel this makes some fundamental conceptual mistakes and is just riding the LLM wave. "Semantics" is literally behavior under execution. This is syntactical analysis by a stochastic language model. I know the NLP literature uses "semantics" to talk about representations but that is an assertion which is contested [1]. Coming back to testing, this implicitly relies on the strong assumption of the LLM correctly associating the code (syntax) with assertions of properties under execution (semantic properties). This is a very risky assumption considering, once again, these things are stochastic…

Paper and discussion history. A mention is not an endorsement.

Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions — HN submission

Fri, 24 May 2024 20:12:02 GMT

A Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions — HN submission

Thu, 07 Sep 2023 17:06:16 GMT

Who answers better? ChatGPT or Stack Overflow for software engineering questions

2 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions — HN submission

Wed, 06 Sep 2023 09:56:01 GMT

Who Answers It Better? In-Depth Analysis of ChatGPT and Stack Overflow Answers

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions — HN submission

Wed, 16 Aug 2023 13:29:02 GMT

ChatGPT vs. Stack Overflow Answers to Sofware Engineering Questions

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions — HN submission

Tue, 15 Aug 2023 14:49:50 GMT

An In-Depth Analysis of ChatGPT and Stack Overflow Answers [pdf]

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions — HN submission

Mon, 14 Aug 2023 13:40:14 GMT

Who Answers It Better? An In-Depth Analysis of ChatGPT

2 points; 2 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions — HN submission

Thu, 10 Aug 2023 11:42:37 GMT

Comparing ChatGPT and Stack Overflow Answers to Software Engineering Questions

4 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions — HN submission

Tue, 08 Aug 2023 14:29:01 GMT

An In-Depth Analysis of ChatGPT and Stack Overflow Answers to SE Questions

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions — mentioned by tkgally

Tue, 08 Aug 2023 00:00:12 GMT

From the paper [1]: > For each of the 517 SO questions, the first two authors manually used the SO question's title, body, and tags to form one question prompt and fed that to the Chat Interface of ChatGPT. The generated answers by ChatGPT are then stored in CSV files. For the additional 2000 SO questions, ChatGPT 3.5 Turbo API is used. I would be interested in seeing the results for GPT-4. For a paper released in August 2023, it’s strange that GPT-4 isn’t even mentioned. [1] https://arxiv.org/abs/2308.02312

Paper and discussion history. A mention is not an endorsement.