Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants — HN submission

Wed, 01 Mar 2023 19:32:09 GMT

Lost at C: A user study on the security of LLM coding assistants

31 points; 13 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants — mentioned by moyix

Mon, 26 Dec 2022 20:52:00 GMT

This is a really nice study! It is very cool that they were able to get professional programmers to participate, this is something that is really hard to set up as an academic team. And yes, 47 participants is a small number, but apparently sufficient in this case to detect the effect (as evidenced by the p-values). It also lines up nicely with work we did last year, which looked at the security of Copilot's completions without any humans in the loop [1] and found that something like 40% of its completions in security sensitive scenarios were vulnerable. One thing I'm not sure of is how well…

Paper and discussion history. A mention is not an endorsement.

Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants · HackerPapers discussions

Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants — HN submission

Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants — mentioned by moyix