Towards A Measure Of General Machine Intelligence · HackerPapers discussions

Towards A Measure Of General Machine Intelligence — mentioned by ahgamut

Sun, 06 Feb 2022 20:09:51 GMT

> To evaluate CoPilot, we should ask questions that are unusual enough they can't be answered through regurgitation of the training corpus. Exactly! It's great that Copilot can generate correct code for a given question, but we cannot gauge its full capability unless we try it on a range of different questions, especially ones that are not found in the training data. I mentioned this in the other AlphaCode post: It would be nice to know how "unusual" a given question is. Maybe an exact replica exists in the training data, or a solution exists but in a different programming language, or a…

Paper and discussion history. A mention is not an endorsement.

Towards A Measure Of General Machine Intelligence — mentioned by ahgamut

Wed, 02 Feb 2022 20:31:44 GMT

I find almost every new advance in deep learning is accompanied by contrasting comments: it's either "AI will soon automate programming/<insert task here>", or "let me know when AI can actually do <some-difficult-task>". There are many views on this spectrum, but these two are sure to be present in every comment section. IIUC, AlphaCode was trained on Github code to solve competitive programming challenges on Codeforces, some of which are "difficult for a human to do". Suppose AlphaCode was trained on Github code that contains the entire set of solutions on Codeforces, is it actually doing…

Paper and discussion history. A mention is not an endorsement.