CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis · HackerPapers discussions

CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis — mentioned by jncraton

Wed, 22 Mar 2023 19:21:00 GMT

What is wrong with the CodeGen model that they are using? It is a reasonably large model (up to 16B params) that has already been trained on both natural language and code. I would expect it to underperform larger models, including GPT-3.5 and GPT-4, but this should still be very useful for autocomplete and boilerplate in simpler cases. It is a bit under trained compared to Chinchilla, T5, or LLaMA, but it still performs well. According to the paper[1], this model is competitive with the largest Codex model that was the basis for Copilot originally. [1] https://arxiv.org/pdf/2203.13474.pdf

Paper and discussion history. A mention is not an endorsement.

CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis — HN submission

Sun, 12 Mar 2023 02:33:36 GMT

CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis — mentioned by jorams

Wed, 03 Aug 2022 06:06:23 GMT

From the paper[1]: > The family of CODEGEN models is trained sequentially on three datasets: THEPILE, BIGQUERY, and BIGPYTHON. > The natural language dataset THEPILE is an 825.18 GiB English text corpus collected by Gao et al. (2020) for language modeling. The dataset is constructed from 22 diverse high-quality subsets, one of which is programming language data collected from GitHub repositories with >100 stars that constitute 7.6% of the dataset. > The multi-lingual dataset BIGQUERY is a subset of Google’s publicly available BigQuery dataset, which consists of code in multiple programming…

Paper and discussion history. A mention is not an endorsement.

CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis — HN submission

Mon, 28 Mar 2022 00:47:27 GMT

A Conversational Paradigm for Program Synthesis

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.