HackerPapers

arXiv:2006.08381cs.AI

DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learning

Kevin Ellis, Catherine Wong, Maxwell Nye, Mathias Sable-Meyer, Luc Cary, Lucas Morales, Luke Hewitt, Armando Solar-Lezama, Joshua B. Tenenbaum

Abstract

Expert problem-solving is driven by powerful languages for thinking about problems and their solutions. Acquiring expertise means learning these languages -- systems of concepts, alongside the skills to use them. We present DreamCoder, a system that learns to solve problems by writing programs. It builds expertise by creating programming languages for expressing domain concepts, together with neural networks to guide the search for programs within these languages. A ``wake-sleep'' learning algorithm alternately extends the language with new symbolic abstractions and trains the neural network on imagined and replayed problems. DreamCoder solves both classic inductive programming tasks and creative tasks such as drawing pictures and building scenes. It rediscovers the basics of modern functional programming, vector algebra and classical physics, including Newton's and Coulomb's laws. Concepts are built compositionally from those learned earlier, yielding multi-layered symbolic representations that are interpretable and transferrable to new tasks, while still growing scalably and flexibly with experience.

arXiv comments
Journal reference
DOI
Subjects
cs.AIArtificial Intelligence cs.LGMachine Learning

On Hacker News

Linked by 12 identifiable commenters in 15 comments across 13 threads, from Dec 2020 to May 2026. Found in 3 HN submissions from Jun 2020 to Dec 2023. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments13 threads

  1. A sleep-like consolidation mechanism for LLMs 1 comment

    • This topic recently came up at the FLANN workshop [1], and seems to periodically be rediscovered [2,3,4] in different contexts. While some have speculated about the biological role it plays (e.g., Pearlmutter & Houghton [5]), we still lack a conclusive theory of sleep, but the…

      bmc7505View on HNParent context

  2. Language is not essential for the cognitive processes that underlie thought 1 comment

    • > So if someone figures out to do this, it will probably take less hardware than an LLM. We have, it's called DreamCoder. There's a paper and everything. Everything needed for AGI exists today, people simply have (incorrect) legacy beliefs about cognition that are holding them…

      erichoceanView on HNParent context

  3. Doug Lenat has died 1 comment

    • Some parts of PAIP might be outdated, but it still has really current material on e.g. embedding Prolog in Lisp or building a term-rewriting system. That's relevant for pursuing current neuro-symbolic research, e.g. https://arxiv.org/pdf/2006.08381.pdf. Other parts like coding…

      nextosView on HNParent context

  4. Ask HN: Who is hiring? (July 2023) 1 comment

    • AI Research Residency Program (1 year) | Zug/Zurich area | Full time | On site | 100-200k CHF/annum I am launching a new AI research organisation in the Zug/Zurich area. The goal of the organisation is to make a meaningful contribution towards solving AGI. I am looking for…

      notacatView on HNParent context

  5. Early Computer Art in the 50s and 60s 1 comment

    • From more recent times, and not specifically an art project, but I liked the work on DreamCoder, particularly the part where it learned to draw in Logo graphics. See figure on page 11 for some graphics: DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep…

      YeGoblynQueenneView on HNParent context

  6. AI Coding with CodeRL: Toward Mastering Program Synthesis with Deep RL 1 comment

    • >> It is current state of art approach. That's not right. The approach described in the article and the linked paper is not "state of the art". It beats one benchmark which has only been used to compare one kind of system: neural program synthesis systems (Codex, AlphaCode…

      YeGoblynQueenneView on HNParent context

  7. AlphaCode as a dog speaking mediocre English 1 comment

    • > I would actually be more impressed if it understood enough about programs to be able to mimic higher-level structure without just memorizing text. Agreed. > Why do you think Python isn't good for comparing program structure? From recent experience I prefer control flow…

      ahgamutView on HNParent context

  8. Competitive Programming with AlphaCode 2 comments

    • >> This is technology that simply didn't exist in any form 2 years ago. A few examples of neural program synthesis from at least 2 years ago: https://sunblaze-ucb.github.io/program-synthesis/index.html Another example from June 2020: DreamCoder: Growing generalizable…

      YeGoblynQueenneView on HNParent context

    • That’s not entirely correct, as I know of at least one approach to neural program synthesis that employs self supervised learning. https://arxiv.org/abs/2006.08381 It’s a slightly different, easier problem: generating programs based on example outputs, rather than natural…

      bglazerView on HNParent context

  9. In the 17th century, Leibniz dreamed of a machine that could calculate ideas 1 comment

    • > if he could find a way to symbolically represent these fundamental concepts and develop a method by which to combine them logically, then he would be able to generate new thoughts on demand Reminds me of some recent neurosymbolic work, like "DreamCoder: Growing generalizable…

      KhoomeiKView on HNParent context

  10. GPT-3 is no longer the only game in town 1 comment

    • Program induction (or inductive programming, program synthesis from incomplete specifications etc, the task of learning programs from examples) is not "headed towards ML", it is a branch of machine learning research - only, one that is dominated by symbolic learning approaches…

      YeGoblynQueenneView on HNParent context

  11. Why AM and Eurisko Appear to Work (1983) [pdf] 1 comment

    • Here is a 1984 article on Eurisko: https://aliciapatterson.org/stories/eurisko-computer-mind-it... From the article we learn Eurisko is more of an example of human machine symbiosis than of AI. "Thus the final crediting of the win should be about 60/40% Lenat/Eurisko," he wrote…

      CybioteView on HNParent context

  12. Kobra – visual programming language for machine learning 1 comment

    • I think https://www.perceptilabs.com/ is a little better. But I hope the future is more like DreamCoder: https://arxiv.org/abs/2006.08381

      swframe2View on HNParent context

  13. Ask HN: What's the best paper you've read in 2020? 2 comments

    • DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learning https://arxiv.org/abs/2006.08381 A killer paper presenting an algorithm capable of inductive learning. ("DreamCoder solves both classic inductive programming tasks and creative…

      macleginnView on HNParent context

    • https://arxiv.org/pdf/2006.08381.pdf A take on Josh Tenenbaum's hard problem of learning

      pinouchonView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.