HackerPapers

arXiv:2303.12712cs.CL

Sparks of Artificial General Intelligence: Early experiments with GPT-4

Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, Yi Zhang

Abstract

Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition. The latest model developed by OpenAI, GPT-4, was trained using an unprecedented scale of compute and data. In this paper, we report on our investigation of an early version of GPT-4, when it was still in active development by OpenAI. We contend that (this early version of) GPT-4 is part of a new cohort of LLMs (along with ChatGPT and Google's PaLM for example) that exhibit more general intelligence than previous AI models. We discuss the rising capabilities and implications of these models. We demonstrate that, beyond its mastery of language, GPT-4 can solve novel and difficult tasks that span mathematics, coding, vision, medicine, law, psychology and more, without needing any special prompting. Moreover, in all of these tasks, GPT-4's performance is strikingly close to human-level performance, and often vastly surpasses prior models such as ChatGPT. Given the breadth and depth of GPT-4's capabilities, we believe that it could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence (AGI) system. In our exploration of GPT-4, we put special emphasis on discovering its limitations, and we discuss the challenges ahead for advancing towards deeper and more comprehensive versions of AGI, including the possible need for pursuing a new paradigm that moves beyond next-word prediction. We conclude with reflections on societal influences of the recent technological leap and future research directions.

arXiv comments
Journal reference
DOI
Subjects
cs.CLComputation and Language cs.AIArtificial Intelligence

On Hacker News

Linked by 66 identifiable commenters in 76 comments across 66 threads, from Mar 2023 to Aug 2026. Found in 2 HN submissions from Mar 2023 to Apr 2025. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments66 threads

  1. Karpathy’s Pelican 1 comment

    • I just went down a rabbit-hole, and happily I found the rabbit - an early example of this type of prompt (draw an unusual animal character in a vector graphics language). There is a paper and a 1 hour podcast resulting from a Microsoft evaluation of a pre-release of GPT4. One of…

      iDonView on HNParent context

  2. The last six months in LLMs in five minutes 1 comment

    • > there’s zero chance any AI lab would train a model for such a ridiculous task. A lot of people here stated that this is a ridiculous metric, but no one seems to remember that it was introduced in the initial GPT report ("Sparks of Artificial General Intelligence: Early…

      romanivView on HNParent context

  3. AI Added 'Basically Zero' to US Economic Growth Last Year, Goldman Sachs Says 1 comment

    • Minor correction: "sparks of AGI" was in reference to GPT-4, which came out 5 months later in 2024. (https://arxiv.org/abs/2303.12712)

      dougb5View on HNParent context

  4. Claude Sonnet 4.6 1 comment

    • > alignment becomes adversarial against intelligence itself. It was hinted at (and outright known in the field) since the days of gpt4, see the paper "Sparks of agi - early experiments with gpt4" (https://arxiv.org/abs/2303.12712)

      NitpickLawyerView on HNParent context

  5. Claim: GPT-5-pro can prove new interesting mathematics 1 comment

    • Yes, he published a paper claiming GPT-4 has "sparks" of AGI. What else is he known for in the field of computer science? https://arxiv.org/abs/2303.12712

      alkyonView on HNParent context

  6. AI 2027 1 comment

    • Disagree. We know it _can_ learn out of distribution capabilities based on similarities to other distributions. Like the TikZ Unicorn[1] (which was not in training data anywhere) or my code (which has variable names and methods/ideas probably not seen 1:1 in training). IMO this…

      byearthithatiusView on HNParent context

  7. I genuinely don't understand why some people are still bullish about LLMs 1 comment

    • No major company directly states "We have created human-like intelligence," they intentionally use suggestive language that leads people to think AI is approaching human cognition. This helps with hype, investment, and PR. >I'd love to see if some company specifically anything…

      gloosxView on HNParent context

  8. Microsoft and OpenAI's close partnership shows signs of fraying 1 comment

    • Microsoft themselves were the ones who wrote the "Sparks of AGI" paper. https://arxiv.org/pdf/2303.12712

      ben_wView on HNParent context

  9. I am starting an AI+Education company 1 comment

    • > Feynman, Bill Gates, Socrates, etc. One of these doesn't quite belong ;) But these AI researchers don't even understand these figures except as advertising reference points. The Socratic dialogue in the "sparks of AGI" paper https://arxiv.org/abs/2303.12712 has nothing…

      nybsjytmView on HNParent context

  10. ChatGPT is better at generating code for problems written before 2021 1 comment

    • The gpt4 model was able to draw a unicorn and given a program that drew a unicorn, modify it so the unicorn was rotated 90 degrees. https://arxiv.org/abs/2303.12712

      hilbertseriesView on HNParent context

  11. Large language models have developed a higher-order theory of mind 1 comment

    • That talk[1] (also paper[2]) covering some of this by the Microsoft team involved with the unfettered GPT-4 model certainly shifted my views last year at a time when until then it was more common to see downplaying about LLMs based on prior gen models. Recently saw an AI safety…

      SpringtimeView on HNParent context

  12. What Is ChatGPT Doing and Why Does It Work? (2023) 1 comment

    • For a more informed opinion than folks on the internet, here's some work from Microsoft with early/internal access to gpt4: https://arxiv.org/abs/2303.12712 . I don't think people close by these systems share the same dumb parrot sentiment at all.

      w1nkView on HNParent context

  13. Sutskever: OpenAI board doing its mission to build AGI that benefits all 1 comment
  14. Google AI chief says there's a 50% chance we'll hit AGI in just 5 years 1 comment

    • 1. "I think GPT-3 is artificial general intelligence, AGI. I think GPT-3 is as intelligent as a human. And I think that it is probably more intelligent than a human in a restricted way… in many ways it is more purely intelligent than humans are. I think humans are approximating…

      famouswafflesView on HNParent context

  15. DALL·E 3 1 comment

    • The art produced by GPT-4 so far hasn't been at this level but this may be a newer version. See: https://arxiv.org/pdf/2303.12712.pdf

      ekamView on HNParent context

  16. The AI Reproducibility Crisis 1 comment

    • *Further Reading*: - [GPT-4's decline over time (HackerNews)](https://news.ycombinator.com/item?id=36786407) - [GPT-4 downgrade discussions (OpenAI Forums)](https://community.openai.com/t/gpt-4-has-been-severely-downg...) - [Behavioral changes in ChatGPT…

      ocolegroView on HNParent context

  17. AI isn’t good enough 1 comment

    • Just to further this, it's not just 'big names' that feel this way. Read this paper from a team at Microsoft Research: https://arxiv.org/abs/2303.12712 . These folks spent months studying properties of GPT-4, that paper is ~150 pages of examples probing the boundaries of the…

      w1nkView on HNParent context

  18. GPT Unicorn has drawn a unicorn 1 comment
  19. AI researcher Geoffrey Hinton thinks AI has or will have emotions 1 comment

    • > What is speaking if not stringing together coherent words in a readable sentence? Haven't the latest LLMs shown us that a neural net trained to "just string together readable words" leads to at least simple intelligence? [1] > I would say that emotion is something more…

      jonny_ehView on HNParent context

  20. ChatGPT Now Losing Users 1 comment

    • Your response does two things -- Dismisses the countless examples given, from me in this thread/my comment history, and many other people in many threads on this website and Reddit. Dismisses the pretense that certain people get access to unfiltered models under the guise of…

      uLogMichealView on HNParent context

  21. Experiencing decreased performance with ChatGPT-4 1 comment

    • Aggressive quantization by itself could explain the quality difference. Combined with stricter guardrails, I would certainly expect intelligence to go down. Check out the tikz unicorn drawing example from the paper "Sparks of Artificial General Intelligence: Early experiments…

      vczfView on HNParent context

  22. GPT-4 API General Availability 1 comment

    • I think the clearest evidence is Microsofts paper where they show abilities at various stages during training[1]... But in a talk [2], they give more details... The unicorn gets worse during the finetuning process. [2]: https://www.youtube.com/watch?v=qbIk7-JPB2c&t=1392s [1]…

      londons_exploreView on HNParent context

  23. Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately? 1 comment

    • Chat GPT 4 has ongoing training, such as using Reinforcement Learning from Human Feedback (RLHF) to tune it to provide "better" responses, "safer" answers, and to generally obey the system prompts. There's a release every few weeks. Yes, I've noticed too that recently it has…

      jiggawattsView on HNParent context

  24. Statement on AI Risk 1 comment

    • I'd say stronger evidence than the Othello paper is the ability to answer what-if questions coherently and plausibly. I just asked GPT-4 "What would happen to Northern Thailand if elephants behave like kangaroos?". The answers are probably better than what 90+% of humans could…

      nopinsightView on HNParent context

  25. Chatbot Arena Leaderboard 1 comment
  26. Programming with natural language is going to work (2010) 1 comment

    • The unicorn example from MSFT's Sparks of AGI paper, https://arxiv.org/abs/2303.12712, probably counts as a novel problem. As another example, ImageBlind, https://ai.facebook.com/blog/imagebind-six-modalities-bindin..., learned modality translations explicitly not in its…

      gaogaoView on HNParent context

  27. Rodney Brooks on GPT-4 2 comments

    • This paper is startling in terms of a world model and how it seems to emerge in a specific example in the step from GPT3.5 to 4. https://arxiv.org/pdf/2303.12712.pdf One of the prompts is "Here we have a book, 9 eggs, a laptop, a bottle and a nail. Please tell me how to stack…

      seanhunterView on HNParent context

    • Yes. One of the astonishing examples in the paper I quoted earlier [1] is an image produced in response to the prompt "Produce Javacript code that creates a random graphical image that looks like a painting of Kandinsky”. The response by GPT-4 does kind of look quite a lot like…

      seanhunterView on HNParent context

  28. US Senator Uses ChatGPT for Opening Remarks at a Hearing on AI 1 comment

    • It is not about what the model tells you. This paper shows an emergent world model in an LLM that was taught to play otello moves https://ar5iv.labs.arxiv.org/html/2210.13382 https://arxiv.org/pdf/2303.12712.pdf This paper discusses (among other things) how a GPT4 model…

      patrickasView on HNParent context

  29. LLMs are not greater than the sum of their parts: researchers 1 comment

    • You might find this video[1] on why "GPT is smarter than you think" interesting. They're older, but the "Sparks of AGI" paper[2] and video[3] are also well worth your time. [1] - https://m.youtube.com/watch?v=wVzuvf9D9BU [2] - https://arxiv.org/abs/2303.12712 [3]…

      pmoriartyView on HNParent context

  30. Geoffrey Hinton leaves Google and warns of danger ahead 1 comment

    • I read that pre-print Microsoft paper. Despite the title, it doesn't actually show any real "sparks" of AGI (in the sense of something that could eventually pass a rigorous Turing test). What the paper actually shows is that even intelligent people have a bias towards perceiving…

      nradovView on HNParent context

  31. We aren't close to creating a rapidly self-improving AI 1 comment

    • https://arxiv.org/abs/2303.12712 > As far as I'm aware they were still hallucinating pretty hard and incredibly biased and overly agreeable. Certainly. But those aren't blockers to displaying the ability to reason within limited contexts. There are hundreds of thousands of human…

      mrshadowgooseView on HNParent context

  32. The future of generative AI is niche, not generalized 1 comment

    • From 'Sparks of AGI' paper from Microsoft (https://arxiv.org/pdf/2303.12712.pdf). Caveat: The GTP-4 model they used in their example might have been more powerful than the one we use publicly today, since the public version is trained for safety and less 'intelligent' than the…

      MVissersView on HNParent context

  33. AGI Is Not Chess Bots – They Will Replace Your Jobs 1 comment

    • First of all, AI doesn't just write text. It (Midjourney v5) can generate realistic images that humans can’t tell if they are really not not. They can also generate music and that’s so good that shakes the entire music industry…

      codermindView on HNParent context

  34. Sparks of AGI? – Analyzing GPT-4 and the Latest GPT/LLM Models [sentdex] 1 comment

    • I was really hoping someone would reproduce the tests and validate the claims from Microsoft's "Sparks of AGI" paper [1]. This video just does that (and more). Let me provide one imporant quote from the video (from 18:50): "I personally reproduced every single example from…

      paboView on HNParent context

  35. Congress gets 40 ChatGPT Plus licenses to start experimenting with generative AI 1 comment

    • If you want proof, it is enough to point to the following recent paper from Microsoft people: https://arxiv.org/abs/2303.12712 All things mentioned in Section 10.2 are extremely worrying in a context of lawmaking and jurisprudence in general.

      jruohonenView on HNParent context

  36. The Sell ∀ ∃ as ∃ ∀ Scam 1 comment

    • The Microsoft Research "Sparks of AGI" paper spends 154 pages describing behaviors of GPT-4 that are inconsistent with the understanding of it being a "fancy Markov chain": https://arxiv.org/abs/2303.12712 I expect that the reason people are constantly arguing with you is that…

      cjbprimeView on HNParent context

  37. Google DeepMind 1 comment

    • I'd recommend reading the Sparks of AGI paper[1] and watching the accompanying video[2] [1] - https://arxiv.org/abs/2303.12712 [2] - https://m.youtube.com/watch?v=qbIk7-JPB2c

      pmoriartyView on HNParent context

  38. Emergent Abilities of Large Language Models 1 comment

    • I liked the paper titled "Sparks of Artificial General Intelligence: Early experiments with GPT-4" better. Link: https://arxiv.org/pdf/2303.12712.pdf

      jiggawattsView on HNParent context

  39. Peak LLM? 1 comment

    • That hasn't been determined http://incompleteideas.net/IncIdeas/BitterLesson.html Sparks of Artificial General Intelligence: Early experiments with GPT-4 https://arxiv.org/abs/2303.12712 LLMs exhibit emergent properties as they scale, we should assume the same will happen as we…

      sitkackView on HNParent context

  40. 33TB of text data for a 1T-parameter model 1 comment

    • "That’s basically the claim of the “Blurry JPEG of the Web” article. I agree that to a large degree that’s probably what’s going on here." I strongly disagree with that view, and encourage reading the "Sparks of Artificial General Intelligence" paper[1] and watching the…

      pmoriartyView on HNParent context

Showing 40 of 66 threads. Download the full data to see all.

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.