HackerPapers

arXiv:2309.01933cs.CY

Provably safe systems: the only path to controllable AGI

Max Tegmark, Steve Omohundro

Abstract

We describe a path to humanity safely thriving with powerful Artificial General Intelligences (AGIs) by building them to provably satisfy human-specified requirements. We argue that this will soon be technically feasible using advanced AI for formal verification and mechanistic interpretability. We further argue that it is the only path which guarantees safe controlled AGI. We end with a list of challenge problems whose solution would contribute to this positive outcome and invite readers to join in this work.

arXiv comments
17 pages
Journal reference
DOI
Subjects
cs.CYComputers and Society cs.AIArtificial Intelligence cs.LGMachine Learning

On Hacker News

Linked by 2 identifiable commenters in 2 comments across 2 threads, from Apr 2024 to Mar 2025. Found in 2 HN submissions from Sep 2023 to Sep 2023. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments2 threads

  1. Richard Sutton and Andrew Barto Win 2024 Turing Award 1 comment

    • >AI can eventually create AI's that are built up out of robust and provable logic That's the approach behind Max Tegmark and Steven Omohundro's "Provably Safe AGI": https://arxiv.org/abs/2309.01933 https://www.youtube.com/watch?v=YhMwkk6uOK8 However, there are issues. How do you…

      optimalsolverView on HNParent context

  2. Meta does everything OpenAI should be 1 comment

    • > When Sam Altman is calling for AI regulation Sure; that's almost certainly not being done in good faith. When numerous AI experts and luminaries who left their jobs in AI are advocating for AI regulation, that's much more likely to be being done in good faith. > What is this…

      JoshTriplettView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.