Is Power-Seeking AI an Existential Risk? · HackerPapers discussions

Is Power-Seeking AI an Existential Risk? — mentioned by simiones

Wed, 17 Jan 2024 13:50:20 GMT

> P1: If intelligent system A cannot give a detailed account of how it would be bested by a more intelligent system B, then A will not be bested by B. P2: Humans (so far) cannot give a detailed account of how a more intelligent AI system would best them. C: So, humans will not be bested by a more intelligent AI system. I don't think anyone seriously believes this. It's very very clear to all humans that have ever played a game of any kind that they can be defeated in unexpected ways. I don't even think that anyone believes the claim "it's impossible for AGI to pose an existential risk to…

Paper and discussion history. A mention is not an endorsement.

Is Power-Seeking AI an Existential Risk? — mentioned by foo3a9c4

Tue, 16 Jan 2024 19:07:36 GMT

> The first links are spiffy little metaphors, but apply just as much at "God could smite all of humanity, even if you don't understand how". They're not making any argument, just assumptions. In particular, they accidentally show how an AI can be superhumanly capable at certain tasks (chess), but be easily defeated by humans at others (anything else, in the case of Stockfish). As I understand it, Yud is actually providing a counterexample to a premise that other people are using to argue that humans will probably not be disempowered by AI systems. The relevant argument looks like this: P1…

Paper and discussion history. A mention is not an endorsement.

Is Power-Seeking AI an Existential Risk? — mentioned by lhl

Fri, 21 Apr 2023 20:30:26 GMT

It seems to me, from a copyright perspective, all commercial use of generative AI depends on whether the output is transformative fair use (vs derived work). While the courts will have its say, ultimately whether new rules are carved out or not is going to be again (as all copyright law is) based on commercial interests - I have the feeling that the potential productivity upside across all industries (and in terms of national interests) is going to be big enough that it'll work itself out largely in the favor of generative AI. That being said, IMO, that's completely separate from the safety…

Paper and discussion history. A mention is not an endorsement.

Is Power-Seeking AI an Existential Risk? — mentioned by lhl

Fri, 07 Apr 2023 18:10:00 GMT

The dangers of AGI I don't think are very related with consciousness or sentience (maybe the saddest outcome if it killed us all and weren't at all). I'm still catching up on my reading, but the existential risk of AGI as I understand it is chiefly that we don't know how to align AI (and keep it aligned) to be beneficial/benevolent for humanity, but the risk is contingent also on a few other things things - that we develop AI that agentic and able to develop and pursue it's own goals and sub-goals, and that it has power-seeking or game-theory strategic awareness/planning. The extra dangerous…

Paper and discussion history. A mention is not an endorsement.

Is Power-Seeking AI an Existential Risk? — HN submission

Tue, 11 Oct 2022 10:54:00 GMT

Is Power-Seeking AI an Existential Risk?

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.