Risks from Learned Optimization in Advanced Machine Learning Systems · HackerPapers discussions

Risks from Learned Optimization in Advanced Machine Learning Systems — mentioned by 0xDEAFBEAD

Mon, 20 Nov 2023 00:02:41 GMT

Read the mesa-optimization paper I linked elsewhere in this thread: https://arxiv.org/pdf/1906.01820.pdf Eliezer's point is that if AI researchers aren't looking for anomalous behavior that could indicate a potential danger, they won't find it.

Paper and discussion history. A mention is not an endorsement.

Risks from Learned Optimization in Advanced Machine Learning Systems — mentioned by 0xDEAFBEAD

Sun, 19 Nov 2023 18:57:58 GMT

I see how some of his tweets could come across as crank-ish if you don't have a background in AI alignment. AI alignment is sort of like computer security in the sense that you're trying to guard against the unknown. If there was a way to push a button which told you the biggest security flaw in the software you're writing, then the task of writing secure software would be far easier. But instead we have to assume the existence of bugs, and apply principles like defense-in-depth and least privilege to mitigate whatever exploits may exist. In the same way, much of AI alignment consists of…

Paper and discussion history. A mention is not an endorsement.

Risks from Learned Optimization in Advanced Machine Learning Systems — HN submission

Wed, 13 Apr 2022 03:57:42 GMT

Risks from Learned Optimization in Advanced Machine Learning Systems

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.