Evaluating Language-Model Agents on Realistic Autonomous Tasks — mentioned by agucova

Wed, 04 Sep 2024 02:56:49 GMT

> My issue with AI safety is that it's an overloaded term. It could mean anything from an llm giving you instructions on how to make an atomic bomb to writing spicy jokes if you prompt it to do so. it's not clear which safety these regulatory agencies would be solving for. I think if you look at the background of the people leading evaluations at the US AISI [1], as well as the existing work on evaluations by the UK AISI [2] and METR [3], you will notice that it's much more the former than the latter. [1]: https://www.nist.gov/people/paul-christiano [2]…

Paper and discussion history. A mention is not an endorsement.

Evaluating Language-Model Agents on Realistic Autonomous Tasks · HackerPapers discussions

Evaluating Language-Model Agents on Realistic Autonomous Tasks — mentioned by agucova