34 - AI Evaluations with Beth Barnes скачать в хорошем качестве

34 - AI Evaluations with Beth Barnes

How can we figure out if AIs are capable enough to pose a threat to humans? When should we make a big effort to mitigate risks of catastrophic AI misbehaviour? In this episode, I chat with Beth Barnes, founder of and head of research at METR, about these questions and more. Patreon:   / axrpodcast   Ko-fi: https://ko-fi.com/axrpodcast The transcript: https://axrp.net/episode/2024/07/28/e... Topics we discuss, and timestamps: 0:00:37 - What is METR? 0:02:44 - What is an "eval"? 0:14:42 - How good are evals? 0:37:25 - Are models showing their full capabilities? 0:53:25 - Evaluating alignment 1:01:38 - Existential safety methodology 1:12:13 - Threat models and capability buffers 1:38:25 - METR's policy work 1:48:19 - METR's relationships with labs 2:04:12 - Related research 2:10:02 - Roles at METR, and following METR's work Links for METR: METR: https://metr.org METR Task Development Guide - Bounty: https://taskdev.metr.org/bounty/ METR - Hiring: https://metr.org/hiring Autonomy evaluation resources: https://metr.org/blog/2024-03-13-auto... Other links: Update on ARC's recent eval efforts (contains GPT-4 taskrabbit captcha story) https://metr.org/blog/2023-03-18-upda... Password-locked models: a stress case for capabilities evaluation: https://www.alignmentforum.org/posts/... Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training: https://arxiv.org/abs/2401.05566 Untrusted smart models and trusted dumb models: https://www.alignmentforum.org/posts/... AI companies aren't really using external evaluators: https://www.lesswrong.com/posts/Wjtnv... Nobody Knows How to Safety-Test AI (Time): https://time.com/6958868/artificial-i... ChatGPT can talk, but OpenAI employees sure can’t: https://www.vox.com/future-perfect/20... Leaked OpenAI documents reveal aggressive tactics toward former employees: https://www.vox.com/future-perfect/35... Beth on her non-disparagement agreement with OpenAI: https://www.lesswrong.com/posts/yRWv5... Sam Altman's statement on OpenAI equity: https://x.com/sama/status/17919368575...