DeepMind 研究科学家 Yan Leike 讨论了他如何使机器学习稳健且有益的工作，以及如何为人工智能安全领域的职业生涯做准备。
Yan Leike, a research scientist at DeepMind, discusses his work on making machine learning robust and beneficial, and how to prepare for a career in AI safety.
要点 · TL;DR
AI 安全研究至关重要且尚未充分探索，人才缺口大于资金缺口。 AI safety research is crucial and under-explored, with a larger talent gap than funding gap.
机器学习博士学位或同等经验是成为 AI 安全研究员的最佳途径。 A PhD in ML or equivalent is the best path to become an AI safety researcher.
实证安全研究机会众多，批判性思维和适应不确定性是关键技能。 Empirical safety research offers many opportunities; critical thinking and comfort with uncertainty are key.
核心观点 · Key points
随着 AI 能力提升，AI 安全至关重要，重点在于对齐和鲁棒性。 AI safety is crucial as AI capabilities improve, focusing on alignment and robustness.
AI 安全的实证研究目前尚未充分探索，提供了许多机会。 Empirical research in AI safety is currently under-explored and offers many opportunities.
机器学习博士学位或同等经验是成为安全研究人员的最佳途径。 A PhD in machine learning or equivalent experience is the best path to become a safety researcher.
批判性思维和适应不确定性是 AI 安全研究的关键技能。 Critical thinking and comfort with uncertainty are key skills for AI safety research.
技术性 AI 安全的人才缺口比资金缺口更大。 The talent gap in technical AI safety is larger than the funding gap.
DeepMind 和 OpenAI 等组织之间的合作对进展很重要。 Collaboration between organizations like DeepMind and OpenAI is important for progress.
反共识 · Contrarian takes
你不需要成为数学天才也能为 AI 安全研究做出贡献。 You don't need to be a math genius to contribute to AI safety research.
过分担心从事能力研究是被过度强调的；应专注于技能积累。 Worrying too much about working on capability research is overemphasized; focus on skill building.
对抗样本即使无法访问原始网络也能跨模型迁移。 Adversarial examples transfer across models even without access to the original network.
如果人类反馈停止，奖励预测器在分布偏移下可能失效。 Reward predictors can fail under distributional shift if human feedback stops.
深度强化学习算法在不同随机种子下以不稳定著称。 Deep reinforcement learning algorithms are notoriously unstable across random seeds.
为了 AI 安全，优先选择更难的数学课程比容易的应用课程更好。 It's better to prioritize harder math courses over easier applied ones for AI safety.
本期章节 · Chapters（共 26）
引言与嘉宾背景Introduction and guest background
Jan 在 DeepMind 的工作与 AI 安全重要性Jan's work at DeepMind and importance of AI safety
AI 安全常见误解Common misconceptions about AI safety
基于人类偏好的深度强化学习研究Specific research: deep reinforcement learning from human preferences
后空翻面条与人类反馈Backflipping Noodle and Human Feedback
扩展与安全Scaling and Safety
对抗样本与黑盒攻击Adversarial Examples and Black-Box Attacks
自动驾驶与防御Self-Driving Cars and Defense
深度强化学习的鲁棒性Robustness in Deep Reinforcement Learning
DeepMind 项目与日常工作DeepMind Projects and Daily Work
DeepMind 日常工作与协作Daily work and collaboration at DeepMind
团队规模与安全协作Team size and collaboration on safety
通往 DeepMind 的职业路径Career path to DeepMind
关键职业决策：从理论到实证Key career decisions and shift from theory to empirics
DeepMind 申请流程Application process at DeepMind
与 Dario Amodei 在研究契合度上的分歧Disagreement with Dario Amodei on research fit
AI 安全研究职业建议Career Advice for AI Safety Research
机器学习博士之外的替代路径Alternatives to a Machine Learning PhD
何时转向 AI 安全研究When to Transition to AI Safety Research
进入 DeepMind 前的中间步骤Intermediate Steps Before DeepMind
AI 安全研究之外的职业选择Career Options Outside AI Safety Research
其他 AI 安全技术方法Other Technical Approaches to AI Safety
挑战与责任Challenges and Responsibilities
关于所需技能的误解Misconceptions About Needed Skills
攻读机器学习博士的建议Advice for Pursuing a PhD in Machine Learning