AI Podcast › 罗欣·沙阿 › 本期AI 对齐可能并非灾难：Rohan Sha 的乐观观点 Why AI Alignment Might Not Be Catastrophic: Rohan Sha's Optimistic Take
罗欣·沙阿 Rohin Shah · 80,000 小时 · 2026-06-02 · 约 168 分钟 · 原视频 ↗
打开互动全文版（中英对照 + 朗读 + 问答）→
本期速览 · Overview 谷歌 DeepMind 的 AGI 对齐与安全负责人 Rohan Sha 解释为何他认为灾难性对齐失败不太可能发生，普通对齐技术很可能成功。
Rohan Sha, head of AGI alignment and safety at Google DeepMind, explains why he believes catastrophic misalignment is unlikely and that ordinary alignment techniques will likely succeed.
要点 · TL;DR DeepMind 的常规对齐技术很可能防止灾难性错位。 Prosaic alignment techniques at DeepMind likely prevent catastrophic misalignment. 由于 Transformer 的透明序列深度低，思维链监控仍将有效。 Chain-of-thought monitoring remains useful due to low opaque serial depth in transformers. 第三方审计和评分卡比广泛承诺更有利于 AI 安全。 Third-party audits and scorecards beat broad commitments for AI safety. 核心观点 · Key points 像 DeepMind 使用的常规对齐技术很可能防止灾难性对齐失败。 Prosaic alignment techniques like those used at DeepMind will likely prevent catastrophic misalignment. 由于 Transformer 的不透明串行深度较低，思维链监控将在数年内保持有用。 Chain-of-thought monitoring will remain useful for years due to low opaque serial depth in transformers. 第三方审计和像 AI Lab Watch 这样的详细评分卡比宽泛承诺更有效。 Third-party audits and detailed scorecards like AI Lab Watch are better than broad commitments. 部署前评估不那么重要，因为 AI 进展是连续的，安全缓冲有效。 Pre-deployment evaluations are less important because AI progress is continuous and safety buffers work. AI 安全研究应关注近期问题，而非试图解决所有未来问题。 AI safety research should focus on near-term problems rather than trying to solve all future issues. 反共识 · Contrarian takes 灾难性对齐失败是可能的，但并非默认情况；认为其不可避免的论点存在漏洞。 Catastrophic misalignment is plausible but not likely by default; arguments for inevitability have holes. 公司应避免严格的安全承诺，因为研究在变化；灵活性更好。 Companies should avoid firm safety commitments because research changes; flexibility is better. 当前模型的可控性并非未来对齐成功的强证据。 Current model steerability is not strong evidence for future alignment success. 智能爆炸可能不会很快发生；像超人编码器这样的工具只是新显微镜。 Intelligence explosion may not happen soon; tools like superhuman coders are just new microscopes. 短视优化配合非短视批准可以在不检测的情况下防止奖励黑客行为。 Myopic optimization with non-myopic approval can prevent reward hacking without detecting it. 通过 AI 加速治理被忽视；这可能比技术对齐更难。 Governance acceleration via AI is neglected; it may be harder than technical alignment. 本期章节 · Chapters（共 53） 引言与对齐乐观 Introduction and optimism about alignment 对齐风险与乐观看法 Views on misalignment risk and optimism 安全与对齐承诺 On commitments to safety and alignment 做出承诺的理由 Arguments for making commitments 谷歌的承诺方式 Google's approach to commitments 信任与外部监督 Trust and external oversight AI 实验室观察评分 AI Lab Watch Scorecard 硬否决与公司动态 On Hard Veto and Company Dynamics 模型发布与透明度 Transparency through model releases 能力突增与对齐研究 Sudden jumps in capability and alignment AI 治理应用 Preparation for future AGI and alignment research 减缓 AI 与证据 Governance use of AI 思维链监控 Slowing down AI vs. evidence 思维链监控为何有效 Chain of thought monitoring 连续思维链与不透明性 Why Chain-of-Thought Monitoring Works 不透明序列深度作为治理目标 Continuous chain of thought and opacity 回应 Maswitch 对前沿安全报告的批评 Opaque serial depth as governance target 评估细节与透明度 Response to Maswitch's criticisms of Frontier Safety Report 目标偏移担忧 Evaluation details and transparency 模型卡与问责目的 Goalpost shifting concern 模型卡与安全信号辩论 Purpose of model cards vs accountability 发表论文与模型卡 Debate on Model Cards and Safety Signaling 近视优化与非近视认可 Publishing papers vs model cards 近视优化与非近视认可 Myopic optimization with non-myopic approval GDM 的技术 AGI 安全方法 Myopic optimization with non-myopic approval 元策略：短中期规划 GDM's approach to technical AGI safety and security 论文坦诚与关键缓解措施 Meta-strategy: short-to-medium-term planning AI 遏制与安全基础设施 Paper's candidness and key mitigations 时间线与递归自我改进 AI Containment and Security Infrastructure 智能爆炸因素 Timelines and Recursive Self-Improvement 区分更好工具与人口增长 Factors for intelligence explosion 基准分数局限性 Distinguishing better tools from population increase 智能爆炸与突发性 Benchmark score limitations 时间线未更新的原因 Intelligence explosion and abruptness 强化学习与泛化性 Why timelines haven't updated 让研究对 AI 公司有用 Reinforcement learning and generalizability 越狱研究与公司立场 Making research useful for AI companies 学术界基线调校不佳 Jailbreak research and company stance 对 GDM 最有用的外部研究 Poorly tuned baselines in academia GDM 最难填补的职位 Most useful external research for GDM 谷歌 DeepMind 招聘 Hardest roles to fill at GDM 保持积极精神 Hiring at Google DeepMind 新书发布 Maintaining a positive spirit 经济模型预测工资增长 10 倍 Book announcement 转行管道工建议受质疑 Economic model predicts 10-fold wage increase 需要与 AI 输出互补的工作 Retraining as plumber advice questioned AI 最暴露的工作年薪 10-20 万 Need jobs complementary to AI outputs Hinton 放射学预测错误；创意工作预测也错 Jobs most exposed to AI earn $100k-$200k 诱导需求不对称：健康与税务合规 Hinton's radiology prediction wrong; creative work prediction also wrong 避免可预见错误的工具 Induced demand asymmetries: health vs tax compliance 具体结论与应避免领域 Tools to avoid predictable mistakes 书籍推广与获取 Concrete conclusions and fields to avoid Book promotion and availability Book promotion and availability 阅读全文双语转录 →