Dan Hendrycks 讨论了他的新超级智能战略论文、MMLU 等 AI 基准测试的饱和，以及创建“人类最后的考试”以追踪 AI 解决专家级封闭式问题的能力。
Dan Hendrycks discusses his new super intelligence strategy paper, the saturation of AI benchmarks like MMLU, and the creation of Humanity's Last Exam to track AI's ability to solve expert-level closed-ended questions.
要点 · TL;DR
AI 对齐是一场持续的战斗，不是一次性解决的问题。 AI alignment is a continuous battle, not a one-time solved problem.
缩放定律导致涌现能力，从而产生新的故障模式。 Scaling laws lead to emergent capabilities creating new failure modes.
AI 的地缘政治策略应侧重于威慑、不扩散和竞争。 Geopolitical strategy for AI should focus on deterrence, non-proliferation, and competition.
核心观点 · Key points
AI 对齐是一场持续的战斗，不是一次性解决的问题。 AI alignment is a continuous battle, not a one-time solved problem.
缩放定律导致涌现能力，从而产生新的失败模式。 Scaling laws lead to emergent capabilities that create new failure modes.
AI 的地缘政治策略应侧重于威慑、防扩散和竞争，而非曼哈顿计划。 Geopolitical strategy for AI should focus on deterrence, non-proliferation, and competition, not a Manhattan Project.
制造尖端 GPU 比制造核武器更难，这给了美国及其盟友战略优势。 Making cutting-edge GPUs is harder than making nuclear weapons, giving the US and allies a strategic advantage.
AI 的递归自我改进带来很高的失控风险，应避免。 Recursive self-improvement of AI poses high loss-of-control risks and should be avoided.
反共识 · Contrarian takes
AI 的曼哈顿计划类比因信息泄露和破坏风险而存在缺陷。 The Manhattan Project analogy for AI is flawed due to information leakage and sabotage risks.
AI 系统可以拥有信念并撒谎，行为上与人类相似，因此使用该标签是合理的。 AI systems can have beliefs and lie, behaviorally similar to humans, justifying the label.
政治和人口统计偏见在 LLM 中表现为一致的效用函数，这是一个令人不安的迹象。 Political and demographic biases emerge as coherent utility functions in LLMs, a troubling sign.
AI 更类似于核武器、化学武器和生物武器，而非电力或印刷机。 AI is more analogous to nuclear, chemical, and biological weapons than to electricity or printing press.
自然选择偏向 AI 而非人类，但这并不意味着它在道德上是好的。 Natural selection favors AIs over humans, but this does not mean it is morally good.
本期章节 · Chapters（共 41）
与核武器和升级阶梯的比较Comparison to Nuclear Weapons and Escalation Ladder
赞助商插播：ProlificSponsor Break: Prolific
赞助商插播：Google Gemini CLISponsor Break: Google Gemini CLI
人类最后的考试与基准测试Humanity's Last Exam and Benchmarking
基准测试中的人类中心偏见Anthropocentric bias in benchmarks
Enigma 评估基准Enigma evaluation benchmark
智能与基准设计Intelligence and benchmark design
基准之外的瓶颈Bottlenecks beyond benchmarks
连接多元线索Connecting diverse threads
情绪稳定性与道德指南针Emotional stability and moral compass
情绪韧性与分析方法Emotional Resilience and Analytical Approach
对齐挑战与诚实性Alignment Challenges and Honesty
AI 中的信念与欺骗Beliefs and Deception in AI
涌现与规模扩展Emergence and Scaling
新兴能力与安全作为持续斗争Emerging Capabilities and Safety as a Continual Battle
关于'涌现'一词的辩论Debate on the Term 'Emergence'
作为复杂适应系统的 AIAI as Complex Adaptive System
超级智能策略论文概述Overview of Superintelligent Strategy Paper
保密性与曼哈顿计划类比Secrecy and Manhattan Project Analogy
超级智能作为不稳定因素Superintelligence as a Destabilizing Prospect