Rob Wiblin 采访 Paul Christiano，讨论 AI 对齐、撤资有害公司的有效性，以及肌酸补充剂和留给未来文明的信息等推测性想法。
Rob Wiblin interviews Paul Christiano about AI alignment, the effectiveness of divesting from harmful companies, and speculative ideas like creatine supplements and messages to future civilizations.
要点 · TL;DR
AI 对齐对文明的长期未来至关重要。 AI alignment is critical for civilization's long-term future.
算力扩展推动 AI 进步，但其未来影响不确定。 Compute scaling drives AI progress, but its future impact is uncertain.
为未来文明留下信息可能具有成本效益，能降低存在风险。 Leaving messages for future civilizations could be cost-effective for reducing existential risk.
核心观点 · Key points
AI 对齐是一个关键问题，可能决定文明的长期未来。 AI alignment is a critical problem that could determine the long-term future of civilization.
AI 对齐的进展需要理论工作和机器学习系统的实际实验相结合。 Progress in AI alignment requires both theoretical work and practical experiments with ML systems.
能力研究和对齐研究之间存在有意义的区别，影响长期发展轨迹。 There is a meaningful distinction between capabilities research and alignment research that affects the long-term trajectory.
对抗性测试、验证和可解释性对短期可靠性和长期安全都很重要。 Adversarial testing, verification, and interpretability are important for both short-term reliability and long-term safety.
算力扩展是 AI 进步的主要驱动力，但其未来影响不确定。 Compute scaling has been a major driver of AI progress, but its future impact is uncertain.
反共识 · Contrarian takes
在人类灭绝后为未来智能生命留下信息，可能是降低存在风险的高性价比方式。 Leaving messages for future intelligent life after human extinction could be cost-effective for reducing existential risk.
肌酸补充可能提升素食者的智商，但这一效应研究不足且被忽视。 Creatine supplementation might boost IQ in vegetarians, but this effect is under-researched and neglected.
房间内高二氧化碳水平可能对认知产生巨大影响，但未得到足够重视。 High CO2 levels in rooms may have absurdly large cognitive effects, yet this is not taken seriously enough.
即使没有对齐的技术解决方案，克制和协调仍可能带来良好结果。 Even without a technical solution to alignment, restraint and coordination could still lead to good outcomes.
安全与能力研究的界限模糊，但关注差异化进展对长期影响至关重要。 The distinction between safety and capabilities research is blurry, but focusing on differential progress is key for long-term impact.
本期章节 · Chapters（共 50）
引言与嘉宾Introduction and Guest
引言与当前工作Introduction and current work
过去一年观点变化Shifts in opinions over the past year
辩论与放大方法进展Progress on debate and amplification methods
低编码研究领域优势Advantages of a less codified research field
学术领域问题与方法错配Mismatch between problems and methods in academic fields
系统故障与约束担忧Concerns about system failures and restraint
AI 对齐思考更新Updates on AI alignment thinking
对齐工作资源分配Resource allocation for alignment work
给未来文明留言Leaving messages for future civilizations
蜥蜴人思想实验Lizard People Thought Experiment
给未来文明留言Leaving messages for future civilizations
与未来文明交流Communicating with future civilizations
从语境解码语言Decoding language from context
项目成本与可行性Cost and feasibility of the project
降低存在风险的被忽视想法Neglected ideas for reducing existential risk
二氧化碳与认知表现Carbon dioxide and cognitive performance
AI 进展与算力扩展观点Perspectives on AI Progress and Compute Scaling
仅靠算力扩展实现 AGI 的可能性Possibility of AGI via Compute Scaling Alone
莫拉维克悖论与算力重要性Moravec's Paradox and Compute Importance
人类与机器任务难度Difficulty of tasks for humans vs machines
硬件与算法进步Hardware vs algorithmic progress
讨论 Pushmeet 观点Discussion of Pushmeet's views
能力与对齐差距The gap between capability and alignment
精选研究问题Being picky about problems to work on
加速 AI 进展的影响The effect of accelerating AI progress
职业建议：通用 AI 岗 vs 安全岗Career advice: taking general AI jobs vs safety-specific roles
长期主义者的职业路径建议Advice on career paths for long-termists
将工作委托给 AI 与灾难性失败Delegating work to AI systems and catastrophic failure
对 AI 安全的乐观态度Optimism about AI safety
单边主义错误与 AI 政策谨慎Unilateralist mistakes and caution in AI policy
从有害公司撤资Divesting from harmful companies
撤资怀疑与成本效益分析Divestment skepticism and cost-benefit analysis
撤资对股价与多样化的影响Divestment Impact on Stock Prices and Diversification