Rich Sutton 讨论阿尔伯塔 AI 研究计划、开放思维研究所的创立以及神经网络的三个历史浪潮。
Rich Sutton discusses the Alberta Plan for AI research, the founding of the Open Mind Research Institute, and the three historical waves of neural networks.
要点 · TL;DR
规模化很容易，找到能规模化的算法才是真正的挑战。 Scaling is easy; finding algorithms that scale is the real challenge.
反向传播调整权重，但不解决表征学习。 Backprop adjusts weights but does not solve representation learning.
强化学习应预测多步未来，而非仅下一观测。 Reinforcement learning should predict multiple steps ahead, not just the next observation.
核心观点 · Key points
Scaling（规模扩张）很容易；找到能扩展的方法才是难点。 Scaling is easy; finding methods that scale is the hard part.
反向传播并未解决表征学习问题；它只是调整权重以解决问题。 Backprop does not solve representation learning; it only adjusts weights to solve the problem.
强化学习应预测多个时间步之后的结果，而不仅仅是下一个观测。 Reinforcement learning should predict multiple steps ahead, not just the next observation.
好的特征能很好地泛化；找到它们是一个尚未解决的元学习问题。 Good features generalize well; finding them is an unsolved meta-learning problem.
想法比规模更重要；简单的实验能揭示基本原理。 Ideas matter more than scale; simple experiments reveal fundamental principles.
从第一性原理思考；你自己独特的想法是你最大的潜在贡献。 Think from first principles; your own unique ideas are your greatest potential contributions.
反共识 · Contrarian takes
Scaling（规模扩张）很容易，并不难；真正的挑战是找到能扩展的算法。 Scaling is easy, not hard; the real challenge is finding algorithms that scale.
反向传播并未解决表征学习问题；这是一个常见的误解。 Backprop does not solve representation learning; it is a common misconception.
预测下一帧视频并不是构建世界模型的正确方法。 Predicting the next video frame is not the right way to build a world model.
无需回放缓冲区的在线学习是可行的，且更符合原理。 Online learning without replay buffers is feasible and more principled.
希望和恐惧等情绪自然源于任何智能体的奖励预测。 Emotions like hope and fear naturally arise from reward prediction in any agent.
智能和意识取决于观察者，而非绝对存在。 Intelligence and consciousness are in the eye of the beholder, not absolute.
本期章节 · Chapters（共 23）
引言与开放思维研究所Introduction and OpenMind Research Institute
阿尔伯塔观点与神经网络浪潮The Alberta Viewpoint and Neural Network Waves
规模扩展 vs 新算法On Scaling vs. New Algorithms
反向传播与表征学习Backprop and Representation Learning
预测世界：时序差分学习 vs 监督学习Predicting the world: TD learning vs supervised learning
预测与控制Prediction and Control
当前热点与过时想法Current Excitements and Out-of-Fashion Ideas
寻找好特征与元学习Finding Good Features and Metalearning
记忆与回放缓冲区Memory and Replay Buffers
想法生成过程Idea Generation Process
时间抽象与选项Abstraction in Time and Options
起点与方法的差异Differences in starting points and approaches
成功观与 AI 作为下一进化Views on success and AI as next evolution
AI 系统中的情感Emotions in AI systems
意识与痛苦On Consciousness and Pain
AI 安全的锁车类比The locked car analogy for AI safety
小规模实验的价值The value of small-scale experiments
扩展时序差分学习与想法的重要性Scaling TD learning and the importance of ideas
给年轻研究者的建议Advice for young researchers
鼓励经典科学思维Encouraging classical scientific thinking
无大规模计算的研究Research without massive compute
跨学科摩擦与强化学习Interdisciplinary friction and RL
最终建议：从第一性原理思考Final advice: think from first principles