演讲者提出了一种通用 AI 架构的愿景，该架构从运行时经验中学习，不依赖领域知识，强调开放式抽象并摒弃设计时承诺。
The speaker presents a vision for a general AI architecture that learns from runtime experience without domain knowledge, emphasizing open-ended abstraction and deprecating design-time commitments.
要点 · TL;DR
AGI 需要一种无领域知识的通用架构，从运行时经验中学习。 AGI needs a general architecture with no domain knowledge, learning from runtime experience.
开放式抽象是 AI 代理发展无限概念和行为的关键。 Open-ended abstraction is key for AI agents to develop unlimited concepts and behaviors.
可靠的持续深度学习是 Oak 的前提，但尚未解决。 Reliable continual deep learning is a prerequisite for Oak but not yet solved.
核心观点 · Key points
AGI 需要一个不包含领域知识、从运行时经验中学习的通用架构。 AGI requires a general architecture with no domain knowledge, learning from runtime experience.
开放式抽象对于 AI 智能体发展无限的概念和行为至关重要。 Open-ended abstraction is crucial for AI agents to develop unlimited concepts and behaviors.
奖励假说：所有目标都可以表述为最大化累积标量奖励。 Reward hypothesis: all goals can be framed as maximizing cumulative scalar reward.
Oak 架构使用选项和知识来实现开放式抽象。 The Oak architecture uses options and knowledge to enable open-ended abstraction.
可靠的持续深度学习是 Oak 的前提条件，但尚未解决。 Reliable continual deep learning is a prerequisite for Oak but not yet solved.
反共识 · Contrarian takes
对于通用智能，设计时领域知识应该被弃用。 Design time domain knowledge should be deprecated for general intelligence.
智能体应仅从运行时经验学习，而非特殊训练数据。 Agents should learn only from runtime experience, not special training data.
在大世界中，所有价值函数和策略都必须是近似的，永远不是最优的。 In a big world, all value functions and policies must be approximate, never optimal.
子问题应该是尊重奖励的特征达成，而非预定义任务。 Subproblems should be reward-respecting feature attainment, not predefined tasks.
持续学习需要随机搜索组件，而不仅仅是梯度下降。 Continual learning requires a random search component, not just gradient descent.
本期章节 · Chapters（共 27）
0. 引言与愿景Introduction and Vision
1. 复杂世界的启示Implications of a Complex World
2. 通用 AI 问题与强化学习The General AI Problem and Reinforcement Learning
3. 奖励假说The Reward Hypothesis
4. 智能体通用模型The Common Model of the Intelligent Agent
5. 通用模型与缺失抽象Common Model and Missing Abstraction
6. Oak 架构：添加辅助子问题Oak Architecture: Adding Auxiliary Subproblems
7. 转移模型与选项Transition Model and Options
8. Oak 架构八步Eight Steps of Oak Architecture
9. 可塑性丧失与持续学习Loss of plasticity and continual learning
10. 从现有状态特征生成新特征Generating new features from existing state features
11. 特征排序与创建子问题Ranking features and creating subproblems
12. 子问题作为特征获取与游戏Subproblems as feature attainment and play
13. 智能体必须自创子问题Agent must create its own subproblems
14. OAK 中的子问题Subproblems in OAK
15. OAK 中的规划Planning in OAK
16. 价值迭代与规划Value Iteration and Planning
17. 使用选项的抽象规划Abstract Planning with Options
18. 规划中的函数逼近Function Approximation in Planning
19. 统计与元数据Statistics and Metadata
20. Oak 中的开放式抽象Open-Ended Abstraction in Oak
21. 子问题选择问题Question on Subproblem Selection
22. 持续学习与反向传播问题Question on Continual Learning and Backpropagation
23. 通过重新初始化持续学习Continual Learning via Reinitialization
24. 工具使用的发现Discovery of Tool Use
25. 自我与世界的边界Boundary Between Self and World
26. 智能体-环境边界与奖励信号Agent-Environment Boundary and Reward Signal