Oriol Vinyals 讨论了 AI 智能体从专门的游戏系统到通用多模态模型的进化，以及强化学习和模仿学习的作用。
Oriol Vinyals discusses the evolution of AI agents from specialized game-playing systems to general-purpose multimodal models, and the role of reinforcement learning and imitation learning.
要点 · TL;DR
从 AlphaGo 到 Gemini，两阶段训练（模仿+强化学习）是 AI 智能体的核心。 Two-phase training (imitation + RL) is the core of AI agents from AlphaGo to Gemini.
工具使用和推理时计算使智能体超越静态模型，实现自主行为。 Tool use and inference-time compute enable autonomous agentic behavior beyond static models.
核心观点 · Key points
两阶段训练（预训练模仿 + 后训练强化学习）是 AlphaGo、AlphaStar 和现代大语言模型共有的核心。 Two-phase training (pre-training imitation + post-training RL) is the constant core across AlphaGo, AlphaStar, and modern LLMs.
预训练的 Scaling（规模扩张）收益递减；进一步突破需要算法创新，而不仅仅是更大的模型。 Scaling pre-training shows diminishing returns; further progress needs algorithmic innovation, not just bigger models.
为模型提供搜索、代码执行和浏览器访问等工具，使其能够超越静态权重，实现自主的智能体式行为。 Giving models tools like search, code execution, and browser access enables autonomous, agentic behavior beyond static weights.
推理时算力（思考）是一个新的扩展维度，可以提升推理能力并突破预训练的极限。 Inference-time compute (thinking) is a new scaling axis that can improve reasoning and break pre-training limits.
长上下文窗口提供了工作记忆以进行详细分析，但信息过多也可能让模型困惑。 Long context windows enable working memory for detailed analysis, but too much information can confuse models.
反共识 · Contrarian takes
缩放定律并未失效，只是需要指数级更多的算力才能获得相同的改进，这给创新带来了压力。 Scaling laws are not dead; they just require exponentially more compute for the same improvement, creating pressure for innovation.
语言领域的后训练强化学习缺乏像游戏那样清晰的奖励信号，因此很难避免奖励欺骗。 Post-training RL for language lacks clean reward signals, unlike games, making it hard to avoid reward hacking.
模型生成的合成数据可能有帮助，但简单重复使用模型输出会产生有害的反馈循环。 Synthetic data from models can help, but naively reusing model output creates harmful feedback loops.
无文本的视频数据可能包含丰富的世界知识，但当前模型无法在没有语言监督的情况下提取它。 Video data without text may contain rich world knowledge, but current models cannot extract it without language supervision.
超级智能可能首先出现在科学等狭窄领域，而非仅通过通用模型实现。 Superintelligence may first appear in narrow domains like science, not through general models alone.
本期章节 · Chapters（共 16）
引言与剧烈研究Introduction and Drastic Research
网络快照与冻结权重Snapshot of the network and frozen weights
规模扩展的收益递减Diminishing Returns of Scaling
超越规模扩展的改进杠杆Levers for Improvement Beyond Scaling
数据限制与合成数据Data Limits and Synthetic Data
视频数据：未开发的知识源Video data as untapped knowledge source
两个阶段：模仿与强化学习Two phases: imitation and reinforcement learning
构建数字身体：推理与工具使用Building the digital body: reasoning and tool use
安全性与工具使用Safety and Tool Use
规划与个性化Planning and Personalization
记忆与上下文Memory and Context
AI 记忆类型：情景、工作与长上下文Memory types in AI: episodic, working, and long context
代理能力与浏览器自动化Agentic capabilities and browser automation