通往 AGI 的路径：架构、计算与超越大语言模型

Pathways to AGI: Architecture, Compute, and Beyond LLMs

本期速览 · Overview

探讨 Transformer 和大语言模型能否通向 AGI，架构与计算的作用，以及超越当前模型所需的巧妙想法。

Exploring whether transformers and LLMs can lead to AGI, the role of architecture and compute, and the need for clever ideas beyond current models.

AGI 可能通过扩展计算、数据和目标实现，而不仅仅是架构。
AGI may be achieved by scaling compute, data, and objectives, not just architecture.
视频预训练在理解世界上比文本有更高的扩展上限。
Video pre-training offers higher scaling ceiling than text for world understanding.
强化学习对于超越人类数据的策略优化至关重要。
Reinforcement learning is crucial for refining strategies beyond human data.

当前前沿模型并非死胡同；我们可以在不从头开始的情况下改进它们。
Current frontier models are not a dead end; we can improve them without starting from scratch.
对于实现 AGI，架构的重要性不如算力、数据和目标函数。
Architecture matters less than compute, data, and objectives for achieving AGI.
长上下文理解和长程推理是关键未解决问题。
Long context understanding and long horizon reasoning are key unsolved problems.
视频预训练在理解世界方面比文本有更高的扩展上限。
Video pre-training offers higher scaling ceiling than text for world understanding.
强化学习对于超越人类数据的策略优化至关重要。
Reinforcement learning is crucial for refining strategies beyond human data.
未来 AI 系统将把多种模态和能力整合到统一模型中。
Future AI systems will integrate multiple modalities and capabilities into unified models.

几乎任何架构（包括 RNN）都能通向 AGI；Transformer 并非唯一。
Almost any architecture, including RNNs, could lead to AGI; transformers are not unique.
我们已经超越了 LLM；当前系统是多模态的，而不仅仅是语言模型。
We are already past LLMs; current systems are multimodal, not just language models.
神经科学可能不是最佳指南；工程和扩展现在更实用。
Neuroscience may not be the best guide; engineering and scaling are more practical now.
上下文学习不够；真正的持续学习需要在线优化。
In-context learning is insufficient; true continual learning requires online optimization.
由于信息密度，视频模型的扩展上限远高于文本模型。
Scaling video models has a much higher ceiling than text models due to information density.
世界模型可以并行模拟机器人训练，减少物理数据收集需求。
World models can simulate robot training in parallel, reducing need for physical data collection.