Google 首席科学家、Gemini 项目联合负责人 Jeff Dean 探讨 AI 的演进、规模与算法的力量，以及他早期在神经网络方面的工作。
Jeff Dean, Google's chief scientist and co-lead of the Gemini project, discusses the evolution of AI, the power of scale and algorithms, and his early work on neural networks.
要点 · TL;DR
扩展计算、数据和模型规模推动了十多年的机器学习进步。 Scaling compute, data, and model size has driven ML progress for over a decade.
Transformer 通过并行化和全局注意力取代了 LSTM。 Transformers replaced LSTMs by enabling parallelization and attention over all tokens.
TPU 等专用硬件对大规模高效训练和推理至关重要。 Specialized hardware like TPUs is critical for efficient training and inference at scale.
核心观点 · Key points
算力、数据和模型规模的扩张持续推动了机器学习过去 13-14 年的进步。 Scaling compute, data, and model size has driven continuous ML improvements for 13-14 years.
算法和架构改进与规模扩张相乘，带来巨大收益。 Algorithmic and architectural improvements multiply with scale, yielding massive gains.
Transformer 通过并行化和对所有词元的注意力机制取代了 LSTM。 Transformers replaced LSTMs by enabling parallelization and attention over all tokens.
文本上的自监督学习通过下一个词预测提供了无限的训练数据。 Self-supervised learning on text provides unlimited training data via next-token prediction.
基于人类或机器反馈的强化学习引导模型行为和能力。 Reinforcement learning from human or machine feedback steers model behavior and capabilities.
TPU 等专用硬件对于大规模高效训练和推理至关重要。 Specialized hardware like TPUs is critical for efficient training and inference at scale.
反共识 · Contrarian takes
AI 生成的逼真媒体导致虚假信息是近期担忧，而非超级智能。 Misinformation from realistic AI-generated media is a near-term worry, not superintelligence.
具有良好趋势的小规模实验比增量式的最优结果更重要。 Small-scale experiments with promising trends matter more than incremental state-of-the-art results.
异步训练虽然在理论上不正确，但在实践中效果良好。 Asynchronous training, though theoretically wrong, works well in practice.
硬件错误导致的静默数据损坏是大规模训练中的主要挑战。 Silent data corruption from hardware errors is a major challenge at large scale.
推测解码在不重新训练或改变输出分布的情况下加速推理。 Speculative decoding speeds inference without retraining or changing output distribution.
未来 AI 将涉及人类协调智能体团队，而非仅单次聊天交互。 Future AI will involve humans coordinating teams of agents, not just single chatbot interactions.
本期章节 · Chapters（共 38）
杰夫·迪恩简介Introduction of Jeff Dean
杰夫开场致辞Jeff's Opening Remarks
AI 进展观察Observations on AI Progress
神经网络与梯度下降Neural Networks and Gradient Descent
ML 进展速览Whirlwind Tour of ML Advances
谷歌大规模神经网络起源Origin of Large-Scale Neural Networks at Google
专用硬件与 TPUv1 动机Motivation for specialized hardware and TPUv1
训练演进与 TPU 代际Evolution to training and TPU generations
开源工具与 TransformerOpen source tools and the Transformer
Transformer 效率与语言建模Transformer Efficiency and Language Modeling
Transformer 在计算机视觉Transformers in Computer Vision
稀疏模型与专家混合Sparse Models and Mixture of Experts
Pathways：分布式 ML 抽象Pathways: Distributed ML Abstractions
大规模硬件可靠性Hardware Reliability at Scale
推理时思考与思维链Inference-Time Thinking and Chain of Thought
蒸馏高效模型Distillation for Efficient Models
蒸馏与强化学习Distillation and Reinforcement Learning
推测解码Speculative Decoding
创新总结Summary of Innovations
Gemini 起源Origin of Gemini
多模态从开始Multimodal from the start
技术基础Technical foundations
大上下文窗口Large context windows
模型规模：Pro 与 FlashModel scales: Pro and Flash
数学问题求解与 IMOMathematical problem solving and IMO
Gemini 3 发布Gemini 3 release
ELO 模型评估Model Evaluation via ELO
搜索与 AI 模式演示Search and AI Mode Demo
图像视频生成模型Generative Models for Images and Video
组织 Gemini 大型项目Organizing a Large Effort like Gemini
团队分布与异步工作Team distribution and async work
未来方向：人机协作与长上下文Future directions: human-AI collaboration and long context
塑造 AI 论文：跨领域影响Shaping AI paper: impacts across domains