诺姆·沙泽尔：从谷歌早期到 AI 的未来

Noam Shazeer: From Google's Early Days to AI's Future

诺姆·沙泽尔 Noam Shazeer · Aarthi & Sriram · 2023-01-21 · 约 63 分钟 · 原视频 ↗

打开互动全文版（中英对照 + 朗读 + 问答）→

本期速览 · Overview

诺姆·沙泽尔，谷歌 21 年老将和 Transformer 论文合著者，讲述他从数学到 AI 的历程、谷歌早期岁月以及大语言模型内部的奥秘。

Noam Shazeer, a 21-year Google veteran and co-author of the Transformers paper, discusses his journey from math to AI, the early days at Google, and the mysteries inside large language models.

要点 · TL;DR

• 大语言模型通过预测下一个词训练，这是一个 AI 完备问题。
  LLMs are trained on next-token prediction, an AI-complete problem.
• Transformer 实现并行训练，扩展计算和数据能提升 AI 智能。
  Transformer enables parallel training, scaling compute and data improves AI.
• 对话是大语言模型的杀手级应用，实现自然交互。
  Dialogue is the killer app for LLMs, enabling natural interaction.

核心观点 · Key points

• LLM 通过下一个词预测训练，这是一个 AI 完备问题。
  LLMs are trained on next-token prediction, which is an AI-complete problem.
• Transformer 在训练时实现并行处理，大幅提升性能。
  Transformer enables parallel processing during training, massively improving performance.
• 通过更多算力和数据扩展模型，AI 变得更智能。
  Scaling models with more compute and data leads to smarter AI.
• 对话是 LLM 的杀手级应用，实现自然的人机交互。
  Dialogue is a killer app for LLMs, enabling natural human-computer interaction.
• LLM 的最佳应用往往出乎创造者的意料。
  The best applications of LLMs are often unforeseen by their creators.

反共识 · Contrarian takes

• 没人真正知道神经网络内部发生了什么；这就像炼金术。
  Nobody really knows what's happening inside neural networks; it's like alchemy.
• 大科技公司因品牌风险和安全顾虑而束手束脚，拖慢了 AI 产品发布。
  Big tech companies are hamstrung by brand risk and safety concerns, slowing AI product launches.
• 产品经理常常阻碍 LLM 开发，因为他们不理解技术的通用性。
  Product managers often hinder LLM development because they don't understand the technology's generality.
• 神经网络的成功很大程度上归功于游戏玩家推动 GPU 并行化，而非算法突破。
  The success of neural networks is largely due to gamers driving GPU parallelism, not algorithmic breakthroughs.
• 仅针对用户参与度优化模型可能不利；需要探索。
  Optimizing models solely for user engagement may not be beneficial; exploration is needed.

本期章节 · Chapters（共 23）

• 引言与背景 Introduction and Background
• 谷歌早期与面试故事 Early Days at Google and the Interview Story
• 谷歌文化与演变 Google's Culture and Evolution
• 早期谷歌与管理哲学 Early Google Days and Management Philosophy
• 谷歌职业里程碑 Career Milestones at Google
• AI 研究历史 History of AI Research
• 硬件与游戏 Hardware and Gaming
• 神经网络基础 Neural Networks Basics
• 什么是 LLM What is an LLM
• 思考与幻觉 Thinking and Hallucination
• LLM 类比演员 Analogy of LLMs as an actor
• 可解释性与调试挑战 Interpretability and debugging challenges
• Attention Is All You Need 论文 Attention Is All You Need paper
• Transformer 与注意力机制简介 Introduction to Transformers and Attention
• 解释注意力机制 Explaining Attention Mechanism
• 在 Character AI 构建圣诞老人机器人 Building a Santa Bot on Character AI
• LaMDA 起源与离开谷歌 Origins of LaMDA and Leaving Google
• AI 领域大厂 vs 创业公司 Big Tech vs Startups in AI
• 未来用例与推测 Future Use Cases and Speculation
• 个人 AI 助手愿景 Personal AI assistant vision
• 进展的边界条件 Bounding conditions for progress
• 创业者的机遇 Opportunities for founders
• 模型优化的存在性问题 Existential questions about model optimization

阅读全文双语转录 →