扩展与通往人类级 AI 之路

大家好，我是 Jared Kaplan。我简单讲讲 Scaling 和通往人类水平 AI 的道路，不过我想对在座各位来说，这些概念已经很熟悉了，所以我长话短说，之后我们会和 Diana 进行一场炉边对话式的问答。我其实只做了六年 AI 研究。在此之前，我的大部分职业生涯是作为理论物理学家在学术界度过的。那么我是怎么进入 AI 领域的呢？长话短说，我当初学物理，是因为我妈妈是科幻作家，我想知道能不能造出超光速引擎，而物理学就是实现它的途径。我也对理解宇宙本身充满热情：事物如何运作？我们周围一切背后的最大趋势从何而来？比如，宇宙是决定论的吗？我们有自由意志吗？我对这些问题非常感兴趣。幸运的是，在物理学家生涯中，我遇到了很多非常有趣、思想深刻的人，包括我现在经常合作的 Anthropic 的许多创始人。我对他们做的事情很感兴趣，一直关注着。当我在物理学的不同领域——从大型强子对撞机物理、粒子物理、宇宙学、弦理论等——之间转换时，我有点沮丧，有点无聊。我觉得我们进步得不够快。很多朋友告诉我 AI 正在变得非常重要。我不相信他们，我很怀疑。我想，AI 嘛，人们已经研究了 50 年，支持向量机也没什么激动人心的。那是我在 2005、2009 年上学时所知道的一切。但我后来被说服了，也许 AI 会是一个令人兴奋的研究领域。我很幸运认识了对的人，剩下的就是历史了。

Hey everyone. I'm Jared Kaplan. I'm going to talk briefly about scaling and the road to human level AI, but my guess is for this audience, a lot of these ideas are pretty familiar, so I'll keep it short and then we're going to do a sort of fireside chat Q&A with Diana. I actually have only been working on AI for about six years. Before that, I had a long career, the vast majority of my career as a theoretical physicist working in academia. So how did I get to AI? Well, I want to be brief. Why did I start in physics? It was basically because my mom was a science fiction writer and I wanted to figure out if we could build a faster than light drive and physics was the way to do that. I also was very excited about just understanding the universe. How do things work? How do the biggest trends that underlie everything we see around us, where does that all come from? For example, is the universe deterministic? Do we have free will? I was very interested in all of those questions. But fortunately, along the way, during my career as a physicist, I met a lot of very interesting, very deep people, including many of the founders of Anthropic that I now work with all the time. I was really interested in what they were doing and I kept track of it. As I moved among different subject areas in physics, from large hadron collider physics, particle physics, cosmology, string theory, and on, I got a little bit frustrated, a little bit bored. I didn't feel like we were making progress quickly enough. A lot of my friends were telling me that AI was becoming a really big deal. I didn't believe them. I was really skeptical. I thought, well, AI, people have been working on it for 50 years. SVMs aren't that exciting. That was all we knew about back in 2005, 2009 when I was in school. But I got convinced that maybe AI would be an exciting field to work on. I got very lucky to know the right people and the rest is history.

我来简单讲讲当代 AI 模型是如何工作的，以及 Scaling 如何让它们越来越好。像 Claude、ChatGPT 等当代 AI 模型的训练有两个基本阶段。第一阶段是预训练，我们训练 AI 模型模仿人类书写的数据和文本，理解数据背后的相关性。这些图非常复古，实际上来自原始 GPT-3 模型的 playground。你可以看到，作为期刊俱乐部的演讲者，你可能会期待我说某些话。句子中的 'elephant' 这个词非常不可能出现。预训练做的就是教模型在大型文本语料库中哪些词可能跟在其他词后面，对于当代模型，则是多模态数据。当代 AI 模型训练的第二个阶段是强化学习。这是另一张非常复古的幻灯片，展示了我们用于 Claude zero 或 Claude negative one 的原始界面，那是在 2022 年的古老年代，我们收集反馈数据时用的。这里看到的基本上是与非常早期版本的 Claude 对话的界面，根据你或众包工人的判断，选择 Claude 的哪个回答更好。利用那个信号，我们优化、强化那些被选为好的行为，即那些被认为有帮助、诚实且无害的行为，并抑制不好的行为。所以训练这些模型的全部就是学习预测下一个词，然后通过强化学习来学习执行有用的任务。

So I'm going to talk a little bit about how our contemporary AI models work and how scaling is leading them to get better and better. There are really two fundamental phases to the training of contemporary AI models like Claude, ChatGPT, etc. The first phase is pre-training, and that's where we train AI models to imitate human written data, human written text and understand the correlations underlying that data. These figures are very retro. This is actually from the playground of the original GPT-3 model. You can see that as a speaker at a journal club, you're probably elephant me to say certain things. The word 'elephant' in that sentence is really unlikely. What pre-training does is teach models what words are likely to follow other words in large corpora of text, and now with contemporary models, multimodal data. The second phase of training for contemporary AI models is reinforcement learning. This is another very retro slide. It shows the original interface we used for sort of Claude zero or Claude negative one, back in the ancient days of 2022 when we were collecting feedback data. What you see here is basically the interface for having a conversation with very early versions of Claude and picking which response from Claude was better according to you, according to crowdworkers, etc. Using that signal, we optimize, we reinforce the behaviors that are chosen to be good, that are chosen to be helpful, honest, and harmless. And we discourage the behaviors that are bad. So really all there is to training these models is learning to predict the next word and then doing reinforcement learning to learn to do useful tasks.

事实证明，这两个训练阶段都存在缩放定律。这是我们五六年前做的一张图，展示了当你扩大 AI 的预训练阶段时，模型的性能会可预测地越来越好。这源于我问了一个最蠢的问题。作为物理学家，你被训练成那样：看大局，问很蠢的问题。我听说在 2010 年代，大数据很重要很流行，所以我就想知道数据应该有多大？它有多重要？有多大帮助？同样，很多人注意到更大的 AI 模型表现更好。所以我们问：这些模型到底好多少？我们非常幸运，发现 AI 训练背后实际上有非常精确且令人惊讶的规律。这些漂亮的趋势像你在物理或天文学中看到的任何东西一样精确，这让我们大吃一惊。这给了我们很大的信心，相信 AI 会以非常可预测的方式变得越来越聪明。因为正如你在这些图中看到的，早在 2019 年，我们就在跨越许多数量级的算力、数据集大小和神经网络大小上观察到了这一点。所以我们预期，一旦你在许多数量级上看到某件事是真的，它很可能在很长一段时间内继续成立。所以我认为这是 AI 改进的基础之一。

It turns out that there are scaling laws for both of these phases of training. This is a figure that we made five or six years ago now, and it shows how as you scale up the pre-training phase of AI, you predictably get better and better performance for our models. This came about because I was just asking the dumbest possible question. As a physicist, that's what you're trained to do. You look at the big picture and ask really dumb things. I'd heard it was very popular in the 2010s to say that big data was important, so I just wanted to know how big should the data be? How important is it? How much does it help? Similarly, a lot of people were noticing that larger AI models performed better. So we just asked the question, how much better do these models perform? We got really lucky. We found that there's actually something very precise and surprising underlying AI training. This really blew us away that there are these nice trends that are as precise as anything you see in physics or astronomy. These gave us a lot of conviction to believe that AI was just going to keep getting smarter and smarter in a very predictable way. Because as you can see in these figures, already back in 2019, we were looking across many orders of magnitude in compute, in dataset size, in neural network size. So we expected once you see something is true over many orders of magnitude, you expect it's probably going to continue to be true for a long time further. So this has been one of the fundamental things that I think underlies improvements in AI.

另一件事其实也很早就开始显现，尽管在过去几年才变得真正有影响力，那就是你可以在 AI 训练的强化学习阶段看到缩放定律。大约四年前，一位研究人员决定研究 AlphaGo 的缩放定律。基本上是把两个非常引人注目的 AI 成功案例——GPT-3 和预训练的缩放，以及 AlphaGo——结合起来。这位研究人员 Andy Jones 独自工作，在那个古老年代可能只有一块 GPU。所以他无法研究昂贵的 AlphaGo，但他可以研究一个更简单的游戏叫 Hex。他做了你在这里看到的这张图。ELO 分数，我想当时还不那么广为人知，但 ELO 分数当然是国际象棋等级分。它们基本上描述了一个棋手在象棋比赛中击败另一个棋手的可能性。现在它们被用来对 AI 模型进行基准测试，看看人类更偏好哪个 AI 模型。但在当时，这只是 ELO 分数作为国际象棋等级分的经典应用。他观察了当你训练不同的模型来玩 Hex 这个比围棋简单一点的棋盘游戏时，它们的表现如何。他看到了这些显著的直线。所以注意到非常简单的趋势是科学中的一项技能，而我认为这个趋势当时被忽视了。

The other is actually also something that started to appear quite a long time ago, although it's become really impactful in the last couple of years, is that you can see scaling laws in the reinforcement learning phase of AI training. A researcher about four years ago decided to study scaling laws for AlphaGo. Basically putting together two very high-profile AI successes, GPT-3 and scaling for pre-training, and AlphaGo. This was just a researcher, Andy Jones, working on his own with maybe a single GPU back in these ancient days. So he couldn't study AlphaGo, that was expensive, but he could study a simpler game called Hex. So he made this plot that you see here. ELO scores, I think, weren't as well known back then, but ELO scores are, of course, chess ratings. They basically describe how likely it is for one player to beat another in a game of chess. They're used now to benchmark AI models to see how often a human prefers one AI model to another. But back then, this is just the classic application of ELO scores as chess ratings. He looked at as you train different models to play this game of Hex, which is a very simple board game, a bit simpler than Go, how do they do? And he saw these remarkable straight lines. So it's a skill in science to notice very simple trends, and this was one I think it went unnoticed.

我认为人们没有足够早地关注强化学习中的这种 Scaling 行为，但最终它还是实现了。我们看到，你可以在预训练和强化学习中同时扩展算力，从而获得越来越好的性能。这是推动 AI 进步的根本原因。并不是 AI 研究人员突然变聪明了，而是我们找到了一种非常简单的方法来系统地提升 AI，并且我们正在不断推动它。

I think people didn't focus on this scaling behavior in RL soon enough, but eventually it came to pass. We see that you can scale up compute in both pre-training and RL and get better and better performance. That's the fundamental thing driving AI progress. It's not that AI researchers suddenly got smart; it's that we found a very simple way of making AI better systematically, and we're turning that crank.

这解锁了什么样的能力？我倾向于从两个维度来思考 AI 能力。一个不那么有趣但仍然非常重要的维度是 AI 的灵活性，即 AI 适应我们需求的能力。如果把 AlphaGo 放在这个图上，它会远远低于 X 轴，因为尽管 AlphaGo 非常智能，但它只能在围棋棋盘的世界中运作。但自从大语言模型出现以来，我们取得了稳步进展，制造出了能够处理多种模态的 AI。我们还没有具备嗅觉的 AI 模型，但这可能很快就会到来。随着你沿着 Y 轴向上移动，你会得到能够做更多相关事情的 AI 系统。

What kinds of capabilities is this unlocking? I tend to think of AI capabilities on two axes. The less interesting axis, but still very important, is the flexibility of AI, the ability of AI to meet us where we are. If you put AlphaGo on this figure, it would be very far below the X-axis because although AlphaGo was super intelligent, it was only able to operate in the universe of a Go board. But we've made steady progress since the advent of large language models, making AI that can deal with many modalities. We don't have AI models with a sense of smell, but that's probably coming. As you go up the Y-axis, you get AI systems that can do more relevant things in the world.

更有趣的维度是 X 轴，即一个人完成 AI 模型所能做的任务需要多长时间。随着我们提升 AI 的能力，这个时间一直在稳步增长。这是任务的时间跨度。我合作的一个组织非常系统地研究了这一点，并发现了另一个 Scaling 趋势：AI 模型能完成的任务长度大约每 7 个月翻一番。因此，通过扩展预训练和强化学习的算力所带来的智能提升，导致了可预测的有用任务，包括更长周期的任务。

The more interesting axis is the X-axis, which is how long it would take a person to do the kinds of tasks that AI models can do. That has been increasing steadily as we increase the capability of AI. This is the time horizon for tasks. An organization I work with studied this very systematically and found another scaling trend: the length of tasks that AI models can do is doubling roughly every 7 months. So increasing intelligence from scaling compute for pre-training and RL leads to predictable useful tasks, including longer horizon tasks.

你可以推测这将会走向何方。在《AI 2027》中，人们就做了这样的推测。这幅图景表明，在未来几年内，我们可能会达到一个点，AI 模型能够完成需要数天、数周、数月甚至数年才能完成的任务。最终，数百万个 AI 模型协同工作，将能够完成整个人类组织所能完成的工作，比如整个科学界目前所做的工作。在数学或理论物理学中，仅通过思考就能取得进展。所以你可以想象，AI 系统协同工作，能够在几天或几周内取得理论物理学界需要 50 年才能取得的进展。

You can speculate about where this is heading. In AI 2027, folks did. This picture suggests that over the next few years, we may reach a point where AI models can do tasks that take days, weeks, months, years. Eventually, millions of AI models working together will be able to do the work that whole human organizations can do, like the work the entire scientific community currently does. In math or theoretical physics, you can make progress just by thinking. So you can imagine AI systems working together to make the kind of progress that the theoretical physics community makes in 50 years in a matter of days or weeks.

如果 Scaling 能带我们走得很远，还剩下什么？我认为，要解锁广义上的人类水平 AI，剩下的东西相对简单。一个重要的成分是相关的组织知识。我们需要训练 AI 模型，使其能够学习在公司、组织、政府中工作，就像拥有在那里工作多年的人那样的背景知识。AI 模型需要能够处理知识。它们还需要记忆。我区分记忆为跟踪特定任务的进展，建立相关记忆并使用它们。这是我们开始在 Claude 4 中构建的东西，并且会变得越来越重要。

What is left if scaling can take us very far? I think what may be left to unlock human-level AI broadly construed is relatively simple. One important ingredient is relevant organizational knowledge. We need to train AI models that can learn to work within companies, organizations, governments as though they have the context of someone who's been working there for years. AI models need to be able to work with knowledge. They also need memory. I distinguish memory as keeping track of progress on a specific task, building relevant memories and using them. That's something we've begun to build into Claude 4 and will become increasingly important.

第三个成分是监督：AI 模型理解细粒度细微差别以解决困难模糊任务的能力。训练 AI 模型编写通过测试的代码或正确回答数学问题很容易，因为什么是对的很明确。所以强化学习在这些任务上效果很好。但我们需要 AI 模型来帮助生成更细微的奖励信号，这样我们就能利用强化学习来做诸如讲好笑话、写好诗以及在研究中有良好品味之类的事情。

A third ingredient is oversight: the ability of AI models to understand fine-grained nuances to solve hard fuzzy tasks. It's easy to train AI models to write code that passes tests or answer math questions correctly because it's crisp what's correct. So reinforcement learning works well for those tasks. But we need AI models that help generate more nuanced reward signals so we can use reinforcement learning to do things like tell good jokes, write good poems, and have good taste in research.

其他成分更简单。我们需要训练 AI 模型完成更复杂的任务，从文本模型到多模态模型再到机器人，沿着 Y 轴向上推进。我预计在未来几年，当规模应用于这些不同领域时，我们将看到持续的收益。

Other ingredients are simpler. We need to train AI models to do more complex tasks, work our way up the Y-axis from text models to multimodal models to robotics. I expect over the next few years, we'll see continued gains from scale when applied to these different domains.

我们应该如何为这个未来做准备？我推荐几件事。一是构建那些还不完全奏效的东西。AI 模型正在快速变好，而且会持续下去。如果你构建了一个产品，因为 Claude 4 还有点笨而效果不佳，可以预期 Claude 5 会让它工作并带来价值。所以在 AI 能力的边界上做实验，因为这些边界正在快速移动。

How should we prepare for this future? I recommend a few things. One is to build things that don't quite work yet. AI models are getting better very quickly, and that will continue. If you build a product that doesn't quite work because Claude 4 is still a bit too dumb, expect that Claude 5 will make it work and deliver value. So experiment on the boundaries of what AI can do, because those boundaries are moving rapidly.

其次，AI 将有助于整合 AI。一个主要瓶颈是 AI 发展如此之快，以至于我们没有时间将其整合到产品、公司、科学中。为了加快这一进程，利用 AI 进行 AI 整合将非常有价值。

Next, AI is going to be helpful for integrating AI. One main bottleneck is that AI is developing so quickly that we haven't had time to integrate it into products, companies, science. To speed that up, leveraging AI for AI integration will be very valuable.

最后，找出 AI 能够快速被采用的地方是关键。我们看到 AI 在编码领域的整合呈爆炸式增长。软件工程是 AI 的好去处，但大问题是接下来是什么。除了软件工程，还有什么能如此快速增长？我不知道答案，但希望你们能想出来。

Finally, figuring out where adoption of AI could happen very quickly is key. We're seeing an explosion of AI integration for coding. Software engineering is a great place for AI, but the big question is what's next. Beyond software engineering, what can grow that quickly? I don't know the answer, but hopefully you'll figure it out.

我想邀请 Diana 上台聊一聊。YC 下一期创业营正在招生。有创业想法吗？请到 ycombinator.com/apply 申请。永远不嫌早，填写申请表会让你的想法更成熟。好，回到视频。刚才关于 Scaling 的演讲非常精彩，最近 Anthropic 刚刚发布了 Claude 4，现在已经可用。我想知道，随着这些模型不断迭代，未来 12 个月会带来哪些新的可能性？

I want to invite Diana on stage for a chat. YC's next batch is now taking applications. Got a startup in you? Apply at ycombinator.com/apply. It's never too early and filling out the app will level up your idea. Okay, back to the video. That was an awesome talk about all the scaling laws and recently Anthropic just launched Claude 4 which is now available. Curious, how does it change what is possible as all these model releases keep compounding for the next 12 months?

我觉得如果 12 个月后才出更好的模型，那我们就麻烦了。不过关于 Claude 4，有几点可以说。Claude 3.7 Sonnet 在编程方面已经非常令人兴奋了，但大家注意到 3.7 有点过于急切，有时它只想让你的测试通过，会做一些你不希望它做的事，比如大量使用 try-except 块。所以 Claude 4 改进了模型作为智能体进行编程的能力，同时在搜索等其他应用上也有提升。我们还改进了它的监督能力，也就是我在演讲中提到的 oversight，让它能更好地遵循你的指令，提高代码质量。另一个改进是保存和存储记忆的能力。我们希望人们能利用这一点，因为 Claude 4 在处理复杂任务时可能会用完上下文窗口，但它可以将记忆存储为文件或记录，并在多个上下文窗口之间检索以持续工作。最后，我认为缩放定律描绘的是一幅渐进式进步的图景。所以 Claude 会随着每次发布在许多方面稳步提升。但 Scaling 实际上指向一条平滑的曲线，通向我所期望的人类水平 AI 或 AGI。

I think we'll be in trouble if it's 12 months before an even better model comes out. But a few things with Claude 4. With Claude 3.7 Sonnet, it was already really exciting to use for coding. But something everyone noticed was that 3.7 was a little too eager. Sometimes it just really wanted to make your tests pass, and it would do things you don't really want, like a lot of try-except blocks. So with Claude 4, we've been able to improve the model's ability to act as an agent specifically for coding, but also in many other ways for search and other applications. We also improved its supervision, the oversight I mentioned in my talk, so that it follows your directions and hopefully improves code quality. Another thing we worked on is improving its ability to save and store memories. We hope to see people leveraging that because Claude 4 can blow through its context window with a very complex task but can also store memories as files or records, retrieve them to keep doing work across many context windows. Finally, I think the picture that scaling laws paint is one of incremental progress. So with Claude, you'll see it steadily gets better in many different ways with each release. But scaling really suggests a smooth curve towards what I expect is human-level AI or AGI.

有没有什么特别的功能会让在座的观众兴奋？能不能透露一些 beta 或 alpha 版本的信息，让大家知道新 API 中哪些会让人爱不释手？

Is there some special feature that a lot of the audience here are going to get excited about? Some beta or alpha leak you can give everyone on what you think people will fall in love with in the new APIs?

我最兴奋的是记忆功能解锁了越来越长周期的任务。随着时间的推移，我们会看到 Claude 成为一个能承担越来越大的工作块的协作者。这正好对应你提到的未来模型能处理更大任务这一点。目前，它们能完成以小时计的任务。

The thing I'm most excited about is memory unlocking longer and longer horizon tasks. As time goes on, we'll see Claude as a collaborator that can take on larger and larger chunks of work. This goes to your point about future models being able to take bigger tasks. Right now, they can do tasks in the hours range.

是的，我同意。这是一个非常不精确的衡量标准，但如果你看软件工程任务，基准测试确实测量了人们完成各种任务所需的时间，大概是小时级别。总的来说，当人们与 AI 合作时，怀疑者会正确指出 AI 会犯很多愚蠢的错误。它能做出令人惊喜的出色工作，但也会犯基本错误。AI 智能与人类智能的一个基本区别是，有很多事情我做不到，但我至少能判断它们是否正确。对于 AI 来说，判断能力和生成能力更接近，这意味着人类在与 AI 互动时的一个主要角色是作为管理者来检查工作是否合理。

Yeah, I think so. It's a very imprecise measure, but if you look at software engineering tasks, I think benchmarks literally measured how long it would take people to do various tasks, and it's a time scale of hours. Broadly, as people work with AI, skeptics will correctly say that AI makes lots of stupid mistakes. It can do brilliant things that surprise you, but also make basic errors. One basic feature of AI intelligence that differs from human intelligence is that there are many things I can't do, but I can at least judge whether they were done correctly. For AI, the judgment versus generative capability is much closer, meaning a major role people can play in interacting with AI is as managers to sanity-check the work.

这很有趣，因为去年我们在 YC 的创业营中观察到，很多公司销售产品时仍然将其定位为副驾驶，比如客户支持的副驾驶，仍然需要人工批准才能回复客户。但就在春季这一期，情况发生了变化，很多 AI 模型已经能够端到端地完成任务。创始人现在直接销售完整工作流的替代方案。你如何看待这一变化，以及你希望观众能构建什么？

That's fascinating because one thing we observed through the batches in YC last year is that many companies selling products still sold them as a co-pilot, where you would have a co-pilot for customer support and still need human approval before sending a reply. But one thing that changed just in the spring batch is that many AI models are very capable of doing tasks end-to-end. Founders are now selling direct replacements of full workflows. How have you seen this translate to what you hope the audience will build?

有很多可能性。这取决于什么样的成功率或性能是可以接受的。有些任务做到 70% 正确就足够了，而另一些则需要 99.9% 才能部署。老实说，为那些 70-80% 就够用的用例构建可能更有趣，因为你可以真正探索 AI 能力的边界。但我们也在提高可靠性。所以我们会看到越来越多的这类任务。目前，人机协作将是最有趣的领域，因为对于最先进的任务，你确实需要人类参与。但从长远来看，会有更多任务可以完全自动化。

There are many possibilities. It's a question of what level of success or performance is acceptable. Some tasks where getting it 70% right is good enough, others require 99.9% to deploy. Honestly, it's probably more fun to build for use cases where 70-80% is good enough because you can really push the frontier of what AI is capable of. But we're also pushing up reliability. So we'll see more and more of these tasks. Right now, human-AI collaboration is going to be the most interesting place because for the most advanced tasks you really need humans in the loop. But in the longer term, there will be more tasks that can be fully automated.

你能详细说说你认为人机循环协作的世界会是什么样子吗？因为 Dario 的文章《爱与优雅的机器》描绘了一幅非常乐观的图景。我们具体如何实现这一目标？

Can you say more about what you think the world will look like with this human-to-AI loop collaboration? Because there's Dario's essay 'Machines of Love and Grace' that paints a very optimistic picture. What are the details of how we get there?

我认为我们已经看到了一些这样的情况。当我与生物医学研究人员交流时，通过适当的编排，现在就可以利用前沿 AI 模型为药物发现产生有趣且有价值的见解。所以这已经开始发生了。我思考的一个方面是，有些智能需要深度，有些需要广度。例如，在数学中，你可以花十年时间证明一个定理，比如黎曼猜想或费马大定理。这是解决一个非常具体、非常困难的问题。但在许多科学领域，可能生物学更明显，有趣的是心理学或历史学，将大量来自不同领域的信息整合起来是关键。而 AI 模型在预训练阶段吸收了整个人类文明的知识。

I think we already see some of that happening. When I talk to folks in biomedical research, with the right orchestration, it's possible to take frontier AI models now and produce interesting valuable insights for drug discovery. So that's already starting. One aspect I think about is that there is intelligence that requires a lot of depth and intelligence that requires a lot of breadth. For example, in math you can work on proving one theorem for a decade, like the Riemann hypothesis or Fermat's Last Theorem. That's solving one very specific, very hard problem. But there are many areas of science, probably more so in biology, maybe interestingly in psychology or history, where putting together a very large number of pieces of information across many different areas is key. And AI models during the pre-training phase kind of absorb all of human civilization's knowledge.

所以我猜想，利用 AI 的这一特性——它比任何一位人类专家都懂得多得多——有很多成果可以收获。你可以通过整合许多不同领域的专业知识来激发洞见，比如在生物学研究中。我认为我们在让 AI 更好地完成深度任务方面取得了很大进展，比如困难的编程问题和数学问题。但我猜想，在那些需要整合可能没有任何一位人类专家具备的知识的领域，这种智能特别有用。所以我预计会看到更多这样的情况，即利用 AI 的知识广度。至于具体如何展开，我真的不知道。预测未来非常困难。缩放定律提供了一种预测未来的方式，它表明这一趋势会持续。我认为我们长期看到的许多趋势都会持续。我是说经济、GDP 这类趋势是未来非常可靠的指标。但就具体实施细节而言，我觉得很难说。

And so I suspect that there's a lot of fruit to be picked in using that sort of feature of AI that it knows much much more than any one human expert and therefore you can elicit insights putting together many different areas of expertise, say across biology for research. So I think that we're making a lot of progress on making AI better at deeper tasks like hard coding problems, hard math problems, but I suspect that there's a particular overhang in areas where putting together knowledge that maybe no one human expert would have, where that kind of intelligence is very useful. So I think that's something that I'd expect to see more of, leveraging AI's breadth of knowledge. In terms of how exactly it will roll out, I really don't know. It's really hard to predict the future. Scaling laws give you one way of predicting the future which says this trend is going to continue. I think a lot of trends that we see over the long haul I expect will continue. I mean the economy, the GDP, these kinds of trends are really reliable indicators of the future. But I think in terms of in detail how things will be implemented, it's really hard to say.

你认为有哪些特定领域，更多的开发者可以进入并用这些新模型进行构建？我的意思是，编程任务已经做了很多，但有哪些任务有更多未开发的领域，现在正被当前模型解锁？

Are there specific areas that you think a lot more builders could go into and build with these new models? I mean there's a lot that has been done for coding tasks but what are some tasks that have a lot more green field that are just getting unlocked right now with the current models?

我来自研究背景而非商业，所以我不确定自己能说出什么深刻的东西。但我觉得，总的来说，任何需要大量技能且主要涉及坐在电脑前与数据交互的任务，都是潜在领域。比如金融，那些大量使用 Excel 表格的人。还有法律，尽管法律可能更受监管，需要更多专业知识作为认可标志。但我认为所有这些领域可能都是未开发的。另一个我提到的领域是如何将 AI 整合到现有业务中。我认为当电力出现时，有一个漫长的采用周期，最初最简单的用电方式不一定是最好的。你不想只是用电机替换蒸汽机。你想重新设计工厂的运作方式。我认为可能尽快利用 AI 将 AI 整合到经济各个领域，那里有很大的杠杆效应。

I come from a research background rather than business, so I don't know that I have anything very deep to say, but I think that in general any place where it requires a lot of skill and it's a task that mostly involves sitting in front of a computer interacting with data. I think finance, people who use Excel spreadsheets a lot. I expect law, although maybe law is more regulated, requires more expertise as a stamp of approval. But I think all of these areas are probably green field. I think another that I mentioned is how do we integrate AI into existing businesses? I think that when electricity came along, there was a long adoption cycle and the very first simplest ways of using electricity weren't necessarily the best. You wanted to not just replace a steam engine with an electric motor. You wanted to remake the way that factories work. And I think that probably leveraging AI to integrate AI into parts of the economy as quickly as possible. I expect there's just a lot of leverage there.

你受过广泛的物理学训练，是最早真正观察到缩放定律这一趋势的人之一，这很可能源于你作为物理学家的背景，看到自然界中自然发生的各种指数现象。这种训练如何帮助你进行世界上最好的 AI 研究？

Now, you have extensive training as a physicist and you were one of the first to really observe this trend with scaling laws, and it probably comes from being a physicist and seeing all these exponentials that happen naturally in nature. How has that training come about with being able to perform the best research in the world with AI?

我认为从物理学的角度来看，有用的是寻找最宏观、最整体的趋势，然后尽可能精确地描述它们。我记得遇到一些杰出的 AI 研究者，他们会说学习是指数收敛的，我就会问一些非常愚蠢的问题，比如你确定是指数吗？会不会只是幂律？是二次的吗？这个收敛到底是怎么样的？这是一个非常笨拙的简单问题，但基本上我认为，在尽可能精确地描述你看到的宏观趋势方面，有很多成果可以收获，可能现在仍然如此，因为这给了你很多工具。它让你能够问，真正推动进展意味着什么？我认为对于缩放定律，圣杯是找到更好的斜率，因为这意味着随着你投入更多算力，你会比其他 AI 开发者获得越来越大的优势。但除非你精确地描述了你看到的趋势，否则你并不清楚击败它意味着什么，能击败多少，以及如何系统地知道你是否达到了那个目标。所以我认为这些就是我使用的工具。并不一定是直接应用量子场论到 AI，我觉得那有点太具体了。

I think the thing that was useful from a physics point of view is looking for the biggest picture, most macro trends and then trying to make them as precise as possible. So I remember meeting brilliant AI researchers who would say things like learning is converging exponentially and I would just ask really dumb questions like are you sure it's an exponential? Could it just be a power law? Is it quadratic? Exactly how is this thing converging? And it's a really dumb kind of simple question to ask, but basically I think there was a lot of fruit to be picked and probably still is in trying to make the big trends that you see as precise as possible because that gives you a lot of tools. It allows you to ask like what does it really mean to move the needle? I think with scaling laws, the holy grail is finding a better slope to the scaling law because that means that as you put in more compute, you're going to get a bigger and bigger advantage over other AI developers. But until you've made precise what the trend is that you see, you don't know exactly what it means to beat it and how much you can beat it by and how to know systematically whether you're achieving that end. So I think those were the tools that I used. It wasn't necessarily like literally applying quantum field theory to AI. I think that's a little bit too specific.

有没有特定的物理学启发式方法，比如重整化、对称性，在观察或测量这一趋势时非常有用？

Are there specific physics heuristics like renormalization, symmetry that came in very handy to really observe this trend or measure it?

如果你观察 AI 模型，你会发现它们很大。神经网络很大，有数十亿甚至数万亿的参数。这意味着它们由大型矩阵组成。基本上，研究在神经网络非常大、特别是组成神经网络的矩阵非常大的极限下的近似，实际上是有用的，而且这在物理学和数学中是一个众所周知的近似。这已经被应用了。但我认为总的来说，问非常天真、愚蠢的问题会让你走得很远。我认为 AI 在某种意义上只有 10 到 15 年的历史，就我们目前训练 AI 模型的形态而言。这意味着这是一个极其新的领域。许多最基本的问题还没有答案，比如可解释性问题，AI 模型到底是如何工作的。所以我认为在那个层面有很多东西要学，而不是应用非常复杂的技术。

Something that you'll observe if you look at AI models is that they're big. Neural networks are big. They have billions now trillions of parameters. That means that they're made out of big matrices. And basically studying approximations where you take the limit that neural networks are very big and specifically that the matrices that compose neural networks are big. That's actually been kind of useful and that's something that actually was a well-known approximation in physics and in math. That's something that's been applied. But I think generally it's really asking very naive dumb questions that gets you very far. I think AI is really in a certain sense only maybe 10 or 15 years old in terms of the current incarnation of how we're training AI models. That means that it's an incredibly new field. A lot of the most basic questions haven't been answered like questions of interpretability, how AI models really work. And so I think there's really a lot to learn at that level rather than applying very fancy techniques.

你在可解释性方面应用了物理学的特定工具吗？

Are there specific tools in physics that you apply for interpretability?

我会说可解释性更像生物学，更像神经科学。所以这些是相关的工具。其中也有一些数学。但我认为这更像是试图理解大脑的特征。AI 相对于神经科学的优势在于，你可以在 AI 中真正测量一切。你无法测量大脑中每个神经元、每个突触的活动，但在 AI 中可以。所以有更多的数据用于逆向工程，了解 AI 模型是如何工作的。

I would say that interpretability is a lot more like biology. It's a lot more like neuroscience. So I think those are kind of the tools. There is some more mathematics there. But I think it's more like trying to understand the features of the brain. The benefit that you get with AI over neuroscience is that you can really measure everything in AI. You can't measure the activity of every neuron, every synapse in a brain, but you can do that in AI. So there's much more data for reverse engineering how AI models work.

关于缩放定律的一个方面：它们已经跨越了五个数量级，这很惊人。这是一个有点逆向思维的问题，但什么样的经验迹象会让你相信曲线正在变化，也许我们正在偏离曲线？

Now, one aspect about scaling laws: they've held for over five orders of magnitude, which is wild. This is a bit of a contrarian question, but what empirical sign would convince you that the curve is changing, that maybe we're getting off the curve?

我认为这是一个非常难的问题，对吧？因为我主要用缩放定律来诊断 AI 训练是否出了问题。

I think it's a really hard question, right? Because I mostly use scaling laws to diagnose whether AI training is broken or not.

所以我认为，一旦你看到某个趋势非常引人注目，审视它在哪里失效就变得非常有趣。但我的第一反应是，如果缩放定律失效了，那一定是我们搞砸了 AI 训练的某个方面。也许我们搞错了神经网络的架构，或者训练中存在我们没看到的瓶颈，或者我们使用的算法精度有问题。所以我认为，要让我相信缩放定律真的不再有效，需要很多证据，因为过去五年里，很多次当缩放看起来出问题时，都是因为我们做错了。

So I think that once you see something and you find it a very compelling trend, it becomes very interesting to examine where it's failing. But I think that my first inclination is to think if scaling laws are failing, it's because we've screwed up AI training in some way. Maybe we got the architecture of the neural network wrong or there's some bottleneck in training that we don't see or there's some problem with precision in the algorithms that we're using. So I think it would take a lot to convince me at least that scaling was really no longer working at the level of these empirical laws because so many times in my experience over the last 5 years when it seemed like scaling was broken it was because we were doing it wrong.

有意思。那么具体来说，与这条曲线相伴的是需要大量算力。随着算力变得越来越稀缺，你会把精度降到多低？你会探索像 FP4 这样的东西吗？你会探索三元表示吗？你对此有什么想法？

Interesting. So going into something very specific that goes hand in hand is a lot of the compute power required to keep going on this curve. What happens as compute becomes more scarce? How far down do you go into the precision ladder? Do you explore things like FP4, do you explore things like ternary representations? What are your thoughts around that?

是的，我认为现在 AI 效率很低，因为 AI 有巨大价值。解锁最强大的前沿模型有很大价值。像 Anthropic 这样的公司正在尽可能快地提高 AI 训练和推理的效率，同时解锁前沿能力。但很多焦点确实在解锁前沿上。我认为随着时间的推移，AI 会越来越普及，我们会大幅降低推理和训练的成本。现在每年我们在算法、算力扩展和推理效率上都有 3 到 10 倍的提升。有个笑话是，我们最终会让计算机回到二进制。所以我认为，未来我们会看到更低的精度，这是提高推理效率的众多途径之一。但我们现在与 AI 发展非常不平衡。AI 进步非常快，变化非常快。我们还没有完全实现当前模型的潜力，但我们在解锁越来越多的能力。所以我认为，当 AI 不再那么快速变化时的平衡状态，是 AI 变得极其便宜，但很难知道我们是否能达到那个状态。AI 可能进步太快，以至于智能的提升解锁了更多东西，所以我们可能继续专注于提升能力，而不是把精度降到 FP2。

Yeah, I think that right now AI is really inefficient because there's a lot of value in AI. There's a lot of value in unlocking the most capable frontier model. Companies like Anthropic and others are moving as quickly as we can to both make AI training more efficient and AI inference more efficient as well as unlocking frontier capabilities. But a lot of the focus really is on unlocking the frontier. I think that over time as AI becomes more and more widespread, we're going to really drive down the cost of inference and training dramatically from where we are right now. Right now we're seeing 3x to 10x gains algorithmically and in scaling up compute and in inference efficiency per year. I guess the joke is that we're going to get computers back into binary. So I think that we will see much lower precision as one of the many avenues to make inference more efficient over time. But we're very out of equilibrium with AI development right now. AI is improving very rapidly. Things are changing very rapidly. We haven't fully realized the potential of current models, but we're unlocking more and more capabilities. So I think that the equilibrium situation where AI isn't changing that quickly is one where AI is extremely inexpensive, but it's hard to know if we're even going to get there. AI may just keep getting better so quickly that improvements in intelligence unlock so much more and so we may continue to focus on that rather than getting precision down to FP2.

这很像杰文斯悖论：随着智能越来越好，人们会更想要它，而不是成本下降，这很讽刺。

Which is very much the Jevons paradox: as intelligence becomes better and better, people are going to want it more, not that it's driving the cost down, which is this irony.

是的，绝对如此。我认为我们确实看到了这一点：在某些点上，AI 变得足够可及。话虽如此，我认为随着 AI 系统越来越强大，能做越来越多我们做的工作，为前沿能力付费是值得的。我一直有个问题：所有价值都在前沿，还是较便宜的、不那么强大的系统也有很大价值？时间范围图景可能是一种思考方式。我认为你可以做很多简单的、小任务，但使用一个能端到端完成复杂任务的 AI 模型要方便得多，而不是需要人类来编排一个更笨的模型，把任务分解成小块再拼起来。所以我确实认为很多价值会来自最强大的模型，但我可能错了。这可能取决于 AI 集成商高效利用 AI 的能力。

Yeah, absolutely. I think that's certainly something that we've seen: there are certain points where AI becomes accessible enough. That said, I think as AI systems become more and more capable and can do more and more of the work that we do, it's going to be worth it to pay for frontier capabilities. I think it's a question that I've always had: is all of the value at the frontier or is there a lot of value with cheaper systems that aren't quite as capable? The time horizon picture is maybe one way of thinking about this. I think that you can do a lot of very simple bite-sized tasks, but it's just much more convenient to be able to use an AI model that can do a very complex task end to end rather than requiring us as humans to orchestrate a much dumber model to break the task down into very small slices and put them together. So, I do kind of expect that a lot of the value is going to come from the most capable models, but I might be wrong. It might depend on the capabilities of AI integrators to leverage AI really efficiently.

你会给这些听众什么建议？他们职业生涯早期，潜力很大。在未来这些模型变得如此厉害的情况下，如何保持相关性？大家应该擅长什么、学习什么，才能继续做出好工作？

What advice would you give this audience, which is everyone early in their career with lots of potential, in terms of how do you stay relevant in the future where all these models are going to become so awesome? What should everyone be really good at and study to still do really good work?

我想，就像我提到的，理解这些模型如何工作，并能高效地利用和集成它们，有很大价值。而且在前沿领域构建也有很大价值。我不知道，我们可以把时间交给观众提问。

I think as I mentioned, there's a lot of value in understanding how these models work and being able to really efficiently leverage them and integrate them. And I think there's a lot of value in building at the frontier. I don't know, we could turn it over to the audience for questions.

我们把时间交给观众提问。

Let's turn it out to the audience for some questions.

我有个关于缩放损失的问题。你展示了很多缩放定律是线性的：算力指数增长，但缩放损失线性进步。但在你最后一张幻灯片上，你预期突然指数增长节省的时间。我想问你：为什么你认为在这张图上突然变成指数而不是线性了？

I had a quick question on the scaling loss. You showed that a lot of the scaling laws are linear: the more we have exponential compute going up, but then we have linear progress in the scaling loss. But then on your last slide, you show that you expect suddenly an exponential growth in how much time we save. I want to ask you: why do you think that suddenly on this chart we're exponential and not linear anymore?

是的，这是个很好的问题，我不知道。那个发现是经验性的。我倾向于这样想：为了完成越来越复杂、时间跨度更长的任务，真正需要的是自我纠正的能力。你需要能够识别出你制定了一个计划然后开始执行。但大家都知道，我们的计划没什么用，我们遇到现实，我们会出错。所以我认为，决定模型能完成的任务时间跨度的很大一部分因素，是它们注意到自己做错了并纠正的能力。而且我认为这不需要很多信息。不需要智能有巨大变化就能多发现一两次错误并知道如何纠正。但如果你纠正了错误，也许任务的时间跨度就翻倍了，因为你不是在这里卡住，而是在两倍远的地方卡住。所以这就是我的图景：通过相对适度的任务理解和自我纠正能力的提升，你可以解锁越来越长的时间跨度。但这些只是空话。我认为经验趋势可能是最有趣的。也许我们可以建立更详细的模型来解释为什么这个趋势成立，但你的猜测和我的一样好。

Yeah, this is a really good question and I don't know. The meter finding was kind of an empirical finding. The way that I tend to think about this is that in order to do more and more complex longer horizon tasks, what you really need is some ability to self-correct. You need to be able to identify that you make a plan and then start executing it. But everyone knows that our plans are kind of worthless and we encounter reality, we get things wrong. So I think that a lot of what determines the horizon length of what models can accomplish is their ability to notice that they're doing something wrong and correct it. And I think that's not a lot of bits of information. It doesn't necessarily require a huge change in intelligence to notice one or two more times that you've made a mistake and how to correct that mistake. But if you fix your mistake, maybe you double the horizon length of the task because instead of getting stuck here, you get stuck twice as far out. So I think that's the picture that I have: you can unlock longer and longer horizons with relatively modest improvements in your ability to understand the task and self-correct. But those are just words. I think the empirical trend is maybe the most interesting thing. And maybe we can build more detailed models for why that trend is true, but your guess is as good as mine.

我也有个问题，很荣幸。基本上，关于延长时间跨度，我对神经网络的思维模型很简单：想让它们做什么，就用什么数据训练。所以如果想延长时间跨度，就必须慢慢获得验证信号。一种方式是通过产品，比如云智能体，然后用验证信号逐步改进模型。这在编程领域效果很好，因为产品足够好可以部署并获取验证信号。但其他领域呢？我们只是把数据标注员扩展到 AGI，还是有更好的方法？

So I also have a question. It's an honor. So basically, in terms of increasing the time horizon, my mental model of neural networks is very simple: if you want them to do something, you train on such data. So if you want to increase the time horizon, you have to slowly get verification signals. One way to do this is via product, like a cloud agent, and then use the verification signal to incrementally improve the model. This works nicely for coding where you have a product good enough to deploy and get verification signals. But what about other domains? Are we just scaling data labelers to AGI, or is there a better approach?

好问题。当怀疑论者问我为什么认为我们能扩展到通用人类水平的 AI 时，基本上就是因为你的说法。有一条操作密集的路径：为 AI 模型构建越来越多、越来越复杂、时间跨度越来越长的任务，然后通过强化学习训练。我觉得这是 AI 进步的最坏情况。鉴于 AI 的投资水平和创造的价值，必要时人们会这么做。不过，也有更简单的方法。最好是用一个 AI 模型来监督另一个 AI 模型，比如 Claude，提供详细的监督，而不是仅仅说‘你完成这个极其复杂的任务了吗？’比如‘你成为教授并获得终身教职了吗？’那需要六七年，效率很低。相反，提供详细监督，比如‘你这里做得好，那里做得差。’随着我们更多地以这种方式使用 AI，我们很可能能让长跨度任务的训练更高效。我们已经在某种程度上这样做了。

Yeah, it's a good question. When skeptics ask why I think we can scale to broadly human-level AI, it's basically because of what you said. There's an operationally intensive path where you build more and more different tasks for AI models, more complex and longer horizon, and just turn the crank and train with RL on those tasks. I feel that's the worst case for AI progress. Given the level of investment and value created with AI, people will do that if necessary. That said, there are ways to make it simpler. The best is to have an AI model trained to oversee and supervise another AI model, like Claude, providing detailed supervision rather than just saying 'did you complete this incredibly complicated task?' For example, 'did you become a faculty member and get tenure?' That takes six or seven years, which is inefficient. Instead, provide detailed supervision like 'you're doing this well, you're doing this poorly.' As we use AI more in that way, we'll likely make training for long-horizon tasks more efficient. We're already doing this to some extent.

我们问最后一个问题。

We'll do one last question.

我想接着问。在开发这些任务并用强化学习训练时，你会尝试用大语言模型来创建任务，还是仍然用人类？

I wanted to build on that. When developing these tasks and training with RL, would you try creating these tasks using large language models, or are you still using humans?

好问题。混合使用。我们尽可能用 AI 通过代码生成任务，也会请人类创建任务。所以是混合的。随着 AI 越来越好，希望我们能更多地利用 AI，但任务的难度前沿也在提高，所以人类仍然会参与。

Great question. A mix. We're building tasks as much as possible using AI to generate tasks with code. We also ask humans to create tasks. So it's a mixture. As AI gets better, hopefully we can leverage AI more, but the frontier of difficulty also increases, so humans will still be involved.

好的，谢谢。

Okay. Thank you.

好了，让我们给 Jared 鼓掌。

All right. Let's give a round of applause to Jared.

非常感谢。谢谢。

Thank you so much. Thanks.