推理足以实现泛化吗？

非常荣幸能邀请到 Transformer 论文的合著者来到播客。我觉得你一直处于 AI 世界诸多重大变革的前沿。我们的目标就是听取你对当今 AI 前沿所有问题的看法。所以，非常感谢你来做客。

It's a pleasure to have a Transformer paper co-author on the podcast. I feel like you've been at the forefront of so many major changes in the AI world. And our goal is really to get your thoughts on all the questions around the AI frontiers today. So, I really appreciate you coming on the podcast.

非常感谢。谢谢你邀请我。

Thank you very much. Thank you for having me.

我觉得从泛化开始最合适不过了，对吧？感觉这是当前最受关注的问题。去年 11 月我听到你说，一个大问题是：推理是否足以实现泛化，还是需要另一种方法？我想知道从那以后你对这个问题的看法有何演变。

I can think of no better place to start than generalization, right? It feels like that's the question in the air right now. In November I heard you say the big question: is reasoning enough to get to generalization or is another method needed? I'm wondering how your thinking on that question has evolved since then.

如果我们看看当前带有推理、智能体以及 shell 访问权限的 Transformer，它们能做出惊人的事情。我们已经取得了难以置信的进展。两年前，甚至在 Transformer 出现之前，我绝不会相信，你只需拿一个下一个词预测器，给它思维链、强化学习和工具，它就能工作——我每天花几个小时与代码助手交谈，它能理解并实现功能。这太不可思议了。另一方面，总有一种感觉，它不太像我们，没有达到我们直觉中它应该达到的边界。我们觉得它应该更好，应该能从更少的数据中泛化，做出更大的跳跃，从更少的信息中获取概念。我最近有个说法：美国人会在穷尽所有其他选项后做正确的事，而大语言模型也会在穷尽所有其他选项后学习一个概念——它们需要一万亿个 token 来学习所有表面层次的东西，只有当这些无法解释时，它们最终才会学习概念。这不是我们学习的方式；我们有时会凭空创造概念，虽然它们并不完美。所以确实感觉存在某种可能泛化得更好的东西，可能有一种更长期的理解形式。但这只是一种感觉。每当我们试图具体指出它时，它似乎就消失了，或者 Transformer 又赶了上来。双方都在成长：Transformer 变得更好，但其他方法的理由也更强了。现在有很多实验室在追求后 Transformer 架构，并看到了有趣的结果。所以谁会赢？我仍然不知道。

If we take the current transformers with reasoning, agents, and access to a shell, they can do amazing things. It's incredible how far we've gotten. Two years ago, even before transformers, I would never have believed that you could take this next-word predictor, give it chain of thought, RL, and tools, and it would work—I spend hours every day talking to code assistants, and it makes sense and implements things. That's incredible. On the other hand, there is this feeling that it is not quite like us, that it's not quite at the edge of what we feel it should be. We feel it should be even better, that we can generalize from less data, make bigger leaps, get concepts from way less. I recently have this saying: Americans will do the right thing after exhausting all other options, and LLMs will learn a concept after exhausting all other options—they need a trillion tokens to learn all the surface-level things, and only when that doesn't explain something will they finally learn the concept. That's not how we learn; we just get concepts from sometimes we make them up and they're not great. So it does feel like there is something else that possibly could generalize much better, that could have a different form of understanding more like long-term. But it's a feeling. Every time we try to put our finger on it, it seems to evaporate, or the transformer just catches up. Both sides have grown: transformers have gotten even better, but the case for something else has also gotten even better. There are now labs pursuing post-transformers and seeing interesting results. So who wins? I still don't know.

我想这对我们的听众来说会很有趣。在最近 NeurIPS 的一次演讲中，你提到了这种“空气中的气息”——有些东西正在发生，激励着新实验室和人们独立出来，研究主流架构的替代方案。这种感觉是什么？是看到了早期结果，还是仅仅是研究者的直觉？你能说得更具体一点吗？

I think it'll be interesting for our listeners. At a talk at NeurIPS more recently, you alluded to this 'whiff in the air'—that there's something happening that inspired new labs and folks to spin out and work on alternatives to the dominant architectures. What is that feeling? Is it seeing early results, or is it just researcher intuition? Can you make it a little more concrete?

这在很大程度上是直觉，你需要意识到，因为很多这样的讨论发生在旧金山的派对和播客上，所以可能在某种程度上是自我驱动的。但我认为有一部分是非常根本的。Yann LeCun 多年来一直在说类似的话。我们现有的模型，被称为神经网络，因为它们本意是模仿我们的大脑，但实际上并非如此。它们非常不同，即使可能有一些相似之处。如果你看看人类是如何学习的，很难不说，从少得多的数据中，我们能比当前模型做得更多。所以感觉我们作为学习机器拥有一种基本能力，而我们的模型目前没有。所以从根本上说，应该存在某种东西，而不仅仅是一种氛围。作为反驳，这些模型总是用一万亿个 token 训练，而人类从未如此。所以我们只是没有针对少量数据训练进行优化。如果你有相同的算力但数据有限，你可以调整 Transformer 使其比今天做得更好。

A lot of this is intuition, and you need to be aware because a lot of this happens in San Francisco at parties and on podcasts, so it may be self-fueled to some extent. But I think there is a part that is very fundamental. Yann LeCun has been saying something like this for years. The models we have, called neural networks because they were meant to imitate our brain, don't really do that. They are quite different, even if they may have some similarities. If you look at how humans learn, it is hard not to say that from much less data we can do much more than our current models. So it feels like there is this fundamental ability that we as learning machines have that our models currently don't. So fundamentally, there should be something there, not just a vibe. As a counterargument, these models always had a trillion tokens to train on, and people never do. So we just didn't optimize them for training with less. If you had the same amount of compute but limited data, you could tweak transformers to do much better than they do today.

所以，有些人会说，为什么要这么做？我们有数据。这是一项大工程。但确实感觉，即使我们试图用尽可能少的数据推进，我们也从视觉事物、在世界上移动、采取行动中获得了大量数据。这是非常不同类型的数据，并非真正可比。这就是为什么很难对此做出确切的科学陈述。但有一种感觉，我们还没有挖掘出机器学习中所有潜力。令人兴奋的是，也许如果我们发现那里有什么，它可能让我们已有的东西更加惊人。也许不会。也许当你有那么多数据时，它就消失了。谁知道呢？但作为研究人员，这绝对非常有趣，我想对很多人来说也是如此。

So, some people say, why would you do that? We have the data. It's a big enterprise. But it does feel that even when we try to push with as little data as possible, we also get a lot of data from visual things, from moving in the world, from taking actions. It's very different kinds of data, not truly comparable. That's why it's hard to make a firm scientific statement about it. But there is this feeling that we have not exploited all that is there in machine learning. The exciting feeling is that maybe if we find out what's out there, it could make what we have even more amazing. Maybe not. Maybe it vanishes when you have that much data. Who knows? But it's definitely extremely interesting to me as a researcher, and I think to many people.

Transformer 令人着迷。它们非常擅长推理。它们能解决研究级别的数学问题。我相信你听说过最近的 ERS 相关事情。作为我过去是一名数学家，这极其令人兴奋。我从未想过在这个时间框架内，计算机会像真正的研究人员一样与我进行高水平的数学对话。这现在已经成为现实，太疯狂了。但作为一名研究人员，我想，好吧，但我们还没有真正搞懂这种学习。有一种感觉，它确实在学习，但需要如此多的数据和算力。这感觉就像我们还没有完全达到目标。这仅仅是一种感觉吗？是一种氛围吗？在某种程度上这似乎是现实，但我们需要拭目以待。

Transformers were fascinating. They're great at reasoning. They can solve research math problems. I'm sure you've heard about the recent ERS things. As a mathematician before in my life, this is extremely exciting. I never thought a computer in this time frame would talk to me about mathematics at a high level as a real researcher. That exists now, and it's insane. But as a researcher, I'm like, okay, but we haven't really figured out this learning. There is this feeling that it learns certainly, but it needs so much data, so much compute. This feels like we're not quite there yet. Is this only a feeling? Is it a vibe? It seems to be reality to some extent, but we'll need to see.

搞清楚这一点的研究吸引力非常有道理。其他人可能会看着它说，就算它不像人类又怎样？我们有数据，我们有有效的方法。显然有些领域数据有限，比如药物开发，从更有限的数据中学习会非常有帮助。但世界上很多问题并没有那么受数据限制。有时我觉得这些方面在自说自话。实验室里的人会对 Yann LeCun 之类的人翻白眼。

The research appeal of figuring that out makes a ton of sense. Other folks might look at it and be like, so what if it's not like people? We have the data, we have a method that works. Obviously there are areas with limited data, like drug development, where learning from more limited data would be really helpful. But so many problems in the world aren't that data constrained. Sometimes I feel like these sides talk past each other. People at the labs will roll their eyes at Yann LeCun or something like that.

我认为这么说很公平。但另一方面，考虑到随着对 AI 的全面投资，事情发展得有多快，不受数据限制的问题会很快得到解决。所以很快，所有剩余的瓶颈都会相当受数据限制，或者已经如此。特别是，确实感觉要在物理世界中良好运作，你至少需要解决其中的一部分，因为物理世界，如果你在一个机器人硬件上训练，它不像虚拟或文本世界那样能很好地扩展数据。所以物理世界是一个相当大的部分。

I think this is fair to say. But on the other hand, given how quickly things move with the whole investment in AI, problems that are not data limited get solved very rapidly. So very soon, all bottlenecks that remain will be quite data limited, or already are. In particular, it does feel that to work well in the physical world, you need to solve some part of it at least, because the physical world, if you train on one robot hardware, it doesn't quite scale data the way the virtual or text worlds do. So the physical world is a sizable chunk.

所以人们当然在尝试用模拟数据和自我中心视频数据这些更便宜的来源。

So people are certainly trying with simulation data and with egocentric video data, cheaper sources.

是的。

Yeah.

我是 Waymo 的超级粉丝。我总是开玩笑：人们说，“我的自动驾驶汽车在哪里？”嗯，我就在开它们。它们就在这里。但后来他们取消了高速公路驾驶，因为他们又无法应对某个施工区。感觉他们多年来一直有施工区的问题，模拟中有数百万英里，实际驾驶中也有不少，但仍然无法泛化到高速公路上的施工区。这感觉不对劲。我不知道具体哪里出了问题，但没有青少年有这个问题，也没有人类有。我们有很多其他问题，但不会是在城市施工区能开，在高速公路上却不能。施工区就是施工区。

I'm a huge fan of Waymo. I always joke: people say, "Where are my self-driving cars?" Well, I drive them. They're here. But then they just canceled the highway driving because they couldn't deal with some construction zone again. It feels like they have had this construction zone thing for years, and there are millions of miles in simulation and quite some in real driving, and it still can't generalize to a construction zone on a highway. This feels off. I don't know what exactly didn't work there, but no teenager has this problem, no human. We have many other problems, but not that we can drive in a construction zone in the city but not on the highway. A construction zone is a construction zone.

你认为这些东西中的一部分能在 Transformer 内解决吗？在未来几年，你期待什么来更好地回答这个问题？

Do you think that some of this stuff could be solved within transformers? What are you looking for in the next few years to get a better answer to this question?

机器学习研究令人兴奋的部分在于它如此广泛。你永远不知道是需要调整架构、数据、损失函数还是优化过程。所有方面都有合理的论点，而且除此之外，可能最终你需要在一定程度上调整所有方面。Transformer 很棒，但它与下一个词预测损失也很搭。你可以让它与 RL 一起工作，但你需要思维链。这些拼图只有组合在一起才能工作。所以有可能，如果出现新东西，可能需要调整一切。但也可能 Transformer 的部分会存活下来，例如，注意力机制可能还会存在。但也许你需要给它添加其他东西。

The exciting part in ML research is that it is so broad. You never know whether you need to tweak the architecture, the data, the loss, or the optimization process. There are fair arguments for all, and on top of that, it might turn out that you need to tweak all of them to some extent. Transformer is great, but it's also great with the next-token prediction loss. You can make it work with RL, but you need chain of thought. These puzzles only work when you click them together. So it is possible that if there is a new thing, there might need to be tweaks to everything. But it's also possible that parts of transformers will survive, for example, probably attention will be somewhere there. But maybe you need other things to it.

是的。

Yeah.

也许，我的机器学习生涯始于 RNN，我当然深深珍视循环。我喜欢它作为一种构造。推理在某种程度上让它回归了，因为你生成的每个新 token 都使用相同的权重。所以从某种意义上说，它回来了。但确实感觉 RL 就像非常稀疏的损失，你做了很多，但它有效。每次我们尝试以其他方式实现循环时，它似乎还没有成功。但总是有一个问题：我们有多努力尝试过？我不知道你或观众是否知道，有一些模型如 TRM 和 HRM，非常小的模型，在数独等问题上表现很好，还有 RKGI。它们有点像玩具测试，但表现相当不错。我认为很多后 Transformer 架构正试图将其与 LLM 融合。

Maybe, I started my machine learning life with RNNs, and I certainly hold recurrence deep in my heart. I like it as a construct. Reasoning kind of brought it back because every new token you produce uses the same weights. So in some sense it's back. But it does feel that RL is like very sparse losses and you do so much, but it works. Every time we try to do recurrence in other ways, it somehow does not seem to click yet. But then there is always the question: how hard have we tried? I don't know if you or the audience know, there are models like TRM and HRM, very small models that turned out to do very well on problems like Sudoku but also RKGI. They are a little bit toy tests, but they do quite well. I think a lot of the post-transformer architectures are trying to merge this with LLMs.

这确实很有趣，对吧？纯 Transformer 在这方面表现不佳，但加入一些递归、一些架构上的小调整，也许再加上一点不同的损失函数，它就能做得很好。所以即使在小的规模上，你也能做很多事情。但问题是，它能否泛化到语言领域，并给出你想要的结果？这将非常值得关注。幸运的是，有很多实验室正在尝试。不过另一件事是，今年我们有了智能体，对我来说，这完全是我作为机器学习研究员工作方式的最大变化，可以说，大概是过去 20 年里的最大变化。

It's interesting, certainly, right? It's like the pure Transformer can't do so well on it, but you add some recurrence, you add some bit of architectural tweaks, maybe a little different loss, and it does really well. So even on the small scale, you can do a lot. But then will it generalize to the language and give you the things you want? It will be very interesting to see. And luckily, there were a number of labs that are trying. The other thing though is like this year we have the agents, and to me this is a totally the biggest change in the way I work as an ML researcher in, I would say, the last 20 years probably.

我不知道你是否尝试量化过，但你觉得它让你提高了多少效率？

I don't know if you try to quantify it, but like how much more productive do you think it makes you?

哦，我完全可以量化，因为我最近尝试在一台私人机器上复现一堆论文，比如我一直很感兴趣的旧论文。嗯，甚至包括一些我丢失了代码的自己的论文。其中至少有一篇我以前尝试复现过，我知道需要大约三周才能让它跑起来，而用 Codex 我两天就搞定了。所以大概是，比如说，从一周缩短到一天。这已经是 10 倍或 5 倍的提升，你知道，也许我当年可以更快，但它确实改变了你的节奏，因为你可以同时处理多件事。而且，你知道，我可以同时启动三个任务并行进行，而以前我通常一次只做一件事，对吧？所以它既加快了速度，又让工作更并行化。

Oh, I can fairly well quantify it because I tried recently, just on a private machine, to reproduce a bunch of papers, like old papers that I was always quite interested in. Um, even some of my papers that I lost code for. And at least one of them I tried to reproduce before, and I knew it took me about three weeks to get to a runnable state, and with Codex I could get there in two days. So it's about, let's say, a week to a day. That's already whether it's a 10x or a 5x, you know, maybe I could have been faster back then, but it's certainly it changes your rhythm because you can just take on things. It's also, you know, I can just start three things in parallel and let it go, while when I was doing I would be just do one thing usually, right? So it both makes it faster and makes it more parallel.

所以它既加快了速度，又让工作更并行化，这真是……

So it both makes it faster and makes it more parallel, which is...

但我的意思是，当我在做私人项目，而不是在生产仓库中时，我基本上不再看代码了。有个朋友问我，你觉得你现在变迟钝了吗？我想了想，觉得恰恰相反，因为虽然我不再看每个类名、每个小函数，但我仍然知道这些智能体可能会出问题。比如有一次，它禁用了某些东西，运行了一些代码，出现了一些辅助损失，然后它就直接加上了，它认为应该再加一个辅助损失，结果完全离谱，完全不合适。所以你需要在大脑中完全掌控它到底在做什么？损失是什么？是什么……但你不需要控制类名是什么？函数里的具体措辞是什么？你能信任智能体，相信它们真的在实现你的想法，这非常令人印象深刻。但我的意思是，有时候我们会检查，它们确实做到了。但由于你需要在脑海中完全掌控机器学习层面实际运行的东西，比如损失是什么、批次是什么，那么我觉得这实际上让我比之前更能在精神上掌控我在做的事情。因为以前我会实现它，但你知道，有时候在运行之前，我不得不稍微忘记大局，专注于小细节，调试，然后再回到大局。到那时，也许我忘记了一些细节，然后当它出错时我才想起来。现在，这是一种美妙的状态，你可以完全沉浸其中，你只需从机器学习角度思考应该发生什么。你告诉它，验证它，它就实现了。是的。所以这不仅仅是节省时间的问题。它让工作变得如此愉快。我想，这算是研究人员中的一种轻度精神病吧，我们就是停不下来。

But like, I mean, when I do private things, not in a production repo, I basically stopped looking at the code. And a friend asked me, like, do you think you're less sharp now? And I gave it some thought and I think it's actually to the contrary, because due to the fact that I don't look at every class name, every small function, but I still know that these agents can go off the rails. Like once it, with disable it runs something and there were some aux losses and it just added, it just thought it should have another auxiliary loss and it was totally off the charts and out of place. So you need to have like a full control in your head of what exactly is it doing? What is the loss? What is the... but you don't need to have control of like what's the name of the class? What's the exact words in the function? It's quite impressive that you can trust the agents to be trustful and like that they're really implementing what you think. But I mean sometimes we check and they are. But since you need to have a full control in your head of what's actually running machine learning wise, what are the losses, what are the batches, what then I feel it gives me actually more mental control over what I'm doing than it was before. Because before I would implement it, but you know, sometimes in the time before running it, I would have to like forget a little bit about what the big picture was exactly and focus on the little things, debug, then go back to the big picture. By that time, maybe I forgot some detail and then I would remember it when it was wrong. Now it's like it's this beautiful thing where you can just be in this flow, like you just think machine learning wise what's supposed to happen. You tell it, verify it, and it's happening. Yeah. So it's not just about the time saved. It just makes the work so nice. It's a mild psychosis, I guess, among researchers these days, we just can't stop.

OpenAI 非常公开地表示，“嘿，我们的目标大概是，在今年 11 月之前达到一个研究级实习生的水平。”作为一个在研究工作中一直使用 Codex 的人，你觉得我们接近那个目标了吗？或者你对这个里程碑有什么看法？

OpenAI very publicly said, 'Hey, our goal is kind of, I think, a research level intern by November of this year.' As someone who plays around with Codex all the time in your research, does it feel like you're close to that, or how are you feeling about that milestone?

确实感觉接近实习生的水平，但你需要非常仔细地检查，正如我所说，它可能会加一个你并没有要求的损失函数，因为它觉得那样合理。嗯，我不知道实习生会不会这样做。也许有时候，我猜，当他们有创意的时候。但比如，我有时会尝试，你知道，我会让它运行一整夜，然后给它一个目标，比如做一个更好的模型来降低困惑度。嗯，这从来都不管用。它只会开始做一些非常琐碎的调整，这些调整并不真正有趣或有用。所以它肯定还没有达到研究人员的水平。

It does feel like close to an intern, but you need to be very carefully checking, as I said, it can just add you a loss that you did not ask for because it seems reasonable to it. Um, I don't know if interns do that. Maybe sometimes, I guess, sometimes when they're creative. But like, I try sometimes, you know, it's like I will just let it go for the night and I give it the goal, you know, make a better model for lower perplexity. Um, that never works. It will just start doing some very trivial tweaks that are not really interesting or useful. So it's certainly not at the level of a researcher.

是的。那么，要让它变得更好，前进的道路是什么？

Yeah. So what's the path forward to make it better there?

这又回到了我们的问题，对吧？很长一段时间里，我都在研究机器学习中的长上下文，甚至可以说在 Transformer 之前，就在研究记忆之类的东西。然后我们用 Transformer 来研究，你知道，上下文变得更长了。我们达到了大约一百万个 token，考虑到注意力机制的作用，这已经非常大了。但现在有了智能体，感觉 grep 或 ripgrep 就是我们解决长上下文的方案：把一堆东西写在文件里，让它能访问 grep 来查找，然后告诉它写索引文件，就像一个小型库。当然，作为一个研究人员，如果你五年前告诉我这个，我会说那不是解决方案，那是个 hack，对吧？但你知道，机器学习中，所有东西都是某种 hack。比如 dropout，我们不去评判，对吧？我们接受有效的方法，而它效果惊人。然后你加入一点强化学习，比如压缩。如果说有一个原因让我更喜欢 Codex 而不是 Claude Code，那就是压缩。你可以继续对话，它很擅长压缩。为什么它擅长压缩？这并不神秘，对吧？人们很好地设计了提示，然后加入了一些强化学习让它工作。如果你几年前告诉我，长上下文问题，你只需要用一点强化学习让它能使用工具、在文件中查找内容，然后总结得足够好以保持上下文，我会说，好吧，那是个权宜之计，不是那种深刻的东西。但你知道，我们不是根据解决方案的外表来评判，而是根据它们的效果来评判，而它确实效果很好。所以回到问题，它能成为研究人员吗？嗯，有些人可能会说，也许不能。也许你需要这种新架构。也许你需要一个后 Transformer 的东西，它有更大的概念并能遵循目标。这是一个合理的论点，对吧？目前感觉它似乎能解决这个问题。

It goes back to our question, right? For a long time, I worked on long context in machine learning, even before Transformers, you could say, on memory and so on. And then we worked on it with Transformers, and you know, the context got longer. We got like a million tokens, which is huge given what attention does. But now with agents, it really does feel like grep or ripgrep is our solution to long context: let's write a bunch of stuff in files and give it access to grep so it can find, and you know, tell it to write index files, and it's like a little library. And of course, to me as a researcher, if you told me five years ago, I'd say that's not a solution, that's a hack, right? But you know, machine learning, everything is a hack of sorts. So like dropout is like, we don't judge, right? We take what works, and it works amazingly. And you add a little bit of RL, like for example, compaction. If there is one reason I like Codex over Claude Code, it is compaction. You can go on with the thread and it's good at compacting. Why is it good at compacting? There is, but I don't think there's anything very mysterious, right? People prompted it well and then put some RL on it to just make it work. And if you told this to me some years ago that the long context, well, you just RL a bit that it can use tools and find stuff in files and then summarize good enough to keep the context, I would tell, okay, that's a band-aid, that's not like a deep thing. But you know, we don't judge solutions by how they look. We judge them by how they work, and it works really well. So to the point of, can it become a researcher? Well, you know, some people would say, well, maybe no. Maybe you'll need this new architecture. Maybe you'll need a post-Transformer thing that has concepts that are bigger and follows goals. And it's a fair argument, right? It currently feels like it can solve this.

但也有人说：你会和 Codex 对话一个月，然后被提示回顾这些对话，找出元模式，写到一些文件里，再思考如何利用它们。如果你有上千人的数据，在上面做一些强化学习，它可能就会开始表现得像个研究员。从某种意义上说，研究员就是这样学习的，对吧？我们看别人怎么做研究，自己做点尝试，看看什么有效。为什么今天还做不到？我相信肯定有人试过。

But then there are other people who say: you will have your conversations with Codex for a month, and then you're going to be prompted to go over them and find meta patterns, write this to some files, and just think how it can use them. And maybe if you have data from over a thousand people and do some RL on it, it will start behaving like a researcher. In some ways, this is how researchers learn, right? We look at how other people do research, we do a bit of our trials, see what works. Why doesn't that work today? I'm sure people have tried that.

哦，我知道。我不认为人们已经非常努力地尝试过。有些人做一些提示词，它们对他们有效。对我来说重要的是——Codex 时代大概是从今年圣诞节开始的，对吧？我的意思是，Codex 以前就存在，我们用过，Claude Code 也存在，我们也用过它的一部分。但似乎只有更新的——不仅仅是模型，还有工具链和一些调整。所以这才半年，还有很多人在我们的 SFAI 圈子之外，完全不明白。他们会说：‘你有点精神病，但为什么呢？’我认为这是个合理的问题。但它最近才开始有效。我们并不真正理解——并不是一次大规模的预训练带来了这么大的变化，尽管大规模预训练也出现了。当我们从 RNN 转向 Transformer 时，很容易把变化归因于这个。

Oh, I know. I don't think people have tried very hard yet. Some people do some prompts and they work for them. It is important to me — the Codex era started like this year at Christmas, right? I mean, Codex existed before and we used it, and Claude Code existed and we also used parts of it. But it seems like only the newer — it's not just the models, it's also the harness and some tweaks. So it's barely half a year, and there are still many people, if you go a bit outside of our SFAI bubble, who totally don't get it. They're like, 'You're a little psychotic, but why?' And I think it's a fair question. But it started working very recently. We don't truly understand — it was not like a big pre-training that changed it that much, even though big pre-trainings came too. When we went from RNN to Transformers, it was very easy to attribute the change to that.

嗯，现在我觉得——然后还有推理，这显然很重要。

Well, now I feel — and then there was reasoning, which clearly is important.

但去年冬天、去年圣诞节的这个变化——有点难以确定。我的意思是，工具链变了，一点后训练变了，然后新的预训练模型来了，这当然让事情变得更好。但感觉像是一个大跳跃，不太容易确定是什么造成的。所以有点混乱，对吧？我们一直在改进一切。但因为它有效，而且感觉如此重要，所以也有必要把它带给人们，让它到处工作，推广它——竞争正在进行。所以我认为在这一切中，人们还没有真正有时间去思考：‘你到底怎么做这个元层次？’人们正在开始，但也感觉因为元层次就像你研究一周，然后得到一些模式并开始应用，这感觉需要几周时间。我们当前的强化学习方法不幸地需要基本上运行所有的 rollout，如果你的 rollout 长达数周，那么你的训练就开始长达数月，这变得有点不切实际。这也许是一个论点，说明人类方面有东西可以学习，因为显然人类可以花数年做研究，而且他们一生只做一次，对吧？或者两次。有些数学家花 20 年解决一个问题，那是他们的杰作，就这样。他们没有 200 个长达 20 年的问题可以学习，但他们 somehow 做到了。这是怎么做到的？这是一个迷人的问题，显然与此相关。我们还没有弄清楚，但另一方面，现在我们将收集数据，因为很多人都在使用它，关于人类数周到数月的数据。有人会运行这个 RL，结果可能只是让你走得更远。

But the change last winter, last Christmas — it's a little hard to pin down. I mean, the harness changed, and the little post-training changed, and then new pre-trained models came, which of course made things better. But it felt like a big jump which is not that easy to pin down what did it. So it's a little messy, right? We improve everything all the time. But then because it works and it feels so important, there is also the necessity to just bring it to people, make it work everywhere, promote it — there is this competition going on. So I think in all of this, people did not truly have the time yet to think like, 'How do you really do this meta level?' And people are starting, but it also feels like because the meta level is something like you research for a week and then you get some patterns and start applying them, that feels like this needs to take weeks. Our current reinforcement learning methods unluckily need to run basically all rollouts on this, and if your rollouts are weeks long, then your training starts to be months long, and that all becomes a little impractical. Which maybe is an argument that the human side has something to learn, because clearly humans can do research over years and they do this once in their life, right? Or twice. Some mathematicians spend 20 years on one problem, that's their magnum opus, and that's it. So they did not have 200 problems 20 years long before to learn from, and somehow they manage. How does this work? It's a fascinating question, clearly with some relevance to this. We haven't figured it out, but on the other hand, now we will gather, since a lot of people work with it, a lot of data on the weeks-to-months-long humans. Someone will run this RL, and it may just turn out that it gets you further.

这是一个非常有趣的观点，因为基本上，当人们 Scaling 预训练或 Scaling 最初的推理模型时，那是相当直接的，或者至少是有意义的——你 Scaling 的向量。然后我们在圣诞节在 Codex 和 Claude Code 上取得的这种大进步——如果你实际上不知道它的来源，或者你对此不完全清楚，那么很难确定你应该推动什么来继续改进这些能力。

It's such an interesting point because, basically, as folks were scaling pre-training or as folks were scaling the original sort of reasoning models, it was kind of straightforward, or at least made sense — the vector you were scaling on. And then this kind of big advance we've had in Codex and Claude Code over Christmas — if you don't actually know what the source of that is, or you're not fully crystal clear on it, it's very hard to then determine what you should be pushing on to continue to improve these capabilities.

是的，这有点令人困惑。我不知道并不意味着没人知道。我认为也许有些人对到底是什么推动了它有更强烈的看法，但我认为目前还不那么清楚。另一方面，它已经改进了一段时间，但发生了什么事。

Yes, it's a little confusing. The fact that I don't know doesn't mean nobody knows. I think maybe some people have stronger opinions on what exactly pushed it through, but I don't think it's that clear at this point. On the other hand, it's been improving for a while, but something happened.

是的，因为以前感觉不可能做到，现在却可以了。

Yeah, because it did not feel possible to do this, and now it does.

在当前这种 RL 侧的 Scaling 机制上，我认为很多人都有一个疑问：我们已经看到了大量的编码改进和数学等可验证领域。我觉得围绕 RL 的两个大问题仍然是：这在不可验证方面效果如何？以及我们能在多大程度上获得泛化，而不必在每个空间上做大量数据？也许我们一个一个来，但先从第一个开始：你如何看待不可验证领域需要解决的问题？以及对于编码和数学之外，下一个可能是什么领域，有什么线索吗？

On this kind of current scaling regime on the RL side, think one question a lot of folks have is: we've seen tons of coding improvement and math and these kind of verifiable domains. And I feel like the two big questions around RL continue to be: how well is this going to work on the non-verifiable side? And also, the extent to which we'll get generalization and not have to keep doing tons of data on each space. Maybe we'll take them one at a time, but starting with the first: how do you think about the problems that need to be solved on the non-verifiable domain side? And any inklings as to which faces might be next beyond code and math?

我确实认为在不可验证方面已经取得了相当进展。如果你看看像 Harvey 和法律，或者医学领域的东西，它们不可验证，但其中很多部分是可验证的，对吧？所以在这方面已经取得了良好进展。我认为 GDP val 是一个基准，在某种意义上也衡量这类事情。我确实认为有很好的进展，也有很好的激励在这些领域取得进展。我不确定称它们为不可验证是否完全公平。

I do think there has been fair progress on the non-verifiable side. If you look at things like Harvey and law, or things in medicine, they're not verifiable, but there are a lot of parts of them that are verifiable, right? So there's been good progress on that. And I think GDP val is one benchmark that in some sense benchmarks things like that too. And I do think there is really good progress and there are really good incentives to make progress in these domains. I'm not sure if it's fully fair to call them non-verifiable.

它们当然不像编码和数学那样完美设置，对吧？

They're certainly not as perfectly set up as coding and math, right?

它们不是编码和数学，对吧？但数学，我认为人们夸大了数学的可验证性。编码在编程竞赛可验证的意义上是相当可验证的。一旦你进入前端编码之类的东西，它也不是那么可验证。但尽管如此，数学——证明并不那么容易或干净。我的意思是，你可以用 Lean，但大多数数学，至少从 GPT 来看，并没有形式化。所以它不是那么可验证。所以这是一个谱系，然后事情变得越来越不可验证。我有一个小项目，把诗歌翻译成波兰语，这似乎相当不可验证。但当你把这些模型作为验证器运行时，它们能搞定不少东西。

They're not coding and math, right? But math, I think people overstate how verifiable math is. Coding is fairly verifiable in the sense that programming competitions are verifiable. Once you go to front-end coding and stuff, it's also not that verifiable. But still, math — the proofs are not that easy or clean. I mean, you can do Lean, but most of the math, at least from GPTs, it's not formalized. So it's not that verifiable. So it's a spectrum, and then things get less and less verifiable. I had this pet project of translating poetry into Polish, which seems fairly not verifiable. But then you run these models as verifiers, and they get a fair bit of stuff.

它们能理解押韵之类的东西，也能理解文化典故。事实证明，一旦你读过人们之前是如何验证事物的，你就能达到一定程度的可验证性。但我认为这个诗歌例子也意在说明，你可以验证很多东西，但仍然没有品味。而且由于它不可验证，就没那么容易了。如果容易用语言描述，那它早就被验证了。

They get like rhyme and things and they can get cultural references. So it turns out once you read how people have verified things before, you can get to some level of verifiability. But then I think what this poetry thing was also meant to show is you can verify a lot of things and still have no taste. And since it's not verifiable, it's not so easy. If it were easy to describe in words, then it would be verified.

但这并不意味着它不存在，对吧？你读到这个，大脑中会有某种东西强化一个想法：它们确实缺少了些什么。

But it doesn't mean it isn't there, right? You read this and there is something in your brain that reinforces this idea that there is something they're missing.

我们基本上是故意把自己逼进这个死胡同的，因为什么是强化学习？它告诉你，每当你有一个老师、一个验证者，有人告诉你这个好、这个坏，我就可以据此训练，然后变得更好。模型就是这么做的。所以每当我来说‘看，我觉得这个做得不太有品味’，就会有人告诉我‘好，那你证明给我看’，然后它就能做到。我认为有些人甚至做了与此相反的实验。对于图像生成，你可以问‘这漂亮吗？’——不可验证，但你只需要让一群人在训练时点击‘漂亮’或‘不漂亮’，然后你瞧，图像就开始变得更漂亮了。所以可验证性这个东西非常弱。当你问人们‘这个好吗？这个不好吗？’时，它只是一个非常稀疏的信号。为什么我觉得这个不太有品味？显然是我的一些经历和我处理信息的方式让我现在说出这句话。那么为什么模型不说呢？有两种可能：一是它没有足够的经历来促使它这样做，二是它没有以正确的方式处理信息。实际上我认为两者都有，但即使以它现有的处理方式，如果你给它更多经历，让一千个人告诉它，它就会变得更好。所以每个漏洞你都可以通过反复敲打来填补，但如果你不需要这样做该多好。因为每填补一个漏洞，它就不再是瓶颈，而新出现的瓶颈又是你尚未填补的漏洞。所以我们陷入了一个有趣的循环。但嘿，如果我们有这种方法，这种类似大脑的方法，它根本不会有那么多需要填补的漏洞，那岂不是很好？

We have driven ourselves into this hole basically on purpose because what is reinforcement learning? It tells you whenever you have a teacher, a validator, someone telling you this is good, this is bad, I can train against it and I'm going to get good. And that's what the models do. So whenever I come and say, 'Look, I don't think this does this very tastefully,' someone will tell me, 'Okay, show me,' and then it will nail it. I think some people even run experiments that go against that. For image generation, you can ask, 'Is this beautiful or not?' Not verifiable, but you just get a bunch of people who during training click 'this is beautiful' or 'this is not,' and lo and behold, the images start to be more beautiful. So the verifiability thing is very weak. It's just a very sparse signal when you ask people, 'Is this nice? Is this not nice?' Why do I think this is not very tasteful? It's clearly some of my experiences and some way that I have processed it that make me say this statement now. So why does the model not say it? There are two possibilities: one is that it hasn't seen enough experience that would make it do it, and the other is that it's not processing it in the right way. I believe in both actually, but even with the way it is processing, if you just put more experience, you ask a thousand people to tell it, then it gets better. So every hole you have you can kind of plug by hammering on it, but it would be so nice if you didn't have to. Because also every hole you plug stops being a bottleneck, and then the bottlenecks that emerge are again the holes that you have not plugged. So we're in this interesting circle. But hey, if we had this method, this brain-like method that would just not have so many holes that need plugging, wouldn't this be great?

这是否意味着，在当前架构下，任何人专注的任何问题领域都可以被解决？只是它可能需要精心策划的数据，并且比未来可能更优雅的方式要手动得多。但并没有一组问题或领域，你觉得在当前强化学习方法下对模型来说太难了？

Does that kind of imply that any problem area that someone focuses on under the current architectures can be figured out? It's just that it probably requires curated data and far more manual than a potentially more beautiful way of doing things down the line. But there's not like a set of problems or domains that you think under the current RL methods would be too hard for the models?

感觉并非如此，但你确实需要考虑经济因素。目前，要让这些模型真正工作得很好，你需要从一个相当强大、相当大且昂贵的模型开始。除此之外，它通常是封闭的，所以你无法真正去做。我的意思是，有强化学习微调 API，我很喜欢 OpenAI 的以及一些类似的，但你并没有真正的完全访问权限。所以即使有 API，也可能有点困难，而且除此之外，你在数据等方面的投入也是巨大的。你自己做不到；你需要一家公司，需要一些合同。如果这足够重要，那是一个公平的方法，但话说回来，如果你能直接和模型对话，它就能自己工作，那岂不是很好？

It does not feel so, but you do need to take economics into account. Currently, to make these models work really well, you need to start from a fairly strong model which is fairly big and expensive. On top of this, it's usually closed, so you can't really do it. I mean, there is the RL fine-tuning API which I quite like from OpenAI and some similar ones, but you don't truly have full access to it. So even with the API, it can be a little hard, and even on top of that, the investment you'll need to make into the data and things is substantial. You couldn't do this yourself; you'd need a company, you need some contracts. Which, if that's important enough, it's a fair method, but then, wouldn't it be great if you could just talk to the model and it would work on its own?

当你这样做时，有没有感觉到通用能力提升的迹象？你可以想象一个世界，我们从代码开始，然后做数学，然后为法律和医疗做这个，你可以逐个攻克每个领域，即使你没有得到任何跨领域的泛化。或者理想情况下，希望是在对一堆不同领域做了强化学习之后，也许在预训练的某个地方，泛化就出现了。

Does it feel like there's any sign of general capability improvement as you do this? You could imagine a world where we start with code, then do math, then do this for legal and healthcare, and you could tackle each of these one by one even if you're not getting any sort of generalization across. Or ideally, the hope would be that at some level of having done reinforcement learning on a bunch of different domains, maybe somewhere in pre-training, generalization emerges.

但我认为泛化在强化学习中确实存在。所以你认为模型已经在全面变好了？哦，是的，它们当然会。如果你看法律领域，它根本不在强化学习流程中，你和 Harvey 或其他人聊，他们会说要么泛化出现了，要么只需要一点点训练，在它上面加一点调整，它就突然学会了。所以肯定有泛化，但泛化有时似乎没有达到我们希望的程度。它甚至不会从数学泛化到数学的其他领域。如果你看 IMO（国际数学奥林匹克竞赛），现在看起来模型离 IMO 还很远……但它会有一些类型的习题。很长一段时间里，几何就是它无法攻克的。它能解决其他领域非常难的问题，但几何你就像‘哦，它没有空间理解’。然后它只是看到了更多数据就开始攻克了，但不是空间理解数据或物理数据，只是更多的几何问题。但它有这种锯齿状。它会从这里泛化到那里，但不会泛化到看起来非常接近的东西，但不知何故，在这些思维链的表示中，它并不是说它离我很近但离模型很远。所以并不是说它没有泛化。它在泛化，但以它那种奇怪的、外星人的方式，而这并没有覆盖我能泛化的一些方式。有可能随着更多数据，它会覆盖更多这个空间，但我也理解那些说当它这样时很难信任或投入的人，因为可能有一个它还没触及的尖峰，所以你需要警惕问题。作为机器学习研究员，我使用它时，我认为它让我非常诚实，因为我需要保持敏锐。所以也许在这方面是好的，但在能力方面并不好，因为你只是希望它没有这些尖锐的边缘。而目前，它确实有。

But I think generalization emerges in reinforcement learning. So you think already the models get better across the board? Oh yes, they certainly do. If you look at law, it's simply not in the RL pipeline at all, and you talk to Harvey or someone, and they say it either emerges or they need just a little training, just a few touches on top of it, and it suddenly catches it. So there is definitely generalization, but the generalization doesn't seem to go as far as we would hope sometimes. It doesn't generalize even from math to other areas of math. If you look at the IMO, right now it seems so far away that models IMO... but it would have some types of exercises. For a long time, it was geometry that it just couldn't crack. It would solve very hard problems in other domains, but geometry you were like, 'Oh, it has no spatial understanding.' And then it just saw more data and started cracking it, but not spatial understanding data or physical, just more geometry problems. But it has this jaggedness. It will generalize from here to here but not to something that seems very close, but somehow in this representation of these chains of thought, it's not like it's close to me but it's not close to the model. So it's not like it's not generalizing. It's generalizing but in its weird alien way, and that just doesn't cover some ways that I can generalize. It's possible that with more data it will just cover more of this space, but I also understand people who say that when it's like that, it's very hard to trust or commit to it, because there may be this spike that it just hasn't gotten, so you need to be on the lookout for problems. As I use it as an ML researcher, I think it keeps me very honest because I need to be sharp. So maybe this is good in this way, but it's not good in a capabilities way because you just hope that it doesn't have these sharp edges. And for now, it does.

你提到了一些应用公司显然从模型进步中受益，我认为有一个大问题：如果你是一家应用公司，现在应该与某个实验室紧密合作，分享所有评估和领域理解，还是最好基于信息构建自己的模型，而不是分享回去？我很好奇你如何看待核心模型之上的应用空间。

You mentioned some of the application companies obviously that benefit from these models getting better and I think there's like this big question of if you're an application company right now should you be working super closely with with one of the labs and sharing kind of all these evals and like understanding you have the domain or you know is that like actually you know are you better off kind of building almost your own model based on information versus you know sharing it back. I'm curious how you think about like the room for you know applications on top of uh you know the core models.

目前来说，确实如此：预训练模型越大越好，你遇到的尖锐问题就越少，生活总体上就越轻松，对吧？无论你在上面做强化学习还是微调，事情都变得更简单。这种情况一直持续，真是不可思议。我记得大约一两年前，人们还在说：“哦，大语言模型已死，小语言模型才是未来，小模型。”我们也有很棒的小模型，比如最近几十亿参数的 Gemma 系列。还记得 GPT-3 吗？人们说：“哦，你做不到，在 1000 亿参数以下无法进行零样本学习。”不，我们现在有 30 亿参数的模型，非常非常棒。这都很了不起，但如果你真想轻松解决大问题，适应你的数据和上下文，似乎没有什么能比得上真正的大模型。当然，它们很昂贵，难以使用，更难训练。

For now. What is certainly true is that the bigger and better your pre-trained model is, the less of these sharp edges you get and generally the easier all of your life becomes, right? Whether you do RL on it or fine-tuning on whatever bigger model, things just get easier. It's insane how this has continued to be the case. We have, I don't know, like a year ago, two years, people were saying, "Oh, LLMs are dead. SLMs are the future. Small models." And we have amazing small models like the Gemmas that recently were like few billion. Remember GPT-3? People said, "Oh, you go, you do no zero-shot learning under 100 billion." No, yeah, we have like 3B models that are so, so, so. That's all amazing, but if you really want to solve big problems easily, adjust to your data and context, yeah, there just doesn't seem to be anything like a really elephant. Well, but they're of course expensive and hard to use and even harder to train.

我认为对听众来说有趣的一点是，也许对前沿领域之外的人来说不太明显的是，新一代硬件带来了什么。我想知道你是否能谈谈这个——显然，对于某些事情，当我们等待 Blackwell 芯片上线时，它们上线了，模型也变好了，但很难说这有多少是因为你现在可以在硬件上做以前做不到的事情，有多少只是时间上的相关性。但也许就谈谈这个，我认为这与我们讨论的架构是否会随着硬件进步而变得更好有关。

One thing I think would be interesting for our listeners is I think something that's maybe less obvious to folks outside the cutting edge is just what's enabled by new generations of hardware, right? And so I'm wondering if you could speak a little bit about to I mean you know obviously it seems like for for for certain things as as you you know uh as we waited for like black wall chips to come online it's like hey they came online and like the models got better and it's always hard to tell like how much of that is is just yes you could now do lots of things on the hardware you couldn't do before how much of that was just timing correlation but maybe just speak to to that and I think it's kind of relevant to this conversation around like are these architectures just going to get better as the as the hardware get better.

硬件在进步，硬件很简单，就是浮点运算和内存访问，对吧？你需要足够快的内存来喂饱浮点运算。但性能这个概念很简单。我最近给自己买了一台个人电脑，配了一块 5090 GPU，心想：“哦，就一块 GPU，放在桌子底下，它能做什么？”于是我做了些测试，结果简直不可思议。5090 大约有 200 teraflops，虽然标称 400，但有些在 BF16 下关闭了。我们研究 Transformer 时用的 GPU 只有 9 teraflops，我们当时有八块 GPU 的机器。按绝对规模算，一台机器实际大约有 70-80 teraflops。现在，我桌子底下的一块 GPU 就相当于五台那样的机器，方便多了。我们当时大概用了十块左右，所以你现在可以在桌子底下用几千块 GPU 做所有 Transformer 研究，就像放在厨房里的普通小塔。才几年，甚至不到十年，它们能做到的事真是惊人。现在我们用 BF16 跑所有东西，当然精度可以更低，尤其是用 MoE 时，推理时能塞进更多。这太棒了。我们运行这些模型的能力大幅提升，也增加了可研究的内容，对吧？你现在可以用很多有趣的方式。它给了你能力，而且还有更多 GPU，对吧？世界上大实验室都在建设，所以你可以用大量非常快的 GPU 训练巨大的模型。Nvidia 跟上了节奏，Google 的 TPU 也跟上了。它们真的在快速提速，数量也在增长，而且这是一个高度可并行的过程，所以我们现在可以更快地训练更大的模型。这太棒了。但我仍然认为更有趣的是我们能做更多研究。我记得加入 Google 时，人们在讨论需要多少浮点运算才能模拟大脑之类的东西，对吧？这是一个非常模糊的问题，因为真正模拟大脑也许不可能，也许仍然非常困难。但几十年来人们一直在做估算，结果总是在 1 到 100 petaflops 之间。我记得当时我们想：“好吧，这需要几十年才能达到。”现在你可以买一块 GPU，这太疯狂了。你有这么一块东西，当然也可以在云上获得带很多 GPU 的机器。所以理论上，你可以用一天时间以一定成本处理相当于人类一年的处理量，对吧？但成本不是数百万，而是几百到几千美元。如果你相信你能找到这个算法，我的意思是，我们是否有数据还是个问题。有些人试图记录儿童的学习过程，对吧？有很多问题。但我们正在达到这样一个水平：大学里的某个人基本上可以运行一个童年。如果你对大脑如何学习有想法，你可以在几天内运行相当于人类 10 年的学习过程，看看它是否有效，也许如果你知道如何评估的话。我认为这比我们能构建这些巨大模型更强大，而构建巨大模型也很强大，因为它们会帮助你实现这一切。我们正在进入一个循环，比如我总觉得 RNN 受限，因为它们非常顺序化。如果你在 torch 中运行它们，它们非常慢，对吧？但你可以写一个特殊的 CUDA 内核让它们变得非常快，但写 CUDA 内核很糟糕，对吧？你真的不想做这个，除非你有一个单元测试确保它和慢版本做完全相同的事，并且有一个智能体为你写它们。它们现在还不太擅长，但已经在做了。你知道，更大的模型可能会好到让你只需说：“用这个硬件发挥最大效能”，几小时后它就出来了。

I mean hardware gets better and hardware, you know, it's easy. It's flops and memory access, right? So, you need memory fast enough to feed the flops. But it's a very simple, can call it performance. And I recently, I got a personal computer. I bought one for myself and I bought a 5090 GPU and it felt like, "Oh, you know, it's one GPU and like you're under your desk. What can this do?" So I did a little bit of some tests and it's just insane to think. So the 5090 it's about 200 teraflops. I mean it says 400 but some are turned off on BF16. So the GPUs we research transformer on they had nine teraflops and we had eight GPU machines. In absolute scaling you could say you know be like 70, 80 teraflops for real on a machine. So now I have under my desk something that's like five of these machines in one GPU, which is much more convenient. But I think we used like around 10 or so, so you could do all of transformer research on this few thousand GPU under your desk that you know you could have in your kitchen like it's just a normal little tower. And oh okay, it's a few years, it's not even a decade though, so it's quite amazing what they can do. And now we run everything in BF-16, but of course you can go lower even in precision, especially with MoEs, then you pack more in inference. This is amazing. So our ability to run these models has dramatically increased and it increases the things you can research, right? You can now run so many interesting ways now. It does give you the ability to just, oh, and also there's more GPUs, right? And the world, like the big labs are building out, so you can train huge models on a huge number of very fast GPUs. And Nvidia has kept the pace, and TPUs at Google have kept the pace. They're really speeding up very quickly, and their numbers are growing, and it's a very parallelizable process, so we can now train much bigger models much faster. That is amazing. I do still think that the even more interesting things is that we can do more research. Like, I remember when I was joining Google, people were talking about how much flops do you need to do something like the brain, right? And it's a very vague question because to really simulate a brain it's maybe impossible, maybe still very much. But people for decades have been doing these estimates and they always fell like somewhere between one and 100 petaflops. And I remember back then we were like, "Okay, so this is going to take like a few decades for us to get there." Now you can buy a single GPU so that is quite insane. You have like this one thing you can, and then of course you can on the cloud get machines with many of them. So potentially you can run like a year of worth of human processing in a day at a cost, right? But it's not a cost of millions, right? It's a cost of hundreds to thousands of dollars. If you believe you can maybe figure out this algorithm, like I mean it's questionable whether we have the data. Some people are trying to do recordings of kids, right? There's a question how well, there's a lot of questions. But we're getting to this level where someone at a university will be basically able to run like a childhood. You know, if you have an idea for how the brain learns, you'll be able to run it in like a few days to the whole like 10 years of learning of a human being and see if it works or doesn't, maybe if you know how to evaluate it. I think this is even more powerful than the fact that we can build these huge models, which is also powerful because they will help you implement this all. And we're getting this loop where I always felt limited with RNNs, for example, because they're very sequential. So if you just run them like in torch, they're very slow, right? But you can write a special CUDA kernel that makes them very fast, but writing CUDA kernels is awful, right? You really don't want to do this, except when you can have a unit test that it does exactly the same thing as your slow thing and an agent that writes them for you. And they're not yet amazing at it, but they're already doing it. And you know, bigger model will probably be so good that you'll just say, "You know, use this hardware as best as it can be," and come a few hours later and here it is.

所以那些瓶颈是因为硬件不符合你的想法。嗯，硬件还是老样子，对吧？它不能随心所欲。它仍然需要并行，但它能做的事情比以前多得多，因为你可以让智能体为它编写内核。

So the bottlenecks that were like because the hardware did not fit your idea. Well, the hardware is still the way it is, right? It can't do anything you want. It still needs to be parallel, but it can do much more than it could do before because you can write like just ask agents to write kernels for it.

是的，这很有趣，因为有些人会说：“天哪，如果没有只有少数地方才有的算力规模，你可能只能做基础研究，但最终还是要看这些技术能否规模化，对吧？”你需要在一个实验室里才能体验到这一点。但听到你对学术界、爱好者以及那些只用单张 GPU 捣鼓的人能在这里做出贡献的机会持乐观态度，真是太棒了。

Yeah, it's so interesting because some people will say, "God, you know, without the scale of compute that exists at only a few places, it's so hard to do maybe you can do basic research, but like ultimately the rubber hits the road on seeing whether these techniques scale, right?" And you need to be in a lab to experience that. But it's awesome to hear your bullishness around the opportunity for academia and hobbyists and folks that are just messing around with single GPUs to be able to contribute here.

嗯，我认为特别是如果你相信应该做一些根本性的改变。你觉得这种情况更可能发生吗？

Well, I think especially if you believe that there are some radical changes that you should do. Do you think it's more likely than not that that's the case?

不，这取决于日子。在我乐观的日子里，我确实相信。研究总是带给我们美好的东西。没有理由认为它不会。但我们现有的技术也似乎工作得如此之好，简直令人震惊。不在这两方面推进将是一个大错误。但幸运的是，有足够的实验室。我感受到了作为学者的兴奋。在加入实验室之前，我曾在学术界，乐趣在于你可以自由发挥你的想法，对吧？你不能大规模扩展，但在较低的规模上——现在这个规模已经不低了——你可以真正地天马行空。你可以尝试完全脱离当前范式的美妙想法，而且你应该这样做。这就是作为研究者的乐趣。很多想法不会成功，有些会在小规模上成功但无法扩展。但在当前 8 GPU 机器的规模上，总会有一些想法在特定规模下有效，但无法进一步扩展。但我认为你现在所处的水平比 5 年前高得多，因为 5 年前这真的是非常小的东西。有很多调整只是非常小规模的调整。现在，即使在一台机器上，你也达到了一个不再是调整的规模。就像，我私下使用 Andre 的 NanoChat。

No, it depends on the day. On my positive days, I do. Research has always brought us beautiful things. There is no reason to think it won't. But then the techniques we have also seem to work so well that it just feels mind-blowing. It would be a big mistake to not push on those two. But luckily there are enough labs. I feel the thrill of being an academic. I was in academia before I joined the labs, and the fun is that you can go wild with your ideas, right? You can't scale up that much, but on the lower scale, which now is not that low, you can go really wild. You can try beautiful ideas that are totally out of the current paradigm, and you should. That's the fun of being a researcher. Not many will work, some will work at small scale and not scale up. But at the scale of current 8-GPU machines, there will always be ideas that work up to a certain scale and don't work further, but I think you're at a much higher level now than 5 years ago, because 5 years ago this was really tiny things. There were a lot of tweaks that were just really small-scale tweaks. Now, even on one machine, you're getting to a scale where it's not tweaks anymore. It's like, I privately use NanoChat from Andre.

是的。这是一个 GPT-2 级别的模型，你在一个盒子上几个小时就能得到，对吧？不幸的是，这些盒子现在有点贵了，但当新一代 GPU 出现时，旧的会变得更便宜。你能做的事情真是令人震惊，是的，并非所有东西都能规模化，但过程中的乐趣是十足的。

Yeah. It's a GPT-2 level model that you get in a few hours on one box, right? These boxes have got a bit more expensive these days unluckily, but when new generation of GPUs will come, the older will get cheaper. It's quite astounding what you can actually do, and yes, not all of this will scale, but the fun you can have on the way is totally.

我想在换话题之前，再问一个研究前沿：多模态模型。我记得你在之前的播客中说过，我们在这方面没有取得太多进展。你现在还这么觉得吗？你对多模态世界的现状怎么看？

I guess one more research frontier, I'd love to get your take on before we shift gears is multimodal models. I think on a previous podcast you said we haven't made a ton of progress there. Do you still feel that's the case? And what's your current state of the union on the multimodal world?

人们当然在取得进展。这可能有点偏向 Japa，但我们在 Transformer 甚至扩散模型中做多模态的方式，最终都是预测周围的每个像素。想想我身处环境中，人类每秒感知到海量信息，但我们的神经元动作很慢，对吧？它们有几百毫秒的处理时间。但我们无时无刻不在接收所有这些感官信息。我们不知何故从这股疯狂的信息流中学习，而不必自回归地预测每个像素。这既更加并行，也更加庞大。所以我觉得我们现有的模型还没有真正处理好这一点。也许需要新的研究。我的意思是，Thinking Machines 最近有了这种多流 Transformer，感觉很容易，对吧？在 Transformer 中，你关注之前的 token。你可以有一堆流来做这件事，对吧？这感觉像是对架构的一个简单调整，但也许这是一个令人惊叹的简单调整。我总是在使用编解码器时，如果忘了什么，我就说出来，但然后它正在执行一些 bash 命令。所以它需要等待我的指令来引导它，这需要 3 分钟，我就想这太不交互了。它应该直接，然后你可以有侧边的东西。又有一堆 hack 让它感觉好一点，但感觉当然所有事情都同时发生在各处。对我们来说，我们同时看和说。这应该是我们模型的行为方式。现在有一个更大的实验室在推动这一点，也许它会实现。但确实感觉我们做多模态时没有进行真正的架构变革来实现并行和吸收。Transformer 目前无法以每秒吸收高分辨率图像的速度工作，对吧？它只是分割它们，而且如此顺序化，以至于根本行不通。这感觉有点不对劲。就像我们不应该把那些小补丁放在那里。它应该直接进入并以某种方式处理。所以我不认为我们在更深层次上已经达到了。但另一方面，感觉有很多人在研究它。

So people are certainly making progress. Maybe this goes a little bit towards Japa, but the way we do multimodal in transformers or even with diffusion models, it's like in the end you predict every pixel of these things around. If you think of me being here in the environment, humans sense an amazing amount of information every second, but we can't act like our neurons are slow, right? They have hundreds of milliseconds process. But we get all these senses everywhere all the time. And we somehow manage to learn from this insane stream without maybe predicting every pixel autoregressively. It's both way more parallel and much larger. So I feel like the models we have have not truly done justice to this yet. Maybe it needs new research. I mean, Thinking Machines recently had this multi-stream transformer, and it feels so easy, right? In a transformer you pay attention to the previous tokens. You could have a bunch of streams that do this, right? That feels like an easy tweak to the architecture, but maybe it's an easy tweak that's amazing. I always, when I work with Codex and I just forget something, I say it, but then it's executing some bash command. So it needs to wait for my thing to steer it and it takes 3 minutes, and I'm like this is just not interactive. It should just, and then you can have the side thing. There's a bunch of hacks again that kind of make it feel better, but it feels like of course everything happens everywhere all at once. For us, we see and talk all at the same time. That should be how our models behave. Now that there is a bigger lab putting pressure on that, maybe it will come. But it does feel like we do multimodal without truly architectural changes to be parallel and absorb. A transformer can't currently at the speed it does absorb a high-resolution image every millisecond, right? It just splits them and is so sequential that it just doesn't work. That feels somehow wrong. It's like we shouldn't be putting these tiny patches there. It should just go in and be processed somehow. So I don't think we have on this deeper level gotten there yet. But on the other hand, it feels like a lot of people are working on it.

是的，那么对于编码来说，我是说，这有那么难说吗？

Yeah, then for coding I mean does it matter all that much harder to say?

完全同意。

Totally.

嗯，我相信它会实现的。我想换个话题，也许聊聊你在 OpenAI 的时光和你的经历。显然过去几年发生了很多事。有几个时刻大家都会想到，所以我很想听听你的看法。也许就 OpenAI 而言，公司有一些非常公开的时刻。我想知道在你任职期间，有哪些艰难的决定真正定义了这家公司？

Well, I'm sure it will come. I'd love to switch gears, maybe just talk a little bit about your time at OpenAI and your journey there. Obviously it's been quite the eventful past years. There are a few moments everyone kind of thinks about, so I'm curious to get your perspective. Maybe just on the OpenAI side, the company's had some very public moments. I'm wondering what were some of the difficult decisions that really defined the company in your time there?

所以我并没有参与最早的那些事情。

So I wasn't there for the earliest things.

我认为在我任职期间，曾有一个重大疑问：是否要转向推理。我觉得公司、领导层和我们所有人勇敢地迈出了这一步，断言推理将与预训练同等重要。我们的模型将是推理模型，它们将被推出。起初，推理模型并不那么健谈，个性更难体现，速度也慢——至今仍有些慢。当时人们会想：我们真的该这么做吗？也许人们只是更喜欢聊天模型。

I think for my time there, there was this big question at some point whether to pivot to reasoning, and I feel it was very brave of the company, the leadership, and all of us to take this plunge and say yes, reasoning will be as important as pre-training. Our models will be reasoning models. They will be launched. At the beginning, the reasoning models were not that chatty. Personality was harder. They were slow, and they still are to some extent. And it was like, should we ever do it? Maybe people just prefer chat models.

嗯。

Yeah.

但 OpenAI 非常擅长下这个重注，并说：是的，我们要推出它，我们要走这条路。我们会想办法管理。同时存在两条模型线，这显然很糟糕，对吧？你希望统一它们。统一花了很长时间，因为一切都在变化。这是一个非常艰难的决定。但如果我们没有坚持推进，就不会有现在所有这些令人惊叹的成果。感觉即使是一些更大的实验室，在强化学习质量上仍难以追赶，所以当你坚定承诺时，确实会有所收获。

But OpenAI was very good at taking this hard bet and saying yes, we're going to launch it. We're going to go this way. We'll try to figure out how to manage. There were two lines of models at the same time. That's obviously awful, right? You want to unify this. The unification took a lot of time because everything is moving. It's a very hard decision. But now we wouldn't have all these amazing things if we didn't push on it. It feels like even some bigger labs still have trouble catching up to the RL quality, so there is some win when you commit to things.

我如今好奇的是，OpenAI 自那以后规模大概增长了 20 倍，变成了一家大得多的公司。所有实验室都是如此。我的意思是，Google 之前就很大，但现在每个人，比如 Anthropic，也变大了。我在 Google 待过很长时间，我认为大公司要下这种疯狂的赌注要难得多，因为你失去的太多，因为你有很多流程。就是更难。我只希望 OpenAI 能保留这种能力，其他实验室也是，因为当前的技术令人惊叹，对吧？它们让我们走得很远。但如果后 Transformer 世界出现火花，这些实验室能抓住机会吗，还是会偏向保守？

I wonder these days, OpenAI has since then grown probably 20 times, become a much bigger company. All the labs have. I mean, Google was big even before, but now everyone, like Anthropic, has become big. Having been at Google before for a long time, I think it's much harder for a big company to take wild bets like that, because you have so much more to lose, because you have processes. It's just harder. I just hope OpenAI retains this ability, and the other labs too, because the current techniques are amazing, right? They get us very far. But if there were sparks of a post-Transformer world, would these labs be able to jump on it, or would they be on the more conservative side?

感觉推理方面有一些早期火花，但显然数据不多。然后这几乎成了一种宗教般的信念：只要我们加倍投入，它就会成功。

It feels that with reasoning there were some early sparks, but obviously not a ton of data. And then it was kind of almost a religious belief that this is just going to work if we double down on it.

我们还没有后继者，至少我不知道。但怀着它会出现的希望，是否需要一个新的实验室来推动它？我认为，如果说有什么的话，OpenAI 擅长下疯狂的赌注。

And we don't have the successor yet, or at least I don't know about it. But with the hope that it will appear, will you need a new lab to push on it? I think if anything, OpenAI is good at wild bets.

看到新实验室的整个趋势显然很有趣，对吧？像 Jerry Tworek 这样的人出来创办公司，说在大实验室之外做这项工作几乎更容易，对吧？然后下一个坚定的赌注。

It's obviously interesting to see this whole trend of new labs, right? And folks like Jerry Tworek spinning out and saying that it's almost easier to do this work outside of a large lab, right? And make one kind of strong convicted bet.

是的，说得有道理。但当你开始看 GPU 数量时，在实验室外面就有点悲哀了。很难获得它们，而且非常昂贵。但 GPU 不是一切。拥有整个生态系统是很好的，对吧？现在既有这些小实验室，也有大实验室。

Yeah, it's a fair point. But then you start looking at the GPU numbers and it's a little sad when you're outside the lab. It's hard to get them and they're very expensive. But GPUs are not everything. It's quite nice to have this whole ecosystem, right? You have both these little labs now and the big labs.

是的，我们会看到的。这太有趣了，因为身处这个 AI 小泡沫中，你清楚地看到竞争激烈，变化正在到来，即使在当前路径上我们也没有穷尽。还有很多技术要做，很多数据和改进，以及更大的模型要训练。然后还有所有这些新事物在酝酿。也许它们还没准备好，但正在用良好的资源积极追求。然后我觉得你走出旧金山，人们对待 AI 就好像它还是 Codex 之前一年那样，永远不会再变了。这是一种错误的对待方式。对我来说，这些编码智能体是如此大的启示，以至于难以释怀。我称之为 AGI，你知道，人们想怎么叫 AGI 都行。我们可能会像超越图灵测试那样超越 AGI，对吧？我们不再真正争论图灵测试了。它通过了吗？没通过？但谁在乎呢？这些用来编码的东西显然是有智能的。

Yeah, we'll see. It's so fun because being in this AI little bubble here, you clearly see that there is a ton of competition, that change is coming, that we have not exhausted even on the current paths. There's still a lot of techniques to do, a lot of data and improvements, and bigger models to train. And then there are all these new things that are bubbling. Maybe they're not ready, but they're very actively pursued with good resources. And then I feel like you step outside of San Francisco and people treat AI basically as if it was from the last year before Codex and would never change again. That is a wrong way of treating it. To me, these coding agents have been such a revelation that it's hard to get over it. I call it AGI, you know, one should call AGI what they want. We may get past AGI the way we got past the Turing test, right? We don't really argue about the Turing test anymore. Is it passed? Is it not passed? But who cares? These things that they code with are clearly intelligent.

在任何有争议的意义上，显然 AI 编码之战现在相当激烈。你认为最终什么会决定这些 AI 编码产品中哪个更好？你如何看待 Codex 和 Claude Code 的下一个前沿？

And in any disputable sense, obviously the AI coding wars are quite fierce right now. What do you think ultimately will determine which of these AI coding products ends up being better? And how do you see the next frontiers for Codex and Claude Code?

我认为编码市场有两个足够大的玩家是好事。我认为更大的问题是它们如何进入其他领域，对吧？我的意思是，编码很棒，对我们很重要，但你可以做很多人的工作。目前，Codex，我试着推荐给一些朋友，但它过去以问题“你的 GitHub 仓库是什么？”开始，这切断了很多人。现在它友好了一些，但仍然叫 Codex。即便如此，人们不太会听到“这是你的会计工具”，对吧？与 ChatGPT 你只需说点什么不同，我认为 Codex 需要一点适应，Claude 更是如此，如果你走编码路线的话。所以我认为有一个问题：你如何将这种能力带给其他职业和地区的人？这可能是更重要的问题。Anthropic 的 Claude Code 基本上是在制作核心代码产品的更友好版本。

I think the coding market is good to have two big enough to have two programmers in it. I think the bigger question will be how well they go to other fields, right? I mean, coding is great and it's important for us, but you could do the work of many people. Currently, Codex, I tried to recommend it to some friends, but it used to start with the question 'What is your GitHub repo?' That cuts off a lot of people. Now it's a little bit friendlier, but it's still called Codex. Even so, people kind of don't hear 'this is your accountant tool,' right? In contrast to ChatGPT where you just say something, I think Codex takes a little bit of getting used to, and Claude even more, if you go on the code side. So I think there is some question: how do you get this power to people in other occupations and places? That may be the more important question. Anthropic with Claude Code and basically making a friendlier version of the core code products.

我当然觉得能力就在那里，对吧？作为一个机器学习从业者，我觉得这些模型显然能做这些事情。它们显然能做 Excel。它们显然能做这个或那个。但然后我像鹰一样盯着它们。你需要投入一定水平的技能才能得到这个。这完全是一种可学习的技能，但我理解人们生活中很忙，不一定想学这个。所以你需要以某种方式让它更平滑。有一些根本性的东西，我认为不会允许你只是让它运行而不加监视。是的，我认为你不想这样做。但另一方面，我认为即使它一开始就超级好，你也不会想这样做，对吧？你需要建立一些信任。

I certainly feel like the abilities are there, right? As an ML person, I feel like these ones obviously can do these things. They obviously can do Excel. They obviously can do this or that. But then I watch them like a hawk. There is some level of skill that you need to put in to get this. This is totally a learnable skill, but I understand that people are busy in their lives and don't necessarily want to learn this. So you need to smooth it in some way. There are some fundamental things that I don't think will allow you to just let it run unwatched. Yeah, I don't think you want to do this. But on the other hand, I don't think you would even want to do this even if it was super good at first, right? You need to gain some trust.

那么问题就变成了，如何说服人们开始投入精力来赢得这种信任？这会带来回报，但在编程方面有一个门槛。为什么你认为 Anthropic 是第一个在编程方面真正成功的？

So the question becomes, how do you convince people to start putting some effort into gaining this trust? It will pay back, but there is a hump on the coding side. Why do you think Anthropic was the first to be really successful on the coding side?

我认为 Anthropic 做了一个非常好的决定，专注于编程。当时 OpenAI 正在做 ChatGPT，这很棒。但我认为 Anthropic 做出这个决定的部分原因是他们无法在聊天领域竞争。他们在其他方面做出了非常好的选择。这要追溯到 AI 经历的这些剧变。你需要押注于不是今天的东西。尽管今天像 ChatGPT 这样的东西很惊人，它是 2025 年最惊人的 AI，但显然不是 2027 年的。也许 2027 年我们会有另一个东西。所以事情变化很快。如果你在别的东西上押对了注，你就能成功。并不是 OpenAI 不做编程；我们做了，这就是为什么它能相当快地赶上。但这并不是重点。这些公司很小。你增长到十亿用户。你有事情要做，所以它就会分崩离析。

I think Anthropic made a very good decision to focus on coding. This was at the time when OpenAI was like, we're doing ChatGPT, and that's great. But I think part of why Anthropic made this decision was that they just could not compete in chat. They made a very good decision on what else to do. This goes back to AI going through these upheavals. You need to put a bet on something that is not what is today. Even though the things today like ChatGPT are amazing, it was the most amazing AI of 2025, but clearly not of 2027. Maybe in 2027 we'll have another thing. So things change quickly. If you put a good bet on something else, you can succeed. It's not like OpenAI didn't do coding; we did, and that's why it could catch up reasonably quickly. But it was just not the focus. These companies are tiny. You grow to a billion users. You have stuff to do, so it falls apart.

你提到了这种紧张关系，一方面要搞定今天有效的东西，另一方面要保持其他领域的开放，这样如果另一个领域出现一线希望，你就可以加倍下注。我想知道你怎么看。显然 OpenAI 现在进入了这个聚焦时刻，你在 Codex 的结果中看到了，也许还有削减 Sora 和其他一些东西。你如何看待这种紧张关系：是真正搞定当下，还是保持这些可能在未来变得非常有趣的火种？

You mentioned this kind of tension between nailing the stuff that's working today and keeping other areas open so that if there's a glimmer of hope in a different area, you double down on that bet. I'm wondering what you make of that. Obviously OpenAI has gone into this focusing moment now, and you've seen it in the results of Codex and maybe slashing Sora and some other things. How do you think about navigating that tension between really nailing the here and now versus keeping these other embers open that could be really interesting down the line?

这取决于文化、规模、资金和视角。众所周知，Google 是那种会保持所有选项开放的实验室。我认为有些人对此相当批评 Google，错过了自己的发明，没有成为从中获利的人。但这对他们来说行得通，因为无论什么好东西出现，都很容易赶上，因为你已经有一个强大的团队在里面。

It's a matter of culture, size, money, and perspective. Famously, Google is the lab that will keep all its options open. I think some people have been quite critical of Google for this, missing their own invention and not being the ones to capitalize on it. But then it works for them because whatever good comes out, it's very easy to catch up since you already have a strong team in it.

你认为他们赶上了吗？我觉得有很多说法声称他们仍然有点落后。

Do you think they've caught up? I feel like there's a lot of discourse claiming that they're still a bit behind.

我认为他们在 ChatGPT 领域已经赶上了。他们还没有赶上……我是说，我不知道你是否看过 Gemini 2.0。我在 IO 之后打开它，我分不清哪个是 Codex，哪个是 Gemini。当然，有很多关于这个的搞笑推文。所以那很棒。我尝试用新的 3.5 Flash 做一些编程，但它就是不行。对我来说它还没有跨过那道门槛，但会的。所以如果你非常广泛，以后需要追赶时会更安全，但你可能不会像 Anthropic 和编程那样立即获胜。你只是第一个搞定它。有实验室去成为第一个搞定它的人，这很好。这很令人兴奋。我觉得这应该是这样的。OpenAI 曾经有很好的下注文化，但现在它也更大了。GPT 有十亿用户，对世界上很多人来说很重要。Google 搜索有三十亿用户。你不希望这些东西受到阻碍和完全破坏。你应该快速前进，但破坏东西并不好。我实际上觉得如果实验室在路上不破坏一切，那是相当好的。

I think they've caught up in the ChatGPT world. They haven't caught up in the... I mean, I don't know if you've seen Gemini 2.0. I opened it after IO and I couldn't tell which one is Codex and which one is Gemini. Of course, there were a lot of funny tweets about that. So that's great. I tried to do some of my coding with the new 3.5 Flash and it just doesn't work. It hasn't crossed the barrier yet for me, but it will. So if you're very broad, it can make it safer later if you need to catch up, but then you may not get the immediate win like Anthropic and coding. You're just the first to nail it. And it's great that there are labs that go and are the first to nail it. That's exciting. I feel like that's how it should be. OpenAI had a good culture of making bets, but now it is also a bigger thing. GPT has a billion users, it's important for many people in the world. Google Search has three billion users. You don't want these things to be hampered and totally broken. You should go fast, but breaking things is not so good. I actually feel like it's quite good if the labs don't break everything on the way.

很多人想知道闭源模型和开源模型之间的差距。感觉有两个不同的东西在向不同方向拉扯。一方面，蒸馏模型相对容易，你看到很多关于中国开源方面的人用闭源提供商做这件事的说法。另一方面，感觉即使在大实验室里，这些模型也越来越大以至于无法服务，所以他们必须在大实验室内部进行蒸馏。你对我们将看到的闭源和开源模型之间的差距有什么直觉？未来几年这个差距是扩大还是缩小？

A lot of people wonder about the gap between closed source models and open source models. It feels like there are two distinct things pulling in different directions. One is it feels relatively easy to distill models, and you've seen a lot of claims around folks doing that on the Chinese open source side with the closed source providers. On the other hand, it feels like more and more of these models even in the big labs are getting too big to serve, so they have to be distilled within the big labs themselves. What's your gut intuition on the gap we'll see between closed source and open source models, and whether that widens or shrinks in the next few years?

这不太容易预测。我觉得更大的模型更好。你可以蒸馏它们，但蒸馏后的模型永远不会那么好。它们很棒，特别是如果你需要模型来赚钱，但它们不如大模型好。我刚才说了 3.5 Flash，我没觉得它能与 5.5 相提并论。也许因为它是蒸馏的 Pro。也许你只需要等待 Pro。即使在实验室内部，我也不记得什么时候用过迷你模型。我认为它们非常好，非常有用。我只是有一段时间没用它们了。每当我用它们时，它们都很好，直到它们出问题，浪费我很多时间，我就回到大模型。所以你可以蒸馏东西。当开源可以蒸馏或不蒸馏时，实验室只是尽量不让你自然地蒸馏一切，但我觉得他们也不会跟你拼命。如果开源有非常落后的模型，那会很可悲，但我不认为有这种风险。有足够的公司，现在还有主权 AI 的概念。我也非常理解，如果你是一个国家，你是否想依赖一个可能宕机的公司？对于警察局或医院用 AI 帮助行政管理，也许你不想依赖一家公司。所以会有很多人想要主权模型，即使它们稍微弱一些，也许任务并不那么难。所以我认为会有足够的激励让开源模型存在，也会有非常好的激励让实验室保持领先。所以人们会继续为此付费。

It is not that easy to predict. I feel bigger models are better. You can distill them, but the distilled models are never quite as good. They're great, especially if you need a model for some money, but they're not quite as good as the big models. I just said the 3.5 Flash, I could not quite feel it's on par with 5.5. Maybe because it's a distilled Pro. Maybe you just need to wait for the Pro. Even within the lab, I don't remember when I have used the mini model. I think they're very good, very useful. I just haven't used them in a while. Whenever I use them, they're fine until they trip and cost me so much time I go back to the big one. So you can distill things. When the open source can distill or not distill, labs just try to not make you distill everything naturally, but I think they also don't fight you to death. It would be very sad if open source had models that are very far behind, but I don't think there is a risk of that. There are enough companies, and now there are notions of sovereign AI. I also very much understand if you're a country, do you want to depend on one company that may just have an outage? For police stations or hospitals running AI to help with administration, maybe you don't want to rely on one company. So there will be a lot of people who want sovereign models even if they're slightly weaker, maybe the tasks are not so hard. So I think there will be enough incentives to have open models that they will exist, and there will be very good incentives for the labs to still keep ahead. So people keep paying for this.

所以，这感觉像是一种会持续一段时间的状态，但你知道，这是 AI 和科技领域的“名言”。你可以说一些话，但结果可能并非如此。我不想做未来预测。当然，但播客如果不试图让你做出预测，那还有什么意义呢？不过，不，这完全说得通。你知道，我们总是喜欢在采访结束时进行一轮快问快答，塞进一堆宽泛的问题。那么首先，我很想知道：在过去一年里，你在 AI 领域改变看法的一件事是什么？

So, it feels like a state that should persist for a while, but you know, it's famous last words in AI and tech. You can say things and they may turn out. I don't want to make future predictions. Of course, but what job is a podcast if not to try and force you into them? But no, that all makes a ton of sense. You know, we always like to end our interviews with a quickfire round where we stuff in a bunch of broad questions at the end. So maybe to start, I just love: what's one thing you've changed your mind on in the AI world in the last year?

嗯，我绝对没想到会这么快出现类似实习生那样的东西，我确实改变了看法。我以前其实不怎么每天和 AI 对话。人们总是问，“那你用 ChatGPT 做什么？”我就说，“嗯，不知道。我昨天问了一个问题，三天前问了一个。”我一直觉得，“我不会和我的电脑说太多话。”但现在我会了，用来处理工作。我也没想到自己会不再用编辑器编程，而现在我不用了。我直接告诉它改代码。是的，那是一个很大的转变。

Well, definitely I did not believe that there would be an intern kind of thing so fast, and I have definitely changed my mind. I actually used to not talk to AI very much every day. People were always like, 'So, how do you use ChatGPT?' And I was like, 'Yeah, I don't know. I asked it one query yesterday and one three days ago.' And I was always like, 'I'm not going to talk to my computer very much.' And now I do about work. I also did not think I'm going to not use an editor for programming, and now I don't. I just tell it to change the code. Yeah, that was a big update.

太棒了。我想，你知道，随着过去几年你更密切地研究这些模型，你对这些模型的存在风险或安全性的担忧是上升了还是下降了？

That's awesome. I guess, you know, as you've worked with these models more closely these past years, have your concerns around existential risk or safety around these models gone up or down?

我认为对我来说没有太大变化。我一直处于不太担心但也不应自满的立场。我仍然觉得，尽管它们现在有编程等技能，但风险是小的，对吧？它们可能入侵我们的一些系统，导致电网瘫痪之类的事情。我仍然觉得这些是我现在应该关注的风险。不是说存在风险不重要。有人思考这个问题是好事。有一些防护措施是好的。最终，我们应该能够关闭这些数据中心，如果我们决定这样做，并对所有这一切保持控制。但即使模型变得更好，我也没有感到任何威胁。

I don't think they have changed very much for me. I was always on the not-too-worried but also we should not be complacent side. And I still feel, with all the skills they have now with programming and so on, I still feel the small risks, right? The risk that they will hack some of our systems, make the grid go down or things like that. I still feel these are the risks I would focus on right now. Not to say that existential risks don't matter. It's good that there are people thinking about it. It's good to have some guardrails. In the end, we should be able to turn off these data centers if we so decide and have control over all of that. But I don't feel any threat from them, even though the models have become much better.

在实验室方面，上周的热门新闻似乎是 Andrej Karpathy 要去 Anthropic 做 RSI，作为那里的一个团队。你怎么看？

On the lab side, it feels like the buzzy news of the last week was that Andrej Karpathy was going to Anthropic to work on RSI, as a team there. What do you make of that?

是的，你知道，我也是这种狂热的一部分，对吧？你可以用这个助手做很多研究，这很惊人，而且你可以让系统的许多部分变得更好，比如更快。所以这当然是真的。但另一方面，当你考虑这些后 Transformer 的东西时，想法的空间是巨大的，不幸的是，大多数都是错的。这就是为什么它被称为研究，对吧？你需要巨大的运气和技巧，但也需要运气才能碰到正确的那个。我们有点觉得也许它就在空气中，但这是研究。可能还要很多年。即使拥有世界上最好的 AGI，它们也只是人类水平，也许是研究员水平，也许会是 10 倍研究员。但多年来，有一个庞大的研究社区试图破解这些东西，但他们没有。所以可能只是非常困难。而且我们对人脑了解甚少，还不能以任何伟大的方式将其与我们的机器学习连接起来。所以，一方面，我认为这很棒。我认为我们会看到当前的东西变得更好。但如果你考虑一个研究突破，它可能只需要一些东西，即使你有了这个，如果你以非常高效的方式搜索，即使你搜索一些有趣的想法，也并不意味着你会找到它，对吧？所有想法的空间如此巨大，以至于即使非常高效的搜索也可能无法到达那里。所以我对这个并不那么担心。

Yeah, you know, I am part of this psychosis, right? It's like you can do so much research with this assistant, and it's amazing, and you can make many parts of the systems better too, like much faster. So that's certainly true. But on the other hand, when you think about these post-Transformer things, the space of ideas is vast and unluckily most of them are wrong. That's why it's called research, right? And you need enormous luck and skill, but also luck to happen upon the right one. We kind of feel like maybe it's somewhere there in the air, but it's research. Maybe years away. And even with the best AGIs of the world, they're like human level, maybe researcher level, maybe they'll be like a 10x researcher. But for years, there was a huge community of researchers trying to crack these things, and they didn't. So it may be just very hard. And we understand very little about the human brain yet, and we cannot connect it to our ML in any great way yet. So I'm like, on the one hand, I think it's great. I think we'll see the current things getting better. But if you're thinking of a research breakthrough, it may just require something that even when you have this, if you're searching in a very efficient way, and even if you're searching some interesting ideas, it still doesn't mean you're going to find it, right? The space of all ideas is so vast that even very efficient searches can just not get there. So I'm not that worried existentially about this.

我发现一件有趣的事是，如果我没记错的话，你所有 Transformer 论文的合著者都去创办公司了。我想知道你是否曾经考虑过这件事。

One thing I found interesting is I think, if I'm correct, all of your Transformer paper co-authors have gone on to start companies. And I'm wondering if that was ever something you thought about.

我当然问过很多很多次。嗯，是的，我很高兴到目前为止我没有这么做。我认为我在 Google 和 OpenAI 的时光都很棒，能待在那里并做这些工作是一种特权。我喜欢技术工作，你知道。每个创办公司的人都想过也许他们不需要花那么多时间在公司工作上，但感觉他们不得不这样做。但有时公司会做出惊人的事情。这是一次精彩的对话。我想确保把最后一句话留给你。有什么想对我们的听众说的，或者想留给他们的想法吗？话筒给你了。

I certainly asked about it many, many times. Um, yeah, I'm very happy that I didn't so far. I thought both my time at Google and my time at OpenAI have been great, and it was a privilege to be there and to be able to do the work. I love technical work, you know. Everyone who started a company has thought maybe they won't need to spend so much time on the company work, and it feels like they had to. But sometimes companies do amazing things. It's been a fascinating conversation. I want to make sure to leave the last word to you. Anything you want to point our listeners to or thoughts you want to leave them with? The mic is yours.

谢谢。

Thank you.

我只想重复我已经说过的话。我觉得现在这个时代，你有强大的 GPU 可以放在桌下，还有编码智能体可以真正帮你把它们推到极限。而所有大东西都在推动 Transformer 的时代，它们很棒，因为很惊人，但还有一丝其他可能性的气息。我认为这仍然是成为机器学习研究员最激动人心的时刻，我想鼓励每个人去尝试自己的想法，向他人学习。如果要说的话，我觉得我们应该发表更多疯狂的东西。当那么多论文都在说，“哦，我们拿一个预训练模型，用稍微不同的方式微调它”时，我总有点难过。我的意思是，这很好，但你不需要追赶已有的东西。你可以做新的事情，即使它们起步更小，即使第一次可能不成功。没有人跟我提过我在《Attention Is All You Need》之前的那篇论文，叫做《You Don't Need Attention》。我前一年有一篇论文说，你只需用主动记忆来替换它。嗯，那不是一个很好的建议，但你需要探索错误的东西，因为它们可能引导你走向正确的东西。这也是模型仍然非常不擅长的，我认为 Jerry 正在推动这一点。模型非常不擅长从完全错误的方向学习，然后真正把它扭转到正确的方向。那是我们人类仍然能做得很好的事情。

I just want to repeat what I said already. I feel this time now that you have powerful GPUs that you can put under your desk and coding agents that can really help you push them to their limits. And the time where all the big things are pushing the Transformers, and great they are because they're amazing, but there is this whiff of possibly other things. I think it is still the most exciting time to be a researcher in machine learning, and I want to encourage everyone to just go and try their ideas, to learn from others. If anything, I feel like we should publish more wild things. I always feel a little sad when so many papers are about, 'Oh, we took a pre-trained model and finetuned it in a slightly different way.' I mean, it's good, but you don't need to catch up with what is there. You can just do new things, even if they'll start smaller, even if maybe it won't work the first time. Nobody talks to me about the paper I had before 'Attention Is All You Need', which is 'You Don't Need Attention'. I had a paper the year before saying you just replace it with active memory. Well, it wasn't quite good advice, but you need to explore the wrong things because they may lead you to the right thing. And this is also what models are still so bad at, which I think Jerry is trying to push. Models are very bad at learning from a totally wrong direction to actually twist it to a right one. That's what we humans can still do very well.

所以，我们应该多做这样的探索。即使失败，也要大胆尝试。我觉得现在如果没有智能体，投入大量个人努力，失败时会非常艰难。但有了智能体，事情反而更容易。所以我想鼓励大家去做研究探索，去失败。只有这样，我们才能找到真正有趣的东西。

So, we should do more of it. We should just do wild explorations even if they fail. I feel now that if you put a lot of your own effort without an agent, it's very hard when it fails. I think with agents it's even easier. So I want to encourage everyone to do research explorations, fail. When it comes to this, this is how we can get to interesting things.

说得太好了。我觉得这正好是完美的结尾。非常感谢你来做客，这期节目很有趣。

I love that. Well, I feel like that's the perfect note to end on. Thank you so much for coming on the pod. This was fun.

非常感谢你的邀请。

Thank you so much for having me.

我是 Jacob Efron，这里是《Unsupervised Learning》播客。在这个节目里，我会和 AI 领域最聪明的人对话，问他们关于模型的最新进展以及对全球商业的影响。我希望大家能感受到，我做这件事非常开心。这是我除了在 Redpoint 做投资人之外，利用晚上和周末做的项目。我们能请到这些出色的嘉宾，全靠像你这样的听众订阅播客、分享给朋友。这才是让这一切运转起来的根本。所以，请考虑这样做。非常感谢你的支持和收听。我们下期再见。

I'm Jacob Efron and this has been Unsupervised Learning, a podcast where I get to talk to the smartest people in AI and ask them tons of questions about what's happening with models and what it means for businesses in the world. As I hope is clear, I have a ton of fun doing this. It's a nights and weekends project in addition to my day job as an investor at Redpoint. But our ability to get these incredible guests on really comes from folks like you subscribing to the podcast, sharing it with friends. It's really what ultimately makes this whole thing work. And so, please consider doing that. And thank you so much for your support and listening. We'll see you next episode.