AI 规模扩展：预训练与强化学习

作为 OpenAI 的研究副总裁，Jerry Tworek 在过去几年参与了 AI 领域许多最大的突破，训练了 01、03、Codex 等模型。他最近离开了 OpenAI，理由是希望研究那些在大实验室里更难推进的领域。我是 Jacob Efron，今天在《无监督学习》播客中，我邀请到 Jerry 来聊聊 AI 领域正在发生的一切以及未来的方向。我们讨论了强化学习、进一步扩展它所需的条件，以及 01 和 03 想法的起源。我们讨论了持续学习和其他研究方法，以及 Jerry 如何看待它们的潜力和需要解决的问题。我们讨论了他对 OpenAI 七年经历的反思、一些关键决策，以及他认为基础模型之间的竞争将如何发展。我们还讨论了优秀研究者的特质，以及为什么研究者会选择在不同的实验室工作。这是一场引人入胜的对话，能够向一位如此接近模型构建、并正在放眼思考下一步的人提问当下最受关注的问题。我想大家会喜欢这次对话。闲话少说，有请 Jerry。Jerry，非常感谢你来做客。我对这期节目期待已久。我觉得你过去几年参与了 AI 领域一些最大的突破，比如 01、03、Codex，现在又有了非常激动人心的下一步。我知道你离开了 OpenAI 去追求新的事业。所以，我真的想不出有谁比你更适合谈论 AI 的现状和所有潜在的发展方向。

As VP of research at OpenAI, Jerry Tworek was a part of many of the biggest advances in AI these past years, training models like 01, 03, Codex. He recently left OpenAI, citing that he wanted to pursue research areas that were harder to pursue within a big lab. I'm Jacob Efron, and today on Unsupervised Learning, I got to sit down with Jerry to talk about everything that's happening in AI right now and where the space is headed. We talked about reinforcement learning, what's required to further scale it up, and the origin of the ideas for 01 and 03. We talked about continual learning and other research approaches and how Jerry is thinking about their promise and the problems to solve there. We talked about his reflections on seven years at OpenAI and some of the key decisions that had to be made, as well as how he sees the race between the foundation models playing out from here. And we talked about what makes good researchers and ultimately why researchers choose to work at different labs. It's just a fascinating conversation to get to ask everything that's top of mind in the space today with someone who's been so close to building these models and is really zooming out to think about what's next. I think folks will really enjoy this conversation. Without further ado, here's Jerry. Well, Jerry, thanks so much for coming on. I've been super excited for this episode for a while. I feel like you've been behind some of the biggest breakthroughs in AI over the past years, you know, 01, 03, Codex, and now have some really exciting next steps. I know you've left OpenAI to pursue something new. So, I really can't think of a better person to talk about the state of where we are with AI today and all the potential future directions.

谢谢，非常感谢。很高兴来到这里。AI 是我最喜欢谈论的话题之一。那我们就开始吧。

Thank you. Thank you very much. Very happy to be here. And AI is one of my favorite topics to talk about. So let's do that.

太棒了。我想从你开始，你显然在 OpenAI 引入推理模型和扩展强化学习方面发挥了关键作用。所以我们也许可以从现有的 Scaling 范式开始，我想听听你的看法。你认为我们在预训练和强化学习这两个现有方向上能扩展多远？从模型能力来看，这能带我们走到哪里？

Amazing. Well, I think I'll start with you obviously played a key part at OpenAI in introducing reasoning models and then scaling reinforcement learning. And so maybe we'll start with the existing scaling paradigms and I'd love to get your temperature. How far do you think we have scaling the current vectors we have of pre-training and RL? Like where does that get us from model capabilities?

它肯定能带我们到某个地方。问题是我们怎么称呼那个地方？怎么命名？我想你可以告诉我。但有一个非常真实、对大多数从业者来说相当惊人的事实：Scaling 的好处是真实、可预测且美好的。每当我们扩展预训练，我们就会得到更好的预训练模型，它们从根本上更了解世界，更好地理解语言，并从根本上构建了周围一切的语言世界模型。同样，扩展强化学习会让模型更好地获得我们想要的技能。在很多方面，这两种情况都是你训练什么就得到什么。如果你想做下一个词预测，你可以大量预训练模型，得到在下一个词预测上非常出色的模型。如果你想要一组特定的技能，你训练强化学习模型，然后它们就会在你训练的任何事情上变得非常出色。基本上没有限制。现在每个人都知道，如果你在乎一项技能，你只需对该技能进行强化学习，就能得到所有技能。这太棒了。就是这么简单，而且有效。人们有时犹豫的地方，卡住的地方是：这些模型如何泛化？这些模型在训练范围之外表现如何？模型如何处理预训练语料库中没有的知识？可能不行。模型如何完成你没有进行强化学习的任务？可能不太好。这些基本上就是 AI 领域剩下的问题，因为我们训练的东西已经变得非常擅长了。

It definitely gets us somewhere. The question is how do we call that place? How do we name it? I figure you could tell me. But it is a very real and for most practitioners pretty striking thing that the benefits to scaling are real and predictable and nice. And whenever we scale up pre-training, we get better pre-trained models which fundamentally know more about the world, which fundamentally understand language better and fundamentally build the linguistic world model of everything around them. And in the same way, scaling up reinforcement learning makes the model better at acquiring skills that we want to do. In many ways, in both those cases, you get what you train for. If you want to do next-token prediction, you can pre-train models very heavily and get really great models in next-token prediction. If you want a specific set of skills, you train reinforcement learning models and then you get them really great at whatever you are training for. There are basically no limits. Everyone knows these days that if you care about a skill, you just do reinforcement learning on that skill and you get them all. That is excellent. It's kind of that simple and works. What people hesitate sometimes, where the moments of holdup are, how do those models generalize? How do those models perform outside of what they've been trained for? How do models do knowledge that is not in a pre-training corpus? Probably not. How do models do tasks that you don't reinforcement learning on? Probably not that great. And these are basically the remaining questions in the world of AI, because what we train for we are getting really good at.

感觉这里有两种思想流派。一种是，看，我们在这个强化学习范式中还处于早期。随着我们扩展它，我们会开始看到更多泛化的迹象。所以也许仅凭这两个 Scaling 向量就能带我们达到我们对 AI 模型的大部分期望。第二种是，我们确实需要一些新的、不同的东西来继续。你在某种程度上倾向于哪一种？

It feels like there's kind of two schools of thought here. One is, look, we're still early in this reinforcement learning paradigm. As we scale this up we'll start to see more signs of generalization. And so maybe these two scaling vectors alone take us to most of what we want from AI models. And then the second being, really we need something new and different to continue. Where do you kind of fall on that in some way?

我几乎认为这是一个经济学问题。很明显，如果你喜欢 Scaling 在很多方面意味着添加数据，那么没有数据 Scaling 就不太有效。所以如果你添加更多你想要的数据，你就会得到更好的模型来做你想做的事。这就是你现在每个季度看到的，每个实验室都在发布更好的模型。这主要意味着它们同时扩展了更多算力。更重要的是，它们向模型添加了更多数据，而最重要的是，这些数据是针对之前模型不擅长的方面。这是一种训练越来越好的模型的非常强大的方法。在这种情况下，显然通过迭代，如果你不断添加数据来让模型获得你想要的技能，你会得到一个能做所有事情的模型。但这个循环在某些方面很慢。而根本问题是可以更快，因为我相信，在当今训练模型的方法范式下，如果我们不断添加我们希望模型擅长的数据，它们就会擅长。这里面有一定程度的泛化。但主要问题是：是否存在某种研究，能用更少的数据获得更多结果，或者更根本地，从它们迄今所见所学中更好地泛化？我知道我们稍后会谈到一些未来的潜在方向。

I almost think it's a question of the economics. It is pretty clear that if you like scaling means in many ways adding data, that scaling doesn't work that much without data. So if you add more data that you want, then you get better models doing what you want. It's what you see right now every quarter, every lab releasing a better model. It mostly means they both together with scaling more compute. More importantly, they added more data to the model and most importantly it was data targeted to what the previous model was bad at. It's a really powerful methodology of training better and better models. In that case, obviously iterating through it will give you a model to do everything that you want if you keep adding the data to do the skills that you want the model to do. But this loop is slow in some aspects. And then the fundamental question can be faster, because I believe it is very natural that with the paradigms of the methods of training models today, if we keep adding the data that we want the models to be good at, they will be good at it. And there is some amount of generalization within it. But the main question is: is there some kind of research that would give us more results with less data, or more fundamentally better ways of generalizing from what they've seen and learned so far? And I know we'll hit on some of those future potential directions later.

也许在强化学习领域，为了给听众设定背景，你如何描述今天的状况，比如什么有效、什么无效？你自己的思维模型是什么？是不是只要有数据就行？我知道很多人谈论容易验证的空间和更难验证的空间之间的区别。

Maybe in the reinforcement learning world, just to kind of set the context for our listeners, how do you characterize today like what works and what doesn't? You know, what's your own mental model? Is it literally anything we have data for? I know a lot of people talk about the difference between spaces that have kind of are easy to verify versus those that are a bit more difficult.

你个人对今天能有效应用强化学习的任务有什么看法？

What's your own mental model for what we can do reinforcement learning on today effectively?

关于容易验证和难以验证的问题，往往与能否获得质量信号密切相关。如果你愿意，我们在 OpenAI 训练那些旨在成为优秀作家的模型方面已经取得了不错的进展，强化学习确实可以在很多有意义的事情上发挥作用。但要知道什么好什么不好非常困难，或者你需要等待很长时间。比如写一本书，有一些简单的方法可以判断它好不好，但大多数情况下，你需要尝试卖掉这本书，看看有多少人愿意读、愿意买。有时这甚至也不是好信号，因为很多评论家会说：“这本书这么好，只是没人买，因为营销没做好。”那么我们如何对写一本好书进行强化学习呢？很难说人们是如何学会写一本好书的，这非常非常难说。类似地，创办公司也是如此：很多公司早期就成立了，我们如何知道哪些是好是坏？只有五到十年后，我们才能看到一些企业家成功了，创办了非常成功的公司，而另一些失败了。这是因为早期采取的行动是好是坏，还是运气使然？直接对此进行强化学习是非常非常困难的事情。任何你能获得反馈的事情，你都可以用来做强化学习。

The question about the easy and hard to verify often comes very close to what is easy to even get a signal on the quality of. If you'd like, at some moment we've made pretty good progress at OpenAI at training models that are meant to become great writers, and it is possible reinforcement learning can really be done on a lot of great things sometimes meaningfully. It is a very hard thing to know what is good or what is not good, or you need to wait a lot of time. If you write a book, there are some easy ways to tell if it's a good book or not. But most likely, getting a signal on that you have to try to sell this book and see how many people will like to read it, how many people want to buy it. And sometimes even that is not a good signal because a lot of critics say, 'Oh, this book is so good, just no one bought it because marketing didn't work well.' So how can we do reinforcement learning on writing a good book? It's hard to say how people learn to write a good book. It is a very, very hard thing to say. Similar with starting companies: there are lots of companies started early on, and how do we know which ones are good or not? Only five to ten years down the line will we see that some entrepreneurs have succeeded and started really successful companies, and some of them failed. Was it because of some actions started early on were good or bad, or maybe it was a stroke of luck? Doing reinforcement learning on that very directly is a very, very hard thing to do. Everything where you get feedback of any kind, you can use that to do reinforcement learning on.

人们对你们开发的模型在编程竞赛、数学竞赛等方面取得的结果感到震惊。我想很多人仍在试图理解他们的直觉：大多数任务看起来像编程和数学，还是像写书、创办公司这类很难构建奖励函数且需要大量测试的事情？比如会计、医学或法律。你直觉上觉得这些更接近前者还是后者？

People have been blown away by the results that the models you worked on have gotten in coding competitions and math competitions and other things. And I think a lot of people are still trying to figure out their intuition for: do most tasks look like coding and math, or do most tasks look like books or starting companies or things that are actually very hard to build rewards and also just test tons of times? You know, maybe we'll take things like accounting or medicine or legal. Do you have a gut instinct on if those are more like the former or the latter?

我认为根本问题在于判断你是否做得好有多容易。因为可以说，即使对人类来说，判断你是否写了一本好书也非常困难。对于许多其他工作，比如你是一名会计经理，能够判断哪个会计做得好，如果有规则，那么利用这些规则你几乎可以训练任何东西。对于医学，我一直在思考外科医生，显然有很多规则，也显然有反馈信号：病人是否在手术中存活下来？这是一个非常好的成功标准。但有时，这是一种非常有趣的方式，真正专家级的、技术高超的人类会违反规则，做一些与以往不同的事情，并因此成功。如果一位外科医生根据经验认为需要以不同于以往所有手术的方式来做这个特定手术，他们就会违反既定做法，做一些全新的事情，从而突然成功，使手术成功。我认为模型也能做到这一点，只要有足够的时间和尝试能力。问题在于今天的模型需要多长时间才能真正做到这一点。

I think the fundamental question is how easy it is to tell if you did a good job or not. Because arguably, even for humans it's a very hard thing to tell if you wrote a good book or not. With a lot of other work, if you can be, let's say, a manager of accountants and be able to tell which accountant is doing a good job or not, if there are rules, then with those rules you can train whatever you want really. For medicine, I've been thinking a lot about surgeons, and there clearly are a number of rules, and there clearly is a sense of feedback: has the human survived the operation? And that is a success criteria that is very good. But sometimes, and this is a very interesting way, really expert, really skilled humans go against the rules to do something differently that has ever been done before, and they succeed through that. If a surgeon, from their experience, sees that they need to do this particular operation differently than all the others before, they go against the established practice and through that they do something completely new that suddenly succeeds and makes the operation successful. I think the models would be able to do that too, with some time and with enough ability to try. The question is how much time would it take today's models to really get to something like that.

当你思考需要解决哪些问题才能让强化学习对人们关心的任务越来越具有泛化能力时，也许可以帮助我们的听众理解这些其他 RL 领域的下一个前沿是什么。

As you think about the problems that need to be solved to make RL more and more generalizable for tasks that people care about, maybe help our listeners understand what are the next frontiers for these other domains of RL.

是的，我认为泛化从根本上说是模型的一个属性。故事是这样的：每当你训练时，你真正影响的是你的训练目标，仅此而已。而训练目标，你得到的是你训练的东西，问题在于你还免费得到了多少其他东西。有些学习方法，甚至下一个词预测，泛化能力很差。比如最近邻分类，一个非常经典的机器学习算法；理论上你可以用它解决任何机器学习问题，但它泛化能力很差，因为它构建的世界表示非常简单。神经网络的神奇之处在于，通过大规模训练进行大规模预训练，它们学会了关于周围世界非常有趣、非常有用的表示。有时这看起来像是免费得来的。为什么通过在大规模互联网上训练一个大型 Transformer，它就能很好地理解世界上的许多概念？这来自哪里？这来自 Transformer 和大量参数，我们通过梯度下降反复锤炼它们。这就是我们从这种模型中获得的泛化能力。问题是：是否存在一种不同的模型能泛化得更好？几乎肯定有。问题在于它是什么样的？

Yeah, generalization I think fundamentally is a property of a model. So the story is: whenever you train, you really affect your training objective, and that's kind of it. And the training objective, you get what you train for, and the question is how many other things you are getting for free. There are clearly methods of learning various bits, even next-token prediction, that generalize poorly. There is nearest neighbor classification, a very classic ML algorithm; you theoretically can use it for any ML problem, it just generalizes poorly because it has a very simple representation it builds of the world. Neural networks, how they work magically, is that through large-scale training for large-scale pre-training, they manage to learn very interesting, very useful representations of the world around it. And sometimes it may seem like we get it for free. Why has it happened that through training a large transformer on the internet, it learns really to understand well a lot of the concepts around the world? Where does it come from? It's a magical part that comes from a transformer and a lot of parameters that we hammer over with gradient descent repeatedly. And this is the type of generalization we get from this model. There's a question: is there a different model that will generalize better? And almost surely there must be. The question is what does it look like?

我之前听你说过，在从事 Scaling（规模扩张）强化学习的工作后，你对 AGI 某些方面的预期时间线可能有所延长。为什么？

I've heard you talk before about how you kind of had updated your timelines maybe to be a bit longer for some of the different aspects of AGI after working on scaling RL. Why was that?

我过去确实是一个非常乐观的人，认为我们对模型进行强化学习就能实现 AGI。也许我们已经做到了，也许它已经是 AGI 了。AGI 的定义非常个人化。

I was definitely a very optimistic person in the sense of thinking we do reinforcement learning on our model and we'll get to AGI. And maybe we did, maybe it already is AGI. The definition of AGI is very personal.

没错，这非常个人化，而且在某种程度上，它就是我们还没有的一切。

Exactly, it's a very personal thing, and in some ways it is everything we still don't have.

所以，那些能解决几乎所有奥赛、任何竞赛问题的模型，那些真正在证明前人未解的新数学问题的模型——我们每周都能从最新的 GPT-5.2 中看到有人实现这一点的例子。这是 AGI 吗？很多人会说是。与此同时，我非常喜欢使用编程模型，但它们仍然会犯错。在某些地方，它们做的事情可能需要我花非常非常长的时间才能做好。所以它们可以成为编程工作的强大倍增器。

So, the models that can solve basically any olympiad, any competitive problem, the models that are meaningfully proving new mathematical problems that no one has solved before — we get with the latest GPT-5.2 examples of that every week, of someone achieving that. Is it an AGI? Many people would say yes. At the same time, I'm a big fan of using coding models, and they still make mistakes. In some places they do things that could take me really, really long time to do well. So they can be an extreme force multiplier on doing programming work.

但与此同时，模型显然也有失败的地方。我认为当前模型最大的局限是，一旦它们失败，你很快就会感到无计可施。有时你可以来回粘贴错误信息，比如“嘿，亲爱的模型，这没成功，再努力试试”，有时我们需要说些鼓励的话。但根本上，模型并没有一种很好的机制来根据失败更新自己的信念和内部知识。这大概是我最大的认识更新。除非我们能让模型自己克服困难、在解决问题时摆脱困境，否则我不认为这能叫 AGI，因为那种无助感——当你遇到现有模型无法解决的障碍时，它们就是解决不了，要么换一个模型，要么你自己动手。智能总能找到出路，智能会不断尝试问题直到解决，而当前模型做不到这一点。

But at the same time, there are clearly places where they fail. And I would say the biggest limitation of the models today is that if they fail, you get kind of hopeless pretty quickly. Sometimes you can do a bit of back and forth between pasting an error message. 'Hey dear model, this didn't work. Like try harder.' Sometimes we need to do words of encouragement. But fundamentally, there isn't a very good mechanism for a model to update its beliefs and its internal knowledge based on failure. This is probably the biggest update on me. Unless we get models that can work themselves through difficulties and get unstuck on solving a problem, I don't think I would call it AGI because of this feeling of hopelessness that if you hit a wall with existing models and they cannot solve it, they just cannot solve it. They either try a different model or do it yourself. Intelligence always finds a way. Intelligence works at the problem and probes it until it solves it, which the current models do not really do.

嗯，这正好可以过渡到其他一些研究领域，也许超越了纯粹的预训练和强化学习的 Scaling。你谈到的很多内容听起来类似于持续学习，这是近来人们公开讨论得越来越多的一个话题。我想知道，从最高层面为我们的听众考虑，你认为需要解决哪些问题才能让持续学习真正成为可能？

Well, it's kind of a great transition to some of the other research areas, maybe beyond the pure scaling of pre-training and RL. A lot of what you're talking about sounds similar to continual learning, a topic that people have been discussing more and more in public these days. I'm wondering, at the highest level for our listeners, how do you think about the set of problems that need to be solved to make something like continual learning actually possible?

是的。最核心的一点是，能够持续训练模型意味着模型不会崩溃、不会进入奇怪模式或出错。深度学习模型训练失败的方式有很多种。如今大实验室的很多工作都是让这些模型保持在“正轨”上，保持训练健康。这本质上是一个脆弱的过程，需要付出努力才能顺利进行。如果不努力，它就会爆炸，最终你得不到一个好模型。这与人类的学习方式根本不同。我认为人类学习在某种程度上更具反脆弱性，它能在学习过程中自我摆脱困境。对于使用强化学习的模型，我常常惊叹于人类很少会崩溃、开始胡言乱语，或者大脑在接收到新信息后陷入某种奇怪状态。而 AI 模型却经常如此，研究人员正在从理论和实践两方面寻找应对方法。我认为训练过程的这种根本鲁棒性是持续学习所必需的。

Yeah. At the very core, being able to continuously train a model means being able to have the model not collapse and not go into a weird mode or error. There are many ways in which training a deep learning model fails horribly. A lot of the work happening in the big labs these days is about keeping those models so-called 'on the rails' and keeping the training healthy. It's fundamentally a fragile process. It is a process that you have to make an effort to go well. And if you don't make that effort, it explodes. You just don't get a good model in the end. That seems fundamentally different to how humans learn. I think human learning is much more anti-fragile in a way. It can get itself unstuck throughout learning. Models with reinforcement learning, I've often marveled at how infrequent it is for humans to crash out, start talking gibberish, and then the human brain, after getting some new information, to spiral into some weird state. AI models do that pretty frequently, and it's something that researchers try to find both theoretical and practical solutions to fight. I think this fundamental robustness of a training process is necessary for continual learning.

关于持续学习的想法，有多少感觉是已经存在一段时间或已被讨论过的，又有多少是完全崭新的研究问题？

How much of the ideas for continual learning feel like they have been around for a bit or have been discussed, versus being an entirely new research problem?

作为研究者，值得问自己的主要问题，也是我经常问的，就是：为什么它还没有被解决？这必须是头号问题。当有人开始研究持续学习这类显然尚未解决的问题时，主要问题就是为什么没有。到目前为止，哪条特定的路径还没有人走过？世界上有很多非常聪明、有很多绝妙想法的研究者，但至今没有人真正攻克持续学习。有很多假设，但我认为一个根本原因是，这很可能需要在规模上——至少在一定规模上——进行研究。目前世界上资金充足的研究实验室数量有限，只能做有限的研究和项目。这很可能是其中一个真正重要的原因。如果存在可以在小规模上进行并从根本上有所发现的研究，那它要么非常复杂，要么理论上非常困难，要么就需要只有极少数人才能获得的模型和算力水平。而且很可能，那极少数实验室还没有朝某个特定方向前进，因为它们忙于其他事情。

The main question worth asking yourself as a researcher, and something I ask a lot, is: why hasn't it been solved yet? That must be the number one question. When someone starts working on problems like continual learning, which clearly hasn't been solved yet, the main question is why not. What particular path has no one taken so far? There are many researchers in the world who are very smart and have a lot of brilliant ideas, and so far no one has really cracked continual learning. There are many hypotheses, but one fundamental one I think is that most likely it is research that needs to happen at scale, at least at a certain scale. There are only so many well-funded research labs in the world right now that can only do so much research and so few research projects. That is most likely one of the really big reasons. If there is research that you could do at a small scale and fundamentally discover something, it's probably either very complex or very theoretically difficult, or it just requires models and levels of compute that are available to very few. And it's very likely that the very few labs didn't go in a particular direction yet because they were busy doing other things.

我之前听你谈到过这样一个观点：AI 中有一些想法，时机未到，但仍然是好想法。我们确实在强化学习上看到了这一点——在拥有大型预训练模型之后，它变得有效得多。所以听起来你的直觉是，有一些非常好的想法，如果真正大规模应用，会对这个领域的问题非常有帮助。

I've heard you talk before about this idea that there are ideas in AI whose time isn't right, but they're still good ideas. We certainly saw this with reinforcement learning becoming much more effective after having large pre-trained models on top of it. So it sounds like your intuition is that there are some really good ideas out there that, if actually applied at scale, would be really helpful toward this domain of problems.

我确实这么认为。

I definitely think so.

我之前听你谈到过，各个实验室真的在趋同，研究非常相似的东西。我不知道过去两三年这是否已成为常态，但似乎当你领导很多工作时，那是一个真正的新事物，很多实验室都措手不及。

I've heard you talk before about the labs really converging on working on pretty similar stuff. I don't know if that feels like it's been common over the past two or three years, but it seems like when you were leading a lot of the work, that was a genuinely new thing that a lot of the labs were caught flatfooted on.

请谈谈过去一年左右发生的这种趋同现象，你对此感到惊讶吗？

Talk a little bit about this convergence that's happened maybe over the last year and was that surprising to you?

是的，即使在用强化学习训练模型时，也存在一个众所周知且有充分记录的权衡：探索与利用。你会想，什么时候是尝试与当前做法不同事物的合适时机，什么时候是优化已知做法的合适时机？这是一个没有真正解决方案的权衡，因为你不知道未知是什么，比如探索是否成功。从根本上说，除非你了解你所做事情的全局，否则这非常困难。所以我不认为这有什么根本性问题，但我记得有人曾问我：为什么所有商用飞机看起来都一样，尽管有几家公司在制造它们？因为最终，这是经济上最高效的设计。在某种程度上，今天所有实验室所做的，经济力量在其中非常强大。如果你想竞争，你需要以最低价格拥有最好的模型，而竞争在客户可以随时切换方面非常高效。在很多方面，客户才是赢家，但这是推动实验室以相当可预测的方式追求更高效率和更好模型的一个因素。然后还有一个探索与利用的问题。我们是否应该扬帆出海，看看外面有什么？我们是否应该训练一个完全不同的模型？这可能会分散对当前事物改进或效率提升的注意力，但也许有 10 倍更好的东西，也许有 100 倍更好的东西。核心在于信念和决心：我们愿意在多大程度上尝试那些其他事物，而不是固守现状。

Yeah, even when training models with reinforcement learning, there is this well-understood and well-documented trade-off of exploration versus exploitation. You wonder when is the right time to try different things than you've been doing so far, and when is the right time to try to optimize very well what you already know how to do. It's a trade-off that has no real solution because you don't know what the unknown is, like whether the exploration is successful or not. Fundamentally, unless you know the landscape of what you are doing, it's a very hard thing to do. So I don't think there's a fundamental question about it, but I remember someone telling me at some point in my life: why do all the commercial planes look the same, even though there are a few companies building them? Because in the end, this is the most economically efficient design. In a way, what all the labs are doing today, the forces of economics are fundamentally very strong. If you want to compete, you need to have the best models at the lowest price, and the competition is pretty efficient there in terms of customers being able to switch whenever they want. It's really the customers that are winning in many ways, but that is one thing that drives the labs to go for higher efficiency and to produce better and better models in a pretty predictable way. And then there is a question of exploration versus exploitation. Should we try to sail over the sea and see what is out there? Should we try to train a model that is completely different? It would probably lose some focus on trying to get the current thing better or more efficient, but maybe there's something 10 times better, maybe something 100 times better. There is a question of belief and conviction at the heart of it: how much do we want to try those other things versus not.

说到你的观点，我的意思是，显然有一条非常清晰的路径：向强化学习和不同领域添加越来越多的数据，从而改进模型以完成经济上有价值的任务。每个实验室如何继续改进其基础模型，这几乎有一条清晰的路线图，这可能使得出去下大赌注变得更加困难。而当预训练似乎放缓时，出去探索各种不同事物反而更容易。

And to your point, I mean, obviously there's such a clear path forward on adding more and more data to RL and different domains, and that improving models for economically valuable tasks. There's kind of a clean roadmap to how each of the labs can continue to improve their underlying models, which maybe makes it harder to go out and make that big bet. Whereas when it felt like maybe pre-training was slowing down, it's easier to go out and explore a bunch of different things.

是的，我认为历史上也有不同的时期。有时人们对此有更大的兴趣，也有更多自由去探索不同维度。竞争格局越激烈，就越困难，因为这几乎变成了一种囚徒困境：尝试做不同的事情会让你面临失去市场份额给其他参与者的巨大风险。

Yeah, I think there are also just different times in history. Sometimes there's more appetite for it and a little bit more freedom to explore various dimensions. The more competitive the landscape becomes, the slightly harder it becomes because it almost becomes something like a prisoner's dilemma situation where trying to do things differently exposes you very heavily to losing market share to other players.

是的。你认为如果下一个重大突破在其他地方被发现，这对实验室来说真的重要吗？我的意思是，让我印象深刻的一点是这些进步的传播。显然，你们在推理方面是 01 的先驱。现在有几个实验室拥有出色的推理模型，我几乎想知道，如果突破发生在别处，实验室是否也能安然无恙，因为这些想法会扩散，最终他们能够将其整合到现有业务中。

Yeah. Do you think it actually matters for the labs if the next big breakthrough is discovered there? I mean, one thing I've been struck by is just the dissemination of a lot of these advances. Obviously, you were kind of the pioneer on the reasoning side with 01. There are a few labs now that have great reasoning models, and I almost wonder if the labs would be just fine if the breakthrough happens somewhere else because these ideas diffuse and eventually they'll be able to plug it into the existing business.

想法会扩散，这是好事，但同时，领先地位带来的先发优势不容小觑。无论如何，我们已经看到，如果你认为 OpenAI 根本不可能成功，但它确实成功了，因为它在大规模预训练 Transformer 方面比任何人都做得好得多，这一领先地位使其成为世界历史上最大、最成功的公司之一。同样，因为 OpenAI 是第一个弄清楚如何做大规模强化学习的，我认为很长一段时间以来，直到今天，它拥有所有实验室中最好的强化学习研究项目，这使其能够比大多数后来才追赶的实验室做得更好、更有雄心。虽然想法会扩散，但领先地位可以是一个非常强大的东西，如果保持下去，可能会永远伴随你。在很多方面，我一直在读一本关于半导体制造的书，其中许多核心初始发明是在美国完成的，然后通过这种方式逐渐传播到世界各地。但同时，也有一些领先的时刻和空间是许多其他国家永远无法匹敌的。对于那些早期押注并真正努力建设的国家来说，优势会随着时间的推移不断累积。并不是只有一个国家拥有成功的半导体业务，但也不是每个国家都有。它并非无处不在。每当业务发生转变时，总会有新的参与者成功，有些会失败，一些老公司会留下来并设法转型，一些老公司会消亡。这就是进步的达尔文主义部分。

Ideas diffuse, and that's a good thing, but at the same time, the lead it gives you to be the first is not something to really discount. In any case, we have seen that if you believe that fundamentally there would be no place in the world for OpenAI to succeed, and it did succeed because it went into pre-training transformer at large scale much better than anyone else, and that lead made it one of the largest and most successful companies in the history of the world. In the same way, because OpenAI was the first to figure out how to do large-scale reinforcement learning, I think for a long time and still today, it has the best reinforcement learning research program out of all labs, which allows it to do things better and more ambitiously than most other labs that had to catch up much later. While ideas diffuse, the lead can be a very powerful thing that, if maintained, will stay with you potentially forever. In many ways, I've been reading a book about semiconductor manufacturing, where many of the core initial parts of the invention were done in the United States, and through that, they slightly disseminated through the world to various places. But at the same time, there have been moments and spaces of lead that a lot of other countries couldn't ever match. The continued compounding of advantages over time for some of the countries that bet on it early and really started hard to build it. It's not that there's only one country having a successful semiconductor business, but also not every country. It doesn't exist everywhere. There always is a place whenever there is a business shift. There are some newcomers that will be successful, some that will be unsuccessful, some old companies that will stay and manage to turn themselves around, and some old companies that will die. That's the Darwinian part of progress.

是的。我觉得消费者和企业总是记得第一个带给他们某种神奇体验的公司。当然，你们在 ChatGPT 上就体验到了这一点。你身上有一件很吸引人的事：你显然在强化学习方面取得了所有这些令人难以置信的进步，并帮助开创了许多这方面的工作，而强化学习的进展仍然活跃且持续，我们在各个领域都取得了很大进展，但你却决定离开 OpenAI。我想你提到了一些你想探索的不同研究领域。

No. And I feel like consumers and enterprises always remember the first company that introduces them to some pretty magical experience. I mean, certainly you guys experienced that with ChatGPT. One thing that's fascinating about you is you obviously made all this incredible progress in RL and helped pioneer a bunch of this, and RL progress is still alive and well and going along, and we're making lots of progress across domains, and you decided to leave OpenAI. I think you cited kind of different research areas you wanted to explore.

我很好奇，你是什么时候开始意识到这可能是你想做的事？最终又是如何做出决定的？

I'm curious like when did you begin to know that might be something you wanted to do? And how did you ultimately make the decision?

这绝对不是一件很快发生的事。它是慢慢在一个人心中滋长的。离开 OpenAI 并不容易，因为我在那里有很多朋友，有很多共同的回忆，我的人生很大一部分都建在那里。很长一段时间里，我真的很努力想让它继续下去，也尝试了各种不同的方式。但到了某个时刻，特别是作为一名研究员，如果你醒来后发现，无论出于什么原因，你不再热爱你的工作，不再对你所做的事感到极度兴奋，那就是时候去探索、去做点别的了。作为一名研究员，如果你不是 100% 兴奋、不是满怀热情地投入，你基本上不可能做出最好的工作。在 OpenAI 有很多天，我对自己的工作充满了无限热情，相信自己正在做完全正确的事，但到了最后阶段，越来越难了。长话短说就是这样。

It's definitely not something that happens very quickly. It's something that slowly grows in a human. And OpenAI is not an easy place to leave because I have many friends there, a lot of shared history, I've built a lot of my life there. For a long time I really tried hard to make it work and tried to see what are the various places to do it. But at some moment, specifically as a researcher, if you wake up and for any reason figure out you don't love your work anymore, you are not incredibly excited about what you are doing, then it's a good moment to try to explore and do something else. It is basically impossible as a researcher to do your best work if you are not 100% excited and if you don't go with full enthusiasm. There were many days at OpenAI when I had basically infinite enthusiasm for the work I was doing and believed I was doing exactly the right things, but somewhere around the end it was getting harder and harder. So that's kind of the long story short.

现在有哪些事情在给你带来能量？

What are some of the things that are giving you energy today?

在最根本的层面上，我所做的一切：当我加入 OpenAI 时，我相信强化学习是通往 AGI 道路上必不可少的元素，我真的很想实现它。将推理能力、推理模型带给世界，对我来说，是我们训练模型范式的一次板块级转变。某种程度上，我想再次追逐那种高峰，尝试做类似的事情。找到目前世界训练模型的方式中缺失的东西，并以某种方式让它成为主流。但一旦你做过那样的事，就很难再通过做别的事获得同样程度的成功。所以这就是我想做的：拥有一些自由，思考如何探索，并尝试攻克最核心、最重要的问题。

On the most fundamental level, what I did and what I've done: when I started at OpenAI, I believed reinforcement learning is a necessary element of a path to AGI, and I really wanted to make it happen. Introducing reasoning to the world, reasoning models to the world, has been to me a tectonic shift in the paradigm of how we train models. In some ways, I want to chase that high again and try to do something similar. Try to find something that is missing in how the world is training models so far and try to make it mainstream in one way or another. But once you've done something like that, it's hard to get another similar hit doing something else. So that's kind of what I would like to do: have a bit of freedom thinking about how to explore and try to attack the most core, the most important problems there are.

你是有很多假设，还是在退后一步，看看外面发生了什么？

How much are you feeling like you've got dozens of hypotheses, or how much are you zooming out and seeing what else has been going on?

总的来说，真正重要的假设和问题，如果你已经从事机器学习七年了，很可能不会是全新的、突然出现在你面前的东西。你很可能知道哪些是重要的问题。主要问题是我之前提到的：你如何用与其他人不同的方式来解决它？因为这意味着还没有人解决它。所以，什么是不一样的，以及什么可以做得与过去的人不同？

In general, the real important hypotheses and the real important problems are most likely not something that will be new and appear to you if you've worked on machine learning for seven years. It's very likely you know what the important problems are. The main question is what I alluded to before: how do you solve it differently than everyone else? Because it means no one has solved it yet. So what is different, and what can be done differently than people in the past?

我确实想聊聊 OpenAI 和你那段了不起的经历。我之前听你说过，每一年都感觉像是一家不同的公司。我希望你能回顾一下这段演变，描述一下过去六七年的历程。

I definitely want to hit on OpenAI and your incredible run there. One thing I've heard you say before was that every year felt like a different company. I'd love if you could walk through that evolution and describe the narrative of the past six or seven years.

从一个只有三四十人的小实验室开始，我们从一开始就非常开放，极其雄心勃勃，相信这里是建造 AGI、为世界创造数字智能福祉的地方。从几个人尝试做一些非常雄心勃勃的酷项目，到今天成为全球最大的公司之一，拥有一个几乎人人皆知、人人都在用、难以想象不用它的产品。这是一段疯狂的旅程。OpenAI 的高管团队多年来发生了很大变化，所以你每天一起工作的人类型变了。公司规模变了。研究的主题也变了。在早期，根本没有预训练。然后有一段时间，它基本上成了预训练公司。再后来，有一段时间它很大程度上成了一家强化学习公司。现在则更加平衡，是这两者的混合，很多人离开 OpenAI 去创业，也有很多新鲜优秀的人加入，做着令人难以置信的研究。这是一家能够自我革新、在所有这些阶段中成长的公司。我一直有一种奇怪的想法，想到那些大获成功的公司，能亲身经历这样的故事该有多不可思议。我觉得我在 OpenAI 经历了不少，这是一种很难与其他经历相比的体验。

Starting with a small lab of 30-40 people, we were always open from the very beginning, extremely ambitious, believing this is the place that will build AGI and create benefits of digital intelligence for the world. But starting from a few people trying to do a few cool projects that were incredibly ambitious, to where it is today, which is one of the largest companies in the world with a product that everyone knows and uses, it's almost hard to imagine not using it. It's been a wild ride. The execs at OpenAI have shifted significantly throughout the years, so the type of people you work with every day changed. The size of the company changed. The themes of research changed. At some moment in the old days, there was no pre-training at all. Then for a while it became kind of the pre-training company. Then for a while it became largely a reinforcement learning company. And now it's a bit more balanced, a mix of those two, with many people leaving OpenAI and building businesses, and many new fresh great people coming in doing incredible research. It's a company that manages to reinvent itself and grow through all those stages. I always had this weird hope, thinking of all those big successful companies, how incredible it would be to live through such a story. I feel like I've been through quite a bit of that at OpenAI, and it's an experience that is very hard to compare with something else.

我想大家都在热切期待关于 OpenAI 这一章的权威故事被写出来。当那些故事被写出来时，人们喜欢关注那些艰难的、51 对 49 的、可能走向任何一边、却真正推动公司前进的决策。有没有什么关键决策让你印象深刻？

I think everyone's eagerly awaiting the definitive stories to be written about this chapter of OpenAI. When those stories do get written, people like to focus on the difficult 51-49 decisions that could have gone either way and really moved the company forward. Are there any pivotal ones that stick out to you?

好问题。我只参与了其中一部分决策。可能有很多决策我只是个背景角色。甚至关于是否向世界发布 ChatGPT 的讨论，你可能也听说过，它的流行和病毒式传播完全出乎内部预料，至少我没听任何人预测到。我认为最终，随着 ChatGPT 和随后发布的 GPT-4，我们创造了一个时刻和势头，这极难预测，但让 OpenAI 在很大程度上成为了今天的样子。

Good question. I was only central to some of them. There were probably many that I was only a background character. Even this discussion about releasing ChatGPT to the world or not, as you may have heard, its popularity and virality was totally not expected internally, at least by no one I heard about. I think in the end, with ChatGPT and GPT-4 released soon thereafter, we created a bit of a moment and momentum that was incredibly hard to predict but made OpenAI largely what it is today.

那绝对是一个在多个维度和多个方面都非常重要的决策。当时投入大量资源训练 GPT-4，伴随着许多权衡，这在 OpenAI 的历史中仍然非常关键。事实证明这是一个非常好的决策。同样，在一个完全不确定的世界里，仅凭第一性原理思考和直觉认为这是正确的事情，就押注推理模型是我们的未来，这让 OpenAI 得以彻底重塑自我。我们说我们现在就在做推理模型，尽管当时没有产品市场契合度，尽管它们看起来只是对解谜有点酷。如果你看 o1，它是一个聪明的模型，但除了在基准测试上大杀四方之外，对任何实际用途都不太擅长。我们有一个还算聪明的模型，真正到了 o3 并且对这些模型投入更多工具使用时，我们才开始构建出对研究和编码极其有用的东西。从那时起，一旦你有了真正产品市场契合度的初步迹象，人类就非常擅长优化那些已经存在并且能看到效果的东西。但达到那个时刻是一段伟大的旅程，值得研究，因为它并不容易。OpenAI 在那一刻真正通过了考验。

That was definitely a very important call on many axes and many dimensions. The decision to pull a lot of resources to train GPT-4 at that moment, with many trade-offs, remained very critical in the history of OpenAI. It turned out to be a really good decision. Similarly, betting on reasoning models as our future in a world where it was completely unsure, just from first principles thinking and intuition that this was the right thing to do, allowed OpenAI to completely reinvent itself. We said we are doing reasoning models right now, even though there was no product-market fit, even though they seemed kind of cool with puzzles. If you look at o1, it was a smart model but not really good for anything practical except for destroying benchmarks. We had a model that was kind of smart, really only with o3 and with a bit more investment into tool use with those models, we were able to start building something that became incredibly useful for research and coding. From there, once you have the first signs of real product-market fit, humans are really good at optimizing something that exists and that they can see works. But getting to that moment was a great journey and something to study because it was not easy. OpenAI at that moment really passed the exam.

是的。不，我是说，我觉得你描述的内容非常有趣，这种想法是你必须持续 Scaling 并投资于某件事，而不太确定它是否真的会成功。这显然也与你未来思考的一些事情非常相关，比如推理模型。在 o1 之后，你是否清楚这不仅仅是好玩和游戏，或者对你来说，那个火花时刻是什么，让你觉得“哦，这真的会成功，我们可以真正地 Scaling 它。”

Yeah. No, I mean, I think what you describe is so interesting, this idea that you have to keep scaling and investing in something without quite knowing if it's really going to work. It's obviously very relevant to some of the things you're thinking about in the future too, with the reasoning models. Was it clear after o1 that this was going to be more than kind of fun and games, or what was the moment the spark for you that you were like, "Oh, this is really going to work and we can really scale this."

我从一开始就有点相信它，仅仅是因为我相信强化学习。再说一次，我在 OpenAI 最初几天的核心信念是，强化学习是达到 AGI 的必要部分。那是主要的方式，而不是是否的问题。问题是我们什么时候准备好去做，以及具体怎么做。我基本上只是默默地知道这就是我们需要的。随着时间的推移和研究的深入，各种实验结果告诉我们这是正确的方法。

I kind of believed in it from the very beginning, just because I believed in reinforcement learning. Again, my core belief from my very first days at OpenAI is that reinforcement learning is a necessary part of getting to AGI. That was the main way, rather than if. The question is when are we ready to do it and how exactly do we do it. I fundamentally, just silently, I knew this is what we need. Over time and over research, various experimental results informed us this is the right way of doing it.

OpenAI 的一个有趣之处在于，我想人们喜欢，我想 Ben Thompson 创造了“意外消费者业务”这个说法。意思是，显然你们是一个追求 AGI 的研究实验室，然后几乎偶然地发现了这个令人难以置信的消费者产品，并立即被更广泛的世界所接受。我想知道，你谈到了在 OpenAI 的不同阶段。我认为每个人都会问 OpenAI 的问题是，这家公司正在做这么多不同的事情：消费者产品、核心研究、Sora、企业产品。内部实际是如何运作的？在研究方面，你是否感觉到被拉向不同方向的紧张感？

One thing that's so interesting about OpenAI is, I think people liked, I think Ben Thompson coined this phrase like the accidental consumer business. The idea that obviously you were a research lab pursuing AGI and then kind of almost accidentally stumbled upon this incredible consumer product that was immediately picked up by the broader world. I'm wondering, you talked about some of the different chapters of being there. I think a question everyone always asks about OpenAI is the company's doing so many different things: the consumer product, core research, Sora, the enterprise product. How does it actually work internally, and do you feel that tension at all on the research side of getting pulled in a bunch of different directions?

有一点很清楚，OpenAI 的研究与产品非常分离，而且从一开始就是这样。OpenAI 的目标和使命是构建智能，这是大多数研究的主要目标和动机。有一个专门针对产品研究的团队，那部分研究针对任何产品指标进行优化。其余的研究主要关注如何让我们的模型更智能，这种紧张感在那里并不存在。我认为真实且有趣的是，OpenAI 处于我们一生中可能最大的技术变革的中心，这意味着有如此多的机会去做各种事情。不去尝试所有这些事情几乎感觉是浪费和不谨慎的，因为基本上世界上的一切都会被 AI 颠覆。但它有一个非常真实且非常成问题的缺点：专注。公司非常不擅长同时成功完成多件困难的事情。公司以成功完成一件非常困难的事情而闻名，然后其他事情做得和其他公司一样好。世界上很少有地方能做到其中几件事，这是一件非常困难的事情。我认为这对 OpenAI 来说是一个非常大的风险：试图做所有事情，然后无法成功。但我们会看到，OpenAI 在半专注的状态下能否很好地执行所有这些方面，或者会有其他公司。众所周知，而且我认为有点可悲的是，OpenAI 在专注于消费者产品时，有相当长一段时间失去了对编码的关注，这已经损失了一些市场份额，现在它正在非常努力地重新夺回。现在的编码模型又非常出色了，但失去专注和失去领先地位肯定是有代价的。所以从某种程度上说，当你在当今世界做一家 AI 公司时，你感觉像个在糖果店里的孩子，因为有太多潜力可以构建对世界极其有价值的东西，很难阻止自己去做所有事情。但每件事都有竞争，问题是谁会以正确的方式做好每一件事。

One thing pretty clear is that OpenAI research operates very separately from the product, and has from the very beginning. OpenAI's goal and mission is to build intelligence, and that is the main goal and motivation of the majority of research there. There is one team specifically built and directed towards product research, and that part of research optimizes for whatever product metrics it is. The rest of the research mainly focuses on how do we make our models more intelligent, and that tension really doesn't exist there. What I think is real and interesting is that OpenAI is at the center of probably the biggest technological shift of our lifetime, which means there is so much opportunity to do various things. It almost feels wasteful and imprudent to not try to do all those things, because basically everything in the world will be disrupted by AI. But it has a downside which is very real and very problematic: focus. Companies are very bad at doing multiple hard things successfully. Companies are well known for succeeding at one very hard thing and then doing others similarly well. There are very few places in the world that can do a few of those things, and that is a very hard thing to do. I think this is a very big risk for OpenAI: trying to do everything and then not succeeding at it. But we will see if OpenAI in a half-focused state can execute well on all those side bits, or there will be other companies. Famously, and I think it's a little bit sad, OpenAI really lost focus on coding for quite a while when it focused on the consumer product, and that has cost a bunch of market share that it is working very hard on regaining right now. Coding models are really great these days again, but lost focus and lost lead definitely has a cost. So in some way, when you are a company doing AI in the world right now, you feel like a kid in a candy store because there is so much potential of extremely valuable things that can be built for the world that it's hard to prevent yourself from doing all of it. But for everything there is competition, and there are questions who will do each one of those things in exactly the right way.

是的。嗯，我是说，我认为这是一个很好的过渡点，可以谈谈当今的整体生态系统。你提到了编码，我认为这是一个非常迷人的领域，值得观察。你认为为什么 Anthropic 在编码方面如此成功？

Yeah. Well, I mean, I think that's a great point to transition to just like the general ecosystem today. And you kind of alluded to coding, which I think has been a really fascinating space to watch play out. Why do you think Anthropic's been so successful at coding?

是专注。我认为专注可以解释 95% 的事情。

It's focus. I think focus can explain 95% of things.

公司是什么？为什么公司能在某些领域取得成功？我从 Anthropic 的创始人们还在 OpenAI 时就认识他们，他们一直非常热爱编程，并始终相信这是 AGI 的必要且关键部分。我只能想象他们多年来对此有多么专注，而他们确实通过最新的模型、Claude Code 和编程智能体，将愿景推进了很远。他们说得没错，如今 Anthropic 内部几乎没人自己手写代码了。

What are companies? Why are companies succeeding in things? I know Anthropic's founders from even the time when they were at OpenAI and they were always extremely fond of coding and they always believed that it's a necessary and critical part of AGI. I think I can only imagine how focused they have been on it over the years and they definitely managed to get their vision very far with the latest model with Claude Code and coding agents. They are speaking truth when they say very few people at Anthropic type code themselves these days.

你认为这几乎是一种预示吗？比如，看看那些主要实验室，每个都可以专注于不同的事情，结果就是不同实验室的模型在某些方面特别擅长。这是个好问题。我认为我们可能处于多种情况。有一种情况是数据很重要，在这种情况下数据是一个零和游戏：你把数据投入到你想要的技能上，你的模型就更擅长那个技能。这时我们会看到市场分裂，基本上你可以调整你投入的数据组合，但最终是以牺牲其他技能为代价的。

Do you think that's kind of a foreshadowing of you know, pick the major labs that are out there, like each can focus on different things and so you end up with models that are particularly good at some things in different labs? It's a good question. I think there are multiple worlds we can be in. There's a world where data matters, and in that case data is a very zero-sum game where you put the data into the skills that you want and your model is better at that skill. In that case we see splintering of the market into basically you can shift the data mix of what you're putting in, but ultimately it's at the cost of some other skill.

没错。而且这不仅仅是调整数据组合，更多的是调整工作。数据是劳动密集型的，资金密集度稍低，但在研究工程方面，它很大程度上是劳动密集型工作，我们姑且这么说。所以你有许多研究人员可以并行工作，为准备下一个数据集做有用的工作。因此，如果数据驱动改进，那么我们会看到不同实验室在不同方面更擅长，形成自然的专业化权衡。但如果研究为王，我认为研究有一种神奇的特性：它很难，高风险高回报。但同时，如果你在研究上有一个好主意，它可以同时提升模型在所有领域的能力。你可以通过训练更好的模型，自然地在所有领域超越所有人。我们现在处于哪种未来，目前很难说。我们将来会看到。

Exactly. And a little bit about shifting the data mix, but mostly shifting the work. Data is labor-intensive and slightly less money-intensive, but it's largely very much labor-intensive work in terms of research engineering, let's call it that way. So you have so many researchers that can work in parallel doing useful work on preparing the next data set. So if data drives the improvements, then we will see different labs being better at different things in natural specialization trade-offs. But if research is king, I think research has this magical property that is hard. It is high risk, high reward. But at the same time, if you have a good idea in research, it could improve your model in all domains at the same time. And you could leapfrog everyone in all domains naturally by just training better models. Which one of the futures we are in, it's very hard to say right now. We will see in the future which one.

我的意思是，当前的范式几乎非常反“苦涩的教训”，对吧？每个人都进入特定领域，进行专业化，并投入其中。你说得对，这几乎有一种直觉上的感觉，那就是一定还有某种更通用的东西。我很确定是有的。问题在于它有多容易找到。所以有一种世界版本，并非完全不可能，尽管对人类有点悲观，它说现在的编程智能体已经非常好了。让我们先把它们带到能够自动化 AI 研究的阶段，然后让模型研究更好的模型，因为也许我们已经到了人类能想出的最后一个模型。不，这并非完全不可能的框架。也许考虑到我们拥有的所有 GPU、那些非常强大的模型以及它们的坚韧性，这是有道理的。也许它们应该研究未来的模型。但你知道，也许人类仍然可以做几件事，我们可以动脑筋去解决。

I mean, this current paradigm feels almost very anti-Bitter Lesson, right? Where everyone's going off into specific domains and specializing and really putting in those. And you're right that it feels there's this intuitive feeling almost that there must be something else that is a bit more generalizable. I'm pretty sure there is. The question is how easy it is to find. So there is one version of the world, which is not completely impossible, although slightly pessimistic on humans, that says coding agents are so good right now. Let's first get them to the moment where they can automate AI research and then have the models research better models, because maybe we are at the last model that humans could have figured out. No, it's not a completely impossible framing. Maybe it makes sense with all those GPUs we have and all those really capable models and their tenacity. Maybe they should be researching future models. But you know, maybe there are still a few things humans can do that we can put our heads to work.

我觉得很多顶尖 AI 研究人员正是出于这个原因在研究编程，对吧？这种信念认为它会加速进展。你显然也研究过编程。我的意思是，你在 Codex 上花了很多时间。你怎么看待这些 AI 编程产品的下一个前沿？我们现在似乎正处于一个疯狂的指数级爬坡阶段。你甚至如何设想这些产品在一年内能做什么？

I feel like a lot of the top AI researchers are working on coding for that very reason, right? This belief that it will speed things up. You've obviously worked on coding too. I mean you spent a lot of time on Codex. How do you think about the next frontiers for these AI coding products? It seems like we're on this crazy exponential hill climb right now. How do you even conceptualize what these products will be able to do in a year?

从根本上说，编程的全部故事在于我们能够以越来越高的抽象层次来编程计算机。我们需要知道的细节越来越少。我需要追踪的细节越来越少。而编程智能体在某种程度上可以被看作是一种更高级的编程语言，它具有与其他所有编程语言截然不同的语义。我认为这是一个将持续下去的趋势。我认为未来我们自己手写代码的可能性非常小。已经很少有人这么做了，而且这是一条单行道。但与此同时，软件很重要。软件需要可靠，如果我们不是编写它的人，甚至可能不是阅读它的人，那么如何确保软件做正确的事情，这方面会有越来越多的进展。从根本上说，我认为这些都是可以解决的问题，我对它们感到非常兴奋。我认为有趣的是，与这些智能体合作所需的核心技能集，是看起来有点像软件工程师，还是真的每个人都变成了项目经理。这实际上只是对要做什么有点想法，然后模型去执行。

Fundamentally, the whole story of coding is that we are able to program computers at a higher and higher level of abstraction in many ways. We need to know less details. I need to track less details. And coding agents in some ways can be thought of as a higher-level programming language that has very different semantics from all the other programming languages. I think it's a trend that will continue. I think it's pretty unlikely that in the future we will be typing code ourselves. Very few people already do, and it's a one-way ticket. But at the same time, software is important. Software needs to be reliable, and there will be more and more progress in how do we get certainty about software doing the right things if we are not the ones typing it and maybe not even the ones reading it. Fundamentally, I think those are all solvable problems, and I'm very excited about them. I think it'll be interesting whether the core skill set to work with these agents looks somewhat similar to a software engineer or really everyone just becomes a PM. It's really just about having some idea of what to go do, and the models going for it.

是的，我认为这是一个有趣的问题，因为我认为目前最重要的技能几乎是成为一名优秀的初级软件工程师经理。很多软件工程师一直有点不愿意进入管理层，而是喜欢在狭窄的领域里专精。我认为很长一段时间里，这确实是正确的事情，深入理解系统非常重要。你需要将深入理解计算机系统与至少能交回一点控制权结合起来。我认为真正最好的经理，他们极其深入地理解团队中人员的工作，但同时能够交回一些控制权，让人们能够掌握自己的命运。这可能是如今与模型合作的最佳方式。

Yeah, I think it's an interesting question because I think almost the most important skill right now is being a good manager of junior software engineers. A lot of software engineers have been a little bit reluctant to go into management and really like to specialize in narrow domains. I think for a long time it was really the right thing to do, and understanding deeply systems is incredibly important. You need to combine deep understanding of computer systems with being able to give back at least a little bit of control. What I think really the best managers are, the best managers understand extremely deeply the work of people on their team, but at the same time are able to give back some control for the people to drive their own destiny. That's probably the best way how to work with models these days.

关于编程，人们也在试图弄清楚各种应用的位置。我想有几个问题，比如，Codex 和 Claude Code 团队就坐在研究团队旁边，他们构建了这些出色的工具和产品。你如何看待像 Cursor 和 Cognition 这样的公司的机会？不在实验室的研究人员旁边，这在多大程度上是一种劣势？

You know around coding one thing people are also trying to figure out is just where the various applications kind of fit in. And I think there's a few questions like one being, hey like you have the Codex, the Claude Code team sitting right by the research teams, they build these great harnesses and products. How do you think about the opportunity for companies like Cursor and Cognition? And to what extent is it like a disadvantage to not be sitting next to the researchers of the labs?

这绝对是一个劣势。我认为世界上那些最成功的公司都在训练自己的模型，这本身就说明了一些问题。我有点认为，就像大型 AI 公司的未来是成为超大规模运营商并运营自己的数据中心而不是租用算力一样，成功的 AI 应用公司的未来是开始自己训练模型。这就是技术栈的运作方式。但你必须从某个地方开始。所以，可能的路径是：你先从一个成功的 AI 应用开始，然后你首先开始后训练自己的模型，然后如果你越来越成功，你就开始预训练自己的模型，然后你开始建造自己的数据中心。这些将是一个好的 AI 企业成功的自然路径和阶段。

It definitely is a disadvantage. I think the fact that the most successful companies of the world are training their own models is telling about something. And I kind of think just like the future of big AI companies is to become hyperscaler and run their own data centers instead of renting compute, the future of successful AI application companies is to start training the models themselves. This is just how the stack works. But you got to start somewhere. So it may be the path that you start with an AI application that is successful, and then you first start post-training your own models, and then you start pre-training your own models if you are more and more successful, and then you start building your own data centers. Those would be the natural paths and stages of success of being a good AI business.

所以你认为这些公司利用自己的用户数据进行强化学习、进行大量后训练是有意义的吗？他们是否有希望赶上你们，毕竟大型实验室拥有无尽的算力和无尽的人才储备？如果你专注于一个非常特定的领域，你有希望在那里构建一个更好的模型吗，还是在某些方面这几乎是一个无望的任务？

So you think it makes sense for these companies to do reinforcement learning on their own user data, post-train a bunch? And do they have any hope of catching you, with endless compute going toward the big labs, endless collection of talent? If you focus on a very specific domain, do you have a hope of building a better model there, or is it almost a hopeless task in some ways?

嗯，没有什么事情是完全无望的。未来尚未确定。但这部分就是我之前暗示的：是数据重要还是研究重要？如果数据重要，你总是可以尝试用数据来差异化自己。但我们现在是否真的处于那个世界，还不清楚。也许存在一个研究重要的世界。但这也会允许较小的公司做一些他们认为更好的研究，并可能通过研究在市场中获胜。

Well, nothing is ever a hopeless task. The future is not determined. But some of that is what I hinted before: is the data important or is the research important? If the data is important, you can always try to differentiate yourself with the data. But it is not clear we are really in that world. Maybe there's a world where research is important. But that also allows the smaller companies to do some research that they think is better and maybe win through research in the market.

但这似乎几乎要求一个世界，其中你有专注于让事情变得更好的特定类型的模型，对吧？而不是泛化。所以，我们进入了一个通用模型的世界。感觉对于一个专注于任何特定任务的应用公司来说，拥有更好的预训练或大型模型变得困难。

But it seems like it almost requires a world where you have specific types of models that are focused on things being better, right? And not generalizing. So, we get into a world of generalized models. It feels like it becomes hard for an application company focused on any specific task to have the better pre-trained or large model.

有时创新来自约束。我认为，一家专注于特定领域的公司，通过看到模型在该领域的缺陷，有可能创建一个总体上更好的模型。我不认为这是不可能的，这可能是这家公司下一层级的成功。如果你试图为 X 做一个非常好的模型，突然之间你通过它做出了其他所有方面最好的模型，然后你成长并成为另一家大型成功公司。

Sometimes innovation comes from constraints. I think it is possible for a company focused on a specific domain, through seeing the deficiencies of the models on this domain, to create a model that is generally better. I don't think it is impossible, and that could be the next layer of success of this company. If you try to do a really good model for X and suddenly through that you make the best model for everything else, and then you grow and become another big successful company.

感觉在过去，问题是你变得更好了一秒钟，然后下一代模型出来了，你会想，哦，我们又落后了。竞争很困难，我们确实在美国科技领域看到了一段时间内大公司拥有大量优势，这是事实，但与此同时，也有新的成功大公司出现，所以并非无望，只是很难。

Feel like in the past the problem's been you get better for like one second and then the next generation of models come out and you're like oh man we're way behind again. Competition is difficult and definitely we have seen for a while in the US tech landscape how big companies have tons of advantages and this is true, but at the same time there are new big successful companies coming up, so it's not hopeless, it's just hard.

嗯，我想把话题转向人才生态系统，也许还有研究本身。显然，你是一位出色的研究员，并与杰出的研究人员合作过。也许首先，如今研究员招聘竞争非常激烈。我知道你可能是将人才引入 OpenAI 的前沿人物。是什么决定了今天研究人员加入哪些公司？

Well, I want to shift gears to the talent ecosystem and maybe research itself. Obviously you've been an incredible researcher and have worked with amazing researchers. Maybe to start, researcher hiring is very competitive today. I know you were probably on the forefront of bringing folks into OpenAI. What determines what companies researchers join today?

该死，这是个好问题。归根结底，人是非常复杂的，肯定比现在的模型复杂得多，这意味着每个人的动机都不同，他们想要的东西也不同。老实说，我无法真正概括。招聘的人不应该想“我如何说服最多的人加入我”以及“我如何成为对研究人员最有吸引力的地方”。这可能是你该问自己的一个好问题，但我认为还有第二个问题，我认为更重要：什么样的研究人员真的想在这里工作？然后找到那些人，因为试图吸引所有人几乎是不可能的，也非常困难，因为偏好、观点和工作方式各不相同。所以，最好尝试建立一个拥有一些共同价值观、共同方法的团队，因为很明显，目标一致、方向一致的团队比不一致的团队行动更快、工作更好。所以，真的应该在双方都进行筛选，努力为合适的团队找到合适的人，这会让每个人都开心，让团队成功，并让团队随着时间的推移更具吸引力。

Damn, it's a good question. In the end, people are very complex, definitely more complex than the models these days, which means everyone's incentives are different, what they want. And I honestly cannot really generalize. Whoever is hiring people should not think about how do I convince the most people to join me and how do I become the most appealing place for researchers. It's probably a good question to be asking yourself, but I think there's a second one which I think is much more important: what type of researchers would really want to work here? And then find those, because it's kind of impossible and very difficult to try to appeal to everyone just because of diverse preferences, diverse viewpoints, and diverse ways of working. So it's much better to try to build a team that has some shared values, some shared approaches, because it's pretty clear that teams that are aligned and have the same goal move faster and work better than teams that are not. So it's really should be filtering on both sides and trying to just find the right people for the right group, and that makes everyone happy, makes the group successful, and makes the group more attractive over time.

是的，但在这方面有一些有趣的实验。我记得 Meta 曾著名地给你开出第一份 offer，这些巨额薪酬包。你对此有什么反应？

Yeah, but there's been some interesting experiments around this right. I think you had Meta famously offering you the first offer, these mega packages. What's your kind of reaction to that?

嗯，建立研究小组有不同的策略。而 Meta 在某个时刻，我想，存在供需曲线，但他们需要让 offer 非常有吸引力，以便在经历了一些失误后开始把人带回来。势头是一件很难阻止的事情。如果在任何时候行业里认为你做得不太好，那么你就招不到人，然后这就会自我循环。

Well, there are different strategies how to build a research group. And Meta at some moment, I think, there are supply and demand curves, but they needed to make offers really attractive to start bringing people back after a few missteps in the world. And momentum is a very hard thing to stop. If at any moment there is a perception in the industry that you are not doing very well, then you will not hire people, and then it can roll itself.

所以，从很多方面来看，我认为这是一个非常好的策略，试图改变这种动态和负面势头，尤其是在 AI 对每个大型企业都如此重要的领域。Meta 已经组建了一个全新的团队，目前正在训练一个新模型，业界很多人都在关注它是否会成功以及如何成功，这将决定这个实验室的未来。但这绝对是一个为 Meta 的努力注入新活力的好时机。

So you know in many ways I think it was a really good strategy to try to change that dynamic and change the negative momentum in a place where AI is so important for every large scale business. Meta has a very much new team built out that is training a new model these days, and a lot of people in the industry are watching whether it will be successful and how it will be successful, and it will determine the future of this lab. But it was definitely a good moment to bring in some new life into the Meta efforts.

你显然做了大量开创性的 AI 研究，也和其他优秀的 AI 研究员合作过。是什么造就了一位伟大的 AI 研究员？

I mean, you've obviously done a ton of groundbreaking AI research. You've worked with other great AI researchers. What makes a great AI researcher?

这是个好问题，也很难回答，因为从很多方面来说，成为一名成功的 AI 研究员就是要恰好在正确的时间出现在正确的地方。但确实有一些要素。在我看来，在我职业生涯的跨度中，如今成为一名伟大 AI 研究员的基础是：在系统和工程层面都非常出色，理解计算机如何工作以及神经网络如何训练，同时掌握神经网络和优化的理论。只做好其中一件事很难取得巨大成功。如果你至少在这两方面都还不错，那你在任何研究工作中都会轻松拥有 10 倍的生产力。我认为这是很重要的一点。另一点是，如果你想成为一名成功的研究员，你必须有独立思考的能力，能够摆脱群体思维。人们有一种自然倾向，会趋同于群体的中位数观点，这几乎扼杀了研究。我有一句话：如果有 100 个研究员想法都一样，那本质上只有一个研究员。作为一名研究员，意味着要一直稍微有点反主流，因为你想研究那些尚未成功、默认人们并不真正相信的东西。反主流意味着很多极其聪明和勤奋的研究员不幸缺少的东西：你可以称之为勇气。在某种程度上，就是要站出来说：“让我们尝试做些不同的事情。让我们用一种大多数人还不相信的方式去做。”这是一件极其困难的事情，尤其是在实验如此昂贵、涉及如此多东西的世界里，在机器学习实验成本堪比好莱坞电影的世界里。执导一部电影需要很多，而且你不知道电影是否会成功。有了大预算，你冒险请明星、做最好的 CGI。在 ML 的世界里，你也试图冒险，尽可能多地积累优势，但最终实验就是实验，注定要进入未知，要么成功要么失败。但我想如果我要总结，那就是：非常了解系统和理论，不太追随群体思维，然后有勇气向别人陈述这些。

It's a good question and it's a difficult one because in many ways being a successful AI researcher is about just being in the right place at the right time. But there is something to that. I think the fundamentals of being a great AI researcher these days, in the span of my career, are one thing: being extremely good at both systems and engineering levels, understanding how computers work and how neural networks are trained together with the theory of neural networks and optimization. It's very hard to be very successful doing only one of those things well. If you are at least okay in both, it makes you easily 10 times more productive in any research endeavor. I think this is an important part. The other one is that if you want to be a successful researcher, you necessarily need to have some ability to think independently, to be able to get away from groupthink. People have a natural tendency of converging on a median viewpoint of a group, which kind of kills research. I have a saying: if you have 100 researchers that think the same thing, you essentially have one researcher. Being a researcher means being slightly contrarian all the time, because you want to work on something that is not working yet and that by default people don't really believe in. Being contrarian means something that a lot of researchers who are extremely brilliant and extremely hardworking are unfortunately missing: you can call it courage. In some way, it is about standing up and saying, "Let's try to do something different. Let's do something in a way that most people don't believe in yet." It's an extremely hard thing to do, especially in a world where experiments are so expensive, involving so many things, in a world of machine learning experiments being the cost of Hollywood movies. Directing one requires a lot, and you don't know if the movie will be successful. With a big budget, you risk buying stars and doing the best CGI you can. In the world of ML, you also try to risk and bring as many advantages on your side, but in the end, experiments are experiments, meant to go into the unknown, and either will be successful or not. But I think if I were to say, it's about knowing both systems and theory very well, not following groupthink that much, and then having the courage to state that to other people.

我们总是喜欢用快问快答环节来结束访谈，我会把其他没来得及问的问题都塞到这里，听听你的快速反应。那么首先，过去一年里你在 AI 方面改变了什么看法？

Well, we always like to end our interviews with a quick fire round where I stuff all the other questions I couldn't fit in elsewhere into the end here and get your rapid reactions. So maybe to start, what's one thing you've changed your mind on in AI in the last year?

可能我最近一次有意义的更新是，我不认为一个静态模型能成为 AGI，持续学习是我们追求的一个必要元素。

Probably the last thing I meaningfully updated on is that I don't think a static model can ever be AGI, that continual learning is a necessary element of what we are pursuing.

这是因为静态模型无法完成某些事情，还是说 AGI 的定义本身就需要包含持续学习的方面？

Is that just because of something it won't be able to accomplish, or is it more that the definition of AGI needs to have a continual learning aspect to it?

这主要是关于揭示我们的模型仍然缺少什么。在很多方面，你会一层层深入，因为我们的模型已经在很多方面很出色了，但尽管它们如此聪明，没有持续学习，对我来说它永远不会感觉那么智能。它仍然是一个需要由能够持续学习的人监督的工具。

It's mostly about uncovering what our models are still missing. In many ways, you go layers and layers deep because our models are already good at so many things, but them being as brilliant as they are, it is clear that without it, it will never feel to me that intelligent. It will still be a tool that needs to be supervised by someone who has the ability to continuously learn.

是的，显然在我们今天没谈到的领域也有很多 AI 进展。你对机器人领域出现类似 ChatGPT 那样的时刻有什么时间线预测？

Yeah, there's obviously a lot of AI progress happening in spaces we didn't talk about today. What's your kind of timeline for a ChatGPT-like moment in the robotics space?

大概从现在起 2 到 3 年。

Probably around 2 to 3 years from now.

这相当乐观。我觉得每个人还在试图弄清楚机器人领域是否存在缩放定律，或者这些东西是否真的有效，是否有足够的数据。

That's pretty bullish. I feel like everyone's still trying to figure out if there are scaling laws in robotics or if the stuff will actually work, if there's enough data.

老实说，私下里我觉得情况比大多数人意识到的要好一点。有很多公司取得了大量进展，但进展总是需要一些时间来显现，也需要更多投资。但我认为机器人技术在未来几年会发展得很好。

I think honestly between you and me, things are slightly better than most people realize. There are tons of companies making tons of progress, but always the progress needs some time to play out and some more investments to happen. But I think robotics will be doing pretty well over the next few years.

是的。那在生物学方面呢？

Yeah. And in a similar sense on biology?

我认为生物学需要更长时间。

I think biology will take longer.

为什么比机器人技术更久？

Why longer than robotics?

只是从需要多少智能、需要多少精度才能成功操纵生物学的角度来看。这是一个更难的问题，需要更多基础性投资才能开始见效。

Just thinking from a perspective of how much intelligence is needed, how much precision to really manipulate biology successfully. It's a harder problem and requires more fundamental investments to start to work.

是的。我想大多数三四岁的孩子都能学会操纵世界上的东西，但他们不是世界级的生物学家。

Yeah. I guess most three or four year olds figure out how to manipulate things in the world, but they're not world-class biologists.

差不多是这样。是的。

Something like that. Yes.

你认为我们目前这种持续改进的模式带来的一个影响是什么，可能我们作为社会低估了或讨论得不够？

What's one impact of this continued model of improvement we're on that you think maybe we're underestimating or not talking about enough as a society?

我知道这很难说，但从根本上讲，广泛部署的工作自动化将在未来几十年成为现实。一方面我们在谈论它，但另一方面我认为我们谈论得不够多、不够认真，因为世界将发生巨大变化，与今天截然不同。如果有些人今天还不清楚，至少对我来说是显而易见的。社会变革是缓慢的，我认为这将非常奇怪。

I know it's hard to say, but fundamentally, widely deployed work automation will be reality over the coming decades. On one hand we are talking about it, but on the other I think we are not talking about it enough and seriously enough, because the world will change very drastically from where it is today. If there are some people for whom it is still not obvious today, it is obvious to me at least. Societal changes are slow, and I think it will be extremely weird.

我认为这在某些方面可能会很痛苦，我们应该想办法让它尽可能不那么痛苦。但我们需要思考，当就业市场与今天截然不同时，世界会是什么样子。与此相关，我感兴趣的一点是，你对这些问题的思考是否影响了你作为父母的行为方式，以及你对养育孩子的看法，AI 是否改变了这一点。

I think it will be probably painful in some ways and we should try to figure out how to make it the least painful. But we need to think about how the world looks where the job market is very different than it is today. And related to that, one thing I'm interested in is whether your thinking about this has impacted the way you act as a parent and how you think about raising kids, and if AI has changed that at all.

有趣的是，我想我在这方面与大多数父母不同，但我不确定。我的女儿们还非常小，所以其实没什么太多可做的。但我现在肯定不会逼她们拼命学习。很难想象你能怎么逼一个七岁的孩子学习。但我不认为我会成为那种说“哦，你必须成为班上数学最好的”的父母。我不认为我会说“哦，你必须赢得所有那些比赛”。我不认为你必须读所有那些书，因为等她们长大后，成为专家这件事可能会变得非常不同，再像那样押注于在就业市场中找到自己的位置已经没有意义了。显然，批判性思考的能力永远重要，因为如果你不为自己思考，没有人会替你思考，即使是 AI 也不会完全按照你想要的方式为你辩护。但考虑到有那么多未知因素，目前我只想让她们拥有快乐的童年，尽可能享受美好时光。我希望如果我们没有搞砸 AI 部署这件事，她们也能拥有快乐的成年生活。

The interesting thing, and I think I differ here from most parents, but I don't know. My daughters are very young and very small, so there isn't really that much. But I'm definitely not pushing them very hard to study these days. It's kind of hard to imagine how hard you can push a seven-year-old to study. But I don't think I will be the parent saying, 'Oh, you have to be the best in math in your class.' I don't think I will be like, 'Oh, you have to win all those competitions.' I don't think you have to read all those books because being a specialist when they grow up will probably look very different, and it doesn't make sense to really bet on trying to find your place in a job market anymore in that way. Clearly, the ability to think critically will always be important because if you don't think for yourself, no one else will, even AI will not fully advocate for you in the way you want. But given how many unknowns there are, at the moment I just want them to have a happy childhood and have a good time as much as they can. I hope they will also have a happy adulthood if we don't screw up this AI deployment thing.

是的，我想每个人都喜欢谈论存在风险这一面。过去几年里，你对这方面的担忧是增加了还是减少了？

Yeah, I guess everyone likes to talk about the existential risk side. Have your worries about that gone up or down these past years?

我基本上认为并希望，没有人希望人类灭绝，这对世界上每个人来说都是一个相当好的激励对齐，使得存在风险不会成为现实。我真的不相信制造一个会把我们都变成回形针的 AI 是那么容易的事。我认为在整个过程中有足够多的人类参与其中，因为这对任何人都没有好处，我们会设法足够成功，而控制世界上最大算力集群的人也会足够负责，因为杀死所有人也不是什么好生意。所以资本主义在这里也是完全对齐的。我认为更反乌托邦、更令人担忧的是，如果我们把娱乐推得太远，以至于它比现实世界更有趣，人类会想要生活在 VR 中，只玩虚拟游戏，这并非不可能。我认为这比我们都灭绝要现实得多，但可能同样糟糕。

I fundamentally think and hope that no human wants humanity to go extinct, which is a pretty good alignment of incentives for everyone in the world for the existential risk not to materialize. I really am not subscribing to the idea that it is that easy to just make a mole that will paperclip all of us. I think there are enough humans in the loop throughout all the process that because it is in no one's interest, we will manage to be successful enough, and the people controlling the biggest clusters in the world will be responsible enough because it's also not a good business to kill everyone. So capitalism is also fully aligned here. I think it's much more dystopian and much more worrisome if we push entertainment so far that it is more interesting than the real world, and humans will want to live in VR and only play virtual games, which is not impossible. I think that is much more realistic than all of us going extinct, but probably similarly bad.

是的。现在我觉得《头号玩家》和《机器人总动员》以及所有这些对那个世界的描绘都似曾相识。

Yeah. Now I feel like Ready Player One and Wall-E and all these kind of depictions of that world before.

是的。所以对我来说，这更令人担忧。但这个问题我们只能自己解决，这不是一个 AI 问题。这是一个人类问题：我们的偏好应该是什么？所以我一直是这么想的。

Yeah. So that to me is much more worrying. But this one we can only solve ourselves really, and this is not an AI problem. This is a human problem: what should our preferences be? So that's kind of how I've been thinking about it.

是的，我喜欢这个观点。我想知道，目前你有什么可以分享的关于你下一步的计划吗？

Yeah, I love that. I guess anything you're comfortable sharing at this point about what's next for you?

现在还非常早期，但我正在考虑。

This is still very early, but I am thinking about it.

太棒了。那么，Jerry，这是一次非常精彩的对话。

Awesome. Well, Jerry, this has been a fascinating conversation.

和你聊天很愉快，非常感谢你邀请我来这里。

It has been nice chatting with you, and thank you very much for inviting me here.

我是 Jacob Efron，您收听的是 Unsupervised Learning，一档探讨 AI 领域最敏锐头脑的节目，内容涉及当今 AI 的真实情况、该领域的发展方向以及它对企业和世界的意义。我很喜欢在担任 Redpoint 董事总经理的同时做这个播客，我在 Redpoint 主导了对 Lora、a bridge 和 physical intelligence 等公司的投资。如果您也喜欢这期节目并觉得有价值，请在 YouTube 上订阅，或在您收听的任何平台上关注我们。这是支持节目的最好方式，有助于我们继续成长并接触到最优秀的嘉宾。非常感谢您的支持和收听。我们下期再见。

I'm Jacob Efron, and you've been listening to Unsupervised Learning, a show where we probe AI's sharpest minds on what's true in AI today, where the space is going, and what it means for businesses in the world. I love doing this podcast alongside my day job as the managing director at Redpoint, where I've led investments in companies like Lora, a bridge, and physical intelligence. If you enjoyed it too and found today's episode valuable, please subscribe on YouTube or follow us on whatever platform you're listening on. It's the best way to support the show, which helps us to continue to grow and get access to the best possible guests. Thank you so much for your support and listening. We'll see you next episode.