LLM 之后会是什么

Yann，非常荣幸。你是 AI 领域的教父之一。我觉得几年前刚开始做这个播客时，就希望有一天能请到像你这样的人。

Yann, this is such a pleasure. You're one of the godfathers of AI. I feel like when I started doing this podcast years ago, I was really hoping we might one day get someone like you on.

我不喜欢这个称呼，因为我住在新泽西。在新泽西当教父，意思可不一样。

I don't like that term because I live in New Jersey. When you're a godfather in New Jersey, it doesn't mean the same thing.

很公平。你在所有人怀疑神经网络时押注它们，今天你又对 LLM 和主流生成式架构做出了类似的押注。你最近创办了一家新公司。我们的目标是让听众了解更多关于 AMI、你在 Tapestry 的工作、你为什么认为这个领域方向错了，以及你对 Meta 和领域发展的反思。

Very fair. You bet on neural nets when everyone doubted them, and today you're making a similar bet against LLMs and the predominant generative architectures. You recently started a new company. Our goal is to leave listeners with more information about AMI, your work at Tapestry, why you think the field is pointed in the wrong direction, and your reflections on Meta and the field's evolution.

首先，LLM 本身没有问题。它们是许多有用 AI 产品的基础，包括我自己也在用。它们在其功能上很出色，但只是通往人类级甚至动物级智能的路径。

First of all, there's nothing wrong with LLMs. They are the basis for many useful AI products, including for me. They're great for what they do. They're just not a path towards human-level or animal-level intelligence.

你帮助构建了最早的一些主要开源模型。

You helped build some of the first major open-source ones.

对，没错。

Right, absolutely.

那么，什么是 AMI？

So, what is AMI?

AMI 代表高级机器智能（Advanced Machine Intelligence）。座右铭是「面向真实世界的 AI」。如今许多 AI 技术擅长语言处理——人类语言、计算机代码、数学或法律术语。但语言很特殊，非常适合大型语言模型这类架构。真实世界要复杂得多：高维、连续、嘈杂、混乱。训练一个系统理解真实世界要困难得多。这正是我们的目标。我职业生涯大部分时间都在研究这个，过去五六年加速推进，最近两年取得了重大进展。创办一家初创公司并全力推进是合理的。去年年底，我清楚 Meta 不是合适的地方，所以我离开并创办了 AMI。

AMI stands for Advanced Machine Intelligence. The motto is 'AI for the real world.' Many AI techniques today are good for language manipulation—human language, computer code, mathematics, or legalese. But language is special and well-suited for architectures like large language models. The real world is much more complicated: high-dimensional, continuous, noisy, messy. Training a system to understand the real world is much harder. That's what we're after. I've been working on this for most of my career, accelerated over the last 5-6 years, with significant progress in the last 2 years. It made sense to start a startup and go into high gear. By the end of last year, it became clear that Meta was not the right place for that, so I left and started AMI.

现在有一种趋势，人们从大公司或研究实验室出来，带着特定的研究方向。你在 FAIR 的经历让你对追求多种研究方向与专注于产品化方向之间的张力有独特的视角。

There's a trend of people spinning out of large companies or research labs with a particular research direction. You have a vantage point from your time at FAIR on the tension between pursuing many research directions versus focusing on something that works for products.

这是一种奇怪的权衡。有两种模式：探索性研究，有很多方向；当某个方向似乎有效时，你就进一步推进。这不再是研究，而是工程和产品推动。这种情况在 Meta 发生过多次，比如 2023 年初 FAIR 启动的一些项目。

It's a strange trade-off. There are two modes: exploratory research with many directions, and when something seems to work, you push it further. That's not research anymore; it becomes engineering and product push. This happened multiple times at Meta with things started at FAIR, like in early 2023.

Llama 是在 FAIR 开发的，Llama 1 非常有前景。Meta 创建了一个完整的组织 GenAI，将其转化为实际产品和一系列产品，并推出了 Llama 2、Llama 3、Llama 4，后者有点令人失望。因为马克·扎克伯格对此感到失望，他重组了整个组织，重新调整，并雇佣了新的人等等。但在过去一年里，Meta 意识到他们有点落后了，因此重新聚焦于追赶行业的战略。可悲的副作用是，很多探索性研究基本上不再被优先考虑。我的意思是，这并不影响我正在做的 Jeppa 和世界模型的工作，因为马克本人、CTO 以及公司里的一群人都对这个项目非常感兴趣，并且相信它的长期影响。但公司的其他部分完全专注于大语言模型，并明确告诉我 Meta 不再是推动那个项目的合适地方。然后我们开始取得好结果，所以很明显我们必须从研究过渡到实际开发技术、扩大规模并构建产品。我们还意识到，大多数应用可能都是 Meta 不太感兴趣的东西。我们一直在做的那种东西的很多应用都在工业领域，比如制造业等等。

Llama, which was developed at FAIR, Llama 1, was very promising. And Meta created a whole organization, GenAI, to turn it into something real and a series of products, and produce Llama 2, Llama 3, Llama 4, which was a bit of a disappointment. And because Mark Zuckerberg was disappointed by it, he rebooted the entire organization, reorganized it, and hired new people, etc. But what also happened over the last year is that basically the company, Meta, realized that they had fallen behind a little bit, and so that refocused the strategy on trying to catch up with the industry. And the sad side effect of it is that a lot of the exploratory research was basically not given high priority anymore. I mean, it didn't concern the stuff I was working on, all the Jeppa and world models, because Mark himself and the CTO and a bunch of other people in the company were really interested in that project and really believed in the long-term impact. But the rest of the company was just entirely focused on LLMs, and made it clear to me that Meta was really not the right place to push on that project anymore. And then we started to have good results, and so it was clear that we had to make that transition between research and actually developing the technology, scaling it up, and building products out of it. And we realized also that most of the applications were probably for things that Meta was not particularly interested in. A lot of applications of the kind of stuff that we have been working on is in the industry, like manufacturing industry and stuff like that.

显然，你在更广阔的世界中追求世界模型。我认为还有其他一些人从更生成式的角度来研究世界模型。所以有谷歌的人和视频模型中的 Genie。有在机器人方面构建 VLA 的人。有 Feifei 和 3D 空间模型。当你思考让你对 Japa 模型感到兴奋的证据，以及你如何将它们与生成式方法进行比较时，你认为我们今天在比较这些架构和方法方面处于什么位置？

Obviously, you are pursuing world models in that broader world. And I think there are other people that have come at the world model pace from a more generative approach. So you have the Google folks and Genie in the video models. You have folks building VLAs on the robotic side. You have Feifei and the 3D spatial models. As you think about the body of evidence that got you excited about the Japa models and how you compare them to what the generative folks have done, where do you think we are today in terms of comparing these architectures and approaches?

好的，世界模型现在正迅速成为一个流行词，在研究领域和某种程度上在工业界也是如此。如果你愿意，可以说有两个派别。我不打算讨论 VLA，因为 VLA 显然被认为没有前途。它确实不起作用。VLA 是视觉语言动作模型，对吧？基本上是利用大语言模型技术训练一个系统来产生动作，用于控制机器人之类的东西。所以输入是视觉和语言，输出是动作。可能也有语言输出。现在这基本上被认为是一个失败。不够可靠，需要太多训练数据，等等。好的，那么还有世界模型。那么什么是世界模型？在区域层面，世界模型是允许一个智能体系统预测自身行为后果的东西。预测自身行为的后果。在我看来，我无法想象你如何能考虑构建一个智能体系统，而该系统没有预测自身行为后果的能力。这是非常基本的，对吧？当我们在世界中行动时，我们有这种能力。当我们不考虑后果就采取行动时，我们是在冒很大的风险。而且很多时候，别人会觉得我们是白痴。目前在国际政治舞台上有很多例子，有些人完全没有预测自己行为后果的能力。所以这就是那个模型。就是这样，对吧？预测自身行为后果的能力。如果你有这种能力，那么你就可以规划一系列行动来完成任务，满足目标。你通过规划、推理、搜索和优化的过程来做到这一点。你不是像真正的 AI 那样自回归地一个接一个地预测动作。你是通过搜索一系列能完成你为自己设定的任务的行动来做到的。所以这个蓝图与目前大语言模型所能做的完全不同。大语言模型没有预测自身行为后果的能力，也没有任何规划能力。因为推理是通过预测下一个词元，对吧？不是通过搜索。所以就在这里，你有我认为对智能行为至关重要的两个特征。预测自身行为后果的能力。第二，通过优化、搜索进行规划的能力。找到一系列能产生正确结果的好的行动。然后还有第三个特征，即你如何预测自身行为的后果？如果我面前有一个水瓶。我意识到有些人只是听这个而没有画面。所以我面前有一个打开、没有盖子的水瓶。如果我推底部，它会在桌子上滑动。如果我推靠近顶部的地方，它可能会翻倒。我们无法精确预测瓶子会朝哪个方向掉落。我们无法精确预测它会如何滑动，水会如何洒出，桌子是否向一个方向倾斜，水会朝一个方向或另一个方向流动。我们无法在像素级别预测这一点。所以我们的大脑世界模型在抽象的表示层次上进行预测。

Okay, so world model is quickly becoming a buzzword right now, in research and also in industry to some extent. And there are two factions, if you want. I am not going to talk about VLA because VLA is clearly now being seen as not going anywhere. It is really not working. So VLA is vision language action models, right? Basically use the LLM technology to train a system to produce actions for controlling a robot or something like this. So you have vision in, language in, action out. Maybe language out, too. And that is pretty much now seen as a failure. Not being reliable enough, requiring too much training data, things like that. Okay, then there is world models. So what is a world model? A world model at a regional level is something that allows an agentic system to anticipate the consequences of its own actions. Predict the consequences of its own actions. From my point of view, I cannot imagine how you can even think of building an agentic system without that system having the ability to predict the consequences of its actions. That is pretty essential, right? When we act in the world, we have this ability. And when we take an action without thinking about the consequences, we are taking a big risk. And very often, other people think we are an idiot. We have plenty of examples on the international political scene at the moment of people who have no ability to predict the consequences of their actions. So that is the one model. That is what it is, right? Ability to predict the consequences of your own actions. If you have this ability, then you can plan a sequence of actions to accomplish a task, to satisfy a goal. And you do this by planning, reasoning, by a process of search and optimization. You do not do this by predicting one action after the other autoregressively, like a real AI we do. You do this by searching for a sequence of actions that will accomplish the task you set for yourself. So the blueprint for this is completely different from what LLMs can do at the moment. LLMs do not have the ability to predict the consequences of their actions, and they do not have any planning abilities. Because inference is by predicting the next token, right? It is not by search. So right there, you have the two characteristics that I think are essential for intelligent behavior. Ability to predict consequences of your actions. And second, ability to plan by optimization, by search. Find a good sequence of actions that will produce the correct outcome. And then there is a third characteristic, which is how do you predict the consequences of your actions? If I have a water bottle in front of me. I realize some people would just listen to this and not have the picture. So I have an open, uncapped water bottle in front of me. If I push at the bottom, it is going to slide on the table. If I push near the top, it is probably going to flip. We cannot predict exactly how the bottle will fall in which direction. We cannot exactly predict how it is going to slide, how the water will spill, whether the table is tilted in one way and the water will flow in one direction or another. There is no way we can predict this at the pixel level. So our mental model of the world predicts that at an abstract level of representation.

当你研究这个架构时，它是否很大程度上受到人脑的启发？我的意思是，显然，你阐述的方式正是我们做事的方式。对吧，或者至少是认知科学，对吧？但能否将其转化为神经架构等等，这中间有很大的差距。

As you were working on this architecture, was a lot of it inspired by the human brain? I mean, obviously, the way you are articulating things is exactly how we do things. Right, or at least by cognitive science, right? Whether you can translate this into a neural architecture and things like this, there is a big gap there.

好的，所以认知科学确实是一个动机，或者说心理系统二，即你在深思熟虑的反思行为中的行为方式是：你想象、预测你行为的后果，并据此进行规划。与系统一相反，系统一只是反应性和本能地行动。所以，是的，有灵感，但也有大量经验证据表明你不想生成像素。

Okay, so that certainly cognitive science was a bit of a motivation, or what psychological system two, which is this idea of the way you behave in deliberate reflective behavior is that you do imagine, predict the consequences of your actions, and you plan accordingly. Contrary to system one, where you just act reactively and instinctively. So, yes, there is an inspiration, but also there is a lot of empirical evidence that you do not want to generate pixels.

我对通过预测来学习世界模型这个问题一直非常感兴趣。大约五年前，我有了一个顿悟，意识到所有成功学习图像和视频表示的架构都是非生成式架构。而所有生成式架构基本上都失败了。所以，VAE，即变分自编码器，或者更一般的自编码器，是一种很自然的学习输入抽象表示的方式。你把一张图像放到神经网络的输入层，然后训练它仅仅在输出层重现输入。但如果只是这样做，你的神经网络不会学到任何有趣的东西，它只会学习恒等函数。完全无趣，行不通。如果你训练一个 VAE 来学习图像的表示，你会得到一些东西，但效果并不好。稀疏自编码器也是如此。然后，还有另一类技术，它们源自一种叫做去噪自编码器的方法，掩码自编码器是它的一个变体。BERT 是自然语言处理中的一个版本。所以，你拿一张图像，以某种方式破坏它，然后训练这个大神经网络来恢复原始图像。FAIR 有一个关于这个的大型项目，叫做 MAE，即掩码自编码器。结果非常令人失望。计算量很大，但结果并不令人满意。与此同时，一些参与 MAE 项目的人，以及巴黎和纽约的其他一些人，正在研究使用非生成式架构的其他技术，即联合嵌入架构。所以，拿一张图像，以某种方式破坏它，然后将这两张图像通过编码器，然后尝试从被破坏图像的表示预测原始图像的表示。这就是 JEPA。JEPA 代表联合嵌入预测架构。所以，你有一个编码器进行一种观察，另一个编码器进行另一种观察。你尝试用一个预测器从第二个表示预测第一个表示。这些技术被证明在表示图像和视频方面要好得多。比如 DINO、DINO V1、V2、V3，这个项目仍在巴黎的 FAIR 进行。还有 I-JEPA 和 V-JEPA 等项目，在此之前还有 SimSiam 和 MoCo 等许多技术，主要来自 Meta。其他团队也有很多。但事实证明，这是一种比预测像素更好的学习图像表示的方法。所以，我（不只是我）突然意识到，这才是正确的方向，而预测像素是一条死胡同。

I've been really interested in that problem of learning models of the world by prediction for a very long time. And then had an epiphany about 5 years ago, realizing that all of the architectures that have been successful to learn representations of images and videos are non-generative architectures. And all the generative ones basically have been failures. So, VAE, variational autoencoders, or autoencoders more generally, is kind of a natural way to think about learning abstract representations of inputs. So, you put an image at the input of a neural net, and then you train it to just reproduce the input on its output. Now, with a big neural net, if you just do it this way, your neural net will not do anything interesting. It will just learn the identity function. Completely uninteresting. It doesn't work. If you train a VAE to learn representations of images, you get something, but it's really not that great. Same with sparse autoencoders. Then, you have another set of techniques, and it's kind of derivative of something called denoising autoencoder, masked autoencoder is a version of this. BERT is a version of this for NLP. So, you take the image, you corrupt it in some way, and then you train this big neural net to recover the original image. There's a huge project at FAIR on this called MAE, masked autoencoder. It was very disappointing. A lot of computation, and not really great satisfying result. Simultaneously, some of the same people working on MAE, and some other people in Paris and in New York were working on other techniques using non-generative architecture, joint embedding architecture. So, take an image, corrupt it in some way, and then run the two images through encoders, and then try to predict the representation of the original image from the representation of the corrupted one. That's JEPA. So, JEPA means joint embedding predictive architecture. So, you have one encoder that makes an observation, another encoder that makes a different observation. You try to predict the representation of the first one from the second one with a predictor. And those techniques turned out to work much better for representing images and video. So, things like DINO, DINO V1, V2, V3, project that is still going on at FAIR in Paris. Projects like I-JEPA and then V-JEPA, and before that there were SimSiam and MoCo and a bunch of different techniques mostly from Meta. There were a bunch of others from other groups. But that turned out to be a much better way of learning representations of images than predicting pixels. And so it just clicked in my mind, not just mine, that this was the way to go and predicting pixels was kind of a losing proposition.

感觉有很多来自一些模型公司的机器人演示越来越令人印象深刻，它们似乎表现出类似规划和推理的能力，即使之前没有见过某个房间或某个任务的特定版本，仍然能够执行该任务。您会对我们的听众说些什么？他们观察到这些，感觉我们正在朝着一些通用方法的真正进步迈进。

It feels like there are all these robotics demos released from some of the model companies that feel increasingly impressive and seem to resemble things like planning and reasoning when they maybe haven't seen a room or a specific version of a task before and are still able to execute that task. What would you say to our listeners that observe that and feel like we're trending toward some real progress with some of the general approaches?

嗯，确实有真正的进步，其中一些演示确实令人印象深刻。但它们是用大量数据训练的，这些数据要么来自远程操作，要么来自人类手持类似夹爪的物体进行操作，或者只是追踪人的手和手指，然后将其转换为机器人的指令。所以这些东西主要是通过模仿学习训练的，还有一点强化学习来微调，主要在模拟环境中。问题在于，你需要大量数据来训练系统进行模仿。这变得昂贵且有点脆弱，因为你需要为每个你想要机器人解决的任务收集大量数据。而如果系统有一个世界模型，允许它预测动作的结果，它就可以规划一个动作来解决新任务，而不需要实际训练来完成这个任务。因此，基于世界模型的系统所能获得的泛化程度要大得多，覆盖更广泛的任务，且需要的训练数据更少，而模仿学习和微调的系统则需要更多数据。

Well, there is real progress and some of those demos are really impressive. But they are trained with enormous amounts of data collected either from teleoperation or from human action with things you hold in your hand that look like grippers, or just tracking hands and fingers of a person and then translating that into commands for a robot. So those things are trained with imitation learning mostly, and a little bit with reinforcement learning to fine-tune mostly in simulation. The issue with this is that you need a lot of data to train the systems to imitation. And it becomes expensive and a little brittle in the sense that you need to collect lots of data for every task you want the robot to solve. Whereas, if the system had a world model that allowed it to predict the outcome of an action, it would just plan an action to solve a new task without actually having to be trained to accomplish this task. So, the degree of generalization you would get with a world model-based system is much larger, a wider spectrum of tasks with less training data required than a system trained with imitation learning and fine-tuning.

毫无疑问，这些方法需要更多数据。我想泛化问题确实是关键问题。我认为有些人已经展示了一些结果，表明在任务 A 上做得更好有助于任务 B，但这显然感觉那些架构仍然存在一个未解答的大问题。

No doubt those approaches require more data. And I guess this question of generalization really is the big question. I think some folks have shown some results around getting better at task A helps with task B, but that obviously feels like there's still the big unanswered question around those architectures.

你会得到任务之间的协同效应。所以，无论你使用什么技术，你训练系统解决的任务越多，它就能用少量数据获得更多任务。但世界模型的希望在于，系统可以零样本解决新任务，而人类完全能够做到这一点，许多动物也是如此。所以这才是真正的希望：用少量训练数据或完全没有训练数据，再加上一点点强化学习式的微调，就能解决更多问题。比如，一个 17 岁的少年怎么能在十几个小时或 20 小时内学会开车？我们有数百万小时的人类驾驶训练数据。我们仍然没有达到五级自动驾驶。所以，模仿学习显然连自动驾驶这一项任务都行不通。

You get this synergy between tasks. So, the more tasks that you train the system to solve, the more tasks it's going to be able to acquire with a small amount of data, regardless of what technique you use. But the hope with world models is that the system can solve new tasks at zero shot, which humans are completely capable of doing, and many animals as well. So that's really the hope: solving a lot more problems with either a small amount of training data or no training data at all, and just a little bit of maybe RL style fine-tuning. Like, how is it that a 17-year-old can learn to drive in a dozen hours or maybe 20 hours? We have millions of hours of training data of people driving cars. We still don't have level five self-driving cars. So, imitation learning obviously does not work even for just the task of autonomous driving.

我想这将是一场竞赛，一边是开发这些能力（可能需要时间和大量数据），另一边是这种架构。我觉得有一个梦想，就是用视频模型生成大量合成数据用于模拟，即使不完美，这些视频模型从物理角度来看也足以帮助改进现实世界中的机器人技术。您如何看待这些方法？显然，我认为英伟达一直专注于这方面。谷歌似乎也在走这条路。我再次问您这个问题：为什么一个 17 岁的少年能在 20 小时内学会开车？你不需要数百万小时的演示。

I guess it'll be a race between the ability to develop some of those capabilities, which may take time and lots of data, versus this kind of architecture. I feel like there's this dream of using video models to just generate tons of synthetic data for simulation, and even if it's not perfect, these video models from a physics perspective are helpful enough to improve robotics in the underlying physical world. What have you made of some of those approaches? Obviously, I think Nvidia's been focused there. Google seems to be going down that road. I'm sort of asking you again the question: why can a 17-year-old learn driving in 20 hours? You don't need millions of hours of demonstration.

这正是关键所在。人类和动物通过观察和互动学习世界模型，而不是通过数百万个带标签的示例。所以关键在于构建能够高效学习世界模型的系统，比如 JEPA，而不是依赖大规模的模仿学习。来自视频模型的合成数据可能会有帮助，但根本挑战在于把架构做对。

That's exactly the point. Humans and animals learn world models through observation and interaction, not through millions of labeled examples. So the key is to build systems that can learn world models efficiently, like JEPA, rather than relying on massive imitation learning. Synthetic data from video models might help, but the fundamental challenge is to get the architecture right.

而且你不需要合成数据。你不需要那些东西。所以，我想要一个能像那样快速学习的系统。如果我们攻克了这一点，那我们就不需要生成数据了，对吧？我的意思是，我们可能需要在模拟中训练系统，但不需要当前系统所需那么多时间或试验。这其实是一个数据效率的问题。

And you don't need synthetic data. You don't need any of that. So, I want a system that can learn as fast as that. If we crack that, then we don't need generated data, right? I mean, we might need to train the system in simulation, but not with the same amount of time or trials as current systems require. It's really a question of data efficiency.

我曾在播客中采访过 Jerry Tworek。他曾在 OpenAI 工作，后来离职创办了自己的实验室。你能感受到一种类似的张力，我认为他可能甚至同意，如果你继续像我们这样扩展强化学习，你会得到更令人印象深刻的结果。但我认为他觉得，「天哪，肯定有某种更高效的方法来做这件事。」这很有趣，因为你可以想象，如果你是 OpenAI，你知道某件事会持续下去，你可以继续扩展它，它会变得更好。从商业角度来看，不一定有很强的动力去做更数据高效的事情。

I was interviewing Jerry Tworek on the podcast. He was at OpenAI and spun out to start his own lab, and you could sense a similar tension where I think he actually might even agree that if you continued scaling RL the way we're scaling, you get more impressive results. But I think he felt, 'God, there's just got to be some way more efficient way to do this.' And it's interesting because you could imagine if you're OpenAI and you know something is going to continue, you could continue scaling it and it will keep getting better. There's not a ton of incentive necessarily from a business perspective to do something more data-efficient.

对。其他公司也没有动力去做不同的事情，因为他们都在追逐同样的东西，他们不能落后于别人，对吧？所以他们都做同样的事情。在硅谷有点从众行为，每个人都在挖同一条沟。所以我特意把 Amy Labs 的总部设在巴黎。美国办公室在纽约，而不是硅谷。

Right. And there's no incentive for the other companies to do anything different either because they're all chasing the same, they can't afford to fall behind the others, right? So they all work on the same thing. There's a bit of herd behavior in Silicon Valley where everybody is digging the same trench. So I purposely set up the headquarters of Amy Labs in Paris. The American office being in New York, not Silicon Valley.

这真的很有趣，因为我认为它指向了当今更广泛生态系统中存在的一种张力：你可以想象另一方确信，也许有更数据高效的方法存在，但几乎谁在乎呢，因为我们可以继续扩展我们已有的东西，取得越来越好的结果。然后显然，我认为从这些模型能实现的新事物，以及作为研究者发现这些新事物的乐趣来看，我理解为什么这些其他架构也如此有吸引力。

It's really interesting because I think it points to a tension that exists in the broader ecosystem today where you could imagine the other side being sure, maybe there are more data-efficient methods out there, but almost who cares because we can keep scaling what we have to better and better results. And then obviously I think from both new things you can accomplish from these models as well as just the joy of being a researcher and finding these new things, I get why there's such an attraction to these other architectures as well.

这是一个赌注。但我们相当有信心，因为我们实际上已经有结果了。

And it's a bet. But we're pretty confident because we have results already, actually.

当你思考 Amy 技术最让你兴奋的初始领域时，你认为这项技术会走向何方，你最兴奋的是什么？

As you think about the initial spaces you're most excited about for the Amy technology, where do you think the technology goes and what are you most excited about?

嗯，AI 应用于现实世界。你的家用机器人在哪里？你的五级自动驾驶汽车在哪里？这就是我兴奋的地方。

Well, AI for the real world. Where is your domestic robot? Where is your level five self-driving car? That's what I'm excited about.

我什么时候才能得到家用机器人？我对此很兴奋。

When am I going to get a domestic robot? I'm excited about this.

嗯，这还需要几年时间。尽管有大量公司在制造机器人，但这些公司实际上都不知道如何让它们足够智能以变得有用，或者能在有婴儿的家中被信任。但即使是相对狭窄的制造任务，除了通过模仿学习少量任务外，它们中没有一个真正知道如何可靠地做到这一点。那么我们如何让这些东西变得有用呢？这是一个相对长期的目标。短期内，工业中有大量应用需要智能系统，能够预测如果我改变这个复杂系统上的控制变量会发生什么，无论是喷气发动机、化工厂、发电厂、生产线、病人还是人体细胞。这些系统足够复杂，无法用少量方程来建模其行为。传统的建模方法行不通。你需要做的是训练一个神经网络、深度学习系统，从数据中建模该系统的动态。最终得到的是该过程、该系统的现象学模型。如果它是动作条件化的，那么你基本上得到了该系统的世界模型，允许你为任何目的优化控制它。我认为这在工业中的应用数量是惊人的。

Well, this is several years down the line. Despite the fact that there are a huge number of companies building robots, none of those companies actually has any idea how to make them smart enough to be useful, or trusted around with a baby in the house. But even for relatively narrow manufacturing tasks, none of them really knows how to do this reliably other than by imitation learning for a small number of tasks. So how do we make those things useful? That's a relatively long-term objective. Shorter term, there is a huge amount of applications in industry where you need an intelligent system that has the ability to predict what's going to happen if I change this control variable on this complex system, be it a jet engine, a chemical plant, a power plant, a manufacturing line, a patient, a human cell. Those are systems that are sufficiently complex that you can't model their behavior with a small number of equations. The traditional way of modeling does not work. What you need to do is train a neural net, deep learning system, to model the dynamics of that system from data. What you get at the end is a phenomenological model of that process, of that system. And if it's action-conditioned, then you get basically a world model of that system that allows you to control it optimally for whatever purpose you have. I think the number of applications of this in industry is mind-boggling.

你认为未来几年我们在通用模型方面会达到什么水平？有没有你可以指出的里程碑，或者你对这里的进步路径有什么看法？

Where do you think we'll be with general models over the next couple years? Are there milestones you'd point to or what's your view of the path of progress here?

好吧，几年有点短。比如，5 年，基本上完全统治世界。这是 Linus Torvalds 的一句话。当人们问他「你对 Linux 的目标是什么？」他说「完全统治世界。」他实际上做到了。粗略地说，世界上每台电脑都运行着 Linux。所以这有点开玩笑。但最终，我认为这是未来智能系统的蓝图。仍然会有一小部分留给大语言模型，基本上作为语言接口。但我们正在设计的是能够思考的系统。它们最初可能不会说话或听，但它们会思考。然后你可以在那之上添加说话和听的能力。

Okay, a couple of years is a little short. Like, 5 years, complete world domination, essentially. This is a quote from Linus Torvalds. When people ask him, 'What's your goal with Linux?' He said, 'Total world domination.' He actually managed to do that. To first approximation, every computer in the world runs Linux. So that's kind of a joke. But in the end, I think this is the blueprint for intelligent systems of the future. There will still be a small place for LLMs, for a language interface basically. But what we're designing are systems that are capable of thinking. They may not be capable of talking or listening initially, but they'll do the thinking. And then you can add the talking and listening on top of that.

我相信你和团队正在热切地努力获得早期的证明点。你如何看待在通往 5 年世界统治的道路上你能展示的中间步骤？

I'm sure you and the team are eagerly working to get the early proof points of this. How do you think about the interim steps of what you'll be able to show on that path to 5-year world domination?

嗯，大约一年内，我们将有一种通用方法来训练非常广泛模态上的分层世界模型。我们知道我们可以在视频上做得很好，使用一些我们不完全满意因为有一些缺点的技术，但我们有我们认为是真正想要的方法的小规模演示。所以我们需要扩大那个方法，使其在视频等事物上达到与其他不那么令人满意的技术相同的性能水平，同时也在我们从行业合作伙伴那里获得的其他类型数据集上。所以我们将有演示，证明我们可以训练世界模型，也许是动作条件化的世界模型，允许我们为许多不同用例进行规划。

Well, within a year or so, we'll have a general methodology to train hierarchical world models on a very wide variety of modalities. We know we can do a good job on video with some techniques that we're not completely happy with because they have some shortcomings, but we have sort of small-scale demonstration of a methodology that we think is really what we want. So we need to scale that one up and get it to the same level of performance as the other techniques that are not as satisfying on things like video, but also on other types of data sets that we would get from industry partners. So we'll have demonstrations that we can train world models, perhaps action-conditioned world models that allow us to plan for a number of different use cases.

其中一些将是机器人技术，一些将是各种类型的工业过程控制，也许还有一些是医疗保健，因为我们在那个领域有合作伙伴。这应该在一两年内，18 个月左右。然后我们将把这种方法论和这些模型推向这些用例，与合作伙伴一起，其中一些已经是我们的投资者，并积累经验，如何构建一个某种程度上通用的世界模型。我的意思是，你显然以前有过这种经历，对神经网络下了一个非常逆向的赌注，并且在历史书中被证明是完全正确的。我想当你思考这个赌注时——我认为如果你和大多数处于 AI 各个前沿领域的人交谈，他们会说这在今天是逆向的——你认为在什么时间框架内它会变得明显是正确的？

Some of them will be robotics, some of them will be industrial process control of various types, maybe some of them in healthcare as well because we have partners in that domain. And that should be within a year or two, 18 months. And then we'll push this methodology and those models into those use cases with partners, some of which are investors already in our company, and gain experience on how to essentially build a somewhat universal world model, if you want. I mean, you've obviously had this experience before of making this really contrarian bet on neural nets and being proven abundantly right in the history books. I guess as you think about this bet, which I think if you talk to the majority of people maybe at the cutting edge of various parts of AI would say is contrarian today. In what time frame do you think it will become apparent that this was right?

我认为它会比预期更快发生，也许是因为你可以看到世界模型已经成为一个流行词，对吧？至少在研究层面是这样。它开始渗透到工业界。很多人意识到 VLM 很糟糕，LLM 不适用于现实世界的数据。工业界已经意识到了这一点，当然在用户端。由于机器人技术的重要性，很多人都在试图弄清楚如何达到那个目标，如何让那些机器人变得有用。所以我认为，需要改变范式的认识正在发生，并且到 2027 年初，人们会完全明白这一点。但这并不意味着到那时我们会有解决方案。我们希望我们有，但走着瞧。

I think it'll happen faster than expected, perhaps because you can see that world model is already becoming a buzzword, right? At least at the research level. And it's starting to permeate into the industry. A lot of people are realizing that VLMs suck and LLMs don't work for real-world data. Industry has realized this already, certainly on the user side. And because of the importance of the robotics industry, a lot of people are trying to figure out how to get there, how to make those robots useful. So I think the realization that you need a change of paradigm is happening as we speak and will become completely obvious to people by early 2027, I think. Now, that doesn't mean we'll have a solution by then. We hope we will, but we'll see.

切换到语言模型方面，你提到了你正在与 Tapestry 合作的一些工作，我认为这对我们的听众来说会非常有趣。所以也许谈谈这个。

Switching gears to the LM side, you mentioned some of this work you're doing with Tapestry, which I think would be really interesting for our listeners. So maybe speak to that a little bit.

好的，这有点与 ML Labs 正交，好像那还不够让你忙似的。这是我过去三年左右一直在酝酿的一个想法：人们越来越多地使用 AI 助手做各种事情。你看到传统搜索引擎的使用在减少，你直接问你最喜欢的 AI 助手一个问题。如果 Meta 和其他公司正在开发的智能眼镜等智能设备的计划得以实现，基本上你只需通过语音与你的 AI 助手对话，对着你的智能眼镜或其他智能设备。那么你所有的信息摄入都将由 AI 助手来中介。如果你在世界某个地方，比如美国或中国以外，你有一个在加州或北京、上海、深圳构建的 AI 助手，这对你不好。你可能说一种这些系统没有经过特别良好训练的语言。你可能有一种不被硅谷和中国的人们很好理解的文化，在互联网上公开可用的训练数据中没有得到很好的体现。你可能有一套完全不被构建这些模型的人所代表的价值观。而且你几乎肯定会有完全不被你能从西海岸科技公司或中国公司得到的少数 AI 助手所代表的政治观点。那么解决方案是什么？你如何服务于印度的一个农民，甚至法国或德国的一个哲学家？你需要的是一个平台，一个开放的、免费的、LLM 风格的基础模型，任何人都可以对其进行微调，以满足说特定语言、拥有特定文化、价值观、政治偏见、信仰等的人们的兴趣。所以你需要的是多样化的 AI 助手。世界上有很多国家既不是美国也不是中国，他们绝对希望 AI 有一定程度的主权，不仅是为了他们的工业，也是为了他们的公民。他们不希望他们的公民被中国模型或加拿大模型洗脑。所以他们想要主权。如何实现？要让这样一个平台达到前沿，你只需在比专有系统更多、更高质量的数据上训练它。如果你和印度、法国、越南、摩洛哥、瑞士、韩国、日本、哈萨克斯坦的人交谈，每个人都想要主权。你告诉他们，你们一直在本地训练你们的模型。你们不必分享数据。这就是 Tapestry 的关键方面。你将拥有国际贡献者，他们为训练一个全球模型做出贡献，这个模型将构成所有世界知识和文化的存储库。但贡献者贡献数据和计算资源，但他们保留对数据的控制权。他们不必与其他贡献者分享数据。他们贡献的是参数向量。有趣。所以这有点像联邦学习风格的东西，你有一堆数据中心。他们从模型的全局共识中获得参数向量。可以把它看作所有贡献者参数向量的平均值。所以所有贡献者定期通过一个中央服务器告诉其他人，这是我的参数向量，你的呢？他们这样交换参数向量。一个本地工作者基本上，每当它更新其参数向量时，它试图使其尽可能接近全局共识向量。随着训练的进行，所有这些参数向量收敛到一个共识模型，本质上是一个所有人类知识的存储库。现在你有一个开放的模型，它和用世界上所有数据训练出来的模型一样好。现在你可以根据自己的目的、自己的政治、文化和语言偏见，或者任何你感兴趣的中心来微调它。我认为有一种自然的力量推动这件事发生，因为大多数既不是美国也不是中国的国家都想要主权，而且因为 AI 正在迅速成为一个平台。

Okay, so this is a little bit orthogonal to ML Labs, as if that wasn't enough to keep you busy. It's an idea I've been forming over the last three years or so: the fact that people increasingly use AI assistants for various things. You see a decrease in the use of traditional search engines and you just ask a question to your favorite AI assistant. And if the plan that Meta and others are developing of having smart devices like smart glasses is realized, basically you'd just be talking to your AI assistant by voice to your smart glasses or some other smart device. So all of your information diet will be mediated by AI assistants. If you are someone somewhere in the world, let's say outside the US or China, and you have an AI assistant built in California or Beijing or Shanghai or Shenzhen, it's not good for you. You may speak a language that those systems haven't been trained to handle particularly well. You may have a culture that is not particularly well understood by people in Silicon Valley and China, not well represented by the training data publicly available on the internet. You may have a value system that is absolutely not represented by people building those models. And certainly you'll almost certainly have political opinions that are absolutely not represented by the handful of AI assistants you might get from West Coast tech companies or from Chinese companies. So what is the solution? How do you serve a farmer in India, or even a philosopher in France or Germany? What you need is a platform which is an open, free foundation model, LLM style, that is fine-tunable by anyone to cater to the interests of people speaking a particular language, having a particular culture, value systems, political biases, creeds, whatever. So what you need is a wide diversity of AI assistants. There are a lot of countries around the world that are neither the US nor China, who absolutely want some level of sovereignty for AI, not just for their industry but also for their citizens. They don't want their citizens to get brainwashed by a Chinese model or a Canadian model. So they want sovereignty. How do you get that? The way you get a platform like this to get to the frontier is you just train it on more and higher quality data than the proprietary systems. If you talk to people in India, France, Vietnam, Morocco, Switzerland, Korea, Japan, Kazakhstan, everyone wants sovereignty. And you tell them, you guys have been training your model locally. You don't have to share your data. That's the crucial aspect of Tapestry. You would have international contributors contributing to training a global model that would constitute a repository of all the world's knowledge and culture. But the contributors would contribute data and computing resources, but they would preserve control on their data. They would not have to share their data with other contributors. What they would contribute is parameter vectors. Interesting. So it would be a kind of federated learning style thing where you have a bunch of data centers. They get the parameter vector from the global consensus of a model. Think of it as an average of all the parameter vectors of all the contributors. So all the contributors periodically tell everyone else through maybe a central server, here is my parameter vector, what is yours? And they exchange parameter vectors like this. A local worker basically, whenever it updates its parameter vector, it tries to make it as close as possible to the global consensus vector. So as the training progresses, all those parameter vectors converge towards a consensus model, essentially a repository of all human knowledge. Now you have an open model that is as good as if it had been trained on all the data in the world. And now you can fine-tune it for your own purpose, your own political, cultural, and linguistic biases, whatever you want or centers of interest. I think there is a natural force for this to happen because most countries that are not the US nor China want sovereignty, but also because AI is fast becoming a platform.

平台有一种自然趋向于开放的趋势。Linux 就是这样，对吧？互联网的软件基础设施或无线网络也是如此。全都是开源的。最初是专有的，但后来都被淘汰了。这是一种非常聪明的方式来规避这种开源减少的趋势。显然，我认为有很多担忧，即随着闭源模型变得更好，它们会被压制并用于训练下一代，这几乎会形成一种闭源模型的「逃逸」场景，让它们变得比开源对手好得多。所以，还记得 1996 年互联网基础设施的主要玩家是谁吗？Sun Microsystems、HP、Dell 和其他几家。Sun Microsystems 销售 Solaris 及其专有硬件。HP 销售 HP-UX，他们声称 Unix 比 Windows 可靠得多，你不会在 Windows 上运行 Web 服务器。Dell 用 Windows NT 做同样的事，但现在谁还用 Windows NT 作为 Web 服务器？所有这一切都被 Linux 彻底淘汰了。整个互联网都运行在 Linux 上。甚至 Azure，甚至微软，都运行 Linux。所以，今天的 OpenAI 和 Anthropic 等公司就是昨天的 Sun Microsystem 和 HP-UX。

And there is a natural tendency for platforms to become open. That's what happened with Linux, right? And that's what happened with the software infrastructure of the internet or the wireless network. It's all open source. It was proprietary initially, but that was all wiped out. It's a really clever way to get around this trend of decreasing open source. Obviously, I think there have been many fears that as closed source models get better, they'll be held back and used to train the next generation, and they'll kind of be this almost like a scape scenario for closed source models where they get so much better than their open source counterparts. So remember who the big players of the internet infrastructure were in 1996: Sun Microsystems, HP, Dell, and a few others. Sun Microsystems was selling Solaris with their proprietary hardware. HP with HP-UX, they were claiming Unix is so much more reliable than Windows, you're not going to run a web server on Windows. Dell was doing this with Windows NT, but who is running Windows NT now as a web server? All of this was totally wiped out by Linux. The entire internet runs on Linux. Even Azure, even Microsoft, it runs Linux. So basically OpenAI and Anthropic etc. of today are the Sun Microsystem and HP-UX of yesterday.

是的，我的意思是，这隐含着你显然认为这些模型的能力有限，只能变得那么好，所以随着时间的推移，开源人士有可能赶上。他们已经用完了数据，对吧？公开可用的文本数据已经全部用完了。没有更多了。所以这些公司正在做的是授权商业版权数据或使用合成数据进行训练。

Yeah, I mean, I guess it's implicit in that is obviously your view of the limitations of what these models can only get so good, and so it'll be possible over time for the open source folks to catch up. They've already run out of data, right? The openly available publicly available text data is already all used. There's not more of it. So what those companies are doing is licensing commercial copyrighted data or training on synthetic data.

我很好奇，因为显然在过去几年里，他们在大规模预训练之后取得了一些令人印象深刻的成果。IMO 金牌，meter task horizon 基准不断提高。好吧，这很有趣。现在，想想这两个领域，对吧？数学和代码。这两个领域语言本身就是推理的基质。它不是唯一的推理基质，但当你做数学时，很多情况下，在纸上进行形式化操作，而不是直觉性的东西，你是在操作语言，对吧？而 LLM 非常擅长这个。所以，证明定理之类的事情，正是 LLM 所擅长的。它们不太擅长提出好的概念和定义之类的东西。更像是，这里有一个问题，解决它。它们是问题解决者。数学不仅仅是解决问题，对吧？大部分实际上是创造性的行为，这些东西做不到。代码也是如此。所以，LLM 是好的程序员。它们不是软件架构师。它们不是计算机科学家，对吧？但它们可以为我们编程。所以，它们还没有达到可以完全取代人类的状态。它改变了人类的世界。所以，人类现在在抽象层次上上升了一层，我们的世界是决定构建什么。但构建它，你可以从 LLM 那里得到帮助。但好吧，重要的一点是，LLM 在语言本身就是推理基质的领域特别成功，而不是其他领域。

I guess I'm curious because obviously there have been some impressive results in the last few years that they have been able to drive, you know, post these large-scale pre-trainings. IMO gold, the meter task horizon benchmark keeps going up. Okay, that's very interesting. Now, think about those two domains, right? Mathematics and code. Those are two domains where the language itself is the substrate of reasoning. It's not the only substrate of reasoning, but a lot of when you do mathematics, the formal way on a piece of paper, not the intuitive stuff, but you manipulate language, right? And LLMs are really good at this. So, proving theorems and stuff like that, that's what LLMs are really good at. They're not so good at coming up with good concepts and definitions and things like that. It's more like, here is a problem, solve it. They're problem solvers. Mathematics is not just problem solving, right? Most of it is actually a creative act that those things don't do. And same for code. So, LLMs are good programmers. They're not software architects. They're not computer scientists, right? But they can program for us. So, they're not in a state where they can just replace humans entirely. It changes the world of humans. So, humans now kind of go one level up in the abstraction hierarchy and our world is to decide what to build. But building it, you can get help from LLMs. But okay, that's the important point is that LLMs are particularly successful at domains where the language itself is the substrate of reasoning, not for anything else.

是的。LLM 需要做什么才能让你改变看法？所以，我的意思是，比如一个零样本的智能体系统，对吧？你有一个智能体系统。给它一个新问题。它没有被训练来解决那个问题。没有针对它的脚本。它能否完成这个从未被训练解决的任务？除非系统有能力预测其行动的后果，并利用这些进行规划，否则它无法做到。而你不能用 LLM 做到这一点。你或许可以用一个显著增强的 LLM 来做到，它能够进行搜索和规划等。目前，做数学和代码的 LLM 实际上做到了这一点。对吧？因为它们搜索能够完成特定任务的 token 序列，并且可以运行代码或验证证明是否正确等等。所以，你有办法检查产生的东西是否正确。但这不是一种非常高效的规划方式。而且它只适用于可以在 token 空间中进行这种搜索的领域。我在 Jeppa 中说的是，你不在 token 空间中做这个。你在抽象思维空间中做这个。

Yeah. What would an LLM need to do to convince you otherwise? So, I mean, like a zero-shot agentic system, right? You have an agentic system. Give it a new problem. It's not been trained to solve that problem. Doesn't have a script for it. Is it going to be able to accomplish this task that it's never been trained to solve? And unless the system has the ability of predicting the consequences of its actions and then using that for planning, it's not going to be able to do it. And you're not going to do this with an LLM. You're going to do this perhaps with a significantly augmented LLM that is capable of search and planning, etc. And currently, LLMs that do math and code actually do this. Right? Because they search for sequences of tokens that actually accomplish a particular task and they can run the code or verify that the proof is correct or whatever. So, you have a way of checking whether something produced is correct. But that's not a very efficient way of doing planning. And it only works in domains where this type of search can be performed in token space. What I'm talking about with Jeppa is you don't do this in token space. You do this in abstract thought space.

我相信一些听众可能会想，好吧，即使它效率不高，但它有效，而且它在 token 空间中完成的事情上有效，这仍然是经济的一大部分。

I'm sure some people listening might think, well, if even if it's inefficient and it works, and it works at things that are done in token space, that's still a large part of the economy.

我的意思是，如果它有效，那就没问题。利用 LLM 做它们擅长的事情并没有错。但这并不是通往人类水平 AI 的道路。你错过了一个巨大的领域。

I mean, if it works, it's fine. There's nothing wrong with using an LLM for what they're good at. It's just not a path towards human level AI. You're missing a huge domain.

你似乎认为它在成为软件架构师之前就会达到极限，而我确信它会达到极限。它只能有限地部署到越来越多的用例中，因为你需要为每个用例收集大量的训练数据。而且你无法让这些系统完全可靠，没有幻觉或危险的东西，除非这些系统有能力预测其行动的后果，这意味着它们必须拥有明确的世界模型。

You seem like you think it's going to tap out before it can become a software architect, whereas I'm sure it's going to tap out. It's just going to have a limited ability to be deployed for an increasingly large number of use cases because you're going to have to collect tons of training data for each of those use cases. And you're not going to be able to make those systems completely reliable, without hallucinations or dangerous stuff, unless those systems have the ability to predict the consequences of their actions, which means they're going to have to have explicit world models.

是的，所以我想这是对 100% 准确性和跨任务泛化的一个赌注。

Yeah, so I guess it's a bet against 100% accuracy and also the generalization across different tasks.

对。我想，这个领域发展方式的一个有趣之处在于，显然你和另外两人共同获得了图灵奖，我觉得他们似乎更相信 LLM 随着时间的推移可能带来的力量、潜在威胁或安全风险。我想知道你的观点是什么时候开始分化的？

Right. I guess, one thing that's so interesting about the way that the field has developed is obviously you share the Turing Award with two others and I feel like they seem much more convinced of maybe the power or potential threats or safety risks of LLMs over time. I'm wondering when did your view start diverging?

2023 年。是什么促使你改变想法？我没有改变想法。

In 2023. And what drove that in your mind? I didn't change my mind.

他们改变了想法，对吧？而且差不多在同一时间，就是 GPT-4 出来的时候。我的意思是，杰夫基本上和那一切没有关联。他从来对 LLM 不感兴趣，直到 2023 年 GPT-4 发布时才注意到。然后他恍然大悟，说：「天哪，这些系统真的接近人类水平的智能，而且它们可能有主观体验。」他快速算了一下：「人类大脑皮层大约有 160 亿个神经元。如果你想做类似反向传播的事情——大脑并不直接做反向传播——但如果它做类似反向传播的事情，比如某种针对某个目标函数的梯度估计，你可能需要几个神经元组成的网络来复制神经网络中一个虚拟神经元的功能。」所以他说：「假设你需要 10 个实际神经元来复制一个反向传播神经元的功能，那么你的大脑皮层突然就只有 16 亿个神经元了。天哪，GPT-4 非常接近这个数字。所以也许它和人类一样聪明。」我完全不相信这个说法。这基本上是杰夫在说：「好了，我可以退休了。我可以宣布胜利。我整个职业生涯都在寻找大脑皮层的学习算法。也许我没有发现它到底是什么，但反向传播似乎是一个很好的替代品。它工作得非常好。所以也许这就是我们所需要的。所以我可以退休，环游世界，谈论 AI 的潜在前景和危险。」我认为这就是他的思想轨迹。他现在对潜在危险的强调比一两年前少多了。他意识到可能有一种方法可以设计出真正智能的系统。所以，首先，他可能意识到当前的 LLM 并没有那么聪明。其次，在达到人类水平的智能之前，可能还需要一些概念上的突破。第三，这些系统的蓝图将与 LLM 截然不同，而且我们可能有办法让它们可控。

They changed their mind, okay? And at about the same time, and it was basically GPT-4. I mean, Jeff basically was not connected to any of that. He was never really interested in LLMs and discovered GPT-4 in 2023 when it came out. And basically he had an epiphany and said, "Oh my god, those systems are really close to human-level intelligence and they possibly have subjective experience." And he did a quick calculation saying, "Okay, the human cortex has about 16 billion neurons. If you want to do something like backprop, the brain doesn't do backprop directly. But if it does something like backprop, like some sort of gradient estimation for some objective function, you would probably need a network of a few neurons to reproduce the functionality of a virtual neuron in a neural net." So he said, "Let's assume maybe you need a circuit of 10 actual neurons to reproduce what a backprop neuron does. Then all of a sudden your cortex is only 1.6 billion neurons. Oh my god, GPT-4 is really close to this. So maybe it's as smart as humans." I do not believe in this claim at all. This is Jeff's way of saying, "Okay, basically I can retire. I can declare victory. I searched for the learning algorithm of the cortex all my career. Maybe I didn't discover what it really was, but backprop seems to be a good substitute for it. It works really well. So maybe that's all we need. So I can retire and go around the world giving talks about the potential promises and dangers of AI." That's basically what I think his intellectual trajectory has been. He's much less vocal about the potential dangers now than he was a year or two ago. He kind of realized there's probably a way to design truly intelligent systems. So, first of all, he probably realized that current LLMs are not that smart. And second, that there's probably a need for a few breakthroughs like conceptual breakthroughs before we get to human-like intelligence. And third, that the blueprint of those systems will be quite different from LLMs and we probably have a way of making them controllable.

我多年来一直在说这个，但好吧，他最近才发现了这一点。是的。和约书亚的情况类似。我认为他们俩都担心的是社会和政体确保 AI 利益最大化的能力，以及 AI 不会仅仅让少数富人更富、加剧不平等、并因滥用导致重大灾难。这不是 AI 接管世界的末日情景，而是糟糕的用户。今天的 LLM 能做什么？这是一种危险，但我不认为它像一些人声称的那样是世界末日。当然不像 Anthropic 声称的那样末日。他们试图游说政府，吓唬政府去监管 AI。我完全不赞同。他们似乎真的相信这一点，但我认为也有一些商业原因让他们相信，并洗脑一些人和政府，让他们认为自己的系统是危险的。

I've been saying this for years, but okay, he's sort of discovered this recently. Yeah. Same kind of thing with Yoshua. I think what they are both worried about is the ability of society and the political system to make sure that the benefits of AI will be maximized, and AI would not just profit a few rich people even richer, accentuate inequalities, and cause major catastrophes because of bad usage. This is not the doomer scenario of AI taking over the world. It's more bad users. What seems possible with the LLMs of today? Which is a danger, but I don't think it's as apocalyptic as some people have claimed. Certainly not as apocalyptic as even Anthropic has claimed. And they're trying to lobby governments into scaring governments into regulating AI because of that. I don't subscribe to this at all. They seem to genuinely believe it, but also I think there are some commercial reasons for them to believe that, and to brainwash some people and governments into thinking their systems are dangerous.

听起来，对于这些其他架构，你认为……显然，尽管你对 LLM 作为一切终局持悲观态度，你对这些新架构也有相当雄心勃勃的时间表。所以似乎你认为我们离一些非常引人注目的能力并不特别遥远。你怎么看待安全性问题——如果这些突破最终来自新架构，这是否应该让我们更放心？

And it sounds like with these other architectures, do you think they're... obviously it doesn't, as maybe bearish as you are on LLMs being the end state of everything, you have some pretty ambitious timelines too for these new architectures. And so it doesn't seem like you think we're particularly far away from some very compelling capabilities. How do you think about the safety around, if these breakthroughs end up coming from new architectures and whether that should make us rest easier or not.

我要说一些可能又有争议的话。当然，我在 Meta 的一些同事不喜欢我这么说，但我认为 LLM 有趣地不安全。我认为它们无法变得可靠和安全。它们无法变得可靠，因为你无法阻止它们产生幻觉。如果它们是智能体式的，你无法保证它们不会采取一个它们没有预测到结果的行动。

I'm going to say something that's again might be controversial. And certainly some of my colleagues at Meta didn't like me saying this, but I think LLMs are interestingly unsafe. I don't think they can be made reliable and safe. They cannot be made reliable because you can't stop them from hallucinating. And if they're agentic, you cannot guarantee they're not going to take an action that they didn't predict the outcome of.

考虑到对可靠性的担忧，它们能完成像 15 小时编码这样的任务，这让你感到惊讶吗？

Does it surprise you they can do these like 15-hour coding tasks given the concerns around reliability?

嗯，但编码是你可以实际验证生成的代码是否满足规范的事情。但并非一切都是编码。而且有例子表明编码智能体擦除了你的硬盘或做了蠢事，导致你损失大量金钱或数据。所以我认为当前形式的 LLM 本质上不安全，因为它们无法预测自己行为的后果，而且它们完成的任务受限于它们的训练。你给它们一个提示，它们会完成与该提示对应的任务，但仅限于它们的训练使它们能够正确执行该任务的程度。没有硬性约束迫使它们完成这个任务并预测任务会被正确完成。

Well, but coding is something where you can actually verify that the code you generate satisfies your specification. But not everything is coding. And there are examples of coding agents wiping your hard drive or doing stupid things that make you lose a lot of money or data. So I think LLMs in their current forms are intrinsically unsafe because they cannot predict the consequences of their actions and because the task they accomplish is determined subject to their training. You give them a prompt and they will accomplish a task corresponding to that prompt only to the extent that their training has conditioned them to do the right task. But there is no hardwired constraint that will force them to accomplish this task and then predict that the task will be accomplished properly.

我的意思是，我认为早期人们就在说，对吧？你问它们一个问题，它们会一直重复那个问题。

I mean, I think people were saying in the early days, right? They would ask them a question and they'd keep asking the question.

对。例如。或者它们也没有常识。一个月前流传的一个笑话：「我需要洗车，洗车店离我家 100 码，我应该走路去吗？」我大约两周前又试了一次。它们都说：「是的，你应该走路。」除了 Gemini。

Right. For example. Or also they don't have common sense. There's the joke that was circulating like a month ago: "I need to wash my car, and the car wash is 100 yards from my house, should I walk?" I tried it again maybe 2 weeks ago. They all say, "Yes, you should walk." Except Gemini.

它们是在你之前发表那个演讲的视频上训练的吗？

They're training on your video of having given that speech before?

那不是我的视频，因为是我提出这个例子的。

It was not my video because I came up with this example.

不管是谁提出的。

Whoever came up with it.

但它们确实在输出句子，对吧？我说过 LLM 能做这个，六个月后人们就在做了，仅仅是因为人们看了我说 LLM 能做这个的播客后，就把它输入 ChatGPT。所以它成了训练集的一部分。下一个版本就有了这个微调数据，当然它能回答这个问题，但并不是因为它突然变聪明了，只是因为它被明确地用那个问题训练过。所以 LLM 本质上是不安全的。我认为在当前范式下没有办法解决这个问题。我一直在提议的是目标驱动型 AI。基本上，你给 AI 系统一个目标：完成这个任务。系统如何知道它会完成这个任务？它有一个世界模型，并预测它想象采取的一系列行动的结果。如果这个结果满足一个描述任务完成程度的成本函数，那么系统通过优化，找到一系列能完成这个任务、根据其世界模型最小化这个成本的行动。它不能做别的。当然，很多地方可能出错。特别是，成本函数可能不准确。你认为衡量任务完成程度的成本函数可能不准确。世界模型可能不准确，所以系统做出的预测不对。因此系统仍然会犯错，但它能在一定程度上预测其行动的后果，这对任何智能体系统都是不可或缺的。你可以给这个系统添加的不仅是一个保证任务完成的成本函数，还可以添加许多其他目标函数或约束，即安全约束。比如「不要在路上伤害任何人」。你不能在抽象层面指定这一点，但你可以有低层次的目标函数，它们共同保证系统不会危险。系统在构造上不能违反这些条件。它必须满足这些条件。LLM 则不然。LLM 总能逃脱。训练误差和测试误差之间总有差距。总会有某个提示让系统做非常愚蠢的事情。

But they are issuing sentences, right? Where I said an LLM can do this, and then 6 months later people are doing it, simply because as soon as people watch the podcast of me saying LLMs can do this, they type it into ChatGPT. So it becomes part of the training set. And the next version has that in the fine-tuning set, and of course it can answer the question, but not because it becomes smart all of a sudden. It's just because it was explicitly trained with that question. So LLMs are intrinsically unsafe. I don't think there is any way to fix that in the current paradigm. What I've been proposing is objective-driven AI. Basically, you give an objective to an AI system: accomplish this task. How does the system know it will accomplish this task? It has a world model and it predicts the outcome of a sequence of actions it imagines taking. If that outcome satisfies a cost function that describes to what extent the task has been accomplished, then the system, by optimization, finds a sequence of actions that accomplishes this task, minimizes this cost according to its world model. It can do nothing else. Of course, many things can go wrong. In particular, the cost function might be inaccurate. It could be that the cost function you think measures the extent of task accomplishment is not accurate. The world model might be inaccurate, so the prediction the system makes is not the right one. So the system can still make mistakes, but it can predict the consequences of its actions to some extent, which is indispensable for any agentic system. You can add to that system not just a cost function that guarantees a task has been accomplished, but also a bunch of other objective functions or constraints that are safety constraints. Say, 'Don't hurt anybody on the way.' You cannot specify this at an abstract level, but you can have low-level objective functions that together guarantee the system will not be dangerous. The system cannot violate those by construction. It will have to satisfy those conditions. Not the case for an LLM. The LLM can always escape. There is a gap between training error and test error. There will always be a prompt where the system does really stupid things.

谈到 LLM 的一个具体领域，我觉得你显然对医疗保健中的 LLM 非常兴奋，人们已经在用 LLM 做各种事情。所以我很好奇，你认为哪些事情上 LLM 在医疗保健中行不通，而你需要一个更好地理解世界的模型。

To talk to one specific space around LLMs, I think you're obviously really excited about LLMs in healthcare, and people have been using LLMs in healthcare for all sorts of things. So I'm curious how you think about the set of things where LLMs are just not going to work in healthcare, and you need a model that understands the world better.

例如，为慢性病甚至非慢性病设计一个针对特定患者的疗程，这个患者可能不完全符合你之前观察到的模板。但如果你对患者生理动态有一个好的心智模型，那么你可能会设计出一个真正能让患者恢复良好状态的疗程。我说患者时，也可以是一个细胞。如何告诉干细胞变成产生胰岛素的胰腺β细胞？你有一个 1 型糖尿病患者。他们的免疫系统基本上吞噬了自己的β细胞。这是自身免疫。如何持续制造β细胞？你能发送信息吗？你有一个人类细胞模型，能让你找出需要向干细胞发送什么序列的信息，使其变成β细胞吗？

Designing a course of treatment for a chronic disease, for example, or even a non-chronic disease, for a particular patient, which may not completely fit into templates you've observed before. But if you have a good mental model of the dynamics of the physiology of the patient, then you might design a course of treatment that will actually bring the patient to a good state. When I say a patient, it can be a cell. How do you tell a stem cell to turn into a pancreas beta cell that produces insulin? You have a patient with type 1 diabetes. Their immune system basically eats up their own beta cells. It's autoimmune. How do you keep making beta cells? Can you send a message? Do you have a model of a human cell that will allow you to figure out what sequence of messages you need to send to a stem cell so that it turns into a beta cell?

不那么推崇 LLM 的阵营和推崇 LLM 的阵营各说各话，但我认为实际上很可能两者都有道理：LLM 能做的，也许是扩展顶级医生的能力，将其推广到全球，这有不可思议的潜在影响。而你说的，对很多这些事情来说当然还在路上，是比顶级医生更好。你怎么做到呢？

The less LLM-pilled camp and the LLM-pilled camp talk past each other, but I think it's actually very possible that both what LLMs can do, which is maybe scaling what a top doctor does, scaling that around the world, has unbelievable potential impact. And then what you're talking about, which is certainly still on the come for a lot of these things, is even better than the top doctor. How do you go do that?

不仅仅是顶级医生，对吧？因为 LLM 擅长的是大致上能复述你在书本上能读到的知识。但如果医学只是积累书本上的陈述性知识，你光读书就能当医生了。你必须做住院医师，实际听心脏、按肚子等等来诊断疾病。

More than just a top doctor, right? Because what the LLM can do well is it can sort of regurgitate knowledge that you can read in books, mostly. But if medicine was only about accumulating declarative knowledge that exists in books, you could be a doctor by just reading books. You have to do residency, actually listen to the heart, press on the belly, and things like that to diagnose a disease.

有意思。我很好奇 LLM 本身能否在全球范围内提供顶级质量的医疗保健。我们得回头再看。看起来它们已经很接近了。

It's interesting. I'll be very curious to see whether LLMs themselves can provide top-quality healthcare globally. We'll have to check back in on that one. It seems they're pretty close.

我当然也想谈谈你在 Meta 的时光，因为你花了十多年建立了世界上最受尊敬的研究实验室之一。你最近离开了。回顾那段时光，你认为在运营 FAIR 时，你做得最对和最错的是什么？

I definitely also want to hit on your time at Meta because you spent over a decade building one of the most respected research labs in the world. You recently left. As you reflect back on the time there, what do you think you got most right and most wrong in running FAIR?

我们做对的是建立了一个真正创新的顶级研究实验室，产生了许多基础方法、科学和工具，比如对整个行业都有用的 PyTorch。整个行业基本上都建立在 PyTorch 之上，除了谷歌的少数人。我认为开放和科学过程的文化是突破性创新所必需的。因为有一条完整的创新链。有蓝天研究、新概念。很多发生在大学里，一些发生在工业界的高级研究实验室，屈指可数。谷歌是一个，FAIR 是一个。希望它仍然是，我不确定。

The thing we got right is building a top research lab that really innovated, produced a lot of the basic methods and science and tools like PyTorch that are useful to the entire industry. The entire industry is built on PyTorch basically, except for a few people at Google. And I think a culture of openness and scientific process, which is necessary for breakthrough innovation. Because there is a whole chain of innovation. You have blue sky research, new concepts. A lot of that takes place in universities, some in advanced research labs in industry which can be counted on the fingers of one hand. Google is a good one, FAIR was a good one. Hopefully it will still be, I'm not sure.

然后你会说，好吧，这是个好主意，但让我们推进它，看看它能否变得有用，但仍在研究层面。从某种意义上说，我们不会自欺欺人。我们不会试图只找到一个能解决这个问题的方案。我们要看看我们想象或从社区其他人那里学到的这个技术能否真正被推进并变得实用——不是作为产品，而是像我们可以证明它在某个任务或基准上打破了记录。然后下一阶段是，托管研究实验室的公司说：「好了，现在我们要按下按钮，投入大量工程努力来实现那个愿景并推进它。」很多项目正是在这里失败的。很多公司也正是在这里没能接住。Meta 在这方面其实做得不错，但远非完美。它不像教科书式的错误案例，比如施乐帕克完全错过了图形用户界面、鼠标和窗口系统，对吧？Meta 基本上错过了一些步骤。这部分是组织问题。部分是因为你需要一个离研究很近、但不完全是产品组织的机构，来接过接力棒，把技术再往前推一点。不是以三个月为期限做产品，而是推进事情。我们曾经在 Facebook 和 Meta 有过这样的阶段。然后我们失去了它。FAIR 基本上在公司内部被孤立，有很多想法但没人采纳。然后在 2023 年，GenAI 组织成立了，最初是从 FAIR 抽调了大约六七十名科学家和工程师，然后逐步壮大。但后来它承受了巨大的短期压力，以至于 GenAI 根本没有时间与 FAIR 沟通。因此，GenAI 非但没有站在 LLM 创新的前沿，反而不得不专注于短期事务，变得非常保守。所以基本上，在研究与产品之间出现了一段脱节期。

Then you have, okay, this is a good idea, but let's push it forward and see if it can be made useful, but still at the research level. In a sense, we're not going to fool ourselves. We're not going to try to just find a solution that just works for this problem. We're going to see if this technique that we imagined or we picked up from other people in the community can actually be pushed and be made practical, not as a product, but like we can show that it beats some record on some task or benchmark. And then the next stage is for the company that hosts the research lab to say, 'Okay, now we're going to push the button, devote a big engineering effort to that vision and push it forward.' That is where a lot of projects fail. That's where a lot of companies kind of fail to pick up. Meta was actually pretty good at this, okay? But far from perfect. It was not like a textbook example of how you do it wrong, like Xerox PARC totally missing out on the GUI interface and mouse and windowing systems, right? Meta kind of missed a few steps, essentially. And it's partly just organizational. It's partly because you need an organization that is pretty close to research, but not completely a product organization, to take the relay of pushing the technology a little further. Not making product with a 3-month deadline, but like pushing things. And we had that at one point at Facebook and Meta. And then we lost it. And FAIR was basically isolated within the company, had lots of ideas that nobody picked up on. And then in 2023, the GenAI organization was created by basically taking about 60 or 70 scientists and engineers from FAIR, initially, and then it built up. But then it was under so much short-term pressure that basically that organization, GenAI, didn't have time to talk to FAIR. And so, instead of being at the forefront and innovating in LLM, GenAI basically had to focus on short-term things and become very conservative. And so, there was a gap, basically, in periods of mismatch between research and the product side.

这就是 Llama 4 的情况吗？

Is that kind of what happened with Llama 4?

是的。嗯，甚至 Llama 3 也是，从 Llama 3 开始。Llama 1 是 FAIR 内部的一个小项目。2023 年初，GenAI 成立了。Llama 团队基本被调到了 GenAI。他们开始做 Llama 2。然后其中一些人意识到他们可以自己创业。这就是 Mistral 的起源。Llama 1 的两位作者基本上和另一位来自 Google 的人一起创立了 Mistral。还有一些人离开去做了别的事情。由于各种原因，那段时间在 Meta 并不愉快。所以很多人离开了。然后 GenAI 组织在一定程度上接管了 Llama 2，而 Llama 3 和 4 承受了巨大的短期压力，变得非常保守。这是团队和领导层压力共同作用的结果。事情出错的方式有很多，不能怪罪任何人，但基本上就是这样。

Yeah. Well, even with Llama 3, starting with Llama 3. So, Llama 1 was a small project within FAIR. Early 2023, GenAI was created. The Llama people were basically moved to GenAI. They started working on Llama 2. And then a bunch of them realized they could do a startup. So that was the genesis of Mistral. Two of the authors of Llama 1 basically created Mistral with another guy from Google. And a few people left and did other things. This was not a happy time at Meta for various reasons. And so a bunch of people left. And then the GenAI organization took over Llama 2 to some extent, and Llama 3 and 4 were under so much short-term pressure that they became very conservative. And it's a combination of the groups and pressure from the leadership. There are many ways things can go wrong and you can't blame anyone in particular, but yeah, that's kind of what happened.

我的意思是，感觉现在很多组织都承受着短期压力，因为竞争异常激烈。所以我很好奇，显然你曾经有 FAIR 这样的架构，谷歌多年来也有类似的，当然 OpenAI 和 Anthropic 也有很多研究人员在尝试各种不同的事情。你认为这种模式未来还有可能吗？还是说唯一的出路就是离开并创办自己的公司？或者你认为行业内是否还有地方，在当前的竞争态势下仍保有 FAIR 最初的那种精神？

I mean, it feels like a lot of these organizations are under short-term pressure right now because there's just an incredible race going on. And so I'm curious, obviously this FAIR setup you had, and there's a similar one at Google for many years, and certainly many researchers running around OpenAI and Anthropic trying many different things. Do you think that is still possible going forward? Or is the only path to leave and do your own company? Or are there still places within the industry that you think have this original ethos of FAIR even amidst the race dynamics that are happening?

我认为在 Google Research 和 DeepMind 内部还有一些地方，人们确实在做研究。但整个行业越来越封闭了，对吧？我的意思是，谷歌肯定收紧了，Meta 和 FAIR 也在朝同样的方向走。现在对发表成果有了更多限制。所以，对于那些真正想做突破性研究的人来说，吸引力降低了。他们得不到那么多资源。如果他们做了中期相关的事情，就会被要求不要谈论。所以，我认为这不是一个有利于突破的好氛围。基本上，要获得我们在 FAIR 早期、贝尔实验室鼎盛时期和施乐帕克那样的突破性研究，最好的办法是雇佣最优秀的人——那些对做什么项目有敏锐嗅觉的人。给他们成功的手段，然后别挡道。请原谅我说话直白。

I think there are a few places within Google Research and DeepMind where people actually do research. But increasingly the industry has become more closed, right? I mean, Google has certainly clamped down, and Meta and FAIR are even going in the same direction. There are restrictions on publication now, more restrictions. And so, it's still less appealing for people who really want to do breakthrough research. They don't get as much resources. If they do something that is relevant in the medium term, they are told not to talk about it. And so, it's not a good atmosphere, I think, for breakthrough. It's not conducive. Basically, the best way to get breakthrough research of the type we were getting in the early days of FAIR, at Bell Labs in the good days, and Xerox PARC, is you hire the best people, and those are people who have a good nose to know what to work on, what projects to attack. You give them the means to succeed and you get out of the way. All right, pardon my French.

是的，我很好奇这最终会对更广泛的研究社区产生什么影响。显然，FAIR 的遗产之一就是培养了大量研究人员，对吧？他们遍布整个生态系统。而现在，那些相当于当年在 FAIR 职业生涯早期进入的人，他们加入的实验室可能更注重短期优先事项和焦点。我想知道，在当前这个生态系统中，很多进入该领域的年轻人被更多地推入这种短期动态中，这是否会改变生态系统演化的方式？

Yeah, I'm curious what impact it then ends up having on the broader research community. So obviously, one of the legacies of FAIR is you trained so many researchers, right? And they're all throughout the ecosystem. And it feels like now the equivalent to those people that came in younger in their careers at FAIR, they're joining these labs with maybe shorter-term priorities and focus. And I guess I'm wondering, in this current ecosystem where it feels like a lot of younger people getting into the field are thrust much more into these short-term dynamics, does that change anything about the way the ecosystem evolves?

嗯，我的意思是，那些倾向于和我一起工作的人，首先都是足够疯狂的人。他们认同一个理念：在学术界和博士期间，你应该研究下一代 AI 系统，而不是当前这一代。比如，如果你现在在学术界研究 LLM，那会非常无聊。至少对我来说很无聊。基本上就是研究 LLM 如何以及为何工作，解释它们为什么有效或有什么局限。这是描述性科学，真的没什么创造性。我觉得这没什么意思。它是有用的。但如果你真的想展示如何用 LLM 做新事情，你不会有所需的 GPU。所以算了吧。如果你在读博士，别研究 LLM。

Well, I mean, the people who tend to want to work with me are generally people who are sufficiently crazy to do it, first of all. And they subscribe to the whole idea that in academia and during your PhD, you should work on the next generation of AI systems. You shouldn't work on the current generation. Like if you work on LLMs in academia now, it's incredibly boring. At least to me it's boring. It's basically studying how and why LLMs work and explaining why they work or what their limitations are. It's descriptive science. It's really not very creative. I don't find that particularly interesting. It's useful. And if you really want to show how to do new things with LLMs, you're not going to have the GPUs you need for that. So forget that. Don't work on LLMs if you're doing a PhD.

你怎么知道是时候离开 Meta 了？有没有一个时刻让你下定决心？

How do you know it was time to leave Meta? Was there a moment that it crystallized?

这是多种因素共同作用的结果。首先，你必须明白，很多人对我之前在 Facebook 和 Meta 的职责有完全错误的理解。我于 2013 年底加入，2014 年初正式开始工作。前四年半，我是 FAIR 的主管。我建立了 FAIR 组织，设定了文化，招聘了关键人员，并负责管理。四年半后，我因多种原因卸任，随后成为首席 AI 科学家。我当时快 60 岁了——实际上是 58 岁——而且我不想再做管理工作了。我愿意做一段时间来启动组织，但我不擅长管理。我更像是科学或技术愿景家、工程师和科学家。其他人在管理方面比我强得多。所以我卸任了，Joelle Pineau 和 Antoine Bordes 接管了 FAIR 的主管职位。我成为首席 AI 科学家，向 CTO 汇报。我启动了一个我认为必要的研究项目，因为 FAIR 的雄心一直是构建智能系统。我在管理 FAIR 期间暂停了自己的研究，我认为设计人类级 AI 系统的架构很重要。我的概念基于从视频等感官信号进行自监督学习，以及世界模型。我在 2016 年 NeurIPS 的主题演讲中说过，AI 研究应该朝着世界模型的方向发展，预测行动的后果并进行规划。强化学习效率太低，监督学习有局限性，所以未来是自监督学习和世界模型。我启动了一些视频预测项目，并提出了通过表示空间预测从视频中进行自监督学习的想法——这就是 JEPA。有了 JEPA，你可以使其以行动为条件，并用于规划。我在 2020 年左右有了这个想法，2022 年我写了一篇长篇愿景论文，倾囊相授，希望能吸引人们。结果奏效了。不仅吸引了纽约大学和巴黎的学生，还吸引了 FAIR 的整个团队。Joelle Pineau 说这应该成为 FAIR 的主要使命，称为高级机器智能。那是内部名称。

It was a combination of things. First, you have to understand that many people have a completely wrong idea about my role at Facebook and Meta. I joined in late 2013, started early 2014. The first four and a half years, I was director of FAIR. I built the FAIR organization, set up the culture, hired key people, and managed it. After four and a half years, I stepped down for several reasons. I then became chief AI scientist. I was getting close to turning 60—actually 58—and I just didn't want to do management anymore. I was willing to do it for a while to get the organization started, but I'm not good at it. I'm more of a scientific or technical visionary, engineer, and scientist. Other people are much better at management. So I stepped down, and Joelle Pineau and Antoine Bordes took over the directorship of FAIR. I became chief AI scientist, reporting to the CTO. I started a research project I thought was necessary because FAIR's ambition was always to build intelligent systems. I had put my own research on hold while running FAIR, and I thought it was important to design the architecture of human-level AI systems. I had the concept based on self-supervised learning from sensory signals like video, and world models. I gave a keynote at NeurIPS in 2016 saying that AI research should go toward world models that predict consequences of actions and plan. RL is too inefficient, supervised learning has limits, so the future is self-supervised learning and world models. I started projects on video prediction, and came up with the idea of training self-supervised learning from video by making predictions in representation space—that's JEPA. With JEPA, you can make it action-conditioned and use it for planning. I had this idea around 2020, and in 2022 I wrote a long vision paper, spilling all my secrets, hoping to rally people. And it worked. Not only did I rally students at NYU and Paris, but also a whole team at FAIR. Joelle Pineau said it should be a major mission of FAIR, called Advanced Machine Intelligence. That was the internal name.

有意思。他们让你带着这个离开，现在它成了公司的名字。

Interesting. They let you leave with it, and now it's the name of the company.

马克·扎克伯格读了那篇论文，并支持这个项目。CTO 安德鲁·博斯沃思也支持。前 CTO 迈克·施罗普费尔，以及我的直接上司、首席产品官克里斯·考克斯也很喜欢这个想法。领导层对这个内部称为 AMI 的项目给予了很大支持。它在视频方面开始见效。但随后公司将所有精力重新聚焦到大型语言模型上。尽管马克和安德鲁支持，但我认为下面的层级看不到价值。政治上变得困难。JEPA 世界模型的应用在于可穿戴智能体和机器人，但 Meta 解散了整个由 Jitendra Malik 领导的机器人 AI 团队，他现在在亚马逊。显然，这不再是合适的环境。大多数应用都在 Meta 不感兴趣的行业。FAIR 越来越受到压力，要帮助 MSL 处理大型语言模型。所以情况变得明朗。当我为新公司筹集资金时，这个故事对投资者也很有效。许多投资者读过我的论文或听过我的演讲，相信我的故事。他们意识到大型语言模型有局限性，并对构建下一代 AI 系统感兴趣。

Mark Zuckerberg read that paper and subscribed to the project. Andrew Bosworth, the CTO, also did. Mike Schroepfer, the previous CTO, and Chris Cox, my direct manager and chief product officer, also loved the idea. There was a lot of support in the leadership for this project, internally called AMI. It started working for video. But then the company refocused all its effort on LLMs. Despite support from Mark and Andrew, the layers below didn't see the point, I think. Politically, it became difficult. The applications of JEPA world models are in wearable agents and robotics, but Meta got rid of its entire robotics AI group led by Jitendra Malik, who is now at Amazon. Clearly, it wasn't the right environment anymore. Most applications were in industries Meta had no interest in. FAIR was increasingly pressured to help MSL with LLMs. So it became clear. And that story worked well with investors when I raised money for the new company. Many investors had read my paper or heard my talks and bought my story. They realized LLMs had limitations and were interested in building next-generation AI systems.

Scale 的收购是否是内部纯粹聚焦大型语言模型的催化剂之一？

Was the Scale acquisition part of the catalyst for the pure LLM focus internally?

当然。可能还有其他原因。我没有内部消息，但有可能马克在 Alex 身上看到了潜在的继任者，一个更年轻的自己。

Definitely. There are probably other reasons too. I don't have inside information, but it's possible that Mark sees in Alex a potential successor, a younger version of himself.

媒体上很多流行说法是，当 Alex 进来后，管理研究组织变得更难了。你有这种感觉吗？

A lot of the popular narrative in the media has been that when Alex comes in, it gets harder to run a research organization. Did you feel that?

这里有一个很大的误解，关于我的角色、我与 Alex 的关系以及 Meta 的 AI 运作方式。我对 Llama 的技术贡献为零，完全没有。我唯一的贡献是主张开源 Llama 2，因为内部有激烈争论。法务部门反对，政策部门反对，通讯部门支持，所有工程方面支持，Boz 也支持。

Here is a big misconception about my role, my relation to Alex, and how AI was run at Meta. I had zero technical contribution to Llama, none whatsoever. My one contribution was to argue for open-sourcing Llama 2 because there was a big internal debate. The legal department was against it, the policy department was against it, the comms department was for it, all the engineering side was for it, Boz was for it.

内部进行了非常高层的大规模讨论，从马克·扎克伯格往下，每周 40 人开会两小时，持续了数月。这是一场内部大辩论。我推动并主张，Boz 也强烈认为，安全风险基本上被夸大了。创造行业的机会非常强大。我们通过开源 Llama 2 来启动 AI 行业，事实也确实如此。但我对 Llama 没有任何正面或负面的贡献。我没有做任何事来阻止或减缓它。FAIR 内部有很多人在做 LLM，这没问题。我从未说过反对的话。只是说这不是通往人类级智能的路径，但它有用。语音识别或翻译也是如此。特别是自 2018 年我卸任 FAIR 主任以来，除了发表我的愿景并召集人们参与我的项目外，我没有直接影响人们的工作。他们和我一起工作是因为他们愿意，而不是因为我是他们的老板。所以我对 Meta 内部的 LLM 没有正面或负面的影响。我对战略有一些影响，但更多是长期的，比如如何维持一个研究实验室。在过去一年里，从 2024 年初开始，尤其是 2025 年，FAIR 被移动和管理的方式不符合我认为必要的，以保持创新、研究、突破和留住优秀人才。很多优秀人才已经离开了。

There were enormous internal discussions at a very high level, 40 people from Mark Zuckerberg down every week for 2 hours for months. It was a big debate internally. I pushed and argued, and Boz was also very vocal, that the safety risks were basically overblown. The opportunities to create an industry were extremely strong. We were going to jump-start the AI industry by open-sourcing Llama 2, and that's exactly what happened. But I had zero contribution to Llama positive or negative. I didn't do anything to stop it or slow it down. There were a lot of people working on LLMs within FAIR, and it was fine. I never said anything against it. Other than saying this is not a path to a human-level intelligence, but it's useful. Same thing for speech recognition or translation. Particularly since 2018 when I stepped down from being director of FAIR, I didn't have any direct influence on what people were working on other than publishing my vision and rallying people around my project. They worked with me because they wanted to, not because I was their boss. So I had no positive or negative influence on LLM within Meta. I had some influence on the strategy, but it was more long-term, like how to maintain a research lab. In the last year, starting early 2024 and certainly in 2025, the way FAIR was moved and managed did not correspond to what I thought was necessary to preserve innovation, research, breakthrough, and good people. A lot of good people have left already.

我想可能很难让人们内部去研究你正在做的事情，而且我肯定你自己也面临压力去研究很多 LLM 相关的东西。

I guess a lot of it probably was harder to get people to work on the stuff you were working on internally, and I'm sure there's pressure for you yourself to work on a lot of the LLM stuff.

是的，但很多其他人也离开了。

Yeah, but a lot of other people also have left.

这很有趣。在整个对话中，我印象深刻的一点是，我觉得你在 FAIR 这样的领域里长期保持着非常一致的观点。你可以回顾你提到的许多早期演讲。显然这是一个快速发展的领域，过去一年发生了很多有趣的事情。在过去一年里，你改变主意的一件事是什么？

It's fascinating. One thing I'm struck by throughout our whole conversation is I feel like you've had a remarkably consistent point of view in the space like FAIR for a long time. You can go back to a bunch of the earlier talks you referenced. Obviously it is a fast-moving space and a ton of interesting things have happened in the last year. What's one thing you've changed your mind on in the last year?

我们过去称之为无监督学习、现在称为自监督学习的整个概念。直到大约 2003 年，无监督预训练的想法是，你为输入数据获得一个好的表示，然后用少量有监督标注数据微调模型。这给了我们一些证据表明这种技术可行。我尝试将其应用于视频，因为我最终想做的是训练一个系统，通过观察世界运转来理解世界如何运作。这是基本想法。我在 2010 年代早期开始主张这一点。我做了一些简单的视频预测工作。我们没有 GPU。然后在 FAIR 成立后更认真地做像素级视频预测，意识到那行不通。但随后主张自监督学习。这个想法是训练一个系统，不是解决特定任务，而是基本上只做预测，然后用这样学到的表示作为下游任务的输入，下游任务可以用监督或强化学习等训练。这大致是我 2016 年 NIPS（当时还叫 NIPS）主题演讲后半部分的话题。然后我继续推动这个想法，并尝试发现一些方法使其工作。让我惊讶的是，它变得非常成功，但不是针对视频，而是针对语言。LLM 基本上是自监督学习的一个极其成功的例子。

The whole idea of what we used to call unsupervised learning that we now call self-supervised learning. Until about 2003, the whole idea of unsupervised pre-training where you get a good representation for the input data and then you either fine-tune the model with a little bit of supervised labeled data. It gave us some evidence that this whole technique could work. I tried to apply this to video because ultimately what I wanted to do is train a system to understand how the world works by just watching the world go by. That's the basic idea. I started to argue for this in the early 2010s. I did some work on simple video prediction. We didn't have GPUs. Then doing this more seriously after the creation of FAIR by doing pixel-level video prediction, realizing that wasn't working. But then arguing for self-supervised learning. This whole idea of training a system generically not to solve a task but to basically just predict and then using the representation that is learned this way as input to a downstream task that you can train supervised or reinforcement or whatever. That was a bit of the topic of my second half of my keynote at NIPS in 2016. It was still called NIPS at the time. And then I kept pushing for this idea and tried to discover some methods to get that to work. What surprised me is that that became incredibly successful but not for video, for language. LLMs basically are a blindingly successful example of self-supervised learning.

不，它们确实是。我觉得这几乎是完美的结束语，但我想确保把最后的话留给你。我觉得我们所有的听众都非常熟悉你，但我想至少给你话筒，让他们看看你认为他们应该关注的一些你正在做的新东西，或者你想指出的任何你的工作。话筒是你的。

No, that they are. Well, I feel like that's almost the perfect note to end on but I want to make sure to leave the last word to you. I feel like all our listeners are very familiar with you but I want to at least give you the mic to point them to anything that you think they should check out with some of the new stuff you're doing or any of your work you want to point to. The mic is yours.

好的，让我告诉你一件事。LLM 之所以有效，是因为当你有一系列离散符号时，进行预测很容易。你的语言中只有有限数量的可能符号，比如 10 万个可能的 token。你可以让神经网络在所有可能的 token 上产生一个概率分布，然后从该分布中采样，将 token 移入输入，并生成下一个 token。你可以进行自回归预测。这是一个特例。如果你面对现实世界，你不能使用生成模型。所以你必须训练一个系统，学习一个表示并在表示空间中进行预测。这有一个大问题，直到大约 5 年前我才认为它容易解决，尽管我在那之前几十年就发明了一种技术来解决它。问题是，如果你取两个输入，比如视频的初始片段和后续片段，或者取一张图像和它的损坏版本，将它们都通过编码器，然后训练一个预测器，从一个的表示预测另一个的表示。有一个非常简单的解决方案：系统基本上预测一个恒定的表示，预测问题就变得微不足道。这称为崩溃，表示崩溃。所以自监督学习对于 JEPA（联合嵌入架构）的大问题是如何防止崩溃。我多年前（1993 年）提出的解决方案是对比学习。基本上，你有应该相互可预测的示例，以及不应该相互可预测的示例。事实证明这种方法有效，但它不随维度扩展。它扩展得不太好。

Okay, let me tell you one thing. An LLM works because when you have a sequence of discrete symbols, making predictions is easy. There's only a finite number of possible symbols in your language. 100,000 possible tokens or something like that. You can have your neural net produce a probability distribution over all possible tokens, then sample from that distribution, shift the token into the input, and produce the next token. You can do auto-regressive prediction. That's a special case. If you have the real world, you can't use a generative model. So you have to train a system that learns a representation and makes prediction in the representation space. There's a big issue with this, which I didn't think until about 5 years ago that was easily solvable, even though I invented one technique to solve it decades before that. It's a problem that if you take two inputs, let's say the initial segment of a video and the continuation of that video, or you take one image and a corrupted version of it, you run them both through an encoder, and you train a predictor to predict the representation of one from the representation of the other. There's a very simple solution where the system basically predicts a constant representation, and the prediction problem becomes trivial. That's called a collapse. Representation collapse. So the big question of self-supervised learning for JEPA, for the joint embedding architecture, is how do you prevent collapse? The solution that I came up with many years ago, 1993, is contrastive learning. So basically you have examples of things that should be predictable from one another, and then examples of things that should not be predictable from one another. It turns out this method works, but it doesn't scale with dimension. It doesn't scale very well.

还有另一种技术，实际上是杰夫·辛顿和苏比亚·埃克在 80 年代末发明的，你让两个网络相互最大化互信息。于尔根·施密德胡伯对我很生气，因为他在 1992 年也提出了一个版本，他说那是 JEPA。其实不是 JEPA，只是防止联合嵌入架构崩溃的另一种方法。这没问题，但我觉得这种特定做法并不特别好。所以，现在有了 JEPA 架构，你需要找到一种好的方法来防止崩溃。有几种方法。如前所述，对比方法我认为不是好方法。还有另一类方法，叫做蒸馏方法。它们确实能防止崩溃，但我们不知道为什么。一个很好的例子是 DINO。这是一种使用蒸馏方法的联合嵌入方法。基本上，其中一个编码器训练另一个，它被用作另一个编码器的教师。被训练的编码器进行反向传播，未被训练的编码器不进行反向传播，但通过指数移动平均与另一个共享权重。这是一个集合配方。DeepMind 有一篇关于它的论文叫 BYOL，它使用了这个技巧。这个技巧源于强化学习的一些直觉。它不知何故防止了崩溃，但我们不知道为什么。有几篇理论论文解释了它在某些简单情况下可能有效的原因，但并不令人满意。你认为你在最小化的代价函数实际上并没有被最小化，所以你无法监控。它在训练时实际上会上升。这说得通。所以我们不喜欢这种方法，但它有效。我们训练的一些模型，大规模视频表示学习系统 VJPA、VJPA2、VJPA2.1，都使用了这种方法。I-JEPA 也是。但我们正在远离这种方法，现在我们有几篇最近发表的论文，关于一种特定的正则化器来防止这种崩溃，它基本上试图最大化编码器输出的信息量。所以它与贝克尔和辛顿 1989 年的工作以及施密德胡伯 1992 年的工作属于同一家族，之后还有很多。在某种程度上也包括对比技术，尽管不是简单的对比。那么问题是如何衡量信息量？如何最大化神经网络输出的信息量？问题在于，如果你想最大化这个量，要么需要能够测量它，要么需要有一个下界。信息量我们只有上界，无法测量，只能得到上界。所以我们取一个上界，然后祈祷。它有点效果。所以最新的一个叫 SigReg，意思是草图各向同性高斯正则化。我们之前有一个叫 VCReg 或 VICReg，方差不变性协方差正则化。SigReg 的东西真的很酷。这是兰德尔·巴莱斯特里罗的工作，他曾是我的博士后，现在是布朗大学的助理教授。它基本上迫使编码器输出的变量分布成为联合高斯分布。所以，如果你想最大化信息，这是一种非常不同的方式，不同于于尔根·施密德胡伯、苏比亚和杰夫·辛顿所做的。在我看来这非常有前景，我们还有它的变体，可以产生稀疏表示。另一个可以产生各向异性表示但不一定是高斯的。我们和兰德尔以及米拉的一个学生卢卡·米斯有一篇论文，我们用这个方法训练了一个世界模型。规模还很小，但我们认为非常有前景。所以如果你想读一篇论文，就读那篇。它叫 Le World Model。

There's another technique that was actually invented by Jeff Hinton and Subbiah Ecker in the late '80s, where you have those two networks and you try to maximize the mutual information between them. Jürgen Schmidhuber is mad at me because he also came up with a version of this in 1992 and he says that's JEPA. It's not JEPA. It's just another way of preventing collapse of a joint embedding architecture. Which is fine, but it's a particular way of doing it which I don't think is particularly good. So, now you have the JEPA architecture. You have to come up with a good way of preventing collapse. And there are a couple ways. As already said, contrastive methods I think is not a good approach. There's another set of methods that are kind of called distillation methods. And they do prevent collapse. We don't know why. A good example of that is DINO. That's a joint embedding method using the distillation method. Basically, one of the encoders trains the other one, it is used as a teacher for the other encoder. The encoder that is being trained, you do backprop to it. The one that is not being trained, you don't do backprop, but you share the weight with the other one with some exponential moving average. It's a collection recipe. There was a paper from DeepMind about it called BYOL, Bootstrap Your Own Latent, which uses this trick. That trick is derived from some intuition from reinforcement learning. And somehow it prevents collapse, but we don't know why. There are a few theoretical papers on it that explain why it possibly might work in some simple cases, but it's not satisfactory. The cost function you think you're minimizing, you're not actually minimizing and so you can't monitor. It actually goes up when you train. It makes sense. So, we don't like this method, but it works. And some of the models we've trained, large scale video representation learning system, VJPA, VJPA2, VJPA2.1, they train using this method. I-JEPA also. But we're moving away from this and now we have a few papers that came out recently on a specific regularizer to prevent this collapse, which basically tries to maximize the information content coming out of the encoder. So, it's in the same family as the Becker and Hinton from '89 and the Schmidhuber 1992 and a bunch of others since then. And to some extent also contrastive techniques, although it's not simple contrastive. And then the question is how do you measure information content? How do you maximize the information content coming out of a neural net? And the problem is if you want to maximize the quantity, you either need to be able to measure it or you need to have a lower bound on it. Information content, we only have upper bounds. We cannot measure it. We can only come up with upper bounds. And so, we take an upper bound and we cross our fingers. And it kind of works. So, the latest one is called SigReg. That means sketch as isotropic Gaussian regularization. We had a previous one called VCReg or VICReg, variance invariance covariance regularization. And the SigReg stuff is really cool. This is some work by Randall Balestriero who was a postdoc with me. He's an assistant professor at Brown now. And it basically consists in forcing the distribution of variables coming out of the encoder to be joint Gaussian essentially. So, maximizing information if you want. It's just a very different way of doing it than what Jürgen Schmidhuber and Subbarao and Jeff Hinton were doing. This is super promising in my opinion and we have variations of it, when that we can produce sparse representations. Another one that can produce anisotropic representations but not necessarily Gaussians. And we have a paper with Randall and a student at Mila, Luca Mice, where we train a world model with this. It's still small scale. But we think it's super promising. So if you want to read one paper, read that paper. It's Le World Model.

太棒了，我一定也会链接到它。

Awesome. I'll definitely link to it, too.

名字不是我起的，是兰德尔取的。

I'm not responsible for the name. Randall picked up the name.

太棒了。扬，真的非常感谢你。能和你共度最后这段时间真是莫大的荣幸，非常感谢你来做客播客。

Amazing. Well, Yann, seriously, thank you so much. It is such a privilege to get to spend the last bit of time with you and really appreciate you coming on the podcast.

谢谢邀请，很有趣。

Thanks for having me, though. It's fun.

我是雅各布·埃夫隆，这里是《无监督学习》播客。在这个播客里，我和人工智能领域最聪明的人交谈，问他们关于模型发展以及这对世界上的企业意味着什么的大量问题。希望很明显，我对此乐在其中。这是我除了在红点资本做投资人之外的夜间和周末项目，但我们能请到这些了不起的嘉宾，全靠像你这样的听众订阅播客、与朋友分享。这最终是让这一切运转的关键。所以请考虑这样做，非常感谢你的支持和收听。我们下期再见。

I'm Jacob Effron and this has been Unsupervised Learning. A podcast where I get to talk to the smartest people on AI and ask them tons of questions about what's happening with models and what it means for businesses in the world. As I hope is clear, I have a ton of fun doing this. It's a nights and weekends project in addition to my day job as an investor at Redpoint, but our ability to get these incredible guests on really comes from folks like you subscribing to the podcast, sharing it with friends. It's really what ultimately makes this whole thing work. And so please consider doing that and thank you so much for your support and listening. We'll see you next episode.