「LLM 是一条死路」

Yann LeCun 是人工智能之父之一，今天他宣布完成了欧洲历史上最大的种子轮融资。他的新创业公司 AMI 已筹集超过 10 亿欧元资金，为人工智能的发展开辟了一条新道路。

Yann LeCun is one of the fathers of artificial intelligence, and today he announced the closing of the largest seed funding round in Europe's history. His new startup, AMI, has raised over 1 billion euros in funding, opening a new path for the development of artificial intelligence.

目前，AI 系统在很多方面都非常愚蠢。我们被误导认为它们很聪明，因为它们能很好地操控语言。但它们不理解物理世界，没有我们那种持久记忆，不能真正推理，也不能规划。而这些是智能行为的基本特征。

Currently, AI systems are in many ways very stupid. We're fooled into thinking they are smart because they can manipulate language very well. But they don't understand the physical world. They don't really have any persistent memory of the type that we have. They can't really reason, and they can't plan. And those are essential characteristics of intelligent behavior.

我和同事们在 FAIR 和 NYU 一直在做的事情之一，是设计一种新型 AI 系统，仍然基于深度学习，但能够理解物理世界，拥有持久记忆，能够推理和规划。在我看来，一旦我们成功按照这个蓝图构建出这些系统，它们将拥有情感。它们会有恐惧、兴奋或喜悦等情感，因为这些是预期。这些系统的基本工作方式是：我们为它们设定一个目标去完成。然后它们会尝试找出可以采取哪些行动来实现目标。如果它们能提前预测到目标会被实现，那就会让它们感到高兴；如果预测到无法实现，就会不高兴。所以从某种程度上说，它们会有情感，因为它们能预测一系列行动的结果。但我们不会硬编码愤怒或嫉妒之类的情感，因为没有理由。但意识是另一回事。我们不知道它到底是什么。没有定义，也没有可测量的东西能告诉我们某物是否有意识。即使观察动物，我们可能都同意猿猴和猴子有意识，也许大象也有。但狗有意识吗？老鼠有意识吗？界限在哪里？因为我们没有好的定义，所以真的无法判断。

One of the things that my colleagues and I have been working on at FAIR and at NYU is designing a new type of AI system, still based on deep learning, that would be capable of understanding the physical world, have persistent memory, be able to reason and plan. In my opinion, once we succeed in building those systems around this blueprint, those systems will have emotions. They'll have emotions like fear or excitement or elation, because those are anticipations. Those systems will basically work by having a goal that we set them to fulfill. We'll give them goals to accomplish. Then they will try to figure out what kind of actions can I take so that I fulfill that goal. If they can predict in advance that this goal will be fulfilled, that will make them happy. Or if they predict that they can't, it will not make them happy. So to some extent, they will have emotions because they'll be able to anticipate the outcome of a sequence of actions. But we will not hardwire into them anything like anger or jealousy because there's no reason. But consciousness is something else. We don't know what it is really. There's no definition of it. There's no kind of measurable thing that can tell us whether something is conscious or not. Even if we observe animals, we would probably all agree that apes and monkeys are conscious and maybe elephants. But is a dog conscious? Is a rat conscious? Where is the barrier? Because we don't have a good definition, we really can't tell.

是什么让你与 Geoffrey Hinton 的争论以及你的深度学习工作成为如此大的变革？

What made your war with Geoffrey Hinton, your deep learning work such a game-changer?

你可能指的是 Geoffrey Hinton、Yoshua Bengio 和我在 2015 年发表在《自然》杂志上的一篇论文。这不是新工作，基本上是一篇宣言或综述，告诉广大科学界和研究者：有一套新技术效果很好，这里列出了它擅长的领域，以及未来的发展方向。所以它标志着深度学习的公开开端和普及。但那篇论文里没有新结果。新结果和大多数其他引用都追溯到我在 80 年代和 90 年代的工作。

You're probably referring to a paper that Geoffrey Hinton, Yoshua Bengio, and I published in Nature in 2015. This was not new work. It was basically a bit of a manifesto, or a review paper, to tell the wide community of science and researchers: there is this new set of techniques that work really well. Here is a list of things where it works well. Here is where the future is going. So it sort of marked the public beginning, the popularization of deep learning. But there was no new result in that paper. The new results and most of the other citations go back to the work that I did in the 1980s and 90s.

你还记得那种流行开始的那一刻吗？

Do you remember the moment when that popularity was beginning?

实际上有两波浪潮。第一次是在 80 年代末，我们开始使用多层神经网络（现在我们称之为深度学习）在图像识别等任务上取得非常好的结果。当时我们还不能识别复杂图像，更多是像手写字符这样的简单图像。但这效果很好。当时我很兴奋，因为我认为这可能会彻底改变模式识别的方式，最终改变计算机视觉，甚至更广泛地改变 AI。所以在 80 年代末到 90 年代中期有一波兴奋浪潮。然后兴趣在 90 年代中期消失了。这是因为我们开发的技术需要大量数据进行训练。而我们只能为少数应用获得好数据，比如手写识别、字符识别和语音识别。仅此而已。而且需要当时非常昂贵的计算机，投资很大。所以兴趣在 90 年代中期消失了。然后兴趣在 2000 年代末再次缓慢上升，并在 2013 年左右完全爆发。2013 年确实是关键的一年，研究界意识到深度学习确实效果很好，并且可以应用于许多不同领域。从那以后它发展得非常快，2015 年是另一个里程碑。

There were two waves really. It happened twice. The first one was in the late 80s where we started to have really good results using multi-layer neural networks, which we now call deep learning, for tasks like image recognition. At the time we could not recognize complex images. It was more like simple images like handwritten characters. But this was working really well. I was really excited at the time when we started getting those results because I thought this may completely change the way we do pattern recognition and eventually computer vision and perhaps AI more generally. So there was a wave of excitement between the late 80s and mid-90s. Then the interest kind of disappeared in the mid-90s. It's because the techniques we had developed required a lot of data for training. And we could only get good data for a few applications, like handwriting recognition, character recognition, and speech recognition. That was about it. And it required computers that were at the time really expensive and it was a big investment. So interest disappeared in the mid-90s. Then interest went up again slowly in the late 2000s, and it totally exploded around 2013. 2013 is really the key year where the research world realized that deep learning really worked well and could be applicable to a lot of different things. It's been growing really quickly since then, and in 2015 was another milestone.

你说机器学习很糟糕。有什么变化吗？

You said machine learning sucks. What's changed?

这正是我们正在努力的方向。我们正在研究构建机器学习系统的新方法，使它们能够像人类和动物一样高效地学习。因为目前还不是这样。我可以告诉你一些过去几十年机器学习发展的历史。实际上有三种机器学习范式。一种是监督学习，这是最经典的。训练监督学习系统的方式是，比如一个旨在识别图像的系统。

That's what we're working on. We're working towards new ways of building machine learning systems so that they can learn as efficiently as humans and animals. Because currently it's not the case. I can tell you a little bit of the history of how machine learning has progressed over the last couple of decades. There are really three paradigms of machine learning. One is called supervised learning, which is the most classical one. The way you train a supervised learning system is that, let's say an image system that is meant to recognize images.

你给它看一张图片，比如说一张桌子，然后你告诉它这是一张桌子。所以这是监督学习，因为你告诉它正确答案是什么。系统计算它的输出，如果它说的不是桌子，那么它就会调整它的参数，也就是它的内部结构，使得它产生的输出更接近你想要的输出。如果你用大量桌子、椅子、汽车、猫和狗的例子不断这样做，最终系统会找到一种方法来识别你训练过的每一张图片，以及它从未见过的、与你训练过的图片相似的图片。这被称为泛化能力。还有另一种范式，人们认为它更接近动物和人类的学习方式，叫做强化学习。在强化学习中，你不告诉系统正确答案是什么。你只告诉它它产生的答案是好的还是坏的。在某种程度上，这可以解释某些类型的人类和动物学习。你试着骑自行车，你不知道怎么骑，过了一会儿你摔倒了。所以你知道你做错了什么，于是你稍微改变一下策略，最终你学会了如何骑自行车。事实证明，强化学习效率极低。如果你想训练一个系统下棋、下围棋或打扑克之类的，它效果很好，因为你可以让系统与自己进行数百万场比赛，并基本上进行微调。但它在现实世界中并不真正有效。如果你想训练一辆汽车自动驾驶，你不会用强化学习来做。它会撞车数千次。如果你想训练一个机器人学习抓取东西，强化学习可以是解决方案的一部分，但不是完整的答案。它是不够的。所以，还有第三种学习形式，叫做自监督学习，正是这种学习使得自然语言理解和聊天机器人最近取得了进展。在自监督学习中，你不训练系统完成任何特定任务，你只是训练它基本上捕捉其输入的结构。例如，对于文本，对于语言，它的使用方式是：你取一段文本，通过例如删除一些单词来以某种方式破坏它，然后训练一个大型神经网络来预测缺失的单词。一个特例是，你取一段文本，该文本的最后一个单词不可见。然后你训练系统预测该文本的最后一个单词。这就是大型语言模型的训练方式。每个聊天机器人都是这样训练的。技术上略有不同，但基本原理就是这样。这被称为自监督学习。你不为任务训练系统，你只是训练它学习输入的内部依赖关系。这一成功令人震惊。它效果惊人地好。你最终得到的系统似乎真正理解语言，并且如果你使用监督学习或强化学习对它们进行微调以正确回答问题，它们就能理解问题。所以，这就是业界每个人一直在做的事情。

You show it a picture, let's say of a table, and you tell it this is a table. So, it's supervised because you tell it what the correct answer is. The system computes its output, and if it says something else than table, then it's going to adjust its parameters, its internal structure, so that the output it produces gets closer to the output you want. And if you keep doing this with lots of examples of tables and chairs and cars and cats and dogs, eventually the system will find a way to recognize every image you trained it on, but also images it's never seen that are similar to the ones you trained it on. This is called the generalization ability. There's another paradigm, which people thought was closer to the way animals and humans learn, called reinforcement learning. In reinforcement learning, you don't tell the system what the correct answer is. You only tell it whether the answer it produced was good or bad. And to some extent, that can explain some type of human and animal learning. You try to ride a bike, and you don't know how to ride the bike, and after a while you fall. So, you know you did something bad, and so you change your strategy a little bit, and eventually, you learn how to ride a bike. It turns out, reinforcement learning is extremely inefficient. It works really well if you want to train a system to play chess or play Go or poker or something like that, because you can have the system play millions and millions of games against itself, and basically fine-tune itself. But it doesn't really work in the real world. If you want to train a car to drive itself, you're not going to do it with reinforcement learning. It's going to crash thousands of times. If you want to train a robot to learn how to grab things, reinforcement learning can be part of the solution, but it's not the complete answer. It's not sufficient. So, there is a third form of learning called self-supervised learning, and this is what has enabled the recent progress in natural language understanding and chatbots. In self-supervised learning, you don't train the system to accomplish any particular task, you just train it to basically capture the structure of its input. The way this is used for text, for example, for language, is that you take a piece of text, you corrupt it in some way by, for example, removing some words, and then you train a big neural net to predict the words that are missing. A special case of this is that you take a piece of text, and the last word in that text is not visible. And so, you train the system to predict the last word in that text. And this is the way large language models are trained. Every chatbot is trained this way. Technically, it's a little different, but that's the basic principle. That's called self-supervised learning. You don't train the system for a task, you just train it to learn the internal dependency of the input. The success of this has been astonishing. It works amazingly well. You get systems in the end that seem to really understand language and be able to understand questions if you fine-tune them to answer questions properly using supervised learning or reinforcement learning. So, this is what everybody has been working on in the industry.

但如果你想让系统理解物理世界，那种模型就不起作用了。缺少了一些东西。是的。只是物理世界比语言难理解得多。我们认为语言是智能的缩影，因为只有人类才能操纵语言，但事实证明语言很简单。它简单是因为它是离散的。它是一系列离散的符号。字典中可能的单词数量是有限的。所以，你永远无法训练一个系统精确预测下一个单词是什么，但你可以训练它为字典中的每个单词生成一个分数，或者每个单词出现在那个位置的概率。这样你就可以处理预测中的不确定性。但你无法训练一个系统预测视频中将要发生的事情。在我尝试了 20 年之后，很多人也尝试过。很多人都有这个想法：如果你能训练一个系统预测视频中将要发生的事情，那么该系统将隐式地理解世界的底层结构、直观物理学，以及任何动物和我们作为婴儿所学习的一切。

But that model does not work if you want the system to understand the physical world. Something is missing. Yes. It's just that the physical world is much more difficult to understand than language. We think of language as the epitome of intelligence because only humans can manipulate language, but it turns out language is simple. And it's simple because it's discrete. It's a sequence of discrete symbols. There's only a finite number of possible words in a dictionary. And so, you can never train a system to exactly predict what word is going to come next, but you can train it to produce something like a score for every word in a dictionary or a probability for every word in a dictionary to appear at that location. And so you can handle the uncertainty in the prediction that way. But you cannot train a system to predict what's going to happen in the video. People have tried to do this after I tried to do this for 20 years. And a lot of people have had this idea that if you could train a system to predict what's going to happen in the video, then that system will implicitly understand the underlying structure of the world, intuitive physics, everything that any animal and any of us as babies learn.

所有物理定律。

All physical rights.

是的，物理直觉。你知道如果我拿起一个物体然后松手，它会掉下去。你已经学会了重力基本上吸引每个物体朝向地面。人类婴儿大约在 9 个月大时学会这个。大约需要 9 个月来学习。

Yeah, physical intuition. You know that if I take an object and I let it go, it's going to fall. You've learned that gravity basically attracts every object toward the ground. Human babies learn this by the age of 9 months, roughly. It takes about 9 months to learn.

也许不是当今人工智能发展的限制，而是我们对现实的认识。我们无法替代我们不知道的东西。

Maybe not the limitation of AI development today, it's our knowledge about the reality. We cannot replace more than we know.

是的，但这是一个简单的问题，因为你的猫或狗可以在几个月内学会重力，对吧？猫在这方面非常擅长。它们可以规划复杂的动作，爬上各种东西，然后跳跃。所以，显然它们对我们所说的直观物理学有很好的直觉理解。我们还不知道如何用计算机重现这一点。原因是这是人工智能研究人员所说的莫拉维克悖论的另一个例子。汉斯·莫拉维克是一位机器人学家，他指出，为什么我们可以让计算机下棋、解决数学难题等等，但却不能让它们做物理性的事情，比如操纵物体，而动物可以做到，或者跳跃之类的。所以，这是这个悖论的另一个例子：离散对象和符号的空间很容易被计算机操纵，但现实世界仍然太复杂。在一个案例中有效的技术，在另一个案例中却不起作用。一个很好的可视化方式是，通过我们的感官（比如视觉或触觉）到达我们的信息量，与我们通过语言获得的信息量相比，是绝对巨大的。这或许可以解释为什么我们有大型语言模型、聊天机器人，它们可以通过律师资格考试，或者解决数学问题，或者写出听起来不错的文本，但我们仍然没有家用机器人。我们仍然没有能够完成猫或狗所能完成任务的机器人。

Yeah, but it's a simple problem because your cat or your dog can learn about gravity in just a few months, right? And cats are really really good at this. They can plan complex actions and climb on all kinds of stuff and jump. So, obviously, they have a very good intuitive understanding of what we call intuitive physics. And we don't know how to reproduce this with computers yet. And the reason is it's another example of what AI researchers have called the Moravec paradox. Hans Moravec was a roboticist and he made the point that how come we can have computers play chess and solve mathematical puzzles and things like this, but we can't get them to do physical things like manipulate objects that animals can do or jump or things like that. So, it's another example of this paradox that the space of discrete objects and symbols is easily manipulated by computers, but the real world is just too complicated yet. And the techniques that work in one case don't work in the other case. A good way to visualize this is that the amount of information that gets to us through our senses, let's say vision or touch, is absolutely enormous compared to the amount of information we can get through language. And this may explain why we have LLMs, chatbots, they can pass the bar exam, or they can solve mathematical problems, or write text that sounds good, but we still don't have domestic robots. We still don't have robots that can accomplish tasks that a cat or a dog can accomplish.

我们仍然没有完全自动驾驶的 L5 级汽车，更不用说像任何 17 岁少年那样只需练习 20 小时就能学会开车的自动驾驶汽车了。所以我们显然缺少一些重要的东西。缺少的是如何训练一个系统理解像视觉这样复杂的感官输入。如果我们想让机器像人类和动物一样熟练地学习，这是必要的。如果我们想要机器拥有类似动物和人类的智能，具备常识，也许在某个时刻拥有意识，但能够真正学习复杂世界的复杂结构，我们就需要解决这个问题。

We still don't have completely autonomous level five self-driving cars, and we certainly don't have self-driving cars that can train themselves to drive in about 20 hours of practice like any 17-year-old. So clearly we're missing something big. What we're missing is how to train a system to understand complex sensory input like vision. This is necessary if we want machines to learn as proficiently as humans and animals. If we want machines that have intelligence similar to that of animals and humans, that have common sense, perhaps at some point have consciousness, but are capable of really learning the complex structure of complex worlds, we need to crack that problem.

让我做一个非常简单的计算。如今一个典型的大语言模型是在大约 20 万亿个 token 上训练的。一个 token 大致相当于一个词。一个 token 通常用三个字节表示。所以 20 或 30 万亿个 token，每个三个字节，大约是 10^14 字节。这是互联网上所有公开文本的总和。我们任何人要读完这些材料都需要几十万年。这是一个巨大的信息量。但将其与我们生命前四年通过视觉系统进入大脑的信息量相比，两者大致相同。在四年里，一个幼儿总共清醒约 16,000 小时。通过视神经进入大脑的信息量约为每秒 2 兆字节。计算一下，大约是 10^14 字节。在四年里，一个幼儿看到的信息量与最大的 LLM 相当。这说明我们永远无法仅通过文本训练达到人类水平的 AI。我们必须让系统理解真实世界。而理解真实世界非常困难。

Let me give you a very simple calculation. A typical large language model today is trained with something on the order of 20 trillion tokens. A token is like a word, more or less. A token is typically represented on three bytes. So 20 or 30 trillion tokens, each on three bytes, that's about 10^14 bytes. This is the totality of all the text available publicly on the internet. It would take any of us several hundred thousand years to read through that material. It's an enormous amount of information. But compare this with the amount of information that gets to our brain through the visual system in the first 4 years of life, and it's about the same amount. In 4 years, a young child has been awake a total of about 16,000 hours. The amount of information getting to the brain through the optic nerve is about 2 megabytes per second. Do the calculation, and that's about 10^14 bytes. In 4 years, a young child has seen as much information as the biggest LLMs. What that tells you is that we're never going to get to human-level AI by just training on text. We're going to have to get systems to understand the real world. And understanding the real world is really hard.

在你的 LinkedIn 和 Facebook 上，你把 AI 和熵联系起来。有什么关联？

On your LinkedIn and Facebook, you are linking AI and entropy. What's the link?

这有点像是我的执念。有一个大问题根植于计算机科学、物理学、信息论以及许多不同领域的很多问题中：如何量化信息。一条消息中包含多少信息？我多次指出，一条消息中的信息量不是一个绝对量，因为它取决于解读这条消息的人。你能从传感器、从别人告诉你的消息中提取多少信息，取决于你如何解读它。所以，认为可以绝对地衡量信息的想法很可能是错误的。每一种信息衡量都是相对于某种特定的信息解读方式。这有非常深远的影响，因为如果没有绝对的信息衡量方式，那就意味着物理学中的许多概念并没有客观的定义，比如熵。熵是对物理系统状态无知程度的度量，而这取决于你对系统了解多少。所以我一直着迷于寻找定义熵、复杂性或信息内容的好方法，这些定义是相对的。

It's been a bit of an obsession of mine. There's a big question at the root of a lot of problems in computer science, physics, information theory, and many different fields: how you quantify information. How much information resides in a message? The point I've made multiple times is that the amount of information in a message is not an absolute quantity because it depends on the person interpreting the message. The amount of information you can extract from sensors, from a message someone tells you, depends on how you can interpret it. So the idea that you can measure information in absolute terms is probably false. Every measure of information is relative to a particular way of interpreting that information. This has very far-ranging consequences because if there is no absolute way of measuring information, that means a lot of notions in physics don't really have objective definitions, like entropy. Entropy is a measure of our ignorance of the state of a physical system, and that depends on how much you know about the system. So I've been obsessed with trying to find good ways of defining entropy, complexity, or information content that is relative.

你不认为我们用来训练 AI 模型的全球数据库已经用尽了吗？

Don't you think that our global database to train AI models is over?

不，我们远未用尽。有大量的文本知识尚未数字化。也许在很多发达国家，很多已经数字化了，但大部分并不公开。例如，有很多医疗数据是不公开的。而且世界上许多地区有大量的文化和历史数据无法以数字形式获取。或者即使以数字形式存在，也是扫描文档的形式，而不是文本。所以数据用尽的说法并不成立。

No, we're not even close. There is a huge amount of textual knowledge that has not been digitized. Maybe in a lot of the developed world, a lot of it has been digitized, but most of it is not public. There is a lot of medical data, for example, that is not public. And there is a lot of cultural and historical data in many regions of the world that is not accessible in digital form. Or if it is in digital form, it's in the form of scanned documents, so it's not text. So it's not true that we've run out of data.

这还涉及现实本质的问题，因为例如，我们不知道物质如何在大脑中转化为意识。所以我们对此没有数据，但也许未来会有。

And that questions about the nature of reality because, for example, we have no idea how matter is transformed into consciousness in a human brain. So we have no data about it, but maybe in the future we will.

嗯，我认为我们不应该对意识问题着迷。我认为它有点像是副现象，我们找不到意识的良好定义，很可能是因为我们没有问对问题。

Well, I don't think we should be obsessed by the question of consciousness. I think it's a bit of an epiphenomenon, and probably the reason we can't find a good definition of consciousness is because we're not asking the right question.

但我觉得世界对此很着迷。世界上有些地方对此很着迷。

But the world is obsessed, I think. Some parts of the world are obsessed by it.

让我举个例子。在 18 或 17 世纪，人们发现视网膜上的像是倒立的。他们完全困惑：为什么我们看到的世界是正的，尽管视网膜上的像是倒立的？那是一个谜题。现在我们意识到这个问题毫无意义。大脑解读图像的方式与图像在视网膜上形成的方向无关。我认为意识有点像这样。我们无法定义它。我们认为它存在，但无法确切指出。

Let me give you an example. In the 18th or 17th century, people discovered that the image on the retina forms upside down. They were completely puzzled: how is it that we see the world right side up even though the image is upside down? That was a puzzle. Now we realize that question makes no sense. The way your brain interprets images is irrelevant to the direction the image forms on your retina. I think consciousness is a bit like this. We can't define it. We think it exists, but we can't put our finger on it.

那是什么让我们成为个体？

And what makes us individuals?

那是另一回事。显然，有很多东西让我们彼此不同。我们有不同的经历，学习不同的东西，在不同的环境中工作。而且我们的大脑布线也略有不同。我们每个人都略有不同。这是进化确保每个个体人类都不同的必要性，因为我们是社会性动物。当同一个部落中不同的人略有不同时，有一个很大的优势，因为他们可以结合各自的专长。如果我们每个人都完全相同，那么数量就不会带来力量。但因为我们是不同的，我们因多样性而更强大。这是进化的结果。

That's different. Obviously, there are many things that make us all different from each other. We have different experiences, we learn different things, we work in different environments. But also our brains are wired slightly differently. All of us are slightly different. That's a necessity for evolution to make sure every individual human is different because we are a social animal. There is a big advantage when different people in the same tribe are slightly different because they can combine their expertise. If every one of us was identical, there would not be strength in numbers. But because we're different, we're stronger because we're diverse. That's a result of evolution.

那么自由推理、抽象思维模型呢？我们能从你的实验室期待这样的成果吗？

What about free reasoning, abstractional thinking models? Can we expect something like this from your laboratory?

从观察中提炼抽象表征的问题是深度学习的关键。深度学习就是关于学习表征的。事实上，深度学习的主要会议之一就是国际学习表征会议（ICLR），这是我和约书亚·本吉奥共同创立的。所以，这说明了学习抽象表征这个问题对于 AI 整体以及深度学习来说有多么核心。现在，如果你想让一个系统能够推理，你需要另一套特性。推理或规划的行为，在经典 AI 中——不仅仅是基于机器学习的 AI，而是自 20 世纪 50 年代以来——包括寻找问题解决方案的方式。例如，如果我给你一个城市列表，让你找出经过所有城市的最短回路，你会思考并说，我应该从附近的城市出发，这样总回路尽可能短。所有可能回路构成一个空间，即城市的所有排列顺序，这是一个巨大的空间。GPS 等算法搜索路径的方式是，在所有可能路径中搜索最短的一条。所有推理系统都基于这种在可能解空间中搜索的想法；你搜索符合目标的那一个。当前系统（如 LLM）进行这种搜索的方式非常原始。它们在所谓的词元空间（即输出空间）中进行。它们基本上让系统或多或少随机地生成大量不同的词元序列，然后用另一个神经网络检查所有这些假设序列，找出看起来最好的一个，然后输出。这极其昂贵，因为需要生成大量输出并选择好的。这不是我们思考的方式。我们不是通过生成大量动作然后查看结果并找出最佳动作来思考的。例如，如果我让你想象一个立方体漂浮在你面前，然后把它绕垂直轴旋转 90°，再想象这个立方体，告诉我它是否和旋转前看起来一样。答案是肯定的，因为你知道立方体旋转 90°后仍然是立方体，你仍然从同一视角看它。

The question of elaborating abstract representations from observation is key to deep learning. Deep learning is all about learning representations. In fact, one of the main conferences on deep learning is called International Conference of Learning Representations, which I created or co-created with Yoshua Bengio. So, this tells you how central the question of learning abstract representations is to AI generally and to deep learning in particular. Now, if you want a system to be able to reason, you need another set of characteristics. The act of reasoning or planning classically in AI, not just in machine learning based AI, but since the 1950s, consists in having a way of searching for a solution to a problem. For example, if I give you a list of cities and I ask you, give me the shortest circuit that goes through all those cities. You're going to think about it and say, I should go from cities that are nearby so that my total circuit is as close as possible. There is a space of all possible circuits, which is the set of all permutations of the cities, in all the orders in which you can go through the cities. It's an enormous space. The way algorithms in your GPS and things like this search for a path is that they search through all possible paths for one that is the shortest. All reasoning systems are based on this idea of a search in a space of possible solutions; you search for one that matches the objective that you want. The way current systems are doing this search, current LLMs, is in a very primitive way. They're doing this in what's called token space, which is the space of outputs. They basically have the system generate lots of different sequences of tokens more or less randomly. Then they have another neural net looking through all of those hypothesized sequences for one that looks the best. Then it outputs that. It's extremely expensive because it requires generating lots and lots of outputs and then selecting good ones. And it's not the way we think. We don't think by generating lots and lots of actions and then looking at the result and figuring out which one is best. If I ask you, for example, imagine a cube floating in the air just in front of you. Now, take that cube and rotate it by 90° around a vertical axis. Now, picture that cube and tell me if it looks like the original cube before you rotate it. The answer is yes, because you know that a cube, if you rotate it by 90°, it's still a cube. You're still seeing it from the same viewpoint.

你是说这是自由推理的幻觉？

You mean that this illusion of free reasoning?

嗯，你是在你的心智状态中进行推理，而不是在你的输出动作空间或物理世界中。你是在一个抽象空间中推理。我们拥有世界的心理模型，这让我们能够预测将要发生的事情，操控现实，预先预测我们行为的后果。如果我们能预测行为的后果，比如将立方体旋转 90°，那么我们就可以规划一系列动作以达到特定目标。每当我们有意识地完成一项任务时，我们全神贯注，思考需要执行哪些动作序列来组装这件宜家家具、用木头建造东西或做任何事情。我们每天用脑做的每一件事都涉及这种规划，而且大多数时候我们是分层规划的。例如，你某个时候要回华沙。如果你现在决定从纽约回华沙，你知道你必须去机场赶飞机。现在你有一个子目标：去机场。这就是分层规划的含义。你为最终目标定义子目标。你的最终目标是去华沙，子目标是去机场。怎么去机场？我们在纽约，所以你要上街打车去机场。怎么上街？你必须离开这栋楼，去电梯，坐电梯下楼，走出去。怎么去电梯？你必须站起来，走到门口，开门，等等。到了某个点，你会达到一个足够接近的目标，不需要规划，比如从椅子上站起来，你不需要规划，因为你太习惯了，可以直接做。你拥有所有必要的信息。所以，我们需要进行分层规划，智能系统也需要进行分层规划，这一点至关重要。我们今天不知道如何用机器做到这一点。这是未来几年的一大挑战。

Well, what you're doing is reasoning in your mental state. You're not reasoning in your output action space. In the physical world. Or in whatever your output state is. You're reasoning in an abstract space. We have mental models of the world that allow us to predict what's going to happen, manipulate reality, predict in advance what the consequences of our actions are going to be. If we can predict the consequences of our actions, like rotating a cube by 90°, then we can plan a sequence of actions to arrive at a particular goal. Whenever we accomplish a task consciously, all of our mind is focused on it and we think about what sequence of actions I have to do to assemble this piece of IKEA furniture or build something out of wood or just do anything. Everything we do every day that uses our mind involves this type of planning, and most of the time we plan hierarchically. For example, you're going to go back to Warsaw at some point. If you decide right now to go back to Warsaw from New York, you know that you have to go to the airport and catch a plane. Now you have a subgoal: going to the airport. This is what hierarchical planning is about. You define subgoals to an ultimate goal. Your ultimate goal is going to Warsaw. Your subgoal is go to the airport. How do you go to the airport? Well, we're in New York, so you go down on the street and hail a taxi to the airport. How do you go down on the street? You have to move out of this building, go to the elevator, take the elevator down, move out. How do you go to the elevator? You have to stand up, go to the door, open the door, etc. At some point, you get down to a goal that is sufficiently close that you don't need to plan, like to stand up from your chair, you don't need to plan because you're so used to doing it, you can just do it. You have all the information necessary for that. So, this idea that we need to do hierarchical planning, that intelligent systems need to do hierarchical planning, is crucial. We have no idea how to do this with machines today. That's a big challenge for the next few years.

这就是为什么你在达沃斯花那么多时间谈论机器人技术。你谈到了即将到来的机器人十年。机器人技术经历了无尽的寒冬。为什么这次不同？

That's why you spend so much time at Davos talking about robotics. You spoke about a coming decade of robotics. Robotics had endless winters. Why is this time different?

机器人可用于相对简单的任务，这些任务可以以非常简单的方式自动化，且感知不需要很复杂。例如，工厂里有喷涂汽车和组装零件的制造机器人。只要一切都在正确的位置，这些机器人基本上就是自动机。但让我们看看另一个任务，比如驾驶。自动驾驶汽车是一种机器人，或者带有驾驶辅助的汽车也是一种机器人。我们仍然没有像人类一样可靠的自动驾驶汽车。我的意思是，我们有，但像这样的公司有点作弊。它们使用的传感器比人类感知要复杂得多。

Robots are usable in tasks that are relatively simple and can be automated in a very simple way, where the sensing doesn't need to be hard. For example, you have manufacturing robots that paint cars in factories and assemble parts. As long as everything is in the right place, those robots are basically just automata. But let's take another task like driving. A self-driving car is a robot, or a car with driving assistance is also a robot. We still don't have self-driving cars that are as reliable as humans yet. I mean, we do, but companies like that cheat a little bit. They use sensors that are much more sophisticated than human sensing.

但马斯克说特斯拉将在未来五年内达到五级自动驾驶。他过去八年一直在这么说。

But Musk said that Tesla would reach level five autonomy within the next five years. He's been saying this for the last 8 years.

他说这事明年就会发生，已经说了 8 年，显然没有。所以你得别再信他了，因为他一直错。要么他以为自己是对的但错了，要么就是在撒谎。我觉得这是他激励团队每年取得可达成进步的一种方式。

He said this is going to happen next year for the last 8 years and obviously it hasn't. So either you clearly have to stop believing him on this because he's been consistently wrong. Either because he thought he was right and turned out to be wrong, or he was just lying. I think it's a way for him to inspire his team to reach attainable progress year after year.

是的，但我认为，当 CEO 告诉工程师或科学家，你们整个职业生涯致力于解决的问题，我们明年就能解决，这其实非常困难。

Yeah, but I think it's actually very difficult for an engineer or scientist to be told by their CEO that the problem you've been devoting your entire career to solve, we're going to solve it next year.

那么，你认为将 AI 与机器人结合是我们这个时代最大的挑战吗？

So, you think that this is the biggest challenge of our era to integrate AI and robotics?

如果我们能够构建理解物理世界、拥有持久记忆、能够推理和规划的 AI 系统，那么我们就有了为机器人提供动力的基础，这些机器人将比现有机器人灵活得多。所以，过去一两年成立了很多机器人公司。他们制造人形机器人等等。所有演示都令人印象深刻，但那些机器人非常笨。它们做不到人类能做的事，不是因为缺乏物理能力，而是因为它们不够聪明，无法应对真实世界。因此，很多公司寄希望于 AI 在未来 3 到 5 年内取得快速进展，这样当他们准备大规模销售和制造这些机器人时，它们就会足够聪明，因为 AI 已经进步了。这是一个很大的赌注。所以我无法告诉你这会在 3 年还是 5 年内发生，但很可能我们会在 AI 方面取得重大进展，从而在未来十年内实现更灵活的机器人，这就是为什么我说下一个十年是机器人的十年。

If we are able to build AI systems that understand the physical world, that have persistent memory, can reason and plan, then we'll have the basis for AI that can power robots that would be much more flexible than current robots. So, there are a lot of robotic companies that have been formed over the last year or two. They build humanoid robots and things like this. And all the demos are really impressive, but those robots are very stupid. They cannot do what a human can do, not because they don't have the physical ability, it's because they just are not smart enough to deal with the real world. And so, a lot of those companies are counting on the fact that AI is going to make fast progress over the next 3 to 5 years, so that when they are ready to sell those robots at a large scale and build them at a large scale, they'll be smart enough because AI would have made progress. It's a big bet. So, I can't tell you whether it's going to happen within the next 3 or 5 years, but it's very likely that we're going to make significant progress in AI that is going to enable more flexible robots within the next decade, which is why I've said the next decade is the decade of robotics.

当你看到今天 AI 的发展，日复一日、夜复一夜的进步，你感到惊讶吗？

Are you surprised when you look at AI development today, the progress day after day, night after night?

并不惊讶。让我惊讶的是它非常不连续。事实是，1980 年代和 90 年代有很多进步，然后停滞了。然后在 2000 年代又有一些进步，但未被察觉。大多数人没有意识到我们在进步。然后当这些进步在 2013 年左右变得可见时，整个领域爆发了，突然很多聪明人开始研究它。很多公司开始投资。兴趣大增。所以现在进步在加速，只是因为更多的投资和更多聪明人在做。但我原本以为从 1980 年代开始的进步会更连续。

Not really. No. What surprised me was the fact that it was highly non-continuous. The fact that there was a lot of progress in the 1980s and '90s, and then nothing. And then some more progress during the 2000s, but it was under the radar. Most people didn't realize we were making progress. And then as soon as those progress became visible around 2013 or so, then the whole field exploded and all of a sudden a lot of smart people started working on it. A lot of companies started investing. There was a lot more interest. So now progress has been accelerating just because there was more investment and more smart people working on it. But I would have thought the progress since the 1980s would have been much more continuous.

如果一项研究或开发被发表，其使用的技术会以论文或白皮书的形式公布。如果代码是开源的，那么全世界都能从中受益，而不仅仅是创造者。创造者或团队获得声望、认可，或许还有投资等等。但全世界都从中受益。这就是开放研究和开源软件的魔力。Meta，我本人和 Meta 整体，一直是开放研究和开源理念的坚定支持者。每当一个实践开放研究和开源的实体产出成果，整个开源社区也会受益。所以人们把这当作竞争，但其实不是。它更像是合作。问题是我们是否希望这种合作是全球性的？我的回答是肯定的，因为好的想法来自世界各地。例如，Llama，Meta 发布的第一个大语言模型。它不是第一个元素；之前我们发布过一些元素，但不太引人注目。它是在巴黎我们的实验室里产生的。这是我 10 年前创建的实验室。这是 FAIR 巴黎。那里有 100 多名研究人员。很多非常好的成果都来自那里。我们的蒙特利尔实验室也出了很多好东西。所以研究社区真的是全球性的。每个人都做出贡献。没有哪个实体垄断好想法，这就是为什么开放合作能让领域进步更快。这就是我们大力支持开放研究和开源的原因，因为当你与其他科学家交流时，整个领域进步更快。

If a piece of research or development is published, the techniques used to produce it are published in a paper or white paper report of some kind. And if the code is open source, then the entire world profits from it. Not just whoever produced it. The person or group who produced it gets prestige, recognition, and perhaps investment or whatever. But the entire world profits from it. This is the magic of open research and open source software. Meta, I mean myself and Meta more generally, have been extremely strong proponents of this idea of open research and open source. And whenever an entity that is practicing open research and open source produces something, the entire community of open source profits from it as well. So people are kind of formulating this as if it's a competition, but it's not. It's more like a cooperation. The question is do we want this cooperation to be worldwide? And my answer to this is yes, because there are good ideas coming from everywhere in the world. Llama, for example, the first LLM that Meta put out. It wasn't the first element; there were elements before that we put out, but they were a little bit under the radar. It was produced in Paris, in our lab in Paris. This is my lab which I created 10 years ago. So, this is FAIR Paris. It has over 100 researchers there working. And a lot of really good stuff came out of that. I mean, a lot of good stuff came out from our lab in Montreal. So, the research community is really worldwide. Everybody contributes. No entity has a monopoly on good ideas, which is why open collaboration makes the field progress faster. That's why we are big proponents of open research and open source, because the entire field progresses faster when you communicate with other scientists.

现在，行业里有些人曾经实践开放研究，但后来停止了。OpenAI 就是这种情况。而 DeepMind 从来就不开放，他们保密一切。Google 从部分开放变成基本开放（主要是因为我们），现在又变成部分封闭。例如，他们没有透露 Gemini 背后的所有技术。他们仍然在做很多开放研究，但更多是基础长期性的。所以，我认为这很可悲，因为很多人基本上把自己排除在世界研究社区之外，不参与、不为进步做贡献。过去 10 年 AI 进步如此之快的原因就是开放研究。

Now, there are some people in the industry who used to practice this open research and then stopped. That's the case for OpenAI. And so, DeepMind was never open, so they keep everything secret. Google kind of went from being partially open to being open essentially because of us, to now being partially closed. They're not revealing all the techniques behind Gemini, for example. They're still doing a lot of open research, but it's more kind of fundamental long-term. So, I think it's sad because a lot of people are basically putting themselves outside of the world research community and not participating, not contributing to progress. The reason why progress in AI has been so fast in the last 10 years is because of open research.

你真的相信吗？

Do you really believe it?

哦，绝对。不，不，这是事实。我不是唯一这么认为的人。这不是信念，是事实。我给你举个例子。几乎整个 AI 行业，至少在研发阶段，都使用一个叫 PyTorch 的软件来构建系统。PyTorch 是开源的。它最初由我在 Meta 的同事在 FAIR 开发，后来更多人参与。几年前，PyTorch 的所有权转移给了 Linux 基金会。所以 Meta 不再拥有它。它仍然是主要贡献者，但不控制它。它基本上由开发者社区控制。整个行业都在用。包括 OpenAI、Anthropic。Google 有自己的东西，但也包括微软、英伟达，所有人。每个人都用 PyTorch。整个学术界研究都用 PyTorch。你可以看到，所有科学文献中的论文，大约 70%提到了 PyTorch。所以这告诉你，AI 的进步是建立在彼此工作之上的。这就是科学和技术进步的方式。

Oh, absolutely. No, no, this is a fact. I'm not the only one. It's not a belief, it's a fact. Let me give you an example. Almost practically the entire AI industry builds, or at least at the research and development stage, uses a software to build systems called PyTorch. PyTorch is open source. It was produced by my colleagues at Meta, at FAIR initially, and then a bigger population. A few years ago, the ownership of PyTorch was transferred to the Linux Foundation. So, Meta does not own it anymore. It's still the main contributor, but it doesn't control it. It's controlled by a community of developers, essentially. The entire industry uses it. That includes OpenAI, Anthropic. Google has their own thing, but it includes Microsoft, Nvidia, everybody. Everybody uses PyTorch. The entire academic world research uses PyTorch. As you can see, all the papers that appear in the scientific literature, PyTorch is mentioned in something like 70% of them. So, what it tells you is that progress in AI builds on each other's work. And that's how you make science and technology progress.

也许星门项目会改变一切。

Maybe Stargate project would change everything.

不会。

No.

现在，所有涉足 AI 的公司都看到了一个未来，一个不远的未来，数十亿人将每天使用 AI 助手。我戴着一副眼镜，不知道你能不能看到，它上面有摄像头。这是 Ray-Ban Meta，由 Meta 制造。你可以和它对话。它连接了一个 AI 助手。你可以问它任何问题，甚至可以让它通过摄像头识别植物种类等等。所以我们看到这样一个未来：人们会戴着智能眼镜，或者使用智能手机和其他智能设备，基本上在日常生活中一直使用 AI 助手。它们会在日常生活中帮助人们。这意味着将有数十亿用户每天多次使用这些 AI 助手。为此，你需要非常庞大的算力基础设施，因为运行一个 LLM 或 AI 系统，无论是什么，都不便宜。所以你需要大量的算力。大部分投资都用于此。Meta 今年将投资约 600 到 650 亿美元用于基础设施，主要用于 AI。微软宣布投资 800 亿美元。而 StarNet 是 5000 亿美元，但那是 5 到 10 年的计划，我们不知道钱从哪里来。所以投资规模是同一数量级的，与微软和 Meta 已经在做的并没有太大不同。其中大部分用于推理，也就是运行 AI 助手来服务数十亿人，而不是训练大模型，训练实际上相对便宜。

Now, all the companies that are involved in AI are seeing a future, pretty near future, where billions of people will want to use AI assistants on a daily basis. I'm wearing a pair of glasses now. I don't know if you can see it, but it's got cameras on it. These are the Ray-Ban Meta, built by Meta. And you can talk to them. There is an AI assistant that is connected to. And you can ask it any question. You can even ask it to recognize the species of plant from the camera and everything. So, we see a future where people would be wearing smart glasses or maybe using their smartphone or other smart devices and basically we use AI assistants all the time in their daily lives. They will help them in their daily lives. Now, that means there are going to be billions of users of those AI assistants using them multiple times a day. And for this you need a very big infrastructure of compute because running an LLM or an AI system, whatever it is, is not cheap. So, you need a lot of compute power. Most of that investment. So, Meta is investing this year on the order of 60 to 65 billion in infrastructure, mostly for AI. Microsoft has announced they're investing 80 billion. And then StarNet is 500 billion, but it's over 5 or 10 years and we don't know where the money is coming from. So, it's on the same order of magnitude of investment. It's really not that different from what Microsoft and Meta are already doing. And most of it is for inference. So, it's for running AI assistant to serve billions of people. It's not for training large models. That is actually relatively cheap.

我有一个观众的问题。你提出了 Transformer 的替代方案，而 Transformer 是 LLM 最重要的部分。G power 世界模型与 Transformer 有何不同？为什么你认为世界模型是未来？

I have a question from our viewer. You propose an alternative to the transformer, which is the most important piece of LLMs. How does G power world model differ from transformers and why do you think world models are the future?

好的。有一种架构，实际上应该被称为宏架构，叫做 JEPA，即联合嵌入预测架构。它并不是 Transformer 的替代品。你可以在 JEPA 内部使用 Transformer。JEPA 是一种宏架构，在其中你可以安排不同的模块，这些模块可以是 Transformer，也可以是其他东西。所以它们是正交的概念，并不对立。JEPA 所替代的是目前没有通用名称但实际上是当前一代大语言模型的东西。在业界，它们被称为自回归仅解码器架构，或者 Transformer，或者 OpenAI 称之为 GPT，即通用预训练 Transformer。所以 GPT 只是一种特定的架构，使用我之前描述的自监督学习技术进行训练：你取一个符号序列，比如文本，一个词序列，然后训练一个系统。系统的组织方式是，为了预测输入中的某个词，它只能看它左边的词。这被称为因果架构。如果你训练一个系统，给它输入文本，并训练它重现输入文本，那么隐含地你就是在训练它预测文本中的下一个词。一旦训练完成，你就可以使用这个系统自回归地一个词一个词地生成，这就是大语言模型。现在，尝试将其应用于现实世界，比如你想训练一个机器人来规划事情或预测世界将发生什么，这行不通。所以，如果你不用词，而是从视频中提取帧，并将这些帧转化为类似词的令牌，然后尝试训练系统预测视频中接下来会发生什么，效果并不好。原因在于世界上有很多事情你根本无法预测。在视频这样的高维空间中，表示你无法精确预测将要发生的事情本质上是一个数学上棘手的问题。在文本这样的离散空间中这是可能的。所以，你无法预测文本中下一个词是什么，但你可以预测所有可能词的概率分布。我们不知道如何对视频做到这一点。我们不知道如何表示所有可能视频帧的分布。因此，用于文本的技术（对文本、DNA 序列和蛋白质效果很好）不适用于视频或其他自然信号。JEPA 就是对此的答案。主要思想是，不在输入空间中进行预测，而是训练系统学习输入的抽象表示，然后在该表示空间中进行预测。这被证明是更好的问题表述方式。因为如果我拍摄你所在房间或任何房间的视频，将相机对准一个位置，然后慢慢转动相机，然后停下，问系统「视频中接下来会发生什么」，系统可能会预测相机将继续转动。但它无法预测相机旋转后视野中的所有细节。那里有一株植物，墙上可能有一幅画，可能有人坐着。它无法预测那些人长什么样，无法预测植物的种类或地板的纹理等等。这些根本无法预测。因此，如果你训练系统进行这些预测，它会花费大量资源试图预测无法预测的事情，并失败。

Okay. So, there is this architecture, which really should be called a macro architecture called JEPA. That means joint embedding predictive architecture. And it is not an alternative to transformers. You can have transformers inside of JEPAs. JEPA is a macro architecture within which you arrange different modules, and those modules could be transformers. They could be other things if you want, but they could be transformers. So it's not those are orthogonal concepts. They are not in opposition if you want. What JEPA is an alternative to is something that doesn't have a common name but basically are the current crop of large language models. In the business they're called autoregressive decoder-only architectures, or transformers, or OpenAI calls them GPTs, general purpose transformer. So a GPT is just a particular architecture that is trained using this self-supervised learning technique I was describing earlier, where you take a sequence of symbols, let's say text, a sequence of words, and you train a system. The system is organized in such a way that to predict a particular word on the input, it can only look at the words that are to the left of it. It's called a causal architecture. And if you train a system to feed it a text and you just train it to reproduce that text on its input, then implicitly you train it to predict the next word in a text. Then you can use that system once it's trained to just produce one word after the other autoregressively, and that's what large language models are. Now, try to apply this to the real world because you want to train a robot to plan things or predict what's going to happen in the world. It doesn't work. So, if instead of words, you take frames from a video and you turn those frames into tokens, like the words, and you try to train the system to predict what's going to happen in the video, it doesn't work very well. And the reason it doesn't work is because there are a lot of things that happen in the world that you simply cannot predict. Representing the fact that you cannot exactly predict what's going to happen is essentially a mathematically intractable problem in high-dimensional space like videos. It's possible in discrete space like text. So, you cannot predict what word comes after a text, but you can predict the probability distribution of all the possible words. We don't know how to do this with videos. We don't know how to represent a distribution over all possible video frames. So, the techniques that are used for text that work really well for text and for DNA sequences and proteins do not work for video or other natural signals. So, JEPA is an answer to this. The main idea is that instead of making that prediction in the space of the inputs, you train the system to learn an abstract representation of the input and then train it to make prediction in that representation space. And that turns out to be a much better way of formulating the problem. Because if I take a video of the room you are in, or any room, and I point the camera at one location, and then I slowly turn the camera, and then I stop and I ask the system, 'Tell what happens next in the video.' The system might predict that the camera is going to keep turning. But there's no way it can predict all the details of what's going to be in the field of view after the camera rotates. There is a plant, there might be a painting on the wall, there might be people sitting. It cannot predict what those people are going to look like. It cannot predict what the species of plant is or what the texture of the floor is going to be and things like that. It's just impossible to predict. And so if you're training a system to make those predictions, it spends a huge amount of resources trying to predict things it cannot predict and it fails.

Yann LeCun 实验室最大的成就是

The greatest achievement of Yann LeCun laboratory is

没有 Yann LeCun 实验室。很难用一个数字来衡量。我出名的是卷积神经网络，这是一种受视觉皮层结构启发的特定架构，旨在处理图像、视频、音频、语音等自然信号。这些系统被广泛应用。所以，如果你的车有任何驾驶辅助系统（现在欧盟销售的大多数汽车都必须配备），至少它们必须有一个系统能在前方有障碍物时自动刹车。那就是你的实验室。

There is no Yann LeCun laboratory. It's hard to put a figure on it. I mean, what I'm known for is something called convolutional neural network, which is a particular architecture inspired by the architecture of the visual cortex, designed to handle natural signals like images, video, audio, speech, things like this. And those systems are used everywhere. So, if you have any kind of driving assistance system in your car and most cars sold in the EU now have to have that, at least they have to have a system that brakes your car automatically when there is an obstacle in front of it. That's your laboratory.

它用的是卷积网络。对，所有这些都是。那是我 1988 年的发明，历史很久远了。所以那是我最出名的成就。最早的应用是字符识别、手写识别、读取邮政编码、读取支票上的金额等等。那是 90 年代初的事。然后大约从 2010 年开始，这方面的应用迅速增长。当你对着手机说话时，语音识别系统的前几层神经网络通常使用卷积网络。当你的手机上有一个应用，可以拍一张植物的照片，然后问它这是什么植物、什么昆虫，或者听鸟鸣识别种类，那都是卷积网络。

It's using convolutional net. Okay. All of them. That's my invention from 1988. It goes back a long time. So, that's what I'm most famous for. The first applications were character recognition, handwriting recognition, reading zip codes, reading checks, the amounts on checks, things like that. That was in the early '90s. And then since 2010 roughly, there's been a very quickly growing set of applications for this. When you talk to your phone, speech recognition systems, the first few layers of the neural net that does this usually uses convolutional nets. When you have an application on your phone that you can take a picture of a plant and then ask your app what is the species of that plant or species of that insect or listening to the song of a bird and tell you what species it is, that's a convolutional net.

欧洲在这场 AI 竞赛中处于什么位置？

What is the place of Europe in that AI race?

我认为欧洲可以发挥非常重要的作用，因为欧洲拥有最困难的东西……

I think Europe has a very important role to play because Europe has the most difficult thing...

实施法规。

Implementing regulations.

嗯，欧盟确实存在这类问题。比如我现在戴的这副眼镜，一个应用是解读摄像头捕捉的图像，这样你可以看波兰语的菜单，或者你可以用波兰语跟我说话，菜单就会被翻译。实际上，这项功能今天已经可以实现。但眼镜本身在欧洲可以买到，只是视觉功能不可用，因为监管不确定。甚至不清楚监管是否会使其非法，只是不明确。但我要说，欧洲拥有巨大的资产和优势。首先是人才。

Well, there are issues of that type in the EU, that's for sure. For example, the glasses I'm wearing right now, one application is interpreting the images that go through the camera so you can look at a menu in Polish or you could be speaking to me in Polish. And there would be translation of the menu. Actually, that's available today. In those glasses, except the glasses are not available. The glasses are available in Europe except the vision feature is not available because of uncertainty about regulation. It's not even clear the regulation would make it illegal, it's just that it's unclear. But let me say that Europe has big assets, big advantages. And the first one is talent.

我们的程序员。

Our programmers.

我们的程序员、数学家、物理学家。更广泛地说，计算机科学家、工程师、物理学家等等。很多顶尖的 AI 科学家，无论他们在世界何处工作，都来自欧洲。我就来自欧洲。

Our programmers, mathematicians, physicists. Computer scientists, engineers more generally, physicists, etc. A lot of top scientists in AI, regardless of where they work in the world, come from Europe. I come from Europe.

我记得在诺贝尔奖新闻发布会上，我问杰弗里·辛顿。我的第一个问题是：辛顿教授，您有什么遗憾吗？我想问您同样的问题。

I remember the press conference Nobel Prize conference when I asked Geoffrey Hinton. And my first question is to Professor Geoffrey Hinton, do you regret something? And I would like to ask you the same question.

我不知道杰夫怎么回答的，但我能猜到。我先说说我的答案。很长一段时间里，我对现在所谓的自监督学习不感兴趣，因为我认为这个问题表述得很糟糕。事实上，我和杰夫·辛顿讨论了很多年，我一直在推动监督学习，而他告诉我最终我们需要弄清楚如何做他所谓的无监督学习，也就是现在自监督学习的一种特殊形式。直到 2000 年代中期我才改变想法，那可能晚了 10 年。所以我应该更早对这个问题产生兴趣。但问题是，90 年代中期到 2000 年代初期，神经网络和深度学习没什么进展，因为整个世界对此完全不感兴趣。所以我们不得不做别的事情。我研究了图像压缩，一个叫 DjVu 的系统，我听说它在波兰挺流行的，实际上在东欧更普遍。所以我认为这是我本可以做得不同的一件事。除此之外，我对事情的发展相当满意。我本可以更努力地让社区在 90 年代末保持对神经网络和机器学习的兴趣，这样就不会有所谓的深度学习寒冬了。我猜杰夫可能回答的一件事是，他在两年前改变了想法。他职业生涯的追求一直是找出大脑皮层的学习算法。他一直认为反向传播——我们今天训练神经网络的主要技术，他和我都参与其中——不是大脑使用的，因为大脑一定用了别的东西，反向传播在生物学上不太合理。所以在过去四年里，他每两年就提出一种新的机器学习方法。两年前他放弃了。他说：「好吧，也许大脑不用反向传播，但反向传播效果很好，也许我们只需要这个。也许它比大脑用的任何东西都更好。」于是他顿悟了，然后退休了。基本上，他可以宣布胜利了。

I don't know what Jeff answered to that question, but I can guess what he answered. Let me give you my answer first. So my answer is for the longest time I was not interested in what we now call self-supervised learning because I thought it was badly formulated as a problem. And in fact, I had those discussions with Jeff Hinton for many years where I was pushing supervised learning and he told me ultimately we need to figure out how to do what he calls unsupervised learning, which is now a particular form of self-supervised learning. And I only changed my mind about this in the mid-2000s. That was probably 10 years too late. So, I should have probably got an interest in that problem earlier. But the thing is, between the mid-90s and the early 2000s not much happened in neural nets and deep learning because the whole world was completely uninterested in that. So, we had to do something else. I worked on something else. I worked on image compression, a system called DjVu, which I heard is pretty popular in Poland, actually, in Eastern Europe more generally. So I think that's one thing I would have done differently. Other than that, I've been pretty happy with things the way they have been going. I would have been a little more forceful also at keeping the interest of the community in neural nets and machine learning in the late 90s than I was. So that there wouldn't have been a winter of deep learning if you want. I'm guessing perhaps one thing that Jeff might have answered is that he had a bit of a change of mind 2 years ago where the quest of his career was to figure out the learning algorithm of the cortex of the brain. He always thought that back propagation, which is the main technique we used to train neural nets today, which he had something to do with and I had something to do with as well. He always thought that was not what the brain used because the brain must be using something else since back propagation is not really biologically plausible. And so he kept coming up with new ways of doing machine learning every 2 years for the last 4 years. And 2 years ago he just gave up. He said, 'Well, maybe the brain doesn't use back propagation, but back propagation works really well and maybe that's all we need. Maybe it works even better than whatever it is that the brain uses.' So he had this epiphany and then retired. Basically, he could declare victory.

非常感谢您抽出时间来到这里。非常荣幸。谢谢。

Thank you very much for your time and having you here. It was a great honor. Thank you.

荣幸之至。

A pleasure.