AI 史上最伟大：Jürgen Schmidhuber 谈欧洲的 AI 遗产

没有 AI 能像水管工那样用双手完成工作。我们离实现这个伟大愿景还有多远？

There's no AI that can do what a plumber can do with his hands. How far are we away to get to this great vision?

我认为我们离得不远了。我可能活着就能看到。今天把 1 万亿美元投进 GPU 的人，会亏掉其中很大一部分。我们正处在一个暂时的泡沫里。一旦你拥有了那个，你就会拥有一种新的生活。那时你将拥有真正能自我改进的机器。那时一个新时代将会来临。

I think we are not very far away. I might live to see it. Guys who are investing today 1,000 billion dollars into GPUs, they are going to lose a lot of that. We are in a momentary bubble. Once you have that, you have a new kind of life. Then you will have truly self-improving machinery. Then a new age will dawn.

欢迎收听播客的新一期节目。今天我们要对话的不是别人，正是 Jürgen Schmidhuber。很多人介绍他是 AI 的先驱之一。《纽约时报》称他为现代 AI 之父。有些人甚至更进一步，称他为 AI 的四骑士之一，当然这是在暗指即将到来的奇点。希望你们喜欢与 Jürgen Schmidhuber 的这场对话。

Welcome to a new episode of the podcast. Today, we're going to talk with none other than Jürgen Schmidhuber. Many people have introduced him as one of the pioneers of AI. The New York Times called him the father of modern AI. Some people even went further, calling him one of the four horsemen of AI, of course alluding to the impending singularity. I hope you enjoy this conversation with Jürgen Schmidhuber.

我想从头开始。那么，欧洲在这场新的 AI 革命中扮演什么角色？你在最近的一篇文章里，把 1991 年的慕尼黑放在了这场新革命的中心。你能详细说说为什么你认为慕尼黑对现代 AI 有这么大的贡献吗？

I'd like to start at the beginning. So, what's the role of Europe in all this new AI revolution and you in a very recent post, you you know, put Munich at the center of of this new revolution in 1991. Can you make elaborate why you think uh Munich contributes such uh a bigger part to modern AI?

一般来说，现在所谓 AI 的基础算法有 90% 是在欧洲开发的，日本也有一点，还有超级大国美国和中国也有一点。但 1991 年是非常强劲的一年，因为当时在短短几个月内，我们就有了如今这些大语言模型的核心要素，这些要素定义了今天人们对 AI 的看法。比如 ChatGPT 里的 P 代表预训练，那就是我们在 1991 年就有的。我记得是 1991 年 3 月或 4 月。然后是 T，代表 Transformer。它是一种特殊的人工神经网络，现在运行着大多数大语言模型，而当时我们有了第一个 Transformer 变体，现在被称为未归一化线性 Transformer。未归一化线性 Transformer，那是在 1991 年。为什么叫线性 Transformer？因为与更现代的 2017 年二次方 Transformer 不同，它线性扩展。什么意思？意思是如果你有 1000 倍的文本，你只需要 1000 倍的算力。而用二次方 Transformer，如果你有 1000 倍的文本，你需要 100 万倍的算力。然后我们还有蒸馏。在同一篇做预训练的论文里，我们也有从一个网络到另一个网络的蒸馏。最近，中国公司被指控通过下载网络数据来蒸馏美国模型对世界的认知，然后训练他们的神经网络，再蒸馏到一个更小的网络里。所有这些都可以追溯到 1991 年。有趣的是，这篇论文——蒸馏技术——差不多四分之一世纪后被其他人重新发表了。他们把它蒸馏成了一篇新论文，却没有提及起源。然后同一年，我们有了第一篇经过同行评审的关于今天被称为生成对抗网络的论文，其中有两个网络互相竞争，一个试图生成让另一个惊讶的输出，另一个当时无法预测这些输出的后果，但随着时间的推移，它学会了预测后果，然后这个网络最小化的误差正是另一个网络最大化的东西，它们互相竞争，这在近年来变得非常流行。这就是生成式 AI 里的 G。那段时间还有一大堆其他东西。我们更早就有了元学习。元学习今天非常重要。递归自我改进。什么意思？意思是有一个系统不仅在这里学一点、那里学一点，还学习学习算法本身。那是在我 1987 年的文凭论文里，第一批用于做自指自我改进算法的构造性算法，能生成自身更好的版本。然后大约同一时间，还有今天有些人称之为 JPA 之类的，不管那是什么意思，它正是我们在 1992 年发表的一系列技术。它叫可预测性最大化。还有一大堆其他技术，当时在我的杰出博士生 Sepp Hochreiter 的文凭论文里自然地交织在一起。有第一个深度残差学习。深度残差学习是让今天的神经网络真正变深的东西。当时深度学习不奏效，但有了那个新方法，它突然就奏效了，这成为了 20 世纪被引用最多的 AI 论文的核心，那篇论文是关于长短期记忆的，其核心正是深度残差学习。它也成为了 21 世纪被引用最多的论文的核心要素。

Generally speaking, 90% of the basic algorithms for what's now called AI were developed in Europe, but also a little bit in Japan and um and also a little bit in the superpowers in um USA and China. But, 1991 was a really strong year because back then, within just a handful of months, we had all kinds of things that are now central for these large language models that kind of define what people think about AI today. So, there's the chat GPT and the P means pre-training and that's what we had back then in 1991. I think it was in in March or in April 1991. And then in the T, it's the transformer. It's a particular kind of artificial neural network is now running most of these large language models and and back then we had the first transformer variant, which is now called the unnormalized linear transformer. The unnormalized linear transformer, that was in 1991. Why is it called the linear transformer? Because unlike the more modern 2017 quadratic transformer, it scales linearly. What does that mean? It means if you have 1,000 times more text, then you need 1,000 times more compute. While with the quadratic transformer, if you have 1,000 times more text, you need 1 million times more compute. And then we had distillation. In the same paper that did the pre-training, we also had distillation from one network to another. And recently, Chinese companies have been accused of distilling what what the American models know about the world by downloading the web and then training their neural networks and then distilling it down into a smaller network. And all of that goes back to 1991. And it's interesting that this paper was um the technique of distillation was you know, almost a quarter century later republished by other guys. So, they kind of distilled it into a new paper without mentioning the origins. And then we had in the same year the first peer-reviewed publication about something that today is called generative adversarial networks where you have two networks that fight each other and one is trying to generate outputs that surprise the other one which is at the moment not able to predict the consequences of these outputs, but then over time it learns to predict the consequences and then the error that this network is minimizing is the the same thing that the other network is maximizing and they are fighting each other and this has become very popular in recent years. That's you know, the the G in generative AI. And you had a whole bunch of other things around that time. We had metal learning even earlier. Metal learning is very important today. Recursive self-improvement. What does that mean? It means that you have a system that not only learns something here and learns something there, but also learns to learn the learning algorithm itself. And that was back then in my diploma thesis in '87 first constructive algorithms for the for doing the self-referential self-improving algorithm that generates better versions of itself. And then um also in around that time something that is today called what some people call it JPA whatever that may mean and it's it's exactly the a of techniques that we published in 1992. It's called predictability maximization. And a whole bunch of other techniques which back then were kind of natural all interlocking in the diploma thesis of my brilliant PhD student Sepp Hochreiter. There was the first deep residual learning. Deep residual learning is the thing that makes today's neural networks really deep. And back then deep learning didn't work, but with that new approach suddenly it did work and and that became the core of the most cited AI paper of the 20th century which was about long short-term memory and the core of that is really deep residual learning. And it became also the the central ingredient of the most cited paper of the 21st century.

你描述了你在过去十年推动领域发展的核心思想，你提到这些来自慕尼黑，那是二十多年前。我想你提到了生成对抗网络，你提到了注意力机制。这些论文可能几年前才登上头条，人们会说。为什么当时的人，或者如果人们试图把功劳归于

You describe the central ideas which you moved the field on for the last decade, you know, and you mentioned those coming from Munich, you know, more than two decades ago. I think you know you mentioned general generative adversarial networks, you mentioned attention. Uh papers which you know hit the the headlines maybe just you know, some years ago people would say. How come people at the time or if people would try to attribute credit

是的。

Yeah.

研究者时，他们最可能把这些归功于最近的贡献，比如你提到的那些。为什么人们错过了那些论文？

to researchers, they would most likely attribute those to the most recent contributions like the one that you mentioned in How come people uh missed out on on those papers?

嗯，有一个非常简单的解释。当时 1991 年前后几年，算力比今天贵几百万倍。因为大约每 5 年，计算机就便宜 10 倍。所以 30 年里，你就获得了 100 万倍的提升。这造成了天壤之别。所以我们不得不等到 2010 年代初，计算机变得足够便宜，让上个千年的所有这些技术快到足以在数十亿人的数十亿部智能手机和其他计算机上铺开。

So um there's a very simple explanation. Back then in 1991 plus minus a couple of years compute was millions of times more expensive than it is today. Because every 5 years, roughly, computer is getting 10 times cheaper. So, in 30 years, you gain a factor of 1 million. And that makes all the difference. So, we had to wait until the early 2010s until computer was cheap enough to make all these techniques from the previous millennium fast enough such that they could be rolled out for billions of people on billions of smartphones and other computers.

所以本质上，你是说，仅仅二十多年后，我们有了 GPU、算力，有些人甚至说还有正确的数据，我们才能真正从所有这些方法中受益，而那些真正展示其适用性的人。最终，那些用新数据发表这些发现的人受到了赞誉，也许不是 90、91 年的先驱。但也许再稍微停留一下，当时有什么我们可以做得更好的吗？我感觉我们做了，你知道，我们在这样的领域里有点被低估了，对吧？

So, essentially, you're saying, you know, only two decades much later, you know, we had the GPUs, the the the compute power, some people even argue in the the right data, and we could really benefit from all these methods, and those were the ones really showing their applicability. And in the end of the days, the guys which published those funding those findings with the this new date were the ones celebrated, and not maybe perhaps the pioneers in uh in 90 91. But maybe to uh to linger on that a little bit, is there anything um that we could have done better at the time? It feels to me like we did, you know, we are sort of under recognized in in in such a field, right?

是的。

Yeah.

所以，我不担心自己不被认可，但我有点担心更古老的基础性贡献被忽视。例如，深度学习是如何开始的？有人声称深度学习始于 20 世纪 80 年代左右。这完全错误。深度学习始于 1965 年，而且是在乌克兰。当时乌克兰是苏联的一部分，乌克兰的 Ivakhnenko 和 Lapa 做出了最早的深度神经网络，其中所有中间层的隐藏单元都能学到东西。他们本应获得该领域的所有奖项，但没有。Ivakhnenko 我想他于 2013 年去世。所以，真正拥有第一个深度学习网络的人，在 1971 年发表了一篇英文论文，用一个八层网络学习预测英国经济的各个方面。他用了一种巧妙的方法，不需要反向传播，而是逐层训练。而另一个人，很久之后，20 年甚至更晚，重新发表了那项工作，却没有正确注明该技术的发明者。他因此获得了奖项，这很荒谬。所以，我认为我们的领域有一个问题，即核心思想的起源被错误归因。

So, I'm not worried about being underrecognized, but I'm a little bit worried about even older fundamental contributions that are being ignored. For example, how did deep learning start? There are people who claim deep learning started in the 1980s or something. It's totally wrong. Deep learning started in 1965 in Ukraine, of all places. By then, Ukraine was part of the Soviet Union, and there were Ivakhnenko and Lapa in Ukraine who had the first deep neural networks where all the hidden units in these intermediate layers learn something. They should have gotten all the awards of the field, but they didn't. Ivakhnenko, I think he died in 2013. So, the guy who really had the first deep learning networks and who had in 1971 an English language paper with an eight-layer network that learned to predict aspects of the British economy. And he did that in a clever way which didn't require backpropagation where you train layer by layer. And another guy, much later, 20 years later or even later than that, republished that without attributing correctly the inventor of that technique. And he got the awards for that, which is ridiculous. So, I think our field has a problem in the sense that the origins of the core ideas are being misattributed.

嗯。好的，我明白了。我的意思是，我认为这是科学的一个普遍问题。还是说——我想理解一下，你知道，

Mhm. Okay, so I understand that. I mean, I think this is a general problem, I guess, of science. Or is there—I'm trying to understand, you know,

这不是科学的普遍问题，因为在科学中，功劳分配大多运作良好，因为整个科学事业都基于此。当你还是个博士生时，你可以通过做出一些酷炫的发明、数学洞见或其他什么来成名，然后你因此获得认可。在数学中，如果你有一个新定理，但使用了别人开创的证明技巧，如果你不明确说明，就会有问题，因为其他数学家会迫使你修改论文等等。机器学习与这些古老科学领域不同，因为它现在可能过于被商业利益主导，各种人都能发表东西，并且可以发表那些本应引用他们重新发表的方法的原始出处的东西，但他们不引用，甚至在后续综述中也不引用。所以我们有很多这样的案例。在传统科学领域，这些人会成为弃儿。他们不会获得奖项或荣誉。所以，最终，真相总会胜利。真相是科学的不动点，是的，只要真相还不清楚，你就在围绕这个不动点转，但它是科学收敛的唯一不动点。所以，最终，所有这些都会水落石出。问题只是那些真正发明这些方法的先驱们，他们是否能活着看到认可。就像有一个人，他没有得到他应得的全部认可，因为他和 Lapa 发明了深度学习。

It's not a general problem of science because mostly in science credit assignment works well because the whole business of science is based on it. Because when you're a little PhD student, you can make a name for yourself by coming up with some cool invention, maybe some mathematical insight or whatever, and then you get credit for that. And in mathematics, if you have a new theorem, but you use a proof technique that was pioneered by somebody else and if you don't make that clear, you get a problem because then the other mathematicians are going to force you to correct your paper and everything. And machine learning is different from these old fields of science in the sense that maybe it is too dominated by commercial interests now and that all kinds of people can publish stuff and get away with publishing stuff that should have cited the originals of the methods that they are republishing, but they don't cite them even in later surveys. And so we have lots of cases like that. And in traditional fields of science, these guys would become outcasts. They would not get prizes or awards or something. So, in the end, the truth is always going to win. The truth is the fixed point of science, and yes, you are circling around this fixed point as long as it's not yet clear what is the truth, but it's the only fixed point towards which science is converging. So, in the end, all of that will pan out. It's just a question whether the guys—the pioneers who really invented these methods—whether they will live to see the acknowledgement. Like I have a man who didn't really get all the recognition that he should have gotten for deep learning, which he and Lapa invented.

是的，当然，这是一个悲伤的故事。我的意思是，即使我们考虑最近的诺贝尔奖，当然颁给了 AI。我们可以争论说，他们获奖时可能关注的是另一个焦点，但这些名字在这段历史中完全没有被提及。

Yeah, of course, this is a sad story. I mean, and even if we maybe consider the latest Nobel Prize awards, which went to AI, of course. We can argue here maybe they had another focus in mind when they got rewarded, but these names are not named at all in this history.

是的，我认为这对诺贝尔奖委员会来说非常尴尬，他们完全忽略了一个事实：那些获得诺贝尔奖的人重新发表了别人更早发表的东西，却没有引用，甚至在后续综述中也没有。有一份著名的报告叫《剽窃的诺贝尔奖》。你可以在那里读到所有细节。

Yeah, that is very embarrassing, I believe, for the Nobel Prize Committee that they completely missed the fact that the guys who got Nobel Prizes republished stuff that was published earlier by other guys whom they didn't cite, not even in later surveys. And there's a well-known report out there, 'A Nobel Prize for Plagiarism.' There you can read all the details about that.

现在，回到正题，你提到的许多关键思想都是在欧洲发展起来的。你能否挑出一个，我们可以讨论它至今的影响，你认为当时它已经具有巨大潜力，但我们直到今天才看到。

Now, circling back, there are many some of the key ideas that you mentioned have been developed here in Europe. Can you perhaps single out one that we can maybe discuss its consequences until today that you think back then had already great potential, but we only came to see it today.

是的，我认为在 1991 年具有巨大潜力的一件事是我所称的快速权重控制器，快速权重编程器，其中一个神经网络学习编程另一个神经网络的权重。所以，通常你有一个单一的神经网络，它通过梯度下降学习改变其权重。一些权重变强，一些变弱，最终它学会创建从输入到期望输出的更好映射。但当时，想法是让系统本身，神经网络本身，学习如何编程另一个网络的权重，以便它可能以比直接梯度下降更好的方式完成这些权重变化。所以你使用梯度下降来训练一个网络生成比梯度下降生成的更好的权重变化。所以，这走向了元学习。而最简单的版本，我当时称之为快速权重控制器，这正是今天所称的非归一化线性 Transformer。非归一化线性 Transformer，它不同于现代二次归一化 Transformer，但原理相同，你有一个网络生成键和值，在另一个网络中创建映射，然后应用于传入的查询。实际上在 1991 年，这个系统是高效的，因为它线性扩展。那么，这意味着什么？如果你有 100 倍的文本，那么你需要 100 倍的算力。而现代二次 Transformer 则有一个不理想的情况，因为如果你有 100 倍的文本，你需要 100 乘以 100 等于 10,000 倍的算力，这就是为什么对数据中心的投资疯狂爆炸的原因之一。是的。所以，当时已经清楚这是一件好事。这是处理通用神经网络的好方法，因为基本上你在做早期计算机先驱分离存储和控制时所做的同样的事情。所以你有一个控制器学习操作另一个神经网络的存储，即快速权重，就像存储一样，你可以读写，然后你可以在第一个网络中学习创建对这个快速存储的良好读取和写入。然后你有了今天所称的测试时训练。当时就已经有了。你基本上有了传统计算机科学中非常成功的存储和控制的分离。是的。

Yeah, one thing that I think had great potential back then in 1991 was what I called a fast weight controller, a fast weight programmer, where there is one neural network that learns to program the weights of another neural network. So, normally, you have one single neural network, and it learns by gradient descent to change its weights. Some of those weights get stronger, some get weaker, such that in the end it learns to create a better mapping from inputs to desired outputs. But back then, the idea was let's have the system itself, the neural network itself, learn how to program the weights of another network such that maybe it can do these weight changes in a better way than those that you can learn by straightforward gradient descent. So, you use gradient descent to train one network to generate weight changes that are better than those generated by gradient descent. So, that goes towards meta-learning. And the simplest version of that, I called it fast weight controller back then, and that's exactly what today is called an unnormalized linear transformer. The unnormalized linear transformer, which is different from the modern quadratic normalized transformer, but the principles are the same and you have one network that generates keys and values to create a mapping in another network which is then applied to the incoming queries. And actually back then in 1991, this system was efficient in the sense that it scaled linearly. So, what does that mean? If you have 100 times more text, then you need 100 times more compute. While with the modern quadratic transformer you have an undesirable situation because if you have 100 times more text, you need 100 times 100 equals 10,000 times more compute, which is one of the reasons why these investments into data centers are exploding like crazy. Yeah. So, back then it was already clear that's a good thing. That's a good way of dealing with general-purpose neural networks because basically you are doing the same thing that the early computer pioneers did when they separated storage and control. So, you have a controller that learns to manipulate the storage, the weights, the fast weights of another neural network, which is like the storage, which you can read and write, and then you can learn in the first network to create good reads and good writes of this fast storage. And then you have all this stuff that today is called test-time training. It was already there back then. And you basically have the separation of storage and control that was so successful in traditional computer science. Yeah.

对我来说，最令人兴奋的一点是，它已经指向了元学习，因为一旦你有了一个可以学习编程另一个网络的网络，你就可以把它闭合到自身，然后你可以构建一个通用的循环神经网络，学习为它自己的所有权重生成权重变化。它通过梯度下降来设置初始权重，从而学习任何在网络上运行并为网络自身产生权重变化的任意计算学习算法。这意味着你已经把所有层级——元学习、元元学习、元元元学习——都折叠进了一个单一网络，这个网络突然不再受限于人类设计的学习算法。不，它可以学习更好的学习算法。而这在当时就已经存在了。所以，2020 年代所谓的未归一化线性 Transformer，只是其中的一个小方面。更普遍的是现在大家都在谈论的递归自我改进，它的根源也在那里。

And to me, the most exciting aspect of that is that it already points towards metalearning because once you have a network that can learn to program another network, then you can also close it on itself, and then you can build a general-purpose recurrent neural network that learns to generate weight changes for all of its own weights. And it learns by gradient descent to set up its initial weights such that it learns any arbitrary computational learning algorithm that is running on the network itself and is creating weight changes there for the network itself. And then this already means that you have collapsed all the levels, you know, the metalearning and the meta-metalearning and the meta-meta-metalearning. You have collapsed all of that into one single network, which suddenly isn't stuck with some human-designed learning algorithm. No, it can learn better learning algorithms. And that was already present back then. So, the unnormalized linear transformer, as it has been called in the 2020s, is just a little aspect of that. And the more general thing is this recursive self-improvement that now everybody's talking about, which also has its roots back there.

在我们继续之前，给瑞士所有学生和应届毕业生说几句。大学教会你很多，但一些最宝贵的经验来自将知识付诸实践。这就是 ETH Juniors 的用武之地。通过真实项目、灵活的兼职角色，以及初创公司、中小企业和西门子、瑞银、诺华、埃森哲等公司的全职机会，在赚钱的同时获得实践经验。免费加入 ETH Juniors 人才库，不到 2 分钟即可完成，并匹配适合你技能的机会。前往 ETH Juniors 网站注册，在你真正需要之前就开始积累经验、人脉和职业生涯。现在，让我们回到对话。

Before we continue, a quick word for all students and recent graduates in Switzerland. University teaches you a lot, but some of the most valuable experience comes from actually putting that knowledge into practice. That's where ETH Juniors comes in. Earn money while gaining hands-on experience through real projects, flexible part-time roles, and full-time opportunities with startups, SMEs, and companies like Siemens, UBS, Novartis, and Accenture. Join the ETH Juniors talent pool for free in under 2 minutes and get matched with opportunities that fit your skills. Head to the ETH Juniors website to sign up and start building your experience, network, and career before you really need them. Now, let's get back to the conversation.

非常令人兴奋。你提到了这种发展，它催生了今天的大规模 Scaling（规模扩张）和缩放定律。你知道，训练系统需要如此多的数据，需要如此多的算力。当时，你已经解释了这个想法以及它导向何处。基于这个最初想法的起源，你认为当前的发展是否在正确的轨道上？这种不断扩张的 Scaling（规模扩张），更多算力、更多数据，如果看 LLM 的话，还有后训练等等。

Very exciting. And you mentioned this development which give rise to this massive scaling, scaling laws of today. You know, you need such amount of data to train the system. You need so much compute to do it. Back then, you just already explained this idea and where this idea led to. Do you think still based on these origins of the initial idea, do you think the current development are on the right track? This ever scaling about more compute, more data, more if we look at LLMs, post-training, and so forth.

这并非根本错误的路线，因为通过下载网络并学习从其他部分预测网络的部分内容，你会学到大量关于人类认为有趣的人类生成数据。为什么你会学到所有关于有趣数据的内容？因为网络上的所有数据都是由某个人上传的，只是因为某个时刻至少有一个家伙认为这很有趣。这就是这些大型语言模型和其他基础模型如何变得超级偏向人类思维。它们了解人类语言的一切，读过所有人类生成的书籍，并且可以续写人类生成的书籍和类似内容，因为它们已经学习了大量关于不同写作风格等等。所有这些都很酷。你可以用它做很多有趣的事情。当然，这对于构建通用 AI 和 AGI（通用人工智能）——现在所谓的人工通用智能——是完全不够的。因为为此你不仅需要一个预测机器。大型语言模型是一个预测机器，它学习根据前面的词预测下一个词。就像 1800 年的高斯和勒让德，他们使用一个简单的两层线性神经网络来预测下一个 token，其中 token 是通过望远镜对小行星的观测，然后你有一个时间序列，你有一堆嘈杂的测量值，你试图预测这颗小行星下次会出现在哪里，以及我需要把望远镜指向哪里才能再次找到它。所以，从先前步骤预测下一步，这是预测机器、线性神经网络最古老的应用。今天我们有更复杂的神经网络，但基本上它们做同样的事情。现在，除此之外，你还需要另一个神经网络或其他设备，一个决策者，它使用预测机器来提出导致目标的动作序列。所以你有一个想要实现的目标。现在你需要一个动作序列来实现目标。现在，如果你使用基础模型或大型语言模型来预测第一个网络的动作的后果，基本上建模世界以及世界如何运作，以及智能体的执行器如何工作等等。如果你有一个基础模型，它是一个预测机器，你可以称之为世界模型，然后第一个家伙可以使用各种技巧来利用第二个家伙——预测机器、世界模型——所学到的任何东西，以便它成为一个更好的问题解决者。所以，但然后你需要这些其他东西，你需要额外的决策者神经网络，它随着时间的推移学习使用第二个家伙，世界模型，进行规划。是的，所以，只要你没有这两个方面，你就离 AGI 很远。所以，当前的大型语言模型，它们只是下载网络上的内容，然后在数据上训练，它们不是决策者。但是，结合其他技术，你可以让它们成为决策者的助手。但是，然后你仍然有这两个方面的问题，即通用问题解决方面，它可以从预测机器、世界模型的存在中获益。

It's not a fundamentally wrong track because by downloading the web and learning to predict parts of the web from other parts, you learn a lot about all the human-generated data which humans consider interesting. Why do you learn everything about interesting data? Because all the data on the web was uploaded by some human only because at some point at least one guy thought this is interesting. And that's how these large language models and other foundation models become super biased towards human thinking. And they know everything about human language and they have read all human generated books and they can continue human generated books and similar stars and because they have learned so much about different stars writing and so on. And all of that is cool. You can do lots of interesting things with it. Of course, it's totally insufficient to build a general purpose AI and AGI and artificial general intelligence as it is called now. Because for that you need not only a predict machine. A large language model is a predict machine that learns to predict the next word given the previous words. Just like Gauss and Legendre in 1800, they used a simple linear neural network with two layers basically to predict the next token where the tokens were observations of asteroids through telescopes and then you have a time series and you have a bunch of noisy measurements and you try to predict where is this asteroid going to show up next time and where do I have to point my telescope to find it again. So, predicting the next step from previous steps, that's the oldest application of predict machines of linear neural networks. And today we have more complicated neural networks, but basically they do the same thing. Now, in addition to that, you need another neural network or some other device that is a decision maker that uses a prediction machine to come up with sequences of actions that lead to goals. So, you have some goal that you want to achieve. And now you need a sequence of actions to achieve the goal. Now, if you use a foundation model or large language model to predict the consequences of the first network of the actions of the first network to basically model the world and how the world works and how the actuators of the agent work and so on. If you have a foundation model, which is a prediction machine, you can call that a world model, and then the first guy can use various tricks to exploit whatever the second guy, the prediction machine, the world model has learned, so that it becomes a better problem solver. So, but then you need these other thing, this you need the additional decision maker neural network, which learns over time to use the second guy, the world model, for planning. Yeah, and so, as long as you don't have these two aspects, you are far from an AGI. So, the current large language models, they just download over the web and then are being trained on the data, and they are not decision makers. But, in conjunction with other techniques, you can make them helpers of decision makers. But, then you have still these two aspects of the problems of the general problem-solving aspect, which can profit from the existence of a prediction machine, of a world model.

出于好奇，我想 ChatGPT 时刻是 2022 年，如果我没记错的话。你自己每天使用 LLM 吗，就像大多数科技人士那样，还是你会说，用这个的时候可以更好？

Just out of curiosity, I think the chat GPT moment is 2022 if I'm mistaken. Do you use LLM yourself on let's say on a daily basis like probably most of the tech people out there or is it something that you say, when you use this could be better?

是的，对于某些常规任务，我会使用大型语言模型。我的学生比我用得更多，他们更清楚什么样的提示会带来好的结果等等。但当然，很明显，LLM 本身离我们真正想要实现的目标还很远，所以当我试图从我的角度在最有趣的研究前沿取得进展时。我不会在大型语言模型上浪费太多时间。

Yeah, so for a certain routine task using large language models. My students use them much more heavily than I do and they know much better which kind of prompt is going to lead to a good result and stuff like that. But of course it's clear that LLMs by themselves are far away from what we really want to achieve and so as I'm trying to make progress on the most interesting research front from my perspective. I'm not wasting too much time with large language models.

有道理。也许回到这两个组件的点上，一个是预测器，另一个等等。

Makes sense. Maybe just to coming back to this point of this two components that one the predictor and the other and so forth.

很多人，我是说几年前，你比我更了解这些，谈论了所有这些学习规则，Hebbian 学习之类的，并认为也许仅靠梯度学习永远无法成功。你觉得这是真的吗，还是说梯度学习不是问题？

Many people, I mean some years ago, you know this much better than I, talked about all these learning rules, Hebbian learning and things like that, and argued that maybe gradient learning would never do the trick just by using gradient learning. Do you think this is true, or would you say gradient learning is not the problem?

嗯，首先，Hebbian 学习实际上应该叫 Konorski 学习，因为有一位波兰科学家，Konorski，如果我发音正确的话，他在 Hebb 之前一年发表了现在所谓的 Hebbian 学习规则。那是什么？就是两个神经元同时活跃，然后它们之间的权重应该增强。所以，这是另一个常见的错误归因。

Well, first of all, Hebbian learning should actually be called Konorski learning because there was a Polish scientist, Konorski, if I correctly pronounce his name, and he published one year before Hebb. He published what is now called the Hebbian learning rule. What is that? It's just two neurons are active at the same time and then the weight between them should strengthen. So, that's another common misattribution.

非常正确。

Very much.

现在你的问题又是，嗯……

And now your question was again, um...

不，那是曾经有过相当多的争论，也许不是现在，而是几年前，你知道梯度下降或随机梯度下降本身不足以实现 AGI。我们需要有另一种学习算法，比如不同于反向传播，也许是一些可以在神经元之间学习的东西，就像人类做的那样，比如 Hebbian 学习。

No, that's there was quite of a debate maybe not right now but some years ago that you know gradient descent or stochastic gradient descent with by itself not be enough to achieve AGI. We need to have another maybe learn algorithm like different than back propagation and maybe something which can learn between neurons like uh this heavyen learn like humans do for instance.

是的。好吧，让我们看看没有梯度下降的深度学习，Evgeniy Golinko 在 1965 年使用过。那是逐层训练。所以，你有输入单元，然后有第一层，你有一些目标函数。也许你想预测下一个词。他实际上就是这么做的，预测下一个词。然后你有一个误差，也许误差太高，你知道，你无法通过让一些权重更强、一些更弱来将其降低到你希望的程度。所以你在上面再加一层。他的激活函数非常通用。它们是所谓的 Kolmogorov-Gabor 多项式。所以，你已经有乘法，今天所谓的门，乘法门。你在这个网络中拥有所有这些。所以你有超级非线性的隐藏层，然后你再加一层，再次尝试最小化误差，也许仍然不够好。然后你再加一层。然后许多这些隐藏单元在过程中被生成，结果证明是无关的、不重要的、多余的。他有一个单独的验证集来移除所有这些多余的单元。所以，他在修剪网络。同时，随着网络变深，它也变薄。权重更少，所以他有点把所有东西压缩成更少的权重。这就是他在 1971 年用来构建一个八层深度网络的方法，这在新千年仍被认为是深度网络。而且没有反向传播，只有逐步，但然后还有这种中间学习，你修剪掉不需要的东西。你得到一个更优雅、更压缩的网络。所以，也许这是一种比你现在所做的更合理的学习方式。我们使用的技术叫做反向传播，它是由一位芬兰人 Seppo Linnainmaa 在赫尔辛基首次发表，1970 年。五年后 Ivan Nechepurenko 做了这个，他有这个技术叫做自动……这是自动微分的反向模式，这正是人们所知道的反向传播。现在它被用来训练巨大的系统，你有神经网络，但还有其他可微分的单元和机制，这样你就可以高效地传播误差，就像莱布尼茨在 1676 年所做的链式法则。基本上，反向传播是实现链式法则以进行梯度下降的一种高效方式。所以，有趣的是，现代 AI 基于 17 世纪的数学。它不像物理学。物理学难得多。在物理学中，你需要 4 年才能理解该领域的基本开放问题。在我们的领域，你也许只需要半年就能理解基本的开放问题，因为我们的领域需要的数学比物理学简单得多。我跑题了。我知道你问了另一个问题。你问梯度下降学习是否过时，或者是否会过时？可能它在某种意义上是必须过时的。因为目前我们拥有的是这些巨大的网络，然后我们通过梯度下降、通过自动微分的反向模式、反向传播技术来训练它们。这意味着，做这些的计算机，它们经常乘以应用一个非常接近零的东西和另一个非常接近零的东西，这完全无关紧要。一切，每个神经元，每个连接始终活跃。或者至少始终被考虑。这不是这个高效的小大脑所做的。因为当我和你说话时，我的 1000 亿个神经元中的大多数是关闭的。只有少数是活跃的。这意味着我有一个不可微分类型的计算机。那个计算机的巨大优势是它只需要，你知道，最多 100 瓦。类似这样。相比 GPU 的 1500 瓦，而 GPU 比这个弱得多。所以，我们想朝这个方向走。这意味着我们必须在某个时候放弃至少经典的梯度下降，因为计算机在计算时变得更快、更热。它们想变得尽可能高效，它们想关闭，它们想学会在大多数时候关闭这些网络中的大多数隐藏单元。只有当需要它们时，你才想打开它们。所有这些都需要不同的技术，也许更像 Ivakhnenko 和 Lapa 在 1965 年所拥有的。

Yeah. Well, let's have a look at deep learning without gradient descent that Evgeniy Golinko used in 1965. It was layer by layer training. So, you have the input units and then there's a first layer and you have some objective function. Maybe you want to predict the next token. That's what he did actually, predict the next token. And then you have an error and maybe the error is too high, you know, you can't reduce it to the extent that you wish by making some of the weights stronger and some of them weaker. And so you add another layer on top of it. And his activation functions were really general. They were what's called Kolmogorov-Gabor polynomials. So, you have had multiplications already, what's called gates today, multiplicative gates. You had all of that in this network. And so you had super non-linear hidden layers and then you add another layer and you try again to minimize the error and maybe it's still not good enough. And then you add another layer. And then many of these hidden units are being generated in the process turned out to be irrelevant, non-important, superfluous. He has a separate validation set for removing all of these superfluous units. So, he's pruning the network. And at the same time as the network is getting deeper, it's also getting thinner. With fewer weights, and so he's kind of compressing everything into a fewer weights. And that's what he used then in 1971 to build a deep network with eight layers, which was still considered deep in the new millennium, you know. And there no back propagation, only step-by-step, but then also this intermediate learning where you prune away the stuff that you don't need. You get a more elegant, a more compressed network. So, maybe that's a more plausible way of learning than what you're currently doing. We are using a technique which is called back propagation, which was invented or which was published first in 1970 by a Finnish guy, by Seppo Linnainmaa in Helsinki. Five years after Ivan Nechepurenko did that, he had this technique which is called the automatic... It's the reverse mode of automatic differentiation, which is exactly what people know as back propagation. And that is now used to train huge systems where you have neural networks, but other differentiable units and machinery in there, so such that you efficiently propagate errors in the sense of what Leibniz did in 1676, the chain rule. Basically, back propagation is an efficient way of implementing the chain rule for doing gradient descent. So, it's fun to see that modern AI is based on mathematics of the 1600s. It's not like physics. Physics is much harder. In physics, you need 4 years to understand the basic open questions of the field. In our field, you need maybe just half a year to understand the basic open questions and because our field requires mathematics that is much simpler than what you need in physics. I'm deviating. I know you asked another thing. You asked is gradient descent learning obsolete or is it going to become obsolete? Probably it will have to become obsolete in some sense. Simply because what at the moment what we have, we have these huge networks and then we train them by gradient descent, by the reverse mode of automatic differentiation, back propagation technique. And this means that the computers who are doing that, they very often multiply applying something that is really close to zero by something else that is really close to zero and it's completely irrelevant. Everything, every neuron, every connection is active all the time. Or is at least being considered all the time. And this is not what this efficient little brain here does. Because as I'm talking to you, most of my 100 billion neurons are switched off. And just a few of them are active. Which means that I have a non-differentiable kind of computer in there. And the great advantage of that computer is that it needs only, you know, max 100 watts. Something like that. Compared to 1,500 watts for a GPU, which is much less powerful than this. So, we want to go in this direction. and this means that we will have to abandon at least the classical gradient descent at some point as computers are getting faster and hotter as they compute stuff. They want to become as efficient as possible, and they want to switch off, they want to learn to switch off most of the hidden units in these networks at most times. Only when they are needed you want to switch them on. And all that requires techniques that are different, that are maybe more like what Ivakhnenko and Lapa had in 1965, you know.

你提到了一些非常有趣的事情。所以，这种智能背后的数学相当简单，我们可以这么说。这种直觉，你知道，对我来说非常共鸣，因为当我第一次学习或研究时，我有同样的印象，对吧？我会假设智能就像人类能力的巅峰，如果我可以这样判断的话，然后我会期望这首先是复杂的，或者你知道，应该有复杂的细节等等。但我们现在为大多数这些算法所拥有的数学本身，你知道，相当直接，当然。它随着复杂性而增加。

You mentioned something very interesting. So, that the math behind this intelligence fairly simple, we could argue. And that intuition, you know, resonates very well to me because when I looked at it when I was learning or studying for the first time, I had the same impression, right? I would assume intelligence is like, you know, the peak of human capabilities if I may judge so, and then I would expect this to be complicated in the first place, or you know, should be intricate details and so forth. But the math itself that we have now for the most of those algorithms is, you know, fairly straightforward, of course. It increases with complexity.

所以，数学并不随着复杂性而增加。只有计算努力随着复杂性而增加，但数学仍然是相同的，你知道，它仍然是相同的小公式，仍然是实现该公式的相同小程序。

So, the math does not increase with complexity. Only the computational effort increases with complexity, but the math is still the same, you know, it's still the same little formula, and it's still the same little program that implements the formula.

只是网络突然变得大得多，有数万亿个权重，这意味着同一个微小的程序现在负责的权重变化比 1990 年左右的小网络要庞大得多。但基本原理仍然非常简单和优雅。所以，我认为重要的是要理解 AI 的基础超级简单。你不需要是天才就能理解它们。高中数学完全足以理解我们 AI 的基本原理以及我们 AI 背后的数学。而且很酷的是，尽管这些基本程序超级简单，只需几行代码，但如果你将它们应用于大量数据，那么就会产生非常复杂的东西，也就是一个或多或少智能的存在。

It's just that the networks are suddenly much bigger and have trillions of weights, which means that the same little tiny program is now responsible for weight changes that are much more massive than for the little networks that we had in 1990 or so. But the basic principles are still very simple and elegant. So, I think it's important to understand that the basics of AI are super simple. And you don't have to be a genius to understand them. High school math is totally sufficient to understand the basic principles of our AIs and of the math behind our AIs. And the cool thing is that although these basic programs are super simple and fit into a few lines of code, if you apply them to lots of data, then something really complex comes out, which is a more or less intelligent being.

我尊重地反驳一下，有一位神经科学家——我可能会念错他的名字，我想是 Jeff Hawkins。也许我错了。所以，你知道，我们使用非常简单的公式。但问题是，我们并不真正能够——也许你不同意——理解人脑或人类智能本身。有一位神经科学家说过类似的话：“如果我们不能先理解人类智能，我们就无法创造 AGI。”我的意思是，我们现在可以争论说，有非常简单的方式来表达我们今天的学习规则，但很难说这些代表了大脑的运作方式。所以，也许这种简单性相当人为。

To push back on this respectfully, there is a neuroscientist—I might now butcher his name, so I think it was Jeff Hawkins. Maybe I'm wrong about that. And so, you know, we use very simple formulas. So, but the question is like, but we didn't—we're not really able, maybe you would disagree with that—to understand the human brain or the human intelligence itself. And there was a statement by a neuroscientist like, "We'll not be able to create AGI, let's say, if we not manage to understand human intelligence first." I mean, we can argue now that there are very simple ways to express the learning rules of us today, but it's hard to make the case that those represent what our brain does. And so, maybe the simplicity is rather artificial.

是的。所以，如果大脑的运作在原则上很复杂，我会非常惊讶。因为，我们从我们的 AI 中知道，它们在许多方面常常超越人类——不是所有方面，但很多方面。它们超级简单，你知道，非常简单的技术、程序，只是高效地计算链式法则。它们有时能产生超人的智能。所以，既然某些类型的解决问题极其不复杂且简单，为什么我们要相信人类擅长而机器还不擅长的其他事情不会以类似的方式简单呢？我确信它们是简单的。即使是我之前提到的更复杂的事情，你知道，两个网络相互作用，一个是世界模型，另一个探索那个世界模型中的信息来规划未来。即使这些也是非常简单的概念。尽管当你扩大规模时，它们会导致真正惊人的结果。你知道，就像那个小程序——计算分形曼德布罗集图像的五行程式。它看起来超级复杂，你看细节，可以放大，整个东西看起来像小细节看起来像整体，它看起来超级复杂，但不，它不是。它非常简单，因为有一个非常短的程序来计算它。我相信核心机器学习原理也是如此。最令人信服的机器学习原理，最成功的那些，它们都是简单的。你知道，就像深度残差学习，它是由我在浦项的杰出学生在 1991 年开创的。它说，看看你在梯度下降和深度网络中遇到的问题。在传统的深度网络中，梯度随着网络越深而变得越来越小。所以 Zep 看了细节，然后他看到，“哦，有一个简单的修复方法。”你只需要一个单元，它有一个到自身的 1.0 连接。它有一个线性激活函数，几乎是你所能想象的最简单的神经网络类型。它是一个漏积分器，所以你发送的一切，它只是保持那个加上在下一个时间步进来的新东西。然后事实证明，如果你有一个像那样超级简单的系统，那么，当你通过由许多这样的单元组成的系统进行反向传播时，误差不会消失，因为它们沿着这些 1.0 线循环，所以它们乘以 1.0，然后它们通过线性激活函数，导数是 1.0。所以，你将 1.0 乘以 1.0，你可以这样做一百万次而不改变任何东西，这意味着突然你可以有非常非常深的网络。所以，一个非常简单但非常强大的见解。就像几乎所有事情——几乎所有好的事情都是简单的事情。反过来不成立。

Yeah. So, I would be really surprised if what our brain does is complicated in principle. Because, we know from our AIs, which are often superhuman in many ways—not in every way, but in many ways. They are super simple, you know, very simple techniques, programs that just compute the chain rule efficiently. They are able to come up with sometimes superhuman intelligence. So, given that certain types of problem solving are extremely uncomplex and simple, why should we believe that the other things that humans are good at which machines are not yet good at aren't simple in a similar way? I'm convinced they are. And even the more complicated things that I mentioned before, you know, two networks interacting with each other, one being a model of the world, and the other one exploring the information in that model of the world to plan the future. Even those are really simple concepts. Although, as you scale them up, they lead to really amazing results. You know, just like the little program—the five-line program that computes the image of a fractal Mandelbrot set. And it looks super complicated, and you look at the details, you can zoom in, and the whole thing looks like the little detail looks like the whole, and it seems super complex, but no, it isn't. It's very simple because there's a very short program that computes it. And the same thing is, I believe, true for the core machine learning principles. And the most convincing machine learning principles, the most successful ones, they are all simple. You know, like deep residual learning which was pioneered by my brilliant students at Pohang there, 1991. Which says looking at the problems that you get with gradient descent and deep networks. And in traditional deep networks, the gradient gets smaller and smaller the deeper the network is. And so Zep looked at the details of that and then he saw, "Oh, there's a simple way of fixing it." You just need a unit that has a 1.0 connection to itself. And it has a linear activation function, pretty much the simplest type of neural network that you can imagine. It's a leaky integrator, and so everything you send in, it just keeps that plus adding the new thing coming in at the next time step. And then it turns out that if you have a super simple system like that, then, as you're performing back propagation through a system consisting of many units like that, the errors do not vanish because they circle along these 1.0 lines and so they are multiplied by 1.0 and then they move through the linear activation function and the derivative is 1.0. So, you multiply 1.0 by 1.0 and you can do that a million times without changing anything, which means suddenly you can have really, really deep networks. So, a very simple but very powerful insight. As almost all things—almost all good things are simple things. The reverse is not true.

好的，你在为简单性提出强有力的理由。我会，你知道，我，你知道，至少今天在这里对我来说有道理。但然后，你知道，我认为人类真正擅长的是——我会说——找到简单的规则来解释世界或物理。我会把智能加进去。那么我们还缺少什么？缺少什么？因为我——你基本上在这里描述了两个层次。一个是我会说的微观层次。所以如何更新权重，你知道，学习规则和反向传播等等。另一个是更宏观的层次。所以你知道你提到了世界模型，你知道另一个帮助决策的。要到达 AGI，还缺少哪个大拼图？

Okay, you're making a strong case for simplicity. I would, you know, I, you know, it makes sense to me at least here today. But then, you know, I think what humans are really good at I would argue finding simple rules explaining the world or physics. I would add intelligence to it. What are we missing then? What is missing? I because you we basically describing two levels here. One is the I would say the micro level. So how to update weights and you know learning rules and back propagation and so forth. And the other are more of the macro level. So how you know you mentioned the model of the world and you know the other one which helps with decision making. Where is still the big puzzle piece missing to get to AGI?

所以在 2000 年代初期，我们有一个项目——那是我获得的一项 SNF 资助，用于 Marcus Hutter 在这项资助上的工作，该工作是关于构建一个最优的、通用的、数学上最优的 AI，适用于所有可计算环境。所以这不仅仅是一个 AGI。不，它强大得多。它对所有它能描述的环境集合是最优的，在它是可计算的意义上，在你可以写一个程序来计算那个环境，或者至少计算这个环境根据其对一个生活在该环境中的智能体的行动做出反应的概率分布的意义上。所以这一切都超级复杂，但从数学角度来看，它的解决方案真的非常简单。所以有一种数学上最优的通用 AI，它有一个世界模型，而世界模型是一个预测机器，它只是学习预测行动者行动的后果。

So in the early 2000s we had a project—it was an SNF grant that I got there for the work that Marcus Hutter did on this grant which was about building an optimal universal mathematically optimal AI that works for all computable environments. So that's not just an AGI. No, it's much more powerful. It's something that is optimal for the set of all environments that it can describe in the sense that it's computable in the sense that you can write a program that computes that environment or at least computes the probability distributions according to which this environment reacts to the actions of an agent living in this environment. So it's all super complicated, but it turns out that the solution of that from a mathematical perspective is really really simple. And so there's a certain kind of mathematically optimal universal AI which has a world model and the world model is a predictive machine which just learns to predict the consequences of the actions of the actor.

本质上在所有可能的世界中。

In all possible worlds essentially.

在所有可能的世界中，它使用一种被称为 Solomonoff 先验的东西，即通用先验，可以追溯到 60 年代，然后它只是对所有可计算先验的求和。每个先验获得一点权重，使得它们的总和不超过 1，这意味着你有一个半概率分布，一个半测度。然后不幸的是，这就是它计算上不可行的原因，但在数学上超级优雅。

In all possible worlds and it's using something which is known as the Solomonoff prior, the universal prior which goes back to the 60s and where then it's just a sum over all computable priors. Each of them gets a little weight such that the sum of them is not bigger than one which means that you have a semi probability distribution, a semi measure. And then you have unfortunately and that's what makes it computationally infeasible but mathematically is super elegant.

然后你对所有这些可能的分布求和，宇宙及其响应可能就是从这些分布中采样出来的，但你并不知道你的宇宙实际遵循的真实分布是什么。尽管如此，如果它是可计算的，那么真实分布就在这个混合之中，而且结果表明，根据 Solomonoff 先验做出的预测会迅速收敛到你若知道宇宙真实先验时会得到的预测。这是一个可以追溯到 Solomonoff 1978 年左右的结果。

Then you have a sum over all these possible distributions from which the universe and its responses might be sampled, but you don't know what the true distribution is according to which your universe is working. Nevertheless, if it is computable, then the true distribution is part of the mix, and it turns out that the predictions according to the Solomonoff prior rapidly converge against the predictions that you would get if you knew the true prior of the universe. This is a result that goes back to Solomonoff, 1978 or something.

但后来 Marcus——Marcus Hutter，我当时带的博士后——他在此基础上把它和实际的选择器结合起来，目标就是最大化奖励。任何计算问题都可以表述为奖励最大化问题，然后他证明了，在这样一个未知世界中行动的最佳方式就是使用 Solomonoff 先验，并对直到你生命终结为止的所有可能动作序列做期望最大化。也许是到目前为止你寿命的两倍。现在你 50 岁，你向前看到 100 岁，然后你考察所有你可能执行的动作序列，并执行那个能带来最高预测奖励的动作序列的第一个动作。结果表明这在数学意义上是——在某种数学意义上是——最优的。它在计算上不可行，因为你谈的是可数无穷多个分布。但从数学角度看，我们有一个非常非常简单的系统，它是最优的。这已经相当令人印象深刻了。

But then Marcus, in addition to that — Marcus Hutter, my postdoc back then — he combined that with the actual selector, and there the goal is to maximize reward. Any computational problem can be formulated as a reward maximization problem, and then he showed that the best way of acting in an unknown world like that is to use the Solomonoff prior and do an expectation max over all the possible action sequences until the end of your life. So maybe twice your current lifespan so far. Now you're 50 years old, you look ahead until you are 100 years, and then you look at all possible action sequences that you could execute, and you execute the first action of the action sequence that leads to the highest predictive reward. And it turns out this is optimal in a mathematical sense. So, in a certain mathematical sense. It's not computationally feasible because you are talking about a countably infinite number of distributions. But from a mathematical perspective, we have a very, very simple system that's optimal. That's already pretty impressive.

与此同时，我还有 Skudl 机器，它以另一种方式最优，但同样是数学上最优的。所以我们在原则上已经知道什么是最优的问题求解器，只不过整个东西还不实用。现在的问题是，你如何用我们更传统的技术让它变得实用，比如人工神经网络、循环网络、类 Transformer 网络、未归一化的线性类 Transformer 网络等等？我们怎么用这些东西？还有像梯度下降这样的有限搜索技术——我们怎么用它们尽可能接近这些最优方法？所以这里有几个开放问题，这也是为什么我们在两端同时工作：不仅是数学上最优但有点不可行的东西，还有非常实用的基于神经网络的东西。我认为最终我们会看到它们如何收敛。

And at the same time, I have the Skudl machine, which is optimal in a different way, but also mathematically optimal. So we already know in principle what an optimal problem solver is, except that the whole thing is not yet practical. And now the question is, how do you make it practical using our more traditional, conventional techniques such as artificial neural networks, recurrent networks, transformer-like networks, unnormalized linear transformer-like networks, or whatever? How do we use that stuff? And limited search techniques such as gradient descent — how do we use that to get as close as we can to these optimal methods? So there are a couple of open questions here, and that's the reason why we are working on both ends: not only the mathematically optimal but kind of infeasible stuff, but also the very practical neural network-based stuff. I think in the end we are going to see how they converge.

我是说，这听起来非常令人兴奋。我觉得你在这里提到的这项工作——我非常好奇它会带我们走向何方，也许甚至沿着这个方向走。我觉得你基本上——我不想替你把话说了。你说过你进入这个领域的原因：你是爱因斯坦的粉丝，你曾想过做物理，但后来你有了这个想法——你怎么能创造，也许有没有一种方法能创造出这个人工科学家，让它替你做科学，并像爱因斯坦那样发现所有这些伟大的发现。你沿着你刚刚铺就的这条路径走了多远？我们离实现这个伟大愿景还有多远？

I mean, it sounds very exciting. I think also this work that you mentioned here — I'm very curious to see where this takes us, and maybe going even in this direction. I think you basically — I don't want to put words in your mouth. You said the reason why you got into the field: you were a fan of Einstein and you thought about doing physics, but then you got this idea — how could you create maybe, would there be a way to create this artificial scientist which does science for you and finds all these great findings like Einstein did. How far are you working along this path that you just paved? How far are we away to get to this great vision?

我认为我们离得并不远，某种意义上我觉得我可能活着就能看到它。我认为我们理解基本原理——就是我之前提到的：行动者和决策者利用世界模型，在给定某个问题后，凭借对数百个先前问题的经验，快速找到更好的动作。然后还有这种好奇心驱动，即人工好奇心，其中控制器有动力去设计实验、动作序列，从而产生世界模型可以用来变得更好的数据，这些数据会让世界模型感到意外，但随后它很快看到规律，就能学会这个规律，然后它就变得无聊了，控制器就必须以创造性的方式生成新的额外实验，以进一步扩展整个系统的视野。

I think we are not very far away, in the sense that I think I might live to see it. I think we understand the basic principles — what I mentioned before: the actor and decision maker who is using the world model to quickly find better actions given a certain problem, after having had experience with hundreds of previous problems. And then having this curiosity drive, the artificial curiosity, where you have an incentive for the controller to come up with experiments, action sequences that lead to data that the world model can use to become a better world model, which surprises the world model, but then quickly it sees the regularity and it can learn the regularity, and then it becomes boring, and the controller will have to generate new additional experiments in a creative fashion to further expand the horizon of the total system.

所以我认为原则上我们理解如何构建它。我们有这样的系统。它们只是不像 ChatGPT 在其远为有限的领域里那样有说服力，在那个领域里它只关乎预测。那么要多久？从人类的角度，从我个人的角度，我希望我能在未来几年、几十年内看到它。我明天可能就心脏病发作，但也许我能再活几十年，我应该能够看到它。

So I think in principle we understand how to build that. We have systems like that. They are just not as convincing as ChatGPT is in its much more limited domain, where it's only about prediction. So how long will it take? From a human perspective, from my personal perspective, I hope I will see it in the next few years and decades. I might have a heart attack tomorrow, but maybe I will live a few additional decades and I should be able to see it.

目前令人担忧、完全不好用的东西，真的是物理世界中的 AI。物理世界中的 AI 根本不行。屏幕背后的 AI 运作得非常好，你有屏幕背后的 AI，计算机里某处的 0 和 1，它们通过了图灵测试。但这只说明图灵测试是衡量智能的一种糟糕方式，因为我总是用水管工作为例子。没有哪个 AI 能做出水管工用双手能做的事，也没有哪个 AI 能做出电工或一个小男孩用足球能做的事，诸如此类。现在，它会到来的，但现实世界的进展比我们在每瑞士法郎算力上看到的进展慢得多。每 5 年，同样价格下计算机快 10 倍。所以 30 年就是一百万倍。然而，当前的机器人并不比 30 年前的机器人好一百万倍，因为 30 年前就已经有会走路的机器人了。它们只是必须小心让重心保持在脚上方之类的事情。但今天的机器人也许比那时的机器人好三四倍，而不是好一百万倍。所以硬件和机械的演化远不如软件和屏幕背后的 AI 那样迅速。

The thing that currently is worrisome, that doesn't work well at all, is really the AI in the physical world. So the AI in the physical world doesn't work at all. The AI behind the screen works really nicely, and you have AIs behind the screen, zeros and ones somewhere in a computer, and they pass the Turing test. But that just means that the Turing test is a bad way of measuring intelligence, because I'm always using the plumber as an example. There's no AI that can do what a plumber can do with his hands, and there's no AI that can do what an electrician or a little boy can do with a football and all of that. Now, it will come, but then the real world progress is much slower than the progress that we have seen in compute per Swiss franc. Every 5 years, 10 times faster computers for the same price. So in 30 years, a factor of a million. However, the current robots are not a million times better than the robots that we had 30 years ago, because 30 years ago there already were walking robots. They just had to be careful to keep the center of gravity above the feet and stuff like that. But today's robots are maybe three or four times better than the robots back then, not a million times better. So hardware and machinery is evolving much less rapidly than the software and the AI behind the screen.

所以你是把这当作一个必要条件来推导吗？AI 必须与真实世界互动才能创造更好的世界模型？你是这个意思吗？

So are you deriving this as a requirement? So AI has to interact with the real world to create a better world model? Is that what you're saying?

嗯，不。甚至不止于此。所以从 1987 年以来，我一直在研究自我改进、递归自我改进的 AI，你改进学习算法，同时也改进学习算法改进自身的方式，等等。但这件事的终点只有当你身处物理世界、AI 也在改进它自己的计算机，以及它从地下高效提取原材料以建造更多更快计算机的方式等等时，才能实现。所以只有当 AI 掌握了真实世界来改进自身，而不仅仅是在屏幕背后、在软件层面。那很容易。

Well, no. Even beyond that. So since 1987, I've been working on self-improving, recursively self-improving AIs, where you improve the learning algorithm, but also the way the learning algorithm improves itself and so on. But the end point of that is achieved only once you are out there in the physical world and the AI is also improving its own computers and the way it extracts raw materials from the ground in efficient ways to build more and faster computers and all of that. So only when the AI has mastery of the real world to improve itself, not only behind the screen in terms of software. That's easy.

我们已经知道怎么做了，但在现实世界里，用机器人去造更好的机器人，让它们学会为各种基础设施建造更好的机械，它们需要建立起不断扩张的 AI 领域和机器人文明。只有到那时，整件事才真正令人震撼、令人信服。否则，它只是屏幕后面的一个小东西，一个花哨的文本编辑器。它不是真正自我指涉、自我改进的物质。

We already know how to do that, but in the real world, with robots that build better robots, that learn how to build better machinery for all kinds of infrastructure, they need to build expanding AI spheres and robot civilizations. Only then have you achieved the point where the whole thing is truly impressive and convincing. Otherwise, it's just a little thing behind the screen, a fancy text editor. It's not truly self-referential, self-improving matter.

我完全明白了。我是说，如果，你知道，我可以放一个小机器人到世界上，让它自我改进，然后你知道——

Got you completely. I mean, if you know, I could let out a small robot into the world which makes itself better and you know—

或者一个机器社会。所以，很可能不是单个机器人。不，会是一大群机器人和其他机械。但你基本上只需要一种机器人，它足够聪明，能学会操作所有其他机器。一旦你有了它，你就拥有了一种新的生命。几个世纪以来，人们一直在谈论自我复制的机械，你知道吧？但现在我们有了一个突破口。突然间我们发现，我们只需要一种会学习的机器人，它的手能和我这双手相媲美。而今天还没有那样的手。没有这样的人造手。然后，一旦你有了这样一台机器，能操作所有现有的其他机器，比如卡车，比如你从原始地面提取矿石所需的机器，把原矿从地里挖出来，运到工厂，在那里材料被精炼、转化成微芯片和机器人肢体等等。一旦你有了它，那么整件事突然间就成了终极的 Scaling（规模扩张）机器，因为它能自我复制。它能自我生长。当然，它不只是自我复制，它还会自我改进，因为他们在机器学习和软件中已经理解得很透彻的所有概念，自我改进之类的，都会被转化到现实世界、转化到物理世界里，然后你就会拥有真正自我改进的机械，那才是真正令人兴奋的开始，那才是一个新时代的黎明。

Or a society of machines. So, probably it's going to be not one single robot. No, it's going to be a whole bunch of robots and other machinery. But all you need is basically one type of robot that is smart enough to learn to operate all the other machines. Once you have that, you have a new kind of life. For centuries, people have talked about self-replicating machinery, you know? But now we have an opening. And suddenly we see we just need one kind of learning robot with hands that are comparable to what I have here. And there are no hands like that today. No artificial hands. And then once you have a machine that can operate all the existing additional machines, like the trucks and the machines that you need to extract the ore from the raw ground, the raw ore from the ground, and ship it to the factories where the material is refined and transformed into microchips and robot limbs and everything. Once you have that, then the whole thing suddenly is the ultimate scaling machine because it can multiply itself. It can grow by itself. And then of course, it's not only going to self-replicate, it's also going to self-improve because all the concepts that they already understand well in machine learning or software, self-improvement and stuff like that, is going to be translated into the real world and into the physical world, and then you will have truly self-improving machinery, and that's where the real excitement starts, and that's when a new age will dawn.

我理解这会非常令人兴奋。我也理解，你知道，如果有人能说服别人说这是必要的。不过，我在这里还是有点没搞明白——为什么这不能在计算机上完成呢？为什么，你知道，你可以创建一个非常逼真的模拟，对吧？你几乎可以学到任何东西。也许你无法创建世界的模拟，因为，你知道，你需要几乎无限的算力，但理论上，难道不可能只是——物理世界到底带来了什么？这就是我在这里没搞明白的地方。是不是有——

I understand that this would be very exciting. I would understand that, you know, if one can convince somebody that this is needed. However, what I'm still sort of missing here—why could this not be done on a computer? Why, you know, you could create a very plausible simulation, right? You could learn anything really. Maybe you cannot create a simulation of the world because, you know, you need very unlimited compute, but in theory would it not be possible just to—What does the physical world bring into the game? That's what I'm missing here to understand. Is there—

几乎整个世界都不在你屏幕后面。现实世界中几乎所有的物质都不在你的计算机里。它在外面的太阳系其他地方，在森林里，在工厂里，在所有环绕着我们、对我们的生存至关重要的物理事物里，还有更远处的月球、水星等等，各种你可以用来建造更多、更大 AI 的材料。而屏幕后面、数据中心里、GPU 里的那一点点，根本不算什么。和其余的一切相比，它什么都不是。所以，如果你想要一个真正令人信服的系统，它必须是一个掌握整条链条的系统，从基本的物理材料，到用这些基本材料制造小型复杂东西的工厂，然后形成微芯片、机器人和额外的计算机，还有改进计算机的实验室，以及所有这一切。所以，我理解今天的情况，因为很多人花大量时间在智能手机后面，每天好几个小时，他们花时间在台式电脑前，每天好几个小时，他们以为那就是世界，差不多就是，那是我的世界。但当然，几乎整个世界都不在你屏幕后面。不在你智能手机后面。它只是你可以定义为有趣的东西，因为那些玩智能手机和社交网络的人找到了把你粘在这部智能手机上的办法。但我们的星辰和现实世界远比这个小东西要多得多，对吧？现实世界仅凭其质量，就几乎无限地多于那些被用来建造计算机的一点点质量碎片。

Almost all of the world is not behind your screen. Almost all of the matter in the real world is not in your computer. It's out there in the rest of the solar system and in the woods and in the factories and all this physical stuff that is surrounding us and which is essential for our survival, and stuff that's further out in the moon, on Mercury and so on, all kinds of material that you can use to build more and bigger AIs. And the little bit behind the screen and in the data centers and the GPUs, that's nothing. It's nothing compared to all the rest. So, if you want to have a truly convincing system, it must be a system that masters the entire chain from the basic physical materials to the factories that create small complicated stuff out of this basic material and then form microchips and robots and additional computers, and labs where the computers are being improved and all this stuff. So, I understand that today, because many people spend so much time behind their smartphones, hours per day, and they spend time in front of their desktop computers, hours per day, and they think that is the world, more or less, and that's my world. But, of course, almost all of the world is not behind your screen. It's not behind your smartphone. It's just something that you can define interesting because the guys who are playing the smartphones and the social networks found ways of gluing you to this smartphone. But our stars and real world is so much more than the little thing, yeah? Just the sheer mass of the real world is almost infinitely more than these little mass fragments that are being used to build computers.

所以，换句话说，你在计算机里找到的只是我们对世界的粗糙表征。它不是真实世界，我们需要让机器走出去，与真实世界互动，才能达到这个 AGI（通用人工智能）。

So, in other words, it's our crude representation of the world that you find in computers. It's not the real world, and we need the machines to get out there interacting with the real world to get to this AGI.

不掌握真实世界，就没有 AGI（通用人工智能）。是的。

No AGI without mastery of the real world. Yeah.

你不能让 AGI（通用人工智能）待在屏幕后面，然后不允许它走出去做真正有趣的事，也就是改变真实世界，建造更多被认为有趣的东西。

You can't have AGI behind the screen and then not allow it to move out there and do the real interesting stuff, which is change the real world and build more of whatever is considered interesting.

你在这场对话中提到这些技术是如何运作的，你知道，其中许多需要大量资源，这也意味着今天需要大量投资，因为许多前沿模型公司仍然没有盈利。我想——当然对我来说，问题就在这里，我想让你评估一下，你知道，你怎么看这件事，当然也要看看欧洲本身，你知道，直到今天——尽管所有这些公司都位于硅谷，对吧？或者说其中许多，不是全部，那样说就太牵强了。很多人说欧洲正在加快步伐，我同意这一点。你认为目前采用的这种模式会成功吗？你认为那些——或者用非常直白的话说，你认为这一切都是泡沫吗？当前的发展。

You mentioned in this conversation how these technologies also work and, you know, many of those require large amounts of resources, and this implies also today large amounts of investment because many of those frontier AI companies are still not profitable. And I would like—and of course there's also the look for me, the questions are here, and I want you to assess, you know, what you think about this, and of course looking at Europe per se, which, you know, till this day—though all of these companies are located in Silicon Valley, right? Or many of those, not all, that would be far-fetched. Many people say that Europe is picking up the pace, which I would agree with. Do you think this model, which is currently employed, will succeed? Do you think those—or putting it in very blunt words, do you think this is all a bubble? The current development.

所以，目前我认为我们有一个泡沫，意思是只要计算机还不够便宜，人们就试图通过砸大量钱来弥补，你知道。当然，如果你等 5 年，你可以用 10% 的价格得到同样的东西。如果你等 10 年，你可以用 1% 的价格得到同样的东西。如果你不着急，那为什么不干脆等几年呢，你知道？一切都会变得越来越便宜，因为所有开源模型也会变得越来越好。而且，你知道，6 个月内它们就会和最好的私有模型、私有公司最好的模型一样好。而且它们会越来越便宜。30 年后，你可以用百万分之一的价格得到同样的东西。因为这个趋势没有中断。每 5 年计算机就便宜 10 倍。

So, at the moment I think we have a bubble in the sense that people, as long as the computer isn't cheap enough, they are trying to compensate by throwing a lot of money at it, you know. Of course, if you wait for 5 years, you can have the same thing for 10% of the price. If you wait for 10 years, you can have the same thing for 1% of the price. And if you are not in a hurry, then why not wait a couple of years, you know? And everything is going to get cheaper and cheaper and cheaper because all the open source models, they are going to become better and better, too. And they, you know, in 6 months they will be as good as the best private model, the best models of private companies. And they're going to get cheaper and cheaper. And in 30 years, you are going to get the same thing for 1 millionth of the price. Because the trend is not breaking. Every 5 years computers get 10 times cheaper.

目前我们看到一个有趣的情况：在计算机还不够便宜的时候，人们愿意在算力上花费数千亿美元，而这些算力可能很快就会过时。因为计算机正变得越来越小、越来越节能。这意味着在不久的将来，今天云端大语言模型能做的所有事情，你都可以在本地桌面上完成，在一台小机器上，甚至可能小到智能手机。越来越小，越来越便宜。最重要的是，你不需要连接到网络。一切都在本地完成。你不必担心网络连接会在你的计算设备上安装各种恶意软件。所以这就像智能手机一样。今天的智能手机与 80 年代的移动电话相比，便宜得令人难以置信。80 年代的移动电话是笨重的大块头，你可以在汽车里找到它们。那种汽车的主人可以通过卫星与其他类似汽车的主人通话。而今天，世界上几乎每个人，无论是在发展中国家还是其他地方，都拥有一部智能手机，比 80 年代那些人在车里用的要好得多。所以仅仅几十年间，它就变得超级便宜，功能也强大得多。我们在 AI 上也将看到同样的事情。所以 AI 会变得越来越便宜。今天在 GPU 上投资 1 万亿美元的人，他们将损失其中的很大一部分，也许在短短 5 年内就会损失 9000 亿美元。因为现在每个人都在尝试实施的商业模式，不足以收回这些损失。所以我们正处于一个暂时的泡沫中。这不会是世界末日。它只是意味着你遇到了这种过度投资，它会正常化，股市会从中吸取教训，也许其他专注于不同领域的公司会成为真正的赢家。很难预测谁会成为那种情况下的赢家。但近几十年的大赢家，曾经是灵活的软件公司，一个小型开发团队，也许 10 个人左右，开发出某个操作系统的新版本，然后推广给数十亿使用自己电脑运行该操作系统和自己智能手机的人。这些公司突然变得像公用事业公司，因为现在他们在建设数据中心和云，他们必须投资核电站、燃气轮机以及各种能源公司，只是为了给整个东西供电。他们变得像公用事业公司，像电力公司。同时，没有护城河保护他们免受那些不断涌现的更便宜的竞争者的攻击，这些竞争者做同样的事情，因为每个人都在用同样的水做饭，他们和大实验室做同样的事情。他们稍微落后一点，但差距不大。在这个意义上，定价面临持续压力，这意味着很多一直在大力投资资本支出的人将会损失很多钱。

At the moment we see this interesting situation where people, while the computer is not cheap enough yet, are willing to spend thousands of billions of dollars on compute in a way that is probably going to be obsolete soon. Because computers are getting smaller and smaller and more energy efficient. This means that in the not so distant future, you will have everything that a large language model in the cloud can do today locally on your desktop, in a small machine, maybe even as small as a smartphone. Getting smaller and less expensive. And most importantly, you won't have to connect it to the web. It will all be done locally. You don't have to fear that connections to the web will install all kinds of malware on your computational device. So it's going to be just like with smartphones. Today smartphones are incredibly cheap compared to mobile phones of the '80s. Mobile phones of the '80s were big chunks of stuff, and you could find them in cars. And the owner of a car like that could talk via satellite to other owners of similar cars. And today, almost every person in the world, in developing countries or elsewhere, has a smartphone that is much better than what these guys in the '80s had in their cars. So within just a few decades, it became super cheap and so much more powerful. And the same thing we are going to see with AI. So AI is going to get really cheap all the time. Guys who are investing today 1,000 billion dollars into GPUs, they are going to lose a lot of that, maybe 900 billion US dollars within only 5 years. Because the business models that everybody is trying to implement now are not good enough to recuperate these losses. So we are in a momentary bubble. It is not going to be the end of the world. It just means you have one of these excesses, and it will normalize out and the stock market will learn from that and maybe other companies who are focusing on different things will be the real winners. And it's hard to predict who exactly is going to be the winner of that situation. But the big winners of recent decades, who used to be nimble software companies where a small team of developers, maybe 10 guys or so, developed a new version of some operating system and rolled it out to billions of people who used their own computers for running that operating system and their own smartphones. These companies suddenly have become like utilities because now they are building data centers and cloud, and they have to invest into nuclear power plants and gas turbines and all kinds of energy companies just to power the whole thing. They're becoming like utilities, like electricity companies. And at the same time, there's no moat protecting them from the attacks from the cheaper guys who come up all the time and do the same thing, because everybody is cooking with the same water and they do the same thing as the big labs. They're a little bit behind but not so much. And in this sense you have constant pressure on the pricing, which means that lots of guys who have been heavily investing into CapEx are going to lose a lot of money.

这非常有趣，因为我正试图从中得出一些潜在的指导方针或建议、推荐，因为德国这里有一个大型倡议，试图向新的 AI 初创公司投资数百万。如今为了保持竞争力，你需要大量算力，甚至要在顶级会议上发表成果，你也必须运行大型实验，大量数据。所以即使只是为了在顶级会议上发表，你也需要大量算力。所以，如果我们不投资、不创造机会，我们该怎么办？因为在这里补充一点，欧洲在 GPU 数量上，与美国和中国相比，要少得多。

I mean, this is very interesting because I'm trying to derive my potential guidelines out of these or suggestions, recommendations because there has been this large initiative here in Germany, trying to invest large amounts of millions into new AI startups right now. And in order to be competitive these days, you need large amounts of compute, even to publish your results in the top conference, you have to run large experiments, a lot of data. So you need a lot of compute even just to publish in the top conferences. So in a way, what should we do if we do not invest in and create, let's say, opportunities? Because just to add on here in that sense, also Europe in terms of the number of GPUs in comparison, say, to the US and China, is much much less.

是的，是的，是的。所以你必须投资一点，只是为了在桌边有个座位。但我认为你不想成为投资最多的人。你想稍微看看最大的投资者在做什么。你要考虑到他们会损失很多钱。同时，当你努力保住桌边的座位时，再保留 5 年，那时你会以同样的价格获得 10 倍的东西。我相信这一切都会实现。当然，有一个经常被重复的论点，也许如果我们现在投资很多钱，我们就能第一个拥有真正的 AGI。然后所有这些投资看起来都不算什么，因为我们的 AI 将接管世界股市，它将比我们投资的那点东西有价值得多。但我认为目前我们几乎没有迹象表明某一个人会取得这样的成功，而不会被其他人几乎立即重复，从而真正侵蚀第一个人的利润。所以如果你不相信当前以大型语言模型为导向的投资会带来超级 AI——我确信它们不会，因为对于 ASI 或 AGI 来说还不够——那么你可能想站在那些稍微等待、更谨慎的人一边。你想在桌边有个座位，但你不想成为那些万亿美元投资的领导者。

Yeah, yeah, yeah. So you have to invest a little bit to just have a seat at the table. But I think you don't want to be the guy who invests most of it. You want to check a little bit what the biggest investors are doing. You want to take into account that they are going to lose a lot of money. And at the same time, as you are trying to keep your seat at the table, keep it for another 5 years when you will get 10 times as much for the same price. All of that is going to pan out, I believe. Of course, there is one argument which has been frequently repeated, which is maybe if we now invest a lot of money we can be the first to have a true AGI. And then all of this investment will look like nothing because our AI is going to take over the world stock market and it will be much more valuable than the little thing that we invested. But I think at the moment we see little indication that one single guy will have a success like that that is not almost immediately going to be repeated by other guys in a way that really eats the margins of the first guys. So if you don't believe that the current large language model oriented investments are going to lead to the super AI — I'm sure they won't because it's not enough for ASI or AGI — then you probably want to be on the side of those who wait a little bit and are a little bit more careful. You want to have a seat at the table but you don't want to be the leader of the also trillion-dollar investments.

所以，你认为这个领域仍将保持高度竞争，因为总有新玩家进入，他们受益于更便宜的计算机，而其他人不得不以更贵的价格购买，因此，如果我们考虑技术发展曲线，我们还没有到达那里。

So, you think this field will still remain largely very competitive because there are always new players coming in and they benefit from cheaper computers that others had to buy more expensive, and therefore, if we think about the technology development curve, we are still not there.

正如我之前提到的，没有对现实世界的掌握，就没有 AGI。现在让一些投资者如此兴奋的所有东西，与真正想要的 AGI 相去甚远——真正的机器人和真正的工厂自己做事情，而不仅仅是屏幕后面通过图灵测试之类的 AI。

You know, as I mentioned before, no mastery, no AGI without mastery of the real world. And all the stuff that is now so exciting to some of the investors, it's really a far cry from what you really want to have an AGI with real robots and real factories that do their own thing and not just AI behind the screen, passing the Turing test and stuff.

再稍微多聊一下这个替代情景，你能不能这样说：也许大型语言模型，这项技术，这种技术会无处不在，我们会看到和 Google 一样的情况——我们整合了所有搜索引擎，LLM 也会如此。它会保持为一个——所有这些不同的提供商会基本上合并成一个。这可能会是那个曾经……

Just to linger a little bit more on this alternative scenario, could you say that maybe a large language model, this technology, this piece of technology is something that would be everywhere, and we will see the same as Google did — we consolidated every search engine, and we will see the same for LLMs. It will stay as a — all these different providers will just basically collapse into one. This could be with the one which was...

是的，但想象一下。那么，会发生什么？在几年或至少几十年内，目前云为你做的所有事情，比如搜索等等，都会在你本地计算机上完成。如果你不想，你就不必连接到互联网。我们还会做 Google 和 Facebook 现在为你做的其他事情。但你不需要用你的数据来支付，也不需要接受大量广告。所以，所有这些对 Google、Facebook 以及其他公司，还有阿里巴巴等正在做这些的公司来说，都将具有挑战性。就像 60 年代的大型数据中心，它们是集中式的，当时 IBM 是主要玩家。而所有这些在几十年内很快就过时了。就像 80 年代的大块头手机，你根本没法随身携带，所以人们把它们放在车里。而今天每个人口袋里都有一个更简单、更小但更强大的东西。未来也会是这样。每个小人物都会拥有大量 AI，包括物理 AI，这将使他的生活更长寿、更健康、更轻松。所以，不会是少数几家大公司获利而穷人受损。不，穷人会赢很多，因为大公司没有真正的护城河。而新的发展正在侵蚀他们以前的护城河。

Yeah, but imagine. So, what is going to happen? In a couple of years or at least decades, all the stuff that the cloud is currently doing for you, like search and so on, it will all be on your local computer. And you won't — if you don't want to, you won't have to connect it to the internet. And we'll do other things that Google and Facebook do for you now. But you won't have to pay with your data and accept lots of ads. So, all of that is going to be challenging for Google and Facebook and others, and Alibaba and others who are doing that. Just like the big data centers in the '60s, which were centralized and back then IBM was the big player. And all of that stuff became obsolete pretty quickly within a few decades. Just like in the '80s the big mobile phones that you really couldn't carry around, so that's why people had them in their cars. And today everybody has a much simpler thing, a much smaller, but much more powerful thing in his pockets. And the future will be like that. Every little guy will have lots of AIs, including physical AIs, that will make his life longer and healthier and easier. So, it's not going to be a few huge companies that profit and the poor guys will lose. No, the poor guys are going to win a lot because the big guys don't have a real moat. And the new developments, they are eating away at their previous moats.

你在这里提出的建议听起来很吸引人。但是，为了从你那里挤出一些建议，我们应该关注什么呢？我的意思是，我们不应该在软件上投资。我们应该关注方法，并尝试找到更少依赖这些资源的巧妙方式吗？

It sounds intriguing what you suggest here. But, in order to squeeze some recommendation out of you, what should we focus on then? I mean, we should not go with the investments in software. Should we focus on methods and try to find clever ways where we rely less on these resources?

是的，所以从欧洲的角度来看，我认为你应该关注那些效果不好的事情，我已经说了十年了。传统上，欧洲一直是工程和制造机器的中心，以及制造机器的机器、工具机和其他各种出口到世界各地的精密设备。日本也类似，日本也是，然后韩国也是，而大多数基础算法和 AI 技术都来自欧洲。所以很明显，你想把这两个方面结合起来，在 2010 年代，我就在报纸上发表过相关文章，试图鼓励像德国这样的国家把这些东西结合起来，还有瑞士，因为瑞士的工业与德国相似，至少与德国南部相似。所以这本来是很明显的事情，但后来我也被邀请到中国，我说了类似的话，区别在于德国和瑞士并没有真正在那里投入任何纳税人的钱，但中国人在上海疯狂投入，上海的经济也许与瑞士相当，尽管我认为他们有 2400 万人口，而瑞士可能只有八九百万左右，但他们当时投资了 150 亿，你必须看到当时 10 亿的杠杆效应可能相当于今天的 1000 亿，不仅因为通货膨胀，还因为新的竞争，因为突然每个人都这样投资。所以欧洲国家并没有真正努力遵循我在 2010 年代给出的建议。但是，现在还不算太晚，因为物理世界中的 AI 还没有被任何人拥有。是的，今天我们有了能做空翻之类花哨动作的机器人。但是，你知道，它们并不能真正做我能用这叠纸巾做的事情。它们也不能用我的手做我能用手做的事情。所以，它们真的非常非常有限。而我的身体完全优于所有这些机器人。所以，对于像瑞士这样的工程国家来说，仍然有一条明显的道路。瑞士在各种自动化、机械、精密机械、机器人等方面都非常强大。我们可能在这个国家现在看到了机器人创业生态的雏形。许多小型机器人初创公司正在涌现。我希望所有这些能自然地融合成真正的物理世界 AI。2014 年，我们已经说过同样的话。这就是为什么我们在 2014 年成立了一家专门从事物理世界 AI 的公司。就像我们做的许多事情一样，也许它超前于时代。但是，现在还不算太晚。所以，物理 AI 领域，比屏幕后的 AI 大得多，还没有被任何人拥有。特别是所有工程国家，所有有工程和机床传统的国家，都应该看到其中的巨大机会。但也许做一点产业政策是有用的。就像中国和美国一样。硅谷不是通过风险投资创建的。硅谷是通过纳税人的钱创建的，以回应 1957 年苏联人造卫星冲击，当时俄罗斯人突然在太空领域领先。当时投入了数千亿甚至数万亿的今日美元，用于创建有竞争力的生态，包括各种研究实验室和大量五角大楼资金，甚至今天五角大楼仍然是美国最大的风险投资来源，有许多小公司获得利润丰厚的合同，几千万美元左右，成长为独角兽，不再容易被收购。中国人完全理解这一点，他们利用产业政策在深圳和中国其他地区创造了世界工厂。所以，当前两个超级大国非常擅长产业政策，从而实现了今天的成就。所以，也许遵循两个超级大国的一些经验教训并不愚蠢。

Yeah, so from a European point of view I would argue that you should focus on those things that don't work well and I have said that for a decade. So traditionally Europe has been the center of engineering and building machines and machines for building machines, tool machines and all kinds of fancy equipment that was exported to the rest of the world. Japan was similar, Japan also and Korea then also, and most of the basic algorithms AI techniques they come from Europe. So it should be obvious that you want to bring these two aspects together and in the 2010s I had newspaper articles just about that trying to encourage nations like Germany to bring these things together but also Switzerland because the industry in Switzerland is similar to the one in Germany at least in southern Germany. So that would have been the obvious thing but then I also got invited in China I said similar things and the difference was Germany and Switzerland didn't really invest any taxpayers money there but the Chinese did like crazy in Shanghai which has an economy which is maybe comparable to the one of Switzerland although it has I think they have 24 million people and Switzerland maybe eight or nine or something but they invested 15 billion back then and you have to see that the leverage of a billion back then is maybe like the leverage of a hundred billion today not only because of inflation but also because of the new competition because suddenly everybody is investing like that. So and the European countries didn't really make a big effort to follow this advice that I gave in the 2010s. But, it's not too late because AI in the physical world is not yet owned by anybody. Yeah, today we have fancy robots that can do, you know, saltos and whatever. But, you know, they can't really do what I can do with this set of paper towels here. And they can do with my hands what I can do with my hands. So, they are really really limited. And my physical body is totally superior to the one of all these robots. So, there still would be an obvious path forward for engineering countries like Switzerland. So, Switzerland is really strong in all kinds of automation and machinery and fine machinery and robots and so on. And we are seeing maybe in this country now the beginnings of a robot startup ecology. Lots of little robot startups are now coming up. And I hope that all of that will naturally merge into true AI for the physical world. In 2014, you know, we already said the same thing. And that was the reason why we founded a company just for AI for the physical world in 2014. And like many things we did, maybe it was ahead of its time. But, it's not too late. So, the field of physical AI, which is much bigger than, you know, AI behind the screen, is not yet owned by anybody. And especially all the engineering countries, all the countries with tradition and engineering and machine tools and so on they should see a huge opportunity in that. But maybe it's useful to do a little bit of industrial policy. Just like China and the US. So Silicon Valley was not created through venture capital. Silicon Valley was created through taxpayers money in response to the 1957 Sputnik shock when the Russians were suddenly ahead in space. And hundreds of billions in today's money, thousands of billions were invested into creating competitive ecology there with all kinds of research labs and lots of Pentagon funding and even today the Pentagon is still by far the biggest venture capital source in the United States and there are lots of little companies that get lucrative contracts a couple of tens of millions or something to grow until they are unicorns and not easily bought any longer. And the Chinese have completely understood that and they used industrial policy to create workbench of the world in Shenzhen and other regions of China. So the two current superpowers they were really good at industrial policy for achieving what they are today. So maybe it would be not stupid to follow some of the lessons learned from the two superpowers.

我认为这是非常鼓舞人心的话。我的意思是，首先，现在还不算太晚，你知道我们可以在这方面有发言权。

I think that's very encouraging words. I mean, first of all, it's not too late and you know we can have a say in this.

有趣的是，在欧洲风险投资业务停滞了几十年之后，突然纳税人的钱进来了，至少在一些国家，通过这些新的国防努力。

It's interesting to see that suddenly after decades of stagnation in the venture capital business in Europe, suddenly taxpayers money is coming in, at least in some countries, through these new defense efforts.

突然之间，德国在基础设施和军事上投入了 5000 亿。突然之间，国防开支的百分比疯狂上升。突然之间，像 Helsing 和 Stark 这样的小公司，估值达到 200 亿左右，按欧洲标准来说已经很高了。它们从纳税人那里拿到数十亿资金，获得好合同，从而实现了那种增长。所以也许，又是军方，或者通过军方，纳税人将资助大量这类研究。这重复了世界历史上多次发生过的事情：军事防御项目带来了许多突破，尽管在有些人看来这很不幸。

And suddenly in Germany, 500 billion are being spent on infrastructure and military. Suddenly the percentage of defense spending is going up like crazy. And suddenly you have little companies such as Helsing and Stark, which are worth 20 billion or something, which is a lot by European standards. And they get billions of taxpayers' money for good contracts that allow that kind of growth. So maybe again, it's the military that will fund, or the taxpayer that will fund, a lot of that research through the military. Repeating what has happened many times before in the history of the world, where military defense projects led to many breakthroughs, as unfortunate as this might seem to some.

国防是一切创新的根源。

Defense is the root of all innovation.

是的。

Yeah.

让我们继续沿着这个思路，谈谈一种非常欧洲的心态：GDPR 监管。我还要补充一点，也许这是一种全球性的恐惧，对 AI 的恐惧。你认为这种恐惧有根据吗？我们是否应该在意？

Let's continue this path and talk about a very European mindset: GDPR regulation. And I would add on to this, maybe that's a global fear, the fear of AI. Do you think this is grounded in something that we should care about?

这让我想起 2010 年代发生的事情，当时人们开始讨论 AI 可能带来的反乌托邦后果，以及如何确保 AI 与人类的目标对齐。当时有各种公开信，我不断收到新的信件，要求我签署暂停 AI 研究的禁令。那时就已经显得很遥远了。同一批人在提议这样的信件。他们秘密地，甚至不那么秘密地，试图推进 AI 的前沿。那些试图改进自己 AI 系统的人，突然呼吁暂停，这看起来很明显，他们只是为了稍微延迟竞争。当然，那时已经很清楚，世界上没有任何情报机构或军方会因为有这样一封信就考虑停止 AI 研究。因为每个人都想，‘如果我们不做，别人就会获得优势。’所以当然，我们必须在军事和情报应用上尽可能推进 AI 的使用。而欧洲某些国家的监管者，已经几十年没有战争局势了，所有这些都显得无关紧要，只会损害当地的竞争力。现在我们又遇到了同样的事情。还有 AI 公司的领导者，他们建议设立某种监督机构来决定什么该做、什么不该做。就像 2010 年代，在 Asilomar 有一次会议，他们有一封信，其中一条写着‘停止递归自我改进的研究。’类似这样。当然，我没有签署，因为自 1987 年我的文凭论文以来，我的主要兴趣和焦点一直是递归自我改进。这是通往真正 AI、通往 AGI 的明显路径。现在我们从这些公司的领导者那里又收到类似的信息，看起来太愚蠢了。

So it reminds me so much of what happened in the 2010s, when people started discussing the possible dystopic consequences of AI and how can we make sure that AI is aligned with the goals of the humans. And there were all these letters, and all the time I got new letters asking me to sign all this moratorium, let's stop AI research for a while. And back then it already seemed so far. The same guys were proposing letters like that. They were secretly, or not even so secretly, trying to push forward the frontiers of AI. And you had the same guys who were trying to improve their own AI systems suddenly calling for a moratorium, and it seemed so obvious that they are just doing that to delay the competition a little bit. And of course back then it was already clear that no secret service or no military in the world would even remotely consider stopping AI research because of a letter like that. Because everybody was thinking, 'Well, if we don't do it, then the others will gain an advantage.' And so of course we will have to push as far as we can for military and secret service applications, the use of AI. And all the regulators that you had in certain countries in Europe, who haven't had a war situation for many decades, all of that seemed so irrelevant and just damaging the local competitiveness. And now we again have the same thing. Also, there are leaders of AI companies and they are making suggestions about how to have some sort of overseeing authority that kind of decides what should be done, what should not be done. Like in the 2010s, there was a conference in Asilomar and they had a letter and one of the things written down there was 'Stop research on recursive self-improvement.' Something like that. And of course, I didn't sign that because since my diploma thesis in 1987, my main interest and my main focus has been on recursive self-improvement. The obvious path to true AI, to AGI. And now we again get similar messages from leaders of these companies and it seems so silly.

我理解，但我认为这与恐惧有关，而恐惧是非理性的。但你今天提到，我用了自我修改代码或自我修改权重这样的术语。你知道，在所有科幻电影中，比如《终结者》等等，这是万恶之源。然后当你启用这些规则时，天网就失控了。我们是否需要关心某种后备机制？我们是否需要关心将安全性整合到未来的 AI 中？还是说 AI 会自我学习？这是否是这类智能系统进化的一部分？我的意思是，很多人以我们人类为例。我们可能是最聪明、最进化的，但我们非常擅长剥削那些不那么发达的。这是我们该关心的吗？

I understand, but I think this goes with fear, which is something non-rational. But you mentioned here today I used the term self-modifying code or self-modifying weights if you like. You know, the root of all evil in all the science fiction movies, you know, Terminator and so forth. And then when you enable these rules, then you know, Skynet breaks loose. Do we need to care to have some sort of fallback mechanism? Do we need to care about integrating safety into future AIs? Or is that something that AI will self-learn? Is it something part of the evolution of such smart systems? I mean, many people make the example with us humans. We were maybe the smartest, the most evolved, but we were very good at exploiting the ones which were not as developed. Is that something that we should care about?

你几乎总是应该关心你所做的一切可能带来的各种潜在弊端。你应该关心。但关心的方式不是签署一封会被世界上所有强大的情报机构和军方忽视的信。你应该始终尝试做理性的事情。当然，你认为理性的事情会与数百万、数十亿其他人认为理性的事情相竞争。如果你把 10 个人放在一个房间里，你会有 10 种不同的意见，关于什么对人类有益，什么对这部分人类有益，什么对那部分人类有益，等等。所以所有这些努力似乎注定要失败，因为存在太多相互冲突的利益，我们现在在乌克兰战争中看到了最极端的形式，AI 被乌克兰无人机用来对抗俄罗斯的 AI 无人机等等。所使用的工具，这些基于 AI 的无人机，都是由人类设定的目标驱动的，但人类的目标完全相反，本质上是为了杀死对方。所以，当然，乌克兰没有理性的人，俄罗斯也没有，会遵循这种一再提出的时髦的暂停令。所有这些似乎都注定要失败。所以你应该考虑其他方式，如果有的话，来应对这种情况。但这真的很难，因为如果你有 80 亿人在地球上，其中许多，数百万的人理解 AI，数百万的人能构建 AI，他们对什么是好的 AI、什么不是好的 AI 都有不同的想法，在这种竞争环境中，许多最重要的参与者是年轻人，你知道，博士生之类的，没有明显的方法来控制这一切。就像没有明显的方法控制人类一样。是的，当然，作为人类父母，我们教育孩子成为社会有用的成员。我们惩罚他们做坏事。我们奖励他们做好事，我们认为的好事。但那边那个人认为的好事可能与我不同。在那种情况下，中央权威或其他东西如何塑造这种进化并不明显。

You should almost always care about all kinds of potential drawbacks of whatever you're doing. You should care about that. But the way of caring about that is not to assign a letter that is going to be ignored by all the powerful secret services and militaries in the world. And you should always try to do the rational thing. Now, of course, whatever you think is the rational thing is going to compete with what millions and billions of other people think is the rational thing. And if you put 10 people in a room, you will have 10 different opinions about what is good for humankind and what is good for this part of humankind versus this part of humankind and so on. So all of these efforts seem doomed just because there are so many conflicting interests and we see it in the most extreme form right now in Ukraine in the war where AI is used by Ukrainian drones to fight AI-based drones of Russia and stuff like that. Where the tools that are being used, these AI-based drones, are both driven by goals that were set by humans, but the humans have totally opposing goals, and it's really about killing the other guys. So, of course, no rational person in Ukraine, but also not in Russia, is going to follow a moratorium of this fashionable kind which has been proposed again and again. And all of that seems doomed. So you should think about other ways, if there is any, to deal with that situation. But it's really difficult because if you have 8 billion people on a planet and many of them and millions of them understand AI, and millions of them can construct AIs, and they all have different ideas about what's a good AI and what's not a good AI, and in this competitive situation where many of the most important players are young guys, you know, PhD students or whatever, there is no obvious way of controlling all of that. Just like there is no obvious way of controlling humans. Yes, of course, we as human parents, we educate our kids to become useful members of society. And we punish them when they do bad things. And we reward them when they do good things, what we think is good. But what this hand over there thinks is good is maybe different from what I think is good. And in a situation like that, it is not obvious how a central authority or something might be able to shape this evolution.

这是一种演化，就像丛林中的演化一样。有一群人正在研究那种 AI，希望它能带来重大突破，并与那边另一群人、甚至全球其他地方的 AI 竞争，其方式很难预见。但就像人类社会中传统的冲突，以及由人类组成的公司一样，当所有这些实体相互竞争、试图通过使用 AI 来改善自身处境时，未来真的很难预测，而它们都不希望 AI 受到限制，以免削弱自己的竞争力。

And it's an evolution, just like the evolution in the jungle, where there's a group of people working on that kind of AI, hoping it will bring a great breakthrough, and it will compete with the other AIs of the other group over there, or maybe around the globe, in a way that is hard to foresee. But just like in traditional conflicts in human societies and companies formed by humans, it's really hard to predict the future as all of these entities compete with each other and try to improve their own situation through the use of AI, which they don't want to be limited in a way that shrinks their competitiveness.

我们正缓慢但确定地接近尾声。Jürgen，我基本上还有两件事要问你，这真是一场愉快的对话。我参加过你的一场演讲，我想那已经是 10 年前了。时间过得真快。

We are slowly but surely coming to an end. I have basically just two more things to ask you, Jürgen, and it's been a delightful conversation. I attended one of your talks, I think it's already 10 years ago. So time passes by.

但你知道，你一定觉得很无聊，因为 10 年前我说了完全一样的话。

But you know, you must be so bored, because 10 years ago I said exactly the same thing.

对我来说，那非常非常激动人心，也非常非常有启发性。所以我觉得，尤其是在这个时代，再次听到其中一些想法是值得的。接着这一点，我想你当年演讲的最后试图描绘未来，描绘这段 AI 之旅将如何演进。如果我能请你再试一次的话。我们现在身处 2026 年。LLM 已经占据了主舞台。从这里开始，它会如何继续？你怎么看？

For me it was very, very exciting and very, very inspiring. So I think it's worthy to hear some of those thoughts again, especially in this time. So to add on to this, I think at the end of your talk back then you tried to picture the future, how this AI journey will evolve. If I could kindly ask you to try to do it again. So, we are here in 2026. LLMs have taken the main stage. How will this continue from here on? What do you think?

如果你想预测未来，看看过去总是好的，因为我们所有的预测能力都基于过去的数据。2014 年，我发现了这条美丽的、历史性的时间线。当我说历史性时间线时，我指的是自 138 亿年前大爆炸以来的全部时间。它之所以美丽，是因为世界和人类演化史上的所有重大发展，似乎都整齐地排列在这条指数级加速的时间线上，而这条线似乎会在我有生之年、希望是在 2042 年，收敛到一个点。有些人认为，实际上很多人认为，届时我们将第一次拥有真正的超级智能，它能在物理世界中做一切事情，而不仅仅是在屏幕后面，其能力超越任何人类所能做的。让我给你展示这条时间线，因为我会从中推导出我的预测。这样讲清楚了吗？

If you want to predict the future, it's always good to look at the past, because all our prediction power is based on past data. And in 2014, I discovered this beautiful historic timeline. And when I talk about historic timelines, I mean all the time since the Big Bang, 13.8 billion years ago. And it's beautiful in the sense that all the major developments in the history of the evolution of the world and of humankind seem to be neatly aligned on this exponentially accelerating timeline, which seems to converge within my lifetime, hopefully, in the year 2042. Where some people think, many people actually think, that for the first time we will have this true superintelligence that can do everything in the physical world, and not only behind the screen, in a way that surpasses what any human can do. And let me show you this timeline, because from there I will derive my predictions. Does that make sense?

当然。

Sure.

它始于 138 亿年前，终止、收敛于 2042 年，就像每一条指数级时间线都会在某个有限时间后收敛一样。我们取这段时间的四分之一，就来到 35 亿年前，那时发生了一件非常重要的事：这个星球上生命的诞生。我们再取这段时间的四分之一，就来到 9 亿年前，那时发生了一件非常重要的事：第一批类似动物的可移动生命。我们再取四分之一，就来到 2.2 亿年前，那时发生了一件非常重要的事：第一批哺乳动物，演化出了我们的祖先。我们再取四分之一，就来到 5500 万年前，那时发生了一件非常重要的事：第一批灵长类，我们的祖先。我们再取四分之一，就来到 1350 万年前，那时发生了一件非常重要的事：第一批人科动物，我们的祖先。我不知道为什么我们总是撞上人类演化中这些伟大的时刻，但我也试过五分之一、三分之一、七分之一以及各种和谐比例，它们都不管用，而四分之一却非常有效。现在我们再取四分之一，就来到 350 万年前，技术的黎明，正如 nature 所称，第一批石器。我们再取四分之一，就来到下一个伟大突破：火或受控火的发明，大约 80 万年前。我们再取四分之一，就来到大约 20 万年前，解剖学意义上的现代人，就像你和我。我们再取四分之一，就来到行为意义上的现代人，5 万年前，离开非洲，殖民世界。我们再取四分之一，就来到 1.3 万年前，那时发生了一件非常重要的事：文明的起源、农业、动物的驯化、最初的定居点。而全部文明史只是宇宙历史的百万分之一。所以它只是一瞬。我们再取四分之一，就来到 3300 年前，铁器时代第一次人口爆炸的开始。我们再取四分之一，就来到收敛点 2042 年，你记得，减去 800 年。那是 13 世纪，铁与火在中国结合，他们发明了火药、火炮、火箭和枪。它彻底改变了世界。我们再取四分之一，就来到 Omega 之前 200 年，2042 年之前。19 世纪，疾病细菌理论、医学革命、农业革命，导致了 1850 年第二次人口爆炸的开始，以及工业革命，铁与火以更复杂的形式结合，驱动新经济。我们再取四分之一。你记得收敛点是 2042 年。现在，我们必须减去 51 年，就来到 1991 年。那正是我们在慕尼黑的奇迹之年，当时我们有了 P 和 T，或者说 Transformer 和大语言模型的 P 和 T 的基础，深度残差学习，最近两个世纪被引用最多的论文，不是最重要，而是被引用最多。还有一大堆其他东西，如今正在驱动现代 AI。我们再取这段时间的四分之一，那是未来的一个点。大约是 2029 年，也就是 2042 减 13。所以，在最后 13 年里，也许我们会看到与过去 1.3 万年、自人科动物出现以来的 1300 万年、自宇宙诞生以来的 138 亿年一样多的变化，主观上的变化。那么，那会是什么？在这最后的 13 年里，从几年后开始，会发生什么？我相信它会是我之前提到的那件事，即真正的物理机械与 AI 的结合，其方式远超人类所能做的。目前，在物理世界中，人类完全优于 AI。这种情况不会一直持续下去。一旦我们有了它们——聪明但并非超级聪明的机器人，聪明到足以操作现有机器、修理现有机器和工具，那么你突然就有了这种新的 Scaling（规模扩张）。

So, it starts 13.8 billion years ago and it ends, it converges, in 2042, like every exponential timeline converges after some finite time. And we take a quarter of that time and we come out 3.5 billion years ago, when something really important happened, which was the invention of life on this planet. And we take a quarter of that time, we come out 0.9 billion years ago, when something really important happened: the first animal-like mobile life. And we take a quarter of that time, we come out 220 million years ago, when something really important happened: the first mammals, and evolved our ancestors. And we take a quarter of the time, we come out 55 million years ago, when something really important happened: the first primates, our ancestors. And we take a quarter of the time, we come out 13.5 million years ago, when something really important happened: the first hominids, our ancestors. I don't know why we keep hitting all these great moments in human evolution, but I also tried fifths and thirds and sevenths and harmonic proportions; none of them works, but quarters work really well. Now we take a quarter of the time, we come out 3.5 million years ago, with the dawn of technology, as nature called it, the first stone tools. And we take a quarter of the time, we come out with the next great breakthrough: the invention of fire, or controlled fire, about 800,000 years ago. And we take a quarter of the time, we come out roughly 200,000 years ago, with anatomically modern man, like you and me. And we take a quarter of the time, we come out with the behaviorally modern man, 50,000 years ago, leaving Africa, colonizing the world. And we take a quarter of the time, we come out 13,000 years ago, when something really important happened: the beginnings of civilization, agriculture, domestication of the animals, first settlements. And all of civilization history is just one millionth of the history of the universe. So it's just a flash. And we take a quarter of the time, we come out 3,300 years ago, with the onset of the first population explosion in the Iron Age. And we take a quarter of the time, we come out at the convergence point, 2042, you remember, minus 800 years. That's in the 1200s, when iron and fire come together in China and they invented gunpowder and cannons and rockets and guns. And it has completely changed the world. And we take a quarter of the time and we come out 200 years before Omega, before 2042. And the 1800s, with the germ theory of disease, medicine revolution, agricultural revolution, which led to the onset of the second population explosion in 1850, and the industrial revolution, iron and fire coming together in a more sophisticated form, powering the new economy. And we take a quarter of the time. And you remember the convergence point is 2042. Now, we have to subtract 51 years of that and we come out in 1991. That's exactly our miraculous year in Munich, when we had P and the T, or the basics of the P and the T of the Transformer of the large language models, the deep residual learning, the most cited, not most important, but most cited papers of the two most recent centuries. And a whole bunch of other things which are now driving modern AI. And we take a quarter of that time, and that is a point in the future. That's roughly 2029, so 2042 minus 13. And so, in the last 13 years, maybe we will see as much change, subjective change, as in the last 13,000 years, as in the last 13 million years since the hominids came up, as in the last 13.8 billion years, since the universe was created. So, what could that be? What will happen in these final 13 years, starting in a few years? And I believe it's going to be the thing that I mentioned before, which is this marriage between true physical machinery and AI in a way that is very superior to what humans can do. At the moment, humans are totally superior in the physical world to AIs. It's not going to stay like that. And once we have them smart, but not super smart robots that are smart enough to operate the existing machines and to repair the existing machines and tools, then you have suddenly this new kind of scaling.

目前，唯一能很好扩展的是软件，复制十亿份软件非常容易，但复制十亿个机器人却非常困难。但一旦你有了自我复制的机器人，那我们就有的聊了。那可能会在 13 年内带来巨大的变化。最终，这个汇聚点将与太阳系的殖民化有很大关系，因为自我复制的机器并不局限于我们的生物圈。它会在小行星带和水星上运作。水星似乎是殖民太阳系的理想起点，因为它没有大气层。这意味着建造电磁炮、所谓的质量驱动器非常简单，你只需要达到 4.5 公里/秒，也就是水星的逃逸速度，就可以把所有这些材料射入太空，那里没有重力，然后你就可以建造所有科幻小说里的东西，比如那些巨大的旋转圆柱体，50 公里长，20 公里厚，你让它们旋转，它们比火星或类似地方更宜居。然后，各种额外的机械和基础设施，不仅对人类，对 AI 和机器人也很重要，太阳系的殖民化大约会在那时开始，它将改变一切。

At the moment, the only thing that scales well is software, and it's very easy to make a billion copies of some software, but it's really difficult to make a billion copies of some robot. But once you have self-replicating robots, then we are talking. And that will then maybe within 13 years lead to so much change. Finally, this convergence point, which will have a lot to do with the colonization of the solar system because a self-replicating machinery is not limited to our biosphere. And it will work on in the asteroid belt and on Mercury. And Mercury seems like an ideal starting point for colonizing the solar system because it has no atmosphere. Which means it's really simple to build, you know, electromagnetic guns, mass drivers they are called, and you just have to achieve 4.5 km a second, which is the escape velocity of Mercury, and you can shoot all this material into space where you have no gravity, and then you can build all the stuff of science fiction, you know, these huge rotating cylinders, you know, 50 km long, 20 km thick, and you rotate them and they are much more livable than Mars or places like that. And then you know, all kinds of additional machinery and infrastructure that will not be irrelevant for humans but for AIs and robots and the beginning of the colonization of the solar system roughly at that time it will start and it will change everything.

是的。然后当然，很明显它不会止步于太阳系，因为几乎所有可以用来建造更多机器人、AI、大型 AI 和更复杂 AI 的物质都不在太阳系，而在银河系的其余部分，不幸的是，银河系直径有 10 万光年，这意味着你需要几十万年才能殖民整个银河系，但它会很快。在宇宙历史中，它会像一道闪光，因为几百万年相对于宇宙历史来说微不足道。然后它不会止步于银河系。现在让我们展望一个时代……到目前为止，我们一直在除以四，四分之一再四分之一。但现在让我们乘以四。让我们展望一个宇宙年龄约为 550 亿年的时代，比现在老四倍。到那时，大部分可见宇宙将被 AI 许可。人类无法跟上，但没关系。主要的决策者将不再是人类。当然，这是一个非常有趣的新发展，也因为通信延迟而有趣，因为如果你这里有一个快速进化的 AI，与一百万光年外的另一个快速进化的 AI 通信会有点无聊。那么这意味着什么？它将如何进化？让我们看看。但是的，所以整个宇宙，所有你能获取的物质和能量，都将被用来使整个宇宙更加复杂，我们只能看到大致的轮廓，但不知道细节意味着什么。但它将改变一切。

Yeah. And then of course, it's clear that it's not going to end with the solar system because almost all of the matter that you could use for building more robots and AIs and big AIs and more complex AIs is not in the solar system but in the rest of the galaxy, which unfortunately is, you know, 100,000 light years of diameter, which means you will need a couple of hundreds of thousands of years to colonize it all, but it's going to be rapid. It's going to be like a flash in the history of the universe because a few million years is nothing compared to the history of the universe. And then it will not stop with the galaxy. And now let's look ahead to a time... So far we have always divided by four, a quarter of a quarter of a quarter. But now let's multiply by four. Let's look ahead to a time when the universe is going to be 55 billion years old or so. Four times older than it is now. By then most of the visible cosmos is going to be permitted by AI. In a way where humans cannot follow, but that's okay. And the main decision-makers will not be humans any longer. And of course it's a very interesting new development which is also interesting because of the communication delays because if you have some rapidly evolving AI here it will be kind of boring to communicate with another rapidly evolving AI that is a million light years away, you know. So what does that mean? How is that going to evolve? Let's see. But yeah, so the entire cosmos, all the matter and energy that you can get hold on is going to be used to make the entire cosmos more complex in a way that we where we can see only the rough outline of all of that, but we don't mean what it means in detail. But it's going to change everything.

说得太美了。Jürgen，你是一个真正的灵感来源。我想我追随你的工作很久了，我会继续追随，我也建议我们的听众这样做。迫不及待想看到接下来会有什么。是的，也要向你致敬，因为你如此关注早期的贡献，确保那些在早期推动该领域的人得到应有的认可。当然，你是一个家喻户晓的名字，但如果你想提请注意某个具体的事情，我现在把舞台或麦克风交给你。

Beautifully put. Jürgen, you are a true inspiration. I think I followed your work for a very long time and I will keep following it and I can just suggest our audience to do the same. Can't wait what next comes out of it. And yeah, also shout out to you for also paying so much attention to the early contributions in making sure credit is due to the people which pushed the field in the very early days. And of course you're a household name, but if you want to draw attention to something specific, I'll give you the stage or the microphone now.

是的。我们涵盖了很多东西。是的，我们可以一直继续下去，但那样我们就需要额外的时间，也许今天已经够了。

Yeah. We covered so many things. Yeah, we could go on and on and on, but then we would need additional hours and maybe maybe it's enough for today.

再次感谢。非常感谢。

Here again. Thank you so much.

谢谢你，Philippe，感谢你所做的一切。

Thank you, Philippe, for all you do.

你是个机器人。

You're a robot.