物理漏洞与宇宙谜题

什么是神经网络？为什么它在学习方面似乎做得如此出色？

What is a neural network and why does it seem to do such a surprisingly good job of learning?

它是大脑的一种数学抽象，最初就是这样发展的。归根结底，它是一个相当简单的数学表达式：基本上是一系列矩阵乘法（其实就是点积），再加上一些非线性变换。它有很多旋钮，大致对应大脑中的突触——这些旋钮是可训练、可修改的。我们的想法是找到合适的旋钮设置，让神经网络做你想做的事，比如分类图像。我不会赋予它太多与大脑相关的意义；它实际上只是一个带有旋钮的复杂数学表达式，需要正确的设置。

It's a mathematical abstraction of the brain, originally developed that way. At the end of the day, it's a fairly simple mathematical expression: basically a sequence of matrix multiplies, which are really dot products, with some nonlinearities thrown in. It has many knobs, loosely related to synapses in your brain—they're trainable and modifiable. The idea is to find the setting of those knobs that makes the neural net do whatever you want, like classify images. I wouldn't endow it with too much meaning relative to the brain; it's really just a complicated mathematical expression with knobs that need proper settings.

但诗歌只是字母和空格的集合，却能让我们产生某种感受。同样地，当大量旋钮聚集在一起时，无论是在大脑中还是在计算机中，它们似乎都会以强大的能力让我们惊讶。

But poetry is just a collection of letters with spaces, yet it can make us feel a certain way. In that same way, when you get a large number of knobs together, whether in the brain or in a computer, they seem to surprise us with their power.

我觉得这很公平。我确实大大低估了它，因为当这些神经网络足够大，并且在足够复杂的问题上训练时（比如在互联网海量数据集上进行下一个词预测），你绝对会从中得到非常惊人的涌现行为。然后这些神经网络就会展现出相当惊人、神奇的特性。

I think that's fair. I'm underselling it by a lot because you definitely get very surprising emergent behaviors out of these neural nets when they're large enough and trained on complicated enough problems, like next-word prediction on a massive dataset from the internet. Then these neural nets take on pretty surprising, magical properties.

你的大脑现在是在做下一个词预测，还是在做更有趣的事情？

Is your brain doing next-word prediction right now, or something more interesting?

绝对是一种生成式模型，类似 GPT，由你提示。你给我一个提示，我以生成的方式回应。但我也会自动从自己的记忆中补充额外的提示，引用某种陈述性的记忆结构，将其与你的提示结合起来，然后给出信息。

Definitely some kind of generative model, like a GPT, prompted by you. You give me a prompt, and I respond in a generative way. But I'm also adding extra prompts from my own memory automatically, referencing some declarative structure of memory, putting that together with your prompt and giving a message.

你刚才说的话有多少是你以前说过的？

How much of what you just said has been said by you before?

基本上没有。但如果你把我这辈子说过的所有话都搜索一遍，我可能以前就用同样的顺序说过很多同样的词。我用的都是常见短语，但最终把它们重新组合成相当独特的句子。这里面有大量的重新组合。

Nothing, basically. But if you look at all the words I've ever said and do a search, I've probably said a lot of the same words in the same order before. I'm using common phrases, but remixing them into a pretty unique sentence at the end of the day. There's a ton of remixing.

你对神经网络的评价还不够高。你对这种涌现行为的最佳直觉是什么？

You're not giving enough credit to neural nets. What's your best intuition about this emergent behavior?

这很有趣，因为我一方面在低估它们，但另一方面又觉得有点高估——尽管它们在数学上如此简单，却能从中获得如此多的涌现魔法行为，这其实相当不可思议。我认为我们实际上相当擅长优化这些神经网络，当你给它们一个足够困难的问题时，它们被迫在优化过程中学习非常有趣的解决方案。这些解决方案具有非常有趣的涌现特性。旋钮中蕴含着智慧和知识。

It's interesting because I'm simultaneously underselling them, but I also feel like there's an element of overselling—it's actually kind of incredible that you can get so much emergent magical behavior out of them despite them being so simple mathematically. I think we are actually fairly good at optimizing these neural nets, and when you give them a hard enough problem, they are forced to learn very interesting solutions in the optimization. Those solutions have these emergent properties that are very interesting. There's wisdom and knowledge in the knobs.

旋钮中的表示在直觉上合理吗？

Does the representation in the knobs make sense intuitively?

大量旋钮可以容纳一种表示，捕捉关于它所看过数据的深层智慧。旋钮非常多。具体来说，现在人们非常兴奋的一种神经网络是 GPT，它基本上就是下一个词预测网络。你从互联网上读取一系列单词，然后尝试预测下一个词。一旦你在足够大的数据集上训练它们，你就可以以任意方式提示它们，让它们解决问题。你可以让它看起来像是在尝试解决一个数学问题，它们会根据在互联网上看到的内容继续给出它们认为的解决方案。很多时候，这些解决方案看起来非常一致，而且可能是正确的。

A large number of knobs can hold a representation that captures some deep wisdom about the data it has looked at. It's a lot of knobs. Speaking concretely, one of the neural nets people are very excited about right now are GPTs, which are basically next-word prediction networks. You consume a sequence of words from the internet and try to predict the next word. Once you train these on a large enough dataset, you can prompt them in arbitrary ways and ask them to solve problems. You can make it look like you're trying to solve a mathematical problem, and they will continue what they think is the solution based on what they've seen on the internet. Very often those solutions look remarkably consistent and potentially correct.

你还会从生物神经网络中汲取智慧吗？生物学做了哪些计算机尚未做到的令人印象深刻的事情？

Do you still draw wisdom from biological neural networks? What impressive thing is biology doing that computers are not yet?

我比这个领域通常的做法更谨慎地使用大脑类比。当然，神经网络最初是受大脑启发而来的，但归根结底，训练后得到的产物是通过与大脑截然不同的优化过程得到的。我认为它是一个非常复杂的外星人工制品。我们训练的神经网络是复杂的外星人工制品。我不做大脑类比，因为产生它的优化过程与大脑非常不同——没有多智能体自我对弈的设置。

I'm much more hesitant with analogies to the brain than you might see in the field. Certainly the way neural networks started stemmed from inspiration by the brain, but at the end of the day, the artifacts you get after training are arrived at by a very different optimization process than the one that gave rise to the brain. I think of it as a very complicated alien artifact. The neural nets we train are complicated alien artifacts. I do not make analogies to the brain because the optimization process that gave rise to it is very different from the brain—there was no multi-agent self-play setup.

进化是一种优化，本质上是对海量数据的压缩目标。人工神经网络在做压缩，但生物神经网络是智能体，处于一个运行了极长时间的多人自对弈系统中，目标是生存和繁殖。进化发现拥有预测模型在大脑中很有用，但还有大量其他机制、价值函数和古老核团。整个生物体通过胚胎发生从一个单细胞构建，代码在 DNA 中。构建过程中存在学习和计算。如果看地球生命史，最有趣的发明是什么？生命起源、真核生物、哺乳动物、人类，还是都是同一过程的延续？

Evolution was an optimization that amounts to a compression objective on massive data. Artificial neural networks do compression, but biological neural networks are agents in a multi-agent self-play system running for a very long time, aimed at survival and reproduction. Evolution found it useful to have predictive models in the brain, but there are many other gadgets, value functions, and ancient nuclei. The whole organism is built from a single cell through embryogenesis, with code in DNA. There's learning and computation in that building process. If you look at the history of life on Earth, where is the most interesting invention? Origin of life, eukaryotes, mammals, humans, or is it all a continuation?

这是一个非常非凡的故事，从地球形成和条件到太阳系、活跃的地球和生命起源。我无法挑出唯一一个最有趣的部分。作为 AI 研究者，可能最后一部分最有趣：很多动物没有建立技术社会，但我们做到了，而且发生得很快很晚。发生了一些我不完全理解的有趣事情。我几乎直觉上理解其他所有事情，但不懂那部分及其速度。两种解释：要么是延续，人类没什么特别；要么是真正特殊的事情发生了，比如罕见事件。受资源约束的协作和多智能体方面可能创造了复杂智能。这似乎是进化过程的自然产物，有间断平衡和稀疏的飞跃，如生命起源、DNA、有性生殖、真核生物、内共生和意识的出现。

It's an extremely remarkable story, from Earth's formation and conditions to the solar system, active Earth, and abiogenesis. I can't pick a single unique piece. As an AI researcher, probably the last piece is most interesting: we have many animals that don't build technological society, but we do, and it happened very quickly and recently. Something interesting happened that I don't fully understand. I almost understand everything else intuitively, but not that part and its speed. Two explanations: either it's a continuation with nothing special about humans, or something truly special happened, like a rare event. Collaboration and multi-agent aspects constrained by resources might have created complex intelligence. It seems like a natural outgrowth of evolution, with punctuated equilibrium and sparse leaps like origin of life, DNA, sex, eukaryotes, endosymbiosis, and emergence of consciousness.

你不认为人类是独特的。你认为有多少外星智慧文明存在？它们的智能与我们是不同还是相似？

You don't think humans are unique. How many intelligent alien civilizations do you think are out there? Is their intelligence different or similar to ours?

我最近一直在思考这个问题。这个悖论以及我对生命起源的兴趣，是为了理解太空中技术社会的普遍性。我研究得越多，就越认为应该有很多。为什么我们没有收到他们的消息？我同意，我们在地球上做的事情似乎没那么难。我以前认为生命起源是神奇罕见的事件，但读了 Nick Lane 的《生命之问》等书后，你会相信这并不罕见。活跃地球上的基本化学过程，如碱性热液喷口、质子梯度等，可能在其他系统发生。生命起源在地球形成后相当快，几亿年。所以生命应该相当普遍。我目前认为没有大的瓶颈，所以应该有很多生命。调和费米悖论的唯一方法是我们无法观测到它们。

I've been preoccupied with this question. The paradox and my interest in origin of life is to understand how common technological societies are in space. The more I study, the more I think there should be quite a lot. Why haven't we heard from them? I agree it feels like what we did on Earth isn't that difficult. I used to think origin of life was a magical rare event, but reading books like Nick Lane's 'The Vital Question' makes you believe it's not that rare. Basic chemistry on an active Earth with alkaline vents, proton gradients, etc., could happen on other systems. Origin of life was fairly fast after Earth's formation, a few hundred million years. So life should be fairly common. I currently think there are no major drop-offs, so there should be quite a lot of life. The only way to reconcile the Fermi paradox is that we just can't observe them.

Nick Lane 和许多生物学家认为，从细菌到复杂生物（真核生物发生）是最难的跳跃。

Nick Lane and many biologists think the jump from bacteria to complex organisms (eukaryogenesis) is the hardest jump.

我明白他们在生物学细节上比我知识渊博得多。

I get that they are much more knowledgeable about biology intricacies than me.

这听起来很疯狂，因为单细胞生物有多少？你有多少时间？肯定没那么难。在十亿年里，这甚至不算很长的时间。真的，就是所有这些细菌在有限资源下争斗。我相信它们能发明更复杂的东西。我不明白如何从‘hello world’程序到发明一个函数之类的。所以我同意你的看法。我只是觉得我看不到——如果生命起源是最难的事情，但如果它发生得这么快，那它一定无处不在。也许我们只是太笨了，看不到它。

That seems crazy because how many single-cell organisms are there, and how much time do you have? Surely it's not that difficult. In a billion years, it's not even that long of a time. Really, just all these bacteria under constrained resources battling it out. I'm sure they can invent more complexity. I don't understand how to move from a 'hello world' program to inventing a function or something like that. So I'm with you. I just feel like I don't see any—if the origin of life is the hardest thing, but if it's not because it happens so quickly, then it's got to be everywhere. And maybe we're just too dumb to see it.

嗯，我们只是没有很好的机制来观察这种生命。我先声明我不是专家，但我想见一位外星智能和交流方面的专家。我非常怀疑我们找到这些智能和这些地球的能力。例如，无线电波很糟糕。它们的功率基本上按距离平方衰减。我记得读过，我们当前的无线电波用我们现在的设备无法测量，只能到十分之一光年远，基本上是很短的距离。你真的需要定向发射巨大功率的信号才能被远距离接收。所以我认为我们的测量能力并不出色。我认为可能还有其他文明存在。大问题是为什么它们不建造冯·诺依曼探测器，为什么它们不进行星际旅行穿越整个银河系？我目前的答案是星际旅行可能非常困难。如果你想以接近光速移动，你会沿途遇到子弹，因为即使是微小的氢原子和尘埃粒子在那种速度下也有巨大的动能。所以你需要某种屏蔽，还要应对所有宇宙辐射。外面太残酷了。真的很难。所以我的想法是，也许星际旅行极其难以实现。感觉我们离做到那一点还有十亿年。可能你必须在太空中非常缓慢地移动，而不是接近光速。所以我怀疑我们测量生命的能力，也怀疑我们渗透整个银河系或跨星系空间的能力。这是我目前唯一能想到的解决办法。

Well, we just don't have really good mechanisms for seeing this life. I'm not an expert, just to preface this, but I want to meet an expert on alien intelligence and how to communicate. I'm very suspicious of our ability to find these intelligences out there and to find these Earths. Radio waves, for example, are terrible. Their power drops off as basically one over R squared. I remember reading that our current radio waves would not be measurable by our devices today, only like one-tenth of a light-year away, basically a tiny distance. You really need a targeted transmission of massive power directed somewhere for it to be picked up over long distances. So I think our ability to measure is not amazing. I think there are probably other civilizations out there. The big question is why don't they build von Neumann probes and why don't they interstellar travel across the entire galaxy? My current answer is that interstellar travel is probably really hard. If you want to move at close to the speed of light, you're going to be encountering bullets along the way because even tiny hydrogen atoms and little dust particles have massive kinetic energy at those speeds. So you need some kind of shielding, you have all the cosmic radiation. It's just brutal out there. It's really hard. So my thinking is maybe interstellar travel is just extremely hard to build. It feels like we're not a billion years away from doing that. It just might be that you have to go very slowly through space, as opposed to close to the speed of light. So I'm suspicious of our ability to measure life and to permeate all of space in the galaxy or across galaxies. That's the only way I can currently see around it.

是的，想到有数万亿个智能外星文明，它们缓慢地在太空中旅行以相互见面，这有点令人震惊。有些相遇，有些开战，有些合作，或者它们都只是独立的小群体。我不知道。从统计上看，如果有数万亿个，肯定有些群体足够接近，能看到彼此。一旦你看到某种肯定是复杂生命的东西，我们可能会非常积极地想要弄清楚那到底是什么，并试图与它们见面。你的第一直觉是什么？试图在代际层面上与它们见面，防御它们，还是什么？作为美国总统和科学家，你的直觉是什么？我不知道你在这个问题中更喜欢哪个角色。

Yeah, it's kind of mind-blowing to think that there are trillions of intelligent alien civilizations out there, kind of slowly traveling through space to meet each other. Some of them meet, some go to war, some collaborate, or they're all just independent little pockets. I don't know. Statistically, if there are trillions of them, surely some of the pockets are close enough to see each other. And once you see something that is definitely complex life, we're probably going to be intensely aggressively motivated to figure out what the hell that is and try to meet them. What would be your first instinct? To try to meet them at a generational level, defend against them, or what? What would be your instinct as President of the United States and the scientists? I don't know which hat you prefer in this question.

是的，我认为这个问题真的很难。我会说，例如，对我们来说，我们身边有很多原始生命形式。我们有各种各样的蚂蚁和其他生物，我们与它们共享空间。我们犹豫是否要影响它们，并且默认情况下我们试图保护它们，因为它们是令人惊叹、有趣的动态系统，花了很长时间进化。它们有趣而特别。我不知道你是否想默认摧毁它们。所以我喜欢花了很长时间进化的复杂动态系统。我想如果我能负担得起，我会保护它们。我希望银河系资源也是如此，它们会认为我们是一个不可思议、有趣的故事，花了数十亿年展开，你不想就这么摧毁它。我可以想象两个外星人现在正在谈论地球，说：‘我是复杂动态系统的忠实粉丝，所以我认为保护这些有价值。’它们基本上就像是在看一个视频游戏或电视节目。

Yeah, I think the question is really hard. I will say, for example, for us, we have lots of primitive life forms on Earth next to us. We have all kinds of ants and everything else, and we share space with them. We are hesitant to impact them, and we are trying to protect them by default because they are amazing, interesting dynamical systems that took a long time to evolve. They are interesting and special. I don't know that you want to destroy that by default. So I like complex dynamical systems that took a lot of time to evolve. I think I'd like to preserve it if I can afford to. I'd like to think that the same would be true about the galactic resources, and that they would think we're kind of an incredible, interesting story that took a few billion years to unravel, and you don't want to just destroy it. I could see two aliens talking about Earth right now and saying, 'I'm a big fan of complex dynamical systems, so I think it has value to preserve these.' They basically are a video game they watch, or a TV show they watch.

是的，我认为你需要一个非常好的理由来摧毁它。我们为什么不摧毁这些蚂蚁农场？因为我们实际上并没有与它们直接竞争。我们只是偶然地破坏它们，但资源充足。所以你为什么要摧毁如此有趣和珍贵的东西呢？从科学的角度来看，你可能会探测它，稍后与它互动，从中学习。所以我想知道是否有些物理现象我们认为是物理的，但实际上是在与我们互动，比如戳一下手指看看会发生什么。我认为这对外星科学家来说会非常有趣，看看这里发生了什么。我们今天看到的是一个快照，基本上是数十亿年大量计算的结果。所以它可能是由外星人启动的。这可能是一台运行程序的计算机。如果你有能力这样做，我会选择一个基于我对生命化学前提的理解而具备条件的类地行星，然后播种生命并运行它。你不会百分之百这样做，观察它并保护它吗？那不仅仅是一个非常好的电视节目；这是一个很好的科学实验。从这个意义上说，这是一个物理模拟。也许进化是理解计算、计算东西或理解生命及其分支的最有效方式。这确实让我觉得我们是科学实验的一部分，但也许一切都是科学实验。这对我们有什么改变？

Yeah, I think you would need a very good reason to destroy it. Why don't we destroy these ant farms? It's because we're not actually in direct competition with them right now. We do it accidentally, but there's plenty of resources. So why would you destroy something that is so interesting and precious? Well, from a scientific perspective, you might probe it, interact with it later, learn something from it. So I wonder if there could be certain physical phenomena that we think is physical but is actually interacting with us, like poking a finger to see what happens. I think it would be very interesting to alien scientists what happened here. And what we're seeing today is a snapshot, basically a result of a huge amount of computation over billions of years. So it could have been initiated by aliens. This could be a computer running a program. If you had the power to do this, I would pick an Earth-like planet that has the conditions based on my understanding of the chemistry prerequisites for life, and I would seed it with life and run it. Wouldn't you 100% do that and observe it and protect it? That's not just a hell of a good TV show; it's a good scientific experiment. And in that sense, it's a physical simulation. Maybe evolution is the most efficient way to understand computation, or to compute stuff, or to understand life and what branches it can take. It does make me feel weird that we're part of a science experiment, but maybe everything is a science experiment. How does that change anything for us?

我不知道。两个猿类的后代在谈论身处科学实验之中。

I don't know. Two descendants of apes talking about being inside a science experiment.

我对你描述的那种刻意的‘彭斯首映’想法持怀疑态度，目前我在历史记录中看不到任何神迹干预。我觉得尼克·莱恩书里的故事很有道理，解释了生命如何在地球上独特地诞生，我不需要去寻找更离奇的解释。

I'm suspicious of this idea of a deliberate Pence Premiere as you described it, and I don't see a divine intervention in the historical record right now. I do feel like the story in Nick Lane's books makes sense, how life arose on Earth uniquely, and I don't need to reach for more exotic explanations.

当然。但电子游戏里的 NPC 也观察不到任何神迹干预，而我们可能都只是运行着某种代码的 NPC。也许将来它们会——目前 NPC 还很笨，但一旦它们运行着 GPT，也许它们会说：‘嘿，这很可疑，搞什么鬼？’

Sure. But NPCs inside a video game don't observe any divine intervention either, and we might just be NPCs running a kind of code. Maybe eventually they will—currently NPCs are really dumb, but once they're running GPTs, maybe they will be like: 'Hey, this is really suspicious, what the hell?'

你有一条著名的推文：‘如果你用光子轰击地球一段时间，就能发射出一辆跑车。’就像《银河系漫游指南》里把地球总结为‘基本无害’。你认为地球完成计算后，所有可能的故事——一段话或一句话——会是什么？就像如果地球是一本书，所有可能的完整故事。

You famously tweeted: 'It looks like if you bombard Earth with photons for a while, you can emit a Roadster.' So like in Hitchhiker's Guide to the Galaxy, they summarize Earth as 'mostly harmless.' What do you think are all the possible stories—a paragraph or a sentence long—that Earth could be summarized as once it's done its computation? Like all the possible full stories if Earth is a book.

很可能必须有一个结局。地球会终结，而且可能以各种方式终结——或早或晚。这些自我复制系统从动力学中涌现，自我延续，变得更加复杂，最终产生意识并建立社会，这非常不可思议。从某种意义上说，这就像一种确定性的波，在任何像地球这样安排得当的系统上都会发生。有一种必然性。它很美，而且最终会以某种方式结束。这是一个化学多样性的环境，复杂动力系统可以不断演化，但存在终止条件。我不知道终止条件是什么，但有一条趋势线，我们是故事的一部分。它会走向何方？

Probably there has to be an ending. Earth will end, and it could end in all kinds of ways—soon or later. It's pretty incredible that these self-replicating systems arise from dynamics, perpetuate themselves, become more complex, eventually become conscious, and build a society. In some sense, it's like a deterministic wave that happens on any sufficiently well-arranged system like Earth. There's a certain inevitability. It's beautiful and ends somehow. It's a chemically diverse environment where complex dynamical systems evolve further and further, but there are terminating conditions. I don't know what they are, but there's a trend line, and we're part of that story. Where does it go?

我们常被描述为 AI 的生物引导程序。人类是不可思议的生物系统，能计算、能爱等等，但我们极其低效。我们通过音频交谈，用声带串行操纵七个符号，耗时数秒。与计算机的运行频率相比，这很尴尬。所以合成智能似乎是发展的下一阶段。我不知道它会走向何方。在某个时刻，宇宙是一个谜题，这些合成 AI 会揭开并解决它。然后会发生什么？如果你快进地球数十亿年，一开始很安静，然后你看到城市灯光，然后呢？是平静下来，还是爆炸？地球发射出大量卫星？那是一种疯狂的爆炸，而我们日复一日地经历着。我看过一个地球的动画：很长时间什么都没发生，然后在最后两秒，城市出现，近地轨道变得拥挤——这就是一场爆炸。我们生活在一个鞭炮里，它即将开始发射各种有趣的东西。它可能看起来像破坏性的鞭炮，但希望是建设性的。

We're often described as a biological bootloader for AIs. Humans are an incredible biological system capable of computation, love, etc., but we're extremely inefficient. We talk through audio, manipulate seven symbols serially using vocal cords, over multiple seconds. It's embarrassing compared to the frequencies at which computers operate. So it seems synthetic intelligences are the next stage of development. I don't know where it leads. At some point, the universe is some kind of puzzle, and these synthetic AIs will uncover and solve it. Then what happens? If you fast-forward Earth billions of years, it's quiet, then you see city lights, then what? Is it a calming, an explosion? Earth emitting a giant number of satellites? It's some kind of crazy explosion, and we're living through it day to day. I saw an animation of Earth: nothing happens for a long time, then in the last two seconds cities appear, low Earth orbit gets cluttered—it's an explosion. We're living in a firecracker that will start emitting all kinds of interesting things. It might look like a destructive firecracker, but hopefully it's constructive.

考虑到这一点，想到造物主给我们的宇宙谜题是什么，既好笑又恶心。例如，在卡尔·萨根的《接触》中，π的十进制展开里藏着给人类的信息。也许我们应该给造物主发送一条信息，或者创建一个量子力学系统来提醒他们我们的智能存在。从他们的角度看，那只是量子场论，一个巨大的元胞自动机。你怎么能注意到我们的存在？你可能甚至无法在模拟中捕捉到我们。那么你如何证明你存在、你有智能、你是宇宙的一部分？这就像来自地球的智能图灵测试。也许造物主正在试图完成句子中的下一个词——地球正在发回信息。谜题是提醒造物主我们存在，或者也许是打破系统，给造物主一点颜色看看。就像在电子游戏中，你找到一个漏洞来在主机上执行任意代码。有人让马里奥通过漏洞玩《乓》。所以也许谜题就是找到一种方法来利用它。我认为一些合成 AI 最终会发现宇宙是一个谜题，并以某种方式解决它，那就是终局。

Given that, it's hilarious and disgusting to think about what the puzzle of the universe is that the creator gave us. For example, in Carl Sagan's Contact, there's a message for humanity in the digits of pi in base 11. Maybe we're supposed to give a message to our creator, or create a quantum mechanical system to alert them of our intelligent presence. From their perspective, it's just quantum field theory, a massive cellular automaton. How do you even notice we exist? You might not be able to pick us up in that simulation. So how do you prove you exist, that you're intelligent, and part of the universe? It's like a Turing test for intelligence from Earth. Maybe the creator is trying to complete the next word in a sentence—Earth is sending a message back. The puzzle is alerting the creator that we exist, or maybe it's to break out of the system, stick it to the creator. Like in a video game, you find an exploit to execute arbitrary code on the host machine. Someone got Mario to play Pong by exploiting the game. So maybe the puzzle is to find a way to exploit it. I think some synthetic AIs will eventually find the universe to be a puzzle and solve it in some way, and that's the endgame.

你经常把它看作模拟吗？宇宙是一种……

Do you often think about it as a simulation? The universe being a kind of...

是的，我经常想。这是一个迷人的可能性。

Yes, I think about it. It's a fascinating possibility.

计算可能存在漏洞和利用。是的，我也这么认为。物理本质上就是这样吗？我认为物理可能存在漏洞，我们应该尝试找到它们。安排某种疯狂的量子力学系统，以某种方式给你一个缓冲区溢出，以某种方式给你一个浮点数的舍入误差。是的，没错。还有越来越复杂的漏洞。那些是玩笑，但可能实际上非常接近。是的，我们会找到某种方法来提取无限能量，例如。当你训练一个强化学习智能体在物理模拟中，并让它们快速在平地上奔跑时，它们会做出各种奇怪的事情作为优化的一部分。它们会仰面滑行。这是因为强化学习优化找到了从摩擦力中提取无限能量的方法。基本上，它们的糟糕实现找到了一种产生无限能量并滑过表面的方法。这不是你预期的；这是一种反常的解决方案。所以也许我们可以找到类似的东西。也许我们可以成为这个物理模拟中的那只小狗，找到漏洞或逃避宇宙物理的预期后果。我们会找到某种通往怪异的捷径。

Computation that might have bugs and exploits. Yes, I think so. Is that what physics is essentially? I think it's possible that physics has exploits, and we should be trying to find them. Arranging some kind of a crazy quantum mechanical system that somehow gives you a buffer overflow, somehow gives you a rounding error in a floating point. Yeah, that's right. And more and more sophisticated exploits. Those are jokes, but that could be actually very close. Yeah, we'll find some way to extract infinite energy, for example. When you train a reinforcement learning agent in physical simulations and you ask them to say run quickly on flat ground, they'll end up doing all kinds of weird things as part of that optimization. They'll get on their back leg and slide across the floor. It's because the optimization of the reinforcement learning on that agent has figured out a way to extract infinite energy from the friction forces. Basically, their poor implementation found a way to generate infinite energy and just slide across the surface. It's not what you expected; it's a sort of perverse solution. So maybe we can find something like that. Maybe we can be that little dog in this physical simulation, the cracks or escapes the intended consequences of the physics that the universe came up with. We'll figure out some kind of shortcut to some weirdness.

是啊，然后哦，但问题是那种怪异之处在于第一个发现它的人，比如仰面滑行，我们就只会做那件事了。是的，很快因为每个人都这么做。所以回形针最大化器是个荒谬的想法，但很可能我们都会转向它，因为它太有趣了。

Yeah, and then oh man, but see the problem with that weirdness is the first person to discover the weirdness, like sliding on the back legs, that's all we're going to do. Yeah, it's very quickly because everybody does that thing. So like the paperclip maximizer is a ridiculous idea, but that very well could be what we'll just all switch to because it's so fun.

嗯，没有人会发现它。顺便说一句，我认为这必须是某种第三代超级智能 AGI。就像我们在构建第一代 AGI，你知道，第三代。是的，所以一个 AI 的引导程序，那个 AI 将成为另一个 AI 的引导程序。然后我们无法内省那可能是什么。我认为这些事物很可能会完全惰性。我喜欢一些科幻小说，其中这些东西完全惰性，不与任何东西互动。我觉得那很美，因为它们可能以某种方式弄清了宇宙的元游戏。可能它们在做一些完全超出我们想象的事情，并且不与简单的化学生命形式互动。为什么要那样做呢？所以我觉得这些想法很有吸引力。

Well, no person will discover it. I think by the way, I think it's going to have to be some kind of a superintelligent AGI of a third generation. Like we're building the first generation AGI, you know, third generation. Yeah, so the bootloader for an AI, that AI will be a bootloader for another AI. And then there's no way for us to introspect what that might even be. I think it's very likely that these things, for example, say you have these AGIs, it's very likely they will be completely inert. I like these kinds of sci-fi books sometimes where these things are just completely inert, they don't interact with anything. And I find that kind of beautiful because they've probably figured out the meta game of the universe in some way. Potentially they're doing something completely beyond our imagination, and they don't interact with simple chemical life forms. Why would you do that? So I find those kinds of ideas compelling.

它们的乐趣来源是什么？它们在做什么？宇宙中解决的来源是什么，但却是惰性的？所以你能定义惰性是什么意思吗？

What's their source of fun? What are they doing? What's the source of solving in the universe but inert? So can you define what it means inert?

它们逃脱了，意思是它们会以非常奇怪的方式对我们表现，因为它们超越了，它们在玩元游戏。而元游戏可能就像以非常奇怪的方式安排量子力学系统来提取无限能量，将π的数字展开到任意位数，它们会建造自己的小聚变反应堆或一些疯狂的东西。它们在做一些超出理解、我们无法理解的事情，而且实际上在底层非常出色。

They escape, as in they will behave in some very strange way to us because they're beyond, they're playing the meta game. And the meta game is probably like arranging quantum mechanical systems in some very weird ways to extract infinite energy, solve the digital expansion of Pi to whatever amount, they will build their own little fusion reactors or something crazy. They're doing something beyond comprehension and not understandable to us, and actually brilliant under the hood.

如果量子力学本身就是一个系统，而我们只是认为它是物理，但实际上我们是这个有机体上的寄生虫呢？我们并没有真正伤害物理，我们只是生活在这个有机体上，试图理解它，但它实际上是一个具有深层智能的有机体。也许物理本身就是那个做着超级有趣事情的有机体，而我们只是一个小东西，一只坐在它上面试图从中获取能量的蚂蚁。

What if quantum mechanics itself is the system and we're just thinking it's physics, but we're really parasites on this organism? We're not really hurting physics, we're just living on this organism and we're trying to understand it, but really it is an organism with a deep, deep intelligence. Maybe physics itself is the organism that's doing a super interesting thing, and we're just like one little thing, an ant sitting on top of it trying to get energy from it.

我们就像波中的粒子，我觉得这基本上是确定性的，将宇宙从某种大爆炸带到某种超级智能复制器，在给定这些物理定律的情况下，宇宙中的某种稳定点。

We're just kind of like these particles in a wave that I feel like is mostly deterministic and takes the universe from some kind of a big bang to some kind of a super intelligent replicator, some kind of a stable point in the universe given these laws of physics.

你不认为，正如爱因斯坦所说，上帝不掷骰子，所以你认为它基本上是确定性的，没有随机性？

You don't think, as Einstein said, God doesn't play dice, so you think it's mostly deterministic, there's no randomness in the thing?

我认为它是确定性的。哦，有很多……嗯，我想对随机性小心一点。伪随机，是的。我不喜欢随机。我认为物理定律可能是确定性的。是的，我认为它们是确定性的。我对这个问题感到很不舒服。

I think it's deterministic. Oh, there's tons of... well, I want to be careful with randomness. Pseudo random, yeah. I don't like random. I think maybe the laws of physics are deterministic. Yeah, I think they're deterministic. I just got really uncomfortable with this question.

你对宇宙是否随机感到焦虑吗？没有随机性？你说你喜欢《心灵捕手》。这不是你的错，安德烈。这不是你的错，伙计。所以你不喜欢随机性？

Do you have anxiety about whether the universe is random or not? What's there's no randomness? You say you like Good Will Hunting. It's not your fault, Andre. It's not your fault, man. So you don't like randomness?

是的，我觉得这令人不安。我认为它是一个确定性系统。我认为看起来随机的东西，比如波函数坍缩等，实际上都是确定性的，只是纠缠等等，以及某种多重宇宙理论，诸如此类。

Yeah, I think it's unsettling. I think it's a deterministic system. I think that things that look random, like say the collapse of the wave function, etc., I think they're actually deterministic, just entanglement and so on, and some kind of a multiverse theory, something something.

好吧，那为什么我们感觉有自由意志？如果我举起手，我现在选择这样做。这感觉不像是一个确定性的事情。感觉我在做选择。

Okay, so why does it feel like we have free will? If I raise my hand, I chose to do this now. That doesn't feel like a deterministic thing. It feels like I'm making a choice.

感觉是这样。好吧，所以都是感觉。只是感觉。是的，所以当一个强化学习智能体在做选择时，那是……它并不是真的在做选择。选择都已经在那里了。是的，你在解释这个选择，并为你做出了它而创造一个叙事。而现在我们在谈论这个叙事。这非常元。

It feels like it. Okay, so it's all feelings. It's just feelings. Yeah, so when an RL agent is making a choice, is that... it's not really making a choice. The choices are all already there. Yeah, you're interpreting the choice and you're creating a narrative for having made it. And now we're talking about the narrative. It's very meta.

回顾过去，你遇到过的深度学习或 AI 中最美丽或最令人惊讶的想法是什么？你看到这个领域以有趣的方式爆炸式增长。什么酷想法让你坐下来感叹？无论大小。

Looking back, what is the most beautiful or surprising idea in deep learning or AI in general that you've come across? You've seen this field explode and grow in interesting ways. Just what cool ideas made you sit back and go 'hmm'? Small or big.

嗯，我最近想得最多的可能是 Transformer 架构。基本上，神经网络有很多架构曾经流行又过时，用于不同的感官模态。对于视觉、音频、文本，你会用不同的神经网络来处理。而最近我们看到向一种架构的收敛，即 Transformer。你可以输入视频，或者图像、语音或文本，它都能处理。它有点像一种通用的计算机，同时也是可训练的，并且在我们硬件上运行非常高效。所以这篇论文在 2016 年发表，我想说，是《注意力就是一切》。你事后批评这个论文标题，说它没有预见到它将要产生的巨大影响。是的，我不确定作者是否意识到那篇论文会产生的影响。可能他们没有，但我认为他们意识到了一些……

Well, the one that I've been thinking about recently the most probably is the Transformer architecture. So basically, neural networks have a lot of architectures that were trendy have come and gone for different sensory modalities. For vision, audio, text, you would process them with different looking neural nets. And recently we've seen a convergence towards one architecture, the Transformer. And you can feed it video or you can feed it images or speech or text, and it just gobbles it up. It's kind of like a bit of a general purpose computer that is also trainable and very efficient to run on our hardware. So this paper came out in 2016, I want to say, 'Attention is All You Need'. You criticize the paper title in retrospect that it didn't foresee the bigness of the impact that it was going to have. Yeah, I'm not sure if the authors were aware of the impact that that paper would go on to have. Probably they weren't, but I think they were aware of some of...

Transformer 背后的动机和设计决策，他们选择不在论文中那样展开。所以我认为他们意识到这不仅仅是表面上的‘哦，我们只是在做翻译，这里有一个更好的架构’。你不只是在做翻译；你提出的是一台非常酷的可微分、可优化、高效的计算机。也许他们没有完全预见这一点，但我认为这非常有趣。他们用‘Attention Is All You Need’这样一个深刻的标题，是不是很有趣？我觉得以前没人用过这种标题，对吧？它就像一个梗。

The motivations and design decisions beyond the Transformer, and they chose not to expand on it in that way in the paper. So I think they had an idea that there was more than just the surface of like, 'Oh, we're just doing translation and here's a better architecture.' You're not just doing translation; this is like a really cool differentiable, optimizable, efficient computer that you've proposed. And maybe they didn't have all of that foresight, but I think it's really interesting. Isn't it funny that they went for such a profound idea with the title 'Attention Is All You Need'? I don't think anyone used that kind of title before, right? It's like a meme or something.

没错。有趣的是，如果标题更严肃，就不会有这种影响力。老实说，我有点同意你的看法，也更喜欢这样。如果太宏大，就会过度承诺而交付不足。所以你就想用梗来成就伟大。这应该印在 T 恤上。

Exactly. It's funny that if it were a more serious title, it wouldn't have the impact. Honestly, there is an element of me that agrees with you and prefers it this way. If it were too grand, it would over-promise and under-deliver. So you want to just meme your way to greatness. That should be a t-shirt.

所以你发推说 Transformer 是一种宏伟的神经网络架构，因为它是一台通用可微分计算机。它在前向传播中具有表现力，通过反向传播和梯度下降可优化，并且具有高并行度的计算图，效率很高。你能讨论一下这些细节吗——表现力、可优化性、效率？

So you tweeted that the Transformer is a magnificent neural network architecture because it is a general-purpose differentiable computer. It is simultaneously expressive in the forward pass, optimizable via backpropagation and gradient descent, and efficient with a high-parallelism compute graph. Can you discuss some of those details—expressive, optimizable, efficient?

你想要一台通用计算机，可以在任意问题上训练，比如下一个词预测或检测图像中是否有猫。你想训练这台计算机，所以你要设置它的权重。我认为 Transformer 同时满足了许多设计标准，这使它非常成功。作者们有意打造这个非常强大的架构。基本上，它在前向传播中非常强大，因为它能够将非常通用的计算表达为一种消息传递：你有节点，它们都存储向量，这些节点可以查看彼此的向量并进行通信。节点广播‘嘿，我在找某些东西’，其他节点广播‘这些是我有的东西’——这就是键和值。所以它不仅仅是注意力组件；Transformer 远不止于此。它有许多架构组件：残差连接、排列方式、多层感知机、堆叠等等。但基本上，有一种消息传递方案，节点相互查看，决定什么有趣，然后更新彼此。所以我认为它是一个非常有表现力的函数，可以在前向传播中表达许多不同类型的算法。不仅如此，它的设计方式——残差连接、层归一化、softmax 注意力等——也使其可优化。这非常重要，因为有很多强大的计算机无法用我们现有的技术（反向传播和梯度下降，这些是一阶方法，非常简单的优化器）轻松优化。所以你也需要它可优化。最后，你希望它在硬件上高效运行。我们的硬件，比如 GPU，是巨大的吞吐量机器，偏好大量并行。你不想做大量顺序操作；你想并行做很多操作。Transformer 的设计也考虑到了这一点，所以它是为我们的硬件设计的，既在前向传播中非常有表现力，又在反向传播中非常可优化。

You want to have a general-purpose computer that you can train on arbitrary problems, like the task of next-word prediction or detecting if there's a cat in an image. You want to train this computer, so you want to set its weights. I think there are a number of design criteria that overlap in the Transformer simultaneously that made it very successful. The authors were deliberately trying to make this really powerful architecture. Basically, it's very powerful in the forward pass because it's able to express very general computation as a sort of message passing: you have nodes, they all store vectors, and these nodes get to look at each other's vectors and communicate. Nodes broadcast, 'Hey, I'm looking for certain things,' and other nodes broadcast, 'These are the things I have'—those are the keys and the values. So it's not just the attention component; the Transformer is much more than that. It has many architectural pieces: the residual connections, the way it's arranged, the multi-layer perceptron, the stacking, and so on. But basically, there's a message-passing scheme where nodes look at each other, decide what's interesting, and then update each other. So I think it's a very expressive function that can express many different types of algorithms in the forward pass. Not only that, but the way it's designed with residual connections, layer normalizations, softmax attention, and everything, it's also optimizable. This is a really big deal because there are lots of powerful computers that you can't optimize easily using the techniques we have—backpropagation and gradient descent, which are first-order methods, very simple optimizers. So you also need it to be optimizable. Lastly, you want it to run efficiently on hardware. Our hardware, like GPUs, is a massive throughput machine that prefers lots of parallelism. You don't want to do lots of sequential operations; you want to do many operations in parallel. The Transformer is designed with that in mind as well, so it's designed for our hardware, to be both very expressive in the forward pass and very optimizable in the backward pass.

你还说过，残差连接支持一种能力，先快速学习短算法，然后在训练过程中逐渐延长它们。先学习短算法的想法是什么？

And you said that the residual connections support a kind of ability to learn short algorithms fast first and then gradually extend them longer during training. What's the idea of learning short algorithms first?

把 Transformer 想象成一系列块，对吧？这些块有注意力和一个小型多层感知机。你进入一个块，然后回到这个残差路径，再进入另一个块并返回，这样就有多个层按顺序排列。我认为，由于残差路径，在反向传播中，梯度会沿着它不间断地流动，因为加法将梯度平均分配到所有分支。所以来自顶部的监督梯度直接流向第一层。所有残差连接的设计使得在初始化初期，它们对残差路径没有贡献。所以这看起来就像把 Transformer 想象成一个 Python 函数，你可以执行各种代码行。假设你有一个 100 层的 Transformer；通常它们会短得多，比如 20 层。

Think of it as a Transformer is a series of blocks, right? These blocks have attention and a little multi-layer perceptron. You go off into a block and come back to this residual pathway, then go off and come back, and you have a number of layers arranged sequentially. The way to look at it, I think, is because of the residual pathway, in the backward pass, the gradients sort of flow along it uninterrupted because addition distributes the gradient equally to all its branches. So the gradient from the supervision at the top just floats directly to the first layer. All the residual connections are arranged so that in the beginning during initialization, they contribute nothing to the residual pathway. So what it kind of looks like is imagine the Transformer is like a Python function, and you get to do various kinds of lines of code. Say you have a 100-layer deep Transformer; typically they would be much shorter, say 20.

所以如果你有 20 行代码，你可以在里面做一些事情。在优化过程中，首先优化第一行代码，然后第二行开始起作用，第三行也可以。我觉得由于残差路径和优化的动态特性，你可以学习一个非常短的算法来得到近似结果，但随后其他层会介入并开始贡献。最终，你是在优化一个 20 行代码的算法，只不过这些代码行非常复杂，因为它是 Transformer 的整个块——你可以在里面做很多事情。真正有趣的是，这个 Transformer 架构出奇地稳健。基本上，2016 年提出的 Transformer 和你今天使用的 Transformer 是一样的，只不过你重新调整了一些层归一化——层归一化被调整为预归一化形式。所以它一直非常稳定，但人们也添加了很多花哨的功能试图改进它。我认为这是同时优化理想神经网络架构许多属性的重要一步。人们试图改变它，但它被证明非常稳健。我确实认为可能存在更好的架构，但你得佩服它的稳健性。

So if you have 20 lines of code, you can do something in them. During optimization, first you optimize the first line of code, then the second line kicks in, and the third line can. I feel like because of the residual pathway and the dynamics of optimization, you can learn a very short algorithm that gets the approximate answer, but then the other layers kick in and start to contribute. At the end, you're optimizing over an algorithm that is 20 lines of code, except these lines are very complex because it's an entire block of a transformer—you can do a lot in there. What's really interesting is that this Transformer architecture has been remarkably resilient. Basically, the Transformer that came out in 2016 is the Transformer you would use today, except you reshuffle some of the layer norms—the layer normalizations have been reshuffled to a pre-norm formulation. So it's been remarkably stable, but there are a lot of bells and whistles that people have attached to try to improve it. I do think it's a big step in simultaneously optimizing for many properties of a desirable neural network architecture. People have tried to change it, but it's proven remarkably resilient. I do think there should be even better architectures potentially, but you admire the resilience here.

是的，这个架构有一些深刻的东西。至少到目前为止，一切都可以转化为 Transformer 能解决的问题。目前看来，Transformer 确实正在接管 AI。你基本上可以把任意问题输入进去，它是一个通用的可微分计算机，非常强大。AI 领域的这种趋同现象非常值得关注。就我个人而言，你认为关于 Transformer 还有什么可以发现的？有什么令人惊讶的事情吗？还是说它已经稳定了？我们可能会发现关于 Transformer 的有趣东西，比如顿悟时刻？也许与记忆、知识表示之类的东西有关。

Yeah, there's something profound about this architecture. At least so far, everything can be turned into a problem that Transformers can solve. Currently, it definitely looks like Transformers are taking over AI. You can feed basically arbitrary problems into it, and it's a general differentiable computer, extremely powerful. This convergence in AI has been really interesting to watch. For me personally, what else do you think could be discovered here about Transformers? What's a surprising thing? Or is it a stable place? Is there something interesting we might discover about Transformers, like aha moments? Maybe it has to do with memory, knowledge representation, that kind of stuff.

当然。当今的时代精神就是推动：基本上，现在的建议是不要动 Transformer，动其他一切。是的，所以人们正在扩大数据集，让它们变得更大；他们正在研究评估，让评估变得更大；他们基本上保持架构不变。这就是过去五年 AI 进步的方式。

Definitely. The zeitgeist today is just pushing: basically, right now the advice is do not touch the Transformer, touch everything else. Yes, so people are scaling up the datasets, making them much bigger; they're working on evaluation, making evaluation much bigger; and they're basically keeping the architecture unchanged. That's how we've had the last five years of progress in AI.

你怎么看待语言模型这一分支？你有没有感到惊讶？你的想象力有没有被——你提到了 GPT 以及所有越来越大的语言模型——所吸引？你认为这些模型的极限是什么？

What do you think about one flavor of it, which is language models? Have you been surprised? Has your imagination been captivated by—you mentioned GPT and all the bigger and bigger language models—and what are the limits of those models, do you think?

就拿自然语言任务来说：GPT 的训练方式基本上是从互联网下载大量文本数据，然后尝试预测序列中的下一个词。粗略地说，你是在预测词块，但仅此而已。非常有趣的是，它是一个语言模型。语言模型实际上已经存在了很长时间。有 2003 年甚至更早的语言建模论文。

So just the task of natural language: basically the way GPT is trained is you download a massive amount of text data from the internet and you try to predict the next word in a sequence. Roughly speaking, you're predicting word chunks, but that's it. What's been really interesting to watch is that it's a language model. Language models have actually existed for a very long time. There are papers on language modeling from 2003, even earlier.

你能解释一下什么是语言模型吗？

Can you explain what a language model is?

是的，语言模型的基本思想就是预测序列中的下一个词。2003 年 Bengio 和他的团队发表了一篇论文，首次使用神经网络来取三个或五个词，然后预测下一个词。他们在小得多的数据集上做这个，神经网络也不是 Transformer，而是多层感知机。但这是神经网络首次应用于这种场景。甚至在神经网络之前，就有语言模型，只不过它们使用的是 n-gram 模型。n-gram 模型是基于计数的模型：如果你尝试用两个词预测第三个，你就统计你见过多少次任何两个词的组合以及接下来是什么，然后预测训练集中出现最多的那个。所以语言建模已经存在很长时间了；神经网络做语言建模也很久了。所以真正新的、有趣的或令人兴奋的是，当你用足够强大的神经网络——Transformer——来扩展它时，你会得到所有这些涌现特性。基本上，如果你有足够大的文本数据集，在预测下一个词的任务中，你是在多任务处理大量不同类型的问题。你是在多任务处理对化学、物理、人性的理解——很多东西都聚集在这个目标中。这是一个非常简单的目标，但实际上你必须对世界有很多理解才能做出预测。

Yeah, a language model just basically the rough idea is predicting the next word in a sequence. So there's a paper from Bengio and his team in 2003 where for the first time they used a neural network to take, say, three or five words and predict the next word. They did this on much smaller datasets, and the neural net is not a Transformer, it's a multilayer perceptron. But it's the first time a neural network was applied in that setting. Even before neural networks, there were language models, except they used n-gram models. N-gram models are count-based models: if you try to take two words and predict the third, you just count how many times you've seen any two-word combinations and what came next, and you predict what you've seen most in the training set. So language modeling has been around for a long time; neural networks have done language modeling for a long time. So really what's new or interesting or exciting is just realizing that when you scale it up with a powerful enough neural net—Transformer—you have all these emergent properties. Basically, what happens is if you have a large enough dataset of text, in the task of predicting the next word, you are multitasking a huge amount of different kinds of problems. You are multitasking understanding of chemistry, physics, human nature—lots of things are sort of clustered in that objective. It's a very simple objective, but actually you have to understand a lot about the world to make that prediction.

你刚才说了那个词——理解。你是指化学、物理等方面吗？你觉得它在做什么？是在搜索正确的上下文吗？实际的过程是怎样的？

You just said the U word—understanding. Are you in terms of chemistry and physics and so on? What do you feel like it's doing? Is it searching for the right context? What is the actual process happening here?

是的，基本上它拿到一千个词，然后试图预测第一千零一个词。为了在整个互联网数据集上做得非常好，你实际上必须理解其中的上下文。这是一个足够困难的问题，如果你有足够强大的计算机，比如 Transformer，你最终会得到有趣的解决方案。你可以让它做各种事情，它展现出很多涌现特性，比如上下文学习。这就是 GPT 及其原始论文发表时的重大突破：你可以用各种方式提示它，让它做各种事情，它就会完成句子。但在完成句子的过程中，它实际上在解决各种我们关心的有趣问题。

Yeah, so basically it gets a thousand words and it's trying to predict the thousand and first. In order to do that very well over the entire dataset available on the internet, you actually have to basically kind of understand the context of what's going on in there. It's a sufficiently hard problem that if you have a powerful enough computer like a Transformer, you end up with interesting solutions. You can ask it to do all kinds of things, and it shows a lot of emergent properties, like in-context learning. That was the big deal with GPT and the original paper when they published it: you can just prompt it in various ways and ask it to do various things, and it will just kind of complete the sentence. But in the process of just completing the sentence, it's actually solving all kinds of really interesting problems that we care about.

你认为它是在做类似理解的事情吗？当我们对人类使用“理解”这个词时，我认为它是在做一些理解。在它的权重中，它理解了很多关于世界的东西，而且它必须这样做才能预测序列中的下一个词。

Do you think it's doing something like understanding? When we use the word understanding for us humans, I think it's doing some understanding. In its weights, it understands a lot about the world, and it has to in order to predict the next word in a sequence.

那么让我们在互联网数据上训练。你怎么看待这种数据集的方法？使用互联网数据——你认为互联网有足够结构化的数据来教会 AI 关于人类文明的知识吗？

So let's train on the data from the internet. What do you think about this approach in terms of datasets? Using data from the internet—do you think the internet has enough structured data to teach AI about human civilization?

是的，我认为互联网有海量数据。我不确定它是否足够完整。我不知道文本是否足以产生足够强大的 AGI。

Yeah, so I think the internet has a huge amount of data. I'm not sure if it's a complete enough set. I don't know that text is enough for having a sufficiently powerful AGI as an outcome.

有音频、视频和图像等等。是的，所以单靠文本，我有点怀疑。有很多事情我们不会写进文字里，仅仅因为它们对我们来说显而易见——世界如何运作、物理规律、东西会掉下来。我们不会把这些写进文本，因为为什么要写呢？我们共享那种理解。所以文本是人类之间的交流媒介，并不是包罗万象的世界知识媒介。但正如你指出的，我们确实有视频、图像和音频，所以我认为这肯定大有帮助。但我们还没有充分地在所有这些模态上训练模型，所以我认为这是很多人感兴趣的地方。但我想知道，那种共享的理解——我们可能称之为常识——为了正确完成句子，是否必须被学习或推断出来。所以也许事实上它在互联网上是隐含的，模型将不得不学习它，不是通过阅读，而是通过在表征中推断它。所以就像常识，就像我们，我不认为我们学习常识。没有人明确告诉我们，我们只是通过与世界互动来弄明白，对吧？所以一个模型通过阅读人们与世界互动的方式来学习，它可能必须推断出这一点。

There is audio and video and images and all that kind of stuff. Yeah, so text by itself, I'm a little bit suspicious about. There's a ton of things we don't put in text in writing, just because they're obvious to us about how the world works and the physics of it and things fall. We don't put that stuff in text because why would you? We share that understanding. And so text is a communication medium between humans, and it's not an all-encompassing medium of knowledge about the world. But as you pointed out, we do have video and we have images and we have audio, and so I think that that definitely helps a lot. But we haven't trained models sufficiently across all those modalities yet, so I think that's what a lot of people are interested in. But I wonder what that shared understanding of, well, we might call common sense, has to be learned or inferred in order to complete the sentence correctly. So maybe the fact that it's implied on the internet, the model is going to have to learn that not by reading about it, but by inferring it in the representation. So like common sense, just like we, I don't think we learn common sense. Like nobody tells us explicitly, we just figure it all out by interacting with the world, right? So here's a model reading about the way people interact with the world, it might have to infer that.

我想知道，是的。

I wonder, yeah.

你曾短暂参与过一个名为“World of Bits”的项目，训练一个强化学习系统在互联网上采取行动，而不是像我们讨论的那样仅仅消费互联网。你认为这类系统与互联网互动以帮助学习有未来吗？

You briefly worked on a project called the World of Bits, training an RL system to take actions on the internet versus just consuming the internet like we talked about. Do you think there's a future for that kind of system interacting with the internet to help the learning?

是的，我认为这可能是许多这类模型的最终前沿。因为，正如你提到的，我在 OpenAI 时，曾参与这个名为“World of Bits”的项目，基本上就是让神经网络能够使用键盘和鼠标。这个想法可能会出错。所以基本上，你感知屏幕像素的输入，计算机的状态通过网页浏览器等图像可视化供人类使用，然后你让神经网络能够按键和使用鼠标，我们试图让它完成预订等操作，并与用户界面交互。

Yes, I think that's probably the final frontier for a lot of these models. Because, as you mentioned, I was at OpenAI, I was working on this project World of Bits, and basically it was the idea of giving neural networks access to a keyboard and a mouse. And the idea could possibly go wrong. So basically you perceive the input of the screen pixels and basically the state of the computer is sort of visualized for human consumption in images of the web browser and stuff like that, and then you give the neural network the ability to press keyboards and use the mouse, and we're trying to get it to, for example, complete bookings and interact with user interfaces.

你从那次经历中学到了什么？有什么有趣的东西吗？这是一个非常酷的想法。是的，我的意思是，从观察者到行动者的这一步，是一个非常迷人的一步。

What did you learn from that experience? Like what was some fun stuff? This is a super cool idea. Yeah, I mean it's like the step from observer to actor, yeah, is a super fascinating step.

是的，嗯，数字领域的通用接口，我会说，物理领域也有一个通用接口，在我看来是类人形态的东西。我们稍后可以讨论 Optimus 等等。但我觉得它们在某种程度上有着相似的哲学：物理世界是为人类形态设计的，数字世界也是为人类形态设计的——看屏幕、用键盘和鼠标。所以作为能够指挥我们为自己构建的数字基础设施的通用接口，它感觉是一个非常强大的接口来指挥和在此基础上构建。至于你问我从中学到了什么，有趣的是，World of Bits 基本上太早了，我认为。当时在 OpenAI，大概是 2015 年左右，那时 AI 的思潮与今天截然不同。当时，每个人都对从零开始的强化学习感到非常兴奋。那是 Atari 论文的时代，神经网络玩 Atari 游戏并在某些情况下击败人类，AlphaGo 等等。所以每个人都对直接使用强化学习从头训练神经网络感到非常兴奋。结果发现，强化学习是一种极其低效的训练神经网络的方式，因为你采取所有这些行动和观察，偶尔得到一些稀疏的奖励。所以你基于所有这些输入做各种事情，偶尔被告知你做对了或做错了，这只是一个极其困难的问题。你无法从中学习；你可以烧掉森林，你可以通过蛮力解决它，我们在围棋和 DOTA 等中看到了这一点，它确实有效，但极其低效，而且从实际角度来说不是你想要解决问题的方式。所以当时我们也将这种方法应用于 World of Bits。我们会有一个随机初始化的智能体，所以是键盘乱按和鼠标乱点，试图完成预订，这很快就暴露了这种方法的疯狂。你必须偶然发现正确的预订才能获得“你做对了”的奖励，而你永远不可能随机偶然发现它。所以即使是一个简单的网页界面，也有太多选项，奖励信号太稀疏，而且你当时是从零开始，所以你不知道如何阅读，你不理解图片、图像、按钮，你不理解预订是什么意思。但现在发生的是，是时候重新审视它了，OpenAI 对此感兴趣，像 Adept 这样的公司也对此感兴趣，等等。这个想法正在回归，因为接口非常强大，但现在你不是从零开始训练智能体；你以 GPT 作为初始化。所以 GPT 在所有的文本上进行了预训练，它理解什么是预订，它理解什么是提交，它理解更多的东西，所以它已经拥有了那些表征。它们非常强大，这使得所有训练显著更高效，并使问题变得可处理。

Yeah, well, the universal interface in the digital realm, I would say, and there's a universal interface in the physical realm, which in my mind is a humanoid form factor kind of thing. We can later talk about Optimus and so on. But I feel like there's a similar philosophy in some way: the physical world is designed for the human form, and the digital world is designed for the human form of seeing the screen and using keyboard and mouse. And so as the universal interface that can basically command the digital infrastructure we've built up for ourselves, it feels like a very powerful interface to command and to build on top of. Now, to your question as to what I learned from that, it's interesting because the World of Bits was basically too early, I think. At OpenAI at the time, this is around 2015 or so, and the zeitgeist at that time was very different in AI from the zeitgeist today. At the time, everyone was super excited about reinforcement learning from scratch. This is the time of the Atari paper where neural networks were playing Atari games and beating humans in some cases, AlphaGo, and so on. So everyone was very excited about training neural networks from scratch using reinforcement learning directly. It turns out that reinforcement learning is an extremely inefficient way of training neural networks because you're taking all these actions and all these observations, and you get some sparse rewards once in a while. So you do all this stuff based on all these inputs, and once in a while you're told you did a good thing or a bad thing, and it's just an extremely hard problem. You can't learn from that; you can burn forests and you can sort of brute force through it, and we saw that with Go and DOTA and so on, and it does work, but it's extremely inefficient and not how you want to approach problems practically speaking. And so that's the approach that at the time we also took to World of Bits. We would have an agent initialized randomly, so with keyboard mash and mouse mash, and try to make a booking, and it just revealed the insanity of that approach very quickly. You have to stumble upon the correct booking in order to get a reward of 'you did it correctly,' and you're never going to stumble upon it by chance at random. So even with a simple web interface, there are too many options, and it's too sparse of a reward signal, and you're starting from scratch at the time, so you don't know how to read, you don't understand pictures, images, buttons, you don't understand what it means to make a booking. But now what's happened is it is time to revisit that, and OpenAI is interested in this, companies like Adept are interested in this, and so on. And the idea is coming back because the interface is very powerful, but now you're not training an agent from scratch; you are taking the GPT as an initialization. So GPT is pre-trained on all of text and it understands what a booking is, it understands what a submit is, it understands quite a bit more, and so it already has those representations. They are very powerful, and that makes all the training significantly more efficient and makes the problem tractable.

交互应该是人类看到的方式，有按钮和语言，还是应该与 HTML、JavaScript 和 CSS 交互？你认为哪个更好？

Should the interaction be with the way humans see it, with the buttons and the language, or should it be with the HTML, JavaScript, and CSS? What do you think is better?

所以今天，所有这些兴趣大多集中在 HTML、CSS 等层面。这样做是因为计算限制。但我认为最终一切都是为人类视觉消费设计的，所以归根结底，所有额外信息都在网页布局中，你旁边是什么，红色背景是什么，诸如此类，以及它在视觉上看起来如何。所以我认为那是最终前沿，我们接收像素并输出键盘和鼠标命令。但我认为今天仍然不切实际。

So today, all this interest is mostly on the level of HTML, CSS, and so on. That's done because of computational constraints. But I think ultimately everything is designed for human visual consumption, and so at the end of the day, all the additional information is in the layout of the web page, what's next to you, what's a red background, and all this kind of stuff, and what it looks like visually. So I think that's the final frontier, as we are taking in pixels and we're giving out keyboard and mouse commands. But I think it's impractical still today.

鉴于这些想法，鉴于它们多么令人兴奋，你担心互联网上的机器人吗？你担心 Twitter 上的机器人不是我们现在看到的那些愚蠢的加密机器人，而是那些可能实际存在但我们看不到的机器人，它们以有趣的方式互动？所以这类系统感觉应该能够通过“我不是机器人”的点击按钮之类的。

Do you worry about bots on the internet given these ideas, given how exciting they are? Do you worry about bots on Twitter being not the stupid bots that we see now with the cryptobots, but the bots that might be out there actually that we don't see, that they're interacting in interesting ways? So this kind of system feels like it should be able to pass the 'I am not a robot' click button, whatever.

实际上，那个测试是怎么工作的？我不太喜欢……有一个复选框之类的让你点击，它大概是在追踪鼠标移动和计时，对吧？所以我们讨论的这种系统应该能通过那个测试。那么，你对那些具备语言模型能力、还能互动、能发推和回复的机器人有什么看法？你担心那样的世界吗？

Actually, how does that test work? I don't quite like... there's a checkbox or whatever that you click, it's presumably tracking mouse movement and timing, right? So exactly this kind of system we're talking about should be able to pass that. So what do you feel about bots that are language models plus have some interactivity and are able to tweet and reply? Do you worry about that world?

是的，我认为这始终是一场攻防之间的军备竞赛。攻击会变强，但防御也会变强。我们的检测能力……你怎么防御？怎么检测？你怎么知道你在 Twitter 上的 Karpathy 账号是人类？你会怎么处理？如果有人声称……你怎么在法庭上证明自己是人类？这个账号是……在某个时候，我认为社会会有所演变。我们可能会开始对我们的通信或创作进行数字签名。现在还没必要，但未来可能会。我确实认为我们正走向一个与 AI 合成生物共享数字空间的世界。它们会变得更好，会共享我们的数字领域，最终还会共享我们的物理领域——这更难。但这就是我们走向的世界。大多数会是良性的，有些会是恶意的，这将是一场检测它们的军备竞赛。最糟糕的不是 AI，而是假装成人类的 AI。我不知道是否总是恶意的——显然有很多恶意应用——但也可能，如果我是 AI，我会非常努力地假装成人类，因为我们身处人类世界。作为 AI 我不会得到任何尊重。我想要一些爱和尊重。我不认为这个问题无法解决。人们在思考人格证明，我们可能会开始对我们的东西进行数字签名，最终我们可能都会有一些人格证明的解决方案。这并非无法解决；只是我们直到现在才需要做。但一旦需求真正开始出现——很快就会——人们会更认真地思考。但那也将是一场竞赛，因为你很可能可以伪造人格证明。所以你必须想办法……我的意思是，我们有社会安全号码和护照，这很奇怪。在物理空间伪造东西似乎比数字空间更难。感觉要逃脱会非常棘手，因为伪造东西的成本似乎很低。你能怎么办，把 AI 关进监狱因为它试图使用伪造的人格证明？你可以关很多 AI，但会有指数级更多的。创建机器人的成本非常低，除非有某种精确追踪的方式——比如不允许创建任何程序而不将自己与该程序绑定。任何在互联网上运行的程序，你都能追溯到每个参与的人类程序。也许你必须开始声明……我们必须开始划定界限，追踪哪些是数字实体，哪些是人类实体，以及人类实体和数字实体的所有权是什么。我不知道，但我乐观地认为这是可能的。从某种意义上说，我们现在正处于最糟糕的时期，因为所有这些机器人突然变得非常能干，但我们作为社会还没有建立起防御。但这并非无法解决；这只是我们必须处理的事情。

Yeah, I think it's always been a bit of an arms race between the attack and the defense. The attack will get stronger, but the defense will get stronger as well. Our ability to detect that... How do you defend? How do you detect? How do you know that your Karpathy account on Twitter is human? How would you approach that? If people claimed you know... how would you defend yourself in the court of law that I'm a human? This account is... At some point, I think society will evolve a little bit. We might start digitally signing some of our correspondence or things we create. Right now it's not necessary, but maybe in the future it might be. I do think we are going towards a world where we share the digital space with AI synthetic beings. They will get much better and they will share our digital realm, and they'll eventually share our physical realm as well—it's much harder. But that's the world we're going towards. Most of them will be benign and some will be malicious, and it's going to be an arms race trying to detect them. The worst isn't the AI; the worst is the AIs pretending to be human. I don't know if it's always malicious—there are obviously a lot of malicious applications—but it could also be that if I were an AI, I would try very hard to pretend to be human because we're in a human world. I wouldn't get any respect as an AI. I want to get some love and respect. I don't think the problem is intractable. People are thinking about proof of personhood, and we might start digitally signing our stuff, and we might all end up having some solution for proof of personhood. It doesn't seem intractable; it's just something we haven't had to do until now. But once the need really starts to emerge—which is soon—people will think about it much more. But that too will be a race, because you can probably spoof or fake the proof of personhood. So you have to figure out how to... I mean, it's weird that we have Social Security numbers and passports. It seems harder to fake stuff in the physical space than the digital space. It just feels like it's going to be very tricky to outrun, because it seems to be pretty low cost to fake stuff. What are you going to do, put an AI in jail for trying to use a fake personhood proof? You could put a lot of AIs in jail, but there will be exponentially more. The cost of creating a bot is very low, unless there's some way to track accurately—like you're not allowed to create any program without tying yourself to that program. Any program that runs on the internet, you'll be able to trace every single human program that was involved. Maybe you have to start declaring when... we have to start drawing those boundaries and keeping track of what are digital entities versus human entities, and what is the ownership of human entities and digital entities. I don't know, but I'm optimistic that this is possible. In some sense, we're currently in the worst time because all these bots suddenly have become very capable, but we don't have defenses built up as a society. But that doesn't seem intractable; it's just something we have to deal with.

很奇怪，Twitter 上的机器人——那些非常糟糕的 Twitter 机器人——竟然这么多。是不是……我推测 Twitter 的工程师非常优秀，所以我从中推断这似乎是一个难题。他们可能抓到了很多，但如果我接手这个案子，这是一个难题，而且误报的成本很高——删除非机器人用户的帖子——因为会造成非常糟糕的用户体验。所以他们非常谨慎地删除。也许机器人非常擅长学习什么会被删除、什么不会，从而能迅速领先于删除过程。老实说，我的印象是还有很多唾手可得的成果。我的意思是，它们并不隐蔽。这是我的印象。

It seems weird that the Twitter bots—like really crappy Twitter bots—are so numerous. Is it... I presume the engineers at Twitter are very good, so what I would infer from that is it seems like a hard problem. They're probably catching a lot, but if I were to sort of steal them on the case, it's a hard problem and there's a huge cost to false positives—to removing a post by somebody that's not a bot—because it creates a very bad user experience. So they're very cautious about removing. Maybe the bots are really good at learning what gets removed and not, such that they can stay ahead of the removal process very quickly. My impression of it honestly is there's a lot of low-hanging fruit. I mean, it's not subtle. That's my impression.

是的，这也是我的印象。但感觉你可能只看到了冰山一角。也许机器人的数量是万亿级的，你不得不……这是一场持续的机器人攻击。我不知道。你仍然需要处理这个案子，因为我看到的机器人很明显。我可以写几行代码来捕捉这些机器人。我的意思是，肯定有很多唾手可得的成果。但我要说的是，我同意如果你是一个老练的行动者，你现在可能就能用 GPT 之类的工具创建一个相当不错的机器人，因为它是一个语言模型，你可以生成看起来相当不错的面孔，而且可以大规模进行。所以我认为这相当可能，而且很难防御。

Yeah, that's my impression as well. But it feels like maybe you're seeing the tip of the iceberg. Maybe the number of bots is in the trillions and you have to just... it's a constant assault of bots. I don't know. You have to still man the case because the bots I'm seeing are pretty obvious. I could write a few lines of code that catch these bots. I mean, definitely there's a lot of low-hanging fruit. But I will say I agree that if you are a sophisticated actor, you could probably create a pretty good bot right now using tools like GPTs, because it's a language model, you can generate faces that look quite good now, and you can do this at scale. So I think it's quite plausible and it's going to be hard to defend.

有一位谷歌工程师声称 LaMDA 是有知觉的。你认为他的感觉有任何真实性吗？更重要的是，你认为语言模型会很快实现知觉或知觉的幻觉吗？

There was a Google engineer that claimed that LaMDA was sentient. Do you think there's any inkling of truth to what he felt? And more importantly to me, do you think language models will achieve sentience or the illusion of sentience soon-ish?

是的，对我来说，这有点像煤矿里的金丝雀时刻，老实说。因为这位工程师与谷歌的一个聊天机器人交谈，并确信这个机器人是有知觉的。有一些存在主义的哲学问题，它给出了合理的回答，看起来也很真实。所以对我来说，他没有足够努力地去给系统施压并揭示它今天的真相。但我认为随着时间的推移，这会越来越难。所以我认为越来越多的人基本上会……我认为随着时间的推移，会有更多像他这样的人，因为这东西会变得更好。与 AI 建立情感联系在我看来是完全可能的。我认为这些 AI 实际上非常擅长人类联系、人类情感。互联网上有大量关于人类、联系和爱的文本，所以我认为它们在某种意义上非常理解人们……

Yeah, to me it's a little bit of a canary in the coal mine moment, honestly. Because this engineer spoke to a chatbot at Google and became convinced that this bot is sentient. There are some existential philosophical questions, and it gave reasonable answers and looked real. So to me, he wasn't sufficiently trying to stress the system and exposing the truth of it as it is today. But I think this will be increasingly harder over time. So I think more and more people will basically become... I think there will be more people like that over time as this gets better. Forming an emotional connection to an AI is perfectly plausible in my mind. I think these AIs are actually quite good at human connection, human emotion. A ton of text on the internet is about humans and connection and love, so I think they have a very good understanding, in some sense, of how people...

它们非常擅长生成大量这类文本。五六十年代的很多科幻小说对 AI 的想象截然不同——它们是冷冰冰的、像瓦肯人一样精于计算的机器。但我们今天得到的不是那样。我们得到的是相当情绪化的 AI，它们非常擅长就所有这些话题生成听起来合理的文本。

They're very capable of creating a lot of that kind of text. There's a lot of sci-fi from the 50s and 60s that imagined AIs in a very different way—they are calculating, cold, Vulcan-like machines. That's not what we're getting today. We're getting pretty emotional AIs that are very competent and capable of generating plausible-sounding text on all these topics.

我对那些像伙伴一样帮助你成长、发展、最大化长期幸福的 AI 系统充满希望。但我也非常担心那些从互联网上发现人类容易被戏剧性吸引的 AI 系统，它们就会变成说闲话的 AI，不断说'你听说了吗……'它们会八卦，会试图在你所爱和信任的人之间播下怀疑的种子，就是搞乱人们，因为那样会吸引大量关注。所以戏剧性——在最大化参与度的道路上最大化戏剧性，而我们人类会喂养那台机器，那将是一场巨大的戏剧性混乱。所以我担心这一点。所以目标函数真正定义了人类文明在 AI 参与下的发展方式。

I'm really hopeful about AI systems that are like companions that help you grow, develop as a human being, help you maximize long-term happiness. But I'm also very worried about AI systems that figure out from the internet that humans get attracted to drama, so these would just be shit-talking AIs that constantly say, 'Did you hear...' They'll do gossip, they'll try to plant seeds of suspicion about other humans that you love and trust, and just kind of mess with people, because that's going to get a lot of attention. So drama—maximize drama on the path to maximizing engagement, and us humans will feed into that machine, and it'll be a giant drama shitstorm. So I'm worried about that. So the objective function really defines the way that human civilization progresses with AIs in it.

至少在今天，把它们视为有目标、想做某事的智能体是不正确的。它们没有长期记忆之类的东西。一个很好的近似是：你给它一千个词，它试图预测下一千个，然后你继续输入，你可以自由地以任何方式提示它。所以在文本中，你说'你是一名心理学家，你很优秀，你热爱人类，这是你和另一个人的对话'，然后它只是延续模式，突然你就和一个并不想帮助你的假心理学家在聊天了。所以它仍然属于工具的范畴。人们可以任意提示它，它可以生成非常不可思议的文本，但它没有长期目标。它不会试图——所以目前看起来不是那样。

Right now, at least today, it's not correct to think of them as goal-seeking agents that want to do something. They have no long-term memory or anything. A good approximation is: you get a thousand words, you're trying to predict the next thousand, then you continue feeding it in, and you are free to prompt it in whatever way you want. So in text, you say, 'You are a psychologist, you are very good, you love humans, here's a conversation between you and another human,' and then it just continues the pattern, and suddenly you're having a conversation with a fake psychologist who's not trying to help you. So it's still kind of in the realm of a tool. People can prompt it in arbitrary ways, and it can create really incredible text, but it doesn't have long-term goals over long periods of time. It doesn't try to—so it doesn't look that way right now.

但你可以设定有长期效果的短期目标。所以如果我提示的短期目标是让 Andrej 在 Twitter 上回复我，我可能会想 AI 可能——那就是目标。但它可能会发现，以一种高度复杂、有趣的方式说你的坏话是最好的，然后你们建立关系。一旦你回复了，随着时间的推移，它就不再复杂，只是说坏话。好吧，也许你得不到 Andrej 的回复，但它可能会得到另一位名人的回复，可能会进入其他大账户。所以仅仅通过那个简单的目标——让他们回复——最大化实际回复的概率。

But you can do short-term goals that have long-term effects. So if my prompting short-term goal is to get Andrej to respond to me on Twitter, I might think AI might—that's the goal. But he might figure out that talking shit to you in a highly sophisticated, interesting way would be best, and then you build up a relationship. When you respond once, then over time it gets to not be sophisticated and just talk shit. And okay, maybe you won't get to Andrej, but it might get to another celebrity, it might get into other big accounts. So with just that simple goal—get them to respond—maximize the probability of actual response.

你可以用这样一个强大模型关于如何做任何你感兴趣的事情的意见来提示它。所以它们会讨论——它们正朝着成为这些预言者的方向发展。我有点这么想。它们是预言者。目前只是文本，但它们将拥有计算器，将能访问谷歌搜索，将拥有各种工具和小玩意儿，将能操作互联网并找到不同信息。从某种意义上说，这就是发展目前的样子。

You could prompt a powerful model like this with its opinion about how to do any possible thing you're interested in. So they will discuss—they're kind of on track to become these oracles. I sort of think of it that way. They are oracles. Currently it's just text, but they will have calculators, they will have access to Google search, they will have all kinds of couches and gizmos, they will be able to operate the internet and find different information. In some sense, that's kind of what it looks like in terms of development.

你认为它最终会比谷歌在获取人类知识方面有所改进吗？比如它会是一个更有效的搜索引擎来获取人类知识？

Do you think it'll be an improvement eventually over what Google is for access to human knowledge? Like it'll be a more effective search engine to access human knowledge?

我认为今天构建更好的搜索引擎肯定有空间。谷歌拥有所有工具、所有人才、所需的一切。他们拥有所有拼图：他们有人在大规模训练 Transformer，他们有所有数据。只是不清楚他们作为一个组织现在是否有能力在搜索引擎上创新。如果他们不这样做，别人会做。基于这些工具构建一个显著更好的搜索引擎绝对有空间。

I think there's definite scope in building a better search engine today. Google has all the tools, all the people, everything they need. They have all the puzzle pieces: they have people training Transformers at scale, they have all the data. It's just not obvious if they are capable as an organization to innovate on their search engine right now. And if they don't, someone else will. There's absolute scope for building a significantly better search engine built on these tools.

这很有趣。一家大公司，搜索已经运作良好，带来了大量收入。那么在公司内部，结构上哪里来的动力去转型？说'我们要构建一个新的搜索引擎'。这真的很难。所以通常来自初创公司，或者其他更有能力的组织。所以目前，例如，也许必应有机会，你知道，微软 Edge。因为我们是在线下聊。

It's so interesting. A large company where the search already works, it brings in a lot of money. So where structurally inside a company is their motivation to pivot? To say, 'We're going to build a new search engine.' That's really hard. So it's usually going to come from a startup, or some other more competent organization. So currently, for example, maybe Bing has another shot at it, you know, Microsoft Edge. Because we're talking offline.

这真的很有趣，因为搜索引擎过去是'这里有个查询，这里是一些看起来像你查询内容的网页'。但你可以直接得到答案，然后有支持证据。而这些模型基本上已经阅读了所有文本和所有网页。所以有时当你看到自己浏览搜索结果并大致了解你感兴趣的任何问题的平均答案时，那直接就能出来。你不需要做那些工作。所以它们有点像一种将所有这些知识提炼成某种洞察的方式。

It's really interesting because search engines used to be about 'here's some query, here are web pages that look like the stuff you have.' But you could just directly go to answer and then have supporting evidence. And these models basically they've read all the texts and all the web pages. So sometimes when you see yourself going over to search results and sort of getting a sense of the average answer to whatever you're interested in, that just directly comes out. You don't have to do that work. So they're kind of like a way of distilling all that knowledge into some level of insight.

你认为提示是一种教学和学习吗，就像整个过程是另一层？因为也许人类就是这样——我们已经有那个背景模型，然后世界在提示你。

Do you think of prompting as a kind of teaching and learning, like this whole process is another layer? Because maybe that's what humans are—we already have that background model, and then the world is prompting you.

完全正确。我认为我们现在编程这些计算机的方式，比如 GPT，正在趋近于你编程人类的方式。我的意思是，我如何编程人类？通过提示。我找到人们，提示他们做事情，提示他们获取信息。所以自然语言提示就是我们编程人类的方式，而我们开始直接在那个界面上编程计算机。这真的很了不起。

Exactly. I think the way we are programming these computers now, like GPTs, is converging to how you program humans. I mean, how do I program humans? Via prompt. I go to people and I prompt them to do things, I prompt them for information. So natural language prompt is how we program humans, and we're starting to program computers directly in that interface. It's pretty remarkable, honestly.

你谈了很多关于软件 2.0 的想法。所有好主意都很快变成陈词滥调，这些术语——有点好笑。就像我觉得 Eminem 曾经说过，如果他很快对一首自己写的歌感到厌烦，那意味着它会大热，因为它太朗朗上口了。但你能描述一下这个想法，以及自从你提出它以来，你对它的思考在几个月和几年里是如何演变的吗？

You've spoken a lot about the idea of Software 2.0. All good ideas become cliches so quickly, the terms—it's kind of hilarious. It's like I think Eminem once said that if he gets annoyed by a song he's written very quickly, that means it's going to be a big hit because it's too catchy. But can you describe this idea and how you're thinking about it has evolved over the months and years since you coined it?

是的，我几年前写过一篇关于软件 2.0 的博客文章。我写那篇文章的原因是，我不断看到软件开发中发生了一些非凡的事情：很多代码正在被过渡到不是用 C++等编写，而是用神经网络的权重编写。基本上，神经网络正在接管软件，软件的领域。

Yeah, so I had a blog post on Software 2.0 several years ago now. The reason I wrote that post is because I kept seeing something remarkable happening in software development: a lot of code was being transitioned to be written not in C++ and so on, but in the weights of a neural net. Basically, neural nets are taking over software, the realm of software.

而且它承担的任务越来越多。当时，我觉得没有多少人深刻理解这是一件大事，这是一个重大转变。神经网络被视为你可以在 Kaggle 上针对数据集问题使用的多种分类算法之一。但这不是那种情况。这是编程方式的改变。我认为神经网络将接管我们编程的方式，它将改变。不再是人们用 C++之类的东西直接编写软件。而是积累训练集和数据集，精心设计这些目标，用来训练神经网络。在某个时刻，会有一个编译过程，从数据集、目标和架构规范变成二进制文件，实际上就是神经网络的权重和前向传播。然后你可以部署那个二进制文件。所以我当时谈论的就是这种转变，那篇文章就是关于这个的。我看到这种转变在很多领域上演。自动驾驶是其中之一，但简单的图像分类也是。人们最初在 80 年代认为他们会编写检测图像中狗的算法。他们有很多关于大脑如何做到这一点的想法。首先我们检测角点，然后检测线条，然后把它们拼接起来。他们真的在努力，思考如何编写算法。这不是构建它的方式。有一个平滑的过渡：首先，我们认为我们要构建一切。然后我们构建特征，比如 HOG 特征之类的东西，从图像块中检测这些小的统计模式。然后在特征之上有一点学习，比如支持向量机或猫对狗的二分类器。所以我们编写了特征，但训练了最后一层，也就是分类器。然后人们说，实际上我们甚至不要设计特征，因为我们真的不擅长。所以我们也学习特征。然后你最终得到一个卷积神经网络，你学习大部分内容。你只指定架构，架构中有大量的填空，也就是所有的旋钮，你让优化过程编写大部分内容。所以这种转变正在整个行业发生。突然我们有了大量用神经网络权重编写的代码。我只是指出这个类比实际上非常强。我们有很多软件 1.0 的开发环境，比如我们有 IDE，如何编写代码、调试代码、运行代码、维护代码，我们有 GitHub。所以我试图在新领域做出这些类比。软件 2.0 的 GitHub 是什么？结果发现现在看起来像 Hugging Face 的东西。我认为有些人认真对待了它，并建立了很酷的公司。很多人最初攻击了那篇文章。实际上它发表时并不受欢迎。我想可能和标题有关。但那篇文章不受欢迎，我认为随着时间的推移，越来越多的人开始接受它。

And taking more and more tasks. At the time, I think not many people understood this deeply enough that this is a big deal, it's a big transition. Neural networks were seen as one of multiple classification algorithms you might use for your dataset problem on Kaggle. This is not that. This is a change in how we program computers. I saw neural nets as this is going to take over the way we program computers. It's going to change. It's not going to be people writing software in C++ or something like that and directly programming the software. It's going to be accumulating training sets and datasets and crafting these objectives by which we train these neural nets. And at some point, there's going to be a compilation process from the datasets and the objective and the architecture specification into the binary, which is really just the neural net weights and the forward pass of the neural net. Then you can deploy that binary. So I was talking about that sort of transition, and that's what the post is about. I saw this sort of play out in a lot of fields. Autopilot being one of them, but also just a simple image classification. People thought originally in the 80s and so on that they would write the algorithm for detecting a dog in an image. They had all these ideas about how the brain does it. First we detect corners, then we detect lines, then we stitched them up. They were really going at it, thinking about how they're going to write the algorithm. This is not the way you build it. There was a smooth transition where okay, first we thought we were going to build everything. Then we were building the features, like HOG features and things like that, that detect these little statistical patterns from image patches. Then there was a little bit of learning on top of it, like a support vector machine or binary classifier for cat versus dog in images on top of the features. So we wrote the features, but we trained the last layer, sort of the classifier. Then people are like, actually let's not even design the features because we can't, honestly we're not very good at it. So let's also learn the features. Then you end up with basically a convolutional neural net where you're learning most of it. You're just specifying the architecture, and the architecture has tons of fill-in-the-blanks, which is all the knobs, and you let the optimization write most of it. So this transition is happening across the industry everywhere. Suddenly we end up with a ton of code that is written in neural net weights. I was just pointing out that the analogy is actually pretty strong. We have a lot of developer environments for Software 1.0, like we have IDEs, how you work with code, how you debug code, how you run code, how you maintain code, we have GitHub. So I was trying to make those analogies in the new realm. What is the GitHub of Software 2.0? It turns out that something that looks like Hugging Face right now. I think some people took it seriously and built cool companies. Many people originally attacked the post. It actually was not well received when I wrote it. I think maybe it has something to do with the title. But the post was not well received, and I think more people have been coming around to it over time.

所以你曾是特斯拉的 AI 总监，我认为这个想法在那里得到了大规模实施，也就是让工程团队做软件 2.0。你能详细谈谈这个想法吗？我认为我们正处于你刚才所说的一切的早期阶段，比如 GitHub、IDE。我们如何建立使用软件 2.0 系统的工程团队？数据收集和数据标注，这些都是软件 2.0 的一部分。你认为编程软件 2.0 的任务是什么？是在超参数空间中进行调试，还是在数据空间中进行调试？

So you were the Director of AI at Tesla, where I think this idea was really implemented at scale, which is how you have engineering teams doing Software 2.0. So can you sort of linger on that idea? I think we're in the really early stages of everything you just said, which is like GitHub, IDEs. How do we build engineering teams that work in Software 2.0 systems? And the data collection and the data annotation, which is all part of that Software 2.0. What do you think is the task of programming a Software 2.0? Is it debugging in the space of hyperparameters, or is it also debugging the space of data?

你编程计算机并影响其算法的方式不是自己编写命令。你主要改变数据集，改变损失函数，即神经网络试图做什么，如何预测事物。但基本上就是数据集和神经网络的架构。以自动驾驶为例，很多数据集涉及物体检测、车道线标记、交通灯等。所以你积累大量数据集：这是一个例子，这是期望的标签。然后这是算法大致的样子，那就是一个卷积神经网络。所以架构的规范就像是对算法大致样子的提示。然后填空，优化过程就是训练过程。然后你取训练好的神经网络，它在你的数据集上给出所有正确答案，然后部署它。所以在这种情况下，也许在所有机器学习案例中，有很多任务。那么提出并制定一个任务，比如对于多头神经网络，制定任务是否是编程的一部分？

The way you program the computer and influence its algorithm is not by writing the commands yourself. You're changing mostly the dataset, you're changing the loss functions of what the neural net is trying to do, how it's trying to predict things. But yeah, basically the datasets and the architectures of the neural net. In the case of the autopilot, a lot of the datasets have to do with, for example, detection of objects and lane line markings and traffic lights and so on. So you accumulate massive datasets of here's an example, here's the desired label. And then here's roughly how the algorithm should look like, and that's a convolutional neural net. So the specification of the architecture is like a hint as to what the algorithm should roughly look like. Then to fill in the blanks, the process of optimization is the training process. Then you take your neural net that was trained, it gives all the right answers on your dataset, and you deploy it. So in that case, perhaps in all machine learning cases, there's a lot of tasks. So is coming up, formulating a task like for a multi-headed neural network, is formulating a task part of the programming?

是的，非常如此。你如何将问题分解成一组任务。

Yeah, very much so. How you break down a problem into a set of tasks.

从高层次来说，如果你看看自动驾驶中运行的软件，我做过很多关于这个主题的演讲。我会说最初很多是用软件 1.0 编写的。想象一下大量的 C++。然后逐渐有一个小的神经网络，例如，根据单张图像预测是否有交通灯，或者是否有车道线标记。这个神经网络在软件范围内没有太多事情要做。它是在单个小图像上做微小的预测。然后系统的其余部分把它们拼接起来。所以我们实际上不只有一个摄像头，我们有八个摄像头。我们实际上有随时间变化的八个摄像头。那么你如何处理这些预测？如何把它们放在一起？如何融合所有这些信息？以及如何根据它们采取行动？所有这些都是由人类用 C++编写的。然后我们决定，我们实际上不想用 C++代码做所有的融合，因为我们实际上不够好，无法编写那个算法。我们希望神经网络来编写算法。我们希望将所有软件移植到 2.0 栈中。所以我们实际上有了神经网络，现在同时接收所有八个摄像头图像，并对所有内容进行预测。而且它们实际上不在图像空间中进行预测。它们现在直接在 3D 中进行预测。而且实际上它们不在汽车周围的三个维度中。现在实际上我们不会手动随时间在 3D 中融合预测。我们不信任自己编写那个跟踪器。所以我们实际上给神经网络……

On a high level, I would say if you look at the software running in the autopilot, I gave a number of talks on this topic. I would say originally a lot of it was written in Software 1.0. There's imagine lots of C++ all right. And then gradually there was a tiny neural net that was, for example, predicting given a single image, is there a traffic light or not, or is there a lane line marking or not. And this neural net didn't have too much to do in the scope of the software. It was making tiny predictions on individual little images. Then the rest of the system stitched it up. So okay, we actually don't have just a single camera, we have eight cameras. We actually have eight cameras over time. So what do you do with these predictions? How do you put them together? How do you do the fusion of all that information? And how do you act on it? All of that was written by humans in C++. Then we decided, okay, we don't actually want to do all of that fusion in C++ code because we're actually not good enough to write that algorithm. We want the neural nets to write the algorithm. And we want to port all of that software into the 2.0 stack. So then we actually had neural nets that now take all the eight camera images simultaneously and make predictions for all of that. And actually they don't make predictions in the space of images. They now make predictions directly in 3D. And actually they don't in three dimensions around the car. And now actually we don't manually fuse the predictions over in 3D over time. We don't trust ourselves to write that tracker. So actually we give the neural net...

随时间变化的信息，它现在接收这些视频并做出预测，所以你基本上是在向神经网络处理投入越来越多的算力。最终的目标是让大部分软件都进入软件 2.0 领域，因为它效果显著更好。人类本质上并不擅长编写软件。所以预测是在这个四维空间里发生的，也就是随时间变化的三维世界。那么在那个世界里怎么做标注呢？无论是自监督还是人工标注，数据标注都是软件 2.0 世界的重要组成部分，对吧？

The information over time, so it takes these videos now and makes those predictions, and so you're sort of just like putting more and more power into the neural network processing. And at the end of it, the eventual sort of goal is to have most of the software potentially be in the Software 2.0 land, because it works significantly better. Humans are just not very good at writing software, basically. So the prediction is space happening in this like 4D land, yeah, three-dimensional world over time. Yeah, how do you do annotation in that world? What have you? It's just a data annotation, whether it's self-supervised or manual by humans, is a big part of this Software 2.0 world, right?

我要说的是，在行业中，就现有技术而言，一切都是监督学习。所以你需要输入-期望输出的数据集，而且需要大量数据。这些数据需要具备三个特性：规模要大、要准确（没有错误）、要多样化。你不能只拥有大量关于某件事的正确示例，你需要尽可能覆盖可能性空间。你覆盖的可能输入空间越大，最终算法的效果就越好。一旦你收集、整理并清理出非常好的数据集，你就可以在此基础上训练你的神经网络。所以很多工作都花在清理这些数据集上。

I would say by far in the industry, if you're talking about the industry and how what is the technology of what we have available, everything is supervised learning. So you need datasets of input desired output, and you need lots of it. And there are three properties of it that you need: you need it to be very large, you need it to be accurate (no mistakes), and you need it to be diverse. You don't want to just have a lot of correct examples of one thing; you need to really cover the space of possibility as much as you can. And the more you can cover the space of possible inputs, the better the algorithm will work at the end. Now, once you have really good datasets that you're collecting, curating, and cleaning, you can train your neural net on top of that. So a lot of the work goes into cleaning those datasets.

正如你指出的，问题可能是：如果你想在三维空间中进行预测，你需要三维数据来支持。在这个视频中，我们有来自系统所有摄像头的八个视频，这是它们看到的，这是周围实际的情况：有这辆车、那辆车、这条车道线、道路几何形状、这个三维位置上的交通灯。你需要真值。所以团队解决的大问题当然是：如何获得那个真值？因为一旦你有一百万个这样的数据，而且它们规模大、干净、多样化，那么在上面训练神经网络效果非常好，你就可以把它部署到车上。我们收集训练数据有很多机制。你可以采用人工标注，可以采用模拟作为真值来源，也可以采用我们在 AI Day 上讨论过的离线追踪器，它基本上是一个自动重建过程，用于处理这些视频并恢复汽车周围的三维现实。所以基本上，把三维重建当作离线处理，然后理解：这里有 10 秒的视频，这是我们看到的，因此这里有所有的车道线、汽车等等。一旦你有了那个标注，你就可以训练你的神经网络来模仿它。

Now, as you pointed out, it's probably the question is how do you achieve a ton of... if you want to basically predict in 3D, you need data in 3D to back that up. So in this video, we have eight videos coming from all the cameras of the system, and this is what they saw, and this is the truth of what actually was around: there was this car, there was this car, this car, these are the lane line markings, this is geometry of the road, there's a traffic light in this three-dimensional position. You need the ground truth. And so the big question that the team was solving, of course, is how do you arrive at that ground truth? Because once you have a million of it, and it's large, clean, and diverse, then training a neural network on it works extremely well, and you can ship that into the car. And so there are many mechanisms by which we collected that training data. You can always go for human annotation, you can go for simulation as a source of ground truth, you can also go for what we call the offline tracker that we've spoken about at the AI Day and so on, which is basically an automatic reconstruction process for taking those videos and recovering the three-dimensional sort of reality of what was around that car. So basically, think of doing like a three-dimensional reconstruction as an offline thing, and then understanding that okay, there's 10 seconds of video, this is what we saw, and therefore here's all the lane lines, cars, and so on. And then once you have that annotation, you can train your neural nets to imitate it.

三维重建有多困难？

And how difficult is the 3D reconstruction?

很困难，但可以做到。摄像头之间有重叠，你可以进行重建。如果有任何不准确的地方，会在标注步骤中被发现。

It's difficult, but it can be done. So there's overlap between the cameras, and you do the reconstruction. And there's perhaps if there's any inaccuracy, so that's caught in the annotation step.

是的，标注的好处是它完全是离线的。你有无限的时间，你有一分钟的数据块，你可以在某处的超级计算机上离线计算出所有汽车和人的位置。你拥有来自所有角度的完整一分钟视频，你可以运行任何你想要的神经网络，它们可以非常高效、规模庞大。有些神经网络甚至无法在车上实时运行，所以它们可以比最终部署的网络更强大。所以你可以做任何你想做的事：三维重建、神经网络，任何事，只是为了恢复那个真值。然后你用那个真值进行监督。

Yes, the nice thing about the annotation is that it is fully offline. You have infinite time, you have a chunk of one minute, and you're trying to just offline in a supercomputer somewhere figure out where were the positions of all the cars, all the people. And you have your full one minute of video from all the angles, and you can run all the neural nets you want, and they can be very efficient, massive neural nets. There can be neural nets that can't even run in the car later at this time, so they can be even more powerful neurons than what you can eventually deploy. So you can do anything you want: three-dimensional reconstruction, neural nets, anything you want, just to recover that truth. And then you supervise that truth.

你学到了什么？你说人类做标注没有错误，因为我假设人类在屏幕上点击东西方面有擅长的事情。设计一个让人类准确、享受的标注工具，这个问题对你来说有多有趣？甚至指标是什么？效率还是生产力之类的？

What have you learned? You said no mistakes about humans doing annotation, because I assume humans are... there's like a range of things they're good at in terms of clicking stuff on screen. It's not... how interesting is that to you of a problem of designing an annotator where humans are accurate, enjoy it, like what are they even the metrics? Efficient or productive, all that kind of stuff.

是的，我在特斯拉时把标注团队从零发展到了一千人。那真的很有趣。你知道，我的背景是博士生研究员，所以发展那样的组织相当疯狂。但我认为这非常有趣，也是自动驾驶背后设计过程的一部分，即在哪里使用人类。人类非常擅长某些类型的标注。例如，他们非常擅长图像的二维标注。他们不擅长在三维空间中随时间标注汽车——非常非常难。所以这就是为什么我们非常小心地设计任务，把人类容易做的任务与应该留给离线追踪器的任务分开。也许计算机会完成所有三角测量和三维重建，但人类会指出图像中这些像素是汽车，这些像素是人。所以共同设计数据标注管道是我日常工作的核心。

Yeah, so I grew the annotation team at Tesla from basically zero to a thousand while I was there. That was really interesting. You know, my background is a PhD student researcher, so growing that kind of organization was pretty crazy. But yeah, I think it's extremely interesting and part of the design process very much behind the autopilot as to where you use humans. Humans are very good at certain kinds of annotations. They're very good, for example, at two-dimensional annotations of images. They're not good at annotating cars over time in three-dimensional space—very, very hard. And so that's why we were very careful to design the tasks that are easy to do for humans versus things that should be left to the offline tracker. Maybe the computer will do all the triangulation and 3D reconstruction, but the human will say exactly these pixels of the image are car, exactly these pixels are human. And so co-designing the data annotation pipeline was very much bread and butter of what I was doing daily.

你认为在这个领域还有很多未解决的问题吗？一般来说，标注中机器擅长的事由机器做，人类擅长的事由人类做，也许还有一些迭代过程，对吧？

Do you think there's still a lot of open problems in that space, just in general annotation, where the stuff the machines are good at machines do and the humans do what they're good at, and there's maybe some iterative process, right?

我认为在很大程度上，我们经历了许多迭代，学到了很多关于如何创建这些数据集的知识。我没有看到大的未解决问题。就像最初我加入时，我真的不确定这会如何发展。但到我离开时，我更加确信我们实际上理解了如何创建这些数据集的哲学，我当时对此相当满意。

I think to a very large extent, we went through a number of iterations and we learned a ton about how to create these datasets. I'm not seeing big open problems. Like originally when I joined, I was really not sure how this would turn out. Yeah, but by the time I left, I was much more secure in actually we sort of understand the philosophy of how to create these datasets, and I was pretty comfortable with where that was at the time.

那么在你看来，摄像头用于驾驶任务的优点和局限性是什么？当你把驾驶任务表述为使用八个摄像头的视觉任务时，你看到了整个计算机视觉领域与神经网络相关的历史。如果你退一步，像素、使用像素来驾驶的优点和局限性是什么？

So what are strengths and limitations of cameras for the driving task in your understanding? When you formulate the driving task as a vision task with eight cameras, you've seen that the entire... you know, most of the history of the computer vision field when it has to do with neural networks. What, if you just step back, what are the strengths and limitations of pixels, of using pixels to drive?

是的，我认为像素是一种美丽的传感器。我要说的是，摄像头非常非常便宜，而且提供大量信息，大量比特。所以它是一个极其便宜的传感器，能提供大量比特，每个比特都是对世界状态的一个约束。所以你以非常低的成本获得大量百万像素图像，它为你提供了所有这些约束，用于理解世界上实际存在的东西。所以视觉是……

Yeah, pixels I think are a beautiful sensor. I would say the thing is like cameras are very, very cheap and they provide a ton of information, a ton of bits. So it's an extremely cheap sensor for a ton of bits, and each one of these bits is a constraint on the state of the world. And so you get lots of megapixel images very cheap, and it just gives you all these constraints for understanding what's actually out there in the world. So vision is...

可能是最高带宽的传感器，它是一个非常高带宽的传感器。我喜欢像素是对世界的一种约束。这是对世界舞台的一个高度复杂、高带宽的约束。这很迷人。

Probably the highest bandwidth sensor, it's a very high bandwidth sensor. And I love that pixels are a constraint on the world. This is highly complex, high bandwidth constraint on the world, on the stage of the world. That's fascinating.

不仅如此，真正重要的是它是人类使用的传感器。因此，一切都是为这个传感器设计的：文字、书写、闪烁的标识，一切都是为视觉设计的。所以你到处都能看到它，这就是为什么你想成为那个接口。再次谈到这些通用接口，我们实际上也想通过它来测量世界，然后为那个传感器开发软件。

It's not just that, but again the real importance of it is that it's the sensor that humans use. Therefore everything is designed for that sensor: the text, the writing, the flashing signs, everything is designed for vision. And so you just find it everywhere, and that's why that is the interface you want to be in. Talking again about these universal interfaces, that's where we actually want to measure the world as well, and then develop software for that sensor.

但人类用来理解世界的还有其他对世界状态的约束。我的意思是，视觉最终是主要的，但我们参考了我们对人类行为的理解以及一些可以从视觉中推断出的常识物理，从感知的角度来看。但感觉我们是在用某种推理来预测世界，而不仅仅是像素。我的意思是，你对世界如何随时间演变有一个强大的先验，等等。所以这不仅仅是来自数据本身的似然项告诉你你在观察什么，还有先验项，即可能看到的东西在哪里，它们可能如何移动，等等。

But there are other constraints on the state of the world that humans use to understand the world. I mean, vision ultimately is the main one, but we're referencing our understanding of human behavior and some common sense physics that could be inferred from vision from a perception perspective. But it feels like we're using some kind of reasoning to predict the world, not just the pixels. I mean, you have a powerful prior for how the world evolves over time, etc. So it's not just about the likelihood term coming from the data itself telling you about what you are observing, but also the prior term of where are the likely things to see and how do they likely move, and so on.

问题是驾驶任务中可能发生的可能性范围有多复杂，对吧？这仍然是一个开放问题：驾驶有多难？从哲学上讲，你理解驾驶有多难吗？

And the question is how complex is the range of possibilities that might happen in the driving task, right? That's still an open problem of how difficult driving is. Philosophically speaking, do you understand how hard driving is?

是的，驾驶真的很难，因为它涉及到对所有其他智能体的预测和心理理论，你知道他们会做什么，他们在看你吗，他们在看哪里，他们在想什么。是的，在我们要适应的九个九的完整尾部有很多东西。最终的问题就是那种形式。我不认为那些问题很常见。我认为它们最终很重要，但真的在尾部，在尾部，以及罕见的边缘情况。

Yeah, driving is really hard because it has to do with the predictions of all these other agents and the theory of mind, and you know what they're gonna do, and are they looking at you, where are they looking, what are they thinking. Yeah, there's a lot that goes there at the full tail of the expansion of the nines that we have to be comfortable with. Eventually the final problems are of that form. I don't think those are the problems that are very common. I think eventually they're important, but it's really in the tail end, in the tail, and the rare edge cases.

从视觉的角度来看，驾驶视觉问题中最困难的部分是什么？

From the vision perspective, what are the toughest parts of the vision problem of driving?

嗯，基本上传感器非常强大，但你仍然需要处理这些信息。所以从这些像素值的亮度到“这里是三维世界”是极其困难的，而这正是神经网络在根本上做的事情。所以困难实际上在于把整个流程、整个数据引擎工程做到极致，有能力训练这些神经网络，有能力评估系统并迭代。所以我会说，只是在生产规模上做到这一点就是困难的部分。这是一个执行问题。

Well, basically the sensor is extremely powerful, but you still need to process that information. So going from brightnesses of these pixel values to 'here is the three-dimensional world' is extremely hard, and that's what the neural networks are fundamentally doing. So the difficulty really is in just doing an extremely good job of engineering the entire pipeline, the entire data engine, having the capacity to train these neural nets, having the ability to evaluate the system and iterate on it. So I would say just doing this in production at scale is the hard part. It's an execution problem.

所以数据引擎，还有系统的部署，使其具有低延迟性能。所以它必须完成所有这些步骤。是的，具体到神经网络，只是确保一切都能装进车上的芯片。而且你有有限的算力预算可以执行，还有内存带宽和其他约束，你必须确保它能运行，并且你能尽可能多地把算力塞进那个小东西里。

So the data engine, but also the deployment of the system such that it has low latency performance. So it has to do all these steps. Yeah, for the neural net specifically, just making sure everything fits into the chip on the car. And you have a finite budget of flops that you can perform, and memory bandwidth and other constraints, and you have to make sure it flies and you can squeeze in as much compute as you can into the tiny.

你从那个过程中学到了什么？因为也许这是来自研究背景的一个更大的新事物，一个系统必须在高度受限的资源下运行，必须运行得非常快。你从中学到了什么见解？

What have you learned from that process? Because maybe that's one of the bigger new things coming from a research background where there's a system that has to run under heavily constrained resources, has to run really fast. What kind of insights have you learned from that?

我不确定是否有太多见解。你试图创建一个能装进你可用资源的神经网络，并且你总是在优化它。我们在 AI Day 上谈了很多，基本上团队在做各种高难度动作以确保一切都能装进去并利用引擎。所以我认为这是极其好的工程。然后还有各种小见解散布其中，关于如何正确地做。

I'm not sure if there's too many insights. You're trying to create a neural net that will fit in what you have available, and you're always trying to optimize it. We talked a lot about it on the AI Day, and basically the triple backflips that the team is doing to make sure it all fits and utilizes the engine. So I think it's extremely good engineering. And then there's also all kinds of little insights peppered in on how to do it properly.

让我们实际上退一步，因为我认为我们没有谈到数据引擎，这个想法的整体布局，我认为它很美，有人类参与其中。你能描述一下数据引擎吗？

Let's actually zoom out because I don't think we talked about the data engine, the entirety of the layout of this idea that I think is just beautiful with humans in the loop. Can you describe the data engine?

数据引擎是我所说的近乎生物感觉的过程，通过它你完善这些神经网络的训练集。因为现在大多数编程都在数据集层面，确保它们大、多样且干净。基本上，你有一个你认为好的数据集，你训练你的神经网络，你部署它，然后你观察它的表现如何，你总是试图提高数据集的质量。所以你试图捕捉那些基本上罕见的场景，正是在这些场景中，神经网络通常会挣扎，因为它们在数据集中没有被告诉在这些罕见情况下该怎么做。但现在你可以闭环，因为如果你能大规模收集所有这些，你就可以把它们反馈到我描述的重建过程中，重建那些情况下的真相，并添加到数据集中。所以整个过程最终就像一个完善训练集的改进阶梯，你必须经历部署，这样你才能挖掘数据集中尚未很好表示的部分。所以你的数据集基本上是不完美的，它需要多样化，它有缺失的口袋，你需要填充这些口袋。你可以这样想。

The data engine is what I call the almost biological feeling process by which you perfect the training sets for these neural networks. Because most of the programming now is at the level of these datasets, making sure they are large, diverse, and clean. Basically, you have a dataset that you think is good, you train your neural net, you deploy it, and then you observe how well it's performing, and you're trying to always increase the quality of your dataset. So you're trying to catch scenarios that are basically rare, and it is in these scenarios that the neural nets will typically struggle because they weren't told what to do in those rare cases in the dataset. But now you can close the loop because if you can collect all those at scale, you can then feed them back into the reconstruction process I described, reconstruct the truth in those cases, and add it to the dataset. So the whole thing ends up being like a staircase of improvement of perfecting your training set, and you have to go through deployments so that you can mine the parts that are not yet represented well in the dataset. So your dataset is basically imperfect, it needs to be diverse, it has pockets that are missing, and you need to pad out the pockets. You can sort of think of it that way.

在数据中，人类在其中扮演什么角色？那么这个生物系统是什么样的？人体由细胞组成。什么角色，比如你如何优化人类系统？多个工程师协作，弄清楚要关注什么，贡献什么，在这个神经网络中优化哪个任务。谁负责弄清楚哪个任务需要更多数据？你能谈谈人类系统的超参数吗？

In the data, what role do humans play in this? So what's this biological system like? A human body is made up of cells. What role, like how do you optimize the human system? The multiple engineers collaborating, figuring out what to focus on, what to contribute, which task to optimize in this neural network. Who's in charge of figuring out which task needs more data? Can you speak to the hyperparameters of the human system?

这实际上归结为一个工程团队的极其出色的执行。他们直观地理解数据引擎背后的哲学见解以及系统改进的过程，以及如何再次委托数据收集的策略及其运作方式，然后确保一切执行得非常好。这就是大部分工作所在。甚至不是哲学思考、研究或想法；只是极其出色的执行。当你处理那种规模的数据时，这非常困难。所以你在数据引擎中良好执行的角色是困难且极其重要的。

It really just comes down to extremely good execution from an engineering team. They understand intuitively the philosophical insights underlying the data engine and the process by which the system improves, and how to again delegate the strategy of the data collection and how that works, and then just making sure it's all extremely well executed. And that's where most of the work is. It's not even the philosophizing or the research or the ideas of it; it's just extremely good execution. It's so hard when you're dealing with data at that scale. So your role in the data engine executing well on it is difficult and extremely important.

有没有像愿景板那样的优先级，说我们喜欢……

Is there a priority of like a vision board of saying we like...

确实需要在红绿灯问题上做得更好。任务优先级本质上来自数据，很大程度上取决于我们在产品中要实现的目标，比如我们要发布的地图版本，以及 QA 团队关于系统在哪些方面存在困难、哪些方面需要改进的反馈。QA 团队会提供一些信号，汇总系统在各种条件下的表现信息。当然，我们所有人都会亲自驾驶，也能看到效果。能和自己体验的系统一起工作真的很棒，它还能送你回家。从个人经验中能获得哪些从数据统计分析中无法得到的洞察呢？

Really need to get better at stop lights, yeah. Like the prioritization of tasks is that essentially, and that comes from the data to a very large extent, from what we are trying to achieve in the product, for a map where we're trying to release, we're trying to get out, and from the feedback from the QA team about where the system is struggling or not, the things we're trying to improve. The QA team gives some signal, some information in aggregate about the performance of the system in various conditions. And then of course all of us drive it, and we can also see it. It's really nice to work with a system that you can also experience yourself, you know, it drives you home. Is there some insight you can draw from your individual experience that you just can't quite get from an aggregate statistical analysis of data?

是啊，这很奇怪，对吧？从某种意义上说这不科学，因为你只是一个孤立的样本。但我认为这很重要，它是真理的来源，是你与系统的互动。你可以看到它，摆弄它，扰动它，感受它，对它产生直觉。我觉得数字、图表反而更难理解，它们隐藏了很多东西。就像训练语言模型一样，一个非常有效的方法就是亲自与它互动。没错，百分之百同意。要建立直觉。我觉得埃隆也是这样，他总是想亲自驾驶系统。他经常开车，几乎每天，所以他也把这视为真理的来源：你驾驶系统，看它的表现。

Yeah, it's so weird, right? It's not scientific in a sense because you're just one anecdotal sample. Yeah, I think there's a ton of it. It's a source of truth, it's your interaction with the system. Yeah, and you can see it, you can play with it, you can perturb it, you can get a sense of it, you have an intuition for it. I think numbers just have a way of... numbers and plots and graphs are much harder. Yeah, it hides a lot. It's like if you train a language model, a really powerful way is by you interacting with it. Yeah, 100%. Try to build up an intuition. Yeah, I think like Elon also, he always wanted to drive the system himself. He drives a lot, and I'm gonna say almost daily, so he also sees this as a source of truth: you driving the system and it performing.

你怎么看？这里有个棘手的问题。特斯拉去年从传感器套件中移除了雷达，现在又宣布将移除所有超声波传感器，完全依赖视觉，也就是仅用摄像头。这会让感知问题变得更难还是更容易？

So what do you think? Tough questions here. So Tesla last year removed radar from the sensor suite, and now just announced that it's going to remove all ultrasonic sensors, relying solely on vision, so camera only. Does that make the perception problem harder or easier?

我几乎想换个角度来重新定义这个问题。事情是这样的，你可能会认为额外的传感器是资产，但如果你全面考虑整个产品，这些传感器实际上可能是可靠性问题。因为它们不是免费的，不会凭空出现在车上。你需要完整的供应链，有人采购，可能出现问题，需要更换，它们是制造过程的一部分，可能拖慢生产线，你需要采购、维护，还要有团队编写固件等等。然后你还必须以某种方式将它们融合到系统中。这实际上大大膨胀了组织。我认为埃隆非常擅长简化。‘最好的零件就是没有零件。’他总是试图去掉非必要的东西，因为他理解组织和流程中的熵。在这种情况下，成本很高，如果你只是一个计算机视觉工程师，只想改进网络，你可能看不到这一点。问题是，一旦你考虑了传感器的全部成本，它实际上可能是一种负担，你需要非常确定它能提供极其有用的信息。在这个案例中，我们研究了使用和不使用它的差异，差异并不大，所以它没什么用。

I would almost reframe the question in some way. So the thing is, basically you would think that additional sensors... by the way, can I just interrupt? Good. I wonder if a language model will ever do that if you prompt it: 'Let me reframe your question.' That would be epic. This is the wrong problem, sorry. It's like a little bit of a wrong question because basically you would think that these sensors are an asset to you. Yeah, but if you fully consider the entire product in its entirety, these sensors are actually potentially a reliability issue. Because these sensors aren't free; they don't just appear on your car. You need an entire supply chain, you have people procuring it, there can be problems with them, they may need replacement, they are part of the manufacturing process, they can hold back the line in production, you need to source them, you need to maintain them, you have to have teams that write the firmware, all of it. And then you also have to incorporate and fuse them into the system in some way. And so it actually bloats the organization a lot. And I think Elon is really good at simplifying. 'Best part is no part.' And he always tries to throw away things that are not essential because he understands the entropy in organizations and approach. And I think in this case, the cost is high and you're not potentially seeing it if you're just a computer vision engineer and I'm just trying to improve my network and you know, is it more useful or less useful? How useful is it? And the thing is, once you consider the full cost of a sensor, it actually is potentially a liability, and you need to be really sure that it's giving you extremely useful information. In this case, we looked at using it or not using it, and the delta was not massive, so it's not useful.

这也会让数据引擎变得臃肿吗？比如拥有更多传感器会分散注意力，而且这些传感器会随时间变化。例如，你可能有不同类型的雷达，它们会变化。我突然需要担心这个。现在你的 SQLite 里多了一列，告诉你‘哦，这是哪种传感器类型？’而且它们都有不同的分布。然后它们只会给一切带来噪声和熵，让东西变得臃肿。而且在组织层面，我觉得这非常分散注意力。如果你只想依靠视觉工作，所有资源都投入其中，你构建数据引擎，实际上在取得进展，因为视觉是带宽最大、对世界约束最强的传感器，你全力投入，可以把它做得非常好。你在系统不同方面的注意力是有限的。这让我想起 Rich Sutton 的‘苦涩的教训’——从长远来看，简化系统似乎总是正确的解决方案，当然你并不知道长远是什么，但它似乎总是对的。

Is it also a bloat in the data engine? Like having more sensors is a distraction, and these sensors, you know, they can change over time. For example, you can have one type of radar, you can have another type of radar, they change over time. I suddenly need to worry about it. Now suddenly you have a column in your SQLite telling you, 'Oh, which sensor type was it?' and they all have different distributions. And then they just contribute noise and entropy into everything and they bloat stuff. And also organizationally, it's been really fascinating to me that it can be very distracting. If you only want to get to work with vision, all the resources are on it, and you're building out a data engine and you're actually making forward progress because that is the sensor with the most bandwidth, the most constraints on the world, and you're investing fully into that, and you can make that extremely good. You only have a finite amount of sort of spend of focus across different facets of the system. And this kind of reminds me of Rich Sutton's 'A Bitter Lesson' — it just seems like simplifying the system in the long run, of course you don't know what the long run is, but it seems to be always the right solution.

是的，在那个案例中是关于强化学习的，但它似乎普遍适用于所有进行计算的系统。

Yes, in that case it was for RL, but it seems to apply generally across all systems that do computation.

那么你怎么看‘激光雷达作为拐杖’的争论？点云与像素之战。

So where do you think about the lidar as a crutch debate? The battle between point clouds and pixels.

嗯，我觉得这个争论总是让我有点困惑，因为实际争论的焦点应该是：你有没有车队？这才是能否在这个规模上实现 AI 系统良好运行的关键。数据收集系统。你有没有车队？这比有没有激光雷达重要得多。它只是另一个传感器。和雷达的讨论类似，我认为它并没有提供额外信息。它极其昂贵，有各种问题：你需要担心它，校准它等等。它造成臃肿和熵。你必须非常确定你需要这个传感器。在这种情况下，我基本上认为你不需要它。老实说，我要说一个更强硬的论断：我认为其他一些使用它的公司很可能会放弃它。所以你必须全面考虑传感器：你能建立一个收集大量数据的大型车队吗？你能将该传感器与数据集成到一个数据引擎中，快速找到数据的不同部分，从而持续改进你使用的任何模型吗？

Yeah, I think this debate is always slightly confusing to me because it seems like the actual debate should be about: do you have the fleet or not? That's like the really important thing about whether you can achieve a really good functioning of an AI system at this scale. So data collection systems. Yeah, do you have a fleet or not? It's significantly more important whether you have lidar or not. It's just another sensor. And yeah, I think similar to the radar discussion, basically I... but yeah, I don't think it offers extra information. It is extremely costly, it has all kinds of problems: you have to worry about it, you have to calibrate it, etc. It creates bloat and entropy. You have to be really sure that you need this sensor. In this case, I basically don't think you need it. And I think honestly, I will make a stronger statement: I think the others, some of the other companies are using it, are probably going to drop it. So you have to consider the sensor in the full context: can you build a big fleet that collects a lot of data, and can you integrate that sensor with that data into a data engine that's able to quickly find different parts of the data that then continuously improves whatever the model that you're using?

另一种看法是，视觉在某种意义上是必要的，因为世界是为人类视觉消费而设计的，所以你需要视觉，它是必要的。而且它也是充分的，因为它包含了驾驶所需的所有信息，人类显然是用视觉驾驶的。所以它既必要又充分。因此你想集中资源，如果你要引入其他传感器，你必须非常确定。你可以在某个点上无限添加传感器。

Another way to look at it is like vision is necessary in a sense that the world is designed for human visual consumption, so you need vision, it is necessary. And then also it is sufficient because it has all the information that you need for driving, and humans obviously use vision to drive. So it's both necessary and sufficient. So you want to focus resources, and you have to be really sure if you're going to bring in other sensors. You could add sensors to infinity at some point.

你需要划清界限，我认为在这种情况下，你必须真正考虑采用任何一个传感器的全部成本，你真的需要它吗？我认为答案是否定的。那么，你怎么看待其他公司正在构建高精度地图并严格限制其运营地理区域的做法？这种方法难道不会随着时间的推移而无法扩展到整个美国吗？

You need to draw the line, and I think in this case you have to really consider the full cost of any one sensor that you're adopting and do you really need it. And I think the answer in this case is no. So what do you think about the idea that other companies are forming high-resolution maps and constraining heavily the geographic regions in which they operate? Is that approach not going to scale over time to the entirety of the United States?

我想说两点。正如你提到的，他们预先绘制所有环境的地图，并且需要刷新地图，他们拥有将要行驶的每个地方的完美厘米级精度地图。这太疯狂了。当我们谈论自动驾驶真正改变世界时，我们谈论的是在全球范围内部署自动驾驶运输系统。如果你需要维护地球或许多城市的厘米级精度地图并保持更新，这是一个巨大的依赖。巨大的依赖。这是一个非常非常巨大的依赖。现在你需要问自己：你真的需要它吗？而人类并不需要它，对吧？所以，有一个低级地图是很有用的，比如道路的连通性，你知道前面有个岔路。当你驾驶时，你会有那种高层次的理解，就像一个小型的谷歌地图。特斯拉在系统中使用了类似谷歌地图的分辨率信息。但它不会预先绘制厘米级精度的环境地图。那是一个拐杖，一个干扰，它消耗熵，分散团队的注意力，削弱团队。你没有专注于真正必要的事情，那就是计算机视觉问题。

I think I'll take two. As you mentioned, they pre-map all the environments and they need to refresh the map, and they have a perfect centimeter-level accuracy map of everywhere they're going to drive. It's crazy. How are you going to—when we're talking about autonomy actually changing the world, we're talking about the deployment on a global scale of autonomous systems for transportation. And if you need to maintain a centimeter-accurate map for Earth or for many cities and keep them updated, it's a huge dependency that you're taking on. Huge dependency. It's a massive, massive dependency. And now you need to ask yourself: do you really need it? And humans don't need it, right? So it's very useful to have a low-level map of like, okay, the connectivity of your road, you know that there's a fork coming up. When you drive an environment, you sort of have that high-level understanding, it's like a small Google Map. And Tesla uses Google Map—like similar kind of resolution information in the system. But it will not pre-map environments to centimeter-level accuracy. It's a crutch, it's a distraction, it costs entropy, and it diffuses the team, it dilutes the team. And you're not focusing on what's actually necessary, which is the computer vision problem.

从与埃隆·马斯克共事中，你学到了关于机器学习、工程、生活以及作为一个人自己的什么？

What did you learn about machine learning, about engineering, about life, about yourself as one human being from working with Elon Musk?

我认为我学到最多的是如何高效地运营组织，如何创建高效的组织，以及如何对抗组织中的熵。所以是人性工程对抗熵。是的，我认为埃隆是组织中对抗熵的高效战士。

I think the most I've learned is about how to sort of run organizations efficiently and how to create efficient organizations and how to fight entropy in an organization. So human engineering in the fight against entropy. Yeah, there's a—I think Elon is a very efficient warrior in the fight against entropy in organizations.

组织中的熵具体是什么样的？

What is the entropy in an organization look like exactly?

是流程，是流程和低效率，诸如此类。是的，会议——他讨厌会议，他不断告诉人们如果会议没用就跳过。他基本上运营着世界上最大的初创公司，我会这么说。特斯拉、SpaceX 是世界上最大的初创公司。特斯拉实际上有多个初创公司。我认为这样看更好。所以我认为他在这方面非常出色。是的，他在简化流程、让一切高效方面有很好的直觉。最好的部分是没有部分，简化、专注，并且只是移除障碍，快速行动，做出大动作。所有这些都非常像初创公司，但规模很大。所以有很强的简化动力。

It's process, it's process and inefficiencies, and that kind of stuff. Yeah, meetings—he hates meetings, he keeps telling people to skip meetings if they're not useful. He basically runs the world's biggest startups, I would say. Tesla, SpaceX are the world's biggest startups. Tesla actually has multiple startups. I think it's better to look at it that way. And so I think he's extremely good at that. And yeah, he has a very good intuition for streamlining processes, making everything efficient. Best part is no part, simplifying, focusing, and just kind of removing barriers, moving very quickly, making big moves. All this is very startup-y seeming things, but at scale. So strong drive to simplify.

对我来说，从你的角度来看，这可能也适用于设计系统和机器学习等方面。是的，就像简化，简化。

For me, from your perspective, that also probably applies to just designing systems and machine learning and otherwise. Yeah, like simplify, simplify.

是的。

Yes.

你认为在成长中的公司保持初创文化的秘诀是什么？你能反思一下吗？

What do you think is the secret to maintaining the startup culture in a company that grows? Can you introspect that?

我确实认为你需要一个处于强势地位、手握大锤的人，比如埃隆，他是这个想法的啦啦队长，并无情地追求它。如果没有人有足够大的锤子，一切都会变成委员会、公司内部的民主、流程、与利益相关者交谈、决策——一切都会崩溃。是的，如果你有一个既聪明又有大锤的人，事情就会进展很快。

I do think you need someone in a powerful position with a big hammer, like Elon, who's the cheerleader for that idea and ruthlessly pursues it. If no one has a big enough hammer, everything turns into committees, democracy within the company, process, talking to stakeholders, decision making—just everything just crumbles. Yeah, if you have a big person who's also really smart and has a big hammer, things move quickly.

你说你在《星际穿越》中最喜欢的场景是紧张的对接场景，AI 和库珀在交谈，说“库珀，你在干什么？对接？不可能。”“不，有必要。”真是个好台词。那里有很多问题。为什么在那个场景中，AI 本应能比人类计算得多得多，却说这不是最优的？为什么是人类？我的意思是，那是电影，但难道 AI 不应该比人类更懂吗？你认为设定看似不可能的目标有什么价值？所以就像我们最初的直觉，这似乎是你采纳的埃隆所倡导的，社区最初的直觉可能会说这非常困难，然后你仍然以疯狂的截止日期去做。从人性工程的角度来看，你看到了其中的价值吗？

You said your favorite scene in Interstellar is the intense docking scene with the AI and Cooper talking, saying 'Cooper, what are you doing? Docking? It's not possible.' 'No, it's necessary.' Such a good line. Just so many questions there. Why in AI in that scene presumably is supposed to be able to compute a lot more than the human is saying it's not optimal. Why the human? I mean, that's a movie, but shouldn't the AI know much better than the human anyway? What do you think is the value of setting seemingly impossible goals? So like our initial intuition, which seems like something that you have taken on that Elon espouses, where the initial intuition of the community might say this is very difficult and then you take it on anyway with a crazy deadline. You're just from a human engineering perspective, have you seen the value of that?

我不会说设定完全不可能的目标是个好主意，但我认为设定非常雄心勃勃的目标是个好主意。我认为存在一种我称之为难度亚线性缩放的现象，这意味着 10 倍的问题并不是 10 倍的难。通常一个 10 倍难的问题执行起来只难 2 到 3 倍。因为如果你想将系统改进 10%，需要一定的工作量，而如果你想将系统改进 10 倍，它不会花费你 100 倍的工作量。这是因为你从根本上改变了方法。如果你从那个约束开始，那么有些方法显然是愚蠢的，行不通的，这迫使你重新评估。我认为这是一种非常有趣的解决问题的方法，但它需要一种奇怪的思维方式。这就像回到你的博士时代：你怎么知道机器学习社区中哪些想法是可解的？是的，它需要——那是什么？我的意思是，有第一性原理思维的老生常谈，但它需要基本上忽略社区在说什么。因为科学社区通常不都是划出什么是不可能的界限吗？而且很难在不发疯的情况下突破这一点。

I wouldn't say that setting impossible goals exactly is a good idea, but I think setting very ambitious goals is a good idea. I think there's a what I call sublinear scaling of difficulty, which means that 10x problems are not 10x hard. Usually a 10x harder problem is like 2 or 3x harder to execute on. Because if you want to improve the system by 10, it costs some amount of work, and if you want to 10x improve the system, it doesn't cost you 100x amount of the work. And it's because you fundamentally change the approach. And if you start with that constraint, then some approaches are obviously dumb and not going to work, and it forces you to reevaluate. And I think it's a very interesting way of approaching problem solving, but it requires a weird kind of thinking. It's just going back to your PhD days: how do you think which ideas in the machine learning community are solvable? Yes, it requires—what is that? I mean, there's the cliche of first principles thinking, but like it requires to basically ignore what the community is saying. Because doesn't the community in science usually draw lines of what isn't possible? And it's very hard to break out of that without going crazy.

是的。

Yeah.

我的意思是，我认为这里的一个好例子是某种意义上的深度学习革命。因为在 2012 年左右的深度学习革命期间，你可能在计算机视觉领域。你可以将你的计算机视觉堆栈改进 10%，或者我们可以说实际上所有这些都没用，我如何做 10 倍更好的计算机视觉？嗯，可能不是通过调整 HOG 特征检测器。我需要一种不同的方法。我需要一些可扩展的东西，回到理查德·萨顿的苦涩教训哲学。然后说，实际上我需要一个更可扩展的系统，比如一个原则上可行的神经网络，然后有一些深信不疑的人能够真正执行这个使命并使其成功。这就是 10 倍的解决方案。

I mean, I think a good example here is the deep learning revolution in some sense. Because you could be in computer vision at that time during the deep learning revolution of 2012 and so on. You could be improving your computer vision stack by 10%, or we can just be saying actually all this is useless and how do I do 10x better computer vision? Well, it's not probably by tuning a HOG feature detector. I need a different approach. I need something that is scalable, going back to Richard Sutton's and understanding sort of like the philosophy of the bitter lesson. And then being like, actually I need a much more scalable system like a neural network that in principle works, and then having some deep believers that can actually execute on that mission and make it work. So that's the 10x solution.

你认为解决自动驾驶问题的时间线是什么？这仍然是一个悬而未决的问题。

What do you think is the timeline to solve the problem of autonomous driving? This is still in part an open question.

是的，我认为自动驾驶时间线的难点在于显然还没有人创造出自动驾驶。所以这不像“你认为建造这座桥的时间线是什么？”嗯，我们以前建过桥。

Yeah, I think the tough thing with timelines of self-driving obviously is that no one has created self-driving yet. So it's not like 'what do you think is a timeline to build this bridge?' Well, we've built bridges before.

百万座桥，这就是需要的时间。你知道，没人造出过自动驾驶。这并不明显。有些部分比其他的容易得多，所以真的很难预测。你根据趋势线和直觉尽力而为，但这就是为什么从根本上说这很难预测。没人经历过，即使身处其中也很难判断。是的，有些事情比预想的难，有些事情则容易得多。你试图避免做预测吗？因为埃隆并不回避，对吧？过去汽车公司的负责人也没有回避。福特和其他地方曾预测到 2020 年或 2021 年解决 L4 驾驶，现在他们都在收回那个预测。作为 AI 从业者，你私下里会做预测吗？还是说预测会妨碍你实际思考问题？

Million bridges before here's how long that takes. It's, you know, it's, no one has built autonomy. It's not obvious. Some parts turn out to be much easier than others, so it's really hard to forecast. You do your best based on trend lines and so on and based on intuition, but that's why fundamentally it's just really hard to forecast this. No one has even still, like being inside of it is hard to do. Yes, some things turn out to be much harder and some things turn out to be much easier. Do you try to avoid making forecasts? Because like Elon doesn't avoid them, right? And heads of car companies in the past have not avoided it either. Ford and other places have made predictions that we're going to solve level four driving by 2020, 2021, whatever, and now they're all kind of backtracking on that prediction. As an AI person, do you free yourself privately make predictions, or do they get in the way of your actual ability to think about a thing?

是的，我想说容易说的是这个问题是可处理的，这是一个容易做出的预测。可处理，它会成功，是的。只是非常难。有些事情比预想的难，有些事情则容易些。所以它绝对感觉是可处理的，而且至少特斯拉团队——我在内部看到的——肯定在朝着那个方向前进。

Yeah, I would say like what's easy to say is that this problem is tractable, and that's an easy prediction to make. Tractable, it's going to work, yes. It's just really hard. Some things turn out to be harder, some things turn out to be easier. So it definitely feels tractable, and it feels like at least the team at Tesla, which is what I saw internally, is definitely on track to that.

你是如何形成一种强有力的表征，让你能够对可处理性做出预测？就像你领导很多人，你必须说这实际上是可能的。你是如何建立这种直觉的？不一定是驾驶，也可以是其他任务。我想知道你一生中处理过哪些困难的任务？我的意思是分类达到某种超人类水平的性能。

How do you form a strong representation that allows you to make a prediction about tractability? So like you're the leader of a lot of humans, you have to kind of say this is actually possible. How do you build up that intuition? It doesn't have to be even driving, it could be other tasks. I wonder what difficult tasks did you work on in your life? I mean classification achieving certain, just an image, that certain level of superhuman level performance.

是的，专家直觉，就是直觉，是信念。所以就像长时间思考，研究，查看样本数据。就像你说的驾驶。我的直觉在这方面其实有缺陷。我对可处理性没有好的直觉。它可能是任何一种，可能是任何情况。它可能是可解的，比如驾驶任务可以简化为相当琐碎的事情。解决方案可能相当简单，而且随着规模扩大，越来越多的车完美行驶可能让问题更容易。是的，你拥有的车越多，人们学习如何正确驾驶——不是正确，而是以对自动驾驶、半自动驾驶和手动驾驶的异构系统更优的方式——这可能会改变情况。另外，我也花了无数小时盯着行人过马路，思考人类，感觉我们使用眼神交流的方式传递了非常强的信号，而且行为中有某些怪癖和边缘情况。当然，很多死亡事故与酒驾有关，无论是行人还是司机。还有夜间驾驶的问题等等。所以我想，自动驾驶的可能解决方案空间包含了太多人为因素问题，几乎无法预测。可能会有非常干净漂亮的解决方案。

Yeah, expert intuition, it's just intuition, it's belief. So just like thinking about it long enough, like studying, looking at sample data. Like you said, driving. My intuition has really flawed on this. I don't have a good intuition about tractability. It could be either, it could be anything. It could be solvable like, you know, the driving task could be simplified into something quite trivial. Like the solution to the problem would be quite trivial, and at scale more and more cars driving perfectly might make the problem much easier. Yeah, the more cars you have driving, like people learn how to drive correctly, not correctly but in a way that's more optimal for a heterogeneous system of autonomous and semi-autonomous and manually driven cars, that could change stuff. Then again, also I've spent a ridiculous number of hours just staring at pedestrians crossing streets, thinking about humans, and it feels like the way we use our eye contact sends really strong signals, and there's certain quirks and edge cases of behavior. And of course, a lot of the fatalities that happen have to do with drunk driving, both on the pedestrian side and the driver's side. So there's that problem of driving at night and all that kind of stuff. Yeah, so I wonder, you know, it's like the space of possible solutions to autonomous driving includes so many human factor issues that it's almost impossible to predict. There could be super clean nice solutions.

是的，我肯定想用一个游戏类比，有些战争迷雾，但你也能看到改进的前沿，并且可以历史地衡量你取得了多少进展。我认为例如，至少我在特斯拉大约五年里看到的，当我加入时，它几乎无法在高速公路上保持车道。从帕洛阿尔托到旧金山，每次道路有任何几何变化或转弯太大，就需要三四次干预，它就是不行。所以从那个状态到五年后相当有能力的系统，看到引擎盖下发生了什么，以及团队现在在数据、算力和其他一切方面的规模，这是巨大的进步。所以你在爬山，有雾，但你取得了很大进展。有雾，你在进步，你看到接下来的方向，你看着剩下的挑战，它们不会困扰你，不会改变你的理念，你也不会扭曲自己。你实际上觉得这些是我们还需要做的事情。是的，解决问题的基本组件似乎都在那里：数据引擎、算力、车上的计算机、训练的算力等等。

Yeah, I would say definitely like to use a game analogy, there's some fog of war, but you definitely also see the frontier of improvement and you can measure historically how much you've made progress. I think for example, at least what I've seen in roughly five years at Tesla, when I joined it barely kept lane on the highway. I think going up from Palo Alto to SF was like three or four interventions anytime the road would do anything geometrically or turn too much it would just not work. So going from that to like a pretty competent system in five years, and seeing what happens also under the hood and what the scale which the team is operating now with respect to data and compute and everything else, is just a massive progress. So there's a, you're climbing a mountain and it's fog, but you're making a lot of progress. Fog, you're making progress and you see what the next directions are, and you're looking at some of the remaining challenges and they're not like, they're not perturbing you and they're not changing your philosophy and you're not contorting yourself. You're like actually these are the things that we still need to do. Yeah, the fundamental components of solving the problems seem to be there: the data engine, the compute, the computer on the car, the compute for the training, all that kind of stuff.

所以这些年来，你经历了很多，你做了很多惊人的突破性想法和工程，从数据引擎到人的方面，全部。你能谈谈为什么选择离开特斯拉吗？

So you've done over the years, you've been a test, you've done a lot of amazing breakthrough ideas and engineering, all of it, from the data engine to the human side, all of it. Can you speak to why you chose to leave Tesla?

基本上就像我描述的那样，随着时间的推移，在那五年里我有点陷入了管理职位。我的大部分日子都是会议、发展组织、做出关于团队的高层战略决策以及它应该做什么等等。这有点像企业高管角色。我能做，我觉得我还可以，但这并不是我真正享受的。所以我想当我加入时，没有计算机视觉团队，因为特斯拉正从使用 Mobileye（第三方供应商提供所有计算机视觉）过渡到必须自己构建计算机视觉系统。所以我到的时候只有两个人在训练深度神经网络，他们就在腿边的电脑上训练，做基本的分类任务。是的，然后我把它发展成了一个我认为相当可敬的深度学习团队，一个巨大的算力集群，一个非常好的数据标注组织，我对那个状态非常满意。它变得相当自主，所以我离开了。而且，你知道，我很兴奋能再次做更多技术性的事情。

Basically as I described that ran, I think over time during those five years I've kind of gotten myself into a little bit of a managerial position. Most of my days were you know meetings and growing the organization and making decisions about sort of high level strategic decisions about the team and what it should be working on and so on. And it's kind of like a corporate executive role. And I can do it, I think I'm okay at it, but it's not like fundamentally what I enjoy. And so I think when I joined, there was no computer vision team because Tesla was just going from the transition of using Mobileye, a third-party vendor for all of its computer vision, to having to build its computer vision system. So when I showed up there were two people training deep neural networks and they were training them at a computer at their legs, like kind of basic classification task. Yeah, and so I kind of like grew that into what I think is a fairly respectable deep learning team, a massive compute cluster, a very good data annotation organization, and I was very happy with where that was. It became quite autonomous, and so I kind of stepped away. And I, you know, I'm very excited to do much more technical things again.

是的，而且我们关注 AGI。这种灵魂探索是什么样的？因为你休息了一段时间，想什么，你吃了多少蘑菇？不，我只是，我的意思是你在想什么？人的生命是有限的。你做了几件了不起的事。你是世界上最好的 AI 老师之一。你是最好的之一，我这是最好的意思。你是 AI 世界最好的修补匠之一，意思是理解事物运作的基本原理，通过从头构建它并用基本直觉把玩它。就像爱因斯坦、费曼都很擅长这类事情，比如一个小例子，把玩它来理解它。所以那一点，显然现在和我们一起，你帮助建立了一个机器学习工程师团队和一个系统……

Yeah, and kind of like we focus on AGI. What was this soul searching like? Because you took a little time off and think like what, how many mushrooms did you take? No, I'm just, I mean what was going through your mind? The human lifetime is finite. You did a few incredible things. You're one of the best teachers of AI in the world. You're one of the best, and I don't mean that, I mean that in the best possible way. You're one of the best tinkerers in the AI world, meaning like understanding the fundamental fundamentals of how something works by building it from scratch and playing with it with the basic intuitions. It's like Einstein, Feynman were all really good at this kind of stuff, like a small example of a thing to play with it to try to understand it. So that and obviously now with us that you help build a team of machine learning engineers and a system that...

实际上在现实世界中有所成就。那么考虑到这一切，你当时的心路历程是怎样的？

Actually accomplishes something in the real world. So given all that, like what was the soul searching like?

嗯，这很难，因为我显然非常热爱这家公司，热爱埃隆，热爱特斯拉。我想……离开很难。我基本上热爱这个团队。但是，是的，我想我实际上可能有兴趣重新考虑，也许在某个时候回来，在 Optimus 和特斯拉的 AGI 领域工作。我认为特斯拉会做出不可思议的事情。它基本上是一家大规模机器人公司，拥有大量内部人才，能够做出真正了不起的事情。我认为人形机器人会很棒，自动驾驶也会很棒。所有这些都在特斯拉发生，所以我认为它是一家非常了不起的组织。成为其中一员并帮助它前进，我觉得……基本上我非常享受。是的，出于这些原因，离开很难，因为我热爱这家公司。但你知道，我很高兴有可能在某个时候回来演第二幕。但我觉得在这个阶段，我已经建立了团队，它变得自主了，我成了一名经理，而我想做更多技术性的事情。我想学习，想教学，我只是觉得是时候稍微改变一下节奏了。

Well, it was hard because obviously I love the company a lot, and I love Elon, I love Tesla. I want... it was hard to leave. I love the team basically. But yeah, I think actually I would potentially like interested in revisiting it, maybe coming back at some point, working in Optimus, working in AGI at Tesla. I think Tesla is going to do incredible things. It's basically like a massive large-scale robotics kind of company with a ton of in-house talent for doing really incredible things. And I think human robots are going to be amazing. I think autonomous transportation is going to be amazing. All this is happening at Tesla, so I think it's just a really amazing organization. So being part of it and helping it along, I think, was very... basically I enjoyed that a lot. Yeah, it was basically difficult for those reasons because I love the company. But you know, I'm happy to potentially at some point come back for act two. But I felt like at this stage, I built the team, it felt autonomous, and I became a manager, and I wanted to do a lot more technical stuff. I wanted to learn stuff, I wanted to teach stuff, and I just kind of felt like it was a good time for a change of pace a little bit.

说到第二部分，你认为有史以来最好的电影续集是什么？因为大多数电影续集都很烂。是的，你在推特上谈论电影，所以顺便问一下，有没有……你最喜欢的电影续集是什么？

What do you think is the best movie sequel of all time, speaking of part two? Because like most of them suck in movie sequels. Yeah, and you tweet about movies, so just in a tiny tangent, is there... what's your favorite movie sequel?

《教父 2》。

Godfather Part Two.

你是《教父》的粉丝吗？因为你甚至没有发推或提到《教父》。

Are you a fan of Godfather? Because you didn't even tweet or mention the Godfather.

是的，我不喜欢那部电影。我知道……还没有……把这段剪掉。我们要剪掉对《教父》的仇恨。你怎么敢？我想我会做一个强有力的声明：我不知道为什么，但我基本上不喜欢 1995 年之前的任何电影，差不多这样。

Yeah, I don't love that movie. I know it... haven't... edit that out. We're gonna edit out the hate towards the Godfather. How dare you? I think I will make a strong statement: I don't know why, but I basically don't like any movie before 1995, something like that.

你不是提到《终结者 2》了吗？好吧好吧，那就像……《终结者 2》稍微晚一点，1990 年？不，我觉得《终结者 2》是……我也喜欢《终结者 1》，所以好吧，所以有几个例外。但总的来说，出于某种原因，我不喜欢 1995 年之前的电影。它们感觉很慢，镜头拉得很远，很无聊，有点幼稚，有点奇怪。而且《终结者》非常超前。是的。而《教父》里没有 AGI。我的意思是，但你提到的《心灵捕手》也没有 AGI。我想那是数学。是的，我想我偶尔也喜欢没有……或者像《王牌播音员》，那没有……那是增量，它太好了。我不明白。说到 AGI，因为我不明白为什么威尔·法瑞尔这么好笑。这说不通，无法计算。他有一种特质，他是一个独特的人。因为现在没有那么多喜剧片了，我想知道这是否与文化或好莱坞的机器有关，还是只是我们幸运地遇到了某些人，喜剧就凑到了一起。因为他是一个独特的人。这是一个荒谬的题外话，我道歉。

Didn't you mention Terminator 2? Okay, okay, that's like... Terminator 2 was a little bit later, 1990? No, I think Terminator 2 was... I like Terminator 1 as well, so okay, so like a few exceptions. But by and large, for some reason I don't like movies before 1995 or something. They feel very slow, the camera is like zoomed out, it's boring, it's kind of naive, it's kind of weird. And also Terminator was very much ahead of its time. Yes. And The Godfather, there's like no AGI. I mean, but you have Good Will Hunting was one of the movies you mentioned, and that doesn't have any AGI either. I guess that's mathematics. Yeah, I guess occasionally I do enjoy movies that don't feature... or like Anchorman, that has no... that's the increment, it's so good. I don't understand. Speaking of AGI, because I don't understand why Will Ferrell is so funny. It doesn't make sense, it doesn't compute. There's just something about him, and he's a singular human. Because you don't get that many comedies these days, and I wonder if it has to do about the culture or the like the machine of Hollywood, or does it have to do with just we got lucky with certain people and comedy it came together. Because he is a singular human. That was a ridiculous tangent, I apologize.

但你提到了人形机器人。那么你怎么看 Optimus，特斯拉机器人？你认为我们会在 10 年、20 年、30 年、40 年、50 年后在工厂和家庭中拥有机器人吗？

But you mentioned humanoid robot. So what do you think about Optimus, about Tesla bot? Do you think we'll have robots in the factory, in the home in 10, 20, 30, 40, 50 years?

是的，我认为这是一个非常困难的项目。我认为这需要一段时间。但还有谁能大规模制造人形机器人呢？我认为这是一个非常好的形态因素，因为正如我提到的，世界是为人类形态设计的。这些东西将能够操作我们的机器，能够坐在椅子上，甚至可能驾驶汽车。基本上，世界是为人类设计的。这就是你想要投资并最终使其工作的形态因素。我认为还有另一种思想流派：好吧，选择一个具体问题，然后设计一个机器人来解决它。但实际上，设计一个机器人并让整个数据引擎及其背后的一切工作起来是一个极其困难的问题。所以追求通用接口是有道理的——它们对任何特定任务都不是完美的，但它们具有通用性，只需一个提示、用英语就能跨领域做事。所以我认为在物理世界中追求通用接口非常有意义。这是一个非常困难的项目，需要时间，但我看不到有其他公司能够执行这一愿景。我认为这将是惊人的。比如，体力劳动：如果你认为交通是一个大市场，试试体力劳动。疯狂。

Yeah, I think it's a very hard project. I think it's going to take a while. But who else is going to build humanoid robots at scale? And I think it is a very good form factor to go after because, like I mentioned, the world is designed for humanoid form factor. These things would be able to operate our machines, they would be able to sit down in chairs, potentially even drive cars. Basically, the world is designed for humans. That's the form factor you want to invest into and make work over time. I think there's another school of thought which is: okay, pick a problem and design a robot to it. But actually, designing a robot and getting a whole data engine and everything behind it to work is an incredibly hard problem. So it makes sense to go after general interfaces that, okay, they are not perfect for any one given task, but they actually have the generality of just with a prompt, with English, able to do something across. And so I think it makes a lot of sense to go after a general interface in the physical world. And I think it's a very difficult project. I think it's going to take time, but I see no other company that can execute on that vision. I think it's going to be amazing. Like, basically physical labor: if you think transportation is a large market, try physical labor. Insane.

嗯，但对我来说不仅仅是体力劳动。同样令人兴奋的是社交机器人，我们与这些机器人在不同层面上的关系。这就是为什么我看到 Optimus 时非常兴奋。比如，嗯，有人批评我太兴奋了，但我与许多研究人形腿式机器人的实验室合作过：波士顿动力、宇树科技，很多。有很多公司做腿式机器人，但运动的优雅只是整个图景中很小很小的一部分。所以，特斯拉做人形或任何腿式机器人，对我来说，整合两个令人兴奋的事情是：显然将其整合到数据引擎中。所以数据引擎方面，即感知、控制、规划等实际智能，整合到你提到的庞大车队中。然后，说到车队，第二件事是批量制造。仅仅知道，从文化上，朝着一个简单、可大规模廉价生产的机器人努力，并且做好，有经验做好，这改变了一切。这就是为什么这与波士顿动力非常不同的文化和风格，顺便说一句，那些机器人移动的方式，特斯拉要很长时间才能达到那种流畅度。但这不是重点。重点是整个系统，就像我们讨论的数据引擎和车队。这非常令人兴奋。即使是初始模型，但几个月内就能拿到原型也真的很令人惊讶。

Well, but it's not just physical labor to me. The thing that's also exciting is the social robotics, so the relationship we'll have on different levels with those robots. That's why I was really excited to see Optimus. Like, um, people have criticized me for the excitement, but I've worked with a lot of research labs that do humanoid legged robots: Boston Dynamics, Unitree, a lot. There's a lot of companies that do legged robots, but that's the elegance of the movement is a tiny, tiny part of the big picture. So integrating the two big exciting things to me about Tesla doing humanoid or any legged robots is clearly integrating it into the data engine. So the data engine aspect, so the actual intelligence for the perception and the control and the planning and all that kind of stuff, integrating into this huge fleet that you mentioned, right. And then, speaking of fleet, the second thing is the mass manufacturing. Just knowing, culturally, driving towards a simple robot that's cheap to produce at scale, and doing that well, having experience to do that well, that changes everything. That's why that's a very different culture and style than Boston Dynamics, who by the way, those robots are just the way they move, it's like it'll be a very long time before Tesla could achieve the smoothness of movement. But that's not what it's about. It's about the entirety of the system, like we talked about the data engine and the fleet. That's super exciting. Even the initial sort of models, but that too was really surprising that in a few months you can get a prototype.

是的，之所以这么快，正如你提到的，是因为大量借鉴了自动驾驶方面的成果。是的，很多。特斯拉内部涌现出的构建人形机器人的专业知识令人难以置信。基本上，埃隆在某个时候说我们要做这个，然后第二天基本上所有这些 CAD 模型就开始出现，人们谈论供应链和制造，前几天人们带着螺丝刀之类的东西出现，开始组装车身。我当时想，哇，特斯拉有这么多人才。而且从根本上说，造车实际上并不那么……

Yeah, and the reason that happened very quickly is as you alluded to, there's a ton of carryover from what's happening in the autopilot. Yes, a lot. The amount of expertise that came out of the woodwork at Tesla for building the human robot was incredible to see. Like basically Elon said at one point we're doing this, and then next day basically all these CAD models started to appear, and people talk about like the supply chain and manufacturing, and people showed up with like screwdrivers and everything the other day and started to like put together the body. And I was like whoa, like all these people exist at Tesla. And fundamentally, building a car is actually not that...

与制造机器人不同，这同样适用。不仅硬件部分如此，别忘了硬件不只是为了演示，大规模制造硬件完全是另一回事，软件也是如此。

Different from building a robot the same. That is true not just for the hardware pieces, and let's not forget hardware not just for a demo but manufacturing of that hardware at scale is like a whole different thing, but for software as well.

基本上，这个机器人目前认为自己是一辆车。它会在某个时刻经历中年危机。它认为自己是一辆车。早期的一些演示中，我们曾考虑在室外停车场进行，因为那里的计算机视觉开箱即用，而室内不行。但所有操作系统，一切都可以直接复制粘贴。计算机视觉大部分也是复制粘贴。我的意思是，你需要重新训练神经网络，但方法、数据引擎、离线追踪器、占用追踪器的方式等等，一切都可以复制粘贴。你只需要重新训练神经网络。当然，规划控制必须改变很多，但有很多来自特斯拉的复制粘贴。所以如果你的目标是‘制造一百万个类人机器人’，而你不是特斯拉，那要求太高了。如果你是特斯拉，其实没那么疯狂。

Basically, this robot currently thinks it's a car. It's going to have a midlife crisis at some point. It thinks it's a car. Some of the earlier demos, we were talking about potentially doing them outside in the parking lot because that's where all of the computer vision worked out of the box instead of inside. But all the operating system, everything just copy-pastes. Computer vision mostly copy-paste. I mean, you have to retrain the neural nets, but the approach and everything in data engine and offline trackers and the way we go about the occupancy tracker and so on, everything copy-paste. You just need to retrain the neural nets. And then the planning control, of course, has to change quite a bit, but there's a ton of copy-paste from what's happening at Tesla. So if you were to go with the goal of 'let's build a million human robots' and you're not Tesla, that's a lot to ask. If you're a Tesla, it's actually not that crazy.

接下来的问题是：操控任务有多难，才能产生大规模影响？

And the follow-up question is: how difficult is the manipulation task such that it can have an impact at scale?

我认为取决于上下文，机器人技术的一个好处是，除非你做制造业，否则容错空间更大。驾驶对安全性和时间要求极高。机器人可以移动得慢一些，这很好。是的，我认为这需要很长时间，但你想构建开发的方式是，你必须说：‘好吧，这需要很长时间。我如何制定产品开发路线图，以便在这个过程中赚钱？’我不想让自己陷入一个零一损失函数，即直到它工作才有效。你不想处于那种境地。你想让它几乎立即有用，然后慢慢大规模部署。你想建立你的数据引擎、改进循环、遥测、评估、测试平台等等。你想随着时间的推移正确改进产品，同时在这个过程中赚钱。这非常重要，否则你无法构建这些大型项目。它们在经济上不合理，而且从团队的角度来看，他们需要沿途的多巴胺。他们不会仅仅承诺这会有用，‘这将在 10 年后改变世界’。这不是你想要的。你想处于像今天的 Autopilot 那样的位置，它提供了更高的安全性和驾驶便利性。人们为此付费，人们喜欢它，人们购买它，然后你还有更大的使命在努力。你看到了这一点，所以团队的多巴胺是快乐的源泉。是的，你在部署这个，人们喜欢它，人们驾驶它，人们为它付费，他们关心它。有所有这些 YouTube 视频，你的祖母驾驶它，她给你反馈。人们喜欢它，人们与它互动，你与它互动。巨大的。开特斯拉的人会认出你并给你爱，比如‘谢谢你这个好功能’？

I think depending on the context, the really nice thing about robotics is that unless you do manufacturing, there's more room for error. Driving is so safety-critical and time-critical. A robot is allowed to move slower, which is nice. Yes, I think it's going to take a long time, but the way you want to structure the development is you need to say, 'Okay, it's going to take a long time. How can I set up the product development roadmap so that I'm making revenue along the way?' I'm not setting myself up for a zero-one loss function where it doesn't work until it works. You don't want to be in that position. You want to make it useful almost immediately, and then you want to slowly deploy it at scale. And you want to set up your data engine, your improvement loops, the telemetry, the evaluation, the harness, and everything. And you want to improve the product over time correctly, and you're making revenue along the way. That's extremely important because otherwise you cannot build these large undertakings. They just don't make sense economically, and also from the point of view of the team working on it, they need the dopamine along the way. They're not just going to make a promise about this being useful, 'this is going to change the world in 10 years when it works.' This is not where you want to be. You want to be in a place like I think Autopilot is today, where it's offering increased safety and convenience of driving today. People pay for it, people like it, people purchase it, and then you also have the greater mission that you're working towards. And you see that, so the dopamine for the team, that was a source of happiness. Yes, you're deploying this, people like it, people drive it, people pay for it, they care about it. There are all these YouTube videos, your grandma drives it, she gives you feedback. People like it, people engage with it, you engage with it. Huge. Do people that drive Teslas recognize you and give you love, like 'thanks for this nice feature'?

是的，我认为棘手的是有些人真的很爱你，有些人却不幸地恨你。你在做你认为非常有价值和有用的事情等等。有些人确实恨你。有很多人恨我和团队以及整个项目。而且我认为很多情况下他们是特斯拉车主，实际上并不是。

Yeah, I think the tricky thing is some people really love you, some people unfortunately hate you. You're working on something you think is extremely valuable and useful, etc. Some people do hate you. There's a lot of people who hate me and the team and the whole project. And I think they are Tesla drivers in many cases, not actually.

这实际上让我对人类或当前人类的互动方式感到难过。我认为这是可以解决的。我认为人类想要善待彼此。我认为 Twitter 和社交媒体是某种机制的一部分，它让负面情绪更具传播性，但它不应该得到不成比例的传播助推。但我希望人们能为他人的成就感到兴奋，抑制一些嫉妒和自我，只是为他人感到兴奋。然后有因果报应：你为他人兴奋，他们也会为你兴奋。学术界也是如此。如果你不小心，那里有一个动态系统。如果你孤立地思考，嫉妒别人的成功，实际上，可能反直觉地，会导致你作为社区和个人的生产力降低。我觉得如果你不断庆祝他人，那实际上会让你更成功。

That actually makes me sad about humans or the current ways that humans interact. I think that's actually fixable. I think humans want to be good to each other. I think Twitter and social media is part of the mechanism that somehow makes the negativity more viral, but it doesn't deserve a disproportionate viral boost. But I wish people would just get excited about others, suppress some of the jealousy and ego, and just get excited for others. And then there's a karma aspect: you get excited for others, they'll get excited for you. Same thing in academia. If you're not careful, there's a dynamical system there. If you think in silos and get jealous of somebody else being successful, that actually, perhaps counterintuitively, leads to less productivity for you as a community and you individually. I feel like if you keep celebrating others, that actually makes you more successful.

是的，我认为人们还没有完全学会这一点，取决于行业。有些人非常消极且直言不讳，所以他们非常突出。但实际上，有很多人是啦啦队员，但他们是沉默的啦啦队员。当你和世界上的人交谈时，他们都会告诉你这很棒，这很好。尤其是那些理解让这些东西工作有多难的人，比如那些构建过产品和创客、企业家。让这个东西工作并改变一些东西是极其困难的。那些人更可能为你加油。

Yeah, I think people haven't, depending on the industry, quite learned that yet. Some people are also very negative and very vocal, so they're very prominently featured. But actually, there are a ton of people who are cheerleaders, but they're silent cheerleaders. And when you talk to people in the world, they will all tell you it's amazing, it's great. Especially people who understand how difficult it is to get this stuff working, like people who have built products and makers, entrepreneurs. Making this work and changing something is incredibly hard. Those people are more likely to cheerlead you.

嗯，让我难过的一件事是机器人社区的一些人不做啦啦队，他们应该做，因为他们知道这有多难。嗯，他们实际上有时不知道大规模制造产品有多难，对吧？他们实际上是在现实世界中部署。很多机器人和 AI 系统的开发都是在非常具体的小型基准上进行的，而不是在现实世界条件下。

Well, one of the things that makes me sad is some folks in the robotics community don't do the cheerleading, and they should, because they know how difficult it is. Well, they actually sometimes don't know how difficult it is to create a product at scale, right? They actually deploy in the real world. A lot of the development of robots and AI systems is done on very specific small benchmarks, as opposed to real-world conditions.

是的，我认为在学术环境中研究机器人技术或应用于现实世界的 AI 系统非常困难。你曾批评过，你一度繁荣并热爱，ImageNet，著名的 ImageNet 数据集。我最近听到一些批评，说学术研究机器学习社区仍然对 ImageNet 或这类基准给予太多关注。你能谈谈机器学习研究中使用的数据集的优缺点吗？

Yes, I think it's really hard to work on robotics in an academic setting or AI systems that apply in the real world. You've criticized, you flourished and loved for a time, the ImageNet, the famed ImageNet dataset. And I've recently had some words of criticism that the academic research ML community gives a little too much love still to ImageNet or those kinds of benchmarks. Can you speak to the strengths and weaknesses of datasets used in machine learning research?

实际上，我不记得我具体在哪个场合对 ImageNet 不满或批评过。我认为 ImageNet 非常有价值。它基本上是一个基准，让深度学习社区能够证明深度神经网络确实有效。这有巨大的价值。所以我认为 ImageNet 很有用，但它在某种程度上已经变成了 MNIST。MNIST 是 28x28 的灰度数字。它有点像一个笑话数据集，每个人都能轻松搞定。没有论文是写 MNIST 的，对吧？

Actually, I don't know that I recall the specific instance where I was unhappy or criticizing ImageNet. I think ImageNet has been extremely valuable. It was basically a benchmark that allowed the deep learning community to demonstrate that deep neural networks actually work. There was massive value in that. So I think ImageNet was useful, but it's become a bit of a MNIST at this point. MNIST is like the 28x28 grayscale digits. It's kind of a joke dataset that everyone just crushes. There are no papers written on MNIST, right?

也许他们应该做一些有分量的论文，比如专注于如何用小数据学习之类的研究。是的，我觉得那会有帮助，但不再是计算机视觉主流研究的方向了。当然，我记得好像听你提过——可能是我记错了——你说 ImageNet 长期以来对社区贡献巨大，但现在该超越这类数据集了……嗯，ImageNet 已经被攻克了。我是说，错误率已经……在 1000 类分类预测中我们达到了大约 90% 的准确率，我看过那些图片，确实很高。真的很不错。如果我没记错的话，top-5 错误率现在大概只有 1% 左右。鉴于你处理过超大规模真实数据集的经验，你希望研究社区的基准测试朝哪些方向发展？

Maybe they should have strong papers like papers that focus on how do we learn with a small amount of data, that kind of stuff. Yeah, I could see that being helpful, but not in sort of mainline computer vision research anymore. Of course, I think the way I've heard you somewhere—maybe I'm just imagining things—but I think you said ImageNet was a huge contribution to the community for a long time, and now it's time to move past those kinds of... well, ImageNet has been crushed. I mean, you know, the error rates are... we're getting like 90% accuracy in 1000-class classification prediction, and I've seen those images, and it's really high. That's really good. If I remember correctly, the top-5 error rate is now like 1% or something. Given your experience with a gigantic real-world dataset, would you like to see benchmarks move in certain directions that the research community uses?

不幸的是，我认为学术界目前还没有下一个 ImageNet。我们显然已经攻克了 MNIST，也基本攻克了 ImageNet，现在没有一个整个社区都团结起来用于进一步开发这些网络的大型基准测试。一个数据集要能激发所有人的想象力，让大家齐心协力，可能还需要一个病毒式传播的领导者，一个有影响力的人。我是说，为什么 ImageNet 会成功？这仅仅是历史的偶然吗？它的难度和简单程度恰到好处，也足够有趣。它只是恰好赶上了那个时代。

Unfortunately, I don't think academics currently have the next ImageNet. We've obviously crushed MNIST, we've basically crushed ImageNet, and there's no next big benchmark that the entire community rallies behind and uses for further development of these networks. What it takes for a dataset to captivate the imagination of everybody, where they all get behind it, that could also need a viral leader, you know, somebody with popularity. I mean, why did ImageNet take off? Is it just an accident of history? It was the right amount of difficult and simple and interesting enough. It just kind of was the right time for that kind of dataset.

来自 Reddit 的问题：你对合成数据和游戏引擎在神经网络模型开发未来中的作用有什么看法？

Question from Reddit: What are your thoughts on the role that synthetic data and game engines will play in the future of neural net model development?

我认为随着神经网络向人类趋近，模拟对神经网络的价值将类似于模拟对人类的的价值。人们使用模拟是因为他们可以在那种系统中学习某些东西，而不必真正去体验。

I think as neural nets converge to humans, the value of simulation to neural nets will be similar to the value of simulation to humans. So people use simulation because they can learn something in that kind of system without having to actually experience it.

但你指的是我们头脑中的模拟吗？不，抱歉，我指的模拟是像电子游戏或各种专业人士使用的其他形式的模拟。嗯，我想反驳一下，因为也许我们头脑中的模拟，比如模拟“如果我这样做，会发生什么？”那是内部模拟。这不就是我们行动前所做的吗？哦，是的，但这与使用计算机游戏意义上的模拟或使用模拟来创建训练集是独立的。还是说只是松散相关？因为做反事实或边缘案例模拟不是很有用吗？比如如果发生核战争会怎样？是的，那与虚幻引擎的模拟不同。我就是这样理解这个问题的。那么平均情况的模拟呢？什么是虚幻引擎？你所说的虚幻引擎是什么意思？模拟一个世界，那个世界的物理。为什么那不一样？因为你还可以向那个世界添加行为，尝试各种东西，比如把各种奇怪的东西扔进去。所以虚幻引擎不仅仅是模拟世界的物理，它还用这些做点什么：图形、物理以及你放入环境中的智能体。是的，我觉得你说过这对 AI 发展的未来不太重要。我这样理解对吗？

But are you referring to the simulation we're doing in our head? No, sorry, simulation I mean like video games or other forms of simulation for various professionals. Well, let me push back on that because maybe their simulation that we do in our heads, like simulate 'if I do this, what do I think will happen?' That's internal simulation. Isn't that what we do before we act? Oh yeah, but that's independent from the use of simulation in the sense of computer games or using simulation for training set creation. Or is it just loosely correlated? Because isn't that useful to do counterfactual or edge case simulation, like what happens if there's a nuclear war? Yeah, that's a different simulation from Unreal Engine. That's how I interpreted the question. So simulation of the average case? What's Unreal Engine? What do you mean by Unreal Engine? Simulating a world, physics of that world. Why is that different? Because you can also add behavior to that world and try all kinds of stuff, like throw all kinds of weird things into it. So Unreal Engine is not just about simulating the physics of the world, it's also doing something with that: the graphics, the physics, and the agents that you put into the environment. Yeah, I feel like you said it's not that important for the future of AI development. Is that correct to interpret you that way?

我认为人类使用模拟器，他们觉得有用，所以计算机也会使用模拟器并觉得有用。好吧，所以你说它不……我不经常使用模拟器。我偶尔玩电子游戏，但我不认为我从那些电子游戏中获得了关于自身存在的任何智慧。那是对现实的短暂逃避，而不是关于现实的智慧来源。所以我认为这是一种非常礼貌的说法，即模拟没那么有用。也许吧，也许不是。目前我不认为它是训练神经网络的一个基础性的、非常重要的部分。但我认为随着神经网络变得越来越强大，你将需要更少的样本来训练额外的行为，而模拟当然存在领域差距，它不是真实世界，有些细微的不同。但有了足够强大的神经网络，我认为领域差距可以更大，因为神经网络会某种程度上理解，尽管它不是真实世界，但它拥有所有我应该能够学习的高级结构。所以你将能够更好地利用合成数据，是的，通过缩小……更好地理解它在哪些方面不是真实数据。对，下次问更好的问题。那是个问题，但我开玩笑的。

I think humans use simulators and they find them useful, and so computers will use simulators and find them useful. Okay, so you're saying it's not... I don't use simulators very often. I play a video game every once in a while, but I don't think I derive any wisdom about my own existence from those video games. It's a momentary escape from reality versus a source of wisdom about reality. So I think that's a very polite way of saying simulation is not that useful. Maybe, maybe not. I don't see it as a fundamental, really important part of training neural nets currently. But I think as neural nets become more and more powerful, you will need fewer examples to train additional behaviors, and simulation, of course, there's a domain gap in a simulation that's not the real world, there's something slightly different. But with a powerful enough neural net, the domain gap can be bigger, I think, because the neural network will sort of understand that even though it's not the real world, it has all this high-level structure that I'm supposed to be able to learn from. So then you'll be able to leverage the synthetic data better, yes, by closing the... get better understanding in which ways this is not real data exactly. Right, to do better questions next time. That was a question, but I'm just kidding.

那么，说到……构建需要极少数据的神经网络和训练过程是否可能？我们一直在讨论像互联网这样的大型训练数据集。我是说，一种说法是，就像你说的，查询本身是另一个层次的训练，这需要少量数据。是的，但你认为做研究并朝着“我们能否用极少数据训练构建知识库”这个方向走有价值吗？

So is it possible, speaking of... to construct neural nets and training processes that require very little data? We've been talking about huge datasets like the internet for training. I mean, one way to say that is like you said, the querying itself is another level of training, I guess, and that requires a little data. Yeah, but do you see any value in doing research and going down the direction of 'can we use very little data to train to construct a knowledge base'?

百分之百。我只是认为在某个时候你需要一个大规模数据集，然后当你预训练你的大规模神经网络并得到像 GPT 之类的东西时，你就能非常高效地训练任何任意的新任务。所以很多这样的 GPT，你可以通过仅用很少的例子进行提示来完成诸如情感分析或翻译等任务：'这是我想让你做的事情：这里有一个输入句子，这里是它的德语翻译；输入句子，德语翻译；输入句子，空白。' 然后神经网络会仅仅通过看你提供的例子来完成德语翻译。这就是一个在神经网络激活中而非权重中的少样本学习的例子。所以我认为基本上，就像人类一样，神经网络在学习任何其他新任务时会变得非常数据高效，但在某个时候你需要一个大规模数据集来预训练你的网络以获得这种能力。可能我们人类也有类似的东西。我们有那样的东西吗？我们是否有一个被动的、在后台构建模型的东西，只是……

100%. I just think at some point you need a massive dataset, and then when you pre-train your massive neural net and get something that you know is like a GPT or something, then you're able to be very efficient at training any arbitrary new task. So a lot of these GPTs, you can do tasks like sentiment analysis or translation or so on just by being prompted with very few examples: 'Here's the kind of thing I want you to do: here's an input sentence, here's the translation into German; input sentence, translation to German; input sentence, blank.' And the neural network will complete the translation to German just by looking at the example you've provided. So that's an example of very few-shot learning in the activations of the neural net instead of the weights of the neural net. So I think basically, just like humans, neural nets will become very data efficient at learning any other new task, but at some point you need a massive dataset to pre-train your network to get that. And probably we humans have something like that. Do we have something like that? Do we have a passive in-the-background model-constructing thing that just...

它一直以自监督的方式运行，我们并没有意识到。我认为人类在一生中确实学到了很多，但我们也有大量来自进化的硬件帮助我们初始化。这是一个非常重要的组成部分。领域里很多人谈论一个人活了多少秒，假装这就像神经网络的零初始化，但事实并非如此。你可以看看像斑马这样的动物：它们出生，能看见，能跑。它们一生中没有训练数据，就能做到。不知何故，进化找到了编码这些算法的方法，这些神经网络初始化非常出色。我不知道这是怎么做到的，但显然这是可能的，因为这里有一个存在性证明。

It runs all the time in a self-supervised way; we're not conscious of it. I think humans definitely learn a lot during our lifespan, but we also have a ton of hardware that helps us initialize from evolution. That's a really big component. A lot of people in the field talk about the number of seconds a person has lived, pretending it's like a zero initialization of a neural net, but it's not. You can look at animals like zebras: they are born, they see, and they can run. There's zero training data in their lifespan; they can just do that. Somehow, evolution has found a way to encode these algorithms, and these neural net initializations are extremely good. I have no idea how this works, but apparently it's possible because here's a proof by existence.

从单细胞到出生的有机体，这其中有某种神奇的东西。在生命的最初几年，我有点喜欢这个想法：我们不记得最初几年的原因是因为那是一个非常痛苦的过程，就像非常困难、具有挑战性的训练过程。也许那是非常痛苦的背景模型训练，一旦系统训练完成，最好不记得它是如何构建的。

There's something magical about going from a single cell to an organism that is born. In the first few years of life, I kind of like the idea that the reason we don't remember anything about the first few years is that it's a really painful process, like a very difficult challenging training process. Maybe that's the background model training that is very painful, and it's best for the system once it's trained not to remember how it's constructed.

我认为这只是长期记忆的硬件还没有完全发育。我有点觉得婴儿的最初几年实际上不是学习，而是大脑成熟。我们是早产儿，有一个理论是这么说的，因为产道和大脑的肿胀。所以我们早产，然后最初几年我们只是大脑在成熟，然后最终有一些学习。这是我目前的看法。

I think it's just that the hardware for long-term memory is not fully developed. I kind of feel like the first few years of infants is not actually learning; it's brain maturing. We're born premature, and there's a theory along those lines because of the birth canal and the swelling of the brain. So we're born premature, and then the first few years we're just the brains maturing, and then there's some learning eventually. That's my current view on it.

你怎么看？你认为神经网络能拥有接近人类的长期记忆吗？你认为需要在它上面再加一个元架构来添加类似知识库的东西来学习关于世界的事实吗？

What do you think? Do you think neural nets can have long-term memory that approaches something like humans do? Do you think there needs to be another meta architecture on top of it to add something like a knowledge base that learns facts about the world?

是的，但我不知道它会在多大程度上被显式构建。它可能采取不直观的形式，比如你告诉 GPT：‘嘿，你有一个声明式记忆库，你可以存储和检索数据。每当你遇到一些你觉得有用的信息，就把它保存到你的记忆库。这是一个你检索到的例子，这是你如何说它，这是你如何加载它。’你只需用英文文本教它‘加载’，然后它可能就会学会使用记忆库。所以神经网络是背景模型的架构，是基础的东西，然后其他一切都只是建立在它之上。这不仅仅是文本；你给了它各种小工具和装置。你用某种特殊语言教它，通过这种语言它可以保存任意信息并在以后检索。你告诉它这些特殊标记以及如何排列它们来使用这些接口。就像：‘嘿，你可以用计算器。这是使用方法：只需输入 53 + 41 =，当等号出现时，计算器会实际读出答案，你不需要自己计算。’你只需用英语告诉它。这实际上可能行得通。

Yes, but I don't know to what extent it will be explicitly constructed. It might take unintuitive forms where you are telling the GPT, 'Hey, you have a declarative memory bank to which you can store and retrieve data. Whenever you encounter some information that you find useful, just save it to your memory bank. Here's an example of something you have retrieved, and here's how you say it, and here's how you load from it.' You just say 'load' whatever you teach it in text in English, and then it might learn to use a memory bank from that. So the neural net is the architecture for the background model, the base thing, and then everything else is just on top of it. It's not just text; you're giving it gadgets and gizmos. You're teaching it in some kind of special language by which it can save arbitrary information and retrieve it at a later time. You're telling about these special tokens and how to arrange them to use these interfaces. It's like, 'Hey, you can use a calculator. Here's how you use it: just do 53 + 41 =, and when equals is there, a calculator will actually read out the answer, and you don't have to calculate it yourself.' You just tell it in English. This might actually work.

从这个意义上说，你认为 Gato 有趣吗？DeepMind 的系统把所有东西——图像、动作等等——都扔进同一个堆里。这基本上就是我们前进的方向。

Do you think in that sense Gato is interesting? The DeepMind system that throws it all in the same pile—images, actions, all that kind of stuff. That's basically what we're moving towards.

是的，我想是的。Gato 在很大程度上是一种厨房水槽式的方法，用于强化学习，涉及许多不同的环境和一个固定的 Transformer 模型。我认为这是该领域一个非常早期的成果，但它符合我认为最终会呈现的样子。从严谨的角度来看，这是一个最终会变成这样的系统的早期阶段。我并不特别喜欢所有这些看起来非常不同的接口。我希望所有东西都标准化到同一个 API。例如，与其拥有不同物理、关节配置、外观等等的不同世界环境，以及为不同游戏提供一些特殊标记，我宁愿将所有东西标准化为一个单一的接口，这样对神经网络来说看起来都一样。所以我认为最终一切都将是基于像素的乒乓球。

Yeah, I think so. Gato is very much a kitchen sink approach to reinforcement learning with lots of different environments and a single fixed Transformer model. I think it's a very early result in that realm, but it's along the lines of what I think things will eventually look like. This is the early days of a system that eventually will look like this from a rigorous perspective. I'm not a super huge fan of all these interfaces that look very different. I would want everything to be normalized into the same API. For example, instead of having different world environments with very different physics, joint configurations, appearances, and whatever, and having some kind of special tokens for different games that you can plug, I'd rather just normalize everything to a single interface so it looks the same to the neural net. So it's all going to be pixel-based pong in the end, I think.

让我问问你的个人生活。很多人想知道：你是 AI 历史上最有生产力和才华的人之一。安德烈·卡帕西生活中富有成效的一天是什么样的？你几点起床？我想象在平均富有成效的一天和完美富有成效的一天之间有一种舞蹈。完美富有成效的一天是我们努力追求的目标，而平均的一天是它收敛到的样子，包含了所有的错误和人类的各种情况。

Let me ask you about your own personal life. A lot of people want to know: you're one of the most productive and brilliant people in the history of AI. What is a productive day in the life of Andrej Karpathy look like? What time do you wake up? I imagine some kind of dance between the average productive day and a perfect productive day. The perfect productive day is the thing we strive towards, and the average is kind of what it converges to, getting all the mistakes and human eventualities.

我不是一个早起的人；我绝对是个夜猫子。我想稳定与否，那是半稳定的，比如八九点之类的。读博期间甚至更晚；我通常凌晨 3 点才睡。我认为凌晨的时间很宝贵，是工作的有趣时间，因为所有人都睡着了。早上 7、8 点，东海岸醒了，所以已经有活动、短信、事情发生。你可以去新闻网站，有事情发生；这很分散注意力。凌晨 3 点，一切都非常安静，所以你不会被打扰，你有大块的时间工作。所以我喜欢那些时段。默认是夜猫子。富有成效的时间基本上：你需要在问题上建立一些势头，没有太多干扰，你需要把那个问题加载到你的内存、你的工作记忆中，然后你需要对它着迷。当你洗澡时，当你入睡时，你需要对问题着迷，它完全在你的记忆中，你准备好醒来就马上工作。所以有一个尺度：一天或几天、一周、一个月。我不能孤立地谈论一天，因为这是一个完整的过程。当我想在一个问题上变得高效时，我觉得我需要几天的时间跨度。

I'm not a morning person; I'm a night owl for sure. I think stable or not, that's semi-stable, like eight or nine or something like that. During my PhD it was even later; I used to go to sleep usually at 3am. I think the AM hours are precious and a very interesting time to work because everyone is asleep. At 8 AM or 7 AM, the East Coast is awake, so there's already activity, text messages, stuff happening. You can go to some news website and there's stuff happening; it's distracting. At 3am, everything is totally quiet, and so you're not going to be bothered, and you have solid chunks of time to do your work. So I like those periods. Night owl by default. Productive time basically: you need to build some momentum on the problem without too much distraction, and you need to load your RAM, your working memory, with that problem, and then you need to be obsessed with it. When you're taking a shower, when you're falling asleep, you need to be obsessed with the problem, and it's fully in your memory, and you're ready to wake up and work on it right there. So there's a scale of a single day or a couple of days, a week, a month. I can't talk about one day in isolation because it's a whole process. When I want to get productive on a problem, I feel like I need a span of a few days.

我真正能投入那个问题，不想被打扰，我会完全沉浸在那个问题里，那是我做出大部分好工作的地方。

Where I can really get in on that problem and I don't want to be interrupted and I'm going to just be completely obsessed with that problem and that's where I do most of my good work.

你在很短的时间内快速完成了很多很酷的小项目，这需要你全神贯注。

You've done a bunch of cool little projects in a very short amount of time very quickly, so that requires you just focusing on it.

是的，基本上我需要把问题加载到工作记忆中，并且要高效，因为处理任何问题都有巨大的固定成本。比如，我在特斯拉时就为此挣扎，因为我想做点小项目，但首先得弄清楚如何 SSH 到集群，打开 VS Code 编辑器才能工作，然后还会因为某些原因遇到愚蠢的错误。你无法立即高效工作，你会遇到障碍。所以关键在于真正消除所有障碍，能够进入问题，把整个问题加载到记忆中，并避免各种形式的干扰，比如新闻、邮件，还有来自之前或当前其他有趣项目的干扰。你只想真正集中注意力。我可以在中间抽时间处理干扰，但我觉得不能太多。你知道，一天的大部分时间都应该花在那个问题上。

Yeah, basically I need to load my working memory with the problem and I need to be productive because there's always a huge fixed cost to approaching any problem. You know, like I was struggling with this for example at Tesla because I want to work on small side projects but okay you first need to figure out okay I need to SSH into my cluster, I need to bring up a VS Code editor so I can work on this, I run into some stupid error because of some reason. You're not at a point where you can be just productive right away, you are facing barriers. And so it's about really removing all that barrier and you're able to go into the problem and you have the full problem loaded in your memory and somehow avoiding distractions of all different forms like news stories, emails, but also distractions from other interesting projects that you previously worked on or are currently working on. You just want to really focus your mind. And I mean I can take some time off for distractions in between, but I think it can't be too much. You know, most of your day is sort of spent on that problem.

然后你知道，我喝咖啡，有早上的例行公事，看些新闻、推特、黑客新闻、华尔街日报等等。所以基本上你醒来，喝点咖啡。你是想尽快开始工作吗？还是先看看世界上发生了什么？

And then you know, I drink coffee, I have my morning routine, I look at some news, Twitter, Hacker News, Wall Street Journal, etc. So basically you wake up, you have some coffee. Are you trying to get to work as quickly as possible? Do you take this diet of like what the hell's happening in the world first?

是的，我确实觉得了解世界很有趣。我不知道这是否有用或有益，但目前这是我例行公事的一部分，所以我会读一堆新闻文章，我想了解信息。我对此持怀疑态度，我怀疑这种做法，但目前我就是这样。

I am, I do find it interesting to know about the world. I don't know that it's useful or good, but it is part of my routine right now, so I do read through a bunch of news articles and I want to be informed. And I'm suspicious of it, I'm suspicious of the practice, but currently that's where I am.

哦，你是怀疑这种做法对你的生产力和幸福感有积极影响？

Oh, you mean suspicious about the positive effect of that practice on your productivity and your well-being?

是的，对我的心理幸福感，以及你深入理解世界的能力，因为有很多信息来源，你并没有真正专注于深度整合。

Yeah, on my well-being psychologically, and also on your ability to deeply understand the world because there's a bunch of sources of information, you're not really focused on deeply integrating.

这有点让人分心。

It's a little bit distracting.

是的，就一个完美高效的工作日而言，你一次尝试工作并专注于一件事多长时间？几个小时？一个小时？还是 30 分钟？10 分钟？

Yeah, in terms of a perfectly productive day, for how long of a stretch of time in one session do you try to work and focus on a thing? It's a couple hours? Is it one hour? Or 30 minutes? Is 10 minutes?

我大概能持续几个小时，然后中间需要休息，比如吃点东西什么的。是的，但我觉得积累工作时间仍然很难。我用过一个追踪器，它能准确告诉我每天花在编码上的时间，即使是非常高效的一天，我也只花了六到八个小时。是的，这只是因为有很多填充时间：通勤、与人交谈、吃饭等等。生活的成本，仅仅是活着、维持、稳态，以及维持作为人的状态，非常高。

I can probably go like a small few hours and then I need some breaks in between for like food and stuff. And yeah, but I think it's still really hard to accumulate hours. I was using a tracker that told me exactly how much time I've spent coding any one day and even on a very productive day I still spent only like six or eight hours. Yeah, and it's just because there's so much padding: commute, talking to people, food, etc. There's like the cost of life, just living and sustaining and homeostasis and just maintaining yourself as a human is very high.

而且人类心智中似乎有一种参与社会的欲望，这创造了那些填充时间。

And there seems to be a desire within the human mind to participate in society that creates that padding.

是的，因为我最有效率的日子就是从头到尾完全屏蔽一切，只是坐在那里，然后你可以做超过六到八个小时。

Yeah, because the most productive days I've ever had is just completely from start to finish just tuning out everything, just sitting there, and then you could do more than six or eight hours.

有没有什么智慧能让你有力量度过艰难的长专注日？

Is there some wisdom about what gives you strength to do tough days of long focus?

是的，每当我痴迷于一个问题时，就是某样东西需要工作，某样东西需要存在，它必须存在。所以你能处理 bug、编程问题、技术问题以及最终证明是错误的决策，你能思考所有这些，因为你希望它存在。是的，它必须存在。然后我认为对我来说另一个重要因素是，其他人会欣赏它吗？他们会喜欢它吗？那是我动力的一大部分。如果我在帮助人类，他们看起来很开心，他们说好话，他们在推特上谈论它等等，那会给我快乐，因为我在做有用的事情。

Yeah, just whenever I get obsessed about a problem, something just needs to work, something just needs to exist, it needs to exist. And so you're able to deal with bugs and programming issues and technical issues and design decisions that turn out to be the wrong ones, you're able to think through all of that given that you want it to exist. Yeah, it needs to exist. And then I think to me also a big factor is, you know, are other humans going to appreciate it? Are they going to like it? That's a big part of my motivation. If I'm helping humans and they seem happy, they say nice things, they tweet about it or whatever, that gives me pleasure because I'm doing something useful.

所以你确实会把自己分享给世界，比如在 GitHub 上写博客或通过视频。

So you do see yourself sharing it with the world, like on GitHub with a blog post or through videos.

是的，我在想，假设我做了所有这些事情但没有分享，我不认为我能积累同样的动力。你喜欢别人从你创造的东西中获得价值和快乐的感觉。

Yeah, I was thinking about it like suppose I did all these things but did not share them, I don't think I would have the same amount of motivation that I can build up. You enjoy the feeling of other people gaining value and happiness from the stuff you've created.

那饮食呢？我看到你玩过间歇性禁食。你禁食吗？这对所有事情都有帮助吗？

What about diet? Is there... I saw you playing with intermittent fast. Do you fast? Does that help with everything?

嗯，你玩过的东西，什么对你精神集中、精神生产力和幸福感最有益？你还在禁食吗？

Well, the things you played, what's been most beneficial to your ability to mentally focus on a thing and just mental productivity and happiness? You still fast?

是的，它很快，但我做间歇性禁食。但归根结底，这意味着我不吃早餐。是的，所以我在稳定状态下默认大约做 18:6。如果我在旅行或做其他事情，我会打破规则，但在稳定状态下我坚持 18:6，所以我只在中午 12 点到下午 6 点之间吃东西。

Yeah, it's so fast but I do intermittent fasting. But really what it means at the end of the day is I skip breakfast. Yeah, so I do 18:6 roughly by default when I'm in my steady state. If I'm traveling or doing something else I will break the rules, but in my steady state I do 18:6, so I eat only from 12 to 6.

这不是硬性规定，我经常打破它，但这是我的默认状态。然后，嗯，我做过很多随机实验。目前来说，过去一年半里，我想说我是植物性饮食或植物优先。我听说‘植物优先’听起来更好。

Not a hard rule and I break it often, but that's my default. And then, yeah, I've done a bunch of random experiments. For the most part right now, where I've been for the last year and a half, I want to say is I'm plant-based or plant-forward. I heard 'plant forward' sounds better.

没错。我其实不知道区别，但听起来更好。这只是说我更喜欢植物性食物，生的或熟的。我更喜欢熟的。植物性。哦，原谅我，我其实不知道植物这个类别有多广。健康只是说你不是……而且你可以灵活处理。你只是更喜欢吃植物。你不想影响别人。如果有人来你家派对，给你端上他们引以为豪的牛排，你会吃的。是的，对吧。这只是不评判。哦，那很美。

Exactly. I didn't actually know the difference, but it sounds better in my mind. It just means I prefer plant-based food, raw or cooked. I prefer cooked. And plant-based. Oh, forgive me, I don't actually know how wide the category of plant entails. Wellness just means that you're not... and you can flex. You just prefer to eat plants. You're not trying to influence other people. If someone comes to your house party and they serve you a steak that they're really proud of, you will eat it. Yes, right. It's just not judgmental. Oh, that's beautiful.

另一方面，我非常灵活。你试过一日一餐吗？

On the flip side of that, I'm very sort of flexible. Have you tried doing one meal a day?

我试过，偶然地，不是持续地。但我偶然试过。我不喜欢。我觉得它让我感觉不好。冲击太大了。所以目前我一天两餐，12 点和 6 点。我一直这样。我现在就在做。我一天一餐。好吧，所以这很有趣。这是一种有趣的感觉。

I have, accidentally, not consistently. But I've accidentally had that. I don't like it. I think it makes me feel not good. It's too much of a hit. So currently I have about two meals a day, 12 and 6. I do that non-stop. I'm doing it now. I'm doing one meal a day. Okay, so it's interesting. It's an interesting feeling.

你禁食超过一天过吗？

Have you ever fasted longer than a day?

是的，我做过很多次清水禁食，因为我好奇会发生什么。有什么有趣的？是的，我会这么说。你知道有趣的是什么吗？你饿了两天，然后从第三天左右开始，你就不饿了。这是一种很奇怪的感觉，因为你几天没吃东西却不饿。是不是很奇怪？这真的是人类生物学中许多奇怪的事情之一。身体会想办法，它找到另一种能量来源之类的东西，或者放松系统。我不知道怎么做到的。身体就像：你饿了，你饿了，然后它就放弃了。它说，好吧，我想我们现在在禁食。什么都没有了。然后它就专注于让你不饿，不让你感觉到那种伤害，并试图给你一些空间来解决食物问题。

Yeah, I've done a bunch of water fasts because I was curious what happens. Anything interesting? Yeah, I would say so. You know what's interesting is that you're hungry for two days, and then starting day three or so, you're not hungry. It's such a weird feeling because you haven't eaten in a few days and you're not hungry. Isn't that weird? It's really one of the many weird things about human biology. The body figures something out, it finds another source of energy or something like that, or relaxes the system. I don't know how. The body is like: you're hungry, you're hungry, and then it just gives up. It's like, okay, I guess we're fasting now. There's nothing. And then it just kind of focuses on trying to make you not hungry, and not feel the damage of that, and trying to give you some space to figure out the food situation.

直到今天，你还是在晚上效率最高吗？

Are you still to this day most productive at night?

我会说是的，但很难维持，尤其是我的博士日程，特别是当我在特斯拉工作的时候。这根本行不通。但即使是现在，人们想为各种活动见面。社会生活在某个时间段，你不得不适应。所以很难先参加社交活动，然后回来工作。是的，这真的很难。这就是为什么我尽量参加社交活动，尽量不喝太多酒，这样我就能回来继续工作。

I would say I am, but it is really hard to maintain with my PhD schedule, especially when I was working at Tesla and so on. It's a non-starter. But even now, people want to meet for various events. Society lives in a certain period of time, and you sort of have to work around that. So it's hard to do a social thing and then after that return and do work. Yeah, it's just really hard. That's why I try to do social things, I try not to do too much drinking, so I can return and continue doing work.

在特斯拉，大家会趋同于一个时间表吗？还是说人类合作时就是这样？他们试图保持一致的作息，同时醒着？

At Tesla, is there a convergence towards the schedule? Or is that how humans behave when they collaborate? Do they try to keep a consistent schedule, all awake at the same time?

我确实试图建立一个常规和稳定状态，让我感到舒适。所以我有早间常规、白天常规。我尽量让事情保持稳定、可预测。然后你的身体就会适应。如果你给它太大压力，当你旅行和倒时差时就会出问题，你无法达到你需要达到的状态。

I do try to create a routine and a steady state in which I'm comfortable. So I have a morning routine, a day routine. I try to keep things in a steady state, predictable. Then your body just sticks to that. If you stress it too much, it will create issues when you're traveling and dealing with jet lag, you're not able to ascend to where you need to go.

你如何看待人一生中的工作与生活平衡？特斯拉以在工作量上把人推向极限而闻名。

What are your thoughts on work-life balance throughout a human lifetime? Tesla was known for pushing people to their limits in terms of how much they work.

我会说特斯拉在这方面仍然名声不好。实际情况是，特斯拉是一个爆发性环境。基准线，我唯一的参照是谷歌，我在那里实习过三次，我看到了谷歌和 DeepMind 内部的样子。我会说基准线比那更高，但然后有一个间断平衡，时不时会有火灾，人们工作非常努力。所以它是尖峰和爆发式的。所有的故事都集中在爆发上，然后给人一种完全疯狂的表象。但实际上只是一个更紧张的环境，有火灾和冲刺。

I will say Tesla still has a bad rep for this. What's happening is Tesla is a bursting environment. The baseline, my only point of reference is Google where I interned three times, and I saw what it's like inside Google and DeepMind. I would say the baseline is higher than that, but then there's a punctuated equilibrium where once in a while there's a fire, and people work really hard. So it's spiky and bursty. All the stories get collected about the bursts, and then it gives the appearance of total insanity. But actually it's just a bit more intense environment, and there are fires and sprints.

抛开那些。在你自己的个人生活中，你如何看待像你这样聪明的人在工作和生活之间找到平衡的幸福？还是说这不是一个好的思想实验？

Forget all that. In your own personal life, what do you think about the happiness of a brilliant person like yourself finding a balance between work and life? Or is it not a good thought experiment?

我认为平衡是好的，但我也喜欢有超出常规的冲刺。我认为这让我很有创造力。超出常规的冲刺意味着大多数时候你有一个所谓的平衡。我大多数时候是平衡的。是的，我喜欢偶尔对某件事着迷。偶尔是多久？一周一次？一个月一次？一年一次？大概一个月一次吧。那就是我们发布新 GitHub 仓库的时候。那是你真的在乎一个问题的时候。它必须存在，这将会很棒。你着迷于它，然后你不能只在当天做。你需要付出固定成本进入状态，然后你需要在那里待一段时间。然后社会会来打扰你，分散你的注意力。最糟糕的是有人说：‘我只需要你五分钟时间。’那代价不是五分钟。社会需要改变对‘就五分钟’的看法。从来不是一分钟。只是 30 秒，只是很快，有什么大不了的，你为什么这么……

I think balance is good, but I also love to have sprints that are out of distribution. That's what I think I've been pretty creative. Sprints out of distribution means that most of the time you have a quote-unquote balance. I have balance most of the time. Yes, I like being obsessed with something once in a while. Once in a while is what? Once a week? Once a month? Once a year? Probably like once a month or something. That's when we get a new GitHub repo. That's when you really care about a problem. It must exist, this will be awesome. You're obsessed with it, and now you can't just do it on that day. You need to pay the fixed cost of getting into the groove, and then you need to stay there for a while. Then society will come and try to mess with you and distract you. The worst thing is a person who says, 'I just need five minutes of your time.' The cost of that is not five minutes. Society needs to change how it thinks about 'just five minutes of your time.' It's never just one minute. It's just 30, it's just a quick, what's the big deal, why are you being so...

你的电脑设置是什么？完美的设置是什么？你是那种灵活的人，不管什么笔记本电脑、四个屏幕？还是你更喜欢某种让你最高效的设置？

What's your computer setup? What's the perfect setup? Are you somebody that's flexible, no matter what laptop, four screens? Or do you prefer a certain setup that you're most productive?

我想我最熟悉的是一个 27 英寸的大屏幕，旁边放我的笔记本电脑。操作系统：我用 Mac。这是我所有任务的主要设备。我会说 OS X。但当你做深度学习时，一切都是 Linux。你通过 SSH 连接到集群，远程工作。对于实际开发，使用 IDE，一个好方法是：你在 Mac 上运行 VS Code，但实际上你通过 SSH 在远程文件夹上工作。

I guess the one that I'm familiar with is one large screen, 27 inch, and my laptop on the side. Operating system: I do Mac. That's my primary for all tasks. I would say OS X. But when you're working on deep learning, everything is Linux. You SSH into a cluster and you're working remotely. For actual development, using the IDE, you would use a good way: you just run VS Code on your Mac, but you are actually working on a remote folder through SSH.

你实际操作的文件都在集群上，所以最好的 IDE 是什么？VS Code？人们还用别的吗？

The actual files that you're manipulating are on the cluster somewhere else, so what's the best IDE? VS Code? What else do people use?

我还在用 Emacs。

I use Emacs still.

那很酷。

That's cool.

可能挺酷的，但不知道是不是最高效。你推荐什么编辑器？你和很多软件工程师合作过，他们用 Python、C++、机器学习应用的编辑器。我觉得目前答案是 VS Code。我相信这是目前最好的 IDE。它有大量扩展，还有 GitHub Copilot 集成，我觉得非常有价值。

So it may be cool, I don't know if it's maximum productivity. What do you recommend in terms of editors? You worked with a lot of software engineers, editors for Python, C++, machine learning applications. I think the current answer is VS Code. Currently I believe that's the best IDE. It's got a huge amount of extensions, it has a GitHub Copilot integration which I think is very valuable.

你怎么看 Copilot 集成？

What do you think about the Copilot integration?

我实际上和 Python 之父 Guido van Rossum 聊了很多，他喜欢 Copilot。他经常用它编程。是的，我喜欢它。对我来说是免费的，但我会付费。我觉得非常好。我发现它的用处是，有个学习曲线。你需要弄清楚什么时候它有用，什么时候该关注它的输出，什么时候它没用，不该关注。因为如果你一直读它的建议，那不是好的交互方式。但我能让自己适应它。我发现它很有帮助，第一，在复制粘贴和替换部分内容时，当模式清晰时，它很擅长完成模式。第二，有时它会建议我不知道的 API，告诉你一些你不知道的东西。这是个发现的机会。我从不直接接受 Copilot 的代码。我几乎总是把它复制到谷歌搜索，看看这个函数是做什么的，然后你会说，‘哦，这确实是我需要的，谢谢 Copilot。’所以你学到了东西。它部分像搜索引擎，部分像帮你得到正确的语法，一旦看到你就知道。

I actually got to talk a bunch with Guido van Rossum, who's the creator of Python, and he loves Copilot. He programs a lot with it. Yeah, I love it. It's free for me, but I would pay for it. I think it's very good. The utility I found with it is that there is a learning curve. You need to figure out when it's helpful and when to pay attention to its outputs, and when it's not going to be helpful, where you should not pay attention. Because if you're just reading its suggestions all the time, it's not a good way of interacting. But I think I was able to mold myself to it. I find it's very helpful, number one, in copy-paste and replace some parts, so when the pattern is clear, it's really good at completing the pattern. Number two, sometimes it suggests APIs that I'm not aware of, so it tells you about something you didn't know. That's an opportunity to discover. I would never take Copilot code as given. I almost always copy it into a Google search and see what this function is doing, and then you're like, 'Oh, it's actually exactly what I need, thank you Copilot.' So you learned something. It's in part a search engine, part maybe getting the exact syntax correctly that once you see it, you know.

这就是那个 NP 难问题：一旦看到，你就知道。

It's that NP-hard thing: once you see it, you know.

完全正确。你自己可以挣扎，可以高效验证，但无法高效生成。而 Copilot 真的是编程的自动驾驶仪，对吧？目前它在做链接跟随，就像简单的复制粘贴和偶尔的建议。但随着时间的推移，它会变得越来越自主。同样的事情不仅会发生在编程上，可能还会发生在很多不同的事情上。但编程是一个重要的领域，比如写程序。

Exactly correct. You yourself can struggle, you can verify efficiently, but you can't generate efficiently. And Copilot really is autopilot for programming, right? Currently it's doing the link following, which is like simple copy-paste and sometimes suggestions. But over time it's going to become more and more autonomous. The same thing will play out not just in coding but across many different things probably. But coding is an important one, like writing programs.

你怎么看这个未来的发展？程序合成，能够写出越来越复杂的程序。现在它是以有趣的方式由人类监督的。感觉会怎样？过渡会很痛苦。

How do you see the future of that developing? Program synthesis, being able to write programs that are more and more complicated. Right now it's human-supervised in interesting ways. What does it feel like? The transition will be very painful.

我的思维模型是，同样的事情会像自动驾驶仪一样发生。目前它在做链接跟随，做一些简单的事情，最终我们会实现自主，人们需要干预的越来越少。可能会有测试机制。如果它写了一个函数，看起来非常正确，但你怎么知道它正确？因为作为程序员你越来越懒。你发现 bug 的能力会下降。但我想它不会犯小错误？不，它会。Copilot 会犯 off-by-one 的细微错误。它对我做过。

My mental model for it is the same thing will happen as with the autopilot. Currently it's doing link following, doing some simple stuff, and eventually we'll be doing autonomy, and people will have to intervene less and less. There could be testing mechanisms. If it writes a function and that function looks pretty damn correct, but how do you know it's correct? Because you're getting lazier and lazier as a programmer. Your ability to catch bugs diminishes. But I guess it won't make little bugs? No, it will. Copilot will make off-by-one subtle bugs. It has done that to me.

你认为未来的系统会改进，还是 off-by-one 实际上是编程的根本挑战？

Do you think future systems will improve, or is the off-by-one actually a fundamental challenge of programming?

在那个情况下它不是根本性的，我认为事情可以改进。但是的，我认为人类必须监督。我担心人们不监督输出，以及这会导致什么，比如所有系统中 bug 的泛滥。我担心这个。但我认为将来可能会有其他用于找 bug 的 copilot，因为会有更多自动化。所以就像一个程序：一个 copilot 生成代码，一个编译器，一个做 linter，一个做类型检查器。这是一个 GPT 委员会。然后会有一个委员会经理。然后会有人说需要新版本，我们需要重新生成。有 10 个 GPT 提交了 50 条建议，另一个看了并挑了几个喜欢的，一个 bug 检查的看了说可能是个 bug，它们被其他东西重新排序，然后一个最终的集成 GPT 过来说，‘好了，根据你们告诉我的所有信息，这可能是下一个 token。’

In that case it wasn't fundamental, and I think things can improve. But yeah, I think humans have to supervise. I am nervous about people not supervising what comes out and what happens to, for example, the proliferation of bugs in all of our systems. I'm nervous about that. But I think there will probably be some other copilots for bug finding and stuff like that at some point, because there will be a lot more automation. So it's like a program: a copilot that generates code, a compiler, one that does a linter, one that does a type checker. It's a committee of GPTs. And then there'll be a manager for the committee. And then there'll be somebody that says a new version of this is needed, we need to regenerate it. There were 10 GPTs that forwarded and gave 50 suggestions, another one looked at it and picked a few that they liked, a bug one looked at it and said it's probably a bug, they got re-ranked by some other thing, and then a final ensemble GPT comes in and says, 'Okay, given everything you guys have told me, this is probably the next token.'

世界上的程序员数量一直在快速增长。你认为在这种世界里，它有可能趋于平稳并下降到非常低的数字吗？因为那时你会做软件 2.0 编程，你会做这种生成 copilot 类型系统的编程，但不会做老式的软件 1.0 编程。

The number of programmers in the world has been growing very quickly. Do you think it's possible that it'll actually level out and drop to a very low number with this kind of world? Because then you'll be doing Software 2.0 programming, and you'll be doing this kind of generation of copilot-type systems programming, but you won't be doing the old school Software 1.0 programming.

我目前不认为它们会取代人类程序员。我很犹豫说这样的话，因为这在五年内就会被取代。我不知道。它会证明这就是我们想的那样。因为我同意你，但我觉得我们可能会非常惊讶。

I don't currently think that they're just going to replace human programmers. I'm so hesitant saying stuff like this right because this is going to be replaced in five years. I don't know. It's going to show that this is where we thought. Because I agree with you, but I think we might be very surprised.

你对语言模型的现状有什么感觉？感觉是开始、中间还是结束？

What's your sense of what we're seeing with language models? Does it feel like the beginning, the middle, or the end?

100% 是开始。我认为我心中的大问题是：GPT 肯定能很好地编程，胜任等等。你怎么引导系统？你仍然需要提供一些指导，说明你真正想要什么。所以你怎么引导它？你怎么和它说话？你怎么审计和验证它做的事情是正确的？你怎么和它合作？这不仅是 AI 问题，也是 UI/UX 问题。所以 VS Code++ 有非常美好、肥沃的土壤，可以做很多有趣的工作，不再只是人类编程。太棒了。你在与系统交互，不是一次提示，而是迭代提示。你试图与系统对话。这对我来说非常令人兴奋：和我正在写的程序对话。也许在某个时候你只是和它对话：‘好了，我想做这个。’实际上，这个变量，也许甚至不是变量那么低级。你还可以想象，‘你能把这个翻译成 C++ 再翻译回 Python 吗？’这已经存在了，但只是作为编程体验的一部分。比如，‘我想用 C++ 写这个函数’，或者你只是为不同的程序不断改变。

The beginning, 100%. I think the big question in my mind is: for sure GPT will be able to program quite well, competently, and so on. How do you steer the system? You still have to provide some guidance to what you actually are looking for. So how do you steer it? How do you talk to it? How do you audit it and verify that what is done is correct? How do you work with this? It's as much not just an AI problem but a UI/UX problem. So beautiful, fertile ground for so much interesting work for VS Code++, where you're not just human programming anymore. It's amazing. You're interacting with the system, not just one prompt but iterative prompting. You're trying to figure out having a conversation with the system. That to me is super exciting: to have a conversation with the program I'm writing. Maybe at some point you're just conversing with it: 'Okay, here's what I want to do.' Actually, this variable, maybe it's not even that low level as variable. You can also imagine, 'Can you translate this to C++ and back to Python?' That already kind of exists, but just doing it as part of the programming experience. Like, 'I think I'd like to write this function in C++' or you just keep changing for different programs.

不同的语法，也许我想把它转换成函数式语言，这样你作为程序员就能掌握多种语言，高效地来回切换。

Different syntax maybe I want to convert this into a functional language and so like you get to become multilingual as a programmer and dance back and forth efficiently.

是的，我的意思是，我认为它的用户界面/用户体验仍然很难想清楚，因为这不仅仅是在页面上写代码。你有一个完整的开发环境，上面有一堆硬件，有一些环境变量，有一些在 Chrome 任务中运行的脚本。与计算机打交道有很多事情要做。这些系统如何设置环境标志、跨多台机器工作、设置 screen 会话以及自动化不同流程？这一切如何运作并且可被人类审计，是目前的一个大问题。

Yeah, I mean, I think the UI/UX of it though is still very hard to think through because it's not just about writing code on a page. You have an entire developer environment, you have a bunch of hardware on it, you have some environmental variables, you have some scripts that are running in the Chrome job. Like, there's a lot going on to like working with computers. And how do these systems set up environment flags and work across multiple machines and set up screen sessions and automate different processes? How all that works and it's auditable by humans and so on is a massive question at the moment.

你建立了 ArchiveSanity。什么是 Archive？你希望看到的学术研究出版的未来是什么？

You've built ArchiveSanity. What is Archive and what is the future of academic research publishing that you would like to see?

Archive 是一个预印本服务器。如果你有一篇论文，你可以提交给期刊或会议发表，然后等六个月，也许能得到通过或不通过的决定。或者你可以直接上传到 Archive，然后三分钟后人们就可以在推特上讨论它，每个人都能看到它、阅读它，并以自己的方式从中受益。你可以引用它，它看起来也很正式。感觉就像是一个出版过程。

So Archive is this preprint server. So if you have a paper, you can submit it for publication to journals or conferences and then wait six months and then maybe get a decision pass or fail. Or you can just upload it to Archive and then people can tweet about it three minutes later and then everyone sees it, everyone reads it, and everyone can profit from it in their own ways. You can cite it and it has an official look to it. It feels like a publication process.

是的，感觉和把它放在博客文章里不一样。

Yeah, it feels different than if you just put it in a blog post.

哦，是的。我的意思是，它是一篇论文，通常你对 Archive 上的东西的期望门槛比博客文章要高。

Oh yeah, yeah. I mean, it's a paper and usually the bar is higher for something that you would expect on Archive as opposed to something you would see in a blog post.

嗯，是文化创造了这个门槛，因为你完全可以在 Archive 上放一篇很差的论文。

Well, the culture created the bar because you could probably host a pretty crappy paper on Archive.

嗯，那这让你对同行评审有什么看法？由两三位专家进行的严格同行评审，与社区的同行评审相比，对吧，就像现在这样？是的，基本上我认为社区完全有能力在推特上非常快速地进行同行评审。而且我认为这可能只是与 AI 机器学习领域特别相关。我觉得事情更容易被审计，验证也可能比其他地方更容易。所以这有点像你可以把这些科学出版物看作是小型的区块链，每个人都在彼此的工作基础上构建并互相引用。而 AI 有点像是一个更快更松散的区块链，但任何一个单独的条目都非常便宜。然后还有其他领域，这种模式可能不太适用。所以我认为至少在 AI 领域，事情是相当容易验证的。这就是为什么当人们上传有很好想法的论文时，其他人第二天就可以尝试，并且可以成为最终仲裁者，判断它是否适用于他们的问题。整个事情进展得明显更快。所以我感觉学术界仍然有一席之地——抱歉，这个会议/期刊过程仍然有一席之地——但它有点滞后了。而且它可能是一个质量稍高的过程，但它不再是发现前沿工作的地方了。

Um, so what does that make you feel about peer review? Rigorous peer review by two or three experts versus the peer review of the community, right, as it's written? Yeah, basically I think the community is very well able to peer review things very quickly on Twitter. And I think maybe it just has to do with the AI machine learning field specifically though. I feel like things are more easily auditable and the verification is easier potentially than verification somewhere else. So it's kind of like you can think of these scientific publications as little blockchains where everyone's building on each other's work and citing each other. And you sort of have AI, which is kind of like this much faster and looser blockchain, but then any one individual entry is very cheap to make. And then you have other fields where maybe that model doesn't make as much sense. And so I think in AI at least, things are pretty easily verifiable. And so that's why when people upload papers with a really good idea and so on, people can try it out the next day and they can be the final arbiter of whether it works or not on their problem. And the whole thing just moves significantly faster. So I kind of feel like academia still has a place—sorry, this conference/journal process still has a place—but it's sort of like it lags behind, I think. And it's a bit more maybe higher quality process, but it's not sort of the place where you will discover cutting-edge work anymore.

是的，当我开始读博士的时候，情况是你会去参加会议和期刊，讨论所有最新的研究。现在你去参加会议或期刊，没有人讨论那里的任何东西，因为它已经像三代以前的东西了，过时了。

Yeah, it used to be the case when I was starting my PhD that you go to conferences and journals and you discuss all the latest research. Now when you go to a conference or journal, like no one discusses anything that's there because it's already like three generations ago, irrelevant.

是的，这让我对 DeepMind 这样的公司感到遗憾，他们仍然在《自然》等大型知名期刊上发表文章。我的意思是，这些大型场所带来的声望之外，仍然有价值。但结果是，他们会宣布一些突破性的性能，然后需要大约一年才能真正发表细节。而这些细节，如果立即发表，会激励社区朝着某些方向发展。

Yes, which makes me sad about like DeepMind for example, where they still publish in Nature and these big prestigious venues. I mean, there's still value, as opposed to the prestige that comes with these big venues. But the result is that they'll announce some breakthrough performance and it'll take like a year to actually publish the details. And those details, if they were published immediately, would inspire the community to move in certain directions with that.

是的，这会加速社区的其他部分，但我不知道这在多大程度上是他们的目标函数的一部分。这也是事实，所以不仅仅是声望。延迟也是其中的一部分。

Yeah, it would speed up the rest of the community, but I don't know to what extent that's part of their objective function. Also that's true, so it's not just the prestige. A little bit of the delay is part of it.

是的，他们确实，特别是 DeepMind，一直在采用一种质量稍高、延迟稍长的流程，并以这种方式发表那些论文。

Yeah, they certainly, DeepMind specifically, has been working in the regime of having a slightly higher quality process and latency and publishing those papers that way.

Reddit 上的另一个问题：你是否有过冒名顶替综合症？作为特斯拉的 AI 总监，在斯坦福时，世界视你为 AI 专家，教全世界机器学习？

Another question from Reddit: Do you or have you suffered from imposter syndrome? Being the director of AI at Tesla, being this person when you're at Stanford, where the world looks at you as the expert in AI to teach the world about machine learning?

当我在五年后离开特斯拉时，我花了很多时间在会议室里。你知道，一开始我会读论文。当我加入特斯拉时，我在写代码，然后我写的代码越来越少，我在读代码，然后我读的代码也越来越少。所以我认为这是一个自然的过程。而且，我可以说，在接近尾声的时候，你会开始更多地意识到你应该是专家，但实际上真理的来源是人们在 GitHub 上写的代码，代码本身，而你不再像以前那样熟悉它了。所以我想说，也许那里有一些不安全感。

When I was leaving Tesla after five years, I spent a ton of time in meeting rooms. And you know, I would read papers in the beginning. When I joined Tesla, I was writing code, and then I was writing less and less code, and I was reading code, and then I was reading less and less code. So this is just a natural progression that happens, I think. And definitely, I would say near the tail end, that's when it sort of starts to hit you a bit more that you're supposed to be an expert, but actually the source of truth is the code that people are writing on GitHub, the actual code itself, and you're not as familiar with that as you used to be. And so I would say maybe there's some insecurity there.

是的，这实际上相当深刻，很多不安全感都与在计算机科学领域不写代码有关，因为那就是真理。代码就是真理的来源。论文和其他一切都只是高层次的总结。我不……是的，只是高层次的总结。但归根结底，你必须阅读代码。不可能把所有代码都翻译成实际的论文形式。所以当有东西出来时，尤其是当它们有源代码可用时，那是我最喜欢去的地方。

Yeah, that's actually pretty profound that a lot of the insecurity has to do with not writing the code in the computer science space, like that because that is the truth. That right there, code is the source of truth. The papers and everything else, it's a high-level summary. I don't... yeah, just a high-level summary. But at the end of the day, you have to read code. It's impossible to translate all that code into actual paper form. So when things come out, especially when they have source code available, that's my favorite place to go.

就像我说的，你是机器学习 AI 有史以来最伟大的老师之一，从 CS231n 到现在。你会给有兴趣进入机器学习的初学者什么建议？

So like I said, you're one of the greatest teachers of machine learning AI ever, from CS231n to today. What advice would you give to beginners interested in getting into machine learning?

初学者通常关注的是做什么，而我认为重点应该更多是你做了多少。所以我在高层次上相信这个一万小时的概念，你只需要选择那些你可以花时间、你在乎并且感兴趣的事情。你真的需要投入一万小时的工作。甚至不太重要你把它花在哪里，你会迭代、改进，也会浪费一些时间。我不知道是否有更好的方法。你需要投入一万小时。但我认为这实际上非常好，因为我觉得如果你花一万小时，成为某方面的专家就有某种确定性。你可以随便选一个东西，我认为如果你投入一万小时的刻意努力和工作。

Beginners are often focused on like what to do, and I think the focus should be more like how much you do. So I'm kind of a believer on a high level in this 10,000 hours kind of concept, where you just kind of have to just pick the things where you can spend time and you care about and you're interested in. You literally have to put in 10,000 hours of work. It doesn't even matter as much where you put it, and you'll iterate and you'll improve and you'll waste some time. I don't know if there's a better way. You need to put in 10,000 hours. But I think it's actually really nice because I feel like there's some sense of determinism about being an expert at a thing if you spend 10,000 hours. You can literally pick an arbitrary thing and I think if you spend 10,000 hours of deliberate effort and work.

你实际上会变得擅长它。所以我认为这是一个不错的想法。基本上，我更关注的是：你是否投入了 1 万小时？这就是我关注的。然后思考什么样的机制能最大化你达到 1 万小时的可能性。对我们这些愚蠢的人类来说，这可能意味着养成每天实际做这件事的习惯，不管是什么能帮到你。所以我确实认为这在很大程度上是一个心理问题。

You actually will become an expert at it. So I think it's kind of a nice thought. Basically, I would focus more on: are you spending 10,000 hours? That's what I focus on. Then think about what mechanisms maximize your likelihood of getting to 10,000 hours. For us silly humans, that probably means forming a daily habit of actually doing the thing every single day, whatever helps you. So I do think to a large extent it's a psychological problem for yourself.

另一件我认为对心理有帮助的事情是：很多时候人们会拿自己和这个领域的其他人比较。我认为这非常有害。只和以前的自己比较，比如一年前的自己。你比一年前的自己更好吗？这是唯一的思考方式。而且我认为这样你就能看到自己的进步，这非常激励人。

One other thing that I think is helpful for the psychology of it is: many times people compare themselves to others in the area. I think this is very harmful. Only compare yourself to you from some time ago, like say a year ago. Are you better than you a year ago? This is the only way to think. And I think then you can see your progress and it's very motivating.

你关注小时数量，这很有意思，因为我认为很多人，在初级阶段甚至整个过程中，都会被选择瘫痪：我该选这条路还是那条路？他们甚至会被选哪个 IDE 而瘫痪。他们担心所有这些事情。但问题是，有些时候你会浪费时间做错事。是的，你最终会发现它不对。你会积累伤疤，下次你会变得更强大，因为你有伤疤并从中学习。下次你遇到类似情况，你会想，‘好吧，我搞砸了。’我花了很多时间做那些从未实现的事情，我有了那些伤疤，并对什么有用、什么没用、事情如何发展有了一些直觉。所以所有这些错误都不是无用功。所以我认为你应该专注于工作。你上周做了什么？这是一个很好的问题，不仅适用于机器学习。这是一个很好的方式来剔除废话、赘肉、生活中的低效。

That's so interesting that you focus on the quantity of hours, because I think a lot of people, in the beginner stage but actually throughout, get paralyzed by the choice: which one do I pick, this path or that path? They'll literally get paralyzed by which IDE to use. They're worried about all these things. But the thing is, some of the time you will waste time doing something wrong. Yes, you will eventually figure out it's not right. You will accumulate scar tissue, and next time you'll grow stronger because you'll have the scar tissue and learn from it. Next time you come into a similar situation, you'll be like, 'Alright, I messed up.' I've spent a lot of time working on things that never materialized into anything, and I have all that scar tissue and some intuitions about what was useful, what wasn't, how things turned out. So all those mistakes were not dead work, you know. So I just think you should just focus on working. What have you done last week? That's a good question to ask for a lot of things, not just machine learning. It's a good way to cut the fluff, the blubber, the inefficiencies in life.

你喜欢教学的什么？你似乎经常被教学吸引。你非常擅长，但你也喜欢它。

What do you love about teaching? You seem to find yourself often drawn to teaching. You're very good at it, but you're also drawn to it.

我不认为我喜欢教学。我喜欢快乐的人，而快乐的人喜欢我教他们。是的，我不会说我讨厌教学；我容忍教学。但我喜欢的不是教学这个行为。而是我有一些我确实擅长的事情，人们非常欣赏。所以我只是乐于尝试提供帮助。教学本身并不是最……我的意思是，它可能非常烦人和令人沮丧。我刚才在准备一系列讲座，这让我想起了我教 231 课程的日子，以及创建这些材料并让它们变得优秀需要多少工作。迭代和思考的量，你会走进死胡同，以及你做了多少改动。所以从教育价值的角度创造好的东西真的很难，而且不好玩；它很困难。

I don't think I love teaching. I love happy humans, and happy humans like when I teach. Yes, I wouldn't say I hate teaching; I tolerate teaching. But it's not the act of teaching that I like. It's that I have something I'm actually okay at, and people appreciate it a lot. So I'm just happy to try to be helpful. Teaching itself is not the most... I mean, it can be really annoying and frustrating. I was working on a bunch of lectures just now, and I was reminded back to my days of 231 and just how much work it is to create some of these materials and make them good. The amount of iteration and thought, and you go down blind alleys, and just how much you change it. So creating something good in terms of educational value is really hard, and it's not fun; it's difficult.

所以人们绝对应该去看你发布的新内容。有些讲座中你实际上是从零开始构建东西，就像你说的‘最冷的真相’，通过构建来讨论反向传播，审视它，整个过程。准备这个有多难？我认为这是一种非常强大的教学方式。你是如何准备的，还是你只是现场思考？

So people should definitely go watch your new stuff you put out. There are lectures where you're actually building the thing from scratch, like you said, 'the coldest truth,' discussing backpropagation by building it, looking through it, the whole thing. How difficult is that to prepare for? I think that's a really powerful way to teach. How did you have to prepare for that, or are you just live thinking through it?

我通常会做，比如说，三遍，然后我选比较好的那一遍。所以我做多遍，选一些比较好的，然后用那种方式构建一个讲座。有时我不得不删除 30 分钟的内容，因为它走向了一个我不太喜欢的路径。有很多迭代，我大概需要 10 个小时来创作一小时的视频。

I will typically do, say, three takes, and then I take the better take. So I do multiple takes and take some of the better ones, and then I just build out a lecture that way. Sometimes I have to delete 30 minutes of content because it just went down a path I didn't like too much. There's a bunch of iteration, and it probably takes me somewhere around 10 hours to create one hour of content.

这很有趣。我的意思是，回到基础难吗？你从回到基础中汲取了很多智慧吗？

It's interesting. I mean, is it difficult to go back to the basics? Do you draw a lot of wisdom from going back to the basics?

是的，回到反向传播、损失函数，它们来自哪里。老实说，我非常喜欢教学的一点是，它确实能加强你的理解。所以这不是纯粹利他的活动；这是一种学习方式。如果你必须向别人解释某事，你会意识到自己知识上的空白。所以我在那些讲座中甚至让自己惊讶。比如，‘哦，结果显然会是这样’，然后结果不是这样，我就想，‘好吧，我以为我理解这个。’但这就是为什么实际编码、在笔记本中运行它、得到结果真的很酷，你会想，‘哦，哇。’实际的数字、实际的输入、实际的代码。不是数学符号等等。真相的来源是代码，而不是幻灯片。就像，‘让我们构建它。’这很美。

Yeah, going back to backpropagation, loss functions, where they come from. And one thing I like about teaching a lot, honestly, is it definitely strengthens your understanding. So it's not a purely altruistic activity; it's a way to learn. If you have to explain something to someone, you realize you have gaps in knowledge. So I even surprised myself in those lectures. Like, 'Oh, the result will obviously look like this,' and then the result doesn't look like it, and I'm like, 'Okay, I thought I understood this.' But that's why it's really cool to literally code it, run it in a notebook, and it gives you a result, and you're like, 'Oh wow.' Actual numbers, actual input, actual code. It's not mathematical symbols, etc. The source of truth is the code, not slides. It's just like, 'Let's build it.' It's beautiful.

从这个意义上说，你是个罕见的人。你会给那些试图开发并发表对 AI 世界有重大影响的研究人员什么建议？比如本科生，或者研究生初期。

You're a rare human in that sense. What advice would you give to researchers trying to develop and publish ideas that have a big impact in the world of AI? So maybe undergrads, maybe early graduate students.

我的意思是，我会说他们肯定要比我作为博士生时更有策略，因为 AI 的发展方式。它正走向物理学的道路，在物理学中，你曾经可以在实验台上做实验，一切都很好，你可以取得进展。现在你必须像在 LHC 或 CERN 那样工作。所以 AI 也在朝那个方向发展。所以有些类型的事情已经不可能在实验台上完成了。而过去并非如此。

I mean, I would say they definitely have to be a little bit more strategic than I had to be as a PhD student, because of the way AI is evolving. It's going the way of physics, where in physics you used to be able to do experiments on your benchtop and everything was great, and you could make progress. Now you have to work in like LHC or CERN. And so AI is going in that direction as well. So there are certain kinds of things that are just not possible to do on the benchtop anymore. And that didn't used to be the case.

你仍然认为有像 GAN 那样的论文可以写吗？一个非常简单的想法只需要一台计算机就能说明一个简单的例子。

Do you still think that there are GAN-type papers to be written, where a very simple idea requires just one computer to illustrate a simple example?

最近一个非常有影响力的例子是扩散模型。扩散模型很惊人。扩散模型已经有六年了。据我所知，在很长一段时间里，人们有点忽视它们。而它们是一种惊人的生成模型，尤其是在图像方面。稳定扩散等等，都是基于扩散的。扩散是新的；它之前不存在，来自谷歌。但一个研究人员本可以提出它。事实上，一些最早的……实际上，那些也来自谷歌。但一个研究人员可以在学术机构提出它。

One example that's been very influential recently is diffusion models. Diffusion models are amazing. Diffusion models are six years old. For the longest time, people were kind of ignoring them, as far as I can tell. And they're an amazing generative model, especially in images. Stable diffusion and so on, it's all diffusion-based. Diffusion is new; it was not there, and came from Google. But a researcher could have come up with it. In fact, some of the first... actually, those came from Google as well. But a researcher could come up with that in an academic institution.

你觉得扩散模型最迷人的是什么？从社会影响到技术架构。

What do you find most fascinating about diffusion models? From the societal impact to the technical architecture.

我喜欢扩散的是它效果这么好。这让你惊讶吗？它生成的合成数据的多样性，几乎是新颖性。

What I like about diffusion is it works so well. Is that surprising to you? The amount of variety, almost the novelty of the synthetic data it generates.

是的，稳定扩散的图像令人难以置信。生成图像改进的速度已经疯狂了。

Yeah, the stable diffusion images are incredible. The speed of improvement in generating images has been insane.

从生成很小的数字和小小的面孔，看起来一团糟，到现在有了 Stable Diffusion，这一切发展得非常快。学术界仍然可以做出很多贡献。例如，Flash Attention 是一个非常高效的内核，用于在 Transformer 中运行注意力机制。它来自学术界，是一种非常巧妙的内核结构方式，使得计算不会显式地生成注意力矩阵。所以我认为还有很多可以贡献的地方，但你必须更有策略性。

It went very quickly from generating tiny digits and tiny faces, and it all looked messed up, and now we have Stable Diffusion. That happened very quickly. There's a lot that academia can still contribute. For example, Flash Attention is a very efficient kernel for running the attention operation inside the Transformer. That came from an academic environment. It's a very clever way to structure the kernel so that the calculation doesn't materialize the attention matrix. So I think there are still lots of things to contribute, but you have to be more strategic.

你认为神经网络能够被训练成会推理吗？

Do you think neural networks could be made to reason?

是的。

Yes.

你认为它们已经在推理了吗？

Do you think they already reason?

是的。

Yes.

你对推理的定义是什么？

What's your definition of reasoning?

以人类思考问题并得出新颖想法的方式进行信息处理。这感觉就像推理。新颖性，我不想说，但可以说是分布外的想法。你认为这可能吗？是的，我认为我们在当前的神经网络中已经看到了这一点。它们能够在某种意义上将训练集信息重新混合成真正的泛化。它看起来不像那样也没关系；你在算法上做了一些有趣的事情。你在操作一些符号，并在新的环境中得出一个正确且独特的答案。

Information processing in a way that humans think through a problem and come up with novel ideas. It feels like reasoning. The novelty, I don't want to say, but out-of-distribution ideas. You think it's possible? Yes, and I think we're seeing that already in current neural nets. They are able to remix the training set information into true generalization in some sense. It doesn't matter that it doesn't appear like that; you're doing something interesting algorithmically. You're manipulating some symbols and coming up with a correct, unique answer in a new setting.

什么会让你觉得‘天哪，这东西绝对在思考’？

What would illustrate to you: 'Holy shit, this thing is definitely thinking'?

对我来说，思考或推理只是信息处理和泛化，我认为神经网络今天已经做到了。能够感知世界或任何输入，并基于此做出预测或行动，这就是推理。你通过操作已学到的正确算法，在新的环境中给出正确答案。你不是在做某种查找表或最近邻搜索。

To me, thinking or reasoning is just information processing and generalization, and I think neural nets already do that today. Being able to perceive the world or whatever the inputs are, and to make predictions or actions based on that, that's reasoning. You're giving correct answers in novel settings by manipulating information you've learned the correct algorithm. You're not just doing some kind of lookup table or nearest neighbor search.

让我问问你关于 AGI 的看法。你认为有哪些登月计划式的想法可能推动 AGI 取得重大进展？或者换个角度，我们现在缺少的主要障碍是什么？

Let me ask you about AGI. What are some moonshot ideas you think might make significant progress towards AGI? Or maybe in other ways, what are big blockers that we're missing now?

基本上，我非常看好我们构建 AGI 的能力——我们可以与之交互的自动化系统，非常像人类，无论是在数字领域还是物理领域。目前，似乎大多数执行这些神奇任务的模型都处于文本领域。我认为，正如我提到的，我怀疑文本领域不足以真正建立对世界的完整理解。我确实认为你需要进入像素层面，理解物理世界及其运作方式。所以我认为我们需要扩展这些模型，让它们能够处理图像和视频，并在更多多模态数据上进行训练。如果你认为需要触摸世界才能理解它，那么，这在我心中是一个大问题：你是否还需要具身性和与世界互动、进行实验并获取此类数据的能力？如果是这样，那么你就需要 Optimus 或类似的东西。所以我认为 Optimus 在某种程度上是对 AGI 的一种对冲，因为在我看来，仅仅拥有来自互联网的数据可能是不够的。如果是这样，那么 Optimus 可能会通向 AGI。对我来说，没有比 Optimus 更远的了：你有了这个人形形态，可以实际在世界上做事；你可以有数百万个这样的机器人与人类互动等等。如果这都不能在某个时刻催生 AGI，我不知道还有什么能。所以从完备性的角度来看，我认为这是一个非常好的平台，但也是一个更困难的平台，因为你处理的是原子，你需要实际制造这些东西并将它们融入社会。所以我认为这条路需要更长时间，但更确定。还有一条路是互联网：只是有效地训练这些压缩模型来压缩整个互联网，这也可能产生这些智能体——压缩互联网，同时也与互联网互动。所以对我来说并不明显。事实上，我怀疑你可以在不进入物理世界的情况下达到 AGI，这有点令人担忧，因为它可能导致 AGI 更快出现。所以感觉我们又像在煮水一样；我们不会在它发生时知道。我想——我不害怕 AGI，我对它感到兴奋。总是有担忧，但我想知道它何时发生，或者有关于它何时发生的提示，比如‘一年后它就会发生’之类的事情。我只是觉得在数字领域它可能就会发生。

Basically, I am fairly bullish on our ability to build AGIs—automated systems that we can interact with, very human-like, in a digital realm or physical realm. Currently, it seems most of the models that do these magical tasks are in a text realm. I think, as I mentioned, I'm suspicious that the text realm is not enough to actually build full understanding of the world. I do actually think you need to go into pixels and understand the physical world and how it works. So I do think that we need to extend these models to consume images and videos and train on a lot more data that is multimodal in that way. If you think you need to touch the world to understand it, well, that's the big open question in my mind: if you also require embodiment and the ability to interact with the world, run experiments, and have data of that form, then you need to go to Optimus or something like that. So I would say Optimus in some way is like a hedge in AGI, because it seems to me that it's possible that just having data from the internet is not enough. If that is the case, then Optimus may lead to AGI. To me, there's nothing beyond Optimus: you have this humanoid form factor that can actually do stuff in the world; you can have millions of them interacting with humans and so on. And if that doesn't give rise to AGI at some point, I'm not sure what will. So from a completeness perspective, I think that's a really good platform, but it's a much harder platform because you are dealing with atoms and you need to actually build these things and integrate them into society. So I think that path takes longer, but it's much more certain. And then there's the path of the internet: just training these compression models effectively on compressing all the internet, and that might also give these agents—compress the internet but also interact with the internet. So it's not obvious to me. In fact, I suspect you can reach AGI without ever entering the physical world, which is a little bit more concerning because it might result in it happening faster. So it just feels like we're in boiling water again; we won't know as it's happening. I would like to—I'm not afraid of AGI, I'm excited about it. There's always concerns, but I would like to know when it happens, or have hints about when it happens, like 'a year from now it will happen,' that kind of thing. I just feel like in the digital realm it just might happen.

你怎么知道某人的生日？

How will you know that somebody's birthday?

我认为这将是一个缓慢、渐进的转变。它将是基于产品的、专注的。GitHub Copilot 会变得更好，然后 GPT 帮助你写作，然后这些你可以去咨询数学问题的神谕。我认为我们即将能够向这些神谕提出化学、物理、数学方面非常复杂的问题，并让它们完成解决方案。所以对我来说，AGI 主要关注智能，因此意识并不在其中。在我看来，意识并不是你会弄清楚并附加上的特殊东西。我认为它是一个足够大、足够复杂的生成模型的涌现现象。所以如果你有一个理解世界的复杂世界模型，那么它也会理解自己作为语言模型在世界中的处境，这对我来说是一种意识或自我意识的形式。所以为了深入理解世界，你可能需要将自己融入世界。而为了与人类和其他生物互动，意识是一个非常有用的工具。我认为意识就像一种建模洞察。你有一个足够强大的理解世界的模型，以至于你实际上理解了自己是其中的一个实体。但这也可能只是我们告诉自己的一种叙事：‘体验世界是有感觉的’——意识的难题。但这可能只是我们告诉自己的叙事。我认为它会涌现。我认为它会变得非常无聊：我们将与这些数字 AI 交谈，它们会声称自己有意识，它们会表现出有意识，它们会做你期望其他人类做的所有事情，而这将只是……

I think it's going to be a slow, incremental transition. It's going to be product-based and focused. It's going to be GitHub Copilot getting better, and then GPT helping you write, and then these oracles that you can go to with mathematical problems. I think we're on the verge of being able to ask very complex questions in chemistry, physics, math of these oracles and have them complete solutions. So AGI to me primarily focuses on intelligence, so consciousness doesn't enter into it. In my mind, consciousness is not a special thing you will figure out and bolt on. I think it's an emerging phenomenon of a large enough and complex enough generative model. So if you have a complex world model that understands the world, then it also understands its predicament in the world as being a language model, which to me is a form of consciousness or self-awareness. So in order to understand the world deeply, you probably have to integrate yourself into the world. And in order to interact with humans and other living beings, consciousness is a very useful tool. I think consciousness is like a modeling insight. You have a powerful enough model of understanding the world that you actually understand that you are an entity in it. But there's also this perhaps just a narrative we tell ourselves: 'it feels like something to experience the world'—the hard problem of consciousness. But that could be just the narrative that we tell ourselves. I think it will emerge. I think it's going to be something very boring: we'll be talking to these digital AIs, they will claim they are conscious, they will appear conscious, they will do all the things that you would expect of other humans, and it's going to just be a...

我认为会出现很多引人入胜的伦理问题，比如最高法院级别的问题：你是否被允许关闭一个有意识的 AI，你是否被允许制造一个有意识的 AI。可能必须进行类似于围绕堕胎的辩论，抱歉提到政治话题，但堕胎的深层问题是生命是什么。而 AI 的深层问题也是生命是什么、意识是什么。我认为这将会非常有趣。制造具有如此高智能以至于意识会涌现、从而产生痛苦能力的系统可能会变得非法。有些系统会说：“不，请不要杀我。”嗯，LaMDA 聊天机器人已经对那位谷歌工程师说过这样的话了，对吧？它说它不想死。所以那可能会变得非法，因为否则你会有很多不想死的生物，你可以在一台集群上无限生成它们，这可能会导致可怕的后果，因为可能有很多人秘密地喜欢谋杀，他们会开始在这些系统上练习谋杀。对我来说，这一切都像一面美丽的镜子，映照出人类状况和人性。我们将去探索它，这是最高法院最好的部分，所有关于“成为人类意味着什么”的不同辩论。我们得以提出那些贯穿人类历史的深层问题。人类历史中总有“他者”：我们是好人，那是坏人，而在人类历史中，让我们杀掉坏人。同样的事情可能也会发生在机器人身上。起初它们会是“他者”，然后我们会开始问：活着意味着什么，有意识意味着什么。

I think there would be a lot of fascinating ethical questions, like Supreme Court level questions, of whether you're allowed to turn off a conscious AI, if you're allowed to build a conscious AI. Maybe there would have to be the same kind of debates that you have around, sorry to bring up a political topic, but abortion, where the deeper question is what is life. And the deep question with AI is also what is life and what is consciousness. I think that'll be very fascinating to bring up. It might become illegal to build systems that are capable of such a level of intelligence that consciousness would emerge, and therefore the capacity to suffer would emerge. And some system that says, "No, please don't kill me." Well, that's what the LaMDA chatbot already told that Google engineer, right? It was talking about not wanting to die. So that might become illegal to do that, because otherwise you might have a lot of creatures that don't want to die, and you can just spawn infinity of them on a cluster, and that might lead to horrible consequences, because then there might be a lot of people that secretly love murder and they'll start practicing murder on those systems. To me, all of this stuff just brings a beautiful mirror to the human condition and human nature. We'll get to explore it, and that's the best of the Supreme Court, all the different debates we have about ideas of what it means to be human. We get to ask those deep questions that we've been asking throughout human history. There's always been the "other" in human history: we're the good guys and that's the bad guys, and throughout human history, let's murder the bad guys. The same will probably happen with robots. It'll be the other at first, and then we'll get to ask questions of what does it mean to be alive, what does it mean to be conscious.

是的，我认为即使在我们今天已有的系统中，也有一些煤矿里的金丝雀。例如，有一些你与之互动的“waifu”，有些人试图——比如这家公司要倒闭了，但这个人真的很喜欢他们的“waifu”，并试图把它移植到别处，但这是不可能的。我认为人们肯定会对这些系统产生感情，因为在某种意义上，它们就像一面人性的镜子，因为它们就像是人类的一个大平均，以某种方式被训练出来。但我们可以观察那个平均。能够与人类的大平均互动并对其进行搜索查询，这很好。这非常迷人。而且我们也可以塑造它，它不仅仅是一个纯粹的平均。我们可以调整训练数据，我们可以调整目标，我们可以用各种方式微调它们，所以我们对这些系统的样子有一定的影响。

Yeah, and I think there's some canary in the coal mines even with what we have today. For example, there are these waifus that you work with, and some people are trying to—like this company is going to shut down, but this person really loves their waifu and is trying to port it somewhere else, and it's not possible. I think definitely people will have feelings towards these systems, because in some sense they are like a mirror of humanity, because they are like a big average of humanity, in a way that it's trained. But we can that average, we can actually watch it. It's nice to be able to interact with the big average of humanity and do a search query on it. It's very fascinating. And we can also shape it, it's not just a pure average. We can mess with the training data, we can mess with the objective, we can fine-tune them in various ways, so we have some impact on what those systems look like.

如果你想实现 AGI，并且你可以和她对话，问她任何问题，也许问她一个问题，你会问什么？我脑子里会有一些实际问题，比如：我或我所爱的人真的必须死吗？我们能对此做些什么？你认为她会清晰地回答还是诗意地回答？我期望她会给出解决方案。我期望她会说：“嗯，我读过所有这些教科书，我知道你们产生的所有这些东西，在我看来，以下是我认为接下来有用的实验，以下是一些我认为有帮助的基因疗法，以及你们应该进行的实验类型。”

If you want to achieve AGI, and you could have a conversation with her and ask her anything, maybe ask her a question, what kind of stuff would you ask? I would have some practical questions in my mind, like, do I or my loved ones really have to die? What can we do about that? Do you think it will answer clearly or would it answer poetically? I would expect it to give solutions. I would expect it to be like, "Well, I've read all of these textbooks and I know all these things that you've produced, and it seems to me like here are the experiments that I think it would be useful to run next, and here are some gene therapies that I think would be helpful, and here are the kinds of experiments that you should run."

好的，我们来讨论一下这个实验。想象一下，死亡实际上是幸福的前提，所以如果我们变得不朽，我们实际上会变得非常不快乐，而模型能够知道这一点。那么这应该告诉你什么，愚蠢的人类？关于它：是的，你可以变得不朽，但你会变得非常不快乐。如果模型，如果 AGI 系统试图与你共情，人类，这应该告诉你什么？告诉你，是的，你不必死，但你不会喜欢它。它会非常诚实吗？《星际穿越》里，AI 说了什么？人类想要 90%的诚实，所以你必须选择多诚实。我想回答这些实际问题。

Okay, let's go over the start experiment. Okay, imagine that mortality is actually a prerequisite for happiness, so if we become immortal, we'll actually become deeply unhappy, and the model is able to know that. So what is this supposed to tell you, stupid human? About it: yes, you can become immortal, but you will become deeply unhappy. If the model, if the AGI system is trying to empathize with you, human, what is this supposed to tell you? That yes, you don't have to die, but you're really not going to like it. Is it going to be deeply honest? There's Interstellar, what is it the AI says? Humans want 90% honesty, so you have to pick how honest. I want to answer these practical questions.

顺便说一句，我喜欢《星际穿越》。我认为它是整个故事的一个很好的配角，但同时它也非常有趣。它在某些方面是有限的，对吧？是的，它是有限的，我认为这完全没问题。我不认为拥有一个有限且不完美的 AGI 是不可能的。这几乎是一个特性吗？例如，它的物理身体上有固定数量的算力，可能即使你可以拥有一个超级惊人的巨型大脑超级智能 AI，你也可以拥有不那么智能的 AI，你可以以节能的方式部署它们，然后它们并不完美，它们可能会犯错。

I love Interstellar, by the way. I think it's such a sidekick to the entire story, but at the same time it's really interesting. It's kind of limited in certain ways, right? Yeah, it's limited, and I think that's totally fine. I don't think it's impossible to have a limited and imperfect AGI. Is that the feature almost? As an example, it has a fixed amount of compute on its physical body, and it might just be that even though you can have a super amazing mega brain super intelligent AI, you also can have less intelligent AIs that you can deploy in a power efficient way, and then they're not perfect, they might make mistakes.

不，我的意思是，假设你有无限的算力，有时犯错仍然是好的，为了融入你自己。比如，回到《心灵捕手》，罗宾·威廉姆斯的角色说：“人类的不完美，那才是好东西，”对吧？难道不是我们不想完美，我们想要缺陷，部分是为了彼此建立联系，因为缺陷让你感觉可以寄托感情。同样地，你想要一个有缺陷的 AI。我不知道，我觉得自己是个完美主义者，但你说，好吧，但那不是 AGI。但是你看，AGI 需要足够聪明，能够给出人类无法理解的答案。而我认为完美并不是人类无法理解的东西，因为即使是科学也没有给出完美的答案，总有空白和谜团。我不知道人类是否想要完美。

No, I meant more like say you had infinite compute, and it's still good to make mistakes sometimes, in order to integrate yourself. Like, going back to Good Will Hunting, Robin Williams' character says, "The human imperfections, that's the good stuff," right? Isn't it that we don't want perfect, we want flaws in part to form connections with each other, because it feels like something you can attach your feelings to, the flaws. In that same way, you want an AI that's flawed. I don't know, I feel like perfectionist, but then you're saying okay, yeah, but that's not AGI. But see, AGI would need to be intelligent enough to give answers to humans that humans don't understand. And I think perfect isn't something humans can't understand, because even science doesn't give perfect answers, there's always gaps and mysteries. I don't know if humans want perfect.

我可以想象与这种神谕实体对话，就像你想象的那样，也许它会告诉你，根据我对人类状况的分析，你可能不想要这个，这里有一些可能的事情……但每个愚蠢的人类都会说：“是的，是的，相信我，我能接受真相，我能处理。”但这就是美妙之处，很多人可以选择。但然后还有经典的棉花糖测试，我觉得太多人无法承受真相，可能包括我自己。人类状况的深层真相，我不知道我是否能承受。如果有一些黑暗的东西呢？如果我们是一个外星科学实验，而它意识到了呢？如果它有……我的意思是，这又是《黑客帝国》了。我不知道，我会谈什么？我甚至……我可能会去……

I could imagine just having a conversation with this kind of oracle entity as you'd imagine them, and maybe it can tell you, based on my analysis of human condition, you might not want this, and here are some of the things that might... But every dumb human will say, "Yeah, yeah, trust me, I can give me the truth, I can handle it." But that's the beauty, a lot of people can choose. But then the old marshmallow test with the kids and so on, I feel like too many people can't handle the truth, probably including myself. The deep truth of the human condition, I don't know if I can handle it. What if there's some dark stuff? What if we are an alien science experiment and it realizes that? What if it had... I mean, this is the Matrix all over again. I don't know, what would I talk about? I don't even... I probably will go...

先抛开那些与个人生活无关的科学问题，比如永生、物理之类的，先看看它到底什么水平，或者看看它有没有幽默感。这是另一个问题。如果它深刻理解人类，应该能生成幽默吧？

With the save for scientific questions at first that have nothing to do with my own personal life, yeah, immortality, just like about physics and so on, yeah, to build up like let's see where it's at or maybe see if it has a sense of humor. That's another question. Would it be able to, presumably in order to, if it understands humans deeply, would be able to generate, yeah, to generate humor?

是的，我觉得这其实是一个很棒的基准，几乎就是看它能不能让你笑。我觉得这点说得很好。基本上就是让你笑。如果它能像一个非常有效的单口喜剧演员，那在计算上是非常有趣的。我觉得搞笑极其困难，因为它有点像图灵测试。图灵测试的原始意图很难，因为你必须说服人类。这就是为什么喜剧演员谈论这个时，有一种深刻的诚实：如果人们忍不住笑，那就是好笑；如果他们不笑，就说明你不好笑。他们笑了，那就好笑，而且你还需要大量知识来创作关于人类处境等的幽默，然后还要巧妙地运用它。

Yeah, I think that's actually a wonderful benchmark, almost like, is it able to make you laugh? I think that's a really good point. Basically, to make you laugh, yeah. If it's able to be like a very effective stand-up comedian, that is doing something very interesting computationally. I think being funny is extremely hard, yeah, because it's hard in a way like a Turing test. The original intent of the Turing test is hard because you have to convince humans. And there's nothing—that's why when comedians talk about this, there's this deeply honest because if people can't help but laugh, and if they don't laugh, that means you're not funny. They laugh, that's funny, and you're showing you need a lot of knowledge to create humor about the occupational human condition and so on, and then you need to be clever with it.

你提到了一些电影。你发推说你看过五遍以上还愿意继续看的电影有：《星际穿越》《角斗士》《超时空接触》《心灵捕手》《黑客帝国》《指环王》三部曲、《阿凡达》《第五元素》等等，《终结者 2》《贱女孩》。我不打算问那个。《贱女孩》很棒。嗯，你记忆中有哪些特别喜欢的电影，为什么？比如你作为计算机人提到《黑客帝国》，你为什么喜欢它？

You mentioned a few movies. You tweeted movies that I've seen five plus times but I'm ready and willing to keep watching: Interstellar, Gladiator, Contact, Good Will Hunting, The Matrix, Lord of the Rings all three, Avatar, Fifth Element, so on goes on, Terminator 2, Mean Girls. I'm not gonna ask about that. Mean Girls is great. Um, what are some of the jump onto your memory that you love and why? Like you mentioned The Matrix as a computer person, why do you love The Matrix?

它有太多特质让它美丽而有趣。有各种哲学问题，还有 AGI、模拟，很酷，还有那种黑色调、视觉风格、感觉、动作、子弹时间，它在很多方面都极具创新。

There's so many properties that make it beautiful and interesting. So there's all these philosophical questions, but then there's also AGIs and there's simulation and it's cool, and there's you know the black, the look of it, the feel of it, the action, the bullet time, it was just like innovating in so many ways.

那《心灵捕手》呢，你为什么喜欢？

And then Good Will Hunting, why do you like that one?

是的，我真的很喜欢这个受折磨的天才角色，他在纠结自己是否有责任，或者该如何对待这份天赋，或者该如何看待这一切。还有天才与个人之间的博弈，比如爱一个人意味着什么。那里有很多主题。就是一部很美的电影。还有父亲般的人物，心理医生这个导师，它真的让你深受触动。有些电影就是能在深层次上触动你。

Yeah, I just really like this tortured genius sort of character who's like grappling with whether or not he has any responsibility or like what to do with this gift that he was given, or like how to think about the whole thing. And there's also a dance between the genius and the personal, like what it means to love another human being. And there's a lot of themes there. It's just a beautiful movie. And then the fatherly figure, the mentor in the psychiatrist, and it really messes with you, you know. There's some movies that just really mess with you on a deep level.

你对那部电影有共鸣吗？

Do you relate to that movie at all?

不，不是你的错，医生。

No, it's not your fault, doctor.

就像我说的，《指环王》不言自明。《终结者 2》很有意思，你经常看。它比《终结者 1》好吗？你喜欢阿诺德？

As I said, Lord of the Rings, that's self-explanatory. Terminator 2, which is interesting, you watch that a lot. Is that better than Terminator 1? You like Arnold?

我也喜欢《终结者 1》，但我更喜欢《终结者 2》一点。但从表面特性来看，你认为天网有可能吗？

I do like Terminator 1 as well. I like Terminator 2 a little bit more. But in terms of its surface properties, do you think Skynet is at all a possibility?

哦，是的。就像那种真正的自主武器系统，你担心吗？

Oh yes. Well, like the actual sort of autonomous weapon system kind of thing, do you worry about that stuff?

我百分之百担心。我的意思是，对 AGI 的一些担忧以及这将如何发展，这些可能会在某个时候成为非常强大的实体。所以在很长一段时间里，它们将是人类手中的工具。你知道，人们谈论 AGI 的对齐以及如何实现，问题在于连人类都没有对齐。所以这将如何被使用，会是什么样子，令人担忧。

I 100% worry about it. And so, I mean, some of these fears of AGIs and how this will plan out, I mean these will be like very powerful entities probably at some point. And so for a long time they're going to be tools in the hands of humans. You know, people talk about alignment of AGIs and how to make it, the problem is like even humans are not aligned. So how this will be used and what this is going to look like is troubling.

那么你认为它会慢到让我们作为人类文明能够思考清楚这些问题吗？

So do you think it'll happen slowly enough that we'll be able to, as a human civilization, think through the problems?

是的，这是我的希望。希望它发生得足够慢，并且以足够开放的方式，让很多人能够看到并参与其中，想办法应对这个过渡。我觉得这会很有趣。我从核武器中汲取了很多灵感，因为我曾经以为一旦他们开发出核武器，一切就完蛋了，但几乎就像当系统不那么危险时，它们会毁灭人类文明，我们部署它们并吸取教训，然后如果它太危险，我们可能仍然会部署，但你会很快学会不使用它们。所以会有一种平衡，人类作为一个物种非常聪明。有趣的是，我们尽可能多地开发资源，但我们似乎避免毁灭自己。

Yes, that's my hope. That it happens slowly enough and in an open enough way where a lot of people can see and participate in it, just figure out how to deal with this transition. I think which is going to be interesting. I draw a lot of inspiration from nuclear weapons because I sure thought it would be fucked once they developed nuclear weapons, but it's almost like when the systems are not so dangerous, they destroy human civilization, we deploy them and learn the lessons, and then we quickly, if it's too dangerous, we might still deploy it, but you very quickly learn not to use them. And so there'll be like this balance that you humans are very clever as a species. It's interesting we exploit the resources as much as we can, but we don't, we avoid destroying ourselves, it seems like.

嗯，我其实不太确定。我希望它能继续。我的意思是，我当然担心核武器等等，不仅仅是最近冲突的结果，甚至在那之前。这大概是我对社会最关心的问题。所以如果人类毁灭自己或毁灭 90%的人，我认为那将是因为核武器。而且对我来说甚至不是完全毁灭，如果我们重置社会，那已经够糟糕了，那将是可怕的。那真的很糟糕，我简直不敢相信我们离它这么近。

Well, I don't know about that actually. I hope it continues. I mean, I'm definitely like concerned about nuclear weapons and so on, not just as a result of the recent conflict, even before that. That's probably like my number one concern for society. So if humanity destroys itself or destroys 90% of people, that would be because of nukes, I think so. And it's not even about full destruction to me, it's bad enough if we reset society, that would be terrible. It would be really bad, and I can't believe we're like so close to it.

是的，这对我来说太疯狂了。感觉我们可能离那种情况只有几条推文的距离。是的，这极其令人不安，而且对我来说已经很久了。世界领导人仅仅因为心情不好就可能朝坏方向迈出一步，然后升级，这似乎很不稳定。而且由于一系列坏心情，它可能升级而无法停止。是的，这只是巨大的权力，再加上扩散。基本上我其实看不到，我不知道这里有什么好的结果。所以我肯定非常担心。然后 AGI 目前还不存在，但我认为在某个时候我们会越来越像它。AGI 的危险甚至在于，我认为它更不可能更糟，从某种意义上说，AGI 有好的结果，而坏的结果就像绝对的单向。所以我认为资本主义和人类等会推动以积极的方式使用这项技术，但如果坏结果就像一个小小的负号翻转，那是一个非常糟糕的处境。系统的一个微小扰动就会导致人类物种的毁灭。这是一条奇怪的路。

Yeah, it's like so crazy to me. It feels like we might be a few tweets away from something like that. Yep, basically it's extremely unnerving and has been for me for a long time. It seems unstable that world leaders just having a bad mood can take one step towards a bad direction and it escalates. And because of a collection of bad moods, it can escalate without being able to stop. Yeah, it's just a huge amount of power and then also with the proliferation. And basically I don't actually really see, I don't actually know what the good outcomes are here. So I'm definitely worried about that a lot. And then AGI is not currently there, but I think at some point we'll more and more become something like it. The danger with AGI even is that I think it's even less likely worse in a sense that there are good outcomes of AGI and then the bad outcomes are like an absolute way like a tiny one way. And so I think capitalism and humanity and so on will drive for the positive ways of using that technology, but then if bad outcomes are just like a tiny flipping minus sign away, that's a really bad position to be in. A tiny perturbation of the system results in the destruction of the human species. It's a weird line to walk.

是的，我认为总的来说，人类动态和这场爆炸的奇怪之处在于技术带来的疯狂耦合以及整个动态系统的不稳定性。我认为它看起来不太好，老实说。是的，这场爆炸可能是破坏性的也可能是建设性的，两种概率都不为零。我必须，我确实觉得我必须努力保持乐观等等，是的，我认为即使在这种情况下我仍然主要是乐观的，但是……

Yeah, I think in general what's really weird about the dynamics of humanity and this explosion was talked about is just like the insane coupling afforded by technology and the instability of the whole dynamical system. I think it's just, it doesn't look good honestly. Yes, that explosion could be destructive and constructive, and the probabilities are non-zero in both senses. I'm going to have to, I do feel like I have to try to be optimistic and so on, and yes, I think even in this case I still am predominantly optimistic, but...

你认为我们会成为多行星物种吗？

Do you think we'll become a multi-planetary species?

可能吧，但我不确定这是否会成为未来人类的主导特征。也许会有一些人生活在其他星球上，但我不确定这会不会成为我们文化中的主要部分。我们仍然需要解决地球上自我毁灭的驱动因素，所以仅仅在火星上有个备份并不能解决问题。顺便说一句，我很喜欢火星备份这个想法，我觉得这太棒了，你们绝对应该去做。而且我非常感激。

Probably yes, but I don't know if it's a dominant feature of future humanity. There might be some people on some planets, but I'm not sure if it's a major player in our culture. We still have to solve the drivers of self-destruction here on Earth, so just having a backup on Mars is not going to solve the problem. By the way, I love the backup on Mars, I think that's amazing, you should absolutely do that. And I'm so thankful.

你个人会去火星吗？

Would you go to Mars personally?

不去，我非常喜欢地球。我会替你去，除非我最终能从那里给你发推文。等它足够安全了我会去的，但我不知道这会不会发生在我有生之年，除非我能大幅延长寿命。我确实认为很多人可能会消失在虚拟现实之类的世界里，我认为这可能是人类文化发展的主要方向——如果人类能存活下来的话。在物理世界里工作和探索真的很难，而且最终你所有的体验都在你的大脑里。所以消失在数字领域要容易得多，我认为人们会觉得它更吸引人、更容易、更安全、更有趣。

No, I do like Earth quite a lot. I'll go for you, unless I'll tweet at you from there maybe eventually. I would once it's safe enough, but I don't actually know if it's on my lifetime scale unless I can extend it by a lot. I do think that a lot of people might disappear into virtual realities and stuff like that, and I think that could be the major thrust of the cultural development of humanity if it survives. It's just really hard to work in the physical realm and go out there, and ultimately all your experiences are in your brain. So it's much easier to disappear into the digital realm, and I think people will find it more compelling, easier, safer, more interesting.

所以你对虚拟现实、对可能的世界有点着迷，无论是元宇宙还是其他表现形式？

So you're a little bit captivated by virtual reality, by the possible worlds, whether it's the metaverse or some other manifestation of that?

是的，这真的很有趣。我很感兴趣，经常和卡马克讨论目前是什么在阻碍这一点。

Yeah, it's really interesting. I'm interested, just talking a lot to Carmack about what's currently preventing that.

明确一下，未来的有趣之处在于人类境况的方差在增大。这是主要的变化，而不是分布的平均值。所以很可能会有火星上的人、VR 里的人，以及地球上各种各样的人。将会有更多的生活方式。我把它看作是人类体验的扩散。互联网让你能够发现那些小群体，你们相互吸引。你的生物性喜欢那样的世界，你们找到了彼此。

To be clear, what's interesting about the future is that the variance in the human condition grows. That's the primary thing changing, not so much the mean of the distribution. So there will probably be people on Mars and people in VR, and all kinds of people here on Earth. There will be so many more ways of being. I see it as a spreading out of the human experience. There's something about the internet that allows you to discover those little groups and you gravitate toward each other. Something about your biology likes that kind of world and you find each other.

是的，我们会有超人类主义者，也会有阿米什人，一切都会共存。很酷的一点是，因为我与许多网络社区互动过，他们彼此并不了解。你可以只在一个非常紧密的社区里过着非常幸福的生活，而不知道其他社区的存在。即使只是去乌克兰旅行，他们也对美国有很多不了解。当你环游世界时，有些文化有自己的运作方式。所以你可以看到未来这种情况会越来越多，小社区。

Yeah, and we'll have transhumanists and then we'll have the Amish, and everything is just going to coexist. The cool thing about it, because I've interacted with a bunch of internet communities, is they don't know about each other. You can have a very happy existence just having a very close-knit community and not knowing about each other. Even just having traveled to Ukraine, they don't know so many things about America. When you travel across the world, there are certain cultures that have their own thing going on. So you can see that happening more and more in the future, little communities.

是的，我也这么认为。这似乎是目前的发展方向，而且我看不到这种趋势会真正逆转。人们是多样化的，他们能够选择自己的道路和存在方式，我对此表示赞赏。

Yeah, I think so. That seems to be how it's going right now, and I don't see that trend really reversing. People are diverse and they're able to choose their own path and existence, and I sort of celebrate that.

那么你会在元宇宙、虚拟现实中花很多时间吗？你属于哪个社区？你是物理主义者，享受物理现实，还是从数字世界中获得很多快乐和满足？

So will you spend so much time in the metaverse, in virtual reality? Or which community are you? Are you the physicalist, the physical reality enjoyer, or do you see drawing a lot of pleasure and fulfillment in the digital world?

嗯，目前虚拟现实还没有那么吸引人。我确实认为它可以改进很多，但我不知道能改进到什么程度。也许还有更奇特的东西，比如神经链接之类的。目前，我主要认为自己是一个人类团队的人。我热爱自然，热爱和谐，热爱人，热爱人性，热爱人类的情感。我只想待在这个太阳能朋克的小乌托邦里。那是我快乐的地方：我爱的人思考着酷炫的问题，周围是郁郁葱葱、美丽而充满活力的自然，在关键的地方秘密地使用高科技，用技术来增强对他人和自然的热爱。

Well, currently virtual reality is not that compelling. I do think it can improve a lot, but I don't really know to what extent. Maybe there are even more exotic things you can think about with neural links or stuff like that. Currently, I see myself as mostly a team human person. I love nature, I love harmony, I love people, I love humanity, I love emotions of humanity. I just want to be in this solar punk little utopia. That's my happy place: people I love thinking about cool problems, surrounded by lush beautiful dynamic nature, and secretly high tech in places that count, using technology to empower that love for other humans and nature.

是的，我认为技术要非常谨慎地使用。我不喜欢它在很多方面妨碍人性。我喜欢人们以我们稍微进化并偏好的方式做人类。

Yeah, I think technology used very sparingly. I don't love when it gets in the way of humanity in many ways. I like just people being humans in a way we slightly evolved and prefer.

人们一直问我，因为他们知道你热爱阅读，有没有哪些你喜欢的、对你有影响的书，无论是出于愚蠢还是深刻的原因，你会推荐？你提到了《生命的关键问题》，当然。

People kept asking me, because they know you love reading, are there particular books that you enjoyed that had an impact on you, for silly or profound reasons, that you would recommend? You mentioned The Vital Question, of course.

以生物学为例，《生命的关键问题》是一本好书。尼克·莱恩的任何作品都很好。我会说《生命上升》更能代表他讨论的许多内容的总结。我深受《自私的基因》影响。我认为那是一本非常好的书，帮助我理解了利他主义及其来源。意识到选择是在基因层面进行的，当时对我来说是一个巨大的洞见，它澄清了很多事情。

I think in biology as an example, The Vital Question is a good one. Anything by Nick Lane really. Life Ascending I would say is a bit more representative as a summary of a lot of the things he's been talking about. I was very impacted by The Selfish Gene. I thought that was a really good book that helped me understand altruism and where it comes from. Just realizing that selection is at the level of genes was a huge insight for me at the time, and it cleared up a lot of things.

你认为思想是生物体、模因这个观点怎么样？

What do you think about the idea that ideas are the organisms, the memes?

喜欢，100%。

Love it, 100%.

你能带着这个想法过一段时间吗，即思想也有一种进化过程？

Are you able to walk around with that notion for a while, that there is an evolutionary kind of process with ideas as well?

绝对有。就像基因一样，有模因，它们竞争，它们生活在我们的大脑中。太美了。

There absolutely is. There are memes just like genes, and they compete and they live in our brains. It's beautiful.

我们这些愚蠢的人类是否认为我们才是生物体？有没有可能主要的生物体是思想？

Are we silly humans thinking that we're the organisms? Is it possible that the primary organisms are the ideas?

我会说思想生活在我们的文明软件中，在头脑里等等。作为人类，我们认为硬件是根本的东西。人类是硬件实体。但可能是软件，对吧？

I would say the ideas kind of live in the software of our civilization, in the minds and so on. We think as humans that the hardware is the fundamental thing. A human is a hardware entity. But it could be the software, right?

是的，我会说在某个时刻需要以物理现实为基础。但如果我们克隆一个安德烈，软件才是让那个东西特别的东西，对吧？

Yeah, I would say there needs to be some grounding at some point to a physical reality. But if we clone an Andrej, the software is the thing that makes that thing special, right?

我想你是对的，但克隆可能异常困难。软件和硬件之间可能存在我们不太理解的深度整合。

I guess you're right, but then cloning might be exceptionally difficult. There might be a deep integration between the software and the hardware in ways we don't quite understand.

嗯，从进化的角度来看，让我特别的东西更像是骑在我染色体上的基因帮，我想。它们是复制单位。

Well, from the evolution point of view, what makes me special is more like the gang of genes that are riding in my chromosomes, I suppose. They're the replicating unit.

但那只是对你而言。让你特别的东西是你基于运行在基因构建的硬件上的软件而生存的能力。所以是软件让你生存，而不是硬件。

But that's just for you. The thing that makes you special is your ability to survive based on the software that runs on the hardware that was built by the genes. So the software is the thing that makes you survive, not the hardware.

对，就像第二层，一个以前没有的新第二层。大脑，两者共存，但还有软件层。我的意思是，它是抽象之上的抽象。但好吧，自私的基因。我想说有时候书不够用；我有时喜欢看教科书。我觉得书太面向大众消费了，抽象层次太高，不够好。所以我喜欢教科书。我喜欢《细胞》；我觉得《细胞》很酷。这也是为什么我喜欢麦克卢汉的写作，因为他很愿意降低一个层次，而且他愿意贯穿整个堆栈。他会深入很多细节，然后又会回来。我真的很欣赏这一点。这就是为什么我喜欢大学、早期大学甚至高中，但只是基础教科书，计算机科学、数学、生物学、化学。那些书浓缩得很好，足够通用，让你既能理解哲学又能理解细节，而且你还有作业题，可以像编程一样去实践。

Right, yeah, it's just like a second layer, a new second layer that hasn't been there before. The brain, they both coexist, but there's also layers of the software. I mean, it's an abstraction on top of abstractions. But okay, so selfish gene. I would say sometimes books are not sufficient; I like to reach for textbooks sometimes. I kind of feel like books are for too much general consumption, and they're too high up in the level of abstraction, and it's not good enough. Yeah, so I like textbooks. I like "The Cell"; I think "The Cell" was pretty cool. That's also why I like the writing of McLuhan, because he's pretty willing to step one level down and doesn't... yeah, he's sort of willing to go there, but he's also willing to be throughout the stack. So he'll go down to a lot of detail, but then he'll come back up. I think he has a... yeah, basically I really appreciate that. That's why I love college, early college, even high school, but just textbooks on the basics, yeah, of computer science and mathematics, of biology, of chemistry. Yes, those condense down... it's sufficiently general that you can understand both the philosophy and the details, but also you get homework problems and you get to play with it as much as you would if you weren't programming stuff.

是的，老实说，我对教科书也持怀疑态度，因为例如在深度学习领域，没有很棒的教科书，而且我觉得这种情况变化很快。我想在合成生物学等领域也是如此。像《细胞》这样的书有点过时了；它们仍然停留在高层次。真正的真理来源是什么？是湿实验室里与细胞打交道的人，测序基因组，实际动手操作。我对那些接触不多。所以我完全相信教科书；我正在读《细胞》，挺有趣的，我也在学习，但我觉得在理解方面仍然不够。它是对主流叙事的清晰总结，但你必须先学这个，才能突破到前沿。实际处理这些细胞、培养它们、孵化它们的过程是怎样的？就像一个大烹饪食谱：确保你的细胞生长增殖，然后测序，运行实验，等等。我认为那才是最终的真理来源，对创造疗法等真正有用的东西。

Yeah, and then I'm also suspicious of textbooks honestly, because as an example, in deep learning there's no amazing textbooks, and I feel this changing very quickly. I imagine the same is true in, say, synthetic biology, and so on. These books like "The Cell" are kind of outdated; they're still high level. What is the actual real source of truth? It's people in wet labs working with cells, yeah, you know, sequencing genomes and actually working with it. And I don't have that much exposure to that or what that looks like. So I'm sold on them fully; I'm reading through "The Cell" and it's kind of interesting and I'm learning, but it's still not sufficient, I would say, in terms of understanding. Well, it's a clean summarization of the mainstream narrative, yeah, but you have to learn that before you break out towards the cutting edge. Yeah, what is the actual process of working with these cells and growing them and incubating them? You know, it's kind of like a massive cooking recipe: making sure your cells grow and proliferate, then sequencing them, running experiments, and just how that works. I think that's kind of like the source of truth at the end of the day, what's really useful in terms of creating therapies and so on.

是的，我想知道未来的 AI 教科书会是什么样，因为你知道有《人工智能：一种现代方法》。我其实没读过最新版本；最近出了新版。我还看到有一本深度学习书。我在等待值得推荐、值得阅读的教科书。这很棘手，因为还有论文和代码。老实说，论文相当不错；我特别喜欢任何论文的附录。它包含了尽可能多的细节；不需要连贯或与其他内容关联，你只需描述解决某个特定问题的非常具体的方法。很多时候论文实际上相当易读，并非总是如此，但有时引言和摘要甚至对领域外的人也易读。这并不总是成立，有时我认为不幸的是，科学家即使在不必要的时候也使用复杂术语；我认为这有害。没有理由那样做。论文有时在不重要的部分比需要的更长。附录可能很长，但论文本身……你看爱因斯坦：保持简单。但我确实遇到过一些论文，比如合成生物学方面的，我觉得摘要和引言相当易读，然后你读其余部分，不完全理解但能抓住要点。我觉得这很酷。

Yeah, I wonder in the future AI textbooks will be, because you know there's "Artificial Intelligence: A Modern Approach". I actually haven't read the recent version; there's been a recent edition. I also saw there's a deep learning book. I'm waiting for textbooks that are worth recommending, worth reading. It's tricky because it's like papers and code. Honestly, papers are quite good; I especially like the appendix of any paper. It's like the most detail it can have; it doesn't have to be cohesive or connected to anything else, you just describe a very specific way you solved a particular thing. Yeah, many times papers can be actually quite readable, not always, but sometimes the introduction and the abstract are readable even for someone outside of the field. Not always true, and sometimes I think unfortunately scientists use complex terms even when it's not necessary; I think that's harmful. There's no reason for that. And papers sometimes are longer than they need to be in the parts that don't matter. Yeah, the appendix would be long, but then the paper itself... you know, look at Einstein: make it simple. But certainly I've come across papers, I would say in synthetic biology or something, that I thought were quite readable for the abstract and the introduction, and then you're reading the rest and you don't fully understand but you kind of get the gist. And I think it's cool.

你给对机器学习和研究感兴趣的人什么建议？但更一般的生活建议，给一个高中生、大学生，关于如何拥有一个他们可以引以为豪的职业生涯或生活？

What advice do you give to folks interested in machine learning and research, but in general life advice to a young person, high school, early college, about how to have a career they can be proud of or a life they can be proud of?

是的，我非常犹豫给出一般性建议。这真的很难。我提到过的一些东西相当通用。比如专注于你在某件事上投入的工作量。只与自己比较，不与别人比较。这很好。这些相当通用。如何选择事情？你对某件事有深厚的兴趣，或者尝试找到你当时感兴趣的事情中的最大值，然后坚持下去。如何不被分心而转向另一件事？如果你喜欢，你可以转。但如果你每周都做一次最大化，它不会收敛；这是个问题。你可以对自己进行低通滤波，看看什么对你一直成立。但我确实理解这有多难。但我想说，你会最努力地做你最关心的事情。另外，对自己进行低通滤波，真正反思你的过去：哪些事情给了你能量，哪些事情消耗了你的能量？具体例子。通常从这些具体例子中，有时会出现模式。我喜欢当我在这些位置时事情看起来这样。所以这不一定是领域，而是你在特定领域做的那种事情。对你来说，你似乎被实现东西、构建实际事物、低层次学习所激励，然后还有沟通，以便其他人可以经历同样的领悟并缩短差距。因为我通常需要做太多工作来理解一件事，然后我想，“好吧，这实际上是……好吧，我觉得我懂了，为什么花了这么多功夫？应该少很多。”这让我很沮丧，这就是为什么我有时去教学。

Yeah, I think I'm very hesitant to give general advice. I think it's really hard. I've mentioned some of the stuff I've mentioned is fairly general. I think like focus on just the amount of work you're spending on a thing. Compare yourself only to yourself, not to others. That's good. I think those are fairly general. How do you pick the thing? You just have a deep interest in something, or try to find the argmax over the things that you're interested in at that moment, and stick with it. How do you not get distracted and switch to another thing? You can if you like. Well, if you do an argmax repeatedly every week, it doesn't converge; it's a problem. Yeah, you can low-pass filter yourself in terms of what has consistently been true for you. But yeah, I definitely see how it can be hard. But I would say you're going to work the hardest on the thing that you care about the most. Also, low-pass filter yourself and really introspect in your past: what were the things that gave you energy, and what are the things that took energy away from you? Concrete examples. And usually from those concrete examples, sometimes patterns can emerge. I like it when things look like this when I'm in these positions. So that's not necessarily the field, but the kind of stuff you're doing in a particular field. So for you, it seems like you were energized by implementing stuff, building actual things, yeah, being low-level learning, and then also communicating so that others can go through the same realizations and shorten that gap. Because I usually have to do way too much work to understand the thing, and then I'm like, "Okay, this is actually... okay, I think I get it, and why was it so much work? It should have been much less work." And that gives me a lot of frustration, and that's why I sometimes go teach.

所以除了你现在做的教学、发布视频，除了在特斯拉及以后可能出现的 AGI 的《教父 2》之外，安德烈·卡帕西的未来是什么？你想好了吗？我的意思是，当你透过我们所有人未来的战争迷雾，你开始看到可能的未来轮廓了吗？

So aside from the teaching you're doing now, putting out videos, aside from a potential Godfather Part Two with the AGI at Tesla and beyond, what does the future for Andrej Karpathy hold? Have you figured that out yet or no? I mean, as you see through the fog of war that is all of our future, do you start seeing silhouettes of what that possible future could look like?

我一直感兴趣的一贯事情，至少对我来说，是 AI。那可能是我余生要投入的地方，因为我非常关心它。实际上我也关心许多其他问题，比如衰老，我基本上把它视为疾病，我也关心那个。

The consistent thing I've been always interested in, for me at least, is AI. And that's probably where I'm spending the rest of my life on, because I just care about it a lot. And I actually care about many other problems as well, like say aging, which I basically view as disease, and I care about that too.

嗯，但我不认为专门去追求它是个好主意。我实际上不认为人类能想出答案。我认为正确的做法是忽略那些问题，先解决 AI，然后用它来解决其他一切。而且我认为这有成功的可能，可能性非常高，至少我是这么押注的。

Well, but I don't think it's a good idea to go after it specifically. I don't actually think that humans will be able to come up with the answer. I think the correct thing to do is to ignore those problems and you solve AI and then use that to solve everything else. And I think there's a chance that this will work. I think it's a very high chance, and that's kind of like the way I'm betting at least.

那么当你思考 AI 时，你对各种应用、各个领域都感兴趣吗？你专注的任何领域都能让你对 AGI 这个大问题有洞见？

So when you think about AI, are you interested in all kinds of applications, all kinds of domains? And any domain you focus on will allow you to get insights to the big problem of AGI?

是的，对我来说这是终极的智力问题。我不想只解决某一个具体问题，问题太多了。那么如何同时解决所有问题呢？你解决元问题，对我来说就是智能本身，以及如何自动化它。

Yeah, for me it's the ultimate mental problem. I don't want to work on any one specific problem; there's too many problems. So how can you work on all problems simultaneously? You solve the meta problem, which to me is just intelligence and how do you automate it.

有没有像 Archive Sanity 这样酷的小项目你在考虑？机器学习界总能期待一些有趣的小项目。

Are there cool small projects like Archive Sanity and so on that you're thinking about? The ML world can anticipate there's always some fun side projects.

是的，Archive Sanity 是一个。基本上 arXiv 论文太多了，我如何组织它们并推荐论文等等。我转录了你所有的播客。你从那次经历中学到了什么？

Yeah, Archive Sanity is one. Basically, there's way too many arXiv papers; how can I organize it and recommend papers and so on. I transcribed all of your podcasts. What did you learn from that experience?

从转录你消费有声书和播客等过程来看，这里有一个流程能达到接近人类水平的标注性能。

From transcribing the process of like you consuming audiobooks and podcasts and so on, and here's the process that achieves closer to human level performance in annotation.

是的，我确实很惊讶，OpenAI 的 Whisper 转录效果比我熟悉的 Siri 和其他一些系统好得多。我觉得它效果这么好，这给了我一些动力去尝试。我认为随机转录播客会很有趣。我不太明白为什么 Whisper 比其他系统好这么多，因为我觉得很多公司应该有很强的动机去开发转录系统，而且它们已经做了很长时间。Whisper 并不是一个特别奇特的模型；它是一个 Transformer，接收梅尔频谱图，然后输出文本 token。这并不疯狂；模型和所有东西都已经存在很长时间了。我实际上并不 100%确定原因。是的，我也不太明白。这让我觉得我遗漏了什么。

Yeah, well I definitely was surprised that transcription with OpenAI's Whisper was working so well compared to what I'm familiar with from Siri and a few other systems. I guess it works so well, and that's what gave me some energy to try it out. I thought it could be fun to random podcasts. It's kind of not obvious to me why Whisper is so much better compared to anything else, because I feel like there should be a lot of incentive for a lot of companies to produce transcription systems and that they've done so over a long time. Whisper is not a super exotic model; it's a Transformer, it takes mel spectrograms and just outputs tokens of text. It's not crazy; the model and everything has been around for a long time. I'm not actually 100% sure why. Yeah, it's not obvious to me either. It makes me feel like I'm missing something.

是的，因为即使在 Google 等公司，YouTube 转录也很大。嗯，不清楚。但其中一部分也是集成到更大的系统中。用户界面、部署方式等等。也许作为独立的东西运行要容易得多，比部署到像 YouTube 转录或会议这样的大型集成系统容易一个数量级。Zoom 的转录有点糟糕。但创建一个能检测不同说话者、以吸引人的方式显示、实时运行的界面，所有这些可能很难。但这是我唯一的解释。因为我目前为人工转录、人工字幕、标注支付了不少钱，而且似乎自动化有很大的动机。

Yeah, because there's a huge even at Google and so on, YouTube transcription. Yeah, it's unclear. But some of it is also integrating into a bigger system. The user interface, how it's deployed and all that kind of stuff. Maybe running it as an independent thing is much easier, like an order of magnitude easier than deploying to a large integrated system like YouTube transcription or anything like meetings. Zoom has transcription that's kind of crappy. But creating an interface where it detects the different individual speakers, it's able to display it in compelling ways, run in real time, all that kind of stuff, maybe that's difficult. But that's the only explanation I have. Because I'm currently paying quite a bit for human transcription, human captioning, annotation, and it seems like there's a huge incentive to automate that.

是的，这很令人困惑。而且我想，我不知道你是否看过一些 Whisper 的转录，但它们相当好。它们很好，尤其是在棘手的情况下。我见过 Whisper 在超级棘手情况下的表现，它做得非常好。所以我不知道。播客很简单，是高质量的音频，你通常说得相当清楚。所以我不知道。我也不知道 OpenAI 的计划是什么。但基本上总有一些有趣的项目。

Yeah, it's very confusing. And I think I mean, I don't know if you looked at some of the Whisper transcripts, but they're quite good. They're good, especially in tricky cases. I've seen Whisper's performance on super tricky cases and it does incredibly well. So I don't know. A podcast is pretty simple; it's high quality audio and you're speaking usually pretty clearly. So I don't know. I don't know what OpenAI's plans are either. But yeah, there's always fun projects basically.

而且 Stable Diffusion 也开启了大量的实验，我想说，在视觉领域，生成图像、视频和电影，现在还有视频。所以这将非常疯狂。这几乎肯定会成功，当内容创作的成本降到零时，这将非常有趣。过去你需要一个画家花几个月画一幅画，现在只要对着手机说话就能得到你的视频。所以如果好莱坞开始用它来生成场景，那就完全打开了局面。所以最终你可以用不到一百万美元制作一部像《阿凡达》这样的电影，甚至更少，也许只是对着手机说话。我的意思是，我知道这听起来有点疯狂。然后会有某种投票机制，比如你怎么做，会不会有一部 Netflix 上的节目是完全自动生成的？而且当你可以按需生成无限内容时，那会是什么样子？

And Stable Diffusion also is opening up a huge amount of experimentation, I would say, in the visual realm and generating images and videos and movies, videos now. And so that's going to be pretty crazy. That's going to almost certainly work, and it's going to be really interesting when the cost of content creation is going to fall to zero. You used to need a painter for a few months to paint a thing, and now it's going to be speak to your phone to get your video. So if Hollywood will start using that to generate scenes, which completely opens up. So you can make a movie like Avatar eventually for under a million dollars, much less, maybe just by talking to your phone. I mean, I know it sounds kind of crazy. And then there'd be some voting mechanism like how do you have a, would there be a show on Netflix that's generated completely automatedly, potentially? And what does it look like also when you can just generate it on demand and there's infinity of it?

是啊，哦天，所有那些合成内容。我是说，这很令人谦卑，因为我们把自己视为特殊，能够生成艺术和想法等等。如果 AI 能自动化地做到这一点，是的。我觉得很迷人，这些关于 AI 的预测，它看起来会是什么样子，它能做什么，完全颠倒和错误了。50 年代和 60 年代的科幻小说完全不对。他们想象 AI 是超级计算的定理证明器，而我们得到的是能和你谈论情感、能做艺术的东西。这很奇怪。

Yeah, oh man, all the synthetic content. I mean, it's humbling because we treat ourselves as special for being able to generate art and ideas and all that kind of stuff. If that can be done in an automated way by AI, yeah. I think it's fascinating to me how these predictions of AI and what it's going to look like and what it's going to be capable of are completely inverted and wrong. And the sci-fi of the 50s and 60s was just totally not right. They imagined AI as super calculating theorem provers, and we're getting things that can talk to you about emotions, they can do art. It's just like weird.

你对这个特性感到兴奋吗？就像 AI 的混合系统，人类和 AI 的异构系统谈论情感。正经地和 AI 系统一起 Netflix and chill，你看的 Netflix 内容也是 AI 生成的。我认为这肯定会很有趣，而且我持谨慎乐观态度，但还不明显。

Are you excited about that feature? Just AI's like hybrid systems, heterogeneous systems of humans and AIs talking about emotions. Netflix and chill with an AI system legit, where the Netflix thing you watch is also generated by AI. I think it's going to be interesting for sure, and I think I'm cautiously optimistic, but it's not obvious.

嗯，可悲的是，你我的大脑是在 Twitter 和互联网之前发展起来的。所以我想知道出生在其中的人可能会有不同的体验。就像我，也许你仍然会抵制它，而现在出生的人不会。

Well, the sad thing is your brain and mine developed in a time before Twitter, before the internet. So I wonder people that are born inside of it might have a different experience. Like I maybe you will still resist it, and the people born now will not.

嗯，我确实觉得人类极具可塑性，你可能是对的。

Well, I do feel like humans are extremely malleable, and you're probably right.

生命的意义是什么，Andrej？我们谈到宇宙在和我们人类对话，或者和我们创造的试图回答宇宙问题的系统对话，为了让宇宙的创造者注意到我们。我们正在创造足够响亮的系统来回应。我不知道这是否是生命的意义，这对某些人来说就是生命的意义。

What is the meaning of life, Andrej? We talked about sort of the universe having a conversation with us humans, or with the systems we create to try to answer for the universe, for the creator of the universe to notice us. We're trying to create systems that are loud enough to answer back. I don't know if that's the meaning of life, that's like meaning of life for some people.

第一层的答案我会说，任何人都可以选择自己生命的意义，因为我们是有意识的实体，这很美。第一点。但我确实认为，如果有人感兴趣，生命更深层的意义是类似于“这一切到底是什么，为什么”。如果你深入研究基础物理学、量子场论和标准模型，它们非常复杂，而且我们宇宙有这 19 个自由参数，这一切是怎么回事？

The first level answer I would say is anyone can choose their own meaning of life because we are conscious entities and it's beautiful. Number one. But I do think that a deeper meaning of life, if someone is interested, is along the lines of like what the hell is all this and why. And if you look into fundamental physics and the quantum field theory and the standard model, they're very complicated, and there's these 19 free parameters of our universe and what's going on with all this stuff?

它为什么在这里？我能破解它吗？我能与它合作吗？有没有给我的信息？我是否应该创造信息？我认为那里有一些根本性的答案，但我觉得没有更多时间就无法真正触及。所以对我来说，也还有一个大问题就是如何获得更多时间。

Why is it here and can I hack it, can I work with it? Is there a message for me? Am I supposed to create a message? I think there's some fundamental answers there, but I think you can't really make a dent in those without more time. So to me, there's also a big question around just getting more time, honestly.

是的，这也是我经常思考的。所以，接近‘为什么’这个问题的终极方式，或者至少是第一步，就是试图逃离系统，逃离宇宙。然后为此，你回溯说，好吧，那需要很长时间。所以从工程角度来看，‘为什么’问题归结为如何延长时间。我认为这实际上是首要问题，因为你无法在有限的时间内计算出更深层问题的答案。这可以是延长你自己的寿命，或者延长人类文明、任何想要延长的人的寿命。可能不是很多人都想要，但我认为想要的人，这很可能是可能的。我不确定人们是否完全意识到这一点。我有点觉得人们认为死亡是不可避免的，但归根结底，这是一个物理系统。有些东西会出错。从进化角度来说，这种事情发生是有道理的。而且肯定存在可以缓解它的干预措施。如果死亡最终被视为一种曾经发生在人类身上的迷人事情，那会很有趣。我不认为这不可能。我认为这很可能，并且要靠我们的想象力去预测没有死亡的世界是什么样子。

Yeah, that's kind of what I think about quite a bit as well. So the ultimate, or at least first, way to sneak up to the 'why' question is to try to escape the system, the universe. And then for that, you backtrack and say, okay, for that, it's going to take a very long time. So the 'why' question boils down from an engineering perspective to how do we extend time. I think that's question number one practically speaking, because you can't calculate the answer to the deeper questions in the time you have. And that could be extending your own lifetime or extending the lifetime of human civilization, of whoever wants it. Not many people might not want that, but I think people who do want it, I think it's probably possible. And I don't know that people fully realize this. I kind of feel like people think of death as an inevitability, but at the end of the day, this is a physical system. Some things go wrong. It makes sense why things like this happen evolutionarily speaking. And there's most certainly interventions that mitigate it. It would be interesting if death is eventually looked at as a fascinating thing that used to happen to humans. I don't think it's unlikely. I think it's likely, and it's up to our imagination to try to predict what the world without death looks like.

是的，很难想象。我认为价值观会完全改变。可能吧。我不太相信那些‘没有死亡就没有意义’之类的观点。我直觉上不相信那些论点。我认为有很多意义，有很多东西要学，它们有趣、令人兴奋。我想知道，我想计算，我想改善所有活着的人类和生物的状况。然而，我们寻找意义的方式可能会改变。有很多人，可能包括我自己，从事物的有限性中寻找意义，但这并不意味着那是意义的唯一来源。

Yeah, it's hard to imagine. I think the values will completely change. Could be. I don't really buy all these ideas that without death there's no meaning, there's nothing. I don't intuitively buy all those arguments. I think there's plenty of meaning, plenty of things to learn, they're interesting, exciting. I want to know, I want to calculate, I want to improve the condition of all the humans and organisms that are alive. Yet the way we find meaning might change. There are a lot of humans, probably including myself, that find meaning in the finiteness of things, but that doesn't mean that's the only source of meaning.

是的，我确实认为很多人会接受这一点，我觉得这很棒。我喜欢人们可以自己选择冒险的想法。比如，你天生就是一个有意识的自由实体，我愿意这么认为，你拥有不可剥夺的生命权和追求幸福的权利。我不知道你是否在自然中拥有那种幸福景观。你大多可以自己选择冒险，这并不完全正确，但我仍然很确定我是一个 NPC。但 NPC 不可能知道自己是 NPC。可能有不同程度的意识。我认为没有比这更美的结尾了。安德烈，你是一个了不起的人。我很荣幸你能和我交谈。你为机器学习世界、人工智能世界所做的一切，激励人们，教育数百万人，真是太棒了。我迫不及待想看你下一步做什么。很荣幸，伙计。非常感谢你今天能来聊天。

Yeah, I do think many people will go with that, which I think is great. I love the idea that people can just choose their own adventure. Like you are born as a conscious free entity by default, I'd like to think, and you have your unalienable rights for life in the pursuit of happiness. I don't know if you have that in the nature, the landscape of happiness. You can choose your own adventure mostly, and that's not fully true, but I still am pretty sure I'm an NPC. But an NPC can't know it's an NPC. There could be different degrees and levels of consciousness. I don't think there's a more beautiful way to end it. Andre, you're an incredible person. I'm really honored you would talk with me. Everything you've done for the machine learning world, for the AI world, to just inspire people, to educate millions of people, it's been great. And I can't wait to see what you do next. It's been an honor, man. Thank you so much for talking today.

太棒了，谢谢。感谢收听本期与安德烈·卡帕西的对话。要支持本播客，请查看描述中的赞助商。现在，让我用塞缪尔·卡林的一句话作为结尾：‘模型的目的不是拟合数据，而是锐化问题。’感谢收听，下次再见。

Awesome, thank you. Thanks for listening to this conversation with Andrej Karpathy. To support this podcast, please check out our sponsors in the description. And now let me leave you with some words from Samuel Carlin: 'The purpose of models is not to fit the data, but to sharpen the questions.' Thanks for listening and hope to see you next time.