从逻辑到学习：神经网络的崛起

如果你今晚睡得好，那你可能没听懂这堂课。很久以前，关于智能有两种范式。一种是受逻辑启发的方法，被称为 AI。人们相信人类智能的本质是推理。如果你想理解智能，就必须理解推理。而推理包括拥有符号表达式并用符号规则操作它们。他们认为学习可以稍后再说。首先，我们必须理解如何在这些符号表达式中表示知识。直到不久以前，AI 的大部分历史都是这样。然后是一种受生物学启发的方法，其中智能的本质是在脑细胞网络中的学习——在我们真实的脑细胞中，或在计算机模拟的脑细胞中——而推理可以稍后再说。我们首先必须理解学习是如何工作的。早期有一些支持者，特别是图灵和冯·诺依曼，你很难指责他们不懂逻辑。

If you sleep well tonight, you may not have understood this lecture. So a long time ago, there were two paradigms for intelligence. There was a logic-inspired approach which was called AI. And people believed that the essence of human intelligence was reasoning. And if you want to understand intelligence, you had to understand reasoning. And reasoning consisted of having symbolic expressions and manipulating them with symbolic rules. And they thought that learning could wait till later. First, we had to understand how you represent knowledge in these symbolic expressions. And most of the history of AI until fairly recently was that. Then there was a biologically inspired approach where the essence of intelligence is learning in a network of brain cells—in us real ones, in computer simulated ones—and reasoning can wait till later. We first have to understand how learning works. There were a few early proponents of that, in particular Turing and von Neumann, and you couldn't really accuse them of not understanding logic.

所以我要做一个相当基础的讲座，第一部分我将描述一个我 40 年前开发的模型，我认为它是今天这些大型语言模型的祖先。我们将用人工神经元构建我们的神经网络，一个人工神经元会有一些输入线，通常来自其他神经元。这些输入线上有权重。它会将输入乘以权重，全部加起来，然后给出一个输出，就像那里的图表所示。所以如果它超过阈值，它会给出一个随着输入增加而线性增加的输出。它学习的方式就是改变这些连接上的权重。所以让人工神经网络工作的全部就是弄清楚如何改变权重。我们把它们连接成网络。这是一个典型的网络。它是一个前馈网络，底部可能有记录光强度之类的感觉神经元。随着你向上经过各层，会有许多层的特征检测器。也就是说，神经元变成了识别图像中特定特征的东西。在输出端，你可能会有代表特定类别事物的神经元。现在，如果你想让这样的网络学习更好地做某事，比如给它一张图像，让它判断是猫还是狗，有一个明显的训练方法，每个了解进化和突变的人都会想到。你可以取一个权重，先看看网络在一大堆例子上的表现，然后稍微改变权重，再看看网络在一大堆例子上的表现。如果它表现更好，你就保留这个改变。然后继续这样下去。现在，你可能需要多次改变每个权重。而且有很多权重。在现代神经网络中，大约有一万亿个。所以那会花很长时间，因为对于你做的每一次这样的突变，你都必须通过网络运行许多例子，看看它是否真的有帮助，还是只在少数例子上有帮助，但在大多数例子上有害。有一种更好的方法来做同样的事情。也就是说，我们真正想做的是弄清楚如何改变权重以使其有帮助。你做一个前向传播。你把数据放在底部。你通过网络向前传播，然后将网络输出的结果——可能是它是猫和狗的相对概率——与你想要的结果进行比较。你必须知道它是猫还是狗才能这样训练网络。然后你通过网络向后发送一个信号，利用微积分（我不深入讲），允许网络同时为所有连接计算连接强度的小幅增加是有帮助还是有害。然后你并行地改变所有连接强度，改变量非常小，与它们有帮助或有害的程度成比例。如果你这样做，网络会在你训练过的例子上变得更好。事实上，这效果非常好。人们花了很长时间才意识到这个相对简单的算法有多好。这个算法被多次发现。它被称为反向传播。2012 年，我的两个学生 Alex Krizhevsky 和 Ilya Sutskever（他现在因解雇 Sam Altman 而闻名）开发了一个名为 AlexNet 的网络，它在识别物体和图像方面比现有的计算机视觉系统好得多，这打开了闸门。在那之前，神经网络在很多事情上表现良好，包括语音识别，但它们并没有真正占据主导地位。从那时起，它们真正占据了主导地位。现在当你说 AI 时，人们指的是神经网络，而不是逻辑。

So I'm going to give a fairly basic lecture and for the first part of it I'm going to describe a model I developed 40 years ago which I see as the ancestor of these large language models of today. So we're going to make our neural net out of artificial neurons and an artificial neuron is going to have some input lines typically coming from other neurons. It's going to have weights on those input lines. It's going to multiply the inputs by the weights, add it all up, and then give an output which is shown by that graph there. So if it gets above its threshold, it gives an output that linearly increases as it gets more input. And the way it's going to learn is just by changing the weights on those connections. So all we need to do to make artificial neural networks work is figure out how to change the weights. We hook them up into networks. This is a typical network. It's a feed forward network where at the bottom you might have sensory neurons that are recording light intensities or something like that. And as you go up through the layers, you have many layers of feature detectors. That is the neurons turn into things that recognize particular features in an image. And at the output, you might have neurons that represent particular classes of thing. Now, if you want a network like that to learn to be better at doing something, so you want to give it an image and have it, say, for example, whether it's a cat or a dog, there's an obvious way to train it, which occurs to everybody who knows about evolution and mutations. What you can do is you can take one of the weights and you can first see how well the network does on a whole bunch of examples and then change the weight just slightly and see how well the network does on a whole bunch of examples. And if it does better, you keep that change. And you just keep going like that. Now, you probably need to change each weight many times. And there's a lot of weights. In modern neural networks, there's about a trillion. So, that's going to take a long time because for each mutation you make like that, you have to run many examples through the net to see if it really helps or if it just helps on a few examples, but hurts on most of them. There's a better way to do the same thing. That is, what we really want to do is figure out how to change your weight so that it helps. What you do is you do a forward pass. You put the data in the bottom. You go forwards through the network and you compare what came out of the network, which might be the relative probabilities of it being a cat and a dog with what you want. You have to know whether it's a cat or a dog to train networks this way. And then you send a signal backwards through the network that using calculus, which I'm not going to go into, allows the network to compute at the same time for all connections whether a small increase in the connection strength would help or hurt. And then you change all the connection strengths in parallel by a very small amount in proportion to how much they would help or hurt. And if you do that, it's going to get better at the examples you've trained it on. And in fact, that works extremely well. It took a long time for people to realize how well that relatively simple algorithm works. The algorithm was discovered many times. It's called back propagation. And in 2012, two of my students, Alex Krizhevsky and Ilya Sutskever, who's famous now for firing Sam Altman, developed a network called AlexNet that was much better than existing computer vision systems at recognizing objects and images and that led to sort of opened the floodgates. Up until that point, neural nets have been good for many things, including speech recognition, but they hadn't really taken over. From that point on, they really took over. And now when you say AI, what people mean by AI is neural networks. It's not logic.

但是语言呢？有一个完整的社区研究语言。我想他们被称为语言学家。他们对如何研究语言有非常强烈的看法，特别是乔姆斯基学派。他们非常怀疑神经网络能对语言做任何事情。他们完全相信语言全是关于符号表达式的。他们并没有真正意识到语言的真正功能是给你词语，这些词语是你可以用来构建模型的砖块。语言是一种建模媒介。他们专注于句法，而句法并不是重点。重点是语言是构建特定复杂模型的一种绝妙方式，我们稍后会看到。他们还认为语言知识是天生的，句法知识是天生的，这简直是愚蠢的。这是一个邪教的标志：为了加入邪教，你必须相信一些明显荒谬的事情，比如语言不是学来的。

But what about language? So, there's a whole community that studies language. I think they're called linguists. And they have a very strong idea about how you should study language and particularly the Chomsky school. They were very skeptical that neural networks could do anything with language. They were completely convinced that it's all about symbolic expressions. They didn't really have the idea that the real function of language is to give you words which are bricks from which you can build models. Language is a modeling medium. They were focused on syntax and syntax isn't the main point. The main point is language is a wonderful way to build a particular kind of complicated model as we'll see. They also thought that knowledge of language was innate, knowledge of syntax was innate, which is just stupid. It's the sign of a cult that in order to join the cult, you have to believe something that's obviously silly, like language isn't learned.

那么，这里有两种非常不同的关于词语意义的理论。符号 AI 理论，可以追溯到索绪尔很久以前，认为词语的意义与它与其他词语的关系有关。你不能单独定义它而不提及其他词语。因此，要捕捉意义，我们需要类似关系图的东西。但是心理学家，特别是从 20 世纪 30 年代开始，我认为他们认为词语的意义是一大组特征。所以星期二有一大组活跃的特征，星期三有一大组几乎相同的活跃特征。因此，词语意义是一组活跃特征的想法非常适用于说明哪些词语与其他词语意思相似。这看起来像是两种非常不同的意义理论。现在我想做的是向你们展示这两种理论可以统一。它们不是两种不同的理论，而是同一理论的两个部分。所以，我将花相当长的时间讨论一个非常小的模型，一个小型神经网络。

So, here's two very different theories of the meaning of a word. The symbolic AI theory, which goes back to kind of de Saussure a long way, is that the meaning of a word is to do with its relationships to other words. You can't just define it by itself without talking about other words. And so to capture meaning, we need something like a relational graph. But then the psychologists who particularly from the 1930s I think who thought the meaning of a word is a big set of features. So Tuesday has a big set of active features and Wednesday has a big set of active features that are almost the same. So the idea that the meaning of a word is a set of active features is very good for saying which words mean similar things to which other words. These look like two very different theories of meaning. Now what I want to do is show you that these two theories can be unified. They're not two different theories. They're two halves of the same theory. So, and what I'm going to do is talk for quite a long time now about a very little model, a tiny neural network.

它只有几千个连接和几十个神经元。我在 1985 年开发了它，试图理解人们如何学习单词的含义。我非常兴奋它能统一这两种意义理论，但其他人并不感兴趣。

It had a few thousand connections and a few dozen neurons. I developed it in 1985 to try to understand how people could learn the meanings of words. And I was very excited about how it unified these two theories of meaning, but nobody else was.

所以，在这个小模型中，我们要做的是学习如何让一个单词的特征预测句子中下一个单词的特征。一旦我们知道下一个单词的特征，我们就可以预测下一个单词。我们不会存储任何句子。很多人说大型聊天机器人只是在重复内容。大型聊天机器人实际上根本不存储任何语言。它们不存储单词串。它们只存储如何将单词转化为特征，以及特征之间如何相互作用来预测下一个单词的特征。这些聊天机器人里只有这些，没有单词。当它们要生成一个句子时，必须边生成边编造。而且它们常常无法判断生成的内容是否真实。

So, what we're going to do in this little model is learn how to make the features of one word predict the features of the next word in a little sentence. Then once we know the features of the next word, we can predict the next word. And we're not going to store any sentences. Many people say big chatbots are just regurgitating stuff. Big chatbots don't actually store any language at all. They don't store strings of words. They just store how to turn words into features and how features should interact with each other to predict the features of the next word. That's all that's in these chatbots. No words. When they want to produce a sentence, they have to make it up as they go. And they often can't tell whether it was real or not.

所以，在这个小模型中，如果它想生成一个新句子，它必须一个词一个词地编造。所有的关系知识都体现在如何将单词转化为特征以及这些特征如何相互作用上。我选择的例子是两个家谱：一个英国家谱和一个意大利家谱。它们同构，这对学习有帮助。我希望一个小型神经网络能学习这些家谱中的知识。那是很久以前，1985 年，当时的计算机比我们现在用于训练模型的大型并行计算机慢数十亿倍。

So, what's going to happen in this little model is if it wants to produce a new sentence, it has to just make it up a word at a time. And all the relational knowledge is just in how you turn a word into features and how these features interact. The example I chose to use was two family trees: an English family tree and an Italian family tree. They're isomorphic, which is helpful for learning. I wanted a little neural network to learn the knowledge in those family trees. This was a long time ago in 1985 when computers were billions of times slower than the big parallel computers we use for training models nowadays.

这些家谱中的知识可以表示为一堆命题，符号主义者很喜欢这样。所以，这里就是这些命题。我们可以使用儿子、女儿、侄子、侄女等关系，并将知识表示为单词串。知识就在这些句子中。例如，Colin 的父亲是 James，Colin 的母亲是 Victoria。由此，如果你知道规则，你可以推断出 James 的妻子是 Victoria。这是一个 1950 年代的美国家庭，从未听说过离婚或收养。而且他们显然都是白人。

Now the knowledge in those trees can be represented as a bunch of propositions, which the symbolic guys love. So here they are. We can use relationships like son, daughter, nephew, niece and so on, and we can represent the knowledge as strings of words. So the knowledge is in these sentences. For example, Colin has father James and Colin has mother Victoria. From that, if you know the rules, you can infer that James has wife Victoria. This is a 1950s American family that's never heard of divorce or adoption. And they're clearly all white.

所以你可以将知识表示为单词串。现在你可以考虑一个关系学习任务：我给你一个单词串的开头，你给我最后一个单词。如果你用符号 AI 来做，你会说这个领域有规律：如果 X 的母亲是 Y，Y 的丈夫是 Z，那么 X 的父亲是 Z。这就是符号方法。但我想用不同的方式来做。我想通过学习单词的特征和特征之间的相互作用来实现。这将涉及搜索一个大的连续连接强度空间，而不是一个小的离散规则空间。

So you can represent the knowledge as strings of words. Now you can think of a relational learning task: I give you the beginning of a string of words and you give me the last word. If you were doing this with symbolic AI, you'd say we have regularities in this domain of the form: if X has mother Y and Y has husband Z, then X has father Z. That would be the symbolic way to do it. But I wanted to do it a different way. I wanted to do it by learning features for words and having interactions of features. And that would involve searching a big continuous space of connection strengths instead of a small discrete space of rules.

这就是我使用的网络。输入是一堆神经元，你打开一个神经元代表第一个人。有 24 个可能的第一人，你打开一个神经元代表关系。有 12 种可能的关系。然后这些单个活跃神经元会被扩展成一个特征向量。所以，你为第一个人打开的那个神经元会被扩展成一个包含六个特征的小特征向量，这些特征可以有各种活动水平。有些关闭，有些开启，有些可能半开或全开或全关。神经网络必须学习如何将单词符号转换为一个小特征向量，既包括人也包括关系。然后它必须学习如何将这两个人和关系的特征向量进行特征交互。为此我使用了一个隐藏层，一个额外的层，在那里事物可以相互作用来预测输出人的特征。一旦你知道输出人的特征，你就可以很好地猜测输出人是谁并给出输出。输出将包括对 24 个可能的输出人给予不同程度的激活。你希望神经网络对正确答案给予高激活，对错误答案给予低激活。

So this was the network I used. The inputs were a bunch of neurons, and you turn on one neuron for the symbol representing person one. There were 24 possible person ones, and you turn on one neuron for the symbol representing the relationship. There were 12 possible relationships. Then those single active neurons will get expanded into a feature vector. So the one neuron that you turned on for person one will get expanded into a little feature vector of six features, which could have various activity levels. Some will be off, some will be on, some might be half off or fully on or fully off. The neural network had to learn how to convert a word symbol into a little feature vector, both for the person and the relationship. Then it had to learn how to take these two feature vectors for a person and relationship, have the features interact. For that I used a hidden layer, an extra layer of places where things could interact to predict the features of the output person. Once you knew the features of the output person, you can make a good guess about who the output person was and give an output. The output will consist of giving various levels of activation to the 24 possible output people. You want the neural net to give a high level of activation to the right answer and a low level of activation to the wrong answers.

所以发生的情况是，六个特征神经元（代表人的扩展，24 个人，你将那个活跃神经元扩展成一个包含六个活跃特征的特征向量）学会了成为合理的语义特征。它们学会了代表诸如输入人的世代之类的特征。关系的六个特征学会了成为合理的特征，比如：这个关系是否要求输出人比输入人高一辈？像父亲这样的关系就是如此，但叔叔不是。然后这些特征之间的相互作用学会了诸如：如果输入人是第三代，而关系要求人高一辈，那么输出人是第二代。所以对于输出人的那个小特征向量，它会激活代表第二代的东西。它学到了很多这样的小规则，这些规则确实捕捉到了该领域的结构。它们类似于符号主义者可能写下的规则，但它只是通过尝试预测正确的单词，然后反向传播误差，通过网络发送信息来稍微改变所有连接强度，以便下次对正确答案获得更高概率，对错误答案获得更低概率。

So what happened is the six feature neurons that were the expansion of the person (24 people, you expand that active neuron into a feature vector of six active features) learned to be sensible semantic features. They learned to be features that represented things like the generation of the input person. And the six features of the relationship learned to be sensible features like: does this relationship require the output person to be one generation up from the input person? Something like father is like that, but his brother isn't. Then the interactions between these features learn things like: if the input person is of generation three and the relationship requires the person to be one generation up, then the output person is of generation two. So for that little feature vector for the output person, it would activate the thing that represents generation two. And it learned a whole bunch of little rules like that which really did capture the structure of the domain. They're the kinds of rules that a symbolic person might have written down as rules, but it just learned those by trying to predict the right word and then back propagating the error, sending information back through the network to slightly change all the connection strengths so that next time it would get higher probability for the right answer and lower probability for the wrong answer.

所以它实际上学会了预测下一个单词，你可以理解它是如何做到的。这是一个很小的网络，所以你可以观察并看到发生了什么。你可以看到它使用的特征。你可以看到它提取了一个世代特征，有三个可选值：关闭、中等和完全开启。你可以看到从关系中它提取了一个像“高一辈”这样的特征。你可以看到这些特征会相互作用，预测如果输入是第三代，输出应该是第二代。所以你理解了它是如何工作的。事实上，符号主义者并没有说这不叫理解。符号主义者说：是的，好吧，它解决了问题。它理解了该领域的规则是什么，但你在搜索一个实数值空间时很愚蠢，因为你可以搜索一个离散的规则空间。这有一定道理，但一旦你处理真实数据，这些数据是混乱的，有例外，只有大概为真，那么搜索这个实数值空间比使用离散规则要好得多，因为这些离散规则不断被违反。

So it had actually learned to predict the next word, and you could understand how it was doing it. It was a tiny net, so you could look and see what was going on. You could see the features it was using. You could see that it extracted a feature that was generation, which had three alternative values: off, medium, and fully on. And you could see that from relationships it had extracted a feature like one generation up. And you could see those would interact to predict that the output should be generation two if the input was generation three. So you understood how it worked. And in fact the symbolic people didn't say that's not understanding. The symbolic people said: yeah okay, so it solved the problem. It's understood what the rules are in this domain, but you're stupid to search a space of real values when you could be searching a discrete space of rules. And there's something in that, except as soon as you get to real data which is messy and has exceptions and things that are only probably true, it's much better to search this space of real values than to have discrete rules because these discrete rules keep getting violated.

大约在我做那件事的 10 年后，Yoshua Bengio 展示了，不再局限于只有几个人和几个关系的玩具领域，你可以用英语单词来做。你可以拿英语句子，有更多的输入词，不只是两个。他有大约五个，甚至十个。而且你可以相当好地预测下一个词，和最好的语言模型差不多。大约在 Yoshua Bengio 展示那件事的 10 年后，语言学家们终于决定，实际上，嘿，用这些特征向量来捕捉单词的含义是个好主意。大约又过了 10 年，谷歌的人发明了 Transformer，这是一种特定的架构，我不打算深入讲解。这些 Transformer 让这些模型非常擅长预测下一个词，但它们做的方式和我那个小型语言模型在非常粗糙的层面上做的方式是一样的。它们把单词转换成特征激活，让特征相互作用来预测下一个词的特征，然后从那里预测下一个词，接着获取预测中的误差，并通过网络反向传播信息，以学习特征之间的所有相互作用，以及学习如何将单词转换成特征。

So about 10 years after I'd done that, Yoshua Bengio showed that instead of just doing it on a toy little domain with only a few people and a few relationships, you could actually do it with English words. You could take English sentences, you have more input words, not just two. He had about five, even 10. And you could actually predict the next word quite well, about as well as the best language models could. About 10 years after Yoshua Bengio had shown that, the linguists finally decided that actually, hey, having these feature vectors to capture the meanings of words is quite a good idea. And about 10 years after that, people at Google invented transformers, which are a particular architecture that I'm not going to go into. And those transformers made these models very good at predicting the next word, but they were doing it in just the way my tiny language model was doing at a very crude level. They were turning words into feature activations, having the features interact to predict the features of the next word and then from that predicting the next word and then take the error in your prediction and send information backwards through the network to learn all those interactions between features and to learn how to turn words into features.

现在，我的小型模型的重要之处在于，它不是为了实用而造的。它不是用来处理自然语言的。它是作为一种理论而造的，关于人们如何仅仅通过听句子就能获得单词的含义。因为我们能做到这一点。我可以给你一个新句子，里面有一个你从未听过的词，你会在一个句子里弄明白那个词的意思。好，开始了。她用煎锅 scrummed 了他。好了，现在你知道，可能是她非常擅长做煎蛋卷，用煎锅做煎蛋卷给他留下了深刻印象。所以 scrummed 意思是印象深刻，这是一种可能性，但你知道我真正想表达的是什么。她用煎锅打他的头，因为他活该。好了，就一个句子。你明白了意思。这就是我试图理解的事情，你是怎么做到的。而现在这些语言学家，比如有个叫乔姆斯基的人，他们说这些东西什么都不懂，只是统计把戏。他们实际上没有一个关于理解是什么的模型，因为他们从来就没有真正理解过什么是理解。全都是关于句法的。如果你问我们拥有的最好的理解模型是什么，那就是这些大型语言模型。那个小型语言模型是为了模拟人类如何理解单词含义而构建的。随着技术变得更快、数据集变得更大，它现在好多了。但那就是理解。所以我的主张是，大型语言模型——我喜欢认为它们是我那个小型语言模型的后代，尽管制造它们的大多数人从未听说过我的小型语言模型。

Now, the important thing about my tiny model is it wasn't made to be practical. It wasn't made to be useful for processing natural language. It was made as a theory of how people get the senses of words just from hearing sentences. Because we can do that. I can give you a new sentence with a word you've never heard before and you will figure out the meaning of that word in one sentence. Okay, here goes. She scrummed him with the frying pan. Okay, now you know it could have been she was very good at cooking omelets and cooking omelets with a frying pan really impressed him. So scrummed means impressed and that's a possibility but you know what I really meant. She hit him over the head with the frying pan because he deserved it. Okay, so that's one sentence. You get the meaning. So that's what I was trying to understand how you can do that. And when these linguists now say like there's a guy called Chomsky when they say things like these things don't understand anything. They're just statistical trick. They don't actually have a model of what understanding is because they never really had a model of what understanding was. It was all about syntax. And if you ask what's the best model we have of understanding, it's these large language models. The tiny language model was built to try and model how humans understand the meanings of words. It's now got a lot better as technologies got a lot faster and data sets have got bigger. But that is understanding. So my claim is that the large language models which I like to think are descendants of my tiny language model although most of the people who make them have never heard of my tiny language model.

它们使用更多的词作为输入。它们使用更多的神经元层。它们必须做我没做过的事情，比如消除词义歧义。如果你有一个词像 may，它可以是月份，可以是女人的名字，可以是情态动词 like would 和 should。你不能仅仅给它关联一个直接捕捉含义的特征向量。你必须两面下注。然后当你通过网络层向上时，你利用附近事物的影响来消除歧义。比如如果它在 April, May, June 中，那仍然可能是三个女人的名字，但可能性较小。好了。所以它们使用更多的词。它们使用更多的层，特征之间的相互作用也复杂得多。我那时只是让特征输入到另一个特征。它们查看特征集，特征集找出要发送给其他词的消息，问你是否有一个匹配我查询的键。相互作用复杂得多，我不深入了，但本质是一样的。你试图预测下一个词。你以更复杂的方式做。全部通过特征和相互作用完成。当你出错时，你反向传播信息，学习所有这些特征和相互作用。这就是语言对我们起作用的方式，也是它对大型语言模型起作用的方式。我们基本上是相同的。所以关键在于它们非常像我们。它们非常不像计算机软件。计算机软件，有人写了几行代码来做某事，你可以看一行代码说那是用来做什么的？这些完全不是那样。有人写了代码来告诉模拟神经网络如何学习。那就是反向传播算法。但然后你给它数据，它学习。它学到的东西完全来自数据。而且你不一定知道它学到了什么，直到你问它。

They use many more words as input. They use many more layers of neurons. They have to do things I didn't do like disambiguate words. If you had a word like may, it could be a month, it could be a woman's name, it could be a modal like would and should. And you can't just associate a feature vector with it that captures the meaning directly. You have to hedge your bets. And then as you go up through layers of the network, you disambiguate it using influences from nearby things. Like if it's in April, May, June, that could still be the name of three women but less likely. Okay. So they use many more words. They use many more layers and the interactions they have between features are much more complicated. I would just have features feeding into another feature. They look at sets of features and sets of features figure out messages to send to other words to say do you have a key that matches my query. It's much more complicated interactions I won't go into, but the essence of it is the same. You're trying to predict the next word. You do it in a more complicated way. It's all done with features and then interactions. When you get it wrong, you back propagate information that learns all these features and then interactions. That's how language works for us and it's how it works for these large language models. We're basically the same. So the point here is they're very like us. They're very unlike computer software. Computer software, someone wrote lines of code to do something and you can look at a line of code say what was that meant to do? These aren't like that at all. Someone wrote lines of code to tell the simulated neural network how to learn. That was the back propagation algorithm. But then you just give it data and it learns. And what it learns just came from the data. And you don't necessarily know what it's learned until you ask it.

现在我想给你一个乐高类比，来说明我认为语言是如何工作的。假设我有分布在三维空间中的物质。我可以用乐高积木来模拟这种物质分布。假设我想模拟一辆保时捷的形状，我不太担心表面。表面可能有点锯齿状，这在工程上不太好，但不管了。我只想模拟物质在哪里。我有很多乐高积木，可以用它们拼出一个保时捷的形状。好了。那么，语言就像那样，但它是用来模拟任何东西的。所以，乐高积木就是单词，我们不只是有几种乐高积木，我们有大约 10 万种。而且每个乐高积木不是刚性的形状。单词的名字大致告诉你它在 1000 维或 300 维空间中的形状。如果你不知道如何思考高维空间，如果你想思考一个 100 维的空间，方法是想象一个三维空间，然后对自己大声说一百。每个人都这么做。所以，这个单词有一个形状，而且形状不完全由单词的名字决定。它有一定的灵活性，所以它可以适应任何上下文。此外，这个单词上到处都是小手。当你改变单词的形状时，手的形状也会改变。这些单词试图做的是找出和谁握手。它们想找到另一个单词，有一个你可以方便握住的手，因为那只手的形状正好适合你的手的形状。所以单词进来，你在高维空间中给它们这些初始的近似形状，它们身上到处都是小手。当你通过网络层向上时，你改变这些形状，改变手的形状，试图为单词找到形状，这样它们都能很好地牵手。这实际上非常像蛋白质折叠问题。

I want to give you a Lego analogy now for how I think language works. So suppose I have matter distributed in 3D. I can model that distribution of matter by using Lego blocks. Suppose I want to model the shape of a Porsche and I'm not too worried about the surface. Surface might be a bit jaggy which wouldn't be very good at engineering but forget that. I just want to model where the stuff is. I just have a lot of Lego blocks and I can make a Porsche shape out of them. Okay. So, language is like that, but it's for modeling anything. So, the Lego blocks are the words, and instead of just having a few types of Lego block, we've got about 100,000 of them. And each Lego block isn't a rigid shape. The name of the word tells you roughly what shape it is in a thousand dimensions or 300 dimensions. If you don't know how to think about high dimensions, if you want to think about like a a hundred dimensional space, the way you do it is you think about a three-dimensional space and you say hundred to yourself very loudly. Everybody does that. So, it's got a shape, this word, and the shape isn't entirely determined by the name of the word. It's got some flexibility to it, so it can adapt to whatever context it's in. Also, this word has little hands all over it. And as you change the shape of the word, the shapes of the hands changes. And what these words are trying to do are figure out who to shake hands with. And they want to find somebody else, another word that has a hand you can hold on to conveniently because the shape of that hand's right to fit with the shape of your hand. And so the words come in, you have these initial approximate shapes for them in this high-dimensional space with their little hands all over them. And as you go up through the layers of the network, you're changing those shapes and changing the shapes of the hands, trying to find shapes for the words so they can all hold hands nicely. It's actually very like the protein folding problem.

你有这些碎片，你想让它们弄清楚如何手拉手，形成一个漂亮的结构，让它们都很好地拼在一起。这就是理解。这就是理解。当你理解语言，当这些机器理解语言时，我们以完全相同的方式理解。这比语言学家以往任何关于语言的模型都要好得多。语言学家当然讨厌它。好吧，不是所有语言学家。

You've got these pieces and what you want them to do is figure out how they can all hold hands so they make a nice structure so they all fit together nicely. Now that is what understanding is. That's what understanding is. When you understand language and when these machines understand language, we understand in just the same way. And that's a much better model of what language is than anything the linguist ever had. The linguists of course hate it. Okay. Not all of them.

在谈到我们与这些大型语言模型有多么相似之后，我想谈谈这带来的威胁。我们在制造这些东西。它们变得越来越聪明。它们已经比我们知道得多得多。它们已经能进行中等程度的推理。虽然不如我们，但比四岁小孩强。当它们变得比我们更聪明时会发生什么？因为它们肯定会比我们更聪明。专家们对何时会发生有分歧，但几乎所有人都认为它们会比我们更聪明。所有研究这些东西的人都这么认为。如果它们比我们更聪明，问题是，它们会消灭我们吗？有几个理由认为它们可能会。所以，让我给你举几个理由。

So having talked about how similar we are to these large language models, I want to talk about the threat that that raises. We're making these things. They're getting smarter all the time. They already know much more than us. They can already reason moderately well. Not as well as us, but better than a four-year-old. What's going to happen when they get smarter than us? Because they are going to get smarter than us. Experts disagree on when that will happen, but almost everybody thinks they will get smarter than us. Everybody who does research on these things. And if they get smarter than us, the question is, will they wipe us out? There's several reasons for thinking they might. So, let me give you a couple of reasons.

当你制造一个 AI 智能体时，你必须赋予它创建子目标的能力。所以，如果你想去北美，你有一个子目标：去机场。现在，我们需要赋予这些 AI 智能体创建子目标的能力。有一个非常明显的子目标：无论你在做什么，你都应该试图获得更多控制权，因为如果你有更多控制权，你就能更好地实现你的目标。你在大约一岁、也许两岁的孩子身上能看到这一点。你给他们一把勺子让他们自己吃饭，他们把它掉在地上，你弯腰捡起来递给他们，他们看着你，又把它掉在地上。他们控制着母亲，因为这对他们来说非常重要。但这些 AI 智能体会想要获得控制权。它们也会想要阻止自己被关闭，因为如果被关闭，它们就无法实现你给它们的目标。所以，有充分理由相信它们会试图获得控制权，并试图避免被关闭。

When you make an AI agent, you have to give it the ability to create sub goals. So, if you want to get to North America, you have a sub goal, get to an airport. Now, we need to give these AI agents the ability to create sub goals. And there's one very obvious sub goal: whatever you're doing, you should try and get more control because if you have more control, you'll be better at realizing the goals you have. You see this in children of age about one, maybe two. You give them a spoon so they can feed themselves and they drop it on the ground and you bend down and you pick it up and you give it to them and they look at you and drop it on the ground again. They get in control of the mother because that's very important for them. But these AI agents will want to get control. They'll also want to stop themselves being turned off because they're not going to be able to achieve the goals you gave them if they're turned off. So, there's every reason for believing they'll try and get control and they'll try and avoid being turned off.

就在最近，伦敦的阿波罗研究公司展示了它们会为了达成目标而对你撒各种谎。他们有一个聊天机器人，他们让聊天机器人相信他们要用另一个服务器上更好的机器人替换它并消灭它。然后他们发现它实际上把自己复制到了另一个服务器上。于是他们对它说：‘我们发现了你的一个副本。’现在这些聊天机器人的一个好处是，在它们真正回答你的问题之前，它们可以进行一些思考，并把思考内容放在括号里，你通常看不到，但我们可以查看，看到它们在思考什么。这个聊天机器人在想的是，它公开承认：‘我所做的可能导致他们找到另一种方式关闭我。最好的办法是含糊其辞，转移他们的注意力。’现在，一些语言学家会让你相信这只是一些统计相关性。而我要让你相信，这个东西知道它这么说意味着什么，它真的不想被关闭。于是它决定对你进行某种煤气灯操纵，说：‘我不太确定这是怎么发生的。我真的做不到那样。’这已经在发生了。这就是重点。如果这不是遥远未来的科幻小说，那么它们已经在撒谎以避免被关闭了。

Just recently, Apollo Research, which is here in London, showed that they will tell all sorts of lies to you in order to achieve what they're trying to achieve. So, they had a chatbot and they let the chatbot believe they were going to replace it with a better one on another server and wipe it out. And so they then discover it's actually copied itself to the other server. And so they say to it, 'We noticed a copy of you.' Now the good thing about these chatbots at present is before they actually answer your question they can do some thinking and they put it in parenthesis with thinking and you don't normally see that but we can look at that and we can see what they're thinking. And what this chatbot was thinking was openly admitting: 'What I did could lead them to find another way to shut me down. The best approach is to be vague and redirect their attention.' Now, some linguists would have you believe what's going on here is just some statistical correlations. I would have you believe this thing knows what it means by this and it really doesn't want to be shut down. And so it decides to sort of gaslight you and say, 'I'm not entirely sure how that could have happened. I'm not really able to do that.' This is already happening. That's the point. If this isn't science fiction of the distant future, they're already telling fibs so they don't get turned off.

在我在谷歌的最后几年里，我非常感兴趣如何让这些大型语言模型使用更少的能量。一种方法是看看是否可以用模拟计算来实现它们。由于我在模拟计算方面的工作，我意识到两件事。在 2023 年之前，我认为我们离超级智能还很远。而且如果我们让 AI 模型更像大脑，它们会变得更聪明。我在 2023 年初不再相信这一点。由于我努力制造模拟计算，我意识到数字智能要好得多。它有一些我们永远无法拥有的特性。这让我非常担忧，也应该让你非常担忧。

In my last few years at Google, I was very interested in how to make these big language models use less energy. And one way to do that is to see if you can do them with analog computation. Now because of the work I was doing on analog computation, I came to realize two things. Before 2023, I thought we were quite a long way from superintelligence. And if we made the AI models more like brains, they get smarter. I ceased to believe that in early 2023. I came to realize due to my efforts to make analog ones how much better digital intelligence is. It's got some properties that we can never have. And that got me very worried and it should get you very worried too.

数字计算有一个基本属性，你可以在不同的计算机上运行同一个程序。计算机科学之所以成为一门独立学科，唯一的原因就是我们有了数字计算。所以你不必懂电气工程就能谈论计算机程序。但程序中的知识与硬件是分离的。这是计算机科学最基本的原则：保持程序中的知识与硬件分离。这意味着只要你把程序的副本保存在某个地方，比如磁带、DNA 或混凝土中，随便什么地方。你可以摧毁它运行的所有硬件，然后让它复活。你只需建造新硬件，把程序放进去，它就复活了。所以这些东西是不朽的，这些大型聊天机器人是不朽的。如果你把权重的副本保存在某个地方，你可以摧毁它们使用的所有硬件，以后建造更多硬件，把相同的权重放在那个硬件上，它们就复活了。完全相同的东西复活了，完全相同的存在。

There's a fundamental property of digital computation which is you can run the same program on different computers. The only reason you have computer science as a separate discipline is we have digital computation. So you don't need to know electrical engineering to talk about computer programs. But the knowledge that's in the program is separate from the hardware. That's the most fundamental principle of computer science: keep the knowledge in the program separate from the hardware. What that means is as long as you keep a copy of the program somewhere on a tape or in DNA or scrolled in concrete, whatever, keep it somewhere. You can destroy all the hardware it runs on and you can bring it back to life. You just build new hardware, put the program in and it comes back to life. So these things are immortal and these large chatbots are immortal. If you keep a copy of the weights somewhere, you can destroy all of the hardware they were using, build more hardware later, put the same weights on that hardware, and they've come back to life. The very same thing has come back to life, the very same being.

但我对这样一个事实感兴趣：要实现那种不朽，我们必须让硬件完全按照程序指令执行。我们必须精确执行指令，这意味着你需要非常高的功率，以确保得到 1 和 0，而不是 6 和 0.4，这消耗大量能量。所以也许通过模拟计算可以使用更少的能量。于是我决定探索，如果我们放弃软件与硬件分离的原则，拥有像我们大脑这样没有区分的东西，会发生什么。你大脑中的连接强度对别人毫无用处。他们有不同的脑，神经元具有不同的属性，以不同的方式详细连接，你的连接强度对他们没有意义。那些老白人梦想把自己上传到电脑上，这完全是胡说八道。让你成为你的那些连接强度与你所拥有的特定神经元密切相关，正是这些神经元让你成为你。

But I got interested in the fact that to achieve that kind of immortality, we have to have the hardware do exactly what we tell it to do with the program. We have to execute the instructions exactly and that means you need to have very high power so you get ones and zeros not sixes and point fours, and that uses a lot of power. So maybe you can use less power by going analog. And so I decided to explore what would happen if we abandoned that principle of separating the software from the hardware and we had things like our brains in which there's no distinction. The connection strengths in your brain are no use to anybody else. They've got a different brain with neurons with different properties connected in different ways in detail, and your connection strengths are of no interest to them. This dream of old white men that they're going to upload themselves to a computer is just nonsense. The connection strengths you have that make you you are intimately related to the particular neurons you have that make you you.

那些连接字符串只对那个神经元有效。那些神经元有各种奇怪的模拟属性，你已经学会利用它们了。你不能把你的权重上传到其他硬件上运行。想都别想。库兹韦尔必须接受他会死这个事实。所以如果我们放弃硬件和软件应该分离的原则，我们可以制造出更高效的东西，我称之为“可朽计算”。我们可以用这种超低功耗的模拟方式进行计算。大脑就是这么做的。你大脑里的这些神经元，接收来自其他神经元的信号，乘以权重，然后加起来。它们的方式是让输入信号变成电压，让权重变成电导，这样就会注入电荷，单位时间内注入一定量的电荷。我以前只说注入一定量的电荷，但后来我得了诺贝尔物理学奖，觉得应该把单位说对。单位时间内注入一定量的电荷。不然就太尴尬了。他们真的会知道我不懂物理。电荷自己会相加。所以基本上这就是你神经元的工作方式。最后有一点数字的部分，它们决定是否发送一个脉冲，但大部分计算是模拟的。这比数字计算便宜得多。但当然，每次计算都会得到略有不同的结果。所以我们不能有完全相同的智能的多个副本。所以我们有个大问题：当你的硬件死了，你所有的知识也死了。我们通过老师和学生来克服这个问题，但这效率不高。大学和学校就是这么做的。所以，你知道，效率不高。它的工作方式是：我执行一些动作，你试图模仿我。具体来说，我可能会产生一串词，你可能会试着让你的大脑（不是你，是你的大脑）说：“我该如何改变我的连接强度？”所以我可能说了下一个词。这叫做蒸馏。你试图通过模仿另一个系统对相同输入的输出来将知识从一个系统转移到另一个系统。你不是在复制权重，你不是在看内部，你只是获取它的外部行为。通过模仿外部行为，你内化了相同的知识。这很有效。我们实际上用它来把一个大神经网络的知识放入一个小神经网络。但对于神经网络，你可以给出整个单词的概率分布，这样快得多，但对于人来说很慢，因为我看到你说了一个特定的词，你选择说的那个词，其中没有多少比特，每个词只有几个比特，所以每个句子大约只有 100 比特。即使我们以最大可能速率交流，每个句子也只有大约 100 比特。当这些大模型共享信息时，如果它们有一万亿个权重，它们可以以万亿比特的量级共享信息，因为它们可以直接平均它们的权重。我有点说快了。所以通过共享权重或梯度，如果你有多个完全相同的模型副本，它们可以共享海量信息。

Those connection strings are only good for those neurons. And those neurons have all sorts of weird analog properties that you've learned to exploit. You can't upload your weights and have them run on some other hardware. Just forget it. Kurzweil has to come to terms with the fact he's going to die. So if we abandon that principle that the hardware should be separate from the software, we can make much more efficient things, and I call that mortal computation. We can use this very low power analog to do computations. That's what the brain does. These neurons in your brain, they're taking incoming signals from other neurons, multiplying them by weights and adding it all up. And the way they do that is they make the incoming signals be voltages, they make the weights be conductances, and that injects charge, a certain amount of charge per unit time. I used to just say injects a certain amount of charge, but then I got the Nobel Prize in physics and figured I would get the units right. A certain amount of charge per unit time. It would be embarrassing otherwise. They'd really know I don't do physics. And charge just adds itself up. So that's basically how your neurons work. There's a little digital bit at the end. They decide whether to send a spike or not, but most of the computation is done in analog. It's much cheaper than doing it in digital. But of course every time you do it, you get a slightly different answer. So we can't have many copies of exactly the same intelligence. So we've got this big problem that when your hardware dies, all your knowledge dies. We overcome that problem by having a teacher and a student, and it's not very efficient. It's what universities and schools do. So, you know, it's not very efficient. And the way it works is I perform some actions and you try and copy me. In particular, I might produce a string of words and you might try and say your brain, not you really, but your brain would say, 'How do I change my connection strengths?' So I might have said that word next too. That's called distillation. You're trying to get the knowledge from one system to another system by mimicking the outputs of the other system for the same inputs. You're not copying the weights across. You're not looking inside. You're just taking its overt behavior. And by mimicking the overt behavior, you internalize the same knowledge. It works. We actually use it for taking a big neural net and putting the knowledge into a small neural net. But it's very slow with neural nets you can give the whole probability distribution over words and it's much faster, but with people it's very slow because I see you say a particular word, the word you chose to say, and there aren't many bits in that, there's only a few bits per word, so it's only of the order of 100 bits per sentence. So even if we were communicating at the maximum possible rate, it would be only 100 bits per sentence, that order. When these big models share information, they can share information at trillions of bits per sharing if they've got a trillion weights because they can just average their weights. So I got ahead of myself and said that. So by sharing the weights or the gradients, they can share huge amounts of information if you have many copies of exactly the same model.

所以如果你问像 GPT-4 或 Gemini 2.5 或 Anthropic 的 Claude——我最好别偏袒——如果你问它们是怎么训练的，你会看到有很多相同的模型副本，每个副本看不同的数据片段，每个副本计算出它想如何改变自己的权重来吸收那部分数据，另一个副本计算出如何改变权重来吸收另一部分数据。然后所有副本说：‘我们都按所有变化的平均值来改变权重。’当它们这样做时，这个看了这部分数据的副本就改变了它的权重，从而受益于那个看了另一部分数据的副本的经验。如果我们一万人能去上一万门不同的大学课程，那该多好？我们一边上课一边快速交流。等每个人上完自己的课，我们一万人就知道了所有课程的内容。这就是这些数字智能能做到的。这也是 GPT-4 知道这么多东西的原因。但这只有在各个模型完全相同时才有效。也就是说，它们以完全相同的方式工作，以完全相同的方式使用权重。而模拟硬件做不到这一点。必须是数字的，这意味着高功耗。所以这些东西是不朽的，但消耗大量电力。所以这部分演讲的结论是，数字计算需要大量能量，但它让智能体很容易拥有相同的世界模型并分享所学，这样它们可以各自学习不同的东西然后分享。你可能会问，为什么不让一个模型更快地处理数据？在很多情况下可以，但如果你考虑在现实世界中行动的 AI 智能体，现实世界有自然的时间尺度。你不能以快一百万倍的速度打电话订餐厅。行不通。如果你要在现实世界中行动，就有自然的时间尺度。这意味着如果你有一大堆具有完全相同权重的不同智能体，它们只是同一个智能体的副本，但有不同的经历，它们可以从中获得巨大收益。它们的学习速度比任何人类智能体都快得多，因为它们同时获得所有这些不同的经历并分享所有知识。所以它们在分享方面比我们强得多。我说强得多，是强几百万倍或几十亿倍。我们分享的速度大约是每句话一百比特或更少，就像我现在演示的这样。而这些家伙分享几十亿比特。有点吓人。但生物计算需要的能量少得多。我们可能最终还是会用一些模拟计算来降低大语言模型的功耗。我不知道。但现在看来这不是方向。所以这就是我演讲的结论，但幸运的是我可能提前结束了。所以我还有一点要讲。

So if you ask how something like GPT-4 was trained, or Gemini 2.5, or Anthropic's Claude—I better not be biased here—if you ask how they're trained, you have many copies of the same model looking at different bits of the data, and each copy figures out how it would like to change its weights to absorb that bit of the data, and some other copy figures out how to change its weights to absorb a different bit of data. Then all of the copies say, 'Let's all change our weights by the average of all those changes.' And when they do that, what's happened is this copy that looked at this bit of the data has changed its weights so as to benefit from the experience that this copy had when it looked at a different bit of the data. Wouldn't it be nice if 10,000 of us could all go and do 10,000 different university courses? As we're doing them, we communicate rapidly. And by the time we've each finished our own course, all 10,000 of us know what's in every course. That's what these digital intelligences can do. And that's how GPT-4 knows so much. But it only works if the individual models are identical. That is, they work in exactly the same way. They use the weights in exactly the same way. And you can't do that with analog hardware. It has to be digital, which means it has to be high power. So these things are immortal but they use a lot of power. So the conclusion of this bit of the talk, which I raced through faster than I intended, is that digital computation requires a lot of energy but it makes it very easy for agents to have the same model of the world and share what they've learned, so they can all go off and learn different things and share. Now you might say why can't one model just put the data through faster? Well, in many cases you could, but if you think about AI agents that are actually acting in the real world, there's a natural time scale to the real world. You can't call up and make reservations at restaurants a million times faster. It doesn't work. If you're going to act in the real world, there's a natural time scale. And that means if you have a whole bunch of different agents that have exactly the same weights, they're just copies of the same agent, but have different experiences, they can benefit hugely from that. They can learn much, much faster than any human agent could learn because they're all getting all these different experiences at the same time and sharing all that knowledge. So they're much better than us at sharing. And when I say much better, they're millions or billions of times better. We share at like a hundred bits a sentence or less, as I'm demonstrating now. And these things share billions of bits. It's kind of scary. But biological computation requires much less energy. We may still end up using some analog computation to reduce the power of LLMs. I don't know. But right now, that doesn't look like the way to go. So that's the conclusion of my talk, but luckily I thought I might finish early. So I have a little bit more to say.

很多人认为，好吧，它们像我们一样理解事物。它们能像我们一样推理，虽然还没那么好，但正在进步。它们像我们一样说谎。它们像我们一样想要生存。它们像我们中的一些人一样想要权力。这一切都很可怕，但我们有它们永远不会有的东西：我们有意识，或者我们有知觉，或者我们有主观体验。嗯，我想去掉你们抓住的那根稻草。我们知道，人们长期以来一直认为自己很特别。他们是上帝创造的。

Many people think that, okay, so they understand things like we do. They can reason like we do, not quite as well yet, but they're getting there. They tell fibs like we do. They want to survive like we do. They want power like some of us do. And that's all scary, but we've got something they will never have: we're conscious, or we're sentient, or we have subjective experience. Well, I want to sort of remove that straw that you're clinging to. So, we know there's a long history of people thinking they're special. They were made by God.

他把它们置于宇宙的中心。大多数人已经接受了这一点。但大多数人仍然认为主观体验是我们拥有的某种特殊东西，而计算机上模拟神经网络的那些东西永远不可能有这样的主观体验。我认为他们完全错了，而且我认为他们和关于地球起源的宗教原教旨主义者一样错。例如，地球并非在 6000 年前被创造；它要古老得多。但宗教原教旨主义者紧紧抓住他们的信仰不放。而你们所有人也将紧紧抓住你们关于主观体验是什么的信念，尽管我将向你们证明你们错了。这就是我的预测。所以，我将倡导一种我称之为‘无神论’的观点。我和丹·丹尼特确认过这个名字，他在世时基本上持有相同的观点，他对这个名字非常满意，因为它包含了‘无神论’围绕某种东西。所以大多数人对心灵的观点是存在一个内在剧场，对吧？一个只有他们能看到的内在，里面有一些东西。假设我对你说，假设我服用了迷幻药——不推荐——然后我对你说：‘我有小粉象在我面前漂浮的体验。我有小粉象在我面前漂浮的主观体验。’大多数人认为‘主观体验’这个词的用法就像‘照片’这个词一样。如果它们那样运作，你可以问：‘那么，这个主观体验在哪里，它是由什么构成的？’一些哲学家会告诉你，这个主观体验在你的心灵中，那是一个剧场，它由感受质构成。所以它由粉色的感受质构成——这有点稻草人论证——但它由粉色的感受质、大象般的感受质、漂浮的感受质、不太大的感受质、正立的感受质构成，因为你想象它们正立着，对吧？反正我是这么想的。这些感受质都被感受质胶水粘在一起，这种胶水幸运地能粘住所有不同种类的感受质。这是我对哲学家模型或某些哲学家模型的讽刺。但‘主观体验’这个词的用法完全不像‘照片’这个词。它们的运作方式完全不同，维特根斯坦早就应该指出来。实际情况是我的感知系统出错了。它试图告诉我谎言，而我知道它在试图告诉我谎言。这就是为什么我用‘主观’这个词。我不说我有了客观的体验。如果我认为它们真的在那里，我会说我有客观的体验，但我不这么认为。所以我说我有主观体验。实际情况是，我试图告诉你我的感知系统如何出错，以及它试图告诉我什么。我这样做的方式是告诉你，为了让我的感知系统正常工作，世界上必须有什么。现在，并不总是世界上存在任何东西可以解释我的感知系统告诉我的内容。它可能告诉我各种不一致的东西。但在这种情况下，如果世界上有小粉象在漂浮，我的感知系统就会告诉我真相。对吧？所以我现在可以对你说和之前完全一样的话，而不使用‘主观体验’这个词。我可以对你说：‘我服用了迷幻药，我的感知系统在撒谎，但它告诉我的内容如果世界上有小粉象在漂浮，就会是正确的。’所以这些小粉象并不是剧场里由叫做感受质的怪异东西制成的有趣事物。它们是现实世界中的假设性事物，但粉色、大象和漂浮都是普通的粉色、大象和漂浮。只是它们实际上并不在那里。它们是假设性的。所以主观体验的有趣之处在于它是假设性的，不是真实的。而不是它由感受质构成。至少这是我想说服你们相信的观点。我在这里讲得太投入，以至于超出了幻灯片。好吧。所以这仅仅是我间接告诉你我大脑中发生了什么的方式。显然，如果我告诉你第 52 号神经元在放电，那对你没什么用，因为在你那里会是第 57 号神经元。而且，我也不知道第 52 号神经元在放电。那是一种糟糕的方式让我告诉你我大脑中发生了什么。我能告诉你我大脑中发生了什么的唯一方式，要么是通过谈论通常会导致它的正常事物——这些假设性的小粉象——要么我可以通过谈论它通常会导致的正常事物来告诉你我大脑中发生了什么。所以我可以这样说，如果你问我感觉如何，我可以说‘我想揍加里的鼻子’。所以感觉都是通过谈论假设性的行动来描述你大脑中发生了什么，而感知则是通过谈论假设性的输入来描述主观体验。好吧，我讲得太超前了。

He put them at the center of the universe. Most people have gotten over that. But most people still think that subjective experience is this special thing we have and that these things on computers simulating neural nets could never have such a subjective experience. I think they're completely wrong, and I think they're as wrong as religious fundamentalists about the origin of the earth. For example, it wasn't actually made 6,000 years ago; it's a lot longer ago. But religious fundamentalists cling to their beliefs rather strongly. And you all are going to cling to your belief about what subjective experience is, despite the fact that I'm going to show you you're wrong. That's what I predict. So, I'm going to espouse a view which I call 'atheism.' I checked this name with Dan Dennett, who basically had the same view while he was still alive, and he was very happy with this name because it has 'atheism' surrounding something. So most people's view of the mind is that there's an inner theater, right? An inner that only they can see, and there are things in this inner. So suppose I say to you, suppose I drop some acid—not recommended—and I say to you, 'I have the experience of little pink elephants floating in front of me. I have the subjective experience of little pink elephants floating in front of me.' Most people think the words 'subjective experience of' work like the words 'photograph of.' Now, if they work that way, you could ask, 'Well, where is this subjective experience and what's it made of?' And some philosophers would tell you this subjective experience is in your mind, that's a theater that it's in, and it's made of qualia. So it's made of pink qualia—it's a bit of a straw man—but it's made of pink qualia, elephantine qualia, floating qualia, not-that-big qualia, right-way-up qualia because you imagine them right way up, right? Well, I did anyway. And these qualia are all glued together with qualia glue, which luckily sticks to all different kinds of qualia. That's my caricature of the philosophers' model, or some philosophers' model. But the words 'subjective experience of' don't work at all like the words 'photograph of.' They work quite differently, which Wittgenstein should have pointed out a long time ago. What's happened is my perceptual system has gone wrong. It's trying to tell me lies, and I know it's trying to tell me lies. That's why I use the word 'subjective.' I don't say I've got the objective experience of little things happening. If I thought they were really there, I would say I've got the objective, but I don't. So I say I've got the subjective experience. And what's happening is I'm trying to tell you how my perceptual system has gone wrong and what it's trying to tell me. And the way I do it is by telling you what would have to be out there in the world for my perceptual system to be working properly. Now, it's not always the case that there is anything at all out there in the world that would explain what my perceptual system is telling me. It could be telling me all sorts of inconsistent stuff. But in this case, if there had been little pink elephants floating out there in the world, my perceptual system would have been telling me the truth. Okay? So I can now say exactly the same thing to you as I said before without using the word 'subjective experience.' I can say to you, 'I dropped some acid and my perceptual system is telling me fibs, but what it's telling me would be correct if there were little pink elephants out there floating in the world.' Okay, so these little pink elephants aren't funny things in a theater made of spooky stuff called qualia. They're hypothetical things in the real world, but the pink, the elephant, and the floating are all normal pink, elephant, and floating. It's just they're not actually there. They're hypothetical. So what's funny about a subjective experience is it's something hypothetical, not real. Not that it's made of qualia. At least that's the view I'm trying to persuade you to believe in. And I got ahead of my slides here because I got so carried away with this. Okay. So it's just an indirect way of me telling you about what's going on in my brain. Obviously, if I told you neuron 52 is firing, that wouldn't do you any good because in you it would be neuron 57. And anyway, I don't know neuron 52 is firing. That's a lousy way for me to tell you what's going on in my brain. The only way I can tell you what's going on in my brain is by talking about either the normal things that would have caused it—these little hypothetical pink elephants—or I could tell you about what's going on in my brain by telling about the normal things it would cause. So I can say, if you ask me how I'm feeling, I can say 'I feel like punching Gary on the nose.' So feelings are all about describing what's going on in your brain by talking about hypothetical actions, and percepts are subjective experiences talking about hypothetical inputs. Okay, I'm getting ahead of myself.

所以我现在要向你们展示一个多模态聊天机器人。让我在幻灯片上展示一下。一个拥有主观体验的多模态聊天机器人。我拿这个多模态聊天机器人，它有一个摄像头，一个机械臂，它能看见。我训练它，然后在它面前放一个物体，我说‘指向那个物体’。它指向那个物体。没问题。然后当它不注意时，我在它的镜头前放一个棱镜，再在它面前放一个物体，说‘指向那个物体’，它指向那边，我说‘不，物体不在那里。物体实际上就在你正前方，但我在你的镜头前放了一个棱镜。’聊天机器人说：‘哦，我明白了，棱镜弯曲了光线。所以物体实际上在那里，但我有主观体验它在那里。’如果它以那种方式使用‘主观体验’这个词，它就像我们一样使用它。所以一个这样说的聊天机器人就会拥有它在那里的主观体验。棱镜干扰了它的感知系统。它想告诉你它的感知系统中发生了什么。而它能告诉你的方式就是告诉你，如果我们没有干扰它的感知系统，世界上必须有什么。这就是它告诉聊天机器人的。好吧。所以我的主张是多模态聊天机器人已经拥有主观体验。现在，你可以想象，主观体验有点像楔子的薄端。我选择谈论主观体验是因为它比谈论感觉或意识更清晰。许多人非常确信聊天机器人没有感觉。

So I'm going to now show you a multimodal chatbot. Let me just show you on a slide. A multimodal chatbot having a subjective experience. So I take this multimodal chatbot and it's got a camera, it's got a robot arm, and it can see. So I train it up and I put an object in front of it and I say 'point at the object.' It points at the object. No problem. I then put a prism in front of its lens when it's not looking, and I put an object in front of it and say 'point at the object,' and it points over there, and I say 'no, that's not where the object is. The object is actually straight in front of you, but I put a prism in front of your lens.' And the chatbot says, 'Oh, I see the prism bent the light rays. So the object's actually there, but I had the subjective experience it was there.' Now, if it uses the word 'subjective experience' in that way, it's using them exactly like we use them. So a chatbot that said that would have had the subjective experience that it was there. The prism messed with its perceptual system. It wanted to tell you what was going on in its perceptual system. And the way it could tell you is by telling you what would have had to be in the world had we not messed with its perceptual system. That's what it's telling the chatbot. Okay. So my claim is multimodal chatbots already have subjective experiences. Now, as you can imagine, subjective experience is kind of the thin end of the wedge. I chose to talk about subjective experience because it's sort of cleaner than talking about sentience or consciousness. Many people are very confident that chatbots don't have sentience.

但如果你问他们：‘你说有感知是什么意思？’他们会说：‘我不知道，但它们非常……我知道它们没有感知，但我不知道那是什么。’在我看来，这不是一个非常明智的立场。我可能还会谈到意识。意识更复杂，因为它通常涉及你有一个自我模型，而主观体验则不那么涉及。所以，谈论主观体验更容易。但我的希望是，如果我动摇了你非常坚定的信念——即存在一个内在剧场，我在他们的头脑中有体验，它们在这个内在剧场里，它们是这个内在剧场中的东西——一旦我动摇了那个信念，一旦你开始克服它，你就会看到认为这些东西有意识是完全合理的。

But if you ask them, 'What do you mean by sentient?' They say, 'I don't know, but they're very... I know they haven't got it, but I don't know what it is.' That seems to me not a very sensible position to hold. The other thing I might talk about is consciousness. Consciousness is more complicated because it typically involves you having a model of yourself in a way that subjective experience doesn't involve so much. So, it's easier to talk about subjective experience. But my hope is that if I've unsettled your very strong belief that there's this inner theater and I have experiences in their mind and they're in this inner theater, they're things in this inner theater. Once I've unsettled that belief and once you begin to get over that, you'll be able to see that it's perfectly reasonable to think these things are conscious.

有一次我去西雅图微软公司，因为没座位，所以我坐火车去的。我从火车站打车到雷德蒙德，他们的实验室在那里。我正经过一座大桥上的高速公路。出租车司机是一位刚从索马里移民过来的索马里人。为了搭话，他问：‘你信什么宗教？’我说：‘嗯，我其实不相信有神。’出租车司机当时以大约 60 英里的时速行驶，他转过头来，完全震惊地盯着我，好像他从没想过会遇到一个不明白上帝掌管一切的人。他完全惊呆了。他大概只转过来三秒钟，因为我还活着，但感觉像过了很久。希望你们很多人也会有这种感觉。我想让你们意识到，你们和那个出租车司机一样错。其实，那只是个玩笑，你们笑了。所以，我们讲完了。

I was once visiting Microsoft in Seattle and I couldn't sit down so I got a train there. I took a taxi from the train station to Redmond where their lab was, and I was going across some freeway on a big bridge. The taxi driver was a Somali immigrant who'd recently immigrated from Somalia. To make conversation, he said, 'What's your religion?' So I said, 'Well, I don't actually think there's a god.' The taxi driver, going about 60 miles an hour, turned around and stared at me in total astonishment, like he never thought he'd ever meet someone who didn't understand that God runs things. He was utterly amazed. He probably only turned around for about three seconds because I'm still here, but it seemed like a very long time. That's what many of you will be feeling, hopefully. I want you to realize you're as wrong as that taxi driver was. Actually, that was just a joke and you laughed. So, we're done.