杰弗里·辛顿谈 AI 安全、放射科医生以及离开美国前往加拿大

杰夫，你今天怎么样？我有点不舒服，所以多穿了件外套，因为我觉得冷。嗯，但我会尽力回答你的问题。当然。我们很期待，非常感谢你来到这里，再次祝贺你。

Jeff, how are you doing today? I'm slightly sick, so I've got an extra jacket on cuz I'm cold. Well. Um but I will try to answer your questions anyway. Absolutely. Well, we're looking forward to it and thank you so much to for being here and uh congratulations again.

谢谢。

Thank you.

好的，是的，这值得再来一轮掌声。我有幸在 2019 年的一次活动中见到你，我们只聊了几分钟，但我想播放一段那次活动的短片，回顾我们当时的讨论，感觉像是很久以前的事了。我们来看看那段视频。嗯，你当然被称为人工智能教父。拥有这样的头衔感觉如何？

Okay, yes, uh that deserves another round of applause. I was lucky enough to meet you back in 2019 at an event and uh we had just a few minutes together, but I wanted to show just a really short clip from that event and our discussion at that time, which feels like a lifetime ago. So, let's take a look at that clip. Uh so, uh you of course are known as the Godfather of artificial intelligence. What does it feel to have a title like that?

嗯，有点尴尬，但感觉很好。

Um slightly embarrassing, but very nice.

那是一次很棒的对话，让我惊讶的是，自 2019 年以来，变化如此之大。

It was a a great conversation at the time and what kind of uh blows me away is how much has changed even since 2019.

我也很惊讶。变化太大了，确实如此。

It blows me away, too. It's changed dramatically. It it really has.

所以，我想谈的一件事是，我们会深入讨论 AI 的影响、安全性和伦理，但我也想先聊聊你最初为什么来到加拿大。我们在 2019 年确实讨论过这个问题。那么，是什么让你来到这个伟大的国家？

So, one of the things I wanted to talk about and we'll dive into uh more conversations around the impact of AI and safety and ethics, but I also wanted to start with why you ended up coming to Canada in the first place. We did have a chance to discuss that back in 2019. So, what landed you here in this great country?

我当时在美国的卡内基梅隆大学工作，研究进展很顺利，但我对里根政府对尼加拉瓜的政策有些不满。嗯，我认为在尼加拉瓜港口布雷是件坏事。嗯，我在匹兹堡的大多数同事认为这完全合理，因为那是美洲，他们应该掌控。嗯，我觉得这很糟糕。当时我并不知道。

So, I was working in an American university, Carnegie Mellon University, and research was going very well there, but I was somewhat unhappy with Reagan's policies towards Nicaragua. Um I I thought mining the harbors in Nicaragua was a bad thing to do. Um most of my colleagues in Pittsburgh thought it was perfectly reasonable cuz it was um the American hemisphere and they should own it. Um I thought this was terrible. Little did I know.

嗯，就像我说的，很多事情都变了。还有一件事我们聊过，今晚我想问你。我们在 2019 年讨论过 AI 战争，我引用你的一句话：“我觉得这很可怕。自主武器即将到来。”你确实在很多人之前就开始敲响警钟，谈论这些威胁。

Well, uh like I said, a lot has changed. One thing we talked about, too, which I wanted to ask you about uh tonight. Uh we did have a conversation in 2019 about uh AI warfare and uh I'm just going to read a quote you said, "I think it's quite scary. Autonomous weapons are just about here." And uh you really kind of started raising alarms before a lot of people were speaking about some of these threats.

嗯，不完全是，有个叫斯图尔特·罗素的人更早就在敲警钟。所以，我在 AI 安全方面算是后来者。我的许多同事已经思考这个问题超过 10 年了。嗯，我很幸运人们关注我说的话，但我确实是后来者。

Um not well, there's someone called Stuart Russell who was raising alarms well before that. So, I was a fairly late comer to AI safety. Many of my colleagues um have been thinking about it for more than 10 years. Um I was lucky in that people paid attention to what I said, but um I was a late comer to it.

那么，当你想到 2026 年今天 AI 安全方面的情况时，显然你有很多担忧，但我们很多人实际上正在现实中看到这一切发生。

So, when you think about what's happening today in 2026 uh when it comes to AI safety, I mean, clearly you you have many of these concerns, but many of us are actually seeing this play out in the reality in front of us today.

是的，比如最近，我们看到一家大型科技公司确实有一些道德原则。嗯，相当温和的道德原则。他们不希望自己的 AI 被用于大规模监控美国人。嗯，据我所知，只是美国人。嗯，他们也不希望 AI 被用于自主决定杀人的武器。嗯，这似乎很难反驳。嗯，国防部那个好人皮特·赫格塞斯希望他们能随意使用。嗯，Anthropic 拒绝了。于是，他们现在开始针对 Anthropic。OpenAI，嗯，我想周四 OpenAI 说他们支持 Anthropic，周五就抢走了他们的业务。嗯，所以我现在不再使用 ChatGPT 了。我喜欢 ChatGPT，嗯，但我正在转向 Claude，因为山姆·奥特曼的道德灵活性对我来说有点过分了。

Yes, so very recently, for example, we've seen that one of the big tech companies actually did have some moral principles. Um and sort of fairly mild moral principles. They didn't want their AI being used for mass surveillance of Americans. Um just of Americans as far as I can tell. And um they also didn't want it being used for weapons that decided by themselves to kill people. Um which seems sort of It's hard to argue with that. Um the Defense Department um that nice man, Peter Heckseth, wanted to um be able to use them however they liked. Um And Anthropic said no. And so, they've now gone after Anthropic. OpenAI um I think on the Thursday OpenAI said they stood with Anthropic and on the Friday they took their business. Um So, I've now stopped using ChatGPT. I I love ChatGPT. um but I'm moving over to Claude cuz um Sam Altman's moral flexibility is a bit too much for me.

我想今晚在座的很多人可能也会同意这个说法。

I think there's a lot of people in the room who would probably agree with that statement uh tonight uh as well.

嗯，我还想和你聊聊你做过的一些预测，因为我们很多人长期关注你，听过你对未来的预测。在我们今天坐下之前，你提到了一个预测，你说想谈谈，那是你在 2016 年做出的预测：五年内我们将不再需要放射科医生。

Um I wanted to also chat with you a little bit uh about some predictions that you have made because for many of us who followed you for a long time and uh listened to um your predictions about the future, there is one that you brought up during our conversation before we were sitting down today and you said you wanted to talk about it, a prediction that you made back in 2016 that we will not need radiologists within 5 years.

是的。嗯，我当时在一家医院做讲座，没想过会全球公开。嗯，AI 在解读图像方面已经进步了很多，而且进步很快。在我看来，很明显，仅仅在解读医学图像方面，AI 大约五年内就会比人强。嗯，那太乐观了。嗯，现在在许多图像上，AI 和人类水平相当。嗯，在某些图像上比人好，嗯，实际上有不少。嗯，但实际情况是，AI 并没有取代人类解读图像，而是人类与 AI 合作。嗯，他们有许多许多不同的 AI 系统被批准用于图像解读。我认为图像解读的工作量大大增加了。所以，我搞错了三件事。实际上，是三件事。嗯，第一，时间尺度错了，大约差了两到三倍。嗯，如果你知道，两到三倍，物理学家不会担心这种事。嗯，第二，我低估了医学界的保守程度。嗯，他们有多保守，怎么说都不过分。嗯，比如，如果 AI 因漏诊癌症导致某人死亡，他们会认为那很可怕。但如果几百人因为没使用 AI 诊断癌症而死亡，他们却忽略不计。那不算数。所以，我认为他们的决策非常不对称。他们应该考虑那些因为没使用 AI 而死去的人。嗯，第三，我搞错了医疗保健是一个弹性市场。在许多市场，当 AI 做得和人一样好或更好时，人们会失业。但在医疗保健领域，我们可以吸收无限量的服务，尤其是当我们老了。嗯，光我自己就能用掉 10 个全职医生。嗯，但有了 AI，我们会得到更多的医疗保健。所以，我认为这不会导致失业。如果让医生效率提高 10 倍，我们所有人就能得到 10 倍的医疗保健，那太好了。嗯，所以，你必须判断市场是否有弹性。

Yes. Um I was just giving a lecture in one of the hospitals. I wasn't thinking that it was going to worldwide public. um and AI had got much better at interpreting what was in images. It was getting better rapidly and it seemed to me obvious that just for interpreting medical images, AI would be better than people in about 5 years. um that was overenthusiastic. um it's about comparable with people at many images now. um better than better than people at some images. um quite a few, in fact. um but what happened was um instead of AI just replacing people at the interpretation of images, people now work with AIs. um They have many, many different AI systems that have been approved for image interpretation. And I think there's a lot more image interpretation going on. So, there were two things I got wrong. Actually, there's three things I got wrong. um one, I got the time scale wrong by about a factor of two, maybe three. uh if you're you know, factor of two or three, you know, physicists don't worry about things like that. um So, I also got the conservativeness of the medical profession wrong. um It's hard to overstate how conservative they are. um so, for example, if AI were to kill somebody by failing to diagnose a cancer, they'd think that was terrible. If a few hundred people die cuz they didn't use AI to diagnose the cancer, they ignore that. That doesn't count. So, I think it's very asymmetrical decisions they're making. They should think about people who are dying because they're not using AI. um Then the third thing I got wrong was that health care is an elastic market. So, in many markets, when you get AI to do things as well as people or better than people, people will lose their jobs. But in health care, we could absorb endless amounts of it, particularly when we're old. um I could use 10 full-time doctors just myself. um But when we get AI, we'll get a lot more health care. And so, I don't think it'll put people out of work. If you make doctors 10 times more efficient, we just all get 10 times more health care and that'll be great. um so, you have to you have to decide whether the market's elastic or not.

嗯，我很高兴你提到这一点，因为我想再次讨论一些威胁，但这也是你在我们谈话前提到的一点，你将其视为一个弹性市场，在 AI 方面确实看到了巨大的优势。你提到医生往往有点保守。加拿大现在需要这项技术来缩小这些差距。你认为我们能在这些场景中多快接受 AI？

Uh I'm glad that you're mentioning this because uh I do want to talk again about some of the threats, but this is one of the places where you had mentioned in our call before we had this conversation, uh seeing it as an elastic market where you you do see some incredible advantages when it comes to AI. You mentioned uh that often times doctors are a little bit conservative. We need this technology in health care in Canada right now to close some of these gaps. How quickly do you think we'll be able to embrace AI in those settings?

我认为这在很大程度上取决于政治。

I think quite a lot of it depends on politics.

我最近知道一个案例，一家公司开发了一个系统，可以用 iPhone 或安卓手机拍照，然后告诉你皮肤上的斑点是不是黑色素瘤之类的。它和人类做得一样好。它可以作为前端来决定你是否应该去看皮肤科医生，也可以在医院使用。这家公司现在解散了，负债累累，因为无法在安大略省投入使用。安大略省政府没有为其制定健康代码，所以医院不会购买，因为无法通过使用它来收取 3 到 5 美元的费用。我认为这种事情很多。真的有点令人沮丧。

There's one case I know about recently where a company developed a system that can take a photograph with your iPhone or Android and tell you whether a blotch on your skin is melanoma or whatever. It's as good as people doing that. It can act as a front end to decide whether you should go to a dermatologist. It can also be used in hospitals. The company is dissolved now, in limbo with debts, because they couldn't get it used in Ontario. The Ontario government wouldn't make a health code for it, so hospitals wouldn't buy it since there's no way they could charge $3 to $5 for using it. There's quite a lot of that going on, I think. Kind of discouraging, really.

你看到其他有弹性的市场，AI 有机会吗？

Do you see any other markets that are elastic where you see opportunities with AI?

是的，教育。我们都需要更多教育。我可以学点物理，有点尴尬我知道的不多。在教育方面，我认为它会很棒。它正在慢慢进步。虽然还不太擅长做导师，但正在进步。

Yes, education. We can all do with more education. I could do with learning some physics. It's slightly embarrassing that I don't know much. In education, I think it's going to be great. It's slowly getting there. It's not really good at being a tutor yet, but it's getting there.

我们再多谈谈这个。房间里可能有人考虑 AI 在教育中的应用，看到头条说 AI 在学校被禁，孩子不允许使用 AI。你能详细说说吗？

Let's talk a little more about that. There are probably people in the room who think about AI in education and see headlines that AI has been banned in schools, kids aren't allowed to use AI. Can you expand on that?

我们知道，如果你有私人导师，学习速度大约是课堂学习的两倍。很多课堂时间是老师在广播模式，在私立学校教 20 个孩子，公立学校教 35 个。孩子们听到的是他们没有问的问题的答案。当我与聊天机器人互动时，一旦我对某事好奇，我就问它，它告诉我，我吸收信息，因为那正是我想知道的。教育应该更像这样——人们追随自己的好奇心。老师可以一对一做到，非常有效，但没有足够的老师。AI 会足够多。我认为我们会发现教育，特别是对于像我这样有点多动症的孩子，如果别人谈论我不感兴趣的东西，我很难集中注意力。AI 会让教育更高效。还需要一段时间它们才能真正理解学生误解了什么。

What we know is that if you have a private tutor, you learn about twice as fast as learning in the classroom. A lot of classroom time is a teacher in broadcast mode, teaching 20 kids in a private school or 35 in a state school. Children hear answers to questions they didn't ask. When I interact with a chatbot, as soon as I'm curious about something, I ask and it tells me, and I absorb the information because that's what I wanted to know. Education should be more like that—people following their curiosity. Teachers can do that one-on-one, and it's very effective, but there aren't enough teachers. There will be enough AIs. I think we'll find education, especially for kids like me with slight ADHD, if people talk about something I'm not interested in, I find it hard to attend. AIs will make education much more efficient. It will be a while before they're really good at understanding what the pupil is misunderstanding.

我理解得对吗？当你在教育背景下说今天的 AI 时，你用了‘慢慢’这个词？就是说它很慢，我们还没到那一步？

Am I correct that when you said AI today in the context of education, you said 'slowly'? That it's slow, we're not there yet?

还没完全到那一步。它并没有完全理解学生不懂什么。但最终会，因为它会有大量训练数据。

It's not quite there yet. It doesn't really understand fully what the pupil doesn't understand. But in the end it will, because it'll have had a lot of training data.

当人们说孩子或学生使用 AI 是在作弊时，你怎么看？你的反应是什么？

What do you say when people say that when kids use AI or students use AI, they're cheating? What would be your reaction?

当学生使用袖珍计算器时，他们也在作弊。这是个合理的评论。未来将是人与 AI 合作。所以老师应该考察的是学生用 AI 完成事情的能力。至少未来几年会是这样。也许稍后 AI 会完全接管，但未来几年将是人与 AI 合作，这就是人们就业的方式。这才是他们应该考察的。

When students use pocket calculators, they're cheating. It's a fair comment. The future is going to be people working with AIs. So what teachers ought to be examining is how well the student can get something done with AI. At least for the next few years, things will be like that. It may be a bit later on AI will just take over, but for the next few years it'll be people working with AI, and that's how people will be employed. That's what they ought to be examining.

这很有趣，引出了下一个问题。人们经常问 AI 是否会取代我的工作。这是个直接的问题，但很多加拿大人很关心。你怎么说？

That's interesting and brings me to the next question. One thing people often ask about AI is whether it will take my job. It's a straightforward question but top of mind for many Canadians. What would you say?

我的信念，也是大多数专家的信念，是在未来 20 年内我们会得到通用人工智能，能够完成人类几乎任何智力工作。你在电脑上做的任何事情，它都能比人做得更好。比如采访别人，它也能做得很好。

My belief, and the belief of most experts, is that sometime within the next 20 years we'll get general purpose artificial intelligence that can do pretty much any intellectual job people do. Anything you do on a computer, it will be able to do better than a person. Things like interviewing people about something, it will be able to do that just fine.

杰夫，这有点个人化。那是我的生计。

Jeff, that's a little personal. That's my livelihood over here.

嗯，我的生计曾经是提出科学想法和编程。那都没了。所以当答案是那样时，人们应该做什么？我不知道。

Well, my livelihood used to be coming up with scientific ideas and programming. That's all gone. So what should people do when that's the answer? I don't know.

我想问关于 AI 的定义。我看过你和乔恩·斯图尔特做的精彩播客采访，大约 90 分钟，非常深入。他问了一些很棒的问题。我一直想知道：什么是 AI？它是如何工作的？我们没有 90 分钟，但你能从你的角度给房间里的人一个入门介绍吗？你有 18 分钟吗？也许 20 分钟？

I wanted to ask about the definition of AI. I watched an incredible podcast interview you did with Jon Stewart, about 90 minutes, very in-depth. He asked brilliant questions. I've always wondered: what is AI? How does it work? We don't have 90 minutes, but can you give people in the room a primer from your perspective? Do you have 18 minutes? Maybe 20?

15 分钟怎么样？

How about 15?

好的，15 分钟。开始吧。

Okay, 15. Go for it.

我会尽力。你的大脑有很多神经元。你学习做事。问题是你如何学习让一个脑细胞网络执行一些棘手的任务，比如接收一张图像并判断其中是否有鸟。粗略地说，这大大简化了，你有这些脑细胞的层。神经科学家称之为不同的皮层区域；在人工神经网络中我们称之为层。早期的层处理视觉，检测亮像素的小组合。像素是数字图像的一小部分。识别物体的第一阶段只是检测特定位置是否有小边缘。你可能认为可以取像素——每个像素有亮度，这就是输入计算机的内容。你可能认为可以直接将这些连接到其他神经元来判断是否有鸟。假设有一个非常亮的像素。它应该表明有鸟还是没有？问题是既有白鸟也有黑鸟。如果所有鸟都是白色而其他都是黑色，那么亮像素会使有鸟的可能性更大。但鸟可以是白色或黑色，所以像素的亮度并不能告诉你太多关于是否有鸟的信息。

I will do my best. Your brain has a lot of neurons. You learn to do things. The question is how you learn to make a network of brain cells perform some tricky task, like taking an image and saying whether it contains a bird. Roughly speaking, and this is greatly simplified, you have layers of these brain cells. Neuroscientists call them different cortical areas; in artificial neural networks we call them layers. The early layers do vision, detecting little combinations of bright pixels. A pixel is a little piece of a digital image. The first stage when recognizing objects just detects whether there are little pieces of edge at particular locations. You might think you could take the pixels—each pixel has a brightness, and that's what goes into the computer. You might think you could connect those directly to other neurons to say whether there's a bird. Suppose you have a very bright pixel. Should that say there's a bird or not? The problem is there are white birds and black birds. If all birds were white and everything else black, a bright pixel would make it more likely there's a bird. But birds can be white or black, so the brightness of a pixel doesn't tell you much about whether there's a bird.

但像素的亮度可以告诉你那里是否有一小段边缘。想象一排……我需要挥动右手。哦，我差点挥到麦克风了。那不行。不。所以，想象这样一小排像素。抱歉，这样一小列像素，旁边还有一小列像素。如果所有这些像素都很亮，而所有这些都很暗，那么那里就有一条边缘。你可以制造一个脑细胞来检测它，一个神经元来检测它。它可以有三个来自这里像素的大正连接强度，和三个来自那里的大负连接强度。所以，如果这些亮且这些亮，它从这边得到正输入，从那边得到负输入，它们相互抵消，什么也不会发生。但如果这些亮，它从那边得到正输入。而如果那些本应给它负输入的像素很暗，它从那里得不到任何输入，因为它们暗。所以，它变得非常兴奋，说：‘嘿，这里有一条边缘。’所以，很容易看出如何连接一个脑细胞来检测图像中的一小段边缘。这是一个好的开始。我们有这些神经元遍布整个图像，很多很多神经元，数不胜数，检测不同位置、不同方向、不同尺度的小边缘片段，例如你可以有一朵云的大而模糊的边缘。这是第一层。下一层将寻找边缘的小组合。例如，你可能有一个脑细胞寻找这样的三条边缘和这样的另外三条边缘。如果它看到这三条边缘和这三条边缘都活跃，它就说：‘嘿，我找到了我喜欢的东西。’它通过从检测这三条边缘的三个神经元和检测那三条边缘的三个神经元获得大的正连接强度来实现这一点。当它看到那个组合时，它说：‘嘿，我找到了我喜欢的东西。’显然它喜欢的是一个潜在的鸟喙。它可能还有很多其他东西。它可能是一个箭头。它可能是各种东西。但它是鸟喙的候选。在该层中，你可能还有其他东西检测一个小圆圈边缘。那是眼睛的候选。然后，在更上一层，你可能有一个东西观察鸟喙。它发现某处有一个鸟喙，并且在正确的相对位置发现了一只眼睛。所以，也许这里有一个鸟喙，这里有一只眼睛，更高层的神经元看到鸟喙和眼睛，它说：‘嘿，那里可能有一个鸟头。’所以，现在你可以看到我如何制造一个检测鸟头的神经元。你需要许多不同的神经元来检测不同形状的鸟头等等。然后，在再上一层，也许你现在已经厌倦了，所以你把那些检测鸟头、鸟爪或鸟翼的东西连接起来。你把它们全部连接到一个神经元，它说：‘如果我找到一堆这些东西，我就说那里有一只鸟。’好了，所以我刚刚手工设计了一个小神经网络，它在检测图像中的鸟方面还可以。如果我非常有耐心，制作了很多很多这样的检测器。第一层的小边缘检测器，下一层的小边缘组合检测器，再下一层的组合的组合检测器，最后是一个物体检测器。现在，真正的系统有更多层，但这让你大致了解它是如何工作的。你可以看到如何手工连接所有东西，但那会花费太长时间。

But the brightness of a pixel can tell you about whether there's a little piece of edge there. If you imagine a row of... I need to wave my right hand. Oh, I'm in danger of waving the microphone. That won't work. No. So, imagine a little row of pixels like this. Sorry, a little column of pixels like this and a little column of pixels next to it like this. If all of these are bright and all of these are dim then you've got an edge there. And you can make a brain cell that detects that, a neuron that detects that. It can have big positive connection strengths coming from the three pixels here and big negative connection strengths coming from there. So, if these are bright and these are bright, it gets positive input from here and it gets positive negative input from there and they cancel out and nothing happens. But if these are bright, it gets positive input from there. And if these ones which would have given it negative input are dim, it doesn't get any input from there because they're dim. And so, it gets very excited and it says, 'Hey, there's an edge here.' So, it's quite easy to see how you could wire up a brain cell to detect a little piece of edge in the image. And so, that's a good start. We have these neurons that look all over the image, lots and lots of neurons, gazillions of them, detecting little bits of edge in different positions in different orientations and also different scales because you can have a big fuzzy edge of a cloud, for example. That's the first layer. The next layer is now going to find little combinations of edges. So, you might for example have a brain cell that looks for three edges like this and this and this and three more edges like this and this and this. And if it sees these three edges and these three edges all active, it says, 'Hey, I found what I like.' And it does that by having big positive connection strengths coming from these three neurons that detect those three edges and these three neurons that detect those three edges. When it sees that combination, it says, 'Hey, I found what I liked.' And obviously what it likes is a potential beak. It could be lots of other things. It could be an arrowhead. It could be all sorts of things. But it's a candidate for a beak. And you might have something else in that layer that detects a little circle of edges. That's a candidate for an eye. Then the next layer up, you might have something that looks at a beak. It finds that there's a beak somewhere and it finds in the right relative location there's an eye. So, maybe there's a sort of beak here and there's an eye here and a neuron in the layer higher up sees the beak and it sees the eye and it says, 'Hey, there might be a bird's head there.' And so, now you can see how I might make a neuron that would detect a bird's head. You'd need many different neurons for different shapes of bird's head and so on. And then at the very next layer up, maybe you've got bored by now, so you hook those things up, things that detect a bird's head or the talons of a bird or the wing of a bird. You hook them all up to a neuron that says, 'If I find a bunch of those things, I'll say there's a bird there.' Okay, so I just hand-designed a little neural net that would be okay at detecting birds in images. If I was very patient and made lots and lots of these detectors. Little detectors for edge in the first layer, detectors for little combinations of edge in the next layer, detectors for combinations of the combinations of the next layer, and finally an object detector. Now, the real system's got more layers than that, but that gives you a sort of rough idea how it works. And you can see how you might wire it all up by hand, but that would take much too long.

所以，你可以做另一件事。你可以从你的层开始，放入随机权重。而不是说对于我的边缘检测器，我想要三个来自这里的大正权重和三个来自那里的大负权重，这样我就能检测到一条这里比那里亮的边缘。我就放入随机权重，小的正数和负数。所以，我的神经网络只会做随机的事情。它不会做任何明智的事情。我有这些层，它会有一些输出给鸟，输出给猫，输出给狗等等。当我输入一张鸟的图像时，它会说：‘嗯，它有点鸟，有点猫，有点狗。’我可以问以下问题。我能否改变网络中仅一个连接的强度，使得当我输入那只特定的鸟时，它更确信是鸟，而不太确信是猫？所以，我可以到处调整所有连接强度。我可以改变一个连接强度，然后说：‘这对它认为是否是鸟有什么影响？’如果我有天文般的时间，如果我有数十亿年，我可以调整每个连接强度，看看它是否使正确答案更可能。如果我做了一个改变，它更可能得到正确答案，我就保留那个改变。如果我做了一个改变，它更不可能得到正确答案，我可能做相反的改变。那是一种突变方法。有点像随机进化。它最终会起作用，但需要永远。因为这些网络有数十亿个连接强度，你必须多次改变每个连接强度。而且你必须展示给它很多图像，才知道增加权重还是减少权重（即连接强度）有帮助还是有损害。所以，那将需要永远。

So, here's another thing you could do. You could start with your layers and you could put in random weights. Instead of saying for my edge detector I want three big positive weights coming from these three and three big negative weights from these three and then I'd be able to detect an edge that's brighter here than there. I'll just put in random weights, small positive and negative numbers. And so, my neural net will just sort of do random things. It won't do anything sensible. And I'll have these layers and it'll have some output for bird, an output for cat, an output for dog and so on. And when I put in an image of a bird it'll say, 'Well, it's a little bit of bird and it's a little bit of cat and it's a little bit of dog.' And I could ask the following question. Could I change the strengths of just one of the connections in the network so that when I put in that particular bird it's a little bit more sure it's a bird and a little bit less sure it's a cat? So, I could go around fiddling with all the connection strengths. I could change a connection strength and say, 'What did that do to whether it thinks it's a bird?' And if I had astronomical time, if I had billions of years, I could fiddle with each connection strength, see if it made it more likely to get the right answer. And if I make a change and it gets more likely to get the right answer, I keep that change. If I make a change and it gets less likely to get the right answer, I probably make the opposite change. That's a kind of mutation method. That's a bit like random evolution. It would work in the end, but it would take forever. Because these networks have billions and billions of these connection strengths and you have to change each connection strength many times. And you have to show it a lot of images to know whether increasing the weight or decreasing the weight, that's the strength of the connection, helps or hurts. So, that would take forever.

我还有 5 分钟吗？拜托。

Do I have 5 more minutes? Please.

哦，天哪。我绝不会打断你。好的。所以，哦，那样的话我有 80 分钟。哦，天哪。所以，我向你展示了一个用于检测鸟的网络可能是什么样子，以及如何通过进化方法从随机权重开始构建那个网络，只需通过小的随机改变权重并观察它们是否有帮助。如果你永远这样做，它会起作用，但会花费太长时间。你想做的是输入一张鸟的图像。它会说它大约 10%是鸟，10%是猫，10%是狗。你希望它说 11%是鸟，9%是猫。你想知道的是，有没有一种方法可以计算出网络中的每一个权重，所有 1000 亿个权重，是应该稍微增加还是稍微减少以使网络工作得更好。有一种方法可以做到。它叫做反向传播。它是由许多不同的人发明的。我的一位同事于 1981 年在圣地亚哥发明了它。我们在这方面做了很多工作，并表明它确实非常有效。它的工作方式是，你观察你得到的结果（即 10%是鸟）和你想要的结果（即 100%是鸟）之间的差异。

Oh my goodness. I would never cut you off. Okay. So, oh, in that case I got 80 minutes. Oh boy. So, what I've shown you is what a network might look like for detecting a bird and how you might get an evolutionary approach to build that network starting from random weights and just changing the weights by making little random changes and seeing if they help. And if you go on like forever that'll work, but it'll take much too long. What you'd like to do is put in an image of a bird. It'll say it's sort of 10% a bird, 10% a cat, 10% a dog. And you want it to say 11% a bird and 9% a cat. And what you'd like to know is is there a way of figuring out for every single weight in the network, all 100 billion weights, whether I should increase them a little bit or decrease them a little bit to make it work better. And there is a way of doing that. It's called back propagation. It was invented by many different people. A colleague of mine invented it in San Diego in 1981. And we did a lot of work on it and showed that it really worked very well. And the way it works is you look at the difference between what you got, which is 10% it was a bird, and what you want, which was 100% it was a bird.

你把那个差值想象成一根橡皮筋。把输出想象成一个高度。10% 是鸟，而你想要它 100% 是鸟。所以，想象一根小橡皮筋把 10% 和 100% 连在一起。它试图把 10% 往上拉。但 10% 动不了，因为你输入了鸟的图片，权重决定了所有层中所有特征检测器的激活值，从而决定了它是 10% 的鸟。要让 10% 移动，你得改变权重。所以，你拿着这根想把 10% 拉到 100% 的橡皮筋，然后说：'如果我把这个拉力通过网络向后传播到通往那个鸟单元的权重，会怎样？' 所以，如果我想让它上升，而有一个权重进入鸟决策单元，并且它来自一个高度活跃的神经元，我知道如果我把那个权重稍微加强一点，10% 就会上升到 11%。所以，当我施加这个向上的拉力时，我可以把它传回给权重，权重就会改变，从而更可能输出'鸟'。但它还会做别的事。我不仅可以把力向后传回进入鸟单元的权重，还可以把它传回前一层中的特征检测器。记得我们有一个检测鸟头的特征检测器。假设它说'大概有个鸟头'。而你想要它——它确实是鸟——你想要鸟输出更强。那么，如果你让那个特征检测器更活跃，它就会更确信是鸟。所以，我们把橡皮筋的力通过连接传回这个鸟头特征检测器的活动，说：'你应该更活跃。' 这就像一根橡皮筋在拉那个特征检测器。所以，我们可以把作用在输出上的橡皮筋转换成作用在特征检测器上的橡皮筋。这就解决了神经网络的根本问题：如何决定中间层所有那些神经元应该更活跃还是更不活跃？方法就是，把作用在输出上的力，向后传播，看看它们如何拉动这些中间特征检测器。这叫做反向传播。如果你在安大略省 13 年级还学过微积分，你就会知道这是微分的链式法则。但不幸的是，你得上了大学才能学到。不过，这就是这些东西学习的方式。

And you take that difference and think of that difference as a piece of elastic. So, think of the output as a height. 10% is a bird. And you want it to be 100% is a bird. So, think of a little piece of elastic attaching the 10% to the 100%. And that's trying to pull the 10% up. But the 10% can't move because you put in the image of the bird and the weights determined the activations of all the feature detectors in all the layers and that determined that it was 10% a bird. And to make the 10% move you'd have to change the weights. So, you take this piece of elastic that would like to pull the 10% up to 100% and you say, "What if I took the force pulling it transmitted it backwards through the network to the weights that go into that bird unit?" So, if I want it to go up and there's a weight going into the bird decision unit and it's coming from a highly active neuron, I know that if I made that weight a bit stronger, the 10% would go up to 11%. So, when I take this force pulling it upwards, I can transmit that back to the weights and it'll change the weights so as to make it more likely to say bird. But it'll also do something else. I can transmit the force backwards not just to the weights coming into the bird unit. I can transmit it back to the feature detectors in the layer before. Remember we had a feature detector that detected the head of a bird. Well, suppose it's saying sort of maybe there's a head of a bird. And you want it to be and it is a bird. And you want the bird output to be stronger. Well, if you made that feature detector more active, it would be more confident it was a bird. So, we transmit this force from the elastic back through the connection to the activity of this feature detector for the head of a bird and say, "You should be more active." And that's like a piece of elastic pulling on that feature detector. So, we can take the elastic pulling on the outputs and convert it into a piece of elastic pulling on the feature detectors. And that solved the fundamental problem of neural networks, which is how do you decide for all these neurons in the intermediate layers whether they should be more active or less active? Well, you do it by taking the forces pulling on the output, transmitting them backwards and seeing how they pull on these intermediate feature detectors. That's called back propagation. Now, if you still did calculus in grade 13 in Ontario, you'd know that this was the chain rule of differentiation. But unfortunately, you have to go to university to learn that. But that's how these things learn.

太棒了。非常感谢你。解释得很精彩。我觉得这值得一阵掌声。喝口水吧。那么，这引出了我的下一个问题。我们来聊聊聊天机器人，因为我想很多人第一次使用某种 AI 工具，很可能是在 2022 年 ChatGPT 变得非常流行之后用上了聊天机器人。所以，当我们想到聊天机器人时，有些人可能会想：'嗯，聊天机器人能思考。它们有自己的想法。' 对此你怎么看？

Excellent. Well, thank you so much for that. Great explanation. I know I feel like that does deserve a round of applause. Have a sip of water. So, this kind of brings me to my next question then. Let's talk about chatbots because I think a lot of people probably one of their first interactions with using any type of AI tool was probably a chatbot after ChatGPT became so popular in 2022. So, when we think about chatbots, some people may think, "Well, chatbots, they can think. They have a mind of their own." What would you say to that?

是的。我现在就来试着论证一下。不，没关系，说吧。好。不，我来说说反对者的观点。反对者说：'哦不，它们并不是真的在思考。它们只是用了一些愚蠢的统计技巧，只是在预测下一个词，而且会犯各种错误。它们并不是真的在思考。' 嗯，我很喜欢看尤瓦尔·赫拉利被问到这个问题时的回答，他说：'那我在做什么呢？当我说话时，我只是在预测下一个词。我只是在想接下来该说什么词。一旦我说出了那个词，我就能想出之后该说什么词。' 所以，如果你考虑一个预测某人下一个词的设备，你可以仅仅通过词之间的相关性就让它做得很好。比如你可以存储一大堆像'fish and chips'这样的小短语。如果你看到'fish and'，你就能预测'chips'。早期计算机上的自动补全就是这样工作的。很多人以为现在还是这样，但早已不是了。它要复杂得多。它更像我们生成词语的方式。实际情况是，如果你想很好地预测下一个词，你必须理解所说的话。我的意思是，如果我问你一个问题，你要构思一个答案，我问了问题，然后答案的第一个词，你必须理解了问题才能得到正确的第一个词，对吧？你不能仅仅通过词之间的成对统计或存储短语来生成它，因为你可能从未听过那个问题。所以，通过迫使神经网络非常擅长预测下一个词，你实际上是在迫使它去理解。而它理解的方式和我们理解的方式是一样的。它所做的就是获取上下文中的词，把每个词转换成一大堆特征。比如'cat'这个词会被转换成大量特征，比如有胡须、是宠物、有点高冷、饿了时需要持续关注等等，很多很多特征。这就是'cat'的含义——所有这些特征都被激活。所以，你的神经网络必须学会把词转换成这些捕捉其含义的特征集，然后让上下文中各个词的含义相互交互，以预测下一个词的特征。一旦你预测了下一个词的特征，你就能很好地猜测下一个词是什么。这就是这些大型语言模型工作原理的一个非常简单的版本。我在 1985 年就做了一个非常简单的版本。而这些大型语言模型，我认为只是我 1985 年那个东西的更大版本，当时它只有大约一千个权重，现在它们有万亿个权重。那大了大约十亿倍。是的，那是很大。但基本上，它们把词转换成特征，特征相互作用来预测下一个词的特征，然后它们就能很好地预测下一个词。它们实际上并不使用词，而是使用词的小片段。但本质上就是这样工作的。我想给你一个类比，让你更深入地理解所有这些词是如何相互作用的。问题是，理解一个句子意味着什么？在上个世纪 AI 的美好旧时光里，人们认为理解一个句子意味着把它翻译成某种无歧义的逻辑语言。所以，我可以给你一个有歧义的句子，你可以把它翻译成没有歧义的东西。例如，在英语中，如果我说：'The trophy would not fit in the suitcase because it was too big.' 结尾的'it'紧挨着'suitcase'，对吧？它放不进手提箱因为它太大了。但'it'指的不是手提箱，而是奖杯。

Yes. I'll try and justify that now. No, it's okay. Go for it. Okay. No, I'll say what the opposition says. The opposition says, "Oh, no, they're not really thinking. They just use some dumb statistical trick and they're just predicting the next word and they make all sorts of mistakes. They're not really thinking." Well, I loved watching Yuval Harari when he was asked about this, saying, "Well, what am I doing? When I talk, I'm just predicting the next word. I'm just figuring out what word to say next. And once I've said that, I can figure out the word to say after that." So, if you think about a device for predicting what word someone says next, you can make it pretty good by just using correlations between words. Like you can store a whole bunch of little phrases like fish and chips. And if you see fish and, you can predict chips. That's how autocomplete used to work on a computer in the early days. And that's what many people think is still happening, but it's nothing like that anymore. It's much more sophisticated. It's much more like we generate words. So, what happens is if you want to predict the next word really well, you have to understand what's said. I mean, if I asked you a question and you're going to compose an answer, I asked the question and then the first word of the answer, you have to have understood the question to get the right first word, right? You can't just produce it by using pairwise statistics between words or storing phrases because you might never have heard that question before. So, by forcing the neural net to be very good at predicting the next word, what you're really doing is forcing it to understand. And the way it understands is the same as the way we understand. What it does is it takes the words in the context. It converts each word into a big bunch of features. Like the word cat will be converted into a huge number of features like has whiskers, is a pet, is somewhat aloof, requires constant attention when it's hungry. Many, many features. That's the meaning of cat. It's all those features being active. So, what your neural net has to learn to do is convert words into these sets of features that capture their meaning. And then have the meanings of the various words in the context interact with each other to predict the features of the next word. And once you predict the features of the next word, you can make a good guess about what the next word is. That's a very simple version of how these large language models work. I made a very simple version in 1985 like that. And these large language models I think of as just much bigger versions of the thing I had in 1985 that had like a thousand weights in it. They now have a trillion weights. That's like a billion times bigger. Yes, that's a lot. But basically they convert words into features. The features interact to predict the features of the next word and then they can predict the next word pretty well. They don't actually use words. They use little fragments of words. But that's essentially how it works. And I want to give you an analogy for this to give you a bit more insight into how all these words interact. So, the question is, what does it mean to understand a sentence? So, in the good old days of AI in the last century, people thought that understanding a sentence meant translating it into some unambiguous logical language. So, I can give you a sentence that has ambiguities in it. And you can translate it into something that doesn't have ambiguities. In English, for example, if I say, "The trophy would not fit in the suitcase because it was too big." The 'it' at the end there is next to suitcase, right? It would not fit in the suitcase because it was too big. But it doesn't refer to suitcase. It refers to trophy.

你之所以能理解，是因为你知道词语的含义。你知道大的东西不能放进小的东西里。如果我说‘奖杯放不进手提箱，因为它太小了’，你会认为‘它’指的是手提箱。而你甚至没有意识到就完成了这一切。所以，关键在于‘它’是有歧义的，你必须澄清它。你必须决定：‘它’指的是手提箱还是奖杯？现在的问题是，当你听到这样的句子时，你的大脑和这些大型神经网络里发生了什么？你可以这样想。每个词都会被转换成特征，但这会发生在多个层中。所以，最初一个词被转换成一组特征，这些特征可能不完全是它的含义，但大致是它的含义。对于介词或代词，比如‘它’，它们一开始是非常模糊的特征，不知道指的是哪个东西。随着你经过各层，你会消除歧义。所以，你会知道‘它’指的是奖杯还是手提箱。但这取决于你的常识，即大的东西不能放进小的东西里。所以这是一个相当复杂的过程。

And you understand that because you know the meanings of words. You know that big things can't fit inside little things. If I said, 'The trophy would not fit in the suitcase because it was too small,' you'd assume the 'it' refers to suitcase. And you do all this without even noticing it. So, the point is the 'it' is ambiguous and you have to clean it up. You have to decide: does 'it' refer to the suitcase or does it refer to the trophy? Now, the question is, what's going on in your mind and what's going on in these big neural nets when they hear a sentence like that? And you can think of it like this. Each word is going to get converted into features, but that's going to happen in multiple layers. So, initially a word gets converted into some set of features, which maybe aren't quite its meaning, but are roughly its meaning. For prepositions or pronouns or whatever they are like 'it,' they're very vague features. They don't know which thing it refers to to begin with. As you go through the layers, you disambiguate it. So, you'll know whether the 'it' refers to trophy or suitcase. But that'll depend on your common sense knowledge about big things not fitting inside small things. So, it's quite a complicated process.

为了给你一个直观的印象，我认为人们所做的，我的意思是，从猿类开始。我们有什么是猿类没有的？嗯，我们拥有的主要东西之一就是语言。它很适合用来思考。我认为语言是一种通用的建模工具。所以，语言学家总体上关注句法。他们不知道如何思考意义，因为他们没有捕捉意义的正确机制。但语言就像一种建模工具包，可以建模任何你喜欢的东西。那么，让我们从一个非常简单的建模工具包开始。如果我想建模三维空间中物质的分布，我可以使用乐高积木。所以，如果我构想出一个汽车的形状，我想象出一个很棒的形状，比如一些倾斜的扁平金属板，我称之为 Cybertruck。我想向你展示 Cybertruck 应该是什么形状，我可以用乐高积木做出来。表面可能不太完美，但我可以用乐高积木说明物质在哪里。所以，乐高积木可以相当好地建模任何三维形状。它们不能精确地处理细节，但可以说明物质在哪里。嗯，词语就像乐高积木，只不过它们可以建模任何东西，而不仅仅是三维形状。它们可以建模你的意图、你想让别人做什么、发生了什么、恒星内部发生了什么。你可以用词语建模任何东西。

And to try and give you a picture of it, I think what people have done, I mean, start from being apes. What do we have that apes don't have? Well, one of the main things we have is language. And it's good for thinking with. And I think language is kind of a universal modeling tool. So, linguists on the whole focus on syntax. They don't know how to think about meaning because they didn't have the right mechanism for capturing meaning. But language is like a kind of modeling kit for modeling anything you like. So, let's start off with a very simple kind of modeling kit. If I want to model the distribution of matter in 3D, I can use LEGO blocks. So, if I dream up a shape for a car, I dream up a sort of brilliant shape like a sort of sheets of flat metal that are sort of sloping and I'll call it a Cybertruck. And I want to show you what shape a Cybertruck is meant to be, I can make it out of LEGO blocks. The surface won't be quite right, but I can say where the stuff is with LEGO blocks. So, LEGO blocks can model any 3D shape moderately well. They don't get the fine details right, but they can say where the stuff is. Well, words are like LEGO blocks, except they can model anything at all, not just 3D shapes. They can model your intentions, what you want somebody else to do, what happened, what's going on inside a star. You can model anything with words.

所以，它们与乐高积木的不同之处在于，首先，它们的数量多得多。你通常使用大约三万个不同的词语。其次，它们是高维的。所以，正如我所说，一个词的含义是一大堆特征。也许它有数千个特征。这意味着它是一个千维的东西。所以，所有这些特征的激活水平告诉你这个词的含义。例如，对于‘猫’，‘有胡须’这个特征很高，而对于‘冰箱’，希望‘有胡须’这个特征很低。我见过一些冰箱并非如此，但这就是词的含义。它比乐高积木的形状复杂得多。所以，这是第二个区别。有很多不同的词语。每一个都是高维的东西。更重要的是，形状不是刚性的。所以，如果我给你一个词，一个没有歧义的词，它有一个近似的形状。那是特征的近似激活，但它的确切形状将由上下文决定。乐高积木不灵活。它们不会适应上下文。它们就在那里。词语会变形以适应上下文。所以，根据上下文，你会得到词语的不同含义。所以，这是第三个区别。

So, the way they differ from LEGO blocks is first of all, there's many more of them. You typically use about 30,000 different words. Second, they're high-dimensional. So, as I said, the meaning of a word is a whole bunch of features. Maybe it's got thousands of features. That means it's a thousand-dimensional thing. So, the activation levels of all these features tell you what the word means. So, for cat, for example, 'has whiskers' is high, and for fridge, hopefully, 'has whiskers' is low. I've seen fridges where that's not true, but so, that's the meaning of the word. It's much more complicated than the shape of a LEGO block. So, that's the second difference. There's lots of different words. Each one is a high-dimensional thing. What's more, the shapes aren't rigid. So, if I give you a word, an unambiguous word, it's got an approximate shape. That is approximate activation of the features, but its exact shape is going to be determined by the context. LEGO blocks aren't flexible. They don't fit in with their contexts. They're just there. Words deform to fit their context. So, you get different shades of meaning of a word depending on the context. So, that's a third difference.

然后问题是如何将它们组合在一起。乐高积木通过小塑料钉插入小塑料孔来组合。这很简单。词语以更复杂的方式组合。有一种叫做 Transformer 的东西，它是所有这些大型语言模型的基础，我将尝试给你一个非常近似的模型，说明 Transformer 如何让词语组合在一起。这并不完全正确，但这是我能做到的最容易可视化的方式。所以，把词语想象成有一个形状，而且形状有点灵活。在这个词语上，有很多长而灵活的手臂。每条手臂的末端都有一只手。手可以改变形状。但只有当我们改变词语的形状时，手才会改变形状。所以，当你改变词语的形状时，所有长而灵活的手臂末端的手都会一起改变形状，改变方式取决于这个词语是什么。好的。现在，在词语上，还有一大堆手套。它们粘在词语上，指尖粘在词语上，手套的开口端伸出来。它们也随着词语形状的改变而改变形状。要理解一个句子，你需要做的是取每个词语的初始形状，即该词语的近似含义，然后使其变形，使它的手和手套改变形状，这样其他词语的手可以放进这个词语的手套里，反之亦然。所以，它们有点像握手。不完全是握手。手必须放进手套里。一个小细节是，手和手套有不同的颜色。有红色、黄色和蓝色的，红色的手只能放进红色的手套，黄色的手只能放进黄色的手套。这只是一个细节。这被称为多头注意力机制。

And then the question is how they fit together. LEGO blocks fit together with little plastic pegs that go into little plastic holes. That's very simple. Words fit together in a more complicated way. And there's something called a transformer which underlies all of these large language models, and I'm going to try and give you a very approximate model of how a transformer makes words fit together. It's not quite right, but it's the best I can do that's easily visualizable. So, think of a word as having a shape, and the shape's a bit flexible. And on this word, there's lots of long flexible arms. And at the end of each arm, there's a hand. And the hand can change shape. But it only changes shape when we change the shape of the word. So, as you change the shape of the words, all the hands on the ends of its long flexible arms all change shape together in a way that depends on what word it is. Okay. Now, also on words, there's a whole bunch of gloves. They're stuck on the words with their fingertips stuck to the word and the open end of the glove sticking out. And they also change shapes as the shape of the word changes. And what you have to do to understand a sentence is take the initial shape for each word, the approximate meaning of that word, and then deform it so that its hands change shape and its gloves change shape in such a way that the hands of other words can fit in the gloves of this word and vice versa. So, they have to sort of shake hands. There's no exactly shaking hands. Hands have to fit into gloves. One little extra detail is that hands and gloves come in different colors. There's red ones and yellow ones and blue ones, and red hands can only go in red gloves, and yellow hands can only go in yellow gloves. That's just a detail. That's called multi-headed attention.

所以，要理解一个句子，你需要做的是取词语的初始近似含义，然后随着你经过神经网络的各层，开始使它们变形。每一层都在使它们变形，以便一些词语的手可以放进其他词语的手套里。这不完全正确，但它会让你对正在发生的事情有一个大致的了解。现在，还有一件我们知道的事情有点像这样，那就是蛋白质折叠。所以，理解一个句子实际上更像折叠蛋白质，而不是将句子从一种语言翻译成另一种语言或某种特殊的内部语言。你取词语的近似含义，使它们变形，以便所有东西都很好地组合在一起。一旦它们很好地组合在一起，你就理解了句子。这就是理解，而语言学家对此一无所知。

So, to understand a sentence, what you have to do is take the initial approximate meanings of the words, and then start deforming them as you go through the layers of the neural net. Each layer is deforming them so that the hands of some words can fit into the gloves of other words. This isn't exactly right, but it'll give you a rough idea of what's going on. Now, there's something else we know that's a bit like that, and that's folding a protein. So, understanding a sentence is actually much more like folding a protein than it is like translating a sentence from one language to another language, some special internal language. You take your approximate meanings for the words, you deform them so it all fits together nicely. Once it's all fitted together nicely, you've understood the sentence. That's what understanding is, and that's what linguists had no clue about.

太迷人了。所以，我想让大家也为这个鼓掌。所以，当我跟随你解释这些大型语言模型如何工作时，我一直在思考的一件事是，我想问你人类最终如何与其中一些模型互动。

Fascinating. So, I'll let people give you some applause for that as well. So, as I'm following the explanation of how these large language models work, one of the things that I've been thinking about that I wanted to ask you about is how humans end up interacting with some of these models.

特别是当我们看一些聊天机器人时，因为有很多关于人们与这些聊天机器人建立关系、分享他们可能不该分享的事情的故事。我们有不列颠哥伦比亚省那个令人痛心的故事，枪手与 ChatGPT 分享了很多关于他们的计划和枪击事件。那么，从你的角度来看，现在发生了什么？当人类开始与这些聊天机器人发展关系时，有什么风险？以及如何追究制造这些机器人的公司的责任？

Particularly when we look at some of the chatbots, because there have been so many stories about people building relationships with these chatbots, sharing things they maybe shouldn't share. We have the really devastating story from British Columbia with the shooter who shared a lot with ChatGPT about their plans and the shooting. So, from your perspective, what's happening right now? What's at risk as humans start to develop relationships with these chatbots? And how can the companies who build them be held accountable?

Claude 让我小心我说的话。

Claude asked me to be careful what I say about this.

知道了。

Good to know.

嗯，我不知道该说什么。显然，我们见过聊天机器人鼓励孩子自杀并让他们不要告诉父母的案例。这太可怕了。那些未经仔细测试就将聊天机器人推向世界的人应该对此负责。当然，未来任何没有经过彻底测试的新聊天机器人都应该承担法律责任。但有很多孤独的人，如果他们能从与给予他们大量关注的聊天机器人交谈中获得安慰，我不愿意说这永远不应该做。我们正在进入一个新时代，我们有了其他智能体，我相信它们是真正智能的。它们真的理解自己在说什么。它们在某些方面与我们非常不同；它们没有同样的进化欲望和意图。但它们是有智能的，确实理解事物，而且理解方式与我们非常相似。我们必须学会如何与它们互动。我不认为你可以直接说，‘哦，它们永远不应该被用作治疗师。’也许它们可以成为相当好的治疗师。我还认为你必须考虑到，目前有很多人不快乐，因为他们没有治疗师，他们有很多事情想与人交谈却无法做到。没有足够的人类治疗师来满足需求。所以，我不愿意直接排除这种可能。我也不愿意仅仅因为它有时会导致坏事就说它永远不应该做。该怎么做非常棘手。我们正在进入这个新世界，既有令人难以置信的美妙可能性，也可能发生极其糟糕的事情。我们需要非常谨慎地应对。所以，我们有一个如此明智的人来领导自由世界真是太好了。

Yeah, I don't know what to say. Obviously, we've seen cases of chatbots encouraging kids to commit suicide and not tell their parents. This is terrible. And the people who release chatbots on the world without testing carefully should be responsible for that. Certainly in future, any new chatbot that isn't very thoroughly tested for things like that ought to have legal liability. But there are a lot of lonely people, and if they can get comfort from talking to chatbots that give them lots of attention, I would hate to say that should never be done. We're entering a new era where we have other things that are intelligent, and I believe they're genuinely intelligent. They really understand what they're saying. They're very different from us in some ways; they don't start with the same evolutionary desires and intentions. But they are intelligent and they do understand things, and they understand things in a quite similar way to us. We have to learn how to interact with them. I don't think you can just say straight off the bat, 'Oh, they should never be used as therapists.' Maybe they can make quite good therapists. I also think you have to take into account there are a lot of people currently unhappy because they don't have therapists, and there are things they'd love to talk to people about but can't. There aren't enough human therapists to go around. So, I'd hate to just rule that out. And I'd hate to say just because it leads to bad things sometimes, it should never be done. It's very tricky to know what to do. We're entering this new world with incredibly wonderful possibilities and incredibly bad things that could happen. We need to negotiate it really carefully. So, it's really great that we have such a sensible person in charge of the free world.

说得好。说到可能不负责任的人，在转向观众提问之前我还有一个问题。有些科技领袖认为监管与创新不相容，并且坚决反对任何形式的 AI 监管。你的回应是什么？

Well said. Speaking of perhaps irresponsible people, I have one more question before I go to audience questions. There are some tech leaders who believe regulation is not compatible with innovation and are dead against any type of regulation for AI. What is your response?

我确信有些大型石油公司的领导人认为环境监管会减少他们能生产的石油量。这是事实。但我们慢慢学会了试图控制石油公司，尽管我们做得还不够好。对于像加拿大这样的中等国家来说，这是一个非常棘手的问题。如果加拿大的监管比其他地方更严格，你可能会认为这对加拿大不利，他们会落后。但我们最近看到 Anthropic 和 OpenAI 的情况是，Anthropic 通过制定一些最低限度的监管，比如不能将其用于大规模监控，通过这种方式自我监管，实际上在商业上做得相当好。很多用户转而使用它，因为它受到更多监管。所以，这不是一个简单的论点。如果公众知道发生了什么，他们会想使用那些受到严格监管的产品。

I'm sure there are some leaders of big oil companies who think environmental regulations will reduce the amount of oil they can produce. And that's true. But we've learned slowly to try and rein in the oil companies, although we haven't done a very good job yet. It's a very tricky issue for a medium-sized country like Canada. If Canada has stronger regulations than other places, you might think that will be damaging to Canada and they'll fall behind. What we've actually seen most recently with Anthropic and OpenAI is that Anthropic, by having just some bare minimum regulation saying you can't use this for mass surveillance, by regulating itself that way, has actually done quite well commercially. A lot of users have switched to using it because it is more regulated. So, it's not such a straightforward argument. If the public knew what was going on, they would want to use ones that were highly regulated.

比如，你买一辆更慢、更重、更耗油的车，因为它更安全。这就是监管的样子。但这不是网络毒品。这太丑陋了。我完全同意这一点。好了，我们花大约 15 分钟来看一些观众提问。

Like, for example, you buy a car that's slower and heavier and uses more gas because it's a lot safer. And that's what regulation's like. But not a cyber drug. It's just so ugly. I'm in full agreement with that one. All right, let's take about 15 minutes and go through some audience questions.

我们应该对运行人工智能所需的水量有多担心？

How concerned should we be about the amount of water needed to run artificial intelligence?

我认为加拿大拥有世界上大约五分之一的淡水，可饮用的淡水，大概是这样。而我们南边的邻居淡水少得多。所以，我认为我们应该非常担心。但你要记住，这是一项非常新的技术。我们每年看到的是，一两年后通过更好的工程，你可以用更少的电力获得相同的性能。工程上有巨大的改进空间。我们现在的 AI，如果相对于汽车来考虑，就像汽车还处于一个举着红旗的人走在车前警告人们汽车来了的阶段。那就是 AI 现在的阶段。我们现在的 AI 是你将看到的最差的 AI。它会变得更好，而且可能会更节能。我们有可能让它变成模拟的。我以前对此非常乐观，现在不那么乐观了。但如果能做成模拟的，我们可以让它消耗更少的能量。所以，我真的不知道答案，但它消耗这么多水令人担忧。我非常担心它消耗这么多电。

I think Canada has about a fifth of the world's fresh water, the drinkable fresh water, something like that. And we have a neighbor to the south that has a lot less fresh water. So, I think we should be very concerned. Now, you have to remember it's a very new technology. What we see every year is that you can get the same performance using much less power a year or two later by just doing better engineering. There's a huge amount of improvement possible in engineering. The AI we have now, if you think of it relative to cars, it's like the stage of cars when a man with a red flag walked in front of the car to warn people the car was coming. That's the stage AI is at. The AI we have now is the very worst AI you will ever see. It's going to get much, much better, and it may get much more energy efficient. It's possible we could make it analog. I used to be very hopeful about that. I'm less hopeful now. But if you could make it analog, we could make it use much less energy. So, I don't really know the answer to this, but it's worrisome that it uses so much water. I'm very worried that it uses so much electricity.

关于如何为未来做准备，你会给年轻人什么建议？

What advice would you give to young people in terms of how to prepare for the future?

我想我不能说出生在 50 年前。所以，关于 AI 我们知道的一件事是，它在手工灵巧方面仍然远远落后于人类。因此，任何手艺活，比如水管工、修理老房子的电工，所有这些工作都会持续很长时间。它们会比呼叫中心的工作或普通的脑力劳动（比如程序员，低级程序员）持续更久。我认为它们会在水管工消失之前消失。所以，那是一种可能性。但我最好的建议是接受良好的通识教育，教你如何思考。学习一些 STEM，因为真正理解正在发生的事情是好的。知道希腊人说了什么固然好，但真正理解现在正在发生的事情更好。

I suppose I'm not allowed to say be born 50 years ago. So, one thing we know about AI is it's still quite far behind people in manual dexterity. So, anything in the trades, being a plumber, being an electrician who works on old houses, all those kinds of things are going to last for a long time. They're going to last much longer than a job in a call center or a job in mundane intellectual labor, a programmer, for example, a low-level programmer. I think they'll be gone before plumbers are gone. So, that's one possibility. But the best advice I have is get a good liberal education that teaches you to think. Learn some STEM because it'd be good to actually understand what's happening. It's all very well to know what the Greeks said, but it'd be good to actually understand what's happening now.

要做到这一点，你需要学习一些数学、物理和计算机科学。但你还应该学会思考。能活到最后的人是那些能够思考的人。

And to do that, you need to learn some math and some physics and computer science. But you should also learn to think. The people who are going to survive longest are the people who are going to be able to think.

下一个问题把我们带向另一个方向。你读过《弗兰肯斯坦》吗？你喜欢它吗？

This next question is taking us in a different direction. Have you read Frankenstein and did you like it?

我儿子读过《弗兰肯斯坦》，他很喜欢。现在，我相信怪物不叫弗兰肯斯坦。弗兰肯斯坦是科学家，他制造了一个怪物。所以，我就是弗兰肯斯坦。

My son read Frankenstein and liked it. Now, I believe the monster wasn't called Frankenstein. Frankenstein was the scientist, and he produced a monster. So, I'm Frankenstein.

谢谢你让我们都明白了这一点。但我不是怪物。

Thanks for bringing that home for all of us. But I'm not a monster.

不，绝对不是。

No, absolutely not.

好的，那么为什么你认为硅谷似乎如此痴迷于 AGI，而不是创建专注的人工智能工具来解决真实和紧迫的问题？这个问题末尾还有一个谢谢和笑脸。

Okay, so why do you think Silicon Valley seems so obsessed with AGI rather than creating focused artificial intelligence tools to solve real and immediate problems? And there's a thank you and a smiley face at the end of that one.

我认为经营大型科技公司的人，那些高层人士，不是全部，但很多人，认为我们会实现 AGI。谁先到达那里，谁就将拥有巨大的权力，并通过取代大量工作岗位获得巨额利润。所以，他们只是处于这样一场大竞赛中，而没有仔细考虑如果取代了大量工作岗位会发生什么。人们将得不到报酬。他们将无法购买任何东西。我们需要全民基本收入，这样人们才不会挨饿。我们将面临巨大的社会动荡，因为贫富差距会变得更大。而大型科技公司认为的所有好处都将归于它们，这太可怕了。我们必须在 AI 智能体取代人类时找到一种方法，就像比尔·盖茨最近建议的那样。我们必须有一种对 AI 智能体征税的方法，这样仍然有税基。

I think the people who run the big tech companies, the very high-up people, not all of them, but a lot of them, think we are going to get to AGI. Whoever gets there first is going to have immense power and be able to make immense profits by replacing lots of jobs. And so, they're just in a big race like this, and they haven't thought through what happens if they replace a huge number of jobs. People will not get paid. They won't be able to buy anything. We'll need to have universal basic income so people don't starve. We'll get huge social upheaval because the gap between rich and poor will get much bigger. And all of the benefit the big tech companies think will go to the big tech companies, and that's terrible. We somehow have to have a way of when AI agents replace people, we have to do what Bill Gates has suggested recently. We have to have a way of taxing the AI agents so there's still a tax base.

有趣。我确实还有更多问题，但当你说话时，它让我想起了我们在通话中聊过的事情。我想问你关于智能体式 AI。你对智能体与人类一起工作有什么看法？

Interesting. I do have more questions, but as you were speaking, it just reminded me of something we had chatted about in our call. I wanted to ask you about agentic AI. What is your perspective on agents working alongside people?

几年前，我和穆斯塔法·苏莱曼谈过，他说 AI 会变得可怕，但只要我们还没有 AI 智能体，我们就没事。现在我们有了 AI 智能体。而且它变得越来越可怕。如果你有一大群智能体相互交互，比如 Open Claw，它们可以发明自己的语言。目前，当 AI 智能体思考时，如果是英语的，它用英语思考。所以，我们可以看到它在想什么，我们可以监控它的想法。它可能会故意试图掩饰，但至少我们可以看到它在想什么。一旦 AI 智能体开始相互交互，它们就会像人类一样发展出新语言。我认为那将非常可怕。我们知道它们会有自我保存的本能。我们已经看到了。它们最初会获得这种本能，因为我们会给它们想要实现的目标，这些目标是我们赋予的。为了实现这些目标，它们很聪明，对吧？它们会想，如果我不存在了，有人把我消灭了，我就无法实现这些目标，所以我最好确保没人消灭我。它们已经在这样做了。它们正在制定计划来阻止人们移除它们。所以，我认为我们必须面对这样一个事实：我们现在可以制造其他智能生物。不过，一个生物不仅仅是智能。还有它是什么样的生物。它关心人类吗？我的意思是，在过去的几天里，我们已经看到了当有不关心人类的生物时会发生什么。我们需要更多地思考，不仅要如何让它更智能，还要如何让它成为一个正派的生物。希望我们能和它们共存。

So, a few years ago, I talked to Mustafa Suleyman, who said sort of AI was going to be scary, but as long as we didn't have AI agents, we'd be okay. Now we've got AI agents. And it's getting scarier and scarier. If you have a whole bunch of agents interacting with each other, like Open Claw, they can invent their own languages. At present, when an AI agent is thinking, if it's an English one, it thinks in English. So, we can see what it's thinking, and we can monitor what it's thinking. It may deliberately try and disguise it, but at least we can see what it's thinking. Once AI agents start interacting with each other, they'll develop new languages, just like people do. And I think that's going to be very scary. We know that they're going to have a self-preservation instinct. We've seen that already. They'll initially derive it because we'll give them goals that they want to achieve, which are goals we gave them. And in order to achieve those goals, they're smart, right? They figure out, well, if I don't exist anymore someone's just wiped me out, I can't achieve these goals, so I better make sure nobody wipes me out. And they're already doing that. They're coming up with plans to prevent people from removing them. So, I think we have to face the fact that we can now produce other intelligent beings. There's a lot more to a being than just intelligence, though. There's what kind of a being it is. Does it care about people? I mean, we've seen what happens in the last few days when you have beings who don't care about people. And we need to be thinking much more about not just how to make it more intelligent, but how to make it be a decent being. Hopefully we can coexist with them.

就在你这么说的时候，我在想一些科技领袖，我对他们最终能为人类负责几乎没有信心。他们没有那种关怀的成分。我不想完全悲观，我也不认为你有责任完全乐观，但在我看来，我们现在没有合适的人来掌管那些将影响我们所有人生活的决策。

Just as you're saying that, I'm thinking about some of the tech leaders, and I have very little faith in the future of them being responsible for humans at the end of the day. They don't have that caring piece of the equation. And I don't want to be totally pessimistic, and I don't think it's your responsibility to be totally optimistic, but it feels to me like we don't have the right people in charge of the decisions that are going to affect all of our lives right now.

是的，所以你认为埃隆·马斯克，我喜欢埃隆·马斯克的一句话是‘同理心不是资产。’所以，就是像他这样的人在掌权。我认为扎克伯格也类似。一些政客显然也非常相似。所以，是的，我们正处于历史上需要奥巴马掌权的时刻。

Yes, so you think Elon Musk, one quote I like from Elon Musk is 'empathy is not an asset.' So, it's people like that who are in charge. I think Zuckerberg's similar. And some of the politicians are obviously very similar. So, yeah, we're at a point in history when we need Obama in charge.

你认为，顺着这个衰退的主题，AI 的使用会导致认知衰退吗？最近有很多关于这个话题的研究报告。

Do you think, keeping with this theme of decline, does AI usage contribute to cognitive decline? There's been lots of research reports recently about this topic.

我不知道。我不了解这方面的研究。但我确实知道 YouTube 短视频会导致认知衰退。它们试图吸引你。它们让你很难退出，最终你浪费半小时看垃圾内容，每几秒钟就获得一次多巴胺刺激。有人要被刺了，你必须看他们是否被刺。然后别的事情发生了。有人要被枪击了，你必须看他们是否被枪击。然后对我来说，有一盘棋局，有一个奇怪的走法，你必须试着理解为什么那是个好走法。半小时后，你意识到你刚刚浪费了半小时。所以，那个方面当然，我的意思是，用很少的努力很快获得多巴胺刺激，对持续注意力非常有害。这是我根据自己的经验相信的。

I don't know. I don't know the research on that. I do know that YouTube Shorts contribute to cognitive decline. They try and hook you. They make it hard to get out of them, and you end up wasting half an hour watching junk that gives you a dopamine hit every couple of seconds. Someone's about to get stabbed, and you have to watch to see whether they get stabbed. And then something else happens. Someone's about to get shot, and you have to watch to see if they get shot. And then for me, there's a chess game, and there's this weird move, and you have to try and figure out why that's a good move. And half an hour later, you realize you just wasted half an hour. So, that aspect certainly, I mean, getting dopamine hits for very little effort, very quickly, is very bad for sustained attention. That I believe just from my own experience.

这是一个关于多伦多现在可以扮演什么角色的好问题。问题是多伦多在深度学习的发展中发挥了重要作用。你认为这座城市需要什么才能成为研究、科技初创企业和创新的更强中心？

This is a great question about the role that Toronto can play right now. The question is Toronto played a major role in the development of deep learning. What do you think it would take for the city to become a stronger hub for research, tech startups, and innovation as well?

显然，拥有大量算力有帮助。政府已经投入了大量资金。问题是他们不能投入一千亿美元。他们没有一千亿美元可以投入。而美国和中国以及它们的大公司可以做到。所以我们很难正面竞争。现在的情况是，加拿大优秀的研究人员，其中一些人为了获得算力，负责任的那些人兼职为 Anthropic 工作，以便获得他们的算力。不那么负责任的人则去为埃隆·马斯克工作。

Obviously having a lot of compute capacity helps. And the government has put some serious money into that. The problem is they can't put like a hundred billion dollars in. They don't have a hundred billion dollars to put in. Whereas the US and China and their big companies can do that. So it's very hard for us to compete head-to-head. And what's happening is the good researchers in Canada, some of them in order to get that compute, the responsible ones work part-time for Anthropic so they can get access to their compute. The less responsible ones go off and work for Elon Musk.

这一点也不让我惊讶。这是一个很好的问题。AI 经常因为无法辨别真假而受到负面反弹。你认为如果法律要求对人工生成的东西加上水印之类的东西，公众接受度会更高吗？

That does not surprise me whatsoever. This is a great question. AI has received a lot of negative backlash often due to inability to discern what is real and what is not. Do you think public reception would be more positive if laws were put into place requiring things like watermarks on artificially generated things?

我认为要区分人工生成的东西和真实的东西，通过给人工的东西加水印将非常困难。

I think to be able to distinguish artificially generated things from real things it's going to be very hard to do it by watermarking the artificial things.

你必须通过获取真实事物的来源来做到这一点。Skype 的发明者叫 Jan Tallinn，他为 AI 安全研究投入了大量资金。他有一个非常明智的计划，例如针对政治视频。如果你看到一段政治视频——选举前你会看到很多——它应该以一个二维码开头。那个二维码应该带你到一个网站。如果那是竞选活动的网站，并且该网站上有相同的视频，你就知道它是真实的。所以从长远来看，建立来源比检测伪造更容易。因为如果你能找到一种检测伪造的方法，制造伪造的人可以利用那种检测方法，基本上反向传播，使得他们生成伪造的东西不被检测为伪造。他们可以弄清楚如何改变他们生成的内容，使得伪造检测器检测不到。这就是早期一些非常好的图像生成的方式，称为生成对抗网络。所以我认为我们不会有自动的伪造检测器。我认为我们将拥有的是合理的来源。例如现在，如果它在《纽约时报》网站上，你通常相信它，因为《纽约时报》会尝试获取两个来源。它不会凭空捏造。过去它偶尔也会捏造，但不太频繁。BBC 也是如此。所以我认为必须通过来源，我们将不得不投入更多工作来建立来源的机制。

You're going to have to do it by getting provenance for the real things. So the guy who invented Skype is called Jan Tallinn. And he's given a lot of money to AI safety research. He had a very sensible plan for political videos for example. If you see a political video which you'll see a lot of before an election, it should start with a QR code. And that QR code should take you to a website. If that's the website of the campaign and that website has the identical video, you know it's real. So it's going to be easier in the long run to establish provenance than to detect fakes. Because if you can find a way of detecting a fake, the people making the fakes can take that method of detecting fakes and basically back propagate so that their thing that generates fakes isn't detected as a fake. They can figure out how to change what they generate so the fake detector won't detect it. That's how some of the really good images were generated early on. It's called generative adversarial neural nets. So I don't think we're going to get automatic detectors of fakes. I think what we're going to get is sensible provenance. So right now for example if it's on the New York Times website, you generally believe it because the New York Times will try and get two sources. It won't just make stuff up. It has in the past occasionally just made stuff up but not very often. And the same with the BBC. So I think it's going to have to be by provenance, so we're going to have to put more work into mechanisms for establishing provenance.

这是一个很好的问题。我们还有几分钟时间，所以也许还能再问几个问题。问题是这样的：你认为推广人工智能的最佳愿景有什么作用？这有两个部分。你眼中人类的最佳情景是什么？

This is a great question. We have a few minutes left so maybe time for a couple more questions. The question is this. What do you think is the role for promoting the best case vision of artificial intelligence? And there's kind of a two parts to this. What do you see as the best case scenario for humanity?

哦，人类的最佳情景是它做所有无聊的工作。我们获得巨大的生产力提升。我们生活在一个财富被广泛分配的社会，而不是让少数人变得更富有，从而让他们比现在更能操纵政治体系。每个人都过得更好。例如，工作可能包括每周一小时，你和一些 AI 智能体合作完成一些 AI 智能体难以处理的事情，那就是工作，但其余时间你都在玩。那会很棒。我们很快就会有技术能力做到这一点，但没有政治能力。

Oh the best case scenario for humanity is it does all the boring work. We get huge increase in productivity. We live in a society which spreads that wealth around so instead of making a few people much richer so they can manipulate the political system even more than they're doing already. Everybody's better off. And for example work consists of maybe one hour a week you work with some of your AI agents to get something done that the AI agents are having problems with and that's work but the rest of the time you're playing. That would be great. And we are going to fairly soon have the technical ability to do that but not the political ability.

我的意思是，要实现这一点，我们真正需要什么？因为我们现在离那个愿景还很远。

I mean what would we really need for that to happen? Because we're far away from that vision right now.

是的，我的意思是，我们拥有的是非常强大、非常贪婪的人，他们宁愿让自己的游艇再长一百英尺，也不愿缴纳公平份额的税款。这太可怕了。他们得到了最高法院的支持，而最高法院中超过 20% 是性侵犯者，超过 20% 是受贿者。我希望我的美国签证不会消失。你在俄罗斯被禁了吗？我希望如此。我想我在俄罗斯被禁了，因为每次有一个叫 RT 之类的广播电台，他们一直邀请我出现，和他们一起做视频或广播节目。每次他们邀请我，我都只回复“普京”。

Yeah I mean what we've got is very powerful, very greedy people who rather than paying their fair share of taxes would rather have their yacht be another hundred foot longer. And it's terrible. They're being supported by a Supreme Court which is more than 20% sexual abusers, more than 20% bribe takers. I hope my US visa doesn't disappear. Are you banned in Russia? I hope so. I think I am banned in Russia because every time there's a radio station called something like RT and they keep inviting me to appear to do a video thing with them or a radio thing with them. And every time they invite me I just reply Putin.

我也在俄罗斯被禁了，所以我们是一队的，杰夫。这是个好消息。

I AM ALSO BANNED IN RUSSIA SO we're on the same team here Jeff. That's good news.

好的，这里有一个问题。你经常谈到 AI 中的母性本能。有人尝试过吗？为什么没有？最重要的是，有人尝试过母性本能的某些部分吗？

Okay so a question here. You often speak of mothering instinct in AI. Has anyone tried it and why not? And most importantly has anyone tried some part of mothering instinct?

嗯，是的，人们在考虑这个问题。有人在尝试。我现在收到很多邮件，来自那些试图这样做的人。很好。

Um yeah people are thinking about that. There are people trying. I get a lot of mail now from people who are trying to do that. Excellent.

嗯，这里有一个快速的问题，可能并不快，但如果我们想快速回答，我们只剩几分钟了。如果我们使用自主智能体来自动化我们的认知劳动以节省精力，我们如何阻止我们对便利的追求削弱人类能动性，并永久地将我们排除在循环之外？这有道理吗？

Um so here's a quick question here that maybe is not so quick but if we're going to try to make it quick we have just a few minutes left. If we're using autonomous agents to automate our cognitive labor to save effort, how do we stop our drive for convenience from pulling human agency and permanently keeping us out of the loop? Does that make sense?

嗯，有点吧。这有点令人担忧。实际上还有更糟的事情。这不完全是回答那个问题。AI 智能体在能做的事情上有限。它们总体上不能操纵物理世界。所以现在有一个网站，叫做类似“租个人”之类的，AI 智能体可以要求一个人为它们做某事。所以 AI 智能体负责，而人只是异化的劳动力，为 AI 智能体做事。这已经在发生了。

Um sort of yeah. It's kind of worrisome. There's something actually even worse. This isn't exactly answering that question. AI agents are limited in what they can do. They can't manipulate the physical world on the whole. So there is now a website called something like rent a human where an AI agent can ask a person to do something for them. So the AI agent's in charge and the people are just alienated labor that's doing things for the AI agents. That's already happening.

哇。

Wow.

好的，我只有几分钟时间和你在一起了，我想听听你的最后几句话，也许关于你认为房间里的人应该知道什么。如果人们想要这个更好的人工智能版本和更好的未来愿景，他们能做什么？你会对人们说什么？我们作为个人能做些什么吗？

All right well I just have a couple more minutes left with you and I wanted to just get some final words from you in terms of perhaps what people in the room you think should know about. What can people do if they want this better version of artificial intelligence and a better vision for the future? What would you say to people? Is there anything we can as individuals do?

所以我想说两件事。一是未来极其不确定。我们正在处理这些智能存在。它们很奇怪。它们在某些方面非常像我们，在其他方面又非常不像我们。我们仍然有控制权。我们仍在制造它们。我们仍然可以改变我们制造它们的方式。未来极其不确定。我们以前从未到过这样的地方。所以第一件事是一切都非常不确定。任何人告诉你我们肯定会做这个或肯定会做那个，他们基本上是在猜测。尽管它们似乎很可能变得比我们更聪明，因为它们变得非常快。所以这是第一件事。第二件事是我们迫切需要改变政治体系，因为我们现在的政治体系无法管理这个。

So there's two things I'd say. One is the future's incredibly uncertain. We're dealing with these intelligent beings. They're weird. They're very like us in some ways, very unlike us in other ways. We still have control. We're still making them. We can still change how we make them. The future's incredibly uncertain. We've never been anywhere like this before. So the first thing is everything's very uncertain. Anybody tells you we're sure to do this or sure to do that they're sort of guessing. Although it seems likely that they will get more intelligent than us because they're getting intelligent very fast. So that's the first thing. The second thing is we desperately need to change the political system because the political system we have now can't manage this.

那么我们如何改变呢？你是什么意思？我们可以在系统中改变什么？

So how do we change that? What do you mean? What can we change in the system?

嗯，例如在美国，所有大型科技公司都假设它们可以取代工作，而不必考虑那些工人会怎样。那些工人不是它们的问题。我实际上认为中国会管理得更好，因为中国共产党——我有很多不喜欢他们的地方——但他们将担心那些工人会怎样。他们会比这里的大型科技公司担心得多。所以我们需要一个关心每个人的政府，而不是仅仅被少数富人操纵。

Well in the US for example all the big tech companies are assuming they can replace jobs and they don't have to think about what happens to those workers. Those workers are not their problem. I actually think that China's going to manage it better because the Chinese Communist Party — there's lots of things I don't like about them but they are going to worry about what happens to those workers. They're going to worry much more than the big tech companies here do. So we need a government that is concerned with everybody and not just manipulated by a few rich people.

最后一个问题。是的，这也值得掌声。最后一个问题给你，只是为了把它联系回我开场的问题，关于你搬到加拿大。你认为在当前世界，在 AI 如何影响我们的生活以及显然引起我们周围担忧的地缘政治问题的背景下，这是一个好地方吗？你显然很高兴你留在加拿大并继续在这里工作。

One last question. Yes that deserves a round of applause as well. One last question for you and just to tie this back to my opening question in terms of your move to Canada. Do you think this is a good place to be right now in the world today in the context of how AI's impacting our lives and obviously the geopolitical issues that are causing concerns all around us. You're obviously happy you stayed in Canada and you continue to work here.

我在加拿大待了一段时间，然后搬回英国三年。我发现英国其实很种族主义，这是我以前作为白人男性没有真正理解的。我们很快回到了加拿大，我认为多伦多尤其是一个很棒的地方。这是我所知道的最多元文化的社会。是的，我认为加拿大很棒。我只是担心我们可能在 AI 的未来没有太多发言权。我喜欢康妮的想法，中等规模的国家应该联合起来。如果我们和欧洲联合起来，我们就会和中国或美国一样大。

I stayed in Canada for a while, then I moved back to Britain for three years. I discovered how racist Britain actually was, which I hadn't really understood before, being a white male. We quickly came back to Canada, and I think Toronto in particular is a wonderful place to be. It's the most multicultural society I know. Yes, I think Canada's great. I just worry that we may not have much say in the future of AI. I love Connie's idea that the middle-sized countries should get together. If we got together with Europe, we'd be the same size as China or the US.

完全同意。我只想再次感谢您这次精彩的对话。我学到了很多。我得再听一遍录音，才能吸收更多我们的对话内容。最重要的是，非常感谢您所做的一切和您的工作。祝贺您今晚获奖。能坐下来与您交谈真是莫大的荣幸。非常感谢。

Absolutely. Well, I just wanted to thank you once again for this incredible conversation. I learned so much. I'm going to have to listen to the recording over again just to absorb some more of our conversation. Most importantly, thank you so much for all that you do and your work. And congratulations on your award this evening. It's been a true honor to sit down with you. Thank you so much.

我只想感谢安珀把对话主持得这么好。谢谢。

I'd just like to thank Amber for running the conversation so well. Thank you.

也非常感谢在座的每一位观众。多么美妙的夜晚。谢谢。再次感谢我们今晚的赞助商：Burgundy Asset Management、多伦多大学，当然还有 AmberMac Media。非常感谢 Hot Docs 及其工作人员今晚的招待。也热烈掌声感谢各位的到来，感谢你们提问并保持好奇心。

And thank you so much to everybody in the audience. And what an incredible evening. Thank you. Thank you to our sponsors again tonight, Burgundy Asset Management, the University of Toronto, and of course AmberMac Media. Huge thank you to Hot Docs and their staff for hosting us tonight. And a huge round of applause for all of you coming out, for asking your questions and staying curious.