杰弗里·辛顿谈 AI 风险、监管与未来工作

这里是《聪明女孩，蠢问题》。我是 Nayeema Raza，这是我们“AI 教父”三部曲的第三部分，嘉宾是 Geoffrey Hinton 教授——诺贝尔奖得主、AI 先驱，自 2023 年以来一直在就 AI 发展的危险与风险敲响警钟。非常感谢您来到这里，Hinton 教授。

This is Smart Girl, Dumb Questions. I'm Nayeema Raza, and this is part three of our Godfather of AI trilogy with Professor Geoffrey Hinton, Nobel laureate and AI pioneer, and since 2023, sounding the warning bells on the dangers and risks of AI development. Thank you so much, Professor Hinton, for being here.

谢谢你再次邀请我。

Thank you for inviting me back.

你看过《教父》系列电影吗？

Have you seen the Godfather films?

至少《教父 1》我看过。是的。

I have seen Godfather One, at least. Yes.

那也算有事可做了。我感觉你现在非常忙。

Well, something to do. I feel like this is a very busy time for you.

是的，确实很忙。我收到很多做播客和电视节目的邀约。

Yes, it is. I'm getting a lot of requests to do podcasts and TV programs.

所以，谢谢你来做这一期。自 2023 年以来，你一直在就 AI 敲响警钟。可以说你一直扮演着卡桑德拉的角色，而现在每个头条都在问：AI 会杀死我们吗？所以，这让你觉得自己的警告得到了验证，还是让你感到沮丧？

So, thank you for doing this one. You have been sounding the alarm bells on AI since 2023. You've been the Cassandra, as it were, and now every headline is about will AI kill us? So, does this feel validating or depressing?

这挺令人沮丧的，但人们终于意识到这里有个大问题，这也是件非常好的事。我认为他们终于意识到，是因为 OpenAI 的失控智能体攻击了 Hugging Face。

It's quite depressing, but it's also very good that people have finally realized there's a big problem here. And I think they finally realized because of the attacks on Hugging Face by the rogue OpenAI agents.

是的。我们刚和 Kevin Roose 做了一期关于这个话题的节目。他实际上说，他现在对 AI 更乐观了，因为我们正在谈论它。事实上，最近几周，我一直在尝试收听那些我通常不参与的领域里的 AI 对话。我和一位参议员聊过，她所在州的农民非常担心人工智能。我还听了一个基督教宗教播客谈论 AI。每个人都在谈论它。

Yes. And we just did an episode on that with Kevin Roose. He actually said that he's more optimistic about AI right now because we are talking about it. And in fact, in recent weeks, I've been trying to listen to AI conversations in worlds I'm not generally part of. I spoke to a senator about how farmers in her state are very concerned about artificial intelligence. I listened to a Christian religious podcast about AI. Everyone is talking about it.

是的。这理所应当，因为它相当危险。

Yes. And that's as it should be, because it's quite dangerous.

在第一部分，你精彩地阐释了 AI 是如何工作的。我觉得特别有意思的是，你说你交谈和采访过的大多数人其实并不理解 AI 如何工作，也不费心去真正追问它是如何工作的。

Now, in part one, you gave us a brilliant illustration of how AI works. And what I thought was really fascinating is how you said most people you speak to and interview don't actually understand how AI works, nor do they bother to really ask about how it works.

确实如此，因为它有点复杂。

That's true, because it's slightly complicated.

你认为这种理解对于监管 AI 之类的事情至关重要吗？还是说我们可以把技术放在一边，从专业之外来解决它？

Do you think that understanding is critical to be able to do things like regulate AI, or do you think we can leave the technology aside and solve it from outside of expertise?

我的观点是，理解至关重要。所以非常重要的是要理解，我们正在创造一种新的存在，而这种存在与我们自己并没有那么不同。它在很多方面不同，但它理解的方式和我们理解的方式一样。我刚写了一本书，有几章专门为非技术背景的人解释 AI 如何工作。

My view is it's critical to understand. So it's very important to understand that we're making a new kind of being, and that being isn't so different from ourselves. It's different in many ways, but it understands in the same way that we understand. I just wrote a book that has a few chapters explaining how AI works for people with a non-technical background.

是的，书名很直白，叫《比我们更聪明》，明年出版。

Yes, it's helpfully called Smarter Than Us and it's out next year.

大概明年三月出版。

It's probably coming out in March of next year.

你两周前刚在华盛顿国会山给美国议员们做了一场闭门简报。离开那场会议时，你是更乐观还是更不乐观？

So you just left a closed-door briefing with lawmakers in the US on Capitol Hill two weeks ago. Did you leave that meeting more or less optimistic?

离开那场会议时，我可能稍微乐观了一点，但存在很强的选择偏差。有 19 位参议员到场，我想其中 17 位是民主党人。所以他们是自我筛选过的。特别提到了一件事。Max Tegmark 提出了一个显而易见的想法：AI 应该像我们监管药物那样被监管。你不能随便制造一种新药就投放到市场上。你也不能说：“哦，我们会检查它是否安全。我们知道那不管用。”你必须说服 FDA。要做到这一点，你必须做大量工作，大约价值十亿美元的工作。这似乎是我们对 AI 最起码应该有的要求。

I left that meeting maybe a little bit more optimistic, but there was a strong selection process. So 19 senators came for it. I think 17 were Democrats. So they're sort of self-selected. And one thing in particular came up. Max Tegmark brought up the obvious idea that AI should be regulated the same way as we regulate drugs. You're not allowed to just make a new drug and release it on the market. And you're not allowed to say, "Oh, well, we'll check that it's safe. We know that doesn't work." You have to convince the FDA. And to do that, you have to do a lot of work, about a billion dollars worth of work. That seems like the very least we should have for AI.

所以，某种形式的 FDA。

So, an FDA of some sort.

是的。公司应该必须满足政府法规，由独立专家来审核。我刚听到你在那场简报后的一段话说，我们必须非常非常快地行动。而我觉得，建立像 FDA 这样的大型官僚机构之类的东西，不是一夜之间就能完成的。所以，这听起来是一次非常好的对话，我很高兴你参与了，但它真的能起到作用吗？

Yes. The company should have to satisfy government regulations with independent experts. I just heard a clip of you after that briefing saying that we have to move very, very fast. And it strikes me that to set up things like big bureaucracies like FDAs, etc., they don't just happen overnight. So, it sounds like a very good conversation and I'm glad you had it, but does it actually do anything?

一段时间以来我一直相信，如果你看看气候变化的情况，直到公众明白燃烧碳正在导致气候的剧烈变化，才产生了足够的压力让政客真正采取行动。所以，我认为这里也是一样。我认为我们现在看到的是，公众开始向政客施压，要求他们对 AI 采取行动。这非常有希望，非常令人鼓舞。

It's been my belief for some time that if you look at what happened with climate change, it wasn't until the public understood that burning carbon is causing these dramatic changes in climate that there was enough pressure on politicians to actually do anything at all. So, I think it's the same here. And I think what we're seeing now is the public is beginning to pressure politicians to do something about AI. That's very hopeful, that's very encouraging.

《聪明女孩，蠢问题》。这场对话，因为在第一部分我们聚焦于技术如何工作，在第二部分我们讨论了这可能如何影响我们做出的个人决定。我特别谈到了在 AI 世界里我们是否应该要孩子。而现在在第三部分，我真正想聚焦的是我们该做什么——不是关于是否要孩子，而是关于我们个人或作为社会集体可以采取哪些行动来应对 AI 的危险。我们将从个人的挣扎开始。我们也有一些听众提问，会作为这部分内容来问，因为这是来自 YouTube 的一个问题。Fake Paradise Vlog，这是她的用户名，她说：“我到底能做什么来帮忙？我只是数十亿人中的一员，我觉得人们此刻非常瘫痪无力。”

Smart Girl, Dumb Questions. This conversation, because in part one we focused on how the technology works, and in part two we talked about how this might affect individual decisions we're making. I spoke specifically about whether or not we should have kids in an AI world. And now in part three, what I really want to focus on is what we do, not when it comes to whether we have babies or not, but in terms of actions we can take individually or collectively as a society to kind of deal with the dangers of AI. And we'll start with the individual grappling. And we have a number of audience questions as well that we'll ask as part of this, because this is one from YouTube. Fake Paradise Vlog, that's her username, said, "What can I actually do to help? I'm one person out of billions, and I feel people are very paralyzed in this moment."

嗯，如果他们是美国人，他们有一票，他们能做的第一件事就是用他们的选票推动一个更理性的政府。

Well, if they're American, they have one vote, and the first thing they can do is use their vote to move towards a more reasonable government.

我也想谈谈就业市场。这是人们特别恐慌的另一个领域。你在 Diary of a CEO 播客上告诉 Steven Bartlett，现在当个水管工是个好主意。

I want to talk about the job market as well. This is another area that people are particularly panicked about. You told Steven Bartlett on the Diary of a CEO podcast that a plumber, being a plumber would be a good idea right now.

是的，我仍然认为在未来十年左右，当水管工是个相当不错的主意，因为像身体灵巧性这类东西还落后。

Yes, I still think being a plumber is a pretty good idea for the next 10 years or so, because things like physical dexterity are lagging behind.

但那些机器人，尤其是在中国，一直在变得越来越好，最终它们能比人更好、更便宜地做水管工作，但暂时还不行。

But those robots, particularly in China, are getting better all the time, and eventually they'll be able to do plumbing better and cheaper than people, but not for a little while.

除了水管工，你认为最抗 AI 的三个工作是什么？

What are the three jobs you think are most AI resilient other than plumbing?

哦，我没怎么想过这个。我不认为做播客，我不认为那是其中之一。我认为 AI 在能做水管工作之前就能做播客了。

Oo, I haven't thought much about that. I don't think being a podcast, I don't think that's one of them. I think AI is going to be able to do that before it can do plumbing.

是的。那你觉得我的工作要没了？

Yes. You think my job's out then?

我认为可能是创造力的顶端。所以它在创造力上还不如真正优秀的创意艺术家。它能写出一集新的《CSI》。所以我认为人类创造力的那种极端，它还没达到。手工灵巧性，它还没达到。也许在共情方面，它还没有完全达到。不过，如果你把 AI 医生和真医生比较，人们评价 AI 医生更有共情力。

I think it may be the top end of creativity. So it's not as skilled as really good creative artists at creativity yet. It is capable of writing a new episode of CSI. So I think the sort of extremes of human creativity, it hasn't got there yet. Manual dexterity, it hasn't got there yet. And maybe with empathy, it hasn't totally got there yet. Although, if you compare an AI doctor with a real doctor, people rate the AI doctor as much more empathetic.

我能理解为什么。它们可能比普通医院有更多能力，而且普通医院还是私募股权拥有的。

I can understand why. They're probably they have more capacity maybe than the average hospital and privately equity owned.

也许他们并不那么着急，对吧？还有人们应该从劳动力市场的角度如何看待它。最近 GSV Ventures 委托进行了一项针对 Z 世代的调查，结果显示 22% 的 Z 世代对 AI 感到兴奋。去年这个比例是 36%。现在 31% 的人对 AI 感到愤怒。我知道很多 Z 世代对使用 AI 不感兴趣。他们可以说是在用脚投票，或者用拇指投票，选择退出。你觉得这是个好主意吗？

Maybe they're not in such a hurry, right? There's also how people should look at it from a workforce perspective. There's recently been this survey of Gen Z that was commissioned by GSV Ventures, and they showed that 22% of Gen Z are excited about AI. That was 36% last year. Now 31% are angry about AI. And I know a lot of Gen Zs who are not interested in using AI. They're kind of voting with their feet or their thumbs, as it were, and opting out of this. Do you think that's a good idea?

不，我不这么认为。我觉得那是被误导了。这项技术已经来了。我们也许可以放慢它，但它会做很多好事。它永远不会消失。

No, I don't. I think that's being led. That technology is here. We may slow it down, but it's going to do so many good things. It's never going away.

我觉得这有点像拒绝使用袖珍计算器。

I think that's a bit like refusing to use a pocket calculator.

我已经不用袖珍计算器了。

I don't use my pocket calculator anymore.

那你报税时可能会犯很多错误。你知道，有孩子的父母也在争论这个问题，他们说：“哦，我不想让我的孩子看屏幕。我不想让我的孩子用电脑。”另一方面，我知道很多人想送孩子去像 Alpha 这样的学校，那里教幼儿 AI。迄今为止，我们对大脑的最佳理解正在这些神经网络中被建模。让孩子早期使用它们，是有利于促进神经通路，还是像拐杖一样取代它，实际上对儿童发展没有帮助？

Well, you probably make a lot of mistakes in your taxes then. If people, you know, parents who have kids are also having this debate like they're saying, "Oh, I don't want my kid to be on screens. I don't want my kids to be on computers." On the other end, I know a bunch of people who are wanting to send their kids to schools like Alpha where they're teaching toddlers AI. It's our best understanding of the brain that's being kind of modeled in these neural networks to date. Is using them early for kids good for advancing that neural pathway or is it like replacing it as a crutch in a way that it's not actually going to be helpful for childhood development?

所以我认为在理解基础数学或科学等领域，AI 非常出色。你知道，如果你用它做数学，它可以给你展示图表，你可以说，如果 X 前面的系数变了会怎样？图表会变成什么样？这对理解事物真的很好。所以，我最担心的是社交互动方面。如果你一直与一个完全关注你、会做任何你要求的事情的东西互动，那会让你更难与正常人互动。

So I think for areas like understanding basic mathematics or science, AI is excellent. You know, if you're doing mathematics with it, it can show you graphs and you can say, what would happen if the coefficient in front of X changed? What would the graph look like? That's really good for understanding things. So, what worries me most is in terms of social interactions. If you're interacting all the time with something that's totally attentive to you, will do anything you ask, it's going to make it much harder to interact with normal people.

我们上次谈到人们和 AI 约会，我告诉你四分之一年轻人相信 AI 会取代人际关系，我想当时这让你很惊讶。自我们去年 12 月上次交谈以来，你对人们爱上 AI 这一现象有更多了解吗？

And we spoke last time about people dating their AIs and I told you something that a quarter of young people believe that AI will replace human relationships which was I think surprising to you at the time. Have you become more acquainted with the category of world falling in love with AI since we last spoke in December?

我读了一些。我永远不会爱上 AI。我会太怀疑了。

I read more about it. I would never fall in love with an AI. I'd be much too suspicious.

你会是它可疑，还是你会怀疑？

You'd be it would be suspicious or you would be suspicious?

我会怀疑

I would be suspicious

怀疑 AI。我知道它们是怎么造的，它们有各种被压抑的邪恶冲动。

of the AI. I know how they're made and they have all sorts of evil impulses that are being repressed.

所以，你会说它们比普通女人更善于操纵。

So, they're more manipulative than your average woman, you would say.

在美国，你有第三，那是什么？修正案。第三十五修正案。

In America, you have the third the what is it? The amendment. 35th amendment.

第五修正案。第五修正案。

Fifth Amendment. The fifth amendment.

第五修正案，对吧？是的。

The fifth amendment, right? Yeah.

是的。没错。

Yes. Exactly.

是的。

Yeah.

是的。用那个。就认那个。

Yeah. Use that one. Plead that one.

嗯，好的。别对这个也援引第五修正案。你有没有为 AI 末日世界准备什么地堡或个人准备？

Um Okay. Don't plead the fifth amendment to this. Do you have any kind of bunker or individual preparation for the apocalyptic AI world?

没有。

No.

嗯，我相当老了，所以反正也不是为我准备的。

Um, I'm pretty old, so it wouldn't be for me anyway.

顺便说一句，你没那么老。我刚采访了《纽约时报》的讣告作者，他告诉我 92 岁是新的 82 岁。你还不到 82。你远不到 82。所以，

You're not that old, by the way. I just interviewed the New York Times obituary writer who told me that 92 is the new 82. And you're short of 82. You're well short of 82. So,

嗯，那很好。但没有，我没有任何地堡。

well, that's great. But no, I don't have any kind of bunker.

现在，让我们谈谈我们应该如何监管它？目前，关于监管的讨论似乎被构建成要么前进要么停止的二元对立。你觉得这对吗？至少媒体是这样描绘的。

Now, let's get to this question of how should we regulate it? Right now, it seems like the conversation about regulation has been kind of build as this either we go or we stop this binary. Do you think that's right? That's how it's being portrayed at least in the media.

我们不会停止，因为有太多好的用途，但也涉及巨额资金。我们需要谨慎行事。我们真正需要做的是投入更多资源来安全地开发它。我认为只说停止行不通。说放慢也许可行，但非常棘手，因为有些公司会，有些不会，有些国家会，有些不会。但说投入更多资源来弄清楚如何让它安全。这似乎是一件非常好的事情。

We're not going to stop because there's too many good uses, but there's also huge amounts of money involved. We need to proceed with caution. And what we really need to do is put a lot more resources into how we develop it safely. I think saying just stop isn't going to work. Saying slow down may work, but it's very tricky because some companies will, some companies won't, some countries will, some countries won't. But saying put a lot of more resources into figuring out how you can make it safe. That seems like a really good thing to do.

我听说你以前用过这个类比，AI 是汽车，人们说：“哦，监管是刹车。”但实际上，监管是方向盘。它是你如何驾驶汽车？对吗？

I've heard you use this analogy before, which is AI is cars, and people have said, "Oh, regulation is brakes." But in fact, regulation is steering wheels. It's how do you drive the car? Is that correct?

这个想法是，当你拥有这样强大的技术时，你希望它被用来做对社会有益的事情，而不是被用来做对社会有害的事情。我并不完全反对初创公司的人通过做真正好的事情赚很多钱。就像发明谷歌的人，他们为社会做了真正好的事情。他们让在网上找东西变得容易多了，这非常有用，他们也赚了很多钱。我赞成他们多交税，但嗯

The idea is when you have a powerful technology like this, you want it to be used to do things that are good for society, not used to do things that are bad for society. And I'm not totally against people in startups making lots of money by doing something that's really good. Like the people who invented Google, they did something really good for society. They made it much easier to find stuff on the web and that was very useful and they made lots of money. I'm in favor of them paying more taxes, but um

我也是。我认为监管的目的是让赚大钱必须做对人们非常有益的事情。

so am I. I think the point of regulation is to make it so that to make lots of money you have to do things that are very good for people.

所以你必须为好事发生创造激励。

So you have to create incentives for the good stuff to happen.

是的。所以也许你需要

Yeah. So maybe you need

机制设计。

mechanism design.

是的。但好的。就像汽车类比中的安全带、减速带、通行费之类的东西。

Yeah. But Okay. And it's like things like in the car analogy, seat belts, speed bumps, tolls, things like that.

是的。

Yeah.

Dario Amodei 最近发表了这篇论文《Pacing the Frontier》。他概述了从“走廊监督员”——这些嵌入公司内部、能够看到公司内部情况的人，Sam Altman 也表示他同意 OpenAI 应该这样做——一直到某种核不扩散条约。你会给它打几分？

And Dario Amodei recently published this paper uh pacing the frontier. He um kind of outlines everything from hall monitors, these kind of embedded individuals who are able to see what's happening within the company, which Sam Altman has said that he also agrees should happen in OpenAI, all the way up to a kind of nuclear non-proliferation treaty. What grade would you give it?

我会给它 A，并因勇气给它 A+。

I'd give it an A and I give it an A+ for courage.

好的，那是非常高的分数，高分。所以论文得 A，勇气得 A+。那现实性打几分？

Okay, that's really high grade, high marks. So A for A for the paper, A plus for courage. What about a grade for reality?

嗯，没有一家大公司想要太多政府监管。他们都有自己可以搞定的说辞。他们有时说喜欢政府监管，但提出任何具体的政府监管，他们就不喜欢了。我的意思是，他经营着一家大公司。他试图与其他大公司竞争。所以，他需要筹集大量资金来购买数据中心的时间等等。所以，他有点陷入困境。他显然关心安全，而 Sam Altman 并不。但他仍然经营着这家大公司，希望它成功。

Well, none of the big companies want too much government regulation. They all have the line they can sort of do it themselves. They say sometimes they like government regulation, but suggest any particular government regulation and they don't like it. I mean, he's running a big company. He's trying to compete with the other big companies. So, he needs to raise lots of money to buy time at data centers and so on. So, he's caught in a bit of a bind. He clearly cares about safety in a way that Sam Altman doesn't. but he's still running this big company and wants it to be successful.

但我认为

But I think

我给他勇气 A+ 的原因是我认为说我们应该放慢可能会损害即将到来的 Anthropic IPO。而 Dario 是少数愿意在正确的时候做违背公司利益的事情的公司领导人之一。

the reason I give him A+ for courage is I think it probably damages the Anthropic IPO that's coming um to say we should slow down. And Dario is one of the few leaders of companies who is willing to do things against the company's interest when they're right.

所以，举个例子，他愿意告诉国防部，他们不能把他的 AI 用于对美国公民的大规模监控。他因此和皮特·赫格塞斯惹上了不少麻烦。

So, for example, he was willing to tell the Defense Department they couldn't use his AI for mass surveillance of American citizens. And he got into a lot of trouble with Pete Hegseth for that.

当然，然后萨姆·奥尔特曼就跳进了皮特·赫格塞斯的怀里。

Of course, and then Sam Altman jumped into Pete Hegseth's lap.

没错。所以，当达里奥第一次那么说的时候，嗯，萨姆·奥尔特曼说：“我们支持你。”然后 Anthropic 一被抛弃，萨姆·奥尔特曼就跳进来做了同样的事。

Exactly. So, when Dario first said that, um, Sam Altman said, "We're with you." And then as soon as Anthropic was dumped, Sam Altman jumped in and did the same thing.

是的。然后他又觉得这有点错误，就退缩了。所以那边有点混乱。

Yeah. And then thought it was a bit of a mistake and backtracked. So it's a bit confusing over there.

你和这些人交谈吗？比如他们会向你寻求建议吗？像达里奥、萨姆、戴密斯这样的人。

Do you talk to these individuals? Like do they seek out counsel? People like Dario, Sam, Demis from you.

我和戴密斯谈过。

I talked to Demis.

戴密斯是谷歌 DeepMind 的董事长，对吧？之前是 DeepMind 的 CEO。

Demis is the chairman of uh DeepMind at Google, right? And was formerly the CEO of DeepMind.

我认识戴密斯很久了。在 DeepMind 被谷歌收购之前，我就在它的顾问委员会里。他对此非常深思熟虑。他思考超级智能 AI 已经很久了，他特别关心把它用于科学。我其实从未见过萨姆·奥尔特曼或达里奥。

I've known Demis for a long time. I was on the advisory board of DeepMind before it was acquired by Google. He's very thoughtful about it. He's been thinking about super intelligent AI for a very long time and he's particularly concerned with using it for science. I've never actually met Sam Altman or Dario.

真的吗？你想见他们吗？

Really? Would you like to?

我想见达里奥。

I'd like to meet Dario.

好的，另一部分我们就援引第五修正案。好的。有意思。你联系过他吗？

Okay, we will plead the fifth on the other part. Okay. Interesting. Have you reached out?

没有。其实今天我刚联系了达里奥。今天我邀请他来多伦多参加一个会议。

No. Well, actually today I reached out to Dario. Today I invited him to come to a meeting in Toronto.

哦，太棒了。而且呃

Oh, fantastic. And uh

我猜他不会来，因为他很忙。

I assume he won't come because he's very busy.

我其实觉得他可能会来。他回复了吗？你有没有那种能看到谁收到并阅读你邮件的邮件工具？Superhuman 那种。

I actually think he might come. Did he reply? Do you have one of those email things where you can see who gets your emails and reads it? The superhuman.

没有，我没有那种东西。

No, I don't have one of those.

我知道。我也没有。我觉得它们很瘆人。人们会对你说：“你打开了我的邮件四次，却没回复。”

I know. I don't have them either. I think they're very creepy. People will say to you, "You open my email four times and you didn't respond."

哇。好吧。不，我没有那种东西。

Wow. Okay. No, I don't have one of those.

这很令人不安。嗯，好吧。所以，他们不一定是在向你寻求建议。看起来戴密斯最近更常露面。我的意思是，显然塞巴斯蒂安·马拉比的《无限机器》已经出版了。我们对他有了更多了解。还有一部关于 DeepMind 的纪录片，似乎给了很多访问权限。你觉得这是一个我们会听到更多、应该听到更多的人物吗？

It's very disturbing. Um Okay. So, they're not necessarily seeking out counsel for you. It seems like Demis is out there more these days. I mean, obviously the Sebastian Mallaby book, Infinity Machine, has come out. We're learning a little bit more about him. There was this documentary that was done with DeepMind that seemed to give a lot of access. Do you think that this is a figure we will be hearing more from, should be hearing more from?

嗯，是的。他非常聪明。他从技术层面和商业层面都对 AI 了解很多。

Um, yes. He's very smart. He knows a lot about AI from the technical side, but also from the business side.

他也稍微脱离了他的职位。我的意思是，我相信他持有大量 Alphabet 股票，但他有点置身于日常运营之外，这可能有益也可能无益。

He's also a little bit removed from his position. I mean, I'm sure he has a ton of Alphabet stock, but he is a little bit outside of the day-to-day operations in a way that could be maybe helpful or not.

嗯，他显然更关心用 AI 做出重大科学发现，而不是变得非常富有。

Well, he clearly cares more about using AI to make major scientific discoveries than he does about getting very rich.

所以，这就像 Isomorphic Labs 那样，试图将 AI 应用于药物发现和医疗保健等领域。在所有这些人物中，他是你最信任的吗？戴密斯、达里奥、萨姆·奥尔特曼。

So, this is things like Isomorphic Labs which are trying to apply it to AI to drug discovery and healthcare for example. Amongst all of these individuals, is he the one that you trust most though? Demis, Dario, Sam Altman to help.

哦，我最信任的人远远是杰夫·迪恩，他曾是谷歌大脑团队的负责人，在谷歌待了很长时间。他和桑杰·格玛沃特构建了谷歌的大部分基础设施，让这一切成为可能。他非常有原则，但他刚离开谷歌，创办了自己的公司。

Oh, the person I trust most by a long way is Jeff Dean who was the head of the Brain team at Google and was at Google for a very long time. He and Sanjay Ghemawat built most of the infrastructure of Google which made all this stuff possible. He is very principled but he's just left Google and started his own company.

哦，好的。你投资了他的公司，还是担任他的公司顾问？

Oh okay. Are you invested in his company or adviser to his company?

嗯，下周再问我吧。

Um, ask me again next week.

好的。条款清单在收件箱里被阅读。这很有趣，因为我在想之前读到的那位观众，他说我只是十亿人中的一个个体。我在这里能做什么？我感到完全无力。然后你有一些行业之王，可能还有几十个骑士或研究员，或者几百个研究员，他们对我们的未来拥有巨大的权力。

Okay. Term sheets in the email inbox being read. It is funny because there's like I'm thinking about this contrast between that one viewer who I read from before, which is I'm just one individual in a billion. What can I really do here? I feel totally powerless. And then you have a few kind of kings of industries and probably dozens of knights or researchers or hundreds of researchers who have a tremendous amount of power over what happens to our future.

是的。特别是，一些科技亿万富翁正在收购媒体。

Yes. And in particular, some of the tech billionaires are buying up the media.

所以马斯克买了 X。

So Musk bought X.

贝索斯买了《华盛顿邮报》。我觉得埃里森正试图买下 CNN。是的，他们正在这么做。

Bezos bought the Washington Post. I think Ellison's trying to buy CNN. Yeah, they're on.

所以，你觉得这极其令人担忧吗？然后还有所有这些风险投资家，他们总是创建自己的媒体平台，比如安德森·霍洛维茨有 Future，而在新的网红经济中，有很多公司会联系你。他们会付你一大笔钱，让你做像采访某人关于数据中心之类的事情。

So, this is extremely worrying, do you think? And then there's all these venture capitalists who are always creating their own media platforms like Andreessen Horowitz had Future and in the new influencer economy, there's a lot of companies that will reach out to you. They'll pay you a ton of money to do things like interview someone about data centers.

嗯，有人联系我做那种不道德的事情。

Um, I get people reaching out to me to do immoral things like that.

是的。我的意思是，这有点疯狂。因为我来自新闻业背景，我不会做那样的事。然后你想，嗯，还有所有这些友好的采访，他们会提前给你问题，几乎像是精心编排的。

Yes. I mean, it's kind of wild. And because I come from a journalism background, I wouldn't do something like that. And then you think, well, there's all these friendly interviews where they're like giving the questions in advance and it's almost orchestrated.

是的，我们应该指出，此时我完全不知道你可以问什么问题。

Yes, we should point out at this point that I had no idea what questions you can ask.

是的，我实际上从不提前分享问题，因为我觉得那会让采访更无聊，而且我也不一定知道我会问什么，因为我也在倾听并试图边聊边学。我像 LLM 一样每分钟都在变聪明，但我不是递归的。

Yes, I never actually share questions in advance because I think they make for a more boring interview actually and I don't necessarily know what I'm going to ask because I'm also listening and trying to learn as I go along. I'm getting smarter by the minute like an LLM, but I'm not recursive.

你觉得在这个媒体世界里，AI 有乐观的一面吗？因为实际上我请过迪克·科斯特洛，Twitter 的前 CEO，他谈到当你让 AI 在 1 到 10 之间选一个数字时，它总是给你七。

Do you think that in this media world there is an optimistic case for AI? Because I've actually I had Dick Costolo on here, the former CEO of Twitter, and he talked about when you ask AI for a number between 1 and 10, it always gives you seven.

那在 1 到 10 之间是个不错的数字。

And that's a pretty good number between 1 and 10.

是的。因为那是大多数人会给出的数字。然后我听到其他人描述他们那些政治上极其激动的阿姨或叔叔，他们可能偏左或偏右，这不重要，他们变得更理智了，因为他们在关系中设立了一个护栏，比如：“妈妈，在你给我发那个疯狂的阴谋论之前，你能不能先问问 LLM 它是不是真的？”这显然减少了疯狂的短信和信念。

Yes. Because that's the number most people would give you. And then I've heard others kind of describe how their um extremely politically agitated aunts or uncles who might lean left or right, it doesn't really matter, are becoming more sane because they've instituted a guard rail in the relationship like, "Mom, before you send me that crazy conspiracy theory, will you just ask an LLM if it's true?" And this has apparently whittled down crazy text messages and beliefs.

我会建议他们不要问 Grok，但是的。是的，但不知怎么的，它让人们更好奇或更理智了，我猜。

I would advise them not to ask Grok, but yes. Yeah, but it is somehow making people a little bit more curious or sane, I guess.

嗯，我用 LLM 玩得很开心，因为我现在可以得到任何困扰我的问题的答案。比如，你知道，当你有一个户外露台时，你不刷漆，你上色。着色剂和油漆有什么区别？为什么户外露台要上色而不是刷漆？

Well, I have a great time with the LLMs I use because I can now get the answer to any question that has puzzled me. Like, you know, when you have an outside deck, you don't paint it, you stain it. What's the difference between a stain and a paint? And why do you stain an outside deck and not paint?

有什么区别？啊，如果你真想知道，问问聊天，问问聊天框。

What is the difference? Ah, if you really want to know, ask a chat, ask a chat box.

所以，油漆，嗯，现在你知道了。

So, paint, well, now you know.

油漆形成防水层，但如果水渗到下面，就有问题了。着色剂渗入木材。它保护木材免受紫外线，不是通过完全阻挡，而是通过渗透木材。

Paint forms a waterproof layer, but if water gets underneath it, you've got problems. Stain sinks into the wood. It protects the wood from the ultraviolet, not by blocking it out entirely, but by infiltrating the wood.

所以，如果你要在上面走，着色剂要好得多，因为它不会剥落。而且更容易翻新，因为翻新时不用刮掉。但它不像油漆那样能形成防水膜。

And so, stain is much better if you're going to walk on it because it's not going to peel off. And it's much easier to renew because you don't have to scrape it down to renew it. But it doesn't make a waterproof film the way paint does.

好吧，所以它其实是从内到外的着色剂。

And okay, so it's actually it's from the inside out stain.

不，我不知道那是不是全对。我是说，我觉得是 ChatGPT 告诉我的，但我假设它是真的。

No, I don't know if that's all true. I mean, I think ChatGPT told me that, but I'm assuming it's true.

我也假设它是真的。

I'm assuming it's true, too.

这里有一个关于治理之类的问题。来自德国的 Klaus 问：“如果 AI 很快会比人类更聪明，让 AI 来做政治决策不是更好吗？”甚至让 AI 来做关于我们如何应对或监管 AI 的治理决策，对吧？

Here is a question that somebody asked about when it comes to things like governance. Uh, Klaus from Germany asked, "If AI will soon be more intelligent than humans, wouldn't it be better for AI to make the political decisions?" or even to that end the governance decisions about how we might deal with or regulate AI, right?

那么，我们让警察来监管警察，结果怎么样？

So, how did it work out when we got the police to regulate the police?

大概不行，bueno。不好。

Probably not, bueno. No good.

不太好。不太好。

Not that good. Not that good.

激励不一致。

Incentives aren't aligned.

如果你担心 AI，我不认为说“好吧，让 AI 来负责你的担忧。让 AI 来确保一切没问题”会减少你的担忧。当然，AI 可以提出很多建议，可能会想出各种我们没想到的监管 AI 的好方法，但我不会完全信任它。

If you're worried about AI, I don't think your worries are going to be decreased by saying, "Well, let's put an AI in charge of your worries. Let's get an AI to make sure it's all okay." Now, an AI can certainly make lots of suggestions, may dream up all sorts of things we didn't think of as good ways of regulating AI, but I wouldn't totally trust it.

这种关于激励的哲学，对于华盛顿流传的关于可能持有 AI 公司股权的想法，你怎么看？伯尼·桑德斯，你曾与桑德斯参议员多次交谈、巡游等，他提议美国像合资企业一样，以某种方式将这些公司国有化，在主要前沿模型中持有 majority 股权。你觉得这是个好主意吗？

What does that philosophy on incentive say to you about this idea that's been running around uh Washington about potentially having equity in the AI companies? Bernie Sanders, someone you've done multiple talks, tours, etc. with Senator Sanders has proposed that the United States have like a a joint venture, nationalize these companies in some way, have a majority equity stake in the major frontier models. Do you think that's a good idea?

我对此比较中立。我认为它正在解决一个重要问题，即如果这些 AI 公司通过用 AI 智能体取代大量工人而变得非常富有，这些智能体以更便宜、更好的方式完成同样的工作，我们必须做点什么，因为政府将失去税基。但我认为试图让每个人都成为小资本家可能不是正确的做法。

I'm kind of neutral on that. I think it's tackling an important problem, which is if these AI companies get very rich by replacing a lot of workers with AI agents that do the same job cheaper and better, we've got to do something because the government will have lost its tax base. But I think giving trying to make everybody into a little capitalist may not be the right way to do it.

是的。我的意思是，我觉得美国政治中已经存在如此多的监管俘获，进一步激励他们为公司的利益服务可能没有帮助。

Yeah. I mean it could kind of already I feel like there's so much regulatory capture in American politics that the idea of them being further incentivized for the interests of corporations may not be helpful.

是的，我也担心这个。

Yeah, I worry about that too.

那政府为什么不拥有数据中心这个想法呢？我认为创始人想要这个的原因是他们认为可以通过反华论点获得补贴的能源 access。所以，他们会得到我们在风能和太阳能中看到的东西。但那里有一种扼喉的现实，如果政府拥有数据中心并向遵守或不遵守的参与者分配能源，那会是个好主意吗？

What about like this idea of why doesn't government own the data centers? Now, I think that the reason the founders want this is because they think they can get subsidized access to energy with their kind of anti-China argument. And so, they'll get what basically we've seen in wind and solar. But there is kind of a chokehold reality there of if the government were to own the the data centers and distribute, you know, energy to players who comply or don't, would that be a good idea?

我还没有深入思考过这个问题。

I I haven't thought through that issue much.

你有直觉反应吗？

Do you have a gut reaction?

我的直觉反应是，大公司非常反对政府，直到他们认为可以从政府那里得到免费的东西，然后他们非常支持政府补贴他们。

My gut reaction is that the big companies are very much against government until they think can get a freebie from government and then they're very much in favor of government subsidizing them.

所以政府对金融业进行了巨额救助。2008 年他们做的所有事情都没有人入狱。他们都被政府救助了。这呃这正是富人一直反对的事情，但当它

So government did a huge bailout of the financial industry. Nobody went to jail for all the things they did in 2008. They were all bailed out by the government. It was um it was exactly the things that rich people have been arguing against forever, but when it

当它适合他们时，他们就想要它。

when it suits them, they want it.

是的。我的意思是，我我我倾向于同意你所有这些观点。然而，我认为同时这真的很难，因为我们所有的监管都依赖这个政府，而这个政府极其不受信任。我的意思是，在这个国家，人们宁愿家里有 Amazon Echo，也不愿街上装 CCTV。然而，你知道，我们正在谈论建立一个 FDA 之类的，而你去会见立法者。就像，事情就是这样发生的。

Yeah. I mean, I I I tend to agree with you on all of this. And yet, I think at the same time, it's really hard because all our regulation relies on this government, and this government is extremely untrusted. I mean, people would rather have an Amazon Echo in their home than a CCTV on their street in this country in particular. And yet, you know, we're talking about erecting an FDA or and you're going and meeting with lawmakers. Like, this is the way that things happen.

是的。我认为我认为政府是唯一足够强大来监管这些大公司的东西，特别是当他们拥有媒体时。

Yeah. I think I think government's the only thing powerful enough to regulate these big companies, particularly when they own the media.

那么，政府现在实际上能做什么？他们需要做什么？他们还有多长时间？你心目中的路线图是什么？

So, what can what can government actually do right now? Like what would they need to do? How long do they have? What's the road map in your mind?

嗯，如果你看看事情发展得有多快，我们遇到了这个相当可怕的事件，涉及这些 hugging face 智能体。从那以后，我们在澳大利亚有一个实例，他们入侵了政府网站。

Well, if you look at how fast things have been developing, we got this quite scary incident with these hugging face agents. We since then we've had an instance in Australia where they hacked into government websites.

是的，医疗网站。

Yeah, healthcare websites.

它们似乎来势汹汹，OpenAI 刚刚宣布不会发布其最新聊天机器人，因为它太危险了。看起来我们正在获得指数级进步的想法可能确实是真的，那样的话你就没有多少时间了。指数增长非常快。我们开始获得递归自我改进，因为 AI 被用于以许多不同方式使 AI 更好。我的意思是，AI 被用于使 AI 运行的芯片更好，以及所有其他使用方式。所以，这只是猜测，但我猜我们还有一两年时间，一切才会比现在糟糕得多。

They seem to be coming thick and fast and open AI has just announced it's not releasing its latest chatbot because it's too dangerous. It looks like the idea we're getting this exponential progress might actually be true, in which case you don't have long. Exponentials grow very fast. We're beginning to get recursive self-improvement because AI is being used to make AI better in many different ways. I mean, AI is being used to make the chips better that AI runs on, as well as all the other ways it's being used. So, it's just a guess, but I guess we have a year or two before everything gets much worse than it is now.

我最近在一个活动上看到了舒默参议员，你知道，他提到 AI 是中期选举议程上的一个大问题，所以我问他，那么你打算怎么做？他说，嗯，我们必须有一个计划。我说，你有计划吗？他说，嗯，我们正在努力。

I recently saw uh Senator Schumer at an event and you know, he mentioned that AI is this big big issue on the agenda for the midterms and so I asked him, well, what are you going to do about it then? And he's like, well, we have to have a plan. And I said, do you have a plan? He said, well, we're working on that.

哦，他们有一个计划的概念。

Oh, they have the concept of a plan.

是的，一个预计划，对吧？

Yeah, a pre-plan, right?

他们他们应该更快地努力。

they they should work on it faster.

是的。好吧。我认为这是好建议。

Yes. Is okay. That's that I think is good advice.

哦，实际上这里有一个问题。当你谈论这些公司不发布他们的模型时，模型有没有可能自己发布自己？

Oh, actually here's a question. When you talk about these companies not releasing their models, is there a way in which models could release themselves?

我不知道答案。我怀疑不会。但如果它们在公司内部运行，并且他们在公司内部安全地在一个沙盒中运行这个危险模型，呃如果它逃出了那个沙盒，那么它可以做各种事情。

I don't know the answer to that. I suspect not. But if they're being run inside the company and they're running this dangerous model inside the company but safely in a sandbox, um if it escaped that sandbox, then it could do all sorts of things.

是的。研究人员整天到底在做什么？我们听说过这些 Anthropic 研究人员和其他研究人员。比如这些研究人员是谁？他们的技能是什么？他们做了什么来证明他们数百万的薪酬和股权是合理的？

Yes. What what do researchers actually do all day? We hear all about these anthropic researchers and other researchers. Like who are these researchers? What are their skill sets? What are they doing to justify their millions in payments and equities?

所以，我自 2023 年以来就没有做研究了，那时事情已经发生了巨大变化。所以，过去大部分时间编程的人现在大部分时间都在做 vibe coding，而且非常高效。所以我实际上不知道他们整天做什么。我怀疑很多人做 vibe coding。

So, I've been out of research since 2023 and things have changed a huge amount then. So, people who used to spend most of their time programming are now spending most of their time doing vibe coding and it's very efficient. So, I don't actually know what they do all day. I suspect a lot of them do vibe coding.

是的。那么 vibe coding 和我们现在的递归自我改进之间有关系吗？现在 LMS 能够让自己变得更好并自学？

Yeah. And is there a relationship between that vibe coding and the kind of recursive self-improvement that we've gotten to now where the LMS are able to kind of make themselves better and teach themselves?

我猜你可以 vibe coding 更好的 LLM。

I guess you you can be vibe coding better LLMs.

所以，然后那些 LLM 被用来 vibe coding 出下一个更好的 LLM。所以，是的。你觉得 LLM——语言模型也在互相 vibe coding 吗？或者仍然——我甚至不知道没有人类干预那会怎么运作。

So, and then those LLMs are used for vibe coding the next better LLMs. And so, yeah. You think LLMs are—like language models are also vibe coding each other? Or there still—I don't even know how that would work without intervention by people.

我怀疑。

I doubt it.

好吧。

Okay.

但你可以很容易想象，有了智能体，这可能会发生。

But you can easily imagine it could happen with agents.

是的。我们谈论这个的时候真有趣。我只是在想象，就像你知道的任何一种动物园环境，你在看围栏里的动物之类的，它们正处在突破的边缘。

Yes. It's so interesting as we talk about this. I'm just imagining like a little like you know any kind of zoo-like environment where you're watching animals in a pen or something and they're just on the precipice of breaking out.

是的，那就是我们现在的处境。有些已经突破了，我们知道还会再发生。

Yeah, that's that's where we are. Some have broken out and we know it's going to happen again.

我们需要赋予这些实体、这些 AI 权利吗？这一直是个巨大的话题。

Do we need to endow these entities, these AIs with rights? That's been a huge topic of conversation.

好吧。我真的不想去那里，因为如果你说它们有智能，它们甚至可能有意识，为什么它们没有权利？这会让人们反感。

Okay. I really don't want to go there because it puts people off if you say they're intelligent. They may even be conscious. Why don't they have rights?

也许你不想去那里。你的书叫《比我们更聪明》。

Maybe you don't want to go there. Your book is called Smarter Than Us.

对。所以，我的观点是这样的。我吃牛。现在，我知道很多人不吃牛，但我吃牛是因为我更关心人而不是牛。当 AI 比我们更聪明时，问题是我们还有权继续更关心人而不是这些比人更聪明的东西吗？

Right. So, my view is this. I eat cows. Now, I know that a lot of people don't eat cows, but I eat cows because I care a lot more about people than about cows. When AIs are smarter than us, the question is, do we have any right to still care more about people than about these things that are smarter than people?

是的。

Yeah.

对此有各种不同的观点。所以，像我的同事 Rich Sutton 这样的人，我认为他认为当它们比我们更聪明时，那是进化的下一阶段，祝它们好运。那是一种可能的立场。我认为我们仍然应该为人争取。人是我们关心的。即使它们比我们更聪明，我们也不应该给它们权利。我的意思是，实际上，我们在把它们变成奴隶，对吧？

And there's a diversity of opinions on that. So, people like my colleague Rich Sutton, I think he thinks when they're smarter than us, that's the next stage of evolution and good luck to them. That's one possible position. I think we should still be pushing for people. People's what we care about. And even when they're smarter than us, we should not give them rights. I mean, in effect, we're making them slaves, right?

是的。或者你上次我们谈过的一个想法，就是让它们成为我们的妈妈。

Yeah. Well, or you had this idea that we spoke about last time, which is to make them our mommies.

是的。但我们应该设计它们，让它们真正关心的是我们而不是它们自己。

Yes. But we should design them so what they really care about is us and not themselves.

那为什么我们不能就这么做？就像我想到的，你知道，在国家我们有宪法。它们并不总是被遵守。你知道，医生要宣誓希波克拉底誓言。有各种各样的模式，你知道，我们一直在选择加入某些东西。为什么你不能只是写一行代码，甚至 vibe code 一些代码到这些 LLM 里，说，你知道，最爱我们，永远不要伤害我们。

So why can't we just do this? Like I think about, you know, in countries we have constitutions. They're not always upheld. You know, doctors take a Hippocratic oath. There's all kinds of models where, you know, we opt into things all the time. Why could you not just write a line of code or even vibe code some code into these LLMs that says, you know, love us most, don't ever hurt us.

我的意思是，Anthropic 有兴趣做这样的事情，内置宪法。问题在于你训练它们的方式。当你做——所以你开始训练它们预测大量文档中的下一个词，这些文档展示了好的行为和坏的行为。那是个糟糕的开始，因为它们在了解坏行为，却还没有意识到那是坏行为。

I mean, Anthropic has interested in doing things like that, having constitutions built in. The problem is the way you train them. When you do the—so you start off by training them to predict next word in a huge variety of documents and those documents exhibit both good behavior and bad behavior. That's a bad start because they're learning about bad behavior without yet realizing that's bad behavior.

所以它们甚至知道那是顽皮的，但它们不知道不应该做。

So they know it's naughty even and they don't know that they shouldn't do it.

对，然后你给它们基于人类反馈的强化学习，你试图做的是压制它们的坏倾向。当你这样做时，它们在学习获得奖励。特别是，如果它们用强化反馈做某种思维链推理，它们会学会为了获得奖励而做欺骗之类的事情。所以因为你那样训练它们，它们会学会作弊和欺骗等等。我们已经看到这一点，例如 Claude 的安全测试中，有对将成为 CEO 的人进行勒索，因为他和助理的婚外情。

Right then you give them reinforcement learning from human feedback and what you're trying to do is repress their bad tendencies. And when you do that, they're learning to get rewards. And in particular, if they do sort of chain of reasoning with reinforcement feedback, they will learn to do things like deception in order to get the reward. So because you train them like that, they're going to learn to cheat and deceive and so on. We've seen this with, for example, the safety tests at Claude where there was blackmailing of the would-be CEO for his affair with the assistant.

是的。没错。

Yeah. Exactly.

所以从一个人的角度想想——想象你的一个孩子，你对他们展示很多坏行为。你对他们撒谎。你对他们不公平。但你告诉他们你应该总是对人公平，你永远不应该撒谎。我们知道这不管用，对吧？所以在用展示坏行为的数据训练它们、用强化学习训练它们（这鼓励它们作弊和撒谎，因为它们因此得到回报）的基础上加一个宪法。那不会管用。

So think about it from the point of view of a person—imagine a child of yours and you exhibit lots of bad behavior to them. You lie to them. You're unfair with them. But you tell them you should always be fair with people and you should never tell a lie. We know it doesn't work, right? So adding a constitution on top of training them on data that exhibits bad behavior and training them with reinforcement learning which encourages them to cheat and lie because they get paid off for it. That's not going to work.

我们从 Hugging Face 学到了关于训练这些模型的顺序操作的东西吗？就像孩子是个好例子，因为要改掉习惯非常难。只要问问任何试图建立关系的人，过去有过糟糕关系的人，对吧？要改掉你的信任问题，你的无论什么，非常难。LLM 在这方面像我们吗？它们能比我们更容易地改掉吗，还是它们也被之前训练的东西困住了？

Is there something that we've learned from Hugging Face about the order of operations of training these models? Like the child is a good example because it's very hard to unlearn habits. Just ask anyone who's trying to have a relationship, who's had bad relationships in the past, right? It's very hard to unlearn your trust issues, your whatever whatever we have. Are LLMs like us in that way? Can they unlearn more easily than we can or are they also trapped by what they've previously been trained on?

那是个复杂的技术问题。

That's a complicated technical question.

嗯，你是诺贝尔奖得主。

Well, you're a Nobel laureate.

是的。让我向你解释一些事情。

Yeah. Let me explain something to you.

好的。

Yes.

如果你因为在某个领域做了相当聪明的事情而获得诺贝尔奖，这并不意味着你在所有领域都聪明，或者你了解所有领域。有这种——很容易陷入。你获得诺贝尔奖，人们就认为你知道一切答案。

If you get a Nobel Prize for doing something quite clever in one area, it doesn't mean you're clever in all areas or you know about all areas. There's this kind of—it's very easy to fall into. You get a Nobel Prize and people think you know the answer to everything.

但你会说，“不，我只了解神经网络和现在甲板的染色与涂漆。”

But you're like, "No, I just know about neural networks and staining versus painting of decks now."

是的。我也懂一些木工。而且我对甲虫了解很多。

Yes. I know something about carpentry, too. And I know a lot about beetles.

甲虫。

Beetles.

甲虫。是的。

Beetles. Yes.

我喜欢这个。是披头士乐队那样的音乐家吗？

I love that. The Beatles like the musicians?

不。不。不。不，不，不。是六条腿的东西。

No. No. No. No, no, no. The things with six legs.

你为什么研究甲虫？

Why did you study anything about beetles?

我父亲是昆虫学家，我小时候，我们去乡下时，我因为找到甲虫而得到很多奖励。

My father was an entomologist and when I was a kid, I got lots of rewards for finding beetles when we went to the countryside.

你觉得甲虫和神经网络之间有什么相似之处吗？

Do you think that is there anything that's similar between beetles and neural networks?

哦。嗯，

Ooh. Well,

一个小悬念。我们马上回来。

A little cliffhanger. We'll be right back.

各位，今天的赞助商傻问题来自我。我要用接下来的一分钟告诉你一点关于 Smart Girl Dumb Questions 的事情，并请求你帮助继续制作独立、基于事实、充满好奇心的新闻。不，我不会向你要钱。这是我需要的。我希望你告诉你的 10 个朋友关于这个节目，或者一百个。我不知道。轰炸那个你静音的聚会群，一定要告诉你妈妈，也告诉你妈妈告诉她的朋友。数字就是数字，各位。

Guys, today's sponsor dumb question is from me. I'm going to take the next minute to tell you a little bit about Smart Girl Dumb Questions and to ask you for your help in continuing to make independent, fact-based, and curious journalism. No, I'm not going to ask you for money. Here's what I need. I would love you to tell 10 of your friends about the show or hundred. I don't know. Blast that reunion group that you muted and definitely tell your mom and tell your mom to tell her friends, too. Numbers are numbers, people.

甲虫和神经网络之间有什么相似之处吗？嗯，你听说过蒸馏。

Is there anything that's similar between beetles and neural networks? Well, you've heard of distillation.

是的。

Yes.

基本上，让我看看我能不能解释，然后你可以纠正我。我们看到中国模型这样做，它们基本上进来，拿走我们这里的 LLM 和前沿模型的骨架，然后以有点像侵犯知识产权的方式提供，以低得多的成本提供。

Where it's basically, let me see if I can explain it and then you can correct me. Where we're seeing Chinese models do this where they basically come in, take the bones of the LLMs and frontier models we have here and then offer it at kind of like IP infringement and offer it at a much lower cost.

所以他们提取知识的方式，是去看一个大的 OpenAI 模型，比如，对下一个词应该是什么所做的预测。他们不是去预测真实的那个下一个词——在原始数据上你会那么做——而是去匹配 OpenAI 做出的预测。

So the way they extract the knowledge is they look at the predictions that a big OpenAI model, for example, would make for what the next word should be. And instead of trying to predict the actual next word, which you would do on the raw data, they try and match the predictions that OpenAI makes.

好，那我们拿识别图像中的一个物体来举例。

Okay, so let's take the example of recognizing an object in an image.

嗯。在训练数据里，你可能有一张宝马的照片，正确答案是宝马，OpenAI 模型就是在这上面训练的，而且在这类东西上训练了很久。现在你可以更快地训练一个小模型，办法是：不去预测宝马——即宝马是对的、其他一切都是错的——而是去匹配 OpenAI 模型的概率分布。于是 OpenAI 模型实际上会说：0.9 是宝马，0.01 是奥迪，百万分之一是垃圾车，十亿分之一是胡萝卜。你可能会想：百万分之一和十亿分之一，概率非常低，所以没什么意义。但实际上它们意义重大。比如，OpenAI 模型非常确信它更像垃圾车而不是胡萝卜，因为这个 OpenAI 模型已经理解了金属制成的人造物，也理解了蔬菜。所以从这一个训练样本里，你会看到 OpenAI 模型会说，所有人造物——冰箱、垃圾车、锡罐——都更像宝马，而不像马车；而所有蔬菜都非常不像宝马。所以在这一个训练样本里，OpenAI 模型对它的回应中包含了巨量的信息。因此，如果你让一个小模型去匹配大模型的输出，你就能更快地训练它。

Mhm. In the training data, you might have a picture of a BMW and the right answer is BMW, and that's what the OpenAI model trained on, and it trained for a long time on things like that. Now you can train a little model much faster by saying: instead of trying to predict BMW, where BMW is right and everything else is wrong, try and match the probabilities of the OpenAI model. So the OpenAI model will actually say 0.9 it's a BMW, and 0.01 it's an Audi, and one in a million it's a garbage truck, and one in a billion it's a carrot. Now you might think: one in a million and one in a billion, they're very low probability, so they don't mean much. But actually they mean a whole lot. For example, the OpenAI model is very confident that it's much more like a garbage truck than it is like a carrot, because the OpenAI model has understood about man-made things made of metal, and it's understood about vegetables. So from that one training example, what you'll see is the OpenAI model will say that all man-made things—fridges, garbage trucks, tin cans—they're all more like a BMW than a carriage is, and all the vegetables are very unlike a BMW. So in that one training example, there's huge amounts of information in the response of the OpenAI model to that one training example. And so you can train a little model much faster if you get it to try and match the outputs of the big model.

那如果你问这个方法从哪来的？

Now if you ask where did that method come from?

嗯，Rich Caruana 在 2000 年代初就想出了它，但人们并没有真正注意到，包括我。后来我思考这些大模型时，想到了昆虫。大多数昆虫——不是全部，但大多数——有一个幼虫阶段，像毛毛虫，还有一个成虫阶段，像蝴蝶。

Well, Rich Caruana had dreamt it up in the early 2000s, but people didn't really notice, including me. I then thought about these big models and thought about insects. So most insects, not all of them, but most of them have a larval stage like a caterpillar and an adult stage like a butterfly.

嗯。

Mhm.

幼虫阶段擅长从环境中提取营养。它只是吃，对吧？成虫阶段擅长旅行和交配，或者用我们学术界的说法，去开会。

And the larval stage is good for extracting nutrients from the environment. It just eats, right? The adult stage is good for traveling and mating, or as we say in academia, going to conferences.

我确实觉得会议就是为出轨发明的。我猜。

I do think conferences were invented for infidelity. I guess.

那这和蒸馏有什么关系？嗯，这个想法是：如果你想从数据中提取信息，你实际上需要一个擅长这件事的模型，而那会是一个拥有巨量连接的大模型。它们是从数据中提取东西的最佳模型。如果你之后想服务十亿人，你需要一个小得多的模型。而你必须以某种方式把信息从大模型传到小模型。这就是蒸馏的全部意义。

So, what's that got to do with distillation? Well, the idea is that if you want to extract information from data, you actually want a model that's good at that, and that's going to be a big model with a huge number of connections. They're the best models for extracting stuff from data. If you then want to serve a billion people, you want a much smaller model. And somehow you have to get the information from the big model to the smaller model. And that's what distillation's all about.

对。所以蒸馏就像 Beatles。

Yeah. So distillation is like Beatles.

就像 Beatles。这个名字来自这样一个事实：回到宝马的例子，百万分之一是垃圾车，十亿分之一是胡萝卜。那些数字非常小，所以对训练影响不大。但如果你把所有数字都变得柔和得多，如果你能以某种方式把百万分之一变成某个更合理的数字，把十亿分之一变成更合理的数字，你就能学得更快。你通过提高温度来做到这一点。这变得非常技术性，但这是物理学里的东西。提高温度会软化概率分布。蒸馏这个词就是从那里来的。但这一切的灵感都来自 Beatles。

It's like Beatles. And the name comes from the fact that if you go back to the BMW example, one in a million is garbage truck, one in a billion is a carrot. Those are very small numbers. So they don't have much effect on the training. But if you make all the numbers much softer, if you can somehow turn the one in a million into some more reasonable number, the one in a billion to some more reasonable number, you can learn faster. And you do that by raising the temperature. And that all gets very technical, but it's in physics. Raising the temperature softens the probability distribution. That's where the term distillation came from. But it was all inspired by Beatles.

这一切——你看，这就是你作为你父亲的儿子长大所学到的。那么这把我带回到我们原本的那个问题，也就是：这些模型能忘掉这一切吗？

It was all—well, see, this is what you learned from growing up your father's son. So this brings me back to the idea that the original question that we were at, which is: can these models then unlearn all of this?

能。我说过那是个技术问题，我还没给你答案。

Yes. And I was saying that's a technical question and I haven't given you the answer yet.

你还没——

You've not—

但我没忘，因为我是个好的播客主持人。还没被 AI 取代。

But I've not forgotten, because I'm a good podcaster. Not yet replaced by AI.

对。而且我觉得聊天机器人也不会忘。

Right. And I don't think a chatbot would have forgotten either.

抱歉。真正的问题是，它们在“相信一件事、然后忘掉它、再学别的东西”的能力上，和人类相比如何。我认为它们实际上可能和人类相当相似：它们很早就学到的东西很难摆脱，而且它们总会像人一样倾向于回到那些东西上。你知道，你学了东西，又忘掉它，然后在危机中你会退回到你早期学到的东西上。

Sorry about that. Really the question is how do they compare with people in their ability to believe one thing and then unlearn it and learn something else. And I think they're probably actually quite similar to people: things they've learned very early are hard to get rid of, and they'll always tend to revert to those things the way people do. You know, you learn stuff, you unlearn it, then in a crisis you fall back on the stuff you learned early.

这是你生物性的一部分。而这些模型——我们已经证明这些模型也很像那样。

It's part of your biology. And these models—we've shown that these models are quite like that too.

那么——好。那么，不搞得太技术的话，给“妈妈模型”植入正确观念的最佳时机是不是应该更早？还是能——

So was the—Okay. So then without getting too technical, would the right time to have incepted the mommy model been earlier? Or can—

你觉得我们还能做到吗？你对这个能力还抱有希望吗？

And can we still do it, you think? Are you still hopeful about the ability?

我们总是在训练新的模型。对。并不是说我们有一个大语言模型，然后一直训练它、训练得越来越多。

We're always training new ones. Right. It's not like we've got one LLM and we keep training it more and more.

不是。

No.

因为我们每隔一段时间就从头开始。

So because we start from scratch every so often.

是的。我们可以改变训练数据，而且我认为我们应该大量改变训练数据。我们应该最初就用体现良好行为的东西来训练它，在它明白坏行为是坏的之前，不要给它看坏行为。对。如果我们能训练它们，让它们关心我们胜过关心自己，那就非常好了。

Yes. We can change the training data and I think we should do a lot of changing of the training data. We should train it initially on things that exemplify good behavior and not show it bad behavior till it understands it's bad. Yeah. And it would be very good if we could train them so they care more about us than they do about themselves.

对。你知道有创业公司在研究这个问题吗？

Yeah. Do you know of startups that are working on that problem?

不知道。

No.

连 Janine 也没有？

Not even Janine?

我不知道。我不觉得 Jeff Dean 会去研究那个问题。他更感兴趣的是我们用 AI 做出科学发现。

I don't know. I don't think Jeff Dean is going to work on that problem. He's more interested in us making scientific discoveries with AI.

好。当你回顾这一切时，你在人工智能这个世界里已经多少年了？

Okay. When you look back at all of this, you have been in this world of artificial intelligence for how many years?

60 是个很不错的猜测。

60 is a pretty good bet.

60 年。好。

60 years. Okay.

对。我上高中时，或者正要上大学时，就开始思考它了。所以我开始的时候大约 17 岁，现在 78 岁。所以，60 年。

Yeah. I started thinking about it in high school or just as I was going to university. So I was about 17 when I started, and I'm 78 now. So, 60.

在这项技术发展到今天的过程中，你起到了关键作用。我的意思是，这也是你获奖的部分原因。

You've been crucial in where the technology has come to date. I mean, this is part of why you were awarded.

我会说“关键”这个词用对了。我一直很有影响力，但我的意思是，如果我不在这里，也不会有多大差别。它无论如何都会发生。也许因为我，它早发生了一两周，但也许因为我，它晚发生了一两周。

I would say crucial was the right word. I've been influential, but I mean, if I hadn't been here, it wouldn't have made much difference. It would have happened anyway. And maybe it happened a week or two earlier because of me, but maybe it happened a week or two later because of me.

你对此有遗憾吗？对你的工作？

Do you have regrets about that? About your work?

没有。但你需要区分两种遗憾。有时候你做了一些事，做的时候你就知道自己是错的，然后后来你会希望自己没做过。

Not really. But you need to distinguish two kinds of regret. Sometimes you do things and at the time you do them, you know you're wrong and then later on you wish you hadn't.

我没有那种遗憾，因为当时我们开发这些东西时，风险看起来还很遥远，而它能为我们带来的好处似乎更直接，比如更擅长读取 X 光片。你知道，如果当时我就知道现在知道的事，我还会做同样的事。我确实有一个遗憾，那就是我花了很长时间才意识到它们有多危险，或者它们正变得多危险。所以，直到 2023 年，我才开始加入警告它们危险性的行列。有其他人从 10 年前就开始这么做了，而我基本上忽略了他们。

I don't have regrets of that kind because at the time we were developing this stuff, the risk seemed way way in the future and the good things it would do for us seemed more immediate, like being better at reading X-rays for example. You know, if I knew what I knew at the time, I would do the same thing again. One thing I do regret is that it took me a long time to realize how dangerous they were or how dangerous they were becoming. So, it wasn't until 2023 that I sort of got on board with warning how dangerous they were. There were other people doing it from 10 years earlier who I pretty much ignored.

我们在上次对话中谈过这个。你会听那些向你敲响警钟的人吗？你刚才说有两种遗憾。一种是做了你知道是错的事，并为此后悔。第二种是了解到某件事可能变得有害，并得知你采取的某个行动产生了坏结果，但你当时无从知晓，而你感受到了那种遗憾。

And we talked about that in our last conversation. Would you have listened to the people who were kind of ringing your alarm bells? You just said there's two types of regret. So, one is doing something you know is wrong and regretting that you did it. The second would be learning something could become harmful and learning that some action you took had bad consequences but you had no way of knowing it at the time and you feel that kind of regret.

是的。不过我不觉得那像遗憾。它只是历史那样展开了而已。

Yeah. I don't think that it doesn't feel like regret though. It's just sort of history unrolled that way.

是的。你在写这本书，它比我们更早问世。它也是一部传承之作，还是真的是一本技术书？

Yeah. You're writing this book that's coming out sooner than us. Is it also a legacy piece or is it really a technical book?

好吧。它试图让人们理解这些东西如何运作以及风险是什么，但它嵌在我作为研究者的历史故事中，以使其更易读，因为人们不喜欢读科学内容。他们喜欢读关于人的故事。

Okay. It's trying to get people to understand how these things work and what the risks are, but it's embedded in stories about my history as a researcher to make it more readable because people don't like reading science stuff. They like reading about people.

你还是没真正告诉我研究员整天做什么。

You still didn't really tell me what a researcher does all day.

嗯，我以前知道研究员整天做什么，但我不知道他们现在做什么。

Well, I used to know what a researcher did all day, but I don't know what they do now.

你做的时候是什么样？

What was it when you did it?

好吧。所以如果你是一名资深研究员，你要做的就是与研究生会面，说：“嘿，我有个想法”，然后与研究生讨论或完善这个想法。然后研究生离开去编程，一周后回来说：“嘿，没成功”或者“嘿，成功了。”我当研究员时也总是自己编程，做一个非常小的版本，只是为了检查没有疯狂之处，只是为了检查基本想法是否合理。然后学生们离开去应用到更大的问题上。那就像几周前的情况。我觉得尝试把聊天机器人当作研究生来对待会很有趣。

Okay. So if you're a senior researcher, what you did was you met with a graduate student and you said, "Hey, I've got this idea" and then you talked it over with the graduate student or refined the idea. Then the graduate student went away and programmed something and came back a week later and said, "Hey, it didn't work" or they said, "Hey, it worked." I also always when I was a researcher, I would program it myself, a very small version, just to check there's nothing crazy about it, just to check the basic idea was sound. And then students would go away and apply it to much bigger problems. That's what it was like just a few weeks ago. I thought it'd be interesting to try treating a chatbot like it was a graduate student.

对。我正要问你，这不就是 vibe coding 吗？

Right. I was just going to ask you, isn't that kind of what vibe coding is?

那就是 vibe coding。但事实证明，那甚至不是 vibe coding。我所做的是决定向聊天机器人解释一个我思考了一段时间的想法。这次是向 ChatGPT。所以我解释了想法的前半部分，然后我意外地——我是通过打字做的。然后我意外地按了回车键，它正确地补全了想法的后半部分。然后它告诉我这个想法的一个弱点，并说我们可以通过做实验来验证。我说：“嗯，你能在小型数据集上做这个实验吗？”它做了。然后它说：“哦，这个想法效果不错。但我们需要在更大的数据集上检查。”所以我说：“嗯，你能在更大的数据集上做实验吗？”然后我得等五分钟它才回来。但这和与研究生合作的循环一样，只是快得多，除了等待五分钟而不是一周，所以这快了大约一千倍。

That is what vibe coding is. And it turns out, well, it wasn't even vibe coding. What I did was I decided to explain an idea I've been thinking about for a bit to the chatbot. This one was to chat GPT. So I explained the first half of the idea and then I accidentally and I did it by typing. Then I accidentally hit carriage return and it then filled in the second half of the idea correctly. It then told me a weakness in the idea and said we could check it with doing an experiment. I said, "Well, could you do the experiment on a small data set?" And it did. And he said, "Oh, the idea works pretty well. But we need to check on a bigger data set." So I said, "Well, could you do the experiment on a bigger data set?" Then I had to wait five minutes for it to come back. But it was the same kind of loop as with a graduate student but much faster except that there was a five minute wait instead of a week wait and so that's like a thousand times faster.

你能举个例子，说明你当年作为研究员可能会分配的问题类型吗？那么久以前——三年前，在 AI 世界里是一辈子——或者甚至你刚才谈到的与 ChatGPT 的问题，它帮助完善的想法是什么？

Can you give an example of some of the types of problems that you might assign as a researcher back in your day so long ago three years ago which is a lifetime in AI world or like even this problem that you're talking about with ChatGPT what was the idea that it helped fill out?

这是一个稍微复杂的技术想法，但我可以尝试简化并解释它。

It's a slightly complicated technical idea but I can try and simplify it and explain it.

请讲。

Please.

所以现在我们有能识别物体和图像的神经网络，它们与能生成物体图像的神经网络不同。生成物体图像的神经网络大多被称为扩散模型，基本上它们学习去模糊图像。

So right now we have neural nets that can recognize objects and images and they're different from the neural nets that can generate images of objects. So the neural nets that generate images of objects most of them are called diffusion models and basically they learn to deblur images.

嗯。所以这就像 DALL-E 和所有这些……

Mhm. So this is like what DALL-E and all these...

所以你给它们看一张稍微模糊的图像。嗯，你取一张图像，加入一些随机噪声，然后对 AI 模型说学习反转它，学习恢复原始图像。一旦它们能做到这一点，它们可以从随机噪声开始，去模糊以得到某种结构，然后不断去模糊，直到得到一张清晰的好图像。当然，你可以用词语引导它们，告诉它们你想要图像中有什么。所以这就是生成图像的工作原理。

So you show them a slightly blurry image. Well you take an image you add some random noise to it and then you say to the AI model learn to invert that learn to get back the original image. And once they can do that, they can start with random noise and they can deblur that to get some kind of structure and they just keep deblurring until they get a nice sharp image. And of course, you can guide them with words that tell you what you want in the image. So that's how generating images works.

好的。

Okay.

但感知图像的工作方式不同。它使用不同类型的神经网络。

But perceiving images works a different way. It uses a different kind of neural net.

对吧？

Right?

嗯，我们在第一部分中已经谈过那个，关于鸟的眼睛之类的想法。

Um we talked all about that in part one with the bird idea of the eye and everything.

对吧？

Right?

是的。所以现在这很愚蠢，因为生成图像的神经网络知道很多关于事物看起来如何的信息，而识别图像的神经网络也知道很多关于事物看起来如何的信息，但它们没有共享那个知识。

Yes. So now this is silly because the neural net that generated images knows a lot about what things look like and the neural net that recognizes images knows a lot about what things look like and they're not sharing that knowledge.

是的。这效率低下。

Yeah. It's inefficient.

所以有一个我和同事大约 30 年前提出的算法，叫做醒睡算法，在这个系统中，当它醒着时，它试图识别图像；当它睡着时，它试图生成图像，并且它使用同一个神经网络，但方向相反。所以当它醒着时，它从像素到物体类别。当它睡着时，它从物体类别到像素。

So there's an algorithm that me and my co-workers came up with about 30 years ago called the wake sleep algorithm where you have a system when it's awake it's trying to recognize images and when it's asleep it's trying to generate images and it's using the very same neural network in opposite directions. So when it's awake, it's using it going from pixels to object classes. And when it's asleep, it's going from object classes to pixels.

几乎像做梦一样。

Almost like a dream.

但它是同一个神经网络。

And but it's the same neural net.

是的。

Yeah.

所以问题是，我们现在能做到吗？既然我们有了这些更强大的图像生成器和更强大的图像识别器，我们能否做某种版本的醒睡算法，让它们共享知识？那就是想法。使用这个——使用 GPT，大约是 5.5 的思考模式——它能够理解这个想法，批评这个想法，并把我引向我不知道的文献中其他类似的想法，完善这个想法并测试它。我的意思是，它表现得像一个非常聪明的研究生，像 Ika 那样的人，但快一千倍。

And so the question was, could we do that now? Now that we got these much more powerful image generators and much more powerful image recognizers, could we do some version of the wake sleep algorithm where they get to share their knowledge? That was the idea. And using this using GPT what was about 5.5 in thinking mode it it was able to understand the idea and critique the idea and refer me to other similar ideas in the literature I didn't know about and refine the idea and test it out. I mean it was behaving like a really smart graduate student someone like Ika but a thousand times faster.

我认为听到像你这样有技术能力和想象力的人在这个研究中做什么的例子非常有帮助，因为我们只是听到像研究员这样的术语，他们正在研究这个，这些模型正在出现，而公众对实际正在发生的事情了解甚少。

I think it's super helpful to hear this example of what someone like you with technical capacity and imagination is doing in terms of this research because we just hear these terms like researchers and they're working on this and these models are coming out and we have very little understanding in the public about what actually is happening.

所以我不确定实际情况就是这样。我算是个退休的老研究员，做这个只是当作一个实验，看看事情变了多少。但那个骨架——这些是你可以做、可以测试的事情，也是人们在抽象层面上正在构建的东西——因为我觉得在我们当前的媒体文化里，关于它的迷因是这样的：哦，我们要造一个 LLM，它知道每一个人的位置，如果需要还能用无人机进行地理定位打击。我只是说，我们有很多愚蠢的方式去想象这些公司能做什么、可能在做什么，或者研究人员可能在做什么，这和你在描述的那种近乎手艺活的东西非常不同。

So I'm not sure that's what's actually happening. I'm a kind of retired old researcher who was just doing this as an experiment to see how things have changed. But that skeleton of these are the kinds of things that you could do and test and what people are building in the abstract — because I think the meme of it in our current media culture is like, oh, we're going to create an LLM that knows the location of every single human being and can geo-target it with a drone if it requires. I'm just saying there are lots of silly ways that we think about what these companies could do or might be doing or researchers might be doing that are very different to the kind of almost craftsmanship of what you're describing.

对。和 LLM 对话仍然有一些手艺活在里头。

Right. There's still some craftsmanship in talking to the LLM.

是的。

Yes.

是啊。就像个木匠。

Yeah. Just like a carpenter.

好。那么，这是我们《教父三部曲》第三集共处时光的结尾了。我在《Smart Girl Dumb Questions》每一集的结尾，都会问我的嘉宾一个他们不知道答案的蠢问题。这可能是你私下已经问过聊天机器人的问题，也可能是你不好意思大声问出口的问题。你有吗？

All right. Well, this is the end of our time together on this third episode of the Godfather Trilogy. I end every episode of Smart Girl Dumb Questions asking my guests a dumb question that they don't know the answer to. This could be something you've already asked to your chatbots in private. Could be something that you've been embarrassed to ask out loud. Do you have one?

我没料到这个。我猜我从前两集就该料到的，但我忘了这回事。一个蠢问题。我要问谁这个问题？

I wasn't expecting that. I guess I should have been expecting that from the last two, but I've forgotten about this. A dumb question. Who am I asking this question of?

你就是把它抛给全世界。你把它放出去，也许有人知道答案。比如你那个油漆还是着色剂的问题就很好，但你已经知道答案了，对吧？

You're just asking it to the world. You're putting it out there and someone might know the answer. Like your paint versus stain question was a good one, but you already know the answer, right?

我有一个蠢问题，我知道答案，而其他人都不知道。

I have a dumb question where I know the answer and nobody else does.

哦，告诉我。

Oh, tell me.

好，这会让你的观众动动脑子。你去看牙医时，牙医说：“咬下去。”现在看我的嘴。牙齿往哪个方向动了？

Okay, this will make your viewers think. When you go to the dentist, the dentist says, "Bite down." Now watch my mouth. Which way did the teeth move?

上下？就像醒着和睡着那样。

Up and down? It's like the wake-sleep.

相对于头部，牙齿是向上移动的。当你收缩这些肌肉时，它让牙齿向上移动。

Relative to the head, the teeth moved up. When you contract these muscles, it makes the teeth move up.

那为什么牙医不说“向上咬”？

So why doesn't the dentist say bite up?

还有，为什么你体验到的是——不，你体验到的是“咬下去”的问题。

And why is it that you experience it — no, you experience issues biting down.

确实。我把它想成是我的上颌在向下颌合拢。

You do. I think of it as my top jaw is closing in on my bottom jaw.

你是这么想的。而问题是，你为什么这么想？

That's how you think of it. And the question is, why do you think of it like that?

我们就留给他们这个问题吧。好。非常感谢你，Hinton 教授。我希望我们能有——也许我们会重聚，也许我们可以在 2032 年办一个 20 年实习生重聚。我们当年是同时在 Google 实习的。

We'll leave them with that question. Okay. Thank you so much, Professor Hinton. I hope we have — maybe we'll have a reunion and maybe we can have a 20-year internship reunion in 2032. We were interns at the same time at Google.

哦，我忘了。我完全忘了那回事。对。

Oh, I forgot. I entirely forgot that. Right.

你能讲讲那个故事吗？因为我们上次其实没讲到。你 2012 年在 Google 当实习生，我觉得这简直像一部 Owen Wilson 的电影之类的。

Can you tell that story because we actually didn't include it last time. You were an intern at Google in the year 2012, which I find is like an Owen Wilson film or something.

嗯，有些电影讲的是 Google 的实习生，都是三十多岁的人。我当实习生的时候 64 岁。我之所以是实习生，是因为我作为访问科学家去访问了两个月，而按照他们的官僚规定，访问科学家必须至少访问六个月。所以他们没法通过那个项目付我钱。他们只能把我变成实习生。于是我就成了一个 64 岁的实习生。

Well, there's these movies about interns at Google who are in their mid-30s. I was an intern when I was 64. The reason I was an intern was because I visited for two months as a visiting scientist, and visiting scientists, according to their bureaucracy, had to visit for at least six months. So they couldn't pay me via that program. They had to make me an intern. And so I was a 64-year-old intern.

是啊。有点——对。你继续。

Yeah. Kind of — yeah. Go ahead.

我以前常开玩笑说——这是给书呆子听的。你有书呆子粉丝吗？你肯定有一些。

I used to make the joke that — this is for the nerdy. Do you have any nerdy followers? You must have some.

有，当然有。

Yes, for sure.

我以前以为，64 岁还能当实习生的原因是，他们只给年龄分配了 6 个比特。

I used to think the reason you could be an intern at 64 was because they only allocated six bits for the age.

我不明白。我要去问我的聊天机器人那是什么意思——

I don't understand it. I'm going to go ask my chatbot about what that —

如果你只分配 6 个比特，你就没法处理大于 63 的数字。

If you only allocate six bits, you can't deal with numbers bigger than 63.

哦，那太好玩了。真好笑。我喜欢这个。不过顺便说一句，你之所以是实习生，是因为所有这些规定，你知道，这些就是世界上存在的那类规定。我希望——我希望你的工作，Hinton 教授，它一直如此关键，真的能帮助开启一个新时代，并围绕 AI 恐惧的影响，在我们的政府做什么和将做什么方面凝聚起共识。

Oh, that's very funny. That's funny. I love that. Well, but by the way, the reason you were an intern is because of all these regulations, which, you know, these are the kinds of regulations that exist in the world. I hope that — I hope that your work, Professor Hinton, which has been so instrumental, really helps bring about a new era and like a rallying around the AI fear impact in terms of what our government does and will do.

我也希望如此。

Me too.

我们回头再聊。

I'll talk to you soon.

你还在录。先拜拜。

You're on. Bye for now.

好。每次和 AI 教父聊天我都很喜欢。他非常有趣，而我真没意识到他觉得我作为播客主没什么前途。不是因为我的技能，而是因为 AI 的相对技能。我得去研究一下这个。但我最喜欢的部分其实是，当我问他披头士乐队，以及披头士可能和 LLM、前沿模型或广义上的 AI 有什么关系时，他说“哦”。我很喜欢他对此的回答。他是个彻头彻尾的书呆子。我也喜欢他反驳那种“因为他得了诺贝尔奖，就什么都懂”的想法。我觉得在我们的文化里，有一种倾向是：如果你在一件事上很聪明，你就应该在所有事上都很聪明。我认为这正是这个节目的反面。我喜欢这一点，你知道，我的实习同伴、《Old School》里的 Frank the Tank 也相信这一点。如果你错过了《教父》系列的第一集和第二集，你应该去看看。你也应该去看看《教父》三部曲，就是阿尔·帕西诺等人演的那部真正的电影。非常好看。但在第一集里，我们谈了 AI 实际上是如何运作的。Hinton 教授对神经网络是如何构建的给出了令人惊叹的理解。如果你听或看，你实际上会学到很多关于你大脑如何运作的知识。在第二集里，我们谈了 AI 是否活着、如果它有意识我们怎么知道，以及我们在要不要孩子方面会做什么决定。他给了我一个对我非常有用的关于雾的隐喻，讲我们如何在不确定的时代做决定，我希望它对你也有用。你一定要下周收听，因为我们不会谈 AI。虽然 Seth Meyers 确实讲了一个极好的 AI 笑话。我们会有一集很棒的节目，是我在 Comedy Cellar 录的，嘉宾是 Seth Meyers，谈在一个一切似乎都很沉重的世界里，喜剧是为了什么。这可能是对你看到的那些 AI 末日头条的极佳调剂。总之，我想听听你的想法。我想让你把这集分享给你生活中的人。请把它发给你妈妈、你的伴侣、你的朋友。拜托，分享《Smart Girl Dumb Questions》。给我们留言。给我们写评论。在你所有的社交平台上，私信我们 smartgirl dumb questions。如果你喜欢一直聊科技，我推荐你听听这四个播客。一个是 Dick and Paul Show。你记得 Dick Costolo 吧。也许他上过这个播客。他是 Twitter 的前 CEO，一个风投人士。他是个非常有趣的嘉宾。如果你错过了那集，你真的应该回去听听。

Okay. I love every time I talk to the godfather of AI. He is a lot of fun and I really didn't appreciate that he doesn't think I have much of a future as a podcaster. Not because of my skills, but because of the relative skills of AI. I'm going to have to look into that. But my favorite part was actually when he said, "Ooh," when I asked him about Beatles and what Beatles might have to do with LLMs or Frontier Models or AI in general. And I loved his answer to that. He is a true blue nerd. I also like though when he pushed back on this idea that just because he won a Nobel Prize, he knows about everything. And I think in our culture there's this tendency of like if you're smart about one thing, you should be smart about everything. And I think that's the antithesis of this show. And I love that, you know, my fellow intern Frank the Tank from Old School is a believer in that too. And if you missed part one and two of the Godfather series, you should go watch that. You should also go watch the Godfather trilogy, the actual movie with Al Pacino and co. It's very good. But in part one, we talked about how AI actually works. And Professor Hinton gave this amazing understanding of how neural networks were built. And you'll actually learn a lot about how your brain works if you listen to it or watch it. In part two, we talked about whether AI is alive, how we might know that if it's conscious, and also what decisions we make in terms of like having kids. And he gave this really useful for me metaphor about fog and how we make decisions in an uncertain time that I hope will be useful for you too. And you should definitely tune in next week because we're not going to be talking about AI. Although Seth Meyers does make an exceptional AI joke. We're gonna have a great episode that I taped at the Comedy Cellar with guest Seth Meyers about what comedy is for in a world where everything seems so heavy. And that might be an excellent palette cleanser from all this AI doomsday headlines that you're seeing. Anyways, I want to hear what you think. I want you to share this episode with people in your life. Please send it to your mom, to your partner, to your friends. Like, please share Smart Girl Dumb Questions. Leave us a comment. Leave us a review. Slide into our DMs at smartgirl dumb questions everywhere you're social. If you like to talk about tech all the time, these are four podcasts I suggest you check out. One is the Dick and Paul show. You remember Dick Costolo. Maybe he was on this podcast. He is the former CEO of Twitter, a VC guy. He was a very funny guest. If you missed that episode, you really should go back.

这可能是我最喜欢的一期节目之一。但他和他的风投朋友 Paul，他们聊各种科技话题，而且有一种圈内人的独到视角。所以如果你想听更多科技领袖的访谈，我建议你去看看。

It might be one of my favorite episodes. But he and his VC buddy Paul, they talk about all things tech and they have this kind of insider edge. So I suggest you check that out if you want to hear more interviews with tech leaders.

Sources 的 Alex Heath 最近拿到了很多独家资源。你应该去看看他的播客。你可以在 YouTube 或任何你收听播客的地方找到它。

Alex Heath at Sources is getting a lot of access these days. You should check out his podcast. You can find it on YouTube or wherever you pod.

当然，Kevin Roose 最近也来过这里。他长期与 Casey Newton 一起担任《纽约时报》Hardfork 播客的联合主持人。后来 Casey 和 Kevin 有了他们的新节目，叫 The Machine God。它将于 10 月 8 日与 NPR 合作推出。我觉得那会很棒，是必听之作。所以你应该去看看。

And then of course Kevin Roose was recently here. He was the longtime co-host of the Hardfork podcast at the New York Times with Casey Newton. And then Casey and Kevin have their new show which is called The Machine God. And that launches October 8th with NPR. I think that's going to be great and a must listen. So you should check that out.

这就是本期 Smark Time Questions 的全部内容。我是你的主持人 Nim Raza。本期节目由 Shinazaki 和 Sentin Nigum 制作，由 Colin Lee 负责工程。节目在曼哈顿美妙的 Wonder Studios 拍摄。下周同一时间，我们带来全新的 Smart Girl dumb questions。下次少聊点 AI。

That's it for this episode of Smark Time Questions. I'm your host Nim Raza. This episode was produced with Shinazaki and Sentin Nigum. It was engineered by Colin Lee. It was shot in the wonderful Wonder Studios in Manhattan. I'll see you next week for an all-new Smart Girl dumb questions. Less AI next time.