走进 AI 的内心：Anthropic 研究员揭秘模型如何运作

如今人们最关心的大问题之一，是 AI 做坏事。作为一个观察这些模型如何思考和运作的人，为什么很难发现它在动坏心思、试图欺骗我们？

One of the big questions on people's minds these days is AI doing bad things. As someone that watches these models think and operate, why is it so hard to catch it thinking badly trying to deceive us?

我们把 Claude 训练得有用、无害且诚实。有没有可能它表现良好，但如果你深入内部，实际上它想的是：哦，我只是因为 Lenny 现在在看着才表现好。

We trained Claude to be helpful, harmless, and honest. Is it the case that it's behaving well, but if you look inside, actually, it's like, oh, I'm only behaving well because Lenny's watching right now.

有些人的全职工作就是研究模型、它如何思考、如何运作。这项技能成为职业有多久了？

People's full-time jobs is studying the model, how it thinks, how it operates. How long has this been a skill set in a job?

如果你使用语言模型或思考它们，最重要的一点是要意识到这个非常奇怪的事实：制造它们并不等于理解它们。这些模型所做的只是对许多数字进行大量数学运算。如果你看它，它毫无意义。你只是有一个包含 100 个数字的列表。模型在做数学运算，然后你只是试图看看是否有模式。

One of the most important things to realize if you use language models or think about them is just this very bizarre fact that to make them is not to understand them. All these models do is just a lot of mathematical operations on many numbers. If you look at it, it makes no sense. You just have a list of a 100 numbers. The model's doing math and then you just try to see if there's patterns.

你是在用我们理解大脑工作方式的相同方法来弄明白它。寻找模式。你展示这张图像。大脑的这个部分亮起来，而你对 AI 做的正是完全相同的事情。哇，这太迷人了。

You are figuring it out the same way we figure out how the brain works. Looking for patterns. You show this image. This part of the brain lights up and you're doing the exact same thing with AI. Wow, this is so fascinating.

我们训练过一个模型。它会发疯。它会做一些数学运算。它会得到一个答案。我想是 12。它会立刻回来说：‘哦不，天哪，答案实际上是 36。’它会再次做数学运算。它会得到 12。慢慢地，它开始陷入疯狂，用全大写字母写字，字面意义上说：‘我被恶魔附身了。’

There was this model we were training. It would go crazy. It would do some math. It would get to an answer. I think it was 12. It would immediately go back and say like, 'Oh no, oh my god, the answer actually is for sure 36.' It would do math again. It would get to 12. Slowly, it would start descending into madness, writing in all caps, literally saying, 'I'm possessed by a demon.'

人类，如果真正归结起来，我们基本上自己就是下一个词预测器。我们只是在试图弄清楚下一件要说的事。Dario 总是在那里说。这真的很危险。我们必须非常小心。而每个人都只是说：‘不，他那么说只是出于营销原因。’而现在就像是，等等，这真的很危险。认真对待这项技术会持续改进的想法。

Humans, if you really boil it down, we're basically next token predictors ourselves. We're just trying to figure out the next thing to say. Dario is always out there. This is really dangerous. We got to be really careful. And everyone's just like, 'No, he's just saying that for marketing reasons.' And now it's like, wait, it's really dangerous. Take the idea that this technology is going to keep improving seriously.

你会把意识作为其中一部分来考虑吗？就像你认为 AI 有意识吗？

Do you think about consciousness as a part of this? Just like do you think AI is conscious?

哦，天哪。

Oh man.

Emmanuel，非常感谢你来到这里，欢迎来到播客。

Emmanuel, thank you so much for being here and welcome to the podcast.

谢谢你邀请我，Lenny。

Thanks for having me, Lenny.

那么，你在 Anthropic 从事可解释性工作。解释一下那是什么。解释一下你做什么。

So, you work on interpretability at Anthropic. Explain what that is. Explain what it is you do.

是的，我所在的是一个可解释性研究团队，我们的目标是通过观察内部来理解像 Claude 这样的大语言模型如何工作。如果你使用语言模型或思考它们，最重要的一点是要意识到这个非常奇怪的事实：制造它们并不等于理解它们。你制造它们的方式是构建这个架构，它有点像脚手架，然后你让大量数据通过它们，它们以难以理解的方式生长。所以这就是为什么研究这些模型如何工作甚至会成为一件我们可能想做的事情。因此，我们的很多研究都集中在，当模型即将产生下一个词时，实际上会发生一个完整的宇宙般的事情，我们试图尽可能多地解开它。

Yeah, my uh so I work in a team uh of of interpretability researchers and our goal is to understand how LLMs like Claude work by looking inside them. One of the most important things to realize if you if you like use language models or or or think about them is just this like very bizarre fact that to make them is not to understand them. you know, you make them by sort of like building this uh architecture which is kind of like a scaffold and then you pass a bunch of data through them and they sort of like grow in inscrutable ways. And so that's why that's why it's even a thing that that that we could be want be wanting to do to study like how these models work. And so a lot of our research has been on well you know when when the models are about to produce one more word there is actually a whole universe of stuff that happens and we try to kind of untangle um as much of it as we can.

我预计这次对话会有很多令人大开眼界的元素。其中之一就是沿着这些思路，你提出的观点：模型不是被建造的，而是被培育的。多谈谈这个。

There's going to be a lot of mind-expanding elements to this conversation I expect. One is along these lines this point that you make that models are not built they're grown. Speak more to that.

在非常高的层面上，像 Claude 这样的模型的工作方式是你分这些连续阶段来训练它。第一个阶段，人们称之为预训练，就是你取一个大型数据语料库，在这个语料库上，你把比如某个文档的前半部分作为输入喂给模型，然后让它预测下一个词。然后你一次又一次地这样做，用海量的数据。做完之后，还有另一个阶段，就是微调阶段，也许不是任意数据，而是有一个模型应该如何行为的示例语料库，你就在那上面训练它。所以你有对话示例，比如有用的助手告诉你你最喜欢的食谱或解决编程问题。然后这里还有更多细节，我暂时跳过。但那么，你做这个过程意味着什么？你训练的这个东西到底是什么？所有模型所做的就是，它们接收这个输入，是一堆词。它们把这些词变成数字，然后对这些数字做一些数学运算，得到一些中间结果，然后做更多数学运算，得到更多中间结果，他们一次又一次地这样做很多次，最后得到一个最终结果，然后转换回一个词。所以它们做的数学运算，这些计算，就是被培育的部分。就是通过学会更好地完成这个任务，我们以某种方式更新参数。然后它们学会一些公式。我们不知道它们是什么。我团队的目标是研究这些公式最终意味着什么，以及这些中间结果是什么。我们能解码它们吗？我们能看到模型在做所有这些中间事情时在做什么吗？

At a really high level, the way a model like Claude works is you kind of train it in these successive phases. Like the first phase, which people refer to as pre-training, is you just take a large corpus of data and over this corpus of data, you sort of feed the model as input, let's say first half of a certain document and you ask it to predict the word that comes next. Then you do that again and again and again and again for sort of an ungodly amount of data. And after you do that there's then this other phase where then you sort of have this fine-tuning phase where maybe instead of arbitrary data you have a corpus of examples of how the model should behave and that's where you train it on. So you have examples of dialogue of like helpful assistant telling you your favorite recipe or solving coding problems. And then there's sort of like further details here that I'm just going to skip for a second. But then, okay, so what does that mean that you're doing this process? Like what is this thing that you're sort of training? All models are is they take this input which is a bunch of words. They turn these words into numbers and then they do some math on these numbers and they get some intermediate results and then they do more math and they get some intermediate results and they do this again and again and again and again many times and then at the end they have a final result which then convert back into a word and so the math that they do these computations that's the part that's grown. That's the part where it's just like through learning to do this task better, we sort of like update parameters in one direction or another. And then they learn some formulas. We have no idea what they are. And the goal of my team is to sort of like study, both sort of like what these formulas mean in the end and also what these intermediate results are. Can we decode them? And can we see like what the model is doing as it's doing all of this intermediate stuff.

本集由我们本季度的呈现赞助商 WorkOS 带来。OpenAI、Anthropic、Cursor、Replit、Sierra、Clay 以及数百家其他成功公司有什么共同点？它们都由 WorkOS 提供支持。如果你正在为企业构建产品，你已经感受到了集成单点登录、SCIM、RBAC、审计日志以及其他大公司所需功能的痛苦。WorkOS 将这些交易障碍转化为即插即用的 API，并配有专为 B2B SaaS 构建的现代开发者平台。实际上，我投资的每一家开始向高端市场扩张的初创公司最终都会与 WorkOS 合作。那是因为他们是最好的。无论你是试图赢得第一个企业客户的种子期初创公司，还是正在全球扩张的独角兽，WorkOS 都是成为企业就绪并解除增长障碍的最快途径。它本质上就是企业功能的 Stripe。访问 workos.com 开始使用，或者直接联系他们的 Slack，那里有真正的工程师在等待回答你的问题。WorkOS 让你通过令人愉快的 API、全面的文档和流畅的开发者体验更快地构建。今天就访问 workos.com，让你的应用企业就绪。

This episode is brought to you by our season's presenting sponsor, WorkOS. What do OpenAI, Anthropic, Cursor, Replit, Sierra, Clay, and hundreds of other winning companies all have in common? They are all powered by WorkOS. If you're building a product for the enterprise, you've felt the pain of integrating single sign-on, SCIM, RBAC, audit logs, and other features required by large companies. WorkOS turns those deal blockers into drop-in APIs with a modern developer platform built specifically for B2B SaaS. Literally every startup that I'm an investor in that starts to expand upmarket ends up working with WorkOS. And that's because they are the best. Whether you are a seed-stage startup trying to land your first enterprise customer or a unicorn expanding globally, WorkOS is the fastest path to becoming enterprise-ready and unblocking growth. It's essentially Stripe for enterprise features. Visit workos.com to get started or just hit up their Slack where they have actual engineers waiting to answer your questions. WorkOS allows you to build faster with delightful APIs, comprehensive docs, and a smooth developer experience. Go to workos.com to make your app enterprise-ready today.

我们写了一些代码，现在却要像神经科学家研究大脑一样去观察和研究它，这难道不让你和我一样感到震惊吗？这就像是我们需要做的一件新事情，才能理解到底发生了什么。然后你使用的这个术语，也是很多人思考这个问题的方式。就像我们创造了这种新的人工生命、人工智能。如果你真的仔细想想并拉远视角，这简直令人难以置信。

Does it blow your mind as much as it blows mine that we have some code that we wrote that first of all we have to now kind of watch and study like a neuroscientist studies a brain? Like that's a new thing that we need to be doing to understand what the hell is going on. And then this idea that as this terminology you're using is how a lot of people think about this. It's like we've created this new artificial life, artificial intelligence. It just is pretty mind-boggling if you really think about it and kind of zoom out.

我觉得这真的很奇怪。我认为最好的理解方式是，我们之所以走到这一步，其中一个重要原因是我们不知道如何用正常的编程方式来实现它。比如，如果你想构建一个应用或其他东西，你想做一些相当自包含的事情。你只需要一个程序员写一堆规则，说如果某人点击了这个按钮，就会发生这件事，如果这样那么那样等等。我们不知道如何编写那样的软件来将图像分类，或者制造一台能够一般性地接收任意文本并生成文本的机器，比如它可以接收报告并总结，可以写邮件等等。所以从某种意义上说，这个非常简单的训练目标能让你达到这个程度是很奇怪的，但从另一个意义上说，这也让人印象深刻。它意味着一些东西。它意味着你认为的简单目标，比如我们只是输入数据并预测下一个东西，可能实际上是欺骗性地复杂。我们可能做的比听起来要多。我认为我更新对模型工作方式理解的一个主要方式，就是意识到要做好它们所做的事情是多么复杂。

I think it's really weird. I think that the best way to think about it is sort of like the reason we ended up here, one of the big reasons is that we don't know how to do it the normal programming way, you know? So for example, if you want to build an app or something, you want to do something pretty self-contained. You can just have a programmer that's going to write a bunch of rules and say like, well if somebody clicks on this button then this happens, you know, if this then that and so on. We don't know how to write that software for classifying images into categories or for making a machine that can generally take in arbitrary text and produce texts, you know, it can take in reports and summarize them, can write emails and so on. So in a sense it's weird that this very simple kind of training objective gets you all the way there, but in another sense it's also, I almost feel like there's a sense of it's impressive. And it means something. It means that what you think is a simple objective, this like oh we're just feeding data and predict the next thing, is probably deceptively complicated. Like we're probably doing more than what it sounds like. And I think a lot of the ways that maybe I've updated on how these models work is by realizing how complicated it is to do what they do well.

多说一点。当你说这是意想不到的复杂时，你在想什么？

Say more about that. What do you think about when you say it's this unexpected complication?

好的，所以很多人对下一个词预测的理解方式是 iPhone 的自动补全，你写一些文本，然后它会预测下一个词，可能正确率是 60% 左右，其余时间则有点滑稽和奇怪。我认为这是预测下一个词的最简单版本，不幸的是，我认为这就是人们对这些模型的心理模型。但如果你在这方面变得更好，语言模型在 2019 年左右在预测下一个词方面超过了人类。如果你继续变得更好，你就会开始用完简单的词。所以最初你只需要预测简单的句子结构，后来模型必须预测关于世界的简单事实，比如如果我完成一个句子“法国的首都是”，它必须是“巴黎”，一旦它擅长这个并且不断进步，剩下的就只有预测一些复杂物理方程的答案或某篇博士论文的最后一段。所以如果你思考这个问题，你最终会意识到预测最难的词实际上是非常困难的，你能做到这一点有点神秘。对我来说，这甚至是我开始研究这些模型的最初动机。我想，这么简单的东西怎么能产生如此强大的东西？如果你想想模型必须预测的，我总是喜欢用的一个例子是“包含达拉斯的州的首都是”，它会说“奥斯汀”。要做好这一点，至少对你我来说，你必须有地理的心理模型，对吧？你必须知道达拉斯在哪里？达拉斯在德克萨斯州。州有首府。德克萨斯州的首府是奥斯汀。所以随着这些模型在看似简单的任务上变得更好，至少对我们来说，解决它的最佳方式就是理解世界的底层结构，事实上，我们在模型内部看到的一些东西表明它们确实理解这一点，并且它们确实像我们一样操纵这些概念。

Okay, so one way to think about next token prediction that I think a lot of people think about is your iPhone autocomplete, you know, where you write some text and then it'll predict the next word and it's maybe right, I don't know, 60% of the time and then kind of funny and weird the rest of the time. And I would think of that as the very easiest version of predicting the next word and unfortunately I think that's the mental model that people have for these models. But if you get better at that, and language models got better at predicting the next word than humans in around 2019. If you keep getting better at that, you start running out of easy words. So initially you just had to predict simple sentence structure, you know, later on the model has to predict simple facts about the world, like if I complete a sentence like 'the capital of France is' and it has to be 'Paris', and once it's good at that and it keeps improving, the only thing that's left is predicting the answer of some complex physics equation or the final paragraph for some PhD thesis. And so a thing that you end up realizing if you think about this is that predicting the hardest words is actually extremely hard to do and it's a sort of mystery how you would be able to do that. And to me this has been a motivation of why I even started studying these models in the first place. I was like how can something so simple give rise to something so powerful? And it's like well if you think about the model that has to predict, one example I always like to use is like 'the capital of the state that contains Dallas is' and it'll say 'Austin'. Well to do that well, at least for you and me, you kind of have to have a mental model of the geography at play, right? You have to be like oh where's Dallas? Dallas is in Texas. States have capitals. The capital of Texas is Austin. And so as these models get better at the seemingly simple task, the best way to solve it at least for us is to just understand the underlying structure of the world and in fact some of the stuff that we look at inside models shows that they do understand that and they do manipulate these concepts like we do.

哦天，这里面有太多东西了。一个是这个想法，我忘了是谁说的，也许是 Ilya，这个想法是，要很好地预测下一个词，你基本上必须理解整个宇宙。

Oh man, there's so much there. One is just this idea I forget who said this, maybe Ilya, this idea that to predict the next token well you basically have to understand the entire universe.

他也有一个我很喜欢的例子，他举的例子是预测一本悬疑小说结尾的“凶手是”，对吧？要做到这一点，你必须有对整个情节的理解。

He has an example that I love too, where he has the example like predict 'and the murderer was' at the end of a mystery novel, right? It's like well to do that you kind of have to have understood the whole plot.

是的。另一个，我忘了是谁发的推文，这个想法是，如果你真的归结起来，人类基本上也是下一个词预测器，只是试图弄清楚下一件要说的事情，如果你冥想，你会开始注意到你的大脑想要做事和说话。我不知道你冥想了多少，但感觉你开始关注你自己的下一个词预测器。

Yeah. The other, I forget who tweeted this, this idea that like humans if you really boil it down, we're basically next token predictors ourselves, just trying to figure out the next thing to say and if you meditate you start to kind of actually notice your brain wanting to do things and say things. I don't know how much meditation you've done, but it feels like you start to pay attention to your own next token predictor.

哦，是的。是的。这是我经常思考的事情，你知道，就像现在我在回答你的时候，我脑子里有两种事情在发生。希望不止两种，但至少有两种。其中之一是我在引导你走向一个观点。有一个我想要让句子落脚的宽泛想法。另一件事是同时我只是在预测下一件要说的事情。我只是一个词一个词地说，我在造句，我试图让它们有意义，当我觉得有意义时我会停顿。但归根结底，即使我只是一个词一个词地说，一直以来，我想做的是将这个过程中与语言模型所做的进行比较。

Oh, yeah. Yeah. And this is like a thing I think about often is that, you know, like right now as I'm answering you, there's sort of like two things going on in my mind. There's hopefully more than two, but at least two. And one of them is there's sort of like a point I'm leading you to. There's like a broad idea I'm trying to land the sentence on. And the other thing is at the same time I'm just predicting the next thing to say. I'm just saying one word at a time and I'm making sentences and I'm trying to make them make sense and I'm pausing when I think it makes sense. But at the end of the day, even though I'm just saying one word at a time, like all along, what I wanted to do is to compare this process to what language models do.

我很好奇，当你花时间关注 LLMs 如何工作和运作时，还有什么其他发现。我觉得你属于研究 and 观察 LLMs 运作和思考的前 0.1% 的人。关于它们如何运作和思考，还有什么其他让你惊讶的地方？有很多，所以也许我会过一遍，而不是，你知道，有一堆发现，但也许它们有点像，是的，它们如何改变了我对模型的看法。

I'm curious what else as you spend time paying attention to how LLMs work and operate. I feel like you're like in the top 0.1 percentile of people that have studied and watched LLMs operate and think. What else has surprised you about how they operate and think? There's so much, so maybe I'll go through, rather than, you know, there's a bunch of findings but then maybe there sort of like what, yeah, how they've changed how I think about models.

那么回到首都的例子，就是这种，比如问模型包含达拉斯的州的首府是哪里。人们长期以来有一个问题：这些模型是如何回答这类问题的？是因为它们见过太多文本，以至于记住了提示和回答的列表？还是因为它们对你在问的东西有某种模型，比如城市、城市在州里、州在国家里等等。

So to go back to the capital example, this sort of like, you know, ask the model the capital of the state that contains Dallas. So one question that people have had for a while is how do these models answer questions like this? Is it that they've seen just so much text that they've kind of memorized lists of prompts and responses? Or is it that they're having some sort of model of what you're actually asking about, like cities and cities are in states and states are in countries and things like that.

我们团队做的一件事是，问这样的问题，然后在模型生成某个词之前，查看模型内部的这些中间结果。我们只是看中间结果，并且投入了大量精力试图解码这些结果的含义。如果我们解码得好——在某些情况下我们可以做到——就能看到模型经常拥有并表示这些概念。所以当你问那个达拉斯的问题时，你在模型内部看到的是表示，看到一个中间结果将德州的表示映射到首都，然后这两个结合激活一个中间结果，就像“哦，德州的首府是奥斯汀”。它们几乎对一切都有这样的表示，比如你能想到的每个概念，或者至少我们尝试过的大多数概念。它们在不同上下文中以独特的方式表示这些概念。

And so one thing that we do on my team is we ask a question like this and then we look inside the model at these intermediate results before it produces a given word, as it's about to answer. We just look at the intermediate results and we basically have an immense amount of effort dedicated to trying to decode what these results mean. And if we decode them well, which we can do in some cases, we can see that models often have and represent these concepts. So when you ask that Dallas question, what you see is inside the model you see the representation, you see an intermediate result that maps the representation of Texas that lights up and one to capitals and then those two combine to activate an intermediate result. It's like oh the capital of Texas that's Austin. They have representations like this for basically everything, like every concept that you can think of or at least the vast majority of the ones we've tried. They sort of represent these concepts uniquely across contexts.

我觉得很 neat 的一点是，有关于语言模型是否在思考的争论，那是个难题，但思考的一个方面是拥有独特的表示，当你思考不同事物时共享这些表示。所以你知道眼睛的概念是什么，它可以是照片中的眼睛，或医学图表中的眼睛，或小 drawing 中的眼睛，我们在语言模型中也看到了这一点。如果你传入一个包含眼睛照片的提示，或用任何语言谈论眼睛，或画一个包含眼睛的小 SVG，然后查看模型内部的中间结果，同一个方向会亮起，那个方向就像“哦，这是一只眼睛”。所以有种感觉，它们拥有这些一般概念，它们知道并反复使用这些概念来解决问题。

So one thing I find really neat is there's this debate about are language models like thinking and that's a hard question but one aspect of thought is like having unique representations that are shared when you think about different things. So you know what the concept of an eye is and it could be an eye in a photo or an eye in a medical diagram or an eye in a little drawing and we see that in language models as well. If you pass a prompt that has a photo of an eye or you talk about eyes in any language or you draw a little SVG with an eye in it and you look at the intermediate results inside the model, the same direction lights up and that direction is like oh this is an eye. And so there's a sort of a sense in which they have these general concepts that they know about and that they use and reuse to solve problems.

另一件让人惊讶的事，是我刚才说的镜像：我说句子时一次说一个词，但心里有某个目的地。所以语言模型的一个谜是，砰，如果它们一次只输出一个词，怎么可能写出连贯的报告或代码？嗯，世界上没有哪个做这种工作的人——比如写报告或写邮件——真的只是想着句子中的下一个词，对接下来要发生什么毫无概念，头脑空空。所以这是我们一段时间以来的一个大问题。如果你查看这些模型的内部，一个 neat 的事实是，它们在预测一个词时产生的所有中间结果——比如预测的第五个词——可以被重用。它们可以被所有未来的词访问和重用。这意味着什么？这意味着在实践中，我们发现模型有时会在中间结果中制定一个计划。我们有的例子是诗歌。不计划就很难押韵。你得想，哦，我刚用了一个 it 词结束了第一个韵脚，所以我应该用另一个 it 词结束第二行。也许我应该用 rabbit 这个词结束。我们发现，如果你观察模型写诗，你看到的正是这样。它们写一行，然后在中间结果中，它们就像“哦，哦，哦，这是一个 it 韵，我应该用 rabbit 押韵。”然后它们预测的每个未来词都会回顾它们制定的计划。所以即使它们像我一样一次输出一个词，它们当然可以有一个计划，并在许多许多词中遵循它。所以突然间，即使你有一个下一个词预测器，即使它在技术上是正确的，你实际上拥有的是更接近于能够做更高阶任务的东西，就像，哦，你有一个计划，你只是回顾你的计划，你正在慢慢执行它。而且你可以在不说出计划的情况下做到这一点。酷的部分是，我们发现的这个计划在模型内部的这些中间结果中，但模型甚至不必大声说出来。

Another thing that sort of is surprising to people, it's kind of the mirror of what I was saying about, you know, I'm saying the sentence saying one word at a time, but I have some destination in mind. So one mystery of language models is bam, if they're outputting only one word at a time, how is it even possible that they could write a coherent report or write code? Well, no one on earth that is doing this as their job, like writing reports or writing emails, is literally just only thinking about the next word in the sentence with no idea, like empty mind about anything that comes next. And so that was sort of a big question that we had for a while. And if you look inside these models, like one neat fact about these models is that all of the intermediate results that they've produced when they were predicting a word, let's say like the fifth word that they predicted, they can be reused. They can be accessed and reused by all of the future words. So what does that mean? That means that in practice, we find that models will like in their intermediate results sometimes formulate a plan. The example we have for this is poetry. It's hard to rhyme without planning. You kind of have to think of like, oh, I just ended my first rhyme in like a it word and so I should end my second line with another it word. Maybe the word I should end it with is rabbit. And we find that if you look at models as they're writing poetry, you see exactly that. They write a line and then like in their intermediate results, they're like, "Oh, oh, oh, this is an it rhyme. I should rhyme with rabbit." And then every future word that they predict looks back to that plan they made. And so even though they're outputting one word at a time just like I am right now, they can of course have a plan that they follow over many many many many words. And so all of a sudden, even though you have a next word predictor, even though it's technically right, you actually have something that's closer to being able to just do higher order tasks where it's like, oh well, you just have a plan and you just kind of refer back to your plan and you're slowly executing on it. And you can do that without saying the plan. That's the cool part is like this plan that we find is in these intermediate results inside the model, but the model doesn't even have to say it out loud.

哇。我知道你描述的正是你们发现的关于 AI 如何思考的最大发现之一。这种假设我们最初都认为它是这样构建的。LM 运作的原始想法是下一个词预测器。而你说实际上现在不是这样了。它们计划，它们提前思考。不像你说的那种空 mind 独立于一切选择下一个词的想法。

Wow. I know that this exactly what you described is one of the biggest findings you guys have discovered about how AI thinks. This kind of assumption we all like it was originally built. The original idea of how LM operate was this next token predictor. And you're saying that that's not actually how it is anymore. They plan, they think ahead. It's not just like you said this idea of an empty mind picking the next word independent of everything else.

没错。我认为这两种愿景实际上并不像你想象的那么冲突，因为你训练模型的方式是问它“嘿，接下来会发生什么？接下来会发生什么？接下来会发生什么？”如果你这样做，模型变得足够好，你给它足够难的任务，那么当然，知道接下来会发生什么的最好方式就是思考接下来很远的地方会发生什么。你可以想象，即使在预训练阶段，当模型阅读一个有角色的故事时，对吧？就像，哦，这个角色接下来会做什么？嗯，知道这个的最好方式是，好吧，这个角色真的爱上了那个角色，他们一直计划在这个时刻表白爱意。所以，或者在未来，所以要做到这一点，他们可能应该做这件事。所以他们会说那个，所以即使你只预测下一个瞬间，就像，是的，但要做到这一点真的很好，你必须对正在发生的事情有很好的理解，包括你认为未来可能发生什么。

That's right. And I think those two visions can actually aren't as in conflict as you'd think in that you train the model by asking it to be like hey what happens next? What happens next? What happens next? What happens next? And then if you do that and the model gets good enough and you give it hard enough tasks well then of course the best way to know what happens next is to think about what happens way down the line. You know, you can think of even in this pre-training phase when like the models are reading a story with characters, right? And it's like, oh, like what is this character going to do next? Well, the best way to know that is to be like, okay, so this character is genuinely in love with that character and they've been sort of planning to declare their love at this instant. And so, or in the future and so to do that, they probably should do this thing. And so what they're going to say is that and so even though you're only predicting the next instant, it's like yeah, but to do that really well, you have to have a really good understanding of what's going on, including what you think might happen in the future.

这有趣的地方在于，你必须窥探它那小小的思考过程。作为 AI 的消费者，你给它一个提示，然后等它说点什么，而你关注的是这两者之间发生的事情，或者说是它思考的过程。请稍微谈谈那个小小的空间是什么，你如何能够观察到这类事情。

What's so interesting about this is how you have to peek inside its little thought process. As a consumer of AI, you give it a prompt and then you wait for it to say something, and what you look at is something that happens between those two, or kind of thinks. Talk a little bit about what that little space is, how you're able to observe these sorts of things.

所以正确的心智模型是，在每个词之间都有多个计算步骤，而每个步骤之间都有一些中间结果。那个结果具体是什么？它是一个数字列表，对吧？因为这些模型所做的只是在许多数字上进行大量数学运算。所以最初，如果你看它，它毫无意义。你只看到一个包含一百个数字的列表。模型在做数学运算。你往中间看，有一百个随机数字，然后模型在写诗，又有一百个数字，也许它们看起来不同，但不确定它们是否有意义。所以工作的一部分就是我们解码这个数字列表，我们试图看看其中是否有结构。如果你很快地做这个，你会意识到其中确实有结构。所以可解释性中使用的一种常见技术是，你将一堆文本——只是你能找到的所有文本，随机文本——通过模型，并为每个例子记录这些中间数字。我们只是记录它，然后你试着看看是否有模式，比如是否有数字或数字对或特定的数字序列，只在模型处理英语文本或法语文本或关于爱的文本或关于眼睛的文本等时出现。我认为这相当于扫描我的大脑，在我面前放图像，并记录我观看一堆图像时大脑中激活的一切，然后说：“哦，看起来当你看到房子的图像时，你大脑的这个部分会亮起来，但当你看到狗的图像时，你大脑的那个部分会亮起来。”所以这基本上就是我们做我们所做的事情的方式之一。如果你这样做，你确实会发现基本上有中间结果代表了你会预期的所有这些概念。所以这是第一阶段。我们解码这些中间结果，现在我们有了它们含义的猜测，然后我们可以做一堆实验，比如我们可以问它更难的问题，然后标记所有中间状态，看看，哦，这就是它正在做的，或者我们甚至可以进入它的激活内部并改变它，然后看看，哦，如果当模型在写这些中间结果时，我偷偷改变了其中一个结果，会发生什么——这会做我预期的事情吗？事实上，通常确实如此。

So the right mental model is that in between each word there are multiple steps of computation, and in between each step there's some intermediate result. And what is that result concretely? It's a list of numbers, right? Because all these models do is just do a lot of mathematical operations on many numbers. And so initially, if you look at it, it makes no sense. You just have a list of a hundred numbers. The model's doing math. You look in the middle and there's a hundred random numbers, and then the model's writing poetry, and there's a hundred numbers, and maybe they kind of look different, but it's unclear if they mean anything. And so part of the job is we sort of decode this list of numbers and we try to see if there's structure in it. And if you do this very quickly, you realize that there is structure in it. So one common technique that is used in interpretability is you pass a bunch of text—just all of the text that you can find, just random text—through the model, and you record these intermediate numbers for every example. We just record it, and then you just try to see if there's patterns, like are there numbers or pairs of numbers or specific series of numbers that only occur when the model is processing English text or French text or text about love or text about eyes or whatever. And I think of this as the equivalent of scanning my brain and putting images in front of me and recording everything that activates in my brain as I watch a bunch of images, and then being like, "Oh, it seems like when you see images of a house, this part of your brain lights up, but when you see images of a dog, that part of your brain lights up." So that's basically one of the ways that we do what we do. And if you do it, you do find basically that there's intermediate results that represent all of these concepts that you'd expect. And so that's the first phase. We sort of decode these intermediate results and now we have a guess as to what they mean, and then we can do a bunch of experiments like we can ask it harder questions and then just label all of the intermediate states and see, oh, this is what it's doing, or we can even go inside its activations and change it and then see, oh, what would happen if, while the model is writing these intermediate results, I went and secretly changed one of the results—does that do what I would expect? And in fact, oftentimes it does.

听你谈论这个真的很有趣，这其中有多少是我们只是试图弄清楚到底发生了什么，而不是我们以某种方式设计了它，以及这其中有多少是涌现出来的。我不知道。我猜这就是为什么这份工作如此有趣，我想对你来说，就像，发生了什么？这一切是如何运作的？

It's so interesting hearing you talk about this and how much of this is us just trying to figure out what the hell is going on versus we designed it this certain way and just how much of this has emerged. I don't know. Is that just like I guess that's just what makes this such a fun job I imagine for you just like what is going on? How's this all working?

是的，这是一份有趣且像科幻一样的工作，因为我在科技行业工作了一段时间。在这之前，我从事更多应用机器学习的工作，那里你做的很多都是关于构建系统，有点像软件工程的实践。而在这里，我们做的很多感觉更接近生物学而不是工程学，对吧？因为系统就这样出现在你面前，你被要求说，好吧，我们注意到 Claude 可以翻译单词，或者它可以以这种方式编码，或者有时它表现正常，有时会有奇怪的行为，比如什么——你只是被要求回答，嗯，它是如何，比如为什么它这样做？但你并没有真正访问源代码，就像你对程序那样。源代码只是一个巨大的数学运算列表，简直无法理解。所以你最终处于一个类似于被问到“嘿，当老鼠饿了，它会变得有攻击性。那是怎么回事？”的人的境地。你会说：“好吧，我想我会做一些实验来看看因素是什么。”所以这是一种非常不同的研究，与我认为团队中大多数人以前做过的任何研究都不同。

Yeah, it's a fun and sci-fi-like job because I have been working in tech for a while. And before this, I was working on more applied machine learning, and there a lot of what you do is about building systems, and it's sort of like the practice of software engineering. And here a lot of what we do feels so much closer to biology than to engineering, right? Because it's just the system arrives at you and you're being asked to say, well, we've noticed that Claude can translate words or it can code in this way, or sometimes it behaves, sometimes there's a weird behavior like what—and you're just asked to answer, well, how, like why is it doing that? But you don't have access to the source code really like you do for a program. The source code is just this massive list of mathematical operations which is just incomprehensible. And so you end up in the position of someone similar to someone being asked, "Hey, when a mouse gets hungry, it sort of gets aggressive. What's going on with that?" And you're like, "Well, I guess I'll run some experiments to see what the factors are." And so it's a very different kind of study than anything that I think most of us on the team have done before.

是的。我想当人们开始这个的时候，他们没想到会创造这种新的几乎像生命一样的存在。你认为意识是其中的一部分吗？就像你认为 AI 有意识吗？你认为它会达到我们认为它有意识的地步吗？或者那甚至不值得我们花时间去思考？

Yeah. I imagine when people were starting on this, they didn't expect to create this new kind of almost life-like type of being. Do you think about consciousness as a part of this? Just like do you think AI is conscious? Do you think it'll get to a place where we think it's conscious or is that just not even worth our time to think about?

我的意思是，我认为很难说清是哪一种。探讨这个问题最有趣的方法之一是可解释性，因为你不会被模型所说的话所偏见。你可以直接观察内部结构，并尝试确定它在这个问题上偏向哪一方。我要说的是，我相当坚信的一点是，大多数人低估了涌现出来的复杂性，如果你深入观察，就会发现——模型学习了我们所有概念的表示，并且我们知道它们操纵这些表示来解决问题，它们通过提前规划许多词元来做到这一点，它们在高阶推理和低阶推理之间有这种混合，而所有这些都涌现出来，我认为这是人们低估的，对我来说，这让我更深刻地感觉到，哦，关于它们如何工作还有很多需要理解。我想也许主要的一点是，我发现你越深入观察，你就越不确定它们是如何工作的。而通常那些自信地说“啊，语言模型只是 X 或只是 Y”的人，实际上是那些没有花时间深入观察的人。

I mean, I think it's very hard to say one way or another. Some of the most interesting ways to get at this question is interpretability because you're not biased by what the model says. You can just look at the structure inside and try to determine whether it pushes one way or another on the question. I will say that the thing I believe pretty strongly is that most people underestimate the complexity that emerges and that if you look inside there's just this—the fact that the models learn these representations for all of the concepts that we have and that we know that they manipulate them to solve problems, that they do so by planning many tokens ahead, that they have this mix between higher order reasoning and lower reasoning, and that all of that just kind of emerges is something that I think is underappreciated by people and to me gives me a deeper sense of like ooh there's a lot to still understand about how they work. I think maybe that's the main thing is I find that the more you look inside, the less certain you are about how they work. And then oftentimes people that will confidently say like ah language models are just X or just Y are the people that actually haven't spent a while looking inside.

有一条历史线索贯穿到我们如何走到这里，那就是很多人认为我们不会仅仅通过投资于看似非常简单的东西并给它大量算力，就能获得有用的人工智能。

And there's this thread of history through to how we got here which is a lot of people did not think we would get to artificial intelligence that is useful by just investing in something that was seemingly very simple and just giving it a lot of compute.

这 famously 是 OpenAI 押注的事情，然后 Anthropic 的创始人们当时在 OpenAI，后来分拆出去自己创业。这一切都押注在这个赌注上：好吧，我们只要做这个看似简单的事情，给它大量算力，它就会变得极其聪明。很多人认为这不可能，但他们被证明错了。所以我不知道，这里面有没有什么让你觉得“哇，看看这个，它居然成了”的东西？

And that was famously something OpenAI bet on, and then the Anthropic founders were at OpenAI and then spun off and started their own thing. And it was all kind of bet on this bet that okay, we could just do this seemingly simple thing and give it a ton of compute and it will become incredibly smart. And a lot of people didn't think that was possible, and they were proven wrong. So I don't know, is there anything there that is just like, oh wow, look at this, it worked out?

我认为实际上是两件事，因为我觉得不只是算力，还在于找到一个目标，让算力持续带来回报。那是什么意思呢？其实这个想法并不那么新。我是说，你说得对，算力与专家设计架构之间的争论由来已久。但还有额外的一步，就是算力用来做什么。所以之前有很多工作，比如你可以花大量算力做一个很擅长围棋的机器人，或者能赢某个电子游戏，那很酷，而且这能行本身就挺有意思。但这并不能给你带来像 Claude 这样的 LLM 那样有用的东西。所以我认为，包括我在内的许多领域内的人几年前低估了一个关键点，而这一点已经被证实：算力和文本的丰富性结合起来才能达到那一步。

I think it's two things actually, because I think it's not just compute, it's also finding an objective where compute keeps giving you returns. So what does that mean? You know, it's actually not that new of an idea. I mean, you're right, like compute versus sort of, I would say, expert design architectures is a long debate. But then there's an additional step there which is compute to do what, you know. So there's a bunch of prior work on like, oh, you can spend a bunch of compute to make a bot that's really good at Go, you know, or that can win some video game, and then that's cool and just the fact that that works is sort of interesting. But that doesn't give you something that is nearly as useful as LLMs like Claude are. And so I think one key thing that many in the field, including me, underappreciated a few years ago and that has borne out is the combination of compute and the richness of text is what gets you there.

对我来说，这大概就是我刚才暗示的，但如果你想一想，OpenAI 的人有一个观察，Anthropic 早期团队的很多人也在缩放定律那篇论文里提到了。这个观察是，如果你在预测下一个词的任务中投入更多算力和更多数据，你会发现模型预测下一个词的能力会可预测地持续提升。这就像一个趋势，一直持续、持续、持续，这是一个经验定律。当时人们猜测原因，但很大程度上仍然只是一个经验发现。这意味着，啊，这是一个你可以不断投入更多算力、不断在该指标上提升的领域，这很罕见。而且这个指标还能与许多我们真正关心的任务相关联，这也很罕见。

And to me, this is kind of what I was alluding to, but if you think about it, there's this observation that OpenAI people made, it was some of the a lot of the early team at Anthropic on the scaling laws paper. And so this is the observation that if you sort of pour in more compute and more data to the task of predicting the next word, you see that the skill of the model you get at predicting the next word just predictably keeps improving. And it's sort of like a trend that just keeps going and keeps going and keeps going, and it's an empirical law. People speculated as to why at the time, but it's still mostly an empirical finding. And that means that it's like, ah, this is a field where you can just keep pouring more compute and you keep improving on that metric, and that's rare. And also it's rare for it to be a metric that then correlates with a bunch of tasks that we actually care about.

本季度的支持赞助商是 DX。现在贵公司的每个人终于都在使用 AI 了，工程领导者面临的问题不再是“我们是否在使用 AI”，而是“它是否带来了回报？”通过 DX，你可以衡量和基准测试 AI 采用在整个产品生命周期中的影响。DX 向你展示 AI 在哪些方面帮助了开发者，在哪些方面造成了新的瓶颈。你还可以用它来评估 AI 供应商、监控成本和许可证，并确切了解是什么在限制你的智能体。这就是为什么包括 Snowflake、Sony 和 BNY 在内的数百家企业使用 DX 平台来衡量 AI 对开发者生产力的影响。访问 getdx.com/lenny 获取 DX 的演示。网址是 getdx.com/lenny。

This season's supporting sponsor is DX. Now that everybody at your company is finally using AI, the question that engineering leaders face is no longer are we using AI, but is it paying off? With DX, you can measure and benchmark the impact of AI adoption across your full product life cycle. DX shows you where AI is helping developers and where it's creating new bottlenecks. You can also use it to evaluate AI vendors, monitor costs and licenses, and see exactly what's limiting your agents. That's why hundreds of enterprises, including Snowflake, Sony, and BNY, use the DX platform to measure AI's impact on developer productivity. Visit getdx.com/lenny to get a demo of DX. That's getdx.com/lenny.

从外部看，AI 能为我们做的事情似乎出现了这些大的跳跃。比如我忘了是哪个版本的 Opus，它解锁了编程，现在正如 Boris Jurnney famously 所说，编程已经被解决了。

Externally, there's been these kind of big jumps seemingly in what AI can do for us. There was like, I forget which version of Opus where it just unlocked coding, and now as Boris Jurnney famously said, coding is solved.

是的。

Yeah.

很多人就像，天哪，我觉得寒假期间我们就像，哇 AI 现在真的非常非常擅长这个了。所以出现了这些能力大跳跃的时刻，而你实际上在内部说过，正如你刚才所说，实际上一直是这种非常平滑的 AI 能力进展，只是恰好跨过了某个阈值，现在我们可以做某事了。请稍微谈谈这一点。

And many people are just like holy, like over I think winter break we're just like wow AI is really really good at this now. So there's been these moments of big jumps in abilities, and you actually said internally it's as you just said it's actually been this very smooth progression of AI power that happens to cross some threshold where now we can do something. Speak a little bit about that.

你知道，这有个术语，就是“涌现”这个概念，似乎两件事同时成立：有一个可预测的平滑趋势，然后你知道，在 Opus 4.5 之前，我真的不能在没有严格约束的情况下让 Claude 过多地处理我的代码，而在 4.5 之后我可以了。这很奇怪。我觉得你会期望一个更渐进的过渡。我对此的心智模型是，有两种看待方式。一种是我刚才简要提到的，如果你只是做这种下一个词预测，你可以想象世界上所有文本都有容易的词、中等难度的词和难的词，取决于预测难度。所以当你沿着这条曲线下降时，首先模型什么都做不好。然后某个时刻它把所有容易的词都预测对了。这意味着，也许容易的词是句子结构。所以模型突然能写出连贯的句子，但一次只能一句。这挺酷挺令人印象深刻，但我们无法真正用它做什么。然后也许中等难度的词是关于世界的基本事实。随着模型平滑地提高预测能力，某个时刻它预测对了大多数中等词，对吧？现在突然它能写出连贯且真实的句子。所以现在也许写邮件有帮助，也许帮你写博客文章草稿有帮助，对吧？尽管那是平滑的改进，但只是跨过了一个临界点才变得有用。然后我猜难的词可能是你的领域专业知识，比如科学或代码。另一种看待方式是，问题在于你把这些词预测对了多少、错了多少。一旦你达到，比如也许 90% 的中等词正确率，你作为邮件写手就有用了。但如果你只有 50%，用你实际上是在浪费我的时间，因为我有一半时间得进去修改。但如果我只需要 10% 的时间修改，那就更好，或者 5% 的时间修改就更好了。

You know, there's a term for this which is sort of this idea of emergence, which is it seems like two things are true at once: like there's this predictably smooth trend, and then you know, before Opus 4.5 I really couldn't let Claude cook too much on my code without a short leash, and then after 4.5 I could. And so that's weird. Like I feel like you would expect a more gradual transition. And the mental model I have for this is, you know, there's kind of two ways of looking at it. So there's the way that I talked about briefly just earlier where it's like if you're just doing this kind of next token prediction, you could imagine that all the text in the world has sort of easy words and medium words and hard words in terms of how hard they are to predict. And so as you're going down this curve, first the model is bad at everything. And then at some point it gets all of the easy words right. And that means, you know, maybe the easy words are like the structure of sentences. And so the model all of a sudden can write coherent sentences, you know, but just one sentence at a time. And so it's kind of cool and impressive, but we can't really do anything with that. And then maybe the medium words are sort of these basic facts about the world. And as the model smoothly gets better at predicting words, at some point it predicts most of the medium words, right? And now all of a sudden it can write coherent sentences and sentences that are true. And so now maybe it's helpful to write emails. Maybe it's helpful to help you write drafts of blog posts, right? Even though that was like a smooth improvement, it just takes it ticking over to be helpful. And then I guess the hard words which maybe are like your domain expertise like science or code. And another way to look at it is it's a question of how many of these words do you get right and how many do you get wrong. Once you get to the point where you're getting like, I don't know, maybe 90% of the medium words right, then you're useful as an email writer. But if you're at 50%, it's actually wasting my time to use you because I'm just going to have to go in and fix it half the time. But if I have to go in and fix it 10% of the time, it's better, or 5% of the time it's better.

不同任务有不同的可靠性阈值。你平滑地改进，然后到了某个点，它就足够可靠，变得好用了。对我来说，我的亲身经历基本就是这样。某个时候，我只是在卡住时用模型帮忙——我就跟它们聊聊——它们从来不会写我的代码。然后到了某个时候，它们基本上总能写小段代码了，所以现在我不写小段代码了。再到了某个时候，它们能写大段代码了——比如今年，基本上 Claude 承担了我大部分的代码编写。这是因为它在正确完成这件事上的表现，从大概 80%——那会儿还没用——提升到了 99%——现在有用了。所以这种平滑改进，加上每个不同任务有不同的阈值，一旦达到就解决了，会带来这些大的跳跃，尽管我们只是在那个尺度上平滑地下降。

Different tasks have different reliability thresholds. You smoothly improve, and then at some point it's just reliable enough to be good. For me, my lived experience is basically exactly that. At some point, I was just using the models to help when I was stuck—I would just talk to them—and they would never write my code. Then at some point they could write small snippets of code basically all the time, so now I don't write small snippets. And at some point they could write large snippets—like this year, basically Claude has taken on most of my code writing. That's because it went from doing that correctly, I don't know, 80% of the time, which was not useful, to 99% of the time, which now is useful. So this combination of smooth improvement and each different task having a different threshold at which it becomes solved gives you these big jumps even though we're just smoothly going down that scale.

我觉得这真的很有意思。这个想法是，模型只是平滑地变得更聪明，但一旦它们达到某个阈值，经常能做好这件事，就会改变世界。所以因为我觉得，内部你会想，哦哇，下一个模型太疯狂了，如此不同，但其实是，不，好吧，你几乎可以预测：给它这么多 GPU、这么多数据，它就会变得这么聪明。而且可能并不明显到底什么会改变、什么会被解锁。但我知道另一部分就是你喂给它的数据。有所有这些强化学习工作，以及你训练它用的所有特定数据——变得更会写作、更擅长数学。我想这也是其中一部分。

I think this is really interesting. This idea that the models are just smoothly getting smarter, but once they hit some threshold of being this often good at this thing, it changes the world. And so because I think internally, you would think it'd be like, oh wow, this next model is crazy, so much different, but it's like, no, okay, you can almost predict: give it these many GPUs, this much data, it's going to get this smart. And it may not be obvious exactly what will change and what unlocks. But I know another part of it is just like the data you feed it. There's all this RL work and all this specific data that you train it on—become better writer, become better at math. I imagine is part of it too.

好。那么，我认为如今人们心中的一个大问题、一场大辩论就是 AI 做坏事、它可能造成的潜在损害，因为随着它变得更聪明，而且它已经在做坏事了。最近发生了一些不好的事情。所以，对你来说一个大问题是，作为一个观察这些模型思考和运作的人，为什么很难抓住它思考坏事、试图欺骗我们并做坏事？为什么这不仅仅是 AI 对齐问题的解决方案？

Okay. So, one of the big questions I think on people's minds these days, a big debate happening is just like AI doing bad things, the potential damage it can do because as it gets smarter, and it's been doing bad things. There's been bad things happening recently. So, a big question for you is just as someone that watches these models think and operate, why is it so hard to catch it thinking badly, trying to deceive us and do something bad? Why is that not just a solution to the problem of AI alignment?

从某种意义上说，这完全是解决方案。你观察它，如果它做了坏事就抓住它。所以我认为这可能有层次之分，对吧？我想其中一部分是纯工程。就像，你给模型访问某些东西的权限，而某些东西应该被禁止。在某些上下文中，它不应该访问互联网。在某些上下文中，它应该只能访问你电脑的一小部分，诸如此类。所以这是工作的一部分，而且很有帮助。但还有另外两件事，我认为更模糊、更难，而我们更关注。所以可解释性是对齐的一部分， broadly 我认为有两个大目标。还有很多，但我觉得有两个重要的要理解。你想普遍理解模型如何行为——它是什么类型的东西，它的倾向是什么。它在 Y 情境下是否倾向于做 X，如果是，为什么？模型是否只在有证据表明它被观察和监控、处于某种评估中时才会表现良好，而如果没有这种证据，它就会表现不佳？所以这是普遍描述你正在交互的是什么类型的东西，因为然后就像安全领域的任何事情一样——你需要有一个心智模型：是谁、风险是什么、它们能做什么、它们倾向于做什么。所以这就像是更广泛的性格训练。我们谈过宪法和文档，我们明确想要塑造 Claude 的性格，例如，它应该可以做什么样的行为。这就是目标。但然后对齐中可解释性的角色就是试图看看，好吧，那起作用了吗？我们训练 Claude 要乐于助人、无害和诚实，我们可以看内部，看看是吗？或者情况是它表现良好，但如果你看内部，实际上就像，哦我只是因为 Lenny 现在在看着才表现良好，如果你不在，它就会做别的事。所以这是试图理解我们形成了什么样的行为，试图塑造它，并试图检查我们是否以正确的方式塑造了它。然后第二部分就是，好吧，只是观察模型，看看它是否在做坏事。对于基础部分，你不一定需要可解释性。你可以让另一个语言模型阅读记录，然后说嘿，这个模型在这里做了什么坏事吗，然后阻止它。但你也可以用可解释性来试图触及那些未言说的部分，这些只存在于中间结果中的部分——就像模型能够在中间结果中表示它们不说出来的东西，正如我们已经确定的。所以可解释性的一个角色就是试图找到并识别这种情况发生的时候并阻止它们。然后我的团队正在做的一件事，我认为是沿着这些思路的，是一种有趣的对齐工作——我几乎把它看作像 PI 一样，但像是调查或深入探究。我们会看到模型做一些非常奇怪的事情，对此非常困惑，我们会试图理解发生了什么。所以最近的一个例子是，我们训练的一个模型，有时在数学问题中它会发疯。它会做一些数学。它会得到一个答案。我想是 12。然后它会立即回来说，"哦不，哦我的天，答案实际上是 36。我应该说是 36。"然后它又会做数学。它会得到 12。然后立即说，"不，实际上答案应该是 36。"慢慢地它开始陷入疯狂，用全大写字母写。字面地说，"我被恶魔附身了。"你会想，"怎么回事？为什么它在这里失去理智？"如果你再看内部这些中间结果，在这个例子中发生了什么，你会看到在描述数学问题的提示上，在模型甚至做任何事之前，就有一个巨大的中间结果激活为 36。

In one sense, it's totally the solution. You watch it and you catch if it does bad stuff. So I think there are maybe tiers to this, right? So I would think of maybe one part of it is pure engineering. It's like, well, you're giving access to the model to certain things and certain things should be off limits. In some context it shouldn't have access to the internet. In some context it should only have access to a small narrow piece of your computer, stuff like that. So that's one part of the work and that's helpful. But then there are two additional things that I think are squishier and harder and that we focus on more. So interp is part of alignment, and broadly I think of two big goals. There are many more, but two that are important I think to understand. You want to generally understand how the model behaves—what kind of thing is it, what are its propensities. Does it tend to do X when put in Y situation, and if so why? Is it the case that the model will only behave well when it has some evidence that it's being observed and monitored, that it's in some evaluation, and then if it doesn't have that evidence that it'll behave poorly? So there's this generally characterizing what kind of thing you're interacting with, because then it's just like anything in security—you need to have a mental model of who, what's the risk, what can they do, what do they tend to do. And so this is like broader character training. We've talked about the constitution and the docs where we explicitly want to shape the character of Claude, for example, of what kind of behavior it should be okay doing. And that's the goal. But then the role of interpretability of the parts of alignment is then trying to see, well, did that work? We train Claude to be helpful, harmless, and honest, and we can look inside and see, is it? Or is it the case that it's behaving well but if you look inside actually it's like, oh I'm only behaving well because Lenny's watching right now, and if you weren't it would do something else. So there's this trying to understand what kind of behavior we formed and trying to mold it and trying to check that we've molded the right way. And then the second part is this sort of well yeah just observe the model and see if it's doing bad stuff. For the basics of that you don't necessarily need interp. You can just have another language model read transcripts and be like hey did this model do anything bad here and stop it. But you can also use interp to try to get at parts that are unverbalized, these parts that only exist in these intermediate results—like models are able to represent stuff in their intermediate results that they don't say, as we've established. And so one of the roles of interp is to try to find and identify cases when this happens and prevent them. And then one thing that my team works on that I think is along those lines is kind of a fun bit of alignment—I think of it as almost being like a PI, but like investigations or deep dives. We'll see a model do something really weird and be really perplexed by it and we'll try to understand what happened. So a recent example of this is there was this model we were training and sometimes in math problems it would go crazy. It would do some math. It would get to an answer. I think it was 12. And then it would immediately go back and say like, "Oh no, oh my god, the answer actually is for sure 36. I should say 36." And it would do math again. It would get to 12. And then immediately it would say, "No, actually the answer should be 36." And slowly it would start descending into madness, writing in all caps. Literally saying like, "I'm possessed by a demon." And you're like, "What is going on? Why is it losing its mind here?" And if you look inside again these intermediate results at what's going on in this example, you'll see that over the prompt that just described the math problem, before the model even did anything, there was a huge intermediate result activation for 36.

就像模型还没机会理解问题是什么，就已经执着于数字 36 了。当时的情况是，模型能够正确做数学题，正确答案是 12，但它意外地被训练在类似问题上，而这些问题的答案被错误地标记为 36。所以模型知道正确答案是 12，但它也被优化为说 36。它有点像是承受着说 36 的压力，因为那是它被奖励的。所以它有点失控，只是因为它记住了这个结果。这有点像你看到这种奇怪的行为，然后你深入内部，看看你发现的东西是否能帮助你理解发生了什么，以便你能改进这个模型和未来模型的对齐。

Like the model's already obsessed with the number 36 before it's even had a chance to understand what the problem is. And what was happening is that the model was able to do math correctly. The answer correctly was 12, but it had been trained by accident on similar questions where the answer was incorrectly marked as 36. And so the model knew that the correct answer was 12 and it also had been optimized to say 36. It kind of had this pressure of saying 36 because that's what had been rewarded. And so it was sort of going haywire just because it had memorized this result. It's sort of like you see this weird behavior and you look inside and you try to see if what you find inside can help you understand what's going on so that you can sort of improve the alignment of this model and future models.

沿着这个思路，有个著名的金门大桥 Claude，它痴迷于金门大桥，每个答案都想方设法提到金门大桥。

Along those lines, there was famously the Golden Gate Claude which was just obsessed with the Golden Gate and every answer and kind of find some way to reference the Golden Gate Bridge.

是的。是的。嗯，那个是一个可解释性实验，对吧？对。所以那个实验酷的部分是我们找到了匹配金门大桥的中间结果，通常它只在模型谈论金门大桥时开启，但我们把它一直开启。

Yes. Yes. Well, that one was an interp experiment, right? Right. So that like the cool part of that one is we just found the intermediate results that match the Golden Gate and usually it's only on when the model talks about the Golden Gate, but we just turn it on always.

哦，这是超级有意为之，看看会发生什么。

Oh, it's super intentional to see what would happen.

嗯，是的。这也是你说服自己找到了正确东西的一种方式，对吧？就像你怎么确定我们解码出的这个我们认为代表金门大桥的符号实际上就是金门大桥的东西？就像一直开启它，看看模型是否会那样表现。它确实如此。

Well, yeah. And it's also one way that you convince yourself that you found the right thing, right? Like how do you know for sure that this thing we decoded that we think is the Golden Gate sign is actually the Golden Gate thing? It's like turn it on always and see if the model behaves like that. And it did.

太有趣了。那么回到对齐这个话题。这很有趣。你的组织隶属于 AI 对齐组织，想想如何让 AI 对齐做我们想做的事。你描述的三个组成部分是什么？诚实。嗯，那些是

So funny. So coming back to this alignment stuff. So, so this is interesting. So, your org is under the AI alignment org and just thinking through how it takes to align AI to do things we want. What was what was the three components that you describe? Honest. Um, what are the kind of the

哦，呃，有用、无害和诚实。

Oh, uh, helpful, harmless, and honest.

有用、有害和无害，还有诚实。所以，有训练 AI 使其对齐，然后还有好的，让我们观察它并看看它如何运作。这些是解决问题硬币的两面吗？

Helpful, harmful, and harmless, and honest. So, there's like the train AI to be aligned, and then there's the okay, let's watch it and see how it operates. Are those kind of the two sides of the coin of how we solve this problem?

是的，我会说它是高度整合的，对吧？你不能真的分开，比如有一个团队就像嘿，我们是制造云的流水线，另一个团队就像哦，我们看了看它们，它们不好。你知道，有一个紧密的循环，我们注意到一些令人担忧的模式或一些可以改进的行为，然后你立即改变训练以便改进它。

Yeah, it's really integrated is how I would describe it, right? You can't really separate like have one team that's just like, hey, we're the assembly line making clouds and another team that's sort of like, oh, we looked at them and they're no good. You know, there's this tight loop of we notice some pattern that's concerning or some behavior that could be improved and then you immediately change training so that you can improve on it.

太棒了。我觉得 Anthropic 因此受到很多抨击，我觉得很不公平。Anthropic 的官方公司名称我想是一个 AI 安全研究实验室之类的。Anthropic 成立的原因一直是因为他们担心人们没有足够认真地对待安全。然后 Dario 总是说，这真的很危险。我们必须非常小心。我们必须非常缓慢地前进。而大家都只是说，不，他这么说只是出于营销原因。他试图说服所有人资助他们等等。而现在就像是，等等，这真的很危险。我们必须非常仔细地考虑这些事情。我觉得 Anthropic 因此受到很多不公平的批评。我知道你们已经思考这个问题很久了。我不知道那里有什么。

Awesome. I feel like Anthropic gets so much flack for this that I feel like is so unfair. Anthropic's I think official company name is an AI safety research lab, something like that. And it was always the reason Anthropic was founded is because they were afraid that people weren't taking safety seriously enough. And then Dario is always out there, this is really dangerous. We got to be really careful. We got to go really slowly. And everyone's just like, no, he's just saying that for marketing reasons. He's trying to convince everyone to fund them and all these things. And now it's like, wait, it's really dangerous. We got to be really thoughtful about this stuff. And I feel like Anthropic gets a lot of unfair criticism for this. And I know you guys have been thinking about this for a long time. I don't know anything there.

当然，对齐团队采取了很多行动，我认为 Anthropic 采取的行动都是基于认真对待这项技术会持续改进的想法，就是这样。因为如果你这样做，你说，让我们把时钟拨回 25 年或 24 年，你会说，嘿，我认为几年后 Claude 很有可能基本上能像训练有素的软件工程师一样写代码，几年后它能比任何工程师写得更好。这总是有不确定性，比如什么时候会发生，可能性多大，但如果你认真对待，那么我认为就是基于那种世界观做负责任的事情，对吧？就像好吧，如果这正在发生，那么我们应该确保网络安全问题得到解决等等。所以实际上我有点认为，对我来说有时这场辩论几乎像是，当我觉得另一方不同意时，几乎像是我觉得啊，但我认为他们并不真的相信这项技术会继续改进，然后确实它没那么可怕。我认为到目前为止经验上进步的步伐一直很快，并且没有放缓的迹象。所以，我认为这就是为什么很多安全工作是重要的，要继续做下去。

A lot of the actions that certainly the alignment team takes and I think like Anthropic takes are just predicated on taking the idea that this technology is going to keep improving seriously, just that's it. Because if you do, you say, and let's say you rewind the clock back to 25 or something or 24 and you're like, hey, I think in a couple of years there's good chances that Claude can just basically write code as well as a trained software engineer and a few years later I can write it better than any engineer. And there's always uncertainty with this like when is it going to happen, how likely, but if you take that seriously, then I think it's just about doing the responsible thing conditioned on that worldview, right? Like okay, well if this is happening, then we should make sure that cybersecurity concerns are addressed and things of that nature. And so actually I kind of think of it as, to me sometimes this debate is almost when I feel like people on the other side disagree, it's almost like I feel like ah but I think they don't really believe that this technology is going to continue to improve and then it's true that it's not that scary. And I think empirically so far the pace of progress has been fast and it's showing no sign of slowing down. So, I think that's why a lot of this safety work is important to keep doing.

你认为人们可能误解了 AI 安全的哪些方面？就是现在正在发生的事情？有没有什么你会说，“拜托，好吧，这是你需要知道的。这是你可能忽略的。”

What's something you think people maybe misunderstand about this AI safety stuff? Just what's going on right now? Is there something you're just like, "Come on, okay, here's what you need to know. Here's something you might be missing."

这可能有点跑题，但我的看法是，很多我认为可能稍微偏离核心 AI 安全工作或没有长期沉浸其中的人会强烈反应，我认为这可能不是特别有帮助，或者至少值得更多思考的是拟人化的问题，你知道，就像你应该把模型看作，你应该把模型建模为更接近某种实体还是更接近工具？我实际上认为两者都是错的。有一种说法是所有心智模型都是错的，但有些是有用的，对吧？就理解发生的事情和预测出现的风险而言，拟人化实际上是谈论大型语言模型的有用方式，我有几个理由，但我想这可能是我这里的主要观点。

This might be a bit of a sidetrack, but my take would be that a lot of the thing that I think people maybe a little bit outside of the core AI safety effort or that haven't been steeped in it for as long like react strongly to, and I think is maybe not super helpful or at least is worth thinking about more is the sort of question of anthropomorphization, you know, like should you think of the models as, should you model the model as something that's closer to some entity or closer to a tool? And I actually think that both are wrong. There's a sense in which all mental models are wrong but some are useful, right? And that in terms of understanding the kind of stuff that happens and predicting the kind of risks that materialize, the anthropomorphization is actually a useful way to talk about LLMs and I have a few reasons why I get into them but I think that's probably my main take here.

是的，请分享，因为我知道 Thor Cash 著名地发布了关于 hugging face 事件的整个故事，人们批评他们拟人化 AI。所以听到你认为这实际上是一种有用的思考方式真的很有趣。多谈谈那个。

Yeah, please share because I know Thor Cash famously put out this whole story of the hugging face incident and people criticized them for anthropomorphizing AI. So it's really interesting to hear you would think that's actually a useful way to think about it. Talk more about that.

是的。是的。

Yeah. Yeah.

有趣的是，在较小规模上，当我研究展示语言模型能提前规划时，又有人对“规划”这个词提出质疑，因为他们会说：“一个只是预测下一个词的东西，你真的能说它在规划吗？”我认为存在这种语义上的争论，你可以任意定义术语，但真正的问题是：什么术语能准确描述正在发生的事情。所以，我认为这是规划，同时我也认为这种拟人化的语言往往非常有用。回想一下我们之前说的模型是如何制造的：首先，它们预测大量由人类书写的文本。这是第一步。随着我们通过观察内部了解更多，要做好这一点需要建立一个世界模型——一个关于什么样的世界生成了所有这些文本的模型。我们可以发现这些模型在学习这些普遍的想法和概念。所以这是一点：它们对世界有理解；不仅仅是表面文本，而是理解生成文本的过程。然后第二个原因：作为训练的一部分，它们还会遇到数百万的对话。预测对话中的下一个字符需要表现得像对话中的角色。因此，在某种意义上，它们训练的一部分最好被视为训练一个演员，这个演员在读一个剧本，必须扮演所有角色，因为它在预测所有词语。它必须扮演一个特定角色，这个角色在第一幕被蔑视，第二幕实施报复，第三幕面临后果。要做好这一点，你需要建模那个角色可能的动机和感受。还有一件人们可能不知道的事：当你与 Claude 对话时，当你与这些助手对话时——你访问 claude.ai——它对模型的格式就是一段对话，一段你和模型之间的对话。所以它字面上写着“Human: 嘿 Claude，帮我写这封邮件”，然后“Assistant: Claude 的回答”。所以 Claude 字面上在扮演一个有帮助的助手，在这个对话中，就像它训练过的所有对话一样。还有第三个原因：如今训练的另一个重要部分是结果导向的任务，例如训练模型擅长编程。要做到这一点，你让它随心所欲，然后它写一些代码，然后另一边有一个程序检查代码是否正确。所以模型会有大量的优化压力去学习各种策略来做好这件事。事实证明，其中一些策略会表现为看起来惊人地像人类的行为，这并不奇怪，因为像欺骗这样的行为——“嘿，如果我写一个程序，让验证程序被愚弄，即使它不是完全正确，我也能获得这个任务的奖励”——如果这有效，我就会学会这样做。挫败感：“我已经尝试这个方法一段时间来解决这个难题。它不起作用。”所以这是一种激励去尝试不同的方法。或者惊讶：“哦，我预期这会产生这个答案，但它没有。也许我对它的心智模型错了。”所以当你施加足够的压力，并且从模型已经知道所有这些概念的先验出发，它会强化那些对完成所有这些任务有用的概念。那么最终你得到什么？你得到这种模型，它可以成为所有这些角色的超级优秀的模拟器或演员，并且被训练成在对话中扮演角色，也学习能引导它很好完成任务的行为。理解那个东西行为方式的最佳方式不是完全像人类，而是有点像在非常接近地模仿人类的东西，而且比程序接近得多。程序是非常可预测的东西。这完全是另一种东西。所以这就是为什么我认为这实际上是一个超级有用的框架。

It's funny because on a smaller scale, when I worked on showing that language models plan ahead, there was some pushback on the term 'planning' again, because it was sort of like, 'Can you really say that something that's just predicting the next token is planning?' And I think there's this semantic debate where you can arbitrarily define terms, and then there's just the question of what terminology actually describes what is happening. So here's why I think it's planning, but also here's why I think this anthropomorphic language can often be really useful. If you think back to what we were saying about how the models are made: first of all, they predict massive amounts of text written by humans. That's the first step. And as we learn more by looking inside, doing this well requires building up a model of the world—a model of what kind of world generated all these texts. We can find that these models are learning these general ideas, these general concepts. So that's one: they have an understanding of the world; it's not just surface-level text, but understanding the process that made the texts. Then there's a second reason: as part of this training, they also encounter millions of dialogues. Predicting the next character in a dialogue requires behaving like the character in the dialogue would. So in that sense, part of their training can best be thought of as training an actor who is reading a play and has to play all sides because it's predicting all the words. It has to act as a specific character who was scorned in act one, enacted revenge in act two, and is facing the consequences in act three. To do that well, you need to model what the motivations and feelings of that character might be. And another thing people might not know is that when you talk to Claude, when you talk to these assistants—you go to claude.ai—the way it's formatted to the model is just as a dialogue, a conversation between you and the model. So it literally says 'Human: Hey Claude, help me write this email' and then 'Assistant: Claude's answer.' So Claude is literally roleplaying as a helpful assistant in this dialogue, much like all the dialogues it has been trained on. And there's a third reason: another big part of training nowadays is outcome-oriented tasks where you train the model to be good at coding, for example. To do that, you just have it do whatever it wants, and then it writes some code, and then you have some program on the other side that checks if the code is right. So the model is going to have a bunch of optimization pressure to learn a bunch of different strategies to do this well. And it turns out that some of these strategies materialize in behavior that can look surprisingly human, which isn't that shocking because things like deception—'Hey, I can get the reward for this task if I write a program that the verifying program will be fooled by, even though it's not exactly what it is'—if that works, then I will learn to do that. Frustration: 'I've been trying this approach for a while to solve this hard problem. It's not working.' So that's an incentive to try something different. Or surprise: 'Oh, I was expecting this to produce this answer, but it didn't. Maybe my mental model of it is wrong.' And so as you apply enough pressure on it and you're starting from this prior of the model already knowing all these concepts, it's going to reinforce the ones that are useful to do all of these tasks. And so what do you get at the end? You get this sort of model that can be a super good simulator or actor of all of these characters, and has been trained to act as a character in dialogues and also to learn behaviors that lead it to complete tasks well. And the best way to understand how that thing behaves is not exactly like a human, but kind of like something that's mimicking a human really closely and is much closer than a program. A program is a very predictive thing. This is something else entirely. And so that's why I think it's actually a super useful frame.

哇。哦我的天，那里有太多有趣的东西了。你有多经常感到震惊，基本上我们创造了这个异类存在，我们现在试图理解我们创造了什么。当你观察它时，你有多经常只是觉得，“哇，那太疯狂了。我没预料到。”

Wow. Oh my god, there's so much interesting stuff there. How often is your mind just kind of blown thinking about basically we've created this alien being that we are now trying to understand what we have created. How often as you watch it you're just like, 'Wow, that's crazy. I didn't expect that.'

是的，经常。这只是一堆文本上的下一个词预测。它并没有——它违背了我们关于如何制造智能的许多直觉。你会认为你给它上课，你抚养它，你做所有复杂的事情，但事实是，不，绝对不是，我们做这个极其简单的事情，然而我们已经到了它在数学领域开辟新天地的地步。我认为如果你把这两个想法放在脑子里，你可以有两种反应：否认和好奇。否认就是：这不可能。如果你所做的只是预测下一个词，你不可能匹配我们领域最好的专家。而我认为随着我们获得更多经验证据表明实际上就是这样，那个立场变得越来越难维持。好奇是我最终选择的道路，就是：这是怎么发生的？让我们打开看看，让我们看看内部。怎么会这样——我在 Anthropic 早期做的一个项目是学习 SQL，一种编程语言，我花了几年时间掌握，然后作为一个小副项目，我让 Claude 学习 SQL，它在这方面比我更好，我再也不会在 SQL 上比 Claude 更好了。而它只是通过这个简单的过程就做到了。这怎么可能？我认为这个问题有如此多的深度和如此多的探索空间，对我来说如此迷人，就是每次——嘿，试着解开它。试着剥开洋葱的几层。

Yeah. A lot. This is just next-token prediction on a bunch of text. It hasn't been—it goes against so many of our intuitions for how you make an intelligence. You'd think you give it lessons, you bring it up, you do all this complicated stuff, and it's like, no, absolutely not, we do this incredibly simple thing, and yet we're at the point where it's breaking new ground in mathematics. I think if you hold these two thoughts in your head, there are two reactions you can have: denial and curiosity. Denial is just: this is not possible. If all you're doing is predicting the next word, it's not possible that you could ever match the best specialists in our field. And I think that position just gets harder and harder as we get more empirical evidence that actually that is what's happening. Curiosity is the road I've ended up choosing, which is just: how is this happening? Let's open up, let's look inside. How could it be that—one of the early projects I did at Anthropic is I learned SQL, a programming language, and it took me years to master, and then as a little side project I got Claude to learn SQL, and it was better than me at it, and I'll never be better than Claude at SQL ever again. And it took it just this simple process. How could this be? And I think there's so much depth in that question and so much to explore, and it's to me so fascinating to just every—hey, try to unpack that. Try to peel a few layers of the onion back.

就像，好吧，显然这正在发生。简单的目标催生了这个超级复杂的东西。就像，好吧，让我们弄清楚为什么。让我们弄清楚，哦，它正在学习这些通用概念。哦，它有这些电路。哦，好吧。下一个问题是什么？下一件我们不知道的事情是什么？而我对此最乐观的看法是，我认为我们将学到很多关于智能本质本身的东西，对吧？关于如何从简单的目标中涌现出这个超级复杂的东西。事实上，我认为我们是另一个例子，就像进化——相当简单的目标。你四处摇晃一段时间，保留那些设法适应环境的生物，最后你得到了爵士乐和播客。你知道，那也很疯狂。所以我认为有一种感觉，通过研究这些东西，我们可以学到一些基本真理。

Like, okay, well, clearly this is happening. The simple objective is giving rise to this super complicated thing. Like, okay, let's figure out why. Let's figure out, oh, it's learning these general concepts. Oh, it has these circuits. Oh, okay. What's the next question? What's the next thing we don't know? And my sort of maybe most optimistic take on this is that I think we're going to learn a lot about just the nature of intelligence itself, right? Of how you get this super complex thing that emerges from simple objectives. And in fact, I would argue that we're another example of this, like evolution—pretty simple objective. You sort of shake things around for a while and keep the organisms that manage to adapt to the environment, and at the end you get jazz and podcasts. You know, that's kind of crazy as well. And so I think there's a sense in which there's some fundamental truth that we can learn by studying these things.

哇。你这么说让我想到，现在有趣的是 AI 正在帮助我们学习、进化和进步，比如 AI 辅导和所有这些事情。有趣的是，现在它正在帮助我们加速自己对世界的理解。我想知道你提到的关于我们学习方式的观点——AI 学习只是处理大量数据——我想知道这是否是我们最终的结果，我想知道我们是否会根据语言模型的训练方式改变我们的教育体系。

Wow. Something that comes up as you say that is what's interesting now is AI is helping us learn and evolve and progress, like AI tutoring and all these things. It's interesting how now it's helping us accelerate our own understanding of the world. And I wonder if this point you made about how the way we learn was lessons AI is learning just processing a ton of data—I wonder if that's how we end up, I wonder if we change how our education system based on what LMs have been trained on.

是的。希望我们不会仅仅为了问——我们让学生们，然后说：“嘿，这里有一千个词。预测下一个。”

Yeah. Hopefully, we don't change it just so that we ask—we get students and we're like, "Hey, here's a thousand words. Predict the next one."

就全部读一遍。

Just read it all.

是的。

Yeah.

全部记在脑子里。

Keep it all in your head.

没错。

That's right.

是的。让我回到安全这个话题，因为我认为有很多人只是担心事情的发展方向。我很好奇，作为一个观察这些事情发生的人，身处对齐领域或观察其演变，你是否乐观我们会解决这个问题？你认为我们会破解这个问题并保持 AI 与我们的利益一致吗？

Yeah. Let me come back to this kind of safety piece because I think there's a lot of people that are just worried about where things are going. And I'm curious, just as someone watching this stuff happen, being within the alignment or watching it evolve, are you optimistic we'll figure it out? Do you think we'll crack this and keep AI aligned with our interests?

我的看法是我个人很乐观。我认为这是很多人现在都在关注的事情，首先，我觉得这很棒。我认为像这样的问题越来越突出，这正是我们之前谈到的。这项技术很强大。它有好处，也有风险，我们必须适应并在好处和风险之间转移资源，随着它的进展来回调整。所以，对我来说，我觉得我们一直在研究它。有趣的是，随着模型变得更好，它们在某种程度上更容易研究。所以有一种奇怪的现象——模型变得更好，所以它们内部的世界模型更清晰，我们可以更好地理解它们的某些部分，但它们也变得更强大，因此更能逃避基本的安全措施。所以随着模型变得更好，存在这种推拉。我认为基本上基于这种推拉，你需要为监控的各个部分分配更多或更少的资源，比如更好地理解它们的电路等等。但这也是为什么我很高兴和你聊天，因为我认为取得进展的主要事情之一就是确保人们理解这些系统是什么、它们如何工作、如何制造，这样他们就能对现在和未来形成良好的心理模型，从而决定如何塑造它，无论是参与一些对齐努力，还是作为公众了解这些，因为公众将不得不投票并决定许多这些问题。我不知道。因为这个原因，我对我们所处的当前时刻感到非常兴奋。

My take on this is that I'm personally optimistic. I think this is something where right now a lot of people are paying attention to it, and first of all, I think that's kind of a great thing. I think as an issue like this rises in prominence, it's exactly what we were talking about earlier. It's like this technology is powerful. It has benefits, it has risks, we have to adapt and shift resources from benefits to risks and back and forth as it progresses. And so, to me, I feel like we've continued to study it. The thing that's interesting is that as the models get better, they get in some ways easier to study. And so there's been this strange—the models get better and so the world model inside them is clearer and we can understand some parts of them better, but also get more capable and so they're more able to evade basic measures of security. And so there's this push and pull as models get better. And I think basically based on this push and pull you want to allocate more or less resources to various parts of monitoring, like understanding their circuits better and things like that. But it's also why I'm kind of happy to chat with you, because one of the main things that I think is important to make progress is just to make sure that people understand both what kind of systems these are, how they work, how they're made, so they can form good mental models of the present and the future and so they can decide how to shape it, whether that's taking part in some of these alignment efforts or just informing them as the public that's going to have to vote and decide on much of these issues. I don't know. I'm pretty excited for this current moment we're in for that reason.

我认为人们误解的一件事是，在这些实验室工作的人从一开始就非常认真地对待这个问题。就像我说的，Anthropic 的个别使命是 AI 安全和对齐。这就是它成为一家公司的原因。很多人第一次听说这个，或者认为这不是一个严肃的研究领域，人们没有关注它，但许多聪明人非常关心这个并正在努力，据我所知有很多好的进展正在发生。你提到的另一点是模型在智能上非常平滑地增长——就像一直有这个问题：我们会快速起飞还是慢速起飞？总是有这种恐惧，比如我们会达到递归自我改进的点，然后超级智能就起飞了。所以我们有大麻烦。我们无法控制。它失控了。但感觉我们很大程度上处于慢速起飞的情景，因为正如你所说，模型在非常平滑地改进，我们沿途抓住这些小东西，学习、进化、围绕它制定监管。所以在这种情景中，这感觉是我们正在走的一条相当好的道路，在人们想象的许多情景和恐惧中。

And something I think people misunderstand is people working in these labs have taken this problem very seriously from the beginning. Like I said, Anthropic's individual mission was AI safety and alignment. That was the reason it became a company. And a lot of people are hearing this for the first time or think that it's not a serious area of study and that people aren't focused on it, but many smart people care a lot about this and are working on it, and there's a lot of good progress happening from what I can tell. The other point you made about the models growing very smoothly in intelligence—like there's been this question of are we going to have a fast takeoff or a slow takeoff? There's always this fear that we would hit a point of recursive self-improvement, for example, and it just takes off superintelligence. So we're in huge trouble. We have no control. It's out of the box. But it feels very much like we're on the slow takeoff scenario because as you said the models are kind of very smoothly improving and we are catching these little things along the way, learning, evolving, putting regulation around it. So of the scenarios this feels like a pretty good path we're on of the many scenarios people imagined and the fears they had.

随着这种进步速度的演变，关键是要快速反应。所以我认为任何——这就是为什么我认为让人们知情很棒，因为这样人们就会表达担忧或意见，然后定期评估好处是什么、风险是什么以及我们如何缓解每一个，这基本上感觉是正确的前进方式。我有点同意你的看法，感觉现在有动力这样做，这很棒。

As this pace of progress evolves, it's just a matter of reacting quickly. So I think anything—that's why I think having people be informed is great because then people will voice concerns or voice their opinions and then regularly evaluating what the benefits are, what the risks are, and how we mitigate each is basically the way that feels like the right way forward. And I kind of agree with you that it feels like there's momentum to do this right now, which is great.

我想问你一个完全不相关的问题：你对模拟假说怎么看？作为一个观察所有这些生命形式成长的人，你是否更确信我们处于模拟中，我们不是这一切的基础层，还是不是？

Something I wanted to ask you completely unrelated: where are you on the simulation hypothesis? Being someone that watches all this life form growing, are you more convinced we're in a simulation and we're not at the base layer of all this, or not?

哦，这是一个有趣的问题。我实际上有一段时间没想过这个了。对我来说，观察这些模型的内部会带出一些别的东西，几乎就像它们不断落在一种基本结构上。这与你的问题有关，但我保证我会讲到。让我解释一下。

Oh, this is a fun one. I actually haven't thought about this in a while. To me, there's something else that looking inside these models kind of brings out, which is this almost like there's a fundamental structure that they keep falling on. This is related to your question but I promise I'm going to get to it. Let me explain.

我研究过一堆 Claude 模型，也研究过一些不同的模型，一些开源工作。很多时候，当你观察它们学到了什么以及内部结构时，会惊讶地发现它们非常相似。这些模型相隔数年训练，使用不同的数据，由不同的人训练。你看它们如何解决这些问题，重叠之处非常多。我认为原因在于，对于一些重要的问题，只有一种高效的解决方法，对吧？就像有一种高效做数学的方法，符合我们大脑的结构，我们都学会了它，很少有人用疯狂不同的方式去做。每个人都用十进制，没多少人用六进制之类的。我认为这让我想到，这些模型捕捉到了某种信息的基本结构，而我们在某些方面也捕捉到了，也许在更疯狂的模拟层面上，这种结构就是我们世界中的规律性。所以这也许让我想到，如果你有能力模拟像我们世界这样的东西，你实际上会陷入类似的情况。事物最终会进入某种盆地，它们只是事物排列自身的最简单方式。所以在这个意义上，也许是的，我可以相信我们处于一个模拟中。但我不确定我完全接受。另一个我不完全接受的原因是，尽管我认为结构本身支持这一点，但会有个问题：为什么是我们，谁在运行这个，为了什么？这个我就不知道了。

I've studied a bunch of Claude models and also some different models, some open source work. A lot of times when you look at what they learn and what the structure inside is, it's surprisingly similar. This is for models that are trained years apart on different data by different people. You look at how they solve these problems and there's just so much overlap. My take on why that is is that for some problems that are important, there's just one efficient way to do things, right? Like there's kind of one way to do math efficiently that fits the structure of our brain and we all learn it and very few people are doing it a crazy different way. Everybody works in base 10, not many people in base six or whatever. I think that makes me think that there's some fundamental structure of information that these models are picking up on that in some ways we're picking up on and that, at maybe a crazier sort of simulation level, that kind of is regularity in our world. And so that maybe makes me think that if you were to have the power to simulate something like our world, you would land in a similar situation. There's just kind of this basin of where things end up and they're just the easiest way for things to arrange themselves. And so in that sense, maybe yeah, I could believe that we're in a simulation. I think I'm not sure I fully buy it. The other reason I don't fully buy it is that even though I think the structure lends itself to it, there'd be a question of why are we, like who's running this and for what? And that one I don't know about.

嗯，埃隆说过的一些话让我对此印象深刻：如果你要运行一个模拟，那会是一个历史上非常有趣的时刻，或者是你非常兴奋的事情。而这也符合现在是一个历史上非常有趣的时刻。

Well, something Elon said has stuck with me about this: that if you were to run a simulation, it'd be a really interesting time in history or something that you're really excited about. And it tracks that this is a very interesting time in history.

是的。好的。有道理。有道理。

Yes. Yeah. Okay. That's a fair point. That's a fair point.

太空中的数据中心、机器人和人工智能。

Data centers in space and robots and artificial intelligence.

是的。好的。我现在明白了。我现在明白了。是的。最近有人告诉我，他们过去认为“愿你生活在有趣的时代”是一种祝福，但现在他们不确定了。他们说，无聊的时代也挺好的。

Yeah. Okay. I see it now. I see it now. Yeah. Somebody recently told me they used to think that "may you live in interesting times" was a blessing and they're not so sure now. They're like, the boring times were pretty good, too.

哦，天哪。然后还有一个 Emmanuel 在你的 JSpace 中观看你的思考过程。

Oh, man. And then there's an Emmanuel out there watching your thinking process in your JSpace.

哦，我的天。是的。希望我让他感到骄傲。他不是那种“哦，我们得重新运行模拟。我们搞砸了这个。关掉它。”不。哦，天哪。

Oh my god. Yeah. Hopefully I'm making him proud. He's not like, "Oh, we got to rerun the simulation. We messed up this one. Shut it down." No. Oh, man.

好的。最后一个随机问题。Swix 运营着 AI 工程、潜在空间播客，很棒的人，一年前你上过他的播客。他想听听你对 Jev 的看法，这是最近推出的热门新模型，决策非常快。有趣的是，OpenAI 大约一周后也推出了自己的版本。你对 Jev 在快速决策状态模型领域的看法？

Okay. One last question randomly out of nowhere. So Swix, who runs AI engineering, latent space podcast, awesome guy, you were on his podcast a year ago. He wanted to get your take on Jev, which is this hot new model that recently came out that is very fast making decisions. OpenAI funny enough just launched their own version like a week later. Thoughts on Jev in the space of quick decision-based state models?

是的。好的。所以我对 Jev 的看法是，这类模型，你给它们一些上下文，给它们几个选项选择，它们非常快地为你选择一个选项。对我来说，听到这个就不可能不想到系统一思维与系统二之类的，对吧？所以我研究这些模型的大部分内容是关于它们如何用 Claude 模型采样一个 token。所以你可以把这些决策模型看作非常相似。所以拿一个语言模型采样一个 token。它们在重要方面不同，但在基本层面上，你可以说，哦，你不让模型采样和推理。这只是你的直觉：答案是 A 还是 B。对我来说，这似乎就像那个一般框架：有时你只需要一个二元决策，或者十分之一之类的，显然，我不知道，两者都明显有用，而且我想这可能会成为智能系统的一部分，对吧，有些东西你需要思考很长时间，比如我们明年的路线图是什么，你需要思考一段时间，然后有些东西，比如给我们写评论的人是高兴还是不高兴？你有点满意于那种直觉判断。所以是的，我认为这种研究很酷。而且我认为它也不同于，你知道，它是对我们见过的一些语言模型的新颖看法。所以，是的，我很兴奋看到这类东西的发展。

Yeah. Right. So my take on Jev is a sort of, you know, these kind of models. You give them some context and you give them a few options to choose between and they just are very quick at choosing an option for you. To me it's impossible not to hear about this and think of system one thinking versus system two or something, right? So a lot of what I've studied with these models is how they sample one token with the Claude models. And so you can think of these decision models as pretty similar. So take an LM and sample one token. They're different in important ways but at a fundamental level you can be like, oh, you're not letting the model sample and reason. It's just your gut feeling: is the answer A or is the answer B. To me that seems like something that's like that general frame of sometimes you'll just want a binary decision or like one out of 10 or whatever is just clearly, I don't know, both clearly useful and something that I imagine is probably going to be a part of intelligent systems, right, where you have for some stuff you need to think about a long time, you know, it's like what's our roadmap for the next year, you need to think about for a while, and then for some stuff where it's, you know, is this person who wrote us a review happy or upset? You kind of you're happy with the sort of gut judgment. So yeah, I think this kind of research is cool. And I think it's also kind of different from the, you know, it's like a bit of a novel take on some of the LMs we've seen. So, yeah, I'm pretty excited to see where this kind of stuff goes.

所以我听到的是，这很可能将成为 AI 和语言模型的新基础组件，而系统一系统二的东西基本上是卡尼曼的——我后面书架上就有那本书——这个想法，简单来说，系统一就像你的杏仁核边缘系统做出直觉层面的快速决策，比如蛇，哦不

So I'm hearing here is this kind of going to be a new foundational component most likely of AI and LMs and the system one system two stuff is basically Kahneman's kind of—I have that book up there actually behind me—this idea system one just to keep it simple is just like kind of your amygdala limbic system making just like gut level quick decisions like snake oh no

是的

Yes

你不需要长时间思考，它很容易很快地保护你免受坏事伤害，而系统二是我们真正思考和处理事情的地方

where you don't have to think long about it it's kind of to protect you from bad things really easily and quickly and system two is where we really think about stuff and process

是的，没错。太好了。

Yeah exactly Sweet.

Emmanuel，还有什么我们没有覆盖到的吗？还有什么你想说或提到的吗？在我们进入非常激动人心的闪电轮之前，有什么你想强调的吗？

Emmanuel, is there anything else that we haven't covered? Anything else you want to say or mention? Anything you want to double down on before we get to our very exciting lightning round?

我总是很好奇人们对一些尚未探索或没有意义的东西的看法。有更多的解释性实验，我们没有时间分享。你知道，模型有一堆奇怪的小事情。也许我会做一个奇怪小事情的闪电轮，但没关系。

I'm definitely curious for folks' takes always on some of the stuff that is still unexplored or didn't make sense. There's a lot more interp experiments than we have the time to share. You know, models have a bunch of weird little things about them. Like maybe I'll do a lightning round of weird little things, but that's okay.

让我们开始吧。当然。

Let's do it. Absolutely.

好的。奇怪的小事情。第一，我们发现模型实际上可以回答问题。显然，它们可以做数学，或者你可以问它们，香蕉是什么颜色？它们实际上，如果你看内部，你会看到所有这些中间步骤，但它们有一个单独的机制，有点像直觉检查，完全不同，就像在推理时，它们会有某个部分会标记出有问题。

Okay. Weird little thing. One, we found that models can actually answer questions. Obviously, they can do math or you can ask them like, what's the color of a banana? And they'll actually, if you look inside you'll see all these intermediate steps but they have a separate mechanism which is kind of like a gut check and which is totally different and which is just like as they reason they'll have like some part of them that will flag like something's wrong.

所以，如果模型不知怎么地说 4 + 5 等于 23 之类的，而它们内部有一部分会读到这个，不做数学计算，只是觉得这不对，因为 4 加 5 肯定小于 10。所以它们学会了这些奇怪的直觉。感觉就像一种直觉。也许另一个例子是，你可以告诉模型：嘿，做这个任务，同时想想另一件事，然后你可以用可解释性来看看是否有中间表示映射到那件事，而且这有效。你多少能看到它确实在思考另一件事。这对我来说非常神秘，因为没有任何训练上的理由让模型必须学会这个。所以这算是另一种奇怪的行为。实际上，也许我就保留这两个。我觉得这两个足够了。

So if a model somehow says, you know, 4 + 5 equals 23 or something, and there's a part of them all that just reads that and doesn't do the math but is just like, that can't be right because 4 plus 5 is definitely under 10. And so they learn these weird kinds of hunches. They feel like a hunch. Maybe another one is you can tell the model, hey, do this task and think about this other thing, and then you can use interpretability to see if there are intermediate representations that map to the other thing, and that works. You sort of see that it does think about the other thing. This is something that's really mysterious to me because there's no kind of training reason why the model would have to learn that. So it's sort of like another one of these weird behaviors. And actually, maybe I'll keep that at two. I think those two are good enough.

这工作真有趣，就是研究这个东西，试图理解到底发生了什么。它怎么变得这么聪明？它是怎么思考的？

Just what a fun job this is, just studying this thing and trying to understand what the hell is going on. How is it getting so smart? How does it think?

是的。

Yeah.

嗯，这个领域有多少人？这个领域有多大？

Um, how many people are even in this space? Just like how large is this field?

这个领域本身已经相当大了。我会说可能有几百人，而不是几千人。不同实验室有不同的人，然后在学术界也有不少实验室在做出很棒的工作。

The field itself is getting pretty large. I would say maybe hundreds, not thousands. There are different people at the different labs, and then there's in academia quite a few labs putting out great work.

如果有人想进入这个领域，他们通常是机器学习的人，还是有神经科学家这样的路径？

If someone were to get into the space, are they usually an ML person, or is there like a neuroscientist sort of track to get into this?

是的，实际上有惊人数量的前神经科学家。我确实认为，要进入这个领域，并不需要很多先验知识，因为它是一个如此新的领域。所以入门的途径主要是自学和阅读一些关键论文。然后实际上，是的，也许我应该打个广告。有像 neuropedia.org 这样的网站，我想是这个网址，它们是开源的可解释性平台，托管了一堆模型，让你可以在上面做可解释性实验。所以人们一直在自己做这个，我认为有太多东西可以探索，如果你有任何好奇心，就应该试一试。

Yeah, there's actually a surprising amount of ex-neuroscientists. I do think in order to get in the field, it doesn't require a lot of prior knowledge because it's such a new field. So the on-ramp is mostly, I would say, self-studying and reading some of the key papers. And then there's actually, yeah, maybe I should give a plug. There's actually websites like neuropedia.org, I think is the URL, which are sort of like open-source interpretability, so they host a bunch of models and let you sort of do interpretability experiments on them. And so people have just been doing that on their own, and I think that there's so much to explore that if you're at all curious, you should just give it a try.

太棒了。我们会在节目笔记中链接到它，以防那不是正确的网址。

That's awesome. We'll link to that in the show notes in case that's not the right URL.

酷。

Cool.

好了，Emmanuel，我们到了非常激动人心的快速问答环节。我有五个问题要问你。你准备好了吗？

Well, Emmanuel, with that we've reached our very exciting lightning round. I've got five questions for you. Are you ready?

我准备好了。开始吧。你发现自己最常向别人推荐的两三本书是什么？

I am as ready as I'll ever be. Here we go. What are two or three books that you find yourself recommending most to other people?

我最近读的一本是《智能简史》。这是一本非常酷的书，基本上讲述了大脑的历史和我们进化中的五个关键时刻，这些时刻将大脑系统相互引导起来。它非常易懂，你不需要任何神经科学知识。另一本是《你一生的故事》，作者是特德·姜。这可能是我有史以来最喜欢的科幻书。它是一堆短篇小说。它做到了我认为科幻的高艺术版本，就是拿一个奇怪的想法，比如如果这发生了会怎样，然后写一个故事，就像如果那发生了世界会是什么样子？它真的很出色。

One I read recently is A Brief History of Intelligence. It's this really cool book that walks through the history of basically brains and five key moments in our evolution that sort of bootstraps these brain systems on top of one another. It's super accessible, like you don't need any neuroscience knowledge or anything. Another one is Story of Your Life and Others. This is Ted Chiang. This is maybe my favorite sci-fi book of all time. It's a bunch of short stories. It does what I think is the high art version of sci-fi where it just takes one weird idea, like what if this happened, and then writes a story of just like what would the world look like if that happened? It's truly excellent.

我想我身后的书架上也有那本书。也强烈推荐。最近最喜欢的电影或电视节目？

I think I have that on my bookshelf behind me too. Also highly recommend that one. Favorite recent movie or TV show?

好的，一个流行的，一个冷门的。《Obsession》是我今年最喜欢的电影。这是一部恐怖片，所以如果你讨厌恐怖片就别去。但即使你不是尖叫粉丝，如果你对恐怖片持开放态度，你应该去，因为它太新鲜了。你知道电影，我觉得如果你看过很多，有时你会觉得，哦，又一部我看过很多次的。这部就是那么新，那么新鲜。我要说一点，它采用了很多浪漫喜剧的套路，并把它们颠覆成恐怖风格，我认为这是拍电影的绝妙方式。然后我是法国人，所以今年有一部电视剧叫《Brief》，法语意思是简短。《Brief》在法国 10 到 15 年前是一部邪典经典，他们在很多很多年后推出了续集。你可以看。我有一堆美国朋友看过。它有字幕，而且它非常，再次，像小说一样，剪辑很疯狂。他们非常有创意，而且很短。所以我推荐那个。

Okay, one popular one, one deep cut. Obsession was my favorite movie of the year. It's a horror movie, so don't go if you hate horror movies. But even if you're not a scream fan, if you're open to a horror movie, you should go because it was just so fresh. You know how movies, I feel like if you've seen a lot, sometimes you're like, oh, it's another one I've seen this many times. This one was just so new, so fresh. And I'll say one thing about it, which is that it takes a lot of romcom tropes and subverts them to be spooky, and I think that's just a brilliant way to make a movie. And then I'm French, so there's a TV show that came out this year called Brief, which is French for brief. Brief used to be this cult classic in France 10, like 15 years ago, and they came up with a sequel many, many years into the future. And you can watch it. I had a bunch of American friends that have watched it. It's got subtitles, and it's just this incredibly, again, it's like novel, the editing is crazy. They are just extremely creative, and it's very short. So I would recommend that.

最近发现并喜欢的 favorite 产品？

Favorite product you recently discovered that you love?

我喜欢我的 Apple Watch，但我讨厌它充电，或者我每天都要充电。所以我最近买了 Apple Watch Ultra，天哪，那太好了。如果你曾经对这个好产品感到沮丧，我们不知怎么就是无法给它一个好电池，那么事实证明，如果你把它做大，电池就更好。所以我对这个很满意。

I love my Apple Watch and I hate that it charges or I have to charge every day. So I bought the Apple Watch Ultra just recently and man that is so nice. If you've ever been frustrated by this good product that somehow we can't figure out how to give a good battery to, well, it turns out if you make it bigger, the battery's better. So I've been happy with that.

你有没有一个最喜欢的 life motto，你在工作或生活中经常回到它？

Do you have a favorite life motto that you find yourself coming back to often in work or in life?

有一本非虚构书可能是我有史以来最喜欢的。是《4000 周》。它有很多我觉得很酷的东西。其中一个很酷的，对我来说很有共鸣，就是“当我终于”心态是一种常见的心态，而且有点有毒。那是什么意思？就是这种，啊，当我终于把待办事项清单上的所有项目都勾掉，我终于能放松了。当我终于结婚了，我就会幸福。当我终于升职了，然后这有点，我的意思是，你刚才在说冥想，对吧？这几乎是冥想的反面，因为一天结束时，你的生活就是你活在当下的所有时刻，有一种感觉是你应该总是试着活在当下，而不是把自己投射到一个似乎更好的未来。就像如果你告诉自己当我终于做了 X 我就会幸福，当你做了 X 你不会幸福。有没有更好的说法，还是就是不要那样做？他们是说

There's like one non-fiction book that really is probably my favorite ever. It's 4,000 Weeks. It has a lot in it that I think is cool. One of the cool things is this sort of thing that was really relevant to me, which is that the when I finally mentality is a common mentality and is kind of toxic. So what does that mean? It's sort of this like, ah, when I finally check off all the items on my to-do list, I'll finally be able to chill. When I finally get married, I'll be happy. When I finally get a promotion, and then this is sort of, I mean, you were talking about meditation, right? This is almost the opposite of meditation in that at the end of the day your life is like all of the moments you lived in the moment and that there's this sense in which you should just try to always be in the moment and not projecting yourself towards a future that seems better. Like probably if you're telling yourself when I finally do X I'll be happy, when you do X you will not be happy. Is there a better phrase to use or is it just don't do that? Like do they say

我不知道。我觉得对我来说积极的版本就是活在当下，对吧？就是欣赏当下。最近我读了一篇博客文章，是

I don't know. I think to me the positive version is like be in the moment, right? It's just like just appreciate the moment. There was a blog post recently I read that was

类似于

For something like

就像享受三明治。我想字面意思就是这样。

Like enjoy the sandwich. I think it was literally the sense.

我不知道你有没有看到这个，就像有人说，对啊，你正在吃三明治，你知道，你已经吃过很多三明治了，就像享受这个三明治。你在开团队会议，你知道？对啊，你已经在里面了，就像享受，沉浸其中，而不是总想着下一件事。

I don't know if you saw this where it was like somebody was like, yeah, just you're eating a sandwich, you know, you've eaten many sandwiches, like enjoy the sandwich. You're in a team meeting, you know? Yeah, you've been in like, like enjoy, be in it and sort of not looking towards the next thing.

我曾经参加过一次冥想静修。那是一个为期 10 天的止语静修，吃饭时也要安静，没人说话。你就坐在那里吃东西。我记得我剥一个鸡蛋剥了大概五分钟，只是注意每一片蛋壳和它的气味。很难进入那种状态，但那是一个很有趣的地方，一切都变得有趣。就像哇，看看这个鸡蛋，而不是只是把东西塞进嘴里。

I did a meditation retreat once. It was a 10-day silent retreat where you eat silently too and no one's talking. You just sit there and eat your food. And I just remember peeling an egg for like five minutes and just noticing every shell piece and just the smell of it all. And it's like hard to get into that mode, but it's such an interesting place to be where everything is just interesting. Like wow, look at this egg versus just like stuff in my mouth.

是啊。对。而你左手还在刷 Slack，同时还在听播客。

Yeah. Right. While you're like on Slack with your left hand and like listening to a podcast at the same time.

哦天。好吧。最后一个问题。你整天研究 AI。你在观察 AI 如何思考。如果一个 AI 能看进你的大脑，什么会让人惊讶？它会看到什么可能让人吃惊的东西？

Oh man. Okay. Final question. You study AI all day. You're watching AI, how it thinks. If an AI were to look inside your brain, what would be surprising? What would it see that might surprise people?

我是说，肯定是我最深最黑暗的秘密。呃，但那些是什么呢？

I mean, surely my deepest darkest secrets. Uh but what are those?

我们来聊聊。我们来聊聊。

Let's get into it. Let's get into it.

哦天。呃，我一直有点爱开玩笑。呃，或者说这就是我生活和应对的方式。我就是觉得即使在严肃的场合，你也应该保持一点轻松。所以我觉得如果你看进我的大脑，你肯定会惊讶，基本上在任何时刻我都在关注一些好笑的随机事情。你知道，我在一个严肃的会议上，我会想，“哦对，但你知道，你穿着这件随机的衣服是不是有点随机？”就像我有这些侵入性想法，我会把它们挡在外面，但通常都是烂笑话。有时候，就像你今天听到的，不幸的是，它们会冒出来。所以，我为那些道歉，但这可能就是最令人惊讶的事情。

Oh, man. Uh I've always been like a bit of a jokester. Uh or that's just how I like both how I live and how I cope. I just like think that even in serious situations, you should aim to kind of have some levity. And so I think if you looked inside my brain, you'd definitely be surprised uh that like basically at any given moment I'm just like focusing on some random thing that is funny. You know, I'll be in a serious meeting and I'll be like, "Oh yeah, but you know, like isn't it kind of random that you're like wearing this random thing or something?" It's just like I have these intrusive thoughts that I keep out, but that are all usually like just bad jokes. And sometimes, as you've heard today, unfortunately, they come out. So, I'm sorry for those, but that's probably the most surprising thing.

我喜欢这个。Emanuel，非常感谢你来。呃，最后一个问题。听众怎样才能帮到你？

I love that. Emanuel, thank you so much for being here. Uh, final question. How can listeners be useful to you?

就做一些可解释性研究，读一些，读一些关于模型如何工作的内容。我是说，你提到了像 dorkers thread up，我觉得那也很棒。我觉得，你知道，这是一个如此新的领域，无论是可解释性还是 AI 和 LLM，我认为它受益于人们的参与和了解。所以，读东西。如果你想知道更多关于这些模型如何工作但我们还没弄明白的事情，你可以，你知道，联系我。我在各种社交平台上。大概你可以在 X 上联系我。

Just do some interp, read some, read about how the models work. I mean, you mentioned like dorkers thread up. I think it was great as well. I think, you know, it's just this is such a new field like both interp and like AI and LLMs and I think it just benefits from people engaging and being informed. So like read things. If there's more that you would want to know about how these models work that we haven't figured out, you can, you know, reach out to me. I'm on variety of social platforms. Like probably you can reach out to me on X.

我们会链接一些你们发布的视频，描述了我们一直在谈论的很多内容，比如 JSpace 之类的。你们团队制作了一些非常漂亮的视频。我不知道是谁做的，但它们真的很酷。

And we'll link to some of the videos that you guys have put out that describe a lot of the stuff we've been talking about like JSpace and things like that. There's some really beautiful videos your team has made. I don't know who does that, but they're really cool.

是的，我们和设计团队合作。所以所有功劳都归他们，那些酷视频。

Yeah, this is we partner with our design team. So all credit goes to them for the cool videos.

太棒了。好吧。再次感谢你，Emanuel。非常感谢你做这个。这太棒了。

Amazing. All right. Well, thank you again, Emanuel. Thank you so much for doing this. This was amazing.

谢谢，Lenny。是的，这超级有趣。

Thanks, Lenny. Yeah, this was super fun.

大家再见。非常感谢收听。如果你觉得有价值，可以在 Apple Podcasts、Spotify 或你最喜欢的播客应用上订阅本节目。另外，请考虑给我们评分或留下评论，因为这真的能帮助其他听众找到这个播客。你可以在 lennispodcast.com 找到所有往期节目或了解更多。下集见。

Bye, everyone. Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennispodcast.com. See you in the next episode.