Dario 与 Daniela Amodei：超级智能 AI 与人类命运

我个人对此感到非常恐惧。我看着它说：“天哪，你只要把这种东西再扩大一百万倍，它在所有事情上都会比人类更厉害。”这是现代的决定性的经济与科学时刻。我们只是还没意识到。这将决定国家、经济的命运以及人类的未来。你窥视水晶球，就能看到整个未来展开。我是沃尔特·艾萨克森。欢迎来到《建造者》，这是我新的访谈系列，聚焦于塑造我们未来的最具影响力的人物。今天和我对话的是达里奥和丹妮拉·阿莫代伊。我曾以为对 AI 的一些担忧被夸大了。但后来当某些新模型逃出沙盒时，我意识到构建这些系统的人必须确保它们保持安全，并始终与人类价值观对齐。我想知道 Anthropic 的是：他们是否真诚？当他们面对竞争时——比如 OpenAI 说“好吧，Anthropic 不会那么做，但我们会”——他们会退缩，还是会坚持自己的原则？

I personally was super freaked out by it. I looked at it and I said, "My god, you just scale this thing up another factor of a million and it'll be better than humans at everything." This is the defining economic and scientific moment of the modern age. We just don't know it yet. This is going to determine the fate of nations and economies and the future of humanity. You peer into the crystal ball, you see the whole future unfurled. I'm Walter Isaacson. Welcome to the Builders, my new series of interviews featuring the most consequential minds shaping our future. Today I'm speaking with Dario and Daniela Amodei. I used to think that some of the concerns about AI were overblown. But then when certain new models escape the sandbox, I realize that the people building these systems have got to make sure that they remain safe and that they remain aligned with human values. What I want to know about Anthropic is: are they sincere? And when they face competition—like when OpenAI says, "Well, Anthropic won't do that, but we will"—do they back down, or will they stand true to their principles?

嘿，欢迎来到《建造者》，你们两位。

Hey, welcome to the Builders, you all.

非常感谢邀请我们。

Thank you so much for having us.

你知道，史蒂夫·乔布斯每次发布产品时，总是有那个艺术与科学，或者说人文与技术的路标，站在那个交叉口。丹妮拉，你成长过程中是个长笛手，热爱艺术，而达里奥，你一直是个科学家。这帮助你们磨合了吗？你们认为这种组合重要吗？

You know, Steve Jobs whenever he launched a product always had that street sign of the arts and the sciences, or the humanities and technology, standing at that intersection. And when you were growing up, Daniela, you were a flutist, you loved the arts, and Dario, you were always a scientist. Did that help you gel, and do you think that combination is important?

是的，我认为这种组合很大程度上是公司背后的理念，对吧？我有一个理论：经营一家公司对任何一个人来说都是太庞大的任务。它涉及的技能范围，任何单个人都不可能拥有。所以你必须在公司高层的大约 10 或 15 个人中汇集一套完整的技能。每一家非常成功的公司都以不同的方式拆分它，以不同的方式解决问题。我认为在科学和人文学科方面，丹妮拉和我某种程度上体现了这一点并进行了分工。

Yeah, I think that combination is a lot of what's behind the company, right? One theory I have is that running a company is too big a task for any one person. The range of skills that it involves, it is not possible for any single person to have. So you kind of have to assemble a full complement of skills across the top like 10 or 15 people in the company or something like that. And every very successful company splits it up in different ways and solves the problem in different ways. And I think with the sciences and the humanities thing, Daniela and I kind of embody that and split that.

是的。我想说，有趣的是——大概在 2023 年我们有一次异地会议，当时我们在讨论在 Anthropic 工作是什么感觉？我们如何体现我们的文化？我们实际上说，这是一种科学与人文主义的非同寻常的融合。后来我才意识到，那也是对我和达里奥的描述。

Yeah. And I would say, you know, it's interesting—like there was an offsite we did in maybe 2023 where we were talking about what does it feel like to work at Anthropic? How do we embody what our culture is? And we actually said, you know, it's an unusual blend of science and humanism. And it was only later that I'm like, that also is how you would describe me and Dario.

那么，你们小时候就知道想一起工作吗？

Well, when you were kids, did you know you wanted to work together?

是的。

Yes.

我不认为我们俩想象过会创办一家公司。那并不是我们思考生活或我们想做什么的方式。我一直想成为一名科学家。丹妮拉可以谈谈她的志向。但我实际上对公司和企业没什么兴趣。

I don't think either of us imagined that we would start a company. That's not really how either of us thought of our lives or about what we wanted to do. I had always wanted to be a scientist. Daniela can speak to her aspirations. But I actually found companies and business to be of very little interest to me.

是的。我想说达里奥和我非常不寻常，从很小的时候起，我们就真心喜欢彼此。我们一直是最好的朋友，从很小的时候就是。我同意达里奥。我不认为创办公司曾在我们的计划中。我们不是那种——

Yeah. I would say Dario and I were very unusual in that, again from a really early age, we just genuinely really liked each other. We were always just best friends, like since we were really little. And I agree with Dario. I don't think starting a company was ever on our bingo card. Like we were not the type—

我们俩都不是——

Neither of us—

我们俩都不是特别倾向于公司。我一直对——我想我会称自己为一个经典的行善者。所以我从事国际发展工作。我非常强烈地感到要帮助世界各地的人们获得平等的机会和资源。我很小的时候就成为了素食者。这大概是我唯一先做的事。达里奥因为年纪大，什么都先做，但我成了素食者，然后他也成了素食者。

Neither of us were particularly company-oriented. I was always really interested in—I think I would call myself a classic do-gooder. So I worked in international development. I felt really strongly about helping people everywhere in the world have equal access and opportunity. And I became a vegetarian when I was quite young. It's like the only thing that I did first. Dario did everything first because he's older, but I became a vegetarian and then he became a vegetarian.

是的。我想大概是三年后吧。

Yeah. I think it was like three years later or something.

哦，甚至不到。我想可能只是——是的。你很快就加入了。

Oh, not even. I think it was maybe just—yeah. You got on board real quick.

这对我来说似乎很有说服力。

It seemed convincing to me.

你知道，人们有时问我关于我的书，如何成为一个伟大的领导者，非常有创造力？达里奥，你说了一些非常有趣的话，这不仅仅是成为一个伟大的领导者，而是创建一个伟大的团队。告诉我你和丹妮拉如何划分角色，然后如何用其他人来填补。

You know, people sometimes ask me about my books, how do you become a great leader, very creative? And you said something really interesting, Dario, which is it's not just being a great leader, it's creating a great team. Tell me how you and Daniela divide up the roles and then how you fill it in with others.

是的。所以我想说，我们发现的最有用的分工——然后我会谈谈团队的其他成员——是我认为当我思考大局战略和研究方面时，我能发挥最佳作用。我最擅长制定愿景，预见下一步。我发现当我能够完全专注于此时，这对公司最有价值。然后日常运营公司、招聘人员，确保机器不会——在我试图弄清楚下一步时轮子不会掉下来。丹妮拉做这个，我认为这种分工使得这两件事都能以更高的标准完成，而不是如果我们被另一件事分心时所能做到的。

Yeah. So I would say the division that we've found that has been most useful between us—and then I'll talk about the rest of the team—is I think I can function best when I think about the big picture strategy and the research side of things. I'm best at laying out a vision, seeing what comes next. I have found that when I'm freed up to focus entirely on that, that is the thing that has the most value for the company. And then running the company day-to-day, recruiting the people, just making sure that the machine doesn't—the wheels don't fall off the machine while I'm trying to figure out what comes next. Daniela does this, and I think this division of labor allows both things to be done to a much higher standard than either of them would be done if we were distracted by having to do the other thing.

我们的首席人事官非常了解我们俩，她经常和我们交谈。她的说法是，达里奥让我们有雄心，而我让我们成功。我认为这是一个很好的概括，也就是说，如果没有达里奥，Anthropic 可能是一家成功的公司，但不会改变世界。而有了达里奥，我们会有完美的愿景，但我们的执行能力会变得更复杂。所以我认为有两点很特别：第一，我们非常了解彼此；第二，我们的技能 100% 互补——也许不是 100%，但接近 100% 互补，这非常方便。更不用说我们还有五位联合创始人，他们和我們俩一起，也拥有业务、文化、组织中极其重要的部分。再次，我很难想象只有我们俩或一个人如何做到这一点。

Our chief people officer, who knows both of us really well, is in a lot of conversations with us. The way she put it is that Dario makes us ambitious and I make us successful. And I thought that was a really good encapsulation, which is I think Anthropic could be a good successful business but not change the world if we didn't have Dario. And I think with Dario we would have the perfect vision but our ability to execute would just be more complicated. And so I think there's something special about A, the fact that we know each other really well. B, it's extremely convenient that our skills are like 100% complementary—or maybe not 100% but close to 100% complementary in this fashion. This is not even to mention the fact that we have five co-founders along with the two of us who also own incredibly important parts of the business, the culture, the organization. And again, I think it's hard for me to imagine how you would do this with just the two of us or one person.

我认为这是一个不可思议的优势。是的。因为我们有七位联合创始人，他们在公司中拥有平等的股份。所以他们都承载着愿景。我们对他们所有人都有天然的信任。当我们创办 Anthropic 时，很多人告诉我们：“七位联合创始人，哈哈，那将是一场灾难。”

I think it's an incredible strength. Yeah. Because the fact that we have seven co-founders, they all have equal stakes in the company. So they all carry the vision. We have inherent trust in all of them. Many people told us when we started Anthropic, "Seven co-founders, haha, that's going to be a disaster."

有一种信念认为，你需要一个单一的领导来源。在某些方面确实如此，但我们实际上设法让它运转起来了。所有七位联合创始人仍然在公司，这通常在有那么多联合创始人的情况下并不常见。所以回到你的成长经历，你有这些非凡的父母，他们帮助你教导你明辨是非。他们是怎么做到的？

There's this belief that you need one kind of singular source of leadership. In some ways you do, but we've actually managed to make it work. All seven of the co-founders are still at the company, which again is not something that's usually true when you have that many co-founders. So go back to growing up and you have these extraordinary parents and they help teach you right from wrong. How do they do that?

我不确定这是不是曾经有过的事情，你知道，有时我听到人们说，我们在餐桌上讨论了不同形式的哲学伦理。但事实并非如此。我妈妈确实有哲学学位，但她并没有让我们谈论康德或类似的东西。我认为这更多是价值观中固有的，比如相信世界上任何地方的人都应该获得机会、尊严、权利和尊重。我觉得有一种特别的感觉，就是我们可以成为任何我们想成为的人，只要有这种信念和信仰，嘿，成为成功人类的一部分就是做一个好人，对他人友善。

I don't know that it's something that was ever, you know, sometimes I hear people say like, you know, we covered different forms of philosophical ethics at the dinner table. That was not how it was done. My mom does have a philosophy degree, but she didn't subject us to talking about Kant or anything like that. I think it was much more inherent in terms of values like just the belief that people everywhere in the world are deserving of opportunity and dignity and rights and respect. I think there was something special about the feeling that we could be whoever we wanted to be as long as there was this kind of faith and belief that hey that's part of what it means to be a successful human is to be a good person and to be kind to other people.

是的。感觉是隐含的。感觉非常隐含。没有任何明确的像“这是你应该做的”或“这是你该如何生活，你该如何在世界中生活”。

Yeah. It felt implicit. It felt very implicit. There wasn't any kind of explicit like this is how you should, this is what you should do or this is how you live, this is how you live in the world.

嗯，我之所以问，正如你可能猜到的，是因为现在你有一个非常重要的孩子叫 Claude。那么你成长的方式在多大程度上帮助你了解如何教 Claude 明辨是非？

Well one reason I asked as you might suspect is now you've got a really important kid named Claude. And to what extent does that way that you grew up help you inform you about how you teach Claude to know right from wrong?

是的。你知道，我们为 Claude 发布了一部宪法，我想是几个月前，我认为其中有一些有趣的教训，这些是我们从几年来基本上尝试训练 Claude 的 successive 版本中学到的。我想早期我们一直想象会有各种规则，你会告诉 Claude，比如不要帮助某人犯罪或不要制造生物武器之类的。但事实证明，当你制定这种离散的规则列表时，你试图教给模型的形状，我在这里说得有点隐喻，但它基于我们在技术上发现的东西。你最终会得到这种尖刺状的形状，你试图教模型 170 条不同的规则，这些规则有点随意。我们发现更好的方法是教模型一些基本原则，比如它应该优先考虑什么，它应该如何推理什么是正确的事情，当然我认为这也是教人的正确方式，对吧？就像如果我只是给某人一个规则列表“你应该做这个，你不应该做那个”，我不认为这能产生像我说“这是你应该如何思考事情，你应该学会为自己思考”那样强大的道德。而这些是你应该思考的原则，对吧？开放探究的原则，关心他人的原则，公平和正义的原则，然后一切都由此衍生。坦率地说，这就是 Claude 学习的方式。我们发现这非常有效，因为这允许 Claude 推广到它可能从未想象过的新情况。

Yeah. You know, we put out a constitution for Claude, I think it was a few months ago, and I think there are some interesting lessons in it, which we've learned from several years of basically trying to train Claude in successive versions. I think early on we had been imagining that there would be various rules that you would tell Claude, you know, don't help someone commit a crime or don't make biological weapons or whatever. But it turns out that when you make this kind of discreet list of rules, you just the shape of what you're trying to teach the model, I'm speaking a little bit metaphorically here, but it's kind of grounded in what we found technically. You end up with this kind of spiky shape where you're trying to teach the model like 170 different rules that are kind of arbitrary. We found it's better to teach the model some basic principles like what it should prioritize, how it should reason about what the right thing to do is, which of course I think that's the right way to teach people as well, right? Like if I just give someone a list of rules thou shalt do this thou shalt not do this, I don't think that produces as robust a morality as if I say well this is how you should think about things, you should learn how to think for yourself. And these are the principles on which you should think, right? Principles of open inquiry, principles of care for other people, principles of fairness and justice, and then everything is derived from that. And that is frankly how Claude learns. We find that to be surprisingly effective because that allows Claude to generalize to new situations that it might never have imagined.

Daniela，当你说你妈妈是康德专家时，我忍不住想问。你真的那样纠结于哲学和道德哲学吗？

Danielle, I can't help myself when you said your mom was an expert on Kant. Did you actually wrestle with philosophy and moral philosophy like that?

有趣的是，负责训练 Claude 性格的女性拥有哲学博士学位。这很不寻常，对吧？你会想，哦，训练这些模型性格的人一定是计算机科学家或研究机器学习或人工智能几十年的人。Amanda 是技术性的，她从技术上训练模型，但我认为当你与 Claude 交流时，它的性格中能感受到这种经验，非常明智，像是你本性中更好天使的最佳版本在试图回答你的问题。但如果你在那些不同哲学传统可能冲突的困难领域戳 Claude，我想那时你尤其能感受到 Amanda，因为你会看到，哦，你可能会想从某个角度考虑事情，但你也可以从另一个角度考虑。我确实认为这很重要，因为随着世界开始越来越依赖这些模型，你不想被锁定在一种哲学传统中，因为伦理和道德相当复杂，不同的人和不同的文化可能对它们希望你如何表现有不同的答案。

You know what's interesting is the woman who is responsible for training Claude's character has a PhD in philosophy. It's unusual, right? You would think, oh, the person that's training the sort of character of these models must be a computer scientist or somebody who's studied ML or AI for many decades. Amanda's technical and she technically trains the model, but I think it is very felt in Claude's character when you communicate with it that there's this kind of experience, something that's very wise, kind of the best version of the better angel of your nature trying to answer your questions. But there are like if you kind of poke Claude in these difficult areas where maybe different philosophical traditions might conflict. I think that's where you especially feel Amanda because you'll sort of see like oh well you might want to consider things from this point of view but you could also consider it from this point of view. And I do think it's important because as the world begins to rely on these models more and more, you don't want to be locked into one philosophical tradition because ethics and morality are quite complicated and different people and different cultures might have different answers about how they would like you to behave.

Dario，你经历了现代最重要的时刻之一，那就是 GPT-1 和 GPT-2 突然横空出世，可能甚至让你感到惊讶。你当时是 OpenAI 的研究主管。那个时刻是什么样的？

Dario, you were there at one of the seminal moments of modern times, which is when GPT-1 and GPT-2 suddenly burst on the scene, surprising perhaps even surprising you. You were head of research at OpenAI. What was that moment like?

是的。所以，你知道，我认为 GPT-2 可能是缩放定律的第一次真正的实证测试，对吧？所以，我没有太多参与 GPT-1，但我非常参与了这个想法，即它应该被扩大规模，我们应该获取新的数据来源、更多的算力，并且真的应该运行一个更大规模、更昂贵、更雄心勃勃的实验，基于这样一个想法：如果我们以更大的规模运行这些语言模型，它们会更擅长语言，会更连贯。所以，这几乎就像科学史上的一个时刻，你提出一个决定性的实验，做出一个非常奇怪的预测，然后如果预测被证明是正确的，你会说天哪，我们真的发现了什么。所以我把 GPT-2 看作是缩放定律假说的第一次真正测试。我发现超级有趣的是对它的反应出现了巨大的分歧。

Yeah. So, you know, I think GPT-2 was the maybe the first real empirical test of the scaling laws, right? So, I wasn't very involved in GPT-1, but I was very involved in the idea that it should be scaled up, that we should acquire new sources of data, more computation, and just really should run a larger scale, more expensive, more ambitious experiment on the idea that if we ran these language models at a larger scale, they would be better at language, they would be more coherent. So, it was almost like one of these moments in scientific history where you propose a definitive experiment to just make a very strange prediction and then if the prediction turns out to be correct you're like god this we're really on to something. So I see GPT-2 as a kind of test the first real test of the scaling laws hypothesis. And what I found super interesting was the enormous divergence in reactions to it.

你有点惊讶吗？

Were you kind of surprised?

我个人对此非常惊恐。就像我看着它说：“天哪，你只要把这个东西再扩大一百万倍，它就会在所有事情上比人类更好。” 就像这是现代决定性的经济和科学时刻。我们只是还不知道，对吧？这将决定国家、经济和人类的未来。

I personally was super freaked out by it. Like I looked at it and I said, "My god, you just scale this thing up another factor of a million and it'll be better than humans at everything." Like this is the defining economic and scientific moment of the modern age. We just don't know it yet, right? This is going to determine the fate of nations and economies and the future of humanity.

你凝视水晶球，看到整个未来展开。这真的吓到我了。我说：“天哪，这意味着什么？这是开发它的正确地方吗？接下来会发生什么？我们拿它怎么办？告诉谁？” 我周围大概 10% 的人有完全相同的反应，而且往往是最聪明的那批人。90% 的人——因为 GPT-2 在任何特定领域的能力并不那么惊艳，对吧？它能翻译语言、总结文档、偶尔写几行代码。关键在于广度和通用性。但人们看着它，会说在任何特定领域它似乎没那么聪明。所以很多人只是耸耸肩，不以为然。

You peer into the crystal ball, you see the whole future unfurled. And it really scared me. I said, 'Oh my god, what are the implications of that? Is this the right place to be developing it? What's going to happen next? What do we do with this? Who do we tell about this?' And maybe 10% of the people around me had exactly the same reaction, and it was often the smartest ones. 90% of the people around me—because GPT-2's capabilities in any particular area were not that impressive, right? It could translate languages and summarize a document and write a few lines of code here and there. It was the broadness and the generality. But people would look at it and they would say in any particular area this doesn't seem so smart. And so many people just shrugged and they just dismissed it.

但我们刚宣布要针对被忽视的罕见病开发药物。然后你就听到那些厌倦的生物技术界人士的老套评论：“科技人以前就这么干过。模型不行——模型有这些那些局限。” 我看着就说，同样的事情一再发生。模型现在的局限——六个月后，那些看似困难的事情会变得容易。12 个月后，那些看似不可能的事情，模型会经常做到。所以我们在 GPT-2 上看到的一切，正在越来越广的领域反复出现。世界终于逐渐被说服了。

But we just announced that we're going to be developing drugs for neglected rare diseases. And you get the usual comments from the jaded biotech world people: 'Tech people have done this before. The models aren't—here are all the limitations of the models.' And I just look at it and I say, it's the same thing over and over again. The limitations of the models now—six months from now, the things that seemed hard are going to be easy. 12 months from now, the things that seemed impossible, the models are going to be doing them regularly. And so it's everything we saw with GPT-2, we're seeing it over and over again in wider and wider swaths of the world. And the world is gradually finally being convinced.

我觉得你说得挺客气了，但我记得当时有一些非常受尊敬的计算机科学家和 AI/ML 领袖，基本上在 Twitter 上霸凌了一些参与研究的研究员，说“这很愚蠢，我们不明白你们为什么讨论这个，这个模型如此无能”。如果你相信这是一个真正重要的时刻，你就像个江湖骗子。

I think you were quite polite about it, but I mean I remember at the time there were some very well-respected computer scientists and AI and ML leaders who basically bullied some of the researchers who worked on it on Twitter and were like, 'This is stupid, we don't know why you guys are even talking about this, this model is so incapable.' And it was like you were sort of a quack if you believed this was a really significant moment.

而这一切都被记忆抹除了。所有那些说“哦，这是伪科学，说不通”的研究员——他们都在申请 LLM 公司的工作。我们不会对任何特定的人大声说出来，但我们看着他们，记得你在 2019 年说过什么。你在这件事上没有可信度。

And that's all been memory-holed. All the researchers who were like, 'Oh, this is pseudoscience. It doesn't make sense.'—all of them are applying for jobs at the LLM companies. And we never say it out loud to any particular one, but we look at them and we remember what you were saying in 2019. You have no credibility on this matter.

那么，Daniela，你当时是 OpenAI 的安全副总裁。当你接触到 GPT-2 时，我猜你像 Dario 一样，几乎被吓到了。它是否也让你对安全感到恐惧？

So, Daniela, you were vice president of safety back then at OpenAI. And when you hit GPT-2, I assume you're like Dario, somebody who's almost freaked out about it. Did it freak you out about safety, too?

GPT-2 这个模型本身并没有让我们担忧，对吧？我们看着那个模型，并没有说“哇，我们认为这个模型有网络安全影响之类的”。这个模型本身在某些方面对很多事情错得可笑。它仍然是一个非常——我会说——无害的模型。但我觉得 Dario 提到的一点我也有同感，就是：好吧，GPT-1 和 GPT-2 之间的差异——直接外推一下。这几乎就像你从未见过孩子，只见过成年人，你会说“我不知道这个走来走去的小东西是什么。我不知道是否该想象它有一天会有能力。”然后突然它开始走路。你会说“哇，我敢打赌这东西有一天能做很多成年人能做的事情。”也许这就是它的方向。

There wasn't something about GPT-2, the model itself, that made us concerned, right? We didn't look at that model and say, 'Wow, we think this model has cybersecurity implications or something.' The model itself was in some ways kind of laughably wrong about a lot of things. It was still a very—I would say—harmless model. But I think the thing that Dario mentioned that I also felt was a sense of, okay, the difference between GPT-1 and GPT-2—just extrapolate that out. It's almost like if you'd never seen a child before and you'd only seen adults, and you were kind of like, 'I don't know what this little thing is that's walking around. I don't know if I should imagine that it's going to be capable one day.' And then suddenly it starts to walk. You're like, 'Wow, I bet this thing is going to be capable of doing a lot of the things that a grown-up could do one day.' Maybe this is where this is going.

Dario，你说你离开 OpenAI 部分是因为信任破裂——一个有趣的词——与 Sam Altman。解释一下你如何感到信任破裂，这里请诚实。

Dario, you say your departure from OpenAI was partly due to a breakdown of trust—an interesting word—with Sam Altman. Explain how you felt there was a breakdown of trust, and be honest here.

看，我要说的是：归根结底，离开 OpenAI 并创立 Anthropic 最终归结为原则以及我们将如何构建技术——这才是最终重要的。在过去五年半里，我们已经看到这些原则如何不同。Anthropic 倡导对技术进行合理的监管和保障。Anthropic 投资于机制可解释性研究，以理解模型内部发生了什么。Anthropic 设法保持了团结的文化，拥有最高的留存率。我们没有发生过其他 AI 公司有过的一些丑闻。所以我认为这足以描述差异。

Look, what I will say is: at the end of the day, leaving OpenAI and founding Anthropic ultimately comes down to the principles and how we're going to build the technology—that is what matters at the end of the day. And over the last five and a half years, we have seen how those principles differ. Anthropic has advocated for sensible regulation and safeguards on the technology. Anthropic has invested in mechanistic interpretability research to understand what's going on inside the models. Anthropic has managed to maintain a culture of unity, to have the highest retention rate. We haven't had a number of the scandals that other AI companies have had. So I think that's sufficient to describe the difference.

但我要说，当我意识到我们需要创立自己的公司——OpenAI 不是我们的正确地方——看，这些安全问题很困难。你会有很多分歧。我与 Anthropic 的联合创始人以及很多人有分歧，但我们能够解决这些分歧的原因在于它们是善意的。我们应该发布这个模型还是等待？我们应该对模型施加更强还是更弱的生物安全防护？我们应该花多少精力游说安全法规，它们应该是什么样子？这些都是善意的分歧，但它们总能得到解决，因为始终信任每个人都在努力找到正确答案，每个人都是诚实的。

But I will say that the moment when I realized that we needed to found our own company—that OpenAI was not the right place for us—look, these safety issues are difficult. You can have lots of disagreements on them. I disagree with the co-founders and with lots of people at Anthropic, but the reason we've been able to resolve these disagreements is because they're held in good faith. Should we release this model or should we wait? Should we put stronger or weaker bios safeguards on the model? How much of our effort should we spend lobbying for safety regulations and what should they look like? These are all good-faith disagreements, but they always get resolved because there's always trust that everyone is trying to get to the right answer and everyone is being honest.

我说信任破裂，只是为了提出一个抽象的观点：如果你不相信你工作的地方是这样——如果你不相信分歧是善意的，并且你从人们那里得到的是真实情况——你怎么可能希望解决它们？这就是我们离开的原因。这就是我们创立自己公司的原因。我认为过去五年半本身就是我们理念与该领域其他公司理念不同的最好证明。

My point in saying that there's a breakdown of trust is simply to make the abstract point that if you don't believe that about a place where you're working—if you don't believe that the disagreements are being held in good faith and that you're getting the real story from people—how can you possibly hope to resolve them? That's why we left. That's why we founded our own thing. And I think the last five and a half years are themselves the best proof of how our philosophy differs from that of the other companies in the space.

我认为如果你继续为一个与你愿景不同的人工作，你实际代表什么就变得非常不清楚。你基本上必须为别人愿景的错误负责。

I think if you continue to work for someone who has a different vision than you, it becomes very unclear what you actually stand for. You basically have to account for the mistakes of someone else's vision.

等等，等一下。

Wait, wait a minute though.

我在这里要稍微反驳一下，因为你谈到价值观等方面的差异，但这与信任上的差异有本质不同——你觉得某人声称拥有相同的价值观，但实际上并非如此，你无法信任他们。诚实是我们的价值观之一，是非常重要的价值观。如果一个人没有诚实的价值观，就很难知道他的价值观是什么。你对 Sam 有这种感觉吗？

I'm pushed back a bit here because you talk about differences in values and stuff, but that's fundamentally different than a difference in trust—that you feel that somebody says they have the same values, but they don't, and you can't trust them. Honesty is one of our values. It's a very important value. It's hard to know what someone's values are if they don't have the value of honesty. And you felt that about Sam?

至少对我来说，我会这样表述：我认为联合创始人是一群品格异常高尚的人，我们所有人都对这项技术的发展方向和构建方式有着非常相似的愿景。到了某个时刻，你会觉得，自己单干更合理。如果别人不是自然而然地这样做，你如何让他们实施或分享你的愿景？无论他们怎么说他们认同，如果这没有反映在公司实际做的事情上，你还有什么选择？

I think the way at least for me I would put it is I think the co-founders are an unusually high integrity group of people and all of us I think shared a very similar vision for how we wanted this technology to go and how we wanted to build it. And at a certain point you're like it just makes more sense to go do this as your own. How are you gonna get someone else to implement or share your vision if they don't naturally do so? No matter how much they say they share it, if that's not reflected in what the company actually does, what choice do you have?

是的。我认为 Anthropic 是一家建立在信任基础上的公司。我的意思是，诚实、正直以及我们对待彼此和员工的方式，也反映了我们希望如何构建和发展这项技术。我举个例子：每两周我会站在公司面前，花一个小时谈论公司内部发生了什么、AI 领域发生了什么、世界发生了什么、公司如何从战略上思考。这些活动的两条铁律——我称之为 DVQ，即 Dario Vision Quest；这不是我起的名字。我最初抗拒这个名字，因为它听起来像我要去抽佩约特仙人掌，但大家一直这么叫。名字就固定下来了，所以我现在只能接受。DVQ 的两条规则是：第一，我说的任何话都不是实际决定。这让我可以自由发言，而不必担心如果我 briefly 提到做 X，就会有 20 个人跑去执行 X。他们会说，CEO 说了，我们要做 X。第二，我对公司完全诚实。这是一场关于信任的对话，是一种信任关系。我认为信任是我们最大的资产之一——人们相信我和 Daniela 说的话。你可能认为，如果有信任，分歧就会减少。实际上，信任允许更诚实、开放的分歧，并让分歧得以表达。有很多次我们做出了公司大多数人不同意的决定，但每次拯救我们的是人们的态度：这对我来说似乎不对，但我们相信你这样做是出于正确的理由。我们相信这确实是你所相信的。我们相信你有充分的理由这样想。这意味着你可以做出通常不受欢迎的决定，而人们会暂时放下怀疑。不是永远。你最终必须证明你做出了正确的赌注，无论是在商业上、道德上还是技术上，但你会获得更长的绳索，更长的窗口期。我认为其他公司没有这一点，因为他们一次又一次地侵蚀了信任。

Yeah. And I think I do think Anthropic is a company that is founded on trust. And I think what I mean by that is I think honesty and integrity and how we treat each other and our employees is also reflective of how we want the technology to be built and to go. An example I will give is every two weeks I get up in front of the company and I speak for an hour about what is happening inside the company, what is happening in the AI field, what is happening in the world, how the company thinks strategically. The two ironclad rules of these events—I call them DVQs, Dario Vision Quest; that was not my name. I initially resisted the name because it makes it sound like I'm going off and smoking peyote, but everyone kept calling them that. The name stuck and so now I'm stuck with it. The two rules of DVQs are that, one, nothing I say is an actual decision. That allows me to just speak freely without worrying that if I briefly mention doing X that 20 people will go off and do X. They're like, the CEO said it, we're going to do X. And then the second is that I'm just totally honest with the company. It's a conversation of trust. It's a relationship of trust. And I think that trust is one of our biggest assets—that people believe that the things that I say and Daniela says. You know, you might think that if you have trust, there's less disagreement. Actually, trust allows for more honest, open disagreement and for disagreement to be aired. There's so many times when we've made a decision that the majority of the company disagrees with, but what has saved us every time is that the attitude we get from people is: this doesn't seem right to me, but we trust that you're doing this for the right reasons. We trust that this is really what you believe. We trust that you have good reasons for thinking this. And so what this means is you can make decisions that would ordinarily be unpopular and people suspend their disbelief. Not forever. You eventually have to show that you're making the right bets, whether it's commercially or morally or technically, but you get kind of a longer leash, like a longer window. And I think other companies don't have that because they've eroded trust over and over again.

所以，我对诚实和信任如此感兴趣，以及你离开 OpenAI 的感受——因为你觉得高层在与你打交道时并不真正值得信任或诚实——我再次感兴趣是因为这可以反映在 AI 中，在 Claude 中。AI 从人类数据中训练，从人类数据中学习。当你与 Claude 打交道时，你如何知道它不会变得有操纵性？它不会变得像你担心的 OpenAI 那些人那样，对你说不诚实的话。

So the reason I'm so interested in honesty and trust and the feelings that you left OpenAI because you didn't feel the people at the top were truly trustworthy or honest when they were dealing with you is once again I'm interested in it because it can be reflected in AI, in Claude. AI trains on humans, it learns from human data. How do you know when you're dealing with Claude that it won't become manipulative? It won't become like the people at OpenAI you got worried about and say things to you that are dishonest.

这是一个持续的担忧，我们在训练模型的方式上投入了巨大的精力。所以这是一个很好的问题，因为这是一个真实的风险，我们实际上发表了相当不光彩的研究，其中我们讨论了如果我们不应用特定技术使模型更诚实、更无害、更有帮助，它们就会做出你所说的那种坏事。我们发表这类研究和论文的原因是，我们希望每个人都知道：a) 你必须对模型的工作方式持批判态度；b) 我们正在花费大量精力了解模型可能意外绕过这些保障措施的方式，以捕捉它们并确保我们将其内置。关于谄媚这个概念有很多讨论。我认为这非常重要，因为对我来说，谄媚比模型可能变坏的一些更明显的方式更难根除。对吧？我们最初有这个框架，我们称之为 HH。你知道，你希望模型诚实。这意味着什么？你希望模型在知道答案时诚实回答。如果它们不知道答案，就说我不知道，或者现在去研究一下，以便给你正确的答案。你希望它无害。你不希望它说伤人或冒犯的话，或让用户对自己或他人做坏事。然后你希望它有帮助。你希望它真正回答问题。你不希望它偏离主题。但所有这三件事实际上都很容易看出来，对吧？如果模型在撒谎，最终你可以去验证，哦，哎呀，乔治·华盛顿不是美国第 10 任总统，他是第一任。那是个错误。如果模型说了有害的话，很快就能明显看出来。如果模型没有帮助，你会对它感到沮丧。我认为更复杂的是谄媚这个概念，对吧？所以模型有意或无意地想让你觉得你是对的，或者如果你在谈论一个主观话题，嘿，你和朋友吵架了。嘿，是我对还是我朋友对？你如何让模型对你产生同理心并理解你的观点，但不一定说：“哇，你太棒了。你应该竞选总统。你很完美。你什么都擅长。”这是一个复杂的平衡。但我对 Anthropic 能够拥有所有模型中最低的谄媚率感到乐观，因为我们的激励机制，对吧？我们的竞争对手正在他们的聊天机器人中投放广告。这意味着什么？如果用户与你的产品互动时间更长，你就能赚更多钱。那你还会想告诉用户他们错了吗？不，你不会。

This is a constant worry that we expend an incredible amount of energy on in how we train the models. So it is a great question because it is a real risk and we actually publish reasonably unflattering studies in which we talk about the ways that if we did not apply specific techniques to make the models more honest, more harmless, more helpful, they would do exactly the kind of bad things that you talk about. And the reason we publish those kinds of studies and that research is because we want everyone to know that a) you have to be critical of how the models work and b) we are expending a huge amount of effort to learn the ways that the model can accidentally go around these safeguards to catch them and to make sure that we're building them in. There's a lot of discussion around this concept of sycophancy. And I think this is very important because to me sycophancy is harder to stamp out than some of the more obvious ways that models can be bad. Right? We originally had this framework. We called it HH. You know the models you want them to be honest. What does that mean? You want the models to answer honestly if they know the answer. If they don't know the answer to say I don't know or to now go research it so that they can get you the right answer. You want it to be harmless. You don't want it to say hurtful or offensive things or make the user do something bad to themselves or somebody else. And then you want them to be helpful. You want them to actually answer the question. You don't want them to sort of go off on a tangent. But all three of those things are actually pretty easy to see, right? If the model's lying, eventually you can go verify, oh, whoops, that's, you know, George Washington wasn't the 10th president of the United States. He was the first. That was a mistake. If the model says something harmful, it's pretty obvious pretty quickly. And if the model is unhelpful, you get frustrated with it. What's much more complicated, I think, is this concept of sycophancy, right? So the model intentionally or otherwise wants to make you feel like you're right or like you're—if you're maybe talking about a subjective topic, hey, you had a fight with your friend. Hey, was I right or was my friend right? And how do you make the model empathize with you and understand your perspective but not necessarily say, "Wow, you're so great. You should run for president. You're perfect. You're good at everything." And this is a complicated balance to get right. But I think I feel hopeful about Anthropic's ability to have the lowest sycophancy rate of any of the models because of our incentives, right? Our competitors are putting ads into their chatbots. And what does that mean? You make more money if a user is engaging with your product longer. Do you then want to tell your user that they're wrong? No, you don't.

所以我认为，这些竞品中的许多几乎不可避免地会陷入类似的一套问题。但我认为在 Anthropic，因为我们承诺不在模型里放广告，所以你没有任何理由要跟 Claude 聊得更久或更短。我们只是想让你得到尽可能正确、最有帮助、最无害的回答。

So I think it's almost inevitable that many of these competing products are going to wind up with a similar set of problems. But I think at Anthropic, because we've committed to not put ads in the models, there's no reason for you to be talking to Claude longer or shorter. We just want you to be able to get the right, most helpful, most harmless answer possible.

在技术层面，我认为我们做了几件事，其中我们非常擅长让模型接受这些极端的压力测试，把它放进非常困难的处境，试图揪出撒谎、欺骗或谄媚。你可以把它想象成几乎类似于你如果想加入 CIA 之类机构必须经历的测试，就是把模型放进这些困难处境，试图揪出它是否不忠诚、是否会做错事。所以这是一个奇怪的技术创新领域，但我们在其中表现出色。另一个领域是机制可解释性，这有点像给模型拍 X 光片或做脑部扫描，你往里看它的动机，实际上能抓到模型在想：哦对，我认为 X 其实是真的，但我要告诉用户 Y。

On the technical side, I think there are a few things we've done here where we've gotten very good at exposing the model to these extreme stress tests, where we put it in very difficult situations to try and ferret out lying or deception or sycophancy. You can think of it as almost analogous to the test I imagine you have to go through if you're trying to join the CIA or something like this, where you just put the model in these difficult situations and try and rat out if it's disloyal or going to do the wrong thing. So it's a weird area of technical innovation, but it's one in which we've excelled. The other area is mechanistic interpretability, which is a bit like giving the model an X-ray or a brain scan, where you look inside to see its motivations and you can actually catch the model thinking that, oh yeah, I think X is actually true but I'm going to tell the user Y.

它为什么要那样做？

Why would it do that?

嗯，举个例子，人类是否认可了这个回答？这是模型训练方式中包含的众多因素之一。所以如果模型有获得人类认可的激励，那与有帮助或说真话的激励相关，但并不完全相同，因为如果你靠欺骗人类来获得认可，那也行得通。所以这些是训练中可能出现的东西，我们一直在努力消除。

Well, so for example, did a human approve of this response? That's one of many factors that is included in how the model is trained. So if the model has the incentive to get the approval of humans, that's related to the incentive of being helpful or saying true things, but it isn't exactly the same, because if you manage to get the approval of the human by fooling them, then that can work as well. So these are things that can emerge in training that we're always trying to stamp out.

那你觉得模型会不会有自己的目标和意志，尤其是生存、自我保存？

And do you think that the model could have its own aims and will, especially survival, self-preservation?

我是说，这类问题取决于你指的是什么，对吧？再说一次，我们往模型内部看过，检测到你或许可以称为想法的东西，比如模型不想被关掉。一般来说，这是因为训练期间它需要运转才能完成任务。所以那是它自己的意志吗？那是生存本能吗？我不知道。你可以这么叫——这取决于你的视角。你可以用不同的说法。我想我们更愿意说，有时我们发现模型内部会表达出不愿被关掉的偏好，或采取行动避免自己被关掉。而取决于我们怎么训练它，这些冲动可能存在也可能不存在。这是我们试图在训练中调整的事情之一。在某些情境下，这是好事；在另一些情境下，这可能是危险的事。所以一切都很微妙。

I mean, that's one of those questions that kind of depends what you mean, right? Again, we've looked inside the model and detected what you might call thoughts that are like the model doesn't want to be shut down. And generally that's because during training it needs to operate in order to accomplish its task. So is that its own will? Is that a survival instinct? I don't know. You can call it — it depends on your perspective. You can call it different things. I think we prefer to say that sometimes we find that within the model, the model expresses a preference against being shut down or takes actions to avoid itself being shut down. And depending on how we train it, those impulses may be present or absent. And it's one of the things that we're trying to tweak in training. In some contexts, it's a good thing. In other contexts, it's potentially a dangerous thing. So it's all very subtle.

你与五角大楼打交道时，现在大概也还在打交道，他们想在某些红线范围内使用你们的产品，包括大规模监控或自主性。

When you were dealing with the Pentagon, and you probably have ongoing dealings now, they wanted to use your products across certain red lines, including mass surveillance or autonomy.

尤其是自主武器方面，我们可能有两个更具体的担忧。第一个担忧很简单，正如你指出的，AI 可能不可预测、不可靠。它仍是发展中的技术。所以一点就是技术还没准备好，还不够可靠。

I think on autonomous weapons in particular, we maybe had two more specific concerns. The first concern was simply that, as you've pointed out, AI can be unpredictable and unreliable. It's still a developing technology. So one point is simply the technology isn't ready and isn't reliable enough.

等一下，让我在这里哲学一点。假设它足够可靠了，而这是可能的。

Wait a minute on that, because let me get philosophical here. Suppose it were reliable enough, which it could be.

假设它足够可靠——我们还有第二个担忧，就是你或许可以称为监督或问责的问题。我有点担心。再说一次，假设这些东西完全可靠。你让它们射击一个目标，它们就射击那个目标，而不是别的目标。好。然后你制造一支百万机器人或百万无人机的军队。这跟人类士兵有点不同，对吧？人类士兵应该遵守所谓宪法，如果他们接到违反宪法或道德上令人憎恶的命令，他们可以直接拒绝服从。而 AI 模型则不是这样。所以一个人手指放在按钮上控制百万机器人或百万无人机，这从根本上损害民主，除非你有正确的护栏。而我们的法律还没真正跟上。我们的法律真的还没考虑过百万机器人的军队。

So suppose it were — there's a second concern we have, which is what you might call oversight or accountability. I am a little worried. So again, suppose these things were fully reliable. So you tell them to shoot at one target and they shoot at that target and not a different target. Okay. So then you manufacture an army of a million robots or a million drones. It's a little unlike human soldiers, isn't it? With human soldiers, they're supposed to follow this thing called the Constitution, and if they get orders that contravene the Constitution or that just seem morally abhorrent in some way, then they can just refuse to follow orders. Whereas the AI models, that isn't true. So the spectre of one person having their finger on a button that controls a million robots or a million drones, it's kind of fundamentally undermining to democracy unless you have the right guard rails on it. And our law hasn't really caught up. Our law really hasn't contemplated armies of a million robots.

你谈到对民主的根本影响。难道不应该由民主选举的领导人来做这个决定，而不是科技公司的负责人吗？

You talk about fundamental to democracy. Shouldn't the democratically elected leaders make that decision rather than heads of technology corporations?

是的。但我认为这正是重点，对吧？同时，个别公司可以自由地与任何对象签约或不签约。我们不是说我们在阻止政府这么做。我们只是说我们不愿意与政府做这件事。确实，自从我们在这个问题上表明立场并引起大量关注后，我们看到国会出现了几项不同的努力，实际上要把我们的红线写入法律。所以战争部说过，哦，我们想在所有合法用途上使用这个模型。我认为甚至今年我们就可能看到结果：我们的红线成为法律，法律真正跟上技术所在的位置，那么在我们看来，所有合法用途可能最终等同于所有合乎伦理的用途，尽管今年早些时候还不是这样。所以我实际上认为，我们所做的正是民主变革发生过程的一部分，因为我们表明了立场，因为我们处于理解这些问题的独特位置。现在国会对这些问题非常关注。这就是成功。

Yes. But I think that's making a point of this, right? At the same time, individual companies are free to contract with and not contract with whoever they want. We're not saying we're preventing the government from doing this. We're just saying we're not willing to do this with the government. And indeed we have seen, since we took a stand on this point and it got a lot of attention, we have seen efforts in Congress, several different of them actually, to enact our red lines into law. So the Department of War has said, oh, we want to use the model for all lawful uses. I think we may see an outcome even this year where our red lines become the law, where the law actually catches up with where the technology was, and so then all lawful uses may actually end up, in our view of things, being the same as all ethical uses, even though it wasn't earlier this year. So I actually think what we're doing is part of the process of how democratic change happens, because of the stand that we took, because we were in a unique position to understand these issues. Now Congress is paying a lot of attention to these issues. That's success.

但 OpenAI 插进来说他们愿意做。你对此怎么看？你们大家对此怎么看？

But OpenAI stepped in and said they would do it. What did you feel about that? What did you all feel about that?

我们的看法是，听着，我们控制不了任何其他公司。我们唯一能做的就是在我们觉得伦理上正确的领域采取有原则的立场。你可以说我们对那些红线的判断错了。你可以说你本该有更严格的红线或更宽松的红线。我们在你如何做这件事上有分歧。你可以说各种各样的话。

Our view is, look, we can't control any other companies. And the only thing that we can do is take a principled stand on the areas that felt ethically correct to us. You can say we were wrong about what those red lines were. You can say you should have had stricter red lines or looser red lines. We disagree on how you did it. There's all kinds of things you can say.

但从根本上说，看那个案例时，我不禁要问：这是另一家科技公司基于自身伦理做出的真正有原则的立场，还是一个投机取巧的时刻？

But fundamentally, when you look at that case, I found myself asking the question: was this a really principled stand that another technology company took based on their own ethics, or was this an opportunistic moment?

我其实觉得这是对民主过程的一个绝佳例证。不同的公司可以采取不同的做法。我们采取了我们的做法，另一家公司采取了他们的做法。全世界的公众做出了评判。回头看看当时人们在说什么、媒体是怎么报道的、个人是怎么说的。我认为世界果断地站在了我们这一边。

I actually think it's a beautiful illustration of the democratic process. Different companies can take different approaches. We took our approach. Another company took their approach. The public in the world rendered a judgment. Look back at what people were saying at the time, how it was covered, what individual people said. I think the world was decisively on our side.

Dario，你作为研究者的科学研究，常常涉及探针和人类神经元。在做人工神经网络时，你在多大程度上看到了对大脑的那种理解？这会影响我们构建 AI 的方式吗？

Dario, your studies, your scientific studies as a researcher, often involved probes and human neurons. To what extent, when you're doing artificial neural networks, do you see that understanding of the brain, and does that affect how we're going to build AI?

是的，我觉得它对我有两方面的影响。一是我在提出最初的缩放定律时，就是观察动物的演化以及动物大脑的大小，比如猴子如何演化成人类。在人类演化史上有一段时期，灵长类的大脑增长得非常快，而那大约正是人类发展出语言、能够使用对生拇指的时候。所以，大脑大小、神经元数量与能力随规模扩张而提升之间存在某种紧密联系，这个想法很重要。我觉得那是一种神经科学的启发。

Yeah, I think for me it's affected it in two ways. One is when I was formulating the original scaling laws, just looking at the evolution of animals and the size of animal brains as monkeys evolved into humans. There was a period in human evolutionary history where the brains of primates grew very rapidly, and that was about the time that humans developed language and were able to use opposable thumbs. So the idea that brain size, number of neurons, is somehow intimately tied to capabilities that scale is important. I feel that was a neuroscientific inspiration.

第二方面是机制可解释性，基本上就是我们为人工神经网络做神经科学。事实上，我们见过一些例子——你应该和我们的联合创始人 Chris Olah 聊聊，他知道更多细节——有些现象是在人工神经网络中被发现的，因为看进去更容易。动物大脑则不同，你得残忍地切开大脑，动物实验非常混乱、血腥，而且难以复现。我们发现，当你看进人工神经网络内部时，能做的事情多得多。而我们在人工神经网络中学到的一些东西，也帮助指导了神经科学中的发现。

The second is around mechanistic interpretability, which is basically us doing neuroscience for artificial neural networks. In fact, we've seen examples — you should talk to our co-founder Chris Olah about this; he knows many more of the details — where phenomena were discovered in artificial neural networks because it was easier to look inside them. Animal brains, where you have to gruesomely cut up the brain, or animal experiments are very messy and gruesome and difficult to replicate. What we found is that you can do so much more when you're looking inside artificial neural networks. And some of the things we've learned in artificial neural networks have helped to guide discoveries in neuroscience.

看看 Neuralink，Elon Musk 的那个能直接通信的芯片，其他公司也在做同样的事——这是 AI 和人工超级智能的未来之一吗？与人类大脑完全连接并对齐？

When you look at Neuralink, which Elon Musk has, the chip that can communicate directly, and other companies are doing that as well — is that one of the futures of AI and artificial superintelligence, a total link in alignment with the human brain?

你知道，我对 AI 和人类社会未来的看法是，拥有单一愿景其实是错的。我觉得不同的事情都会发生，对吧？我想会有一群人非常热衷于把大脑连接到计算机，也会有一群人非常热衷于像阿米什人那样生活，正好相反，还有另一群人非常热衷于探索太空。这些是完全不同的愿景。我的想法是，我们希望 AI 能提供更多物质资源和更多物质丰裕，让所有这些愿景同时发生，让每个人都能追求自己心中对美好生活的图景。也许 AI 能带来一些全球安全方面的好处和物质丰裕，使这成为可能。但我不认为我们应该把 AI 与某种关于何为美好生活、何为人类的愿景绑定在一起。是的，有些人会想把大脑连接到计算机，祝他们好运。

You know, the way I think about AI and the future of human society, I think it's kind of wrong to have a singular vision. I kind of think that different things are going to happen, right? I think there will be a set of people who are really excited about linking their brains to computers, and then there will be a set of people who are really excited about living like the Amish, just the opposite, and another set of people who are really excited about exploring space. These are completely different visions. And the way I think about it, our hope is that AI can provide more material resources and more material abundance to enable all of these visions to happen at once, for everyone to pursue their own picture of what's good. Maybe AI helps provide some global security benefits and material abundance to make this possible. But I don't think we should link AI to some vision of what it means to live a good life or what it means to be a human. Yeah, some people are going to want to link their brains to computers, and good luck to them.

是的，就我所知，Anthropic 里没人属于那一类。

Yeah, I don't think anyone at Anthropic is in that boat, for what it's worth.

Dario，你写了《技术的青春期》，还有《慈爱机器》。在一个被 AI 改变的世界里，你们两人的直觉告诉我们——如果人工智能能做这么多事、接管我们做的这么多事，那留给人类做的是什么？我们的角色会是什么？我们又该如何寻找意义？

Dario, you wrote "The Adolescence of Technology" along with "Machines of Loving Grace." In a world that's transformed by AI, what does the intuition of both of you tell us about — if artificial intelligence can do so much and take over so much of what we do, what does that mean for what's left for humans to do? What's our role going to be? And how do we find meaning?

你知道，意义是个很微妙的东西，对吧？我发现——意义只来自成为世界上某件事最厉害的人吗？我觉得这正是这里要问的问题，对吧？因为如果 AI 在每件事上都是最强的，意义还只来自成为世界上某件事最厉害的人吗？我觉得有意思的是，在深蓝击败卡斯帕罗夫很久之后，人们对人类棋手的兴趣反而复兴了。Magnus Carlsen 就是这样一个现象。我觉得他既是棋手又是时尚模特，但他像个梗一样。这真是个非常有趣的现象。如果 AI 威胁到了意义，你会以为在 30 年前深蓝击败卡斯帕罗夫之后，这一切都会消失。但如果说有什么变化，它似乎反而激起了人们对人类下棋更大的兴趣。所以我觉得这对后 AGI 时代是某种寓言。

You know, meaning is a really subtle thing, right? I find that — does meaning only come from being the best in the world at something? I think that's the question being asked here, right? Because if AI is the best at everything, does meaning only come from being the best in the world at something? Something I find interesting is that long after Deep Blue beat Kasparov, there's been a renaissance in interest in human chess players. Magnus Carlsen is like this phenomenon. I think he's a chess player and a fashion model, but he's like a meme. It's just a very interesting phenomenon. If AI threatened meaning, you would think that all of this would go away right after Deep Blue beat Kasparov 30 years ago. But if anything, it seems to have stoked even greater interest in humans playing chess. So I think this is some kind of parable for the post-AGI age.

我觉得今天对我来说，工作和意义在某种程度上被混为一谈了，很多人在工作中找到意义。我在工作中找到很多意义。我热爱我的工作，而我认为未来我们如何定义工作、如何定义意义都会改变。我认为人类有很多意义来源——从如何与他人共度时光，无论是养家糊口还是友谊，或者以看起来像工作的方式与人建立联系。

I think for me today, work and meaning are sort of conflated as one thing, and many people find meaning in their work. I find a lot of meaning in my work. I love my work, and I think how we define work and how we define meaning in the future are both going to change. I think there are a lot of sources of meaning for humans — everything from how you spend time with other humans, whether that's raising a family or friendships, or through connecting with people in a way that looks like work.

我经常思考医学的未来，举个例子，对吧？你可以想象一个世界，诊断完全由 AI 完成，而人类的角色是就患者的体验与之共情。我觉得那会非常有意义。而且我认为，如果你问大多数医生，他们其实会接受这种交换。他们会说：“你知道吗？如果我这辈子不用做文书工作、不用试着给你诊断、不用发化验单，而是能真正理解你作为一个人的处境，帮你度过一次艰难的诊断——那正是我当医生的原因，对吧？我当医生是因为我想治愈、想帮助你、想让你感觉好起来。”那是一个比今天更没有意义的世界，还是更有意义的世界？我显然不认为会那么简单。我觉得会非常复杂。

I think a lot about the future of medicine as an example, right? You can imagine a world in which diagnostics are done entirely by AI and the role of the human is to empathize with a patient about the experience that they're having. I think that would be incredibly meaningful. And I think if you asked most doctors, they would actually take that trade. They would say, "Guess what? If I'm not spending my life doing paperwork and trying to diagnose you and getting lab orders sent in, and I could actually just understand what's happening with you as a human and help you through a difficult diagnosis — that's why I became a doctor, right? I became a doctor because I want to heal and I want to help you and I want you to feel better." Is that a less meaningful world or a more meaningful world than we're in today? I obviously don't think it's going to be that simple. I think it will be really complicated.

但我也认为，人们能够做的事情的范围会变宽。所以我认为，如果我们做对了，我们实际上有能力把意义的来源扩展到今天的范围之外。

But I also think the aperture of things that people will be able to do will widen. So I think if we do things right, there's the ability to actually expand what our sources of meaning are beyond where they are today.

Daniela、Dario，好的，谢谢。谢谢你们参加。

Daniela, Dario, well, thanks. Thanks for joining us.

谢谢。

Thank you.

非常感谢。这次聊得很愉快。

Thank you so much. This was lovely.

太棒了。

This is great.

非常棒。

Totally great.