Anthropic CEO Dario Amodei 谈 AI 的紧迫性与被误解的警告

当别人称我为‘末日论者’时，我非常生气。有人说‘这家伙是个末日论者，他想放慢脚步。’你听到了我刚才说的。我父亲因为本可以晚几年出现的疗法而去世。我理解这项技术的好处。

I get very angry when people call me a doomer. When someone says, 'This guy's a doomer. He wants to slow things down.' You heard what I just said. My father died because of cures that could have happened a few years later. I understand the benefit of this technology.

我相信你听过像 Jensen 这样的人的批评，他们说：‘Dario 认为只有他能安全地构建这个，因此想控制整个行业。’

I'm sure you've heard the criticism from people like Jensen who say, 'Well, Dario thinks he's the only one who can build this safely and therefore wants to control the entire industry.'

我从未说过那样的话。那是一个离谱的谎言。那是我听过的最离谱的谎言。

I've never said anything like that. That's an outrageous lie. That's the most outrageous lie I've ever heard.

Anthropic CEO Dario Amodei 加入我们，讨论人工智能的前进道路，生成式 AI 是否是一门好生意，并回击那些称他为末日论者的人。他在旧金山 Anthropic 总部的演播室与我们在一起。Dario，很高兴再次见到你。欢迎来到节目。

Anthropic CEO Dario Amodei joins us to talk about the path forward for artificial intelligence, whether generative AI is a good business, and to fire back at those who call him a doomer. He's here with us in studio at Anthropic headquarters in San Francisco. Dario, it's great to see you again. Welcome to the show.

谢谢邀请。

Thank you for having me.

那么，我们来回顾一下过去几个月。你说 AI 可能消灭一半的初级白领工作。当你得知 OpenAI 要收购 Windsurf 时，你切断了 Windsurf 对 Anthropic 顶级模型的访问。你要求政府实施出口管制，还惹恼了 Nvidia CEO Jensen Huang。你怎么了？

So, let's recap the past couple months for you. You said AI could wipe out half of entry-level white collar jobs. You cut off Windsurf's access to Anthropic's top tier models when you learned that OpenAI was going to acquire them. You asked the government for export controls and annoyed Nvidia CEO Jensen Huang. What's gotten into you?

我认为 Anthropic，我和 Anthropic，始终专注于尝试做和说我们相信的事情。随着我们越来越接近更强大的 AI 系统，我想更加强有力、更公开地说出这些观点，以使观点更清晰。多年来我一直在说，我们有这些缩放定律。AI 系统正变得越来越强大。从几年前几乎不连贯，到几年前达到聪明高中生的水平。现在我们正在达到聪明大学生、博士的水平，并且它们开始应用于整个经济。因此，所有与 AI 相关的问题，从国家安全到经济问题，都开始变得非常接近我们实际要面对它们的时候。随着这些问题越来越近，尽管 Anthropic 已经说了一段时间，但紧迫性增加了。我想确保我们说出我们的信念，并警告世界可能的下行风险，尽管没有人能预测会发生什么。我们在说我们认为可能发生的事情，我们认为很可能发生的事情。我们尽可能提供支持，尽管这通常是对未来的推断，没有人能确定。我认为我们认为自己有责任警告世界将要发生的事情。这并不是说 AI 没有令人难以置信的积极应用。我一直在谈论这一点。我写了那篇文章《爱的机器》。我觉得 Anthropic 和我往往比那些自称乐观主义者或加速主义者的人更能阐述 AI 的好处。所以我认为我们可能比任何人都更欣赏这些好处。但正是出于同样的原因，因为如果我们把一切都做对，我们可以拥有一个如此美好的世界，我觉得有义务警告风险。

I think Anthropic, myself and Anthropic, are always focused on trying to do and say the things that we believe. As we've gotten closer to AI systems that are more powerful, I've wanted to say those things more forcefully, more publicly to make the point clearer. I've been saying for many years that we have these scaling laws. AI systems are getting more powerful. They're going from barely coherent a few years ago to the level of a smart high school student a couple years ago. Now we're getting to smart college student, PhD, and they're starting to apply across the economy. So all the issues related to AI, ranging from national security to economic issues, are starting to become quite near to where we're actually going to face them. As these problems have come closer, even though Anthropic has been saying these things for a while, the urgency has gone up. I want to make sure we say what we believe and warn the world about the possible downsides, even though no one can say what's going to happen. We're saying what we think might happen, what we think is likely to happen. We back it up as best we can, although it's often extrapolations about the future where no one can be sure. I think we see ourselves as having a duty to warn the world about what's going to happen. That's not to say there aren't incredible positive applications of AI. I've continued to talk about that. I wrote this essay, 'Machines of Loving Grace.' I feel that Anthropic and I have often been able to do a better job of articulating the benefits of AI than some of the people who call themselves optimists or accelerationists. So I think we probably appreciate the benefits more than anyone. But for exactly the same reason, because we can have such a good world if we get everything right, I feel obligated to warn about the risks.

所以这一切都源于你的时间线。基本上，看起来你的时间线比大多数人更短，因此你感到紧迫感，要站出来，因为你认为这迫在眉睫。

So all of this is coming from your timeline. Basically, it seems like you have a shorter timeline than most and so you were feeling a sense of urgency to get out there because you think that this is imminent.

是的，我不确定。我认为预测非常困难，尤其是在社会方面。所以如果你说人们什么时候会部署 AI，或者公司什么时候会使用 X 美元的 AI 支出，或者 AI 什么时候会用于这些应用，或者什么时候会推动这些医疗疗法，这更难说。我认为底层技术更可预测，但仍然不确定。没有人知道。但在底层技术上，我开始变得更加自信。存在不确定性。我认为我们正在经历的指数增长可能仍然会逐渐消失。我认为有 20%或 25%的可能性，在未来 2 年内的某个时候，模型会开始停止变得更好，原因我们不明白，或者我们明白的原因，比如数据或算力可用性，然后我所说的一切都显得非常愚蠢，每个人都会嘲笑我发出的所有警告。鉴于我所看到的分布，我完全接受这一点。

Yes, I'm not sure. I think it's very hard to predict particularly on the societal side. So if you say when are people going to deploy AI or when are companies going to use X dollars of spend of AI or when will AI be used in these applications or when will it drive these medical cures, that's harder to say. I think the underlying technology is more predictable but still uncertain. No one knows. But on the underlying technology, I've started to become more confident. There is uncertainty about it. I think the exponential that we're on could still peter out. I think there's maybe 20 or 25% chance that sometime in the next 2 years the models just start getting stop getting better for reasons we don't understand or maybe reasons we do understand like data or compute availability, and then everything I'm saying just seems totally silly and everyone makes fun of me for all the warnings I've made. I'm totally fine with that given the distribution that I see.

我应该说，这是我们对话的一部分，是我正在写的关于你的特写的一部分。我已经与二十多位与你共事过、认识你、与你竞争过的人交谈过，我会在节目笔记中链接那篇文章，供大家免费阅读。但与我交谈过的每个人都有一个共同点，那就是你的时间线是所有主要实验室领导者中最短的，你刚才也提到了。那么，为什么你的时间线这么短，我们为什么要相信你的？

I should say that this is part of our conversation as part of a profile I'm writing about you. I've spoken with more than two dozen people who've worked with you, who know you, who've competed with you, and I'm going to link that in the show notes if anybody wants to read it. It's free to read. But one of the themes that has come through across everybody I've spoken with is that you have about the shortest timeline of any of the major lab leaders and you just referenced it just now. So, why do you have such a short timeline and why should we believe in yours?

这确实取决于你对时间线的理解。多年来我一直保持一致的一点是，AI 领域有一些术语，比如 AGI 和超级智能。你会听到公司领导者说我们已经实现了 AGI，我们正在转向超级智能，或者有人停止研究 AGI 开始研究超级智能，这真的很令人兴奋。我认为这些术语完全没有意义。我不知道 AGI 是什么。我不知道超级智能是什么。这听起来像一个营销术语，旨在激活人们的多巴胺。所以你会看到我在公开场合从不使用这些术语，而且我谨慎地批评这些术语的使用。

It really depends what you mean by timeline. One thing I've been consistent on over the years is that there are these terms in the AI world like AGI and superintelligence. You'll hear leaders of companies say we've achieved AGI, we're moving on to superintelligence, or it's really exciting that someone stopped working on AGI and started working on superintelligence. I think these terms are totally meaningless. I don't know what AGI is. I don't know what superintelligence is. It sounds like a marketing term, something designed to activate people's dopamine. So you'll see in public I never use those terms and I'm careful to criticize the use of those terms.

嗯，但尽管如此，我确实是 AI 能力快速提升的最乐观者之一。我一直强调的，是指数增长。每几个月我们就能得到一个比之前更好的 AI 模型，通过投入更多算力、更多数据、更多新型训练方式实现。最初是通过预训练，也就是把互联网上的大量数据喂给模型。现在有了第二阶段，即强化学习或测试时计算或推理，随便你怎么叫。我认为这是一个涉及强化学习的第二阶段。现在这两方面都在同步扩展，从我们的模型和其他公司的模型都能看到，我看不到任何阻碍进一步扩展的因素。在 RL 方面，如何拓宽任务范围是个问题。我们在数学和代码上取得了更多进展，模型已经接近高水平专业人士，而在更主观的任务上进展较少，但我觉得这只是一个暂时的障碍。所以当我看到这个指数增长时，我说，人们不太擅长理解指数。如果某样东西每六个月翻一番，那么在它发生前两年，它看起来只走了 1/16 的路程。我们现在是 2025 年中，模型在经济层面真的开始爆发了。看模型能力，它们开始饱和所有基准测试。看收入，Anthropic 的收入每年增长 10 倍。每年我们都保守地说这次不可能再增长 10 倍了。我从不假设任何事情，总是非常保守地说业务端会放缓，但我们从 2023 年的 0 到 1 亿，2024 年从 1 亿到 10 亿，今年上半年从 10 亿到了，今天说话时，已经远超过 40 亿，可能是 45 亿。所以想想看，假设这个指数增长再持续两年——我不是说一定会——但假设持续两年，那就到了数千亿。我不是说这会发生。我是说，当你处于指数增长中时，你很容易被迷惑。在指数变得完全疯狂的两年前，它看起来才刚刚开始。这就是基本动态。我们在 90 年代的互联网上看到过，网络速度和计算机底层速度在加快，几年内就在之前不可能的基础上建成了全球数字通信网络，除了少数人，几乎没人预见到其影响和速度。这就是我的出发点，我的想法。当然，如果一堆卫星坠毁，互联网可能花更长时间；如果经济崩溃，也可能更慢。所以我们不能确定确切的时间线，但我认为人们被指数迷惑了，没有意识到它可能有多快。我认为它很可能很快，虽然我不确定。

Um, but I think despite that, I am indeed one of the most bullish about AI capabilities improving very fast. The thing I think is real that I've said over and over again is the exponential. The idea that every few months we get an AI model that is better than the AI model we got before. And that we get that by investing more compute in AI models, more data, more new types of training models. Initially, this was done by what's called pre-training, which is when you just feed a bunch of data from the internet into the model. Now we have a second stage that's reinforcement learning or test time compute or reasoning or whatever you want to call it. I think of it as a second stage that involves reinforcement learning. Now both of those things are scaling up together as we've seen with our models and as we've seen with models from other companies, and I don't see anything blocking the further scaling of that. There's some stuff about how do we broaden the tasks on the RL side of it. We've seen more progress on say math and code where the models are getting pretty close to a high professional level, and less on more subjective tasks, but I think that is very much a temporary obstacle. So when I look at it, I see this exponential and I say look, people aren't very good at making sense of exponentials. If something is doubling every six months, then two years before it happens it looks like it's only 1/16th of the way there. And so we are sitting here in the middle of 2025, and the models are really starting to explode in terms of the economy. If you look at the capabilities of the model, they're starting to saturate all the benchmarks. If you look at revenue, Anthropic's revenue every year has grown 10x. Every year we're kind of conservative and say it can't grow 10x this time. I never assume anything and always am very conservative in saying I think it's going to slow down on the business side, but we went from zero to 100 million in 2023, from 100 million to a billion in 2024, and this year in the first half of the year we've gone from 1 billion to, I think as of speaking today, it's well above four, it might be 4.5. So if you think about it, suppose that exponential continued for two years—I'm not saying it will—but suppose it continued for two years, you're well into the hundred billions. I'm not saying that'll happen. I'm saying the situation is that when you're on an exponential, you can really get fooled by it. Two years away from when the exponential goes totally crazy, it looks like it's just starting to be a thing. And so that's the fundamental dynamic. We saw that with the internet in the '90s, where networking speeds and the underlying speed of the computers were getting fast, and over a few years it became possible to build a digital global communications network on top of all this when it wasn't possible just a few years ago, and almost no one except for a few people really saw the implications of that and how fast it would happen. So that's where I'm coming from, that's what I think. Now, I don't know, if a bunch of satellites crashed maybe the internet would have taken longer. If there was an economic crash, maybe it would have taken a little longer. So we can't be sure of the exact timelines, but I think people are getting fooled by the exponential and not realizing how fast it might be. How fast I think it probably will, although I'm not sure.

但 AI 行业很多人都在谈论 Scaling 的收益递减。这跟你刚才描述的愿景不太吻合。他们错了吗？

But so many folks in the AI industry are talking about diminishing returns from scaling. Now, that really doesn't fit with the vision you just laid out. Are they wrong?

是的。从我们看到的来说，我只能谈 Anthropic 的模型。但就 Anthropic 的模型而言，如果看编程——编程是 Anthropic 模型进步很快、采用率也很高的领域。我们不只是一家编程公司，我们计划扩展到很多领域，但就编程来说，我们发布了 3.5 Sonnet，一个叫 3.5 Sonnet V2 的模型，现在可以叫 3.6 Sonnet，然后是 3.7 Sonnet，接着是 4.0 Sonnet 和 4.0 Opus。这一系列四五个模型，每个在编程上都比上一个好很多。如果看基准测试，SweetBench 从 18 个月前的 3%左右，增长到了 72%到 80%，取决于你怎么衡量。实际使用量也呈指数增长，我们越来越朝着自主使用这些模型的方向发展。我认为 Anthropic 编写的大部分代码，现在都是由 Claude 模型编写或至少参与编写的。其他多家公司也说了类似的话。所以我们看到进步非常快，指数增长在继续，我们没有看到任何收益递减。

Yeah. From what we've seen, I can only speak in terms of the models at Anthropic, but what I think seen in terms of the models at Anthropic, if we look at coding—coding is one area where Anthropic models have advanced very quickly, adoption has been very quick. We're not just a coding company, we're planning to expand to many areas, but if you look at coding, we release 3.5 Sonnet, a model we call 3.5 Sonnet V2, which let's call it 3.6 Sonnet now, 3.7 Sonnet, and then 4.0 Sonnet and 4.0 Opus. That series of four or five models, each one got substantially better at coding than the last. If you want to look at benchmarks, you can look at SweetBench growing from, I think 18 months ago was at like 3% or something, growing all the way to 72 to 80% depending on how you measure it. And the real usage has grown exponentially as well, where we're heading more and more towards autonomously you can just use these models. I think the actual majority of code written at Anthropic is, at this point, written by or at least with the involvement of one of the Claude models. And various other companies have said similar statements. So we see the progress as being very fast and the exponential is continuing, and we don't see any diminishing returns.

但大型语言模型似乎有一些缺陷。比如持续学习。几周前我们请了 Dark Kesh，他是这么说的，也在他的 Substack 上写过。缺乏持续学习是一个巨大的问题。语言模型在很多任务上的基线可能比普通人高，但你只能使用开箱即用的能力。你造出模型就完了，它不会学习。这似乎是一个明显的缺陷。你怎么看？

But there are some liabilities it seems like with large language models. For instance, continual learning. We had Dark Kesh on a couple weeks ago. Here's how he put it and he wrote about it in his Substack. The lack of continual learning is a huge huge problem. The LM baseline at many tasks might be higher than an average human, but you're stuck with the abilities you get out of the box. So you just make the model and that's it. It doesn't learn. That seems like a glaring liability. What do you think about that?

首先，我想说，即使我们永远不解决持续学习，即使我们永远不解决持续学习和记忆，我认为 LLMs 在影响经济规模方面仍有巨大潜力。想想我以前从事的领域，生物学和医学。假设我有一个非常聪明的诺贝尔奖得主，我说，好吧，你发现了所有这些，你有非常聪明的头脑，但你不能读新教科书或吸收任何新信息。这当然会有困难，但如果你有 1000 万个这样的人，他们仍然会取得很多生物学突破。他们会有局限，但依然很强大。

So, first of all, I would say even if we never solved continual learning, even if we never solve continual learning and memory, I think that the potential for the LLMs to do incredibly well, to affect things at the scale of the economy will be very high. If I think of the field I used to be in, biology and medicine, let's say I had a very smart Nobel prize winner and I said okay, you've discovered all these things, you have this incredibly smart mind, but you can't read new textbooks or absorb any new information. I mean that would be difficult, but still if you had like 10 million of those, they're still going to make a lot of biology breakthroughs. They're going to be limited.

它们将能够做一些人类做不到的事情，而人类也能做一些它们做不到的事情。但即使我们把这个作为上限，那也已经非常令人印象深刻且具有变革性了。即使我说你永远无法解决那个问题，我认为人们也低估了其影响。但你看，上下文窗口正在变长，模型实际上在上下文窗口内也能学习。所以当我在上下文窗口内与模型交谈时，我们进行对话，它会吸收信息。模型的底层权重可能不会改变，但就像我现在和你说话，我们进行对话，我倾听你说的话，然后回应你。模型也能做到这一点。从机器学习和人工智能的角度来看，我们今天没有理由不能把上下文长度做到一亿个词，这大致相当于人类一生中听到的词汇量。我们没有理由做不到。这实际上是推理支持的问题。所以，即使这样也填补了许多空白，不是全部，但填补了许多。然后还有一些像学习和记忆这样的东西，确实允许我们更新权重。所以有很多关于强化学习训练的方法。我们过去常讨论内循环和外循环。内循环就像我有一个片段，我在那个片段中学到一些东西，并试图优化那个片段的生命周期。而外循环是智能体跨片段学习。所以我认为这种内循环外循环结构是学习持续学习的一种方式。我们在人工智能中学到的一件事是，每当感觉存在某种根本性障碍时，比如两年前我们认为推理存在根本性障碍，结果发现只是强化学习的问题：你用强化学习训练，让模型写下一些东西来尝试解决客观数学问题。不具体说，我认为我们已经有一些证据表明，这是另一个看起来很难但实际上没那么难的问题，它会随着规模扩张加上稍微不同的思维方式而得到解决。

They're going to be able to do some things humans can't and there are some things humans can do that they can't. But even if we impose that as a ceiling, that's pretty damned impressive and transformative. And even if I said you never solve that, I think people are underestimating the impact. But look, context windows are getting longer and models actually do learn during the context window. So as I talk to the model during the context window, I have a conversation, it absorbs information. The underlying weights of the model may not change, but just like I'm talking to you here and we're having a conversation, I listen to the things you say and I respond to them. The models are able to do that. And from a machine learning perspective, from an AI perspective, there's no reason we can't make the context length a hundred million words today, which is roughly what a human hears in their lifetime. There's no reason that we can't do that. It's really inference support. And so again, even that fills in many of the gaps, not all the gaps, but it fills in many of the gaps. And then there are a number of things like learning and memory that do allow us to update the weights. So there are a number of things around types of reinforcement learning training. We used to talk about inner loops and outer loops. The inner loop is like I have some episode and I learn some things in that episode and I'm trying to optimize for the lifetime of that episode. And the outer loop is the agent learning over episodes. So I think that inner loop outer loop structure is a way to learn continual learning. One thing we learned in AI is whenever it feels like there's some fundamental obstacle, like two years ago we thought there was this fundamental obstacle around reasoning. Turned out it was just RL: you train with RL and you let the model write some stuff down to try and figure out objective math problems. Without being too specific, I think we already have some evidence to suggest that this is another of those problems that is not as difficult as it seems, that will fall to scale plus a slightly different way of thinking about things.

你认为你对规模扩张的痴迷会不会让你忽视一些新技术？就像德米斯·哈萨比斯说的，要达到 AGI，或者你称之为超级强大的 AGI，无论我们所说的人类水平智能是什么，我们可能需要一些新技术才能实现。

Do you think your obsession with scale might blind you to some of the new techniques? Like Demis Hassabis says, to get to AGI or you might call it super powerful AGI, whatever human level intelligence is, we might need a couple new techniques for that to happen.

我们每天都在开发新技术。Claude 非常擅长编码，但我们对外不太谈论为什么 Claude 如此擅长编码。

We're developing new techniques every day. Claude is very good at code and we don't really talk externally that much about why Claude is so good at code.

为什么它这么擅长编码？

Why is it so good at code?

就像我说的，我们对外不谈这个。所以我们制作的每一个新版本的 Claude 都在架构、输入的数据以及训练方法上有所改进。所以我们一直在开发新技术。新技术是我们构建的每个模型的一部分。这就是为什么我说我们尽可能优化人才密度。你需要那种人才密度才能发明新技术。

Like I said, we don't talk externally about it. So every new version of Claude that we make has improvements to the architecture, improvements to the data that we put into it, improvements to the methods that we use to train it. So we're developing new techniques all the time. New techniques are a part of every model that we build. And that's why I've said things like we're trying to optimize for talent density as much as possible. You need that talent density in order to invent the new techniques.

有一件事一直悬在这场对话中，那就是也许 Anthropic 是想法正确的公司，但资源不足。因为你看 xAI 和 Meta 内部的情况，埃隆建造了他的大型集群，马克·扎克伯格正在建造一个 5 吉瓦的数据中心，他们投入了大量资源进行规模扩张。这可能吗？我的意思是，Anthropic 显然已经筹集了数十亿美元，但这些都是万亿美元级别的公司。

There's one thing that's been hanging over this conversation, which is that maybe Anthropic is the company with the right idea, but the wrong resources. Because you look at what's happening with xAI and inside Meta where Elon built his massive cluster, Mark Zuckerberg is building this 5 gigawatt data center and they are putting so much resources towards scaling up. Is it possible? I mean Anthropic obviously you have raised billions of dollars but these are trillion dollar companies.

是的。所以我认为到目前为止我们已经筹集了接近 200 亿美元。这还不错。我还要说，如果你看看我们正在与亚马逊等公司建设的数据中心的规模，我不认为我们的数据中心规模扩张比该领域的其他公司小多少。在很多情况下，这些事情受到能源和资本的限制。当人们谈论这些大笔资金时，他们说的是几年内的投入，对吧？当你听到一些公告时，有时它们还没有资金支持。我们已经看到了其他公司正在建设的数据中心的规模，我们实际上相当有信心，我们的数据中心规模将大致在他们建设的规模范围内。

Yeah. So we've raised I think at this point a little short of $20 billion. It's not bad. And I would also say if you look at the size of the data centers that we're building with for example Amazon, I don't think our data center scaling is substantially smaller than that of any of the other companies in the space. In many cases these things are limited by energy, they're limited by capitalization. When people talk about these large amounts of money, they're talking about it over several years, right? And when you hear some of these announcements, sometimes they're not funded yet. We've seen the size of the data centers that folks are building and we're actually pretty confident that we will be within a rough range of the size of data centers they build.

你谈到了人才密度。你怎么看马克·扎克伯格在人才密度方面的做法？我的意思是，结合那些庞大的数据中心，他似乎能够竞争。

You talked about talent density. What do you think about what Mark Zuckerberg is doing on the talent density front? I mean, combining that with these massive data centers, it seems like he's going to be able to compete.

是的。这实际上非常有趣，因为我们注意到，相对于其他公司，Anthropic 很少有人被这些挖角所吸引。这并不是因为他们没有尝试。我和很多在 Anthropic 收到这些挖角邀请的人谈过，他们都拒绝了，甚至不愿与马克·扎克伯格交谈，他们说‘不，我留在 Anthropic’。我们对此的总体回应是，我在全公司的 Slack 上发了一条消息，说我们不愿意为了个别回应这些挖角而损害我们的薪酬原则和公平原则。Anthropic 的运作方式是一系列级别。候选人进来后，会被分配一个级别，我们不会就那个级别进行谈判。因为我们认为这不公平。我们想要一个系统化的方式。如果马克·扎克伯格向飞镖盘扔飞镖，正好扎中你的名字，那并不意味着你应该比旁边同样熟练、同样有才华的人多拿 10 倍的薪水。

Yeah. So this is actually very interesting because one thing we noticed is that relative to other companies, very few people from Anthropic have been caught by these offers. And it's not for lack of trying. I've talked to plenty of people who got these offers at Anthropic and who just turned them down, who wouldn't even talk to Mark Zuckerberg, who said no, I'm staying at Anthropic. And our general response to this was I posted something to the whole company Slack where I said look, we are not willing to compromise our compensation principles, our principles of fairness, to respond individually to these offers. The way things work at Anthropic is there's a series of levels. One candidate comes in, they get assigned a level, and we don't negotiate that level. Because we think it's unfair. We want to have a systematic way. If Mark Zuckerberg throws a dart at a dart board and hits your name, that doesn't mean that you should be paid 10 times more than the guy next to you who's just as skilled, who's just as talented.

我认为，真正能伤害你的唯一方式，就是让恐慌摧毁公司文化，为了保卫公司而不公平地对待员工。实际上，这对公司来说是一个团结的时刻，我们没有屈服。我们拒绝妥协原则，因为我们有信心，人们充满热情是因为他们真正相信使命。这就是我的看法。我认为他们试图购买无法购买的东西——那就是对使命的认同。这里存在选择效应。他们是否得到了最热情、最认同使命、最兴奋的人？

Um, and my view of the situation is that the only way you can really be hurt by this is if you allow it to destroy the culture of your company by panicking, by treating people unfairly in an attempt to defend the company. I think this was a unifying moment for the company where we didn't give in. We refused to compromise our principles because we had the confidence that people are enthusiastic because they truly believe in the mission. That gets to how I see this. I think what they are doing is trying to buy something that cannot be bought, and that is alignment with the mission. There are selection effects here. Are they getting the people who are most enthusiastic, most mission-aligned, most excited?

但他们有人才和 GPU。你没有低估他们。

But they have talent and GPUs. You're not underestimating them.

我们拭目以待。我对他们的做法相当悲观。

We'll see how it plays out. I am pretty bearish on what they're trying to do.

我们来谈谈你的业务。很多人都在想：生成式 AI 的业务是真的吗？我也很好奇。你提到你筹集了近 200 亿美元。你从 Google 筹集了 30 亿，从 Amazon 筹集了 80 亿，从 Lightspeed 领投的新一轮中筹集了 35 亿。你的卖点是什么？因为你不在大型科技公司里，你是独立作战。你只是拿出缩放定律说‘能给点钱吗’？

So let's talk a little bit about your business. A lot of people have been wondering: is the business of generative AI a real thing? I'm also curious. You talked about how much money you've raised, close to 20 billion. You've raised 3 billion from Google, 8 billion from Amazon, 3.5 billion from a new round led by Lightspeed. What is your pitch? Because you are not part of a big tech company. You're out there on your own. Do you just bring the scaling laws and say, 'Can I have some money?'

我一直认为人才是最重要的。三年前，我们只筹集了几亿美元。OpenAI 已经从微软筹集了 130 亿。大型科技公司坐拥 1000 亿、2000 亿。我们当时的卖点是：我们比别人更懂得如何让这些模型变得更好。缩放定律可能存在一条曲线。但如果我们能用 1 亿做到别人用 10 亿才能做到的事，用 100 亿做到别人用 1000 亿才能做到的事，那么投资 Anthropic 的资本效率是其他公司的 10 倍。你是愿意以十分之一的成本做任何事，还是从一大笔钱开始？如果你能以十分之一的成本做事，钱只是一个暂时的缺陷，你可以弥补。如果你有这种内在能力，以同样的价格做得更好，或者以更低的价格做得一样好，投资者理解资本效率的概念。三年前，差距是千倍级别。现在有了 200 亿，你能和 1000 亿竞争吗？我的答案是肯定的，因为人才密度。Anthropic 实际上是历史上在这个规模上增长最快的软件公司。我们从 0 增长到 2023 年的 1 亿，2024 年从 1 亿到 10 亿，今年从 10 亿到 45 亿。每年增长 10 倍。每年我都怀疑我们能否保持这个规模，每年我都不敢公开说，因为我觉得不可能再次发生。这种规模的增长本身就说明了我们与巨头竞争的能力。

My view has always been that talent is the most important thing. Three years ago, we had raised mere hundreds of millions. OpenAI had already raised 13 billion from Microsoft. The large tech companies were sitting on 100 billion, 200 billion. The pitch we made then is: we know how to make these models better than others do. There may be a curve of scaling laws. But if we can do for 100 million what others can do for a billion, and for 10 billion what they can do for 100 billion, then it's 10 times more capital efficient to invest in Anthropic than in these other companies. Would you rather be able to do things 10 times cheaper, or start with a large pile of money? If you can do things 10 times cheaper, the money is a temporary defect that you can remedy. If you have this intrinsic ability to build things much better for the same price, or as good for much lower price, investors understand the concept of capital efficiency. Three years ago, the differences were like a thousand times. Now with 20 billion, can you compete with 100 billion? My answer is yes, because of talent density. Anthropic is actually the fastest growing software company in history at this scale. We grew from zero to 100 million in 2023, 100 million to a billion in 2024, and this year from 1 billion to 4.5 billion. That's 10x a year. Every year I suspect we'll grow at that scale, and every year I'm almost afraid to say it publicly because I think it couldn't possibly happen again. The growth at that scale speaks for itself in terms of our ability to compete with the big players.

CNBC 称 Anthropic 60% 到 75% 的销售来自 API。这是根据内部文件。现在还准确吗？

Okay. So CNBC says 60 to 75% of Anthropic sales are through the API. That was according to internal documents. Is that still accurate?

我不会给出确切数字，但大部分确实来自 API，尽管我们也有蓬勃发展的应用业务。最近，有高级用户使用的 Max 层级，以及程序员使用的 Claude Code。我们的应用业务发展迅速，但大部分收入仍来自 API。

I won't give exact numbers, but the majority does come through the API, although we also have a flourishing apps business. More recently, the Max tier which power users use, as well as Claude Code which coders use. We have a thriving and fast growing apps business, but yes, the majority comes through the API.

所以你是在纯粹押注这项技术。OpenAI 可能押注 ChatGPT，Google 可能押注整合到 Gmail 和日历。为什么你选择纯粹押注技术本身？

So you're making the most pure bet on this technology. OpenAI might be betting on ChatGPT, and Google might be betting on integrating into Gmail and Calendar. Why have you made this pure bet on the tech itself?

我不完全这么认为。我更倾向于说我们押注的是模型的商业用例，而不仅仅是 API 本身。只是最初的商业用例是通过 API 实现的。OpenAI 非常关注消费者端，Google 非常关注现有产品。我们的观点是，AI 的企业用途将大于消费者用途，或者说是商业用途，因为涉及企业、初创公司、开发者和使用模型提高生产力的高级用户。我还认为，专注于商业用例能给我们更好的动力去改进模型。做一个思想实验：假设我有一个模型，在生物化学方面相当于本科生水平。然后我把它改进到博士生水平。如果我去找消费者说‘好消息，我把模型从本科水平提升到了研究生水平’，可能只有 1% 的消费者在乎。99% 的人会说‘反正我也不懂’。但如果我去找辉瑞，说我把它从本科生物化学提升到了研究生水平，那将是天大的事。他们可能会为此多付 10 倍的钱。这对他们的价值可能高出 10 倍。

I wouldn't quite put it that way. I'd describe it more as we've bet on business use cases of the model, more so than on the API per se. It's just that the first business use cases come through the API. OpenAI is very focused on the consumer side. Google is very focused on existing products. Our view is that the enterprise use of AI is going to be greater than the consumer use, or I should say the business use, because it's enterprise, startups, developers, and power users using the model for productivity. I also think that being focused on business use cases gives us better incentives to make the models better. A thought experiment: suppose I have a model that's as good as an undergrad in biochemistry. Then I improve it to be as good as a PhD student. If I go to a consumer and say, 'Great news, I've improved the model from undergrad to graduate level in biochemistry,' maybe 1% of consumers care. 99% will say, 'I don't understand it either way.' But if I go to Pfizer and say I've improved it from undergrad to graduate biochemistry, that's the biggest deal in the world. They might pay 10 times more for that. It might have 10 times more value to them.

所以总体目标是让模型解决世界上的问题，让它们越来越聪明，同时也能带来许多积极的应用——就像我在《Machines of Loving Grace》里写的那样：解决生物医学问题、解决地缘政治问题、解决经济发展问题，以及更平常的金融、法律、生产力或保险等领域。我认为这为尽可能开发模型提供了更好的激励，而且在很多方面，这甚至可能是一个更积极的商业。所以我会说，我们押注于 AI 的商业用途，因为它最符合指数级增长。

And so the general aim of making the models solve the problems of the world to make them smarter and smarter but also able to bring many of the positive applications, the things I wrote about in 'Machines of Loving Grace' like solving the problems of bio medicine, solving the problems of geopolitics, solving the problems of economic development, as well as more prosaic things like finance or legal or productivity or insurance. I think it gives a better incentive to develop the models as far as possible and I think in many ways it may even be a more positive business. So I would say we're making a bet on the business use of AI because it's most aligned with the exponential.

好的。那么简单说一下，你们是如何决定选择编程这个用例的？

Okay. Then briefly, how did you decide to go with the coding use case?

是的。最初，和大多数事情一样，我们试图优化模型在多个方面的能力，而编程尤其突出，因为它非常有价值。我和数千名工程师合作过，大约一年半前，我合作过的最优秀的一位工程师说，之前所有的编程模型对他都没用，而这个模型终于能做一些他做不到的事了。发布之后，它很快就被采用。那段时间，很多编程公司比如 Cursor、Windsurf、GitHub、Augment Code 开始迅速流行，我们看到它这么受欢迎，就加倍投入了。我的看法是，编程特别有趣，因为采用速度快，而且通过模型提高编程能力实际上有助于开发下一代模型。所以它有很多优势。

Yes. So originally, as happens with most things, we're trying to optimize for making the model better at a bunch of stuff, and coding particularly stood out in terms of how valuable it was. I've worked with thousands of engineers, and there was a point about a year and a half ago where one of the best I'd ever worked with said every previous coding model has been useless to me, and this one finally was able to do something I wasn't able to do. And then after we released it, it started getting quick adoption. This was around the time that a lot of the coding companies like Cursor, Windsurf, GitHub, Augment Code started exploding in popularity, and then when we saw how popular it was, we kind of doubled down on it. My view is that coding is particularly interesting because the adoption is fast, and getting better at coding with the models actually helps you to develop the next model. So it has a number of advantages.

现在你们通过 Claude Code 销售 AI 编程。但定价模式让一些人感到困惑。你可以每月花 200 美元，却获得相当于每月 6000 美元的 API 用量——我和一位开发者聊过，他就是这样。Ed Zitron 指出，你的模型越受欢迎，如果人们是重度用户，你就会亏得越多。这怎么说得通呢？

And now you're selling your AI coding through Claude Code. But it's very interesting the pricing model has been confounding to some. You can spend $200 a month and get the equivalent I spoke to one developer they got the equivalent of $6,000 a month from your API. Ed Zitron has pointed out the more popular that your models get, the more money you're going to lose if people are super users of this technology. So, how does that make sense?

实际上，定价方案和速率限制出奇地复杂。所以这基本上是我们发布 Claude Code 和最高套餐（后来我们合并了）时，没有完全理解人们使用模型的方式以及他们实际能获得多少的结果。所以在过去几天里，截至本次采访，我们已经做了调整，特别是在 Opus 这样的大模型上，我认为现在不可能通过 200 美元的订阅获得那么多用量了。未来可能还会有更多变化，但我们始终会有大量用户和适量用户的分布。有些用户通过消费者订阅获得了比 API 产品更划算的待遇，这并不一定意味着我们在亏钱。这里面有很多假设。我可以告诉你，其中一些假设是错误的。我们实际上并没有亏钱。

So, actually pricing schemes and rate limits are surprisingly complicated. So some of this is basically the result of when we released Claude Code and the max tier which we eventually tied together, not fully understanding the implications of the ways in which people could use the models and how much they were actually able to get. So over the last few days, as of the time of this interview, we've adjusted that particularly on the larger models like Opus, I think it's no longer possible to spend that much with a $200 subscription. And it's possible more changes will come in the future, but we're always going to have a distribution of users who use a lot and users who use some amount. And it doesn't necessarily mean we're losing money that there are some users who get more, if you were to measure via API credits spend, get a better deal on the consumer subscription than they would on the API products. There's a lot of assumptions there. And I can tell you that some of them are wrong. We are not in fact losing money.

但我想还有一个问题：你们能否继续服务这些用例而不涨价？我给你几个数据：有些开发者很不满，因为在 Cursor 中使用 Anthropic 的新模型比以往任何时候都贵。我和一些初创公司聊过，他们说 Anthropic 表现不佳，因为他们拿不到 GPU。至少他们是这么猜测的。我刚刚和 Replit 的 Amjad Masad 做了一次采访，下周会播出，他说有一段时间每 token 的价格在下降，但后来停止了。所以是不是这些模型对 Anthropic 来说运行成本太高，以至于碰到了自己的天花板？

But I guess there's another question about whether you can continue to serve these use cases and not raise prices. So just to give you a couple stats, there are some developers that are upset because using Anthropic's newer models in Cursor is costing them more than it ever has. Startups that I've spoken with say Anthropic is down a bunch because they can't get access to the GPUs. At least that's what they imagine is happening. And I was just with Amjad Masad at Replit in an interview that we're going to air next week who said there was a period of time where the price per token to use these models was coming down and it stopped coming down. So is it what's happening that these models are just so expensive for Anthropic to run that it's hitting a wall of its own?

再说一次，我认为你在这里做了假设。

Again, I think you're making assumptions here.

所以我才会问 CEO 啊。

That's why I'm asking the CEO.

嗯，我的思考方式是，我们从模型创造多少价值的角度来看待它们。随着模型越来越好，我考虑的是它们创造了多少价值，而价值如何在模型制造者、芯片制造者和底层应用制造者之间分配，是另一个问题。所以再说一次，不具体展开，我认为你的问题中有一些假设不一定正确。

Yeah, the way I think about it is we think about the models in terms of how much value they are creating. So as the models get better and better, I think about how much value they create, and there's a separate question about how the value is distributed between those who make the model, those who make the chips, and those who make the underlying applications. So again, without being too specific, I think there are some assumptions in your question that are not necessarily correct.

告诉我哪些假设。

Tell me which ones.

我会这么说：我确实预期提供特定智能水平的成本会下降。我预期提供前沿智能的成本——它会带来不断增加的经济价值——可能会上升，也可能下降。我的猜测是它大概会保持在目前水平。但同样，创造的价值会大幅上升。所以两年后，我的猜测是，我们将拥有成本与今天大致相同的模型，但它们将能够更自主、更广泛地完成工作，远超今天的能力。

I'll say this. I do expect the price of providing a given level of intelligence to go down. I expect the price of providing the frontier of intelligence, which will provide increasing economic value, that might go up or it might go down. My guess is it probably stays about where it is. But again the value that's created goes way up. So two years from now, my guess is that we'll have models that cost of the same order of magnitude that they cost today, except they'll be much more capable of doing work much more autonomously and much more broadly than they are capable of today.

Amjad 提到的一点是，他认为更大的模型由于架构和我们讨论过的一些技术——只激活模型的某些部分——运行起来并不那么密集，或者说根据其规模并不更密集。所以他的想法（我希望我传达得准确）是，Anthropic 可以在后端不增加太多负担的情况下运行这些模型，但仍然保持价格不变。而我想划清界限的是，也许为了达到软件利润率——有报道称 Anthropic 略低于软件毛利率——你必须对这些模型收取稍高的费用。

One of the things that Amjad mentioned was he thinks that the bigger models are not as intensive to run or more intensive to run given their size because of the architecture and some of these techniques that we talked about that they're lighting up only certain sections of the model. So his idea, I'm hopefully conveying this truthfully, is that Anthropic can run these models without too much bulk on the back end but is still keeping those prices where they are. And I think the line that I'm going to draw there is maybe that to get to software margins, there were some reports that Anthropic is slightly below software gross margins. You're going to have to charge a little bit more for these models.

所以，是的，再说一次，我认为大模型比小模型运行成本更高。你提到的技术可能是混合专家模型之类的。

So, yeah, again, I think larger models cost more to run than smaller models. I think the technique you're referring to is maybe mixture of experts or something like that.

所以无论你的模型是否采用混合专家模型，混合专家模型是一种以更低的成本运行具有给定参数数量的模型的方法。它是一种训练模型的方式。但如果你不使用这种技术，那么不使用该技术的较大模型比不使用该技术的较小模型运行成本更高。如果你使用这种技术，使用该技术的较大模型比使用该技术的较小模型运行成本更高。所以我认为这有点扭曲了实际情况。基本上，我只是在猜测，想从你这里了解真相。

So whether your models are mixture of experts or not, mixture of experts is like a way to run models more cheaply that have a given number of parameters. It's a way to train models. But if you're not using that technique, then larger models that don't use that technique cost more to run than smaller models that don't use that technique. And if you're using that technique, larger models that use that technique cost more to run than smaller models that are using that technique. So I think that's sort of a distortion of the situation. Basically, I'm just guessing and I'm trying to find out what the truth is from you.

是的，关于模型成本，有一件事会让你惊讶：人们往往会认为，‘哦，要把利润率从 x%提高到 y%真的很难。’但我们一直在进行改进，让模型的效率比以前提高 50%。我们才刚刚开始优化推理。推理能力从几年前到现在已经有了巨大的提升。这就是价格下降的原因。

Yeah, look, so in terms of the cost of the models, one thing you'd be surprised by: people kind of impute this thing like, 'Oh man, it's going to be really hard to get the margins from x% to y%.' We make improvements all the time that make the models 50% more efficient than they were before. We are just at the beginning of optimizing inference. Inference has improved a huge amount from where it was a couple of years ago to where it is now. That's why the prices are coming down.

那么需要多长时间才能盈利？因为我认为今年的亏损将达到 30 亿美元。他们会区分不同的事情。好吧。

And then how long is it going to take to be profitable? Because I think the loss is going to be like three billion this year. That's what they would distinguish different things. Okay.

运行模型是有成本的，对吧？所以模型每赚一美元，都需要一定的成本。实际上，这已经相当有利可图了。还有其他方面：支付员工和建筑物的成本在整体中并不大。最大的成本是训练下一个模型。我认为公司亏损且不盈利的想法有点误导人。当你查看缩放定律时，你会更好地理解这一点。作为一个思想实验，这些数字并不精确，甚至不接近 Anthropic 的情况。假设在 2023 年，你训练了一个成本为 1 亿美元的模型。然后在 2024 年，你部署了 2023 年的模型，它带来了 2 亿美元的收入，但你花了 10 亿美元来训练 2024 年的新模型。然后在 2025 年，这个 10 亿美元的模型带来了 20 亿美元的收入，你又花了 100 亿美元来训练下一个模型。所以公司每年都不盈利：2024 年亏损 8 亿美元，2025 年亏损 80 亿美元。这看起来像是一个巨额亏损的企业，但如果你换个角度，认为每个模型都是盈利的——把每个模型看作一个风险项目。我投资了 1 亿美元在模型上，第二年从中获得了 2 亿美元。所以那个模型有 50%的利润率，让我赚了 1 亿美元。公司投资了 10 亿美元，赚了 20 亿美元。每个模型都盈利，但公司每年都不盈利。这是一个简化的例子；我不是在声称这些数字适用于 Anthropic 或这些事实，但这种总体动态解释了正在发生的事情。在任何时候，如果模型停止改进，或者公司停止投资下一个模型，那么现有模型可能就能形成一个可行的业务。但每个人都在投资下一个模型，所以最终它会达到一定的规模。我们花费更多来投资下一个模型的事实表明，业务的规模明年将比今年更大。当然，可能发生的情况是模型停止改进，这就像一次性的浪费，我们花了一大笔钱，但然后行业会回到这个盈利平台，或者指数增长可以继续。所以我认为这是一个冗长的说法，表明我不认为这是思考问题的正确方式。

There's the cost of running the model, right? So for every dollar the model makes, it costs a certain amount. That is actually already fairly profitable. There are separate things: the cost of paying people and buildings is not that large in the scheme of things. The big cost is the cost of training the next model. And I think this idea of companies losing money and not being profitable is a little bit misleading. You start to understand it better when you look at the scaling laws. As a thought exercise, these numbers are not exact or even close for Anthropic. Let's imagine that in 2023 you train a model that costs $100 million. Then in 2024, you deploy the 2023 model and it makes $200 million in revenue, but you spend a billion dollars to train a new model in 2024. Then in 2025, the billion-dollar model makes $2 billion in revenue and you spend $10 billion to train the next model. So the company every year is unprofitable: it lost $800 million in 2024 and $8 billion in 2025. This looks like a hugely unprofitable enterprise, but if instead you think in terms of each model being profitable—think of each model as a venture. I invested $100 million in the model and then got $200 million out of it the next year. So that model had 50% margins and made me $100 million. The company invested a billion dollars and made $2 billion. Every model is profitable, but the company is unprofitable every year. This is a stylized example; I'm not claiming these numbers for Anthropic or these facts, but this general dynamic is the explanation for what is going on. At any time, if the models stopped getting better or if a company stopped investing in the next model, you would probably have a viable business with the existing models. But everyone is investing in the next model, so eventually it'll get to some scale. The fact that we're spending more to invest in the next model suggests that the scale of the business is going to be larger the next year than it was the year before. Now, of course, what could happen is the models stop getting better and there's this kind of one-time cost that's a boondoggle, and we spend a bunch of money, but then the industry will return to this plateau of profitability, or the exponential can keep going. So I think that's a long-winded way to say I don't think it's really the right way to think about things.

对吧？但开源呢？因为如果你停止投资模型，而开源赶上来，那么人们可以换成开源。现在，我很想听听你的看法，因为当谈到 Anthropic 的业务时，人们跟我讨论的一件事是，最终存在开源变得足够好，以至于你可以把 Anthropic 替换掉，用开源取而代之的风险。

Right? But what about open source? Because if you stopped investing in the models and open source caught up, then people could swap in open source. Now, I'd love to hear your perspective on this because one of the things people have talked to me about when it comes to the Anthropic business is there is that risk eventually that open source gets good enough that you can take Anthropic out and put open source in.

是的。你知道，人们——我认为这个行业的一个特点是，我在 AI 的早期历史中就看到了。AI 经历过的每个社区都有一套关于事物如何运作的启发式方法。比如早在 2014 年我从事 AI 时，就有一个现有的 AI 和机器学习研究社区，他们以某种方式思考问题，并认为‘这只是一时流行，这是新东西，这行不通，这无法规模化。’然后由于指数增长，所有这些都被证明是错误的。随后类似的事情发生在人们将 AI 部署到公司各种应用时。然后在创业生态系统中也有同样的想法。我认为现在我们正处于这样一个阶段：全球的商业领袖，比如投资者和企业家，他们有一套关于商品化的词汇，关于价值将累积到哪个层次，而开源是一种你可以看到一切的想法，它具有削弱商业的意义。而我实际上发现，作为一个完全不是来自那个世界、从未用那种词汇思考的人，在这种情况下，一无所知往往能让你比那些用上一代技术思维方式思考的人做出更好的预测。我认为这是一个冗长的说法，表明我不认为开源在 AI 领域会像在其他领域那样起作用。

Yeah. So, you know, people have—I think one of the things that's been true of this industry is that, and I saw it early in the history of AI. Every community that AI has gone through has this set of heuristics about how things work. Like back when I was in AI in 2014, there was an existing AI and machine learning research community that thought about things in a certain way and were like, 'This is just a fad, this is a new thing, this can't work, this can't scale.' And then because of the exponential, all those things turned out to be false. Then a similar thing happened with people deploying AI within companies to various applications. Then there was the same thought in the startup ecosystem. And I think now we're at the phase where the world's business leaders, like the investors and the business, they have this whole lexicon of commoditization, modes which layer is the value going to accrue to, and open source is this idea that you can kind of see everything that's going on, that it has a significance that undermines business. And I actually find, as someone who didn't come from that world at all, who never thought in terms of that lexicon, this is one of these situations where not knowing anything often leads you to make better predictions than the people who have their way of thinking about things from the last generation of tech. And I think this is all a long-winded way of saying I don't think open source works the same way in AI that it has worked in other areas.

主要是因为开源模型你可以看到源代码，而我们这里看不到模型内部，所以通常称为开放权重而非开源以示区别。但很多好处，比如很多人可以协作、具有累加性，在开放权重上并不完全适用。所以每当新模型出现时，我总觉得开源是个烟雾弹。我不在乎它是否开源。比如 DeepSeek，我认为开源与否并不重要。我问的是：这是个好模型吗？它在我们关心的方面比我们强吗？那才是我唯一在意的。开源与否其实无所谓，因为最终你必须在云端托管它。托管的人做推理。这些模型很大，推理很难。反过来，很多你能通过看权重做到的事情，我们正越来越多地在云上提供，比如微调模型。我们甚至在研究如何通过可解释性接口来探查模型的激活状态。上次我们做了一些关于引导的小实验。所以我认为在竞争时，用开源来思考是错误的维度。我考虑的是哪些模型在我们做的任务上表现好。开源其实是个烟雾弹。

Primarily because with open source, you can see the source code of the model. Here we can't see inside the model; it's often called open weights instead of open source to distinguish that. But a lot of the benefits, like many people can work on it and it's additive, don't quite work the same way. So I've always seen it as a red herring when a new model comes out. I don't care whether it's open source or not. If we talk about DeepSeek, I don't think it mattered that it's open source. I ask: is it a good model? Is it better than us at the things we care about? That's the only thing I care about. It doesn't matter either way, because ultimately you have to host it on the cloud. The people who host it do inference. These are big models; they're hard to do inference on. Conversely, many things you can do when you see the weights, we're increasingly offering on clouds where you can fine-tune the model. We're even looking at ways to investigate the activations of the model as part of an interpretability interface. We did some little things around steering last time. So I think it's the wrong axis to think in terms of competition. I think about which models are good at the tasks we do. Open source is actually a red herring.

但如果它免费且运行成本低……

But if it's free and cheap to run...

它并不免费。你必须在推理上运行它，而且得有人让推理变快。

It's not free. You have to run it on inference, and someone has to make it fast on inference.

好的。我想多了解一点 Dario 这个人。我们还有一点时间。所以我有一些关于你早年生活以及你如何成为现在这样的问题。在旧金山长大是什么感觉？

All right. So I want to learn a little bit more about Dario the person. We have a little bit of time left. So I have some questions for you about early life and then how you became who you are. So, what was it like growing up in San Francisco?

是的。我小时候，这座城市还没有那么中产阶级化。我成长时，科技热潮还没到来。它是在我上高中时发生的。实际上，我对此毫无兴趣。我觉得它很无聊。我感兴趣的是当科学家，研究物理和数学。写网站或创办公司这类事情我一点兴趣都没有。我感兴趣的是发现基本的科学真理，以及做一些让世界变得更好的事。所以我看着科技热潮在我周围发生，但我觉得我本可以从中学习到很多现在有用的东西，但我当时根本没注意，也没有兴趣，尽管我就处在中心。

Yeah. The city when I first grew up here had not really gentrified that much. When I grew up, the tech boom hadn't happened yet. It happened as I was going through high school. And actually, I had no interest in it. It was totally boring to me. I was interested in being a scientist, in physics and math. The idea of writing some website had no interest to me whatsoever, or founding a company. Those weren't things I was interested in at all. I was interested in discovering fundamental scientific truth and in doing something that makes the world better. So I watched the tech boom happen around me, but I feel like there were all kinds of things I probably could have learned from it that would have been helpful now, but I just wasn't paying attention and had no interest in it, even though I was right at the center of it.

所以你是犹太母亲的儿子……

So you were the son of a Jewish mother...

意大利父亲，没错。

Italian father, that is true.

在我来自的长岛，我们管这叫披萨贝果。

From where I'm from in Long Island, we call that a pizza bagel.

披萨贝果。我从没听过这个词。

A pizza bagel. I've never heard that term before.

那你和父母的关系怎么样？

So what was your relationship with your parents like?

是的，我一直和他们很亲近。我觉得他们给了我一种对错感和什么在世界上是重要的。灌输强烈的责任感可能是我记忆最深的。他们总是有那种责任感，想让世界变得更好。我觉得那是我从他们身上学到的主要东西之一。我们一直是一个非常充满爱、非常关怀的家庭。我和妹妹 Daniela 很亲近，她后来当然成了我的联合创始人。我想我们很早就决定要以某种方式一起工作。我不知道我们是否想象过它会达到现在的规模，但那是我们很早就决定要做的事。

Yeah, I mean, I was always pretty close with them. I feel like they gave me a sense of right and wrong and what was important in the world. Imbuing a strong sense of responsibility is maybe the thing I remember most. They were always people who felt that sense of responsibility and wanted to make the world better. And I feel like that's one of the main things I learned from them. It was always a very loving family, a very caring family. I was very close with my sister Daniela, who of course became my co-founder. And I think we decided very early that we wanted to work together in some capacity. I don't know if we imagined it would happen at quite the scale that it has, but that was something we kind of decided early that we wanted to do.

这些年认识你的人告诉我，你父亲的病对你影响很大。你能分享一下吗？

The people that I've spoken with that have known you through the years have told me that your father's illness had a big impact on you. Can you share a little bit about that?

是的。他病了很长时间，最终在 2006 年去世。那实际上是促使我在进入 AI 之前转向生物学的因素之一。我去了普林斯顿，想成为理论物理学家，头几个月做了一些宇宙学工作。在我父亲去世前后，那对我产生了影响，说服我进入生物学，以解决人类疾病和生物学问题。所以我开始和普林斯顿从事生物物理和计算神经科学的人交谈，这导致我转向生物学和计算神经科学。然后最终我进入了 AI。我进入 AI 的原因是那个动机的延续：在生物学多年后，我意识到潜在问题的复杂性似乎超出了人类规模。要完全理解它，需要成百上千的人类研究人员，而他们常常难以协作或整合知识。AI——我当时刚刚开始看到它的发现——感觉是唯一能弥合这一差距的技术，能带我们超越人类规模，完全理解和解决生物学问题。所以这里有一条连贯的线索。

Yes. He was ill for a long time and eventually died in 2006. That was actually one of the things that drove me to go into biology before AI. I'd gone to Princeton wanting to be a theoretical physicist and did some work in cosmology for the first few months. Around the time my father died, that had an influence on me and convinced me to go into biology to address human illnesses and biological problems. So I started talking to folks who worked on biophysics and computational neuroscience at Princeton, and that led to the switch to biology and computational neuroscience. Then eventually I went into AI. The reason I went into AI was a continuation of that motivation: as I spent many years in biology, I realized the complexity of the underlying problems felt beyond human scale. To understand it all, you needed hundreds or thousands of human researchers, and they often had a hard time collaborating or combining their knowledge. AI, which I was just starting to see discoveries in, felt like the only technology that could bridge that gap, bring us beyond human scale to fully understand and solve the problems of biology. So there is a through line there.

我可能搞错了，但我听说他的病在当时基本无法治愈。

And I could have this wrong, but one thing I heard was that his illness was largely incurable when he had it.

而且已经有一些进展了……你能再多说一点吗？

And there have been advances that have been... Can you share a little bit more?

是的。有一些进展让今天的情况变得更容易处理了。

Yes. There are advances that have made it much more manageable today.

是的。确实如此。实际上，在他去世后大概三四年，他所患的那种疾病的治愈率从 50%上升到了大约 95%。

Yes. That is true. Actually, only in the maybe 3 or 4 years after he died, the cure rate for the disease that he had went from 50% to roughly 95%.

是啊。我是说，你父亲被一种本可以治愈的病带走，这一定让你觉得很不公平。

Yeah. I mean, it has to have felt so unjust to have your father taken away by something that could have been cured.

当然会。但这也告诉你要解决相关问题的紧迫性，对吧？有人致力于研究这种疾病的疗法，最终治好了它，拯救了很多人的生命，但如果他们能早几年找到那种疗法，本可以拯救更多人的生命。我认为这就是其中的一个矛盾点，对吧？AI 有所有这些好处。我希望每个人都能尽快获得这些好处。我可能比几乎任何人都更了解这些好处的紧迫性。所以我真的理解其中的利害关系。当我公开谈论 AI 存在这些风险、我担心这些风险时，当人们叫我‘末日论者’时，我非常生气。当有人说‘这家伙是个末日论者，他想放慢速度’时，我真的很生气。你听到了我刚才说的：我父亲因为本可以晚几年才出现的疗法而去世了。我理解这项技术的好处。当我坐下来写《Machines of Loving Grace》时，我列出了这项技术可以改善数十亿人生活的所有方式。推特上那些为加速欢呼的人，我不认为他们对技术的好处有人文主义的理解。他们的大脑里充满了肾上腺素，他们想为某件事欢呼，他们想加速。我不觉得他们在乎。所以当这些人叫我末日论者时，我认为他们这样做完全缺乏道德可信度。这真的让我对他们失去了尊重。

It does, of course. But it also tells you of the urgency of solving the relevant problems, right? There was someone who worked on the cure to this disease, managed to cure it and save a bunch of people's lives, but could have saved even more people's lives if they had managed to find that cure a few years earlier than they did. I think that's one of the tensions here, right? AI has all these benefits. I want everyone to get those benefits as soon as possible. I probably understand better than almost anyone how urgent those benefits are. So I really understand the stakes. When I speak out about AI having these risks and I'm worried about these risks, I get very angry when people call me a doomer. I got really angry when someone says, 'This guy's a doomer. He wants to slow things down.' You heard what I just said: my father died because of cures that could have happened a few years later. I understand the benefit of this technology. When I sat down to write 'Machines of Loving Grace', I wrote out all the ways that billions of people's lives could be better with this technology. Some of these people on Twitter who cheer for acceleration, I don't think they have a humanistic sense of the benefit of the technology. Their brain's just full of adrenaline and they want to cheer for something, they want to accelerate. I don't get the sense they care. So when these people call me a doomer, I think they completely lack any moral credibility in doing that. It really makes me lose respect for them.

我一直在想‘影响力’这个词是怎么回事，因为它经常出现。你身边的人都说你特别执着于产生影响力。事实上，我和一个很了解你的人聊过，他说你不看《权力的游戏》，因为它和影响力无关，是浪费时间，你想专注于影响力。

I've been wondering what this word 'impact' has been, because it's come up so often. Those who have been around you have said you've been singularly obsessed with having impact. In fact, I spoke with someone who knew you well who said you wouldn't watch Game of Thrones because it wasn't tied to impact, that it was a waste of time and you wanted to be focused on impact.

实际上，这不完全对。我不看是因为它太负和了。人们在玩非常负面的游戏。这些人一开始，部分是由于环境，部分是因为他们本身就是可怕的人。他们制造了一种局面，最后每个人都比以前更糟。我真的很热衷于创造正和局面。

Actually, that's not quite right. I wouldn't watch it because it was so negative sum. People were playing such negative games. It was like these people start off, partly the situation and partly because they're just horrible people. They create this situation where at the end everyone is worse off than before. I'm really excited about creating positive sum situations.

我推荐你看。这是一部很棒的剧。但我听到了一些部分。我只是非常不情愿，很久都没看。

I recommend you watch it. It's a great show. But I hear some parts of it. I was just very reluctant and didn't watch it for a long time.

让我们回到影响力的话题。

Let's get back to the impact.

让我们回到影响力。所以这就是影响力。实际上，你的职业生涯就是追求这种影响力，能够防止其他人陷入类似的情况。如果我说的太过分了，请告诉我。

Let's get back to the impact. So that's what impact is. Effectively your career has been this quest to have that impact, to be able to prevent other people from being in similar situations. Tell me if I'm going too far.

我认为这是其中的一部分。我见过很多帮助他人的尝试，有些比其他的更有效。我认为我一直试图在背后有策略，有头脑地去帮助别人。这通常意味着一条漫长的道路，对吧？它可能贯穿一家公司，以及许多技术性的、与你想产生的影响力没有直接联系的活动。但轨迹是，我总是在努力让轨迹朝那个方向弯曲。这就是我对它的理解。这就是我进入这一行的真正原因。和进入 AI 领域的原因类似：我认为生物学的问题如果没有 AI 几乎是无法解决的，或者至少进展太慢。我创办公司的原因是，我在其他公司工作过，我只是觉得那些公司的运营方式并没有真正以追求那种影响力为导向。有一个围绕它的故事经常被用于招聘，但多年来我逐渐明白，那个故事并不真诚。

I think that's a piece of it. I have looked at many attempts to help people, and some of them are more effective than others. I think I've always tried to have strategy behind it, brains behind trying to help people. Which often means there's a long path to it, right? It can run through a company and many activities that are technical and not immediately tied to the kind of impact you're trying to have. But the arc is I'm always trying to bend the arc towards that. That's my picture of it. That's really why I got into this. Similar to the reason to get into AI: I saw the problems of biology as almost intractable without it, or at least too slow moving. My reason to start a company was that I had worked at other companies and I just didn't feel like the way those companies were run was really oriented towards trying to have that impact. There was a story around it that was often used for recruiting, but it became clear to me over the years that story was not sincere.

我要稍微绕回来一点，因为很明显你这里指的是 OpenAI。据我所知，你拥有 OpenAI 50%的算力。我是说，你负责 GPT-3 项目。所以如果有人要专注于影响力和安全，那不就是你吗？

I'm going to circle around a little bit because it's clear that you're referring to OpenAI here. From what I understand, you had 50% of OpenAI's compute. I mean, you ran the GPT-3 project. So if anyone was going to be focused on impact and safety, wouldn't it have been you?

是的，我是。有一段时间确实如此。但并非一直如此。例如，在我们扩展 GPT-3 的时候。当我在 OpenAI 时，我和很多同事，包括最终创立 Anthropic 的人，熊猫们……

Yes, I was. There was a period during which that was true. That wasn't true the entire time. That was, for example, when we were scaling up GPT-3. When I was at OpenAI, I and a lot of my colleagues, including the people who eventually founded Anthropic, the pandas...

熊猫们。

The pandas.

熊猫们。那不是我给他们的名字。

The pandas. That isn't a name I gave them.

他们取的名字。

The name they took.

那不是他们取的名字。

That isn't a name they took.

那是别人叫他们的名字。

That's the name other people called them.

也许是别人叫他们的名字。我从没用过这个名字称呼我的团队。

Maybe it's a name other people called them. That's not a name I ever used for my team.

好的，抱歉。请继续。

Okay, sorry. Go ahead.

我们参与了这些模型的扩展。实际上，构建 GPT-2 和 GPT-3 的最初原因是我们正在做的 AI 对齐工作的延伸。我自己、Paul Christiano 和一些 Anthropic 的联合创始人发明了一种叫做基于人类反馈的强化学习（RLHF）的技术。它旨在帮助引导模型朝着遵循人类意图的方向发展。这实际上是一个前身。我们试图扩展另一种叫做可扩展监督的方法，我认为多年后它才刚刚开始奏效，以帮助模型遵循更可扩展的人类意图。

We were involved in scaling up these models. Actually, the original reason for building GPT-2 and GPT-3 was an outgrowth of the AI alignment work we were doing. Myself, Paul Christiano, and some of the Anthropic co-founders had invented this technique called RL from human feedback (RLHF). That was designed to help steer models in a direction to follow human intent. It was actually a precursor. We were trying to scale up another method called scalable supervision, which I think is just starting to work many years later, to help models follow more scalable human intent.

但我们发现，即使是用更原始的基于人类反馈的强化学习技术，在 GPT-1 这样的小语言模型上也不奏效，那是 OpenAI 其他人构建的。所以扩展 GPT-2 和 GPT-3 就是为了研究这些技术，并在大规模上应用基于人类反馈的强化学习。这指向一点：在这个领域，AI 系统的对齐和能力总是以我们想象不到的方式交织在一起。实际上，这让我意识到，很难将 AI 系统的安全性和能力分开研究。我认为，以更积极的方式影响这个领域的价值和途径来自于组织层面的决策：何时发布、何时内部研究、对系统做什么工作。这也是我和其他 Anthropic 创始人决定另起炉灶的原因之一。但如果你在 OpenAI 内部推动前沿模型，你知道他们仍然会做这些事。

But what we found is even with the more primitive technique RL from human feedback, it wasn't working with the small language models, like GPT-1 that we applied it to, built by other people at OpenAI. So the scaling up of GPT-2 and GPT-3 was done to study these techniques and apply RL from human feedback at scale. This goes to one thing: in this field, the alignment of AI systems and the capability of AI systems are intertwined in a way that always ends up being more tied than we think. Actually, this made me realize it's very hard to work on the safety and capability of AI systems separately. I think the value and the way to inflect the field positively comes from organizational-level decisions: when to release things, when to study things internally, what work to do on systems. That was one of the things that motivated me and some of the other Anthropic founders to go off and do it our own way. But if you were driving the cutting-edge models within OpenAI, you knew they would still be a company doing this stuff.

看起来如果你在推动能力，你就处于主导地位，可以按自己的方式确保安全。

It seems like if you're driving the capabilities, you'd be in the driver's seat to help it be safe the way you wanted.

再说一遍：关于发布模型、公司治理、人事运作、公司对外形象、部署决策、对社会运作方式的声明——这些很多都不是仅仅通过训练模型就能控制的。我认为信任非常重要。公司领导者必须是值得信赖的人，动机真诚。无论你在技术上如何推动公司，如果你为一个动机不真诚、不诚实、并非真心想让世界变得更好的人工作，那行不通——你只是在助长坏事。

Again, I will say: if there's a decision on releasing a model, on the governance of the company, on how the personnel works, how the company represents itself externally, the decisions it makes with respect to deployment, the claims it makes about how it operates with respect to society—many of those things are not controlled just by training the model. I think trust is really important. The leaders of a company have to be trustworthy people, with sincere motivations. No matter how much you drive the company technically, if you're working for someone whose motivations are not sincere, who is not an honest person, who does not truly want to make the world better, it's not going to work—you're just contributing to something bad.

那么，你肯定听过像 Jensen 这样的人的批评，他们说：‘Dario 认为自己是唯一能安全构建这个的人，因此，说到控制这个词，他想控制整个行业。’

So, I'm sure you've heard the criticism from people like Jensen who say, 'Well, Dario thinks he's the only one who can build this safely and therefore, speaking of that word control, wants to control the entire industry.'

我从未说过类似的话。那是个离谱的谎言。那是我听过最离谱的谎言。

I've never said anything like that. That's an outrageous lie. That's the most outrageous lie I've ever heard.

顺便说一句，如果我曲解了 Jensen 的话，我很抱歉，但是……

By the way, I'm sorry if I got Jensen's words wrong, but...

不，不，不。话是没错。但那些话很离谱。事实上，我多次说过，而且我认为 Anthropic 的行动也证明了，我们追求的是所谓的‘竞向顶端’。我多年来在播客上说过，Anthropic 的行动也证明了这一点。在‘竞向底端’中，每个人都竞相尽快推出产品。当你是‘竞向底端’时，谁赢不重要，所有人都输——因为你制造了一个不安全的系统，帮助了对手，引发了经济问题，或者从对齐角度看是不安全的。我对‘竞向顶端’的理解是：谁赢不重要，所有人都赢。所以‘竞向顶端’的方式是，你为这个领域树立榜样。我们是第一个发布负责任扩展政策的。我们没有说其他人都应该这样做，或者你们是坏人。我们发布出来，并鼓励其他人也这样做。几个月后，我们发现其他公司内部也有人试图推出负责任扩展政策，但因为我们先做了，这给了那些人向领导层论证的许可：‘Anthropic 在这样做，所以我们也应该这样做。’在投资可解释性方面也是如此——我们把研究公开给所有人，允许其他公司复制，尽管有时这有商业优势。同样的事情还有宪法 AI，以及危险能力评估的测量。我们试图为这个领域树立榜样，但其中存在相互作用：成为一个强大的商业竞争对手是有帮助的。我从未说过任何接近‘这家公司应该是唯一构建这项技术的公司’这样的话。我不知道有人怎么能从我说的任何话中得出这个结论。这简直是一个难以置信的、恶意的曲解。

No, no, no. The words were correct. But the words are outrageous. In fact, I've said multiple times, and I think Anthropic's actions have shown it, that we're aiming for something we call a race to the top. I've said this on podcasts over the years, and Anthropic's actions have shown it. With a race to the bottom, everyone is competing to get things out as fast as possible. When you have a race to the bottom, it doesn't matter who wins, everyone loses—because you make an unsafe system that helps your adversary, causes economic problems, or is unsafe from an alignment perspective. The way I think about the race to the top is that it doesn't matter who wins, everyone wins. So the way the race to the top works is you set an example for the field. We were the first to put out a responsible scaling policy. We didn't say everyone else should do this or you're bad guys. We put it out and encouraged everyone else to do it. In the months after, we discovered that people within other companies were trying to put out responsible scaling policies, but the fact that we had done it gave those people permission to make the argument to leadership: 'Anthropic is doing this, so we should too.' The same has been true of investing in interpretability—we release our research to everyone and allow other companies to copy it, even though it sometimes has commercial advantages. Same with things like constitutional AI, and the measurement of dangerous capabilities evals. We're trying to set an example for the field, but there's an interplay where it helps to be a powerful commercial competitor. I've said nothing that anywhere near resembles the idea that this company should be the only one to build the technology. I don't know how anyone could derive that from anything I've said. It's just an incredible and bad faith distortion.

好了，在我们进入最后一个问题之前，我们快速问一两个。SBF 是怎么回事？

All right, let's see if we can lightning round one or two before I ask you the last one. What happened with SBF?

我说不上来。我大概见过他四五次。所以我对 SBF 的心理或者他为什么做出那么愚蠢或不道德的事情没有深刻见解。我唯一提前看到的是，有几个人跟我提到他很难共事，有点‘快速行动，打破常规’的那种人。

I couldn't tell you. I probably met the guy four or five times. So I have no great insight into the psychology of SBF or why he did things as stupid or immoral as he did. The only thing I had ever seen ahead of time was that a couple people mentioned to me that he was hard to work with, that he was a bit of a move fast and break things guy.

欢迎来到硅谷。

Welcome to Silicon Valley.

是啊。欢迎来到硅谷。所以我记得我说过，好吧，我给他非投票权股份。我不会让他进入董事会。他听起来像个每天打交道很糟糕的人。但他对 AI 和 AI 安全很兴奋。

Yeah. Welcome to Silicon Valley. So I remember saying, okay, I'm going to give this guy non-voting shares. I'm not going to put him on the board. He sounds like a bad person to deal with every day. But he's excited about AI and AI safety.

好，我们就在这结束吧。你找到了你的影响力，你现在基本上是在做梦想中的工作。想想 AI 在生物学上的各种应用，这只是个开始。你也说这是一项危险的技术，我很好奇，你对影响力的渴望是否在推动你加速这项技术，同时可能低估了控制它可能不可行的可能性。

Okay. So let's end here. So you found your impact I mean you're working the dream pretty much right now. I mean think about all the ways that AI can be used for biology just a start. You also say that this is a dangerous technology and I'm curious if your desire for impact could be pushing you to accelerate this technology while potentially devaluing the possibility that controlling it might not be feasible.

我认为我比业内任何人都更多地警告过这项技术的危险。对吧？我们刚刚花了 10 到 20 分钟谈论那些掌管万亿美元公司的人批评我谈论这些技术的危险，对吧？我有美国政府官员，有掌管 4 万亿美元公司的人批评我谈论技术的危险，对吧？他们给我强加各种奇怪的动机，这些动机与我所说的一切毫无关系，也与我做过的一切毫无支持。然而，我还会继续这样做。我实际上认为，随着收入，随着 AI 的经济业务加速增长，而且是指数级增长，如果我是对的，几年内它将成为世界上最大的收入来源，对吧？它将成为世界上最大的产业。而那些经营公司的人已经这么想了。所以我们实际上处于一个可怕的情况，数千亿到数万亿，我甚至说可能 20 万亿的资本站在尽可能快加速 AI 的一边。我们这家公司绝对值上非常有价值，但与那相比看起来很小，对吧？600 亿美元。我一直在发声，即使这让一些人不满，有些文章说，美国政府的一些人对我们不满，例如，因为我们反对暂停 AI 监管，支持对中国的芯片出口管制，谈论 AI 的经济影响。每次我这样做，我都会受到许多同行的攻击。

I think I have more than anyone else in the industry warned about the dangers of the technology. Right? We just spent 10 20 minutes talking about the frightening large array of people who run trillion dollar companies criticizing me for talking about the dangers of these technologies, right? I have US government officials. I have people who run $4 trillion companies criticizing me for talking about the dangers of the technology, right? imputing all these bizarre motives that bear no relationship to anything I've ever said, not supported in anything I've ever done. And yet, I'm going to continue to do it. I actually think that as the revenues, as the economic business of AI ramps up, and it's ramping up exponentially, if I'm right, in a couple years, it'll be the biggest source of revenue in the world, right? It'll be the biggest industry in the world. And people who run companies already think it. So we actually have this terrifying situation where hundreds of billions to trillions to I would say maybe 20 trillion of capitals on the side of accelerate AI as fast as possible. we have this company that's very valuable in absolute terms, but looks very small compared to that, right? 60 $60 60 billion. And I keep speaking up even if it makes folks in, there have been these articles, some folks in the US government are upset at us, for example, for opposing the moratorium on AI regulation, for being in favor of export controls for chips on China, for talking about the economic impacts of AI. Every time I do that, I get attacked by many of my peers.

对。但你仍然假设我们可以控制它。这就是我指出的。

Right. But you're still assuming that we can control it. That's what I'm pointing out.

但我只是在告诉你，我付出了多少努力，多少坚持，尽管有这么多阻碍，尽管有这么多危险，尽管对公司有风险，我仍然愿意发声。这就是为什么我说，如果我认为没有办法控制这项技术，对吧？如果我认为这只是一场赌博，对吧？有些人会说，‘哦，你认为 AI 有 5%或 10%的出错概率，你只是在掷骰子。’我不是这么想的。这是一个多步骤的游戏，对吧？你走一步，构建下一步最强大的模型，你有一个更密集的测试机制。随着我们越来越接近更强大的模型，我越来越大声地发声，采取越来越激烈的行动，因为我担心 AI 的风险越来越近。我们正在努力解决它们。我们取得了一定的进展，但当我担心我们在风险上取得的进展没有与技术的速度同步时，我就会加速，然后更大声地发声。所以，你问我为什么谈论这个？你开始这个采访时说‘你怎么了？你为什么谈论这个？’这是因为指数增长到了这样一个点，我担心我们可能面临一种情况，即我们处理风险的能力跟不上技术的速度。这就是我的回应。如果我相信没有办法控制一项技术——我完全没有看到任何证据支持这个命题——我们每发布一个模型，在控制模型方面都做得更好，对吧？所有这些事情都会出错，但你真的必须对模型进行非常严格的压力测试。这并不意味着不会出现突发的坏行为。我认为，如果我们只用现在的对齐技术就得到更强大的模型，那么我会非常担心。那时我会站出来说每个人都应该停止建造这些东西。甚至中国也应该停止建造。我不认为他们会听我的，这也是我认为出口管制是更好措施的原因之一。但如果我们在模型上领先几年，却只有今天的对齐和引导技术，那么我肯定会主张我们大幅放缓。我警告风险的原因正是为了我们不必放缓。这样我们就可以投资于安全技术，并继续该领域的进步。即使一家公司愿意放缓技术，这也将是一个巨大的经济代价。你知道这并不能阻止所有其他公司，也不能阻止我们的地缘政治对手，对他们来说这是一场生存之战。所以，这里几乎没有回旋余地，对吧？我们被困在技术的好处、加速竞赛以及这是一个多方竞赛的事实之间。所以，我正在做我能做的最好的事情，那就是投资于安全技术以加速安全的进步。我写过关于可解释性重要性的文章，关于安全各个方向的重要性。我们公开所有安全工作，因为我们认为这是公共物品。这是每个人都需要分享的东西。所以，如果你有更好的策略来平衡好处、技术的必然性和它面临的风险，我非常愿意听取，因为我每晚睡觉都在想这个问题，因为我对利害关系有如此深刻的理解——从好处、它能做什么、它能拯救的生命来看。我亲身经历过这些。我也亲身经历过风险。

But I'm just telling you how much effort, how much persistence, how much despite everything that stacked up, despite all the dangers, despite the risk that it has to the company of being willing to speak up, I'm willing to do it. And that's why I'm saying that look if I thought that there was no way to control the technology, right? If I thought even even if I thought this is just a gamble, right? Some people are like, 'Oh, you think there's a five or 10% chance that AI could go wrong, you're just rolling the dice.' That's not the way I think about it. This is a multi-step game, right? You take one step, you build the next step of most powerful models, you have a more intensive testing regime. As we get closer and closer to the more powerful models, I'm speaking up more and more and I'm taking more and more drastic actions because I'm concerned that the risks of AI are getting closer and closer. We're working to address them. we've made a certain amount of progress, but when I worry that the progress that we've made on the risks does not you know is not fully aligned with the um uh you know is not going as fast as we need to go for the speed of the technology then I speed up then I then I speak up louder. And so, you know, you're asking why am I why am I talk, you know, what, you know, you started this interview by saying what's gotten into you? Why are you talking about this? It's because the exponential is getting to the point that that I worry that we may have a situation that our ability to handle the risk is not keeping up with the speed of the technology. And that's how I'm responding to it. If I believe that there was no way to control a technology, which I see absolutely no evidence for that proposition, we've gotten better at controlling models with every model that we release, right? All these things go wrong, but like you really you really have to stress test the models pretty hard. That doesn't mean you can't have emergent bad behavior. And I think, you know, if we got to much more powerful models with only the alignment techniques we have now, then I'd be very concerned. then I'd be out there saying everyone should stop building these things. Even China should stop building these. I don't think they'd listen to me, which is one reason I think export controls is a better measure. But if we got a few years ahead in models and had only the alignment and steering techniques we had today, then you know, I would definitely be advocating for us to slow down a lot. The reason I'm warning about the risk is so that we don't have to slow down. So that we can invest in safety techniques and can continue the progress of the field. It would be a huge economic effort even if one company was willing to slow down the technology. You know that doesn't stop all the other companies that doesn't stop our geopolitical adversaries to whom this is a existential fight for survival. So, you know, there's very little latitude here, right? We're stuck between all the benefits of the technology, the race to accelerate it and the fact that that is a multi-party race. And so, I am doing the best thing I can do, which is to invest in safety technology to speed up the progress of safety. I've written essays on the importance of interpretability on how important various directions in safety are. We release all of our safety work openly because we think that's the thing that's a public good. That's the thing that everyone needs to share. So if you have a better strategy for balancing the benefits, the inevitability of the technology and the risks that it face, I am very open to hear it because I go to sleep every night thinking about it because I have such an incredible understanding of the stakes in terms of the benefits in terms of what it can do, the lives that it can save. I've seen that personally. I also have seen the risks personally.

我们已经看到模型出过问题。比如 Grock 就是一个例子。人们对此不以为然，但当模型开始采取行动、进行制造、负责医疗干预时，他们就不会再笑了，对吧？模型只是聊天时，人们可以嘲笑风险。但我认为这非常严重。所以我认为当前形势要求我们非常严肃地理解风险和收益。这些都是高风险的决策，必须以严肃的态度来对待。

We've already seen things go wrong with the models. You know, we have an example of that with Grock. And you know, people dismiss this, but they're not going to laugh anymore when the models are taking actions, when they're manufacturing, and when they're in charge of medical interventions, right? People can laugh at the risks when the models are just talking. But I think it's very serious. And so I think what this situation demands is a very serious understanding of both the risks and the benefits. These are high-stakes decisions. They need to be made with seriousness.

让我非常担忧的是，一方面有一批纯粹的末日论者。有人叫我末日论者，但我不是。确实有末日论者，他们声称知道无法安全构建这些模型。我看过他们的论点，简直是一派胡言。模型确实有危险，包括对整个人类的危险，这我能理解。但认为可以逻辑上证明无法让它们安全，这在我看来是无稽之谈。所以我认为这是对当前形势在智力和道德上都不严肃的回应。我也认为，那些坐拥 20 万亿美元资本、因为利益一致而联合起来的人，他们眼里全是美元符号，却坐在那里说我们不应该在 10 年内监管这项技术，这也是智力和道德上不严肃的。有人说担心模型安全的人只是想自己控制技术，这种说法是荒谬的，也是道德上不严肃的。

And I think something that makes me very concerned is that on one hand we have a cadre of people who are just doomers. People call me a doomer. I'm not. But there are doomers out there. People who say they know there's no way to build this safely. I've looked at their arguments. They're a bunch of gobbledegook. The idea that these models have dangers associated with them, including dangers to humanity as a whole, that makes sense to me. The idea that we can kind of logically prove that there's no way to make them safe, that seems like nonsense to me. So I think that is an intellectually and morally unserious way to respond to the situation we're in. I also think it is intellectually and morally unserious for people who are sitting on 20 trillion dollars of capital who all work together because their incentives are all in the same way. There are dollar signs in all of their eyes to sit there and say we shouldn't regulate this technology for 10 years. Anyone who says that we should worry about the safety of these models is someone who just wants to control the technology themselves. That's an outrageous claim and it's a morally unserious claim.

我们坐在这里，做了所有可能的研究。我们在认为合适的时候发声。我们在声称 AI 的经济影响时，努力提供依据。我们有一个经济研究委员会，还有一个经济指数来实时追踪模型。我们还提供资助，让人们理解这项技术的经济影响。我认为，那些比我更从技术成功中获利的人，轻率地进行人身攻击，这与末日论者的立场一样在智力和道德上不严肃。我认为我们需要的是更多的深思熟虑、更多的诚实、更多愿意违背自身利益的人，不要轻率的推特争吵或热点评论。我们需要人们真正投入去理解形势，真正去做工作，真正发表研究，真正为我们所处的形势带来一些光明和洞见。我正在努力这样做。我不认为我做得完美，没有人能做到。我尽力而为。如果有其他人也尝试做同样的事，那将非常有帮助。

We've sat here and we've done every possible piece of research. We speak up when we believe it's appropriate to do so. We've tried to back up when we make claims about the economic impact of AI. We have an economic research council. We have an economic index that we use to track the model in real time. And we're giving grants for people to understand the economic impact of the technology. I think for people who are far more financially invested in the success of the technology than I am to just breezily lob ad hominem attacks, I think that is just as intellectually and morally unserious as the doomer's position. I think what we need here is more thoughtfulness, more honesty, more people willing to go against their interest, willing to not have breezy Twitter fights or hot takes. We need people to actually invest in understanding the situation, actually do the work, actually put out the research, and actually add some light and insight to the situation that we're in. I am trying to do that. I don't think I'm doing that perfectly as no human can. I'm trying to do it as well as I can. It would be very helpful if there were others who would try to do the same thing.

嗯，Dario，我在镜头外说过，但在结束时我想确保说出来。我很感激 Anthropic 发表了那么多内容。我们从实验中学习了很多，从红队测试模型到自动售货机 Claude，我们今天没机会聊到。我认为世界听到这里发生的一切会变得更好。就此而言，谢谢你和我坐下来，花了这么多时间。

Well, Dario, I said this off camera, but I want to make sure to say it on as we're wrapping up. I appreciate how much Anthropic publishes. We have learned a ton from the experiments, everything from red teaming the models to vending machine Claude, which we didn't have a chance to speak about today. I think the world is better off just to hear everything going on here. And to that note, thank you for sitting down with me and spending so much time together.

谢谢你邀请我。

Thanks for having me.

感谢大家的收听和观看。我们下次在 Big Technology Podcast 再见。

Thanks everybody for listening and watching. And we'll see you next time on Big Technology Podcast.