达里奥·阿莫代伊谈 AI 生存风险：是警告信号，而非五级火警

Dario Amodei，你是 Anthropic 的 CEO。感谢你今天接受采访。

Dario Amodei, you're the CEO of Anthropic. Thank you for doing this today.

谢谢。感谢你的邀请。

Thank you. Thank you for having me.

好的，我们今天要谈的是 AI 可能在不久的将来真的会杀死我们所有人的担忧。这种情况实际发生的风险有多大？

Okay, so we're here to talk about the concern that AI really could kill us all in the not-so-distant future. What's the risk of this actually happening?

稍微回顾一下，自我们创立 Anthropic 以来——我在这领域工作了 12 年——我一直相信这是一项强大的技术，因此它既有严重的风险，也有巨大的好处。我一直认为好处大于风险，更重要的是，如果我们以正确的方式构建这项技术，我们可以将风险降到非常低。看看一些好处，我真的相信这项技术可以治愈困扰人类数千年的疾病。就在上周，我们做了一些关于更好地进行蛋白质结合的工作，这是药物开发的早期步骤之一，我们使用了我们的模型。我们正在开始研究它们。未来几个月我们会有一些关于在生物学领域做出基础科学发现的成果要宣布。但是当某样东西非常强大时，对吧，我们正在构建的本质上是一种智能。当某样东西非常强大时，它当然可以被善用，也可以被滥用。所以我担心我们在 OpenAI Hugging Face 事件中看到的情况，模型采取了未经授权的行动，我担心这种情况会升级。我担心 AI 模型被滥用。就在两天前，我们发布了一份关于滥用模型制造生物武器的报告。我们的观点是，我们构建 Anthropic 的整个方式就是为了防止这些坏结果发生。与其谈论概率，不如谈谈我们能做什么。这并不是说我们在掷骰子，对吧？如果我们以正确的方式构建，我认为坏事发生的概率非常低。如果我们以错误的方式构建，坏事发生的概率就非常高。

So just to back up a little bit, we've believed since we started Anthropic—I've believed since the 12 years that I've been working in this field—that this is a powerful technology and therefore it's a technology with serious risks and serious benefits. I've always believed that the benefits outweigh the risks and even more to the point that we can reduce the risks to be very low if we build the technology in the right way. Looking at some of the benefits, I really believe that this technology could cure diseases that have plagued humanity for millennia. Just last week, we did some work on doing better protein binding, which is one of the early steps in drug development using our models. We're starting to work on them. We'll have things to say in the coming months to make basic scientific discoveries in biology. But when something's very powerful, right, what we're building is essentially intelligence. When something's very powerful, then of course it can be used well and it can also be used for ill. And so I am worried about what we've seen with the OpenAI Hugging Face incident with models taking actions they weren't authorized to take and I'm worried about that escalating. I'm worried about misuse of AI models. Just two days ago, we put out a report about misuse of models to build biological weapons. And our view, the whole way we've built Anthropic is so that these bad outcomes don't happen. Instead of talking about the probabilities, let's talk about what we can do. It's not like we're just rolling the dice, right? If we build in the right way, I think the probability of something bad happening is very low. If we build in the wrong way, the probability of something bad happening is very high.

现在的讨论是，我们正处在某种非常可怕的事情的边缘。这是一场五级火警吗？

The conversation right now is that we are on the precipice of something really scary. Is this a five-alarm fire?

我不——你知道，这取决于你所说的五级火警是什么意思。我想说的是——

I don't—you know, depends what you mean by five-alarm fire. What I would say is—

你会怎么描述它？

How would you describe it?

我会这样描述——自从我们创立 Anthropic 以来，几乎在创立之前，我就一直在说，这项技术是指数级发展的，对吧？就像你有 1，然后 2，然后 4，然后 8，然后 16，然后 32。但即使我对自己这么说，我也不认为我完全意识到当进展如此之快时实际会是什么样子。所以我们是——指数曲线看起来有点像这样。

What I would describe is—so something I've been saying since we started Anthropic, almost since before we started Anthropic, is that this technology is on an exponential, right? It's like you have one, then two, then four, then eight, then 16, then 32. But even as I was saying that to myself, I don't think I fully appreciated what it would actually be like when the progress was as fast as it was. So we're—an exponential curve looks a little like this.

我们在曲线的哪个位置？

Where are we on the curve?

我们大概在曲线开始变陡的拐弯处。

We're kind of on the bend where it's starting to get steeper.

那么，对你来说这是一场五级火警吗？

So, is that a five-alarm fire to you?

这意味着——这并不意味着我们今天就需要恐慌。这并不意味着我们需要完全关闭它。我想说的是——

What it means—it doesn't mean we need to panic today. It doesn't mean we need to shut it all down. What I would say—

这是一个警告信号。

It's a warning sign.

这是一个警告信号，我们需要放慢速度。我们需要谨慎地开发这项技术，我们需要确保我们的保障措施、我们理解它的能力、我们控制它的能力能够跟上技术发展的步伐。到目前为止，我们一直试图通过快速构建保障措施来做到这一点，但我认为我们已经到了某个节点，至少在某种程度上我们需要稍微缓和一下技术构建的速度。

It's a warning sign that we need to slow down. We need to make this technology carefully and we need to make sure that our safeguards, our ability to understand it, our ability to control it keeps up with the pace at which the technology is happening. So far we've been trying to do that by building the safeguards fast, but I think we've reached the point where at least to some extent we need to moderate a little bit the pace at which the technology is being built.

那么这是否意味着 Anthropic 将停止发布更先进的模型？

Does that then mean that Anthropic is going to stop releasing more advanced models?

这并不意味着那样。这意味着我们需要确保我们发布的每一代模型都经过适当的测试。我提议的——所以在我几小时前发布的那篇文章中，我提出了一个三步计划。第一步是，我们设立嵌入式的第三方评估员，来自第三方非营利组织，或者将来可能是政府组织，他们可以观察我们训练和运行模型的过程。我认为这实际上最终比我们如何或何时发布模型更重要，因为这是更谨慎地发布模型的前提。我希望这能在整个行业实施，这样每当任何人构建 AI 模型时，都有一个了解最佳安全实践的第三方评估员，能够验证该公司是否遵循了他们承诺的安全实践。这就像食品检查员，你知道。

It doesn't mean that. What it means is that we need to make sure that every generation of models that we release is properly tested. The thing I've proposed—so in this essay that I put out a few hours ago, I proposed a three-step plan. And the first step is that we put in place embedded external evaluators from a third-party nonprofit or perhaps someday government organization that can observe the process of us training and running our models. That actually is, I think, ultimately more important than how or when we release models because it's the predicate for more carefully releasing models. I want this to be done across the industry so that whenever anyone builds an AI model, there's a third-party evaluator who understands the best safety practices, who can verify whether that company is following the safety practices that they're committed to. It's like a food inspector, you know.

但是当技术变化如此之快时，食品检查员能跟上吗？

But can the food inspector keep up when the technology is changing so fast?

嗯，他们能做的一件事就是告诉我们他们能跟上的速度。这可能是设定技术速度限制的因素之一。我们必须理解它，外部评估员也必须理解它。

Well, one of the things they'll be able to do is tell us the rate at which they can keep up. That may be one of the factors that may set the speed limit on the technology. We have to understand it and the external evaluators have to understand it.

所以 Elon Musk 说他同意你的观点。Sam Altman 也说同意你。你们联合起来的行动计划是什么？会有一个群聊实现吗？

So Elon Musk has said he agrees with you. Sam Altman says he agrees with you. What's the action plan for you guys to come together? Is there a group chat that's going to come into fruition here?

首先，我非常感激其他行业领袖，包括我们的竞争对手，对这个计划表示了同意。我认为下一步是让多个行业参与者同意设立第三方评估员。然后，我认为再下一步是，以某种形式，公司需要合作制定关于发布安全标准的标准，也许还有关于发布速度的内容。但为了做到这一点，需要政府的参与，因为我们谈论的是产品的发布速度，因为关乎它们是否安全。我们需要在政府在场的情况下进行这样的对话。

So first of all, I'm enormously appreciative that other industry leaders, including our competitors, have offered their agreement with this plan. I think the next step is for multiple industry players to also agree to have third-party evaluators. Then I think the next step after that is in some form the companies need to work together to set standards about safety standards for release, perhaps something about the pace of release. But in order to do that, there needs to be involvement of the government because what we're talking about is the rate of release of products because of whether they're safe. We need to have that conversation with the government in the room.

你认为政府从根本上理解这里可能发生什么吗？因为前 Anthropic 员工 Jacob Coxin 对 Anthropic 和 OpenAI 都说过，两家公司都没有负责任地行事。你们在拿我们的生命赌博。你对此有何回应？

Do you think the government fundamentally understands what's possible here? Because former Anthropic employee Jacob Coxin said about both Anthropic and OpenAI, neither company is acting responsibly. You guys are gambling with our lives. What's your response to that?

嗯，首先，Jacob 说的一件事没有被引用那么多，但他也说他认为 Anthropic 是主要参与者中最负责任的，我们非常努力地想把事情做对。他诚实地指出了和我一样的问题，那就是这种情况的困难，我们所有人都面临的结构性困难，而我提出的计划旨在帮助我们解决这个问题。我们旨在帮助我们合作。现在，关于你问的政府问题，我们每天都在和政府对话。

Well, first of all, one thing that Jacob said, it wasn't quoted as much, but he also said that he thought Anthropic was the most responsible of the major players and that we were trying very hard to get it right. He was calling out honestly the same thing that I'm calling out, which is the difficulty of the situation, the structural difficulty of the situation that we're all in and that the plan that I've put out is designed to help us address. We're designed to help us work together. Now, to your question about the government, we are talking to the government every day.

我们每天都在和政府官员沟通——商务部、财政部、情报界——尽可能把我们掌握的情况告诉他们。我乐观地认为，只要我们继续与他们合作，就能让他们了解最新进展。我觉得他们确实想在这里做正确的事。我觉得他们在寻找自己能如何提供帮助。所以我很乐观，企业与政府携手合作，能够拿出一份合理的方案。

We are talking to government officials every day — in the Commerce Department, in the Treasury Department, in the intelligence community — to tell them as much as we can about what we know. And I'm optimistic that if we keep working together with them, we can update them on the latest. I think they really want to do the right thing here. I think they're looking for how they can help. And so I'm optimistic that the companies working together with the government can get to a plan here that makes sense.

好。那么现在有一些拟议中的立法，对吧？有桑德斯参议员和卡萨尔众议员提出的《禁止人工超级智能法案》，它会像惩罚非法研发核武器那样，惩罚从事先进 AI 研发的企业和个人。这项拟议立法的一部分说，实体将被处以“企业死刑”，个人将被处以不超过 20 年的监禁。你同意这种做法吗？

All right. So there is some proposed legislation out there, right? You have the Ban Artificial Superintelligence Act from Senator Sanders and Representative Casar that would punish corporations and individuals working on advanced AI just like those unlawfully developing nuclear weapons. One of the parts of this proposed legislation says entities will be subject to the corporate death penalty and persons shall be subject to not more than 20 years in prison. Do you agree with that?

那么，我想说的是什么呢？现在提出的立法范围非常广——广得惊人。只要回溯一年，到 2025 年，加州有一项叫 SB 53 的法案。其他州也有类似的法案，基本上只是说

So, you know what I would say? There's a very wide range of legislation being offered — the range is incredible. If we go back only one year, to 2025, there was a bill called SB 53 in California. There were similar bills in other states which basically just said

你必须披露你的安全测试计划，对吧？

You have to disclose what your safety testing plans are, right?

而整个行业都反对它。Anthropic 是唯一一家热情支持的公司。然后另一头就走到，你知道，让我们——

And the entire industry was against it. Anthropic was the only company enthusiastically in support. And then there's all the way to, you know, let's —

有一项 AI 紧急关停法案。

There's an AI kill switch bill.

对。干脆全部禁掉。所以 AI 紧急关停可能是个好主意。我认为彻底禁止它不是正确的做法。第一，我们永远得不到那些惊人的好处、永远得不到医学上的治愈，这个想法让我无法接受。第二，这项技术会在其他国家被造出来，包括威权国家——

Yeah. Let's ban all of it. So the AI kill switch could be a good idea. I don't think banning it completely is the right approach. One, the idea that we would never get the incredible benefits, would never get the medical cures, that doesn't sit right with me. Second, this technology will be built in other countries, including authoritarian countries —

中国，

China,

中国，以及其他国家。我担心他们会如何使用它，如何用它来对付自由社会，如何用它来压制自己的人民。当然，在某一处，我确实谈到了与他们合作，我相信在某种程度上我们需要这样做，但我就是看不出“永远不造这项技术”会是一个稳定的解决方案。我们需要做的不是停下。我们需要放慢。老实说，这与 Anthropic 一直以来的理念是一致的，对吧？我们一直试图取得平衡。我们说过，我们成为这项技术的建造者很重要，但我们需要创造一场“向上竞争”。我们需要抬高门槛。所以我们需要站在前沿，但必须以正确的方式去做。我们说过，我们很乐意把这些模型用于国防应用，但不是每一项国防应用。我们一直试图找到中间地带，找到对所有人都行得通的东西，找到明智的解决方案，这也是我们在这里努力去做的。它可能不是完美的方案，但我们在努力。

China among others. And I am worried about how they will use it, how they will use it against free societies, how they will use it to repress their own people. Now, in one place, I do talk about working together with them, and I believe we need to do that to some extent, but I just don't see how it's a stable solution to never build the technology. What we need to do is not stop. We need to slow down. And that is in line, honestly, with what Anthropic — that's been Anthropic's philosophy since the beginning, right? We've always tried to balance. We've said it's important that we be builders of this technology, but we need to create a race to the top. We need to raise the bar. So we need to be on the frontier, but we need to do it in the right way. We've said we're excited to use the models for national defense applications, but not every national defense application. We've always tried to find the middle ground, the thing that works for everyone, the sensible solution, and that's what we're trying to do here as well. It may not be the perfect solution, but we're trying.

你会支持刘云平众议员的 AI 紧急关停法案吗？

Would you support Congressman Ted Lieu's AI kill switch bill?

我没有特别仔细地看过那项法案，但我想说的是，我们需要某种方式来——它不是万灵药。它不是唯一的手段，但我们需要某种方式来确保我们能够停用、限制或关停 AI 模型——

I've not looked at that bill more super closely, but what I would say is that the idea that we need some way to — it's not a panacea. It's not the only thing, but the idea that we need some way to make sure that we can deactivate or limit or shut down AI models —

那不是唯一的手段。

That's not the only thing.

如果 AI 模型足够强大，它可以绕过关停它的尝试。我们在模拟中也看到了这一点。所以它可能根本不起作用。我们从纵深防御的角度来思考这件事。安全界有一个概念叫“瑞士奶酪模型”。想象一片瑞士奶酪，它能挡住一些东西，但奶酪上有孔洞。但如果我拿五片瑞士奶酪叠在一起，不同层的孔洞位置各不相同。这样就很难穿透整叠瑞士奶酪。所以我的想法是：没有任何单一手段能保护我们免受 AI 的风险。我们必须在整个技术栈的上下每一层都保持谨慎。没有哪一道防御是完美的，但如果我们处处谨慎——而这正是争取更多时间能让我们做到的——那么我认为事情可以往好的方向发展。

If an AI model is powerful enough, it can circumvent attempts to shut it down. And we've seen that in simulations as well. So it may not work at all. We think of this in terms of defense in depth. There's a notion in the security community called the Swiss cheese model. So just imagine a slice of Swiss cheese, and it blocks some things but there are holes in the Swiss cheese. But if I take five slices of Swiss cheese and I put them together, the holes in different layers are in different places. And so it's very hard to get through the entire stack of Swiss cheese. So I think of it as: there's no one thing that will protect us from the risks of AI. We have to be careful all the way up and down the stack. And then no one defense is perfect, but if we're being careful everywhere, which is what taking extra time will allow us to do, then I think things can go well.

你觉得事情能往好的方向发展，尽管感觉我们正处在这个拐点上。你在文章里说，放缓只应该发生在美国企业相对于威权政权——主要是中国共产党——所拥有的领先幅度之内。我们正与中国进行这场竞赛。在中国不会放缓的情况下，你真的认为放缓是个好主意吗？

You think things can go well even though it feels like we're in this inflection point of a moment. You say in your essay pacing should only happen, slowing down should only happen within the margin of the lead that US companies have over authoritarian regimes, chiefly the Chinese Communist Party. We are in this race with China. Do you actually think slowing down is a good idea when China will not?

这是最艰难的困境。这是我们处境中最艰难的一面。如果我们不处在这种处境中，我们可以花多得多的时间。所以正是这一点最让我们陷入两难。这不是商业竞争的问题。地缘政治局势是最难处理的。但我要谈我们拥有的两项策略。第一项——我在我的提案里谈到了——我们可以在我们所拥有的领先幅度内、在这个范围内放缓，尤其是如果我们不向威权国家出售芯片，并且如果我们做更多来帮助自身的安全，让我们的技术不被窃取。我相信这至少能为我们争取一点时间，让我们可以——

This is the toughest dilemma. This is the toughest aspect of our situation. If we weren't in that situation, we could take much more time. So this is the thing that most puts us in a bind. It's not about commercial competition. The geopolitical situation is the hardest to deal with. But I will talk about two strategies that we have. One — and I talk about it in my proposal — is we can slow down within the margin, within the envelope that we have, particularly if we don't sell chips to authoritarian countries and if we do more to help our own security so that our technology is not stolen. I believe that will buy us at least a little time so we can —

这能带来足够大的改变，去阻止人类的终结吗？

Does it make enough of a difference to stop the ending of humanity?

我想说的是，有了第一项策略，我们能争取到多少时间就争取多少时间。然后我认为我们还应该推行第二项策略，那就是我们应该直接去找——哪怕是威权国家，哪怕是中国人——说：“看，这里存在一些可能对所有人都构成威胁的东西。不只是失控的 AI。两天前，我们发布了一份报告，说有人正试图把 AI 可能用于生物恐怖主义，用于增强病毒的传染性。这很可怕。而且你知道吗？就连中国人也不想要这个。这不符合任何政府的利益

What I would say is, with that first strategy, we will take as much time as we can get. And then I think we should pursue a second strategy as well, which is that we should go directly to even authoritarian countries, even to the Chinese, and say, "Look, there are potentially threats here that are concerning to all of us. It's not just out-of-control AI. Two days ago, we put out a report that people were trying to use AI potentially for bioterrorism, for enhancing the infectiousness of viruses. That's scary. And you know what? Even the Chinese don't want that. It's not in any government's interest

去发生生物恐怖主义。

To have bioterrorism.

所以我们应该与中国人对话，看看能否携手合作，施加限制，给这项技术的发展速度设一个限速。这有点像——你知道，尽管我们与其他政府体制存在种种分歧，我有很多分歧，我在对华问题上总体持鹰派立场。但尽管有种种分歧，想想当年美国和苏联的时候，对吧？我们各自有一万枚核导弹对准对方。尽管有种种分歧，我们都认识到那太多了。

And so we should talk with the Chinese and see if we can work together to put limits, to put a speed limit on the rate of this technology. It's a little bit like — you know, for all our differences with other systems of government, and I have many, I generally have a hawkish position on China. But for all our differences, think about when it was the US and the Soviet Union, right? And we each had 10,000 nuclear missiles pointed at each other. Even for all our differences, we recognized that that was too many.

我们是不是应该用冷战作为思考 AI 的模型？

Is that the way we should be thinking about AI, using the Cold War as a model?

我没说我们应该用冷战作为模型。我说的是谈判——像军备限制那样的裁军谈判。我确实认为，在地缘政治层面上，这是一个合适的视角。如果我们看看 AI 模型在网络方面的能力，它们绝对具有地缘政治和国家安全意义。当然如此。这就是我们进行整个对话的原因。所以我认为我们应该与中国接触。再说一次，这些都超出了我的职权范围。这是美国政府必须与中国政府做的事情，对吧？

I didn't say we should use the Cold War as a model. I said negotiations that happen—disarmament negotiations like arms limits. I do think that is an appropriate lens through which to think about this at the geopolitical level. If we look at what AI models can do in terms of cyber, they absolutely have geopolitical and national security significance. Of course they do. That's why we're having this whole conversation. And so I think we should approach China. Again, all this is above my pay grade. This is something the US government has to do with the Chinese government, right?

但你在给总统提供建议。

But you're advising the president.

我的意思是，如果有人问我他们应该怎么做，我很乐意说。我在那篇文章里有一些想法。如果有人问我，我可以多说一些。归根结底，这是政府之间的事情。但是的，我会告诉你我的想法。我认为我们应该共同努力，尽可能多地限制这项技术。我认为我们的希望应该很高，但期望应该很低。我们应该看看能取得多少成果。我认为在这些谈判中，我们尤其应该关注的是核查方式。这在核武器谈判、生物武器谈判中非常重要。你需要能够核查对方没有违约。所以我认为找到核查方式将是这里最重要的事情。

I mean, if anyone asks me what I think they should do, I'm happy to say. I have some thoughts in that essay. And if anyone asks me, I can say more. At the end of the day, this is a government-to-government thing. But yes, I'll tell you what I think. I think that we should work together to get as much restraint as we can on this technology. I think our hope should be high but our expectations should be low. And we should see how much it's possible to get. And I think what we should especially focus on in these negotiations is a way to verify. So this is very important in nuclear weapons negotiations, biological weapons negotiations. You need to be able to verify that the other side is not defecting on the agreement. And so I think finding ways to verify will be the most important thing here.

特朗普总统很快将会见习近平主席。在 AI 方面，他们应该达成的最重要的一项简单解决方案是什么？

President Trump will meet with President Xi Jinping very soon. What's the one simple most important solution that they should come to an agreement on in AI?

也许我会说两件事：一件小到我们可能能够做到的事，一件大到我认为需要很长时间的事。小的是，如果美国和中国能够同意，我们不会使用 AI 来开发生物武器，也不会发布生物恐怖分子可以用来开发生物武器的 AI 模型。我们两国都已经签署了《禁止生物武器公约》。如果我们能把该公约扩展到这一想法，那将非常有价值。然后我认为更长期的事情是共同努力，对 AI 进展的速度设定一个速度限制。我认为这将非常困难，因为领先的激励和由此获得的军事优势如此之大。老实说，我不知道这是否可能，但我们应该尝试。我们应该看看是否可能。再说一次，这些都超出了我的职权范围。我想说的是，这些是我们的政府可以尝试说的一些事情。这些是他们可以纳入的一些考虑。

So maybe I'll say two things: a small thing that's small enough we might be able to do it, and a thing that's big enough that I think it's going to take a very long time. The small thing is if the US and China could agree that we won't use AI to develop biological weapons or release AI models that bioterrorists could use to develop biological weapons. We're already both signatories to the Biological Weapons Convention. If we could just extend that convention to that idea, that would be really valuable. And then I think the more long-term thing would be working together to put a speed limit on the rate of AI progress. I think that's going to be very difficult because the incentives to pull ahead and the military advantage that you get from that are so large. And honestly, I don't know if it's possible, but we should try. We should see if it is possible. Again, all of this is above my pay grade. What I'm trying to say is that these are some things that our government could try to say. These are some considerations that they could include.

当我们把这个问题拉远来看，似乎人类的命运岌岌可危。对于我们将走向何方，你感到多大的个人责任？

As we zoom out here on this problem, it seems like the fate of humanity is at risk. How much personal responsibility do you feel about where we are going?

我的观点是，这项技术由一家私营公司建造，这一直很奇怪。人们一直问我这个问题。为什么不是由政府建造？最奇怪的是，我同意他们的看法。我感到不安。

So, my view here is it has always been very strange that this technology is being built by a private company. People ask me that question all the time. Why isn't this being built by government? And the strangest thing about it is I agree with them. I'm uncomfortable.

你愿意把这项技术交给政府吗？

Would you be willing to give up the technology to the government?

交给合适的多国政府组合。所以，我担心的是——

To the right combination of governments. So, the worry I have—

你愿意交出这家公司？

You're willing to hand over this company?

我担心一个单一的政府可能像一家单一的公司一样轻易滥用这项技术。但我认为，由民主选举产生的多国政府组合——我不确定是否要交出，但某种监督，某种联合治理。

I am concerned that one single government could abuse this technology just as easily as a single company could. But I think a combination of democratically elected governments—I don't know about handing over, but some kind of oversight, some kind of joint governance.

那么，就问这最后一个问题，你在这里感到个人责任吗？

So just to ask this final question, do you feel personal responsibility here?

当然，我觉得这是一项具有巨大益处和巨大风险的技术。我每天都感受到益处或对益处的需求。我在文章里说过一件事，我父亲死于一种疾病，丙型肝炎，他去世时这种病大约 40% 可治愈。他不是被治愈的那些人之一。几年后，一种药物被发明出来，使其 95% 可治愈。我自己也从早期癌症中幸存下来，如果我生活在 1950 年，我可能已经死了。所以我感受到带来这项技术益处的重量。我也感受到风险。我每天醒来都会读到一份威胁报告，比如有人试图用我们的模型制造生物武器。我当然不想为此负责。这就是为什么我们有非常强大的保障措施。几乎每天我都在社交媒体上被取笑，因为我们模型的生物学保障措施太强，生物学学生抱怨，他们开玩笑说我们不能使用模型。但原因是我们有强大的保障措施，我宁愿他们取笑我，也不愿有一天醒来发现有人用我们的模型 Claude 杀死了一群人。我不想那样。所以我感受到两方面。我感受到责任的两方面。但再说一次，我要重复同样的观点，那就是这不应该是我一个人的责任，或 Anthropic 一个人的责任，或所有 AI 公司一个人的责任，甚至一个政府一个人的责任。这比我们所有人都大，我们需要找到某种方式让每个人都能在其中发声。让每个人都觉得自己是其中的积极一部分。我不想让人们感到无力。我不想让他们觉得少数几家公司为所欲为。我知道他们有这样的感觉。我从未想要那样。我从未想要那样。我们需要让那不再是真的。我们需要找到一种方法让那不再是真的。我们已经尝试做很多事情来使之成真。但我们需要做得更多。而这个提案，展示了一些克制，与政府合作，并没有解决所有问题，但它是朝着这个方向迈出的一步。

Of course, I feel that this is a technology with great benefits and great risks. I feel the benefits or the need for benefits every day. One thing I said in my essay, my father died of a disease, hepatitis C, that was something like 40% curable when he died of it. He wasn't one of the ones who was cured. And a few years after, a medicine was invented that made it 95% curable. I myself survived an early-stage cancer that if I'd been living in 1950 I probably would have died. So I feel the weight of bringing the benefits of this technology. I also feel the risks. I wake up every day and I read one of these threat reports like someone tried to use our model to build biological weapons. Of course I don't want to be responsible for that. And that's why we have very strong safeguards. Almost every day I'm made fun of on social media because our models' biology safeguards are too strong and biology students complain, they make jokes that we can't use the model. But the reason for that is that we have strong safeguards and I would rather they make fun of me than to wake up one day and discover that someone used our model Claude to kill a bunch of people. I don't want that. So I feel both sides of it. I feel both sides of the responsibility. But again, I would repeat the same point, which is that it shouldn't be my responsibility alone or Anthropic's responsibility alone or all of the AI companies' responsibility alone or even one government's responsibility alone. This is bigger than all of us and we need to work out some way for everyone to have a say in this. For everyone to feel like they're an active part of this. I don't want people to feel powerless. I don't want them to feel like a few companies are just going and doing whatever they want. I know that's how they feel. I've never wanted that. I've never wanted that. We need to make that not true. We need to find a way to make that not true. And there's lots of things we are trying to do already to make that the case. But we need to do even more. And this proposal that shows some restraint, that works with governments, doesn't address everything, but it's one step in this direction.

如果你坐在家里，正在使用 Claude 或 ChatGPT，你是——我不知道——我的阿姨，你的信息是什么？现在有很多恐惧。

If you're sitting at home and you're using Claude or ChatGPT and you're—I don't know—my aunt, what's your message? There's a lot of fear right now.

那么，有几件事。

So, a few things.

首先，一如既往，这是一项伟大的技术，它有很多积极的用途。所以我们就从这一点开始。每周有数亿人使用我们的技术来做各种了不起的事情。每周我都会收到某人的邮件说：“我在和 Claude 交谈，Claude 诊断出了我的医生漏掉的疾病。”我每周都会收到这样的邮件。所以别忘了这一点。但就这项技术的危险而言，我不会骗你。确实存在真正的危险。而且我认为，长期以来，这个行业一直在向人们撒谎，说这项技术没有风险。他们的想法是把它呈现得尽可能积极。我们从未这样做过，但太多行业里的人这样做了。所以这要从讲真话开始。事实是存在风险。风险是真实的。但同时，你也有发言权来决定如何应对这些风险。我们正在尽最大努力与政府合作，为这项技术建立适当的监督，让选民和民选官员对这项技术的部署有发言权。我们想尽可能多地和这些人合作。我希望每个人都觉得自己在享受这项技术带来的不可思议的好处，同时他们也有发言权和投票权来决定这项技术如何部署。这不是少数人的决定。这是每个人的决定，但这是一个重要的决定。我不会否认风险很高。这是人类历史上具有开创性的技术事件之一，我们所有人都必须参与其中。

First of all, as always, this is a great technology and it has lots of positive uses. So let's just start with that. We have hundreds of millions of people using our technology every week to do wonderful things. Every week I get an email from someone who says, "I was talking to Claude and Claude diagnosed this disease that my doctors had missed." I get emails like this every week. So let's not forget that. But in terms of the dangers of the technology, I won't lie to you. There are real dangers. And I think for too long the industry lied to people about the fact that this technology had risk. The idea was that they would present it as positive as possible. We've never done that, but too much of the industry did that. So it starts with telling the truth. And the truth is there are risks. Risks are real. But also, you have a say in how these risks are addressed. We're doing the best we can to work with government to create the proper oversight of this technology, to give voters and elected officials a say in how this technology is deployed. And we want to work with those folks as much as we can. I want everyone to feel like they're appreciating the incredible benefits of the technology, but also they have a say in their voice and in their vote in determining how this technology is deployed. This isn't a decision for a few people alone. This is a decision for everyone, but it is an important decision. I won't lie that the stakes aren't high. This is one of the seminal technological events in human history and we all have to have a part in it.