Anthropic CEO Dario Amodei：人类需要觉醒面对 AI 风险

Dario Amodei 是 Anthropic 的 CEO。我认为他也是最直言不讳的真相讲述者之一，关于 AI 的好、坏以及潜在的丑陋和毁灭。我们临时有机会与 Dario 交谈。他的备忘录已经发布。我们说想和他深入聊聊。用 Dario 的话说，五个字：人类需要觉醒。去年他发出警告，称 50% 的白领工作可能因 AI 在几年内过时，这引发了一场全国性讨论。在这份备忘录中，他采取了不同的方向。他试图说明这不只是工作问题。它可能关乎你的国家安全，你的生活方式。这份备忘录写得很有煽动性，也确实如此。他最大的警告之一，连同威权政府，是针对 AI 公司。他说：“作为 AI 公司的负责人，我这么说很尴尬，但看看他们拥有的用户数量，看看他们的数据中心，看看他们拥有的权力。如果他们洗脑这个庞大的消费者用户群呢？”这对我来说是新的。Jim，这是一个人们需要回应的警钟。他们需要倾听。你可能持怀疑态度，可能害怕，可能热情。Dario 所说的很重要，而且它以一种煽动性但准确的方式综合了我们与其他人在一次次对话中听到的内容。这不是他歇斯底里，也不是他在炒作技术。我认为他反映了一种我们在私下对话中反复听到的担忧，来自那些构建和使用这些技术的人。我们想超越备忘录。我们想具体谈谈，如果以最直白的方式传达，Dario 想向国会议员和联邦政府成员传达什么信息？因为他们在监管技术以及告知公民和选民方面扮演着不可或缺的角色。你希望国会现在做哪三件事？还有，如果他们真正了解情况并完全坦诚，你希望他们告诉选民什么？

Dario Amodei is the CEO of Anthropic. He's also, I think, one of the most vocal truthtellers about the good, the bad, and the potential ugly and destruction from AI. We had a spur of the moment chance to talk to Dario. His memo was out. We said we wanted to go a little deeper with him. The way that Dario says it, five words, humanity needs to wake up. Last year when he made that warning that 50% of white collar jobs could be obsolete within a couple years because of AI, he ignited a national conversation. With this memo, he's taking a different track. He's trying to say it's not just jobs. It could be your national security. It could be your way of life. And it was written to be provocative. It is provocative. One of his biggest warnings along with authoritarian governments was AI companies. He said, "It's awkward for me to say this as the head of an AI company, but look at all of the users that they have. Look at all the data centers they have. Look at the all the power they have. And what if they were to brainwash this massive consumer use base?" That was new to me. Jim, this is a wakeup call that people need to answer. They need to listen. You might be skeptical. You might be scared. You might be enthusiastic. What Dario has to say is important and by the way it synthesizes provocatively but I think accurately what we hear in conversation after conversation with other people. It's not him just being hysterical. I don't think it's him just hyping the technology. I think he reflects a concern we hear time and time again at least in off-the-record conversations with the people that are building these technologies and using them. And we wanted to go beyond the memo. We wanted to talk specifically about what message, if it was delivered in its bluntest form, would Dario want to deliver to members of Congress and members of the federal government? Because they play such an integral role in regulating the technology, but also in informing their citizens and their constituents. What are the three things that you wish Congress would do now? And then also what do you wish they would tell their constituents if they were really fluent on what's going on and they were completely leveling with them?

是的。所以我认为现在要做三件事。第一是尽可能强有力的透明度立法。你们运行了哪些测试？你们在模型方面看到了什么？公司有能力研究这些东西，而且他们经常这样做。所以要求他们不仅研究这些风险，还要向公众展示这些风险，在产品上贴上标签——我认为这对消费者非常有帮助，也有助于公司之间相互学习。如果每家公司都独自研究这些东西，并且因为竞争而不敢向他人展示自己的发现，我们就无法作为一个科学界来了解这些。第二件事，我已经说过很多次，就是美国公司之间要应对这场疯狂的商业竞赛已经够难了。但理论上，我们可以通过我刚才描述的法律来约束公司。然而，如果我们有一个几乎和我们一样快速构建技术的威权对手，这几乎是不可能的，对吧？这造成了一个可怕的困境。我认为我们需要切断供应链。我们在芯片方面领先他们多年。我们确实可以切断供应链。这给了我们时间和缓冲来妥善处理这些危险。第三，我认为我们需要考虑这项技术收益的分配。我看到 AI 创造了一个经济巨大增长的世界。我们可以治愈癌症，可以更便宜地开发能源，可以开发巨大的新材料。这些东西将极大地增长经济。但正因为 AI 做了许多当前白领工人所做的工作，财富将从劳动力向资本集中。所以我们将面临一个我们从未见过的奇怪世界，我们拥有巨大的财富，但分配是个问题。那是一个不同的世界。我不认为这是意识形态问题，但我认为我们需要适应那个世界。

Yeah. So I think three things to do now. One would be transparency legislation as robust as possible. What tests did you run? What are you seeing with respect to your model? Companies have the capability to study these things and they often do. So requiring they not only study these risks but show those risks to the public, put a label on the product — I think that's really helpful to the consumer and it's also helpful in that it allows companies to learn from each other. If each company is studying these things on its own and is afraid to show what it's finding to others because of competition, we can't learn about these things as a scientific community. I think the second thing, and I've said it many times, is it's hard enough between the companies in the US to handle this crazy commercial race. But in theory, we could pass laws like I just described that help to reign the companies in. But it's almost impossible to do that if we have an authoritarian adversary who's out there building the technology almost as fast as we are, right? It creates a terrible dilemma. And I think we need to cut off the supply chain. We're years ahead of them in chips. We really can cut off the supply chain. And that gives us the time and the buffer to deal with these dangers properly. And then third, I think we need to think about the distribution of benefits of this technology. I see AI creating a world where there's enormous economic growth. We can cure cancer. We can develop energy for cheaper. We can develop enormous new materials. And those things will grow the economy enormously. But precisely because AI does the jobs that many current white collar workers do, there's going to be some concentration of this wealth from labor to capital. And so we're going to have this weird world that we've really never seen before where we have enormous wealth, but distribution is a problem. That's a different world. I don't think it's an ideological thing, but I think we just need to adjust to that world.

现在就适应？比如什么？显然，我们还没有。那不是今天的现实。那么国会如何让国家做好准备，以免我们措手不及，不得不事后补救？

Adjust to that now. Like what? Obviously, we don't have that. That's not the reality today. So how would Congress prepare the country to do that so we're not caught napping and having to do it retroactively?

也许最明显的一点是，我们需要考虑更强有力的税收政策。我不认为这是过去的税收政策。这是为人们成为万亿富翁的世界准备的。我们离那一步已经不远了，比如 Elon Musk。我认为 AI 和 AI 公司的影响将使这种情况更加极端。我这么说是因为我也是受益者之一，对吧？如果我们找不到一个设计良好的答案来解决这个问题，我们可能会得到设计糟糕的答案。我们可能会得到非常激进、设计糟糕的答案。所以我的要求是：看，财富将出现这种倾斜和分配不均。有哪些经济上明智且合理的方式来处理它，以免我们出现这种疯狂的膝跳反应？

Maybe the most obvious one is we need to think about more robust tax policies. I don't think this is the tax policies of old. This is for a world where people are trillionaires. We're almost there already with Elon Musk. And I think the effect of AI and the effect of the AI companies is going to make that more extreme. And I say that as one of the people who's benefiting from it, right? If we don't find a well-designed answer to this problem, we may get poorly designed answers. We may get very aggressive, poorly designed answers. So my ask would be: look, there's going to be this skew and distribution of wealth. What are ways of handling it that are economically literate and economically sensible so that we don't get this crazy knee-jerk stuff?

Dario，这些都是艰巨的任务。我交谈过的国会议员甚至害怕谈论这个问题。他们的选民要么担心自己的工作，要么对电费感到愤怒，要么觉得这很恶心或不安。你如何说服宾夕法尼亚大道两端的政策制定者，他们可以而且必须讨论这些问题并深入探讨？

Dario, these are heavy lifts. Members of Congress I talked to are afraid to even talk about this issue. Their constituents either are worried about their jobs or pissed about their power bills or they think it's icky or they're queasy. How do you convince policy lawmakers at both ends of Pennsylvania Avenue that they can and must talk about these issues and dig in?

这不会一天之内发生。但我要说的是，随着我们看到 AI 的影响，我预计公众会理解 AI 正在给我们带来所有这些奇迹，所有这些医学奇迹，所有这些富足。最终，我们将拥有便宜的机器人，它们会做所有事情。但这些问题会出现。人们会说：“我的工作呢？”人们会说：“为什么那个人是万亿富翁，而我的工资却因为技能过时而下降了？”人们会问这些问题。我认为最好提前应对，现在就开始思考。顺便说一句，我不认为这会是党派问题。现在甚至都不是党派问题。

So it's not going to happen in a day. But what I will say is as we see the effects of AI, I expect the public to understand that AI is bringing us all these wonders, all these medical wonders, all this abundance. Eventually, we'll get cheap robots that will do everything. But these problems will emerge. People will say, "Where are my jobs?" People will say, "Why is that person a trillionaire and my wage has gone down because I've been deskilled?" People will ask these questions. And I think it's better if you get ahead of it and you start to think about it now. And by the way, I don't think it'll be a partisan thing. It's not even a partisan thing now.

我聊过的政治光谱两端的人，他们说的话惊人地相似。你觉得你那些未来的万亿富翁同行们会讨论这个问题，还是会抵制它？

Even people on the two extremes of the political spectrum I've talked to, it's remarkable how similar the things they say are. Do you think that any of your fellow future trillionaires will discuss this or will they fight it?

我无法预测别人会怎么做。我只能告诉你，很多人现在还没到这个认识。

I can't say what anyone else is going to do. I can tell you, a lot of them aren't there now.

你能做些什么来让他们跟上你的思路？

What can you do to bring them along with how you're thinking?

我同意。很多人现在还没到这个认识。观点分歧很大。我不能替别人发言，但我想重申我之前说的：你不能到处说我们要创造巨大的富足，大部分归我们，我们要成为万亿富翁，没人会抱怨。如果你的回答只是‘去你的，我们无能为力也不该做什么’，那会引发大量不满。已经开始了。我们正看到苗头，而且会越来越糟。我的观点是，我们应该这样做，因为这是正确的事。但如果这对别人没有说服力，那么我会说：看，如果你不以正确的方式来做，你会招来一群暴民。如果你不以正确的方式做，事情会以非常错误的方式发生。

I agree. Many are not there now. There's a wide range of views. I can't speak for anyone else, but I'd say the thing I said before: you can't just go around saying we're going to create all this abundance, a lot of it is going to go to us, and we're going to be trillionaires, and no one's going to complain. If your answer is just 'screw you, there's nothing we can or should do,' that's going to create a lot of discontent. It already has. We're starting to see the beginnings of it, and it's going to get worse. My view is we should do this because it's the right thing to do. But if that isn't compelling to others, then I'd say: look, you're going to get a mob coming for you if you don't do this in the right way. If you don't do it in the right way, it's going to happen in a very wrong way.

国会议员应该告诉选民关于 AI 的现状以及未来一年走向的什么信息？

What should members of Congress be telling their constituents about the state of AI and where we're headed over the next year?

AI 领域的情况很有意思。人们对此感到担忧。总体而言，这种担忧是有充分理由的，但我不确定它是否完全针对正确的问题。比如 AI 的用水量——如果你仔细研究，AI 实际上并不消耗那么多水。AI 有很多问题，但这不是其中之一。当然，人们担心他们的电费账单，我认为这可以理解，而且某种程度上是切中要害的。但从长远来看，问题不在于电费账单，而在于巨大的富足以及他们能否分得一杯羹。电费账单只是其中很小的一部分。所以我认为选民们是担忧的。但帮助他们了解未来的走向，带领他们一起前进——因为再说一遍，如果你不引领，如果你不说事情会如何发展，我们在努力寻找解决方案，即使我们还没有所有答案，比如我们支持你们，我们在努力寻找解决方案——我认为这比说没什么好担心的，或者只盯着这些非常有限的问题要好得多。

We have an interesting situation in AI. People are concerned about it. Broadly, that concern is well justified, but I don't know that it's all that well targeted. There are risks like the water use of AI—if you look into it, AI actually doesn't use that much water. There are many problems with AI, but that's not one of them. And of course people are worried about their power bills, which I think is understandable and kind of well targeted. But in the long run, it's not about power bills; it's about enormous abundance and whether they get their piece of it. Power bills are a tiny piece of that. So I'd say constituents are concerned. But helping to educate them about where things are going, helping to bring them along—because again, if you don't lead, if you don't say this is where things are going and we're looking hard for solutions, even if we don't have all the answers yet, like we've got your back, we're trying to find the solutions—I think that will end much better than saying there's nothing to worry about or only looking at these very limited problems.

华盛顿的假设是，因为特朗普总统、戴维·萨克斯等人希望放手让 AI 发展，让美国赢得与中国的竞赛，国会似乎没有干预的意愿。等三年再采取行动的风险是什么？这似乎是目前最可能的情况。

The assumption in Washington is because President Trump, David Sachs, and others want to be hands-off on AI and have the US win the race against China, Congress seems to have no appetite to intervene. What are the risks of waiting three years to do anything, which seems like the most likely scenario right now?

如果我们等三年，这项技术正在呈指数级发展。三年前的 2023 年，模型可能只相当于一个聪明的高中生。现在，Anthropic 的工程师们，模型为他们编写所有代码，工程师可能只是编辑一下。我们非常接近中高级专业水平了。这仅仅过了三年。如果再等三年，我认为我们会得到我在文章中所谓的‘数据中心里的天才国度’。可能还不用三年。三年在这个领域是永恒。所以我认为我们绝对需要在此之前采取行动。我真正抱有希望的一点是，我认为当这些问题开始显现时，它们不会带有党派色彩。可能一开始某个政党或一方持有反监管的意识形态，但随着问题变得真实，所有人都会产生需求。

If we wait three years, this technology is progressing exponentially. Three years ago in 2023, the models were maybe as smart as a smart high school student. Now, we have engineers at Anthropic where the model writes all the code for them, and the engineer maybe edits it. We're very close to mid to high professional level. That was just in three years. If we wait another three years, I think we'll get what I call in the essay 'a country of geniuses in a data center.' Maybe less than three years. Three years is an eternity in this field. So I think we absolutely need to act before then. One place where I really have hope is I think as these problems start to manifest, they're not going to be partisan. It may start with one party or one side having an anti-regulatory ideology, but as these problems become real, there's going to be a demand among everyone.

在文章中你列出了不同的风险，无论是生物恐怖主义还是威权政权拥有太多工具进行颠覆行为。你有多担心？你显然足够担心才说出来并提出来。但在你看来，这种结果的可能性有多大，特别是如果我们未来三年什么都不做？是 1%的概率，还是说如果三年什么都不做，我们就完蛋了？

In the note you outline different risks, whether it's bioterror or authoritarian regimes with too many tools to do subversive behavior. How worried are you? You're obviously worried enough to state it and raise it. But in your mind, how likely is that outcome, particularly if we don't do anything for the next three years? Is it like 1% or if you don't do anything for three years, we could be screwed?

这总是很难说。我在文章中说过，我们就是不知道。我们可能会回顾说，‘哈哈，AI 驱动的生物恐怖主义，当时听起来可能发生，但根本没发生。’这非常不可预测。我想说的是，我们在运营行为上采取了一种偏执的立场。我们总是假设所有可能出错的事情都会出错。这就是你构建可靠东西的方式。如果你在造火箭，你不会说，‘哦，我确定这个部件没问题。’你会说，‘不，我要对这个和那个做情景分析。’我不会想当然。如果政府介入并采取适当行动，那么我认为我们成功的机会会大大增加。即使没有，我们也会尽力而为。但我认为，如果我们的政策制定者不玩忽职守，很多事情会容易得多。

It's always hard to tell. One of the things I say in the essay is we just don't know. We could look back and say, 'Haha, AI-driven bioterror, that sounded like it could happen at the time, but it just didn't happen at all.' It's very unpredictable. The way I would say it is we're taking a paranoid stance with respect to our operational behavior. We always assume that everything that can go wrong does go wrong. That's how you build things that are reliable. If you're building a rocket, you're not like, 'Oh, I'm sure this part will work out.' You're like, 'No, I'm going to do a scenario analysis of this and that.' I'm not going to take anything for granted. If government steps in and takes the appropriate actions, then I think our chances of success go up a lot. We'll do the best we can even if that doesn't happen. But I think a lot of things get easier if our policymakers are not asleep at the wheel.

我想我们时间到了，达里奥。感谢你抽出时间来做这个。备忘录很吸引人，宣言也很棒。我们很感激。

I think we're getting the hook, Dario. We appreciate you taking time to do this. The memo is fascinating. The manifesto is great. So, we appreciate it.

谢谢你的对话。

Thank you for the conversation.

你花了多长时间写这份备忘录？

How long did you work on the memo?

我在寒假期间 72 小时内写出了初稿。

I wrote the first draft in 72 hours over winter break.

嗯，老实说，我的寒假就是先花一周放空打游戏，然后在最后三天才想，哎呀，我得做点正事。然后我几乎不眠不休地写了 72 小时。

Um, I honestly, my winter break is like I spend a week just zoning out and playing video games, and then like in the last three days of winter break I was like, oh man, I should try and get something right. And so I wrote for like 72 hours almost without sleeping.

其中有多少是 Claude 写的？

How much of it was Claude?

Claude 一个字都没写。不过 Claude 帮我做了不少研究，还给了我反馈。我觉得我是作者，Claude 更像是我的编辑和研究助理。

Claude did not write any of it. Claude helped me though to do a fair amount of research and Claude gave feedback. I would say I was the writer and Claude was kind of my editor and my research assistant.

说得太好了。感谢你的时间。

Drop the mic. Thanks for the time.