AI 安全与可解释性：对话 Dario Amodei

大家好，欢迎来到《In Good Company》。今天非常激动，我们请来了 Anthropic 的 CEO 兼联合创始人 Dario Amodei。Dario 是 AI 界的巨星，他和团队开发了 Claude 语言模型，这是目前最顶尖的模型之一，并且得到了亚马逊和谷歌的支持。Dario，你是 AI 安全和伦理领域的领军人物，甚至中断了假期来和我们交流。非常感谢你的到来。

Hi everybody, welcome to In Good Company. Today, really exciting, we have Dario Amodei, the CEO and co-founder of Anthropic, visiting now. Dario, he is a superstar in the AI world, and together with his team has developed the Claude language model, one of the best out there, and they are backed by Amazon and Google. Now, you are a leading figure, Dario, on AI safety and ethics, and you even interrupted your holiday to come here and talk to us. So big thanks for coming.

谢谢你的邀请。

Thank you for having me.

那么，AI 领域最新的突破是什么？

Now, what are the latest breakthroughs in AI?

是的，我可以谈几点。首先，我认为 AI 的 Scaling 趋势仍在继续。未来一年，我们会看到更大、更强大的模型，能够完成更复杂的任务。事实上，在这个播客播出时，Anthropic 将发布一个新模型，它很可能是世界上最智能、最强大的模型。但我特别兴奋的一个领域是模型的可解释性——能够看到 AI 模型内部，理解它们为何做出某些决策。过去几年，这主要是一个研究领域，现在才刚刚开始有实际应用。所以这是我非常看好的一个方向。

Yes, so a few things I could talk about. One is, I think the scaling trends of AI are continuing. So I think we're going to see over the next year much bigger and more powerful models that are able to do greater tasks. In fact, by the time this podcast airs, a new model will be out from Anthropic that will probably be the most intelligent and powerful model in the world. But one area I'm particularly excited about that we're developing in parallel with that is interpretability of models—the ability to see inside our AI models and see why they make the decisions they make. That area has been mainly a research area for the last few years, and it's just at the beginning of starting to have practical applications. So that's one area I'm very excited about.

为什么这如此重要？

Why is that so important?

看看今天 AI 模型的表现，你往往不明白它为什么这么做。我午餐时刚和一个人聊过。假设你想训练一个 AI 模型，用一些数据来预测某组金融数据的结果。训练时的一个问题是，如果模型用历史数据训练，它可能已经记住了结果——因为它知道未来。在这种情况下，可解释性可以帮你区分：模型是在推导答案，还是在记忆答案？同样，如果模型表现出对某个群体的偏见，或者看起来如此，我们能否审视它的推理过程？它真的是被偏见驱动的吗？此外，还有一些法律要求，比如欧盟有“解释权”。所以可解释性——能够看到模型内部——可以帮助我们理解模型为什么说和做那些事，甚至进行干预并改变它们的行为。

If you look at what AI models do today, often you won't understand why an AI model does what it does. I was just talking to someone at lunch. Let's say you want an AI model to be trained on some data to be able to predict what happened with a particular set of financial data. One problem you have with training a model to work on that is that if you train it on data from the past, the model might have memorized it because it basically knows what happens—it knows the future. In that case, interpretability might allow you to tell the difference: is the model deducing the answer to the question, or is it memorizing the answer? Similarly, if a model acts in a way that shows prejudice against a particular group, or appears to do so, can we look at the reasoning of the model? Is it really being driven by prejudice? There are also a number of legal requirements, right? In the EU, there's a right to explanation. So interpretability—being able to see inside the model—could help us understand why models do and say the things they do and say, and even to intervene and change what they do and say.

你之前说过，我们仍然不知道高级 AI 模型是如何工作的。这是否意味着可解释性会解决这个问题？

So a while back you stated that we still don't know how advanced AI models work. Does this mean that interpretability will solve this problem?

我不会说“解决”。我们才刚刚开始。也许我们现在只理解了它们工作方式的 3%。我们目前能做到的是，可以观察模型内部，找到对应复杂概念的特征。一个特征可能代表“诚实”、“犹豫”、某种音乐类型、角色所处的某种隐喻情境，或者对某些群体的偏见。我们找到了这些特征，但可能只是冰山一角。我们仍然不明白所有这些特征如何相互作用，产生我们每天看到的模型行为。这有点像大脑，对吧？我们可以做脑部扫描，了解一些人类大脑的信息，但我们没有它的规格表。我们无法说，‘这就是那个人为什么那么做的原因。’所以，我们最终能完全理解它们吗？我不确定能否细致到每一个细节，但我认为进展很快，我对达到这个目标持乐观态度。

I wouldn't say solve. I would say we're at the beginning. Maybe we now understand 3% of how they work, really. We're at the level where we can look inside the model and find features inside it that correspond to very complex concepts. One feature might represent the concept of honesty, hesitating, a particular genre of music, a particular type of metaphorical situation a character could be in, or the idea of prejudice for or against various groups. So we have all of these features, but we think we've only found a small fraction of what there is. And what we still don't understand is how all of these things interact to give us the behaviors we see in models every day. It's a little like the brain, right? We can do brain scans, we can say a little about the human brain, but we don't have a spec sheet for it. We can't go and say, 'This is why that person did exactly what they did.' So will we ever understand fully how they work? I don't know about down to the last detail, but I think progress is happening fast, and I'm optimistic about getting there.

但进展的速度是否快于新模型的复杂度？

But is progress happening faster than the complexity of the new models?

这是个好问题，也是我们正在应对的挑战。我们正在投入大量资源进行语言模型的可解释性研究，试图跟上模型复杂度增长的速度。我认为这是该领域最大的挑战之一。这个领域发展太快了，包括我们自己的努力，我们希望确保我们的理解能跟上我们创造强大模型的能力。

That is a great question, and that is the thing we're contending with. So we are putting a lot of resources behind interpretability of language models to try and keep pace with the rate at which the complexity of the models is increasing. I think this is one of the biggest challenges in the field. The field is moving so fast, including by our own efforts, that we want to make sure our understanding keeps pace with our abilities—our capabilities to produce powerful models.

你们的模型好在哪里？

What's so good about your model?

这是 Claude 模型，对吧？我们最近发布了一系列 Claude 3 模型，分别叫 Opus、Sonnet 和 Haiku。它们在能力、智能、速度和低成本之间做了不同的权衡，同时保持智能。Opus 发布时，它实际上是世界上最好的全能模型。但我认为它特别好的一个原因是，我们在它的性格上投入了大量工程。我们最近发了一篇文章，介绍如何设计 Claude 的性格。人们普遍觉得 Claude 模型更温暖、更人性化，他们更喜欢与它互动。其他一些模型听起来更机械、更缺乏灵感。我们正在快速创新，正如我所说，到这个播客播出时，我们可能至少会推出新一代模型的一部分。

So this is the Claude model, right? These are Claude models. To give some context, we recently released a set of Claude 3 models. They're called Opus, Sonnet, and Haiku. They offer different trade-offs between power and intelligence, and speed and low cost, while still being intelligent. At the time Opus was released, it was actually the best all-around model in the world. But I think one thing that particularly made it good is that we put a lot of engineering into its character. We recently put out a post about how we design Claude's character. People have generally found the Claude models are warmer, more human; they enjoy interacting with them more. Some of the other models sound more robotic, more uninspired. We're continuing to innovate quickly, and as I said, by the time this podcast comes out, we'll probably have at least part of a new generation of models out.

给我讲讲新模型吧。

Tell me about the new one.

我不能透露太多，但如果非要说一点的话，那就是我们正在推动前沿。速度、低成本和质量之间存在权衡。你可以想象一条权衡曲线，一个前沿。新一代模型将把这条前沿向外推。所以到这个播客播出时，我们会给它起个名字，至少是其中一些模型。你会看到，以前需要最强大模型才能完成的任务，现在可以用中端或低端模型来完成，它们更快、更便宜，甚至比上一代更强大。

I can't say too much about it, but if I had to say a bit, I would say that we're pushing the frontier right now. There's a trade-off between speed, low cost, and quality. You can imagine that as a trade-off curve, a frontier. There's going to be a new generation of models that pushes that frontier outward. So by the time this podcast is out, we'll have a name for it, at least for some of those models. And we'll see that things you needed the most powerful model to do, you'll be able to do with some of the mid-tier or low-tier models that are faster, cheaper, and even more capable than the previous generation.

那么 Dario，这里的“哇”因素是什么？当我拿到这个模型时，它会给我带来什么？

So Dario, what's going to be the wow factor here? When I get this model, what is it going to do to me?

你会看到模型在代码、数学、推理等方面更出色。我最喜欢的一个领域是生物学和医学——这是我对新模型最兴奋的应用之一。我们今天的模型在很多知识方面就像大学低年级学生，或者实习生。我认为我们正在把这个边界推向高年级本科生，甚至更远。

You're going to see models that are better at things like code, math, reasoning. One of my favorites is biology and medicine—that's one of the sets of applications I'm most excited about for the new models. The models we have today are kind of like early undergrads in their knowledge of many things, or like interns. I think we're starting to push that boundary towards advanced undergrads or even...

研究生级别的知识，所以当我们考虑模型用于药物开发，或者在你自己的行业里，用模型来思考投资甚至交易时，我认为模型在这些任务上会变得更加复杂。我们希望每隔几个月就能发布一个新模型，不断突破这些界限。最近加速的一件事就是我们如何将 AI 融入我们所做的一切。

Graduate level knowledge and so when we think of use of models for drug development or, you know, in your own industry, use of the models for thinking about investing or even trading, I think the models are just going to get a good deal more sophisticated at those tasks. And we're hoping that every few months we can release a new model that pushes those boundaries further and further. Now, one of the things which has accelerated lately is just how we weave AI into everything we do.

是的，最近苹果和 OpenAI 的公告，你怎么看？

Yes, and with the recent announcement from Apple and OpenAI, how do you look at this?

Anthropic 更倾向于为企业提供服务，而非面向消费者。所以我们正在思考如何将 AI 融入工作环境。如果你想想现在的模型和聊天机器人，在企业环境中使用它们，就像你从街上随便拉一个很聪明但对公司一无所知的人，然后请他们给建议。我真正想要的是一个更像 AI 模型的东西，它就像一个经过多年公司知识培训的人。所以我们正在努力将我们的 AI 模型连接到知识数据库，让它们能引用工作内容，能使用企业内部工具，真正作为员工的虚拟助手融入企业。这就是我推动整合的一种方式。

Anthropic thinks of itself more as providing services to enterprises than on the consumer side. So we're thinking a lot about how to integrate AI in work settings. If you think about today's models, today's chatbots, it's a bit like if I use them in an enterprise setting, it's like if I took some random person on the street who was very smart but knew nothing about your company, and I brought them in and asked them for advice. What I really like is someone that acts more like an AI model that acts more like someone that's been trained with knowledge of your company for many years. So we're working on connecting our AI models to knowledge databases, having them cite work, having them be able to use internal enterprise tools, and really integrate with the enterprise as a sort of virtual assistant to an employee. So that's one way I think about driving the integration.

如果看 Anthropic 的长期目标，是什么？

If you look at the long-term goal of Anthropic, what is the long-term goal?

我们只有三年半的历史，是前沿模型领域最新的参与者。我们是一家公益公司，我认为我们的长期目标是确保这一切顺利进行。这是通过公司这个载体来实现的，但如果你考虑我们的长期战略，我们真正想做的是创造所谓的“竞相向上”。竞相向下是众所周知的，每个人都为了竞争而偷工减料。我们认为有一种方式可以产生相反的效果：如果你能制定更高的标准，以更道德的方式创新，那么其他人就会效仿。他们要么受到启发，要么被自己的员工或公众舆论逼迫，最终法律也会朝那个方向发展。所以我们希望提供一个如何正确做 AI 的榜样，并带动整个行业。这背后有很多工作，比如我们的可解释性工作、安全工作，以及我们对负责任 Scaling 的思考。我们有一个负责任 Scaling 政策。所以我们的总体目标是帮助整个行业变得更好。

We're only three and a half years old, by far the newest player in the space that's been able to build models on the frontier. We're a public benefit corporation, and I think our long-term goal is to make sure all of this goes well. That's being done through the vehicle of a company, but if you think about our long-term strategy, what we're really trying to do is create what we call a race to the top. Race to the bottom is a well-known thing where everyone fights to cut corners because market competition is so intense. We think there's a way to have the reverse effect: if you're able to produce higher standards, innovate in ways that make the technology more ethical, then others will follow suit. They'll either be inspired by it or bullied into it by their own employees or public sentiment, or ultimately the law will go in that direction. So we're hoping to provide an example of how to do AI right and pull the rest of the industry along with us. That's a lot of the work behind our interpretability work, our safety work, and how we think about responsible scaling. We have something called a responsible scaling policy. So our overall goal is to try to help the whole industry be better.

所以你把自己定位为好人了？

So you kind of pitch yourself as the good guys?

我不会说得那么宏大。我更倾向于从激励和结构的角度思考，而不是好人和坏人。我想帮助改变激励，这样每个人都能成为好人。

I wouldn't say anything that grandiose. It's more like I think in terms of incentives and structures more than good guys and bad guys. I want to help change the incentives so that everyone can be the good guys.

你认为我们会在意与哪个模型互动吗，还是会有某个智能体为我们选择最适合的模型？比尔·盖茨在播客上就是这么说的。

Do you think we will care which model we interact with, or are we going to have just one agent who picks the model that's best for that purpose? That was kind of what Bill Gates said when he was on the podcast.

我认为这完全取决于场景。有几点：一是我们越来越倾向于模型各有所长。例如，我刚才谈到 Claude 的性格。Claude 更温暖友好，互动起来很舒服。对于很多应用场景来说，这非常理想。对于其他应用，专注于不同方面的模型可能更有帮助。有些人走智能体路线，有些人走擅长代码的模型路线。比如 Claude 还擅长创意写作。所以我认为我们会有一个生态系统，人们根据不同的目的使用不同的模型。实际上，这是否意味着有东西为你选择模型？我认为在某些消费者场景中会是这样。在其他场景中，有人会说，‘哦，我做的这份工作或者我这种性格，我想一直用这个特定模型。’

I think it really depends on the setting. A few points: one is that we are increasingly going in the direction where models are good at different things. For example, I was just talking about Claude's character. Claude is more warm and friendly to interact with. For a lot of applications and use cases, that's very desirable. For other applications, a model which focuses on different things might be helpful. Some people are going in the direction of agents, some people are going in the direction of models that are good at code. Claude, for example, is also good at creative writing. So I think we're going to have an ecosystem where people use different models for different purposes. In practice, does that mean there's something choosing models for you? I think in some consumer contexts, that will be the case. In other contexts, someone will say, 'Oh, the job I'm doing or the kind of person I am, I want to use this particular model all the time.'

但什么让模型温暖？你怎么让模型友好？是更幽默还是更礼貌，或者只是加一些红心？

But what makes a warm model? I mean, how can you make a model friendly? Is it more humorous or more polite, or just putting some red hearts in between?

我们实际上尽量避免太多表情符号，因为会烦人。但如果你上 Twitter，看到人们与 Claude 互动的一些评论，它听起来更像人类。我认为很多机器人都有某些习惯，比如模型会说：‘抱歉，作为一个 AI 语言模型，我不能做 X、Y 和 Z。’这是常见说法。我们帮助模型更多样化地思考，听起来更像人类。

We actually try to avoid too many emojis because it gets annoying. But if you go on Twitter and see some of the comments when people interact with Claude, it just kind of sounds more like a human. I think a lot of these bots have certain ticks, like models will say, 'I apologize, but as an AI language model, I can't do X, Y, and Z.' That's a common phrase. We've helped the model to vary their thinking more, to sound more like a human.

当你发布新模型时，你能很好地预测它的准确性，对吧？这是参数数量的函数等等。那么要达到 AGI，我们还有多远？这是通用智能，比这更智能。

When you launch new models, you get pretty good predictions on how accurate it will be, right? It's a function of number of parameters and so on. Now to get to AGI, how far out are we? This is the general intelligence, so more intelligent than this.

我说过几次，但十年前当这一切还是科幻时，我经常谈论 AGI。我现在有不同的看法，我不认为它是一个时间点。我只是认为我们处在一个平滑的指数曲线上，模型随着时间的推移越来越好。没有一个点说：‘哦，模型以前不是通用智能，现在是。’我只是觉得，就像人类孩子学习和成长一样，它们越来越好，越来越聪明，知识越来越丰富。我不认为会有一个单一的标志性时刻。但我认为有一种现象正在发生，随着时间的推移，这些模型甚至比最优秀的人类还要好。我确实认为，如果我们继续增加规模，模型的资金投入，比如说达到 100 亿……那么现在一个模型要花多少钱？1 亿？现在大概是 1 亿。今天正在训练的模型有些已经接近 10 亿。我认为如果我们达到 100 亿或 1000 亿，而且我认为……

I've said this a few times, but back 10 years ago when all of this was kind of science fiction, I used to talk about AGI a lot. I now have a different perspective where I don't think of it as one point in time. I just think we're on this smooth exponential, the models are getting better and better over time. There's no one point where it's like, 'Oh, the models weren't generally intelligent and now they are.' I just think, like a human child learning and developing, they're getting better and better, smarter and smarter, more and more knowledgeable. I don't think there will be any single point of note. But I think there's a phenomenon happening where over time these models are getting better and better than even the best humans. I do think that if we continue to increase the scale, the amount of funding for the models, if it goes to say 10 billion... so now a model would cost what, 100 million? Right now 100 million. There are models in training today that are more like a billion. I think if we go to 10 or 100 billion, and I think...

那会在 2025 年、2026 年或 2027 年发生。算法改进和芯片改进都在快速推进。我认为到那时，我们很有可能会得到在大多数事情上比大多数人类更优秀的模型。所以 100 亿，你认为明年就会有这样的模型？

That will happen in 2025, 2026, maybe 2027. And the algorithmic improvements continue apace, and the chip improvements continue apace. Then I think there is, in my mind, a good chance that by that time we'll be able to get models that are better than most humans at most things. So 10 billion, you think a model will be next year?

我认为训练一个 100 亿美元的模型可能会在 2025 年的某个时候开始。没有多少人能参与这场竞赛。不，不，不，不。当然，我认为会有一个充满活力的下游生态系统，也会有一个小型模型的生态系统。你没有那么多钱。我的意思是，我们大概有那么多。我们迄今为止筹集了，我相信，略超过 80 亿美元，对吧？所以大致在那个数量级。当然，我们总是对达到下一个规模水平感兴趣。这当然也取决于芯片。我们刚刚得知英伟达缩短了发布周期，对吧？过去是每两年一次，现在差不多每年一次。这有什么影响？

I think that the training of a $10 billion model could start sometime in 2025. Not many people can participate in that race. No, no, no, no. And you know, of course I think there's going to be a vibrant downstream ecosystem, and there's going to be an ecosystem for small models. You don't have that much money. I mean, we have on the order of that. We've raised, I believe, a little over $8 billion to date, right? So generally on the order of that. And of course, we're always interested in getting to the next level of scale. Now this is, of course, also a function of the chips. And we just learned that Nvidia has shortened the time between launches, right? So in the past, every other year, now it's more like every year. What are the implications of this?

是的，我认为这是认识到芯片将变得极其重要的自然结果，对吧？同时也面临竞争。谷歌正在制造自己的芯片，我们知道。亚马逊也在制造自己的芯片。Anthropic 正在与两者合作，使用这些芯片。不具体说，我能说的是芯片行业竞争非常激烈，有一些非常强大的产品。谷歌和亚马逊在芯片开发上落后多少？我知道这不是我能说的，但只是某种指示。再次，我只想重复，我认为现在有多家厂商提供了强大的产品，这些产品对我们有用，并且将在不同方面对我们有用。

Yeah, I think that is a natural consequence of the recognition that chips are going to be super important, right? And also facing competition. Google is building their own chips, as we know. Amazon is building their own chips. Anthropic is collaborating with both to work with those chips. And without getting specific, what I can say is that the chip industry is getting very competitive, and there are some very strong offerings. How far behind are Google and Amazon in chip development? I know that's not something I could say, but just some kind of indication. Again, I would just repeat that I think there are now strong offerings from multiple players that have been useful to us and will be useful to us in different ways.

好的，所以不再只是英伟达的事了？

Okay, so it's not only about Nvidia anymore?

我认为不再只是英伟达的事了。但当然，你看看他们的股票估值，看看他们的股价，你肯定知道，我认为这既是他们自身的指标，也是行业的指标。

I don't think it's only about Nvidia anymore. But of course, you look at their stock valuation, you look at their stock price, which you are certainly aware of, and it's an indicator, I think, both about them and the industry.

你提到你更偏向企业端，不一定在消费端。但最近，关于手机芯片和 AI PC 的讨论越来越多。你怎么看？

You mentioned that you were more on the enterprise side and not necessarily on the consumer side. But just lately, there has been more talk about having chips in phones and we talk about AI PC and so on. How do you look at this?

是的，不，我认为这将是一个重要的发展。再次，如果我们回到我谈到的曲线，对吧？在强大、智能但相对昂贵和慢的模型，与超级便宜、超级快但相对于其速度和成本非常智能的模型之间的权衡曲线。随着这条曲线向外移动，我们将拥有非常快且便宜的模型，它们比今天最好的模型更智能，尽管那时最好的模型会更智能。我认为我们将能够把这些模型放在手机和移动芯片上。它们将越过某个阈值，今天需要调用云或服务器的事情，可以在本地完成。所以我对这其中的影响感到非常兴奋。当然，我对推动前沿更兴奋。但这条曲线向外移动，意味着两者都会发生。

Yeah, no, I think that's going to be an important development. And again, if we go back to the curve I talked about, right? The trade-off curve between powerful, smart but relatively expensive and slow models, and models that are super cheap, super fast, but very smart for how fast and cheap they are. As that curve shifts outward, we are going to have models that are very fast and cheap that are smarter than the best models of today, even though the best models then will be even smarter than that. And I think we'll be able to put those models on phones and on mobile chips. They'll pass some threshold where the things that you need to call to a cloud or server for today, you can do there. So I'm very excited about the implications of that. Of course, I'm even more excited about pushing the frontier of where things will go. But this curve shifts outward, an implication is that both things will happen.

我们听说法国竞争对手 Mistral 开发了一些非常高效、低成本或较低成本的模型。你怎么看？

We heard from Mistral, the French competitor, that they have developed some really efficient, kind of low-cost or lower-cost models. What do you think of that?

我无法评论其他公司的情况，但我认为我们看到这种曲线整体移动。所以确实我们看到了高效的低成本模型。但我认为这与其说是事情趋于平稳、成本下降，不如说是曲线向右移动。我们可以用更少的资源做更多的事，但也可以用更多的资源做更多的事。所以我认为这两种趋势并存。

I can't comment on what's going on at other companies, but I think we are seeing this kind of general moving of the curve. So it is definitely true we're seeing efficient low-cost models. But I think of it less as things are leveling out, costs are going down, and more as the curve is shifting right. We can do more with less, but we can also do even more with more resources. So I think both trends coexist.

我们换个话题。你的背景：你从物理学开始？

Let's change topic a bit. Your background: you kicked off in physics?

是的，我本科是物理学，然后读了神经科学的研究生。

Yes, I was an undergrad in physics, and then did grad school in neuroscience.

那你最后怎么进入了 AI 领域？

So how come you ended up in AI?

是的，当我完成物理学学位时，我想做一些对世界人类有影响的事情。我觉得理解智能是其中一个重要组成部分，这显然是塑造我们世界的东西之一。那是在 2000 年代中期，说实话，当时我对当时的 AI 并不特别兴奋。所以我觉得当时研究智能的最佳方式是研究人脑。于是我进入神经科学读研究生，计算神经科学，用到了我的一些物理背景，研究神经元的集体特性。但到研究生结束时，经过短暂的博士后，AI 真的开始起作用了。我们看到了深度学习革命，我看到了 Ilya Sutskever 当时的工作。所以我基于此决定进入 AI 领域。我在不同的地方工作过：在百度待过一段时间，在谷歌待了一年，在 OpenAI 工作了五年。

Yeah, so when I finished my physics degree, I wanted to do something that would have an impact on humanity in the world. I felt that an important component of that would be understanding intelligence, that that's one of the things that has obviously shaped our world. That was back in the mid-2000s, and in those days, I wasn't particularly excited about the AI of the day, to be honest. So I felt like the best way to study intelligence in those days was to study the human brain. So I went into neuroscience for grad school, computational neuroscience, that used some of my physics background, and studied collective properties of neurons. But by the end of grad school, after a short postdoc, AI was really starting to work. We saw the deep learning revolution, I saw the work of Ilya Sutskever back then. So I decided based on that to go into the AI field. I worked at different places: I was at Baidu for a bit, I was at Google for a year, I worked at OpenAI for five years.

你在开发 ChatGPT 2 和 3 中发挥了关键作用，对吗？

And you were instrumental in developing ChatGPT 2 and 3, right?

是的，是的，我领导了这两个的开发。

Yes, yes, I led the development of both of those.

你为什么离开？

Why did you leave?

我们在 2020 年底左右达到了一个点，我们这些在这些领域从事这些项目的人有了自己做事的方式。所以我们有了这个图景，我想我已经隐含地描述过了：一是对 Scaling 假设的真正信念，二是安全性和可解释性的重要性。

We had reached around the end of 2020, we had kind of reached a point where the set of us who worked on these projects in these areas had our own vision for how to do things. So we had this picture that I think I've already implicitly laid out: one, real belief in the scaling hypothesis, and two, the importance of safety and interpretability.

所以是安全方面让你离开的？

So it was the safety side that made you leave?

我认为我们只是有自己的愿景。我们有一群联合创始人，真的觉得我们意见一致，真的觉得我们互相信任，真的觉得我们只是想一起做点什么。

I think we just had our own vision of things. There were a set of us who were co-founders who really felt like we were on the same page, really felt like we trusted each other, really felt like we just wanted to do something together.

但你以前比现在更倾向于 AI 末日论？

But you were a bit more AI doomsday before than you are now?

我不会这么说。我的观点一直是存在重要的风险和收益，随着技术呈指数级发展，风险变得更大，收益也变得更大。所以我们在 Anthropic 非常关注这些灾难性风险的问题。我们有所谓的负责任 Scaling 政策，基本上就是在每一步衡量模型的灾难性风险。

I wouldn't say that. My view has always been that there are important risks and there are benefits, and as the technology goes on its exponential, the risks become greater and the benefits become greater. So we at Anthropic are very interested in these questions of catastrophic risk. We have this thing called responsible scaling policy, and that's basically about measuring models at each step for catastrophic risk.

什么是灾难性风险？

What is catastrophic risk?

我会把它分为两类。一类是模型的滥用，可能涉及生物、网络或大规模选举操作等领域——那些对社会极具破坏性的事情。这是一个类别。另一类是模型的自主意外行为。今天可能只是模型做出一些意想不到的事情，但随着模型越来越多地在世界中行动，我们不得不担心它们以你无法预料的方式行事。

I would put it in two categories. One is misuse of the models, which could include things in the realm of biology or cyber or election operations at scale—things that are really disruptive to society. That's one bucket. The other bucket is autonomous unintended behavior of the model. Today it might be the model doing something unexpected, but increasingly as models act in the world, we have to worry about them behaving in ways you wouldn't expect.

你在 GPT-2 或 GPT-3 中具体看到了什么，让你对此特别担忧？

What was it that you saw exactly with GPT-2 or GPT-3 that made you particularly concerned about this?

这不是针对某个特定模型。回到 2016 年，在我加入 OpenAI 之前，我在谷歌工作时，和几位同事（其中一些现在是 Anthropic 的联合创始人）合写了一篇论文，题为《AI 安全的具体问题》。那篇论文提出了一个担忧：我们有这些强大的 AI 模型、神经网络，但它们本质上是统计系统，这就会带来可预测性和不确定性的问题。再加上 Scaling 假设——我在研究 GPT-2 和 GPT-3 时真正开始相信它——这两件事加在一起告诉我：好吧，我们将拥有强大的东西，而控制它绝非易事。所以把这两者结合起来，让我觉得这是一个我们必须解决的重要问题。

It wasn't about any particular model. Going back to 2016, before I worked at OpenAI, when I was at Google, I wrote a paper with some colleagues—some of whom are now Anthropic co-founders—called 'Concrete Problems in AI Safety.' That paper laid out the concern that we have these powerful AI models, neural nets, but they're fundamentally statistical systems, which creates problems about predictability and uncertainty. Combine that with the scaling hypothesis—and I really came to believe in it as I worked on GPT-2 and GPT-3—those two things together told me: okay, we're going to have something powerful, and it's not going to be trivial to control it. So we put those two things together, and that makes me think this is an important problem we have to solve.

你们如何解决这两个灾难性风险问题？

How do you solve the two catastrophic risk problems?

在 Anthropic，我们最大的工具之一是我们的 RSP，即负责任扩展政策。它的运作方式是：每当我们有一个新模型，代表了显著的飞跃——算力超过旧模型一定量——我们就会测量它的滥用风险和自主自我复制风险。

At Anthropic, one of the biggest tools for this is our RSP, our Responsible Scaling Policy. The way it works is: every time we have a new model that represents a significant leap—a certain amount of compute above an old model—we measure it for both the misuse risks and the autonomous self-replication risks.

你们怎么做？

How do you do that?

我们有一套评估体系。对于滥用风险，我们与国家安全领域的人合作。例如，我们与一家名为 Griffin Biosciences 的公司合作，该公司与美国政府签约，从事生物安全工作。他们是应对生物风险的专家。他们会说：哪些东西不在互联网上，但如果模型知道了就会令人担忧？他们进行测试。到目前为止，每次他们都说：嗯，模型在任务上比以前更好了，但还没有达到严重担忧的水平。

We have a set of evaluations that we run. For misuse risks, we've worked with folks in the national security community. For example, we've worked with a company called Griffin Biosciences that contracts with the US government and does biosecurity work. They are the experts on responding to biological risk. They say: what is the stuff that's not on the internet that if the model knew it would be concerning? They run their tests. Every time so far, they've said: well, it's better at the task than before, but it's not yet at the level where it's a serious concern.

所以滥用测试就是如果我输入‘嘿，你能想出一个毁灭地球的病毒吗’？这是一个例子，对吧？

So a misuse test would be if I put in 'hey, can you come up with a virus that's going to wipe out the Earth'? That's an example, right?

概念上是的，尽管这更多不是回答一个问题，而是模型能否完成整个工作流程。一个恶意行为者能否在数周内利用这个模型在现实世界中做坏事？模型能否给他们提示，长期帮助他们完成任务？

Conceptually yes, although it's less about answering one question and more about whether the model can go through a whole workflow. Could a bad actor over weeks use this model to do something nefarious in the real world? Could the model give them hints, help them through the task over a long period?

所以你的意思是，到目前为止，AI 模型还做不到这一点。它们知道一些令人担忧的孤立事情，每次我们发布新模型它们都会变得更好，但还没有达到那个点。另一个呢，自主风险？

So what you're saying is that the AI models so far cannot do this. They know individual isolated things that are concerning, and they get better every time we release a new model, but they haven't reached this point yet. What about the other one, the autonomous risk?

我们测试模型的能力，比如训练自己的模型、配置云算力账户并在这些账户上采取行动、注册账户并进行金融交易——这些只是衡量模型能否摆脱束缚并采取行动的一些指标。

We test the models for things like ability to train their own models, provision cloud compute accounts and take actions on those accounts, sign up for accounts and engage in financial transactions—just some measures of things that would unbind the model and enable them to take actions.

你认为我们离那还有多远？

How far are we away from that, do you think?

我认为这和滥用的情况一样：它们在任务的各个部分上越来越好。有明显的能力提升趋势，但我们还没到那一步。我再次指向 2025、2026，可能 2027 年的时间窗口，就像我认为很多极端的 AI 积极经济应用将在那时到来一样。我认为一些负面担忧也可能在那时开始出现。但我也不是水晶球。

I think it's the same story as with misuse: they're getting better and better at individual pieces of the task. There's a clear trend towards ability to do that, but we're not there yet. I again point to the 2025, 2026, maybe 2027 window, just as I think a lot of the extreme positive economic applications of AI will arrive sometime around then. I think some of the negative concerns may start to arise then as well. But I'm not a crystal ball.

那你们怎么办？装一个终止开关之类的？

What do you do then? Do you build in a kill switch or what?

有很多事情可以做。在自主行为方面，我们很多关于可解释性的工作，很多关于宪法 AI 的工作——那是我们为 AI 系统提供价值观和原则的另一种方式。对于自主风险，我们真正想做的是理解模型内部发生了什么，确保我们设计它并能够迭代，使它不会做我们不想让它做的危险事情。对于滥用风险，更多的是在模型中设置防护措施，让人们无法要求它做危险的事情，并且我们可以监控人们何时试图用它做危险的事情。

There are a number of things. On the autonomous behavior, a lot of our work on interpretability, a lot of our work on constitutional AI—that's another way we provide values and principles for the AI system. On the autonomous risk, what we really want to do is understand what's going on inside the model and make sure we design it and can iterate on it so that it doesn't do these dangerous things we don't want it to do. On misuse risk, it's more about putting safeguards into the model so that people can't ask it to do dangerous things and we can monitor when people try to use it to do dangerous things.

总的来说，关于这一点有很多讨论，但如何监管 AI？公司能自我监管吗？

Generally speaking, there's been a lot of talk about this, but how can one regulate AI? Can companies self-regulate?

我思考的一个方式是：RSP，我描述的负责任扩展政策，可能是一个过程的开始。它代表了自愿的自我监管。去年九月我们实施了 RSP。此后，其他公司如谷歌和 OpenAI 也实施了类似的框架——它们有不同的名称，但运作方式大致相同。现在我们听说亚马逊、微软，甚至 Meta 据报道至少正在考虑类似的框架。我希望这个过程继续下去，让公司有时间尝试不同的自愿自我监管方式。通过公众压力和实验，会形成某种共识，关于哪些是不必要的，哪些是真正需要的。然后我想事情真正的发展方式是：一旦有了共识，一旦有了行业最佳实践，立法的角色可能就是介入并说：‘嘿，有件事 80% 的公司已经在做了，这是如何确保安全的共识。立法的工作就是强制那 20% 不做的公司去做，并强制公司如实报告他们的行为。’我不认为监管擅长提出一堆人们应该遵循的新概念。

One way I think about it is: the RSP, the Responsible Scaling Policy I described, is maybe the beginning of a process. It represents voluntary self-regulation. Last September we put in place our RSP. Since then, other companies like Google and OpenAI have put in place similar frameworks—they've given them different names, but they operate roughly the same way. Now we've heard Amazon, Microsoft, even Meta reportedly are at least considering similar frameworks. I would like it if that process continues, where we have some time for companies to experiment with different ways of voluntarily self-regulating. Some kind of consensus emerges from a mixture of public pressure and experimentation about what is unnecessary versus what is really needed. Then I would imagine the real way for things to go is: once there's some consensus, once there's industry best practices, probably the role for legislation is to look in and say, 'Hey, there's this thing that 80% of companies are already doing, it's a consensus for how to make it safe. The job of legislation is just to enforce that 20% who aren't doing it, and force companies to tell the truth about what they're doing.' I don't think regulation is good at coming up with a bunch of new concepts that people should follow.

那么你如何看待欧盟 AI 法案和加州安全法案？

So how do you view the EU AI Act and the California safety bill as well?

首先我要说，欧盟 AI 法案……

I should say first of all that the EU AI Act...

你知道，尽管法案通过了，但细节仍在制定中，很多事情都取决于细节。加州法案有一些结构很像 RSP。我认为在某个时候，类似这样的结构可能是好事。但我的担忧是，我们在这个过程里还太早了。我描述一个过程：先是一家公司有 RSP，然后很多公司有 RSP，接着行业共识形成。我的问题是：我们在这个过程里是不是太早了，不适合监管？也许监管应该是一系列步骤的最后一步。

You know, the details are still being worked out even though the Act was passed. A lot depends on the details. The California bill has some structures that are very much like the RSP. I think something resembling that structure could be a good thing at some point. But my concern is that we're very early in the process. I describe a process: first one company has an RSP, then many have RSPs, then industry consensus comes into place. My question is: are we too early in that process for regulation? Maybe regulation should be the last step of a series of steps.

过早监管有什么危险？

What's the danger of regulating too early?

我可以举我们自己的 RSP 经验为例。我们在 9 月写了一份 RSP，之后部署了一个模型，很快又要部署另一个。你看到很多我们在 RSP 中没有预料到的事情。比如，你可以在模型上运行各种 A/B 测试，这些测试对安全性有参考价值，但我们的 RSP 没有说明什么时候可以、什么时候不可以。所以我们正在更新 RSP 来处理我们从未想过的问题。在早期，这种灵活性很容易。如果你没有这种灵活性——如果你的 RSP 是由第三方编写的，而且你不能轻易修改——它可能产生一个既不能防范风险又非常繁琐的版本。然后人们可能会说：‘所有这些监管的东西，所有这些灾难性的东西，都是胡说八道，都是麻烦。’所以我不反对监管，但你必须谨慎地、按正确的顺序来做。

One thing I can say is look at our own experience with RSPs. We wrote an RSP in September, and since then we've deployed one model and are soon deploying another. You see so many things we didn't anticipate in the RSP. For example, there are various A/B tests you can run on your models that are informative about safety, and our RSP didn't speak to when those are okay and when not. So we're updating our RSP to handle issues we never even thought of. In the early days, that flexibility is easy. If you don't have that flexibility—if your RSP was written by a third party and you couldn't change it easily—it could create a version that doesn't protect against risks but is very onerous. Then people might say, 'All this regulation stuff, all this catastrophic stuff, it's all nonsense, it's all a pain.' So I'm not against it, but you have to do it delicately and in the right order.

但我们将 AI 融入到大国竞争之中——融入武器、汽车、医学研究，一切。当它成为世界力量平衡的一部分时，你怎么监管？

But we build AI into the race between superpowers—into weapons, cars, medical research, everything. How can you regulate when it's part of the power balance in the world?

这是不同的问题。一个是国内使用如何监管。这方面有历史可循。我打个比方，就像汽车和飞机的监管。在美国，这还算合理。每个人都明白有巨大的经济价值，每个人都明白这些东西很危险、会致人死亡，每个人都明白必须进行基本的安全测试。这经过多年演变，进展还算顺利，虽然不完美。所以对于国内监管，我们应该朝这个方向努力。事情发展很快，但我们应该尝试走完所有步骤。从国际角度看，这完全是另一个问题。这更多是关于国际逐底竞争，而不是监管。你如何处理这种逐底竞争？这本身就很难。一方面，我们不想不顾一切地尽快建造，尤其是在武器方面。另一方面，作为在挪威的美国公民，另一个民主国家，我非常担心如果专制政权在这个技术上领先，那将非常危险。

There are different questions. One is how to regulate use domestically. There's a history there. An analogy I would make is how cars and airplanes are regulated. In the US, that's been a reasonable story. Everyone understands there's huge economic value, everyone understands these things are dangerous and can kill people, and everyone understands you have to do basic safety testing. That has evolved over years and gone reasonably well, not perfect. So for domestic regulation, that's what we should aim for. Things are moving fast, but we should try to go through all the steps. From an international point of view, that's a completely different question. It's less about regulation and more about an international race to the bottom. How do you handle that race to the bottom? It's inherently difficult. On one hand, we don't want to recklessly build as fast as we can, particularly on the weapon side. On the other hand, as a citizen of the US here in Norway, another democracy, I'm very worried if autocratic regimes were to lead in this technology. That would be very dangerous.

他们现在落后多少？还是说他们并不落后？

How far behind are they now? Or are they behind?

很难说。由于一些限制措施，例如对俄罗斯和中国的芯片和设备出口限制，如果美国政府操作得当，这些国家可能落后两三年。这并没有给我们留下太多余地。

It's hard to say. With some restrictions put in place, for example on shipment of chips and equipment to Russia and China, if the US government plays its cards right, those countries could be kept behind maybe two or three years. That doesn't give us much margin.

说到民主国家，AI 会影响美国大选吗？

Talking about democracies, will AI impact the US election?

是的，我对此很担忧。Anthropic 实际上刚刚发布了一篇文章，说明我们正在做什么来应对选举干预。它可能如何干预？回顾 2016 年大选，有大量的人被雇佣来提供内容。我不知道那最终有多有效，很难衡量。但很多由付费水军做的事情现在可以由 AI 完成。这倒不是说你能够制造出人们必然相信的内容；而是你可以用大量低质量内容淹没信息生态系统，让人们难以相信真正真实的东西。

Yes, I am concerned about that. Anthropic actually just put out a post about what we're doing to counter election interference. How could it interfere? If we look back at the 2016 election, there were large numbers of people being paid to provide content. I don't know how effective that was, it's very hard to measure. But a lot of the things that were done by farms of paid people could now be done by AI. It's less that you could make content people necessarily believe; it's more that you could flood the information ecosystem with very low-quality content, making it hard for people to believe things that really are true.

这在印度或欧洲选举中发生了吗？今年真的会发生吗？

Did that happen in India or the European election? Is it really happening this year?

我们没有特别证据表明我们的模型被使用。我们禁止它们用于竞选活动，并监控使用情况。偶尔我们会关闭一些东西，但我不认为我们见过超大规模的操作。我只能就我们模型的使用情况发言，但我不认为我们见过超大规模的操作。

We don't have particular evidence of the use of our models. We've banned their use for electioneering and we monitor usage. Occasionally we shut things down, but I don't think we've ever seen a super large-scale operation. I can only speak for use of our models, but I don't think we've ever seen a super large-scale operation.

稍微换个话题，你提到你认为我们将在 2025-2026 年看到 AI 的一些极端积极影响。这些极端积极的事情是什么？

Changing topic slightly, you mentioned that you thought we were going to see some extreme positive effects of AI in 2025-2026. What are these extremely positive things?

再次回到那个类比：今天的模型像本科生。如果模型达到研究生水平或强专业人士水平，想想生物学和药物发现。想象一个模型，它像诺贝尔奖得主科学家或大型制药公司的药物发现主管一样强大。看看所有已经发明的东西：CRISPR，基因编辑能力；CAR-T 疗法，它治愈了某些癌症。可能还有几十个这样的发现。

Again, if we go back to the analogy: today's models are like undergraduates. If we get to the point where models are graduate-level or strong professional-level, think of biology and drug discovery. Think of a model that is as strong as a Nobel Prize-winning scientist or the head of drug discovery at a major pharmaceutical company. Look at all the things that have been invented: CRISPR, the ability to edit genes; CAR-T therapies which have cured certain kinds of cancers. There are probably dozens of discoveries like that.

如果我们有一百万个 AI 系统，它们和那些发明这些东西的科学家一样知识渊博、富有创造力，那么我认为这些发现的速度会真正加快。我们一些长期存在的疾病可能会得到解决甚至治愈。我不认为所有这些都会在 2025 年或 2026 年实现。最多，我认为能够启动解决所有这些问题的 AI 水平届时可能准备就绪。但应用它们、通过监管体系是另一个问题。

If we had a million copies of an AI system that are as knowledgeable and creative about the field as all those scientists that invented those things, then I think the rate of those discoveries could really proliferate. Some of our really longstanding diseases could be addressed or even cured. I don't think all of it will come to fruition in 2025 or 2026. At most, I think the caliber of AI capable of starting the process of addressing all those things could be ready then. It's another question of applying it all, putting it through the regulatory system.

你能为社会生产力做些什么？我想到虚拟助手，比如每个人都有一个参谋长。我有一个参谋长，但不是每个人都有。每个人都能有一个参谋长来帮助他们处理桌上的一切吗？如果每个人都有，你会怎么做？你能给生产力增长一个数字吗？

What could you do to productivity in society? I think of virtual assistants, like a chief of staff for everyone. I have a chief of staff, but not everyone has one. Could everyone have a chief of staff who helps them deal with everything that lands on their desk? If everybody had that, what would you do? Could you put a number on productivity gain?

我不是经济学家，我无法告诉你 X%。但如果我们看指数增长，AI 公司的收入似乎每年增长大约 10 倍。你可以想象在两三年内达到数千亿，甚至每年数万亿，这是任何公司都未曾达到的。对于社会生产力，这取决于这取代了多少已经完成的事情，还是做新的事情。在生物学方面，我们可能会做新的事情。如果你将人们的生产工作能力延长 10 年，那可能占整个经济的 16%。

I'm not an economist, I couldn't tell you X percent. But if we look at the exponential, revenues for AI companies seem to have been growing roughly 10x a year. You could imagine getting to hundreds of billions in two to three years, and even to trillions per year, which no company has reached. For productivity in society, it depends on how much this is replacing something already being done versus doing new things. With biology, we're probably going to be doing new things. If you extend people's productive ability to work by 10 years, that could be 16% of the whole economy.

你认为这是一个现实的目标吗？

Do you think that's a realistic target?

我懂一些生物学，我知道一些 AI 模型将如何发展。我无法确切告诉你会发生什么，但我可以讲一个可能的故事。所以 15%？我们什么时候能增加相当于 10 年的寿命？同样，这涉及太多未知数。如果我试图给出一个确切的数字，那听起来就像炒作。但我可以想象的是：从现在起两三年后，我们有能够做出那种发现的 AI 系统；五年后，这些发现实际上正在实现；再过五年，它们都通过了监管机构。所以我们谈论的是十年多一点。但我只是凭空猜测。我对药物发现或生物学了解不多。虽然我发明了 AI Scaling，但我对此也了解不多。我无法预测。

I know some biology, I know something about how the AI models are going to happen. I wouldn't be able to tell you exactly what would happen, but I can tell a story where it's possible. So 15%? When could we have added the equivalent of 10 years? Again, this involves so many unknowns. If I try to give an exact number, it's just going to sound like hype. But a thing I can imagine is: two to three years from now, we have AI systems capable of making that kind of discovery; five years from now, those discoveries are actually being made; and five years after that, it's all gone through the regulatory apparatus. So we're talking about a little over a decade. But I'm just pulling things out of my hat. I don't know that much about drug discovery or biology. Although I invented AI scaling, I don't know that much about that either. I can't predict it.

你比我们大多数人都更了解这些事情，但预测起来仍然困难。你有没有想过 AI 会对通胀产生什么影响？

You know more about these things than most of us, yet it is also hard to predict. Have you thought about what AI could do to inflation?

同样，我不是经济学家。用我有限的经济推理，如果我们有非常大的实际生产力增长，那往往会带来通缩而非通胀。你就能用更少的钱做更多的事；美元会更值钱。所以方向性上，这暗示着通胀放缓。但幅度如何？你比我更专业；也许我应该请你预测一下。

Again, I'm not an economist. Using my limited economic reasoning, if we had very large real productivity gains, that would tend to be deflationary rather than inflationary. You would be able to do more with less; the dollar would go further. So directionally, that suggests disinflation. But what kind of magnitude? You're more the expert than I am; maybe I should ask you to predict that.

你如何与超大规模云服务商合作？你们的一些股东，比如谷歌和亚马逊。

How do you work with the hyperscalers? Some of your shareholders like Google and Amazon.

它们被称为超大规模云服务商，是因为它们在估值上是超大规模的公司，而且它们也建造非常大的 AI 数据中心。这种关系是有道理的，因为我们有互补的输入：它们提供芯片和云服务，我们提供模型。这个模型可以卖给云上的客户。所以这是一个分层蛋糕，我们提供一些层，它们提供其他层。这些合作伙伴关系在多个层面上都有意义。同时，我们一直非常谨慎。我们有自己的公司价值观，自己的做事方式，所以我们尽量保持独立。我们做的一件事是与多个云服务商建立关系——我们同时与谷歌和亚马逊合作——这让我们有了灵活性，确保没有太多排他性，我们可以自由地在多个平台上部署我们的模型。

These are called hyperscalers because they are hyper-scale companies in terms of valuation, but also they make very large AI data centers. The relationship makes sense because we have complementary inputs: they provide the chips and the cloud, and we provide the model. That model can be sold to customers on the cloud. So there's a layered cake where we provide some layers and they provide the others. These partnerships make sense on multiple grounds. At the same time, we've always been very careful. We have our own values as a company, our own way of doing things, so we try to stay as independent as possible. One thing we've done is have relationships with multiple cloud providers—we work with both Google and Amazon—which has allowed us flexibility to ensure there isn't too much exclusivity, and we're free to deploy our models on multiple surfaces.

这些公司变得如此强大，会带来什么样的系统性风险？

The fact that these companies are becoming so incredibly powerful, what kind of systemic risk does that pose?

这可能比 AI 更广泛；它关系到我们生活的时代。历史上有些时代，强大的技术或经济力量倾向于集中资源。19 世纪可能也发生过同样的事情。所以我认为确保利益被所有人共享很重要。我想到的一件事是，在发展中世界的某些地区，比如撒哈拉以南非洲，AI 和语言模型的渗透非常少。我们如何将这些模型带到这些地区？我们如何帮助解决健康或教育等挑战？我完全同意我们生活在一个财富更加集中的时代，这是一个令人担忧的领域。我们应该尽我们所能找到制衡力量。

This is maybe broader than AI; it relates to the era we're living in. There are certain eras in history where a powerful technology or economic force tends to concentrate resources. Probably the same thing happened in the 19th century. So I think it's important to make sure the benefits are shared by all. One thing on my mind is there has been very little penetration of AI and language models in some parts of the developing world, like sub-Saharan Africa. How do we bring these models to those areas? How do we help with challenges like health or education? I definitely agree we're living in an era of more concentrated wealth, and that's an area of concern. We should do what we can to find countervailing forces.

这些公司现在变得比国家和政府更强大，风险是什么？

What's the risk that these companies are now becoming more powerful than countries and governments?

这就是我关于监管所说的。AI 是一项非常强大的技术，我们的民主政府确实……

This is what I said about regulation. AI is a very powerful technology, and our democratic governments do...

需要介入并制定一些基本的规则。这需要按正确的顺序进行，不能过于束缚，但我认为确实需要这样做。因为我们正在接近一个权力集中程度可能超过国家经济和国家政府的节点。我们不希望这种情况发生。归根结底，国家的人民和所有实体，包括在其中运作的公司，最终都必须对民主进程负责。没有其他办法。

Need to step in and set some basic rules of the road. It needs to be done in the right order, it can't be stifling, but I think it does need to be done. Because we're getting to a point where the concentration of power can be greater than that of national economies, national governments. We don't want that to happen. At the end of the day, all the people of the country and all entities, including companies that work in it, ultimately have to be accountable to democratic processes. There's no other way.

人工智能会扩大还是缩小富国和穷国之间的差距？

Will AI increase or decrease the difference between rich and poor countries?

我认为这取决于我们选择如何利用它。从前进的道路来看，我想说我们正在寻找不让差距扩大的方法。但这种情况正在发生吗？鉴于技术的部署方式，现在说还为时过早。我确实看到了与此相关的一些令人担忧的事情，我们正在努力应对。如果你看看这项技术的自然应用，最热切的客户——因为我们是硅谷公司——往往是其他技术先进的硅谷公司，它们也使用这项技术。所以存在一种闭环的危险：一家人工智能公司供应一家人工智能法律公司，后者供应一家人工智能生产力公司，再供应硅谷的其他公司。这是一个完全由最受过高等教育的人使用的封闭生态系统。我们如何打破这个循环？我们考虑过多种方法。我谈论生物学和健康的原因之一是，健康领域的创新，如果我们能很好地分配，可以惠及每个人。教育之类的事情可以有所帮助。另一个我非常兴奋的领域是使用人工智能提供日常政府服务。在美国，每次你与车管局、国税局、各种社会服务机构打交道时，人们几乎总是有糟糕的体验，这导致了对政府角色的愤世嫉俗。我希望我们能够现代化每个人都在使用的政府服务，这样它们才能真正提供世界各地人们需要的东西。我不得不说，在这个国家，我们很幸运，因为我们人口不多，而且高度数字化。你们在这方面可能比我们做得好得多。我是在根据我在美国的经历做出反应，我认为美国可以做得更好。

I think that depends on what we choose to do with it. The way you look at the path forward, I would say that we are looking for ways for it not to increase the gap. But is that happening? It's too early to say with how the technology is being deployed. I definitely see something related to it that's a little worrying to me, and we're trying to counter. If you look at the natural applications of the technology, the most eager customers that come to us—because we're a Silicon Valley company—are often other technologically forward Silicon Valley companies that also use the technology. So there's this danger of a kind of closed loop: an AI company supplies an AI legal company, which supplies an AI productivity company, which supplies some other company in Silicon Valley. It's all a closed ecosystem being used by the most highly educated people. How do we break out of that loop? We've thought about a number of ways. One of the reasons I talk about biology and health is that innovations in health, assuming we distribute them well, can apply to everyone. Things like education can help here. Another area I'm very excited about is use of AI for provision of everyday government services. In the US, every time you interact with the DMV, the IRS, various social services, people almost always have a bad experience, and it drives cynicism about the role of government. I would love it if we can modernize government services that everyone uses so they can actually deliver what people across the world need. I have to say that in this country, we are fortunate in that we are not so many people and we are heavily digitalized. You are probably much better than we are at this. I'm reacting to my experience in the United States, which I think could be better.

那么总的来说，你怎么看？十年后，贫富差距会更大还是更小？

So net net, what do you think? In 10 years' time, will the gap between rich and poor be bigger or smaller?

我只能说，如果我们以正确的方式处理——我听到你说的——正确的方式，我们可以缩小差距。

I just have to say, if we handle this the right way—I hear what you say—the right way, we can narrow the gap.

我听到你说的了。你认为会发生什么？

I hear what you're saying. What do you think will happen?

我不知道我认为会发生什么。我知道如果我们对此不极其深思熟虑，不极其审慎，那么是的，差距会扩大。

I don't know what I think will happen. I know that if we are not extremely thoughtful about this, if we're not extremely deliberate about it, then yes, it will increase the gap.

谁会从人工智能中赚到最多的钱？会是芯片制造商，还是你们，还是规模扩张者，还是所有的消费者或公司？

Who will make the most money on AI? Will it be the chip manufacturers, or you guys, or the scalers, or all the consumers or companies?

我无聊的答案是，我认为它会分布在所有人之间，而且蛋糕会非常大，以至于在某种程度上甚至可能无关紧要。当然现在，芯片公司赚的钱最多。我认为这是因为模型训练先于模型部署，而部署先于收入。所以我的看法是：芯片公司的估值是领先指标，人工智能公司的估值可能是当前指标，而许多下游事物的估值是滞后指标。但浪潮会波及每个人。

My boring answer is that I think it's going to be distributed among all of them, and the pie is going to be so large that in some ways it may not even matter. Certainly right now, the chip companies are making the most money. I think that's because training of models comes before deployment of models, which comes before revenue. So the way I think about it: the valuation of the chip companies is a leading indicator, the valuation of the AI companies is maybe a present indicator, and the valuation of lots of things downstream is a lagging indicator. But the wave is going to reach everyone.

当你看到英伟达的市值时，那是一个指标。我的意思是，你把它乘以多少才能找到人工智能的潜在影响？那是三万亿，对吧？几乎是这个基金规模的两倍，而这是世界上最大的主权财富基金。

When you look at the market cap of Nvidia, for instance, that's an indicator. I mean, what do you multiply that by to find the potential impact of AI? That's three trillion dollars, right? Nearly twice the size of this fund, which is the largest sovereign wealth fund in the world.

非常抽象和概念性地讲，是什么驱动了这一点？可能是预期的需求。人们正在建造非常大的人工智能集群；这些集群为英伟达带来了大量收入。像我们这样的公司大概在为这些集群付费，因为他们认为用它们构建的模型会产生大量收入，但这些收入尚未出现。所以到目前为止我们看到的只是人们想买很多芯片。当然，有可能这一切都会失败——模型并没有那么强大，像 Anthropic 和其他公司表现不如预期，因为模型没有持续变好。这种情况总是可能发生。但这不是我的赌注；我不认为会发生这种情况。我认为会发生的是这些模型将产生大量收入，然后对芯片的需求会更大。英伟达的价值会上升，人工智能公司的价值会上升，所有这些下游公司——这就是我领导这家公司所押注的看涨情景。但我不确定；也可能走向反面。我认为没人知道。

Speaking very abstractly and conceptually, what's that driven by? Probably anticipated demand. People are building very large AI clusters; those clusters involve lots of revenue for NVIDIA. Presumably companies like us are paying for those clusters because they think the models they build with them will generate lots of revenue, but that revenue is not present yet. So what we're seeing so far is just that people want to buy a lot of chips. Of course, it's possible that all of this will be a bust—the models don't turn out to be that powerful, companies like Anthropic and others don't do as well as expected because models don't keep getting better. That always could happen. That's not my bet; that's not what I think is going to happen. What I think is going to happen is that these models are going to produce a great deal of revenue, and then there's going to be even more demand for chips. Nvidia's value will go up, the AI companies' value will go up, all these downstream companies—that's the bullish scenario that I'm betting on by leading this company. But I'm not sure; it could go the other way. I don't think anyone knows.

目前最大的制约因素是什么？是芯片、人才、算法还是电力？

Where is the biggest constraint just now? Is it in chips, talent, algorithms, electricity?

我们正在处理的主要瓶颈是数据。但正如我在别处所说，我们和其他公司正在非常努力地研究合成数据，我认为这个瓶颈将被解除。

My big bottleneck we're dealing with is data. But as I've said elsewhere, we and other companies are working very hard on synthetic data, and I think that bottleneck is going to be lifted.

那么数据——澄清一下——就是你输入模型的信息？

So data—just to get that straight—that's just information you feed into your models?

是的，那是输入模型的信息。但我们在合成数据方面越来越擅长。

Yes, that's information fed into the models. But we're getting increasingly good at synthesizing the data.

告诉我，什么是合成数据？

Tell me, what is synthetic data?

我喜欢举的例子是七年前，作为谷歌一部分的 DeepMind 制作了 AlphaGo 模型，它能够击败围棋世界冠军。有一个版本叫 AlphaGo Zero，它没有经过任何人类下棋的训练。它所做的就是模型与自己下棋，从中生成自己的数据，学习，然后变得超人类。这就是合成数据：由模型自身生成的数据，而不是来自现实世界。

The example I like to give is seven years ago, DeepMind as part of Google produced the AlphaGo model, which was able to beat the world champion in Go. There was a version called AlphaGo Zero that was not trained on any humans playing Go. All it did was the model played Go against itself, and from that it generated its own data, learned, and became superhuman. That's synthetic data: data generated by the model itself rather than from the real world.

与自己对抗很长时间，基本上永远。仅凭围棋的微小规则，模型相互对弈、相互推动，利用那条规则，它们能变得越来越好，直到超越任何人类。所以你可以认为这些模型是在由其他模型生成的合成数据上训练的，借助了围棋规则的逻辑结构。我认为语言模型也可以做类似的事情。

Against itself for a long time, basically forever. With just the tiny rules of Go and the models playing against each other, pushing against each other using that rule, they were able to get better and better to the level where they were better than any human. So you can think of those models as having been trained on synthetic data created by other models with the help of the logical structure of the rules of Go. I think there are things analogous to that that can be done for language models.

你认为 AI 将如何影响地缘政治？

How do you think AI will affect geopolitics?

我认为这是个重大问题。我的看法是，如果我们达到 AI 系统在广泛任务上超越最优秀专业人员的水平，那么军事和情报等任务也将位列其中。我们不应天真；每个人都会试图部署这些系统。我认为我们应该尽可能创造合作与约束，但在许多情况下这不可能。当不可能时，我站在自由世界民主国家一边。我希望确保未来是民主的，尽可能多的世界是民主的，并且民主国家在世界舞台上拥有领先和优势。强大 AI 加专制政权的想法让我恐惧，我不希望它发生。

I think that's a big one. My view is that if we get to the level of AI systems that are better than the best professionals at a wide range of tasks, then tasks like military and intelligence are going to be among those tasks. We shouldn't be naive; everyone is going to try to deploy those. I think we should try to create cooperation and restraints where we can, but in many cases that won't be possible. And when it isn't possible, I'm on the side of democracies in the Free World. I want to make sure that the future is democratic, that as much as possible of the world is democratic, and that democracies have a lead and an advantage on the world stage. The idea of powerful AI plus autocracies terrifies me, and I don't want it to happen.

每个国家都应该有自己的语言模型吗？

Should each country have its own language model?

这确实取决于你的目标。从国家安全角度看，每个国家拥有语言模型可能是有意义的。一个可能可行的想法是设想某种民主联盟或合作，民主国家共同努力提供相互安全、保护彼此、保护民主进程的完整性。也许它们集中资源制造极少数的超大型语言模型是有意义的。但去中心化也可能有价值。我不强烈认为哪个更好。

It really depends on what you're aiming to do. It may make sense from a national security perspective for every country to have language models. An idea that might work is imagining some kind of democratic coalition or cooperation in which democratic countries work together to provide for their mutual security, to protect each other, to protect the integrity of their democratic processes. Maybe it makes sense for them all to pull their resources and make a very small number of very large language models. But then there may also be value in decentralization. I don't have a strong opinion on which of those is better.

美国控制 AI 是国家安全问题吗？欧洲应该担心吗？

Is it a national security issue that the US controls AI? Should Europe be worried about this?

每个国家都必须担心自己的安全，即使与盟友分开。我认为这更像是各国政府的问题。我可能会这样想——这是一个有争议的类比——有点像核武器。有些国家，即使是盟友，也觉得需要拥有自己的核武器，例如法国。其他国家则说，不，我们相信我们受到美国、英国和法国的保护。我认为这些更强大的模型可能有些类似。而且我认为，民主世界内部有多少个模型并不那么重要，重要的是民主世界相对于专制政权处于强势地位。

Each country has to worry about its own security, even separately from its allies. I think that's more of a question for individual governments. I would think of it probably—this is a provocative analogy—a little like nuclear weapons. Some countries, even though they're allies, feel the need to have their own nuclear weapons, for example France. Other countries say no, we trust that we're being protected by the US, the UK, and France. I think it may be somewhat similar with these more powerful models. And I think it's less important how many of them exist within the democratic world as that the democratic world is in a strong position relative to autocracies.

你谈到合作和伙伴。你们 Anthropic 的人真的互相喜欢吗？

You talk about cooperation and partners. Do you guys at Anthropic actually like each other?

我们进行过多次合作。很早以前我在 OpenAI 时，我推动了最初的基于人类反馈的强化学习论文，这被认为是安全的工作，最终成为 DeepMind 和 OpenAI 之间的合作。我们还在前沿模型论坛等组织中合作。话虽如此，老实说，我不认为这个领域的每家公司都同样重视安全和责任问题。但与其指责，不如系统性地思考。向上竞争的理念：让我们设定标准，而不是指责做坏事的人。让我们做好事，很多时候人们就会跟随。就在几周前，我们提出了一种可解释性创新，能够看到模型内部。几周后，我们从 OpenAI 那里看到了类似的东西。我们内部看到其他公司提高了对它的优先级。所以很多时候你只需要做好事，就能激励别人也做好事。如果你做了很多这样的事，设定了这些标准，如果它们成为行业标准，然后有人不遵守，那就有问题了，那时你才可以谈论指责。

We've done a number of collaborations. Very early on when I was at OpenAI, I drove the original RL from Human Feedback paper, which was considered safe work, and this ended up being a collaboration between DeepMind and OpenAI. And we've worked together in organizations like the Frontier Model Forum to collaborate with each other. That said, I'll be honest, I don't think every company in this space takes issues of safety and responsibility equally seriously. But instead of pointing fingers, let's think systemically. The idea of a race to the top: let's set standards instead of pointing fingers at people doing something bad. Let's do something good, and then a lot of the time people just follow along. We invented an interpretability idea just a few weeks ago, being able to see inside the model. A few weeks later, we got similar things from OpenAI. We've seen internally other companies increase their prioritization on it. So a lot of the time you can just do something good and inspire others to do something good. Now if you've done a lot of that, set these standards, if they're industry standards, and then there's someone who's not complying with them, there's something that's really wrong, then you can talk about pointing fingers.

我们花几分钟谈谈文化。你们 Anthropic 有多少人？

Let's spend a few minutes talking about culture. How many people are you at Anthropic?

几周前我们大约有 600 人。我一直在休假，所以现在可能更高了。

We are about 600 as of a couple weeks ago. I've been on vacation, so it may be even higher now.

文化是什么样的？

What's the culture like?

我会描述几个元素。一个元素是我所说的“做简单有效的事”。Anthropic 的许多人曾是物理学家，因为我自己有那个背景，我的几位联合创始人也有，其中一位在共同创立 Anthropic 之前实际上是物理学教授。物理学家寻找事物的简单解释。所以我们文化的一个元素是不要把事情搞得太复杂。很多学术机器学习研究倾向于过度复杂化。我们追求尽可能简单且有效的方法。我们在工程上也有同样的观点，在安全、伦理、可解释性、我们的宪法 AI 方法上也是如此。它们都是极其简单的想法，我们只是尽力推动它们。甚至这个向上竞争的理念：一两句话就能说清楚。它并不复杂。你不需要一篇 100 页的论文来讨论它。这是一个简单的策略：做好事，并鼓励他人效仿。

I would describe a few elements. One element is what I describe as 'do the stupid simple thing that works.' A number of folks at Anthropic are ex-physicists, because I myself had that background and a couple of my co-founders had that background, including one person who was actually a professor of physics before co-founding Anthropic. Physicists look for simple explanations of things. So one element of our culture is don't do something overcomplicated. A lot of academic ML research tends to overcomplicate things. We go for the simplest thing possible that works. We have the same view in engineering, and again on things like safety and ethics, on interpretability, on our constitutional AI methods. They're all incredibly simple ideas that we just try and push as far as we can. Even this race to the top thing: you can say it in a sentence or two. It's not complicated. You don't need a 100-page paper to talk about it. It's a simple strategy: do good things and try to encourage others to follow.

当你在 3 年内招聘 600 人时，你如何确信他们是好人？

When you hire 600 people in 3 years, how can you be confident that they are good?

坦率地说，我认为 AI 行业的一个挑战是一切发展太快。在正常的初创公司，事情……

I think candidly, one challenge of the AI industry is how fast everything moves. In a normal startup, things...

每年可能增长 1.5 倍或 2 倍。我们认识到这个领域发展如此之快，需要更快的增长才能满足市场需求，最终导致比平时更快的增长。一开始我确实担心过这个问题。我说：‘天哪，我们面临这个困境，该怎么应对？’总体而言，我对我们迄今为止的处理能力感到惊喜——我们能够很好地扩展招聘流程，而且我觉得每个人都既有技术天赋和知识，又普遍善良且有同情心，我认为这与招聘技术人才同样重要。

Might grow 1.5x or 2x a year. We recognize that in this field things move so fast that faster growth is required in order to meet the needs of the market, and that ends up entailing faster growth than usual. I was actually worried about this at the beginning. I said, 'Oh my God, we have this dilemma, how do we deal with it?' I have generally been positively surprised at how well we've been able to handle it so far, how good we've been able to scale hiring processes, how much I feel everyone is both technically talented and knowledgeable, and just generally kind and compassionate people, which I think are equally important as hiring technical talent.

那你寻找什么样的人？现在我坐在这里，你正在面试我这个职位。你寻找什么？

So what do you look for? Here I'm sitting, you are interviewing me now for that position. What do you look for?

我们看重的是愿意做简单有效的事情。我们看重才华。我们不一定看人工智能领域的经验年限。我们雇佣的许多人都是物理学家或其他自然科学家，他们可能只做过一个月的人工智能，只是自己做过一个项目。所以我们看重学习能力、好奇心、快速抓住问题核心的能力。在价值观方面，我们看重从公共利益角度思考。我们并不对 Anthropic 的正确政策或世界上该做什么有特定看法，而是希望随着公司扩张保持一种精神，这随着公司变大越来越难。我们希望人们有公共精神，一方面理解 Anthropic 需要成为商业实体才能接近中心并产生影响，另一方面也理解长期目标是公共利益和社会影响。

We look for willingness to do the simple thing that works. We look for talent. We don't necessarily look at years of experience in the AI field. A number of folks we hire are physicists or other natural scientists who have maybe only been doing AI for a month or so, only have been doing a project on their own. So we look for ability to learn, curiosity, ability to quickly get to the heart of the matter. In terms of values, we look for thinking in terms of the public benefit. It's less that we have particular opinions on what the right policies for Anthropic are or what the right things to do in the world. It's more that we want to carry a spirit as we scale the company, and it gets increasingly hard as the company gets bigger. We want people who carry some amount of public spirit, who understand on one hand that Anthropic needs to be a commercial entity to be close enough to the center of this to have an impact, but also understand that in the long run we're aiming for this public benefit, this societal impact.

招聘时，你是否觉得资金有限？

When you hire, do you feel you have a limited amount of money?

算力几乎是我们所有的支出。我不会给出确切数字，但超过 80%。所以薪水其实不重要。在薪酬方面，我们更考虑公平。我们想做公平、符合市场、善待员工的事情。我们不太考虑花了多少钱，因为算力是最大支出。更重要的是如何创造一个每个人都感到公平、同工同酬的环境。

Compute is almost all of our expenses. I won't give an exact number, but it's more than 80%. So salaries don't really matter. In terms of paying people, we think more about what is fair. We want to do something that's fair, that meets the market, that treats people well. It's less of a consideration of how much money we are spending because compute is the biggest expenditure. It's more how we can create a place where everyone feels they're treated fairly and people who do equal work get equal pay.

你和这些才华横溢的人、天才，甚至可能有些难搞的人一起工作。管理或领导他们的最佳方式是什么？

Now you work with all these brilliant minds and kind of geniuses, and perhaps even some prima donnas. What's the best way to manage them or lead them?

我觉得他们不能被管理，所以你需要领导。最重要的原则之一是让创造力自然发生。如果过于自上而下，人们就很难充分发挥创造力。看看过去十年机器学习领域的许多重大创新，比如 Transformer 的发明，谷歌没有人下令把它作为一个项目。那是一种去中心化的努力。与此同时，你必须做出产品，每个人都必须合作完成一件事。我认为，需要新想法和需要每个人都为同一件事做贡献之间的创造性张力，正是魔力所在——找到正确的组合，从而两全其美。

I guess they can't be managed, so you need to lead. One of the most important principles is letting creativity happen. If things are too top-down, then it's hard for people to be fully creative. If you look at a lot of the big innovations in the ML field over the last 10 years, like the invention of the Transformer, no one at Google ordered it as a project. It was a decentralized effort. At the same time, you have to make a product and everyone has to work together to make a single thing. I think that creative tension between needing new ideas but needing everyone to contribute to one thing is where the magic is—finding the right combination so that you can get the best of both worlds.

你和你的姐姐一起经营这家公司，对吗？

You run this company together with your sister, right?

是的。我们都在 OpenAI 工作过，然后一起创立了 Anthropic。这真的很棒。真正的分工是她负责大部分日常运营公司的事情：管理员工、规划公司结构、确保我们有 CFO、首席产品官、确保薪酬设置合理、确保文化良好。我更多考虑想法和战略。每隔几周我会给公司做一次演讲，基本上是愿景演讲，我说一些我们在战略上思考的事情。这些不是决定，而是领导层在想什么：我们认为明年什么会很重要，商业、研究和公益方面的发展方向。

Yes. We both worked at OpenAI and then we both founded Anthropic together. It's really great. The real division of labor is she does most of the things you would describe as running the company day-to-day: managing people, figuring out the structure of the company, making sure we have a CFO, a chief product officer, making sure comp is set up in a reasonable way, making sure the culture is good. I think more in terms of ideas and strategy. Every couple weeks I'll give a talk to the company, basically a vision talk where I say here's some things we're thinking about strategically. These aren't decisions, this is a picture of what leadership is thinking about: what do we think is going to be big in the next year, where do we think things are going on the commercial side, the research side, the public benefit side.

她比你小还是大？

Is she younger or older than you?

她比我小四岁。

She is four years younger than me.

她比你聪明吗？

Is she cleverer than you?

我们在不同方面都非常有才华。

We are both extremely skilled in different ways.

你父母是做什么的？

What did your parents do?

我父亲已故，他以前是个工匠。我母亲退休了，她曾是公共图书馆的项目经理。

My father is deceased. He was previously a craftsman. My mother is retired. She was a project manager for public libraries.

你是如何被抚养长大的？

How were you raised?

非常注重社会责任和帮助世界。这对我的父母来说很重要。他们真的思考如何让事情变得更好，如何让出生在幸运位置的人履行责任，并将其传递给不那么幸运的人。你可以从公司的公益导向中看到这一点。

There was a big focus on social responsibility and helping the world. That was a big thing for my parents. They really thought about how to make things better, how people who have been born in a fortunate position reflect their responsibilities and deliver them to those who are less fortunate. You can kind of see that in the public benefit orientation of the company.

那么 14 岁的达里奥在做什么？

So the 14-year-old Dario, what was he up to?

我非常喜欢数学和科学。我参加数学竞赛之类的。但我也在思考如何运用这些技能发明一些能帮助人的东西。

I was really into math and science. I did math competitions and all of that. But I was also thinking about how I could apply those skills to invent something that would help people.

你有朋友吗？

Did you have any friends?

比我想要的少。我有点内向。但当时认识的一些人我现在仍然认识。

Less than I would have liked. I was a little bit introverted. But there were people who I knew back then who I still know now.

Anthropic 是书呆子的复仇吗？

Is Anthropic like revenge of the nerds?

我不太会这么说。我不太愿意把不同群体对立起来。不同类型的人擅长不同的事情。我们有一个完整的销售团队。

I wouldn't really put it in those terms. I'm kind of reluctant to set different groups against each other. People are different kinds of people are good at different things. We have a whole sales team.

他们擅长的东西跟我完全不同。当然，我是 CEO，所以也得学点销售，但技能确实千差万别。在公司里你会意识到，不同类型的人拥有截然不同的技能——你会认识到各种技能的价值，包括那些你自己完全不具备的能力。

They're good at a whole different set of things than I am. Of course, I'm the CEO, so I have to learn how to do some sales as well, but there are just very different skills. One of the things you realize in a company is that different kinds of people with very different kinds of skills—you recognize the value of a very wide range of skills, including ones that you have no ability in yourself.

那现在是什么在驱动你？

So what drives you now?

我认为我们正处于 AI 领域一个非常特殊的时期。我曾说过 2025 或 2026 年事情可能会变得多么疯狂。我觉得把这件事做对很重要。运营 Anthropic 只是其中一小部分；还有其他公司，有些比我们更大或更知名。一方面，我们只能扮演一个小角色，但考虑到这对经济和人类的重要性，我认为我们有一个重要的机会来确保事情顺利发展。结果有很多种可能性，而我们有能力影响它。日常我们要发展业务、招人、卖产品——这很重要，这样公司才有影响力。但从长远来看，驱动我的是抓住那些可能性并推动事情向好的方向发展的愿望。

I think we're in a very special time in the AI world. I've said things about how crazy things could be in 2025 or 2026. I think it's important to get that right. Running Anthropic is only one small piece of that; there are other companies, some bigger or better known than we are. On one hand, we have only a small part to play, but given the importance of what's happening for the economy and for humanity, I think we have an important opportunity to make sure things go well. There's a lot of variance in how things could go, and I think we have the ability to affect that. Day to day, we have to grow the business, hire people, sell products—that's important so the company is relevant. But in the long run, what drives me is the desire to capture some of that variance and push things in a good direction.

你怎么放松？

How do you relax?

我现在在挪威，但这可不是放松。我是从意大利的假期过来的。每年我都会休几周假来放松和思考更深层的概念。我每天游泳。实际上，我和我妹妹还在玩电子游戏，就像高中时那样。我都 40 多岁了，她也是……我们最近买了新的《最终幻想》游戏。我们高中时玩《最终幻想》，那是 90 年代的游戏，最近出了重制版。所以我们开始玩新版本，有着 20 年 GPU 进步带来的华丽画面。我们注意到，哇，我们高中时做这个，现在我们在运营这家公司。

I'm in Norway now, but this is not relaxing. I came here from my vacation in Italy. Every year I take a few weeks off to relax and think about deeper concepts. I go swimming every day. Actually, my sister and I still play video games, like we did since high school. I'm over 40 and she's... well, we recently got the new Final Fantasy game. We played Final Fantasy in high school, it was a game made in the 90s, and they recently made a remake. So we started playing the new version with all the fancy graphics from 20 years of progress in GPUs. We noticed it was like, wow, we used to do this in high school, now we're running this company.

很高兴听到有些人永远不会长大。

I'm glad to hear that some people never grow up.

我觉得我们在某些方面没长大。希望在其他方面长大了。

I don't think we've grown up in a certain way. Hopefully we have in others.

你对年轻人熟悉这些新 AI 技术有什么建议？

What kind of advice do you have for young people to gain familiarity with these new AI technologies?

我不会给出那种老生常谈，说确切知道哪些工作会热门哪些不会。我觉得我们不知道，而且 AI 可能会触及每个领域。但可以肯定的是，人类在使用这些技术并与它们协作方面会有一席之地，至少要在公共辩论中理解它们。另一件我想说的，而且这已经很重要但会变得更加重要，就是对信息的怀疑能力。随着 AI 生成越来越多的信息和内容，辨别这些信息将变得更加重要和必要。我希望我们会有 AI 系统帮助我们筛选一切、理解世界，这样我们就不那么容易受到攻击。但归根结底，这必须来自你自己——你必须有一些基本的渴望、好奇心和辨别力。所以我认为培养这一点很重要。

I'm not going to offer some bromide about knowing exactly which jobs will be big and which won't. I think we don't know that, and also AI might touch every area. But it's safe to say there will be a role for humans in using these technologies and working alongside them, at the very least understanding them in the public debate. The other thing I would say, and this is already important but will become more so, is the faculty of skepticism about information. As AI generates more information and content, being discerning about that information will become more important and necessary. I hope we'll have AI systems that help us sift through everything and understand the world, so we're less vulnerable to attacks. But at the end of the day, it has to come from you—you have to have some basic desire, curiosity, and discernment. So I think developing that is important.

这真是很好的建议。非常感谢，这次访谈非常愉快。祝你一切顺利，回意大利好好休息，继续深入思考概念。

Well, that's really great advice. Big thanks, this has been a true blast. I wish you all the best, get back to Italy, get some more rest, and do some more deep conceptual thinking.

非常感谢你邀请我上播客。

Thank you so much for having me on the podcast.