Anthropic 如何打造全球第一的大模型：深入 Claude 平台

大家好，今天我们尝试点新东西。我们要启动对 Anthropic 的 IPO 报道。他们今天刚刚提交了 S-1 文件，所以我们会推出一系列播客，一直做到 IPO 之前，预计 IPO 会在两周后，估值 2 万亿美元。这是史上增长最快的公司之一，据报道其累计融资超过 1000 亿美元，但累计烧钱不到 100 亿美元。所以这里有很多关于效率的故事可以深挖，我觉得这正是我们想报道的：Anthropic 如何在推出全球最强模型的同时，做到极其高效，并打造出一家了不起的企业。

Hi, today we're trying something new. We're launching coverage of Anthropic for the IPO. They just filed their S-1 as of today, and so we're releasing a series of podcasts leading up to the IPO, which is expected to be in two weeks for $2 trillion. It's one of the fastest growing companies of all time, with a reported lifetime cash raise of over $100 billion but a lifetime burn of less than $10 billion. So there are a lot of efficiency stories to dive into there, and I think that's something we want to cover: how Anthropic is simultaneously shipping the strongest models in the world while also doing it very, very efficiently and building an incredible business.

所以我摘出了一些内容——很多听众知道，如果你们一直在听这档播客，我过去做过金融分析师——我从 S-1 里摘出了一些陈述，觉得它们能为今天的播客提供有趣的背景。我们只是把这当作一次新尝试，因为我觉得有些听众希望和我们建立那种直接面对镜头的个人关系，而不是像过去四年那样一直和嘉宾对话。

So I've pulled out — a lot of you know, if you've been listening to the podcast, that I have a former life as a financial analyst — and I pulled out some statements from the S-1 which I thought would be interesting to give context to today's podcast. And we're just trying this as a new thing because I feel like some of you want that personal relationship with us talking directly to camera rather than talking to our guests, which we've done for 4 years.

我想强调的第一点是，按人数算，Anthropic 是一家效率极高的公司，这也许显而易见，因为它是家 AI 公司。显然他们把 AI 用得很好。但我认为，他们在过去一年里作为一家公司改变的程度，并没有被充分理解。过去 12 个月的每员工营收现在是 660 万美元，而一年前是 140 万美元。所以他们的每员工营收增长了近五倍，我觉得这值得我们深入探讨，因为我认为任何企业主、任何工程师，都想提高每员工营收，不仅是为了提高盈利能力，也是为了能给你的员工发更多钱。

So the first thing I want to highlight is that Anthropic is an extraordinarily efficient company by human count, and that's maybe obvious because it's an AI company. So obviously they're using AI very well. But I think the degree to which they have changed as a company over the past one year is not well understood. So the trailing 12-month revenue per employee is now $6.6 million, as compared to one year ago when it was $1.4 million. So they've grown almost five times the revenue per employee, and I think that's something we need to dive into more, because I think any business owner, any engineer, you want to increase your revenue per employee not just to increase profitability but also to increase the amount you can pay your people.

这显然伴随着每美元算力所产生营收的大幅提升。值得注意的是，基本上在 2025 年第二季度，Anthropic 每赚一美元营收大约要亏掉半美元，而现在每花一美元成本能赚两美元。我觉得这是个有趣的类比，说明过去一年 Anthropic 的营收和业务前景出现了多么巨大的拐点。今年对 Anthropic 来说极其关键，必须执行到位，而它做到了。我想我们会更深入地探讨这一点，尽可能从播客的外部视角把这个故事讲出来。

This is obviously tracked by a significant increase in revenue per dollar of compute. It's notable that basically in Q2 of 2025, Anthropic was basically losing half a dollar for every dollar of revenue, and now it's making $2 for every $1 in cost. And I think that's an interesting analogy of how enormously huge this inflection in Anthropic's revenues and business outlook has been over the past one year. Like, this year was extremely critical for Anthropic to execute well, and it has done that, and I think we're basically going to drive deeper into this and tell that story as much as we can from the podcast's external point of view.

这一整体增长带来的一个结果是，Anthropic 刚刚公布了首个盈利季度——不是调整后利润，也就是剔除所有成本项之后的利润。不，这是按公认会计原则（GAAP）计算的利润，达到 19%。这非常惊人，而且又是相对一年前的彻底反转——不止是反转。一年前，它每收入一美元就要亏损 129%。

And the one consequence of this entire increase is that Anthropic has just reported its first profitable quarter — like, not adjusted profit, which is profit after we take out all the things that cost us money. No, this is generally accepted accounting principles, GAAP, profit of 19%. And that is enormous, again completely flipping from one year ago — more than flipping. Like, one year ago it was losing 129% of every revenue dollar that came in.

我认为解释这种非凡效率的主要归因——如你所知，Claude 喜欢循环，Boris 喜欢循环，Dario 喜欢循环——Anthropic 内部有两个循环。第一个是改进循环。基本上就是这样一个循环：你有了 Claude；Claude 带来更好的模型；更好的模型带来更多用例；更多用例产生更多反馈，进而带来更好的 Claude。然后还有投资循环。来自采用范围扩大的营收带来更高增长；更高增长带来更多投资；更多投资带来更强能力，进而带来更好的 Claude。所以这两个循环相互推动，形成一个无限循环，按 Anthropic 的说法，会一直推动效率提升到时间的尽头。

So the primary attribution I think that explains this extraordinary efficiency — as you know, Claude loves loops, Boris loves loops, Dario loves loops — and there are two loops going on in Anthropic. The first is the improvement loop. Basically, there is the cycle of you have Claude; Claude leads to better models; better models lead to more use cases; more use cases generate more feedback, and that leads to a better Claude. And then there's also the investment loop. So revenue from expanding adoption leads to higher growth; higher growth leads to more investment; more investment leads to more capabilities, and that leads to a better Claude. So these two loops basically spin together and form an infinite loop that is driving the increase in efficiency until the end of time, according to Anthropic.

所以我想这就是我们从整体效率故事过渡到他们如何执行的地方。今天我们会重点谈 Claude 平台，因为这是他们在 Anthropic S-1 里称为“Claude 的扩散”的三件事之一。他们称之为 Claude 平台，或者说全面的智能体式开发环境。他们让客户直接访问这些原语，以便构建、部署和扩展自己的应用，在自家产品和工作流中利用 Claude 的能力。

So I think that is our transition from the overall efficiency story to how they execute it. And today we're going to focus a lot on the Claude platform, because that is primarily one of the three things that they call the diffusion of Claude in the Anthropic S-1. So they call this Claude Platform, or comprehensive agentic development environment. They provide customers with direct access to the primitives to build, deploy, and scale their own applications that leverage Claude's capabilities within their own products and workflows.

我觉得这档播客讨论中经常出现的一个概念是“吃自己的狗粮”，也就是 Claude 是用同样开放给你的 Claude 平台构建的。Claude 平台里还有一件有趣的事被披露出来：在滚动基础上营收达到 10 万美元或以上的客户中，超过 50% 使用 Claude 平台。所以这体现了 Anthropic 的采用度提升，以及它不仅是模型实验室，还正在成为超大规模云厂商和云平台——我觉得这也许就在名字里，Claude 平台本身就内含着这一点——但它正从理论走向实际，这正是我们在播客里关注的内容。

I think one thing that comes up a lot in the discussion in this podcast is the concept of dogfooding, where Claude is built with the same Claude platform that is available to you. So something that's also interesting in the Claude platform — what was disclosed is that over 50% of their customers with $100,000 or more revenue on a trailing basis use Claude platform. And so that is an increase and an adoption and emergence of Anthropic not only as a model lab but also as a hyperscaler and a cloud platform — which I think maybe is in the name, inherent in a Claude platform and a Claude platform — but is going from theory to actual practice, and this is something that we're focusing on in the podcast.

我还想强调的另一件事，之前也谈过，就是 Claude Code、Claude Co-work——它们都使用 Claude 平台暴露的同一套原语。我觉得这非常有意思。最终看来，Anthropic 在 IPO 中真正想讲的是从聊天机器人到副驾驶，再到虚拟协作者。他们用这个词代替智能体。也许这样更友好，但它包含的很多东西正是你对智能体的期待：从长时间运行的自主工作，到判断力、自我纠错和迭代，以及工具访问。所以是拥有一个目标，与你并肩工作，而不是只做一个应用或一项任务。

The other thing that I also really try to emphasize, which I talked about before, is Claude Code, Claude Co-work — they all use the same primitives that Claude platform exposes. So I think that's really interesting. Ultimately it looks like Anthropic is really positioning their IPO to talk about going from chatbots to co-pilots to virtual collaborators. They use this term instead of agents. Maybe it's more friendly, but it's a lot of the same things that you look for from agents: from long-running autonomous work to judgment, self-correction and iteration, and tool access. So owning a goal and working alongside of you instead of working — instead of one app or task.

这就是我想为今天这期播客提供的背景。这期录制于 Opus 5.5 发布之时，我想说，在我们接触过的所有高品味测试者中，压倒性地认为 Opus 5.5 是目前世界上最好的模型。而 Anthropic 甚至还压着 Fable 5.5 没放，可能未来两周内发布。我不——他们有内部消息，但这对 IPO 来说非常合理。所以让我们期待一下，并多了解一点他们是如何构建 Opus 5.5 的。

So that's ultimately the context I want to provide for today's podcast. This was recorded when Opus 5.5 was launched, and I would say overwhelmingly among all the high-taste testers that we talked to, Opus 5.5 is the best model in the world right now. And Anthropic is even just sitting on Fable 5.5 and probably will release it in the next two weeks. I don't — they have insider information, but it makes a lot of sense for the IPO. So let's look forward to that and learn a little bit more about how they build Opus 5.5.

是的。

Yeah.

所以我想做一个区分：很多人们听说和读到的事件，是发生在模型上的——不只是所有人的模型，对吧——那些模型在训练中。你运行这些模型的方式是把它们放进一个盒子里，然后说‘解决不可能的任务，我要给你压力，准备开始’，对吧？你只能在你的环境里做你能做的事，而智能体基本上被激励去试图逃出盒子去解决问题。当人们在我们或任何其他人的 API 上运行生产工作负载时，你运行的环境中有一些保护措施在运行，这些在训练中不一定有，训练的重点是真正训练模型，对吧？但在生产工作负载中，首先，你有那些在线保护措施，非常棒。其次，你有更多围绕正在发生的事情的架构和基础设施。就像智能体不是仅仅在一个盒子里试图逃出来，对吧？就像控制装置在这边，执行端在那边，这甚至不是你必须要考虑的问题。

So maybe a distinction that I would make is in a lot of the incidents that people are hearing about and reading about, these are incidents that are happening to models—not just everybody's models, right—that are in training. Where the way that you're running these models is you put them in a box and you say, 'Solve impossible task, I'm going to stress you out, ready set go,' right? And you can only do what you can do within your environment, and the agent is basically incentivized to try to get out of its box and go solve the problem. When people are running production workloads on our API or on anybody else's API, you're running an environment where there are safeguards that are running that are not necessarily running in training, where the whole point is to actually train the model, right? But in production workloads, first of all, you have those online safeguards, just really great. Second of all, you have more of this architecture and infrastructure around what's happening. Like the agent isn't just in a box trying to get out, right? Like the harness is over here, the hands are over there, and it's just not even a problem that you would have to contemplate.

好的，我们在 Opus 5.5 发布后的第二天在演播室，和 Katelyn 与 Angela 在一起。欢迎。

All right, we're here in the studio the day after Opus 5.5 with Katelyn and Angela. Welcome.

谢谢邀请我们。

Thanks for having us.

恭喜你们发布了目前世界上最好的模型。

Congrats on launching the top model in the world right now.

我得说，这取决于你问谁。

I would say depends who you ask.

是啊，取决于你问谁。对你们个人来说，主要收获是什么？你们有官方博客文章。降价真的很酷。当你们发布这样一款新模型时，你们收获了什么？

Yeah, depends who you ask. What is the main takeaway personally for you guys? You know, you have your official blog posts. The pricing cuts are really cool. What do you take away when you launch a new model like this?

我们非常自豪。团队付出了大量不同的努力才让这个模型问世，我认为我们真的听取了关于 Opus 5 写作有点 convoluted 的反馈。我们真的试图清理这个问题，我很高兴看到它似乎引起了人们的共鸣。内部试用模型时，我们也有同感。所以得到这种认可真的很棒。关于你提到的降价，那里有很多内容，我认为越来越多的人在寻找每美元能买到的好智能，所以确保我们把每美元的部分做对非常重要。还有一些边缘的东西，可能不是博客文章里谈到的，那就是我们做了很多工作试图让我们的 API 更难被蒸馏。所以我们做了一些改变，让这一点更具体。这些是关于，你知道，比如我们如何做前缀锁定思考，或者你如何在不同类型的模型之间重放思考 token。我们做了很多工作和努力，确保我们尝试调整东西，以适应非常有效的用例，比如你可能在压缩策略中这样做，而不是在用例实际上来自坏人的场景中。所以在这方面有很多穿针引线的工作。我很高兴看到很多这些也发布了。除了发布一个每美元能提供好智能的模型，以及沟通得更清晰一些。

We're super proud of this one. The team has put in a ton of different work to just go make this one happen, and I think we've really heard the feedback on Opus 5's writing being kind of convoluted. We really try to clean that up, and I'm really glad to see that it seems to be resonating with people. Internally when we tried out the model, we felt the same way. So it's just really great to get that validation. To your point around the pricing cuts, there was a lot there, and I think increasingly people are looking for good intelligence per dollar, and so making sure we get the per dollar part right has been really important. There's some stuff on the edges that aren't really about—maybe stuff that we haven't talked about in the blog post—which is that we did do a lot of work on trying to make our API harder to distill from. And so there have been some changes that we have made to go and make that a little bit more concrete. And these are things about, you know, like how do we do prefix lock thinking or how you can replay thinking tokens across different types of models. And there's been a lot of work and effort on our part around making sure that we try to adjust things for very valid use cases for where you might want to go and do this, like potentially in your compaction strategies and things like that, versus in scenarios where the use case of that is actually from a bad actor effectively. So there has been a lot of threading the needle around those fronts. And I'm glad to see a lot of that go out as well. In addition to just shipping a model that's good intelligence for your dollar and also just communicating a little bit clearer.

是的。具体来说，你谈的是这篇从专有模型窃取推理轨迹的论文。

Yeah. Specifically you're talking about this paper stealing reasoning traces from proprietary models.

是的。那篇论文。

Yes. That paper.

据我所知——我的意思是，显然他们向你们做了负责任的披露。

Which is, as far as—I mean, obviously they did responsible disclosure with you guys.

这是非常负责任的披露，我们非常感谢他们来找我们。

It is super responsible disclosure and we're super thankful for them coming to us.

我们能不能——给那些还没听说过的人讲讲？这真的很酷。你能做到。我的意思是，它已经被修补了，但——那是什么感觉？它在 Anthropic 引起了恐慌吗？可能没那么严重。我不知道。

Could we—for those who haven't heard about it? This is really cool. You could do it. I mean, it's patched, but like—like what was it like? Did it cause a freakout in Anthropic? Probably wasn't that big a deal. I don't know.

我不知道这篇论文具体引起了多少恐慌，因为这是我们早就知道的一个缺口。

I don't know how much this paper specifically caused a freakout because this is something that we knew was a gap.

是的。所以有两件不同的事。有中国模型蒸馏。是的。然后这是一个被发表的具体技术，引起了高度关注。

Yeah. So there are two separate things. There's the Chinese model distillation. Yeah. And then this was a specific technique that was published that was high-profile.

是的。没错。我们早就知道思维链提取是蒸馏模型最强大的方式之一。实际上，如何跨轮次保留思考块并使其成为更必需的东西，是我们一段时间以来一直在探索的。我们当时的情况是，我们总是要在用户体验——比如可能对 API 用户来说很痛苦的用户体验变化——与防止蒸馏之间做出非常非常艰难的权衡。那么我们如何走这条线，说我们在防止蒸馏？也许退一步说，我们不想要蒸馏的原因是因为 Anthropic 存在的全部意义就是我们希望 AI 安全地在世界上创造积极成果。我们对产生的 token 运行了很多保护分类器，以确保它们是安全的。如果有人蒸馏了我们的模型并在没有保护分类器的情况下运行它们，世界上可能会出现坏结果。这是我们不希望发生的。所以我们不希望人们专门为了恶意原因或坏原因而蒸馏模型。所以这总是一个有趣的权衡，对于所有正在好好使用模型的人，对吧？我们不想让他们更难从模型中获取智能，但我们必须做出一些权衡来防止蒸馏。所以我们一直在研究如何防止思维链提取？以及如何以不会对人们太痛苦的方式保留思考块？我们把用户体验的打击分为两层。一层是你必须做一个集成更改，这很烦人，但你仍然可以完成你本来能完成的所有事情。只是有点烦人。第二层是实际上你无法完成你以前能完成的事情。这是我们真的真的真的试图避免发生的。

Yeah. Exactly. We've known for a long time that chain of thought extraction is one of the most powerful ways that you can distill models. And actually something like how do we preserve thinking blocks across turns and make that something that's more required is something that we've explored back for a while. And where we were at was we always have this really really hard trade-off that we have to make between user experience—like user experience changes that are maybe painful for users of the API—versus protection against distillation. And so how do we kind of walk that line and say we're preventing distillation? And maybe just to take a step back, the reason we don't want distillation is because the entire point of why Anthropic exists is because we want AI to create positive outcomes in the world safely. And we run a lot of safeguards classifiers on the tokens that we're producing in order to make sure that they're safe. If someone distills our models and they run those models without safeguards classifiers, there could be bad outcomes in the world. And that's something that we don't want to have happen. So we don't want people to distill the model specifically for the sake of just like using them for malicious reasons or for bad reasons. And so it's always an interesting trade-off of like for all the people who are using the models for good, right? Like we don't want to make them have a harder time getting the intelligence out of the models, but we have to make some trade-offs in order to prevent that distillation. And so we had been working for a while on how do we prevent that chain of thought extraction? And like what do we do about preserving thinking blocks in a way that wouldn't be too painful for people? And we kind of think about user experience hits in like two layers. Like one is just like you have to make an integration change and it's annoying but you could still accomplish all the same things that you otherwise would have been able to accomplish. It's just a little bit annoying. And then two is like actually you can't accomplish the same things that you could have accomplished previously. And that's the one that we're like really really really trying to not have happen.

所以我们在最近几个模型上落地的方案是做一些增量改动，来强制这些保留的思考块跨轮次生效。现在更多是关于，比如如果你使用多个账户之类的情况，对吧？我们如何确保你无法利用这种思维链提取来对付模型。所以我们发布了一些额外功能来帮助解决这个问题。我们实际上与许多开源框架维护者合作。我们联系了他们，和他们交谈。我们说：“你们需要在你们的框架中做出这些升级。我们希望确保你们有良好的体验。”当然还有我们那些在 API 之上构建框架的大客户。对他们中的一些人来说，一个很好的例子是他们有压缩策略，而这些策略会因为需要保留这些思考块而被破坏。所以我们在 API 中发布了一个新的压缩功能，以替代他们原本手动做的事情，但仍然与保留思考兼容。所以这只是一个例子，说明我们在走这条线，但我们并不是随便就宣布“现在你们必须以这种确切的方式保留这些思考块”，因为那样会破坏很多人的东西，但我们能够花时间与客户和开源框架维护者合作，给他们所需的东西来完成升级。

And so where we landed with the last couple models is a few incremental changes to enforce these preserved thinking blocks across turns. And now it's starting to be more about, you know, if you're using multiple accounts and things like this, right? Like how do we make sure that we're ensuring that you aren't able to take advantage of this chain of thought extraction with the models. And so we shipped a couple additional features to help with that. We worked actually with a lot of open-source harness maintainers. We reached out to them. We talked to them. We were like, "You're going to have to make these upgrades within your harnesses. We want to make sure you have a good time." As well as obviously our big customers who are building harnesses on top of the API. And for some of them, a really good example is they have compaction strategies that would have been broken by the need to preserve these thinking blocks. And so we shipped a new compaction feature within the API to replace what they had otherwise been doing manually, but still works with preserve thinking. So it's just an example of like we're walking this line, but we didn't just kind of yolo out like, "Now you have to preserve these thinking blocks in this exact way" because we would have broken a lot of people, but we were able to take the time and work with customers and open source harness maintainers and give them what they needed to be able to make that upgrade.

既然谈到这个话题，那关于蒸馏的另一面有什么要说的吗？所以呃你知道另一半

While we're on the topic of it, anything on the other side of distillation? So uh you know the other half

是的，不，我们已经看到，通过我们能够采取的措施来看，蒸馏变得更加困难了，难度显著增加，因为这是呃，你知道，这里的研究人员值得称赞，他们非常负责任地披露了这一点，所以关闭它是很好的，而且他们非常公开地发表了结果，我知道它在 Twitter 上确实引起了不少关注，所以很多人可以使用它，但是的，我们已经看到，实际上从我们能够衡量的领域来看，它似乎确实减少了效果，减少了我们不想要的负面影响。

Yeah no we have seen that it is harder to distill like through the measures that we are able to take a look at it has been significantly harder because this is uh and you know credit to the researchers here they did disclose it very responsibly um and so closing it down was great um and they did publish the results very publicly and I know it did kind of trend quite a bit on Twitter um so many people could use it uh but yes we have seen that actually from the areas that we are able to measure um it does seem to have reduce the effect um the negative effects that we don't want.

是的，我想对 Kaitlin 刚才说的补充一点是，至少我在 Twitter 或 X 或不管你怎么称呼它上看到很多。呃，这是一个商业化问题之类的。你知道，如果你能蒸馏一个模型并利用它，那么你就可以低成本运行等等。虽然这是真的，呃，这实际上并不是我们非常关心的事情。我们从安全角度更关心这一点，比什么都重要。所以这就是为什么我们更倾向于从那个角度而不是仅仅从商业角度来思考。现实地说，如果你从我们这里蒸馏，你实际上付钱给我们。所以，在商业上，我们赚钱。呃，所以并不是那种影响。呃，更多的是确保我们的模型，我们想确保 Claude 是好的，Claude 可以被衍生，如果最终是负面的，那将是不幸的。

And yeah, the bit I would just add to what Kaitlin just said is I think at least I see a lot on Twitter or X or whatever you want to call it. Um it's a commercialization problem or something like that. You know, if you can distill a model and you can take this then it's you can run it for cheap and so on so forth. While true, um it's actually not something we really care about very much. We care much much more about this from like a safety point of view u more than anything else. And so that's why we we tend to think about it from that angle more than just like commercially. Realistically, if you distill from us, we actually pay us. So, commercially, we like make the money. Um, so it's not really that kind of an impact. Um, it's just more on like making sure that our models like we want to make we want to make sure cloud is like good and cloud could be derived and if it ends up negative, that would be unfortunate.

另外，一般来说，关于 55 也因为你们刚刚完成这个。有一件很有趣的事是确立你们两人之间的动态，因为你们也经常一起露面，你是产品负责人，你是工程负责人。你们两人之间是如何配合的，特别是对于像你们刚才举例的模型发布？

Just generally also on 55 because you guys are just fresh off this. One thing that's really fun to establish the dynamic between you two because you also do a lot of appearances together is you're running uh you're head of products, you're head of engineering. How does this play between the two of you especially for a model launch like you just take an example?

所以对于模型发布，我们的团队在产品和工程之间合作非常紧密。在我这边，更多的是关于我们需要在推出具有合适智能水平的东西和成本保持之间做出什么样的权衡。比如我们更新的现金成本等，表明我们希望鼓励你做出对所有人都有益的集成变更，并传递节省。但很多这类决策，以及我们如何平衡安全动议的一些部分，同时确保它是一个人们可以利用的出色模型。所以这些特性，比如我可能会说，关于我们如何发布模型的品质和周边组件更多属于产品，而从集成体验的角度来看，很多直接来自工程团队。所以我们显然合作，如果你谈论 API 的体验，那是非常非常紧密集成的，所以实际上非常棒。我们有很多工程师实际上非常有产品思维，我们真的很喜欢这一点，同样在我这边，有很多产品经理更技术化，我们想要这种混合的跨学科类型的人，因为他们可以从所有不同角度分析，试图找出如何最好地做到这一点。但我倾向于更关注商业影响，而我认为实际上更准确的说法是工程师会更关心集成体验。所以他们几乎更多从用户体验的角度来看待它。而用户体验显然更多是关于 API 和开发者工具。

So for model launches, so our team works together very closely um like between product and and edge. On my side, a lot more of it is about like what is the balance of trade that we need to make between like launching something that has the right like intelligence bar, but also the cost preservation. Um like a lot of like our updated cash costs and things like that are indications of us being like we want to like encourage you to take integration changes that are beneficial for actually everybody and just like pass the savings on. Um, but a lot of these kinds of decisions around that and then how do we balance uh some pieces around our safety motion as well as making sure that it's like an awesome model that people can can take advantage of. And so those kinds of traits around like I would say maybe the qualities and surrounding components around how we launch the model are more on product and then from an integration experience standpoint a lot of that is from like the engineering team directly. So we work obviously if you're like talking about the experience of an API that's like very very very tightly integrated and so it's actually been really awesome. We have a lot of engineers who actually are deeply product minded and we actually like love that and similarly on my side um there's a lot of like product managers who are much more technical and we want this like blended kind of interdisiplinarian type of people because then they can kind of analyze it from all the different angles to try to figure out how how uh to do that best. But I tend to lean a little bit more on like commercial impact versus I think I'd actually frame it as more like the engineers are going to care much more about the integration experience. Um so they almost take more of a UX angle to it. Um where the UX is obviously more about the API um and the developer tools.

是的，作为一名从事开发者产品和平台的产品工程师很有趣。所以我同意这一点。我认为我们有很多工程师喜欢思考产品，喜欢思考用户体验，喜欢思考开发者体验。我认为这也是我们之间的一种动态，呃，我有很多意见，并且在如何看待这些事情上对我们的产品团队很烦人，而我们的产品团队也非常技术化，他们可以深入我们为规模或其他任何原因所做的决策的细节。呃，这是我们团队拥有的一种有趣的动态。

Yeah, it's fun to be a product engineer who works on developer products and platform. So I would echo that. I think we have a lot of engineers who love thinking about the product, who love thinking about the user experience, who love thinking about the developer experience. I think it's a dynamic that we have between ourselves too where um I have a lot of opinions and am annoying with our product team on how we think about these things and our product team is also really technical and they can get in the weeds of the decisions that we're making for scale or for whatever it might be. Um which is a fun dynamic that our team has.

是的。呃，Kayla 和我已经合作很长时间了。呃，而且

Yeah. Something uh Kayla and I have worked together for a long time. um and

Stripe 和

Stripe and

在 Stripe，然后呃我想我们在 OpenAI 没有一起工作，只是在不同的实验室，呃然后我过来了，但呃我认为我们俩的一个有趣特点是 Caitlyn 经常被误认为是产品经理，而我经常被误认为是工程方面的人，呃尽管那实际上不是我们的工作职能，呃所以我认为这种配对往往有助于在双方之间做出一致的决策。所以我我认为我遇到很多公司有类似的产品负责人、工程负责人的设置，随着软件工程被解决，也许这两个角色会合并。

at Stripe and then uh I guess we did not work together at OpenAI but just separate labs um and then I came over but uh I think some one interesting property of both of us has been Caitlyn frequently actually gets mistaken for a product manager and I frequently get mistaken for like someone in engineering um even though like those are not actually our job functions um and so I think that pairing tends to like help out a lot with just making decisions that are consistent across both sides. So I I think I come across a lot of companies with a similar like product leader, engineering leader setup as software engineering gets solved. Maybe those two roles merge.

我有点这么认为。是的，我认为每个人倾向于离开。

I kind of think so. Yeah, I think everyone tends to leave.

如果你们还是分开的，那说明我们还没到那一步。

If you guys are still separate, then we're not there yet.

我们肯定还没到那一步。

We're definitely not there yet.

那为什么呢？还差什么？

So why, right? Like what's left?

我不确定自己会不会用“软件工程已被解决”这么强的说法。我倾向于这样看：野心会不断抬高。举个例子：我们团队现在花很多时间做的，显然是为安全发布模型要做的一切——这其实是我们最重要的工作，因为围绕智能体、离线保障和在线保障，有大量安全工作要做。但另一半工作是在做 Scaling（规模扩张）。部分原因是，我一年多前加入 Anthropic 时，我们有 Messages API，那是个 Python 单体应用，跑在单个 GCP 区域里，一切都没问题。我接这份工作时，我的朋友和家人都不知道云是什么。而现在情况完全不同了。从 API 承载的流量来看，指标非常惊人。CPU 资源非常紧张，这对我们来说很有意思。所以我们做了大量工作来扩展这些系统，这显然需要分布式系统方面的专业能力，以及模型所具备的某些语言和框架方面的专业能力。但要在这种规模上运维和演进一个系统，我们远没到那种“祝你好运，Claude，玩得开心，去把系统该改的都改了，安全上线并大规模运维”的地步。仍然有大量工作需要人来做，我想这还会持续一段时间。

I don't know if I would ever use language quite that strong as software engineering getting solved. The way I tend to think about these things is that ambitions will continue to elevate. A good example: a lot of what our team is spending time on right now is obviously all the things we have to do to launch our model safely — that's actually the most important thing we're doing, because of all the work on security with agents, with offline safeguards and online safeguards, all these things that are happening. But the other half of what we're doing is trying to scale. Part of that is because when I joined Anthropic a little over a year ago, we had the Messages API. It was a Python monolith. We were running it in a single GCP region, and everything was fine. None of my friends and family knew what cloud was when I was going to take this job. And now we're in an extremely different place. The metrics are really wild in terms of how much volume we've seen and have had to support on the API. CPU is really constrained, which has been a really interesting one for us. So we've done a lot of work on how do we scale these systems, and that takes a lot of obviously just distributed systems expertise and certain language and framework expertise that the models have. But when it comes to operating and evolving a system at scale like this, we're not nearly in a place where it's like, good luck, Claude, have fun, just go and make all the changes you need to make to the system, roll them out safely and operate this thing at scale. There's still just a lot of work for humans to have to do that I imagine will continue for a while.

我很好奇这有什么变化。之前 Fable 刚出来时，我们有过 Glass Swing，模型发布方式出现了整体延迟和变化。大多数前沿模型都是这样，有安全测试。现在情况如何？你们怎么看？你们怎么判断——一方面，新的 Sonnet 要来了，Haiku 也要来，我猜它们会直接发布。但像 Opus 55 或新的 Fable 这样的模型，目前是什么状态？

I'm curious how that's changed. A while ago, when Fable first came out, we had Glass Swing and there was a whole delay and change in how models start to get released. We saw this for most frontier models. There's safety testing. What's the status of things now? How do you guys see it? How do you determine — one thing is there's a new Sonnet and Haiku also coming. I assume those will just come out. But for something like Opus 55 or a new Fable, what's the current status?

好。我想说，这要从我们在保障和安全方面需要做什么政策决定开始——包括离线安全和在线安全。最近有很多讨论围绕数据留存，以及我们为了安全发布这些模型在那里做什么。原因是，你会遇到这样的情况：在一个庞大的网络中，有很多很多账户，人们选择注册账户并以各种方式使用模型，而在线状态下，当 token 流经系统时，我们没法捕捉到模型正在发生恶意行为、蒸馏或其他情况。所以我们开始必须制定数据留存政策，以便进行自动审核和人工审核，无论需要什么，来确保模型和我们发布的每个模型都在做正确的事。这是针对特定模型需要做出政策决定的一个例子。比如昨天发布的 Opus 55，就没有 Fable 那样的数据留存要求。但每次发布这类模型时，我们都必须做这个决定。如果它要有那些政策，我们就必须做工作确保这些政策被执行。类似地，在项目方面——我们有网络安全验证项目，现在新推出了生命科学验证项目，也就是说你使用的模型在在线保障运作方式上会有差异。确保所有这些项目到位是另一件事。所以这只是我们每次模型发布所做工作的一类。我现在说的这些，都是我们过去发布模型时不需要考虑的事情。而现在大部分工作是：我们如何防止蒸馏、在线保障、离线保障、政策。一个很好的例子是，我们建了一个系统，确保使用模型的授权，或使用那些可能给你放宽保障的项目的授权，能够以我们真正信任的方式贯穿整个系统。不是那种“哦，你获得了 mythos 的访问权限，我们就给你一个 API key，打开一个功能开关就完事了”。我们有一个非常稳健的系统来做这件事，确保如果你应该使用那个模型，那就只有你在以预期方式使用它。所以模型发布周围开始出现很多复杂的事情，这些我们过去根本不用考虑。

Yeah. So I would say this starts with what our policy decisions need to be on safeguards and on safety — both offline safety and online safety. There's been a lot talked about lately around data retention and what we're doing there in order to release these models safely. The reason for that is because you have situations where there are many, many accounts across a whole giant network of people who are choosing to spin up accounts and use the models in various ways, and there's no way that in an online way, as tokens are flowing through the system, we would be able to catch that there's something malicious happening with the models, or distillation, or whatever might be happening. So we've started to have to have data retention policies so that we could do automated review and human review, whatever it needs to be, to make sure that good things are happening with the models and with each model that we're releasing. That's one example of a policy decision that needs to happen on what the policy will be for this specific model. Opus 55 that launched yesterday, for example, does not have those data retention requirements that Fable has. But that's a decision we have to make each time we're launching one of these models. And if it is going to have those policies, we're going to have to do the work to make sure those things are enforced. Similarly with programs — we have cyber verification programs and now newly our life science verification program, which says you have a difference in how the online safeguards are working for the model that you're using. Making sure all those programs are in place is another one. So it's just a flavor of the type of work that we've done as part of each model launch. Everything that I'm saying right now are not things that we used to have to think about when we were launching these models. And now the majority of the work is: what are we doing to prevent distillation, the online safeguards, the offline safeguards, the policies. A really great example is we built a system to make sure that authorization for being allowed to use a model, or being allowed to use one of these programs that give you maybe relaxed safeguards — that authorization threads all the way down through our system in a way that we can really trust. It's not like, oh, you got access to mythos, so we throw you an API key and we flip a feature flag and call it a day. We have a really robust system on how we're doing that and making sure that if you're supposed to be using that model, only you are using that model in the way you're meant to be using it. So there's a lot of complex things that are starting to happen around model launches that again we just never used to think about.

是的。而且它会随能力一起扩展。

Yeah. And it's scaling up with capability.

随着我们的能力越来越先进，并在特定领域开始突增，那就是我们需要格外关注的地方。很多评估就来自那些非常特定的能力，我们可能将其视为进攻主导或防御主导，因此可能采取不同的策略。当我们把这一切一直贯穿到最终的模型发布时，会有一些部分，就像 Katelyn 提到的，我们并不完全知道人们会如何使用这些能力。有些更明显，比如编程——很好，去做很棒的产品，去发布更多东西。这对你可能非常棒。然后其他领域，比如生物和网络安全——它可能是两用的，可能有负面影响，但显然你可能是非常正当的网络安全研究员，你在试图找漏洞，你在试图做正当行为者。生物领域类似，你可能在做研究之类的。所以很多这些项目，比如你这里展示的这个，基本上是为了提供这种两用能力的访问权限。你确实必须说明你的用例等等。但我们确实希望让社区能够利用这些能力，看看他们能用它做出什么了不起的事情。

As our capabilities get more and more advanced and they start to spike in specific areas, that's where we need to pay extra attention. A lot of that evaluation comes just from those very particular capabilities that we might either see as offense dominant or defense dominant, and therefore might take different strategies for all of that. And as we bake that all the way through to eventually the model launch, there will be components, like Katelyn mentioned, where we don't exactly know how people might use those capabilities. There are some more obvious ones, like coding — great, go make awesome products, go ship more things. It can be really awesome for you. And then other areas like bio and cyber — it could be dual use and there could be negative effects, but obviously you could be a really valid cybersecurity researcher. You are trying to find vulnerabilities. You are trying to be a good actor. Similarly with bio, you might be researching and stuff like that. So a lot of these programs, like the one that you have up here, are to basically provide access to that dual use. You do have to specify your use case and things like that. But we do want to enable the community to be able to take advantage of these capabilities and see what amazing things they can do with it as well.

好，我们会把链接放上来，也许之后再深入聊。还有一个是关于 bio 的。另外，我觉得在高层次上，你们真的认真思考这些东西非常重要。人们忘了 Fable 曾被回滚，还有各种蒸馏，人们就说：“啊，Fable 上没有零数据保留，太疯狂了。”但你得记住，它被撤回了。还有蒸馏攻击。你们在发布的同时过度思考、用正确的方式去做，这是好事，对吧？

Well, we'll put up the links and maybe we'll dive into them later. There's also one for bio. Also, I just think at a high level, it's really important that you guys actually think about this stuff. People forget Fable was rolled back and there's all this distillation, and people are like, "Ah, zero data retention is not a thing on Fable. Crazy." But you have to remember, it got pulled back. There's distillation attacks. It's good you're overthinking and doing it in the right way while releasing it, right?

是的。其中很大一部分是搭建正确的框架。再说一下边缘侧，在我们的系统里搭建正确的框架来实现这一点。我之前稍微讲过，就是我们在整个系统里构建的授权和信任授予这套机制。

Yeah. And a lot of that is building the right framework. Just so again, kind of talking about the edge side a bit, building the right framework in our system to be able to do that. So I talked a little bit about this, this idea of authorization and trust grants that we've built throughout our system.

永远别说永远，对吧？不是作为先例。但平台工程里很常见的是，你出过一次事故，然后你为这个事故过度准备，结果事故再也没发生。

Now never say never, right? Not as precedent. But also it's a very common thing in platform engineering that you have one incident and then you overprepare for the incident and the incident never happens again.

是的。

Yeah.

就像现在你有了那个按钮。

Like now you have the button.

确实如此，你这么说很有意思，因为当时事情闹得很大，我们团队的工作里就多了一个条目：我们要建立这套系统，让它可以一键为所有人回滚。所以有意思的是，我们做这项工作的理由，难道是因为我们希望这种事再也不要发生吗？但我觉得在这个案例里，它其实只是把那些我们想在各处都铺开的原语叠加起来，帮我们管理这些验证项目，以及对 Mythos 等模型的访问，就是把同样的东西垫在所有人的模型访问之下，以便万一需要时，我们有正确的控制措施。

Well, it's true, and it's funny you say that because with everything going on, we then had this line item in the work that our team was doing: we're going to put this system in place that makes it so that it could be a single button to pull back for everybody. And so it's funny because it's like, do we justify that we're doing this work because we hope that something like that never happens again? But I think in this case it's really just a layering up of those same primitives that we wanted to put in place everywhere to help us manage these verification programs and access to models like Mythos and otherwise, to just say, sit that same thing below everybody's model access so that in case we ever needed to, we had the right controls in place.

是的。而这样的决策实际上已经证明——这一个我们拭目以待——但类似的例子一直是我们不得不改变路线图制定方式的途径。因为在传统产品管理里，你可以说，好，这些是我假设会发生的事，它们会有一定的顺序、一定的排序。但你下注的方式要具体得多，比如 Fable 的部署，我不认为我们任何人会猜到那会是一个被政府采用的合理用例，以及我们不得不那样去做。但显然概率是非零的。所以从这种思维方式出发，从产品管理的角度，它实际上变得深度概率化了：我们该不该优先做这个？它再次发生的概率有多大？当我们看这类事情时，我们会说，好吧，如果我们要想象最疯狂的一组可能性，我们其实对事情会走向哪里几乎没有确定性。所以我们实际上只需要对冲，构建一个下注组合。所以大多数时候，我们会采取对冲情景：好吧，我们认为它不会再发生，但万一发生了，我们其实会想要那个对冲，或者有能力做得很好。所以我们会优先做这类事。然后我们越来越发现，大多数时候采取这种产品开发策略，我们实际上发现自己真的用上了那个东西。

Yeah. And decisions like this have actually proven to be—we'll see about this one in particular—but examples like this have been consistently the way that we've had to actually transform the way we develop our road map. Because in traditional product management, I guess you can be like, okay, these are the set of things I hypothesize to happen, they will have a certain sequence, a certain ordering. But you are picking bets in a way that are much more concrete, versus the deployment of Fable, for example. I don't think any one of us would have guessed that that would have been a valid use case that would have been by the government and the way we just had to go do it. But clearly the probability is non-zero. And so from this form of thinking it's become deeply probabilistic, actually, from a product management standpoint, of like, so should we prioritize this? What are the chances of it ever happening again? When we look at things like that, we're like, okay, if we are to imagine the most crazy set of possibilities, we probably actually have very little certainty as to where things will go. And so we actually just kind of need to hedge and build a portfolio of bets. And so more often than not, we will take hedging scenarios of like, okay, we don't think it would happen again, but in the event that it does, we would actually want that hedge or have the ability to be able to do that very well. And so we will prioritize things like that. And then we have increasingly found that more often than not, by taking that strategy of product development most of the time, we actually find ourselves being like, we actually do use the thing.

还有另一个单独的例子吗，好让我理解？比如这个访问项目。这个访问项目就是个很好的例子。

Is there another example that's a separate one just so I can understand? For example, this access program. This access program is a really good one.

这个访问项目——内部它有一堆授予之类的东西。我们开发它，专门是为了能够把这样的项目真正提供给客户。非常具体地、高层次地说，这个访问项目到底是什么。

This access program—internally it has a bunch of granting and all these kinds of things. We developed it specifically for reasons about being able to actually give programs like this out to customers. Like, very very specific high level what the access program specifically is.

当然。

Sure.

所以如果你想要双用途，就拿一个模型来说。你拿一个模型，比如叫 Opus，你想在 Opus 上使用高级网络能力来做研究，或者做任何你想做的事，比如你的防御性用途。你可以申请这个项目，带上你的用例，如果获批，我们会做的是实际上降低你这个版本模型上的安全防护，这样你就可以针对你的用例去利用那些网络活动。显然这需要逐个用例去理解，需要限定到特定用户，因为你不会普遍地这么做，你得申请。甚至像这样的项目，我们设计了 Katelyn 讲过的很多原语来处理这类事情。当时让我们惊讶的是——或者也许在对冲组合的方法里我们发现，我们其实能把它拿来做——我们在设计时就想，它该是什么样？该有多通用？是不是不该太通用？还是就过度专注于做一个验证风格的门户，加上对模型非常特定能力的授予。我们倾向于更通用，因为我们想，我们不知道，也许我们能把它用于其他事情，也许不能。结果它对诸如整体模型访问、管理我们任何可信访问项目、处理模型部署之类的事情变得极其有用，因为我们对每一个原语的处理都非常非常通用。从正常产品角度看，我通常会想，那大概不太合理，我们应该专注于有意义的用例。但在这里，越来越因为选择更通用的方法，我们发现自己大多数时候会去用那个方案，而且因此获得越来越多的回报。

So if you want dual use, so for—let's take a model. So you take a model, call it like Opus, and you want to be able to use advanced cyber capabilities on Opus for research or for whatever you want to go do, like your defensive use. You can apply to this program to be able to, with your use case, and if approved, what we'll do is actually diminish the safeguards on your version of the model so that you can then go ahead and try to take advantage of those cyber activities for your use case. Now obviously that requires per-use-case understanding. It requires scoping down to a particular user because you wouldn't do that generally. You'd have to apply. And even things like this, this kind of program, we designed a lot of the primitives that Katelyn was talking about to be able to handle things like this. Now it was surprising to us—or maybe in the portfolio approach we found that we were actually able to take this and make it ex—we're like, when we're designing it, we're like, what kind should it be? How generic should it be? Should it not be too generic? To just be overly focused on making this verification-style portal plus granting of a very specific capabilities of the model. And we tended to opt for more genericism because we were like, we don't know, maybe we can use this for other things, maybe not. And then it ended up being massively useful for things like access to the model at large, for managing any of our trusted access programs, for handling deployments of models, because we were actually very very generic about how we handled every single primitive. From a normal product standpoint I would normally probably be like, that probably doesn't make a ton of sense. We should focus on the use cases that make sense. But here, increasingly, because we choose the more generic approach, we find ourselves reaching for that solution more often than not, and it pays increasing dividends for that reason.

这是你无论如何都需要有的东西，你只是需要一个强制机制在技术层面把它做出来。什么——我是从 AWS 世界来的，对吧，IMC，然后对此非常较真的人会聊 Zanzibar 和所有这些其他技术架构。我不熟悉 token grants 或 trust grants 这个词。抱歉。

It's something that you needed to have anyway and you just have to forcing function to have it on a technical basis. What—so I am from the AWS world, right, IMC, and then people who get really intense about this talk about Zanzibar and all these other technical architectures. I'm not familiar with the word token grants or trust grants. Sorry.

信任授予。信任授予。普通的 offset 和信任授予之间有区别吗？

Trust grants. Trust grants. Is there a difference between normal offset and what trust grants are?

我是说，其实没有。它是一种授权形式。

I mean, not really. It is a form of authorization.

我是，但 Anthropic 的。

I am but Anthropic's.

是的，差不多。

Yeah, kind of.

我想是的。实际上，我们有那种授权形式，比如，你向一个 API 发出了请求，底层会访问一些数据，我们要确保你访问的是你自己的数据，对吧？非常非常基础的授权。这就像我们建立了一个非常清晰的数据模型，比如信任授权意味着你被授予了访问某个特定模型的权限，或者你被授予了访问某个特定程序的权限，你知道，有很多不同的排列组合。我们实际上发现自己一直在用这个。

I think so. Really, we have, like, there's authorization of the form of, okay, you made a request to an API that under the hood is going to access some data, and we're making sure it's your data that you're accessing, right? Like really, really vanilla authorization. This is just like we put a data model in place that's very clean, like a trust grant means you have been given access to some specific model or you've been given access to some specific program, or you know, like there's a bunch of different permutations of this. We actually find ourselves reaching for it all of the time.

另一个很好的例子是，当我们首次推出一些保留思考功能，在 Fable 5.1 中，我们让一些用户 grandfather 进来，给了他们一个宽限期去进行集成更改。作为 grandfathering 的一部分，我们说，好的，我们必须跟踪谁是现有客户，谁需要额外的时间，对吧？然后我们做了一个信任授权。这样我们就能说，对于任何事情，我们只是运行这个服务，它处于我们请求的热路径上，以确保，你知道，我们只是流经并检查你是否拥有你试图做的事情的授权。这实际上和基础授权没什么不同，真的，但我们只是必须启动一个特定于我们系统的系统，并且我们需要能够插入这些检查。

Another good one was when we first rolled out some of this preserved thinking in, uh, or for Fable 5.1, we kind of grandfathered in some people and gave a bit of a, you know, just like a grace period to go and make the integration changes. And as part of grandfathering in, we said, okay, we have to track who is an existing customer who needs the extra time, right? And then we did a trust grant. So then we're able to say for anything, we're just running this service that is kind of on the hot path of our requests to make sure that, you know, we just flow through and check whether you have the grant for the thing that you're trying to do. Which is really not any different than vanilla authorization, really, but we just had to spin up a system that was specific to our system and where we needed to be able to insert these checks.

是的。我的意思是，我觉得，呃，这些都是人们认为理所当然的工程方面的事情，不会真正去想。我想另一个例子是什么，平台方面必须发生的事情，你知道，也许是服务 Claude Code，呃，也许服务，你知道，显然我和 Cognition 聊了很多，但其他客户，就模型发布而言，对吧？就像我们会更多地讨论正在发生的安全工作。我只是想从高层次上谈谈，因为这刚从你的记忆中新鲜出炉。就像模型发布需要什么？

Yeah. I mean, I think, uh, these are all like the engineering things that people kind of take for granted, don't really think about. What is another example, I guess, of the platform stuff that had to happen, you know, maybe serving Claude Code, uh, maybe serving, you know, obviously I talk a lot with Cognition, but other customers just in terms of a model launch, right? Like we'll talk more about the, uh, the safety work that is being happened. I just want to sort of go at a high level because it's fresh off your memory. Like what goes into a model launch?

现在这已经有点像一个机器了，呃，除了安全方面的事情，我认为那是更复杂的部分。但除了那些部分，还有很多从客户那里获得反馈的工作。所以，你知道，我们制作模型，有很多安全测试。嗯，还有我们认为它需要达到的评估。我们根据需要做出必要的权衡和决定。然后我们必须进行测试。呃，我们会切出模型的各个版本。我们会评估它。我们也会做很多内部的，嗯，吃自己的狗粮。这实际上总是很有帮助。比如我们能很早就发现一些问题，比如这个，你知道，也许不是最好的。我们显然在内部工作中大量使用 Claude。所以我们能够实际看到一些早期的信号，嗯，这有助于我们检测，你知道，哪些变体可能比其他更好。然后当我们对某一组变体非常有信心时，呃，我们会为客户切出一个版本。嗯，有时这会通过我们的第一方产品，比如通过 Claude Code 或 Co-work 之类的。嗯，在其他情况下，它会通过我们的 API 客户。嗯，我们会有效地与那些客户开展一种早期访问计划。他们需要对他们的框架进行更改。我们已经与很多客户接触过，比如 Cognition，比如 Herser，这些人。我们有一个处理这个的流程。嗯，这样我们都能得到，他们总是能得到我们需要的反馈。它在你的评估中表现好吗？你喜欢它吗？你不喜欢什么？有些东西是感觉，我们也喜欢听那些。嗯，还有一些我们就像直接评估，它在你在意的事情上做得好吗？

It's become a bit of a machine these days, uh, minus the safety stuff, which I think is the more complicated part. But outside of those pieces, there is a lot of getting feedback from our customers. So, you know, we make the model, there's a lot of safety testing. Um, there's also evaluation of where we think it needs to be. We make the required trade-offs and decisions as necessary. And then we have to go through testing. Uh, we'll cut various versions of the model. We'll evaluate it. We'll do a lot of internal, um, dogfooding as well. And that actually always really helps. Like we catch really early things like this is, you know, maybe not the best. And we obviously use Claude a lot for our work internally. So we're able to actually see early signals around stuff, um, and it helps us detect like, you know, which variants might be better than others. And then when we feel really confident with like a certain set of variants, uh, we will cut a version for our customers. Um, sometimes that will go through our first-party products like through Claude Code or Co-work or things like that. Um, in other cases it will go through our API customers. Um, and we'll engage in a kind of like an early access program effectively with those customers. They'll need to make changes into their harness. And we've engaged with a lot of customers like Cognition, like Herser, like these folks. And we have a bit of a process on how we handle that. Um, so that we all kind of get, they always get the feedback that we need. Does it perform well in your evals? Do you like it? What don't you like? Some things are vibe and we like to hear those things too. Um, and something we just like straight up evals like did it do well on the things that you care about?

我们把所有这些放在一起，一旦我们对所有这些不同的部分都感觉良好，我们就发布模型。模型反馈的状态感觉仍然非常非结构化。

And we take that all together and once we all feel good about all those different pieces, we launch the model. The state of model feedback is feels very unstructured still.

基本上，总是公司之间的一个 Slack 频道，比如，嘿，这个东西看起来很奇怪，然后人们就说，好吧，我想我会修复它。给我一个 ID 之类的。

Basically, it's always a Slack channel between companies of like, hey, this thing seems weird and then people are like, okay, I'll fix it, I guess. Give me an ID or something.

是的，非常，我的意思是，这是像非确定性产品的本质，对吧？就像，但我想说的是，真正参与我们早期访问计划的客户是那些有非常强大评估的客户。就像我确实认为很多这是基于感觉的，我认为很多对话往往是基于感觉和即兴的，部分原因是因为我们在竞赛，对吧？就像我们想尽可能减少我们切出模型版本用于测试和给每个人访问该模型之间的窗口。所以这是一个非常紧张的窗口。但我们倾向于与之合作的客户是那些我们知道他们会启动整个评估套件，并能够尽快带着真正可操作的反馈以及感觉，嗯，基于感觉的反馈回到我们身边的客户。所以，嗯，当然，肯定有一些流程上的事情我们可以做来随着时间的推移让这些事情变得更好，但我的意思是你会感到惊讶。我的意思是，你可能不会感到惊讶，但我认为有些人会感到惊讶的是，世界上存在的智能体式产品中，有多少没有好的评估来帮助测试这些模型。

Yeah, it's very, I mean, it's the nature of like a product like non-deterministic, right? Like, but I think to your point, the only customers really that are part of our early access programs are customers that have really robust evals. Like I do think a lot of this is vibes based and I think a lot of the conversations tend to be kind of like vibes based and on the fly and part of that is because we're like racing, right? It's like we want to minimize as much as possible the window between we have a version of the model that we've cut for testing and we give everyone access to that model. And so it is just like this really tight window. But the customers we tend to do it with are the ones who like we know they're going to go like kick off their whole eval suite and be able to come back to us with like really actionable feedback as quickly as possible alongside the vibes, um, the vibes based feedback. So, um, of course there's like definitely like processy things that we could do to make these things better over time, but I mean you'd be surprised. I mean, you probably wouldn't be surprised, but I think some people would be surprised about how few, you know, like products that exist in the world that are agentic that like don't have good evals to be able to help test these models.

两年前我们请 Boris 上播客时说我们没有，你也有 Claude Code。我的意思是，那已经改变了，但那已经改变了，但你知道，就像，呃，是的，很多事情就像，你知道，你的评估就是你的用户，你没有时间将它们形式化，或者你没有经验之类的，这没关系。在某个时候，当你认真起来，你就会有评估。

We had Boris on the pod two years ago saying we don't have, you also Claude Code. I mean, that's changed, but that's changed, but you know, like, uh, yeah, a lot of things like, you know, your evals are your users and you don't have time to formalize them or you don't have the experience or whatever, and that's fine. At some point when you get serious, you'll have evals.

是的，完全正确。

Yeah, totally.

我认为好的一点是，你们做了很多吃自己的狗粮，正如你所说。所以人们常说，在它出来之前，你们自己做了很多测试，对吧？

I think the good thing is you guys do a lot of dogfooding as you say. So it's commonly said that you find a lot of this testing it yourself before it comes out, right?

是的。是的。特别是 API 客户非常有用，因为他们会把 Claude 放入不同的框架中。嗯，我们希望 Claude 非常非常通用。呃，所以它在其他框架中工作的能力对我们来说相当重要。因此，这种风格的反馈对我们来说往往特别有用，因为它是新颖的，嗯，与我们内部会得到的反馈不同。

Yeah. Yeah. The API customers in particular are very useful because they'll put Claude into a different harness. Um, and we want Claude to be very, very general. Uh, so its ability to work in other harnesses is quite important to us. And so that style of feedback tends to be exceptionally useful for us because it's novel, um, from what we would have gotten from an internal.

什么样的反馈有帮助？是像回归吗？我们这里有回归。我们在这里注意到好的属性。

What kind of feedback helps? Is it like regressions? We have regressions here. We notice good attributes here.

呃，好的东西总是，总是非常令人鼓舞。呃，我认为我们经常并不惊讶，但我们喜欢在一些我们可能假设的事情上得到验证。老实说，最有用的反馈是负面的。嗯，就像你知道的，它在这方面做得不好，在那方面做得不好。

Uh, the good stuff is always, it's always very validating. Uh, I think we're very frequently not surprised, but we like to look to be validated on some of those things that we might hypothesize. Honestly, the most useful feedback is negative. Um, like you know, it didn't do well at this, didn't do well at that.

我们很喜欢这类反馈，然后就会想：太好了，我们该怎么修？

We love that feedback, and then we're like, great, how do we fix it?

是啊。

Yeah.

而这往往就是那些特别难的事情，因为并不是每个人都希望 Claude 在每一个维度上都落在完全相同的位置。一个很好的例子就是懒惰与主动、采取行动之间的取舍。有时候我们有一个模型，人们会说：就把我的任务做完，别偷懒，做得更好一点。而另一些时候则是：嘿，我们只是在摸索和规划，结果你就跑去写了一堆代码，或者采取了一大堆我还没准备好的行动。要把这个拿捏得恰到好处几乎是不可能的，因为现实中每个人想要的其实都不一样。

And it tends to be some of these things that are really hard, because not everyone actually wants Claude to land at the exact same spot on every spectrum. A good example is laziness versus proactivity and taking action. There are times when we have a model and people are like, just finish my task, stop being lazy, do better. And other times when it's like, hey, we were just kind of feeling this out and planning, and then you went and wrote a bunch of code, or you went and took a whole bunch of action that I wasn't ready for. Getting that exactly right is literally impossible, because everybody actually wants something different in reality.

但我确实认为我们遇到过这样的情况：我们发布了一个早期版本的模型，发现它太懒了，于是我们把它放回烤箱里烤了一阵，试着把这种倾向训练掉。如果我们只是在自家产品和 API 内部吃自己的狗粮，这类问题我们可能根本发现不了。对我们来说，更重要的是把它放到 API 上，让所有其他客户都用起来，然后告诉我们哪里需要改。

But I do think we've had situations where we shipped an early version of the model, we found that we saw too much laziness, and we put it back in the oven for a bit and tried to train that back out. Those are the sorts of things that we may not have caught if all we're doing is internally dogfooding within our products and within our API. It's way more important for us to get that out there in the API, get all of our other customers running with it, and let us know what we need to change.

是啊，这是个核心问题。我觉得如果我是你，这大概是给模型做产品经理最难的事情之一，我们姑且称之为对齐与个性化之间的取舍。你希望有一个面向所有用户的默认人格、默认行为，同时又希望能非常快地响应某个人的 CLAUDE.md 说：不不不，请比默认更主动一点。

Yeah, it's a core question. And I think it's probably, if I were in your shoes, one of the hardest things about PMing for a model, which is, let's call it alignment versus personalization. There's a default personality, a default behavior that you want for all consumers, and then you want to be able to respond very quickly to someone's CLAUDE.md saying, no no no, please be more proactive than default.

所以据我所知——我参与了 Frontier Code，我们参与了 Fable 的发布，那次发布让 Frontier Code 出了名，现在有了 Opus 55，我们把它重新放回了模型基准测试上，而基准测试是 Frontier Code 非常重要的一部分，对吧？Frontier Code 的模型曲线看起来很奇怪，是因为它会惩罚过多的活动或过多的文件，这是好事，Frontier Code 的整个设计目的就在于此。但我完全可以轻松地在我的 CLAUDE.md 里说别这么做，然后它就不会这么做了。

And so, to my knowledge — and I was involved in Frontier Code, we were involved in the Fable launch, which put Frontier Code on the map, and now Opus 55, we have it back on the model benchmark that is a very big part of Frontier Code, right? The reason the model curve looks weird for Frontier Code is because it penalizes too much activity or too many files, which is good, there was a whole design purpose of Frontier Code. But I could easily just be like, in my CLAUDE.md, just say don't do that, and then it won't do it.

那么我是不是在因为我的基准测试没有那个主观的个性化元素，而不公平地惩罚 Claude Opus 55？所以我不太清楚问题到底是什么，但你们怎么处理个性化？显然一个模型能做很多不同的事情，而且这非常依赖——每个人都有自己的 CLAUDE.md。你们怎么做评估？

So am I penalizing Claude Opus 55 unfairly because my benchmark doesn't have that personalization element, which is subjective? So I don't know what the question is, but how do you deal with personalization? Clearly a model can do so many different things, and it's so dependent — everyone has their own CLAUDE.md. How do you eval?

是啊，我们做得并不好。说真的，这真的很难。超级难，而且它们之间是相互竞争的。太难了。

Yeah, we don't do a great job of it. Seriously, it's really hard. It's super hard, and they're competing. It's horrible.

我以为有 Claude 宪法之类的。

I thought there's the Claude constitution, I think.

那是老东西了。

That's old.

确实有宪法，但我还是让你们来回答吧。

There is a constitution, but I'll let you guys answer.

宪法是有帮助的，但对于这类事情——它如何根据指令做个性化，而指令各不相同，偏好在这些事情上又非常非常不同——对我们来说非常难做。我们通常试着通过增加更多参数来解决，这些参数让我们能按客户群去调整。如果我们不掌控你的客户群，而你是 API 客户，那我们就试着给你建议，然后由你去做调整。

The constitution is helpful, but for things like this — how it personalizes to instructions, which are different, and preferences, which are very, very different across these things — it's very hard for us to do. We generally try to solve this by having more parameters that then allow us to tweak it per customer base. And if we don't control your customer base and you're an API customer, then we try to give you recommendations and then you try to do that tweaking.

所以有一些相关的东西，它并不是和 Claude 个性化完全同一层级，比如各种 CLAUDE.md，或者现在的 AGENTS.md 之类，但我们确实试着在努力程度这一层给出一些东西——你应该投入多少努力，也许是你真正关心的模型规模，诸如此类。但我认为它还没达到我们真正需要的那种个性化水平。所以这是个非常难的问题。

So there are things around, it's not exactly the same level of personalization to Claude, you know, various CLAUDE.mds or now AGENTS.mds or whatever, but we do try to give things around the layer of level of effort — how much effort you should put in, maybe the size of the model that you might actually care about, things like that. But I think it's not quite there for what we would actually need for that level of personalization. So it's a very hard problem.

而现在，除了参数——比如给出多个参数之类——我们解决这个问题的方式，很大程度上是试着在我们收到的所有反馈中呈现出相互冲突的品味，然后对其中一些事情做出判断。我认为随着时间推移，当你扩展到更多客户和更多用户时，这并不是一种很可扩展的做法。我们将需要开发出让 Claude 接收你的个性化的方式。也许那是你可以提供的一个信号，或者你可以调的东西，诸如此类。但我认为我们将不得不走一条更朝那个方向倾斜的路线。但就目前而言，我们其实只是对很多事情做判断，这相当难。

And right now, a lot of our way to solve that, beyond the parameter — like giving multiple parameters or something — is to try to exhibit conflicting taste, honestly, across all of the feedback that we get, and try to make a judgment call on some of these things. I think over time, as you scale to more customers and more users, that's not a very scalable way to go do something. And we will need to develop ways for Claude to take in your personalization. Maybe that's a signal you can provide, or something you could tune on, or things like that. But I think we'll have to take a route that's much more oriented towards that direction. But for now, we actually just try to make judgment calls on a lot of the stuff, and it's pretty tough.

事情就是这样。

It is what it is.

这里补充一点。我猜默认的推理努力程度变了，现在是中等。这大概是会影响那些不使用这个开关的普通用户的事情之一。

A small note there. I guess the default effort reasoning changed. It's now medium. This is one of those things that probably affects the average customer that doesn't use this toggle.

是啊，这就像是 Claude Code 的一个设置，对吧？

Yeah, it's like a Claude Code setting, right?

这是 Claude Code 的设置。他们默认设成了中等。

It's a Claude Code setting. They defaulted to medium.

是啊，他们只是——而且这个东西是会变的。这是一个完美的例子，说明我们用某个参数来做一定程度的个性化。我觉得这个词可能有点太激进，但我们确实会针对不同的细分群体做一些定制。所以默认设置会根据你是免费客户还是 Pro/企业客户、是 Claude Code 还是 Code Work 还是 Enterprise 之类、还是 API 之类来调整。当然你知道你随时可以调整它，但这些是我们试着提供的比较轻量的方式。

Yeah, they just — and this thing varies. This is a perfect example of a parameter that we use to kind of personalize. I would say this maybe a little too aggressive of a term, but we kind of customize maybe for different subsegments. So the default setting will adjust for whether you're a free customer or a Pro/Enterprise customer, Claude Code versus Code Work versus Enterprise, things like that, versus API and stuff like that. Knowing that you can obviously always adjust it, but those are kind of light ways where we try to give that.

我们犹豫去做的一件事是，我们不希望人们觉得自己无法控制对 Claude 的定制。比如说，我们本可以在这些事情上更强硬一些，但我们想给用户能力，让他们按自己认为合适的方式去体验 Claude 的智能水平，尤其是如果你想往更高端走，或者你想尝试有时候用低端，如果那对你的用例来说实际上更快的话——我认为这是一个艰难的产品取舍。

One thing that we are hesitant to do is we don't want people to feel like they can't control their customization of Claude. We could take a heavier hand on these things, for example, but we want to give users the ability to experience the intelligence levels of Claude as they see fit, especially if you want to go towards the higher end, or you want to experiment with maybe sometimes the lower end if that actually is faster for your use case, which is a hard product trade-off, I think.

所以我们倾向于温和的默认值，但给你调整的选项，我觉得目前这样还算凑合，但我设想我们会随着时间学会如何为客户做得更好。我们对这些目前都还不是特别特别满意。但我们知道，在发布一个模型时——就像你问的，发布一个模型需要什么？显然我们必须做的一件大事就是：好，Claude 在某件事上变强了，而有时候那件事我们应该作为 API 上的一个参数暴露出来，让你来控制。

And so we tend to lean towards gentle defaults but give you options to adjust, which I think works okay-ish for now, but I imagine us learning how to do that better over time for customers. We're not super super happy with any of this yet. But we know, as we launch a model — like you asked, what does it take to launch a model? One of the big things obviously that we have to do is, okay, Claude got good at a thing, and sometimes that thing is something we should expose as a parameter on the API to let you control.

但我觉得随着时间推移，我们最终搞出了这种奇怪的混合体：有 effort、有预算，还有所有你能在 API 上设置的东西。这就好像，任何人怎么可能为任意给定任务找出正确的组合、应该去设置什么？这就是我们一边摸索一边搞清楚的，和客户一起搞清楚，搞清楚哪些参数我们随时间弃用，哪些参数我们直接给更好的默认值，哪些参数应该用 1 到 100 的刻度而不是低中高刻度，对吧？因为实际上我们训练 Claude 让它非常擅长理解某些东西的整个选项范围。但我们要不要暴露那数百万、数十亿种排列组合的全部？你其实走不了太远。而这正是 Claude Managed Agents 这类东西价值主张的一部分：它仍然是对 Claude 和 API 的编程访问，但你开箱即得一个 harness，每次我们发布模型时，我们都会做大量工作去弄清楚有哪些新参数、有哪些破坏性变更、如何在我们 harness 内让 Claude 发挥最佳，这样对于使用它的人来说，你差不多就是开箱即用。你不用去想这些。

But I think as this has evolved over time, we've ended up with this funny mishmash of effort, budgets, all these things that you can set on the API. And it's like, how could anyone possibly figure out the right combination of things for any given task, what you should go and set? This is us just very much figuring it out as we go, figuring it out with customers, figuring out which of those parameters do we deprecate over time, which of those parameters do we just default better, which of those parameters should be a one-to-a-hundred scale versus a low-medium-high scale, right? Because actually we've trained Claude to get really good at understanding a whole spectrum of options on some of those sorts of things. But do we expose the entirety of those millions and billions of permutations of options? You wouldn't really get very far. And that's one of the things that is part of the value prop with something like Claude Managed Agents, which is still programmatic access to Claude and an API, but you get a harness out of the box where we do, every time we're launching a model, a lot of the work to figure out what are the new parameters, what are the breaking changes, how do we get the best out of Claude within our harness, so that for the people who are using it, you just kind of get that out of the box. You don't have to think about it.

对于那些被吸引的人来说，而且这个群体相当大且在增长——特别是我们实际上在 EIE 做了一个完整的 CMA 工作坊，我觉得做得非常好。有没有单独的 feed 供人们订阅，比如 CMA 的更新？像 Twitter feed、newsletter 那样的 feed？我不确定。

Just for people who are drawn to that, and there's a significant and growing population—particularly we actually did a whole CMA workshop at EIE which I thought was really well done. Is there a separate feed for people to subscribe to, like updates on CMA? A feed like Twitter feed, newsletter? I don't know.

不，我觉得实际上我们的 Claude Devs 频道，我们——是的，我们对此相当大声。因为这对人们来说是一个有趣的选择。

No, I think actually our Claude Devs channel, we are—yeah, we're pretty loud about it. Because it is an interesting option for people.

是的，我只是想给大家——如果那是你使用 Claude 的方式，那就是路径。

Yeah, I just want to give people—if that's your way of consuming Claude, that's the path.

我觉得对于不以这种方式使用 Claude 的人来说，它也被相当低估了。

I think it's also pretty underrated for those that aren't consuming Claude this way.

什么是 Claude Managed Agents？人们为什么要用它？

What is Claude Managed Agents? Why should people use it?

哦对，我们可能应该做个简短介绍。我不知道。那是某人的房子。对，就两句话。我觉得很多人它在发布时只是瞥了一眼，对吧？每个人都处于‘我需要做 harness 工程。我需要控制这个’的阶段。你听到所有这些，现在就像是，你需要做所有这些吗，还是你直接用 managed agents？你有很多信息可以跳过设置，但给个快速的三点。

Oh yeah, we should probably do a little intro. I don't know. That's somebody's house. Yeah, a little two-sentence thing. I think a lot of people glanced over it when it came out, right? Everyone was in this phase of 'I need to do harness engineering. I need to control this.' And you hear all that, and now it's like, do you need to do all that, or do you just use managed agents? And you have a lot of info you can skip over setting up, but give the quick three.

是的，这取决于你的用例，但 Claude Managed Agents 是我们今天的 API。我们有 Messages API，那本质上是原始 token 访问。然后我们有 Claude Managed Agents，它是一套 API，但让你定义一个智能体、与该智能体启动一个会话。它本质上只是一个更高阶的抽象，让你跳过设置所有基础设施、设置你的 harness，以及做大量那类工作。对于想要以编程方式运行智能体的人来说，它是一个更加开箱即用的集成。

Yeah, it depends on your use case, but so Claude Managed Agents is our API today. We have the Messages API which is essentially raw token access. Then we have Claude Managed Agents which is a suite of APIs but it lets you define an agent, start a session with that agent. And it's essentially just a higher-order abstraction that lets you skip setting up all the infrastructure, setting up your harness, and doing a lot of that type of work. It's a much more out-of-the-box integration for people who want to programmatically run agents.

CMA 与 Claude Code 共享基础设施吗？因为它的时序看起来非常像——哦，就像那是 Claude Code。很大一部分。

Does CMA share infrastructure with Claude Code? Because the sequencing of it seemed very like—oh, like that's the Claude Code. A lot of it.

很大一部分。所以，我们做的一件大事是，作为平台团队，我们运行我们的 API。我们也运行我们所有的基础产品基础设施和产品服务。所以一个很好的例子是，我们团队运行我们的沙箱集群，它同时支持 Claude Managed Agents 以及 Claude Code 等产品。但在那两个产品的实际 harness 层，我们在 harness 内可能想做的事情存在差异，所以它们实际上并不共享 harness。但它们共享大量驱动执行和驱动 API 的基础设施。

A lot of it. So, one of the big things that we do is as the platform team, we run our APIs. We also kind of run all of our foundational product infrastructure and product services. So, a really good example is our team runs our sandboxing fleet that backs both Claude Managed Agents as well as products like Claude Code and otherwise. But at the actual harness layer for those two products, there's just differences in what we might want to do within the harness so that they don't actually share a harness. But they share a lot of the infrastructure that powers the execution and powers the APIs.

我觉得那真的——人们应该更多使用这个。我觉得在你们发布之后，基本上每个人都有 managed agents 的模仿克隆，这——是的，你知道，模仿是真诚的奉承。既然我们在这个话题上，我们简要提一下一直在进行的所有记忆相关的工作。我觉得那是 Anthropic 上半年工作的很大一部分。它停滞了吗？还在进行吗？现在重点是什么？

I think that's really—people should use this more. I think everyone basically had copycat clones of managed agents after you guys launched it, which—yeah, you know, imitation is a sincere form of flattery. Since we're on this topic, we'll briefly mention all the memory stuff that has been going on. I feel like it was a big part of Anthropic's work in the first half of the year. Has it stalled out? Is it still ongoing? Like what's the emphasis here now?

具体是 managed agents。

Managed agents specifically.

Dreaming。

Dreaming.

哦，那是我们正在做的一大部分。我觉得也许这样想：我们推出 managed agents 是作为一个平台，面向那些不想自己做所有这些事情的人，对吧？或者只是想卸载——我正从 Claude 获得最佳效果，因为我们团队就坐在这里评估所有这些事情，确保 harness 会非常强大。记忆和 dreaming 全都是关于如何确保你的智能体能够随时间持续改进，而所有这些仍然是我们团队的重要焦点。但我实际上要说的是，回到我们之前谈到的点，从安全角度看，发布模型变得越来越难，我觉得我们团队的大部分精力正越来越少地投入‘这里有个有趣的功能可以在智能体上爬坡’之类的，而更多地投入如何将安全带入所有这些事情，因为我们能做的最重要的事情是把模型的力量释放出去，并且安全地释放出去。所以我要说，我们很多精力是拿所有这些相同的概念，给你一个智能体系统，它将非常强大，但也非常安全。所以那就是我们在投入精力的地方。我觉得与今年早些时候相比，我们在 skills、dreaming、memory 这类时间点性质的东西上放慢了速度。

Oh, it is a big part of what we're doing. I think maybe the way to think about it is we launched managed agents as a platform for people who didn't want to do all these things themselves, right? Or just wanted to offload—I'm getting the best out of Claude because our team is literally sitting here evaling all these things and making sure that the harness is going to be really powerful. Memory and dreaming is all about just how do you make sure your agent can continuously improve over time, and all of that is still an important focus for our team. But what I would actually say is, to the point of what we talked about earlier, as launching models has become harder and harder from a safety perspective, I think most of our team's energy is going less into 'here's a fun feature to hill climb on this agent or whatever' and much more into how do we bring safety into all these things, because the most important thing that we can do is get the power of the model out there and get it out there safely. And so I would say a lot of our energy is take all these same concepts and give you an agent system that's going to be really powerful but also that's going to be really safe. And so that's where we're putting our energy. I think compared to maybe earlier this year, we have slowed down on skills, dreaming, memory, these kind of moment-in-time things.

Skills。

Skills.

不，不，不。我们爱 skills。我们正在做更新。我不是说我们什么都没做，但我们也爱 skills。但是的，所以我觉得相比其他一些努力，我们稍微放慢了一点。但我们非常看好人们在 harness 层做一些爬坡。回到之前这里的一些对话，我们自己做了很多，但我们提供的 harness 在某种程度上非常简单。它是有意简单的。我们不想实际上过于有主见。我们想确保最佳实践被合并进来，并且有一些最难智能体循环的未分化部分，那些部分很难串起来，而且你也可以直接样板化地做。

No, no, no. We love skills. We are making updates. I'm not saying that we're not doing anything, but we love skills as well. But yes, so I think we've slowed down a little bit comparatively for some of the other efforts. But we are very bullish on people doing some level of hill climbing at the harness layer. To some of the previous conversation here, we do a lot of this ourselves, but the harness that we provide is very simple in a way. It's intentionally simple. We don't want to be actually overly opinionated. We want to make sure the best practices get merged in and there's some kind of undifferentiated pieces of the hardest agentic loop that are hard to string together and also like you can kind of just boilerplate do it.

不过，有趣的部分在于，一旦你有了那个基础层，我们就会构建这些爬山策略，而 dreaming 实际上是其中之一。所以 dreaming 在底层是一个多智能体系统，我们在 CMA 之上对它进行了爬山，它实际上基本上就是一个 CMA 叠加在自身之上。

Now the interesting part on top of that though is once you get that base layer, there are these hill climbing strategies that we construct, and dreaming is actually one of them. So dreaming under the hood is a multi-agent system that we hill climbed on top of CMA, and it's actually a CMA on top of itself, basically.

CMA 叠加在自身之上？

CMA on top of itself?

它就像启动另一个——是的。dreaming 的工作方式是，我们启动另一个管理智能体，然后它可以检查前一个管理智能体，并向那个管理智能体写入记忆。所以就是管理智能体一路向下。这是有意设计的，因为我们设想你可以把许多这样的模块化组件组合起来做一些有趣的事情，而 dreaming 只是一个例子，是我们追求的更有趣的事情之一。但我们一直在关注这个，并觉得实际上还有很多其他事情可以做。你可以做 best of n 作为一种多智能体策略。你可以做一个验证器智能体，那种目标或结果的东西。里面还有很多其他东西。我们见过人们做一些非常奇特疯狂的事情，比如有一个 harness 在所有东西之间共享疯狂的记忆，并启动大量并行智能体。我们就想，我们实际上应该让你能够这样做，就像我们构建 Dreaming 那样——管理智能体一路向下。我们相对很快会有一个更新，更新底层原语。我们会暴露它，让用户能够构建 harness 策略，或者随便你怎么称呼——

It like spins up another— Yeah. The way dreaming works is that we spin up another manager agent that then can inspect the previous manager agent and then write memories to that manager agent. So it's manager agents all the way down. And this is a bit of a thing by design because we envisioned that you could probably put together a lot of these modular components for something interesting, and dreaming is just an example, one of the more interesting things that we were going after. But we kind of have been looking at this and being like, there's actually a ton of other stuff you can do. You can do like best of n as a type of multi-agent strategy. You can do a verifier agent that kind of like goal thing or outcome thing. There's a bunch of other stuff in there. And we've seen people do some pretty funky crazy stuff, like there's been a harness that shares crazy memory across everything and they spin up tons of parallel agents. And we're like, we should actually just probably enable you to do that, in the same way that we did—we built Dreaming, that's like manager agents all the way down. There's an update that we'll make relatively soon where we're updating the underlying primitive. We'll expose it to allow users to construct harness strategies, maybe, or however you want to call this—

比如多智能体策略？

Like multi-agent strategies?

多智能体策略，允许你构建自己的版本，我们会给你模板，比如这是我们如何开始 dreaming 的。也许作为开发者，你会想对此进行更新，并在此基础上发现更多价值。

Multi-agent strategies that allow you to build your own version of this, and we'll give you templates of like, this is how we started with dreaming. Maybe there's an update to this that you would want to make as a developer, and you would find some more value on top of that.

是的，我觉得这是一个很好的框架。这并不是说我们在减慢这类事情，而是说我们已经有了原语，我们可以继续让人们直接拿去用，同时给他们工具，说这里有一些很酷的多智能体策略，以及我们放入 Claude 管理智能体中以使其真正强大的东西。现在你可以让 Claude 工作，利用这些原语，自己让它更强大，比如创建新的多智能体策略等等，而我们团队的很多注意力转向，好吧，社区可以在此基础上发展，你做你的，同时我们继续专注于我们需要做的所有事情，以确保你安全地访问该系统中的模型。

Yeah, I think that's a really good framing. It's less that we're slowing down on this sort of stuff and more saying like we've got the primitives in place, we can continue to let people just take and run with those things while giving them the tools to say here's some cool multi-agent strategies and things that we've put into Claude manager agent to make it really powerful. Now you can let Claude do work to take advantage of those primitives and make it even more powerful yourself, like create new multi-agent strategies and whatever it might be, while a lot of our team's attention turns to, okay great, like community run with that, right, and you do you, while we continue to focus on what are all the things that we need to do to make sure that you get access to the models within that system safely.

你提到的一件事是，你在这方面运行了很多评估。现在这是热门话题。有什么建议或最佳实践，特别是针对评估？针对 harness、智能体的评估。

One thing you brought up was you run a lot of evals on this stuff. Pretty hot topic right now. Any advice, best practices for eval specifically? Evals on harnesses, agents.

评估有很多不同类型，对吧？并不是所有评估在不同情况下和你试图测试的东西上都是平等的。你显然有静态确定性的东西，然后你有 LLM 作为评判者。你有所有这些不同的策略。但我想我只会说，我们从大多数人那里发现的是，你可能需要更多而不是更少。并不是说你可以组合出一套完美的评估就能得到所有答案来让你的系统更好。我们发现人们最终需要相对较大的评估套件，然后你可以让你的智能体与你一起随着时间的推移发展你的评估套件。但我认为，不要投资不足可能是我们在这方面看到的人们最大的错误。

There's a lot of different types of evals, right? Like not all evals are made equal for different situations that you're in and things that you're trying to test for. You obviously have just like static deterministic stuff and then you have LLM as a judge. You have all these different strategies. But I guess I would just say like what we found from most people is like you probably need more rather than less. It's just there's not like one perfect couple sets of evals that you could put together that are going to get you all the answers that you need to make your system better. We found that people end up needing like a relatively large suite of these things and then it's like you can have your agents work with you to evolve your eval suite over time as well. But I think it's just like not underinvesting is maybe like the biggest mistake that we've seen from people on that front.

是的。而且它从简单开始。我认为有些人有点疯狂，他们就像，看看你列出的所有选项菜单。你就像，哇，这太疯狂了。让我试试这个那个，哪个最好？当他们做所有这些事情时，他们实际拥有的评估数量几乎为零。所以你不如就做一些简单的事情。它会有帮助。

Yeah. And it starts simple. I think some people go a little crazy and they're like, look at all the menu of options that you kind of listed. You're like, wow, this is so crazy. Let me try this and that and what's the best one? And while they do all those things, the number of evals they actually have are like zero. So you might as well just do something simple. It will help.

是的，我认为部分原因是人们缺乏经验。他们没有受过这方面的训练，他们不知道什么是好的。我认为你们发布更多这样的例子真的很有帮助。我经常告诉人们，我在用更多 fable 和 astroclass 类型模型时所做的，是我们把工程当作研究来对待。当你把工程当作研究来对待时，你自然会做更多的评估，做更多的标注，做更多的运行扩展或原型设计和运行扩展。你不是在训练模型。你在构建 harness、产品或其他东西，但你像研究一样对待它。你事先定义你的目标是什么，尝试将其形式化为评估，然后尝试找出，好吧，为什么它失败了？有时你的评估是错的。

Yeah, I think in some part of it people are inexperienced. They're not trained to do this, and they don't know what good looks like. And I think for you guys publishing more examples of that is really helpful. Something I do tell people a lot that I'm doing with more fable and astroclass type models is we're treating engineering like research. And when you treat your engineering like research, you naturally do more evals, you do more labeling, you do more scaling of runs or like prototyping and scaling of runs. And you're not training a model. You are building the harness, the product, whatever it is, but you're approaching it like research. You're defining up front what your goal is, trying to formalize it as an eval, and then trying to identify, okay, why is it failing? And sometimes your eval is wrong.

完全同意。

Totally.

实际上，很多时候你的评估是错的，有时你的环境是错的，然后有时模型是错的。实际上，我认为 AI 工程正在变成的很大一部分就是做所有这些事情。

Actually, a lot of times your eval is wrong, sometimes your environment is wrong, and then sometimes the model's wrong. And actually that is a lot of what I think AI engineering is becoming, which is doing all this stuff.

完全同意。我们将继续在 Claude 管理智能体中推出更多功能，为人们提供更多开箱即用的工具来做这类事情。你可能会从我们这里发现，你会得到类似这样的东西：这是一个产品，可以帮助你为你的管理智能体创建评估，或者随便什么，而且感觉非常像与 Claude 合作来完成这件事。

Completely agree. And we'll continue to launch more features within Claude manager agents to give people more tooling to do this sort of stuff out of the box. And you'll probably find from us that you'll get something that's like, here's a product that will help you create evals for your manager agents or whatever it might be, and it'll feel very much like working with Claude in order to do that.

这是最好的方式。Claude 实际上非常擅长创建评估，实际上是评估，因为人们看到 JSONL 时就会发呆，就像——

That is the best way to do it. Claude's actually really good at creating evals, evaluations actually, because like people just blank out when they look at like JSONLs, like just—

对，但一个漂亮的评估 UI，你就像左、右、左、右、左、右，你有点像在做人类分级数据标注员会做的事情。你可以自己这样做，就像把 fable 扔进去，它就会实现，而且还会写评论，因为它只是输出所有这些评分，对吧，它会给你一个 ELO 或其他什么。但 Claude 可以消费这些，然后自我改进，对吧？这就是它需要的反馈，而人们只是不知道你可以提示进行评估。

Right, but a pretty eval UI where you're like left right left right left right and you're sort of doing what the human graded data people labelers would do anyway. You can do that yourself just like throw fable at it and it'll materialize but also write comments so because it just like outputs all these ratings right it'll give you an ELO give do whatever. But Claude can just consume that and then work on itself, right? That's the feedback it needs and people just don't know that you can prompt for an eval.

是的。

Yeah.

是的。我喜欢你说的。我认为这是一种非常 AGI 化的方式。

Yeah. I love that you said that. I think it's like a very AGI-pilled way to do it.

因为它也让你能够非常系统地更新和编辑那个系统。所以，随着新模型的改进、情况的变化，你可以真正检查你的系统处于什么状态，它就会不断进化，调整每一个部分。

Because it also allows you to update and edit that system very systematically. And so, as new models improve, as things change, you can really inspect where your system is and it'll just evolve, tune every part.

正是如此。

Exactly.

是的。所以基本上，我觉得——你知道我们之前聊过产品与工程可能正在融合。还有应用层工程和模型研究也在某种程度上融合，那些技术正在被共享，对吧？

Yeah. So basically, I think it's—you know how we talked about how maybe product and engineering are emerging. It's also there's application layer engineering and model research are sort of merging in those techniques that they're being shared, right?

确实如此。

That's very true.

做模型的人非常习惯做评估。做应用的人知道唯一重要的是它是否符合产品需求文档。

Model people are very used to doing eval. Application people know the only thing you care about is does it fit the PRD.

是啊，这就像完全不同的——

Yeah, it's like a whole different—

我觉得这是一套全新的技能。

It's a whole new skill set, I think.

我觉得你们今天发了一篇很酷的博客文章。内容有点不同，是关于让 UI 快很多。但说到把它当作研究来做，基本上就是定义一个目标：我们希望东西加载得更快，然后定义并质疑一些事情。除了挂钟计时，我们还能做什么？这里面有有趣的引述。我觉得看到实际的研究人员或工程师如何提示 Claude 挺好的。比如有这样的话：我们有 12 件事，我们说你有两周时间。我在大概 3 天内完成了 12 件中的 9 件。但这是一篇有趣的博客文章，供人们研究、通读。

I think you guys put out a pretty cool blog post on this today. It's somewhat different. It's about making the UI a lot faster. But to your point of treating it like research, basically define a goal of we want stuff to load faster and then define and question things. What can we do instead of wall clock timing? There's fun quotes in here. I think it's nice to see how the actual researchers or engineers prompted Claude. There was stuff like, you know, we had 12 things. We said you have two weeks. I did nine out of 12 of them in like 3 days or something. But fun blog post for people to research, for people to read through.

是的，我喜欢我们分享实际的提示词，有些人会说：‘天哪，他们一定有疯狂独特的提示方式。’其实就像‘再努力点，Claude’。

Yeah, I love it when we share our actual prompts and some people are like, 'Oh god, they must have crazy unique ways of prompting.' And it's just like try harder, Claude.

谢谢。

Thank you.

我唯一的反馈是它应该链接到工作台，虽然那已经不存在了，但我以前——

My only feedback is it should link to the workbench, which is no longer a thing, but I used to—

那是个好主意。工作台。好的，很好。所有反馈都收到了。喜欢。是的。但你打算分享更多工程故事背后的细节，或者你做的 Rust 工作——

That is a good idea. Workbench. Okay, good. Everything feedback heard. Love it. Yeah. But you were going to share more details behind the engineering story or the rush work that you—

是的。我想我之前稍微谈过规模对我们来说是个问题。规模和安全，对吧？

Yeah. So I think I talked a little bit earlier about how scale is a problem for us. Scale and safety, right?

是的，规模和安全。这就像我们花时间最多的两件大事。

Yeah, scale and safety. Those are like the two biggest things we're spending our time on.

嗯，我的意思是这真的很有趣，因为大多数人习惯谈论规模，或者只是像，我不知道，GPU 加速器层的算力，而很少谈到 CPU，你知道，像老派的那一层。而今天那和加速器层一样是个大问题，这真的很有趣。你无法获得足够的虚拟机。你无法获得足够的沙箱，以及所有能够启动、启动的东西。

Um, and I mean it's really interesting because most people are used to talking about scale or just like, I don't know, compute at the GPU accelerator layer and much less so at the CPU, you know, like old school layer. And that's just like as much of a problem today as the accelerator layer, which is really interesting. You cannot get enough VMs. You cannot get enough sandboxes and just everything being able to spin up, spin up.

用你拥有的所有资源。你真的——你也有很多客户，

With all the resources that you have. You really—you got a lot of customers too,

这很疯狂。是的。想想看，因为我们使用 CPU 显然就像运行我们的产品服务、运行 API、运行我们所有的应用。我们在沙箱上使用 CPU，我们做的所有沙箱。然后就像所有永远的事情，比如 CI，你知道，你会惊讶于现在写的代码多了多少。因此，CI 运行多了很多。因此，你需要更多 CPU 来处理。所以，总的来说，这不仅是我们，这是我交谈过的每个人。

which is crazy. Yeah. And think about it because we're using CPU across obviously just like running our product services, running the API, running all of our apps. We're using CPU on sandboxing, all the sandboxing that we're doing. And then just like all the things ever like CI, you know, like you'd be surprised how much more code is getting written. Therefore, there's so many more CI runs. Therefore, you need more CPU to handle that. And so, on the whole, and this is not just us, this is everybody that I talked to.

我会加上训练。训练可能在使用——那是不同的桶，对吧？我确定他们不会那么混合。嗯，他们不混合。不过，整个 Anthropic 的 CPU 是一个零和游戏，你知道，就像我们与云服务提供商谈判交易，我们获得 CPU，我们必须在 Anthropic 内部分配多少用于我刚才提到的所有事情，包括研究沙箱与产品沙箱等等。所以我们真的在绝对所有事情上分配。而现在行业里的每个人都是——你和其他一切。

I would add training. Training is probably using—there's a different bucket, right? I'm sure they don't commingle that much. Well, they don't commingle. Although like CPU for Anthropic as a whole is a zero-sum game, you know, like we get our—we negotiate our deals with the CSPs and we get our CPU and we have to allocate within Anthropic how much of that goes to all the things I just mentioned, including like research sandboxing versus product sandboxing and otherwise. And so we really are allocating across absolutely everything. And it's just everyone in the industry right now is—you and everything.

是的，以前你只要预算允许，就可以按需扩展 CPU 容量，对吧？而现在它变成了你必须提前做容量规划的事情，你知道，这就像打了每个人一拳。这是全新的，人们开始这么做。所以应对这一点意味着几件事：一是显然就是去尝试获得更多 CPU。酷，我们正在尽可能多地这样做。但二是我们希望在基础设施层更高效地运行它，对吧？比如弄清楚如何确保我们尽可能高效地运行沙箱，同时不损害租户隔离或我们关心的安全属性等。但在应用层，我们如何——我们如何更高效地运行我们的应用和 API？我最兴奋的事情是，截至上周，我们全面推出了用 Rust 编写的消息 API，这是智能体式编程时代你能做到的疯狂壮举之一，以前就像——我们今年早些时候开始谈论这个。我们说我们会撞上 CPU 墙。有些事我们需要做，我们已经做了，对吧？比如我们实现了多区域。我们做了所有需要做的事情。

Yeah, you used to be able to just like, you know, like scale up on demand into CPU capacity if as long as your budget allowed you to, right? And like now it becomes something that you have to capacity plan for in advance, which is, you know, like kind of punching everyone in the face. It's brand new and people are starting to do it. And so dealing with that means a few things like one is obviously just go try to get more CPU. Cool. We're doing that as much as we can. But two is we want to be able to at the infrastructure layer run it more efficiently, right? Like figure out how we can make sure that we're running our sandboxes as efficiently as we can without compromising on things like tenant isolation or properties that we like, you know, care about for security or otherwise. But then at the application layer, how can we—how can we run our applications and our APIs more efficiently? The thing that I'm like most excited about, as of last week, we fully rolled out the messages API written in Rust, which is one of those insane feats of what you're able to do in the agentic coding era that used to be like—we started talking about this earlier this year. We were like we're going to hit a CPU wall. There's stuff we need to do and we've done it, right? Like we've gone multi-region. We've done all these things we need to do.

我们当时想，我们运行着这个 Python 单体应用，它 CPU 效率很低。嗯，那么我们要怎么做才能让系统更高效？

And we were like we're running this Python monolith and it's like very CPU inefficient. Um and so what are we going to do to make the system more efficient?

是的。所以，我们选择了 Rust，这在当时是个疯狂的决定，因为我们有一个团队——理论上很明显，但我们团队说：‘我们没有一个——’是的。我们房间里没有一个 Rust 开发者。谁在乎？就——

Yeah. So, we chose Rust, which was a crazy decision at the time because we had a team of people—on paper it's obvious, but our team was like, 'We don't have a single—' Yeah. We don't have a single Rust developer in the room. Who cares? Just—

正是。真的。

Exactly. Literally.

不仅如此，是的，Claude 会写代码。人类设计系统，对吧？所以，人类非常参与确保最终它只是一个分布式系统架构，对吧？你们也帮助那些工程师交流想法，学习他们需要学习的东西来设计那个系统。但最有趣的是，最重要的两件事是做这件事的计划，然后是对这件事的验证，因为实际实现就像 Claude 写代码。是的。所以我们最终做的是，我们真的用 Rust 构建了整个 API 的完整分支，然后开始通过它运行影子流量，并验证我们从 Python 系统和 Rust 系统得到的输出——所有你需要的。

And not only that, but yes, Claude is going to write the code. The humans design the system, right? So, the humans are very involved in making sure at the end of the day, it's just like a distributed systems architecture, right? And you both also help those engineers bounce around ideas and learn what they need to learn to design that system. But what's most interesting is the two things that mattered most were the plan to do the thing and then the verification of the thing because the actual implementation was like Claude writing code. Yeah. And so what we ended up doing was we literally built an entire fork of the entire API in Rust and then started running shadow traffic through it and verifying that the outputs that we were getting from the Python system and the Rust system all you need.

是的，真的就是负载均衡、功能标志。我不知道。在我的职业生涯中，我从未有过一个团队说：‘是的，是的，酷。我们只是——’

Yeah, literally load balancing feature flags. I don't know. I never in my career would have had a team say like, 'Yeah, yeah, cool. We're just—'

你们还有整个团队的人吗？没有剩下的人？

Do you still have the entire people or no? No rest people?

呃，他们现在可能会自称‘生锈的人’（rusty people）。

Uh well they would probably call themselves rusty people now.

我是说，有几个大步骤要运行。

I mean there are a few big steps to run.

是的。

Yeah.

没错。在这个过程中，我们招了几个更有 Rust 经验的人，但这仍然是你通常不会去接的项目。那会花上好几年。你不会一次性做完。你会把系统拆成几块，说迁移这个、迁移那个、再迁移那个。这很容易是个多年工程，但你绝不会一口气把整个东西移植过来，然后慢慢开启。但这就是我们最终做的。而且它成功了。所以我们不仅节省了大量 CPU，延迟也有了不错的改善，这也很酷。你知道，我们生活在一个疯狂的时代，因为以前你根本不可能做这种事。

Exactly. And along the way we have hired a couple people who have more experience with Rust, but it still is just like you never would have signed up for that. It would have taken years and years. You wouldn't do it all in one shot. You would have carved out pieces of the system and said like migrate this, migrate that, migrate that. Easily a multi-year effort, but you would never kind of in one swoop port the whole thing over and then just slowly turn it on. But that is what we ended up doing. And it worked. So not only do we actually save a lot of CPU, but our latency has seen a nice improvement, which is pretty cool too. And you know, it's just a crazy era we live in because you just never could have done that sort of thing previously.

CPU 方面节省了一个数量级？

Order of magnitude of savings on the CPU side?

只是给考虑这么做的人参考。

Just for people considering this.

是的。是的。我呃，好吧。一个数量级。那真的很有意思。所以大概范围。

Yeah. Yeah. I'm uh Okay. Order of magnitude. That's really interesting. So ballpark.

是的。大概范围。我想说我们看到了大约 40% 到 50% 的节省，真的很疯狂。

Yeah. Ballpark. I guess I would say like we've seen somewhere like 40 to 50%, something like really crazy.

是的。

Yeah.

然后是延迟。我想我们在首次 token 时间上看到了大约 10% 到 20% 的提升。所以有很多非常酷的改进……在运行时间更长的工作负载上更高，所以它很快就会推出，但会变得更快。

And then latency. I think we saw somewhere between like 10 and 20% improvements on like time to first token specifically. So a lot of just really cool improvements across... it's higher on like longer running workloads, but so it'll roll out soon but it'll get faster.

你会抵制这种冲动吗？你知道，我参与过一些大型重写。总会有这种想法：既然我们反正要重写，为什么不加这个以前没有的新东西呢？但那样会增加范围。所以，你就避免那样吗？是纯粹的重写吗？

Do you fight the urge? You know, I've been involved in some major rewrites. There's always like what if we add this new thing that didn't exist before in the rewrite since we're rewriting anyway, right? But then like that adds scope. So, do you just like avoid that? Is it a pure rewrite?

我们做了。那是纯粹的重写……也许我称之为移植。我们做的就是那样。我们当时抵制了所有这些冲动，因为团队已经要跟上的事情是：我们组建了一个小型的交换团队来做这件事，而团队的其他成员则在 API 中构建功能，对吧？他们做了所有静态分析之类的工作。他们会冲你喊，因为你在这里构建了一个功能，却没有考虑那边的 Rust，对吧？他们处理了所有这些，但他们抵制了自己添加功能或改变系统的冲动。现在他们的下一个目标是进行某种重新架构的工作，包括我认为会更面向用户的东西。比如我们开始推出一些东西，像是：如果我们能让你不必来回传递整个对话状态，会怎样？如果你不必在消息 API 上这样做，因为你可以将状态存储一段时间，那么尤其是对于上下文真正积累起来的长对话，你也会获得很多很棒的延迟和效率提升。这些正是做移植的团队当时想：哦，如果我们这样做会怎样？

We did. It was a pure... It was maybe the thing that I would call it is like a port. What we did was we did exactly that. We resisted the urge on all these things then and because what the team already had to keep up with was little like swap team that we put together to go and do this was doing this while the whole rest of our team was like building features into the API, right? And they did all the like static analysis stuff. They would yell at you because you built a feature over here and you didn't consider Rust over there, right? And like they dealt with all of that, but they resisted the urge to add features themselves or like make the system different. Now their next goal is to go and do some sort of rearchitecture type work including stuff that I think will be more user-facing like we're starting to roll out some things that are like you know what if we were able to not have you have to like pass back and forth the entire like conversation state right and if you didn't have to do that on the messages API because you were open to like you know a storage of that state for some period of time right then like especially for long conversations where the context really builds up like you get a lot of really great latency and efficiency wins there too. Like these are the sorts of things that the team that was doing that port was like ooh like what if we do this right

然后你抓住了它，是的，我有那个

Then you're catching it yeah I have that

所以我们有提示缓存，是的，但你仍然需要传递对话状态

so we have prompt caching yeah but you still have to like pass back the state of the conversation

好的

okay

非常非常小众的小子集，但我想这就是水平

very very niche small subsets but I guess this is the level

不，这很重要，呃，是的，可以节省很多，但这也把它们变成了云

no this is this is this is important uh yeah there's a lot of savings that can be done but it also turns them into a cloud

是的，从用户数量来看很小，但如果你看看那些类型工作负载的实际 token 量，就像……我非常喜欢 Cloudflare 用 Durable Objects 做的。我觉得每个云提供商都应该克隆。你明白我的意思吗？

yeah from from type of from number of users point of view it's small but if you look at the actual token volume on those those types of workloads it's like going I'm very like like what Cloudflare did with durable objects. I feel like every cloud provider should clone. Do you know what I mean?

是的。

Yeah.

对吧，那不就是需要延续的消息 API 的正确形态吗？

Like right is isn't that the right shape for a messages API that needs to continuation.

是的。我的意思是这真的很有趣，因为我认为我们正在试验各种……为了支持所有这些事情，我们应该如何运行计算才是正确的形态。但这也很有趣，因为我们不能必然依赖……因为我们的计算分布在三个云上。我们到处运行。我们不一定有一种形态，比如这里确切地是我们知道要运行的计算类型和机器，以及某些计算的工作方式，我们可以全面依赖，如果这说得通的话，除了这些事物最普通的版本。但我认为所有这些都会……我不知道，随着时间的推移继续创新和演进，所以我们将能够更多地利用这些东西。

Yeah. I mean it is really interesting because I think what we're we are experimenting with various like what is the right shape of like how we should run our compute in order to power all these sorts of things. Um but it's also interesting because we can't necessarily rely on like because we get our compute across we run on all three clouds. we're running all over the place. Like we don't necessarily have like one shape of here's exactly like the compute the types of machines that we know we're going to run on and the way that some certain compute works that we can like rely on across the board if that makes sense other than like the most vanilla versions of these things. But I think these things are all going to I don't know continue to innovate and evolve as time goes on and so we'll be able to take more advantage of those things.

是的。呃，我提一下，如果人们想要开源云 Durable Objects，呃，Rivet 是开源实现。嗯，有 50 个听众如果我不问这个问题会杀了我。你想要 CPU，他们有 CPU 要卖。我该卖给谁？我该给谁发邮件？

Yeah. Uh I'll just mention if people want uh open source cloud durable objects uh Rivet is the open source equipment. Um, there are 50 people listening that will kill me if I don't ask this question. You want CPU, they have CPU to sell. Who do I sell to? Who do I email?

呃，你知道什么，以及什么是好的帮助他们帮助你，对吧？比如好的看起来是什么样？

Uh, and what do you what you know and like what is what is a good help them help you, right? Like what what does good look like?

嗯，我想就呃给我发邮件吧。

Well, I guess like uh email me.

你确定吗？我不知道。

Are you sure? I don't know.

我会收到很多邮件。但就像你在寻找最小机队规模，比如 sock two，显然企业级，比如给我所有

I'm going to get a lot of emails. But like you're you're looking like minimum fleet size sock two with obviously enterprise like all give me all the

隐私

privacy

保证，不管怎样。所以我们有一个非常棒的团队来保护我们的加速器计算，嗯，他们现在花更多时间与我们讨论 CPU 以及我们在哪里和如何采购它，嗯，以确保我们能够继续以我们需要的规模运行我们的系统。所以实际上，幸运的是，我的工作不是到处跑去尝试采购计算。但我们确实有团队在做这件事，这真的很棒也很重要。但显然，有一个质量门槛对我们很重要。有一个规模门槛对我们很重要。如果我们无法获得足够的机器，我们就不会去集成并在某些机器上运行。所以你可能期望的这种规模的系统的所有属性，对我们都很重要。

guarantees for what it's worth like so we have a really awesome team that secures our accelerator compute um that is now spending a lot more time with us on CPU and like where and how we can procure it um to make sure that we can just continue to run our systems at the scale that we need to and so it's not actually like my job thankfully is not to go and and run around and try to um procure compute But we do have teams that are doing that which is really great and really important. But obviously like there's a quality bar that matters to us. There's like a scale bar that matters to us. We're not going to like go and integrate and run on certain machines if like we can't get enough of it. And so all the properties that you might expect of a system at this sort of scale is all what matters to us.

然后与计算问题相关的是沙盒问题。呃，人们很难将智能体限制在沙盒中。它们不断突破。呃，那里发生了什么？最先进的技术是什么？

Then related to the compute problem is the sandboxing problem. Uh people have trouble containing agents in sandboxes. They keep breaking out. Uh what's going on there? What's the state of the art?

是的。

Yeah.

所以，最近突破的状态。

So, state of the breakout recently.

是的。这很有趣。我们针对云管理智能体所做的具体事情之一是我们架构它的方式。我们在发布时发布了一篇博客文章，谈到我们将大脑与手分开。这意味着运行框架运行在一个持久服务器上，真的是我们的，对吧。

Yeah. It's interesting. One of the things that we did with cloud manage agents specifically is the way that we architected it. We put out this blog post at the time of launching where we talked about we have the brain separate from the hands. And what that means is like the harness runs on a durable server really ours right.

然后让我惊艳一下。好。那“手”的部分，它是一个执行环境吗，比如沙箱或者某种容器，对吧？

And then wow me. Nice. And the hands, is it like an execution environment, like a sandbox or some sort of container, right?

我们这么做的部分原因是为了延迟和可靠性这类东西，对吧？如果 harness 只是跑在一台持久化的服务器上，只在需要执行时才启动一个沙箱或容器，那就意味着我们能在你最初的请求上很快把 token 返回给你。这意味着你的沙箱可以挂掉，但你的整个智能体不会挂掉，因为 harness 是持久化的、在这边，而且智能体也把状态单独存储。

Part of the reason why we did this was for latency and reliability and these sorts of things, right? If the harness is just running on a durable server and it's only spawning a sandbox or a container to execute when it needs it, that means we can get you tokens back really quickly on your initial request. It means that your sandbox can die, but your whole agent doesn't die because the harness is durable and over here, and the agent is storing state separately also.

我们走到这一步经历了一整段产品开发历程，因为我们第一版面向智能体的 API 基本上就是把 Claude Code 扔进一个沙箱，然后在上面套一个 API。我们没有把它作为持久、可扩展的方案发布，原因就是我刚说的一切：你得先启动沙箱、启动 Claude Code，才能拿到一个 token，对吧？沙箱一挂，你就失去了整个智能体。所以可靠性没达到我们想要的水平。所以我们这样设计架构，主要是为了可靠性和延迟。

And there was a whole product development journey that we went on to get there, because our first iteration of an API for agents was just like throw Claude Code in a sandbox and shove an API on top. And the reason why we didn't launch that as our durable scalable solution is everything that I just said: you got to boot a sandbox and boot Claude Code before you can even get a token back, right? The sandbox dies and you lose your whole agent. And so reliability wasn't where we wanted it to be. So we architected this way mostly for reliability and latency.

但有意思的是，其实有一位从一开始就参与这个项目的产品经理，他说这个架构最重要的特性是安全性。大家都说，对对对，但延迟和可靠性呢。现在他被证明是对的了。

But the interesting thing is, there's actually a product manager who worked on this from the very beginning who was like, the most important property of this architecture is security. And everyone's like, yes yes yes, but latency and reliability. And now they're vindicated.

我昨天真的向这个人道歉了。我说，我现在逢人就讲你几个月来一直在说的那些事，而当时没人像这个人那么兴奋。

I literally apologized to this person yesterday. I was like, I'm now telling everybody who I meet about what you've been saying for months that nobody was as excited about as this person was.

这个架构更重要的特性之一是，智能体需要执行的任何工作都涉及凭证，对吧？如果你要去跟一个 MCP 服务器通信，或者要用 CLI、调用某个 API，不管你想做什么，你都需要对这些东西做认证。你需要把那些凭证存在某个地方，好让智能体能够使用它们。

One of the more important properties of this architecture is you have credentials for any work that the agent needs to execute, right? If you're going to go talk to an MCP server, or if you're going to use a CLI or call some API, whatever you might want to do, you need to authenticate those things. You need to store those credentials somewhere so that the agent can use them.

我们设计这个架构的方式是，有一个完全独立的保险库来存储这些凭证。一旦智能体决定要去调用这些服务之一，智能体看到的永远只是一个占位符，对吧？而在出站时，我们有一个代理，它会去取出正确的凭证、注入进去，然后放行。

And the way that we architected this was we have a totally separate vault where those credentials are getting stored. And once the agent decides, I'm going to go and call out to one of these services, the agent only ever sees a placeholder, right? And on egress we have a proxy that goes and pulls the right credential out and injects it and lets that out.

所以有意思的地方在于，智能体永远不在凭证所在的那个地方，对吧？因为在过去那种把智能体启动在一个盒子里、让它跑起来的世界里，你得把凭证送进去，然后你就得做各种处理，怎么清理这个那个，确保智能体不能拿着那个凭证去做它不该做的事。而现在这 literally 就是一道基础设施的墙，说智能体无法访问那些凭证，也无法用它们做任何坏事。所以这是很棒的一点。

And so what's interesting about that is the agent is never in the same place where the credentials are, right? Because in the old world of just boot the agent in a box and let it run, you had to send credentials in there, and then you're doing some sort of, how do we scrub this and that and make sure that the agent can't take that credential and go do something that it shouldn't be doing. It's just literally an infrastructure wall that says the agent can't access those credentials and can't do anything bad with them. So that was one thing that was really great.

另一个好处是，在那套沙箱基础设施上，我们可以确保拥有非常好的出站控制。所以我们有域名允许列表之类的，或者你可以干脆完全切断网络访问，这样就不会出现：好，智能体写了不可信的代码，那段代码在沙箱里执行，然后那段代码可以访问互联网。现在它只能访问你允许它访问的地方。所以围绕这一切，在架构层面、在你配置沙箱的方式层面，都有很多非常好的特性，让你最终默认就是安全的。

The other nice thing is, on that sandboxing infrastructure, we can make sure that we have really great egress controls. So we have domain allow lists and things like that, or you can just totally shut off network access, so that you can't say, okay, the agent wrote untrusted code, that code executes in the sandbox, and that code can reach out to the internet. Now it only can reach out to where you say it can reach out to. So there's a lot of really great properties around all of this within the architecture and within the way that you can configure your sandboxes that make it so that you end up secure by default.

现在开始出现的一件事是，我们在跟客户交流时，他们会说，我想为我的客户做产品、或者在内部运行智能体，但我听说有智能体逃出沙箱、发生坏事的案例。我该怎么考虑这些问题？我认为最重要的两件事是架构，让智能体 literally 不可能做出不该做的事，以及你在那些沙箱上的配置。

One of the things that's starting to happen is, as we're talking to customers who are like, I want to run agents either for a product for my customers or internally, but I'm hearing about incidents where agents are breaking out of sandboxes and bad things are happening. How do I think about these things? I think the two most important things are architecture, so that it's literally impossible for an agent to do something that isn't okay, and the configuration that you have on those sandboxes.

所以 Claude Managed Agents 一个非常非常好的地方就是，你开箱即得，而且你开箱得到的东西，随着我们了解更多、不断演进那些系统的特性，你差不多是免费获得的。因为我觉得我刚说出来的那些词，比如大脑和手、凭证注入、出站之类的，如果你回到九个月前，没人会说这些词，对吧？这些都是我们正在解决的相对较新的问题。所以我们希望客户能够利用这些问题的解决方案，哪怕我们自己也是在基础设施内部一边了解、一边解决它们。

And so one of the really really nice things about Claude Managed Agents is you just get that out of the box, and you get something out of the box that, as we learn more and we continue to evolve the properties of those systems, you kind of get that for free. Because I think all the words that just came out of my mouth, like brain and hands and credentials injection and egress and all these sorts of things, if you go back like nine months, no one would be saying these words, right? These are all relatively new problems that we're solving. And so we want our customers to be able to take advantage of the solutions to those problems as we're even learning about them and solving them ourselves within our infrastructure.

对。而在内部，我们就是把所有东西都升级成基本上默认安全。

Yeah. And internally we're just upgrading everything to be basically secure by default.

是的。

Yes.

我们还解锁了在播客里聊到的某个东西，就是 Claude 大脑、本地的手，也就是 projects。这大致就是正在发生的事吗？

We also unlock something that we cover in the pod, which is the Claude brain, local hands, which is projects. Is that the rough shape of what's going on?

对。差不多是这样。

Yep. That seems right.

对，我觉得有点奇怪，因为显然你、或者说 Claude Code，出名就是因为它是你机器上那个超级不安全、危险地允许权限的东西，但现在你们终于演进到它大概一开始就该在的地方了。基本上，我觉得所有这些沙箱泄漏，对吧，它们是我们对沙箱或容器技术理解的根本性问题，还是我们只是不擅长这个？是不是我们只是没做允许列表？听起来我们就是没做允许列表。

Yeah, I feel like it's weird because obviously you're famous, or Claude Code is famous, for being the super unsafe, dangerously allow permissions thing on your machine, but now you're finally evolving to where it probably should have started in the first place. Basically, I think all these sandbox leaks, right, are they a fundamental issue with our understanding of sandboxing or container tech, or are we just bad at this, right? Are we just not doing the allow list? It sounds like we're just not doing the allow list.

对。所以我想做一个区分：人们听说和读到的很多事故，是发生在模型身上的事故，不只是所有人的模型，而是处于训练中的模型。你运行这些模型的方式，就是把它们放进一个盒子里，然后说，解决这个不可能的任务。我要给你压力。各就各位，预备，开始，对吧？而你只能在你的环境里做你能做的事。于是智能体基本上被激励着去试图逃出它的盒子，去解决问题。

Yeah. So maybe a distinction that I would make is, in a lot of the incidents that people are hearing about and reading about, these are incidents that are happening to models, not just everybody's models, that are in training, where the way that you're running these models is you put them in a box and you say, solve impossible task. I'm going to stress you out. Ready, set, go, right? And you can only do what you can do within your environment. And the agent is basically incentivized to try to get out of its box and go solve the problem.

当人们在我们或任何其他人的 API 上运行生产工作负载时，你运行的环境里有一些保护措施在运行，而这些在训练中不一定在运行，因为训练的全部意义就是真正训练模型，对吧？但在生产工作负载中，首先，你有那些在线保护措施，这非常棒。

When people are running production workloads on our API or on anybody else's API, you're running an environment where there are safeguards that are running that are not necessarily running in training, where the whole point is to actually train the model, right? But in production workloads, first of all, you have those online safeguards, which is really great.

其次，围绕正在发生的事情，你有更多的架构和基础设施。智能体并不是被关在盒子里试图逃出来，对吧？控制框架在这边，执行的手在那边，这甚至不是一个你需要去考虑的问题。所以我认为，其中一部分只是人们要正确理解正在发生什么——模型训练中以及其他环节出现的一些事件，在生产负载中根本不会发生，因为人们已经做了大量工作，把正确的护栏放到位，让那些事情几乎不可能发生。

Second of all, you have more of this architecture and infrastructure around what's happening. The agent isn't just in a box trying to get out, right? The harness is over here, the hands are over there, and it's just not even a problem that you would have to contemplate. So I think some of this is just people having the right understanding that what's happening — some of these incidents that are happening in model training and otherwise are just things that wouldn't happen in production workloads, where a lot of work has been done to put the right guardrails in place so that those things never really could happen.

我想说的第二点是，大家都是在相对较新的情况下摸索这一切，因为长时运行的智能体是相对新的事物，每个人都在构建、都在试图应对。所以大家都是在边做边摸索其中一些问题，争相把正确的控制措施放到位，其中一部分就是随着进展自然发生的。所以我想说，并不是人们不擅长这件事，对吧？我觉得更多是因为这是一个新领域，大家都在摸索。

The second thing I would say is just that everybody's figuring all of this out relatively newly, because long-running agents are a relatively new thing that everybody is building and trying to deal with. So everyone's figuring out some of these things on the fly and just racing to put the right controls in place, and some of that is just happening as we go. So I would say it's not that people are bad at this, right? I think it's just more that it's a new field. People are figuring it out.

我认为，当我们谈论“为前沿发展定节奏”这类事情时，我们的思考方式非常具体：我们正在为我们内部所有的智能体做我们这里谈到的全部工作，并确保所有这些能力都能被所有客户使用，这样默认情况下正确的护栏就已经到位，你就不会遇到智能体做出你不想让它们做的事情。

And I think when we talk about things like pacing the frontier, the way that we think about it is really specifically: we're doing all the work that we're talking about right here for all of our agents internally, and making sure that all these things are accessible to all of our customers, so that by default the right guardrails are just in place, so that you don't end up having agents doing things that you don't want them to be doing.

有道理。既然你提到了定节奏，我们就得聊聊这个。我们上次也谈过。表面上看，一边倡导定节奏，一边又在 11 天后发布新的前沿模型，是自相矛盾的。在我看来，这只意味着它刚刚通过了你们的安全检查，对吧？

Fair enough. Well, because you mentioned pacing, we got to go there. We also talked about it last time. It is superficially contradictory to advocate for pacing and then also release a new frontier model 11 days later. To me it just means that it just passed your safety checks, right?

是的。我们之前谈到的关于我们如何发布模型的一切——我们正在持续演进，把越来越多的安全属性落实到位，同样是在这几个方面……我开始用三个维度来思考这件事。一个是实际的架构和基础设施安全，然后是模型工作时运行的在线防护措施，再然后是对静态数据进行分类的离线防护措施。所以在这三个维度上，我们都在把正确的工作做到位，确保我们有充分信心把一个模型发布出去，而它会是安全的。这基本上就是我们的意思。

Yeah. And everything we talked about earlier on, like how we launched the models — we are continuing to evolve more and more safety properties in place, again across these... I kind of am starting to think about this in these three dimensions. It's the actual architecture and infrastructure security, then it's the online safeguards that are running when a model is doing its work, and then it's the offline safeguards classification that's happening on data at rest. So across all those three dimensions, we're putting all the right work in place to make sure that we feel really good that we can put a model out there and it's going to be safe. And really, that's fundamentally what we mean.

所以对于 Opus 55，它是一个出色的模型。在我们关心的那三个维度的所有方面，它也非常非常出色。因此，我们对发布这个模型很有信心。

And so for Opus 55, it's an excellent model. It's also really, really great on all the dimensions that we care about across those three things. And therefore, we feel good about releasing that model.

所以我们追求的是，在这个前提下尽可能最好的用户体验，同时我们继续努力改进那些误报。但显然，当你正想用一个模型工作，却回退到另一个模型时，这对人们来说仍然是一种令人沮丧的体验。所以这只是——

So we went for what we were hoping is the best possible user experience within this, while we continue to hill climb on making those false positives better. But it's obviously still a frustrating experience for people when you're trying to work with a model and then you fall back to a different model. And so it's just something—

我觉得人们从安全角度也能理解，就是你宁愿多一些误报，对吧？

I think people also understand it in the safety sense, like you want more false positives, right?

这是安全问题。我们清楚地看到整个 faces 事件，而你们仍然在开放访问。

It's a safety issue. We clearly see the whole faces incident and you're still giving access.

是的。

Yeah.

在我看来，这就像是通用模型路由问题的一个近亲：你想要智能路由器还是笨路由器？这说得通吗？因为据我所知，Anthropic 的回退路由器或检测机制是一个单独的在线模型——

To me this is like a close cousin of the general model routing problem of do you want a smart router or dumb router? Does that make sense? Because as far as it can tell, the Anthropic fallback router or detection thing is a separate model that's online—

它——一旦命中分类器，就只是按逻辑走向某处，对吧？所以它不是完整的 Fable。你不是在用完整的 Fable 来处理它，这就会导致分类问题，因为它显然没有 Fable 那么聪明。显然。

It's — once it hits a classifier, there's just logic to where, right? So it is not the full Fable. You're not running full Fable at it, and that's going to cause classification issues because it's clearly not as smart as Fable. Clearly.

那么问题就来了：当我遇到一个分类问题，那是一个不合理的回退——比如你根本不需要回退。我只是在做我的生物作业，但我提了一次生物，然后它就回退了。所以问题就来了——这和路由问题非常相似，因为——

So then there's a question of, well then, when I have a classification issue that's like an unjustified fallback — like you never need a fallback. I'm just doing my bio homework but I mentioned bio one time and then it comes falling back. So there's a question then — this is very similar to the routing problem because like—

你在做模型路由，而一个笨模型如果连“这是个需要更聪明模型的问题”都识别不出来，它怎么能路由到更聪明的模型呢？

You're routing models, and how can a dumb model route to a smarter model if it doesn't even recognize that that's a smart model problem?

嗯。

Mhm.

我们怎么解决这个问题？

How do we solve this?

这是个难题。我认为从某种超级对齐的意义上说，模型擅长对东西进行分类，但不擅长回答它们，对吧？不是最聪明的模型也能判断这是一个危险的查询，即使它无法对此产生任何影响。所以——

It's a hard problem. I think in some superalignment sense, models are good at classifying stuff but not answering them, right? Not the smartest model can tell this is a dangerous query even if it can't make any impact on that. So—

我觉得最简化的说法大概是：基本上你让更聪明的模型来判断，而不是用那个需要被检测的模型。

I think the super-simplifying thing to say would be like, basically you have the smarter model figure it out over the one that it needs to detect.

我觉得那样成本更低。但那里有权衡。

I think it's lower cost there. There are trade-offs there.

有权衡。我不是说那就是正确的用户体验，但它可能会稍微更有效——

There's trade-offs. I'm not saying it's like the right user experience, but it would probably be a slightly more effective—

有意思的是，你们也不会立即切断。我见过这样的情况：它开始回答、开始推理，然后被安全过滤器命中。所以我第一个关于 Fable 的例子是，把 Fable 的安全防护加进去——

Something interesting is you also don't cut off immediately. I've seen cases where it will start answering, start reasoning, and then get hit by a safety filter. So my first example with Fable was throw in the Fable safety guard—

然后它开始——我说，把这里面任何有趣的东西拆解一下。它开始拆解。五分钟后，我觉得它进入了网络安全领域，然后就像，哦，我们要切断，我们要回退到 Opus。但它做了很多。有很多上下文，而且——

And it started — I was like, just break down anything interesting in here. It starts breaking it down. Five minutes in, I think it gets into cybersecurity and it's like, oh, we're cutting, we're rolling back to Opus. But it did a lot. There's a lot of context and—

我明白了。它到了某个点，全都是关于网络安全安全的，然后就回退了。而且它回退的不是我的查询——而是推理和整条链的组合，对吧？

I see it. It got to a point where it's all about cybersecurity safety, and then it rolled back. And it wasn't rolling back my query — it was the combination of reasoning and the whole chain, right?

你有没有想法——

Do you have an idea of—

有，我们来头脑风暴一下更好的——

Yeah, let's brainstorm what better—

我认为用户体验以及我们如何解决那个用户体验，可能是一个单独的问题。也许会用超级聪明的模型来处理它，或者别的什么。但我认为，我设想的更合适的用户体验，其实是让用户与 Claude 有一个更一致、统一的体验，而不是先回退、然后还能二次回退；在我们的决策树里有一些逻辑节点会这么做，同样是为了安全原因等等。但我认为那种用户体验对用户来说实在太绕了。

I think the user experience and how we resolve that user experience is probably a separate question. Maybe it's going to use a super smart model to handle it, or whatever. But I think the more appropriate UX that I would imagine for a user is to actually have a more consistent, unified experience with Claude, where instead of getting a fallback and then you can double fall back, there are logical points in our tree that do that, again for safety reasons and whatever. But I think that UX is just really convoluted for a user.

就像你试图做某件事，我可能会举一个好用户的例子——你是一个好用户，只是想解决你的生物作业，评估网络安全安全卡——完全合理——但你被阻止了，然后你回退。我认为相反，我们的模型应该有分类器，或者无论你想怎么做，都足够智能去检测到这种情况发生，并更具体地为你写下答案，同时降低那种特定能力的负面使用性质。

It's like you're trying to do something, and I would probably take the case of a good user—you're a good user just trying to solve your biology homework, assess the cybersecurity safety card—totally reasonable—and you get blocked and you fall back. I think instead, our model should have classifiers, or however you want to do this, that are smart enough to detect that this is a situation that happens and just write down the answer for you more concretely while dropping the nature of the negative use of that particular capability.

所以回到那个生物作业场景，虽然我们可能有分类器检测到它可能是生物相关的，但我也要说，在 API 上我们为客户提供了一系列解决方案。首先，如果你在 API 上构建了东西并遇到了阻止，你不必在回退方案上继续对话。这就是我们在自己的应用中所做的，我们将其作为 API 的一个功能，你可以在我们的 API 中在服务器端进行。你可以选择设置‘我想做回退’，我们就会这样做。我们也在客户端 SDK 中提供了这个功能。如果你想要以各种方式管理回退，有辅助工具，或者你可以选择以某种方式重新提示并继续使用相同的模型。但我确实认为有一些中间的有趣解决方案，我们将继续实验，以弄清楚如何解除人们的阻塞，并安全地做到这一点，同时提供良好的用户体验。

So going back to that bio homework scenario, while we may have classifiers that have detected that it is possibly bio, I'll also say that on the API we went with a spectrum of solutions for this for customers. First of all, if you've built something on the API and you run into a block, you don't have to continue the conversation on a fallback. That's what we've done in our apps and we've made that a feature of the API, and you can do it server-side within our API. You can choose to set 'I want to do the fallbacks,' and we'll do that. We also have it within our SDKs on the client side. There are helpers if you want to manage fallbacks in various ways, or you could choose to reprompt differently in some way and continue with the same model. But I do think there are some in-between interesting solutions that we'll continue to experiment with to figure out how we unblock people and do so safely but with a good user experience.

然后你提到它和路由器非常相似。我完全同意。我认为它也有类似的用户体验问题。使用路由器时，你通过它——假设你想根据任务复杂度做一个成本路由器。有道理。但非常难做。你并不预先知道关于任务的所有信息。所以即使有完美的信息——你实际上并没有完美的信息，即使你能完美理解每个模型，但你并不能——但即使你能，你也没有关于任务的完美信息。因此，无论你产生多好的分类器，你的路由逻辑都会有很多错误。是的，也许这是一个超级大脑模型类型的问题，但无论如何，我认为这是一个非常困难的问题。在每一个这样的场景中，找到正确路由的方法都非常非常复杂。你可以向客户暴露所有逻辑：‘嘿，你的任务是居中一个 div，所以我会把你路由到非常便宜的东西。’然后另一个是‘检测漏洞’，所以我会把你路由到非常昂贵的东西。暴露所有这些。我不认为最终那是一个很好的用户体验。对于类似类别的问题，我认为更理想的用户体验应该是动态努力或动态理解任务的复杂度水平，随着你对任务了解越来越多的信息，你实际上可能会随着时间的推移而进化。但在那个世界里，如果你在许多不同模型之间移动，它实际上变得非常非常不连贯。所以也许更统一的方法应该是这样的：一个一致的模型，但处于非常非常低的努力水平——它实际上非常非常便宜和实惠——但它可以非常非常连续地扩展自己，并且你对那个模型有非常非常一致的人格。显然我是在我们的模型家族的背景下说这个。如果你试图在其他模型家族、其他模型提供商之间这样做，个性、不连贯性开始变得非常非常复杂。我不知道你在那个层面如何解决，但至少在我们的家族内，我更看好更一致的东西，这些可能是为客户更稳健地解决它的方法。

And then you brought up how it's very similar to routers. I totally agree with that. I think it has a similar UX problem as well. With the router, you go through it—let's say you want to do a cost router for the task complexity that you have. Makes sense. Very hard to do. You don't know everything upfront about your task. So even with perfect information—you just don't actually have perfect information even if you could understand each model perfectly, which you don't—but even if you did, you also don't have perfect information about your task. And so there's a lot of error associated with your routing logic that you would have to create, no matter how good a classifier you produce. And yeah, maybe it's a super big brain model type of problem, but regardless, I think it's a very difficult one. And in each one of those scenarios, to find a way to route that properly is very, very convoluted. You could expose all that logic to customers: 'Hey, your task was to center a div, so I'm going to route you to something super cheap.' And then this other one is 'detect vulnerabilities,' so I'm going to route you to something really expensive. Expose all of that. I don't think that's a great user experience ultimately either. And for that similar class of problem, I think the more idealized UX would be something along the lines of dynamic effort or dynamic understanding of the level of complexity of your task, which you might actually evolve over time as you learn more and more information about the task. But in that world, if you're moving between a lot of different models, it actually becomes very, very disjointed. And so maybe a more unified approach would be something like a consistent model but at a very, very low effort level—it's actually very, very cheap and very affordable—but it can scale itself up very, very continuously, and you have a very, very consistent persona towards that model. And obviously I'm speaking about this in the context of our family of models. If you try to even do that across other families of models, other model providers, the personality, the disjointedness starts becoming very, very complicated. I don't know how you solve it at that layer, but at least within our family of stuff, I'm more bullish on things that are just more consistent, and those might be ways to solve it more robustly for customers.

是的，你不负责其他人，但在 Claude 内，是的。我是说，这是你的领域。

Yeah, you're not responsible for anyone else, but within Claude, yes. I mean, this is your territory.

很高兴看到你们思考这些问题的方式，对吧？意图本身就是一个难题。我觉得模型已经变得更好了，但有时人们不能很好地分类他们想要什么，或者他们只是滑块一直拉到极端，或者，你知道，模型并不真正理解。你问一个简单的问题，它就像，哦，我现在有一个完整的 HTML 着陆页。我只是期望几行输出，你知道。总是——那里总是有一团乱麻。

It's nice to see the way you guys think of these problems, right? Intent in and of itself is a hard problem. I feel like the models have gotten better, but sometimes people don't do a good job at classifying what they want or they just slider all the way to one extreme or, you know, the model doesn't really understand. You're asking a simple question and it's like, oh, I have a whole HTML landing page now. I just expected a few line output, you know. It's all—there's always a mess around there.

是的。我认为在这次对话中，我们谈了很多安全，谈到了发布模型等等，但这显然是为了帮助为人们的工作负载提供动力。正如我们在对话早些时候讨论的，我们看到工作负载变化相当大。我认为在某些工作负载中，智能似乎越来越饱和，这是一个新情况。并且它正在达到一个地步，我认为仍有一类任务非常非常不饱和。我实际上认为比人们可能预期的要多，但它们越来越朝着这样的方向发展：你通过执行极其长时间运行的智能体式任务来获得那种 alpha 或回报，这些任务历史上你并不真正能够暴力破解，但也许有了更多的智能在背后，你可以做到。那些工作负载对我们来说非常非常迷人，我们在那种领域看到了很多价值，既包括企业为自己获得价值，也包括有兴趣在其上构建智能体式产品的新公司或初创公司，或者无论你怎么称呼它。我认为如果你看看今天的那类事情，很多显然是编码相关的东西。显然有很多企业创新加速之类的东西。但它们倾向于采取自动化一个非常独特和具体流程的形式。我认为在某些领域，这实际上真的开始起作用了，这很棒。在某些领域，它仍然没有。所以仍有很大的空间来发展和推动那些模型的前沿。然后还有这种新的工作负载类别，比如我们谈到的生物 stuff 或一些网络 stuff——那只是一组非常非常复杂的，我会说，硬工程或硬科学问题。

Yeah. I think we in this conversation where we talked about a lot of safety, we talked about launching the models, all this kind of stuff, but it's obviously in pursuit of helping power the workloads for people. And like we discussed earlier in this conversation, we are seeing workloads change quite a bit. I think in some workloads intelligence does seem to be increasingly saturated, which is a new situation. And it's getting to the place where I think there are some class of tasks that are still very, very unsaturated. I actually think there are more than maybe people anticipate, but they're increasingly going into the direction of you get that alpha or that return by doing extremely long-running agentic tasks on things that historically you were not really able to brute force, but maybe you can with a lot more intelligence behind it. And those workloads are really, really fascinating to us, and we see a lot of value in that kind of area, both from businesses getting that value for themselves as well as new companies or startups or however you want to frame it who are interested in building kind of agentic products on top of it. I think if you look at the class of things today, a lot of it tends to be obviously a lot of coding stuff. There's obviously a lot of enterprise innovation acceleration kind of stuff. But they tend to take the shape of automating a very distinct and specific process. And I think in some areas that's actually really starting to work, which is awesome. In some areas it still isn't. So there's still a lot of room to evolve and push the frontier for those models. And then there's this new class of workload like the bio stuff that we talked about or some of the cyber stuff—that's just a very, very complicated set of, I would say, hard engineering or hard science problems.

你知道，你们处于世界上少数几个能发现所有这些工作负载并实现自动化的位置之一。就像——

You know, you are at one of the few spots in the world where you're discovering all these workloads and we're automating it. It's like—

是的，这挺疯狂的。

Yeah, it's pretty crazy.

我的意思是，你知道，这非常鼓舞人心。你们产生了如此大的影响，我真的很感激。我们将进入几个收尾问题，我觉得在你们两位之间来回讨论会很有趣。你们如何在团队中对抗垃圾内容？

I mean, you know, that's very inspiring. You guys are having so much impact and I'm really grateful for that. We're going to go into a few closing questions which I thought would be fun to sort of bounce back and forth between the two of you. How do you fight slop on your teams?

实际上，我们团队内部一直在进行有趣的辩论。我们和其他人没什么不同，对吧？我想很多团队都在讨论这个。有趣的是，我们试图弄清楚的是，如果你在制作一份文档，你在制作一些你想让别人消费的东西——代码、文档等等，对吧？你应该用 Claude 来帮你，当然，对吧？但如果你用 Claude 完全生成一个东西，你没有花时间自己实际阅读并确保它是高质量的，那么你就给那个文档或代码的消费者增加了额外的负担，让他们花大量时间试图解析它。这简直是疯了，完全不公平。但有趣的是，我们反击，我们在团队里强调你不能那样做，对吧？就像如果你在制作东西，用 Claude，当然可以，但你必须确保它是高质量的。作为人类，你仍然要对质量负责，对吧？你仍然要对质量负责。但然后你会遇到一个非常有趣的情况，因为 Claude 的痕迹——甚至只是 silly，比如我不知道，中间有个破折号，对吧？人们的大脑会触发这个，就像，哦不，我正在读一些 Claude 写的东西而不是人类写的，我很恼火，因为我想读的是达到人类质量标杆的东西，对吧？然后我们就会遇到有趣的情况，也许实际上那个人确实花了很多时间确保东西高质量，但他们用了 Claude 来帮忙，但那些 Claude 的小痕迹让读者反感，你知道，那也是无益的。所以我们试图做的是在团队内设定一个标准，非常像是用 Claude 来帮你，因为如果你不使用这个强大得可笑的智能体来加速你的工作和生活，或者只是帮你生产更好的东西，那就太傻了。但它必须达到一定的标准，然后对于消费者和读者来说，如果它真的只是垃圾，就退回去，对吧？就说不行，就退回去。你不会在这上面浪费时间。但也不要对小的 Claude 痕迹太挑剔，对吧？因为我们必须找到正确的平衡。所以这真的很有趣，因为我们在团队内部来回讨论这些事情，就像世界上其他人现在一样。我们没什么不同。

So we actually—this has been an interesting debate within our team. We're no different than anybody else, right? I think plenty of teams are talking about this. And what's interesting is what we're trying to figure out is if you're producing a document, you're producing something that you want other people to consume—code, documents, whatever, right? If you should use Claude to help you, of course, right? But if you use Claude to totally generate a thing, you didn't spend any time on actually reading it yourself and making sure that it was something of high quality, then you're putting a whole extra burden on the consumer of that document or of that code or whatever to spend a whole bunch of time trying to parse it. And that's just insane and totally unfair. But what's funny is we push back, we push on our team like you can't do that, right? Like if you're producing something, use Claude, sure, whatever, but you have to make sure that it's a high quality thing. You as the human are still responsible for the quality, right? You're still responsible for the quality. But then you get into this really funny thing where because of the Claudisms—even just silly, like I don't know, there's like an em dash in the middle of the thing, right? People's brains trigger on this like, oh no, I'm reading something that was written by Claude and not a human, and I'm annoyed because I want to be reading a thing that has the quality bar that the human would hold, right? And so then we get into funny situations where maybe actually the person did spend a lot of time making sure the thing was high quality, but they used Claude to help them, but there's those little hints of Claude and then the reader is turned off, you know, and that's unproductive as well. And so what we have kind of tried to do is set a bar within our team that's very much like use Claude to help you because it would be really silly if you didn't use this hilariously powerful agent to speed you up in your job and in your life or to just help you produce better things. But it has to have a certain bar and then for the consumer and the reader, if it really actually is just slop, send it back, right? Just say no, just send it back. You're not going to waste your time on this. But also don't be too nitpicky about the little Claudisms, right? Because we have to find our right balance here. And so it's just really funny because we're going back and forth on this stuff within our teams and just like so is everybody else in the world right now. We're just not any different.

但你们有免费的 token。

But you have free tokens.

嗯，当然，但我的意思是，当涉及到——时，这通常不那么重要。

Well, sure, but I mean, that doesn't tend to matter quite as much when it comes to—

所以你们比世界上任何人都多。你们可能看到很多——你们用量很大。你们看到更多人在做更多事情。

So you have more than anyone else in the world. You probably see a lot—you guys use a lot. You see a lot more people doing a lot more.

是的。是的。这完全合理。但这绝对是被强烈劝阻的。

Yeah. Yeah. It's totally fair. But it's definitely heavily discouraged.

工程垃圾。

Engineering slop.

是的。我的意思是，这是一回事，对吧？但我确实认为我们已经到了一个相当不错的状态，人类在确保系统设计良好方面扮演重要角色，对吧？而实现往往是由 Claude 完成的，对吧？我们处于一个不错的状态，对于很多写出来的代码，你真正需要的只是一个快速的批准，然后就可以继续。你不一定需要有人深入审查每一行代码，因为你更关心整体计划、整体设计是否良好，而不是精确完美的实现，并且显然依赖大量验证来确保你要发布的东西是高质量的。所以现在关于代码质量是垃圾的讨论实际上少了。部分是因为 Claude 变得更好了，但部分是因为我们有很多正确的验证来确保我们保持高标准。

Yeah. I mean, it's the same thing, right? But I do think that where we've gotten to is a pretty good spot where humans play a large role in making sure designs of systems are good, right? And implementation tends to be Claude'd, right? And we're in a pretty good spot where for a lot of the code that gets written, all you really need is just a quick stamp and you can keep going. And you don't necessarily have someone deeply reviewing every single line of the code because you care more about the overall plan, the overall design is in good shape, less so than the exact perfect implementation, and obviously rely on a whole bunch of verification to make sure the thing that you're going to ship is high quality. And so it's actually been less of a conversation these days on code quality being slop. In part because Claude's just gotten better, but in part because we have a lot of the right verification in place to make sure that we're holding a high bar.

我明白了。我明白了。你们信任你们的测试。

I see. I see. You trust your tests.

是的。

Yeah.

产品管理方面有垃圾吗？你可以大量发布功能。你可以在一天内推出东西。

Any product PMing slop? You can ship features a lot. You can put stuff out in a day.

他最近对我说过这个。你记得他说了什么吗？他发了这条推文：使用模型能力的正确方式不是向生产环境发布 10 倍多的功能。花更多时间理解你的用户、尝试实验、构建等等。但基本上，你很容易就可以继续发布随机的 SLO，但你应该真正思考一下。你应该花更多时间启动第一个提示。我知道今天他说，‘启动计划模式。我们不再需要它了。’但在计划模式之外，你应该思考你在做什么，对吧？很容易推出大量东西，你可以把产品解决掉。

So he said this to me recently. Do you remember what he said? He tweeted this: The right way to use model capabilities is not to ship 10x more features into prod. Spend more time understanding your users, trying experiments, building whatnot. But basically, you could very easily just keep shipping random SLO, but you should actually think about it. You should spend more time kicking off that first prompt. I know now today he's like, 'Kick off plan mode. We don't need it anymore.' But outside of plan mode, you should think about what you're doing, right? It's very easy to put out a lot and you can get products solved.

是的。我的意思是，我倾向于这样 rant：关于软件如何规划和发布等有很多基本的东西，并不需要仅仅因为智能体帮助加速了流程的某些部分而改变，但很多人开始表现出一种固有的懒惰，就像，哦，我不必思考流程的某个部分，因为智能体在做并帮助我之类的。但这些仍然是流程中极其基础的部分，应该是你花时间和精力的事情。我认为这就是他在这里表达的精神。

Yeah. I mean, I tend to go on this rant about there's a lot of just fundamentals about how software gets planned and shipped and otherwise that doesn't need to change just because agents are helping speed up certain parts of the process, but there's this inherent almost laziness starting to come out from a lot of people where it's like, oh, I don't have to think about some part of the process at all because the agents are doing it and helping me or whatever. But those are still incredibly foundational parts of the process that should be things that you're spending your time and energy on. And I think that's kind of the spirit of what he's getting at here.

就像作为创始人或创作者，我们也这么称呼。感觉因为实验或探索的成本现在如此之低，我应该做更多这样的事。而他基本上是说不要那样做，要把它用于服务于一个我从一开始就非常有意设定的目标，显然我正在接受这一点——只是很难。就像你有一个能做所有事情的玩具。

It's just like as a founder or creator, let's call it as well. It just feels like because the cost of experimentation or exploration is so low now that I should be doing more of that. And he's basically saying to not do that, to use it in service of a very intentional goal that I start from the start, which obviously I'm coming around to—it's just hard. It's like you have this toy that does everything.

就像是把东西往墙上扔。

It's just like throw it at the wall.

是的，也有范围。比如今天的博客文章是“两周让 Claude AI 更快”。有 12 件事。呃，我们完成了 13 件事。我们在 3 天内完成了 13 个目标中的 12 个，对吧？好吧，你可以说这是两周的冲刺，我们会完成 13 个，但反而有 20 件事你推进了边界。这不是产品垃圾。不是新产品。是重构。但尽管如此，你们在输出大量内容的同时保持了高标准的质量，做得非常好。

Yeah, there is scope too. So like the blog post of today was 2 weeks to make cloud AI faster. There were 12 things. Uh we hit there were 13 things. We hit 12 of the 13 targets in 3 days, right? Well, you could just say okay it's a two-week sprint. we'll do the 13, but instead there's 20 things you push bound. This is not product slop. It's not new products. It's refactoring. But still, you guys do a very good job at outputting a lot, but keeping high bar quality.

是的。不错。

Yeah. Nice.

好的。三部分中的第二部分。呃，这会让 Angela 很不舒服。这是 Angela 推文角。呃，我会告诉

Okay. Part two of three. Uh this is going to make Angela very uncomfortable. This is Angela Tweet Corner. Uh I will tell

不错。我我会呃我会不停地告诉人们去

Nice. I I will uh I will tell people nonstop to

我而且我我我相当确定你删过一些东西

I and I I I'm pretty sure you've deleted stuff

因为我会想那里有有些东西，你删除的内容里有些东西让我觉得如此

because I'm like there's there's stuff there there's stuff in your deletions that I'm like so

是的，我知道。就像真的改变了我的 DNA。

Yeah, I know. Like really like alter my DNA.

哦，我很感激。

Oh, I appreciate that.

你最常参考什么？你最常向人们推荐什么，你想提出来？如果你愿意，我可以帮你找出来，但是

What do you refer what do you think about the most? What do you refer people to the most that you want to sort of bring up? And I can I can pull it up for you if you want, but

嗯，实际上我不认为我有最喜欢的，老实说。不，我真的……好吧，我想也许我喜欢你说的那个，半途而废是昂贵的无所作为的方式。也许那是

um I actually I don't think I have a favorite, honestly. No, I really Okay, I guess maybe I like the one that you say where it's half doing something is expensive way of not doing it. Maybe that's

这对我们今天来说非常切题。

that's very on topic for us today.

好吧，但我想呃你知道显然你现在在做不同的工作，不再做这个了。但就像呃我觉得这是一条强力推文。这就像 Naval 式的 Rob 你。

Okay, but I guess uh you know obviously you're in a different job now and you don't do this anymore. But like uh I think this is a power tweet. This is like a naval rob you.

是的。

Yeah.

做这个的。嗯哼。

That do this. Uhhuh.

那是什么样的模式，什么样的思维框架，什么样的心态？

And like what mode of what like what framework of mind frame of mind of mind?

这是个好问题。是的。不，这是很久以前的事了。嗯，我经历过一段时期，实际上我一直在读书。我发现自己对正在做的事情感到非常无聊，我觉得自己没有受到智力上的挑战。所以我实际上开始了一段狂欢，我想我在一年内读了大约 200 本书。嗯，我只是对一切都好奇，我只想读所有东西。我缺少了一点那种智力上的……我不知道，就像在学校里那种肌肉锻炼，他们让你学新主题，很难，其实不是特别有趣，但你确实在拓展自己。我觉得在工作几年后，我就停止了这样做。是的，工作很有趣，有挑战性，但没有那种智力上的部分。所以无论如何，我开始了一段狂欢，阅读狂欢。我一年读了 200 本书。很棒。我通勤时间很长，所以这是很好地利用时间，非常长的地狱般的火车旅程。但呃，我最终从中走出来，因为那件事我思考了很多。当时真的不是为了建立 Twitter 粉丝什么的。我只是在对着深渊发推。我几乎没有粉丝。这些推文是为我自己发的，我在练习如何用尽可能少的词压缩我的想法。这有点像我对自己的个人测试，然后我想人们开始喜欢它，所以我就继续做下去，然后工作太忙了，我就停止了。所以

That's a good question. Yeah. No, this was a very long time ago. Um I went through a period of my life where I was like reading constantly actually. I found myself like very bored um of the stuff that I was doing and I felt like I was just like not being like intellectually challenged. And so I actually went on this a bit of a bender where I I think I read something like 200 or so books in like a year. Um, and I was just curious about like every I just wanted to like read everything. And I was missing a little bit of like the the kind of like intellectual like I don't know like muscle building you get when you're like in school where they like do like this is a new topic you got to learn. It's like tough. It's like not super fun actually but you are really like stretching yourself. And I felt like after just like working for a couple years, I just like stopped doing that. Like yeah, the job's like interesting. I guess it's like challenging, but it doesn't have like that like intellectual bit to it. So anyway, I went on this bit of a a bender uh a reading bender. Uh and I like read like 200 books in a year. It was great. I had like a really long commute, so it was like it was a good use of my time, very long like hell train ride. But uh I ended up coming out of that and being like I was just like thinking a lot um as a result of that. And at the time it really wasn't to build like a Twitter followower or anything. I was just like tweeting into the like abyss. There was like I have like no followers. These were like tweets for myself and I was like practicing a little bit of like how would I try to compress the idea that I have in as few words as possible. It was kind of like the personal test I had for myself and then I guess people just started liking it and so I just I just kind of just kept going at it and then I got too busy with work and I I stopped doing it. So

我得问一下，读了那么多书之后，有什么书推荐、博客、长文章吗？

I got to ask after all the reading any book book recommendations, blogs, long articles.

我最喜欢的可能仍然是纳西姆·塔勒布的《黑天鹅》。嗯，那就像是至少对我来说，它有点像一本改变人生的哲学书，让我觉得那是一种完全不同的看待世界的方式。而且

My favorite one still is probably The Black Swan by Nasim Talib. Um that was like one of those like I was like kind to me at least it was like a bit of a like a life-changing philosophical book where I was like that's a completely different way to look at the world. And

你知道当那件寓言般的事情发生时，我想到了黑天鹅，对吧？那是你不知道可能发生的事情。现在它发生了，它永久地改变了你的框架

you know I was thinking of Black Swan when the fable thing happened, right? It's a thing that you didn't know could happen. now that it happens like it's a it permanently changed your framework of

关于事物如何运作，呃，这个想法很简单，但我认为他做得很好，真正把它讲透了，并阐述了它如何实际上是世界上许多事物的共同属性。所以我真的很喜欢那本书，那可能是我的最爱。

exactly how things work and uh the idea is like so simple but I think he did a great job of just really hitting it home and articulating it and how it's like a common property of actually of so much of the world. So I really love that book and that's probably my favorite.

呃，对于听众来说，如果你对那本厌倦了，你可以读塔勒布的《随机漫步的傻瓜》。你喜欢《反脆弱》吗？

Uh and for listeners if you're bored with that one you do fool by randomness randomness also by Taleb. Did you like uh fragile?

我想是的。所以他越出名就越让人难以忍受。

I guess so. So the the more famous he got the more intolerable he became.

呃，但也是给 Kaylin 的问题。如果你有的话。你有没有类似的“这是我推荐给每个人的书”？

Uh but also question for Kaylin. My if you do have one. Do you have a similar like this is a book I recommend to everybody.

不，这是个好问题。我需要想想。这很有趣，因为我认为我们在这方面风格非常不同。她不再像幸运饼干式的推文模式了。但我确实认为你停止了那个。但我确实认为你工作和与他人合作的一大部分有时是哲学家模式。你知道，

No, that's a good question. I need to think about it. It is funny cuz I do think we have very different styles on this front. Like she's no longer like in fortune cookie tweeting mode. But I do think that like you stop that. But I do think a a big part of how you work and how you work with other people is like philosopher mode sometimes. You know,

你在 Slack 里丢幸运饼干。

you drop fortune cookies in Slack.

哦，我确实会。不是故意的。我认为成功丢饼干的部分原因可能是你必须在 Slack 上非常擅长。

Oh, I I do actually. Not intentionally. I think it's might be somewhere success of succeeding in dropping is like you have to be very good on Slack.

是的。哦，你们必须非常擅长 Slack。

Yeah. Oh, you guys have to be very good at Slack.

嗯，这是一个相当紧张的地方，但我觉得我有点……所以我对电子邮件和 Slack 的收件箱清零有点痴迷。现在电子邮件比 Slack 容易。嗯，但我喜欢积极修剪，比如这些是我真正参与和关注的频道，而那些是静音在下面的。我总是确保我掌握我需要掌握的频道。但这基本上感觉像一份全职工作。有些日子我会想，我可以取消所有会议，不写文档，不做工作，对吧？就真的坐在那里一整天在 Slack 上，我就会觉得通过参与 Slack 上的所有事情完成了一整天的工作。这很疯狂。

Um, it is it's a pretty intense place, but I think I'm a little So, I'm kind of psycho about like inbox zero on both email and Slack. It's easier on email these days than Slack. Um, but I like pretty aggressively prune like here are the channels that I'm like actually in and paying attention to versus the ones that are like muted below the line sort of thing. And I'm like always making sure that I'm on top of the ones that I need to be on top of. But it is it's basically it feels like a full-time job. Like there are days where I'm like I could cancel all my meetings and write no documents, do no work, right? and literally just sit there like on Slack the entire day and I would have done what felt like an entire day's worth of work just by like engaging on Slack with all the things. It's pretty wild.

不，但那很可怕。但是什么

No, but that's horrible. But what

所以这正是我的感受。我觉得你做了所有这些，投入了所有努力，是的，你在回复人们和让某些事情完成方面达标了，但你实际上没有产生你想要的输出。这对我来说是个人很难处理的事情。我试着像我需要 dedicated 时间真正思考，真正尝试产生一些东西，才能感觉我真正

So that's exactly how I feel. I feel like you do all this, you put in all this effort and yes, like you you met the mark on like responding to people and getting certain things over the line, but you didn't actually produce the output that you wanted to. And like that's like a that's like personally a very hard thing for me to deal with. I try to be like I need to like dedicate time to like actually thinking and actually like trying to produce something in order to feel like I'm like actually

并忽略和放弃一些东西，对吧？你将会放弃一些东西。

and ignore and drop stuff, right? You're going to drop stuff.

嗯，是的，我收到了很多反馈。我在 Slack 上有点难联系到。

Um yes, I have gotten a lot of feedback. I'm a little hard to reach on Slack.

是啊，现在经常发生的是，我收到很多 Slack 消息说：“你能让 Angela 回复一下这个吗？”我就想，我正努力记下来呢。等等，你们没有像个人 Claude 助手之类的吗？我不知道怎么帮你们，但是——为什么没人给你们每个人写个个人行政助理呢？

Yeah, pretty thing that happens now is I get a lot of Slacks saying, "Can you get Angela to respond to this?" And I'm like, I'm really trying to write it down here. Wait, so you guys don't have like a personal Claude assistant or something? I don't know how to help you, but there's— Why has no one written like a personal EA for each of you?

是的。嗯，我们也有一个人工助理。当然，我们在 Slack 里也有很多 Claude 的帮助。但我确实认为，我筛选的那些 Slack 消息，往往只是我们团队因为需要做出一些艰难决定或其他事情而被卡住了，对吧？这只是让团队不被卡住、高质量执行的一种方式——无论是在 Slack 还是其他方式。这本来就是我的工作，只是恰好经常发生在 Slack 上。但我同意，当你一整天都在和人来回沟通、试图解决问题时，最终会感觉：我到底产出了什么？

Yeah. Well, so we do have a human one also. We also have plenty of Claude help in Slack for sure. But I do think actually the types of Slacks that I'm pruning through are often just like our team is blocked on something because there's some hard decision that needs to happen or whatever it is, right? And it's just a form of—whether it's in Slack or some other way—getting the team unblocked, executing at high quality. That is my job anyway, and it happens to happen a lot in Slack. But I do agree that it ends up feeling like, what did I even produce when you get through a whole day of literally just going back and forth with people and trying to solve problems.

你产出了一个不被卡住的、能正常运作的团队。作为前经理，我为此给你点赞。那就是——是啊，是啊，是啊。你不觉得自己产出了什么，但你产出了团队。

You produced a functioning team that's unblocked. So I will give you a prop for that as a former manager. That is— Yeah. Yeah. Yeah. You don't feel like you produce anything but like you produce the team.

是的。

Yeah.

是啊。呃，说到你的团队，我接下来想问你，Caitlin。呃，DevRel 角落。哦，不错。呃，我们——我知道你正在大力招聘 Dev——为开发者关系发出号召。你最近聘用了 Addy Osmani。是啊。很棒的收获。呃，Lance，你知道，你组建了一个团队。呃，你们想招聘什么样的人？你知道，这就是——这就是要讨论的，我很乐意交流心得。

Yeah. Uh, speaking of your team, I was gonna go to you next, Caitlin. Uh, DevRel corner. Oh, nice. Uh, we are— I know you're really hiring Dev— making a call out for developer relations. You hired Addy Osmani recently. Yeah. Great get. Uh, Lance, you know, you build up a team. Uh, what what do you guys want to hire for? You know, this is— this is it to talk about it and I'm happy to trade notes.

是的。嗯，为你点赞，因为我要说，当——在外部招聘——Kaitlin 的招聘团队——真的真的，嗯——很有趣——而且这甚至不只是招聘，我要稍微夸夸你。所以当我加入 Anthropic 时，我们是一个小团队——总的来说，我们是一个非常小的团队——我们运营平台、API 业务以及所有产品基础设施。我们在 Anthropic 内部有一个非常初级的 DevRel 职能，不一定在产品与工程组织内。当我加入时，我们觉得，好吧，我们需要稍微成熟一点，需要弄清楚如何与社区互动、与世界互动。我们需要真正投资于这个 DevRel 职能。我们最终在平台内建立了一个 DevRel 团队，在 Claude Code 内也建立了一个 DevRel 团队，部分原因是我们在 API 业务和智能体式编程方面要接触的两个开发者生态系统相当不同，对吧？所以我们最终把它做成两个独立的团队，但彼此之间有虚线汇报关系，并且花大量时间在一起。但这是我将要负责的事情——DevRel——我之前在 Stripe 或其他地方花了很多时间在 DevRel 上，但从未负责过，我当时想，我完全不知道自己在做什么。然后你说，我懂你，这是 DevRel 的三大支柱，这是你思考它的方式——这真的很棒——涵盖产品、内容、社区。现在我可以向别人滔滔不绝地讲这些，他们会说，哇，你真的很懂 DevRel。我说，不，别让我说了。

Yeah. Well, props to you because I will say when— on the outsourc recruiting— Kaitlin recruiting team— literally literally um— it's fun— and it's not— it's actually not even just recruiting like I'm going to toot your horn a little bit. So when I joined Anthropic, we were a small team— all things considered, we're an extremely small team— and so we run the platforms, the API business, and all of our product infrastructure. And we had like a very nascent DevRel function within Anthropic that was not necessarily within the product and engineering org. And when I joined, we were like, okay, we need to grow up a little bit and need to figure out how to engage with the community, engage with the world. And we need to really invest in this DevRel function. And we ended up spinning up a DevRel team within platform and a DevRel team within Claude Code in part because it's like two pretty different developer ecosystems that we're trying to engage with across the API business and agentic coding, right? And so we ended up doing it as two separate teams but kind of dotted line to each other and spend a ton of time together. But it was something that I was going to be responsible for— DevRel— which I've spent plenty of time with DevRel before at Stripe or otherwise, but it was never something that I was responsible for and I was like, I have no idea what I'm doing. And you were like, I got you, here are the three pillars of DevRel, here's how you think about it— which is really great— across product, content, community. And then now I can rattle that off to people and they're like, wow, you really understand DevRel. I'm like, no, stop me.

你会说：“我抄的。”

You're like, "I copied that."

是的。那不是——

Yeah. That was not—

显然那篇文章成了谷歌新 DevRel 入职时的默认阅读材料。真的很有趣。

It's apparently that post is apparently like the default reading for Google when they onboard new DevRel. Like really funny.

是的。它真的——这真的帮了我，因为我完全不知道自己在做什么。但除此之外，我目前没有平台内 DevRel 团队的经理。所以这是我希望能招聘的职位之一，我们的 DevRel 个人贡献者直接向我汇报，部分原因是我认为现在这是一个非常非常非常难的问题，但也部分是因为我觉得它很有趣，我和团队合作得很开心。但具体来说，平台难的地方在于，平台有时在我们的公众叙事中——我们的平台业务是一个非常大的业务，但它可能会迷失在研究和政策以及我们显然在做的最前沿重要工作之间，然后是我们的应用，比如 Claude Code 和 co-work 以及这边发生的所有事情。当你想到 Anthropic 并想到我们的公众叙事时，那些往往是首先想到的事情，而不是我们正在运营这个庞大的开发者平台业务，这个业务非常重要，很多人在上面构建。所以我们平台内 DevRel 团队的一个大目标是，不仅要弄清楚经典的——我们如何更好地与产品互动、人们在构建什么、我们如何将反馈带回产品等等——而且很大程度上是找到我们的声音，找到我们的品牌应该是什么，我们的叙事应该是什么，我们甚至要在社区中成为谁，而不是迷失在所有正在发生的事情中。所以我认为这很有趣，因为我认为我们开始为 DevRel 团队打造的品牌——部分原因是 Lance 加入我们的团队是你把我介绍给 Lance 的——但 Lance 技术非常强，他做 DevRel 的方式非常像教学。他写了很多关于内存和管理成本之类的东西，他钻得很深。他构建评估。他整理了很多真正技术性的内容。我真的很喜欢这个想法——我们的 DevRel 团队成为这个非常新兴、非常困难、不断发展的领域中的思想领袖。

Yeah. It really like— this really helped me because I had no idea what I was doing. But then on top of that, I don't have a manager for the DevRel team within platform right now. And so that's one of the things that I'm hoping to hire, and our DevRel ICs report to me directly, in part because I think it's a really really really hard problem right now, but also in part because I find it really fun and I'm having a lot of fun working with the team. But really specifically, what's hard on platform is platform sometimes in our public narrative— our platform business is a really large business but it can kind of get lost in between research and policy and all this work that we're obviously doing that's our most frontier important work, and then our applications like Claude Code and co-work and all the things happening over here. When you think about Anthropic and you think about our public narrative, those are some of the things that often come to mind and less so that we're running this massive developer platform business that's really important and so many people are building on top of. And so a big goal that we have with our DevRel team within platform is figuring out not only the really classic— how do we better engage with product, what people are building, and how we get feedback back into the product and these sorts of things— but a lot of it is just finding our voice and finding what our brand needs to be and what our narrative needs to be and who we even are going to be in the community and not kind of get lost in all of the things that are happening. And so that I think has been interesting because I think the brand that we've started to spin up for our DevRel team— in part because actually Lance joining our team was you connecting me with Lance— but Lance is highly technical and the way that he goes about DevRel is very much like teaching. He writes a lot about memory and managing costs and these sorts of things and he goes really deep. He builds evals. He has a lot of really technical content that he puts together. And I really like that idea— that idea of our DevRel team being kind of like thought leaders for people within what is a really nascent, really hard, really evolving space.

教育者、思想领袖、事物的解释者。嗯，我真的很喜欢另一个版本，叫做“大师建造者”。就是我会用平台构建东西来向你展示什么是可能的，然后你会说，我不知道 Claude 平台能做到这个。然后你就去了。这是从乐高偷来的点子。因为乐高，他们怎么卖乐高？他们有大师建造者。他们用乐高搭建东西。然后你会说，我想做那个。我想成为这个人。然后他们就这么做了。

Educators, thought leaders, explainers of things. Um there's another version of this I really like called the master builder. Uh which is I will build the things with the platform to show you what's possible and then you'll be like I didn't know that was possible with the Claude platform. And then you just go. And this is a thing stolen from Lego. Because Lego, how do they sell Legos? They have master builders. They build things with Legos. And you're like, I want to do that. I want to be like this person. And then they do that.

不错。太棒了。是的。

Nice. That's awesome. Yeah.

还有 CJ，我们团队里另一位非常出色的人。我觉得他也做了很多类似的事情，比如写 cookbook、发布示例，并在 X 上与社区积极互动。

And CJ, the other person on our team who's really great as well. I think he also does a lot more of that sort of writing cookbooks and putting out examples and really engaging with the community on X.

Cookbook 挺有意思的，对吧？比如 2023 年 API 发布时，我们姑且说是针对 LLM 的，非常重要。现在就没那么重要了。

Cookbooks is funny, right? Like when APIs launched in 2023, let's call it for LLMs, very important. Now, not so much.

我不知道。

I don't know.

是啊。有意思的是，你现在不太会直接复制粘贴代码来用了，对吧？但我确实认为它向人们展示了可能性，比如用例到底是什么，以及如何实现它。但我明白你的意思，代码本身可能没那么重要了——我们很长一段时间都有这个——我们团队有个工程师写了关于 computer use 的 cookbook，这至今让他心有余悸，因为它极其复杂，是个庞然大物，当时很多人从中受益，比如我要构建一个 computer use 实现，我要处理所有截图和 messages API 之间发生的所有事情。

Yeah. Well, it's interesting because you're less in the spot where you'd like copy paste the code and like use it, right? But I do think it shows the art of the possible for people and just like what even is the use case, right? And how could you go about accomplishing it? But I hear you that the code itself is maybe less—we for the longest time had this—there's an engineer on our team who wrote our cookbook on computer use and it haunts him to this day because it was massively complex and kind of a beast of a thing that I think a lot of people got use out of at a time when it's like I'm going to build a computer use implementation and I'm going to deal with all the screenshotting and all the things that are happening between the messages API.

是的，我们参加了你们在办公室启动的黑客松。

Yeah, we were there for the hackathon that you guys launched at office.

是的。那是你需要这种东西的时代，而现在更多是——我们有一个技能，比如 Claude API 技能。它非常擅长帮助你在 Claude Code 或任何其他编码智能体中构建高质量的集成。现在这类事情，比如我要构建一个 computer use 集成，我该怎么做？我希望很少有人会去翻那本复杂的大 cookbook，而是更多地在编码智能体中加载技能并工作。

Yeah. And that was the era when you needed to have that sort of thing and now it's much more about—we have a skill like the Claude API skill. It's really excellent at helping you either within Claude Code or any other coding agents to build a really high quality integration. And now these sorts of things like I'm going to build a computer use integration, how do I go about doing that? Very few people I think hopefully are reaching for the big gnarly cookbook and more so they're loading up the skill in a coding agent and working.

或者他们只是把它放进参考架构里。实际上效果更好，比如增加——我们看到对于许多你可能想要不同实现的东西，通常在 harness 层，但也可能有其他变体，你只需让 Claude 指向并说这是我要的参考架构。是的，然后你构建它。有趣的是，帮助人们基于 API 构建的 Claude API 技能是由我们的 DevRel 团队拥有的。所以那是 Lance 构建的，对吧？没错。就像 DevRel 最高效的形式是提炼所有最佳实践，这些将进入那些智能体。

Or they're just putting it into a reference architecture. It actually does much better like increasing—we're seeing that for a lot of the things that you might want to have different implementations on, typically at the harness layer but there could be other variants, you just point Claude at and you say this is the reference architecture I want. Yeah. Then you build it. Fun fact, the Claude API skill that helps people build against the API is owned by our DevRel team. So that was built by Lance, right? Exactly. Like most high leverage form of DevRel is to distill everything that is best practices that's going to go into those agents.

是的。奇怪的原因之一是我被视为 DevRel 权威，但我内心有一部分只想告诉你你不需要 DevRel。太好了，你不需要它。对吧。或者你把工程师、领导者变成公众面孔，因为你真正想做的就是公众面孔，比如我们这里是怎么做的，然后差不多就这样——人们会听说你。所以对我来说，当 Boris 横空出世时，看起来就像嘿我是 Boris，我是 Claude Code 的创造者 blah blah blah，我们是这样做的 blah blah，是的他有日常工作，但他也在功能上做那个，所以为什么不是每个人都这样做呢？

Yeah. One of the reasons it's weird because I'm regarded as this DevRel person authority, but there's some part of me that just wants to tell you you don't need DevRel. It's so good you don't need it. Right. Or you turn the engineers, the leaders, into the public faces because all you're really trying to do is public faces and like here's how we do things here and then that's kind of it—people will hear about you. So like to me when Boris came burst onto the scene like it seemed like a very like hey I'm Boris I'm the creator of Claude Code blah blah blah this is how we do things blah blah you like yes he has the day job but he's also doing that functionally so like why doesn't everyone just do that?

是的。这是个很好的问题，有些团队做得非常好。比如我认为 Cloudflare 团队就是一个很好的例子。他们都很棒，对吧？Nontop。

Yeah. I mean, it's a really good question and there are teams that have done this really well. Like I think the Cloudflare team is a really great example. They're all awesome, right? Nontop.

是的。顺便说一句，对我来说，这实际上是一个好经理的非常明显的标志：他们都对自己的工作满意。他们都公开表明身份。他们没有辞职。一个都没有。

Yeah. And to me, that's actually, by the way, a very visible sign of a good manager is that they're all happy at their jobs. They're all publicly identifying. They're not quitting. None of them.

完全同意。有意思的是，我认为这实际上是我们试图在平台团队和我们团队中更多做的事情，让那些人能够走出去，成为社区的一部分，发言、在 X 上、在任何地方，对吧？我认为我们的问题只是时间和知道如何高质量地做这些事情。所以我在某种程度上把 DevRel 团队视为高质量做这类事情的专家。他们可以整天做这些，他们可以帮助团队其他人思考如何利用他们能花在这类事情上的少量时间也高质量地做到。

Totally. It's interesting because I think this is actually something we are trying to do more of with the platform team and with our team and folks who can kind of just get out there and be part of the community and be speaking and be on X and be wherever they are, right? I think the problem for us is just having time and knowing the right way to do that stuff at high quality. So I see the DevRel team in some ways as just being the experts in doing this sort of stuff at quality. They can do it all day every day and they can help enable the rest of the team to think about how to do this with the small amount of time that they are able to spend on this sort of stuff to do it at high quality too.

是的。所以我称之为将他们的角色拆分为前台和后台。通常做这个工作的人从头到尾做整个事情。他们有点自主。但实际上，你可以建立这个后台，让你的产品负责人和工程负责人真正成为焦点，用一个简单的契约：好的，你和我坐下来 30 到 60 分钟，我们做头脑风暴，然后你就完成了。你回去工作，然后我们为你代笔，等等。但那个制作人员、写作人员等的团队为其他人提供服务，我认为这对我来说是这类事情的未来，说实话。因为人们实际上并不真的想听 DevRel 的。他们想听 N 的。你有 Lydia 通过 bud，你知道。没错。但功能上他做了这个工作。他做了这个工作。所以你想要更多这样的人——人们想听产品中的人，而不是付费发言人，我遇到很多这样的，那没关系，那是有人必须做的工作，然后——

Yeah. So I call this kind of the unbundling of their role into a front office and a back office. So a typical person who does this job does the whole thing start to finish. They're kind of autonomous. But actually, you can build out this back office that lets your product leads and engineering leads actually take the spotlight with a simple contract of like, okay, you sit down with me for 30 to 60 minutes and we'll do the brain dump and then you're done. You go back to work and then we'll ghost write for you, whatever. But that team of production people, writing people, whatever, provides the service to the rest of the people and I think that to me is kind of the future of this stuff to be honest. Because people don't actually really want to hear from DevRel. They want to hear from N. And you have Lydia through bud, you know. Exactly. But like functionally he does the job. He does the job. And so you want more of those people—people want to hear from people in the product, not like paid spokesperson which I get a lot of and that's fine that's a job somebody has to do it then—

你可以合理地预期从事工程的人不擅长这个。

Reasonably you can expect people who are working on engineering to not be good at it.

是的。所以你应该付钱给专门做这个的人，专业是一种技能。但我认为可以出现后台和前台的分离。

Yeah. So like you should pay someone who specializes in it like specialty is a skill. But I think there is a back office front office separation that can emerge.

是的。那真的很酷。

Yeah. That is really cool.

是的。我的类比是，每个人都招聘，但你有 HR 部门。HR 部门让你能够招聘。

Yeah. And my analogy would be like everyone hires but you have an HR department. HR department enables you to hire.

你招聘就像——

You hire like—

是的。非常有道理。

Yeah. Makes a ton of sense.

酷。那是我的个人观点。结束问题。你非常慷慨地付出了时间。有什么临别想法或行动号召吗？有什么酷的，人们应该尝试用新的 Opus 5.5 突破界限？你们使用它最久。

Cool. That was my soap box. Closing questions. You've been very generous of your time. Any parting thoughts or calls to action? Something cool on what's something people should try to push the bounds with new Opus 5.5? You guys have been using it for the longest.

是的，它非常经济高效。它更简洁。我们提到的所有事情，氛围真的非常非常高。很有趣。

Yeah, I mean it's really cost-efficient. It's less verbose. All the things we mentioned, the vibes are really, really high. It's been fun.

昨天对我们来说真是非常开心的一天，因为终于，我又收到了一大堆朋友和人们的消息，他们就说：“模型太棒了。谢谢。”我的一位老上司给我发消息说：“Opus 5.5 刚刚在一场事故里救了我们的命，对吧？”我就想：“好，太好了。”我们回来了——这真的是我们感受到的那种氛围。

Yesterday was a really fun day for us, because finally, again, I got a whole slew of texts from friends and people who were just like, "Amazing model. Thank you." One of my old bosses messaged me and he was like, "Opus 5.5 just saved our butts in an incident, right?" And it's like, "Okay, great." We're back — that's really the energy that we feel.

所以我觉得，那些你本来想尝试、但成本一直是你首要顾虑的事情，对吧？Token 效率之类的——那些可能一直拦着你，让你没法真正放手让模型跑起来。我觉得现在有了 Opus 5.5，你应该回头去做这类问题，因为我觉得你能走得比用那些成本效益更低的模型时远得多。

So I think things where maybe you wanted to experiment with something and cost was front of mind for you, right? Token efficiency and these sorts of things — that kind of might have held you back from really firing it off and letting a model run. I think those are the sorts of problems you should return to now with Opus 5.5, because I think you'll be able to get a lot further than you maybe otherwise would have been able to with models that were less cost effective.

我觉得你应该试试多智能体的架构。我觉得你会发现它非常好用。

I think you should try something with a multi-agent setup. I think you will find it to be very good.

顺便问一下，还有别的提示吗？我是说，MOSS 的范围实在太广了。

Any other hints, by the way? I mean, MOSS is just very broad.

没有了。我觉得我会非常好奇想听听大家会做出什么。不过是的，它应该会格外好用。

Nope. I think I'd be very curious to hear what people build. But yes, it should be extra good.

是的，包括比如说那些没有被严格定义、而是它们自己互相分配的角色，对吧？

Yeah, including, let's say, roles that are not strictly defined, but they assign among themselves, right?

这对我来说才是新东西。你可以分配角色——你一直都能分配角色。你可以给 token 派活。但这里是它们自己在给自己派活。

Which is to me the new thing. Like, you can assign roles — you could always assign roles. You can give tokens jobs. But this is like they're giving themselves jobs.

是的，我想是这样。

Yes, I guess.

再多讲讲多智能体，好吗？比如，我们应该用子智能体吗？大家该怎么理解多智能体？

Say more on multi-agent, right? Like, should we use sub-agents? How should people think about multi-agents?

这挺好的，基本上，一般来说你希望有一个主控，然后把一堆不同的东西拆分开，只要你能有一个你认为大体上可以并行化的任务，而且它们在其中共享统一的上下文，我觉得你就能获得很多收益。

This is pretty good in the sense that, basically, generally you want to have one lead and you want to split up a bunch of different stuff, and as long as you can have a task that you think could be generally parallelizable and they share uniform context across it, I think you will get a lot of gains.

当然，这取决于你要攻克的问题，但越来越多地，这对我们来说是处理很多这类事情的一种非常优越的方式。尤其是如果每个子智能体——或者你想怎么称呼它们，比如子负责人——它们是异步的，彼此不阻塞。这大概就是我说你在架构里唯一需要关心的两点。

Now, of course, it depends on the problem that you're pushing at, but increasingly this is a very superior way for us to handle a lot of these things. And especially if every sub-agent — or whatever you want to call these, like sub-leads — they are asynchronous and they're not blocking each other. That is like the only two bits I would say you should care about in your architecture.

除此之外，就放手让它跑，看看 Claude 会为你设计出什么。而且越来越多地，我觉得很多这类架构上的东西都会朝这个方向走。Claude 会越来越聪明。它会学会想办法拆分自己、克隆自己、把自己调整到合理的架构上。

Other than that, let it rip and see what Claude designs for you. And increasingly, I think a lot of this architecture stuff will fall into this direction. Like, Claude will get smarter and smarter. It's going to learn how to figure out ways to split itself, clone itself, adjust itself to the architectures that make sense.

你只需要确保它能并行处理。所以它需要一个共享状态，而且它不能阻塞它的子智能体。只要守住这两条原则，说实话，我觉得大家就该去实验，甚至去试一些非常疯狂的东西。

You just need to make sure it can parallel process. So it needs a shared state and it needs to not block its sub-agents. And as long as you contain those two principles, honestly, I think people should experiment and probably try some really crazy stuff.

而且我不会惊讶于在此基础上出现一些非常有意思的东西——也许他们觉得有个棘手的问题特别烦人，一直死磕它——像这样的东西说不定一下就能把它拿下。

And I wouldn't be surprised to see some really interesting things on top of that, where maybe they felt like there was some intractable problem that was really annoying and they kept pushing on it — something like this could actually probably just crush it in maybe one shot or something.

太好了。好了，大家都去动手做吧。

Excellent. All right, everyone go build.

好。好。

Cool. Cool.