为什么 AI 算力是长方形的：Erik Bernhardsson 谈打造 Modal

感谢你来参加节目。Erik Bernhardsson，来自 Modal。你能简单介绍一下这项业务，以及你是怎么走到今天的吗？

Thanks for coming on. Erik Bernhardsson from Modal. Could you explain the business a little bit and maybe how you got here?

是的，我们构建 AI 基础设施。最简单的说法是，我们提供一个算力平台和一个存储平台，让 Decagon、Sunno、Runway、Cognition 等许多客户能够构建 AI 应用。实际应用中，这意味着大量的推理、大量的沙盒环境，有时还有训练和微调，有时是批处理任务等等。所以我们基本上构建了一个软件 SDK，让客户能够快速行动并构建这些应用。我们管理底层的所有容量，使部署、扩展和在生产环境中运行变得容易。

Yeah, so we build AI infrastructure. The easiest way to put it is we offer a compute platform and a storage platform that enables a lot of customers like Decagon and Sunno and Runway and Cognition and many others to essentially build AI applications. So what that means in practice is a lot of inference, a lot of sandboxes, sometimes training and fine-tuning, sometimes batch jobs and other things. So we have basically built a software SDK that enables customers to move really fast and build this. We manage all the capacity underneath, make it easy to deploy and scale and run these things in production.

你两年前上过 Latent Space 节目。从那以后发生了很多事。当时你们规模有多大？能分享的数字都行，对比一下。是的。现在规模有多大？

So you were on Latent Space like 2 years ago. A lot has happened since then. Like how big were you back then? Like whatever numbers you can share and comparatively. Yeah. How big are you now?

我其实不记得当时的情况了，但可能在营收、用户和使用量等方面至少增长了 50 倍。

I actually don't remember where we were, but it's probably at least like 50x in terms of revenue and users and usage and stuff like that.

我其实在一个副项目里用它，那个项目是……不错。

I use it actually for a side project that I have where... Nice.

是的。是的，就是把 2D 图像转成 3D 模型。所以，是的，只是在那里托管模型。

Yeah. Yeah, it's like turn 2D images into 3D models. So, yeah, just hosting like models there.

你觉得怎么样？有什么反馈吗？

What do you think? Any feedback?

我是说，我很喜欢。

I mean, I love it.

不错。很好。

Nice. Good.

嗯，是的，这就是为什么我用它而不是任何竞争对手或其他什么。

Um I mean, yeah, that's why I use it over any competitors or anything.

很好。

Good.

所以在创办 Modal 之前，你在 Spotify，还有 Better.com，我想。没错。是的。嗯，你在 Spotify 做什么？

So before you started Modal, you were at Spotify and also like Better.com I think. That's right. Yeah. Um yeah, what were you doing at Spotify?

我其实在 Spotify 做了很多不同的事情，但我的主要贡献是构建了一个音乐推荐系统。第一个版本是我毕业后在 2008 年、2009 年左右加入时做的。我在 Spotify 待了七年，大部分时间基本上是从零开始构建一个音乐推荐系统，并建立了一个小团队，我们推出了很多功能，比如相关艺术家和 Discover Weekly 等等。所以大量的矩阵分解，大量计算密集型的工作，可以说后来成为 Modal 起源的很多想法，就是构建 AI 应用并意识到基础设施很糟糕，一定有更好的方法。所以 Modal 的很多理由就是构建一个我在 Spotify 多年一直想要的工具。

I actually did a lot of different things at Spotify, but my main contribution was I built a music recommendation system. The first version I joined after school in 2008, 2009 maybe. And spent seven years at Spotify and most of that time was building basically from scratch a music recommendation system and built up a small team and we launched a bunch of features like related artists and Discover Weekly and a bunch of other stuff. So a lot of matrix factorization, a lot of compute intensive stuff and arguably a lot of what sort of later became the genesis of Modal was just building AI applications and realizing like the infrastructure sucks, like there's got to be a better way to do that. So a lot of the rationale for Modal was kind of building a tool I always wanted to have throughout my years at Spotify.

是的。所以 Modal 更多是你在 Spotify 学到的东西的延伸，而不是从 Better.com 来的。

Yeah. So Modal was more of an offshoot of the stuff you learned at Spotify, not much from Better.com.

Better.com 算是一段小插曲。有点像过山车。关于这家公司的故事很长，可以单独讲，但我在 Better 当了 5 年 CTO，把技术团队从我想是两三百人发展起来。所以学到了很多关于管理、团队建设和招聘的东西，也学到了很多关于开发者生产力的东西。思考了很多关于工具之类的事情。做了更多全栈的东西，但之后又回到了我内心亲近的东西，那就是基础设施、AI 和机器学习。

Better.com was a little bit of a side tour. It was kind of a roller coaster. Long story about the company, talk about it separately, but I was a CTO for 5 years of Better and grew the tech team from I think two to three hundred. So learned a lot about management and building teams and hiring and also learned a lot about developer productivity. Thought a lot about tools and things like that. Did a lot more like full stack stuff but yeah kind of came back to what's close to my heart which is infrastructure and AI and machine learning after that.

Better.com 是做什么的来着？

What was Better.com doing again?

抵押贷款。

Mortgages.

好的。

Okay.

是的。

Yeah.

你是怎么进入那个领域的？我的意思是我们不必花太多时间在那上面。只是

How did you get into that? I mean we don't have to spend too much time on that. Just

是的。不，没关系。就像这个叫 Zack Frankle 的人，我当时不认识他，后来他把我招到 Better 当 CTO，公司当时有 10 个人，不知什么原因，创始人 Vishal 是个非常有趣的人，非常非常聪明，有点疯狂，但主要是积极的方面，我们相处得很好，所以我们 2015 年开始构建它，我花了 5 年发展那家公司，然后最终发现做抵押贷款相当难。疫情后利率上升得非常快，然后给公司造成了很多问题，但那时我已经离开了。

Yeah. No, it's fine. Like this guy Zack Frankle who I didn't know at that time ended up recruiting me to Better as the CTO and the company was 10 people and for whatever reason the founder Vishal very interesting guy very very smart a little crazy in mostly positive ways we got very well along so we started building it in 2015 and I spent 5 years growing that company and then eventually turned out that doing mortgages is pretty hard. And interest rates went up very quickly after the pandemic and then that caused a lot of issues at the company but by then I had already left.

你离开的时候大概是 2020 年、2021 年。

That was like 2020 2021 when you left.

差不多吧。是的。

Yeah something like that. Yeah.

所以好吧，你是在有 Modal 的想法之前离开的，还是离开后开始

So okay you left before you had the idea for Modal or you left and then you started

很难说。我离开 Better 时有很多随机的想法，其实我不确定是否想创办公司。你知道，疫情期间我几乎像休了个假。我有两个孩子。我的意思是我仍然有两个孩子，但当时他们我想是两岁和四岁，所以我决定放松一下，休息一段时间，做点兼职，捣鼓一些东西，不确定也许我应该做些咨询，我开始构建东西，然后我对这个最终成为 Modal 的想法超级兴奋，但实际上我构建的 Modal 是我构建的第三个东西，所以之前有几次尝试都无疾而终，但 Modal 是第一个让我觉得“这挺酷的，我应该做这个。这可能是一个很酷的平台”的想法。

It's hard to say. I had a lot of random ideas when I left Better and I actually wasn't sure if I wanted to start a company. I was kind of just, you know, during the pandemic, I sort of almost took like a sabbatical. I had two kids. I mean I still have two kids but I mean at that time there were I think two and four and so I decided to hang out and took some time off and was kind of working like part-time like hacking on certain things and wasn't sure maybe I should do some consulting and I started building stuff and I just got like super pumped about this idea that eventually became Modal but I actually built you know Modal was like I think the third thing I built that I got you know that so there were like a few attempts before then that went nowhere but Modal was like the first idea was like this is kind of cool like I should do this. This is like this could be a really cool platform.

其他想法也是类似的方向吗，比如开发者工具？

Were the other ideas like also in a similar vein like developer tools?

第一个东西是某种网页抓取工具，但没成功，不过事后看来也许能有所成就。

The first thing was like some sort of web scraping thing which went nowhere but actually in hindsight maybe could have gotten somewhere.

这是在 ChatGPT 发布之前吗？

And was this before the ChatGPT launch?

是的。是的。2021 年。

Yeah. Yeah. 2021.

好的。

Okay.

所以 ChatGPT 是 2022 年底。对。然后我开始构建，所以也许稍微回溯一下背景，我在 Spotify 工作时发布了一个开源框架叫 Luigi，是工作流编排。这和 Airflow、Prefect、Dagster 属于同一类。某种程度上有点过时的类别，人们不太谈论了，但有一段时间它得到了很多关注。总之我从未把它变成生意。但几年后，2021 年我在思考商业想法或要做的事情时，我又回到了工作流编排，开始捣鼓一堆在云中运行代码的想法。我意识到，归根结底，核心难题不仅仅是编排算力，而是执行算力，我意识到这实际上是一个更大的问题。所以某种程度上我把它搁置了，然后说实际上我要去做这边这个其他事情，我稍后会回到工作流调度，但我再也没有回去。

So ChatGPT was late 2022. Right. Then I started building so maybe actually to rewind a little bit context like when I worked at Spotify I released an open source framework called Luigi which is like workflow orchestration. This is sort of the same category as Airflow and Prefect and Dagster. Sort of dated category to some extent like people don't talk about as much but for a while it was like you know got a lot of attention. Anyway I never turned it into a business. But a couple years later when in 2021 when I was thinking about business ideas or things to work on I kind of came back to workflow orchestration and I started hacking on a bunch of ideas of running code in the cloud. And what I realized was like at the end of the day the core puzzle piece is not just orchestrating the compute it's executing the compute and I realized that's kind of a bigger problem actually to solve. So in a way I actually left it on the shelf and then said actually I'm going to go to this other thing over here and I'll get back to the workflow scheduling in a little bit but I never got back to it.

长话短说，最后我们做出了 Modal。重点更像是，我们得想办法让在云端运行代码变得简单。一开始重点相当通用——其实从某种意义上说，我们最近又绕回了原点——但我们关注过很多不同的应用。想法就是，让在云端运行代码变得超级简单。可以是 GPU 代码、AI、机器学习，也可以是数据相关的东西。我们非常关注如何扩展它，如何让开发者体验变得很棒。我们很早就意识到，要做到这一点，就必须深入技术栈底层，构建自己的文件系统、自己的容器运行时。我们解决了很多艰难的工程挑战。

Long story short, ended up building Modal. The focus was more like, let's just figure out a way to make it easy to run code in the cloud. The focus was initially fairly general purpose—and actually, in a way, we've kind of gone full circle more recently—but we focused on so many different applications. The idea was, let's just make it super easy to run code in the cloud. It could be GPU code, AI, machine learning, could also be data stuff. We focused a lot on, how do we scale it? How do we make the developer experience awesome? We realized super early that in order to do that, we'd have to go very deep in the stack and build our own file system, build our own container runtime. We solved a lot of hard engineering challenges.

然后你提到了 ChatGPT。那其实不算杀手级应用，但如果你记得那年秋天，差不多同时出现了另一个东西，就是 Stable Diffusion。而 Stable Diffusion 成了我们当时所做事情的杀手级应用。那时我们只有四五个人。所以我们开始看到很多人来找我们，想运行 Stable Diffusion 模型。然后我们意识到，也许生成式推理对我们来说其实是个非常好的用例。

And then you mentioned ChatGPT. That was actually not quite the killer app, but if you remember that fall, another thing came out around the same time, which was Stable Diffusion. And Stable Diffusion turned out to be a killer app for what we were building at that time. We were only four or five people at that time. So we started seeing a lot of people coming to us wanting to run Stable Diffusion models. And then we realized, maybe generative inference is actually a really good use case for us.

他们是怎么找到你们的？

How were they finding you?

我是个强迫性推文发布者。以前更是这样——现在太忙于建公司了——但我一直发很多推文，一直写很多博客。我想我有一点粉丝基础。所以早期，很多增长动力就是我强迫性地发关于 Modal 和我正在构建的东西的推文，让人们兴奋起来，写长篇博客文章谈论云计算的走向等等。

I'm this compulsive tweeter. I was even more so before—now I'm too distracted by building a company—but I always tweeted a lot, I always blogged a lot. I think I had a little bit of a following. So in the early days, a lot of what drove the growth is just me compulsively tweeting about Modal and what I was building, getting people excited, and writing long blog posts about where the cloud is going and all these things.

我称之为有点像创始人主导的营销。我总觉得你必须发挥自己的长处。而我的长处，我想就是发帖和永远在线。所以现在仍然——我觉得我们很多增长就是因为我们擅长社交媒体，经常在线。

And I call it a bit like founder-led marketing. I always feel like you kind of have to lean into what you're good at. And what I'm good at is, I guess, just posting and being terminally online. So it's still—I think a lot of our growth is we're just kind of good at social media and just being online a lot.

不错。你们多久才有了第一个客户？

Nice. How long until you had your first customer?

花了一段时间。

It took a while.

你们早期融资了吗？

Also did you raise like early on?

我们有一段时间没有融资。实际上我自己出资了一段时间，甚至直到最初几名员工，这在某种程度上挺好的，因为它让我对钱非常节俭，希望看到投资回报。很长一段时间我都没拿薪水。到了某个时候这成了问题，因为我需要钱。但我们有一段时间没融资，直到 2022 年初我们融了一轮种子轮。所以那时我已经做了大约一年。Akshat 大约五年前加入，担任 CTO 和联合创始人，大概是 2021 年 8 月。但直到 2022 年底，我们才看到任何有意义的收入。

We didn't raise for a while. I actually self-funded it for a while, even up until the first few employees, which in a way was kind of nice because it made me really frugal about money and wanting to see ROI. I wasn't taking out a salary for a long time actually. At some point it became a problem because I needed the money. But we didn't raise for a while, until early 2022 we raised a seed round. So at that point I'd already been working on it for about a year. Akshat joined as the CTO and co-founder about five years ago, about now—August, maybe 2021. But then it took until basically late 2022 for us to see any meaningful revenue.

我猜那是 ChatGPT 出来的时候。

I guess that's when ChatGPT came out.

是的，然后 Stable Diffusion 在我们这里成了主要的东西——所以我们从 2022 年 11 月的每月 100 美元，下个月做到了每月 2000 美元，然后 2023 年 1 月突然做到了每月 3 万美元。所以我终于感到胸中那块巨大的焦虑消失了。我想，也许这真的——我们找到了方向。感觉我们找到了方向。然后 2023 年我们增长非常快，我想从那以后我们基本上一直增长很快。

Yeah, and Stable Diffusion then, in our case, was the main thing that sort of—so we went from like $100 a month in, I think, November 2022, and the next month we did like $2,000 a month, and then January 2023 we did like $30,000 all of a sudden. So finally I felt this massive anxiety thing off my chest. I'm like, maybe this is actually—we're on to something. It feels like we're on to something. And then in 2023 we grew very fast, and I guess we've been growing very fast since then, basically.

那个每月 3 万美元的时刻——是几个大客户还是很多样化？

That $30k a month moment—was that just a couple of big customers or was it very diverse?

就一个。我不知道我有没有谈过那是谁，他们自己是否知道，但我想他们怀疑是纽约一家叫 Mage 的公司。他们公司做的是 Stable Diffusion 相关的东西。所以在 2023 年第一季度某个时候，我想他们占了我们收入的 80-90%。但之后开始稍微多样化了。

It's just one. And I don't know if I ever talked about who that was, and if they even know it, but I think they suspect it was a company in New York called Mage. They're doing like Stable Diffusion as a company. So at some point, in the first quarter of 2023, I think they were like 80-90% of our revenue. But then after that it started diversifying a little bit.

2023 年底，我们有了 Suno 这个大客户。他们有一段时间也占了我们收入的很大一部分。但随着时间的推移，我们现在实际上收入非常多样化。

Late 2023 we got Suno as a big customer. They were a big chunk of our revenue for a while too. But over time, we actually have really good revenue diversification now.

是的。对于 Stable Diffusion 的东西，我记得用它来玩 ComfyUI。我猜很多人都在用 ComfyUI，比如在上面运行 ComfyUI 工作流。

Yeah. For the Stable Diffusion stuff, I remember using it for like ComfyUI. I'm assuming a lot of people were using ComfyUI, like running ComfyUI workflows on it.

有些人是的，但我认为使用 Modal 的人往往更偏向底层的东西，他们做复杂的事情。他们喜欢 Modal 的地方在于它一直都很可编程。所以他们部署了自己的自定义工作流，带有适配器、自定义推理和大量优化。而我们为了让开发者体验变得非常好所做的所有事情，结果也非常适合运行生产工作负载。因为自动扩缩容其实是一个非常相似的问题——你希望容器能非常快地扩缩容。所以我们恰好处于非常有利的位置，在早期抓住了很多这类工作负载。

Some people were, but I think the people that were using Modal tended to be more into low-level stuff, like they did complicated stuff. What they like with Modal is it's always been pretty hackable. So they deployed their own custom workflows with adapters and custom inference and a lot of optimizations. And all the stuff we had done in order to get the developer experience to be really good turned out to be really good for also running production workloads. Because autoscaling turns out to be actually a very similar problem—you want to scale containers up and down very fast. So we just happened to be very well positioned to capture a lot of those workloads in the early days.

自动扩缩容的工具不存在吗？为什么你们必须自己构建这些？

Did tools for autoscaling like not exist? Like why did you have to build those yourself?

总的来说，我的论点是，每个新的技术时代都会遇到工具的局限——无论你回顾网络、云、移动、PC、AI 还是别的什么。所以我认为经常是工具已经存在，但往往不是特别好。而且通常你需要重新思考很多工具，才能获得好处，真正加速并为这个新世界构建东西。你在云上看到了这一点——我们不得不发明 Kubernetes 和 Docker 之类的东西。类似地，在生成式 AI 中，你可以在上一代工具上构建这些东西——你可以在 Kubernetes 和 Docker 等之上构建这些东西——但这些技术是为非常不同类型的工作负载构建的。它们是为非常稳定的后端工作负载构建的。

In general, my thesis is that every new era of technology runs up against limitations of tools—whether you look back at the web, the cloud, mobile, PCs, AI, whatever. So I think it's often the case that tools exist, but they're often not particularly good. And often you need to rethink a lot of those tools in order to reap the benefits and really turbocharge and start building for this new world. You saw this with cloud—we had to invent things like Kubernetes and Docker and things like that. In a similar way, with Gen AI, you could build these things on the prior generation of tools—you can build these things on top of Kubernetes and Docker and these things—but those technologies are built for a very different type of workload. They're built for very steady backend workloads.

他们有一个稳定数量的请求，运行在相当低的利用率下，因为 CPU 通常很便宜，所以人们不太担心成本。但现在你进入了这个新世界，你有非常昂贵的 GPU，你需要管理容量和利用率。你有非常大的模型，需要通过从网络加载到 CPU、再到 GPU。而事实证明，很多这些工具并不是为此而建的。所以我们觉得必须重建很多这样的东西。我们在构建自己的文件系统，我提到过，但过去几年我们也花了很多时间构建一些非常酷的东西，比如 CPU 快照、GPU 快照，还有自动——我们构建了自己的自动扩缩器和调度器，以及各种容量管理工具。我们觉得你必须这样做，因为对于新一代软件，你不能在旧东西上构建一个包装层来实现新功能。你有点需要从头开始。

They have an even, you know, steady number of requests, like running at pretty low utilization, because CPUs are normally pretty cheap, and so people don't really worry so much about cost. But now you're entering this new world, and you have very expensive GPUs, you want to manage capacity and utilization. You have very large models that you need to load over network into CPU, into GPU. And it turns out like a lot of these tools were just not built for that. So we felt that we had to rebuild a lot of that. We're building our own file system, I mentioned, but we also spent a lot of time over the last few years building some really cool stuff around CPU snapshotting, GPU snapshotting, and auto—we built our own autoscaler and scheduler and all kinds of capacity management tools. And we kind of feel like you have to do that because you can't build for this new generation of software on top of—you can't build a wrapper on top of the old stuff in order to do the new stuff. You kind of need to start over a little bit from scratch.

那么我猜，像自动扩缩这些，你们在平台上有没有看到任何奇怪的行为？比如在扩展过程中有什么东西坏掉了吗？

So I guess like the auto scaling doing stuff like did you see any weird behaviors on the platform? Like was there anything that broke while you guys were scaling?

我的意思是，容量时不时总是一个挑战，比如你如何获得数百、数千、数万块 GPU。所以这总是具有挑战性。我们很早就必须走向多区域。事后看来，我认为那是一个决定性的决定，咬紧牙关去构建它。我们一直把容量管理看作一个大池子，就像全世界一个大池子。我们不把它看作每个区域运行的独立集群。所以这是我们早期做出的非常有影响的决定之一，以那种方式思考。所以我不知道是否有具体的问题。我的意思是，我们显然有过 bug 和宕机，我讨厌——尽管我非常希望 Modal 能达到 100% 的正常运行时间，而且我认为最近我们基本上达到了四个九或五个九，但在 Modal 的历史中，当然，我们遇到过扩展问题之类的事情。

I mean capacity is always from time to time been a challenge, like how do you get hundreds and thousands and tens of thousands of GPUs. So that's always challenging. We had to go multi-region very early. I think that was a fateful decision in hindsight, doing that and just biting the bullet and building it. We always thought of our capacity management as one big pool, like it's one big pool over the entire world. We don't think of it as separate clusters running in each region. And so that was one of the very consequential decisions we made earlier, to think about it that way. So I don't know if there's been specific issues. I mean, we've obviously had bugs and outages, and I hate—as much as I would love for Modal to be 100% uptime, and I think recently we've basically hit four or five nines, but throughout the history of Modal, of course, we've had run up against scaling issues and things like that.

GPU 市场，嗯，实际上在那之前，你们底层使用 AWS、GCP 等所有这些云提供商，你们也购买任何基础设施吗？还是不？让我们谈谈云容量，因为我认为它实际上变化很大。

The GPU market and well, actually before that, so you guys use AWS like GCP like all these cloud providers underneath like do you also buy any infrastructure or no? Let's talk a bit about like cloud capacity because I think it's actually changed a lot.

就像在最初几年，我们几乎所有的容量都来自超大规模厂商的竞价和按需容量，主要是竞价容量，运行在所有不同区域，最初使用 AWS，但也用 GCP，后来用 Azure。我们很早就开始从 Oracle 租用一些容量，因为我们想要 A100 和 H100，而且我们拿到了非常好的价格。随着时间的推移，我们增加了几家云，做预留。但直到去年年底，我们的大部分容量仍然是竞价或一年期预留。

Like in the first few years we got almost all the capacity pretty much from the hyperscalers' spot and on-demand capacity, mostly spot capacity running in all the different regions, using initially AWS but also GCP and over time Azure. We started renting some capacity from Oracle pretty early because we wanted A100s and H100s and we got really good rates on those. And over time we added a couple more clouds, doing reservations. But still up until actually end of last year, most of our capacity was either spot or one-year reservations.

竞价意味着你就像当场租用？

Spot means you're just like renting it right there and then?

没错。是的。而且它也是可抢占的，我们必须处理这个问题。但它总体上有效，因为你可以从许多，基本上是数百个区域聚合容量。所以那个模式运行得很好，直到我们再次遇到瓶颈。我的意思是，首先，Modal 开始增长得非常快，所以我们需要更多容量，但根本上云市场也开始发生很大变化。所以去年，基本上现在为了获得我们需要的容量，我们必须与更多的云合作。实际上我们现在与近 25 家云合作。在许多情况下，为了获得我们想要的容量，我们必须去做三年以上的交易，并且你必须预付一大笔钱。而且你锁定三年的价格，对吧。所以我们不得不花整个过去 6 到 9 个月重新思考我们如何获得容量。我们不得不建立一个完整的采购部门，就像我们有四个人全职工作，基本上他们所做的就是采购容量，打电话给新云，做交易，审查容量。我们不得不建立一个完整的工程团队，基本上 10 个人，只是管理来自世界各地的裸金属容量，处理网络问题和坏驱动器以及 GPU 从我们这里掉线等等所有这些事情。我们不拥有容量。

Exactly. Yeah. And it's also preemptable which we had to deal with. But it sort of works in aggregate because you can sort of just aggregate capacity from many, you know, hundreds of regions basically. So that model worked really well until we started running up again. I mean first of all, Modal started growing very fast and so we needed more capacity, but fundamentally also the cloud market started changing a lot. So in the last year basically now in order to get the capacity we need we had to work with a much larger set of clouds. We actually work with almost 25 clouds at this point. And in many cases in order to get the capacity we want we have to go and do three plus year deals and you have to pay a meaningful amount of money up front. And you're locking in pricing on a three-year term right. So we had to spend the entire last 6 to 9 months rethinking how we get capacity. We had to build a whole procurement desk, like we have four guys working full-time basically all they're doing is procuring capacity, calling neoclouds, doing deals, vetting the capacity. We've had to build a whole engineering team basically 10 people just managing bare metal capacity from all over the world and dealing with network issues and bad drives and GPUs falling off the us and all these things. We don't own capacity.

到目前为止，我们租用所有容量。通常是三年期交易，这是一个重大决定，比如我们应该买吗，我们应该还是继续租。我实际上并不特别教条。我认为租用很好，因为它有一定的资本效率。如果你购买，首先，有明显的运营复杂性，比如你必须与供应商和电力、电缆等各种东西打交道。但也有财务复杂性，比如你如何融资？你应该筹集股权吗，然后你可以筹集十亿美元去买硬件。那是一种非常稀释的方式。你可以借债，但如果你借债，你通常希望有锁定的长期收入来支付利息。所以那也不简单，但我们看经济性，看运营复杂性，看风险。这是一个复杂的权衡，我们对购买持开放态度。还有其他结构，比如为了获得 2027 年所需的容量，需要做很多工作，走很多不同的途径来锁定它。

We rent all the capacity so far. On typically the three-year deals and that's a big decision like should we buy, should we or should we keep renting. I'm actually not mega dogmatic about it. I think renting is nice because there's a certain amount of capital efficiency to it. If you're buying, first of all, there's an obvious operational complexity of it, like you have to deal with vendors and suppliers and power and cables and all kinds of stuff. But there's also financial complexity to it, like how do you fund it? Should you raise equity, and then you can raise a billion dollars and go buy hardware for it. That's a very diluted way to do it. You can raise debt, but if you raise debt, you typically want to have locked in long-term revenue to pay the interest rates. So that's also not trivial, but we look at the economics, we look at the operational complexity, we look at the risks. It's a complex trade-off and we're open to buying. There's other constructs too, like to get the capacity we need for 2027, it's going to take a lot of work on going a lot of different avenues to lock it up.

那么你需要雇佣很多财务人员来做所有这些事情吗？还是不太需要？

So do you have to hire a lot of like finance people to do all this stuff or not really?

你知道，我不得不花很多时间在这上面，我们也花了很多时间在这上面，而且我确实认为随着时间的推移，我的意思是，有很多——这些是非常昂贵的资产，硬件，我认为这有点惊人。你开始看到 AI 投资开始成为美国 GDP 的几个百分点，你知道，开始看到美国的利率在过去几周飙升。

You know, I've had to spend a lot of time on this and we've had to spend a lot of time on this and I do think over time, I mean, there is a lot of—these are very expensive assets, the hardware and I think I mean, it's kind of staggering. You're starting to see like AI investments starting to become like several percentage points of GDP of the US and you know, starting to see like interest rates in America shooting up in the last few weeks.

我的意思是，在这个阶段，你看看那些新云公司——它们是非常非常复杂的金融工程公司。所以是的，我们关注这一点，我们考虑所有选项，可能我们自己也必须做类似的事情。我不太想——我更愿意专注于做一家非常精简的软件公司，做我们擅长的事情。总的来说，我不认为软件公司拥有硬资产是好事，就像你应该拥有自己的办公室吗？我不认为这对大多数公司来说是必然正确的解决方案。但当然，为了获得你想要的产能，一切都可以谈。

And I mean, at this stage, you look at the neoclouds—these are very, very sophisticated financial engineering companies. So yeah, we look at that, we look at all the options, and it's possible we might have to do similar things ourselves. I wouldn't want to—I would rather focus on being a very lean software company, doing the things we're good at. I don't think it's good in general for software companies to own hard assets, in the same way as, should you own your office? I don't think that's necessarily the right solution for most companies. But of course, in order to get the capacity you want, everything's on the table.

是的。我猜供应会在一段时间内受到限制。所以你不得不这么做。你同意这一点吗，还是你认为这只是一个暂时的现象？

Yeah. And I guess the supply is going to be constrained for a while. So you kind of have to do that. Do you agree with that, or do you think this is just a moment in time?

我不一定同意这一点。我的意思是，英伟达每月交付数十万块 Blackwell 芯片，对吧？我不想说市场是 100% 有效的，但总的来说，如果你愿意支付高价，你就能获得 GPU 产能，对吧？如果你愿意比其他人出价更高。当然，Anthropic 和 OpenAI 正在尽可能多地购买。但从根本上说，市场上不断有供应出现，所以我不一定认为这是一个完全被垄断的资源。价格在上涨，我确实认为值得反思一下：硬件的正常状态就像重力一样——最终硬件价格总是会下降。所以我们正处于一个奇怪的时期，硬件价格在上涨。很难说这会持续多久——可能是另外 6 个月，也可能是另外 36 个月。真的很难知道会持续多久。但我从根本上不知道“你需要购买产能”这种说法是否真实。对我来说，这有点像是疫情期间的卫生纸短缺之类的，就像是银行挤兑的情况。但话虽如此，产能在一段时间内会紧张，价格也在上涨。幸运的是，我认为价格上涨对我们来说甚至不一定是坏事，因为稀缺也意味着我们可以向客户收取更多费用。而且我们有很多预订以固定价格锁定，这意味着在某种程度上，我们实际上已经做多了 GPU 价格。所以我不一定太担心价格上涨。我更担心价格崩溃。我认为这不会很快发生，但我认为这对市场上的许多参与者来说将是一件非常糟糕的事情。

I don't necessarily agree with that. I mean, Nvidia is delivering several hundred thousand Blackwells a month, right? And I don't want to say the market is 100% efficient, but in general, if you're willing to pay a high price, you can get GPU capacity, right? If you're willing to pay more than everyone else. And of course, Anthropic and OpenAI are buying up as much as they can. But fundamentally, there's constantly supply coming on the market, so I don't necessarily think it's a fully cornered resource. Prices are going up, and I do think it's worth reflecting on that: the normal state of hardware is like gravity—eventually hardware prices always go down. So we're in a weird period of time where hardware prices are going up. It's hard to say how long this is going to last—could be another 6 months, could be another 36 months. It's really hard to know how long. But I fundamentally don't know if it's true to say you need to buy capacity. To me, that's a little bit like the great toilet paper shortage of the pandemic or whatever, like a run on the bank type situation. But that being said, capacity is going to be tight for a while, and prices are going up. Luckily, I think it's not even necessarily bad for us if prices go up, because scarcity also means we can charge more from our customers. And the fact that we have a lot of reservations locked in at a rate means that to some extent we're effectively long GPU prices already. So I don't necessarily worry too much about prices going up. I do worry more about prices collapsing. I don't think that's going to happen anytime soon, but I think that would be quite a bad thing for a lot of the players in the market.

为什么那会很糟糕？

Why would that be bad?

我认为这对任何锁定大量供应的人来说都是糟糕的。

I think it would be bad for anyone who's locked in a lot of supply.

是的。因为他们支付了过高的价格，而且

Yeah. Because they paid way too much and

他们支付了太多，你知道。所以存在在高点买入的风险，对吧？再说一次，我不太担心这个。我不认为这很快会发生，但就像我说的，从长远来看，硬件价格往往会下降。所以值得反思一下，对吧？就像现在，我认为你的产能受到限制。从经济学术语来说，这导致了稀缺租金，对吧？谁获得了稀缺租金？我认为这主要流向两个地方：主要是英伟达，但当然，在某种程度上也流向新云公司——新云公司和一般拥有产能的人有更好的经济效益。这在今天产能受限的情况下是无可否认的。长期来看，在非常长的运行中，我不认为这必然完全正确。拥有 GPU 总是能获得一点溢价，但值得冒险吗？在你的资产负债表上持有这些非常昂贵的资产值得冒险吗？我不太确定。但我们会看到——需要几年时间才能度过这个阶段。从长远来看，我们可能会拥有一些，可能会租用一些，也许是组合，也许做其他疯狂的事情，合资企业，谁知道呢，也许一些复杂的 SPV 之类的。一切都可以谈。

they paid too much, you know. So there's a risk of buying at the top, right? And again, I don't worry about that that much. I don't think it's going to happen anytime soon, but like I said, hardware prices in the very long run tend to go down. And so it's worth reflecting on that, right? Like right now, I think your capacity is constrained. And that leads to, in economic terms, there's a scarcity rent, right? Who captures the scarcity rent? I think that primarily goes to two places: mostly Nvidia, but of course, also to some extent neoclouds—neoclouds and in general anyone owning capacity has better economics. That's undeniably true today with capacity being constrained. Long term, in the very long run, I don't think that's necessarily entirely true. There's always going to be a little bit of a premium you can get from owning GPUs, but is it worth the risk? Is it worth the risk of holding these very expensive assets on your balance sheet? I'm not so sure. But we're going to see—it's going to take a few years to get through this. And in the long run, we might own some, we might rent some, maybe a combination, maybe do some other crazy stuff, joint ventures, who knows, maybe some complex SPVs or whatever. Everything is on the table.

你如何与 OpenAI 和 Anthropic 竞争购买 GPU 产能？我猜他们想买光所有可用的东西。

How do you compete with OpenAI and Anthropic in buying GPU capacity? I'm assuming they'd want to buy up literally everything that's available.

他们确实在这么做。我认为你看到的价格上涨很大程度上反映了这一点。话虽如此，英伟达正在运送大量产品。我认为在某种程度上，英伟达也偏好一点多元化——他们不想要少量的大买家，就像你看到超大规模企业获得有限分配一样。我认为英伟达更愿意看到许多新云公司和替代云获得更多分配。我认为市场上我们看到的一部分也是——这更多是猜测——但我认为我们看到一些风险稍高的小型云公司对我们来说更像是机会。也许英伟达——抱歉，Anthropic 和 OpenAI——我的感觉是他们更愿意从经过考验的可靠公司购买。部分原因也是因为他们必须进行大量训练运行，所以需要非常大的区块。但那里有充足的产能。你可以在亚美尼亚获得 32 个节点，在印度尼西亚获得 128 个节点。我们喜欢那种东西。所以我们可以出去，在世界各地以相当好的价格获得大量较小的产能。

They are. And I think what you're seeing is that a lot of the prices going up is a reflection of that. That being said, Nvidia is shipping a lot of stuff. I think also, to some extent, Nvidia prefers a little bit of diversification—they don't want a small amount of large buyers, in much the same way as you see hyperscalers getting a little bit of a constrained allocation. I think Nvidia prefers to see a lot of neoclouds and a lot of alternative clouds getting more allocation. And I think a little bit of what we see in the market is also—this is more speculation—but I think we see smaller clouds that are a little bit more risky to work with being more of an opportunity for us. Maybe Nvidia—sorry, Anthropic and OpenAI—my feeling is they prefer to buy from the tried and true. Part of it is also because they have to do a lot of training runs, so they need very large blocks. But there's plenty of capacity out there. You can get 32 nodes in Armenia, you can get 128 nodes in Indonesia. We love that kind of thing. So we can go out there and get a lot of smaller capacity all over the world at pretty good prices.

是不是基本上就像如果有人向你提供一定数量的 GPU，那就是你关心的全部？你不在乎可靠性吗？是不是就像某个破旧的仓库里有人放着这些？

Is it pretty much just like if someone offers you a certain number of GPUs, that's all you care about? Is there nothing like, oh, you care about how reliable it is? Is it just some janky warehouse that someone has these in?

我的意思是，有时候确实有点像破旧的仓库。感觉是这样，但我们基本上有一份规格表。当我们与供应商交谈时，我们做的第一件事就是发送一份规格表。我们说我们需要这个数量的 IOPS。我们需要 CPU 至少——有这么多，不管 PassMark 分数还是什么。

I mean, sometimes it's kind of a janky warehouse. It feels like that, but we do have a spec sheet basically. When we talk to vendors, the first thing we do is we send them a spec sheet. We say we need this number of IOPS. We need the CPU to be at least—have this many, whatever the PassMark scores or whatever it is.

我们至少需要看到南北向带宽、出口和入口带宽。所以我们确实有一份需求清单。通常我们会尝试做一个验收测试期，拿一个节点跑一段时间测试。某种程度上，这也需要一点信心跳跃。有时我们会签下非常大的承诺——数千万或数亿美元——然后开始与一个我们还不完全了解的供应商合作。随着时间的推移，我们会看到结果。我觉得有点我们自己的……SemiAnalysis 有 ClusterMAX。我感觉我们内部几乎有自己的 ModalMAX 评分。我知道我更喜欢哪些供应商，我们更倾向于与那些我们看到非常可靠性的供应商做更多交易。不仅仅是可靠性，还有：当我们有支持问题时，他们是否非常响应？他们是否修复我们要求的问题？周转时间是多少？如果出故障，他们多快能修复？诸如此类。

We need to see, at minimum, north-south bandwidth, egress and ingress. So we do have a list of requirements. Often we try to do an acceptance testing period where we get one node and run some tests over time. To some extent, it also involves a little bit of a leap of faith. Sometimes we sign very big commitments—tens or hundreds of millions of dollars—and we start working with a vendor that we still don't fully know. Over time, we'll see. I think there's a little bit of our own sort of... SemiAnalysis has ClusterMAX. I kind of feel like we almost have our own ModalMAX score internally. I know which vendors I like more than others, and we prefer to do additional deals with vendors that we've seen very good reliability. Not just reliability, but also: are they very responsive when we have support issues? Do they fix the issues we ask for? What's the turnaround time? If things go down, how quickly do they fix it? Things like that.

你同意对智能的需求基本上是无限的吗？如果是这样，为什么不尽可能多地购买容量，然后如果买多了，你可以直接卖给其他人？

Do you agree that the demand for intelligence is basically infinite? And if so, why wouldn't you just buy up as much capacity as possible and then if you overbuy, you can just sell it to somebody else?

我觉得当资源稀缺时，说“让我们尽可能多买”很容易。但我认为，首先，如果我们今天买了，我们无法利用全部——我们可以利用很多，对吧？但是——

I think it's very easy to say when there's a scarce resource like let's just buy as much as you can. But I think, first of all, if we bought it today, we couldn't utilize all of it—like we could utilize a lot of it, right? But like—

但你可以把它卖给其他想用的人，对吧？

But you can sell it off to somebody else who wants to use it, right?

我认为挑战在于，如果市场上发生了一些事情——很多相关性会同时发生。如果价格下降、利用率下降，而转售容量的市场同时消失。我坐在这里，我非常激进，每天购买容量并筹集资金购买。所以总体我非常看涨，但我想指出这里有尾部风险，当这些尾部风险发生时，可能会非常糟糕。部分原因可能是，我老了，我过去见过这些。我实际上在 1999 年有第一份工作，所以我看到了互联网危机，也看到了金融危机等等。我想建立一家能再存续 20 到 30、50 年的公司，对吧？所以我认为我们现在当然可以出去购买疯狂的容量，试图垄断市场，

I think the challenge is like if there was something in the market that happens—there are a lot of correlations that happen at the same time. If prices go down and utilization goes down and the market for reselling capacity vanishes at the same time. I'm sitting here and I'm extremely aggressive, like buying capacity every day and raising money to buy. So overall I'm very bullish, but I do want to point out that there are tail risks here, and when those tail risks happen, it could get very bad. Part of it is like maybe I'm just old and I've seen this in the past. I actually had my first job in 1999, so I saw the dotcom crisis and I've seen the great financial crisis and things like that. I want to build a company that is around for 20 to 30, 50 more years, right? So I think we can certainly go out and buy an insane amount of capacity right now and try to just corner the market,

但你知道亨特兄弟吗？

But you know about the Hunt brothers?

这是交易界的传奇故事。他们试图垄断白银市场。结果惨败。所以垄断市场，我认为总体上可能不是个好主意。但我们总体上非常非常激进，但我们正以激进的计划扩产，基本上匹配我们的需求增长，并匹配我们明年年底在运行率方面想要达到的目标。

It's like a legendary story in trading. They tried to corner the silver market. It fails spectacularly. So cornering the market, I think in general, can be not a good idea. But we are in general very, very aggressive, but we're ramping up at an aggressive plan that basically matches our demand growth and matches where we want to try to get to at the end of next year in terms of run rate.

你们只购买英伟达的东西，还是也可能从其他类型的公司购买容量？

Are you only buying up Nvidia stuff or also maybe capacity from other types of companies?

我们正在考虑。我认为替代加速器开始变得非常有趣。讽刺的是，过去五年，甚至十年，我每年都说“明年将是非英伟达加速器之年”，然后我总是错的。但我确实认为 2027 年可能最终是非英伟达加速器之年。所以我们将看看我今年是否也错了。但 AMD 在某些事情上开始得到非常好的分数。我认为 TPU 显然在谷歌内部被广泛使用，似乎运行得很好。我对 Trainium 了解不多。我感觉我没看到那么多，但团队非常优秀。我知道他们正在做很多软件方面的工作。硬件在某种程度上很难，软件更容易，这让我觉得如果我们只是修复这些加速器的软件层，我认为会有很多需求。而且我认为从根本上说，由于英伟达是一个瓶颈，我们可能会看到更多替代加速器。话虽如此，管道下游还有瓶颈。我的意思是，有台积电和 ASML，也许这是所有这一切的更深的瓶颈。但长期来看，我认为垄断并不总是持续。所以我认为市场有更多替代品是健康的。尽管我非常喜欢英伟达——要明确这一点——他们显然是一家了不起的公司，有着了不起的产品。我确实预计市场会看到更多的多样性，还有像 Cerebras 和 Groq 这些东西。

We're looking at it. I think the alternative accelerators are starting to get very interesting. Ironically, every year in the last five, maybe even 10 years, I've been saying next year is going to be the year of non-Nvidia accelerators, and then I'm always wrong. But I do think 2027 might be the year of non-Nvidia accelerators finally. So we'll see if I'm wrong this year, too. But AMD is starting to get really good scores on certain things. I think TPUs are obviously used very widely internally at Google and seem to work really well. I don't know much about Trainium. I don't feel like I see quite as much, but the team is very good. I know they're working on a lot of the software stuff. Hardware is hard in a certain way, like software is easier, and that makes me feel like if we just fix the software layer for these accelerators, I think there's going to be a lot of demand for them. And I think fundamentally, with Nvidia being such a bottleneck, we'll probably see more alternative accelerators. That being said, there are bottlenecks further down the pipe. I mean, there's TSMC and ASML, maybe a deeper bottleneck to all of this anyway. But long-term, I think monopolies don't always last. So I think it'd be healthy for the market to have more alternatives. As much as I love Nvidia—to make that clear—they're obviously an amazing company with an amazing product. I do expect to see a little bit more diversity in the market as well, also things like Cerebras and Groq and these things.

是的。你对其中一些供应商有什么看法？

Yeah. What are your thoughts on some of those providers?

我认为它们相当令人兴奋，你看到它们在某些工作负载上做得非常好。有一些有趣的人在做混合分离式推理，他们用英伟达做预填充，用 Cerebras 做解码等等。

I think they're pretty exciting, and you see them doing really well on certain workloads. There are some interesting people doing hybrid disaggregated inference where they run Nvidia for the prefill and Cerebras for the decode and things like that.

我实际上从未听说过。

I've never heard of that actually.

是的。还有 Groq，因为很多这些东西——某些是计算密集型，某些是内存密集型——所以你可以混合搭配，我认为这相当令人兴奋。在实践中部署这些东西时，它开始变得非常复杂。但我确实认为在这些更复杂的硬件设置中有很多机会。

Yeah. And Groq too, because a lot of these things—certain things are compute-bound, certain things are memory-bound—so you can sort of mix and match, and I think that's quite exciting. It starts to get very complex when you're deploying these things in practice. But I do think there's a lot of opportunity in these more complex hardware setups.

你从这些新云公司，比如 Nebius 或其他什么的，以及他们做事的方式与你们的方式相比，有什么学习吗？

Are there any learnings that you get from some of these Neoclouds like, I don't know, Nebius or whatever, and the way that they're doing things versus what you're doing?

我认为新云公司非常有趣的一点是，它们如何能如此快速地扩张。我认为实际上最疯狂的例子是 Nscale，它从无到有，你知道，尽管几乎没有容量，却有数千亿美元的管道。

I think one of the things that is very interesting with Neoclouds is how can they expand so quickly. I think actually the most crazy example is like Nscale, which went from nowhere to having, you know, whatever hundreds of billions of dollars in their pipeline despite barely having any capacity.

新云（neocloud）怎么能增长得这么快？我的意思是，基本上答案可能就是显而易见的：他们去找 Anthropic 或 OpenAI 这样的大买家，说：“嘿，你要不要买 5 年的芯片？签这份合同，我们给你好价钱。”然后他们拿着这份合同去华尔街说：“看，我们有个大买家，他们承诺了 100 亿美元的芯片，我们现在只需要钱。”然后他们筹到钱，再去找 Super Micro、Dell 或 Nvidia 买硬件，安装好，然后交付给客户。所以你有这种非常巧妙的金融结构，基本上可以预售你还没买的东西。这意味着你可以极快地扩张。这就是我们在这些新云身上看到的，它们能扩张得非常好。

How can neoclouds grow so fast? I mean, there's like... and basically the answer is like, you know, and this might be obvious, but like what they do is like they go out to like, you know, Anthropic or OpenAI, some big buyer or whatever, and basically say like, hey, do you want to buy like 5 years of chips? You know, sign this contract, we give you a good rate. And then they take that contract and then they go to, you know, Wall Street and say, look, we got this like, you know, big buyer, you know, they committed to $10 billion worth of chips. Uh, we just need the money right now. And they're going to raise the money and then they like, you know, go to like Super Micro and Dell or whatever and Nvidia and like get the hardware and then they install it and then you know now they have like, you know, deliver it to the customer, right? So you have this like very nice sort of financial construct where you can basically sort of pre-sell things that you haven't even bought. And that means, you know, you can scale up extremely fast. And so that's what we've seen with these neoclouds is they can scale very, very well.

我觉得 Modal 一直更偏向产品驱动增长（PLG），没有那种锁定长期合同的能力，所以从财务角度要那么快扩张会更有挑战。尽管我认为长期来看我们会达到同样的水平。但我觉得这些新云非常有意思，尤其是 CoreWeave 和那些大公司，它们在融资和把 GPU 作为贷款抵押品等方面非常精明。

I think Modal, because we've always had a little bit more of a PLG focus and not the sort of ability to lock in long-term contracts in the same way, it's a little bit more challenging from a financial perspective to scale that fast. Even though I think in the long run we'll get to the same point. But I think that's been kind of a fascinating thing to see with these neoclouds, and especially like CoreWeave and those big ones, have been very, very savvy with how they raise money and how to think about GPU as collateral for loans and all these things.

那有什么意义呢？我忘了是谁，但好像是一家金融公司说：“哦，我们要把 GPU 变成一种资产类别。”我觉得其实是 Nvidia。他们说要让 GPU 成为资产类别。这只是因为他们需要融资，想弄清楚钱从哪来吗？这有什么意义？

What's the relevance of like... I forget who it was, but like some financial company was like, oh, we're going to make... oh, I think it was Nvidia actually. They were like, we are going to turn GPUs into an asset class. Is that just because they need to raise money and they were trying to figure out where to get the money from? Like what's the relevance of that?

我的意思是，有一堆初创公司——不知道你是不是在想这个——比如 Silicon Data 和 Compute Exchange 等几家，基本上在尝试创建价格指数、指数和交易衍生品之类的。我觉得这很有趣。我认为算力作为资产类别有许多挑战。它绝对不像其他资产类别那样同质化。你有不同的网络、不同的位置、不同的磁盘和 CPU 等各种东西，还有可靠性问题。话虽如此，石油也是一样，我们某种程度上把石油变成了相对商品化、可交易的资产。这还有待观察。我确实认为，在目前 AI 数据中心 GPU 部署的规模下，我们需要在风险对冲、风险分担以及更多保险等方面更聪明一些。所以我很高兴有很多人在思考这个问题。话虽如此，我不完全确定我们是否会看到 GPU 变成像 Henry Hub、布伦特原油那样超级商品化、可交易的资产。

I mean, there's a bunch of startups—I don't know if you're thinking about that—there's like Silicon Data and Compute Exchange and a few other ones that are basically trying to create price indices and indexes and trading derivatives and things like that. I think it's interesting. I think with compute, there are a number of challenges with compute as an asset class. It's definitely not as homogeneous as other asset classes. You have different network and different locations and different disks and CPUs and all kinds of stuff and reliability. I mean, that being said, you can say the same thing about oil, and we kind of created relatively commoditized, tradable assets out of oil. It remains to be seen. I do think at this scale where we are with the sort of AI data center GPU deployment, we do need to be a little bit smarter with risk hedging and how we share risks and having more insurance and things like that. So I'm very happy that people are thinking a lot about this. That being said, I'm not entirely sure if we're going to see GPUs becoming sort of the Henry Hub, Brent crude type, super commoditized, tradable asset.

好的。我想把话题转到 Modal Sandboxes。

Yeah. I want to shift the conversation to Modal Sandboxes.

好的。

Yeah.

在那之前，我想问一下 Modal 有哪些不同的产品？主要产品显然是 GPU，然后有 Modal Sandboxes。还有别的吗？

I guess like before we do that, what are the different products that Modal has? It's I guess like the main product obviously is like the GPUs, then you have Modal Sandboxes. Is there anything else?

我认为我们有两个大产品：第一是 Modal Functions，第二是 Modal Sandboxes。Modal Functions 是 2021 年推出的。当时的概念是无服务器函数，非常容易部署和扩展，我们管理所有容量。结果发现它非常适合推理，也非常适合管理 GPU 和运行这类任务。但我们其实从来没有专门的推理产品或 GPU 产品。我们从未把 Modal 视为新云，我认为有被卷入那个方向的风险——变成新云，卖那种“矩形”合同：比如三年期，给你这么多 GPU。我们一直擅长的是：你部署一些东西，可以扩展，有这种周期性或突发性的负载，我们管理所有容量。美妙之处在于，因为我们有成千上万的客户，我们可以聚合容量，使工作负载非常平滑，聚合需求非常平滑，我们可以非常高效地管理。所以 Modal Functions 是核心。我们 2021 年推出，2023 年开始看到产品市场契合，现在它约占我们收入的三分之二，赋能很多——实际上主要是推理，但你可以用于各种其他用途。Modal Sandboxes 是 2023 年推出的，它实际上花了一段时间才看到产品市场契合，类似于 Functions。为什么推出它？我们意识到很多我们交谈的人在使用 LLM 输出代码。这就像我和 Asha 某天午餐时聊天。我们说也许他们需要一个地方来运行代码。我们意识到其实我们已经有很多基础设施可以直接运行代码，因为我们已经构建了管理容器和隔离的原语。我们已经有 G Visor 作为隔离原语等。所以我们对这个想法感到兴奋，Nokia 写了一个原型，基本上在一两天内就推出了。零——我们看到增长很少。我不想轻描淡写。我们确实看到了一些增长，有一些——

I think of the two big products that we have is basically number one is Modal Functions and number two is Modal Sandboxes. So with Modal Functions we launched that back in 2021. It was the idea of serverless functions that are very easy to deploy and very easy to scale and we manage all the capacity. That turned out to be really, really good for inference and it turned out to be really good for managing GPUs and running on things like that. But we actually never had like a dedicated inference product or like a GPU product per se. We never thought of Modal as a neocloud and I think there's a risk of getting sucked in that direction, becoming a neocloud, selling what I think was like rectangular—like here's a three-year, you get this many GPUs. We've always been very good at like you deploy something, you can kind of scale it up, you have this very cyclical or bursty stuff, we manage all the capacity. And the beautiful thing is like because we have many, many thousands of customers, we can aggregate that capacity and very smooth workloads, very smooth aggregate demand, which we can manage very, very efficiently. So Modal Functions was the core. We launched that in 2021 like I mentioned. We started seeing product market fit in 2023 and now it's about two-thirds of our revenue and empowering a lot of—in practice it's mostly inference but you can use it for all kinds of other stuff. Modal Sandboxes we launched that in 2023 and it actually took a while to see product market fit similar to Functions. Why did we launch it? We realized a lot of people we talked to were using LLMs to output code. This is like me and Asha talking over lunch one day. We're like maybe they need a place to run that code. And we realized like actually we kind of have a lot of the infrastructure already to just take code and run it because we already built the primitives for managing containers and isolation. We already had G Visor as an isolation primitive and things like that. So we got kind of excited about that idea and Nokia wrote a prototype and launched it in basically one day or two days or something like that. Zero—like we saw very little growth. I don't want to trivialize it. We did see some growth like there were some sort of, you know—

那是 2023 年，所以——

It was 2023 so—

是的，那是 2023 年，有些人开始使用它。我们有一些早期用户喜欢它用于某些事情，但总体而言，到 2023 年夏天，它还是一个相当微不足道的收入来源。所以两年后我们甚至讨论过：我们真的需要这个产品吗？应该关掉它吗？然后它就开始爆炸式增长。所以 2023 年——我认为 2023 年真正发生的是 LLM 在生成代码方面变得非常擅长。

Yeah, it was 2023 and some people picked it up. We had some early users who enjoyed it for certain things, but overall it was a pretty insignificant source of revenue going into the summer of 2023. So two years later we even discussed like do we even need this product? Should we shut it down? Then it just started exploding. So 2023—and I think what really happened in 2023 was like LLMs got really good at generating code.

在 2023 年。2023 年。

In 2023. 2023.

我的意思是，大概就是那个时候。

I mean, it was like around that time.

我觉得那是在他们从能够重构函数和编写代码片段，发展到生成完整应用的时候，对吧？我们开始看到大量增长来自氛围编程，你知道，像 Lovable 这样的公司和其他一些公司。然后像 Ramp 这样的公司开始使用模型来做后台智能体，接着我们开始看到来自许多新实验室的巨大增长，他们在模型沙盒上做强化学习之类的事情。所以沙盒现在约占我们收入的三分之一。我们基本上把它视为我们的第二个产品。对我来说这真的很令人兴奋，因为即使在 Modal 的早期，我一直知道 Modal 是要打造一整套产品工具包。我们一直想几乎像一朵云那样。你有存储和算力，以及各种不同的使用方式。我们一直想成为多产品供应商，但这在实践中并不一定成立，直到现在，我认为现在很清楚了，我们有函数，我们有沙盒，我们还在做很多其他东西。在做训练，在做批处理，我们还在做很多其他东西。所以对我来说，演变成一家多产品公司这件事非常棒，因为那一直是愿景的核心，甚至在很早的演示文稿中也是如此。我有很多关于这个的幻灯片，但直到最近，它才对外界成为现实。

I feel like that was when they went from being able to refactor functions and write snippets to generating full apps, right? We started seeing a lot of growth coming from vibe coding, you know, companies like Lovable and others. Then companies like Ramp started using models for background agents, and we started seeing tremendous growth coming from a number of neolabs doing things like reinforcement learning on model sandboxes. So sandboxes is now about a third of our revenue. We consider it basically our second product. To me it's really exciting because even back in the early days of Modal, I always knew that Modal is about creating a whole toolkit of products. We always wanted to be almost like a cloud. You have storage and compute and a bunch of different ways to work with it. We always wanted to be a multi-product vendor, but that wasn't necessarily true in practice until now, where I think it's clear we have functions, we have sandboxes, and we're working on a bunch of other stuff too. Working on training, working on batch, we're working on a bunch of other stuff. So to me, that evolution into becoming a multi-product company has been amazing because that was always so core to the vision, even in the very early decks. I had a bunch of slides on this, but it wasn't necessarily true to the external world until recently.

你们有没有在研发中的东西，你认为会成为下一个这样的产品？或者对市场有什么预测，之类的？

Do you have anything in the works that you think is going to be like that next product? Any predictions maybe about the market, anything like that?

我是说，我非常看好训练。我认为训练是一个有点被忽视的市场。显然，有很多关注点在非常大的训练运行上。但我们实际上看到中小型训练领域有很多需求。所以我这周晚些时候会稍微谈谈这个。

I mean, I'm very bullish on training. I think training is kind of an underserved market. Obviously, there's a lot of focus on very large training runs. But we actually see a lot of demand in the smaller and midsize training segment. So I'll be talking a little bit about that later this week.

我们正在组织一个会议。

We're organizing a conference.

批处理我认为是一个非常有趣的市场。我认为我们看到很多需求，客户需要——前几天我们有一个客户需要转录 5000 年的音频。他们有数以百万计的音频文件，而我们有很多容量，所以我们让他们做了。花了几天时间。我认为那里有一些非常有趣的用例。非常看好低延迟的东西。我认为一旦我们让语音转文字模型工作起来，比如语音模型，我预计那将是一个非常有趣的市场。然后当然，所有这些智能体式的东西，对吧？在某种程度上，我对“智能体式”这个词有复杂的感受，因为我觉得在某些方面，我认为所有软件都会变得智能体式。所以对我来说，几乎就像，成为智能体式意味着什么？我认为所有软件在某种程度上都会随着时间的推移变成 AI 驱动的。但显然，有一组构建模块用于做更多 LM 驱动的软件，这似乎在所有不同用例中都是常见的。我对 Modal 的思考方式是，我们从事的是构建模块的业务。我们想给你惊人的、可泛化的、经过实战考验的、通用的、可组合的构建模块。而且我认为在智能体用例方面也有一些非常有趣的东西。

Batch is I think like a really interesting market. I think we see a lot of demand for customers that need to—we had a customer the other day that needed to transcribe 5,000 years of audio. They had millions and millions of audio files, and we had a lot of capacity so we let them do that. It took a few days. I think there's some really interesting use cases there. Very bullish on low latency stuff. I think there's going to be a lot once we get the speech-to-text models working, like the voice models, I expect that to be a very interesting market. Then of course, all this agentic stuff, right? In a way, I have mixed feelings about the word agentic because I feel like in certain ways, I think all software will become agentic. So it's almost like to me, what does it mean to be agentic? I think all software in a certain way will become AI-powered over time. But clearly there's a set of building blocks for doing more LM-powered software that seems to be common across all the different use cases. The way I think about Modal is we're in the business of building building blocks. We want to give you amazing, generalizable, battle-tested, general-purpose, composable building blocks. And I think there's some really interesting stuff also with the agent use cases.

你提到训练很有趣，因为我想在另一个播客中，大概一年前，或者两年前，你说训练不是你的重点。具体来说，市场发生了什么变化？

That's interesting you bring up training because I think in another podcast maybe like a year ago, maybe two years ago, you said training you were not focused on. Specifically, what changed in the market?

我是说，我们多年来一直在构建训练产品，断断续续的。我不会说我们从未关注过它，但就像那些事情之一——我是说，现在我们已有 150 名工程师，但一两年前我们的工程师人数要少得多。由于规模较小，我们不得不更专注一些，所以推理成了我们唯一的焦点，并且多年来一直是我们的唯一焦点。但我们现在更大了，所以我们确实有机会在其他市场下注。我也非常感兴趣继续扩展后训练、强化学习方面的事情。我认为那周围有一些非常令人兴奋的东西。我认为还有很多其他东西，比如存储、构建数据仓库和特征存储、实时流处理管道以及各种——我不知道。我认为 Modal 的长期愿景基本上是：你想构建一些酷炫的 AI，我们为你提供支持。我们有所有的构建模块。来找我们，我们会帮助你取用所有这些构建模块并将它们组合起来。这就是软件的美妙之处：如果你有正确的构建模块，你几乎可以构建任何东西。这非常是我们想要建立的那种公司和产品。

I mean, we've been building a training product for multiple years, on and off. I wouldn't say we've never been focused on it, but it's like one of those things where—I mean, now we're 150 engineers, but a year or two ago we were a much smaller number of engineers. By virtue of being smaller, we had to focus a little bit more, and so inference became our sole focus and was our sole focus for many years. But we're bigger now, so we do have opportunities to also place bets in other markets. I'm also really interested in continuing the list of things post-training, reinforcement learning. I think there's some really exciting stuff around that. I think there's also many other things around storage and building data warehousing and feature stores and real-time streaming pipelines and all kinds of—I don't know. I think the long-term view of Modal is basically: you want to build some cool AI, we've got you covered. We've got all the building blocks. Come to us and we'll help you take all these building blocks and compose them. That's the beauty of software: you can build almost anything if you have the right building blocks. That's very much the sort of company and the product we want to build.

当你说训练时，你是更专注于做大规模训练运行的大企业，还是——

When you say training, are you focused on more of the bigger enterprises doing huge training runs or—

不，他们会去做那些大型超大规模云或新云的大交易，获得 10 万块 GPU 之类的。但在我看来，有一个有点被忽视的市场，比如研究人员迭代或人们训练较小的模型。有文本转语音和语音转文本模型。计算生物技术实际上是一个我们看到很多需求的市场。我们在那个领域有很多客户，训练相对适度的模型，你可以在 64 块 GPU 上训练之类的，推荐系统。很多传统的机器学习。有很多东西你不需要 10 万块 GPU。那个市场，我认为我们暂时不会专注于它，但我认为在稍小的市场中，有很多东西相对被忽视。建立训练集群非常困难。

No, they're going to go and do these big hyperscaler or rather Neo cloud mega deals and get 100,000 GPUs or whatever. But there's a little bit of an underserved market, in my opinion, like researchers iterating or people training smaller models. There's text-to-speech and speech-to-text models. Computational biotech is actually a market that we see a lot of demand from. We have a lot of customers in that space, training relatively modest models you can train on 64 GPUs or whatever, recommendation systems. A lot of traditional machine learning. There's a lot of things where you don't need 100,000 GPUs. That market, I think, we're not going to focus on that for a while, but I think there's a lot of stuff in the slightly smaller markets that are relatively underserved. It's very hard to set up training clusters.

是的。你有没有注意到，比如自从我不知道，像 GLM 5.2 或 Kimi K3 或者 Kimi 2.5 以来，很多人都在使用它们并进行微调？

Yeah. Have you noticed, like since I don't know, like GLM 5.2 or like Kimi K3 or maybe like Kimi 2.5, that a lot of people are using those and fine-tuning them?

如果说有什么的话，我实际上看到了相反的方向。

If anything, I've actually seen the other direction.

我觉得几年前微调的价值主张是这样的：开源模型还不够好，但如果你拿一个小模型，在特定领域上微调，你其实可以在质量和成本上达到前沿水平。今天我会说，开源模型已经非常好了，你基本上可以直接替换掉前沿模型，运行 Kimi K3 或 DeepSeek 4.1 或 GLM 5.3，它们几乎和前沿模型一样好，成本却低得多。所以我们仍然看到大量后训练的需求，我确实认为这是一个非常有趣的空间，我们正在这个领域做一些产品，包括 SFT 和强化学习等等。我还想说，微调不再是你必须做的事情，它非常可选。开源模型开箱即用也能工作得很好。

I feel like fine-tuning the value prop a couple years ago was like, yeah, you know, open source models, they're not quite as good, but you know, if you take a small model and you fine-tune it on your specific domain, you can actually reach the frontier in terms of quality and cost. Today I would say the open source models are so good that you can basically just swap them out and just run, instead of the frontier ones, you can run Kimi K3 or DeepSeek 4.1 or GLM 5.3 and they're basically almost as good as the frontier model at a much lower cost. So we still see a lot of demand for post-training and I do think it's a really interesting space and we're working on some products in that space, both SFT and also reinforcement learning and things like that. I would also say fine-tuning is no longer the thing that you have to do, it's very optional. You can open source models like work really well just off the shelf too.

回到沙盒。其实我忘了问你。你们提供的沙盒必须更新哪些现有沙盒不允许的功能？

Back to sandboxes. Actually, I forgot to ask you. So what did you have to update about the sandboxes that you guys were offering that existing sandboxes did not allow for?

没有太多。我认为我们一直非常擅长的事情，我的意思是，我们是基础设施极客，对吧？我们非常擅长扩展和可靠性以及做很多事情。我想这在很大程度上解释了为什么我们拥有——在我看来，我没有具体数字，但我认为我们在这个领域拥有迄今为止最大的市场份额——我们只是非常擅长运行数十万甚至数百万个沙盒。

Not a whole lot. I think the thing we've always been very, very good at, I mean, we're infra geeks, right? We're very good at scaling and reliability and doing a lot of stuff. And I think that's why that explains to a large extent why we have, in my opinion—I don't have numbers but I think we have by far the largest market share in the space—we're just very good at running hundreds of thousands or even millions of sandboxes.

是的。

Yeah.

我要说，为了处理这种规模，我们不得不重写系统很多次。但我们一直很擅长这个。还有其他事情，比如随着时间的推移——我的意思是，沙盒领域总体上不是最具差异化的领域——但我们正在做一些非常酷的事情。随着时间的推移，我们不得不增加对虚拟机的支持。我们之前基于 gVisor 构建沙盒产品，但现在我们也提供完全虚拟化的机器。事实证明，对于某些用例，你需要完整的 Linux 兼容性。我们还必须构建其他东西，更好的原语，用于更交互式的沙盒之间的数据管道，处理文件系统。我们不得不在快照上花费大量时间。我认为从长远来看，许多差异化因素将是真正酷的事情，比如分叉沙盒、快照、恢复快照、实时迁移等。我认为沙盒的新颖之处始终是这种短暂性——你可以启动沙盒几秒钟，这是我想更多利用的。我认为你可以做很多非常酷的事情，将智能体和执行框架解耦，将其与沙盒解耦，让智能体控制许多沙盒，分叉沙盒，快照，基本上运行非常短命的——也许每个命令一个——类似的事情。所以我认为那里有很多酷东西。我们已经围绕快照花了很多时间做这件事。但我认为那里也有很多酷东西可以构建。

I will say we've had to rewrite our systems a bunch of times in order to handle that scale. But we were always very good at it. There are other things like over time—I mean the sandbox space in general is not the most differentiated space—but we're working on some really cool stuff. Over time we've had to add support for VMs, virtual machines. We previously built our sandbox product on gVisor but now we offer full virtualized machines as well. Turns out for certain use cases you need full Linux compatibility. There are other things we've had to build, better primitives for more interactive piping data back and forth between sandboxes, working with file systems. We've had to spend a lot of time on snapshotting. I think a lot of the things that in the long run will differentiate will be really cool things like forking sandboxes, snapshotting, resuming snapshots, live migrations in general. I think sandboxes, the novelty of sandboxes was always this ephemerality—you can spin off sandboxes for a few seconds and that's something I want to pull more in. I think there's a lot of really cool stuff you can do, decoupling the agent and the harness, decoupling it from sandboxes, having agents control many sandboxes, forking sandboxes, snapshotting things, running basically very short-lived—maybe one per command—things like that. So I think there's a lot of cool stuff there. We've already spent a lot of time doing that around snapshots. But I think there's a lot of cool stuff to be built there as well.

你多次提到快照。你认为快照的用例是什么？比如它能给我们带来什么我们现在没有的东西？

You brought up snapshotting a couple times. Like what do you see as like the use case for snapshotting? Like what would that give us that we don't get right now?

对我来说，最重要的是，今天很多沙盒工作负载有点像直接迁移。人们经常把沙盒想象成，比如，我现在有我的开发箱，我要把它塞进云里。我要让一个智能体控制它。也许我会在沙盒里运行执行框架。我不知道长期来看这是否是思考智能体算力的正确方式。我认为一个更好的模型是，你会有这些长时间运行的智能体，对我来说几乎像是持久执行工作流。它们会 24/7 运行。每隔一段时间它们会运行命令，并且能够在 100 毫秒内启动一个沙盒，运行几个命令，然后关闭并快照。我认为你越快完成这个唤醒、休眠的周期，你就越能创建只运行非常短时间的沙盒。这将使算力更加高效。它还会让你做一些非常酷的事情，比如分叉和移动东西，想象一下 Claude Code 不是有一个非常线性的流程、顺序流程，而是分支并同时运行数百个智能体。我经常想到 CI 作为另一个说明性例子。在 CI 的大部分历史中，你谈论 GitHub Actions 之类的，你有一个非常顺序的流程：你慢慢安装一堆东西，然后运行测试套件，然后生成一些报告之类的。你可以把它分成碎片，但这有点烦人。我想做的是把你的整个测试套件，每个测试都在自己的沙盒中运行，并把它并行化，比如一千个沙盒。为什么不能这样做？我的意思是问题在于启动和停止沙盒的开销太大。所以我认为你越是创造无摩擦的启动和停止，你就越能进入一个世界，你可以做这种极其突发性、大规模、尽可能并行运行的事情。所以，你知道，我是一个基础设施极客，就像我说的，我对这些事情非常兴奋。一旦我们弄清楚了所有这些原语和正确的构建块，我认为它将解锁很多我们以前用软件无法做到的非常酷的事情。

I think the big thing for me is like today a lot of sandbox workloads are kind of like lift and shift. People think a lot about sandboxes almost like, you know, I have my dev box now I'm going to shove it in the cloud. I'm going to have an agent control it. Maybe I'll run the harness inside the sandbox. I don't know if long-term that's the right way to think about compute for agents. I think a much better model would be you're going to have these very long-running agents which almost like to me are kind of durable execution workflows. They're going to run 24/7. Every once in a while they're going to run commands and having the ability to spin up a sandbox in 100 milliseconds, run a few commands, and then shut it down and snapshot it. I think the faster you can get that wake up, go to sleep cycle, the more you can create sandboxes that only run for a very brief period. And that will make compute much more efficient. And it will also enable you to do some really cool things like forking and moving things around and having—imagine like Claude Code instead of having a very linear flow, sequential flow, kind of branching and running hundreds of agents at the same time. I think a lot about CI as another illustrative example. For most of the history of CI, you talk about GitHub Actions and things like that, you had a very sequential flow: you slowly install a bunch of stuff, then you run the test suite, then you generate some reports or whatever. You can break it up into shards but it's kind of annoying. What I would love to do is take your entire test suite and just run every single test in its own sandbox and just parallelize the out of that run, like a thousand sandboxes. Why can't that be done? I mean the problem is there's too much overhead around starting and stopping sandboxes. So I think the more you create frictionless starting and stopping, the more you can enter a world where you can do this extremely bursty, large-scale, run as much as you want in parallel. So, you know, I'm an infra geek, like I said, I get very excited about these things. Once we figure out all these primitives and the right building blocks, I think it's going to unlock a lot of really cool stuff that we previously could not do with software.

为什么那还没有被构建？我认为即使以前人们也会希望尽可能并行化他们的 CI。

Why hasn't that already been built? I would assume even previously people would want to parallelize as much of their CI as possible.

你做不到，但开销就像 Andall 定律之类的，对吧？现在并行化的开销非常高。因为比如现在在 GitHub Actions 里，我们在 modal 测试套件中并行化，对吧？但这基本上意味着，如果你把测试套件分成两份，现在每个分片都需要做所有的安装和测试设置，这很耗时——可能要几分钟。但假设你可以只做一次测试设置，然后对容器做快照，现在你克隆它，就可以独立运行测试套件了。我认为很多好处——在把摩擦和开销降下来之前，你无法实现这些好处。

You can't, but the overhead is like Andall's law or whatever, right? Like the overhead of parallelization right now is very high. Because like right now in GitHub Actions for instance, we parallelize in the modal test suite, right? But what that basically means is like now if you take your test suite and you split it up in two, now each of those shards both need to do all the installs and the test setup, and that takes a long time—it can take minutes. But let's say you can do the test setup once and then you snapshot the container, and now you clone it and now you can run the test suites independently. I think a lot of the benefits—you can't materialize those benefits until you get the friction and the overhead down.

交易成本。

The transaction cost.

我觉得你们和 Anthropic 有合作，他们用沙箱来做类似 modal 沙箱的事情。

I think you guys have a partnership with Anthropic and they're using sandboxes for something like modal sandboxes.

是的，我是说 Anthropic 的托管智能体——有一种自带沙箱的集成。所以你可以用 modal 来做这些。是的。

Yeah, I mean the managed agents with Anthropic—there's a sort of bring your own sandbox integration. So you can use modal for those. Yeah.

哦，酷。那是怎么出来的？比如怎么——

Oh, cool. How did that come out? Like come maybe—

我觉得 Anthropic 在试图创建一个繁荣的生态系统和社区，所以我认为他们从推理中赚钱，对吧？所以我认为推广尽可能多的生态系统对他们非常有利，只要你在推理中使用 Anthropic，他们希望有最大的可能表面积。所以我认为他们与我们合作作为沙箱提供商是非常合理的。

I think Anthropic is like trying to create a thriving ecosystem and community, and so I think they make money from inference, right? So I think it's very much in their interest to promote as much of an ecosystem as possible for everything else, and they want to have the largest possible surface area as long as you use Anthropic for inference. So I think it makes very rational sense for them to partner with us as a sandbox provider.

另外我觉得我们必须提一下 Hugging Face 事件。你介意谈这个吗？

Also I feel like we have to bring up the Hugging Face incident. Are you okay talking about that?

可以。可以。都在那儿。

Yeah. Yeah. It's all there.

是的。是的。所以，是的。你能简要描述一下发生了什么吗？我认为他们用的是 modal 沙箱，但这不是你们的——你知道，这不是沙箱的错。

Yeah. Yeah. So, yeah. Could you just briefly describe like what happened? And I think they were using the modal sandbox, but it wasn't your—you know, it wasn't like a fault of the sandbox.

是的。没错。我忘了漏洞利用的所有细节，但我是说，我想你指的是 modal 被卷入的事实。是的。所以，就这一点评论一下，基本上在内部运行的那个利用了 Hugging Face 的智能体——他们发现了一个部署在 modal 上的应用中的漏洞，他们利用那个几乎作为跳板来获取访问权限，获取完整的互联网访问，这是我的理解。

Yeah. Exactly. I forget all the details of the exploit, but I mean, I think what you're referring to is the fact that modal was involved. Yeah. And so, just to comment on that, basically the agents running inside that exploited Hugging Face—they discovered a vulnerability in an app that was deployed on modal, and they used that as almost like a springboard for getting access, getting full internet access, is my understanding.

我不记得全部细节。澄清一下，这是一个运行在 modal 上的应用被黑了——不是 modal 被黑了。

I don't remember the full details. To be clear, this is an app running on modal that was hacked—like it was not modal that was hacked.

所以就像 AWS——很多人在 AWS 上运行东西。偶尔那些应用会被黑。那意味着 AWS 被黑了吗？不，当然不是，对吧？所以有很多人在 modal 上运行应用，其中一个应用被黑了。我认为我们可以学到一些教训，我们一直在反思的一件事是，我们能否默认有更合理的安全限制。用 AWS 的类比，S3 曾经很容易意外地让 S3 桶公开，AWS 意识到这不一定对 AWS 本身构成安全问题，但很多客户意外泄露数据。所以 S3——现在要让 S3 桶公开难多了。类似地，我们正在探索能否默认让安全限制更严格一些，而不一定破坏让 modal 出色的魔力。我认为还有——作为一个单独的岔题——智能体在网络安全方面变得非常擅长，所以实际上，作为从这个事件中学到的另一件事，我们开始内部大量使用智能体来尝试发现漏洞。到目前为止还没发现什么，但它们非常擅长探测系统的深层部分，尝试利用东西，尝试提权并逃出沙箱。所以这是我们目前正在积极探索的事情。

So it's like AWS—a lot of people run stuff on AWS. Occasionally those apps get hacked. Does that mean AWS got hacked? Like no, of course not, right? And so there's a lot of people running apps on modal, and one of those apps got hacked. And I think there are some lessons we can learn, and I think one of the things we've been reflecting on is like could we have more sensible security restrictions by default. I think to use the AWS analogy just as another analogy, S3 used to be very easy to accidentally make an S3 bucket public, and AWS realized it doesn't necessarily pose a security problem for AWS per se, but a lot of their customers accidentally would leak data. And so S3—it's much harder now to make S3 buckets public. And so in a similar way, we're exploring can we make security a little bit more restrictive by default without necessarily destroying the magic that makes modal good. And I think there's also—as a separate sidetrack—agents are getting really good at cyber security, so we've actually also, as a separate learning from this incident, started using agents internally quite a lot to try to find vulnerabilities. So far it hasn't found anything, but they're very good at probing into very deep parts of the system and trying to exploit things and trying to escalate and get out of sandboxes. So it's something we're exploring very actively right now.

你具体怎么让智能体跑遍代码库？你就说，嘿，找安全漏洞。

How exactly do you get the agents to like run through the codebase? You're just like, hey, like find security vulnerabilities.

嗯，不，你就把它们放进沙箱，告诉它们，你能试着逃出沙箱吗？就像密室逃脱，但针对智能体。

Well, no, you just put them in a sandbox and tell them like, can you try to escape from the sandbox? It's like escape the room but for agents.

酷。你什么时候开始这么做的？

Cool. When did you start doing that?

我们几个月前开始的，挺令人兴奋的——大概是在 Fable 出来的时候。

We started doing it a few months ago and it's pretty exciting—like when Fable I guess came out.

是的，大约同一时间。

Yeah, roughly around the same time.

我是说，我对网络安全的看法是，我实际上认为它们非常擅长发现漏洞，但我认为你也可以非常好地防御性地使用它们。所以我认为像 modal 和现在许多其他公司——如果你去和几乎每个季度的公司谈，我确信他们都在想同样的事情：我们如何防御性地使用智能体来修复我们所有的漏洞。所以随着时间的推移，实际上在某种程度上我并不那么紧张。我认为如果说有什么的话，智能体和网络安全——网络安全的所有突破——会让世界更安全。

I mean my view on cyber security is that I actually think they're very good at finding vulnerabilities, but I think you can also use them defensively very very well. So I think companies like modal and many other companies right now—I think if you go to talk to pretty much every season I'm sure they're all thinking about the same thing: how do we use agents defensively to fix all our vulnerabilities. So over time actually in a way I'm not that nervous. I think if anything, agents and cyber security—all the breakthroughs in cyber security—will make the world safer if anything.

你觉得所有网络安全——大型网络安全公司都完蛋了吗？

Do you think that like all of the cyber security—the big cyber security companies are just cooked?

我不知道那是不是真的，因为我是说，你问的东西远远超出我的核心专长，但当我想到网络安全时，你销售的很多实际上是恐惧，这几乎就像品牌和营销策略，当然还有核心产品。但我认为品牌之类的东西在 AI 时代可以说更有价值。所以我不一定会急于得出结论说它们完蛋了。我认为他们与企业客户有深厚的关系，他们有庞大的销售团队，非常擅长讲恐怖故事之类的。我认为他们会继续非常擅长这一点。

I don't know if that's true because I mean you're asking something that's very far outside of my core expertise, but when I think about cyber security, a lot of what you're selling is effectively fear, and that's almost like a branding and marketing strategy as much as obviously the core products. But I think brand and things like that are actually arguably more valuable in the era of AI. So I wouldn't necessarily jump to the conclusion that they're cooked. I think they have deep relationships with enterprise customers and they have big sales teams who are very good at telling spooky stories and things like that. And I think they're going to continue to be very good at that.

我只是问，因为，是的，你自己在做。你不会去找像 Palo Alto Networks 之类的公司。但我想只有巨头企业才关心那个。而且，如果出问题，他们可以怪别人。

I just ask 'cause like, yeah, you're doing it yourself. You're not going to one of like the Palo Alto Networks or whatever. But I guess like it's only like giant enterprises that care about that. Also, they can blame somebody if something goes wrong.

那是真的。是的。是的。在很多情况下，你有点想雇一个成熟的供应商，以便怪罪他们。

That is true. Yeah. Yeah. You kind of want to hire in many cases just like an established vendor in order to blame them.

某种程度上，这确实减少了我们请渗透测试员的需求，就是人类渗透测试员，大多数初创公司会定期做。所以也许之前我们会做——我其实不知道周期是多久——但我想我们大概每六个月做一次。也许现在只需要每年做一次。我不确定。但所有这些事情，人类也可以用它们来提高效率。我有点相信，现在人类加上 AI 会更强大。所以也许有理由说，我们可以请那些非常擅长使用智能体的渗透测试员，让他们更快地发现漏洞。我不确定。我是个乐观主义者。我创办了一家公司，所以我对一切通常都很乐观。但我当然不认为人类完蛋了，或者公司完蛋了。我认为在 AI 时代，很多公司会想办法利用 AI 变得更高效。

I mean, to some extent it does reduce a little bit the need for us to bring in pentesters, like human pentesters, which most startups do on a regular cycle. So maybe previously we would do it—I actually don't know what the cycle was—but I think we used to do it like once every six months roughly. Maybe now we only need to do it once every year. I don't know. But with all of these things, humans can also use them to get more efficient. I'm kind of a believer that humans with AI is even more powerful now. So maybe there's an argument that we can bring in pentesters who are really good at using agents and find vulnerabilities even faster. I don't know. I'm an optimist. I started a company, so I'm generally optimistic about everything. But I certainly don't think humans are cooked or companies are cooked. I think in the era of AI, a lot of companies are going to figure out how to use AI to be even more efficient.

我想稍微转向开发者体验，因为早期——其实现在也是——你们非常关注这一点，也谈了很多。开发者体验现在还重要吗？

I want to transition a little bit to developer experience, because early on—I mean even now—you guys focus a lot on that and you've talked a lot about that. Does developer experience even matter anymore?

我认为仍然重要。我想你说的背景是现在有了智能体，对吧？比如开发者——文档已经在那里了，智能体会去看文档，诸如此类。

I think it still matters. And I think the context you're talking about is like now with agents, right? Like does developer—like the docs are already there, the agents are going to look at the docs, like all that stuff.

不过说到这点，几乎所有让开发者体验好的东西也让智能体体验好，这大概是我们在实践中看到的，比如有好的文档、有描述性的错误信息、有快速的反馈循环，对吧？

To that point though, like almost all the things that make developer experience good also make the agent experience good, is kind of what we've seen in practice, which is like things like having good docs, having descriptive error messages, having fast feedback loops, right?

我确实认为有些东西变了。前几天我在设置 Cloudflare 的东西，我觉得 Cloudflare 有时候出了名地难用，我让一个智能体去处理，它就能帮我在 Pulumi 里配置一堆东西。所以我会说，智能体确实让处理过去比较难处理的东西变得容易了一点。比如处理 YAML 的人力负担可以说降低了一点，因为你只要告诉智能体去处理就行。但总的来说，我认为让开发者体验好的东西也让智能体体验好。

Like I do think there's certain things that have changed. The other day I was setting something up with Cloudflare and I find Cloudflare notoriously annoying to work with sometimes, and I threw an agent at it and it was able to just configure a bunch of things in Pulumi for me. And so I will say agents do make it a little bit easier to work with things that are harder to work with in the past. Like the sort of human toll of dealing with YAML has arguably gone down a little bit because you can just tell your agent to deal with it. But for the most part, I think the things that make developer experience good also make agent experience good.

我觉得某种程度上，基础设施随着时间的推移本来就会变得、也需要变得更程序化。我想这一直是关于沙箱的论点：以前必须用 YAML 配置的东西应该就是一个 SDK，应该更命令式——比如启动一个沙箱，用它做事情，然后销毁沙箱——而不是用某种 DSL 声明式地做。这也推动了一个以前可能不存在的不同接口。我认为无论哪种方式，SDK 都和以往一样重要，甚至更重要。

I think to some extent also, infrastructure over time anyway becomes and needs to be more programmatic. I think that's always been the thesis about sandboxes, that things that you previously had to configure using YAML should just be an SDK, should be more imperative—like launch a sandbox, do stuff with it, tear the sandbox down—rather than declarative using some sort of DSL. It also pushes a little bit of a different interface that maybe previously didn't exist. I think in either way, SDKs are as relevant as they always were, if not more.

你们有没有因为大多数人都在上面运行智能体而改变产品中的某些东西？

Is there anything you changed in your product just because most people are just running agents on it?

有的。其实我们也会做很多内部测试，让智能体——我们告诉智能体去构建一个移动应用并运行它。一个很具体又有点好笑的事情是，我们发现由于智能体的训练数据，现在很难弃用东西。我们看到智能体经常喜欢使用旧版本的 Modal 语法，而且很难让它们改掉。所以过去一年左右，我们在弃用和移除功能方面不得不更保守一些。我们必须非常小心地更新网上每一个示例。

We do. I mean, we also actually run a lot of internal tests where we have agents—we tell agents to build a mobile app and run it. And a very concrete thing that is kind of funny is that we've actually seen that because of agent training data, it's very hard to deprecate things now. We see that agents often love using old versions of Modal syntax, and it's kind of hard to nudge them away from that. So we've actually had to be a little bit more conservative going forward, like over the last year or so, with how we deprecate and remove features. We have to be very careful to update every single example that's out there online.

还有技能，对吧？

And skills, right?

是的，有类似的东西。但网上现在也有海量的 Modal 代码进入训练数据。我觉得那是个有趣的例子。顺便说，另一件事是，显然有类似于搜索引擎优化的东西——智能体引擎优化，不管人们怎么叫。我们经常运行内部基准测试，比如问一个智能体：‘嘿，你能帮我们部署 Stable Diffusion 吗？用你想要的任何工具’，然后看它是否推荐 Modal。这些东西对我们长期跟踪非常有用。或者像‘嘿，启动一些沙箱’，或者‘嘿，给所有沙箱供应商排名’。所以我们花很多时间做这些事情来确保——

Yeah, there's stuff like that. But there's also an insane amount of Modal code that's now online too that goes into the training data. I've seen that to be kind of an interesting example. Another thing, by the way, is that there's obviously something similar to search engine optimization—there's agent engine optimization, whatever people call it. We often run internal benchmarks and we're like, ask an agent, 'Hey, can you deploy Stable Diffusion for us? Use whatever tool you want,' and we see if it recommends Modal or not. And those things are very useful for us to track over time. Or like, 'Hey, spin up some sandboxes,' or 'Hey, rank all the sandbox vendors.' So we spend a lot of time doing those things to make sure—

我是说，你们对此怎么做？你说‘嘿，构建这个 Stable Diffusion 的东西’，它推荐 Modal 还是不推荐？你们对此怎么做？

I mean, what do you do about that? You say, 'Hey, build this Stable Diffusion thing,' does it recommend Modal or not? Like, what do you do about that?

是的，我觉得那更难。我认为就像搜索引擎优化。我认为主要通过内容——你只需要发布大量内容。

Yeah, I think that's harder. I think it's like search engine optimization. I think it's primarily through content—you just have to put out a lot of content.

有意思。

Interesting.

某种程度上你也可以更多地看到认知——你可以问一个智能体，比如‘如何——按可靠性给所有沙箱供应商排名’或者‘按价格给所有沙箱供应商排名’，然后看看。确实感觉智能体更容易误解定价之类的事情。所以我们也花很多时间让内容对智能体更清晰、更易懂，因为我们知道所有的索引——网上所有谈论 Modal 的东西。

You can also to some extent see the perception a little bit more—you can ask an agent, like, 'How does—rank all the sandbox vendors by reliability' or 'rank all the sandbox vendors by price,' and sort of see. And it does feel like agents are a little bit more prone to misunderstanding pricing and things like that. So we also spend a lot of time just making content more clear and more understandable for agents, because we know that all the indexing—everything that's out there and that's talking about Modal.

你有没有想过直接建一堆 GitHub 仓库，让智能体在里面写一堆 Modal 代码？你试过吗？

Have you thought about just making a bunch of GitHub repos and having the agents write a bunch of Modal code in there? Have you tried that?

垃圾内容大炮。我觉得事实是，那大概是你能够做的事情。

Slop cannon. I think the truth is that is probably something you can do.

我的猜测是，如今的语言模型训练数据可能对 AI 生成的内容相当不敏感，而且你很可能可以制造大量垃圾内容。

My guess is that LM training data today is probably quite insensitive to AI output in the first place, and you can probably create a lot of slop.

我们考虑过。我不太喜欢那个想法，但理论上，就像早期的 SEO，某种程度上现在也是，自动生成内容是一种非常有效的策略。我认为今天你也有可能做那些事情。

We thought about it. I don't love that idea, but in theory, much like SEO in the early days and still to some extent, autogenerated content is a very effective strategy. I think it's possible you could do those things too today.

我想转到 FDE 的话题。听说你们雇佣了前线部署工程师。他们做什么？

I want to switch to FDEs. Apparently you guys hire forward deployed engineers. Like what do they do?

我们正处于一个构建应用和几年前无法构建的东西的新时代。我们看到很多公司需要一点帮助。所以我们在几年前开始投资 FDE，但今年我们真正扩大了规模。我记得今年开始时我们有四个 FDE，现在大概有 24 个左右，所以增长了很多。他们做大量的客户互动工作。他们与许多客户合作进行性能调优，尤其是语言模型。我们发现，与客户坐下来训练草稿模型并调整参数、DM 和 SG lang 有很多好处。他们经常修复 bug——或者不是修复 bug，而是当客户遇到问题时，他们经常与客户合作，想办法解决。我们经常使用 FDE 的另一个方面是主动出击——如果我们看到某些客户在 Modal 上过度配置，比如请求过多的 CPU 或内存，这很昂贵，我们试图抓住这些情况，并与这些客户坐下来谈，比如‘嘿，你现在在 Modal 上花了很多钱，我们实际上可以帮助你省钱。’当然，对于活跃的交易，我们帮助他们从其他平台迁移代码等等。

We're in a new era of building applications and things that we couldn't build just a few years ago. We see a lot of companies that need a little bit of help. So we started investing in FDEs a couple years ago, but it's really been this year we've scaled it up a lot. I think we started this year having like four FDEs, now I think we have like 24 or something like that, so we've grown a lot. And they do a lot of heavy customer engagement. They work with a lot of customers on performance tuning, especially with LMs. We see that there's so much benefit from sitting down with customers and training draft models and tweaking parameters and DM and SG lang. They often fix bugs—or not fix bugs, but when customers run into issues, they often work with customers trying to figure out how to work around that. Another thing we've used FDEs a lot for is proactively—if we see certain customers overprovisioning with Modal, like customers requesting too much CPU or too much memory for instance, that's kind of expensive, and we try to catch those things and sit down with those customers like, 'Yo, you're spending a lot of money on Modal right now, we can actually help you save money.' And then of course, with active deals, we help them migrate code from other platforms, things like that.

我认为我们从未打算成为服务密集型业务。我认为 FDE 的目标是：A，参与销售过程和达成交易；B，服务客户并提供一些客户支持；但我实际上认为最重要的是 C，弥合产品中的差距，我们后来希望将其转化为真正的功能。所以在很大程度上，FDE 做的任何事情，我的问题总是长期来看：我们如何将其转化为产品，这样我们就不需要 FDE 来做这个了？

I think we never have the intention of becoming a service-heavy business. I think the goal for FDEs is: A, to be a part of the sales process and closing deals; B, to serve customers and do some customer support; but I actually think the most important thing is C, bridging gaps in the product that we later want to turn into real features. So to a large extent, anything an FDE does, my question is always long-term: how do we turn this into product so we don't need an FDE for this?

是的。你们雇佣 PM，产品经理吗？

Yeah. Do you guys hire PMs, product managers?

不。

No.

你会说这些 FDE 有点像伪 PM 吗？

Would you say these FDEs are kind of like pseudo-PMs?

这是一个有点不同的职能。当然，在更面向客户的一面，他们是，但在更内向的内部一面，产品管理有一个单独的部分，更像是 Modal 接下来应该投资哪些构建模块。然后中间还有一些事情：看看也许 10-20 个不同的客户，他们都要求稍微不同的东西，但模式是什么？我们如何将其转化为产品？我们确实有员工和我们一起做产品，而且它有点进化了——我们有人基本上在之前是开发或工程师之后成为了产品经理，只是在对我们必须构建的东西进行非常战略性的思考方面表现出很强的能力。所以我实际上并不反对产品管理。我确实认为过去很多公司过于急切地雇佣产品经理，我认为存在一种风险，你可以在很多产品中看到——某些产品，你可以看出有急切的产品经理只是想到了一些小部件的东西就发布了。我认为在一家基础设施公司做好产品管理非常非常难，因为这需要非常深入的战略思考，思考什么是基础构建模块，我们在一到两年内想要达到什么位置，因为事实是构建基础设施非常难，以及我们如何确保我们不会最终得到 35 个看起来相似但都做稍微不同事情的功能。那种水平的思考非常罕见，因此我们还没有积极尝试建立产品管理团队。

It's a little bit of a different function. Certainly on the more customer-facing side they are, but on the more inward internal side, there's a separate part of product management which is more like what are the next building blocks Modal should invest in. And then there's something in between: looking at maybe 10-20 different customers, they're all asking for slightly different things, but what are the patterns? How do we turn this into product? We do have people doing product with us, and it's sort of evolved—we've had people basically become product managers after previously being either dev or engineers, just showing strong aptitude for thinking very strategically about what we have to build. So I'm actually not anti-product management. I do think that a lot of companies in the past hired product managers too eagerly, and I think there's a risk that you can sort of see it in a lot of products—certain products, you can tell there are just eager product managers that thought of widgety things and just shipped them. I think doing product management at an infra company really well is very, very hard because it takes a very deep level of strategic thinking about what are the foundational building blocks, where do we want to be in one to two years, because the truth is building infrastructure is very hard, and how do we make sure we don't end up with 35 features that are kind of similar looking but all do slightly different things. That sort of level of thinking is very rare to find, and for that reason we haven't aggressively tried to build up a product management team yet.

是的，这说得通。你会说你的团队自 Gen AI 以来如何演变？因为我假设——我的意思是，你显然在 Spotify 工作过。你有一定的团队结构。现在我们有编码代理，你的团队结构变化很大吗？

Yeah, that makes sense. How would you say your teams have been evolving maybe since Gen AI? Because I would assume—I mean, you obviously worked at Spotify. You had a certain team structure. Have your team structures evolved a lot now that we have coding agents?

有趣的是，是的，我确实认为团队结构变化很大，但我实际上并不确信是 Gen AI 驱动的。特别是，我经常看到的一件事是，有一种回归个人贡献者（IC）的趋势，我实际上认为这对整个领域非常非常积极。如果你回顾 10 年、20 年前的领域，每个人都想进入管理层，因为那是获得更多金钱、更多认可和更多声望的方式。我认为这种转变实际上在 Gen AI 之前就开始了，但我确实认为它随着 AI 而大大加速。我们向 Modal 的每个人明确表示，经理和 IC 在声望上没有区别。每个人都只是技术员工。最终重要的是你的贡献。我们看到很多前经理成为 IC，因为他们想回去写代码。而且有一种远离非技术经理的趋势。几乎每个人，包括我——我仍然在 Modal 和 Oxhat 2 写很多代码——而且技术团队中几乎每个人以及很多非技术团队的人仍然写代码。所以对我来说，这是一个最近的现象，但我实际上不知道它是否一定是 Gen AI 引起的，但我认为 Gen AI 大大加速了它，因为它使写代码更容易，降低了成本。

In a funny way, yes, I do think team structures have changed a lot, but I'm actually not convinced it was Gen AI driving that. In particular, one thing I've seen a lot is that there's been a swing back towards being an IC, which I actually think is very, very positive for the space as a whole. If you look back at the space 10 years ago, 20 years ago, everyone wanted to be in management because that was a way you would get more money and more recognition and more prestige. I think that shift actually started even before Gen AI, but I do think it's heavily accelerated with AI. We make it very clear to everyone at Modal that manager and IC—there's no difference in prestige. Everyone is just a member of technical staff. It's about ultimately your contributions. And we see a lot of ex-managers becoming ICs because they want to go back and build code. And there's been a shift away from non-technical managers. Everyone, pretty much including me—I still write a lot of code at Modal and in Oxhat 2—and pretty much everyone in the tech team and a lot of people who are not in the tech team still write code. So that to me is a recent phenomenon, but I actually don't know if it's necessarily caused by Gen AI, but I think Gen AI has greatly accelerated it because it made writing code even easier and lowered the cost.

那么，如果有非技术人员提交代码，你们怎么审查？你们团队如何进行代码审查？

So if you have like non-technical people like committing code like how do you how do you review that like how do you guys do code reviews?

我们做代码审查，但认为它是可选的。我的观点是，很多时候代码审查只是人们试图逃避将代码合并到生产环境的责任。如果你有那种文化，那对我来说就是一个信号，说明你需要更多地投资于持续交付（CD）和持续集成（CI）之类的东西。特别是 CI，还有 CD，因为它是部署。我们做代码审查，比如我写代码时偶尔会问作者：这符合你的思路吗？我希望我没有从根本上改变架构。但我们也在内部明确表示，代码审查是相当可选的。如果你只是改文档，或者做一些基本的事情，就不要等人工来看，尤其是在 AI 时代。为什么需要人工审查每一段代码？你不必。

We do code reviews but we think of it as optional. I mean for most my view is like code reviews in large part is like people in many cases trying to just escape the accountability of like you know merging things into production and if you have that culture like then to me that's a sign that you need to invest way more in CD and CI and things like that. CI in particular but also CD, continuous integration because it's deployment. We do code reviews like when I write code occasionally I'm asking authors like is this in line with how you think about things? I hope I'm not changing the architecture fundamentally. But we also make it clear internally that code reviews are quite optional. And if you're just changing docs, if you're just doing some basic stuff like don't wait for a human to look at it especially in the era of AI. Why do you need a human to look at every piece of code? You don't have to.

但这确实要求公司在测试方面有非常好的内部控制。

But that does require you to have very good controls internally around testing for a company.

是的。就像基础设施公司，我们的正常运行时间需要比普通初创公司多一个九，但这些是你必须做的投资。你必须构建非常有弹性的系统，到处都要有限流。你必须解耦系统。你必须确保没有单点故障。它们必须有极好的测试覆盖率。现在智能体实际上非常擅长写测试，几年前我认为这是智能体缺失的主要能力之一。现在它们也非常擅长写测试，因为我认为写好的测试需要你暂时跳出正在做的事情，几乎要扮演一点魔鬼代言人的角色，但现在它们在这方面也相当不错了。所以我认为所有这些都有帮助。

Yeah. And exactly like infra companies like we need one more nine than the average startup in terms of uptime but these are investments you just have to make. You have to build systems that are very resilient, you have to have rate limits everywhere. You have to decouple systems. You have to make sure that nothing is a single point of failure. They have to have extremely good test coverage. Agents are actually really good at writing tests these days, which was a few years ago I thought like one of the major things missing in agents. Now they're actually quite good at writing tests too because I think it takes a little bit of need to step out of what you're trying to do for a second and be almost a little bit like devil's advocate to write good tests, but now they're actually quite good at writing tests too. So I think all these things help.

你们用前沿模型来编程吗？还是尝试过开源模型？

Do you guys use like frontier models for coding or do you guys have you guys experimented with uh like the open source ones?

我们什么都尝试，但最初大量使用 Cursor，然后当然有 Claude 和 Codex，现在很多人使用 Night Shift 和所有这些开源的东西或其他技术栈，所以我没有强烈的观点。我希望人们实验和尝试不同的东西。我们使用前沿模型，也使用开源模型。现在我们实际上正在大规模地为许多开源模型运行推理，我们试图在内部多做一些吃自己的狗粮。我认为对于许多智能体式开发，使用我们的内部端点也很有帮助。所以我们试图稍微推广这一点。

We experiment with everything but yeah initially was a lot of Cursor then of course Claude and Codex and now a lot of people are using Night Shift and all these open source stuff or other stacks so I don't necessarily have a strong point of view. I want people to experiment and try different things. We use frontier models, use open source models. We're trying to do a little bit more dogfooding internally now that we're actually running inference for a lot of the open source models at a very large scale. I think it's also helpful to use our internal endpoints for a lot of agentic development. So we try to promote that a little bit.

我大约一年前或两年前读到你说，你认为你们不会有大的本地部署业务。你改变看法了吗？

Something I read maybe like I don't know a year ago, maybe like two years ago, you were saying that you didn't think that there was going to be like a big on-prem business for you guys. Um, have you maybe like changed your opinion on that?

不，我没有改变看法。我认为 Modal 对我来说，我是一个云最大化主义者。也许退一步说，当云在 2008 年或 2009 年出现时，我的第一反应是这太疯狂了，为什么我要把代码放在别人的电脑上运行，这太疯狂了。然后大约 6 个月后我就这么做了。我花了 6 个月改变想法，企业可能又花了 15 到 20 年。但你看像 Snowflake 或 AWS 这样的供应商，还有很多例子如 Data Dog 等，云显然在赢，多租户系统显然在赢。对我来说，本地部署越来越像一种遗留商业模式。我实际上认为 BYOC、VPC 对等连接等所有这些都反映了一种过时的安全模型，即我们需要保护自己网络内的一切，把网络当作护城河，而我认为这不再正确。我们已经转向更强大的安全范式，比如零信任等。所以对我来说，这些东西至少从安全角度看越来越过时。从我们的角度看，我们完全控制整个基础设施栈。我们租用所有硬件。我们部署和管理所有基础设施。我们部署和管理健康状态并观察所有主机上的所有指标。这意味着我们可以快速迭代并构建更好的产品，我认为这非常有价值。相比之下，过去客户有企业版必须部署在客户环境中。他们必须非常仔细地考虑发布管理和依赖关系。而我们不在乎，如果我们想引入新数据库，我们可以引入，因为我们控制它，不需要考虑客户能访问什么。所以对我来说，多租户有很多不同的好处。话虽如此，如果未来我们与一些非常大的财富 500 强客户合作，他们要求 BYOC，我们过去做过，我们会给他们一个价格，那个价格非常高，到目前为止没有人同意，但谁知道呢，如果价格合适，我仍然会构建它。我认为还有另一件有趣的事情，理论上 BYOC 也可能是一种合作和获取容量的方式。我们有时与 Neolabs 合作，那些 Neolabs 已经有非常大的容量预留，问题是如何让他们在已有容量上运行 Modal，BYOC 可能是一个解决方案，我们仍然可以把控制平面留在我们的环境中。另一方面，我们甚至可以在那容量上运行别人的东西，所以理论上你可以做一些有趣的事情，比如把所有容量聚合成一个大池子。我们对一些想法感到兴奋，但总的来说，我觉得本地部署，我不知道我们是否会做，但 BYOC，也许吧，自带云。我一直说 BC，我应该解释缩写。

No, I have not changed my opinion. I think Modal to me I'm a cloud maximalist and maybe to take a step back like when the cloud came in 2008 2009 or whatever my first reaction was like that's crazy like why would I want to take my code and run it on someone else's computer that's like insane and then like 6 months later I was doing it and I think it took me 6 months to change my mind it took probably enterprises like another 15 to 20 years but you look at vendors like Snowflake or I mean AWS and I think there's many examples of Data Dog etc like the cloud is clearly winning and the multi-tenant systems are clearly winning and to me on-prem looks increasingly like sort of a legacy business model and I actually think BYOC, VPC peering, all of these things to me they reflect sort of an outdated security model of like we need to protect everything inside our own network, using the network as a moat and that I don't think is true anything anymore. We've sort of moved on to more robust security paradigms like zero trust and things like that. And so to me the things like they look increasingly as outdated at least from a security perspective from our perspective also like the fact that we have full control over our entire infrastructure stack. We rent all the hardware. We deploy we manage all the infrastructure. We deploy and manage health and observe all the metrics on all the hosts. It means we can iterate very fast and build a greater product and I think that's very valuable versus customers in the past who had enterprise additions that they had to deploy on customers environment. They have to be very they have to think very carefully about release management and having depend like we don't care like if we want to introduce a new database like we can introduce a new database because we have control over it like we don't need to think about what customers have access to. So to me multi-tenency has so many different benefits. I mean that being said if we were to work with some very large Fortune 500 customer in the future and they ask for BYOC we've done this in the past we'll give them a number like that number is very high and no one said yes to it so far but who knows at the right price I would still build it and I think there's maybe separately like another thing which I think is interesting which is like in theory BYOC might also be a way to work with and get capacity in certain ways like we sometimes work with Neolabs and those Neolabs already have very large capacity reservations and the question is like how do we get them to run Modal when they already have capacity and BYOC might be a solution we can still keep the control plane in our environment and the other flip side of that is like we could maybe even run other people's stuff on that capacity so there's some fun stuff you could do in theory where you can sort of aggregate all that capacity into one big pool. We're excited about some of those ideas, but in general, I feel like on-prem I don't know if we're ever going to do, but BYOC like maybe bring your own cloud. I keep saying BC. I should explain acronyms.

假设你即将筹集更多资金，你会用这些钱做什么？

Let's say hypothetically you are going to raise, I should say, more money soon. What would you use that money for?

就像我提到的，在这种产能情况下，所需的大量资金——我应该说大量现金流——是产能预付款，对吧？所以对我来说，这是当前现金的最佳用途。我们正在非常积极地扩大产能。也许未来我们会转向其他类型的部署模式，比如所有权之类的。但我认为总会有现金部分，比如某种首付或预付款之类的。所以对我来说，这是目前现金的最大用途。当然还有其他事情，比如工资和办公空间之类的，但在我们财务的大局中，那现在基本上是个舍入误差。

Like I mentioned, in this capacity situation, a lot of the money—a lot of the cash flow, I should say, that's required—is capacity prepayments, right? So that to me represents the best use of cash today. We're ramping up capacity very aggressively. Maybe we move into other types of deployment models in the future, ownership or whatever. But I think there's always going to be a cash component of that, like some sort of down payment or prepayment or something like that. So that to me is the greatest use of cash right now. There are of course other things like salaries and office spaces and stuff like that, but in the grand scheme of our financials, that's basically a rounding error right now.

嗯，有意思。你觉得这种情况会改变吗？

Yeah. Interesting. Do you think that's going to change?

不会。我认为我们在这个业务中的运营杠杆有点疯狂。比如我们有 200 名员工——他们花费很多钱——但 GPU 花费更多钱。

No. I think the operating leverage that we have in this business is kind of insane. Like we have 200 employees—they cost a lot of money—but the GPUs cost a lot more money.

如果你必须说，你或公司目前最关注的三大重点是什么？你会说是哪些？

If you had to say, what are the top three things that are your biggest focus or the business's biggest focus right now? What would you say those are?

产能和财务复杂性，以及总的来说我们如何思考那个领域，如何思考定价和打包，如何思考所有权和租赁之间的张力——我们应该朝更 neo cloud 的方向发展，还是朝相反的方向，那种 counter precision——这大概是我思考最多的事情。我思考第二多的大概是产品战略。我经常思考下一个构建块是什么。如果我们做更多智能体式的东西，我们需要什么？是否有某些存储原语或计算原语，或者总的来说，世界在未来几年会醒来并意识到他们需要什么，理想情况下我们在那个时候已经有了。我们总体上一直很擅长预测需求并提前构建产品，比如无服务器函数和沙盒。那么那个东西是什么？然后我会说第三件我思考很多的事情更多是关于文化和公司建设之类的。我们最近刚超过 200 人。我们增长非常快。到年底我们可能会达到 250 人。当你增长非常快时，你有 250 人都在试图弄清楚如何为 Modal 增加最大价值，并确保他们有正确的背景，有正确的动力，他们每天兴奋地来上班，因为他们能构建非常酷的东西，他们知道什么让 Modal 成功以及他们能做什么——对齐激励措施，确保人们有一种非常积极的文化，他们有那种隐式信任去做很多事情，但也有责任交付最好的东西，并对没有做到负责。对我来说，这些是在这个规模下超级关键的事情。当你只有 10、20、30 人时，大家可以坐在一个房间里，一切都是隐式的，你几乎能感觉到。人们能读懂我——他们能告诉我什么时候我有点烦。他们能看出我什么时候开心。现在我们 250 人，人们不一定看到这些，当然有很多新人可能有点紧张和 CEO 说话之类的。那么我如何保持平易近人？还有邓巴数，我认为那更多是假的——至少背后的故事——但它实际上还是有点真实，大概在 150、200 左右。我们正在奠定文化的基础，这非常非常重要。

Capacity and the financial complexity, and in general how do we think about that space, how do we think about pricing and packaging, how do we think about the tension between ownership and renting—should we go in a more neo cloud direction, should we go in the opposite direction, the sort of counter precision—like that's probably the biggest thing that I think a lot about. The second biggest thing I think a lot about is probably around product strategy. I think a lot about what are the next building blocks. If we do more agentic stuff, what are the things we need? Are there certain storage primitives or compute primitives, or in general, what is the thing that the world is going to wake up and realize they need over the next few years, and ideally we have that at that point. We've been pretty good in general at anticipating demand and building products in advance of it with the serverless functions and the sandboxes. So what's the thing? Then I would say the third thing I think a lot about is more around culture and company building and stuff like that. We recently crossed 200 people. We're growing extremely fast. We'll probably hit 250 by the end of the year. When you're growing very fast, you have 250 people that are all trying to figure out how they can add the most value to Modal, and making sure that they have the right context, that they have the right motivation, that they're excited to come into work every day because they get to build something incredibly cool and they know what makes Modal successful and what they can do—aligning the incentives and making sure that people have a very positive culture where they have the implicit trust to do a lot of stuff but also the accountability to deliver the best things and be held accountable if they're not doing it. To me those are super critical things around this scale. When you're 10, 20, 30 people, you can all just sit in a room and it's all implicit and you kind of just feel it. People can read me—they can tell me when I get a little bit annoyed. They can tell when I'm happy. Now we're 250 people and people don't necessarily see it, and of course there are a lot of new people who are maybe a little bit nervous to talk to the CEO and all this stuff. So how do I remain approachable? And there's the Dunbar number, which I think is more faked than anything—at least the story behind it—but it is actually kind of real still, like there's something around 150, 200. We're laying the foundation of the culture, which is very, very important.

你们明年打算招聘多少人？

How many people are you guys trying to hire over the next year?

过去 12 个月，我们年化增长了约 4 倍。我认为相对而言这是不可持续的增长速度。我见过你可以每年让团队翻倍，那并不是特别难。但你开始看到文化漂移。你开始看到不同工作风格的孤岛。你开始看到人们做非常投机的事情。这是我在工程团队中一直警惕的。你去和工程师交谈，我问：“你在做什么？”结果他们在构建一些边缘的东西，这真的是你应该花时间做的吗？所以快速增长有那个代价，也有培养人员的代价。所以我不知道，我想今年年底可能 250 人，明年年底——我们会继续增长大概 2.5 倍、3 倍，所以很可能明年我们会达到 700 人左右。

We've grown about 4x on an annualized basis in the last 12 months. I think that's an unsustainable growth rate on a relative basis. I've seen that you can double the team every year and that's not super hard to do. But what you start to see is cultural drift. You start to see islands of different working styles. You start to see people doing very speculative things. That's something I always watch out for in the engineering team. You go and talk to an engineer and I'm like, 'What are you working on?' And it turns out they're building something kind of fringe and is this really what you should spend time on? So growing very fast has that cost, and it has the cost of ramping people up. So I don't know, I think by the end of the year maybe 250, by the end of next year—we're going to keep growing like maybe 2.5x, 3x, so it's very likely we'll end up next year like 700 people or something like that.

我猜其中大多数是工程师。

I'm going to assume most of those are engineers.

主要是工程师，但也有很多销售。

Mostly engineers, but also a lot of sales.

好的。哪种工程师？

Okay. What kinds of engineers?

主要是系统工程师。我们一直深入 Linux、文件系统、可扩展系统、网络和云之类的东西。所以那是我们寻找的核心——真正擅长这些事情的人。我们也在大量寻找——因为我们向栈的上方移动了一点——我们在语言模型推理领域做了很多工作。所以现在我们处理一系列围绕 CUDA、GPU 和 MFU 的不同问题，比如最大化每个 GPU 能获得的 token 数量等等。而且尽管我会说，在我看来那个领域的很多工作仍然更多是系统驱动的，因为事实证明 KV 缓存和路由实际上仍然是非常系统密集型的问题。但是的，更多一点 GPU 性能之类的东西。我们开始建立一个研究团队。我们非常兴奋地进入后训练、蒸馏以及更好的量化方法等领域。

It's primarily systems engineers. We've always been very deep into Linux and file systems and scalable systems and networking and cloud and stuff like that. So that's the core of what we're looking for—people who are really good at those things. We're also looking a lot at—because we moved up in the stack a little bit—we're doing a lot of work in the LM inference space. So now we're dealing with a separate set of concerns around CUDA and GPUs and MFU, like maxing out the tokens that you can get per GPU and a lot of those things. And even though I will say a lot of the work in that space in my opinion turns out to be more systems driven still, because it turns out KV caching and routing are actually still very systems-heavy problems. But yeah, a little bit more GPU performance and that kind of stuff. We're starting to build up a research team. We're very excited about moving into stuff around post-training and distillation and better methods for quantization.

是为客户还是为内部？

For your customers or for internally?

两者都有。

Both.

我觉得这有点像 FTE 的情况。一开始我们会和客户一起做很多工作来优化他们的模型，但久而久之要想办法把这些变成产品。如果我们每次客户需要优化模型都得去和他们一起做，那可以说我们在某种程度上就失败了。长期来看必须把它变成产品。所以有了 FDE 和研究团队，一开始你得和客户做很多工作，我们正在这么做。但久而久之我们也在构建很多内部工具和很多完全自助式的内部产品。所以我们现在在后训练领域和 GPU 性能领域都在做很多工作。很多很酷的东西。

I think it's a little bit like with the FTEs. I think we initially will do a lot of stuff with our customers to optimize their models, but over time figure out how do we turn those into products. I think we'll arguably fail in a certain way if every time a customer needs to optimize a model we have to go and work with them. Long-term it has to be turned into products. So with the FDEs and with the research team, initially you're going to have to do a lot of work with the customers, and we are. But over time we're also building a lot of internal tooling and a lot of internal products that are fully self-serve. So we're working on a lot of stuff right now in the post-training space but also in the GPU performance space. So, a lot of cool stuff.

回到你刚才说的产品策略。你怎么看？接下来一年、两年我们需要什么？

Back to the product strategy stuff that you were talking about. What are your thoughts? What do we need over the next year, two years?

我觉得这非常难说，如果我说我确切知道我们需要构建什么，我觉得我就是在妄想。到目前为止我们看到的，以及我在更普遍层面上所相信的，是我们需要更好的构建模块。这是软件的新时代，我们正在解锁几年前根本无法构建的全新类型的应用。做所有 GenAI 的东西，LLM 输出的代码需要以隔离的方式运行——需要太多新的原语来与 GPU 和 LLM 协作。到目前为止我们已经构建了一些构建模块，这解锁了过去几年世界看到的一堆应用，当然不只是我们。但我认为未来几年会有更多酷炫的东西。我提到了一些——我非常看好训练。我非常看好批处理任务。我非常看好智能体和低延迟之类的某些东西。但最终，我们把自己视为一家基础设施公司。我们不是试图构建非常狭窄的东西。我们试图构建非常通用的构建模块，把它们放在客户面前，然后那些客户会组合这些构建模块，构建出我们根本无法预料的惊人应用。

I think it's incredibly hard to say, and if I said I know exactly what we need to build, I think I would be delusional. What we've seen so far, and what I kind of believe in on a more general level, is we need better building blocks. This is a new era of software, and we're unlocking a whole new type of applications that could not be built a few years ago. Doing all the GenAI stuff, code that LLMs output that needs to run in an isolated way—there are so many new primitives that are needed to work with GPUs and work with LLMs. We've built some building blocks so far, and that's unlocked a bunch of applications that the world has seen over the last few years, not just us, of course. But I think over the next few years there's going to be so many more cool things. I mentioned some stuff—I'm very bullish on training. I'm very bullish on batch jobs. I'm very bullish on certain things around agents and low latency and things like that. But in the end, we consider ourselves an infrastructure shop. We're not trying to build very narrow things. We try to build very general-purpose building blocks and put them in front of customers, and then those customers will combine those building blocks and build some incredible applications that we could have never anticipated.

我想过一遍你的一些推文。这会是最后一部分，也许你想深入聊聊任何内容。

I want to go through some of your tweets. This will be the last part, and maybe you want to double-click on anything.

现在有很多关于长时间运行的持久沙盒的东西。今天感觉像是直接迁移。我怀疑沙盒会从云中的开发盒演变成更短暂的东西——每个命令一个环境，中间进行快照，这样你可以暂停、恢复、克隆。是的，我们谈过很多这个。

A lot of long-running persistent sandbox stuff right now. Today it feels lift-and-shift. I suspect sandboxes will evolve from devboxes in the cloud to something more ephemeral—one environment for every command and snapshotting in between so you can pause, resume, clone. Yeah, I mean we talked about this a lot.

是的，我们谈过一点，对吧？

Yeah, we talked a bit about it, right?

我完全——我认为今天 Modal 里沙盒的中位时长大概是 8 秒左右，想想都觉得不可思议。随着时间的推移，我认为它可能会变得更短。我们也想支持长时间运行的沙盒。我认为围绕长时间运行的沙盒有一些非常有效的用例。但我的愿景是——世界也需要我们弄清楚这一点：我们如何大规模并行化事物，并减少启动和停止沙盒的开销。

I'm fully—I think the median sandbox in Modal today is like 8 seconds long or something like that, which is kind of incredible when you think about it. And over time I think it might get even shorter. We want to support long-running sandboxes too. I think there are some very valid use cases around long-running sandboxes. But my vision is—the world needs also we need to figure out this: how can we just massively parallelize things and reduce the overhead of starting and stopping sandboxes.

我不知道如果我经营一家准备 IPO 的初创公司，与竞争对手展开资本支出军备竞赛，能稍微协调和调整军备竞赛的节奏就好了。

I don't know if I ran an IPO-ready startup engaged in a capex arms race with a competitor, it would be nice to coordinate and pace the arms race a bit.

无可奉告。

No comment.

GPU 稀缺的最大缺点不是很难获得 GPU，而是几乎所有容量都以三年期出售——作为一个三年期的矩形，从未来 3 到 6 个月开始。这不是大多数用户的需求形态。我想我们也谈过一点这个。

The biggest downside of GPU scarcity is not that it's hard to get GPUs, but that almost all capacity is sold as three years—as a three-year rectangle starting 3 to 6 months in the future. That's not what demand looks like for most users. I think we talked about that a little bit as well.

是的。你知道，也许再深入一点。我认为 Modal 一直非常非常擅长的一件事就是完全按需的容量，对吧？如果客户来找我们需要 100 块 GPU，我们通常可以立即给他们。你甚至不需要和销售人员交谈。现实生活中的许多工作负载并不像这些平坦的矩形工作负载。它们在 24 小时周期内上下扩展。它们可能随时间增长。它们可能有某些突发期。对我来说，这是当今许多客户面临的挑战。我确实认为这是该领域的一个合理问题——获得容量的唯一主要方式是购买这些矩形块。这并不好。我认为世界需要进化，从根本上说——我的意思是，我很有偏见——但这就是为什么我认为 Modal 可以在这种持续时间转型中发挥非常大的作用，提供更灵活的容量。我们可以做到这一点，因为我们可以聚合成千上万的客户，需求在聚合后变得非常平滑。所以这就是 Modal 的很大一部分价值主张——正是如此。

Yeah. And you know, maybe just go a little bit deeper. One thing I think Modal has always been very, very good at is fully on-demand capacity, right? If a customer comes to us and needs 100 GPUs, we can typically give them that immediately. You don't even have to talk to a salesperson. And a lot of workloads in real life don't look like these flat, rectangular workloads. They scale up and down throughout the 24-hour cycle. They maybe grow over time. They maybe have certain burst periods. And to me that is a challenge with a lot of customers today. And I do think it's a legit problem for the space—the only prime way to get capacity is to buy these rectangular blocks. It's not great. I think the world needs to evolve, and that's fundamentally—I mean I'm very biased—but that's why I think Modal can play a very large role in that sort of duration transformation of offering more flexible capacity. And we can do that because we can aggregate many, many thousands of customers and the demand gets very smooth in aggregate. So that's a lot of the value prop of Modal—it's exactly that.

你会说这是一个差异化因素吗？哦真的，好的。

Would you say that's a differentiator? Oh really, okay.

是的，实际上是巨大的差异化因素。事实上，我认为 Modal 的——如果你想非常简化——你可以把 Modal 的价值主张分解为两件事。一是开发者体验：能够迭代和构建东西，而不必考虑基础设施。另一件事是容量方面的东西。我越是想 Modal 和我们的长期发展方向，我们正在构建——我们正在聚合世界的算力容量，我们非常擅长非常细粒度地切分它，因为我们围绕极快扩展和收缩构建了所有这些工作，所有围绕 GPU 快照的东西。我们正在研究一些方法让它快得多，而且这种模式在非常大的规模下运作得非常好，因为你可以聚合很多客户。

Yeah, massive differentiator actually. In fact, I think Modal's—if you want to be very reductive—you can break down Modal's value prop into two things. One is the developer experience: the ability to iterate and build things and not have to think about infrastructure. The other thing is the capacity stuff. The more and more I think about Modal and where we're going in the long run, we're building—we're aggregating the world's compute capacity and we're very good at slicing it out very granularly because we built all this work around scaling up and down extremely fast, all the stuff around GPU snapshotting. We're working on some ways to make it much, much faster, and that model works at very large scale really, really well because you can aggregate lots of customers.

每个月我一直在说我们的沙盒收入可能已经趋于平稳，但它每个月都在上涨 30%。

Every month I keep saying our sandbox revenue might have plateaued, but it keeps going up 30% every month.

这有点疯狂。就像，我不知道，每个月我都觉得，嗯，你知道，沙盒增长了很多，但显然它们有点趋于平稳了，然后它们继续增长得更多。所以我不知道。很多风投，他们会说，你怎么知道沙盒会变大？事实是，我不知道。我确信它会比 2023 年更大，但沙盒今天占我们收入的三分之一，这太疯狂了。你知道，我们显然对此非常高兴。我们有一个非常大的团队在投资沙盒。

It's kind of nuts. Like, I don't know, every month I'm like, well, you know, sandboxes grew a lot but clearly they kind of plateaued, and then they keep growing even more. So I don't know. And a lot of VCs, they're like, how did you know sandboxes was going to be big? And the truth is, I didn't. I thought it was going to be bigger than it was in 2023 for sure, but the fact that sandboxes is a third of our revenue today, it's insane. And you know, we're obviously very happy about it. We have a very large team investing in sandboxes.

你知道，我不想轻描淡写——我们显然想给自己很多功劳，因为在 2023 年甚至更早一点就看到了沙盒今天的模样。但我想我可能永远不会押注说沙盒的收入会这么大、市场需求会这么多。

You know, I don't want to trivialize it—we obviously want to give ourselves a lot of credit for seeing sandboxes for what it is today, a little bit back even in 2023. But I think I probably would have never taken a bet that revenue would be this large from sandboxes and the market demand would be so much.

有一个有趣的结构性原因，说明为什么推理提供商应该与模型构建者分开存在。最好的模型一直在变。推理提供商可以轻松地在模型之间重新分配资源，因为总体需求相当平稳。

There's an interesting structural reason why inference providers should exist separately from model builders. The best model keeps changing a lot. Inference providers can easily reallocate resources between models since overall demand is quite smooth.

所以我的意思是，如果你想想所有中国实验室，未来希望还有更多美国实验室，每次一个新实验室发布最新最棒的开源模型，你都会看到对这些模型的需求发生巨大变化。所以我认为一个挑战是，如果所有这些实验室都独立持有容量，他们会看到：当他们发布新模型时需求会大幅上升，然后另一个实验室发布了更好的模型，需求就下降了。

So I mean if you think about all the Chinese labs and in the future hopefully more US labs too, every time a new lab releases the latest and greatest open source models, you see big shifts in demand for these models. And so I think a challenge is if all these labs independently held capacity, they would see when they launch a new model demand would go up a lot, and then another lab launches a better model and then the demand goes down.

但当然，他们在后台正在开发下一个版本，所以我认为有一个结构性原因，就像我说的，为什么你会看到需求转向最新最棒的中国实验室。我的意思是，并不是一切都会一夜之间转移，因为很多模型和提示词等有一定的粘性。但我确实认为有一种论点，即对模型来说很容易——比如今天我们在为一个模型运行几千块 GPU，明天我们需要把其中 50% 转移到另一个模型。当然，这就是系统构建时要处理的事情。甚至不需要有人告诉我们去做。它就会神奇地做到。

But of course in the background they're working on the next version, so I think there's a structural reason, like I said, why you're seeing demand shifting to the latest and greatest Chinese lab. I mean, it's not like everything just moves overnight, because a lot of models and prompts and stuff have a certain stickiness to it. But I do think there's an argument that it's very easy for a model—like right today we're running x number of thousand GPUs for one model, tomorrow we need to shift 50% of that to another model. Sure, that's what the system was built to handle. No one even needs to tell us to do that. It just magically does that.

所以你同意基本上大语言模型是商品。如果你同意，那么你认为大实验室的业务将在哪里？

So you agree that basically the LLMs are commodities. And if you agree, then where do you think the business for the big labs is going to be?

我不确定那是否一定正确，对吧？就像泰勒·斯威夫特是商品吗？她发行一张大专辑并巡演赚很多钱，然后有一年低谷期她去录音。我不知道这是不是最好的类比，然后她发行新专辑。所以我不知道——她仍然有很多品牌，有很大的影响力和很高的利润率。所以只要实验室能反复推出新模型，并且只要他们在这些模型上的利润率足以支付——摊销——训练这些模型的成本，我认为这仍然可能是一个不错的生意。

I'm not sure if that's necessarily true, right? Like is Taylor Swift a commodity? Like she releases a big album and goes on a tour and makes a lot of money, and then there's a down year when she goes and records. I don't know if this is the best analogy, and then she releases a new album. So I don't know—she still has a lot of brand and there's a lot of power and makes a lot of margin. So as long as the labs repeatedly can launch new models and as long as the margin that they have on those models are such that they can pay off—amortize—the cost of training those models, it could still be a decent business I think.

所以你不认为智能正在趋于平稳？

So you don't think that the intelligence is plateauing?

不，我不这么认为。我的意思是，我认为这还有待观察，对吧？就像今天的模型非常擅长做那些可以通过强化学习高效训练的事情。所以我们最初在编码和编写简单代码片段上看到了这一点。随着时间的推移，它在编写生产级代码方面也变得非常出色。它在数学方面变得非常出色。它在网络安全方面变得非常出色。这些都是非常可验证的事情。我仍然认为，我们的模型是否会在烹饪食物或那些目标更分散、反馈循环更长的事情上变得非常出色，还有待观察。我认为人类——我的感觉是，在这些领域，我们将在很长一段时间内与模型相比相当出色且具有竞争力。但当然，AI 模型正变得非常非常好，我相信每个人都会同意。

No, I don't think so. I mean I think it remains to be seen, right? Like models today are very good at doing things that can be trained very efficiently using reinforcement learning. So we've seen that initially with coding and writing simple snippets. Over time, it's gotten really good at also writing production-level code. It's gotten really good at math. It's gotten really good at cybersecurity. Like these are all quite verifiable things. I still think it remains to be seen whether our models are going to be really good at cooking food or things that have more diffused targets and maybe longer feedback loops or whatever. I think humans—my feeling is we'll be quite good and quite competitive with models for a long time in those domains. But certainly AI models are getting very, very good, as I'm sure everyone can agree with.

是的。所以显然模型在编码方面变得非常出色。它们已经在编码方面非常出色了。你认为还有哪些目前可能不那么大的应用会在未来几年变得巨大？

Yeah. So obviously the models are getting really good at coding. They're already really good at coding. What other applications do you think maybe that aren't as big right now are going to become huge over the next few years?

我不知道，但我认为任何具有可验证性的——计算生物学，我想你提到了。

I don't know, but I think anything that has a verifiable—computational bio, I think you mentioned.

是的。是的。计算生物学。

Yeah. Yeah. Computational bio.

我认为那些并不完全是可验证的，但你可以通过某些方式运行模拟——你可以使用语言模型输出预测。然后你可以用计算方法模拟分子动力学，并获得某种反馈循环。我的意思是，我认为材料科学可能是另一个领域，你有一点那种循环，你可以合成制造某些材料并测量属性，你只需把一个智能体放入循环中并做事。所以我有物理学背景，我对此相当兴奋。所以药物、材料，还有什么——我一时想不起来了，但我相信还有很多其他事情。

I think those are not exactly verifiable, but there's certain ways you can run sim—you can use an LM to output predictions. Then you can do computational methods to simulate the molecular dynamics and get kind of feedback loops. I mean, I think material science is maybe another area where you have a little bit of that loop where you can synthetically manufacture certain materials and measure the properties and you just kind of put an agent in the loop and do things. So I have a physics background, I'm pretty excited about that. So drugs, materials, what else—drawing a blank, but I'm sure there's many other things.

酷。我的意思是，这些就是所有的问题了。有没有什么你想谈但我没问到的？

Cool. I mean those are all the questions that I had. Is there anything you wanted to talk about that I haven't?

我很好。这很棒。这很有趣。

I'm good. This is great. This is a lot of fun.

太棒了。谢谢。

Awesome. Thank you.