Modal CEO Erik 谈 Agent Maxxing、推理的未来与 GPU 市场的混沌

现在的算力市场有多大程度是有组织的市场，而不是像狂野西部那样？

How much of an organized marketplace is compute these days versus like the wild west?

我去年去了 GTC。那就像是一个露天的毒品市场，但交易的是 GPU，你走来走去跟人交谈，试图搞到 GPU，在酒店大堂见人，交换电话号码。这离有效市场还差得远。我是说，有些人在做这方面的努力。算力有很多不同的形态或形式。我还觉得从结构上讲，它在很长一段时间内都会有点笨重。就像它不是那种你可以在交易所买卖的东西，因为它不是一种可替代的合约。

I went to GTC last year. It was like an open air drug market, but like GPUs, walking around like talking to people, trying to get GPUs and meeting people in hotel lobbies, exchanging numbers. It's very far from an efficient market. I mean, some people are working on this. Compute comes in many different shapes or forms. I also feel like structurally it will be a little bit like sort of unwieldy for a long time. Like it's just not something you can like sort of buy and sell in an exchange because it's not like a fungible contract.

欢迎来到 Unsupervised Learning。从我的声音你大概能听出来，我不是 Jacob。我是 Patrick Chase，Redpoint 的普通合伙人。非常高兴能第二次邀请 Erik 来到 Unsupervised Learning。他是 Modal 的创始人兼 CEO。过去 3 年与他合作让我学到了很多。他是我最喜欢一起探讨 AI 想法的人之一。他有一种不可思议的能力，能从开发者交付代码的最低视角无缝切换到市场走向和构建缺失之物的最高视角。我们聊了 Modal 过去 3 年的历程、他们为实现这一切而构建的疯狂技术、推理沙盒以及接下来的发展。希望你喜欢这期节目。

Welcome to Unsupervised Learning. As you can probably tell from my voice, I am not Jacob. I am Patrick Chase, GP at Redpoint. So awesome to have Erik on Unsupervised Learning for the second time. He is the founder and CEO at Modal. I have learned so much from working with him over the last 3 years. He is one of my favorite people to jam on AI ideas with. He has an incredible ability to move seamlessly from the lowest altitude of developer shipping code to the highest altitude of where the market is going and building what is missing. We talk about the Modal journey over the last 3 years, the crazy technology they have built to make it all work, inference sandboxes and what's coming next. Hope you enjoy the episode.

显然有大量的 AI 算力需求，很多人出去试图预订 GPU 和 CPU。你们现在在市场上找算力是什么感觉？

There is obviously a ton of AI compute demand, and lots of people going out and trying to reserve GPUs and CPUs. What's it like for you guys finding compute in the market today?

相当艰难。我是说，不是那种我们真的找不到算力的艰难。我觉得有一点误解，我们在某些方面确实算力稀缺，但你能获得容量。你只需要看得更远一点，接受更高的价格，接受更长的期限等等。我们经常这样做。我是说，我们现在可能每周做两到三笔算力交易。所以我们不断堆积越来越多的算力。我真的认为这就是 Modal 的美妙之处，我们可以聚合所有这些需求，并更高效地以聚合方式管理容量。Modal 的价值主张一直是：别考虑容量，我们处理所有那些烦人的事情，那种日常的争夺容量的刀战。我们现在与 25 家不同的云合作，并且一直在增加。

It's pretty rough. I mean, not rough in the sense that we can actually find compute. I think there's a little bit of a misconception that we are certainly scarce in terms of compute in certain ways, but there is capacity you can get. You just have to look a little bit further, accept higher prices, accept longer terms, etc. And we do that a lot. I mean, we probably do like two compute deals every week at this point, if not three. And so we keep just stacking up more and more compute. And I really think that's the beauty of Modal is we can aggregate all this demand and manage the capacity in aggregate much more efficiently. And the value prop of Modal has always been like don't think about capacity, like we handle all of that annoying stuff, the day-to-day knife fighting of getting capacity and working. We work with 25 different clouds at this point and adding more all the time.

现在的算力市场有多大程度是有组织的市场，而不是像狂野西部那样，随便什么人 somehow 有容量，还有深夜短信？

How much of an organized marketplace is compute these days versus like the wild west like random people that somehow have capacity and you know late night text messages?

有很多深夜的 WhatsApp 群，而且相当无组织。我是说，我去年去了 GTC，或者今年早些时候在圣何塞，英伟达办的大型会议，那就像是一个露天的毒品市场，但交易的是 GPU。你走来走去跟人交谈，试图搞到 GPU，在酒店大堂见人，交换电话号码。在某个时候，我想我们联系上了一个有 GPU 的人，因为我们销售副总裁的表亲跟某人打高尔夫，然后我们联系上了。有很多这样的案例。这离有效市场还差得远。我是说，有些人在做这方面的努力，我认为随着时间的推移会变得更有效率一点，但它非常异质。算力有很多不同的形态或形式。所以我也觉得从结构上讲，它在很长一段时间内都会有点笨重。就像它不是那种你可以在交易所买卖的东西，因为它不是一种可替代的合约。

It's a lot of late night WhatsApp groups and it's pretty unorganized. I mean, I went to GTC last year or was earlier this year in San Jose, the Nvidia big conference they were doing, and it was like an open air drug market but like GPUs. You're just walking around talking to people trying to get GPUs and meeting people in hotel lobbies and exchanging numbers. And at some point, I think we got connected with someone who had GPUs because our VP of sales' cousin was golfing with someone and we got connected. There's many such cases. It's very far from an efficient market. I mean, some people are working on this and I think over time it will get a little bit more efficient, but it's very heterogeneous. Compute comes in many different shapes or forms. So I also feel like structurally it will be a little bit unwieldy for a long time. Like it's just not something you can buy and sell on an exchange because it's not like a fungible contract.

是的。是的，我是说关键区别在于，我猜是实际的数据中心外壳或集群规模，以及芯片本身，或者像你说的，有很多人一直试图构建，嘿，你知道算力是一种可替代的单一资产，你可以围绕它建立市场。你观察到的一些关键区别是什么？

Yeah. Yeah, I mean the key differences being like I guess the actual powered shell or size of the cluster and like the chips themselves or to your point there's a lot of people that have been trying to build like hey you know compute is this fungible single asset that you could build a marketplace around. What are some of the key differences you observe?

我认为很多实际上归结为可靠性。我是说，首先磁盘和网络有差异，当然 CPU 和内存等等，所有这些都不同，还有区域、位置，它们遍布世界各地，我们当然在美国各地有 GPU，但我们也在印度尼西亚等地有 GPU，遍布欧洲等等。所以位置是那些重要因素之一，但还有可靠性，那是我们做得特别好的另一件事，我们聚合所有这些容量，然后当 NeoClouds 死掉或出现故障时——它们经常有故障，因为有些是前加密货币矿工，他们并没有真正考虑过运行 24/7 运营——我们可以直接将其移出轮换，或者当 GPU 掉线之类的，我们在很多情况下只是自动提交工单，然后有人去查看。

I think a lot of this actually boils down to reliability. I mean first of all there's differences in disk and networking and of course CPU and memory and things like all these things vary and region like location like they're all over the world like we have GPUs in the US of course like all over the US but we also have GPUs in like Indonesia like all over Europe etc. So location is one of those big factors and then but also reliability and that's another thing we do exceptionally well is like we aggregate all this capacity and then when NeoClouds die or like when they have failures which they often have because some of these are like former crypto miners who haven't really thought about running a 24/7 operation we can just take it out of rotation or like when GPUs fall off the bus or whatever like we in many cases we just file a ticket automatically and someone goes and looks at it.

你觉得那些数据提供商，提供 GPU 定价之类信息的公司，你觉得它们有帮助吗？

Have you found the data providers, the companies that provide pricing for GPU and that sort of things, have you found them to be helpful?

我觉得中等有帮助。我认为定价方面不如它们与许多提供商建立联系的能力。我今天不完全信任定价。但我认为其中一些提供商实际上非常擅长了解所有 NeoClouds，我们有时与他们合作来获得与云提供商的联系。我认为定价随着时间的推移会变得更准确。就像我们内部定价与我们购买容量时公开看到的不匹配，但我认为随着时间的推移，希望我们会有更有效的像外部指数。我认为那实际上会非常有用。

I think moderately helpful. I think the pricing less so than just their ability to connect with a lot of providers. I don't fully trust the pricing today. But I think some of these providers are actually very good at knowing all the neo clouds and we work with them sometimes to get connections with cloud providers. The pricing I think over time will get more accurate. Like our internal pricing does not match what we see publicly when we buy capacity but I think over time hopefully we will have more efficient like external indices. I think that would actually be very useful.

过去几年你经历了火箭般的旅程，所以有很多事情我们很想为听众深入探讨，在商业方面。也许从这开始，你知道，我认为 Modal 历史上有趣的事情之一是，有一段时间感觉你公开说过，我想在你第一次 Unsupervised Learning 节目中，你不太关注这些 LLM 推理调用，特别是像简单的 API 调用模型。你知道，你最近发布了一个产品。也许谈谈这段旅程和最初的犹豫，以及是什么让你决定现在真的是加倍投入的合适时机。

You've been on this like rocket ship journey these past years and so lots of things we'd love to dig into for our listeners on the business side. Maybe starting, you know, I think one of the interesting things about the history of Modal is you know for a while it felt like you publicly said I think on your first Unsupervised Learning episode that you were less focused, you know, on these LLM inference calls, specifically like a simple API call for a model. And you know, you kind of recently released a product there. Maybe talk a little bit about the journey and kind of that initial reticence and what made you decide that like now is the right time to really, you know, double down on this.

是的。

Yeah.

我们大概在 2021 年底、或者那前后开始做这个东西。最初的产品是一个无服务器函数，我们觉得它在数据、AI、机器学习这些广泛领域会非常有用。我们看了很多不同的用例，但后来 2022 年底聊天应用出现了，Stable Diffusion 也差不多同时出现。

So we started building this in late 2021, basically, or early somewhere around that time. The initial product was this serverless function, and we thought it would be really useful for a wide range of stuff around data, AI, machine learning broadly speaking. We looked at a lot of different use cases, but then chat came in late 2022 and Stable Diffusion also came around the same time.

对，对。

Right. Right.

结果发现我们在托管一批 Stable Diffusion 应用上位置特别好。所以第一年的增长基本全是 Stable Diffusion。然后我们开始扩展到相邻的用例，比如音频、视频之类的。那时候是早期团队在部署推理，他们不想处理底层基础设施，只想快速推进。我们开始跨很多不同的云聚合大量不同的 GPU。即便在那时，我们也一直知道我们想做多产品，但我们从无服务器这个计算原语开始，因为我们觉得它足够通用。

It turned out we were super well positioned to host a number of Stable Diffusion applications. So that was like the first year of growth — it was all Stable Diffusion. Then we started expanding into adjacent use cases like audio and video and stuff like that. And it was early teams that were deploying inference; they didn't want to deal with underlying infrastructure, they wanted to move fast. We started aggregating a lot of different GPUs across a lot of different clouds. Even back then, we always knew we wanted to be multi-product, but we started with the serverless compute primitive because we felt like it was so general purpose.

我们很聚焦于我认为的高代码用例——想写代码的机器学习工程师和研究者——并且很聚焦于待在技术栈更底层，对吧？很早我们还建了一个存储层、一个文件系统来存数据，尤其是容器镜像。然后 2023 年我们推出了沙盒产品。当时推出它的原因是，我们开始看到人们在用大语言模型生成代码，我们觉得我们在基础设施侧建的很多东西非常适合运行这些代码，而他们会需要一个隔离的地方来跑。

We focused a lot on what I think were high-code use cases — machine learning engineers and researchers who wanted to write code — and focused a lot on being lower in the stack, right? Pretty early we also built a storage layer, a file system to store data, in particular container images. Then we launched a sandbox product in 2023. The reason we launched it back then was we started seeing people messing around with LLMs to generate code, and we figured a lot of stuff we built on the infrastructure side was really well suited to run that code, and they were going to need a place to run it in an isolated way.

它不是一夜成功，但基本上到 2025 年它就爆发了，因为我觉得那时候模型写代码变得非常强。所以大约一年前。那是我们迈向多产品厂商的一步。从大约 12 个月前开始，Sandboxes 一路猛涨，涨了大概一千倍还是多少。它现在约占我们营收的三分之一。单独算，Sandboxes 是一个 3 亿美元年化收入的业务。所以就成为多产品厂商而言，这非常惊人。

It wasn't an overnight success, but basically in 2025 it kind of blew up because I think models got really good at writing code back then. So about a year ago. That was sort of our venture into becoming a multi-product vendor. Starting about 12 months ago, Sandboxes has been on a tear and grown, I don't know, almost a thousand-x or something like that. It's about a third of our revenue right now. Standalone it's a 300 million run-rate business — Sandboxes. So that's been incredible in terms of becoming a multi-product vendor.

我们刚还推出了一个训练产品。所以我们在尝试铺开所有不同的积木，想清楚这套工具长什么样——在这个 AI 应用的新世界里，工具箱里有什么？它们都会需要一堆新的原语，我们想提供所有这些原语。那会是什么样？

We just launched a training product too. So we're trying to lay out all the different building blocks and figure out what that suite of tools looks like — what are the toolboxes in this new world of AI applications? They're all going to need a bunch of new primitives, and we want to offer all those primitives. What does that look like?

LLM 推理这块我们有一阵子有点置身事外，直到——我会说是 DeepSeek 时刻。那大约是一年半前，2025 年初，DeepSeek 出来的时候，它算是第一个真正很强的中国开源模型。然后我就想，“管他呢，我们得进场。”我们开始组建团队做大量推理优化。

LLM inference was something we were a little bit on the sidelines for a while, until actually — I would say the DeepSeek moment. That was about a year and a half ago, early 2025, when DeepSeek came out, which was like the first Chinese open source model that was actually really good. And then I was like, "Fuck it, we got to get in here." We started building up a team to do a lot of inference optimization.

Modal 一直很擅长 LLM 推理，但我们不一定有对应的产品。事实上，直到最近我们甚至都没有推理产品。我们只是很擅长托管推理，包括 LLM 推理，但那有点偏定制化，一两年前很多想跑 LLM 推理的团队得自己部署，或者我们跟他们合作。但随着 DeepSeek 时刻，我们开始看到大量需求。过去 18 个月我们开始看到很多增长。LLM 推理现在也约占我们营收的三分之一，所以这块增长非常惊人。

Modal has always been really good at LLM inference, but we didn't necessarily have a product for it. In fact, we didn't even have an inference product until very recently. We were just really good at hosting inference, including LLM inference, but it was kind of custom, and a lot of the teams that wanted to run LLM inference a year or two ago had to deploy it themselves, or we would work with them. But we started seeing a lot of demand with the DeepSeek moment. Over the last 18 months we started seeing a lot of growth. LLM inference is now about a third of our revenue too, so we've seen incredible growth with that.

几个月前我们发布了一批 LLM 专属功能，因为我们认为我们已经铺好了这些基础层——存储、计算、无服务器函数、文件系统等等。我觉得往上走有很大的机会。所以我们现在开始往技术栈上层走，推出了专用推理产品。你可以拿到单租户模型——所有最先进的中国模型，或者一堆美国模型——一键或一条 CLI 命令就能部署到 Modal 上，还有一堆共享端点，你可以直接调用现成模型而不用自己部署，完全按 token 计价。

A few months ago we shipped a number of LLM-specific features because we think we've laid these foundational layers — the storage, the compute, the serverless functions, the file system, etc. I think there's a lot of opportunity to move up in the stack. So we started moving up in the stack now and launched a dedicated inference product. You can get single-tenant models — all the state-of-the-art Chinese models or a bunch of US ones too — with one single click or CLI command, you can just deploy on Modal, as well as a bunch of shared endpoints where you can just call models off the shelf without having to deploy them, with fully token-based pricing.

我觉得这些是未来很多产品中的第一批，位置稍微更靠上层。我们认为机会很大，因为我们在计算和存储上坐拥非常扎实的基础。

I think those are the first of many more products to come, slightly higher up in the layer. We think there's a lot of opportunity because we're sitting on this very solid foundation in the compute and the storage.

也许你们会从技术栈底部开始，一路往上走。但回到过去，你提到过——你们早期投入了一些非常硬核的技术。你们自建了文件系统、容器运行时这类东西。这些在今天的产品里怎么体现？

Maybe you guys will start at the bottom of the stack and work your way up. But going back in time, you touched on it — you guys invested in some really hardcore technology in the early days. You built your own file system, container runtime, those types of things. How does that show up in the product today?

对，我觉得这个市场里一个巨大的 alpha 来源就是不害怕做难的事。当我们意识到，为了交付我们早期想要的开发者体验，我们得自建文件系统来把容器镜像做得非常快时，我们就全力投入了。所以我们在很早的时候就开始建这个。

Yeah, I think a massive source of alpha in this market is just not being afraid to do hard things. We went all in when we realized that in order to deliver on the developer experience we wanted in the very early days, we kind of had to build our own file system to do container images really fast. So we started building that in the very early days.

而这成了——在接下来几年里，它仍然是一个巨大的 alpha 来源。这肯定是一段更长的旅程。我们不得不走一条长得多的路：自建文件系统、自建容器运行时、自建调度器——我们不用 Kubernetes 和 Docker。我们用很少的现成软件。我们用 gVisor，那是 Google 一个很棒的隔离产品；最近也开始在某些任务上用 KVM。但我们很早就决定，很多东西我们得自己建。

And that became — over the next few years, it still is a very massive source of alpha. It's been a longer journey for sure. We've had to go a much longer route: building our own file system, building our own container runtime, our own scheduler — we don't use Kubernetes and Docker. We use very little off-the-shelf software. We use gVisor, which is a fantastic Google product for isolation; recently started using KVM for certain tasks too. But we decided pretty early on that a lot of the stuff we're going to have to build ourselves.

别人也做了一些很酷的东西。我觉得这体现得最好的例子之一，是我们冷启动可以极快。结果发现，我们最初想要的开发者体验——在云上启动东西时非常快的反馈循环——在构建需要快速扩缩容的自动伸缩应用时也非常有用。

Others do some really cool stuff. I think one of the best examples where that shows up is we can cold start extremely fast. It turns out the initial developer experience we wanted — a very fast feedback loop when you're launching stuff in the cloud — is also very helpful when you're building autoscaling applications that need to scale up and down very fast.

首先，容器的加载非常快。但随着时间的推移，我们还推出了几个非常酷的功能，围绕 CPU 快照和 GPU 快照。所以我们可以非常快地把数据从磁盘或网络直接加载到 GPU，甚至在几秒内启动这些大模型，这真的很酷。而在过去六个月里，我们构建了这套极其复杂的系统，基本上利用节点之间的点对点 RDMA 来非常快地复制数据。所以我觉得很多东西都在积累，并且真正开始产生回报。我们造了一些很疯狂的东西，现在我们真的在收获那些我们投入了很多很多年的投资所带来的红利。那是一条更长的路，但现在我认为我们拥有一些极其独特、非常难以复制的技术。

First of all, the loading of containers is very fast. But over time we also launched a couple of really cool features around CPU snapshotting and GPU snapshotting. So we can very fast load data from the disk or from the network straight into the GPU and fire up even these large models in a few seconds, which is really cool. And over the last six months we built this incredibly complex system that basically uses RDMA over peer-to-peer between the nodes to copy data very fast. So I think a lot of this stuff is accumulating and really starting to pay off. We built some crazy stuff, and now we're really reaping the dividends of those investments that we made for many, many years. It was a longer route, but now I think we have some incredibly unique technology that's very hard to replicate.

然后往上说到推理——你们在这方面也做了大量投资和优化。你认为推理领域、或者说推理栈里，还有哪些未解决的问题？

And then moving up to inference — you guys have made tons of investments there as well and optimizations. What do you view as the unsolved problems in the inference world, or in the inference stack?

我认为推理还处在相当早期的阶段。很多东西就像那种非常经验性的科学——人们在尝试不同的做法，看什么有效。随着时间推移，人们已经搞明白了很多东西。就拿分离式预填充来说吧，这是个非常不直观的想法：你在一组节点上做预填充，然后通过 RDMA 把数据复制过去做解码，有点像交换。有一些非常不直观的东西，但我们在这些方面投入了很多。KV 缓存——当然，围绕 KV 缓存有大量复杂性，还有运行更高效的 kernel，这是大家做得很多的事。有很多这类东西。我们现在有一个大概 20 到 30 人的团队，极度专注于语言模型性能优化。这需要大量工作：你得跟上研究进展，得读论文，得看所有不断涌现的软件，试着看什么有效、什么无效。它仍然有点像经验科学。随着时间推移，我认为模式会浮现出来。我也认为模型本身会变化得非常快——现在我们有混合专家模型和更复杂的东西。随着时间推移，谁知道呢，也许我们会把 Transformer 换掉，换成别的东西，那会改变 KV 缓存的工作方式。所有这些事情都可能在某个晚上就变了，如果有人想出一种非常不同的范式的话。也许他们会，也许不会，我不知道。显然还有大量围绕量化和其他东西的工作。所以是的，外面是个狂野的世界，但我们在玩所有这些。

I think we're still pretty early with inference. A lot of stuff is like one of those very empirical sciences — people are trying different things and seeing what works. Over time people have figured out a lot of stuff. Take something like disaggregated prefill, right? It's a very non-obvious idea: you do the prefill on a set of nodes and then you copy the data over RDMA and do the decode and kind of swap. There are some very non-obvious things, but we've invested a lot in those types of things. KV caching — there's a lot of complexity around KV caching, of course, and running more efficient kernels, that's something people do a lot. There are a lot of these things. We have a team of, I think, 20 to 30 people right now who are extremely focused on just LM performance optimization. And it takes a lot of work: you have to stay up with research, you have to read papers, you have to look at all the software coming out all the time and try to see what works and what doesn't work. It's still kind of an empirical science. Over time I think patterns will emerge. I think also the models themselves will change very quickly — now we have mixture of experts and more complex things. Over time, who knows, maybe we'll swap out the transformers for something else, and that would change how KV caching works. There are all these things that could change overnight if someone figures out a very different paradigm for all these things. Maybe they will, maybe they won't, I don't know. There's also obviously a lot of stuff around quantization and other things. So yeah, it's a wild world out there, but we're playing around with all these things.

但感觉有很多人在研究改进推理的不同部分。而且显然，过去两三年非常有意思的一点是，很多这些优化和改进都发生在开源里，对吧？也许通过研究分享给更广泛的社区。作为 CEO，你如何权衡你想让团队去解决的那类问题的层次，对比说，嘿，我们就受益于整个社区推动这个领域前进？

But it feels like there are so many folks working on different parts of improving inference. And obviously what's been really interesting the last two or three years is a lot of those optimizations and improvements have been in open source, right? And shared with the broader community, maybe via research. As a CEO, how do you think about the level of the kinds of problems you want your team to go solve versus, hey, we're going to benefit from just the entire community pushing this space forward?

我们其实很喜欢开源，因为我认为我们的很多 alpha 在专有的一侧，在基础设施层——比如围绕自动扩缩容的那些很酷的东西。所以我认为，和其他推理提供商不同，我们其实非常重度地投入开源。我们和 vLLM 团队以及 SGLang 团队都合作得非常紧密。我们非常积极地给这两个仓库做贡献，并和他们一起做大量优化。所以我觉得这可能和其他推理提供商的做法有点不同。当然，有些东西我们会保留在内部，但几乎所有的东西我们都会推到开源。我认为这是因为我们本来在基础设施上就有很多 alpha，那我们不如直接和外面那些了不起的开源引擎合作——它们几年前可能还不是最先进的，但现在已经是了。坦白说，这是我们很想去利用的一件了不起的事。

We actually love open source because I think a lot of our alpha is on the proprietary side, in the infrastructure layer — all this cool stuff around autoscaling, for instance. So I think unlike other inference providers, we've actually leaned very heavily into open source. We work very closely with both the vLLM team and the SGLang team. We contribute very actively to both those repos and do a lot of optimizations with them. So I think this is maybe a little bit of a different approach than maybe other inference providers. Of course there are going to be certain things that we keep internal, but almost everything we push to open source. And I think it's because we have so much alpha in the infrastructure anyway, we might as well just work with the amazing open source engines that are out there — that maybe weren't state-of-the-art a few years ago, but now they are. And that's an incredible thing that we'd love to leverage, frankly.

你之前谈到异构算力，以及你们从不同地方获取算力的方式，还有推理栈正在如何变化。在你们考虑供给、或者你们的服务栈随着时间会是什么样子时，芯片的长尾——非 Nvidia 的芯片之类的——对你们有多重要？

You were talking about heterogeneous compute and the different places where you're getting compute from, and also how the inference stack is changing. How important is the long tail of chips to you guys — the non-Nvidia chips and things like that — as you think about your supply or what your serving stack looks like over time?

我是说，我认为 Nvidia 仍然是这里的王者。他们拥有最好的芯片。对 Nvidia 的需求远远超过其他任何东西。我们对其他一些芯片很好奇——我们做过一些实验，看了看——但 Nvidia 肯定有最好的芯片。不过看看 AMD、Trainium、TPU 等等，它们开始变得相当不错了，所以我们一直对此很好奇。当然还有很多更老的芯片。我认为挑战在于，对很多语言模型来说，你一直在推能塞进多少参数。所以通常最新最强的语言模型总是需要一整个节点——也许未来甚至需要多个节点，谁知道呢？比如八个 B300，它们需要最新的芯片，而且要八个，因为一个节点上就能装这么多。但除此之外还有很多模型——更老的语言模型、更小的、为性能优化的、参数可能更少的模型。

I mean, I think Nvidia is still the king here. They have the best chips. There's way more demand for Nvidia versus anything else. We're curious about some of the other chips — we've made some experiments, taken a look at it — but Nvidia definitely has the best chips. But looking at AMD, Trainium, TPUs, etc., they're starting to be pretty good, so we're always curious about that. And then of course there's a lot of older chips too. I think the challenge has been that for a lot of LMs, you're constantly pushing how many parameters you can fit. So typically the latest and greatest LMs will always require a full node — maybe in the future even multiple nodes, who knows? So like eight B300s, they will require the latest chip and eight of them, because that's how many you can fit on a node. But there are plenty of models outside of that — older LMs, smaller ones that are optimized for performance, that might have fewer parameters.

然后，在语言之外，还有很多围绕语音、音频转录、语音合成，或者制药、生物等领域的东西。其中很多实际上并不需要最新最强的 GPU。它们在 Hopper 甚至 A100 或不同类型的 L40S 或 RTX Pro 6000 等芯片上服务反而更具成本效益。所以我们正在非常积极地扩大 Blackwell 的规模，主要是 GB200，但我们也看到旧款芯片上有大量需求，我觉得这很令人兴奋。有太多事情在发生，人们忘记了 Hopper，但它们目前仍然是 Modal 上最常见的类型——H100。我们基于 Hopper 运行一些非常大的生产用例。

And then outside language, there's a lot of stuff around voice, audio transcription, speech synthesis, or pharma, bio stuff. A lot of that stuff actually doesn't require the latest and greatest GPUs. They actually get more cost efficient serving on Hopper or even A100s or different types of L40S or RTX Pro 6000 stuff like that. So we are ramping up very aggressively on Blackwell, primarily GB200s now, but we see plenty of demand also on the older chips, which I think is quite exciting. There's just so much stuff going on. People forget about the Hopper, but they are the most common type right now still at Modal—H100s. We run some very large production use cases based on Hopper.

如今整个行业的模型与芯片匹配效率如何？你觉得大多数人是否从价格角度为他们所做的工作使用了最优芯片，还是说这方面仍有很大的改进空间？

How efficient is model-to-chip matching today in the industry at large? Do you feel like most people are using the optimal chip from a price perspective for what they're working on, or is there still a lot of room for improvement there?

有一点改进空间。我们随着时间的推移想要做的一件事就是为人们自动化这一点。在实践中，问题没有看起来那么大，因为通常你受限于内存。所以知道模型需要多少内存后，你基本上选择能装下模型的最便宜的 GPU。这通常是几乎在所有情况下都有效的启发式方法。所以如果你的模型是 80 GB，你可以把它放在 Hopper 或更旧的 A100 80 GB 上。在那种情况下，这有点取决于 FLOPS。但在许多情况下，就是选择能装下你模型的最小的 GPU。随着时间的推移，我们一直在考虑很多事情：你只需部署一个模型，然后我们管理不同的芯片，也许自动给你 GPU 数量，因为我们有越多选项在多种芯片上运行它，就能更好地聚合容量并以更高的利用率运行。然后我们可以在不同 GPU 类型之间切换，因为它们存在利用率不足的情况。所以这是我们未来六个月非常感兴趣做的事情：给人们一个选项，以某种方式部署东西，我们在底层自动切换类型，同时仍然符合你的每请求美元或每 token 美元之类的。我认为你可以在那里做一些非常酷的事情。

There's a little bit of room to improve. One of the things we want to do over time is automate that for people. In practice, it's less of a problem than it might seem because often you're memory constrained. So knowing how much memory you need for a model, you basically pick the cheapest GPU that fits the model. That's usually the heuristic that works in almost all cases. So if your model is 80 gigs, you can either fit it on a Hopper or an older A100 80 gig. In that case, it depends a little bit on the FLOPS. But in many cases, it's whatever GPU is the smallest that fits your model. Over time, there's a bunch of stuff we've been thinking about: you just deploy a model and then we manage different chips and maybe automatically give you GPU counts because the more options we have to run it on a lot of different chips, the better we can aggregate capacity and run things at higher utilization. Then we can switch between different GPU types as there's underutilization on those. So that's something we're quite interested in doing over the next six months: giving people an option to deploy things in a way where we automatically switch types underneath in a way that's still consistent with your dollar per request or dollar per token or whatever. I think there's some incredibly cool stuff you can do there.

这非常有趣。我想在你的角色中有一件非常有趣的事情是，你显然在预测你的算力集群应该是什么样子，而且你显然看到了今天的工作负载是什么，你可能对哪些模型在哪些芯片上运行得好有一些了解。然后你还在试图预测，嘿，一两年后我们会有什么样的工作负载？我认为有很多不同的争论，关于大多数运行的 token 是会来自真正的大型模型，还是实际上我们已经饱和了大多数任务，这些模型会变得越来越小。当你思考未来一两年你的集群的演变时，你对哪里会看到相对更多不同种类的工作负载、哪里会更少有什么看法？

It's super interesting. And one thing I imagine is so interesting in your role too is you're obviously predicting forward what your compute fleet should be, and obviously you see what the workloads are today, and you probably have some sense of what models run well on what chips. Then you're also trying to predict, hey, a year or two from now, what kind of workloads will we have? I think there's lots of different debates about whether most tokens running will be really large models, or actually we've saturated most tasks and these models will get smaller and smaller. As you think about the evolution of your fleet in the next year or two, what's your perspective on where you'll see relatively more different kinds of workloads and less?

我认为过去三四年基本上有一个模型越来越大的趋势。与此相反——我确实觉得那个趋势会继续。我们将需要——如果 B300 有更多内存，我确信 Kimi K3 会更大。人们训练模型的方式就是让它们能装在这些芯片上。所以我认为至少在语言模型方面——语言模型在某种程度上是一种粗糙的世界模型；你试图在这些语言模型中编码尽可能多的信息。你需要非常大的、几乎像是事实查找表的东西，在某种程度上——这是我思考它的一种概念——而这在很大程度上推高了内存需求。在某个时候可能会逐渐减弱，谁知道呢，但就目前而言，人们会利用每一份可能的内存。内存越多，他们肯定会训练更多参数。

I think there's been basically a trend of larger and larger models over the last three or four years. Against that—I definitely feel like that trend is going to continue. We're going to need—if B300s had more memory, I'm sure Kimi K3 would be bigger. People are training models in the way that they fit on these chips. So I think at least on the LM side—LMs are in a way a crude type of world model; you're trying to encode as much information as you can in these LMs. You need very big, almost like lookup tables of facts, in a way—this kind of conception I think about it—and that sort of pushes the memory requirements a lot. At some point it might taper off, who knows, but for now people will make use of every possible memory. The more memory, they'll just train more parameters for sure.

你显然在某种程度上处于一个非常令人兴奋、庞大且竞争激烈的市场的飓风眼中。所以我很好奇，当你思考最终什么会让这个领域的参与者脱颖而出并获胜时。我认为有很多人坚信整体推理市场的规模，很多人都在问最终到底什么会胜出。我知道你对此有一个很好的论点，所以我很想让你和我们的听众分享。

You obviously sit in some ways in the eye of the hurricane of a very exciting, large, and competitive market. So I'm curious as you think about what ultimately will differentiate players in the space and win out. I think there are so many true believers in the size of the overall inference market, and lots of folks are asking what actually wins out at the end of the day. I know you have a good thesis on this, so I would love for you to share it with our listeners.

我认为推理市场的大部分会相当艰难。那种直截了当的只是提供 token 服务——你已经看到 OpenRouter 基本上在使底层提供商商品化。我倾向于认为真正的 alpha 或真正的差异化点——仅仅在更高效的服务方面就会有很多差异化，但这有点逐底竞争。而且我认为会有很多机会保持领先于竞争。你必须为此付出很多努力。

I think a lot of the inference market will be quite rough. The sort of straight down the middle just serving tokens—you already see OpenRouter basically commoditizing the underlying providers. I tend to think that the real alpha or the real point of differentiation—there will be a lot of differentiation just in more efficient serving, but it is a little bit of a race to the bottom. And I think there's going to be a lot of opportunities to stay ahead of competition. You're going to have to put a lot of effort into that.

但我认为更持久的差异化一定在于提供一整套平台，涵盖很多不同的东西，而我觉得这正是我们很有优势去做的事。我们有沙盒，有强化学习，很多客户也在做常规的微调。很多客户显然还在部署其他类型的模型。我们有批处理。我总体的看法是，客户不想买只会一招的东西，他们想买的是多产品的工具组合。所以对我来说，这最终可能就是最大的差异化来源：你如何提供那一整套工具？你如何提供这种非常通用的东西，让工程师能快速迭代、交付价值？然后日常层面，我们仍然得去打那种白刃战，在性能上保持领先于竞争对手，用最新的 kernel 和更高效的 KV cache。这类事情还会持续很久，因为里面钱太多了。但我想指出的是，我认为这作为优势没那么持久，因为随着时间推移，这些技巧会扩散，人们会想办法复制。所以更持久的东西，我认为是围绕推理的一整套产品组合，而不只是推理本身，拥有所有模型，以及围绕强化学习之类的东西。

But I think the more durable differentiation will definitely be in providing the entire platform of a lot of different things, which I think is something we are quite well positioned to do. We have the sandboxes, we have reinforcement learning, a lot of customers doing also normal fine-tuning. A lot of customers obviously deploying other types of models. We have batch processing. I think my view in general is that customers don't want to buy single-trick ponies, they want to buy a multi-product set of tools. So to me that's ultimately probably the biggest source of differentiation: how do you provide that entire set of tools? How do you provide this very general-purpose thing that lets engineers iterate very fast, deliver value? And then the day-to-day, we're still going to have to go and do the knife fighting of staying ahead of your competitors in terms of performance and use the latest kernels and the much more efficient KV cache. There's going to be a lot of that stuff for a long time because there's so much money in it. But I'm kind of pointing out that I think that's less durable as an advantage, because over time these tricks will spread and people will figure out how to copy them. So the more durable thing I think is being that portfolio of a lot of stuff around inference, not just the inference in itself, having all the models and a lot of stuff around reinforcement learning, maybe, or other things.

Modal 最大的客户都在用多个产品。也许你可以讲讲几个比较有意思的客户？

The biggest Modal customers are on multiple products. Maybe can you touch on a few of the more interesting ones?

是的，我们有一些非常酷的客户。Cognition 在大规模使用我们，既用于推理，也用于沙盒，来做大量模型上的强化学习。他们用沙盒做所有的 rollout。像 Ramp 这样的客户用我们做后台智能体，用于他们内部的编码智能体，那个智能体他们今天刚作为独立产品发布并开始售卖。像 Suno 和 Runway 这样的客户用我们做生成式视频和音频。Decagon 是另一个客户，用我们做大量客服相关的东西，语音和文本都有。DoorDash 用我们做一堆内部应用，主要是 LLM，但也有各种其他东西。我们在计算生物技术领域其实有很多客户，用我们做各种类型的药物发现，用分子模拟和序列比对之类的。我们在机器人领域有一批客户，1X 和其他几家，用我们训练机器人模型，做大量训练，也做视频批处理之类的事情。数据生成领域也有很多客户，就是那些构建环境、在模型上跑基准测试的客户，经常用沙盒。

Yeah, we have some incredibly cool customers. Cognition is using us at large scale for both inference and sandboxes to do a lot of reinforcement learning on models. They use sandboxes for all the rollouts. We have customers like Ramp using us for background agents for the internal coding agent that they actually launched as a standalone product today that they're selling. We have customers like Suno and Runway that use us for generative video and audio. Decagon is another customer that uses us for a lot of customer support stuff, voice and text. DoorDash uses us for a bunch of internal applications, mostly LLMs, but also all kinds of other stuff. We actually have a lot of customers in computational biotech using us for various types of drug discovery, using molecular simulations and sequence alignment and all kinds of stuff. We have a bunch of customers in robotics, 1X and a few other ones, using us to train robot models, doing a lot of training, but also batch processing of video and things like that. A lot of customers in the data generation space, customers who are building environments and running benchmarks on models, often using sandboxes.

你提到的一点，这个产品组合，我觉得这是你在构建的业务中最有意思的部分之一，而且我想说实话，最难的部分之一一定是对所有你能构建的东西做优先级排序。大概会不断冒出很多新东西，很多能为开发者构建的非常令人兴奋的工具。我很好奇：显然在沙盒方面，你有围绕编码智能体、围绕强化学习的出色产品。你怎么考虑沿着那个方向往下做？还有一些竞争对手似乎优先做了微调或 all-as-a-service 之类的东西。你怎么考虑在这个产品组合内部该优先做什么？

One thing you kind of mentioned, this portfolio of products, and I think that's one of the most interesting parts of the business you're building, and I imagine honestly one of the hardest parts must be just prioritization of all the different things you could build. There's probably so many new things that come up, really exciting tools you can build for developers. I guess I'm curious: obviously with sandboxes you have incredible products around coding agents, around reinforcement learning. How do you think about building down that vector? There's also the fine-tuning or all-as-a-service stuff that some of your competitors seem to have prioritized. Like, how do you think about where to prioritize within that portfolio?

我觉得这在大多数初创公司都是极其困难的问题，尤其是多产品公司，对吧？首先是平衡：什么时候该走向多产品，然后你的赌注押在哪里。我们今天大概有 150 名工程师，我每次、每天都为此纠结。我希望我们能做那个，还有那个。但我多少知道，即使我们有 1000 名或 10000 名工程师，也还是这样——就像我希望我们能造火箭上月球之类的。这种事永远不会消失。所以你必须对优先级极其无情，真正去想什么 ROI 最高、线画在哪里。有时候你还得相当强势地说：其实我知道我们启动了这个项目，我知道你真的很兴奋，但另一件事冒出来了，所以我们得先暂停这个，之后再回来做。我们也很多次不得不这么做。发生的时候很难受。但我真的认为，优先级排序以及如何分配资源——资源分配基本上是我认为 CEO 在公司里必须真正无情对待的头号事情，把工作分配给正确的资源能创造巨大的价值。

I think it's an incredibly hard problem at most startups, but especially multi-product companies, right? The balance of, first of all, when do you go multi-product, and then where do you put your bets. We're 150 engineers or something like that today, and I struggle with it every time, every day. I wish we could do that, and whatever. But I kind of know that even if we had a thousand engineers or 10,000 engineers, it's all you — it's like, I wish we could build a rocket and go to the moon and whatever. It just never goes away. So you kind of have to be just ruthless with prioritization and really think about what's the highest ROI and where do you draw the line. And you have to be sometimes pretty aggressive also saying, actually, I know we started on this project, I know you're really excited about it, but this other thing kind of came up, so we're going to have to pause this for a second. We'll get back to it later. We've had to do that many times too. It sucks when it happens. But I really think that prioritization and how do you allocate resources — I mean, resource allocation is basically the number one thing that I think a CEO has to really be ruthless about at a company, and so much value is being created by allocating the work to the right resources.

每次董事会上 Modal 的路线图都很长，也很吓人，我总是很佩服团队能交付那么多软件，硬核软件，同时还能让系统保持运行。你们是在 ChatGPT 之前就在建这家公司，然后经历了 ChatGPT 之后的时刻。这改变了多少，也许在产品侧，你们在做的事情，还有你们思考构建的方式，以及你们交付软件的速度？

The Modal roadmap in each board meeting is extensive and very daunting, and I'm always impressed by the amount of software that the team can ship, hardcore software, while keeping the system running. How much has — you guys were building the company pre-ChatGPT, and then post-ChatGPT moment. How much has that changed, maybe on the product side, what you guys were doing, but then also when you were thinking about building and the velocity that you can ship software at?

是的，我的意思是，我们显然受益于在 AI 爆发的时候处于相当有利的位置，看到这些非常不可思议，我们今天在支撑那么多不同的 AI 应用，而四五年前我们创办 Modal 时，我根本不知道它们会存在。这挺不可思议的。就像你说的，软件本身也——我们构建软件的方式已经变了很多。我觉得到现在，Modal 合并的代码行里大概 95% 在某种程度上是智能体写的。基础设施方面，我感觉仍然很依赖——智能体在这方面还算擅长。

Yeah, I mean we've obviously benefited from being quite well positioned at a time where AI kind of blew up, and that's been incredible to see, and we're powering so many different AI applications today that I had no idea would exist four years ago, five years ago when we started Modal. That's been kind of incredible. And like you said, software in itself has also — how we build software has changed a lot. I think probably at this point 95% of the lines of code that get merged at Modal is probably written by agents to some extent. Infrastructure still, I feel like, relies a lot on — agents are kind of good at it.

其实过去几天——其实是过去几个月——我一直在疯狂地用智能体写大量代码。它们能构建的东西令人难以置信，但你仍然需要人类大量监督。基础设施在可靠性、确保扩展性等方面有很多尾部风险。所以至少目前，我认为软件工程师相当安全，我们仍然受益于拥有非常优秀的工程师来监督这些智能体。可能未来几年都是如此。

I've actually been on a tear the last few days—actually the last few months—writing an insane amount of code using agents. It's kind of incredible what they can build, but you still need humans to supervise them quite a lot. Infrastructure has a lot of tail risks around reliability and making sure things scale. So at least for now, I think software engineers are pretty safe, and we still benefit from having incredibly good engineers who can supervise these agents. Probably for the next few years.

另一件有趣的事是，采用模式实际上与我的预期相反。我以为采用智能体最多的是初级人员。但实际上几乎相反——是那些非常资深的人。因为对他们来说，这就像给了他们 10 个实习生。他们长期以来受限于打字速度，但脑子里有很多想法，现在用智能体就能更快地实现。他们非常擅长构建测试套件。给智能体一个详尽的测试套件——它实现了文件系统的所有 POSIX 标准——然后去写一个新的文件系统，它就会去写一个文件系统，大概花三天时间完成。所以我们看到一些非常资深的工程师完全沉浸在智能体中。你能用智能体做的事情令人难以置信——你可以非常快速地推进。归根结底，未来几年人类作为软件工程师似乎仍然安全。我不知道。我们拭目以待。

Another thing that's been interesting to see is that the adoption pattern is actually the opposite of what I would have expected. I thought it would be junior people adopting agents the most. I've actually seen almost the other way around—it's the very senior people. Because for them it's like giving them 10 interns. They've been constrained by typing speed for a long time, but they have all these ideas in their heads, and using agents now they can get them out much faster. They're really good at building a test suite. Give the agents an extensive test suite—it implements all of the POSIX standard for file systems—now go write a new file system, and it just goes and writes a file system, and it takes like three days to do it. So we've seen some of the very senior engineers go complete agent maxing. It's kind of incredible what you can do with agents—you can move very fast with them. At the end of the day, it seems like humans are still safe as software engineers for the next few years. I don't know. We'll see.

你觉得软件交付的速度变化如何？你们在代码审查、CI/CD 或部署方面需要做些什么？开发生命周期中的这些环节有变化吗？

How have you found the velocity of shipping software has changed? What you guys need to do with code review or CI/CD or deployment? Have those pieces of your development life cycle changed at all?

我觉得有一点变化。我们开始遇到一些限制——我们不得不在 CI/CD 上花更多时间。甚至在智能体出现之前很久，我做了多年 CTO，一直痴迷于 CI/CD，认为它非常非常重要。从这个意义上说，我不知道我的观点是否变了——你真的需要有自动化部署、自动化测试，所有这些。顺便说一句，现在智能体也很擅长写测试；一年前它们还不行。所以我不确定我们是否已经遇到了根本性限制，但 Anthropic 几天前宣布他们要离开 Git——至少是 GitHub。当我们每天的提交量达到 10 倍时，我们可能某个时候会遇到那些限制。

I think a little bit. We're coming up against some of those constraints—we've had to spend a lot more time on CI/CD. Even long before agents, I was a CTO for many years and was always obsessed with CI/CD as very, very important. In that sense, I don't know if my view has changed—you really need to have automated deployments, automated tests, all these things. Now, by the way, agents are pretty good at writing tests too; they weren't a year ago. So I don't know if we've necessarily run up against the foundational limits yet, but Anthropic a couple days ago announced they're moving off Git—or at least GitHub. We might run up against those limits at some point when we have 10x the amount of commits every day.

想到这些真正强大的编码智能体让世界变得多么不同，这很有趣。显然，我想你的联合创始人谈到的部分内容是，随着时间的推移，Modal 的消费者甚至从开发者转变为智能体本身。我很好奇你如何看待这对 Modal 作为一家企业意味着什么，以及随着越来越多的人通过智能体或完全端到端的智能体与 Modal 交互，你可能会做出哪些不同的决策。

It's interesting to think about how different the world is with these really powerful coding agents. Obviously, I think part of what your co-founders talked about is the shift even in who the consumer of Modal is over time—from a developer to agents themselves. I'm curious how you think about what that means for Modal as a business and some of the decisions you might make differently as more and more folks interacting with Modal are either doing it via an agent or purely end-to-end agent.

实际上变化比我想象的要小得多。事实证明，智能体想要的东西是一样的。早期我们构建 Modal 是为了给工程师更好的开发者体验和更快行动的能力。事实证明，智能体也想要同样的东西——它们想要快速反馈循环、良好的错误信息、良好的文档。很多工程师想要的东西也是如此。所以大部分东西其实非常相似。我确实认为某些事情有点更难。例如，我们发现弃用功能要麻烦得多，因为智能体通常是在网上存在的大量旧代码上训练的。

It's actually changed a lot less than I would have expected. It turns out agents want the same thing. In the early days, we built Modal to give engineers a better developer experience and the ability to move faster. It turns out agents kind of want the same thing—they want fast feedback loops, good error messages, good documentation. A lot of the stuff that engineers want too. So most of the stuff is actually quite similar. I do think certain things are a little bit harder. One of the things we've found, for instance, is deprecations are much more annoying because agents are often trained on a lot of old code that exists online.

所以移除功能对我们来说难得多。我们不能只是从文档中移除然后等三个月，因为它在训练数据里。人们会用 LLM 生成 Modal 代码，而那会使用已弃用的功能，这很难改变。所以我们不得不大幅调整弃用窗口。我们开始经常做的另一件事是建立自动化管道，基本上让 LLM 持续编写 Modal 代码，并确保它能构建和运行，试图捕捉智能体难以处理的问题。然后我们花很多时间的另一件事更像是智能体优化，这实际上更类似于搜索引擎优化。你可以去问 Claude Code 类似‘嘿，我想用推理做点事。我想部署这个模型’，而不提 Modal，看看它推荐哪个工具。显然，我们希望它尽可能推荐 Modal。所以这是我们内部跟踪的事情，以确保随着时间的推移——我实际上认为这会稍微改变软件工程，也许几乎是往坏的方向。我觉得在某种程度上，由于训练数据和训练循环，现在软件中有更多的在位者优势——有更多的持久力。如果你进入训练数据并让模型推荐某个框架，我听到有人说，‘哦，我就用 React 做所有事情，因为智能体很擅长它。我不会尝试这些新的花哨的前端框架。’我认为今天很多框架都有类似的情况。这种持久力现在实际上非常非常有价值。所以我们显然想确保智能体尽可能推荐 Modal，因为这创造了一个小小的飞轮。

So removing features is much harder for us. We can't just rely on removing it from the documentation and waiting three months, because it's in the training data. People will use LLMs to generate Modal code, and that will be using deprecated features, and that's quite hard to change. So we've had to change our deprecation window quite a lot. Another thing we've started doing quite a lot is setting up automated pipelines to basically have LLMs continuously write Modal code and make sure it builds and runs, trying to catch things that agents struggle with. And then maybe another thing we spend a lot of time on is more like agent optimization stuff, which is actually more similar to search engine optimization. You can go and ask Claude Code something like, 'Hey, I want to do something with inference. I want to deploy this model,' and not say anything about Modal, and just see which tool it recommends. Obviously, we want it to recommend Modal as much as possible. So that's something we track internally to make sure that over time—and I actually think that's something that will change software engineering a little bit, maybe almost for the worst. I feel like in a way, because of the training data and the training loops, there's more incumbency power in a way—there's more staying power in software today. If you get inside the training data and you get the models to recommend a certain framework, I've heard people say, 'Oh, I'm just going to use React for everything because agents are really good at it. I'm not going to try these new fancy front-end frameworks.' I think there's something similar with a lot of the frameworks today. That staying power is actually very, very valuable now. So we obviously want to make sure that agents recommend Modal as much as possible because that creates a little bit of a flywheel.

我想到的一点是，你们在 2023 年对沙盒做出了非常有先见之明的押注，对吧？听起来你们当时就有这种直觉：嘿，这将成为编码智能体所需要的东西。然后过了几年它才真正起飞，之后它变成了一个非常重要的基础构件。我很好奇，你们今天还在做哪些事情，你们觉得有点早，但同样兴奋？以及你们的产品组合中有多少是这类押注，多少是‘我们知道这是火烧眉毛的当下问题’？

I mean, one thing I'm struck by is you obviously had this very prescient bet in 2023 on sandboxes, right? It sounds like you guys had this instinct of like, hey, this is going to be something that's required for coding agents, and then it took a few years for it to really take off, and then it became this really important primitive. I'm curious: is there anything else that you guys are working on today that you feel like you're a little early on but equally as excited about? And how much of your product portfolio is those kind of bets versus, okay, we know this is a hair-on-fire, here-and-now problem?

Jacob，你是在套我们下一步该投资什么吗？

Jacob, are you fishing for the next thing we should be investing in?

百分之百，百分之百。我就是这么厚脸皮。

100%, 100%. I'm nothing if not shameless.

我觉得沙盒确实给我们设定了一个有点高的门槛。大家都在问，下一件事是什么？你们在搞什么秘方？实验室里在煮什么？我希望我能轻松回答：是的，百分之百确信，这个东西是……

I feel like sandboxes did create a little bit of a high bar for us. Everyone's like, what's the next thing? What's the secret sauce you're working on? What's cooking in the lab? And I wish I had an easy answer like, yeah, 100% confidence, this thing is...

你在 23 年就会回答沙盒吗？还是它只是你们当时在做的众多事情之一？

Would you have answered sandboxes in '23? Or was it just part of a bunch of things you were doing?

不，因为当时沙盒所代表的是一个有点不对称的押注。当时对我们来说，构建沙盒并不是一件很难的事。做出沙盒的原型只花了几天时间，然后我花了一点时间改进 SDK，其他人开始构建，然后我们开始看到一些客户。我们重写了它，然后在 2025 年它开始爆发时，我们不得不重写系统五次，只是为了处理越来越大的规模，让它更快、更可靠等等。但最初的投入其实相当基础。所以从这个意义上说，它有点像一个不对称的押注，下行风险低、上行空间大。我不知道今天我们在做的事情中是否有类似的。我的意思是，就举几个我非常看好的例子，这些可能代表更大的投资、更大的风险，就押注规模而言：低延迟。我认为会有很多语音对语音、很多实时视频类的东西，需要我们重新思考很多关于如何在全球管理算力、如何减少跳数、如何在不同区域扩展和收缩，几乎像是为推理构建一个 CDN。所以我对这个很兴奋。我认为训练实际上是一个被低估的事情。我认为在训练中型模型和强化学习方面有很多机会。过去几年我们一直在大力押注，比如让所有 RDMA InfiniBand 之类的东西运转起来。还有什么？我认为很多虚拟机相关的东西相当有趣。比如支持完整的操作系统，如 Mac、Windows、Android 和 iPhone，可能会很酷。我认为那里有很多上行空间。所以我脑子里有很多这些想法，我们正在构建很多这样的东西。我不知道是否有任何东西能像沙盒那样具有爆发性潜力。我的意思是，也许如果我们 3 年后再次做这个播客，我们会回顾并……

No, because what sandboxes represented back then was a little bit of an asymmetric bet. Sandboxes was not a very hard thing for us to build back then. Getting out a prototype of sandboxes took like a few days, and then I spent a little bit of time improving the SDK, and some other people started building, and then we started seeing some customers. We rewrote it, and then in 2025 when it started exploding, we had to rewrite the system like five times just to handle more and more scale and make it faster and more reliable and things like that. But the initial investment was actually pretty basic. So in that sense, it was a little bit of an asymmetric bet, like a low downside, high upside bet. I don't know if there's anything similar today that we're working on. I mean, just to call out a couple of things that I'm quite bullish on, and these probably represent way larger investments, larger risks, in terms of the bet size: low latency. I think there's going to be a lot of voice-to-voice, a lot of real-time video type stuff that requires us to rethink a lot of how we manage compute globally, how do we get down the number of hops, how do we scale up and down in different regions, almost like building a CDN for inference. So I'm pretty excited about that. I think training is actually kind of an underrated thing. I think there's a lot of opportunity in training midsize models and reinforcement learning. That's been a big bet for the last couple years, like getting all the RDMA InfiniBand type stuff working. What else? I think a lot of VM stuff is quite interesting. Like supporting full-blown operating systems like Mac and Windows and Android and iPhone could be kind of cool. I think there's a lot of upside there. So I have a lot of those ideas in my head and we're building a lot of those things. I don't know if there's anything that represents the same sort of explosive potential as sandboxes. I mean, maybe if we have this podcast again in like 3 years, we'll look back and...

到那时我们会把它变成常设节目。所以我们期待重新讨论这个话题。

We're going to make it a recurring thing at this point. So we'll look forward to revisiting that.

你知道，显然沙盒已经成为编码智能体的基础构件，感觉这把你带入了一个世界，我想象你看着编码智能体的现状，你会觉得，天哪，还有这么多工具能帮助这些智能体，让它们对使用者更有效。你如何考虑是否进一步构建那个世界，或者那里对你来说有趣的一系列事情？

You know, I guess obviously sandboxes have become this fundamental primitive for coding agents, and it feels like that then brings you into this world where there's like, I imagine you look at the current state of coding agents, you're like, god, there's so many more tools that would be helpful for these agents to make them more effective for people that use them. How do you think about whether you further build into that world or the set of things that are interesting for you there?

我们确实考虑过，但我认为我们更多是从基础设施的角度来考虑。我觉得很容易迷失在当下最新的东西里。当我看着智能体框架的架构以及我们如何构建智能体等等，我仍然觉得我们太早了，这些架构演进得太快。在某种程度上，我们的工作是试图弄清楚，我们今天可以推出哪些核心构件，它们会持久，人们在未来 5 年都会需要它们。这对我来说不那么明显。我们对智能体编排作为一个类别感兴趣。我认为那里有很多有趣的东西。我们对评估网关以及围绕智能体学习和迭代的整个生命周期之类的各种东西感兴趣。但我不太清楚架构具体会怎样……我的观点是，它会更加以云为中心，可能更多是持久对象、工作流引擎类型驱动的，我们有一些在那方面构建的想法。但我觉得很难准确描绘 5 年后智能体式开发会是什么样子。部分原因是我认为在某种程度上，所有软件基本上都会是智能体式的。我有点讨厌‘智能体式’这个词，因为我觉得它就像是，好吧，所以它是用 AI 来做决策，那么，难道所有软件不应该都这样吗？所以我几乎觉得，如果你看五年视野，我认为所有软件都会变得智能体式，那么那时软件是什么样子？我不知道，但我希望我们能构建很多构件，来支持软件使用 AI 的各个方面。

We do think about it, but I think we think about it more from an infrastructure perspective. I think it's very easy to get lost in whatever is the latest today. When I look at the architecture of agent harnesses and how we're building agents and stuff like that, I still kind of feel like we're so early and these architectures are evolving so fast. And what our job to some extent is, is trying to figure out what are the core building blocks that we can put out today that will last, that people would need them for the next 5 years. And that to me is less obvious. We are interested in agent orchestration as a category. I think there's a lot of interesting stuff there. We're interested in eval gateways and all kinds of other stuff around the whole life cycle of learning from agents and iterating on agents. But I think it's less clear exactly how the architecture... My view is it will be more cloud-centric and it will be probably more durable objects, kind of workflow engine type driven, and we have some ideas of stuff to build in that. But I find it harder to lay out exactly what agentic development will look like in 5 years. Partly because I also think that to some extent all software will basically be agentic. I kind of hate the word agentic in certain ways because I feel like it's like, okay, so it's using AI to make decisions, like, well, shouldn't all software do that? So I almost feel like if you look at a five-year horizon, I think all software will become agentic, and then what does software look like then? I don't know, but I hope we can build a lot of the building blocks to support all kinds of aspects of just how software uses AI.

是的。我觉得这是为公司构建时最难的部分之一，对吧？对许多人来说，在公司里构建 AI，什么才是真正持久的，对吧？而不是一种模式，比如两年前的 RAG 之类的。

Yeah. I mean this feels like one of the hardest parts of building for a company, right? Of you know across the board for many people building AI in for companies of what actually persists, right? Versus is a pattern, you know, is like RAG from two years ago or whatever it is.

RAG 是个好例子。没错。对吧。就像如果我们两年前押注 RAG，那会是一件有点糟糕的事情。

RAG is a good example. Exactly. Right. Like if we had made a bet on RAG like two years ago, it would have been kind of a bad thing to do.

所以是的，但我也想说的是，构建基础设施的好处之一就是你可以有更长远的视角，不必听所有的噪音。我的意思是，你确实得听噪音，但你得忽略其中很多，试着弄清楚：好吧，这里的模式是什么？

So yeah, but I would also say one of the benefits of building an infrastructure is that you can take a longer view and you don't have to listen to all the noise. I mean, you have to listen to the noise, but you kind of have to ignore a lot of it and try to figure out, okay, what are the patterns here?

你刚才提到了一些其他工作负载，比如低延迟工作负载，像语音到语音和实时视频。现在有很多关于多模态推理、机器人推理这些其他类别的讨论。我想听你随便聊聊，这些问题在多大程度上类似于推理市场以前面临的那组问题？又在多大程度上是不同且全新的？

You kind of alluded to some of these other workloads, like low latency workloads like voice-to-voice and real-time video. There's a lot of chatter in the air about multimodal inference, robotics inference, these other categories. I'd love to hear you just riff a little bit on to what extent are those problems similar to the set of problems the inference markets had before. To what extent are they like different and new?

我觉得它们都很相似，但就像雪花一样，各有各的独特之处。机器人技术当然很多都会在设备端。所以这是一个很大的区别。我不太设想我们会支持机器人技术的大部分形态，因为很多都会在设备本身上。但我认为我们能够支持很多机器人技术的测试、开发和训练，以及很多仿真环境。所以这是我非常兴奋的事情。在多模态方面，有音频、音乐、图像之类的。人们谈论得没那么多，但那个领域也在疯狂增长。我觉得过去几年 LLM 主导了讨论空间，但我对所有不同的应用都无比兴奋。比如为什么 2026 年了语音助手还这么差？我不理解。它们就是很糟糕。我有一个 Google Home，每个问题都搞得一团糟。我就是不明白它在听什么。所以我干脆用 ChatGPT。为什么不能把所有这些端到端连接起来，训练一个系统让它能做语音到语音？我觉得我们会在这个领域看到很多非常酷的应用。我总是在想，回顾四年前、三年前，你能猜到 Instinct 或 Sumo 或 Cognition 这些东西吗？三年前这些都不太容易猜到。

I think they're all quite similar, but they're like snowflakes, unique in their own way. Robotics is going to be a lot of it on device, of course. So that's a big difference. And I don't sort of envision that we're going to support most of what robotics looks like because a lot of it is going to be on the devices themselves. But I think we're going to be able to support a lot of the testing and development and training of robotics and a lot of the simulation environments. So that's something I'm very excited about. On the multimodal stuff, I mean there's audio and music and image and stuff like that. People don't talk about it as much but that keeps growing like crazy too. I feel like LLMs have kind of dominated the airspace in the last few years, but I'm just incredibly excited about all the different applications. Like why are voice assistants so bad in 2026? I don't get it. They're just horrible. I have a Google Home and every question is just bungled. I just don't understand what I'm saying. So I just go to ChatGPT. Why can't you just connect all of it end to end and train a system so that it can do speech to speech? I think we're going to see a lot of incredibly cool applications in that space. I always think about, you look back four years ago, three years ago, could you have guessed things like Instinct or Sumo or Cognition? None of those things are very easy to guess three years ago.

所以我也觉得，如果你展望未来三年，大多数人都不知道三年后世界会是什么样。这太难知道了。但我知道的是，从过去三年来看，肯定会有一些非常酷的应用。我只是不知道会是什么，但它们会非常酷，希望它们会用 Modal。

So I kind of feel like also if you look down the road like three years, most people don't have an idea of what the world is going to look like in 3 years. It's so hard to know. But what I do know is, judging by the past three years, there's going to be some really cool applications. I just don't know what it's going to be, but they're going to be very cool and hopefully they'll use Modal.

我回看了你发给我的一封邮件，附了 Modal A 轮融资的 pitch deck，你是这么说的。那是在我们签了 term sheet 之后。顺便说一句，我们其实从没看过 Modal 的 deck，但这里是我们之前做的东西，我刚更新了最新数据。也许某个时候可以看看。你发这个的时候，年化收入大概 200 万美元。现在你已经突破了 10 亿美元年化收入，这太疯狂了，就在三年内。在短短几年里，支持这种规模的最大变化是什么？

I was looking back at an email you sent me with the Modal Series A pitch deck and this is what you said. This was after we had signed a term sheet. By the way, we never actually reviewed a Modal deck, but here is something we made earlier that I just updated with the latest numbers. Could be good to review at some point. When you sent this, you were at roughly 2 million in run rate. You now have crossed a billion run rate, which is insane, in three years. What's been the biggest change to supporting that kind of scale in just a couple years?

首先，我前几天意识到，我在另一个播客上说我从没做过 pitch deck，然后我仔细想了想，也许我在 A 轮之后确实做了一个 pitch deck，因为我们有几个天使投资人进来，他们在问这个。所以我想，好吧，那我就整理一个吧。所以我意识到我确实撒谎了。我确实做了一个 pitch deck，但那是在之后，而且你从来没给我们看过，所以我不知道那算不算。

So first of all, I actually realized the other day I said on another podcast that I've never made a pitch deck and then I was actually thinking about it and I was like maybe I did make a pitch deck after the Series A because we had a couple of angels that came in and they were asking about it. So I was like, oh fine, I guess I'll put something together. So I realized I did lie. I did make a pitch deck, but it was after and you never showed it to us, so I don't know that it counts.

是啊。所以我不知道那算不算。我从来没给投资人发过 pitch deck。我其实只发给过几个天使投资人。

Yeah. So I don't know if it counts. Like I never sent a pitch deck to an investor. I sent it to a few angels actually.

但那其实是我唯一做过的 pitch。不过就规模而言，是的，我们增长了 500 倍，我猜按收入算，差不多是这样，在过去三年里。所以增长很大。我觉得在某种程度上，我不知道，也许我的经验主要是在超高速增长的创业公司，比如我在 Spotify 待了七年，还有一些其他地方。所以对我来说，这种快速增长就是我唯一见过的。就像每六个月你就得重写所有系统，你得重新思考你在做的一切，你得改变所有流程。你总是觉得自己落后了。我不知道。我其实挺喜欢这种混乱的。

But that's the only pitch I've ever made actually. But in terms of the scale, yeah, we've grown 500x I guess in terms of revenue, something like that since then over the last three years. So a lot of growth. I think in a way, I don't know, maybe my experience has mostly been at hyper-scale startups like I was at Spotify for seven years and some other places. So for me, this fast growth is kind of all I've seen. Like every six months you have to rewrite all the systems, you have to rethink everything you're doing, you have to change all the processes. You always feel like you're behind. I don't know. I kind of like the mess.

我想既然我们在播客上有了你的见解，我们很想拉远一点，也许聊聊更广泛的基础设施市场以及最近新闻里的一些事情。也许最近几周最受关注的是，我觉得有大量关于 Neoclouds 的讨论，对吧？比如债务交易情况，它们下跌了，现在又稍微回升了一点。我很好奇，你认为这个格局随时间会怎样演变？

I guess we would love to, since we have your mind on the podcast, would love to zoom out, maybe talk a little bit about the broader infra market and some things that have been in the news of late. Maybe most top of mind these last few weeks, I feel like there's been tons of discourse about Neoclouds, right? They like, you know, how the debt's trading, they traded down. They're kind of back up a little bit now. And I'm curious, what do you think happens to this landscape over time?

我觉得 Neoclouds 是可行的商业模式。我觉得人们有点混淆了。他们说，哇，需求这么大，收入会增长得这么快。然后有一点 VC 思维，用收入倍数来思考。但我认为所有这些 Neoclouds，增长对它们来说其实相当昂贵，因为它们必须去筹集大量债务。所以我不确定是否仅仅因为它们增长很多，就非常资本低效。这些公司本质上就像融资工具，它们在租赁硬资产，这本身就很难扩展资本。现在市场在说，哇，我们向 Neocloud 投入了大量资本，也许先等等。我们没有那么多资本可以借出，所以利率开始上升。但顺便说一句，这些公司的杠杆率非常高。所以当然会有更大的波动性。剩余股权价值在股票市场上会随着 GPU 价格等东西上下波动得更快。

I think the neoclouds are fine business models. I think people have been confusing it a little bit. They're like, wow, there's so much demand, revenue is going to grow so fast. And then there's a little bit of a VC mindset of thinking in revenue multiples. But I think all these neoclouds, growth is actually quite expensive for them because they have to go and raise a lot of debt. So I don't know if necessarily just because they grow a lot, it's very capital inefficient. These companies at their core are like financing vehicles, they're doing leasing of hard assets, and that's inherently harder to scale the capital. Now that market is kind of saying whoa, we deployed a lot of capital into neocloud, maybe hold on for a second. We don't have that much more capital to lend and so the interest is starting to go up. But also by the way, these companies have incredibly high leverage. So of course they're going to have more volatility. The residual equity value is going to go up and down much faster on the stock market with GPU prices and stuff like that.

但它们的商业模式还不错。它们锁定了客户合同，并以此举债，现在正在偿还债务。在很多情况下，残值高于人们的预期。GPU 的寿命比人们预期的要长。所以一旦还清债务，它们可以以高于人们预期的价格转售。它们会从中赚很多钱。这需要几年时间才能实现，但我认为从这个意义上说，新一类公司会表现不错。它们并没有特别差异化。所以长期来看，我认为它们会开始……

But they're like fine business models. They kind of locked in customer contracts and raised debt against it, and now they're paying off that debt. In many cases, the residual value has been higher than people expected. The lifetime of GPUs has been longer than people expected. So they can resell once they've paid it off at a higher price than people thought. They're going to make a bunch of money from that. It takes a few years to get there, but I think in that sense the new class will do well. They're not particularly differentiated. So long term, I think they're going to start...

你和很多这类公司合作。你看到任何差异化吗？

You work with a lot of them. Is there any differentiation you see?

我是说，有一点，但核心上这是一个相对缺乏差异化的领域。所以我的观点是，埃隆前几天发推说所有的新云都会死掉。我觉得这有点……我确实认为埃隆有道理，他暗示 SpaceX 的资本成本要低得多。如果你看看三家成功的超大规模云厂商——亚马逊、Azure 或 AWS、Azure 和 GCP——它们都依附于现金流非常稳健的公司，而且都是上市公司，资本成本非常低。因为如果你要构建资本密集型的东西，这从根本上让生活轻松很多。所以总的来说，我认为这是 NeoClouds 会挣扎的地方。它们的资本成本会很高，因此很难快速增长。所以市场方面我们拭目以待，但它们的商业模式还不错。我认为可能存在逢低买入的机会。我不知道。

I mean, it's like a little bit, but at its core it's a relatively undifferentiated space. So my view is like, Elon was tweeting the other day that all the new clouds are going to die. I think that's kind of... I do think Elon had a point when he implicitly was saying SpaceX has a much lower cost of capital. I don't think there's a coincidence that if you look at the three successful hyperscalers—Amazon, Azure or AWS, Azure, and GCP—they're all attached to companies that had very solid cash flow and they're all public companies that have very low cost of capital. Because that at its core makes life a lot easier if you want to build something that's very capital-intensive. So I think in general that is something that NeoClouds will struggle with. They're going to have a high cost of capital and it's going to be hard for them to grow fast because of that. So we'll see about the market, but they're decent business models. I think there might be an opportunity to buy the dip. I don't know.

你发过一条有趣的推文，说很多 NeoLab 如果模型不成功，可能会变成 NeoCloud，但它们已经锁定了这些合同。我绝对能想象世界朝那个方向发展，如果你有有价值的算力，由于需求巨大而持续升值。

You had an interesting tweet that a lot of the Neolabs could become NeoClouds if their model doesn't work out, but they have locked in these contracts. I definitely could see the world going that way if you have valuable compute that continues to appreciate because there's so much demand.

是的，我是说，Mrol 就有点这么做了，对吧？而且看起来还不错。我认为很多人在做这些非常大的购买决策时，会合理化说，好吧，如果你不需要 GPU，我们可以转售，对吧？这几乎是一种保险形式。你必须假设所有这些 NeoLab——外面有那么多 NeoLab，每家都融了 10 亿美元。不知为何他们总是恰好融 10 亿美元。我不知道为什么。我猜他们喜欢这个数字。然后购买 GPU……

Yeah, I mean, Mrol kind of did that, right? And it seems to be fine. I think a lot of people are making these very large buying decisions and they rationalize it like, well, if you don't need the GPUs we can just resell them, right? That's almost like a form of insurance. And you have to assume with all these Neolabs—there are so many Neolabs out there that have each raised $1 billion. They always raised exactly $1 billion for some reason. I don't know why. I guess they like that number. And then buying GPUs...

非常精确的财务模型指向最优金额。

Very precise financial modeling that points to the optimal amount.

是的。是的。所以我认为其中一些可能会非常成功，但我想在大局中有些不会成功。如果它们不成功，那它们现在坐拥大量 GPU。让我们转售这些 GPU，对吧？特别是如果出现风险投资资金回撤的情况。因为我认为这些承诺有点可怕的地方在于，今天当你做出 GPU 承诺时，你只需要预付大约 30%。有时多一点，有时少一点。但这意味着你可以融 10 亿美元，然后出去获得 30 亿美元的 GPU，只预付 10 亿美元——数学可能不精确——但问题是一年后你还要承担另外几亿美元，现在每个月都要付款。所以我认为这在很多公司中造成了一点隐性杠杆，并依赖后续融资。只要它们能去融风险投资的钱来支付 GPU 账单，这就行得通。但到了某个时候可能就行不通了，在那种情况下，我认为你会看到更多二级市场，一些 NeoLab 可能会转向做更多类似 NeoCloud 的事情——比如我们有一堆 GPU——或者直接转售回云。我对 AI 整体非常看好，但我总是寻找这些结构性风险。所以总的来说，我认为会有不可思议的应用、不可思议的需求，但 100% 的 NeoLab 都会成功并成为万亿美元公司吗？可能不会。

Yeah. Yeah. So some of them I think will probably work out really well, but I imagine in the grand scheme some of them will not work out. And if they don't work out, well, now they're sitting on a ton of GPUs. Let's resell those GPUs, right? Especially if there's ever a scenario where VC funding pulls back. Because I think what's kind of a scary thing about some of these commitments is that effectively what happens today when you make a GPU commitment is you only have to pay about 30% upfront. Sometimes a little bit more, sometimes a little bit less. But what that means is you can raise a billion dollars, you can go out and get GPUs for $3 billion and only pay a billion dollars up front—maybe not exact math—but the problem is in a year you're on the hook for another few hundred million and every month now you have to pay. So I think that creates a little bit of implicit leverage in a lot of these companies and a dependence on subsequent fundraising. And that works as long as they can go and raise VC dollars to fund the GPU bills. But at some point it might not work, and in that case I think you're going to see a little bit more of a secondary market where some of these Neolabs might pivot to doing a little bit more sort of neo cloud style—like we have a bunch of GPUs—or just resell it back to the cloud. I'm very bullish on AI as a whole, but I always look for these sort of structural risks. So overall I think there's going to be incredible applications, incredible demand, but is 100% of the Neolabs going to succeed and be a trillion-dollar company? Probably not.

我觉得在 AI 基础设施方面，一个备受关注的大类别是 RL 即服务市场，很多人几乎在前置部署，帮助公司构建自己的模型。你也有像 Satcha 这样的人说：“嘿，每家公司都会有自己的模型。”你认同吗？你对整个市场怎么看？

I feel like on the AI infra side, one big category that has gotten a lot of buzz is this RL as a service market and a bunch of folks almost forward deploying and helping companies build their own models. You have had folks like Satcha saying, "Hey, every company's going to have their own model." Do you subscribe to that? And what do you think of that overall market?

不，我认为有些公司会有自己的模型。我认为按美元收入加权，大多数模型可能会是后训练的。但按公司数量加权，大多数公司不需要自己做后训练。但大公司，比如 Cognition 正在训练自己的代码模型，Cursor 也做了，所有这些公司都在做，而且它们也处理大量 token。所以我认为当你观察强化学习和后训练时，某些部分非常有吸引力。你可以与公司合作，提供更好的模型，以更低的成本和更高的吞吐量等等。有些看起来只是服务业务。所以我认为未来几年后训练方面的很多紧张关系将在于：你能把它变成产品吗？你能把那个产品卖给这些公司吗？这是我们思考很多的事情——我们如何创造这个你可以直接销售的惊人产品，而不是把它变成服务业务。

No, I think some companies will have their own models. I think that weighted by dollar revenue, most models will probably be post-trained. But weighted by company—in terms of numbers of companies—most companies will not have the need to do their own post-training. But the large ones, like Cognition is training their own code models, and Cursor did, and all these companies are, and those are the ones also serving a lot of tokens. So I think when you look at reinforcement learning and post-training, certain types of parts of it are very attractive. You can work with companies and serve better models and do it at lower cost and higher throughput and all of this stuff. Some of it looks like it's just a service business. So I think a lot of the tension over the next few years when you look at post-training is going to be: can you turn this into a product? Can you sell that product to these companies? This is something we think a lot about—how do we create this amazing product you can just sell and not turn it into a service business.

然后我想在进入快问快答环节之前，最后一个问题是关于在 AI 世界中，有多少工具存在于单一基础设施提供商之下，而不是分开的类别。每次和你交谈，我总是印象深刻的是你多么想去构建，以及所有这些构建模块。

And then I guess last one before we move to the quickfire round would just be this question about how many tools live under a single infra provider in the AI world versus separate categories. I'm always struck when we talk to you about just how much you want to go build and all these building blocks.

我想知道，有没有哪些 AI 基础设施的类别，你认为会和你正在做的东西不同、独特，而且你看好它们，它们仍然会是很大的独立机会？

I'm wondering if there are categories of AI infra that you think will be unique and distinct from the stuff you're building, that you're optimistic on and that will live as still large standalone opportunities.

嗯，我们不会去做模型训练。我觉得 TypeSafe 加上 Jav 会——这类定制模型大概会有很多。我确信在其他垂直领域也是，其他模态比如音频、视频和音乐。显然在应用侧会有很多了不起的公司。然后大概也会有很多公司更专注于软件开发之类的东西。我对这些事情的一般看法是，市场地图上的格子总是比你预期的多。但我也确实认为，总体而言，每一家最终成功的公司在某种程度上都必须成为一家多产品公司。所以我不知道这算不算一种张力。我觉得我们显然想找到一组非常相邻的东西，并以最大化营收和股东价值的方式把它们全做了。但我们绝不会试图扩张到所有东西上。我觉得人们也有这种错误的假设——你看看 Anthropic 或 OpenAI，他们就说，为什么它们不往栈上走，开自己的律所，开自己的这个那个？它们非常擅长服务这些模型并大规模地做这件事——它们就是这样实现规模的。而且它们非常擅长做这个。我不认为每家公司都该什么都做。所以对我们来说也一样，我们不会什么都做，但我们会做一大簇相邻的东西，并把它做到极其出色。我认为市场上会有很大空间，去创造一堆非常大的、千亿美元或万亿美元级别的机会。

Yeah, I mean we're not going to go into model training. I think type safe with Jav is going to be — there's probably going to be a lot of these custom models. I'm sure in other verticals too, other modalities like audio and video and music. And obviously on the application side there's going to be a lot of incredible companies. And then there's probably going to be a lot of companies focusing more on software development and stuff like that. My general view on these things is there's always more boxes on the market map than you expect. But I also do think that overall every company that is successful in the end, to some extent, has to become a multi-product company too. So I don't know if that's a tension or not. I think we obviously want to find a cluster of very adjacent things and do all of it in a way that maximizes the revenue and the shareholder value. But by no means are we going to try to expand into everything. I think people kind of make this false assumption also — you look at Anthropic or OpenAI and they're like, why don't they move up the stack and launch their own law firm and launch their own whatever? They're very good at serving these models and doing that at scale — that's how they achieve scale. And they're very good at doing that. I don't think it makes sense for every company to do everything. And so similarly for us, we're not going to do everything, but we're going to do a very large cluster of adjacent things and do it exceptionally well. And I think there's going to be a lot of room in the market to create a bunch of very large hundred-billion or trillion-dollar opportunities.

我们最后来做快问快答环节，塞进去的问题大概和三年前问你的那套差不多，也许稍微进化一点。先来一个：我想听一件过去一年里你在 AI 上改变了看法的事。

We'll end on our quickfire round where we stuff in probably a similar set of questions to what we asked you three years ago. Maybe a little bit of an evolution. To start, I'd love just one thing you've changed your mind on in AI in the last year.

嗯，沙盒的机会规模显然有点疯狂，对吧？我没想到它会接近——我是说，

Well, the opportunity size of sandboxes has obviously been kind of crazy, right? I didn't think it would be anywhere close to — I mean,

我以为你 23 年就知道这一点，所以才下了那个大注。

I thought you knew that in '23 and that's why you made the big bet.

是啊。所以那挺不可思议的。

Yeah. So that's been kind of incredible.

编程智能体改变了很多「当工程师意味着什么」。它有没有改变、以及在多大程度上改变了你对招聘的看法？

Coding agents have changed a lot of what it means to be an engineer. How, if at all, has it changed how you think about hiring?

没怎么变。我觉得很长一段时间内我们仍然需要人。我认为对人的需求只会随着他们生产力提高而上升。过去 30、40、50 年的软件工程我们都看到了——不管是编译器、高级编程语言还是 CI/CD，每次你让工程师更有生产力，需求就只是上升。所以我认为我们会需要多得多的工程师。我确实认为短期内对新毕业生来说会稍微难一点，除了最顶尖的那一档。但长期来看我仍然非常看好——计算机科学，其实讽刺的是我自己没学计算机科学——但总体而言我非常看好学计算机科学。如果我的孩子想学，我肯定会鼓励他们。

Not much. I think we're still going to need humans for a long time. I think the demand for humans is just going up the more productive they get. We've seen this for the last 30, 40, 50 years of software engineering — whether it's compilers or high-level programming languages or CI/CD, every time you make engineers more productive, the demand just goes up. And so I think we're going to need a lot more engineers. I do think maybe in the short term it's going to be a little bit harder for new grads, except for the very high end of the spectrum. But in the long run I'm very bullish on still — computer science, actually, ironically I didn't study computer science — but I'm very bullish on studying computer science in general. If my kids wanted to do it, I would definitely encourage them.

到目前为止，你在打造 Lovable 的过程中犯的最大错误是什么？

What's the biggest mistake you've made in building Lovable so far?

我不知道。我觉得我过去犯的一个错误是，有时候你会想按顺序做事。你会说，好，我们先得做这个，这是阻塞项，我们之后再弄那个，因为那个被这个卡住了。但在一个一切都快速变化的世界里，我学到的一件事是，有时候你得同时做两件事。你得假设，哦，我们正在做这件事来解除阻塞——那你不妨同时把另一件事也做了，就为了快速推进。我天生不是特别激进，这有点讽刺，因为我们过去 12 个月增长了 15 倍。我不得不重新训练我的大脑，变得稍微更激进一点，说：这是一个巨大的市场机会，我们需要更——部署、招聘、投入资本，不管是什么——大规模地，就像把所有能调动的都调动起来。

I don't know. I think one mistake I made in the past is that sometimes you kind of want to sequentially do things. You're like, well first we need to do this, this is the blocker, we'll get to this other thing later because this other thing is blocked on this thing. But in a world where everything is moving fast, one of the things I've learned is sometimes you kind of need to do both at the same time. You need to assume that, oh, we're doing this thing to unblock it — you might as well do this other thing at the same time, just in the interest of moving fast. That's something I'm naturally not mega aggressive, which is kind of ironic because we've grown 15x in the last 12 months. I sort of had to retrain my brain to be a little bit more aggressive and saying, this is a massive market opportunity, we need to be even more — deploy, hiring, deploying capital, whatever it is — massively, just like paralyzing everything we can.

瑞典涌现了大量有趣的初创公司。你怎么看背后的原因？瑞典黑手党（Swedish House Mafia）是不是已经被瑞典科技黑手党取代了？

Sweden has a ton of interesting startups that have emerged. What are your thoughts on why that is, and has the Swedish House Mafia been replaced by the Swedish tech mafia?

你知道 Swedish House Mafia 的成员之一是我同学吗？这是个有趣的冷知识。

Did you know that one of the Swedish House Mafia members was my classmate? It's a fun trivia.

真的吗？我不知道。

Really? I did not know that.

是啊。90 年代初我在瑞典长大，我八岁的时候，他说我的自行车很丑。我当时特别难过。嗯，我们在瑞典有很多科技公司——Spotify、Klarna、一堆游戏工作室。最近显然有 Lovable 和 Gora 之类的东西。我对这些事的理论是，成功本身会创造成功。很多人进入科技行业，是因为他们知道某个表亲的朋友在科技行业赚了很多钱之类的事。所以我觉得很大一部分就是一点飞轮效应。我确实认为瑞典作为一个欧洲国家一直相当有创业精神和商业气息。所以我觉得有一点那个味道——我们一直有点，也许追溯到维京人，就是「我们去征服世界」。过去也许是通过音乐和家具之类的东西，但现在越来越多是通过科技。所以有那个因素。我觉得瑞典有一种心态——斯德哥尔摩很有意思，因为斯德哥尔摩每个人都觉得自己是这座超级全球城市，是世界的一部分，而我其实觉得他们并不是。但我觉得他们自认为如此全球化这件事还挺好的。他们有这种愿景，觉得所有人都在乎斯德哥尔摩。但我其实挺喜欢这一点，因为这意味着他们在外面造东西，而且从第一天就说：我们要征服世界。我们要做这个初创公司，它会很牛。

Yeah. Back in the early 90s growing up in Sweden, when I was eight, he told me my bike was ugly. I was very upset. Yeah, I mean, we've had a lot of tech in Sweden — Spotify, Klarna, a bunch of gaming studios. More recently obviously Lovable and the Gora stuff like that. I think my theory about a lot of this stuff is success kind of just creates success. A lot of what makes people go into tech is they know about some cousin's friend who made a lot of money in tech or something like that. So a lot of it I think is just a little bit of a flywheel effect. I do think that Sweden as a European country has always been pretty entrepreneurial and commercial. So I do think there's a little bit of that — we've always been a little sort of, maybe going back to the Vikings, like, let's go conquer the world. In the past, maybe it's been through music and furniture and stuff like that, but now increasingly with tech. So there's that. I think there's sort of a mentality in Sweden — Stockholm is funny because everyone in Stockholm thinks they're this mega global city that's part of the world in a way where I actually think they aren't. But I think the fact that they think they're so global is kind of good. They have this vision that everyone cares about Stockholm. But I actually kind of like that because it means they're out there building things and they're saying from day one, we're going to conquer the world. We're going to do this startup and it's going to be awesome.

我猜这是个不太客气的问题。你觉得 12 个月后 GPU 租赁价格会更高还是更低？

I guess a shameless question. Do you think GPU rental prices 12 months from now will be higher or lower?

我觉得会更高。说来有趣，我其实在过去 48 小时里改变了看法。我告诉你为什么。我读了 Anthropic 的 S-1 文件，其实是让一个智能体帮我总结的。那家公司盈利能力之强有点惊人，超出了我的预期。48 小时前，我的论点是：Anthropic 的毛利率确实很好，但他们在训练上花了太多钱，所以一旦上市，他们就不得不放慢训练和后训练的扩张速度，推理可能会继续增长，但其他部分不会那么快。但后来我意识到，他们基本上已经是一家盈利的公司了。所以他们完全没有动力放慢算力采购。我猜 OpenAI 也有类似的动态。而 Anthropic 和 OpenAI 显然是这个市场上最大的两个算力买家。所以我认为他们不会放慢采购，因为从财务上看他们做得非常好，这意味着需求会大得多，因此价格在未来 12 个月很可能会上涨。

I think higher. You know, it's funny. I actually changed my mind on this in the last 48 hours. And I'll tell you why. I read the Anthropic S-1 and I actually had an agent summarize it for me. But it was kind of incredible how profitable that company is, to a degree that I hadn't anticipated. In my mind, 48 hours ago, my argument was like, oh, Anthropic, they have really good gross margins, but they're spending so much money on training that as soon as they go public they're going to have to slow down how they grow training and post-training, and they're going to maybe grow inference as much but not the other stuff as much. But then I realized they're basically a profitable company. So they have zero incentive to slow down on capacity acquisition. And I imagine there's a similar dynamic going on in OpenAI. And Anthropic and OpenAI have obviously been the biggest two buyers of capacity out there. So the fact that I don't think they're going to slow down buying, because clearly financially they're doing very well, I think it will mean that there's going to be a lot more demand, and I think for that reason prices will probably go up in the next 12 months.

我听你说过，你一直把 Modal 当作你的 20 年项目。现在已经五年了。第 10 到第 20 年会是什么样子？

I've heard you say that you've always thought of Modal as your 20-year project. You're five years in. What do years 10 through 20 look like?

哇。我希望我能招到一批人，替我分担很多事，因为我现在什么都亲力亲为，陷得太深了。我不知道，我就是热爱这些东西，喜欢一直做下去。我觉得第 10 到 15 年，我们会更像一个企业集团，会有很多不同的产品，可能还会有一些疯狂的赌注，比如上月球、上火星之类的。但我仍然希望能深度参与高层的资源分配，决定我们要押注什么，日常事务我大概会少参与一些，比如跟某个 GPU 供应商签合同之类的。但那种高层的资源分配，对我来说是最有意思的，我希望还能再参与 20 年，也许更久。

Wow. I mean, hopefully I'll hire a bunch of people that can do a lot of the sort of, you know, I'm too in the weeds of everything right now. So I don't know, I just love this stuff. I love to keep working on this. I think year 10 to 15 we're going to look more like a conglomerate. We're going to have a lot of different products. We're going to have probably some crazy bets going to the moon and Mars or whatever. But I still hope to be quite involved in the high level, like resource allocation and what are the things we're betting on, and I'll probably be less involved in day-to-day like, you know, let's get a contract from this GPU vendor or whatever. But I think that high level resource allocation and stuff like that, to me that's the most fun, and I hope to stay involved in that for another 20 years, hopefully maybe longer.

是啊。我是说，如果保持现在的节奏，那段时间里至少还能再做五期 Unsupervised Learning，对吧？

Yeah. I mean, if we keep current pacing, at least five more unsupervised learning episodes in that time frame, right?

对，没错。

Yeah. Exactly.

Erik，非常感谢你来参加节目，我们真的很感激。最后请你给听众一个行动号召。他们可以去哪里了解更多关于 Modal 的信息，或者你有什么想推荐他们去看的？

Well, Erik, thank you so much for joining us. We really appreciate it. We'll give you the last call to action for the audience. Where can they go to learn more about Modal, or anything you'd direct them to?

显然应该去 modal.com。也可以在终端里运行 pip install modal，然后玩一玩。也许有很多听众正在创业，或者正考虑创业，我很想看看他们会做出什么酷东西，希望 Modal 能帮到这些创业公司。所以告诉我，随时联系我。顺便说一句，我一直很喜欢和客户聊天。总之，尽管去玩 Modal，期待看到大家会做出什么。

Well, you should obviously go to modal.com. You should also run pip install modal in terminal and play around with it. There are a lot of people maybe listening to the show who are building startups or thinking about building startups, and I'd love to see all the cool stuff they're going to build, and hopefully at Modal we can help you with those startups. So let me know and feel free to reach out. By the way, I always love talking to customers. But feel free to play around with Modal and looking forward to seeing what everyone's going to build.

我是 Jacob Efron，这里是 Unsupervised Learning，一档让我能和 AI 领域最聪明的人对话的播客，我会问他们大量关于模型进展的问题，以及这对世界上的企业意味着什么。希望你能看出来，我做这件事非常开心。这是我在 Redpoint 做投资人本职工作之外，利用晚上和周末做的项目。但我们能请到这些了不起的嘉宾，真的离不开像你这样的听众订阅播客、分享给朋友。归根结底，正是这些让整个节目得以运转。所以，请考虑这么做。非常感谢你的支持和收听。我们下期再见。

I'm Jacob Efron and this has been Unsupervised Learning, a podcast where I get to talk to the smartest people in AI and ask them tons of questions about what's happening with models and what it means for businesses in the world. As I hope is clear, I have a ton of fun doing this. It's a nights and weekends project in addition to my day job as an investor at Redpoint. But our ability to get these incredible guests on really comes from folks like you subscribing to the podcast, sharing it with friends. It's really what ultimately makes this whole thing work. And so, please consider doing that. And thank you so much for your support and listening. We'll see you next episode.