AWS 觉醒前的窗口期：Nebius CTO 谈与超大规模云厂商的竞速

我想先聊聊今天更宏观的 AI 云市场。我觉得市面上有几家规模较大、比较知名的上市“新云”（neo cloud）。然后感觉每天都有新的公司在融资。我知道你不喜欢这个词。但也许你可以谈谈你怎么看差异化——我们有些听众可能在想，真正重要的只是能不能拿到 GPU，哪里能拿到就去哪里。也许可以把这个当作一个靶子，告诉我们为什么事实并非如此。

I guess I'd love to talk just maybe about the broader AI cloud market today. And I think that there's a few maybe larger and prominent public, I guess termed neo clouds. And then it feels like there's a new one getting funded every day. I know you don't like that term. But maybe just talk a little bit about how you think about what differentiates — some of our listeners are probably thinking, all that really matters is just access to GPUs, and wherever you can get them, you go. Maybe use that as a straw man and tell us why that's not the case.

嗯，我显然有偏见，因为首先我是个做软件的人。我们的产品和业务从一开始就建立在“软件是核心”这个理念上，对吧？也就是说，你既能服务那些在一个互联集群里消耗数万块 GPU 的客户，又能用同一套平台服务一个只跑半小时单卡 GPU 的自助客户——这就是我们最初的价值主张。所以我们相信这极其重要。能够大规模建设，同时又有能力几乎像零售一样把它卖出去。我们认为这正是我们与市场上所有人的区别所在。补充一句：新云们非常痛恨这个词，因为最初它是一个区分——指那些在做这种奇怪新事物的云公司。但后来所有人都想进入这个市场。所以我们刚起步时，这个领域大概有五到七家公司被称为新云。现在我觉得已经接近三百到五百家了。

Well, I'm obviously biased because, first of all, I'm a software person. And our product and business was launched on the idea that software is the mode, right? So your ability to serve the customers that consume tens of thousands of GPUs in one interconnected cluster with the same platform as you serve a self-service customer that runs one GPU for half an hour — that was our value prop from the very beginning. So we believe that this is extremely important. Being able to build at large scale, but have the possibility to almost sell it in retail, in a sense. This is what we think definitely differentiates us from everyone on the market. Small remark: neo clouds hate that term with a passion, because it started as a distinction — these are cloud companies that are doing this weird, strange new thing. But then everyone wanted in that market. So when we started there were, you know, five, seven companies in this area that were called neo clouds. Now I think it's closer to 3 to 500.

是啊。

Yeah.

这取决于你愿意把什么叫做新云，取决于规模。我们真的不再喜欢这个被拉伸的伞形术语了。我们是 AI 云。我们不在乎。让别人去做新云吧。我们是 AI 云。这是我们的心智模型。我们不是在和新云市场竞争。我们相信我们已经过了那个阶段。我们的下一个竞争点是挑战 AWS 和 Google Cloud。是的，我们规模小得多。是的，我们的营收现在大概是 AWS 的 1%。但我们跑得很快。我们服务一个非常特定的市场。我们押注这个市场会增长、会爆发。所以人们担心这些投资巨大。人们往基础设施里投了很多钱，等等。我的看法是，如果把今天市场上的情况与两年前相比，我们才刚刚触及表面。我认为未来三年的整体需求会增长十倍、甚至百倍。所以我们正处在又一次爆发式增长的边缘。现在，我们看到这些巨大的设施，对吧？但其中很多是私有拥有的，用于某些公司的内部需求。比如 Elon Musk 在谈他的 Colossus 集群，规模极其庞大。真的，那是一个实打实的大规模设施。但它不是面向更广泛市场的，对吧？它是为了他们的内部需求。也许还会分享给一些他们决定合作的伙伴。

Depending on what you're even ready to call a neo cloud, to the scale. And we really don't like this stretched umbrella term anymore. We are the AI cloud. We don't care. Let others be neo clouds. We are the AI cloud. And this is our mental model. We are not competing with the neo cloud market. We kind of believe we are past that. Our next competition point is taking on AWS and Google Cloud. Yes, we are way smaller. Yes, our revenue is, I think, like 1% of AWS nowadays. But we are running fast. We serve a very particular market. We have a bet that this is the market that is going to grow, that is going to explode. So people are concerned that these investments are huge. People put a lot into the infrastructure, etc. My view, if I compare what we see in the market with what was there two years ago, we are barely scratching the surface. I think the overall demand in the next three years will increase tenfold, hundredfold maybe. So we are at the verge of another explosive growth. Right now, we see these huge facilities, right? But a lot of those are privately owned for internal needs of certain companies. Elon Musk is talking about his Colossus cluster and it's immense scale. Really, it's a real solid large-scale thing. But it's not for the broader market, right? It's for their internal needs. Maybe for some partners that they decide to share with.

是啊。

Yeah.

Anthropic 和 OpenAI 在囤积算力，但那是为了他们自己的产品。这完全是另一个话题——这些公司本质上想成为一切。他们想成为互联网，把一切、每个产品和每个算力角度都装进自己体内。还有 Google、Meta 和 Microsoft 的问题，他们拥有大量算力，但一直在纠结和挣扎：是把它放到市场上出租，还是给内部团队用。因为这些公司都极度依赖机器学习和 AI。他们需要大量算力用于内部使用、运行时和研究。所以他们必须不断权衡。因此我们押注的是，存在一个利基市场，属于一个独立的——即不隶属于拥有巨大内部消耗的大公司——玩家，来服务那些想成为大公司的媒体玩家的市场。

Anthropic and OpenAI are amassing the capacity, but they do it for their own product. And this is a wholly separate discussion — these companies will essentially want to be everything. They will want to be the internet and contain everything, every product and every capacity angle within themselves. There is a question of Google, Meta, and Microsoft, who have a lot of capacity but are constantly juggling and struggling between giving it out to the market to rent or giving it out to internal teams. Because all of these companies are extremely machine learning, AI-enabled. They need a ton of capacity for internal usage, for runtime, and for research. So they have to constantly juggle this. So our bet is that there is a niche for an independent — as in not affiliated with a large company with huge internal consumption — player that will serve the markets of the media players who want to be the large companies.

我觉得对我们的听众来说，聊聊 Nebius 的历史会很有意思。我感觉你们走到今天这一步的路径非常有趣。所以也许可以简单讲讲这家公司是怎么来的。

I thought it would be interesting for our listeners to talk a little bit about the history of Nebius. I feel like it's a really interesting path that you had to end up here. And so maybe just tell a little bit about how the business came to be.

我从我加入的那一刻讲起吧，对吧？我是公司的 CTO，但我从最开始就在这家公司，那时它还不叫 Nebius。新公司是在 2024 年成立的，我想。交易也最终完成了。我们终于宣布与 Yandex 分离。不再与俄罗斯资产有任何关联，一切从头开始，等等。就是这样。

I will start with the moment I joined, right? So I am CTO of the company, but I've been with the company from the very beginning, since before it was called Nebius. So the new company was established, I think, during 2024. Also the deal was finalized. We finally announced our separation from Yandex. No connections to the Russian assets anymore, clean slate, etc. And that was it.

你知道，人们进入这些 AI 云业务的方式各不相同。显然我觉得 Yandex 团队在工程方面是传奇般的强大。

You know, people have come to these AI cloud businesses in different ways. And obviously I felt like the Yandex team was legendarily really, really strong on the engineering side.

你知道很多构建 AI 云的人来自加密货币或能源行业，或者不一定有相同的技术背景。Yandex 的经验在多大程度上真正相关，又在多大程度上是这种情况：结果发现没人知道如何构建 GPU 云，因为这一切都是新的，每个人都在边做边学？

You know a lot of folks building out AI clouds came from like crypto or energy or like not necessarily the same technical heritage. To what extent was the Yandex experience really relevant and to what extent was it like, you know, it turns out no one knows how to build a GPU cloud because it's all kind of new and everyone's learning on the fly?

首先，我们过去和现在都有一个核心团队支持业务的硬件方面，他们来自 Yandex，拥有大规模运营硬件基础设施的经验，真正的规模。数十万、数百万台服务器分布在多个设施中，等等。这些人是我们初始建设和扩张的核心，他们仍然在公司。所以这肯定有帮助。我认为如果没有那个核心团队，我们会远没有那么成功。

First, we had and still have the core team that supports the hardware side of the business that came from Yandex that had experience operating hardware like infrastructure at scale, at real scale. Hundreds of thousands, millions of servers spread over multiple facilities, etc. These people were the core of our initial build-outs and expansion, and they are still with the company. So this definitely helps. I think we would be way less successful if not for that core team.

是的。

Yeah.

在软件方面，这很有趣，因为它是双重的。对于硬件，非常直接。我们需要有经验的人，我们有有经验的人，他们从一开始就开始建设。在软件方面，情况有点奇怪。首先，我们必须从头重写一切。所以我们之前的经验有点混乱。其次，我们必须增长得如此之快，以至于例如到 2024 年底新平台推出时，我们软件工程团队的很大一部分不再来自 Yandex，因为在此期间我们不得不大量招聘。作为一个团队，我们从最初的想法开始就一直在开发这个云平台。所以我们就这样坚持下来，并加倍投入软件。我们不做托管裸金属，我们不做第三方服务的拼凑。我们想要构建一个整体的云体验，一个整体的云平台。我们加倍投入，即使整个平台从头重写。当然，团队中有大量有才华的工程师显然有帮助。就规模而言：你提到来自加密货币或其他途径的人。他们主要集中在硬件上。他们的想法是你需要购买 GPU，你需要它们运行，然后你会找到买家。在我们的案例中，例如，在 2024 年，当我们的硬件团队采购 GPU、进行建设、开发我们自己的服务器和机箱、编写固件、构建网络时——整个团队大约有 40 到 50 人。

On the software side, it's interesting because it's twofold. With hardware, it's very straightforward. We needed people with experience, we had people with experience, they started building out from the get-go. On the software side, it was a weird one. First of all, we had to rewrite everything from scratch. So our previous experience was kind of garbled. Second, we had to grow so fast that, for example, by the end of 2024 when the new platform launched, a significant portion of our software engineering team was not from Yandex anymore because we had to hire extensively during this period. As a team, we were working on this cloud platform from the initial idea. So we kind of just went with it and doubled down on software. We are not doing managed bare metal, we are not doing a patchwork of third-party services. We want to build a holistic cloud experience, a holistic cloud platform. And we doubled down on that even with the whole rewriting of the platform from scratch. And of course, having a ton of talented engineers on the team helped. Just for scale: you mentioned people who came from crypto or other avenues. They are mostly concentrated on hardware. Their idea is you need to buy GPUs, you need them to run, and then you'll find the buyer. In our case, for example, in 2024, when our hardware team that procured GPUs, did the build-outs, worked on our own servers and chassis, wrote firmware, built the network—that whole team was roughly 40 to 50 people.

是的。

Yeah.

软件组织已经有 200 人了。所以我们当然可以认为自己是一家软件优先的公司。我们正在坚实的硬件基础上构建云。

The software organization was already 200. So we can certainly think of ourselves as a software-first company. We are building the cloud on a solid foundation of hardware.

是的。

Yeah.

我们不是一家硬件优先的公司。

We're not a hardware-first company.

完全合理。我想我很想带我们的听众深入了解 Nebulas，我觉得业务中有很多有趣的部分，但也许一个起点是：我想象当你获得新的 Nvidia 芯片时，业务内部肯定有很多事情要做才能让这一切运行起来。所以那看起来像什么，从，我不知道，你和 Nvidia 的人交谈，他们给你关于下一代芯片的早期见解，到实际为一批客户上线这些芯片？

Makes total sense. I guess I'd love to take our listeners inside Nebulas and I feel there's so many interesting parts of the business, but maybe one place to start would just be: I imagine as you're getting new Nvidia chips that come in, there must be a bunch of stuff you have to do within the business to get this all up and running. So what does that look like from, I don't know, you're talking to Nvidia folks and they're giving you early insight into the next generation of chips to actually getting these live for a set of customers?

那是一场混乱。不，我是说，以好的方式。整个市场、整个领域都以这样的速度运行，每个人都在努力追赶。而 Nvidia，尽管处于领先地位，本质上引领和展示这个市场的方向，也不例外。首先，Nvidia 的人很早就会向你展示他们明年想要构建的原型。他们向你展示原理图，内部如何优化，PCI 连接在哪里，那看起来像什么，未来平台会是什么样子。然后过一段时间，他们来找你说，‘哦，我们优化了东西。现在会有所不同。’你说，‘好吧。’然后他们来找你说，‘哦，我们有第一批实验室样品。你想看看吗？’是的，我们当然想看看。‘哦，顺便说一下，这看起来不像你两个月前给我们看的东西。’‘哦，我们改变了东西，因为我们发现……’因为他们时间如此压缩，和其他人一样，他们在这些芯片生产过程中不断迭代。所以不像消费市场，你今天看到的苹果 iPhone 实际上是三年前计划的，最终设计大约一年前就确定了，他们用最后一年来打磨它，构建这个产品。AI 市场以这样的速度运行，他们宣布的任何东西都是指示性的。然后团队不断迭代，而且……

That is a mess. No, I mean, in a good way. This whole market, this whole field is running at such a pace that everyone is trying to catch up. And Nvidia, despite being in the lead, essentially leading and showing the direction for this market, is not the exception. First, Nvidia folks, way before, show you the prototypes of what they want to build in the next year. They show you the schematics, how it will be optimized internally, where your PCI connections are, how that looks, what the future platform would look like. Then sometime later, they come to you and say, 'Oh, we've optimized things. It's now going to be different.' You're like, 'All right.' Then they come to you and say, 'Oh, we have the first lab samples. Do you want to have a look?' Yes, of course we do want to have a look. 'Oh, by the way, this doesn't look like the stuff you showed us 2 months ago.' 'Oh, we've changed stuff because we found...' Because they are so compressed in time, same as everyone else, they are constantly iterating while these chips are being produced. So it's not like in the consumer market where an Apple iPhone that you see today was actually planned 3 years ago and the final design more or less was settled a year ago, and they take the final year to polish it, to build this product. The AI market is running at such a pace that whatever they announce is indicative. And then teams are constantly iterating and...

工作流程在变化，对吧？所以我确信他们随着进展在迭代。

The workflows are changing, right? So I'm sure they're iterating as that goes.

是的，他们在迭代，越来越深入。而 Nvidia 现在是一个庞然大物。他们做的任何事情都会影响整个市场、供应链、一切。所以他们必须考虑这一点。所以在某个时候，你会得到一个样品，它或多或少就像你将在生产中得到的东西。然后你开始构建，你在测试一切。你将对新硬件的支持、新的限制或能力整合到软件平台中。你从一开始就找到隔离工作负载的方法。我们是一个云，我们不提供裸金属主机。所以我们需要拆分，我们需要确保我们的虚拟化工作，我们的存储工作，我们的网络工作。有时我们处于最前沿，所以对于某些代际，我们可能是第一个团队，也许是第一批实际构建了某种规模集群的团队之一。所以我们遇到了一些以前没人见过的软件问题。所以我们不得不向 Nvidia 报告，提出补丁，到达那里，修复它，确保它运行，等等。所以我们觉得自己处于最前沿。但总的来说，这是一个相当正常化的过程。听起来可能很混乱，对吧？有多次迭代，你来回走动，你与 Nvidia 硬件人员、Nvidia 软件人员保持持续联系。

Yes, they're iterating, going further and further. And Nvidia is a behemoth now. Whatever they do influences the whole market, supply chains, everything. So they have to look at that. So at some point, you'll get a sample, and it's more or less like what you'll get in production. Then you start building, you are testing everything. You incorporate support for the new hardware, for the new restrictions or capabilities into the software platform. You figure out a way to isolate workloads from the start. We are a cloud, we are not providing bare metal hosts. So we need to split, we need to make sure that our virtualization works, that our storage works, that our networking works. Sometimes we are on the bleeding edge of this, so for some of the generations, we were probably the first team, maybe one of the first teams that actually built a cluster of a certain size. So we've hit some issues with the software that no one has seen before. So we had to report that back to Nvidia, propose a patch, get there, fix from it, make sure that it runs, etc. So we kind of feel like we are at the forefront of it. But all in all, it's a pretty normalized process. It may sound messy, right? There are multiple iterations, you go back and forth, you're in constant contact with Nvidia hardware folks, with Nvidia software folks.

嗨，我是 Anne Marie，是 Unsupervised Learning 的制作人。非常感谢你收听本节目，如果你喜欢这次对话，请订阅。订阅有助于我们成长，帮助我们继续邀请到一些很棒的嘉宾。非常感谢你的收听，我们下期再见。

Hi, I'm Anne Marie and I'm the producer of Unsupervised Learning. Thank you so much for listening to the show and if you've enjoyed the conversation, please subscribe. Subscribing helps us grow, which helps us continue to lock down some amazing guests. Thank you so much for your listenership and see you next episode.

我觉得有一件事想听听你的看法，就是终端客户最终想要的东西的差异。显然，人们很容易认为，好吧，每个人都只是想来找你，得到类似的东西，但我想你会对客户进行细分，或者思考对不同客户来说什么是重要的。你今天如何看待这些客户需求的关键差异？

One thing I think would be interesting to get your thoughts on is just the differences in what end customers end up wanting. Like, obviously it can be easy to think, okay, everyone just wants to come to you and get similar things, but I imagine you segment your customers or think about differences in what is important to different customers. How do you think about the key differences in these customer desires today?

说实话，并没有那么不同，因为每个客户都像是一朵独特的花，对吧？但与此同时，工作负载原则上并没有那么不同。要么是训练，你大概知道训练是什么样子；要么是推理，你大概知道推理是什么样子。回想几年前，例如，我们有点低估了存储组件。我们认为 GPU 是最重要的东西，它们确实是。但伴随着整个训练角度，你看到对存储的巨大需求。而且不是你能提供给客户的虚拟机上的硬盘数量，而是吞吐量，是延迟。在我之前的职业生涯中，AI 之前的工作负载和基础设施建设中，如果谈到网络附加存储，我们说的是每秒千兆字节，对吧？那是相当的。现在，我们的客户经常达到我们的公共共享，比如一个卷，连接到许多 GPU 服务器，吞吐量达到 5、6、700 GB 每秒。这太极端了。就像两年前，我甚至没想过这个。而现在这或多或少是我们所期望的。同时，如果你针对最高需求，比如最苛刻的客户，你的基线如此之高，你几乎无法服务任何人。

It's not that different, to be honest, because every customer is like its own unique flower, right? But at the same time, workloads in principle are not that different. It's either training and you kind of know how training looks like, or inference and you kind of know how inference looks like. Circling back a couple of years ago, for example, we kind of underestimated the storage component. We thought GPUs are the most important thing and they are. But alongside the whole training angle, you see a massive demand for storage. And it's not for the amount of HDs on VMs that you can provide to the customer. It's about throughput. It's about latency. In my prior life, pre-AI workloads and infrastructure build outs, I was talking about gigabytes per second if we were talking about network attached storage, right? It was like it's equivalent. Right now, we have customers routinely hitting our common share like one volume, right? Connected to a number of GPU servers with like 5, 6, 700 gigabytes per second of throughput. This is extreme. Like 2 years ago, I didn't even think about that. And now this is more or less something we expect. At the same time, if you are targeting these numbers for the highest demand like the most demanding customers, your baseline is so high, you kind of can't serve anyone.

是的。

Yeah.

对吧？所以，如果你遇到这样的用例：客户需要从 3 PB 的共享文件存储中获得每秒 700 GB 的吞吐量，连接到一千台服务器或 8,000 个 GPU，你大概知道如何服务一个庞大的群体。所以，我认为就性能或功能本身而言，客户并没有那么不同。然后你需要另一个子集来了解客户真正想要什么。因为确实有不同的客户，不是他们的规模、大小或工作负载不同，而是他们的成熟度不同。他们对客户的义务是什么，对吧？因为我们可以谈论任何实验室。新实验室现在经常有数十亿美元的种子轮融资，对吧？所以，大部分预算都用于算力。所以，你可以与一个新成立的 Neo 实验室签订十亿美元的合同。他们除了 GPU、存储、性能、可用性之外，不太关心其他任何事情。就这样。他们通常甚至不关心那些 GPU 的位置。他们不关心是在美国、欧洲，比如法国或冰岛。为什么费心呢？互联网连接很好，存储在那里，GPU 正常工作，而我们谈论的是十亿美元的合同。对吧？与此同时，我们现在看到并开始接纳企业客户。这些公司与我们签订的合同会更小，有时会小得多。但他们更成熟。他们有客户，他们对那些客户有义务。他们有流程。他们需要确保遵守多个国家的多项法规。

Right? So, if you hit this use case where the customer needs 700 gigabytes per second from a 3 petabytes of shared file storage connected to a thousand of servers or like 8,000 GPUs, you kind of know how to serve a massive cohort. So, I'd say in terms of performance or features per se, the customers are not that different. Then you need another subset of what the customers really want. Because there are definitely different customers in terms of not their scale, not their size, not their workloads, but rather how mature they are. What are their obligations like for their clients, right? Because we can talk about any labs. New labs routinely have seed rounds of in the billions nowadays, right? So, most of that budget goes into compute. So, you can have a contract for a billion dollars with a newly founded Neo lab. Who like they don't care that much about anything but GPU, storage, performance, availability. This is it. They often times don't even care about the location of those GPUs. They don't care if that's US, Europe, like France or Iceland. Why bother? Internet connection is good, storage is there, GPUs are working as a and we are talking about a billion-dollar contract. Right? At the same time, we now see and start onboarding enterprise customers. These are companies whose contract with us will be smaller. Sometimes significantly smaller. But they are more established. They have customers and they have obligations for those customers. They have procedures. They need to make sure they're compliant to a number of regulations in multiple countries.

是的。

Yeah.

所以，现在我们进入了成为真正云服务的整个角度，你知道？比如认证、合规、安全态势检查，让我们看看你的源代码。你如何解决问题 X？你如何保护你的客户免受攻击向量 Y？等等等等。我认为，如果你谈论差异化客户，这是最具差异化的东西，而不是你从规模或问题中所期望的。

So, now we are getting into that whole angle of being the real cloud, you know? Like certifications, compliance, security posture checks, let us look at your source code. How do you solve problem X? How do you protect your customers from attack vector Y? Etc. etc. etc. And I'd say if you're talking about differentiated customers, this is the most differentiating thing as opposed to what you would expect from the size or the problems.

你知道，我见过很多人思考吞吐量与延迟之间的权衡，或者延迟与成本之间的权衡。我想你有一些客户的工作负载对延迟超级敏感，他们会做小批量或任何必要的事情来达到那个延迟。然后你还有其他对延迟不太敏感的客户，也许是后台智能体之类的。这一切都只是在软件层面决定的吗？我猜如果你有那些不同的客户，你如何确定运行这些工作负载的批量大小，或者你实际上会根据那些工作负载以任何不同的方式构建集群吗？

You know, I've seen a lot of people thinking through this trade-off of throughput latency or a trade-off of latency and cost. And I imagine you have some customers that have workloads that are super latency-sensitive and they'll do small batches or whatever is required to get to that latency. And then you have other customers that are much less latency-sensitive, maybe background agents or something. Is that all just determined on the software side? I guess if you have those different customers, how do you determine the batch size that you're running these workloads in, or do you actually construct the clusters in any different ways based on those workloads?

不。不。不。不。这是从一开始的前提。我们正在构建通用云平台，我们需要确保我们的基线足够高，以适应大多数工作负载。

No. No. No. No. This was the premise from the very beginning. We are building the universal cloud platform, and we need to make sure that our baseline is high enough to fit most of the workloads.

是的。

Yeah.

然后，还有 SLA 的玩法，对吧？所以，客户来了，例如，对于托管推理业务部分，客户有不同的需求，然后你就进入了这种基准测试游戏，对吧？他们希望基于这种特定流量模式的特定足迹符合 SLA 在延迟方面的要求，或者比如首词元时间延迟，或整个请求延迟。你开始为此优化。但是，这从来不是在硬件层面完成的，因为硬件已经足够好了。这很少在较低的基础设施层面完成，因为它是通用的，而且再次，足够好了。不妥协性能，等等等等。

Then, there is the SLA play, right? So, customers come with, for example, for the managed inference part of the business, customers come with different demands, and then you are in this kind of a benchmarks game, right? They want this particular footprint based on this particular traffic pattern fits this SLAs in terms of latency, or like, latency as time to first token, or the whole request latency. And you start optimizing for that. But, this is never done on the hardware level, because hardware is good enough. This is rarely done on the lower infrastructure level, because it's universal, and again, good enough. No compromise for performance, etc., etc.

你还在往编排层、模型层走得更远。沿着整个技术栈往上，一直到客户的请求（比如端点）究竟如何到达推理服务器，以及沿途有哪些性能损耗。

And you are going further away to the orchestration level, to the model level. Way further up the stack, all the way to how exactly does the request from the customer, like the endpoint, gets to the inference servers, and what are your performance hits along the way.

是的，有道理。

Yeah. That makes sense.

对，所以这几乎 100% 是软件问题。

Yeah. So, this is like 100% software.

我和很多人聊过，感觉现在拆分预填充和解码（disagg）这个话题很热，很多人都在讨论。你们现在会考虑这个吗？或者至少，你们未来会怎么考虑？

I guess a lot of people I've talked to, you know, feels like this disagg, you know, splitting prefill and decode is very in the air these days, and a lot of people talking about that. Like, is that something you guys think about now, or at least, how do you think about that kind of in the future?

我认为我们正处于一个人人都在谈论、但没人能正确实现的阶段。因为这是一个复杂的问题。尤其是在推理中，你还有很多其他问题，比如很多更容易摘到的低垂果实，你可以修复、解决并达到那些 SLA。所以，我们对此相当谨慎。例如，我们并没有把它作为每个客户、每个工作负载的默认解决方案，因为首先它并不合理。其次，它真的很难操作和维护，要围绕它提供适当的 SLA 在今天看来相当困难。我们在原型服务中取得了一些有希望的结果，但今天还没有。同时，我们现在为最苛刻的客户在生产中部署了一些拆分式设置。这只是一个更广泛的手动方法。

I think we are in the stage where everyone's talking about that, and no one's doing that properly. So, because like, it is a complex problem to solve. And particularly in inference, you have a lot of other problems, like way lower hanging fruits that you can fix, address, and hit those SLAs. So, we are quite cautious about that. We don't have it, for example, as a go-to solution for every customer for every workload because like first it doesn't make sense. Second, it's really hard to operate and work with like provides proper SLAs around that is like proving quite difficult as of today. We have some promising results in our like prototype services but not as of today. At the same time, we have some like disaggregated setups for the most demanding customers in production right now. It's just a more you know, wide glove manual approach.

是的。

Yeah.

但尤其是对于这些更新的架构，比如你知道的 Grace Blackwell，整个节点有非常快的互连，如果你能把它装进一个机架的 footprint 里，以前拆分式设置遇到的很多挑战就迎刃而解了，因为吞吐量快得多，互连快得多，你就不再那么在意了，这只是一种平衡配置文件和代码节点以匹配你从客户那里看到的特定模式的方式。

But especially for these newer architectures of like you know, those Grace Blackwalls, whole node that has really fast interconnect, a lot of challenges that you've had before with disaggregated setups are kind of solved if you're able to fit them in the footprint of a rack because like the throughput is so much faster, like interconnect is so much faster, you kind of stop caring and that this is a way of just balancing the profile and the code nodes to match the particular pattern that you see from the customer.

对于那些确实能装进一个机架的工作负载，你的心智模型是什么？

What's your mental model for like the workloads that do fit in a rack?

我不知道。这非常因情况而异。嗯，这些绝对是显著的工作负载，比如我们说的是每秒数千个请求，对吧？我们说的是数十亿个 token 流过。但同时，因为我们是云，我们可以例如在两个工作负载之间共享机架，而不必太担心，对吧？所以，例如在云方面，对于这个特定产品，我们只允许以机架级别消费。你不能订购两个 Grace Blackwell 节点。那没有意义。

I don't know. It's a very case-by-case. Well, these are definitely significant workloads like we are talking like thousands of requests per second, right? We are talking like billions of tokens that flow through. But at the same time, because we are the cloud, we can, for example, like share the rack between two workloads and not be that bothered about that, right? So, for example, on the cloud side, we for this particular product, we only allow to consume it like a track level. You cannot order two Grace Blackwalls nodes. It doesn't make sense.

是的。

Yeah.

对吧？就像你必须分配整个机架，然后在其中组织工作负载。但例如，我们只是作为这种解决方案的客户，就像我们做托管推理一样。你可以共置工作负载，你可以找到方法做到这一点。所以，从某种意义上说，你不需要把任何东西装进机架。只是有没有工作负载会从这种不平衡中受益，比如有意让预填充和代码节点不平衡。

Right? Like you'll have to allocate the whole rack and then like structure workloads within it. But for example, we only were a client of such a solution as we do with managed inference, for example. You can collocate workloads, and you can find a way to do that. So, in a sense, you don't need to fit anything in the rack. It's just is there a workload that will benefit from this disbalance, like in intentional disbalance of prefilling the code nodes.

那你能在一个机架里装不止一个吗？你们如何解决打包问题？

And can you fit more than one into a single rack, and how do you solve the packing problem?

是的，有道理。

Yeah, that makes sense.

你提到在推理方面有更容易摘的低垂果实，也许比做那个更简单。今天你认为我们作为一个行业还没有优化的一些低垂果实是什么？

You kind of mentioned there's like lower hanging fruit on the inference side, you know, that than than doing that maybe. What are like some of the lower hanging fruits today that you still think we're not as an industry optimized on?

嗯，首先，我认为作为一个行业，我们还没有优化。所以，就像你知道的，请原谅我的粗话，但从某种意义上说，我坚信运行推理，特别是当今可用的各种开放权重、开源类模型，并不是一个已解决的问题。比如，如果你只是按照发布时的样子运行，就像一些开源推理服务器中的初始支持那样，用新颖的模型运行，你的结果可能会令人失望。它没有那么快，也没有那么便宜，尤其是考虑到当前基础设施的价格。更容易的是去使用，比如我不知道，Fable 5.1、Astra 6 等等。但是，由于开放架构，由于我们今天所知的一切，你可以优化处理这些请求的方式。你可以优化推理服务器的配置方式。你在内核优化上投入大量精力，有时基于你在流量中看到的特定模式进行内核过度优化。对吧？在模型权重优化方面有很多未被探索的潜力。我不仅仅是在说像推测解码之类的东西。我说的是模型剪枝、选择性剪枝、压缩。因为我相信，而且我们看到，在我们的客户群中，95% 的客户不运行原版开源模型。这些是修改过的、微调过的等等。这意味着这些模型开始解决客户拥有的特定用例。特定用例通常意味着有限领域、范围有限的领域。范围有限的领域通常意味着你需要为特定子集任务保留前沿智能和推理能力，而你不关心其余部分。现在我们在这方面有所进展，也许你不再需要那么大的模型了。也许你只需要一个经过选择性压缩的模型，它在特定任务上有效，而在其余任务上不那么好。当然，这不再适用于开放式深度研究或广泛的编码任务。但我坚信，对于真正的 AI 采用，当我们谈论严肃的企业业务时，当我们谈论真正解决问题的实用 AI 时，我们越来越进入这些专业化的智能体、有限领域，你甚至不再关心成本。你更关心结果的可预测性，你关心时间，即获得结果所需的真实秒数和分钟数。

Well, first of all, I think as an industry we didn't optimize. So, like, you know, pardon my French, but in a sense, I strongly believe that running inference, particularly in the like variety of open weight for open source-ish models that are available today, is not a solved problem. Like, if you take something just to run with it as it's published like in the initial support in some of the open source inference servers, run with the novel models, your results will probably be underwhelming. It's not that fast, it's not that cheap, especially with like current prices on the infrastructure. It's easier to go and use like uh I don't know, Fable 5.1, Astra 6, etc., etc., etc. But, because of the open architecture, because like uh of everything that we know as of today, you can optimize the way you process those requests. You can optimize the way the inference server is configured. You put a lot of effort in kernel optimizations, sometimes kernel over optimizations based like on the particular pattern that you see in the traffic. Right? You There is a lot of underexplored potential in model weights optimization. And I'm not just talking about like stuff like speculative decoding, for example. I'm talking about like model pruning, selective pruning, compression. Uh because I believe that uh And I And I And we see that uh within our customer base, 95% of our customers do not run vanilla open-source models. These are modified, fine-tuned, etc., etc., etc. It means uh that these are the models that uh uh start to address a particular use case that the customer has. Particular use case usually means limited domain, scoped domain. Scoped domain usually means that you need to retain like frontier intelligence and reasoning for a particular subset of tasks, and you don't care about the rest. And now we are getting somewhere in terms of like maybe you don't need such a large model anymore. Maybe you need just like uh model that is selectively compressed in a way where it works for a particular task, and it doesn't work as good for the rest of it. Now, of course, that doesn't work with like open-ended, you know, deep research or uh broad coding tasks anymore. But I strongly believe that for that for real AI adoption, when we are talking about serious enterprise business, when we are talking about like practical AI that actually solves problems, we are getting more and more into these specialized agents, limited domains, where you even stop caring about the costs. You care more about predictability of the outcome, and you care about time as in like real seconds and minutes till we get that outcome.

是的。

Yeah.

我认为那里有很多潜力，我们仍然会看到一些不错的突破。

And I think there is a lot of potential there, and we still will we will still see a couple of nice breakthroughs there.

围绕推理优化有大量工作，其中很多是在开源中完成的，对吧？或者像你知道的，在整个社区共享的贡献。显然，有很多关于更小的专用模型和压缩以及一堆这类事情的工作。

There's a ton of work around inference optimization, and a lot of that gets done in open source, right? Or like, you know, contributions that share throughout the community. There's obviously a bunch of work on like smaller purpose-built models and compaction and a bunch of these things.

在所有这些影响企业使用推理的问题中，哪些是 Nvidia 应该提供解决方案或产品的，哪些是你觉得‘嘿，这很好，我们会受益，但我们不需要站在最前沿’的？

Across all these problems, which do impact how enterprises consume inference, which ones should Nvidia have a solution or offer a product around, and which ones are you like, 'Hey, it's great that that exists and we'll benefit from that, but we don't need to be at the forefront.'

要站在一切的最前沿。在某种意义上，这些是相加的，甚至可能是相乘的，对吧？如果你针对特定用例引入了正确的优化子集，这意味着你可以更容易、更好、更快、更可预测地达到目标。所以，我们在所有途径上投资。比如我们在夏初的一次收购就是专门针对这个的。我们基本上是为了模型优化、模型压缩、推测解码等方面的人才而收购了那家公司。结合现有的人才，我们有一个两倍大小的团队。我们有严肃的人力，有大量的人在研究这些问题。我们有专门的内核优化小组，专门的模型压缩问题小组。因为我觉得如果你想在严肃的层面上谈论推理即服务，这是如今唯一的方法。

To be at the forefront of everything. In a sense, these are additive and even maybe multiplicative, right? Like if you have a right subset of optimizations that you introduce for a particular use case, it means that you can reach the targets easier, better, faster, in a more predictable fashion. So, we invest in all avenues. Like one of our acquisitions back in the beginning of summer was targeting specifically that. Like, we basically bought the company for the talent in model optimization, model compression, speculative decoding, etc., etc. Combined with existing talent, we have a team that is 2x the size. And we have serious manpower, as in tons and tons of people who are working on these problems. We have specialized groups for kernel optimization, specialized groups for model compression problems. Because I feel if you want to talk about running inference as a service at a serious level, this is the only way you can do it nowadays.

沿着这些思路，你们进行了一些收购，其中引起一些人注意的是网络搜索领域的 Tavily。也许可以谈谈背后的理由，以及你如何考虑将其融入更广泛的平台。

And I guess along those lines you guys have done a few acquisitions, and one of them that caught a few people's eyes was Tavily in the web search space. Maybe talk a little bit about the rationale for that, how you think about that fitting into the broader platform.

首先，我们对他们的产品和市场推广策略印象深刻。那几乎是快一年前的事了，对吧？但我们觉得我们知道如何用我们的专业知识、技术和底层的东西帮助他们跑得更快。因为实际上运行一个网络搜索引擎并不是一件简单的事。从软件开发的角度，也不是从……

So, first we were very impressed by the product and their go-to-market motion. It was almost a year ago now, right? But we felt like we knew how to help them run even faster with our expertise and technology and low-level stuff. Because actually running a web search engine is not a very simple task. Not from the software development perspective nor from the...

因为你们很多人曾经运行过一个。

Cuz a lot of you used to run one.

我们中有些人在前世也涉足过一点。是的，对吧？所以，这就是前提。我们有一个伟大的团队，在市场渗透和一般市场推广方面跑得很快。我们知道如何用我们在底层东西和运行大规模基础设施（例如大规模网络索引）方面的专业知识使其互补。它并不完全嵌入到我们今天现有的其他产品中。我们正在探索，并且有一些正在酝酿的东西，用于托管推理与智能体工具的结合，因为例如，我们现在也开发沙盒，对吧？所以，这是构建智能体的另一个构建块。在这方面，我们还有几个其他产品处于早期开发阶段。

And some of us dabbled a little bit in our previous lives. Yes. Right? So, this was the premise. Like, we have a great team that is running real fast in terms of market penetration and in general go-to-market motion. And we know how to make it complementary with our expertise on the low-level stuff and running large-scale infrastructure and for example large-scale web index. It's not exactly embedded into the rest of the offerings that we have now today. We are exploring and we have something cooking for the managed inference combined with agentic tools because for example, we also develop sandboxes nowadays, right? So, this is another building block for building agents. We have several other products that are in the earlier stages of development in that regard, too.

是的，我的意思是，显然有一组工具人们会用来构建智能体和基于开源构建，看起来你想拥有一堆这样的工具提供给最终客户。而网络搜索是任何长期运行的智能体的关键原语。仅拉取信息就会消耗相当数量的 token。所以，是的。然后我想换个话题，谈谈更广泛的行业建设……我认为人们有很多问题在问。我想一个起点可能对我们的听众来说很有趣，那就是你们合作的数据中心在多大程度上是相似的，就像有一个参考设计，它们都……我不知道，对你来说哪个是哪个并不重要。以及它们之间有哪些差异实际上会影响在其上运行 AI 云的业务。

Yeah, I mean, it seems like there's obviously a set of tools that folks are going to use to build agents and build on open source and it seems like you want to have a bunch of those tools that you will offer to your end customers. And web search kind of being a key primitive for any long-running agent. We'll consume a decent amount of tokens on just pulling information in. So, yeah. And then I'd love to switch gears to just like the broader industry buildout and kind of the... I think there's lots of questions folks are asking about that. I figured one place to start that actually might just be interesting for our listeners is to what extent are the data centers that you work with similar and like there's a reference design and they all kind of like... I don't know, to you it doesn't really matter which one's which. And like what are some of the differences in them that actually impact the business of running an AI cloud on top of them.

嗯，坦白说，我不知道设施墙壁是什么材料做的。所以它们可能不同，但那没什么帮助。说正经的，墙内的一切是我们完全控制的。我们不租用数据中心，比如有人自己建了网络或带来了一些 GPU 或服务器等。如果我们不拥有设施而租用它，例如我们租用建筑、电力，但仅此而已。所有内部建设都由我们控制。按照我们的标准完成。所以，从软件方面来说，我们运行哪个数据中心实际上没有区别。是我们自己的设施？是在租用设施内建造的？或者，例如我们最近宣布的所谓轻资产模式，第三方带来资本来建立设施和 GPU，我们在其上运行我们的平台。结果是一样的。这些数据中心按照我们的规格建造，按照我们的规格装修。硬件是我们熟悉的，因为未知硬件很难采用并确保提供适当的 SLA。就这样。所以我们不在那里妥协。我们从不从别人那里租东西并把它当作我们自己的云来呈现。如果你在 Nebius 云中启动一个虚拟机，你可以确信它从上到下运行在我们的技术栈上。

Well, to be frank with you, I have no idea what the walls of the facility are made of. So, they're probably different, but that doesn't really help. In all seriousness, what's within the walls is something that we fully control. We do not rent data centers when someone like have built the network on their own or brought some GPUs or servers, etc. If we don't own the facility and we rent it, for example, we rent the building, electricity, but that's it. All the internal build-out is controlled by us. It's done according to our standards. So, from the software side of things, there is literally no difference which data center we're running. Is it our own facility? Is it something that is built within a rented facility? Or, for example, we've recently announced the so-called asset-light model when a third party brings capital in order to set up the facility and GPUs, and we run our platform on top of that. It's going to be the same. These data centers are built according to our spec, fitted out according to our spec. Hardware is something that is familiar to us because unknown hardware is quite hard to adopt and make sure that you provide proper SLAs. And that's it. So, we don't compromise there. We never rent something from people and present it as our own cloud. If you launch a virtual machine in Nebius cloud, you can be sure that it runs on our stack top to bottom.

是的，有道理。但我想今天有很多关于 AI 建设瓶颈的对话，人们谈论各种不同的瓶颈，无论是电力还是实际制造数据中心的劳动力、冷却、网络、HBM、GPU 整体。比如，如果你要对所有瓶颈进行排序，你认为关键瓶颈是什么，哪几个现在感觉最突出？

Yeah, makes sense. But, I guess there's a lot of dialogue around the bottlenecks to the AI build-out today, and I think people talk about a bunch of different ones, whether it's power or the labor to actually make data centers, cooling, networking, HBM, GPUs as overall. Like, how do you think about what the key bottlenecks are there if you were to rank across all of them the few that feel most poignant right now?

很抱歉让你失望。只有一个瓶颈，那就是你的执行能力。因为是的，所有这些都是问题。市场上有 GPU 短缺，组件短缺。如今市场上有 CPU 短缺。市场上有 NVMe 短缺。没有足够的设施，没有足够的电力，没有足够的合格劳动力来帮助你建设。没有足够的小组件，例如网络收发器。还有什么不丰富？哦，你知道，那些大规模的……我不确定我知道正确的英语术语，但你知道那些实际转换电力的机器，比如变压器。

I'm sorry to disappoint. There is one single bottleneck and that's your ability to execute. Because yes, all of those are issues. Like there are GPU shortages in the markets, component shortages in the market. Nowadays CPUs shortages in the market. Like NVMe shortages in the market. There are not enough facilities, not enough electricity, not enough qualified workforce to help you build it out. Not enough small components like network transceivers for example. What else is not in abundance? Oh, you know, like those large scale... I'm not sure I know the proper English terminology but like you know those machines that actually convert electricity and place like transformators.

变压器。

The transformers.

变压器。是的，你知道现在那些的交货时间吗？哦，几年。对吧？所以一切都有问题。

Transformers. Yeah, like do you know the lead times on those these days? Ooh, years. Right? So like everything is a problem.

我认为唯一的瓶颈，同时也是你唯一能施加的模式，就是你围绕所有这些问题和关切执行的能力。你的采购能力，你在适当的地方走捷径的能力。你整体的执行能力。你在设施之间调动资源的灵活性和能力。好，我们把这个从这里移到那里，然后再移回来。是的，我们会推迟这个，但这意味着我们能赶上这个。这很重要，等等等等。所以我认为在这个特定领域，有经验的项目经理可能是我整个职业生涯中见过的需求最高的。

And I think the single bottleneck and at the same time the single mode that you can impose on that is your ability to execute around all these problems and concerns. Your ability to source, your ability to cut corners where appropriate. Your ability to execute in general. Your ability and flexibility to move stuff between facilities. Okay, we're going to move this one from here to there and then we'll move it back. And yes, we'll postpone this one but this means we'll be in time with this one. This is important, etc. etc. So I think that in this particular field an experienced project manager is probably in highest demand I've ever seen throughout my career.

是的，我猜瓶颈是真正优秀的产品经理，这

Yeah, that's I guess the bottleneck is really good product managers, which

是的，没错。就像你整体上能够在脑中掌握各处问题的大局并驾驭它，这变得最为重要。

Yeah, exactly. Like your overall ability to have this large context of problems everywhere in mind and navigate that becomes the most.

是的。这很有趣，因为显然硬件本身在变化。你有一堆工作负载在变化。显然，我们有更多长时运行的智能体。我们有更多的后训练在进行。你知道，人们在运行开源模型。这实际上如何影响集群本身的整体构建方式？在多大程度上是“嘿，工作负载就是工作负载，我们可以运行它”？

Yeah. It's interesting cuz you obviously have the hardware itself changing. You have a bunch of workloads changing. Obviously, we have much more of long-run agents. We have more post-training going on. You know, people running open source models. How does that actually impact the way that the overall build-out of the clusters themselves? To what extent is it like, "Hey, you know, a workload's a workload and we can run it."

不，工作负载就是工作负载。就像你说的。它略有不同，但某种意义上，你需要足够高的基线才不会被它限制。然后剩下的就是你的优化问题。你打包的方式，你拆分的方式，对吧？所以，对于某些工作负载，你不需要整个节点。所以，你需要共置工作负载。你需要有正确的原语。你需要通过平台暴露这些，等等等等。但某种意义上，差异化的工作负载几乎从不会对你在最低层的构建方式施加任何改变。是的，当然，如今有了这些新架构，比如叫什么？Nvidia LPX，像是 Groq 交易的产物。对吧？这增加了复杂性。但某种意义上，如果你有运行其他代际的经验，这不过是又一个 0.5 代，对吧？是的，这会有点新颖，与之前相比连接方式有点奇怪，并且对你能运行和不能运行的内容有一些限制。但效率提升。这只是平台的一个小众本地补充，对你向更广泛市场提供的东西。但总的来说，不。你预期的定制解决方案要少得多。工作负载就是工作负载。

No, the workload's a workload. Like exactly like you said. It is slightly different, but in a sense, you need a high enough baseline to not be constrained by that. And then the rest is just your optimization problem. The way you pack things, the way you split things, right? So, you don't need whole nodes for some of the workloads. So, you need to collocate workloads. You need to have the right primitives. You need to expose those through the platform, etc. etc. But in a sense, differentiated workloads almost never impose any changes on how you build at the lowest level. Yes, of course, nowadays with these new architectures like what is it called? The Nvidia LPX like the outcome of the Groq deal. Right? This adds to it. But in a sense, if you have experience running other generations, this is just another 0.5 generation, right? Yes, this will be a little bit novel and it's connected in a weird way compared to the previous ones and there are some restrictions on what you can run and not run on that. But efficiency gains. This is just a niche local addition to the platform to what you offer to the broader market. But in general, no. There are way less custom solutions that you would expect from that. The workload is a workload.

为什么超大规模云厂商没有更快地进入这个市场，或者至少在第一章早期更有说服力地赢得这个市场？

Why didn't the hyperscalers move into this market faster or win this market more convincingly early on in this first chapter at least?

首先，有时有一片海洋他们甚至没有尝试。超大规模云厂商从这个市场获得了大量业务。比如巨额交易。他们非常有竞争力。有时人们说，“哦，你必须看看 AWS 的价格。”嗯，这些是公开价格。我向你保证，对于合适的交易，他们给出合适的价格，他们非常有竞争力，他们非常了解市场。我认为某种意义上，有两个角度。首先，就像我之前提到的，很多这些公司需要平衡内部消耗和可以投放广泛市场的部分。而且我们处于芯片短缺的世界，对吧？所以，如果 Nvidia 能生产出 10 倍的芯片，可能整个市场的形态会不同。因为像微软和谷歌这样的公司会既为自己购买容量，也为更广泛的市场运营购买容量。这可能只是没有足够的容量满足他们的规模，对吧？因为没有足够的容量满足他们的规模，这对他们来说是一个相对较小的业务。AWS 收入大概 1000 亿左右？我的意思是，一个二三十亿的业务，绝不是无足轻重的，但与他们保守的大规模成熟业务相比，也不那么容易。

First of all, sometimes there is an ocean that they didn't even try. There is a solid amount of business that hyperscalers get out of this market. Like huge deals. They are very competitive. Sometimes people say, "Oh, you have to see AWS prices." Well, these are public prices. I assure you for proper deals they are doing proper prices and they are very competitive and they are very aware of the market. I think in a sense there are two angles to that. First, like what I've mentioned before, a lot of these companies need to juggle their internal consumption and what they can put for the broad market. And there is like we are in chip shortage world, right? So, if Nvidia could have produced like 10x the chips, probably this whole market would have been shaped differently. Because they would like Microsofts and Googles of the world would both buy capacity for themselves and for the broader market to operate. This is probably simply not enough capacity for their scale, right? Because there is not enough capacity for their scale, this is a relatively small business for them. AWS revenue is what like 100-billion or something? I mean like a business for like two, three, four billion, it's not insignificant by any margin, but compared to what they're conservative large-scale established businesses, it's not that easy, either.

是的。但你的观点基本上是，他们获得的芯片分配没有他们想要的那么多。他们必须将其中一部分，很大一部分用于自己训练。考虑到所有这些，推理业务或云业务只能做到那么大。是的，弄清楚如何优先排序很有趣。你是追求少数拥有大合同的客户，还是尝试获取中端企业、新兴细分市场？但我想，如果你只有一定数量的芯片，无论哪种方式都有点注定要失败，

Yeah. But, your point is basically like they get a chip allocation that is not as large as they'd want. They have to use some of that, a large amount of that for training themselves. There's only so big that inference business or that cloud business can be given all that. And yeah, it's interesting to figure out how to prioritize. Do you go after the few folks with large contracts or do you try and get this mid-market enterprise, up-and-coming segment? But, I guess you're kind of doomed either way if you only have a certain number of chips to

是的，没错。然后同时，这成为你收入组合中相对较小的一部分，你会想，我甚至需要为此分配资源吗？这是一个特别紧迫的问题，因为对于超大规模云厂商来说，这不仅仅是销售问题。这也是改变他们为其他类型工作负载大规模构建的基础设施的问题。网络、吞吐性能、存储性能，顺便说一下，对吧，我之前提到过。所以，如果你想要在整个技术栈中普遍实现这一点，这需要他们进行大规模投资，而投资需要由潜在业务的规模来证明合理性。他们还没有做到。我认为如果我们开始摆脱芯片短缺，我不知道，两、三、五年后，也许，我们肯定会看到这种转变。他们会更加关注，因为这将成为越来越大的业务块。他们会开始投资，他们会重组和重新设计内部结构以更适应这些需求，等等等等。然后凭借他们全部的销售力量，他们会去尝试赢得这个市场。问题是像我们这样的新玩家，例如，是否能够足够快地成长，在他们到达时形成重要的竞争。

Yes, exactly. And then at the same time, this becomes a relatively small part of your revenue portfolio and you're like, do I even need to allocate resources to that? And that is especially a pressing question because for the hyperscalers, it's not just a matter of sales. It's also a matter of changing their infrastructure that was built at scale for other kinds of workloads. The networking, the throughput performance, the storage performance, by the way, right, that I've mentioned earlier. So, this requires, if you want to have it universally implemented through the whole stack, this is a massive investment on their part that needs to be justified by the size of a potential business. And it's not that they're yet. I think if we start getting out of this chip shortage in, I don't know, two, three, five years, maybe, we will definitely see this shift. And they will pay more attention because this will become an increasingly large chunk of business. And they will start investing, they will reorganize and redesign their internals to be more accommodating to these demands, etc. etc. And then with their whole sales power, they'll go and try to win this market. And the question is whether the novel players like us, for example, will be fast enough to grow and represent significant competition when they get there.

是的。

Yeah.

我 100% 确定他们未来会更加关注。

And I'm 100% sure that they will pay more attention in the future.

是的。

Yeah.

我们的赌注是我们会足够快，与他们同处一个联盟并平等竞争，而不是一个小玩家试图追随他们的足迹。

Our bet is that we'll be fast enough to be in the league with them and fight as equals rather than a small player trying to run in their trails.

有道理。

Makes sense.

我的意思是，你觉得会发生什么？你提到有大约 300 到 500 家新云公司，每个人都想进入这个市场。就像你提到的，你认为芯片短缺会在某个时候结束。如果你把时间拉远到 5 年、10 年后，这些企业中有多少还会存在？或者这个市场的最终状态会是什么样？

I mean, what do you think happens to like you mentioned there's like these like, you know, 300 to 500 like, you know, neo clouds everyone trying to get into this market. Like as you kind of, you know, you mentioned you kind of think like the chip shortage comes to an end at some point. Like as you zoom 5 years, 10 years in the future, like how many of these businesses are still around or like what is the end state of of of this market?

听着，我认为整个 AI 消费都会增长。它会被普遍使用，就像 AI 无处不在。现在 Web 服务无处不在，AI 服务也会无处不在。所以从某种意义上说，你有 AWS，对吧？但你也有 Hetzner，也有 DigitalOcean，也有 Linode，还有成千上万的本地托管提供商，它们提供虚拟机、裸金属等。所以这个市场会非常广阔。当前这 300 到 500 家公司都能在未来 5 年存活吗？我不确定。但我很确定，未来会有非常长尾的提供商的一席之地。一些现有玩家会整合，一些会消亡，新玩家会出现，等等。现在这不是一个糟糕的市场，也不是一个糟糕的生意。所以它会不断吸引资本、关注和投资，等等。我相信 5 年后这个市场仍然会有数百个玩家，他们找到了正确的投资，获得了数亿美元，并且能够向市场中的某些人提供一些东西。就像现在的小规模云如 DigitalOcean 一样，对吧？DigitalOcean 并没有因为 AWS 的增长而消亡。

Listen, I think the whole AI consumption is going to grow. And it will be universally used everywhere like AI is everywhere. Like right now web services are everywhere, AI services will be everywhere. So in a sense, you have AWS, right? But you also have Hetzner. You also have DigitalOcean. You also have Linode. You also have tons and tons and tons of local hosting providers that are giving you virtual machines, bare metals, etc. So this market is going to be vast. Will all of the current 300 to 500 companies survive the next 5 years? I'm not sure. But I am pretty sure that there will be a place for the very long tail of providers, down the line. Some of the current players will consolidate, some of them will die off, new players will be launched, etc. It's not a bad market right now. It's not a bad business to be in. So it will constantly attract capital and attention and investments, etc. And I trust that in 5 years time there's still going to be hundreds of players in this market who found the investment into the right ones and several hundred million dollars and they can offer something to someone in the market. Same as it is with the small scale clouds like DigitalOcean these days, right? DigitalOcean did not die because AWS is growing.

他们显然在内部做了很多事情，比如沙箱、网络搜索，以及随着时间的推移帮助人们使用他们的模型。我认为很多人都在问的一个问题是，AI 基础设施栈的哪些部分最终会被这些未来的 AI 超大规模厂商吞并，并整合成一个综合产品，然后哪些独立的 AI 基础设施部分会作为公司持续存在？

They're obviously doing a lot of things in-house like sandboxes and you know, web search and you know, helping people with their models over time and you know, I think one question a lot of people are asking is you know, what parts of the AI infra stack end up just being subsumed into these future AI hyperscalers and like live, you know, in just one combined offering and then what are like the standalone pieces of AI infrastructure that will persist as companies?

从某种意义上说，我认为它会模仿传统一代云、上一代云所发生的情况，对吧？当你的增长要求你开始涉足新的领域、新的市场等，否则你的增长就会受限，你无法展示你的预测，无法证明估值的合理性，等等。所以，大公司会有一种自然的推动力去开始增加他们的产品组合，尝试进入新产品、新服务，等等。这并不意味着会有一个一刀切的解决方案。因为从某种意义上说，如果你担心这个，你只需要停止做我们正在做的事情，然后等待 OpenAI 统治世界，Anthropic 统治世界。因为这些公司最终会——你确实意识到 OpenAI 有模型，运行推理，他们有自己的搜索，他们并不完全公开，但如果你在他们的平台上构建智能体，它就可以访问搜索。他们有各种沙箱、本地智能体、云智能体，等等。OpenAI 和 Anthropic 现在都在投资前置部署组织，并且将去做传统上——我不知道，德勤或埃森哲的那块蛋糕。对吧？所以他们会无处不在。我认为这是大公司开始进入更广阔市场的自然举动。但这并不意味着他们将是唯一的玩家。所以即使在今天，独立对象存储玩家也有一席之地。突然之间，对吧？或者独立的算力玩家。或者独立的小规模推理玩家。独立的数据库玩家。就像——所以看起来谷歌有 BigQuery 之类的各种数据东西，但后来 Snowflake 不知从哪儿冒了出来。AWS 有各种托管数据库、大规模数据库，比如 Aurora、Redshift 等。但后来 MongoDB 出现了，而且相当成功。我不能说它是超大规模，按照现在的标准可能永远也不会是，因为现在他们起步就是万亿美元级别的大公司。对吧？但就像两年前，它是一家非常稳健和非常成功的公司。或者像 ClickHouse 这样的公司带着他们的云产品出现，对吧？也是一个大型托管数据库，一家独立公司找到了自己的利基市场并做得很好。所以，我认为大公司、更大的云会增长。他们会在每个领域都有产品，并且因为捆绑、因为销售能力等原因，在销售这些产品方面会相当成功。但利基玩家总有一席之地。而且因为市场巨大，利基并不意味着小。利基并不意味着小规模或低收入或低估值。这些可以是稳健的公司。

In a sense, I think it will mimic what's happened with the legacy generation of clouds, previous generation of clouds, right? When your growth dictates that you need to start addressing new sectors, new markets, etc. because otherwise your growth is bounded and you can't show your projections, can't justify evaluations, etc. So there will be a natural push for the larger companies to start increasing their portfolio, trying to get into new products, into new services, etc. It doesn't mean that there will be a one-size-fits-all solution. Because in a sense, if you're concerned by that, you will like we just need to stop doing whatever we are doing and just wait for OpenAI to rule the world, Anthropic to rule the world. Because these companies will eventually like you do realize that OpenAI has models, runs inference, they have their own search that they don't exactly expose it, but if you're building the agent on their platform, it has access to search. They have all kinds of sandboxing, local agents, cloud agents, etc. Both OpenAI and Anthropic are now investing in forward deployed organizations and are going to go and do the work that was traditionally like I don't know, Deloitte's or Accenture's piece of pie. Right? So they will be everywhere. And I think this is a natural motion for the large company to start getting into broader markets. But it doesn't mean they are going to be the only ones. So even nowadays, there is a place for independent object storage players. All of a sudden. Right? Or independent compute players. Or independent small-scale inference players for that matter. Independent database players. Like when so like it seemed like Google had big query and whatever, all kinds of data stuff, but then Snowflake kind of appears from out of nowhere. AWS has all kinds of managed databases, large-scale databases, like Aurora, Redshift, etc. But then MongoDB appears and is pretty successful. I can't say it's super large-scale and probably by nowadays standards will never be because nowadays they started for the large companies like trillion plus. Right? But like 2 years ago, it was a very solid and very successful company. Or someone like ClickHouse appears with their cloud offering, right? Also, a massive managed database, an independent company that finds its niche and works it real well. So, I think larger company, larger clouds will grow. They will have products in every sector and will be pretty successful at selling them those because of bundling, because of their sales power, etc. But there will always be a place for niche players. And because the market is huge, niche doesn't mean small. Niche doesn't mean small scale or low revenue or low valuation. These can be solid companies.

是的。但有趣的是你提到 OpenAI 和 Anthropic 也有这种广泛的产品套件。我的意思是，在很多方面他们显然在构建自己的 AI 云。我认为最近一件有趣的事情是，显然这些 AI 云与其他云相比的局限性在于只有一个模型家族，对吧？只能调用一个。但我觉得 OpenAI 宣布与 Baseten 的合作真的很有趣。我很好奇你对此的看法。我的意思是，这是他们试图进入这个通用的、跨模型的超大规模游戏吗？

Yeah. But it's interesting you mentioned like, you know, OpenAI and Anthropic having this kind of broad suite of offerings, too. I mean, in many ways they're obviously building their own AI clouds. And I thought one thing that was interesting recently was, you know, obviously limitation of those AI clouds versus others is the having just one model family, right? To be able to call from. Um but I thought it was really interesting that OpenAI announced that partnership with Base 10. I'm curious what your take on was on that. I mean, is that them just trying to get into this general, you know, hyperscaler game across models?

从某种意义上说，我们看到很多推动力和担忧，人们声称 OpenAI 模型或 Anthropic 模型太贵了。每个人月底收到账单时都感到困惑或惊讶。所以，这可能是对此的回应。好吧，如果你觉得贵，那就去试试开源模型。同时，开源模型仍然落后于闭源模型，对吧？无论那些模型多好，它们仍然落后。给人们一个尝试较低规模的选择，并能够切换回顶级前沿模型，这也是一个好的激励。总之，我认为 OpenAI 通过那次合作做了一个非常有趣的举动。我非常喜欢，因为这显示了灵活性，对吧？他们不是一成不变的。他们正在探索不同的选择，他们能或不能做的不同事情。

In a sense, we see a lot of push and concerns about people claiming that OpenAI models or Anthropic models are too expensive. Everyone is puzzled or surprised by the bills they get at the end of the months. So, this might be an answer to that. Well, if you're like, well, go try open models. At the same time, open models are still behind the closed source models, right? However good those models are, they're still lagging behind. And given people an option to try lower scale with the ability to switch back to the top frontier model, it's also a good incentive. Anyway, I think OpenAI is doing a very interesting move with that partnership. And I like it really much because this kind of shows flexibility, right? They're not set in stone. They're exploring different options, different things that they can or cannot do.

就像寻找不同的选项，但我认为在某种意义上，这一切都归结为一件事。OpenAI 想要的和 Anthropic 一样。他们想成为单一的接触点。他们想成为一切事情的单一入口。即使那样也没问题，但顺便说一句，有时我想在我的 Codex 控制环境中使用开源模型。所以对他们来说很好，但如果公司、个人、用户仍然在你的生态系统中，你仍然有很多选项来引导他们、追加销售他们、向他们介绍新服务等等。所以如果这能缓解其中一个担忧，那就这样吧。为什么不呢？

Like looking for different options, but I think in a sense this all plays into one single thing. OpenAI wants the same as Anthropic. They want to be one single touch point. They want to be one single thing for everything. Even that is okay, but sometimes I want to use open source models within my Codex controlled environment, by the way. So good for them, but if the company, the person, the user is still within your ecosystem, you still have a lot of options to steer them, to upsell them, to introduce them to new services, etc. So if that alleviates one of the concerns, so be it. Why not?

嗯，我总是喜欢在采访最后塞一些过于宽泛的问题。所以如果你能配合我一下，我想第一个问题就是：在过去一年里，你在 AI 方面改变看法的一件事是什么？

Well, I always like to end these interviews with some overly broad questions that I stuff into the very end. So if you'll humor me, I guess the first question is just: what's one thing that you've changed your mind on in AI in the last year?

过去一年？哦，过去一年发生了很多事。显而易见的答案会是软件工程。很多人忘了，但一年前我们生活在一个完全不同的世界。

In the last year? Oh, a lot has happened in the last year. The obvious answer would be software engineering. A lot of people forget, but a year ago we lived in a totally different world.

是啊，太惊人了。就像 AI 世界的狗年。

Yeah, it's amazing. Like dog years in AI world.

是的，能力差远了。是的，在模型能力以及现代工具和框架的水平方面都差远了。就像 AI 编程。因为一年前它完全是……现在，它已经是你日常工具中坚实的一部分。但例如，如果你不谈行业趋势，或者更谈我个人的情况，老实说，我已经开始更多地依赖 AI 支持的本地智能体和框架来卸载一部分认知负担。

Yeah, way less capable. Yes, way less capable in terms of the model capabilities and in terms of how good the modern harnesses and tools are. It's like AI coding. Because a year ago it was total. Right now, it's a solid part of your everyday tooling. But for example, if you're not talking about the industry trends or talking about more like my personal thing, I've started relying more on AI-enabled local agents and harnesses to offload a part of my cognitive load, to be honest.

嗯，显然你管理着一个非常庞大的技术组织，我想每个人都在试图弄清楚——显然模型最近才在编程方面变得非常出色——所以每个人都在试图弄清楚，在具备新模型能力的情况下组建团队和运营组织意味着什么。我很好奇你最终是怎么定的，以及你是如何思考这个问题的。

Well, obviously you run a really large technical organization, and I think everyone's trying to figure out—obviously the models only recently got really good on the coding side—and so everyone's trying to figure out what it means to build teams with the new model capabilities and run an organization. I'm curious where you've settled and how you've thought about that.

谨慎地。保守地。我非常担心现代模型及其提供的吸引人的用户体验——现代智能体、现代模型等等——会导致我们出现问题。你会看到垃圾内容在聊天、合并请求中随处可见。我试图粘贴这些，并试图对此保持非常敏锐和警惕。所以，例如，我们不限制 token 使用，对吧？人们可以随意使用。我们的预算相当可观。我们不施加每人或每部门的限制。但与此同时，我们试图查看使用情况，弄清楚它用于什么，查看人们使用它的最低效方式。我们制定了一套政策，强调无论你用智能体做了什么，它仍然是你的结果。这是你的责任。这是你的拉取请求。这是你的代码。是你待命。如果出了问题，是你半夜醒来，等等。如果某件事是由完全自主的智能体创建的，就会这样标记。完全自主的智能体暂时不会接触核心服务。因为这是越界的。有时人们会惊讶，我在一家参与这场 AI 革命的公司工作，却非常保守，但同时我见过香肠是如何制作的，我会保持谨慎。

Cautiously. Conservatively. I'm very concerned that modern models and the attractive user experience that they provide—modern agents, modern models, etc.—is something that will lead us to issues. You see slop being revealed in chats, in merge requests, all over. And I try to paste that, and I try to be very observant and vigilant about that. So, for example, we don't restrict token usage, right? People can use whatever they want. Our budget is quite significant. We don't impose per person limitations or per department limitations. But at the same point, we try to look at the usage, figure out what it is used for, look at the most ineffective ways people use it. We constructed a set of policies that doubles down on the idea that whatever you've done with the agent, it's still your result. It's your responsibility. It's your pull request. It's your code. It's you who is on call. It's you who's going to wake up in the middle of the night if something goes awry, etc. And if something was created by a complete autonomous agent, it is marked like that. Complete autonomous agents will not touch the core services for a while. Because this is out of bounds. Sometimes people are surprised that I work in a company that kind of participates in this whole AI revolution, and I'm very conservative, but at the same time I've seen how the sausage is made, and I'll be cautious.

你觉得迄今为止在打造 Nebius 的过程中犯的最大错误是什么？显然，很多事都做对了，但我很好奇——回顾过去，你希望你们当初能做得不一样的地方。

What's the biggest mistake you feel like you've made in building Nebius so far? Obviously, a lot's gone right, but I'm curious—something that in retrospect you wish you guys had done differently.

现在听起来可能有点好笑，但我们低估了规模。这不是关于公司的增长。我们从一开始就相当乐观，但我个人——在最初的重要阶段，这是我的强烈偏见——我低估了当今单个客户运营的规模。因为 5 年前，如果你谈论一个 30 兆瓦的数据中心，那是一个严肃的设施。它托管了数十万台服务器，规模宏大。到现在，30 兆瓦有时对单个客户来说都不够。我还在适应这一点。我提到过存储，对吧？就像 4 年前，当你谈论网络附加存储时，你谈论的是每秒数百兆字节，也许几千兆字节的吞吐量。现在，我们的一些客户经常从单个共享文件系统获取，我不知道，每秒 700 千兆字节。而我仍然——我一开始低估了它，我还在适应这个规模。这很奇怪。我是一个基础设施人。在我的职业生涯中，我多次看到这些数字。突然之间，你在末尾加几个零，就称之为新常态。

It might sound funny these days, but we underestimated the scale. And it's not about the growth of the company. We were pretty optimistic from the very beginning, but I personally—and that was my strong bias for the first significant part—I underestimated the scale that a single client operates at nowadays. Because 5 years ago, if you were talking about a 30 megawatt data center, it was a serious facility. It hosted hundreds of thousands of servers, like gloriously large. By now, 30 megawatt sometimes is not enough for a single client. Which I'm still getting used to. The storage I've mentioned, right? Like 4 years ago, when you were talking about network attached storage, you were talking about hundreds of megabytes, maybe gigabytes per second of throughput. Right now, some of our clients routinely fetch, I don't know, 700 gigabytes per second from a single shared file system. And I'm still—I underestimated it in the beginning and I'm still getting used to this scale. It's weird. I'm an infrastructure person. I've worked and I've seen these numbers throughout my career multiple times. And all of a sudden you add a couple of zeros to the end and call it a new norm.

是啊。

Yeah.

仍然令人困惑。

Still puzzling.

这太疯狂了。嗯，我想最后一个问题是，显然当今 AI 领域有很多未解决的问题，很多人试图弄清楚世界将走向何方。我很好奇你在关注什么，我想从这样一个角度——如果它朝一个或另一个方向发展，真的会改变 Nebius 整体的战略。在 AI 世界走向的这些未解决的问题上，你最关注什么？

It's crazy. Um and I guess lastly, there's obviously a lot of unsettled questions in the AI field today that lots of people are trying to figure out where the world goes. I'm curious what you're paying attention to, and I guess with a lens toward—if it goes one way or another, it would really change the strategy for Nebius as a whole. What are you most paying attention to on these unsettled questions of where the AI world's going?

嗯，我不再关心新模型是否更大或更聪明。我只是接受这是理所当然的。我有点试图寻找新的新颖架构和方法，但到今天为止我还没看到。我希望在某个 stealth 实验室的深处，有新的东西正在酝酿，但坦率地说，目前我没有看到任何我甚至会下小赌注的东西。但我更感兴趣的是智能体方面的整个结果优化部分。因为我们习惯于谈论模型、模型大小和模型家族等等。但现在我们已经转向这些多轮长期运行的智能体，我们谈论的不是请求，而是结果。公司们在谈论按结果竞价。我对那部分将如何发展非常感兴趣，实际上，我们在 Nebius 内部有一个小型研究小组正在研究这个问题。

Well, I stopped caring about the new models being larger or smarter. I just accepted that it's a given. I'm kind of trying to look out for new novel architectures and approaches, but I don't see any as of today. I hope somewhere deep down in a stealth lab something new is cooking, but for now I frankly don't see anything that I would even place a small bet on. But I'm more interested in the whole outcome optimization part on the agent side. Because we are used to talking about models and size of models and families of models, etc. But now that we've shifted to these multi-turn long-running agents, we are talking not about requests but rather outcomes. Companies are talking about bidding for per outcome. I'm very interested in how that part will shape up, and we have a small research group internally within Nebius that's working on this problem, actually.

如何提高达到那个结果的概率？不是靠增大模型或改变模型。那当然也是讨论的一部分。但我认为，在引入更复杂、或者也许更简化的工具这一环节上，有很大的优化空间。你引入的 harness 会开始根据特定任务演化。这某种程度上又让我们回到显著的 CPU 消耗，因为很多现代 AI 工作负载实际上又回到了代码。它是代码化的，非常形式化。你会看到质量门禁，看到复杂的工具，等等。有时候你的智能体连续工作好几分钟，甚至根本不向 LLM 发出后端请求，因为工作负载开始集中在这一部分。而这正是真正让我感兴趣的地方，因为我认为未来 AI 的很多采用，取决于你的智能体式场景有多可靠，以及你有多可靠地达到那些结果。而模型之外的整个优化循环，是其中重要的一环。

How do you increase the chances of getting to that outcome? Not by increasing the model or changing the model. That is, of course, part of the discussion. But I believe there is a lot of optimization on the part where you introduce more sophisticated or maybe more simplified tools. You introduce the harnesses that start to evolve according to a particular task. And this kind of gets us back to significant CPU consumption, because a lot of modern AI workloads actually go back to becoming code. It's codified, it's very formal. You see quality gates, you see sophisticated tools, etc. And sometimes your agent works for minutes and minutes and minutes without even issuing a backend request to an LLM, because the workloads start to sit on this part. And this is the part that really interests me, because I think a lot of future AI adoption lies in how reliable your agentic scenarios are and how reliably you reach those outcomes. And I think the whole optimization loop outside the model is an important part of that.

百分之百同意。嗯，这是一次非常棒的对话。非常感谢你来上节目，也谢谢你抽时间来。

100%. Well, this has been an awesome conversation. Really appreciate you coming on the show, and yeah, thanks so much for coming on.

谢谢你邀请我。

Thank you for having me.

我是 Jacob Effron，这里是 Unsupervised Learning，一档让我与 AI 领域最聪明的人对话、向他们提出大量关于模型进展以及这对企业和世界意味着什么的问题的播客。希望你们看得出来，我做这件事非常开心。这是我在 Redpoint 做投资人本职工作之外，利用夜晚和周末做的项目，但我们能请到这些了不起的嘉宾，真的离不开像你这样的听众订阅播客、把它分享给朋友。正是这些最终让这一切运转起来。所以，请考虑这样做，也非常感谢你的支持和收听。我们下期再见。

I'm Jacob Effron and this has been Unsupervised Learning, a podcast where I get to talk to the smartest people in AI and ask them tons of questions about what's happening with models and what it means for businesses and the world. As I hope is clear, I have a ton of fun doing this. It's a nights and weekends project in addition to my day job as an investor at Redpoint, but our ability to get these incredible guests on really comes from folks like you subscribing to the podcast and sharing it with friends. It's really what ultimately makes this whole thing work. And so, please consider doing that, and thank you so much for your support and listening. We'll see you next episode.