扩散 LLM：比自回归模型更快、更便宜、扩展性更好

好了，各位。欢迎收听新一期的 Twilio AI 播客。我是主持人 Sam Charrington。今天，我们邀请到了 Stefano Ermon。Stefano 是斯坦福大学的副教授，也是 Inception 的 CEO。在开始之前，请务必在收听节目的地方花点时间点击订阅按钮。Stefano，欢迎回到播客。好久不见了。

All right, everyone. Welcome to another episode of the Twilio AI podcast. I am your host, Sam Charrington. Today, I'm joined by Stefano Ermon. Stefano is associate professor at Stanford University and the CEO of Inception. Before we get going, be sure to take a moment to hit that subscribe button wherever you're listening to today's show. Stefano, welcome back to the podcast. It has been a while.

是的，谢谢你再次邀请我。确实，自从我们上次聊天已经很久了。我想大概有八年了吧。当然，很多事情都变了。我们会谈到其中一些，特别是你在扩散模型方面的工作。但首先，不如你跟我们说说过去八年你都在忙些什么？

Yeah, thank you for hosting me again. Yeah, it's been a very long time since we last chatted. Yeah, I think about eight years or so. Certainly lots has changed. And we'll get into some of that, in particular, what you've been doing with diffusion models. But to get us started, why don't you tell us a little bit about what you've been up to for the last eight years, maybe?

是的，我一直在同一个领域工作。可以说，我整个职业生涯、整个人生都在研究生成模型。现在，变化在于这个领域真正起飞了。我想现在它被称为生成式 AI，每个人都在关注它，它成了每个人都关注并试图进入的领域。所以，看到这个领域的成长和这些模型的能力，真的很令人兴奋。当我大约在 2014 年开始时，我们几乎只能对 MNIST 图像建模，而且图像非常模糊，那已经算是一个重大成果了。现在，这些模型的能力标准已经有所提升。所以，这很令人兴奋。更具体地说，我在斯坦福的实验室一直处于这些模型能力的前沿，一直在模型层面、架构层面、ML 系统层面进行创新。所以，我在 2019 年就开始了扩散模型的早期工作，那时每个人都在使用生成对抗网络，如果你还记得的话。是的，我们提出了这种替代方法，也就是现在所谓的扩散模型，它现在几乎用于图像、视频、音乐的每一个生成解决方案。从那时起，我一直在尝试让这些模型处理文本、代码和 DNA 等离散对象。最终，我们用这种方法取得了一些非常好的结果，这就是我在 Inception 所做的事情。我目前是一家名为 Inception 的初创公司的 CEO 和联合创始人，我们正在开发一种基于扩散的新型 LLM。这些新型 LLM 速度更快、效率更高、质量更高。所以，我们几天前刚刚发布了最新的模型 Mercury 2。如果你想尝试一种不同的 LLM，一种在生成文本和代码方式上根本不同的 LLM，那就试试吧。它非常快。它是一个很好的解决方案，特别是当你考虑对延迟敏感的 LLM 应用，且延迟预算非常紧张时。这些模型非常快速，并且能给出高质量的回答。所以，很多开发者已经在它们之上构建实时 AI 应用，这是我目前最兴奋的事情。我花了很多时间思考如何让这些模型表现得更好。

Yeah, so I've been working still in the same space. So, I've been working in generative models, I guess, my whole career, my whole life. Now, what has changed is that the field really took off. I guess now it's called generative AI and everybody is paying attention to it and it's become the thing that everybody is looking at and everybody's trying to get into. So, yeah, it's been exciting to see the growth of the field and the capabilities of these models. When I started back in 2014 or so, we were barely able to model MNIST images and it was all very blurry and that was already like a big result. And now, the bar has shifted a little bit in terms of what these models can do. So, it's been exciting and more specifically, my lab at Stanford has always been at the forefront of what these models can do and always been innovating at the model level, at the architecture level, at the MLCs level. So, I did early work on diffusion models back in 2019 when everybody was using generative adversarial networks, if you still remember. Yeah, so we came up with this alternative approach that is what's now called a diffusion model, which is now used in pretty much every generative solution for images, video, music. And since back then, I've been trying to get these models to work on text and code and DNA, like discrete objects. And finally been able to get some really good results with this approach and that's what I've been doing at Inception. I'm currently the CEO and one of the founders of a startup called Inception where we are developing a new kind of LLM that is based on diffusion. And these new LLMs are way faster, more efficient, higher quality. So, we just launched our newest model, Mercury 2, a couple of days ago. And so, if you want to play with a different kind of LLM, something that is fundamentally different in the way it generates text and code, give it a try. It's really fast. It's a great solution, especially if you're thinking about latency-sensitive applications of LLMs, with very tight latency budgets. These models are really quick and they get really high quality answers. So, a lot of developers are already building a bunch of real-time AI applications on top of them and that's what I'm most excited about today. That's what I've been spending a lot of my time figuring out how to get these models to work even better.

带我们回到扩散模型的创建过程。比如，灵感来自哪里？

Take us back to the creation of diffusion models. Like, where did the inspiration come from?

是的，当时这个领域由 GAN（生成对抗网络）主导。这是一种旧方法，有两个神经网络。一个生成图像，另一个试图区分图像是真是假，然后你让它们相互对抗训练。这是一个非常不稳定且具有挑战性的优化问题，因为存在博弈论方面的因素，这两个网络需要相互竞争，非常不稳定，很难让它很好地工作。需要很多技巧。所以，我们在实验室尝试探索替代方案。其中一个替代方案是常见的自回归方法，即一次生成一个像素，这种方法对于图像和视频从来都不是特别有效，现在仍然如此。它非常慢，而且不太准确。于是，我们提出了这种替代方法，现在称为扩散模型，其本质是从噪声开始生成图像，然后迭代地细化，直到最终得到一个清晰、漂亮的图像，与提示一致。关键优势在于训练目标非常稳定。神经网络只是试图对图像进行去噪。所以，你拿图像，添加噪声，然后训练一个神经网络去除噪声。这是一个相当标准、相对简单的优化问题，你可以用它在大数据集上训练神经网络，效果相当不错。然后，你可以在推理时使用这个神经网络生成图像，因为网络已经训练好去除噪声、改进样本、纠正错误。事实证明，你可以从纯噪声开始，然后多次应用这个去噪网络，最终得到一张非常好的图像。关键优势在于，当时人们并没有考虑测试时推理之类的事情，但它具有一种神经网络的风格，即在推理时有一个非常深的推理路径，将神经网络的多次评估串联起来。所以，你有一个非常深的计算图，可以完成非常强大的事情。但在训练时它仍然非常可扩展，因为你不需要在训练时展开所有这些计算。在训练时，你只是训练模型去除噪声。所以，你基本上只需要在训练时进行一次神经网络评估。

Yeah, so back then, the field was dominated by GANs, generative adversarial networks. And that's the old approach where there's two neural networks. There is one that generates images and there's one that is trying to discriminate and figure out if the images are real or fake and then you train them one against each other and it's a very unstable and challenging kind of optimization problem because there's this game-theoretic aspect to it where they need to outcompete each other, these two neural networks, and it was very unstable, very difficult to get it to work well. A lot of tricks were needed. And so, we were trying in my lab to experiment with alternatives. And one alternative was the usual auto-regressive approach where you generate the image one pixel at a time and that's never worked particularly well for images and video and it still doesn't. It's just very slow and not very accurate. And so, we came up with this alternative approach which is now called a diffusion model where essentially you generate an image by starting from noise and then iteratively refining it until you get a crisp nice image that is consistent with the prompt at the end. And the key benefit is that the training objective is very stable. The neural network is trying to just denoise an image. So, you take images, you add noise and you train a neural network to remove noise. It's a fairly standard, relatively easy optimization problem that you can use to train neural networks over large data sets and it works reasonably well. And essentially, you can then use this neural network at inference time to generate images because the network has been trained to remove noise, to improve the samples, to correct mistakes. And so, it turns out that you can just start with pure noise and then you apply this denoising network a bunch of times and at the end, you get a really nice image. And the key benefit, I mean, back then people were not thinking about test time inference and those kind of things, but it has the flavor of a neural network where you have a very deep inference path where you're chaining together many evaluations of this neural network at inference time. So, you have a very deep computation graph that can do very powerful things. But it's still very scalable during training because you don't have to unroll all of this computation during training. During training, you just train the model to remove noise. So, you just basically need a single neural network evaluation during training.

所以，这种训练成本低、推理时却非常强大的想法一直在我脑海里，我一直在思考如何高效地进行这类计算。现在，这种想法在 LLM 的背景下以另一种形式出现，人们对思维链以及能够在推理时调整算力感到非常兴奋。我觉得这是一个类似的想法，尽管它是在一种截然不同的生成模型之上实现的。

So, this idea of having something that is cheap to train, yet very powerful at inference time, has always been something that was on the back of my mind and trying to think about ways to do these sorts of computations efficiently, which is now kind of showing up in a different form in the context of LLMs where people are very excited about chain of thought and being able to adjust the amount of compute at inference time. I feel like it's a similar idea, although implemented on top of a very different kind of generative model.

请谈谈从图像扩散模型到文本扩散模型的路径。是的，这花了一段时间。在图像生成上取得良好结果后——最初我们证明这些模型优于 GAN，然后该领域迅速转向扩散模型，Stable Diffusion 和 Midjourney 相继出现，并很快基本接管了整个领域。从那时起，我就开始思考：我们如何让这些想法适用于文本和代码，因为我们希望模型能够生成离散对象。

Talk a little bit about the path to getting from diffusion models for images to diffusion models for text. Yeah, so that took a while. So immediately after getting the good results on image generation where initially we showed that these models were better than GANs, and then very quickly the field switched to diffusion models and Stable Diffusion came out and Midjourney, and then quickly basically took over the whole field. And since basically back then, I started thinking about, okay, how do we get these kinds of ideas to work for text and code, when we wanted the model to somehow generate discrete objects.

你多次提到离散，相对于连续。能谈谈为什么这对扩散模型构成挑战吗？

You've mentioned discrete a couple of times, as opposed to continuous. Can you talk about why that presents a challenge for diffusion models?

当然。如果你考虑一张图像，甚至一个像素，它是一堆颜色，有趣的是，如果你稍微改变颜色，含义并不会改变。具体来说，你可以考虑一个像素的两种可能颜色，它们之间的所有颜色仍然有意义，并且不会以任何戏剧性的方式改变图像的含义，对吧？但如果你考虑文本，取两个词，那么两个不同词之间的含义是什么就不清楚了，对吧？所以可能的 token 或词的空间没有真正的几何结构。这使得去噪的想法更具挑战性，因为不清楚对文本添加噪声意味着什么。不清楚如何构建——整个几何结构都不存在。因此，许多为让扩散模型在图像和视频上工作而发明的概念，都严重依赖于这样一个事实：存在某种可能的图像连续体，你可以在它们之间插值，让模型平滑地从一张图像移动到另一张图像是有意义的。在文本和代码的背景下，一切都是离散的，因此为连续空间开发的数学如何直接转化到离散空间并不明显。

Yeah, of course. So if you think about an image or even a single pixel, it's a bunch of colors, and the interesting thing is that if you change the colors a little bit, the meaning doesn't change. So in particular, you can think about two possible colors for a pixel and all the things in between them still make sense and they don't change the meaning of the image in any dramatic way, right? But if you think about text, and you take two words, then it's not clear what's in between the meaning of two different words, right? And so there is no real geometry to the space of possible tokens or possible words. And so that makes the idea of denoising much more challenging because it's not clear what it means to perturb noise to text. It's not clear how you build the whole geometry does not exist. And so a lot of the concepts that were invented to get diffusion models to work on images and video were relying very heavily on the fact that there is some kind of continuum of possible images and you can interpolate between them and it makes sense to get the model to smoothly move from one image to another. In the context of text and code, everything is very discrete, and so it's not obvious how you get the mathematics that were developed for continuous spaces to translate immediately to discrete spaces.

当你谈到点之间的词和邻域中的词时，这让人联想到嵌入空间之类的。我想这在多大程度上被尝试过，或者也许是让它在文本上工作的最终解决方案的一部分？

When you talk about the idea of words between points and words in a neighborhood, it calls to mind embedding spaces and the like. To what degree I imagine that's been tried, or maybe part of the ultimate solution to getting it to work for text.

是的，有一些方法本质上试图在嵌入空间中为语言生成构建扩散模型，即你把所有东西都嵌入，然后构建一个扩散模型，但问题在于你最终必须解码回文本，对吧？你最终不能给用户或客户一个嵌入。所以问题始终是：最终，扩散模型会犯一些小错误，可能不会恰好落在对应于字典中某个现有词的点上。因此，让这些扩散模型在潜在空间中良好工作实际上相当具有挑战性。有很多论文，包括来自学术界和工业实验室的，但并不是很成功，但这是人们尝试过的方法之一。

Yeah, so there are approaches that essentially try to build diffusion models for language generation in the embedding space, so where you embed everything and you build a diffusion model, and then the problem is that essentially you have to eventually decode back to text, right? Eventually you cannot give an embedding to your users or your customers, and so that's always the problem that at the end of the day the diffusion model will make some small mistakes and it might not end up exactly in a point that corresponds to one of the existing words in the dictionary. And so it's actually pretty challenging to get these diffusion models to work well in latent spaces. There's been a number of papers including from academia and industrial labs, but it's not been very successful, but it is one of the approaches that people have taken.

那么，对于文本扩散，什么被证明是有效的呢？

So what has been demonstrated to work for you text with diffusion?

最初的结果仍然是在学术环境中。实际上，又是来自我的实验室，我们几年前发表了一篇论文，首次证明可以训练一个基于 Transformer 的模型——你基本上拿一个 GPT-2 大小的模型，然后以通常的方式自回归地训练它，像其他人训练 LLM 一样训练它预测下一个 token，同时你可以将同一个神经网络训练为扩散模型。在那篇论文中，我们首次展示了能够匹配质量。所以从困惑度、两个模型生成文本的质量来看，大致相同，但扩散模型明显更快——你可以用大约十分之一的数量生成相同质量的文本，即神经网络评估次数减少 10 倍。所以在 GPT-2 规模上，扩散模型效率显著更高。

So the initial results were still sort of in the academic setting. It was actually again from my lab where we had a paper a couple of years ago essentially showing that for the first time it was possible to train a transformer-based model — so you basically took a GPT-2 size model and then you train it autoregressively the usual way, you train it to predict the next token the way everybody else is training LLMs, and you can train the same neural network as a diffusion model. And in that paper we showed that for the first time we were able to match the quality. So in terms of perplexity, in terms of the quality of the text that these two models are able to generate, it was about the same, but the diffusion model was significantly faster — like you could generate the same quality of text in about 10x less, so 10 times less number of neural network evaluations. So diffusion models were significantly more efficient at the GPT-2 scale.

为了让我理解设置，你是说你用自回归方式训练它，又用扩散方式训练它？这是你在比较的两个不同模型，还是你依次用自回归然后扩散训练它？

And just so I understand the setup, are you saying you train it autoregressively and train it via diffusion? Are those two different models that you're comparing, or are you sequentially training it autoregressively and then with diffusion?

这实际上就像一次 AB 测试，是一个非常公平的比较：你采用完全相同的神经网络架构，参数数量相同，在相同数据量上训练，一方面你把它作为典型的自回归模型训练，只预测下一个 token，推理时也这样使用；另一方面，你把它作为扩散模型训练。因此，性能差异完全归因于不同的建模范式：扩散模型与自回归模型。

So it's really just like an almost AB test where it's a very fair comparison in the sense that you take the same exact neural network architecture with the same number of parameters, you train it on the same amount of data, you just train it on the one hand as a typical autoregressive model where you just predict the next token and that's how you use it at inference time, and then on the other hand you can train it as a diffusion model. And so at that point, the difference in performance is entirely due to the different modeling paradigm: diffusion versus autoregressive model.

你是如何克服训练该模型时的离散挑战的？

How did you overcome the discrete challenge in training that model?

是的，这基本上是那篇论文的主要思想——有一些新的数学方法，基本上弄清楚了在离散文本类对象背景下进行扩散意味着什么，然后它被证明在实践中效果良好，直到 GPT-2 规模。

Yeah, so that was kind of the main idea in that paper — there were some new mathematics, some new methods of basically figuring out what it means to do diffusion in the context of discrete text-like objects, and then it was demonstrated to actually work well in practice up to the GPT-2 scale.

下一步是我们在 Inception 所做的——我对那些结果非常兴奋，于是创办了 Inception 公司，一直在扩大规模。我们训练了商业级的扩散语言模型，模型规模更大，数据也更多。现在结果非常好。比如我们本周发布的最新模型 Mercury 2，在质量上已经能匹配一些前沿实验室最快的优化模型。想想 OpenAI 的 Haiku 模型、Flash 模型、mini 模型。它达到了那个质量水平，但速度要快 5 到 10 倍——用扩散模型获得答案的时间比自回归模型短得多。

The next step was that we've at Inception and we, you know, I was very excited about those results and so I started a company called Inception where we've been scaling that up. And so, you know, we've trained commercial scale diffusion language models, so much larger models, some more data. And now the results are extremely good. Like the latest model that we announced this week, Mercury 2, is actually matching in quality some of the best speed-optimized models from frontier labs. So we'll think about the Haiku models, the Flash models, mini models from OpenAI. So it's at that quality level, but again, it's about 5 to 10x faster in terms of like the time it takes you to get an answer using a diffusion model versus an autoregressive model.

你能给我们概述或总结一下让这一切成立的数学原理吗？

Are you able to give us an overview or a summary of some of the mathematics that make this work?

在直觉层面上，它们有些相似：仍然有一个神经网络被训练来去除噪声。只是噪声过程不再像给像素强度加小数值那样。实际上有不同类型的噪声过程可用。一种效果不错的方法是掩码掉 token，也就是隐藏它们。你拿一个句子，去掉一些 token，对神经网络隐藏它们，然后问网络：你能预测这些 token 是什么吗？所以这在某种意义上类似于下一个词预测，只是顺序是乱的，网络需要利用左右上下文，以有趣的方式组合信息，来预测句子中所有缺失的 token。

At an intuitive level, it's all somewhat similar in the sense that there is still a neural network that is trained to remove noise. It's just that the noise process is no longer kind of like adding small numbers to the pixel intensities. It's more like there are different kinds of noise processes that you can use. One that works pretty well is basically one where you mask out tokens, so you kind of like hide them. You take a sentence and then you remove some of the tokens, you hide them from the neural network, and then you ask the neural network: can you predict what those tokens were? And so it's similar in some sense to next-token prediction, except that things are done out of order and the network needs to be able to use information from—it needs to use context to the left and to the right and combine it in some interesting ways to figure out how to predict all these missing tokens from the sentence.

所以某种程度上，你是在改变噪声的定义，使其在文本语境中更有意义。

So in some ways you're changing the definition of noise to one that makes sense in the context of text.

完全正确。事实上，这种训练目标与多年前的 BERT 风格模型非常相似。但当时它在自然语言处理中被广泛使用。人们正是用同样的目标训练这些神经网络：让网络预测一些缺失的 token。为了做到这一点，它必须理解其他 token 的含义，这是获取表示的好方法。在我提到的那篇 ICML 论文中，我们展示了：一旦你能做到这一点，你还可以从头生成内容。因为本质上，你可以从一个所有内容都被掩码的句子开始，然后让神经网络填充片段，但它是乱序进行的。所以不是从左到右一次生成一个 token，而是任意顺序。关键是，网络可以一次输出多个 token。这就是这些模型快得多的原因。因为在自回归世界中，要生成一千个 token，就需要一千次神经网络评估。而在扩散语言模型中，神经网络每一步可以输出许多 token。只要不需要太多步骤——比如 20 步去噪——这些模型就可以高效得多。

Exactly. And in fact, that kind of training objective is very similar to the BERT style models from many years ago. But that was the thing that for a while was sort of widely used in natural language processing. People were training these neural networks exactly on the same objective: this idea of, oh, let's train the network to predict some of the missing tokens. In order to do that, it really needs to understand the meaning of the other tokens, and that's a good way to get representations. In that ICML paper that I mentioned, basically we showed that once you can do that, you can also generate content from scratch. Because essentially you can start with a sentence where everything is masked, and then you can let the neural network figure out how to fill in pieces, but it does so out of order. So instead of generating left to right one token at a time, it does it in any order. And crucially, the network can output more than one token at a time. That's why these models are so much faster. Because in the autoregressive world, if you want to generate a thousand tokens, you need a thousand neural network evaluations. In the context of a diffusion language model, the neural network can output many tokens at every step. And to the extent that you don't need too many steps—20 denoising steps—then these models can be much more efficient.

在图像领域，我们熟悉那种逐步增强的图像，能看到图像逐渐成形。文本也能看到类似现象吗？文本是不是一开始很糟糕，然后逐渐变好？

In the image world, I think we're familiar with these progressively enhanced images where you see the image taking shape. Are you able to see the same thing with text? Does text start out horrible and get better over time?

是的，确实有类似的过程。事实上，如果你访问我们的网站，可以看到一些动画，让你了解背后的情况。不过，我认为它不像图像领域那样可解释——在图像中你能看到细节逐渐浮现。文本对我来说总是有点难以解释。但你可以——尤其是在代码中——有时看到结构逐渐浮现。至少你能看到一些有趣的模式，了解模型是如何生成答案的。而且，能够通过改进步骤数、去噪步骤数来控制答案质量，这非常令人兴奋，因为它提供了另一种进行测试时 Scaling、测试时推理的方向——在推理时使用算力来控制答案质量。如果你有一个自回归模型，控制答案质量的唯一方法是产生越来越长的思考痕迹。这正是这些推理模型所做的：在给出正确答案之前，它们会产生一个思考痕迹。让它们思考的时间越长，答案质量通常越好，但成本也越高，速度也越慢。扩散语言模型有另一种方式来实现类似效果：你可以控制去噪步骤数、迭代次数。迭代次数越多，质量越高。但所有编辑基本上都是原地进行的，所以你不必让痕迹变长。模型实际上能够进行错误修正；它能够改进自己的答案，而不必让它越来越长，这节省了内存，并且效率更高。

Yeah, there is definitely something like that going on. In fact, if you go on our website, you can see some little animations to give you a sense of what's going on under the hood. It's not as interpretable, I would say, as what you see in the image space, where you really see the details emerging as you go through the process. I think text has always been a little bit less interpretable to me. But you can definitely—especially in code—you can see the structure kind of emerging sometimes. At least you're able to see some interesting patterns in terms of how the model is producing the answer. And for sure, this idea of being able to control the quality of the answer as a function of the number of improvement steps, the number of denoising steps, is actually very exciting because it gives you another direction to do test-time scaling, test-time inference, using compute at inference time to control the quality of your answers. So if you have an autoregressive model, the only way you can actually control the quality of the answer is by producing a longer and longer thinking trace. That's what these reasoning models are doing: they produce a thinking trace before actually providing the right answer. The longer you let them think, the better the quality of the answer usually is, and the more expensive and slower it becomes. A diffusion language model has a different kind of access to do something similar, where you can control the number of denoising steps, the number of iterations. The more iterations you do, the higher the quality becomes. But all the edits are essentially happening in place, so you don't necessarily have to make the trace longer. The model is actually able to do error correction; it's able to improve its own answer without having to make it longer and longer, which saves memory and is significantly more efficient.

当你谈论推理模型和扩散语境中的思考时，除了改变去噪步骤数这个想法，我们是否应该以与自回归模型相同的方式来思考扩散模型中的思考？你们仍然有思考痕迹吗，还是扩散模型中的所有思考都是带外进行的？

When you talk about reasoning models and thinking in the context of diffusion, even beyond this idea that you can change the number of denoising steps, should we be thinking about thinking in the same way with diffusion models as autoregressive models? Do you still have thinking traces, or is all thought in diffusion models out of band?

是的，这是个好问题。事实上，推理和扩散语言模型这个领域还很新。Mercury 是第一个——Mercury 2，我们本周发布的模型，是第一个具有推理能力的商业级扩散语言模型。所以这项能力、这项技术都是全新的。在我们的案例中，我们仍然有推理痕迹。它们只是以不同的方式产生，模型通过去噪、通过一种不同的训练过程来生成它们。但推理痕迹的概念仍然存在。事实上，如果用户想要，我们可以提供推理痕迹的摘要。所以实际上非常相似。

Yeah, it's a great question. And in fact, the space of reasoning and diffusion language models is pretty new. Mercury is the first—Mercury 2, the model we released this week, is the first commercial scale diffusion language model with reasoning capabilities. So it's still this capability, this technology, it's all brand new. In our case, we do still have reasoning traces. They're just produced in a different way, and the models have been trained to generate them through denoising, through a different kind of training process. But the idea of a reasoning trace is still there. And in fact, we're able to provide summaries of the reasoning trace to our users if they want. So it's actually pretty similar.

实际上，所有 API 都与 OpenAI 兼容，你仍然可以使用一些参数来决定和控制你希望答案的速度，以及在推理时用多少算力来换取质量。在比较这两种模型时，我想到的另一件事是，对于传统的自回归 LLM，要获得更多文本，你只需继续生成 token，因为它是自回归的，是下一个词预测。对于扩散模型，这意味着什么？对上下文窗口之类的东西有什么影响？你是做滚动窗口生成，还是这一切如何转化？

And in fact all the API is OpenAI compatible and you can still use some parameters to decide and to control how quickly you want your answer, how much you want to trade off compute for quality at inference time. Another thing that I'm thinking about in comparing these two types of models is that with your traditional LLMs, autoregressive LLMs, to get more text you just continue generating tokens because it's autoregressive, it's next token. For diffusion models, what does that mean and what are the implications on things like context windows? Like are you doing rolling windows of generation or how does that all translate?

是的，这是另一种能力——处理可变长度输出的方法有很多种。我们在 Inception 找到了一种方法。我相信还有其他想法，比如做滚动块，这也有道理，并且已经在文献中发表。所以处理可变长度有不同的方法。用扩散语言模型是可能的。它不会影响相对于上下文大小的 Scaling。这更多受架构影响，而架构与训练目标完全正交。所以在 Inception，我们仍然使用 Transformer 作为底层神经网络。我们使用注意力机制，因此在注意力机制如何随序列长度扩展方面，我们有同样的优点和缺点。但这是一个正交的方向。有可能训练，事实上我们有扩散语言模型的原型，其中骨干网络不是 Transformer，而是基于状态空间或 Mamba 的，这样在上下文长度方面有更好的扩展，次二次扩展。但对于我们的主要模型，我们仍然基于 Transformer。

Yeah, that's another capability that is not—there are many different ways of handling outputs of variable length. We have figured out a way to do it at Inception. I'm sure there are other ideas, like doing rolling blocks, which also makes sense and has been published in the literature. So there are different ways of handling variable length. It is possible to do it with a diffusion language model. It does not affect the scaling with respect to context size. That's more affected by the architecture, which is completely orthogonal to the training objective. So at Inception, we are still using transformers as the underlying neural network. We are using attention, so we have the same benefits and downsides of attention in terms of how it scales with sequence length. But that's an orthogonal direction. It is possible to train, and in fact we have prototypes of diffusion language models where the backbone network is not a transformer but maybe state space based or Mamba based, so that you have better scaling with respect to context length, sub-quadratic scaling. But for our main models, we are still transformer based.

我想我想到的问题是，为什么是现在？这个模型是由该领域正在发生的其他特定事情促成的，还是只是你达到这一点所需的时间？

I guess the question that is coming to mind for me is why now? Is this model enabled by particular other things that are happening in the space, or is it just the time that it took you to get to this point?

是的，这是两者的结合。一方面，我们花了一段时间才弄清楚如何做到这一点，而且现在是扩大规模的合适时机，因为事情终于至少在学术基准和学术规模上奏效了。另一方面，人们开始意识到这一切都关乎推理 Scaling。有一段时间，人们关心的主要轴心是缩放定律，即事物在训练时间、预训练时间如何扩展。现在一切都转向了推理时间 Scaling，原因有几个。一是那是你看到最大收益的地方，比如强化学习后训练、测试时间推理，还有经济学：如果你需要扩展这些模型并且它们实际上进入生产，每个 token 的价格或每个 token 所需的瓦特数就成为关键指标。我们在扩散语言模型中看到的是，它们在推理时比自回归模型扩展得更好。它们服务成本更低，速度更快，每个 GPU 能获得更多 token，这意味着价格实际上更低。这就是为什么我们觉得现在是时候了，因为如果我们能在智能能力上匹配自回归模型，同时用在一个重要轴——成本和速度——上扩展得更好的解决方案，那么我们就有了非常有价值的东西，客户会趋之若鹜。事实上，我们看到对速度和成本的需求很大，而且我们看到其他竞争对手试图获得他们模型的快速版本，与 Cerebras 和 Groq 等 AI 推理芯片公司合作，试图推出最快的模型。但我们的解决方案是基于软件的，因此更具可扩展性；我们仍然在 GPU 上运行。所以你可以获得尽可能多的 GPU 容量，这相对于专门的 AI 推理芯片来说更容易。

Yeah, it's a combination of both. One is that it took us a while to figure out how to do it, and it was the right timing to scale things up because finally things were working at least on academic benchmarks at academic scale. The other one is that people are starting to realize that it's all about inference scaling. For a while, the main axis that people cared about was scaling laws in terms of how things scale at training time, at pre-training time. Now everything has shifted to inference time scaling, for several reasons. One is that that's where you're seeing the biggest benefits, like RL post training, test time inference, but also the economics: if you need to scale up these models and they're actually getting into production, the price per token or the watts needed per token becomes the key metric. What we're seeing with diffusion language models is that they scale better than autoregressive models at inference time. They're cheaper to serve, they're faster, you get more tokens per GPU, which means the price is actually lower. That's why we felt this is the time to do it, because if we can match the capabilities in terms of intelligence of autoregressive models with a solution that scales better along the axis that matters—cost and speed—then we would have something that can be very valuable and that customers would jump to. In fact, we're seeing a lot of demand for speed and cost, and we're seeing other competitors trying to get fast versions of their models, partnering with AI inference chip companies like Cerebras and Groq, to try to get the fastest models out there. Except our solution is software based, so it's much more scalable; we're still running on GPUs. So you can get as much capacity as you can get GPUs for, which is relatively easier compared to specialized AI inference chips.

我们现在在后训练中学习并经常应用的所有技术，在多大程度上也适用于这些扩散模型？

To what degree do all of the techniques that we have learned about and apply regularly now in post training apply also to these diffusion models?

有些适用，有些我们不得不从头重新发明。如果你考虑预训练、中期训练、监督微调，很多都相对简单，因为你可以基本上使用相同类型的数据集。架构不需要改变太多，损失函数——你只需要将损失函数从下一个词预测改为去噪。我认为强化学习，那才是事情变得更有趣的地方。因为在强化学习的背景下，无论你是从人类偏好做，还是你有某种可验证或不可验证的奖励需要优化，采样过程非常不同，你将信息传播回网络的方式也不同。事实上，这对扩散语言模型是有利的，因为在强化学习后训练的背景下，真正的瓶颈是推理。如果你在做 LLM 的强化学习后训练，你会花大部分时间做 rollout。你让模型提供一堆候选解决方案，然后用你的奖励函数给它们打分，然后你找出如何教模型将更多概率质量放在好的 rollout 上，避免被奖励函数评估为不好的 rollout。由于扩散语言模型在推理时快得多，你可以在强化学习后训练阶段做不同的事情。这就是我们现在花很多时间的地方：找出对扩散模型进行强化学习后训练的正确方法。

Some do, some we had to reinvent from scratch. If you think about pre-training, mid-training, SFT, a lot of that is relatively simple in the sense that you can essentially use the same kind of datasets. The architectures don't have to change too much, and the loss—you just need to change the loss function essentially from next-token prediction to denoising. I think about reinforcement learning, that's where things become more interesting. Because in the context of reinforcement learning, whether you do it from human preferences or you have some kind of verifiable or non-verifiable reward that you need to optimize for, the sampling process is quite different, and the way you would propagate that information back into the network is different. In fact, it's actually beneficial to diffusion language models because in the context of RL post training, the real bottleneck is inference. If you're doing RL post training of an LLM, you're going to spend most of your time doing rollouts. You get the model to provide a bunch of candidate solutions, then you score them using your reward function, and then you figure out how to teach the model to put more probability mass on the rollouts that were good and avoid rollouts that were not good as evaluated by the reward function. Because diffusion language models are so much faster at inference time, you can do different things in the RL post training stage. That's where we're spending a lot of our time right now: figuring out the right way to do RL post training for a diffusion model.

在预训练方面，这些模型是从头开始预训练的吗？

In terms of pre-training, are these models pre-trained from scratch?

是的，我们训练自己的模型。我们有自己的一套流程。我们没有透露太多关于我们如何做的细节。但我们有自己的配方，有自己的训练模型的技术栈。

Yeah, we're training our own models. We have our own pipeline. We have not disclosed a lot of detail in terms of how we do it. But we have our own recipe, we have our own stack for training our models.

那么这些方法是否与损失函数之外的东西有本质不同？或者说，如果你眯起眼睛看，是否还能看到我们训练自回归模型方式的影子？

And are the recipes substantially different from, you know, beyond the loss function? Or would they be like, you know, if you squint, you can kind of see the echoes of the way we train autoregressive models?

有一些相似之处，但我要说，要找到让这些模型正常工作的正确方法并非易事。事实上，多年来一直有人尝试让扩散语言模型工作，包括谷歌和其他地方，但很长一段时间都不成功。因此，弄清楚如何很好地训练它们，如何以最佳方式让它们扩展，并充分利用你所能获得的数据和算力，这绝非易事。

There are some similarities, but I would say it's been non-trivial to figure out the right way to get these models to work. In fact, there have been attempts for years to get diffusion language models to work, including from Google and other places, and they were not successful for a long time. So it is non-trivial to figure out how to train them well, how to get them to scale in the best possible way, and make the best possible use of the data and the flops you have access to.

是否已经注定无法从一个预训练的自回归模型出发，然后通过某种神奇变换将其转化为一个可以进一步进行扩散训练的基础模型？考虑到训练传统模型投入了如此多的精力和投资，这似乎会非常有趣。

Is it a foregone conclusion that there's no way to start from a pre-trained autoregressive model and somehow, through some magic transform, turn that into a base that you can then diffusion train? It seems like that would be really interesting given how much energy and investment has been placed in training traditional models.

学术界已经有很多论文尝试了各种方法来实现这一点。在某种程度上，嵌入可以重用，网络也可以部分重用。真正的挑战在于，传统自回归模型中使用的注意力掩码是因果的，因此模型在决定下一步做什么时，只知道如何使用左侧的上下文。而在扩散语言模型中，当你决定要改变什么时，你确实希望能够同时访问左侧和右侧的上下文。这是使这些模型可能比自回归模型质量高得多的关键特性之一。所以这是难点所在。人们探索了通过退火注意力掩码，使其从因果逐渐变为非因果的方法，让模型慢慢偏离初始的自回归状态。还有一些数学上更复杂的方法，可以将自回归模型的似然转换为分数函数，而这正是扩散模型所需要的。更广泛地说，总有一种方法有效，那就是从自回归模型中获取样本。这至少是生成合成数据的好方法，然后你可以用这些数据来训练你的扩散语言模型。这几乎就像一个黑盒，你总是可以用它来组合并尝试从现有模型中提取知识。正如我们所知，对于 Distil Gate，我认为很多实验室和研究人员都在考虑这个问题，而且它似乎正在其他地方以相当大的规模进行，但这总是可能的。

There have been a number of papers in the academic literature trying out various recipes for doing exactly that. To some extent, embeddings can be reused, and to some extent, networks. The real challenge is that the attention mask used in a traditional autoregressive model is causal, so the model only knows how to use context to the left as it figures out what to do next. In a diffusion language model, you really want to have access to context both to the left and to the right as you decide what to change. That's one of the key properties that make these models potentially much higher quality than autoregressive models. So that's the challenging bit. People have explored ways of annealing the attention mask to make it go from causal to non-causal slowly, kind of making the model drift away from the initial autoregressive state. There are more mathematically sophisticated ways of converting the likelihood from an autoregressive model to a score function, which is what you need in the context of a diffusion model. More broadly, one thing that always works is that you can get samples from the autoregressive model. That's usually a good way to at the very least generate synthetic data that you can then use to train your diffusion language model. That's almost like a black box that you can always use to combine and try to get knowledge out of an existing model. As we know, with Distil Gate, I think that has been on the mind of a lot of labs and researchers, and it seems to be something that is really going on at a pretty massive scale in other places, but that's always possible.

扩散模型的推理服务设置有何不同？

How does the serving setup change for diffusion models?

这是个好问题。这是另一个相当具有挑战性的方面。我认为，目前还没有其他提供商能够在生产中提供扩散语言模型服务的原因之一是，你无法在现有的推理服务引擎上运行扩散语言模型。想想 vLLM、SGLang、TensorRT 这些现有的开源框架，它们在高效服务自回归大语言模型方面非常出色。它们会为你处理连续批处理之类的事情，比如当有大量请求涌入时，如何将它们批处理在一起以高效服务。一旦你拥有多个 GPU 和大量请求，就需要进行各种优化。对于自回归模型，已经有很多现有的框架和出色的工作。而扩散语言模型的空间则远未成熟。因此，我们不得不构建自己的推理服务引擎。在过去一两个月里，SGLang 开始为社区开发的开源扩散语言模型提供一些支持。所以，开源社区中开始出现一些生态系统、工具和社区支持，但还远不及自回归模型那么成熟。

That's a great question. It's another pretty challenging aspect. I think one of the reasons why there are still no other providers that are able to serve diffusion language models in production today is that you cannot run a diffusion language model on existing serving engines. If you think about vLLM, SGLang, TensorRT, these frameworks that exist and are open source, they are really good at serving autoregressive LLMs very efficiently. They handle things like continuous batching for you, like when there is a stream of requests coming in, how do you batch them together to serve them efficiently. There are all kinds of optimizations you need to do once you have access to multiple GPUs and many requests. There is a lot of existing frameworks and great work that has been done for autoregressive models. The space for diffusion language models is much less developed. So we had to build our own serving engine. Over the last maybe month or two, there's been some support for diffusion language models in SGLang for the open source models that have been developed by the community. So there is starting to be a little bit of ecosystem, tooling, and community support for diffusion language models in the open source community, but it's still not nearly as developed as for autoregressive models.

你之前提到过改变细化步骤数量的能力，以及这在扩散模型自身的推理时间扩展中有多强大。目前，在静态到动态的频谱上，它是完全动态的吗？完全静态的吗？还是每个请求静态的？你如何看待那里的控制旋钮？

You talked earlier about the ability to change the number of refinement steps and how powerful that is in diffusion's own type of inference time scaling. Is that something that is currently, you know, on the static to dynamic spectrum, is it fully dynamic? Is it fully static? Is it per request static? How do you think about the knobs there?

这是一个设计选择。在某种程度上，这是我们作为开发者做出的选择，以决定如何向用户暴露这一功能。目前，我们的 Mercury 模型允许你选择不同的“努力程度”。我们试图保持与现有的类似 OpenAI 的自回归框架的兼容性，这样人们就可以非常容易地将我们的扩散语言模型插入到他们现有的应用或 IDE 中，并且可以无缝使用。你只需要更改 API 密钥，一切都能正常工作。所以我们基本上仍然使用 reasoning effort 参数来控制底层使用的算力。但理论上，你可以考虑其他暴露这个旋钮的方式。只是目前市场已经非常成熟，因此对我们来说，向后兼容非常重要，这样我们的客户就可以非常快速地将他们之前使用的任何模型切换为扩散语言模型。人们很容易尝试我们的模型，看看它们有多快。所以这是我们做出的设计选择，因为它使我们更容易将扩散语言模型推向市场。

It's a design choice. To some extent, it's a choice that we, as developers, made to figure out how to expose this functionality to the user. Right now, our Mercury models allow you to select different kinds of efforts essentially. We tried to keep it compatible with the existing autoregressive OpenAI-like frameworks so that it's very easy for people to plug in our diffusion language models into their existing apps or IDEs, and they can just be used seamlessly. You just need to change the API key, everything works. So we still basically use the reasoning effort parameter to control how much compute is used under the hood. But potentially, you could think about alternative ways of exposing that knob. It's just that there is already a very well-developed market right now, and so for us, it's very important to be backwards compatible so that our customers can very quickly switch out whatever they were using before to a diffusion language model. It's very easy for people to try our models and see how fast they are. So that was a design choice we made because it makes it easier for us to go to market with diffusion language models.

除了我们讨论过的速度和成本这些指标之外，还有质量。

Beyond speed and cost, which are these metrics that we've talked about, there's also quality.

你是给用户提供全部三个，还是他们需要做出取舍？他们在哪里做出取舍？你怎么知道，或者他们怎么知道取舍是什么？还有，你最强的证据是什么，证明速度提升在真实生产负载下能保持可接受的质量？

Uh and are you giving the user all three or are they sacrificing? Where are they sacrificing? How do you know or how do they know what the sacrifices are? And you know, what's the strongest evidence you have that the speed gains survive under real production load at an acceptable quality?

是的，这是个好问题。我认为归根结底，考虑 LLM 时有三件事很重要：质量、速度和成本，对吧？这三者之间总是需要权衡。实际上，你可以绘制现有 LLM 在这三个维度上的位置。如果你去 Artificial Analysis 或者看那些对 LLM 进行基准测试的提供商，你会发现它们根据能力、成本、价格、速度和质量来评估。所以我们用这套现有方法对我们的模型进行了基准测试。当然，测量速度很容易，测量成本也很容易。质量总是最难的那个，对吧？比如，一个模型比另一个好意味着什么？要真正衡量质量是非常棘手的。但通常的做法是，有一系列已建立的基准测试，试图衡量人们关心的东西，比如编码能力、问答、指令遵循、工具使用等。所以我们做的就是将这些现有基准测试上的质量进行比较。例如，我们实际上把模型交给了 Artificial Analysis。他们做了独立评估，用他们自己的一套基准测试来得出一个智能分数，这正是一个质量指标。它基本上是在看不同模型在这些反映真实用例的基准测试上的能力对比。结果再次表明，我们最新的扩散语言模型 Mercury 2 在质量上与前沿实验室的速度优化模型相当，比如 Haiku、Mini、Flash 模型，但速度显著更快，根据对比对象不同快 5 到 10 倍。所以最大的限制是它并非最高质量。如果你有一个工作负载需要最智能的模型，比如最新的 Opus 模型或 Google Gemini Pro 的最新 Pro 模型，我们还没有达到那个质量水平。所以我们还没有训练出能与前沿实验室最佳模型质量匹配的扩散语言模型。这目前是主要限制。我们已经证明可以在速度优化模型层面移动质量与速度的帕累托前沿，但我们需要做更多工作来持续提升模型质量，训练更大的扩散语言模型，使用更多数据，找到更好的训练技术来缩小差距。到那时，我们将拥有真正非常有价值的东西。

Yeah, that's a great question. And I think it boils down to, there are three things that matter when you think about LLMs. It's quality, speed, and cost, right? And it's always a trade-off between those things. And there is, you can actually plot this, where existing LLMs stand in terms of these three things. And I know that's what you find if you go on Artificial Analysis or you look at providers that are benchmarking LLMs in terms of capabilities, cost, price, speed, and quality. And so we've benchmarked our models using this existing methodology. Of course, measuring speed is easy. Measuring cost is also easy. Quality is always the hard one, right? Like what does it mean that a model is better than another one? It's all very tricky to actually measure quality in a good way. But the way it's usually done is, there are a number of benchmarks that have been established that try to measure things that people care about, like coding ability, question answering, instruction following, tool use. And so what we do is we basically just compare the quality of our models on these existing benchmarks. And we've actually given our models, for example, to Artificial Analysis. Artificial Analysis did their independent evaluation. They tried the model on a bunch of benchmarks that they use to come up with their own intelligence score, which is exactly like a quality metric. It's basically trying to see how different models compare in terms of their capabilities on these benchmarks, which reflect real-world use cases. And again, the result is that our latest diffusion language model, Mercury 2, is comparable in quality, about the same as the speed-optimized models from Frontier Labs. So Haiku's, Mini, Flash models, but significantly faster. 5-10x faster depending on which one you compare against. So the big limitation is that it's not the highest possible quality. So if you have a workload where you want to have the most intelligent model, like the latest Opus model or the latest Pro model from Google's Gemini Pro or something like that, we are not at that quality level. So we have not yet trained a diffusion language model that matches the quality of the best models from Frontier Labs. That's kind of like the key limitation at the moment. So we've been able to show that we can shift the Pareto frontier of quality versus speed at the level of the speed-optimized models from Frontier Labs, but we need to do more work to basically keep increasing the quality of our models, train bigger diffusion language models, use more data, figure out better training techniques to close that gap. And at that point, we would have something really really valuable.

你在实证比较这些模型时，是否发现生成的类型之间存在任何定性差异？

Do you find when you're comparing these models empirically, do you find any qualitative differences between the types of generations that you see?

我们从用户和客户那里听到一些传闻，说确实感觉不同。但这又很难量化。我的意思是，你可以用基准测试作为衡量模型表现的好方法。有一些任务我认为基本上什么都可以。比如，想想自动补全或编辑建议，这类任务直觉上，如果你在编辑器里做自动补全，你真的很想利用左右两边的上下文。确实，我们看到扩散模型在这方面表现得非常好。所以有一个叫 Copilot Arena 的东西，有点像代码生成模型的 LLM Arena，那里会有一个 Elo 分数来评估代码生成。它实际上是一个 IDE，开发者会看到两个模型的自动补全建议，他们不知道模型是什么，然后给它们排名，哪个更好。在扩散语言模型生成的补全质量方面，我们排名第一。速度非常非常快。所以我们现在看到大量使用。我们的模型已经嵌入到许多 IDE 中，比如 Continues 和其他一些，还有 Kilocode，开发者们真的很喜欢这种体验，喜欢我们生成的质量和速度。

We heard anecdotally from our users and customers that yeah, it does feel different. But it's hard to quantify again. I mean, you can use the benchmarks as a good way to measure how well these models do. There are some that I think are basically anything like tasks. So if you think about autocomplete or edit suggestions, that's the kind of task where intuitively, you really want to be able to use context to the left and to the right if you're doing autocomplete in an editor. And indeed, we're seeing that diffusion models do really really well. So there is this thing called Copilot Arena, which is kind of like the LLM Arena for code generative models, where there's basically an Elo score that they come up with for code generations. So it's literally an IDE, and developers get to see autocomplete suggestions from two models. They don't know what the models are, and then they rank them, which one is better. And we are at the top of that ranking in terms of the quality of the completions that you get from a diffusion language model. It's really, really fast. So, we're seeing a lot of usage right now. Our models are already embedded in a number of IDEs, Continues, a bunch of others, Kilocode, and developers are actually loving the experience, the quality of the generations that we get, and the speed at which we can provide.

相对于传统模型，扩散模型在哪些方面存在困难？假设在一致的层级上，比如将更小、更快的模型与 Mercury 2 比较？比如长程连贯性或大海捞针？

Are there areas where diffusion struggles relative to traditional models, granted at a consistent tier like comparing the smaller, faster models to Mercury 2? Like long horizon coherence or needle in a big haystack?

是的，我们的 Mercury 2 模型有 128k 上下文。所以如果你有一个任务可能需要更多，那可能不是最佳用例。再说一次，我不认为这是扩散语言模型的根本限制。只是我们还没有训练更长上下文的模型。我们还不是多模态的。所以这是目前另一个限制。如果你有一个任务需要视觉输入，或者考虑音频、输出图像和视频等多模态内容，我们目前还不支持这些功能。我的意思是，没有技术上的根本原因我们不能做，只是我们还没有时间训练多模态模型。

Yeah, so the context that our Mercury 2 model has is 128k context. So that could be, if you have a task where you maybe need more than that, that's again probably not the best use case. Again, I don't think it's a fundamental limitation of diffusion language models. It's just like we haven't trained models with longer context. We're not multimodal yet. So that's another limitation at least right now. So if you have a task where you need vision inputs or you're thinking about audio or outputting images and video or something multimodal, we yet do not yet support those kind of functionalities. I mean, there's no technical fundamental reason we cannot do. It's just like we didn't have the time to train multimodal models yet.

在扩大规模方面，对你来说是什么样子的？关键障碍和步骤是什么？

And in terms of getting to larger scale, like what does that look like for you, and what are the key impediments, steps, that kind of thing?

是的，这是一个过程。这是一项新技术，所以很多时候我们仍然需要重新发明新东西，在最大规模上做所有研发没有意义。所以如果我们在中小型模型规模上尝试我们的想法和方法，迭代会快得多。

Yeah, it's a process. So it's a new technology, and so a lot of the time we still have to reinvent new things, and it doesn't make sense to do all the R&D at the largest possible scale. So we can iterate much more quickly if we try out our ideas, try out our methods at small to medium scale model size just because iteration is faster.

所以，在我们能直接说“好，我们只管扩大规模”之前，还有很多研发工作要做。但根本上，还有一些科学问题需要解决，当然也有工程问题。数据和参数每增加 10 倍，都会带来大量工程挑战，通常意味着你必须大幅改变基础设施，因为之前规模下没有出现的问题，到了下一个规模就变得重要了。在这个过程中，我们学到了很多关于扩展到更多 GPU 和更大数据集的知识，以及各种工程和基础设施问题。技术风险不大，只是需要时间在内部找到解决方案。你能谈谈一些开放的科学问题吗？

And so, there's still a lot of R&D to be done before we can just say, 'Okay, let's just scale up.' But fundamentally, there are some science questions that still need to be solved, and there's engineering, of course. Every 10x in data and parameters comes with a lot of engineering challenges, and it often means that you have to change a lot of the infrastructure because new problems appear at the next scale. As we go through this process, we're learning a lot about scaling up to much larger numbers of GPUs and bigger datasets, and various kinds of engineering and infrastructure problems. There's not a lot of technical risk, it just takes time to figure out how to come up with a solution internally. Can you talk about some of the open science questions?

是的，关于训练这些模型的最佳方式，仍然非常开放。去噪过程是什么？有很多选择。我们有一些可行的方法，但可能有更好的。如果考虑推理，扩散语言模型的有趣之处在于训练和推理是解耦的。在自回归模型中，你训练来预测下一个词，推理时只能反复使用相同的过程。在扩散语言模型中，你实际上是在求解一个微分方程来生成样本。至少对于图像和视频生成，有很多方法可以加速采样，比如花哨的 ODE 求解器或随机微分方程求解器。这些技术已被移植到机器学习中，为传统的连续扩散模型带来了非常快速、高质量的采样算法。离散语言扩散语言模型的空间仍然是“狂野西部”。没有人知道最佳做法。从架构上看，我认为还有很多可以改变的地方。如果考虑强化学习，使用扩散语言模型进行强化学习的正确方式是什么？即使在传统的图像和视频模型背景下，仍然有很多开放的研究。通过扩散过程整合信息的最佳方式是什么？最有效的方式是什么？我仍然通过我在斯坦福的实验室参与一些研究项目，与 Nvidia 合作训练大型视频模型 Cosmos。我们一直在努力找出这些已经存在六七年、更标准的扩散模型的强化学习正确配方。语言、离散空间仍然远未成熟，所以那里也有很多研究要做。

Yeah, it's still pretty open in terms of what is the best way to train one of these models. What is the denoising process? There are many choices there. We have some things that work, but there could be better ones. If you think about inference, the interesting thing about diffusion language models is that training and inference are decoupled. In an autoregressive model, you train to predict the next token, and at inference time, the only thing you can do is to basically reuse the same process over and over. In a diffusion language model, you're essentially solving a differential equation to generate samples. At least for image and video generation, there are many methods to accelerate sampling, like fancy ODE solvers or stochastic differential equation solvers. Those techniques have been ported over to machine learning and led to very fast, high-quality sampling algorithms for traditional continuous diffusion models. The space of discrete language diffusion language models is still the Wild West. Nobody knows the best way to do things. Architecture-wise, I think there's still a lot that can be changed. If you think about RL, what is the right way to do RL using a diffusion language model? Even in the context of traditional image and video models, there's still a lot of open research. What is the best way to incorporate information through the diffusion process? What's the most efficient way? I'm still involved through my lab at Stanford in some research projects, some collaborations with Nvidia where we're training big video models, Cosmos. We've been working on figuring out the right recipe for RL on these more standard diffusion models that have been around for six or seven years. The space for language, discrete, is still much less mature, so there is a lot of research to be done there as well.

而且，因为最终基于 Transformer 模型，传统训练的大语言模型的所有限制在扩散模型中是否类似？比如幻觉就是一个例子。

And presumably, because you're ultimately based on transformer models, all of the limitations of traditionally trained LLMs are similar in diffusion models. Is that the case? Hallucination is one that comes to mind, for example.

是的，幻觉问题确实存在。我认为这不一定是架构的问题。我认为这仅仅是拟合统计模型时的基本问题。有数据，你拟合一个统计模型。存在一个插值区域，你得到的答案可能是可靠的。但总会有外推区域，那时就会出现错误。我认为这在某种程度上是不可避免的，无论它是扩散模型还是自回归模型，无论架构是什么。我们从有限的数据中学习。我们需要这个模型进行泛化，但没有人真正理解深度学习中的泛化是如何工作的。即使对于分类问题，非常聪明的人也花了大量时间试图理解深度学习中的泛化机制。进展非常有限。直到今天，还没有预测性理论能告诉你“这个神经网络会泛化吗？”一切都是经验性的。你必须尝试，然后看它是否有效。但没有任何在人们关心的合理尺度上具有预测性的理论，能告诉你神经网络在实践中表现如何，即使对于分类问题也是如此。对于生成模型，情况更糟。这个问题从根本上来说应该是无法解决的，对吧？存在维度灾难。有一些很好的论据说明为什么这些模型所做的应该是不可能的，但它们却成功了。所以，我觉得从科学角度来看，在理解这些模型如何工作、为什么工作以及在什么条件下会工作方面，还缺少一些根本性的东西。这仍然非常、非常开放。

Yeah, so hallucinations, yes. I think it's not necessarily an issue with the architecture. I think that's just a fundamental issue whenever you fit a statistical model. There is data, you're fitting a statistical model. There is a regime where you're going to be interpolating, and maybe the answers you get are reliable. And then there's always going to be a regime where you're extrapolating, and at that point, there are going to be mistakes. I think that's to some extent unavoidable, no matter whether it's a diffusion or autoregressive model, no matter what the architecture is. We're learning from limited data. We need this model to generalize, and nobody really understands how generalization in deep learning works. Even for classification, very smart people have spent a lot of time trying to understand how generalization works in deep learning. The progress has been very limited. To this day, there is no predictive theory that can tell you, 'Will this neural network generalize?' It's all very empirical. You have to try and see if it worked or not. But there is no theory at any reasonable scale that people would care about that is predictive and would tell you how well a neural network would work in practice, even for classification. For generation, generative models, it's even worse. It's a problem that fundamentally should be impossible to solve, right? There is a curse of dimensionality. There are some pretty good arguments for why what these models are doing should not be possible, yet they work. So, I feel like there is something fundamentally missing from a scientific point of view in terms of understanding how these models work, why they work, and under what conditions they will work. It's still very, very open.

那么可解释性或模型估计自身不确定性的能力呢？在这些核心维度上，扩散模型或基于 Transformer 的模型是否存在任何根本性差异？

And how about things like explainability or the model's ability to estimate its uncertainty? Are there any fundamental differences either to the benefit of diffusion models or to transformer-based models in these core dimensions?

是的，我们没有探索太多可解释性，我也不期望有特别的差异，因为这是另一个领域，一旦你开始使用深度网络，我个人对整个可解释性研究方向持怀疑态度。所以，我们目前没有在这方面投入。我认为一个令人兴奋且实际相关的方向是可控性。在这个领域，人们确实关心能够控制这些模型的输出，通常通过提示词或末尾的一些护栏来实现。对于自回归模型，有一整套你能做和不能做的事情。扩散模型，至少对于图像来说，被认为更适合可控生成。

Yeah, so we've not explored much interpretability, and I would not expect particular differences in the sense that it's yet again one of the spaces where if you start using deep networks, I'm personally pretty skeptical about the whole interpretability research direction. And so, that's not something that we've invested in at the moment. One interesting direction that I think is actually exciting, and it's also practically relevant, is controllability. That's a space where people do care about being able to control the outputs of these models, and usually that's done through a prompt, maybe some guardrails at the end. There is a certain stack and a certain set of things you can and cannot do with an autoregressive model. A diffusion model, at least for images, is known to be much more suitable for controllable generation.

原因在于，因为目标对象——比如你正在生成的图像——从一开始就对模型可见，模型很容易检查它生成的对象是否与某些约束或控制信号一致，从而确保输出符合你的要求。你不仅可以检查它是否满足条件，还可以引导生成过程，使其与这些外部约束保持一致。这之所以可能，是因为你从一开始就拥有完整的对象，而不是逐词生成，那样只能到最后才能检查是否满足约束。这就是为什么扩散模型被广泛用作解决医学成像中逆问题的先验。有很多应用场景中，这种通过外部信号控制输出的能力非常重要。我参与过一些医学成像的论文，其思路是：当你做 CT 扫描时，你实际上是在获取身体横截面的投影，然后根据机器测量的数据重建身体图像。测量数据越多，重建质量越高，但这也意味着患者接受的辐射更多，对吧？但如果你有一个好的身体外观先验模型——可以由扩散模型提供——那么你就可以强制模型说：“生成一个看起来像真实人体的东西，但同时要与我从这位患者身上获得的测量数据一致。”这可以显著减少达到相同质量所需的测量次数，从而减少患者的辐射。还有很多类似的问题，扩散模型表现得非常好。那么，如何将这种方法应用到文本上呢？这方面也有一些工作，但我认为这会非常令人兴奋，对吧？因为人们关心的是能够保持品牌一致性或满足安全约束。在很多场景中，你确实希望能够控制模型的输出。所以我认为这是一个令人兴奋的可能性，这是扩散语言模型独有的。

And the reason is that because the object, let's say the image that you're generating, is available to the model from the very beginning, it's very easy for the model to check whether or not this object that it's generating is consistent with, say, some constraints or some kind of control signal that you want to use to make sure that the output is consistent with whatever you want the model to generate. And not only you can check whether it matches your conditions, but you can also steer the generation process in a direction that makes it consistent with these external constraints. And that's only possible because you have the object from the very beginning, the full object, as opposed to generating it token by token, where you can only check whether or not it satisfies the constraint at the end. And so that's why diffusion models have been used a lot as priors for solving inverse problems in medical imaging. Like there is a lot of applications where this ability of controlling the output through some external signal has been really really important. So I was on some papers where we're doing medical imaging and the idea is that when you do a CT scan, you're basically taking some projections of your body cross-section and then you're trying to reconstruct what your body looks like from some measurements that you get from the machine. And the more measurements you get, the higher the quality of the reconstruction, but it also means more radiation for the patient, right? But if you had a good prior model of what the body looks like, which can be given by a diffusion model, then you can kind of force the model to say, "Okay, produce something that is likely to correspond to an actual human body, but it's also consistent with these measurements that I'm getting for this particular patient." And that can significantly reduce the number of measurements that you need to take for the same quality level, which means less radiation for the patients. And there are a number of problems that have that flavor where diffusion models have been really really good. And so now, how to do that for text? There's some work again there, but I think it'd be pretty exciting, right? Because people care about being able to stay on brand or be of course safety constraints. Like there's a bunch of settings where you do want to be able to control the output of the model. And so I think that's an exciting possibility that is pretty unique to diffusion language models.

展望未来，你的时间线是怎样的？也许我应该问得更开放一些。你最终认为扩散模型会在前沿规模上挑战自回归模型吗？

Looking forward, what's your mental timeline for — maybe I should even ask this in a more open-ended way. Do you ultimately see diffusion challenging auto-regressive models at frontier scale?

是的，是的。我认为这是我们的赌注。我认为没有理由不行。我不知道我们需要多长时间才能达到那个水平。我猜挑战在于前沿在不断移动。如果你告诉我，这就是前沿，你需要多长时间达到？我大概能给出一个合理的估计。问题是它一直在变，对吧？模型越来越好，速度越来越快。所以很难预测需要多长时间。而且，还有大量的研发工作，不幸的是，这有很多风险，但也有很大的上升空间。完全有可能我们提出一种比现有算法好得多的新算法，这可能会大大加速进展。尤其是因为扩散语言空间还非常未被探索。我认为还有很多唾手可得的成果，很多改进空间，很多比我们当前做法好得多的解决方案。所以很难预测需要多长时间。也很难预测如果我们扩展到那些规模，效果会如何。令人兴奋的是，不太可能一种架构完全主导另一种。所以最好的情况是短时间。扩散模型更好，每个人都会切换，它将成为未来其他实验室的架构。即使没有发生，也一定会有一些用例，比如延迟敏感、设备端。会有一些用例，替代架构会更好。而且市场会非常大，即使最坏的情况对我们来说也相当不错，对吧？因为会有很多其他实验室的用例，只要我们能赢得其中合理的一部分，那仍然非常有价值。

Yeah, yeah. I think that's our bet. I think there is no reason it shouldn't. I don't know how long it's going to take us to get there. And I guess the challenge a bit is that the frontier keeps moving. Like if you tell me, this is the frontier, how long do you need to get there? I think I would probably come up with a reasonable estimate for that. The problem is that it keeps shifting, right? And so the models keep getting better and the speed keeps accelerating. So it's hard to predict how long it's going to take. And again, there is still a lot of R&D, unfortunately, which has a lot of risks, but also a lot of upside. Like it's entirely possible that we come up with a new algorithm that is way better than what we have. And so that could accelerate progress by a lot. Especially because the diffusion language space is very very unexplored. I think there is still a lot of low-hanging fruits, a lot of room for improvement, a lot of room for wildly better solutions to what we're currently doing. So it's hard to predict how quickly it's going to take. It's hard to predict how well it's going to work if we were to scale up to those sizes. What's exciting is that it's unlikely that one architecture is going to dominate the other one. So maybe the best-case scenario is short. You know, diffusion models are better, everybody will switch, that will become the architecture for other labs in the future. Even if that doesn't happen, there's got to be some use cases, latency sensitive, on device. Like there's going to be some use cases where an alternative architecture is just going to be better. And it's going to be such a big market that even the worst-case scenario is actually pretty good for us, right? Because there's going to be just so many use cases of these other labs that as long as we can win on a reasonable subset of them, that's still going to be extremely valuable.

是的，当我想到延迟敏感时，最先想到的是语音交互。但然后我想到围绕智能体的所有活动，以及它们如何运行循环。任何时候你有循环，如果你能压缩一次循环的时间，那就会产生复合效应。你们在扩散模型和智能体应用方面做了很多工作或看到了很多吗？它们现在足够强大，可以用于智能体吗？

Yeah, when I think about latency sensitive, the things that come to mind most immediately are things like voice interactions. But then I think about all the activity around agents and how they're running a loop. And anytime you have looping, if you can compress the time for one run through that loop, then that compounds. Are you doing a lot or seeing a lot with regards to these diffusion models and agentic applications? Are they powerful enough to be used in agents now?

绝对是的，是的。我们已经看到了很多使用。我是说，你准确指出了我们看到的两个主要领域。语音和大量的语音客户支持、教育类智能体。人们喜欢扩散语言模型的速度。他们一直有个问题，就是他们想使用思考模型，比如推理模型，但通常延迟不够。所以除非他们使用专门的 AI 推理芯片，但那太贵了，无法大规模扩展。所以我们有一批客户在扩散语言模型之上构建语音智能体。智能体是另一个，就是通用智能体。例如，Mercury 在 open claw 中实际上表现很好。所以是的，你可以使用它，你可以插入它。它已经——是的，你也可以用它来编码、客户端、杀手级代码。所以它什么都能做，它可以使用工具，可以推理，而且非常快。所以特别是，如果你想着“好吧，我让它运行 24 小时，然后回来看它是否解决了我的问题”，那它可能不是最好的模型。

Absolutely, yes, yes. So we're already seeing a lot of usage. I mean, you nailed the two main ones that we're seeing. Voice and a lot of voice customer support, educational kind of like agents. People love the speed of the diffusion language models. They always had this issue that they would want to be able to use a thinking model, like a reasoning model, but usually the latency is just not enough. And so maybe they use unless they use a specialized AI inference chips, but that's too expensive and it cannot scale to large volumes. So we had a bunch of customers that are building voice agents on top of diffusion language models. And agents, that's another one, just general agents. Mercury works actually pretty well in open claw, for example. So yeah, you can use it, you can plug it in. It's already — yeah, you can use it also for coding, client, killer code. So it is all, you know, it can use tools, it can reason, and it's really quick. So especially kind of like it's not the best model if you're thinking of, "Okay, I'm going to let it run for 24 hours, I'm going to come back and see where it solves my problem." Then it's probably not a good use case for that.

但如果你考虑快速的智能体式交互和循环，其中你实时参与并希望快速得到答案，且有人类在回路中，那么这是一个非常好的模型，因为正如你所说，它明显更快，你可以更快地迭代，最终在更短的时间内得到最终结果，这才是开发者真正关心的。去年在 Google IO 上，Google 宣布并预览了他们在该领域的布局。从那以后我没看到太多进展。你追踪过他们的动向吗？能给我们总结一下吗？

But if you think about fast agentic interactions and loops where you're actually there and you want to get an answer quickly and there is a human in the loop, then it's a really good model because as you said, it's significantly faster, you can iterate more quickly, and so eventually you can get to the final result in less time, which is the thing that actually matters to developers. I think last year at Google IO, Google announced and kind of previewed their play in the space. I haven't seen much of it since then. Have you tracked what they've been up to? And can you give us a summary?

是的，我没有任何内部消息，但正如你所说，他们在我们发布第一个 Mercury 模型几个月后，也宣布了一个扩散语言模型 Gemini Diffusion。所以我愿意认为这可能产生了一点影响，促使他们展示自己也有成果。我认为他们当时发布的数据与我们的初始 Mercury 1 模型非常接近。所以我不知道他们是否有所改进。我所知道的是，它还没有投入生产，也没有对客户开放。所以我猜他们可能还没搞清楚，或者仍在研究如何高效提供服务以及最佳用例。我的感觉是存在很大的转换成本。他们非常专注于 Gemini 和他们的主要模型。所以这可能是这些大型实验室的问题：他们只朝一个方向前进，很难真正专注于另一个方向。作为初创公司，我们更有优势做到这一点，因为我们专注于一件事，能够真正交付并构建成功所需的一切。但在大公司或大实验室里，通过那些大型展示会有点困难。我认为他们已经确定了方向，想要转换会有很大的机会成本。

Yeah, so I mean, I don't have any inside information in terms of what they're doing, but as you said, they also announced a diffusion language model, Gemini Diffusion, a few months after we announced our first Mercury model. So I'd like to think that maybe that had a little bit of influence and impact there, pushing them to actually show they have something too. I think what they published back then, those numbers were very comparable to our initial Mercury 1 model. So I don't know whether they've been able to improve. What I know is that it's not yet in production and not yet available to customers. So I'm guessing they've maybe not figured out or they're still working on figuring out how to serve it efficiently and what the best use cases are. My sense is that there is a big switching cost. Like they're very focused on Gemini and their main model. So it could be that that's the issue with these big labs: they're only going in one direction and then it's hard for them to really focus on an alternative direction. As a startup, we're much better positioned to do that because we are laser-focused on one thing and we can really deliver and build everything that's needed to get that technology to succeed. But it's a little harder through these big displays, I think, in a big company or big lab. I think they already have their direction set, and there's a big opportunity cost if you want to switch.

你关注哪些其他实验室或团队（学术或工业界）在做扩散方面的有趣工作？

What other labs or teams, academic or industry, do you kind of keep an eye on for doing interesting things with diffusion?

是的，中国有很多优秀的工作，比如 LLaDA 模型。这些是几所中国大学与阿里巴巴合作的项目。他们从工业界获得了大量算力和资金。他们在思考模型、架构以及如何训练方面做得很好。这些 LLaDA 模型与我们内部模型之间仍有巨大差距，但他们在研究和推动领域发展方面做得很好。字节跳动内部有非常认真的努力。他们也发表了几篇关于内部构建的扩散语言模型的论文。字节跳动 Seed（他们内部的基础研究组）有很多聪明人和优秀研究员。他们在这个领域也做得很好。总的来说，整个学术界有很多有趣的论文。我去年 12 月参加了 NeurIPS，看到有多少关于扩散语言模型的论文，真是疯狂。如果你画个图，会发现自从我们团队 2024 年的那篇原始论文以来，出现了爆炸式增长。现在每个人都在关注这个新范式。当然，这很令人兴奋，因为有一种方法对图像、视频和音乐非常有效，而另一种方法对文本和代码很有效。那么什么会是最终的解决方案？有没有办法统一所有东西，得到一个在所有模态上都表现良好的单一生成模型？每个人都对 LLM 感到兴奋，但令人惊讶的是，前沿实验室的所有不同模型都非常相似。它们几乎都是彼此的克隆，差异很小。所以现在有了一种替代方法，一条替代路径。这在研究界引起了很大兴奋，因为这是一个做新事情并影响该方法概念基础的机会。

Yeah, there is a lot of good work coming out from China, like the LLaDA models. These are several Chinese universities collaborating with Alibaba. So they get a lot of compute and funding from industry. And they're doing good work in terms of thinking about models, architectures, how to train them. There's still a huge gap between these LLaDA models and what we have internally, but they've been doing good work in terms of doing research and pushing the field forward. ByteDance has pretty serious efforts internally. They've also published a few papers with some internally built diffusion language models. ByteDance Seed, which is kind of their fundamental research group within ByteDance, has a lot of smart people and good researchers. They've been doing good work in this space too. And then generally, the whole academic community, there are a bunch of interesting papers coming out. I was at NeurIPS in December and it was crazy to see how many papers are there on diffusion language models. If you were to plot it, you'd see there's been an explosion since that original paper from my group in 2024. Now everyone is looking at this new paradigm. Of course, it's exciting because there's this approach that works really well for image, video, and music, and then there's this other approach that works well for text and code. And then what's going to be the winning solution? Is there a way to unify everything and have a single generative model that works well across all modalities? Everyone is excited about LLMs, but it's surprising how similar all the different models from frontier labs are. They're all kind of clones of each other, with very little differences. So now there is an alternative approach, an alternative path. That is generating a lot of excitement in the research community because it's an opportunity to do something new and have an impact on the conceptual foundations for this approach.

你认为图像和文本是两条不同的路径，还是可以从图像方面借鉴技术来促进文本方面的工作？

Do you see image and text as these two divergent paths, or are there techniques from the image side that you can pull over to facilitate your work on the text side?

是的，我认为有很多交叉借鉴。我自己就是从图像开始研究的。我们团队中的很多研究人员，因为当时研究语言扩散的人不多，实际上最初是纯粹的图像扩散或视频扩散研究人员。所以很多专业知识确实很好地转移了过来。因此，我们也密切关注那个社区在蒸馏、加速模型的技术以及使扩散模型更快的推理技巧方面的进展。所有这些进步都非常令人兴奋。

Yeah, there is a lot of cross-pollination, I would say. I myself started out working on images. A lot of the researchers in our team, because there were not many researchers working on diffusion for language, actually started out as pure diffusion for images or diffusion for video researchers. And so, a lot of the know-how did indeed transfer reasonably well. So, we also pay close attention to what's happening in that community in terms of distillation, techniques to accelerate the models, and inference tricks to make diffusion models go even faster. All those advances have been pretty exciting.

在 Inception 或其他地方，是否有工作指向基于扩散的可靠多模态方法？

Is there any work happening either at Inception or elsewhere that points to a credible multimodal approach based on diffusion?

在 Inception，我们还没有优先考虑多模态，但在学术界，过去一年左右有很多论文，包括我的一位联合创始人 Aditya（他曾是博士生，现在有自己的实验室）的工作。他在展示如何构建真正多模态的扩散模型方面做了非常出色的工作。所以在学术界，基于扩散的统一模型处理不同模态方面取得了一些非常好的成果。

So, at Inception we've not been prioritizing our multimodal yet, but in the academic community, there have been a number of papers over the last year or so, including from one of my co-founders, Aditya, who was a former PhD student with his lab. He's done some really good work in terms of showing how to build diffusion models that are truly multimodal. So there have been some really good results in the academic space on getting that unifying model based on diffusion that can handle different modalities.

好了，Stefano，很高兴和你交流，全面了解了文本扩散。我现在感觉跟上了。非常感谢你参与并分享你正在做的工作。

Well, Stefano, it's been great catching up with you and getting the complete download on text diffusion. I feel caught up now. Thanks so much for jumping on and sharing a bit about what you've been working on.

是的，非常感谢你邀请我。这次聊天非常愉快。

Yeah, thank you so much for hosting me. That was really fun chat.

一样。谢谢。

Same. Thank you.