302 个神经元的线虫启发了他们的架构——Ramin Hasani，Liquid AI

好的，我们在远程演播室，有新来的编辑主管 Richard。打个招呼吧。

Okay, we're in a remote studio with our new head of editorial, Richard. Say hi.

大家好。

Hi, everyone.

从英国连线。还有 Ramin，Liquid 的 CEO。欢迎。

Dialing in from the UK. And Ramin, CEO of Liquid. Welcome.

谢谢邀请。

Thanks for having me.

大多数人听说 Liquid 时，他们知道 LFM 或听说过 LFS。他们可能没有亲自尝试过。另一个有趣的公开数据点是，它非常受蠕虫 DNA 或类似东西的启发。我从未真正关注过这一点，因为显然我认为那是一个类比，但你可以告诉我们该多认真地对待它。

Most people when they've heard about Liquid, they know about LFM or they've heard of LFS. They maybe haven't personally tried it. The other interesting public data point is that it is very inspired by worm DNA or something. I never really pay much attention to that because obviously I think that that's an analogy but you can tell us how seriously to take it.

是的，当然。你知道，大约 12 年前我们开始研究时，我们想看看能否将大脑的灵感带回机器学习。所以一切就是这样开始的。然后当我们开始研究人脑时，我们意识到——当我说我们时，指的是我现在的 CTO 和我自己——我们最初在维也纳科技大学，简称 TUNI。然后我开始读博士，我的联合创始人当时是那里的硕士生。我们想理解如何将连续时间数学引入机器学习系统。这就是我们所做的。所以我们基本上研究神经元，神经元如何交换信息。这由非常简单的 一阶微分方程控制，这就是神经系统动力学在神经元和突触层面的实际工作方式。然后我们的第一个重点是想从人脑开始，但人脑有脉冲神经元，实际上很难学习，我们仍然对人脑一无所知，比如竞争是如何发生的。我们现在对果蝇的大脑了解得更多，但对人脑了解不多。但我可以告诉你，我们之所以研究这种动物，是因为我们想从第一性原理出发，我们找到了这种动物——秀丽隐杆线虫，它体内只有 302 个神经细胞，大约 2 毫米长。这种蠕虫的一个好处是它的神经元行为非常类似于人工神经元，它们不发放脉冲。你知道，这种动物神经元的这种分级行为，在神经科学中被称为神经元的电紧张行为。神经元不发放脉冲。这种行为对我们来说非常有趣，因为如果你构建受这种蠕虫大脑启发的系统，你就能构建非常连续时间的系统，而且它们是可微的。所以你可以进行反向传播和其他各种操作。这就是我们当时研究的模式生物。最让我们着迷的是，用 302 个神经元，它能控制 95 个肌肉细胞，比我们地球上任何机器人系统都要好，你知道，这是 2016 和 2017 年。然后我们想，好吧。关于这种蠕虫的另一个有趣的事情是，6 亿年前，在进化树上，人类实际上是从这种蠕虫分裂出来的。所以神经科学家发现的关于这种蠕虫大脑或细胞结构的任何东西，实际上在一定程度上也可以转移到人类身上，与人类基因组有 78% 的相似性。所以它实际上是一种非常重要的蠕虫，到目前为止，它已经为人类赢得了四项诺贝尔奖。所以，你可以想象，科学家们真的很幸运能研究这种动物。所以，它是一种非常非常受欢迎的动物，这就是我们的研究。我们甚至开始与研究这种神经元大脑的神经科学家合作。长话短说，我们基于我们所知道的灵感和关于蠕虫大脑的知识，数学建模了两个神经元如何交换信息。然后我们得到了这些数学，并构建了称为液体神经网络的东西——液体时间常数神经网络。Liquid 就是基本的液体，代表灵活性，因为我们意识到的一件事是，这些是在前向传播中由微分方程控制的高度非线性系统，而且它们是可微的。所以你可以运行反向传播，你可以很好地学习它们，因为它们有多个反馈结构。它们真的是表示学习器，它们从数据中学习表示非常好。我们意识到的一件事是，用这些神经网络的一小部分，你就可以控制机器人，非常类似于蠕虫本身的做法。当我说一小部分时，我们说的是几十到几百个神经元，非常非常小的子集。然后我们非常兴奋，开始将这些应用于控制机器人。在机器人上你没有那么多算力，所以你能将那么多智能打包进机器人内部以自主控制某种行为，这对我们来说是一个非常有吸引力的特性。所以这种计算的效率以及将智能带入最小的计算单元一直是我们研究的一个主题。然后 2017 年，MIT 的 Daniela Rus 教授——她是我在维也纳的教授联系了她，她也是我的联合创始人之一。她可以说是机器人学之母。她在机器人学、分布式机器人和软体机器人领域做出了巨大贡献。她开始了解我们的研究，然后她被震撼了，她说，你们考虑来 MIT 继续这项研究，在现实世界中真正应用这项技术，比如汽车、无人机和许多不同类型的车辆。我们俩，Matias 和我，加入了 MIT。所以自 2017 年以来，我一直与 Daniela 合作。我遇到了我的第四位联合创始人 Alexander Amini。Alexander 当时是 MIT 的博士生。Matias 和我都在那里，我们四个人自 2017 年以来开始研究这些主题，再次是连续时间动力系统和连续时间学习系统，以及所有这些的 Scaling（规模扩张），将它们应用到许多不同的领域。

Yeah, absolutely. Well, you know like when we when we started uh our kind of research about 12 years ago, we wanted to see if we can bring inspirations from brains back to machine learning. So that's that's where everything got started. And then when we started looking into human brain, we realized like when I say we, my current CTO and myself, we started back in Vienna University of Technology. It's called TUNI. And then so I started my PhD. My co-founder was the a master student there. And then what we wanted to do, we wanted to understand how we can um bring in like continuous time math inside machine learning kind of system. And that's what we did. So so we we basically look into neurons, how neurons exchange information with each other. And then um this was governed by very simplistic f first order differential equations and that's that's kind of how how the nervous system dynamics are actually working at the level of curve neurons and synapses and then uh our first focus we wanted to start with human brain but human brain has like a spiking neurons and and it's very difficult to actually learn like we still don't know anything about human brain how how competition actually happens. We know a lot more uh now about a fruit flies brain but we don't know that much about human brain but uh but I can tell you that um we got to this animal because we wanted to do it first principles we we got to this animal C elegance worm which has uh only 302 nerve cells in its body and it's like it has like two it's 2 mm long and uh one of the nice thing about the worm was the fact that its neurons behave very similar to artificial neurons they don't they don't spike You know, so that graded kind of behavior in these neurons of this animal, the it's called in in neuroscience they call it electroonic behavior of a neuron. You know, like the neurons do not spike. And that behavior was very very interesting for us because if you build systems inspired by the brain of this worm, you would be able to build like very continuous time kind of systems and they're differentiable. So you can you can do back prop and all sort of other things on top of these things. This was this was the model organism that we were looking at at that time. The most fascinating thing for us was that with 302 neurons it could control 95 muscle cells better than any robotic systems that we had on the planet you know and this is 2016 and 2017. So then we thought that okay so and also another thing another interesting thing about the worm is the fact that 600 million years ago we got a split humans actually got split from this worm in the tree of evolution you know so it's actually like whatever whatever neuroscientists are discovering about the brain of this worm or the cell structure of this worm is actually transferable like to to humans as well to some extent 78% similarity to human genome you know so it is actually a very very important uh uh worm for us and so far it has won four nobel prices for humanity, you know. So, like you can imagine like scientists are really lucky to study this animal. So, it's a very very popular animal, you know, and um and that's why like that's our our research. We started even working with neuroscientists, you know, themselves that are working on this uh brain of these neurons. So, long story short, we mathematically modeled how two neurons exchange information based on the inspirations that we know and the things that we know about the brain of the worm. And then we got those mathematics and we built something called liquid neural networks. liquid time constant neural networks. Liquid was like basic liquid for flexibility because one of the things that we realized is that these are highly nonlinear systems governed by differential equations in a forward pass and they are differentiable. So you can actually run them backward pass and you can basically learn them like really nicely because of the multiple feedback structures that they had. They are really representation learners you know like they learn representation from data really really well. One of the things that we realized is that with a small kind of subset of these neural networks, you can control robots like you know very similar to how the the worm itself was doing it and uh when I say small subset we were talking about tens to hundreds of neurons you know like we talking about very very small subset of neurons and then we we we got very excited so we started applying these things to controlling robots on the robot you don't have that much compute so you bring like int the fact that you can actually pack that much intelligence to control autonomously a kind behavior inside a robot that was like a very attractive feature for us. So efficiency of this uh computation and bringing intelligence into like smallest kind of unit of compute that has always became kind of a a kind of research topic for us and then uh 2017 um professor Daniela Roose of MIT cell so she's she's my uh so she she reached out to my professor in Vienna and then she's one of my co-founders as well. She's the mother of robotics arguably. So you know she's like she has done tremendous contribution to the field of robotics and distributed robots and soft robots as a whole. She started learning about our research and then she she was like blown away and she said okay you guys would you guys consider coming to MIT and continue this research in real world like really applying this technology into let's say cars and drones and many many different kind of vehicles and we were like both Matias and I we joined MIT. So since 2017 I've been like um working with Daniela. I met my force co-founder uh uh Alexander Amini. So Alexander is although was was a PhD student at that time at MIT. So Matias and I were both there and the four of us since 2017 started working on these topics of um again continuous time dynamical systems and continuous time learning systems that and and then the scaling of all of these things applying them into many many different domains.

预测性序列模型，比如序列建模，成了我们的专长。我们开始做贡献，然后在连续时间这个领域变得更好一些，因为很多来自不同地方的研究者——比如 Yashu Benji 的实验室、David Duban 的实验室、多伦多大学，还有一些来自斯坦福的人——大家都在为连续时间动力系统这个领域做贡献。想象一下循环神经网络，它们是离散化的动力系统，基本上一次计算一步，但一步到另一步之间的时间差变化不大。但这种连续时间动力系统让你对事件以离散方式发生的时间有一点控制；你也会考虑事件发生之间的时间。所以对于连续演化的序列，你在时间维度上也有更多的适应性。所以这种循环和反馈计算——这就是我们带到机器学习中的直觉。再次强调，不仅是我们 MIT 这边，还有很多其他人同时在做神经 ODE、状态空间模型等主题。你可以想想人们开始给这些动力系统整体贴上的所有其他名字。

Predictive sequence models, like sequence modeling, became kind of our thing. We started doing contributions, and then in this field of continuous time it became a little bit better because a lot of other researchers from different places—like from Yashu Benji's lab, David Duban's lab, from University of Toronto, and then some people from Stanford—everybody was contributing to this field of continuous time dynamical systems. Imagine recurrent neural networks that are discretized kind of dynamical systems that are basically computing one step at a time, but the time difference between one step and the other doesn't change that much. But this type of continuous time dynamical systems allows you to have a little bit of control also over when events are happening in a discrete way; you also account for the time between the events that are happening. So for sequences that are continuously evolving, you have a lot more adaptability in the time dimension as well. So that aspect of recurrence and feedback kind of calculation—this was kind of the intuitions that we brought to machine learning. Again, not only us from the MIT side but also a lot of other people were simultaneously working on this with topics such as neural ODEs, state space models. You can think about all these other names that people started putting on top of these dynamical systems as a whole.

然后在 2020 年左右，每个人都在谈论可扩展性。当时，从 MIT 这边来看，我们从研究角度在与世界上的 OpenAI 和 Google 竞争。我们提出的算法高效，在机器人环境中的新学习场景下表现非常好。如果你记得 2017 到 2020 年间，OpenAI 在强化学习方面非常活跃，也做了很多机器人工作，比如 OpenAI Gym，他们发布的一些东西。所以我们试图在这些连续时间算法控制机器人方面发挥创意，并展示如何真正把学习带到分布之外。我们如何拥有小型的学习系统，同时在分布外泛化上做得更好？这就是整个论点。但后来我们开始思考规模，因为每个人也都开始思考规模。我们想从这种蠕虫启发的智能进化到更复杂的东西，并进入人类可理解的信号，比如音频、视觉和文本作为我们关心的数据语料。一旦我们开始扩展连续时间和循环的高度非线性循环，你就会意识到执行计算有多困难。它在计算上变得非常不可处理。所以计算变得非常困难。你需要发挥创意来解决如何扩展这类系统。循环神经网络有一个主要障碍：顺序计算。当循环算子是非线性时，并行化它们非常困难，尤其是在 GPU 上。这是 Transformer 架构的美妙之处之一，因为它们是无偏的。它们基本上是矩阵乘法，你可以扩展它们——这是硬件彩票。这就是 Transformer 架构在规模和释放通用行为方面的美妙之处。但循环本身如此有吸引力，因为计算不会爆炸。如果你想想注意力，当你消耗越来越多的数据时，注意力的二次成本是人们已经关注的问题。所以我们一直意识到，循环实际上降低了计算量，使其成为亚二次或线性的，随着你消耗更多数据，更接近线性计算。所以这些算法更高效，但这只是理论上的。如果你不能真正扩展并并行执行操作，你就无法扩展这些系统。

Then around 2020, everybody was talking about scalability. At the time, from the MIT side, we were competing with the OpenAIs of the world and the Googles of the world from a research point of view. We were putting algorithms that are efficient and perform really well in new learning scenarios in robotics environments. If you remember between 2017 and 2020, OpenAI was very active in reinforcement learning and also doing a lot of robotics, like OpenAI Gym, for example, some of the things they were putting out. So we were trying to get creative around controlling robots with these types of continuous time algorithms and showcase how you can actually take learning out of distribution. How can we have learning systems that are small but at the same time do a lot better in out-of-distribution generalization? That was the whole thesis. But then we started thinking about scale because everybody started thinking about scale as well. We wanted to evolve from this worm-inspired kind of intelligence to something more complex and get into human-understandable signals, like audio, vision, and text being a corpus of data that we cared about. As soon as we start scaling continuous time and recurrent kind of highly nonlinear loops, you realize how difficult it is to perform computation. It becomes really computationally not tractable. So it becomes very difficult to compute. You need to get creative around how to scale these kinds of systems. Recurrent neural networks have had a major blocker: sequential computation. Parallelizing recurrent operators when they are nonlinear is really difficult, especially on GPUs. This is one of the beauties of the Transformer architecture because they are unbiased. They are basically matrix multiplications and you can scale them—it's the hardware lottery. That's the beauty of the Transformer architecture for scale and for unlocking general-purpose behavior. But recurrence itself is so attractive because the computation doesn't explode. If you think about attention, when you consume more and more data, the quadratic cost of attention is something that people have paid attention to. So we've always realized that recurrence actually brings this computation lower and makes it sub-quadratic or linear, closer to linear kind of computation as you consume more data. So these algorithms are more efficient, but this is only theoretical. If you cannot really scale and do the operations in parallel, you cannot scale these systems.

我们尝试的一件事是，解决这类非线性系统变为线性的一种方法是线性化它们。当我说线性时，你动态地减少系统的所有非线性。这样你就可以在系统上应用线性代数技巧，使系统实际上变得可计算。你可以并行执行操作，线性代数操作。这就是为什么状态空间模型是循环神经网络的线性版本，它们失去了参数之间的非线性关系。你为什么要这样做？因为你想使计算可扩展。所以 SSM 正成为动力系统或学习算法的一种非常流行的版本，因为你现在可以并行化其他形式的操作，比如不仅仅是注意力，你还可以有并行扫描，并且可以应用线性代数技巧来扩展另一种操作格式。线性化这个动力系统让我们得以扩展。所以如果你考虑 SSM 的观点，但问题再次出现，计算机科学中没有免费的午餐。当你线性化一个复杂的动力学时，你失去了表达能力。所以系统不会像非线性形式那样具有表达力。然后我们意识到，当我们应用这些状态空间模型来解决语言问题时，它们在长上下文学习依赖关系方面有困难。它们可以非常高效地计算，但不能非常高效地表达。所以这些神经网络的表达能力总是比不上 Transformer。凭借我们从液态神经网络、动力系统、SSM、卷积神经网络中学到的所有非线性数学，你可以将许多不同的算子变体结合在一起，构建一个通用计算机。

One of the things we tried was that one way to solve for these kinds of nonlinear systems to become linear is to linearize them. When I say linear, you dynamically reduce all the nonlinearities of the system. So you can apply linear algebra tricks on the system so that the system actually becomes computable. You can perform operations, linear algebraic operations, in parallel. That's something that, for example, state space models are linear versions of recurrent neural networks, and they lost that nonlinearity, nonlinear relationship between the parameters. The reason behind why you want to do that is because you want to make a scalable computation. So SSMs are becoming a very popular version of a dynamical system or a learning algorithm because you can now parallelize some other form of operations as well, like not just attention but you can have parallel scan and you can apply linear algebra tricks as I said to scale another format of operations. Linearizing this dynamical system allowed us to scale. So if you think about the SSM view, but the problem again that arises is that there's no free lunch in computer science. When you linearize a complicated dynamics, you're losing expressivity. So the system is not going to be as expressive as this nonlinear form. Then we realized that when we apply, let's say, these state space models to solving language problems, they have troubles learning dependencies at long context. They can compute really efficiently but they cannot express very efficiently. So the expressive power of these neural networks always falls short of Transformers. With all the nonlinear kind of mathematics that we learned from liquid neural networks, from dynamical systems, from SSMs, from convolutional neural networks, there are many different variations of operators that you can bring in together to build a general-purpose computer.

结合我们学到的一切，我们想：不要让自己陷入偏见，说我们的架构是圣杯，每个人都必须构建替代架构并押注单一架构。我们在 Liquid 的第一天就这么想，大约三年前——我们在 Liquid 所做的，就是把替代架构的复仇者联盟聚集到一个地方。正如我告诉你的，我们有来自 Yashua Bengio 实验室、斯坦福、MIT 的人——他们都是各种技术的发明者。有 Hyena Hierarchy，有斯坦福 Jimmy Smith 等人的状态空间模型，还有我们自己研究液体神经网络。所以我们把所有这些团队聚集在一起，说：让我们构建一种无偏见的方式，让我们构建一个元 AI 系统，在操作空间中搜索。所以我们构建了一个元系统，根据部署环境搜索感兴趣的算子。想象你想把这些解决方案部署在 GPU 或 NPU 上。所以我们在构建混合架构的基础上应用了这个大规模搜索算法，让你优化四件事。第一，不牺牲质量。有一堆质量指标，比如推理能力、语言理解、知识。我们希望模型做很多不同的事情——像 100 个不同的标准。我们认为：不牺牲相对于纯 Transformer 模型的质量。最小化内存消耗，最小化延迟，最大化计算速度。这些是我们在基于搜索的算法中优化的四个标准。我们发表了一篇关于这个科学的论文——它叫 STAR，定制混合架构的合成设计。所以如果你看这个，搜索 STAR Liquid，你就能找到。这就是我们如何系统地设计架构背后的科学。那个框架，那个元算法，是一种递归自我改进的系统。想想看，三年半前，我们应用它来有一个搜索算法，为给定芯片遍历各种混合架构。然后这个系统成为了不同代 Liquid 基础模型的计算图。Liquid 基础模型第一版是这个架构搜索的第一个实例。它有一些卷积元素，一些注意力机制元素，一些原始液体元素。它是一团混乱。那是第一代。第二代，我们专门优化了它在 CPU 上运行得非常好。LFM2 模型实例——如果你看它们，它们有一个非常优雅和简单的架构。架构基本上是 80% 双门控卷积，1D 卷积，加上一些分组查询注意力，基本上 20% 的查询。所以架构只是一个非常简化、非常快、高质量的基础模型，你可以在 CPU 上运行得很好。现在这个架构搜索的美妙之处在于，想象架构空间或智能基本构建块中出现了新创新或新发明——我们把它加入这个巨大的元 AI 搜索。所以我们推出的每一代 Liquid 基础模型肯定比上一版有改进，但算子是我们选择的，目标设备基本上决定了混合架构应该是什么。这就是我们开始设计这些 Liquid 基础模型的方式。你今天看到的是我们处于 LFM 2.5 模型实例。我们是一个基础模型实验室。我们自己从头构建一切——我们预训练这些模型，我们运行这些模型的缩放定律，我们把模型从数千万参数扩展到 700 亿参数。这是我们通常运行缩放定律的范围。到目前为止，我们发布了 LFM2 和 LFM 2.5 的较小实例。下一代将是 LFM3，是这些东西的稍微修改版本，这样我们可以增加更多质量、更多稳定性、更好的延迟控制，在我们关心的部署框架集上。

With everything that we learned, we thought: let's not put ourselves into a biased way and say our architecture is the holy grail and everybody has to build an alternative architecture and bet on a single architecture. What we thought day one at Liquid, about three years ago—what we did at Liquid—we started bringing avengers of alternative architectures all into one place. As I told you, we had people from Yashua Bengio's lab, from Stanford, from MIT—they're all inventors of various types of technology. There was Hyena Hierarchy, there were state space models like Jimmy Smith of Stanford, and ourselves on liquid neural networks. So we brought all of these teams into one place and we said: let's build an unbiased way, let's build a meta AI system that searches through the operation space. So we built a meta system that searches through operators of interest given the deployment environment. Imagine you want to deploy these solutions on a GPU or NPU. So we applied basically this massive search algorithm on top of building hybrid architectures that allows you to optimize four things. Number one, no sacrifice on quality. There are a bunch of quality metrics like reasoning capability, language understanding, knowledge. There are so many different things that we want the model to do—like 100 different criteria. We thought: no sacrificing quality against the pure Transformer model. Minimizing memory consumption, minimizing latency, and maximizing the speed of computation. These are the four criteria that we optimize in a search-based algorithm. There's a paper that we published on just the science of this thing—it's called STAR, the synthetic design of tailored hybrid architectures. So if you look at this, just search STAR Liquid, you can find it. So that's the science behind how systematically we are designing architecture. That framework, that meta algorithm, is a recursive self-improving kind of system. If you think about it, three and a half years ago, we applied it to have a search algorithm to go through various hybrid architectures for a given chip. And then this system became the computational graph of different generations of Liquid Foundation Models. Liquid Foundation Model version one was the first instance that came out of this architectural search. It had some elements of convolution, some elements of attention, some elements of original liquid. It was a mess of everything. That was the first generation. The second generation, we optimized it specifically for running really well on CPUs. The LFM2 instances of the models—if you look at them, they have a very elegant and simple architecture. The architecture is basically 80% double-gated convolutions, 1D convolutions, plus some group query attention, basically 20% of query. So the architecture is just a very simplified, very fast, and high-quality foundation model that you can run on a CPU really well. Now the beauty of this architecture search is that imagine a new innovation or new invention in the architecture space or fundamental building blocks of intelligence happens—we add them to this massive meta AI search. So every generation of Liquid Foundation Model that we put out will definitely have an improvement over the previous version, but the operators are chosen from those we choose, and the target device implies what that hybrid architecture should be. So that's how we started designing these Liquid Foundation Models. What you see today is we are at the LFM 2.5 instances of the models. We are a foundation model lab. We're building everything from scratch ourselves—we pre-train these models, we run the scaling laws of these models, we scale these models from tens of millions of parameters now to 70 billion parameters. That's the range that we have, usually running scaling laws on top of models. And so far we've released the smaller instances of the LFM2 and LFM 2.5 instances. The next generation would be LFM3, which is a slightly modified version of these things so that we can have added more quality, added more stability, added better latency control on the set of deployment frameworks that we care about.

你刚刚给了我们一个关于 Liquid 的博士论文摘要。我有很多要深入探讨的——神经架构搜索、无梯度进化，所有这些——但为了商业方面的听众，让我们也更新一下。上次我们在播客上覆盖 Liquid 是和 Shopify 的 Mikel 一起，他是你们的大粉丝。我相信还有更多，所以我想只有你知道，你会让人们了解——这不仅仅是一个研究项目，对吧？人们实际上在有效使用这个。基准测试不仅仅是目标追求；它们实际上反映了世界。

You just gave us a PhD abstract of what Liquid has. I have so much to dive into—neuro architecture search, gradient-free evolution, all these things—but just for the people also listening on the business side, let's also give an update. The last time we covered Liquid on the podcast was with Mikel from Shopify, who's a big fan of you guys. I'm sure there's a lot more, so I think only you would know and you would sort of bring people up to speed on—this is not just a research project, right? People are actually effectively using this. The benchmarks are not just goal seeks; they're actually reflective of the world.

绝对。绝对。所以是的，我必须说 Mikel 现在是我的导师之一。Mikel 是行业中最重要的人物之一。他是那种不废话的人。我真的很喜欢他如何非常直接地给出建议。他告诉我们的一件事是：你们在 Shopify 有一个非常能干的团队，我们在构建模型，所以如果你的模型——如果你告诉我你有模型，我必须看看。我们的团队——

Absolutely. Absolutely. So yes, I have to say Mikel is one of my mentors now. Mikel is one of the biggest people in the industry. He's one of those no BS kind of people. I really like how he gives his advice very directly. One of the things he told us is: you guys have a very competent team at Shopify and we're building models, so if your model—if you're telling me you have models, I have to see. Our team—

他是个雇佣兵；他只关心输入。

He's a mercenary; he just cares about the input.

就是这样。你带来技术，我们会测试它，我们会告诉你它是好是坏。然后他们第一次测试，他们对我们的模型质量和效率感到惊喜。所以我认为在替代架构领域，我们真的有非常好的模型。所以回到商业方面，当我们谈论我们操作的这一系列模型时，我告诉过你我们也在为 CPU 优化。我们正在考虑将智能带到数据中心之外。我们开始与关心数据的企业合作，这些数据也部署在数据中心之外。

That's it. You bring in the technology, we will test it and we will tell you if it's good or bad. Then they tested it first time and they were positively surprised by the quality and also the efficiencies of our models. So I think in the space of alternative architectures, we have really really good models. So back to the business side of things, when we talk about this range of models that we are operating and I told you we are optimizing for CPUs too. We are looking into bringing intelligence outside of data centers. We started working with enterprises that care about data that goes to deployments that are outside of data centers as well.

所以我们开始与汽车制造商、机器人制造商、笔记本电脑生产商合作，也与移动公司合作。他们的算力部署在数据中心之外的某个地方。所以我们介入，我们的基础模型非常出色，可以为各种类型的应用在这些设备上带来最佳的智能。我们的合作伙伴例子包括 AI PC 领域的 AMD。全球每年产生 3 亿台 AI PC，也就是笔记本电脑，3 亿到 4 亿台笔记本电脑。我认为其中大约 1 亿台可能是 AMD 驱动的。基本上核心构建模块是 AMD。所以我们正致力于让我们的液态基础模型直接在笔记本电脑上运行，实现某种本地智能部署，它们可以解决多种不同的问题。

So we started working with car manufacturers, with robot manufacturers, with laptop producers, we work with mobile companies. So they have deployments of compute somewhere outside of data centers. So we go in there and our foundation models are really good and you can bring the best of intelligence on top of these devices for various types of applications. Examples of partners of ours would be like AMD on the AI PC side. 300 million AI PCs are getting generated on the planet every year, like these are laptops, 300 to 400 million laptops. And I think from there about 100 million of it could be AMD powered. Basically the core of the building blocks is AMD. So we're working towards bringing our liquid foundation models directly running on the laptop, enabling some sort of local deployment of intelligence, and they can solve multiple different problems.

这可以用于隐私过滤器，它们可以是一种智能方式来守护你想从电脑中传出的内容。想象一下，你想把某些东西发送到 ChatGPT 终端或你的云代码智能体。那些你想发送的大量敏感数据，你可以先让它们通过一个液态基础模型，该模型能以智能方式屏蔽敏感信息，然后你就可以分享那些信息了。这是一方面。

This could be for privacy filters, they can be a smart way to guardrail what wants to get out of your computer. Imagine you want to send something to the ChatGPT terminal or to your Claude Code agent. Those kinds of bulk of sensitive data that you want to send, you can pass them first through a liquid foundation model that basically shields out sensitive information in a smart way, and then you can share that information. That's like one thing.

另一件你可以做的事是，让小型模型成为直接在你的设备上运行的主动式智能体。所以想象一下，如果你有一个智能体，它可以访问你笔记本电脑上目前所有类型的信息，然后执行某种分析，并把分析结果给你，就像你在云端能实现的一切，你也希望把它作为一种体验直接带到笔记本电脑上运行。这也会给你带来离线能力，以及气隙隔离能力。

The other thing that you can do is you can have the small models be proactive agents directly running on your device. So imagine if you have an agent that has access to all sorts of information that is right now on your laptop and then performing some sort of analysis and gives you that analysis, like everything that you can achieve in the cloud you'd want to bring it as an experience also directly running on a laptop. And that would give you also offline capabilities, also airgapped capabilities.

所以你可以想象，当我们进入制造业时，我们试图做的是为智能的端侧部分提供动力。今天当你想到设备智能时，存在混合解决方案：有一个模型实际上在云端，还有一些模型在设备上运行，这两者总是有协作。通常现在如果你想想最常用的手机，比如 Apple Intelligence、Galaxy AI，所有这些，90% 的调用都去往云端。所以 90% 去云端，10% 在设备上。原因是设备模型不够强大或不够可靠，无法真正从那些云端用例中分担更多。

So you can imagine when you're going into manufacturing, what we are trying to do is we are trying to power the on-device portion of intelligence. Today when you think about device intelligence, there are hybrid solutions out there: there is one model that is actually in the cloud and there are models that are running on the device, and there's always a collaboration of these two. Usually right now if you think about the most used phones, like from Apple Intelligence, from Galaxy AI, of all of these things, 90% of the calls are going to the cloud. So these are 90% goes to the cloud, 10% on the device. The reason is because the device models are not powerful enough or reliable enough to really take more off of those cloud use cases.

为什么公司想要在设备上运行这些 AI？第一个原因是成本，纯粹的成本。如果你有 4 亿部手机，或者假设数十亿客户，你必须支持他们并给他们免费的智能访问，而你必须在云端的某个地方提供这种智能，那将变得非常非常高昂。所以我们所做的，我们试图真正——这些公司真的在努力把 AI 工作负载直接带到设备上，因为那样你就是在为自己创造利润。所以这是第一个也是最简单的解决方案。

Why do companies want to run these AIs on the device? The first reason would be cost, sheer cost. If you have 400 million phones or let's say billions of clients that you have to support and give them free intelligence access, and you have to serve that intelligence somewhere in the cloud, that's going to become really really prohibitive cost. So what we do, we try to really—and these companies are really trying hard to bring the AI workload directly on the device, because in that way you're just creating margins for yourself. So that's the first and simplest solution.

然后除此之外，你会有主动的——你知道，正如我提到的，离线功能，比如隐私敏感话题，然后你可以在设备上带来许多许多不同的优势。但为设备构建模型并使其达到生产质量是非常困难的。极其困难。我可以告诉你，这不像只是从开源下载一个模型，然后试图把它放到设备上，然后你就成功了。远不止如此。你必须做热控制。模型在长上下文中的行为实际上会变化。你真的必须增加一定程度的可靠性。真正达到那个点真的很难。

And then apart from that, you would have active—you know, as I mentioned, offline features like privacy sensitive topics, and then many many different advantages that you can bring on the device. But it is very difficult building a model for the device and getting it to production quality. It's extremely difficult. I can tell you, it's not like just downloading a model from open source and then trying to put it on the device and then now you're successful. It goes a lot more than that. You have to do thermal control. The behavior of a model in the long context actually changes. You really have to have a degree of reliability to add that. It's really difficult to actually get to that point.

在过去的两年半里，我们经历了许多生产周期，我可以告诉你这真的很难。生产级 AI 远不止是在设备上跑一些质量基准测试。它超越于此。

And we have gone through over the last two and a half years, we've gone through many cycles of production and I can tell you it is really difficult. Production grade AI is way more than just benchmarking some quality benchmarks on the device. It goes beyond that.

然后另一个例子是汽车制造商，比如梅赛德斯-奔驰是我们的客户之一。我们在汽车领域所做的是带来车内智能。想象一下，我们去年四月宣布了一项合作，在汽车领域这实际上发生得非常快，这是一项重要的合作，我们考虑的是车内智能。所以我想给你一些见解。想象一下，你坐在车里，你可以和你的车说话，你可以有各种“嘿，梅赛德斯”功能，你在和你的车交流等等。这些都是通过直接进入车内的基础模型实现的。

And then another example of these things would be car manufacturers, like Mercedes-Benz is one of our clients. What we do in the car space is we bring in-car intelligence. So imagine we announced a partnership last April that actually happened really fast in the space of automotive, it's a significant partnership that we thought about inside the car intelligence. So I want to give you some insight. So imagine you're sitting in your car, you can talk to your car, you can have all sorts of 'Hey Mercedes' features, you are communicating with your car and stuff. Those are all happening with the foundation model that goes directly inside the car.

这个系统，我们提供动力的模型，是一个多模态解决方案，可以连接到摄像头，它实际上可以进行语音交互，并且通过基于文本的模型具有推理能力。这个模型本身大约 600 兆字节。所以想象一下，如果你有一个 600 兆字节的智能进入每辆车，你可以对地球上的每辆车进行空中更新。我们在梅赛德斯上的首次部署，将在今年发生，将是在所有第三代北美梅赛德斯-奔驰汽车上。我们将首次大规模部署 AI 在汽车内部，这运行在一个成本大约 100 美元的芯片上。

And this system, the model that we are powering, it's a multimodal solution that can get connected to the cameras and it can actually do a voice interchange and also it has reasoning powering through the text based models. And this model itself is about 600 megabytes. So imagine if you have a 600 megabyte intelligence that goes inside every car, you can do an over-the-air update of every car on the planet. The first deployments that we have on Mercedes, which is going to happen this year, it's going to be on all North America Mercedes-Benz cars that are generation 3. We are going to have the first deployment at scale of AI actually inside the cars, and this is running on a chip that is probably 100 bucks in cost.

所以我想说的是，真正思考带来真正高质量的生产级 AI 绝对是疯狂的。实际上，朝着我们所谈论的效率水平努力，然后把这些东西带入汽车领域的生产，感觉真的非常好。然后我可以告诉你我们与 Shopify 做的数百万件事情。Shopify 一直是我们不可思议的合作伙伴。他们不做端侧 AI，但他们关心延迟。他们关心延迟和质量。

So what I want to say is that it's absolutely insane to really think about bringing a really high quality production grade AI. It feels really really good to actually work towards this efficiency at the levels that we talk about and then taking these things and bringing them into production in the car space. And then I can tell you about millions of things that we've done with Shopify. Shopify has been an unbelievable partner for us. They are not doing on-device AI but they care about latency. They care about latency and quality.

他们的做法是，他们有一些私有部署，必须通过 Shop 应用来提供服务，包括面向商家的产品，也包括面向客户的产品，覆盖 Shopify 所做的许多不同业务。我们在生产环境中部署了很多 Liquid 基础模型。一个例子是：如果你今天打开 Shop 应用，在里面输入一些东西，你实际上会把信号发送到一个 Liquid 基础模型。所以那里确实有一个 Liquid 基础模型在跨整个技术栈提供服务，这非常好。而且我们正在看到一些惊人的数据，在使用量上也是指数级的数据。看起来客户真的很喜欢这些功能，现在我们每月已经指数级地超过 10 亿次请求。所以这就是通过 Liquid 基础模型、跨 Shop 应用的请求量——就是这一类指标。

So what they do is they have private deployments that they have to serve through the Shop app, the merchant-facing and also the client-facing products of Shopify, across many different kinds of things that Shopify does. We have a lot of liquid foundation models deployed in production. One example would be: if you go to the Shop app today and type in something in the Shop app, you would actually send the signal to a liquid foundation model. So there is actually a liquid foundation model there that is serving across the stack, which is very nice. And we are passing through some amazing statistics, exponential statistics as well, in terms of use. It seems like clients are really enjoying those features, and now we are exponentially at over a billion requests per month. So that's the amount of requests that goes through the liquid foundation models across the Shop app — just that kind of metric.

只是 Shopify，明白了。

Just Shopify, got it.

是的，只是那个。然后在模型本身的受欢迎程度方面，如你所知，我们开放——我们向社区交付 Liquid 基础模型，而且我们真的把它们开源了。我认为这些模型现在非常受欢迎。它们总下载量超过 4000 万次，而且我们每周大约有 150 万次下载。所以这些模型被下载得非常多。很多开发者把这些东西拿去投入生产。到目前为止我们发布的 Liquid 基础模型参数量范围在 1 亿参数到 240 亿参数之间，其中更大的那些是混合专家架构。而且这一系列模型也在加入多模态能力，正如我提到的。模型可以同时处理音频、视觉和文本作为输入，然后生成音频和文本作为输出。所以这就是我们所做的大致产品组合。

Yeah, it's just that. And then in terms of popularity of the models themselves, as you know, we open — we deliver liquid foundation models to the community, and we really open source them. I think the models are very popular right now. They have an overall over 40 million downloads, and we have about 1.5 million downloads per week. So the models are getting downloaded a lot. A lot of developers are taking these things and putting them in production. The range of parameters of liquid foundation models that we've released so far is between 100 million parameters to 24 billion parameters, and the bigger ones are a mixture-of-experts kind of architecture. And this range of models — they're also adding multimodal capabilities, as I mentioned. The models could be processing audio, vision, and text at the same time as an input, and then generating audio and text as an output. So that's kind of the portfolio of things that we do.

我想问问机器人，因为那是你创业时的灵感之一。我注意到你们有多模态模型。你们有视觉语言模型，也有音频模型。那么这些是否被用于机器人目的？你能不能谈谈其中一些实验？

I was wondering about robotics, because that was kind of one of your inspirations when you started. And I noticed you've got multimodal models. You've got like a vision-language model and audio models. So are those being used for robotics purposes, and can you maybe talk about some of those experiments?

当然可以。是的。当你拥有视觉语言模型时，你实际上可以解决视觉—动作类问题。你可以让它们变成机器人，看见世界，然后根据机器人看到的内容采取行动。你也可以把它们用作监控器。想象一下，在工厂里有摄像头监控整个工厂的运作。我们通常考虑的是机器人的工业运作——不主要是人形机器人，也包括工业机器人。我们和一家叫 Rootech 的公司合作过——我觉得你应该能找到，如果你直接搜 Rootech、AMD 和 Liquid，你会找到我们在那里做过的一个演示。还有一家公司叫 Gbionics，我想是在意大利，这些公司也有模块化机器人，他们在使用我们的视觉语言模型，用于机器人身体的一些动作空间。机器人身体上实际上有一个摄像头，他们用这些东西来实现，比如说，指令跟随类能力——基本上就是视觉—动作模型。所以用例大多在工厂内部。然后显然——如果你考虑多模态智能，正如你提到的，这些模型更小的实例用于机器人中的低延迟应用；它们真的非常受欢迎。有很多新的合作正在找上门来。但机器人是我们从第一天起就刻意暂停的事情。背后的原因是，围绕机器人的市场推广动作，以及真正围绕机器人建立业务，都非常困难，而且会被延迟，因为如果你考虑生产级机器人，你需要掉进技术验证与确认的兔子洞，然后进入部署的监管层面。所以我们一直——因为我们自己来自机器人背景，我们知道真正把某个机器人垂直领域产品化是什么样子。这就是为什么我们想先构建一个横向的智能层，先放在我们可以拥有 OEM 的地方，然后再进入动作世界，在那里验证与确认会敏感得多，作为公司更后期的一步。但我认为现在是个好时机，因为我们的模型现在成熟了一些，我们对数据以及这些基础模型面向某些类型应用的后训练和定制过程有了更多控制。我认为现在对我们来说，是一个非常合适的时机，可以比以前更认真地启动机器人垂直领域。

Absolutely. Yes. So when you have vision-language models, you can actually solve vision-action kind of problems. You can turn them into robots seeing the world and then taking an action based on what the robot sees. And you can also use them as monitors. So imagine in factories there are cameras that are monitoring the entire operations of a factory. And we usually are thinking about industrial operations of robots — not mostly humanoids, but also just industrial robots. We worked with a company called Rootech — I think you should find, if you just say Rootech, AMD, and Liquid, you would find a demo of what we've done there. There's another company called Gbionics, I think it's in Italy, and these companies also have modular robots, and they're using our vision-language model for some action space for the body of their robot. There's a camera actually on the body, and they are using these things for, let's say, instruction-following kind of capabilities — basically vision-action models. So use cases mostly go inside factories. And then there are obviously — if you think about multimodal intelligence, as you mentioned, the smaller instances of the models are for low-latency applications in robotics; they're really, really popular. There's a lot of new engagements that are coming our way. But robotics has been something that we deliberately put on pause day one. The reason behind it was that the go-to-market motion around robotics, and also really building a business around robotics, is very difficult, and it is delayed, because if you think about production-grade robotics, you need to go down the rabbit hole of verification and validation of the technology, and then you get into the regulatory aspect of it for deployment. So we always — because we are coming from a robotics background ourselves, we know how it is to really productize some robotics kind of vertical. That's why we wanted to build a horizontal layer of intelligence on top of places where we can have OEMs first, and then enter into the action world, where verification and validation becomes much more sensitive, as a later step of the company. But I think now it is a good time, because our models are matured a little bit better now, we have a lot more control over data and the post-training and customization process of these foundation models for certain types of applications. I think it's a very good time for us to really also start the robotic vertical much more seriously than before.

就这一点，因为我本来不打算——我没想到会在机器人上花这么多时间，但既然你提到了，而且你说这是个好时机，你们基本上是使用同一个 LFM 系列吗？我觉得很多人把用于机器人的动作模型或世界模型有多不同说得很重要。我不知道其中有多少只是营销，还是从根本上其实就是同一回事。

Just on this, because I wasn't going to — I wasn't expecting to spend that much time on robotics, but since you mentioned it and you said it's a good time, do you basically use the same LFM family? I think a lot of people make a big deal about how different action models are, or world models, for robotics. I don't know how much of that is just marketing versus fundamentally it is actually the same thing.

这是个很好的观点。那我告诉你一些事情。在基础模型领域发生的、会被用于任何垂直领域的创新——我会把它们分成三类。第一是智能的模型架构，第二是智能的算法层面，也就是那些有创新性的算法，最后是数据处理类机制——你想做哪种数据工作。这棵树上的创新——从我们研究的角度看，我们一直关注的是，到目前为止谈到的是模型架构创新，对吧？就是关于架构，比如模型的效率，以及基本上成为通用计算机的能力。所以这是模型视角。然后是算法。你训练模型的方式——你可以用自回归损失来训练模型。你可以在扩散过程中训练模型。你可以在世界建模的语境中训练模型。

That's a great point. So I will tell you something. Innovations that happen in the space of foundation models that are getting used in any vertical — I would categorize them in three different classes. There is model architecture of intelligence, then there is the algorithmic aspect of intelligence, like algorithms that are innovative, and then the last thing would be data processing kind of mechanisms — what kind of data work do you want to do. Innovation across this tree — what we've been focusing on from our research point of view, what we saw so far talked about, was about model architecture innovation, right? It was just about the architecture, like efficiency of a model and the ability to become a general-purpose computer, basically. So that's the model kind of perspective. Then there are algorithms. The way you train a model — you can train a model with an autoregressive kind of loss. You can train a model in diffusion processes. You can train a model in a world modeling context.

所以你可以把那种架构拿过来，带进我所说的世界模型。它像是一种无监督的学习系统的方式，真正在中间构建一个表征，这样从那个表征中你就有了一个世界观，对世界有了理解。所以当然，我会把世界模型归到智能的算法一侧，而不是模型架构的改变。你技术上可以拿一些 LFM 架构，用在世界模型的流程里训练它们，让它们更好地理解世界的物理规律。我一直认为，不同的算法让你以不同方式学习世界的某些物理结构。世界模型让你从物理世界中提取出一组不同的原语，相比下一个词预测。你不会说下一个词预测没有物理理解，但它和人类理解物理世界的方式非常不同。所以我只想说那个物理——而且关于哪些学习算法会解锁哪些能力、从而让机器人迎来 ChatGPT 时刻，仍然存在争论。这在算法一侧是个大大的问号。在架构一侧，我会说你可以用任何通用计算机、任何通用的算法或架构，用在我提到的那类框架里。

So you can take that architecture and bring them into what I would call world modeling. It's like an unsupervised way of learning systems, really building a representation in between, so that from that representation you have a world view, an understanding of the world. So of course I would attribute world modeling on the algorithmic side of intelligence rather than on the model architecture change. You could technically take some of these LFM architectures and use them in a world modeling pipeline and train them to have a better understanding of the physics of the world. I would always argue that various algorithms allow you to learn some physical structure of the world in different ways. World modeling allows you to extract a different set of primitives from the physical world compared to next-token prediction. You wouldn't say next-token prediction would not have a physical understanding, but it's very different than the human way of understanding the physical world. So I would just say that physical—and there is still a debate between what learning algorithms are going to unlock what set of capabilities needed for having the ChatGPT moments for robots. That's a big question mark on the algorithmic side. On the architecture side, I would say you could use any general-purpose computer, any general-purpose algorithm or architecture, to be used in those kinds of frameworks that I mentioned.

在我们回到更多研究之前，先让我理解一下商业模式这一侧。我假设这些都是授权协议，因为显然它是气隙隔离的、是在设备上的，诸如此类。你们要花多少时间来定制一个模型？假设我带着一组非常特殊的算力约束或别的什么来找你。是你们的人来做吗？是合作吗？这是怎么运作的？

Just so I understand the business model side before we go back to a bit more research. I assume these are all licensing agreements because obviously it's airgapped, it's on-device, it's all these things. How much time do you have to spend to customize a model? Let's say I come to you with a very special set of compute constraints or whatever. Is it you got your guys doing it? Is it a collaboration? How does this work?

非常好的问题。实际上我们早期的合作通常是与设计合作伙伴。这些设计合作伙伴，我们会配一个应用机器学习团队与客户合作，我们也会对那段合作期收费。然后一旦解决方案建成，我们就部署解决方案，然后按年、按设备收费，取决于这东西部署在哪里。所以我们把模型作为经常性成本来收费。然后我们在某个垂直领域建好解决方案后，该垂直领域下一个客户的服务时间就会减少。所以想象一下，如果梅赛德斯-奔驰是我们的设计合作伙伴，我们围绕我们的产品构建了一个解决方案，那么进入汽车垂直领域的下一个客户，我们这边提供的服务量就会减少，就变成了单纯的解决方案部署。所以我们在每个垂直领域都做解决方案，这些解决方案实际上让我们能更快地销售。话虽如此，作为一家基础模型公司，我们经历过从零构建模型并在生产中部署这些模型的过程。我们经历过这个过程很多很多次。我们现在正在构建一个模型开发平台，那个平台是自助式的。所以你可以访问这个平台，然后使用这个平台，你就能自动完成那项工作。所以这是我们正在构建的东西。

Fantastic question. So this is actually our early engagements is usually with design partners. These design partners are the ones that we associate an applied ML team to work with the client, and we charge also for that duration of engagement. And then once the solution is built, then we deploy the solution and then we charge licenses on a yearly basis, on a device basis, depending on where this thing is getting deployed. So we are charging for models as a recurrent cost. Then after we build a solution in a vertical, the time of service reduces for the next customer in that vertical. So imagine if Mercedes-Benz is our design partner and we build a solution around our products, then the next customers that are coming in the automotive vertical, the amount of service that we provide reduces from our side and it becomes just the solution deployment. So we make solutions in every vertical and those solutions are the ones that are actually allowing us to sell faster. That being said, because as a foundation model company, we've gone through the process of building a model from scratch and deploying these models in production. We've gone through this process many, many times. We're building now a platform for model development and that platform is self-served. So you can actually get access to this platform and then using this platform you would be able to automatically do that job. So this is something that we are building.

哇。这个平台是全面推出了，还是在 beta 阶段？这是我第一次听说这个。

Wow. Is this platform fully rolled out, is it in beta? This is my first time hearing about this.

不，现在还在 beta 阶段。它在 beta 阶段，我们还在测试。我们对此非常兴奋。如果你想想，定制和持续学习是个非常有趣的问题。这是 AI 部署非常非常重要的下一步。现在大约 90% 的市场都围绕推理 token，比如前沿实验室，如果你想想前沿实验室，甚至现在的推理提供商，比如 Fireworks 之类的，他们在做什么？他们托管模型，然后基本上对推理收费。推理 token 才是赚钱的。但我认为下一波公司——你应该知道，我可以列举几个：有 Core Automation，有 Thinking Machines，你想想还有什么，Trajectory——在这个定制类别里有很多不同的公司。所以很多人正在转向定制 token，因为现在也有很多好的基础开源模型，所以企业实际上可以获得——顺便说一句，我们说的是企业 AI——所以在企业 AI 领域，定制 token 的变现是我认为人们会去利用的，它会是非常非常有趣的东西，它会是从这类类别中涌现的下一波十角兽，其中一些已经是十角兽了。比如 Jeff Dean 刚刚又创办了一个实验室，做同样类型的话题。所以你可以想象定制会变得非常——当我说定制时，不要只想到后训练一个模型。想想从预训练到推理的定制深度。那就是定制。你可以做预训练，你可以做中期训练，你可以做后训练，你可以做数据生成，你可以做强化学习。所有这些都属于模型定制的范畴，然后让智能体来做这些，然后把整个东西循环起来，这样你实际上可以有一个递归循环，让客户拥有自己的智能。这是我们计划要做的事情，我们正在向客户提供这个平台，这样他们就能真正构建出和我们能构建的质量一样的 Liquid 基础模型。这是第一点。然后一旦他们把它们投入生产，他们就能让它们始终保持最新，始终达到他们希望它们表现的水平。

No, it's in beta right now. It's in beta, we're still testing it out. We're very excited about it. It's a very interesting kind of problem if you think about it, this customization and continual learning. It's a very, very important next step for the deployment of AI. Right now like 90% of the market is around inference tokens, like frontier labs, if you think about frontier labs and even inference providers right now like Fireworks of the world, what are they doing? They host the models and then they basically charge the inference. The inference tokens are the ones that are making money. But I think the next wave of companies—you should know, I can name a couple of them: there's like Core Automation, there's like Thinking Machines, you think about what else do we have, Trajectory—there's like many different companies in this kind of category of customization. So a lot of people are turning into customization tokens because now there's a lot of good base open source models as well, so enterprises could actually get—we are talking about enterprise AI by the way—so in the space of enterprise AI, monetization of customization tokens is something that I think people are going to capitalize on and it's going to be something that is very, very interesting and it's going to be the next wave of decacorns that are going to emerge from these kinds of categories, which some of them are already decacorns. Like Jeff Dean just started a lab again on the same type of topics. So you can imagine customization is going to become very—and when I say customization, don't only think about post-training a model. Think about the depths of customization from pre-training to inference. That's customization. You can do pre-training, you can do mid-training, you can do post-training, you can do data gen, you can do RL. All of these things fall into this category of customization of a model, and then allowing agents to do that and then looping the whole thing so that you can actually have a recursive loop that allows the customers to own their own intelligence. That's kind of something that we are planning to do and we are providing this platform to customers so that they can really build liquid foundation models at the quality that we can build. That's number one. And then once they put them in production, they can keep them always up to date and always at the level that they want them to perform.

那么在我们进入研究话题之前，我能先问一下你们的开发者平台 LEAP 吗？开发者目前是怎么使用这个平台的，他们用什么流程来部署你们的产品？

So just before we head to the research, can I ask a little bit about your developer platform, LEAP? How are developers using this platform at the moment, and what kind of process do they use to deploy your products?

如果你把 LEAP 想成一个库，我们有一个 cookbook，在 Liquid 这就像一个开源的地方。你可以去我们的 cookbook，它吸引了很多关注。里面内容非常丰富，我们甚至提供了很多洞见。这是手册式的，是为人类构建的，对吧？LEAP 也是为人类构建的。你是一个开发者，你想用 Liquid 基础模型，想微调它们，你去那里，可以拿一些 LEAP 微调，比如 LEAP 微调就是你可以微调我们某个模型的地方之一。然后是部署。LEAP 还允许你提取一个可以部署的包，比如推理就绪的 GGUF 格式，像 llama.cpp 就绪的系统，可以直接放到设备上。如果你想做，比如说 CPU 部署，你可以去我们的 LEAP 平台，它允许你提取这些就绪的包来部署。所以这就是 LEAP 做的事。再说一次，想想我们现在正在构建的平台，它实际上就是我们一直在用 LEAP 做的事情的自动化版本。LEAP 有一些功能，也有一些功能没有。我们想把 LEAP 平台作为工具补全，然后也能为企业提供能力，以我们使用这些工具的方式来做，但这些是独立的、分段的工具，你可以用，或者给你的云智能体，让它拿去微调模型。话虽如此，Liquid 基础模型也在非常流行的地方得到支持，比如 Ollama 和 Hugging Face。他们有 TRL 微调，你可以用任何这些开源包。也因为我们现在试图真正与开源社区连接。我们喜欢 Hugging Face 的人，我们喜欢 SGLang 的人，我们与他们合作非常紧密，真正带来支持。而 llama.cpp 项目本身就是一个了不起的努力，真正把智能部署到数据中心之外。

So if you think about LEAP, it's like a library. We have a cookbook, you know, like at Liquid this is like an open source kind of place. You can go to our cookbook, got a lot of attention. There is so much going on in there. We literally provide even a lot of insights. This is manual, this is built for humans, right? LEAP is also built for humans. You're a developer, you want to use Liquid foundation models, you want to fine-tune them, you go there, you can take some of the LEAP fine-tune, for example, is one of those places where you can fine-tune one of our models. Then there are deployments. LEAP allows you to also extract a bundle that you can deploy, like an inference-ready GGUF format, like a llama.cpp-ready system that can go directly on the device. If you want to have, let's say, a CPU deployment, you can go to our LEAP platform and it allows you to extract one of these bundles that are ready to be deployed. So that's what LEAP does. And again, think about our current platform that we are building right now, it's literally the automated version of what we have been doing with LEAP. There are some functionalities that LEAP has and some functionalities that it doesn't. So the ones that we want to complete the LEAP platform as tools, and then be able to also provide enterprises with the capability to do that in a way that we use those tools, but those are kind of individual, segmented tools that you can use or give your cloud agent to really take them and use them to fine-tune a model. That being said, there are also support of Liquid foundation models in the very popular places like Ollama and Hugging Face. They have the TRL fine-tuning, you can use any of those open source packages. Also because we now try to be really connected to the open source community. We love the Hugging Face guys, we love the SGLang guys, we work very, very closely with them, and really bringing support. And the llama.cpp project itself is an amazing effort to really deploy intelligence outside of data centers.

是啊，我有一阵没听到 TRL 了。每年或每两年就有个新热门。现在是 Unsloth，但我们过去也报道过 Axolotl。有一点，你说到自助服务时我眉毛都挑起来了。我原本以为你们会做很多服务，也许现在叫前置部署，我不知道，反正就是那个热门词。但是不是有一个成熟度或层级？基本上我的想法是，好吧，大家都在谈持续学习，但可能大多数人根本没准备好。比如你们发布量化感知蒸馏，对吧？这个我能做，看起来简单，看起来像自助服务。好，你给我一些护栏，告诉我怎么做，比如我可以把大模型蒸馏成小模型，对吧？是不是有一个阶段，好，你从这里开始，然后一步步往上走？

Yeah, I haven't heard TRL in a while. There's all these, every year or two there's like a new hotness. Right now it's Unsloth, but we've also covered Axolotl in the past as well. One thing, my eyebrows just went up when you said self-serve. I had expected that you would do a lot of services, maybe these days it's called forward deployed, I don't know, whatever the hot term is. But is there a maturity or hierarchy of things? Like, so basically the way I think about it is, okay, everyone's talking about continual learning, but probably most people are not even ready. You release, for example, quantization-aware distillation, right? And like, that I can do, that seems easy, that seems like self-serve. Okay, you give me some guardrails on how to do this, like I can distill big model into a small model, right? Is there a stage of, okay, you start here and then you work your way up?

绝对有。是的。你触及了一个非常非常重要的问题。我告诉你一个更深层的视角。我们大概已经与 200 家《财富》500 强公司合作过。我们真的与所有企业保持联系，而企业已经精疲力竭。他们经历了很长时间构建自己智能的过程。他们试图自己拥有应用机器学习团队。我们说的是成熟企业，对吧？这些成熟企业，他们尝试过智能，从开源下载模型，从头构建自己的模型，用于自己的主题等等，但这并没有带来实际的生产线，他们总是退回到云解决方案。所以他们从未达到他们想要的生产质量。有些产品确实存在，但我可以告诉你比例，大约 80% 的生产级东西没有，80% 没有达到生产水平。所以这是我看到的令人精疲力竭的事情之一。我看到的第二件事是，现在开发者有巨大需求，他们想获得最高形式的智能用于开发。他们想 vibe code 东西，好像每个人都想 token max。有这类事情，它就像一种病毒在开发者头脑中，每个人都是，再说一次，当你做价值归因时，这些 vibe code 的仓库在成熟组织中产生了多少价值，你又会发现 90% 的 token 是无用 token。所以当企业看到这一点，并且也想控制成本时，现在有巨大需求，让他们拥有一个自助平台，能够尽快达到他们想达到的点，获得他们想获得的价值，同时满足自己开发者的需求。所以我们尝试做的，我们尝试构建这个平台，不是作为另一个 UI，另一种新类型的东西。我们的平台实际上会坐在客户端里面，非常类似于你与 Claude Code 交互的方式，非常类似，你拿起你的智能体框架，给它 Liquid AI 的工具，这些工具就能让你自己最喜欢的智能体框架以逐步方式引导你开发、构建和部署一个生产质量级别的 Liquid 基础模型，就像我们会做的那样。但这个过程目前是与开发者的互动过程。人在回路方面非常重要，就像你提到的。你提到他们准备好了吗，你必须引导客户，你必须引导他们，引导开发者真正达到那个点。所以这就是自动化的东西。你也可以给它加把劲，我们有，你可以把整个平台放到自动模式，说一键点击，然后就直接交付。

Absolutely. Yes. You touched on a very, very important problem. I'll tell you a little bit a deeper perspective. We worked probably with 200 of the Fortune 500 companies so far. We really are in touch with all the enterprises, and what enterprises are exhausted. They went through the process of building their own intelligence for a long time. They try to have applied ML teams themselves. We're talking about mature enterprises, right? These mature enterprises, they tried out intelligence, downloading models from open source, building their own models from scratch for their own topics and stuff, and this did not result in actual production lines, and they always defaulted back to the cloud solutions. So they never got to the production quality that they want. Some of these products are out there, but I can tell you the ratio is like 80% of the production-grade stuff doesn't, the PC is 80% of them do not hit the production level. So that's one of the exhaustive things that I see. The second thing that I see is that there is massive demand for developers right now that they want to have access to the highest form of intelligence for development. They want to vibe code stuff, like everybody wants to token max. There was like these kind of things, it's such a virus that is in the head of developers, and everybody is, and again when you do the value attributing, how much value was generated off of this vibe-coded, vibe-coded repositories and stuff that is happening at mature organizations, you will again end up with 90% of these tokens are useless tokens. So when you actually have enterprises see this and they want to also control the cost that they have, there is a massive demand now for them to have a self-served platform that allows them to really get to the point that they want to get as soon as possible, to a value that they want to receive, while satisfying the demand of their own developers. So what we try to do, we try to build the platform not as another UI, another new type of set or something. So our platform is literally going to sit inside a client, very similar to how you interface with your Claude Code, very similar, that you take your agentic harness and you just give it the tools of Liquid AI, and these tools would be able, your own favorite agentic harness would be able to, in a step-by-step manner, walk you through developing and building and deploying a Liquid foundation model at production quality level, the way that we would do. But this process is an interactive process with the developer at the moment. Human-in-the-loop aspect of it is very important to the point that you mentioned. You mentioned like are they ready to do this, you got to guide the customers, you got to guide them, the developers, to really get to that point. So that is the automated kind of stuff. You can also put it on steroids, we have like you can put it on auto mode, the whole platform, and say click one click and then just deliver this.

我不想看到任何中间环节，但显然最终模型的质量不会像开发者真正交互式地构建出来的那么好。所以我们试着让自己……

I don't want to see any intermediary stuff, but obviously the quality of the resulting model is not going to be as good as when you're having an interactive kind of a developer actually interactively build something with it. So we try to get ourselves...

让我反驳你一下，对吧？如果我有好的评估，我为什么要在意过程是什么？它应该自己自动研究。我为什么第一次还得去看日志？

Let me push back on you, right? If I have good evals, why do I care what the process is? It should just auto research itself. Why do I have to look at the logs for the first time?

第一次部署时我完全同意你，对吧？想象一下，你根据现有的评估把模型投入生产。现在模型上线了，然后新数据会进来。模型在生产环境里，真实的生产环境。现在想象新数据进来，新的请求，突然之间出现巨大的分布偏移，比如说某个社区对你的模型有了不同的需求。你怎么处理？你必须能处理这些事，因为你的评估会过时。静态评估会过时。所以你需要某种动态性，而这只有在你拥有一个持续演进的系统时才会发生。评估不可能保持静态，这就是关键。第一次迭代时，我完全同意你。但一旦进入生产，评估就不会保持不变了，你总是得改变标准。

I fully agree with you for the first deployment, right? So imagine if you put the model in production based on the evals that you had. Now you put the model in production, then new data is going to come in. The model is in production, actual production environment. Now imagine new data is coming in, new requests, all of a sudden there's a massive distribution shift of, let's say, a community wanting different things from your models. How do you handle that? You got to be able to handle these things, because your evals are going to become obsolete. Static evals are going to become obsolete. So you need to have some sort of a dynamism there, and this would only happen if you have a continuously evolving system. Evals cannot stay static. That's the thing. For the first iteration, I fully agree with you. But if you go to production, then the evals are not going to stay the same. You always have to change the criteria.

这一点我完全同意。我觉得这就是为什么我们在转向环境、harbor 风格的任务等等这些东西。你在后训练里还看到哪些趋势？我们自己的环境？那里有很多东西，Maxim 显然谈过，但实质上是不是客户——我们姑且这么说——在做越来越多的后训练，而你们在做架构方面的工作？分工是不是大概这样？

I definitely agree on that. I think that's why we're moving towards environments and harbor style tasks and all these things. What other trends are you seeing in post-training? Our own environments? There's a lot there which obviously Maxim has talked about, but effectively are we customers, let's call it, doing more and more of the post-training and you are doing the architectural work? Is that the division of labor, something like that?

是的，我会说不只是后训练。就像我说的，别忘了不只是后训练。我谈的是从零开始的整个模型开发。我们希望能够……一切，包括架构搜索。

Yes, I would say not just post. As I said, don't forget about just post-training. I talked about the whole model development from scratch. We want to be able to... everything, including the architecture search.

是的，如果你像高通那样成熟，比如说他们想为下一代芯片做自己的设计，他们想要一个专用的计算图，能在他们正在设计的 NPU 上、在高通芯片上跑得最好。或者也有一些公司把神经网络的权重直接蚀刻进芯片里。这些事情正在发生。

Yeah, if you're as sophisticated as, let's say, Qualcomm, that wants to design their own, let's say, for the next generation of their chips, they want to have a dedicated computation graph that works the best on top of Qualcomm chips on an NPU that they're designing. Or there are also some companies that are etching kind of weights of a neural network directly inside the chips. These things are happening.

我报道过 Taalas，是的，Taalas 和 Etched 之前都报道过。

I covered Taalas, yeah, Taalas and Etched before.

所以你知道，硅片这个世界里有太多东西，也是因为那些人会比任何人都更对我们的架构搜索感兴趣。

So as you know, there's so much in the silicon kind of world that is also because those guys would be interested in our architecture search more than anybody.

所以硅片其实极其……与硅片合作伙伴保持紧密关系对我们非常重要，因为我们的架构搜索是硬件感知的。为什么？因为他们把他们的硬件，或者下一代硬件，放进循环里，来找出对于我在意的标准来说什么是最好的架构。所以想象你启动这样一个流程。硅片公司真的很喜欢我们摆在他们面前的这种自动化基础设计平台。所以那项技术，然后正如你提到的，我同意定制化会越来越多地走向企业。企业将能够拥有自己的模型开发和部署。这就是两件事。这才是主权 AI 的真正含义。每个人都能够拥有自己的智能，然后希望再进一步，所有客户，地球上每一个人，也能够拥有自己的智能。所以那是下一个层次，个人智能。希望我们能解锁它。

So silicon is actually extremely... staying close with silicon partners is actually very important for us, because our architecture search is hardware aware. Why? Because they are putting their hardware, or maybe next generation of the hardware, in the loop of finding out what is the best architecture for the criteria that I care about. So imagine you launch a process like that. Silicon companies, they really like this automated foundation design kind of platform that we are putting in front of them. So that technology, and then as you mentioned, I agree customization wants to get more and more to enterprises. Enterprises would be able to own their own model development and deployment. These are the two things. This is kind of the true meaning of sovereign AI. Everybody would be able to own their own intelligence, and then hopefully one step further would be all the customers, like every single person on the planet, would be able to own their own intelligence as well. So that's kind of the next level, personal intelligence. Hopefully we can unlock that.

我想快速聊几点，因为你刚才留了太多东西在里面。其实有很多可以深入。就神经架构搜索而言，你是第一家，我想也许你是第二或第三家公司。我们最近报道了 Poolside，他们谈到了他们的模型工厂。我相信你也谈过一些。然后还有一点，我之前也对搜索的工具非常兴奋。Apple 有一篇关于 Taria 的论文，那是他们的一种 LM ops，像是逐层检查和消融工具。你在你们新的架构搜索或模型工厂之类的东西里发现了什么，是你以为会有效但其实无效的，或者出乎意料地有效的？有没有什么你想从你们的流程或研究中强调的？

I just wanted to quickly touch on things, because you left so much in there. There's actually a lot to dive into. In terms of neural architecture search, you're the first, I think maybe you're like second or third company. We recently covered Poolside where they talked about their model factory. I'm sure you've talked a little bit about that. And then also a bit back I was also very excited by the tooling for search. Apple had this paper on Taria, which is their sort of LM ops, like layer-by-layer inspection and ablation tool. What have you found in your new architecture search or model factory type things that you thought would work but doesn't work, or is surprisingly effective? Anything that you want to highlight from your process or research?

当我们在基础模型领域做架构搜索时，我们会跑一个缩放损失。想象对每一个架构候选，我们试着跑一定类型、到一定数量的 token，我们跑一个缩放损失。所以这就像是在某个固定规模的模型上投入更多数据，或者增大模型规模，然后看给定这个候选架构它能训练得多好。我们学到的是，你做的神经网络越大，如果你想构建通用智能，架构就应该越无偏。所以你必须让你的系统去偏。比如你看出来的 LFM2 架构，它没有任何这些门控 delta 网络之类的东西。它非常非结构化，就是 1D 卷积，字面意义上的 1D 卷积。这是你能做的最简单的东西。它是一种非结构化的方式。注意力机制本身就是非常非结构化的，就像我说的，当架构没有偏置时，比如注意力机制、比如 Transformer 架构。你就能够扩展它们。所以在规模上，我会说架构内部需要的偏置要少得多。你不需要添加特征、外循环之类的……你把架构做得越大，就越要简化它。在光谱的另一端，想象你想解决非常非常小规模的问题，而你负担不起，比如你有内存限制，比如说 8 GB 的内存限制。在那里你可以在架构方面更有创意。你可以让模型有多个反馈回路。你可以给系统增加多得多的复杂度。在小模型领域，你给系统加的偏置越多，你得到的表达动态就越丰富。在更大规模上，你去掉那些偏置。这就是我们学到的。

When we do architecture search in the space of foundation models, we run a scaling loss. Imagine for every architecture candidate, we try to run a certain type up to a certain number of tokens, we run a scaling loss. So this being like throwing more data at a certain fixed size of the model, or increasing the size of the model, and see how well it can actually train given that candidate architecture. The thing that we learned is that the larger the neural network you are making, the architecture should be, if you want to build general purpose intelligence, it should be unbiased. So you have to unbias your system. For example, if you look at LFM2 architecture that came out, this doesn't have any of these gated delta nets and all these things. It's like a very unstructured, it's like 1D convolutions, literally like 1D convolutions. This is like the simplest thing that you can actually do. It's like an unstructured way. Attention itself is like a very unstructured, as I said, when architectures are not biased, like attention, like transformers architecture. You would be able to scale them. So at scale, I would say a lot less bias is needed inside the architecture. You don't need to add features, out loops, at some sort of... You got to simplify the architecture the larger you make them. On the other side of the spectrum, imagine if you want to solve problems at a very, very small scale and you cannot afford to have, you know, like you have like memory limit, let's say like 8 GB of memory limit. There you can get more creative on the architecture side. You can make the models to have multiple feedback loops. You can add like a lot more complexity to the system. In the small model regime, the more biases you add to the system, the more expressive dynamics you get. In the larger scale, you remove those biases. This is what we learned.

我要告诉你的第二个洞见是数据模态。有些数据模态，比如音频。在音频中，你可以使用连续时间动力系统或 SSM，它们在音频中非常有效，但在文本上，它们很糟糕。SSM 在文本上表现不佳。当你考虑，比如说你只想要一个音频模型，在需要达到人类级别、真正高质量音频输出的规模时，加入循环神经网络，这时我会说循环网络会大放异彩。同样在小规模下，当我们谈论我们搜索的这类模型，比如低于 200 亿参数时，循环可以非常非常有效。但我会称它们为更有偏置的架构。它们有更多反馈，有结构门控，在它们之上有某种架构原型，然后随着规模扩大，你移除这些东西。这是我们的一般理解。毫不奇怪，我想 Jason 我们有过这个帖子，在小规模时你加入偏置，最终你必须把它们去掉。

Second insight I would tell you is data modality. There are some data modalities like audio. In audio, you could use continuous time dynamical systems or SSMs for example, they're very effective in audio, but on text, they suck. SSMs don't work really well on text. And when you think about, let's say if you just want to have an audio model, throwing in a recurrent neural network at the scale that is needed for having a really good quality audio output, this is the place where I would say recurrent network would be very much shining. Again at small scale, when we talk about this category of models that we search below let's say 20 billion parameters, recurrence can be really really effective there. But I would call them more biased architectures. They have more feedback, there's a structural gate, they have a certain architecture archetype on top of them, and then you remove those things as you scale. That's the general understanding that we have. No surprise that I think Jason we had this post where at small scale you add in biases and then eventually you have to take them out.

我还认为层循环，或者我不知道这是否叫层循环，你叫它层循环还是什么？

I also think layer looping, or I don't know if this is called layer looping, do you call it layer looping or whatever?

是的是的是的，循环层。

Yeah yeah yeah, looped layer.

那个有点不直观，为什么它在小规模有效但大规模无效。看起来它应该有效。我想这主要是效率问题。所以也许这只是我的解释。我不知道。但我会说，循环仍然是一种偏置，像任何形式的循环。它是你添加到系统中的偏置。你让它复杂化，然后你必须为它找到基础设施。所以我们没有正确的基础设施；我们当前的 GPU 基础设施是为这种工作负载优化的，并行矩阵乘法，一切你可以并行化的。想象如果你有光子学。我认为光子学这个竞赛会是一个惊人的空间。如果你想想，如果你能真正以光速通信，现在以串行方式计算。当你以光速移动时，串行计算的速度是多少？人类无法感知，对吧？所以我说的是，我认为光子学可能是我们前进的一种难以置信的方式，如果你考虑介质变化。所以那也可能解锁非线性算子。我知道有公司在研究光子学，然后是量子计算机。每个人都在这个替代智能或计算介质空间工作。这些是我看到的另一个机会，我对光子学非常兴奋，因为我认为有很多，因为顺序计算，从线性代数的角度来看，我们为非线性系统学习它，因为我们不能真正并行化非线性系统。但想象如果你有光速计算速度用于非线性算子，那么你实际上可以构建下一代架构，因为那时你不受你托管智能的基质的限制。

That one is a bit unintuitive why it works at small scale but not at larger scale. It seems like it would work. It's mostly an efficiency thing I guess. So maybe that's just my explanation. I don't know. But I would say still, looping is again a bias, like any format of looping. It is a bias that you're adding to your system. You make it complicated and then you have to find out an infrastructure for it. So we don't have the correct infra; the current GPU infrastructure that we have is optimized for this type of workload, parallel matrix multiplication, everything that you can parallelize. Imagine if you have photonics. I think this race of photonics would be an amazing space. If you think about it, if you can actually communicate with the speed of light, now compute in a serial way. What is the speed of serial computation when you're moving with the speed of light? It's humanly not perceivable, right? So what I'm saying is that I think photonics could be an unbelievable way for us to move if you think about medium change. So that could also unlock nonlinear operators. I know companies that are working on photonics and then quantum computers. Everybody's working in this space of alternative mediums for intelligence or compute in general. Those are another opportunities that I see opening up, and I'm really excited about photonics because I think there's a lot to be, because sequential computations, this is something that from a linear algebra point of view we learned it for nonlinear systems because we cannot really parallelize nonlinear systems. But imagine if you have speed of light computational speed for nonlinear operators, then you can actually build the next generation of architectures because then you're not limited by the substrate you're hosting intelligence on.

所以，我给你引用一句，我想是 Christopher K 写的关于神经科学的书。我想我大约 14 年前读过。这不是引用。他在一个会议上说，智能是碳的属性。因为人类的基质是碳的属性。所以他说，好吧，自然生命发生在不同的基质上，我们现在在硅上构建智能。所以智能的基础非常不同。我认为基质将在下一代架构中发挥作用，如果你真的想想，因为它会解锁各种计算格式，而不仅仅是通过 GPU 的并行计算。那也是我想提的另一个空间。

So, I'll give you one quote from I think it was from Christopher K who was writing some book on neuroscience. I think I read it about 14 years ago. It wasn't a quote. He was at a conference and he was saying intelligence is a property of carbon. So because the substrate of human is a property of carbon. So he was saying like okay so natural life is happening on a different substrate and we're building intelligence right now on silicon. So the base of intelligence is very different. I think substrate is going to play a role into even the next generation of architectures if you really think about it because it would unlock various formats of computation apart from just parallel computation through GPUs. That's also another space that I would just bring in.

好的，我们没有太多时间提这个，但你提到了，所以我必须问。我会说，实际上我在旧金山圈子里有惊人数量的朋友对量子计算并不那么兴奋。如果我们真的拥有大规模的量子比特，你真的认为它会改变什么吗？有些人不同意，所以我想听听一些看法。

Okay, we don't have a ton of time to mention this, but you brought this up so I have to ask. I will say actually a surprising number of people in my circles in SF are not that excited by quantum computing. If we did have at scale like a bunch of qubits, do you actually think that it would change anything? There's some people that disagree, so I want to hear a few.

我认为它唯一会大规模改变的是搜索，因为我认为搜索是那些非常重要的算法之一，想象如果你能进行并行搜索。

I think the only thing that it changes massively would be search because I think search is one of those algorithms that matters so much and imagine if you can have parallel search.

而优化最终是什么？

And what is optimization at the end of it?

我们现在让自己服从梯度下降，因为我们不知道任何更好的方法来大规模执行大规模搜索。它是在参数空间中的搜索。如果你考虑量子计算机，我认为搜索将大规模进化。希望这一点没有争议。但是的，非常有趣的是，有许多其他瓶颈对我们很重要，实际上并没有被量子解决。

We are submitting ourselves to gradient descent right now because we don't know any better way to perform massive search at scale. It is a search in a parameter space. And if you think about quantum computers, I think search is going to evolve massively. That one hopefully is not controversial. But yeah, it's very interesting that there are many other bottlenecks that matter to us that actually is not solved by quantum.

绝对，但我不认为我们被关闭了。

Absolutely, but I don't think we are closed.

因为我想 2017 年我在一次 NeurIPS 会议上，我和我想是微软或 IBM 的人交谈，他们，哦不，是 IBM，他们带来，他们展示他们的量子比特，像悬挂在天花板上的东西，他们在会议上展示，然后我和那个人交谈，我说好的，那么什么时候每个人都能真正使用这样的东西？当时 2017 年他告诉我 2035 年。好的。

Because I think 2017 I had I was at one of these NeurIPS conferences and I was talking to I think it was Microsoft or IBM, like they were, oh no it was IBM, that they were bringing, they were showing off their quantum bits like the thing that is hanging from the ceiling and they were showing it at the conference and then I was talking to the guy and I said okay so how can, when can everybody actually use something like this? At that time 2017 he told me 2035. Okay.

所以，即将到来。

So, coming.

我认为现在还太早，所以说什么都还早。

I think it's still too early so to say anything.

是的。是的。好的。实际上，回到更近的地方，为听众们。我们做过一些相关的播客。我们 RWKV，你知道，是另一个被广泛采用的非 Transformer 架构，有点松散地受 RNN 启发。所以我们做了几集关于这些东西。我们实际上请过 Tree Dow。我们还没有请过 Cartisia，但就像你提到 SSM 对语音非常适应，起初当 Tree 和我谈论它，Karan 和我谈论它时，我不理解为什么语音对我来说，实际上简单的解释就是没有那么多状态。

Yes. Yes. Okay. Coming back closer actually for listeners. We've done a bunch of related podcasts. We RWKV, you know, is another like pretty widely adopted non-transformer architecture that's like sort of loosely inspired by RNNs. So we've done a couple episodes on those things. We've actually had Tree Dow on. And I we haven't had Cartisia on yet, but like when you mentioned how SSMs are very attuned to voice, at first when I when Tree talked about it to me and Karan talked about it to me, I didn't understand like why voice to me actually like the simple explanation is there's just not that much state.

语音本身就是在持续推理。所以这其实非常理想。它是一个连续的序列，而序列就是时间序列。如果你用一个专门处理时间序列数据的动力系统，那基本上就是你能用来建模时间序列的最佳版本。所以这就是为什么 SSM 相当有效。SSM、RNN 和连续时间系统在处理基本上属于序列类的数据时非常有效。

Voice is like continuously inferencing anyway. So this is actually pretty ideal. It's a continuous sequence, and sequences that are time series. If you throw a dynamical system that is designed to handle time series data, that's like the best version of the thing that you can actually model time series with. So that's why SSMs are pretty effective. SSMs and RNNs and continuous-time systems are very effective in basically sequential kind of data.

你看，你提到了时间序列。我背景里带进这件事的另一点是，我以前是期权交易员，我们当时得做随机微积分，解那类常微分方程，那是另一种连续时间金融，也就是我们刚才谈到的。

You see, you brought out time series. The other thing about my background that I bring into this is I used to be an options trader, where we actually would have to do stochastic calculus and solve those kinds of ODEs, which is another kind of continuous-time finance, which is what we talked about there.

是啊。我只是觉得奇怪，这件事到现在才成为优先事项，而现在一旦你解释清楚，就相对明显了，这是我们本该利用的东西。它就像是一个我们应该利用的维度。你心里还有别的维度吗？显然你正在把这条路线规模化，也许可以说你已经走了 10 年，但好吧，如果我们对连续时间的判断错了，还有什么？

Yeah. I just think it's weird that this hasn't been a priority till now, and now it seems like once you explain it, it's relatively obvious that this is a thing that we should exploit. It's like a dimension that we should exploit. Are there any others in your mind? Obviously you're scaling out this, you're maybe, let's call it 10 years into this journey, but okay, if we're wrong about continuous time, what else?

这是个好问题。我觉得有很多。当我们最初把 Liquid 作为一个想法启动时，我们在思考智能的效率和智能的适应性。我觉得我们在智能的适应性上探索得还不够多。对我们来说，前向传播基本上就是对模型做一次前向传播。很多前瞻性的竞争实际上就是前向传播的竞争。那么，如果能在运行中适应呢？很多人会告诉你，上下文学习正在解决这个问题，因为上下文系统在测试时引入更多数据时，它们会以某种方式求解某种反向传播，一种伪的、非常类似反向传播的情形。它就像最小二乘算法，因为上下文学习发生在测试时算力上。但总的来说，我认为能把前向传播和反向传播同时结合的方法，会是新一代架构，我认为这件事会有一个演进过程。

That's a good question. So I think there's a lot. When we started Liquid as an idea, we were thinking about efficiency of intelligence and adaptability of intelligence. I think we haven't explored too much on the adaptability of intelligence. Forward pass for us is basically a forward pass to a model. A lot of forward-looking competition is literally just a forward pass competition. So what if adaptability on the go? A lot of people would tell you that in-context learning is solving that problem, because in-context systems, when you bring more data at test time, they would somehow be solving for some sort of a backprop, a pseudo, very similar kind of backprop situation. It's like a least squares algorithm, as in-context learning is happening on the test-time compute. But in general, I think approaches where we can combine forward pass and backward pass at the same time, this would be a new generation of architectures, and I think there will be an evolution of this thing.

我听说 SSI 也在非常投入地研究类似液态神经网络最初想法的架构。这是我听说的。但我们拭目以待，拭目以待。

I know that I've heard that SSI is also working very much on similar architectures to the original ideas of liquid neural networks. So that's what I heard. But we will see, we'll see.

他们怎么还没联系你？拜托，怎么回事？

How have they not reached out? Come on, like what's going on?

但总的来说，我知道所有基础模型实验室，Anthropic、OpenAI 以及其他所有人，都在大规模投入。他们都有团队在研究替代架构。我还想说的另一点是，你为什么要一个替代架构？这是你必须问的问题。对我们来说，首先是效率，而现在其他人似乎都在关注事后优化，或者比如投机解码。人们尝试的一个领域是量化，另一个是训练的量化。这些是人们试图压缩信息的地方，你可以在那里做很多事，因为这是内核科学。我认为随着 AI 变得更好，我期待下一代 AI 在今天所做的事情上大幅提升，人们真的可以优化当前架构，从模型中获得效率。

But in general, I know that all the foundation model labs, Anthropic and OpenAI and everybody else, are massively invested. They have teams that are working on alternative architectures. Another thing that I want to say is, again, why do you want an alternative architecture? This is the question you have to ask. The first thing for us would be efficiency, and everybody else now seems to be focusing on post hoc optimizations, or really like speculative decoding, for example. One of the spaces people try is quantization, another quantization of training. These are the kind of places where people try to pack information, and you can do a lot in there, because these are kernel science. And I think with better AIs, I'm hoping for the next generation of AIs to get a lot better at what they do today, people can really optimize the current architectures to get efficiency out of the models.

现在，如果你想解锁下一代能力，那就是另一回事了。那可能是一种元算法。所以也想想，现在我们的编程工作流是什么样子？有一个编排器，还有很多智能体。所以未来，多智能体式的架构，那是我能想到的另一件事。我们可以从原子层面思考，也可以从系统层面思考。所以你还可以构建由更小计算单元组成的新系统。比如说，一个计算单元可以是神经元和突触，也可以是一个 LFM，或者它本身就是一个小的模型。然后你可以使用、捆绑，基本上构建这些系统的系统。所以这是架构的另一个方向，我会把它们称为认知架构，在那里你有多个智能体，你可以把它们捆绑在一起，构建一个完整的、全功能的系统。所以我认为我指出的一堆方向，我认为可能是令人兴奋的未来一步。

Now, if you want to unlock the next generation of capabilities, that's a different story. That could be a meta algorithm. So think about it also like, right now, how does our workflows of coding today look like? There's an orchestrator and there are a lot of agents. So in the future, multi-agentic kind of architectures, that's another thing that I can think about. We can think about it at the atom level or we can think about it at a system level. So you can also build new systems that are composed of smaller kind of units of compute. Let's say a unit of compute could be neurons and synapses, or it could be an LFM, or it could be a small model itself. And then you can use and bundle and you can build basically these systems of systems. So that's kind of another direction that architecture, and I would call them cognitive architectures, where you have multiple kind of agents that you can actually bundle them together to build a whole full-blown system as a whole. So I think a bunch of directions I pointed out that I think could be an exciting future step.

如果我转述一下，也许智能体应该能够把训练模型当作工作的一部分随手完成，显然 LFM 会相当有适应性，而且实际上不是 LFM，而是 LEAP，因为你还需要完整的平台，才能适应你部署的任何情境。

If I were to paraphrase, maybe agents should be able to train models casually as part of their job, and obviously LFM would be quite adaptable, and it's not actually the LFM, it would be LEAP, because you need the full platform as well in order to adapt to whatever the situation you're deploying in.

当然。

Definitely.

你列出了所有这些方向。你可能也有更近期的研究方向。你会说 Liquid 当前的研究议程是什么？

You've named all these directions. You probably also have more near-term research directions. What would you say is the current research agenda for Liquid?

我们非常专注于大规模多模态系统。我们在思考同时训练所有数据模态，同时也思考更长视野的推理。当然，这是我们想做的方向之一。想象一下，如果你想做可靠的端侧智能服务，比如助手。所有这些东西都需要模型有更高程度的可靠性。所以我们在这个方向上投入很多。

We are very much focused on massively multimodal systems. We're thinking about training all the data modalities at the same time, and at the same time longer-horizon kind of reasoning. Of course, this is one of the directions that we want to do. Imagine if you want to do reliable on-device intelligence kind of services like an assistant. All of those things require higher degrees of reliability for the models. So we work a lot on that direction.

我们投入很多。我们有一个效率部门。我们在大量层面的效率上工作。你可以想到内核设计，它延伸到推理，延伸到基础设施，快得多的强化学习基础设施，让我们从预训练到把模型交付到生产的所有流程，如何让所有基础设施更高效。这是公司的重点。

We work a lot. We have a department for efficiency. We work on massive layers of efficiency. You can think about kernel design, it goes to inference, this goes to infrastructure, RL infrastructure that is much faster, making all of our processes that are happening from pre-training to delivering a model to production, how to make all the infrastructure more efficient. That's the focus of the company.

然后，从研究的角度，我们也在看非人类可理解的数据。例如，想想信号，你也可以想想生物学。我们也有一些生物模型。例如，我们在构建 DNA 基础模型。你的 DNA，用你的眼睛，你基本上真的能读它。

Then, from a research point of view, we're looking also at non-human-understandable data. For example, think about signals, you can think about biology as well. We have some biomodels as well. For example, we're building DNA foundation models. Your DNA with your eyes, you kind of really read it.

它们是长上下文数据。它们不像词汇那样密集——你处理的其实只有四个“词汇”，也就是字母。但从序列长度的角度看，它们非常庞大。在那个领域有很多让我们兴奋的应用。我认为我们这类技术天然会推动我们去研究这种替代类型的数据，还有我们的架构——我们谈过动力系统和更长的数据序列。所以 DNA 数据可能是我们能带来价值的领域之一。而且正如我提到的，多模态研究目前在 Liquid 极其火热。

They're long-context data. They're not as dense as, let's say, in the vocabulary — they're like four vocabularies, letters that you're dealing with. But from the sequence-length point of view, they're very large. There are a lot of applications that we are excited about in that realm. I think our type of technology naturally pushes us into looking into this alternative type of data, as well as our architectures — we have talked about dynamical systems and longer sequences of data. So DNA data could be one of those places where we can bring value. And as I mentioned, multimodal research is extremely hot right now at Liquid.

对，对，对。你说视频、音频输入，视频、音频输出。视频、音频、文本输入，音频输出——

Yeah, yeah, yeah. You said video, audio in, and video, audio out. Video, audio, text in, audio out —

文本是默认的。文本是默认的。没错。

Text is assumed. Text is assumed. Exactly.

输入没问题。输出才是新的。

The in is fine. The out is new.

对。没错。那是——

Right. Correct. That is —

通常我不会指望像你这样的人会在意这个。从一个 7B 模型输出可用的图像、音频、视频，这现实吗？

Typically I would never expect someone like you to care. Is it realistic to have a usable image out, audio out, video out from a 7B model?

我不认为能从中得到一段像样的、非常高质量、高保真的视频。但从方向上说，如果你想解决物理问题、对世界有模糊的理解——想象你在训练这些 LFM，比如在 7B 及以下的范围内。如果你想用世界建模的语境训练一个 Liquid 基础模型，你能够在模型内部编码某种物理知识，但模型无法输出非常高保真的东西。它会增进自身的理解，但无法输出那个。它基本上能拥有稍微更好的内部表征。如果你用世界建模重新训练它们，尤其是视频生成很重要的视频应用——不只是娱乐，而是机器人在工厂里作业这种落地的产业工作——视频理解其实被严重低估了。目前还远远不够；大多只是插值图像。

I don't think it is feasible to get a proper, very high-quality, high-fidelity video out of that. But directionally, if you want to solve physical problems and have a vague understanding of the world — imagine you're training these LFMs, let's say in the range of 7B and below. If you want to train a Liquid foundation model with a world-modeling kind of context, you would be able to encode some sort of physical knowledge inside the model, but the model would not be able to output something very high-fidelity. It would add to its own understanding, but it would not be able to output that. It can basically have a little bit of a better internal representation. If you retrain them with world modeling, especially for video applications where video generation matters — not just for entertainment, but grounded industry work of a robot operating in a factory — video understanding is actually so underrated. There's still not enough; mostly it's just interpolated images.

我们想要更多。

We want more.

我告诉你，在多模态智能这个领域，到目前为止真的有很多要学，因为真正高度多模态的系统其实并不多。对整个基础模型领域来说，这是一个非常非常开放的研究课题。它将开启某种完全不同的东西。

I'll tell you, in the space of multimodal intelligence there's really a lot to be learned so far, because there really aren't that many highly multimodal systems. This is a very, very open research topic for foundation models as a whole. It's going to enable something completely different.

我想再确认一件事，跟你领域相邻的，就是 Modular。他们刚被收购——被高通收了。他们刚发布了 1.0 开源版。你们以从 JAX 切换到 PyTorch 闻名。你们探索过 Mojo 吗？它本该是给你们这类人用的——你们在做自定义内核工作，你们是以 Python 为核心的。如果它没成，为什么？

One thing I wanted to double check is something adjacent to your space, which is Modular. They just got bought — went to Qualcomm. They just released their 1.0 open source. You guys famously switched from JAX to PyTorch. Did you explore Mojo? It's supposed to be for you guys — you're doing custom kernel work, you're Python-centric. If it didn't work, why?

Modular 基本上是在内核这一层运作——他们在基础模型下面一层。他们试图像我们一样，像瑞士一样中立，跟所有硬件厂商合作，但他们现在被锁进高通了。其实几周前我跟 Chris 同台过，在一个 AMD 的小组讨论上。我可以告诉你，他是我的英雄之一——Chris Lattner，他太了不起了。

Modular is basically operating on a space of kernels — they're one layer below the foundation models. They try to be like us, like Switzerland, working with all the hardware providers, but they now got locked into Qualcomm. I was on a panel with Chris actually, some weeks ago, on an AMD panel. And I can tell you, he's one of my heroes — Chris Lattner, he's unbelievable.

对，他可能正在听。

Yeah, he's probably listening.

是的。但我能告诉你的是，我们在 Liquid 从第一性原理出发努力做的一件事，就是不想重新造轮子。比如，有很多推理公司试图取代 llama.cpp，或者围绕 llama.cpp 构建封装来做端侧 AI。几乎所有那些没有被锁进某一个生态、又试图成为适用于所有硬件的通用语言的东西——它们全都更差。如果你深挖，它们比原版 llama.cpp 更差。所以我们不想换基础。有一些成功的案例是锁进单一生态的，我觉得 Mojo 会走向类似的方向。既然他们正被锁进高通生态，我认为会有多得多的成功。但平台本身让在那个层面引入智能——承载智能——变得极其容易。所以他们提供的东西其实是有用的，但我们自己的部署还没用上这些。

Yes. But what I can tell you is that one thing we try to do at Liquid from first principles is we don't want to reinvent the wheel. For example, there are a lot of inference companies trying to replace llama.cpp or building wrappers around llama.cpp for on-device AI. Almost every single one of these things that are not locked into one ecosystem and try to be a general-purpose language for all hardware out there — all of them are worse. If you dig in, they're worse than the original llama.cpp. So we don't want to change base. There are some successful ones that lock into one ecosystem, and I feel like Mojo is going to go in a similar direction. Now that they're getting locked into the Qualcomm ecosystem, I think there's going to be a lot more success. But the platform itself makes it extremely easy to bring intelligence at that layer — for hosting intelligence. So what they provide is actually useful, but we haven't been using those for our own deployments yet.

你说 llama.cpp 作为基线更好，但这只是生产环境的打磨，还是这种方法有什么根本性的东西？我想弄清楚。

You say llama.cpp is better as a baseline, but is it just production hardening, or is this something fundamental about the approach? I'm trying to figure it out.

它是一个编译器。它是高度优化的 C++ 内核，就是这样。

It's a compiler. It's very optimized C++ kernels, that's what it is.

是的，但 Chris Lattner 懂 C++——这不是问题。

Yes, but Chris Lattner knows C++ — this is not an issue.

正如我说的，要统一它极其困难。比如，我不相信 Mojo 那种平台的统一性。你需要做自定义工作，而这正是 Modular 团队实际投入的工作。所以运行在，比如说，AMD 上的那个 Mojo 版本——你必须为 Mojo 设计特定的内核，让它们针对 AMD，或者针对高通。他们做的所有这些底层工作都是非常扎实的工作，如果真有人能做到，那基本上就是 Chris 的团队。这就是我要说的。但我会把这些看作——当然，Chris 做的，Jonathan Ross 在 TPU 上做的，以及他们正在做的所有这些事。这些是他们做的非常基础性的工作。但我要告诉你，我们会坚持用最原生的语言、最原生的平台，让我们能尽可能贴近地接触计算机。生态永远会赢。这就是一个非常残酷的答案。这对新进入者不公平。不公平。这就是个难题。

As I said, it's extremely difficult to unify it. I don't believe, for example, in the uniformity of the Mojo kind of platform. You need to do custom work, and this is the work that the Modular team actually put into. So the version of Mojo that runs on, let's say, AMD — you have to design certain kernels for Mojo so that they're AMD-specific, or Qualcomm-specific. All that underlying work they do is very solid work, and if anyone can pull it off, it's basically Chris's team. That's what I would say. But I would consider those, of course — what Chris does, what Jonathan Ross did with TPUs, and all those kinds of things they're doing. These are the very fundamental works that they do. But I'll tell you, we would stick to the most native language, the most native platform that allows us to touch the computer as close as possible. Ecosystem always wins. It's just a really brutal answer. It's not fair to new entrants. It's not fair. It's just a hard problem.

我也想鼓励人们——你知道，现在有很多创业者——我现在已经创业四年了。很多朋友和人来问我，嘿，创业旅程怎么样？我只想说，如果一个问题很难解决，那它实际上值得去做；这让它更有吸引力。它应该对你更有吸引力，因为简单的问题——其他人都会去解决。

And I would also encourage people—you know, a lot of entrepreneurs these days—I'm now four years into our entrepreneurial journey. A lot of my friends and people come and ask, hey, how's the journey? I would just say, if a problem is hard to solve, it's actually worth it; it makes it more attractive. It should make it a lot more attractive for you, because easy problems—everybody else is going to go and solve them.

也许我本来想用这个问题来结束这期节目——你知道，明年就是 Transformer 的十周年了。注意力机制就是你所需要的一切吗？

Maybe the question I was going to end the episode with is mostly—you know, it's going to be the 10-year anniversary of the Transformer next year. Is attention all you need?

注意力机制绝对是你需要的一个要素。我认为我们将在很长一段时间内与基于注意力的系统共存。所以注意力机制肯定会存在于任何——就像，你知道，你的眼睛是如何工作的？但你需要你的身体。我偶然注意到——实际上，你并没有看到整个视野；你的眼睛一直在四处跳动，而你的大脑在进行合成，对吧？是的，你看到的是两帧之间的差异。顺便说一句，这就是你所看到的——所以基本上你看到的是差分。

Attention is definitely one element that you need. And I think it will be a long time that we will be living with attention-based systems. So attention is definitely going to be inside any of the—it's like, you know, how do you see how your eyes are working? But you need your body. I randomly noticed—actually, you're not seeing the whole view; your eyes are just bouncing around all the time and your brain is compositing, right? Yes, you see the difference between two frames. By the way, this is what you see—so it's basically you see the differential.

哦，左眼和右眼？不，就像你基本上——人类看东西的方式——你不是看到一帧。你看到两帧，两帧的差异。

Oh, left and right eye? No, like you're basically the way that humans see—you don't see one frame. You see two frames, the difference of two frames.

我得去查一下这个。查一下这个。

I need to go look this one up. Look this one up.

所以注意力机制是一个组成部分。

So attention is one component.

注意力机制绝对是一个组成部分。但话说回来——

Attention is definitely one component. But then again—

硬件意识、定制化。

Hardware awareness, customization.

是的。具身——还有另一个方面。所以我会说，智能想要部署到数据中心之外的社会中，就必须有某种具身。实际上在虚拟世界内部的智能体，它们也对自己所操作的虚拟物理环境有所理解。所以顺便说一句，具身对虚拟智能体也适用。所以你必须在其中包含这些方面。所以具身智能和具身推理是非常重要的。然后自适应智能是我们真正想要添加的另一个层次。

Yeah. Embodiment—like there's another aspect. So I would say intelligence that wants to get deployed in the society outside of data centers, you got to have some sort of embodiment. Agents that are actually inside the virtual world, they have also an understanding of their virtual physical kind of environments that they are operating in. So embodiment is also true for virtual agents, by the way. So you got to have those kind of aspects in there. So embodied intelligence and embodied reasoning is something that is very important. And then adaptive intelligence is another level that we really want to add.

那和路由有关吗？或者你说的自适应是什么意思？

Is that routing related or what do you mean by adaptive?

哦，自适应我指的是像能够不断进化的系统，比如持续学习，不断改进的系统。

Oh, adaptive I mean like systems that can always evolve like continual learning, continuously kind of improving systems.

好的。嗯，这真是一次很好的深入探讨。有什么我们应该问你但没问到的问题吗？

Okay. Well, that was a really good deep dive. Is there anything that we should have asked you that we didn't cover?

没有，我觉得这是一次很棒的交流。非常感谢。

No, I think that was a great session. Thank you so much.