关系基础模型与 AI 虚拟细胞

我们最近取得的突破，刚刚发布了第二个版本，就是我们所谓的“关系基础模型”。这是一个预训练的基础模型，能够对结构化关系数据进行推理。这个模型的能力非常惊人，它可以在任何数据库和任何预测任务上做出准确预测，而无需任何模型训练。

The recent breakthrough that we had and we just released in the second version is our what we call a relational foundation model. And that's a pre-trained foundation model that can reason over structured relational data. And it's crazy what this model can do. It can make accurate predictions on any database and any predictive task without any model training.

好了，各位。欢迎收听新一期的 TwiML AI 播客。我是主持人 Sam Cherington。今天和我一起的是 Jure Leskovec。Jure 是 Kumo 的联合创始人兼首席科学家，也是斯坦福大学的教授。在开始之前，请务必在您收听节目的地方点击订阅按钮。Jure，欢迎来到播客，很高兴终于能和你连线。

All right, everyone. Welcome to another episode of the TwiML AI podcast. I am your host, Sam Cherington. Today I'm joined by Jure Leskovec. Jure is co-founder and chief scientist at Kumo and a professor at Stanford University. Before we get going, be sure to hit that subscribe button wherever you're listening to today's show. Jure, welcome to the podcast. It's great to finally connect with you.

嗯，很高兴来到这里。

Yeah, great to be here.

我很期待我们的对话。我们将深入探讨你在关系学习方面的工作，以及你在斯坦福大学和 AI for Science 等领域做的其他有趣事情。但让我们从那里开始吧。请跟我们谈谈你的研究重点。

I'm looking forward to our chat. We're going to be digging into your work on relational learning as well as some of the other interesting things you're up to at Stanford and around AI for science and more. But let's start there. Tell us a little bit about your research focus.

嗯，很好。我是斯坦福大学计算机科学系的教授。我喜欢说，这里是未来发生的地方。所以总有令人兴奋的研究在进行。我们最近的重点可以说是两个领域。首先是 AI for Science。具体来说，我们有一个项目叫做 AI 虚拟细胞，我们基本上在构建下一代基础模型，这些模型能够表示人类细胞、患者以及细胞内的单个分子，并让我们能够跨这些复杂的生物医学数据进行推理，用于发现新的癌症疗法、分子设计，推理所有不同的生物医学数据模态和对象，以及它们如何相互作用，以帮助加速科学。所以它涵盖了从理解蛋白质的底层基础模型，到聚合细胞内分子以表示单个细胞的模型，再到下一级模型，即组织或患者是细胞的集合，细胞是分子的集合，让我们构建模型，将所有知识聚合起来，形成对患者的忠实表示。这非常有帮助，因为现在我们拥有的表示更加稳健，完全由数据驱动。模型中没有插入任何生物学知识，一切都是从数据中涌现出来的。我们能从中学习到的东西令人惊叹。所以，我认为这是我们一直在做的一条工作线。

Yeah, great. So I'm a professor at Stanford here in the computer science department. You know, where the future happens, I like to say. So there's always exciting research going on. Our focus recently has been, I would say, on two areas. First is AI for science. And in particular, we have a project that we call AI Virtual Cell, where we are basically building next-generation foundation models that allow us to represent human cells, patients, as well as individual molecules in cells, and allow us to reason across this complex biomedical data for discovering new cancer therapies, molecule design, reasoning about all different biomedical data modalities and objects, and how they interact with each other to help speed up science. So it's everything from foundation models at the lower level of understanding proteins, to then models that aggregate the molecules in the cell to represent a single cell, and then the next level models that now say, a tissue or a patient is a collection of cells, cells are a collection of molecules, let's build models that just aggregate all this knowledge in a very faithful representation of a patient. And that helps a lot because now the representations we have are much more robust, driven purely from the data. No biology is inserted in the model. Everything is emergent out of the data. And it's amazing how much we can learn from that. So that, I would say, is one line of work we've been working on.

我忍不住要暂停一下问：你是端到端地训练所有这些，还是逐个训练单个模型或表示，然后在定义好这些模型后再进行聚合？

And I can't help but hit pause and ask: do you train this all end to end, or are you training an individual model or representation at a time and then aggregating it after you've got these models defined?

好问题。我们现在做的方式实际上是：第一个科学问题是：这到底可不可能？对吧？你能说一个细胞是细胞内分子的表示吗？那么，假设细胞内的分子是蛋白质。我可以使用蛋白质语言模型来表示细胞中的每个蛋白质。现在细胞需要聚合所有这些蛋白质的信息来说：我是一个细胞，这是我的状态。好了。现在你有了细胞的表示，你可以构建一个患者级别的模型，说患者是给定状态下细胞的集合，这些细胞由其中的蛋白质组成。我们能否跨这些数量级不同的尺度收集信息，以获得一个强大的数据驱动的患者表示？有趣的是，这完全是可行的，并且完全以无监督、自监督的方式训练。对吧？所以你不需要插入任何人类偏见，任何人类生物学知识。生物学从数据本身涌现出来。比如细胞类型、细胞状态、它们之间的关系。那种人类生物学，我们如何描述它，实际上直接从数据中涌现出来。所以模型学会了如何最好地描述底层过程和现象，而无需我们从顶层推动。这就是令人兴奋的涌现能力。

Great question. So the way we are doing it right now is actually the first scientific question was: is this even possible? Right? Could you say a cell is a representation of molecules that are inside the cell? So now let's say molecules inside the cell are the proteins. I can use the protein language model to now represent every protein in the cell. And now the cell needs to aggregate information from all these proteins to say: I am a cell, this is my state. All right. Now that you have a representation of the cell, you can build a patient-level model that says a patient is a collection of cells in given states that are composed from the proteins that are in there. And can we kind of collect this information over these orders of magnitude different scales to get a strong data-driven representation of the underlying patient in this example? And the interesting thing is that this is purely doable and it's trained purely in an unsupervised, self-supervised way. Right? So you don't need to insert any human bias, any human knowledge of biology. The biology emerges from the data itself. Like cell types, cell states, relationships between them. That kind of human biology, how we describe it, actually emerges directly from the data. So the model learns how to best describe the underlying processes and phenomena without us pushing it from the top. That's the exciting emergent capability there.

告诉我这个问题是否有道理。我认为它与你描述训练过程的方式有关。但你训练的数据集本质上是机制性的还是行为性的？也就是说，你是观察细胞的一些行为，然后在该数据上训练，然后某种机制的信实表示是涌现出来的？还是数据本身具有机制属性？

And tell me if this question makes sense. I think it's related to the way you're describing the training process. But is the dataset that you're training on mechanistic in nature or behavioral in nature? In the sense of, are you observing some behaviors of cells and then training on that data, and then some kind of faithful representation of mechanisms are emergent? Or does the data have mechanistic properties to it?

我们在这个案例中使用的数据叫做单细胞 RNA 测序数据。这是大型国际联盟正在收集的数据。但基本上，它说的是你可以从某个组织中取一个样本，然后对该样本中的每个细胞，测量该细胞内不同蛋白质分子的数量。所以每个细胞现在由一个 20,000 维的向量表示，告诉我该特定细胞中特定蛋白质的丰度。每个细胞根据其类型、状态等，这些蛋白质的比例不同。这就是原始输入数据。然后，当然，因为我们知道蛋白质是什么，我们实际上可以通过 ESM 或 AlphaFold 引入蛋白质信息，然后事情就变得非常有趣了。

The data we are using in this case is called single-cell RNA-seq data. This is data that large international consortia are collecting. But basically, what it says is that you can take some sample from some tissue, and then for every cell in that sample, you measure the number of different protein molecules inside that cell. So every cell is now represented by a 20,000-dimensional vector that tells me the abundance of that specific protein in that specific cell. And every cell has different ratios of these proteins depending on its type, depending on its state, and things like that. So that's the raw input data. And then, of course, because we know what the protein is, we can actually bring the protein information through ESM or through AlphaFold, and then it gets very interesting.

所以基于蛋白质，这既带来了机制也带来了行为。

So being protein-based, that brings in both mechanism and behavior.

完全正确。完全正确。完全正确。然后，当然，你可以一直连接到表型，因为我们现在做的是，我们可以从患者身上取一滴血，而不是进行传统的血液筛查，我们可以做这种单细胞 RNA 测序分析。所以现在我们基本上可以分析一滴血中的每一个细胞。为什么血液有趣？因为它循环全身。它某种程度上捕捉了整个身体的免疫状态。所以我们能够检测疾病、理解患者轨迹等，仅仅通过这一滴血的数字孪生。

Exactly. Exactly. Exactly. And then, of course, you can connect this all the way to the phenotype because what we are doing now is we can take a single drop of blood from a patient and, rather than running a classical blood screen, we can do this single-cell RNA-seq analysis. So now we basically can profile every single cell inside the drop of blood. And why blood is interesting is because it circulates through the entire body. It kind of captures the immune state of the entire body. So we are able to detect diseases, understand patient trajectories, and things like that just from this digital twin of a single drop of blood.

非常有趣。也与你关注的另一个领域——关系数据——非常不同。

Super interesting. Also very different from the other thing that you focus on, which is relational data.

嗯。让我给你讲个故事，说明为什么这并不那么不同。

Yeah. Let me tell you a story why this is not so different.

好的。

Okay.

我真正感到兴奋的，是从根本上拆解事物，理解不同部分如何相互作用和协同工作。我最初是在计算社会科学领域这样做的。我对人们如何互动感到兴奋。当我开始研究时，社交媒体刚刚兴起。我的观点是，我可以把社交媒体当作望远镜来观察人类行为。通过人们使用手机和社交媒体留下的数字痕迹，我可以研究人类行为。这一切都关乎网络，即人们互动的图。这不仅让我们理解社交媒体上的现象，还能极其精确地模拟新冠疫情的传播。我们通过计算分析和预测了经济重启后病毒将如何传播。如果我们提高餐厅、健身房、教堂或其他地点的客容量，就能预测病毒的传播方式。其底层结构就是网络。那么，生物学是什么？生物学也是一个网络。它关乎分子在细胞中聚集并发挥作用。组织又是什么？是细胞聚集、相互交流、以特定方式组织起来，使我的皮肤具有特定结构和层次。这本质上也是一个网络，一个交互图。然后是关系数据——每个企业都有的数据库表格中的数据，这是最有价值的数据。它也是一个图，捕捉了组织内不同实体之间的交互。

So what I'm really excited about fundamentally is to always take things apart and understand how different parts interact and work together. I started doing this in computational social science. I was excited about how people interact with each other. When I started my research career, social media was just emerging as a phenomenon. My view was that I could use social media as a telescope into human behavior. I could study human behavior through the digital traces people produce by using cell phones and social media. It's all about networks, graphs of people interacting. This allowed us not only to understand phenomena on social media, but also to model the spread of the COVID pandemic super accurately. We computationally analyzed and predicted how the virus would spread as we reopened the economy. If we increased occupancy levels at restaurants, gyms, churches, or other locations, we could say how the virus would spread. That underlying structure was a network. Now, what is biology? Biology is a network. It's about molecules coming together to do something in a cell. What is a tissue? It's cells coming together, talking to each other, organizing in a given way so that my skin has a given structure and layers. That's essentially a network, a graph of interactions. Then there's relational data—data in tables in a database that every enterprise has, the most valuable data. That's also a graph capturing interactions of different entities inside an organization.

当我看到你在关系深度学习方面的一些工作时，我想起了其他一些关于表格数据深度学习的讨论。但那些讨论往往集中在单张表上，而不是企业数据中出现的那些关系——不同表通过键关联。你能谈谈这两个研究和实践领域是如何相互关联的吗？

When I look at some of the work you're doing around relational deep learning, it calls to mind other conversations I've had that focus on deep learning for tabular data. But that tends to be focused on a single table, as opposed to these relationships that arise in enterprise data where you have different tables linked by keys. Can you talk a little bit about how these two areas of research and practice relate to one another?

我来解释一下。首先，想想机器学习，在过去 30 年里它并没有真正改变。我们有过决策树、支持向量机、逻辑回归、神经网络、梯度提升树等等。但本质上始终是：你有数据，进行特征工程，得到一张特征表，加上标签，然后训练一个监督模型，从特征预测标签。我们一直在重复这个过程。预测模型可能是深度神经网络，但我认为，AI 并没有像彻底改变计算机视觉或自然语言理解那样，改变结构化数据领域。让我具体说明。计算机视觉和自然语言理解的重大突破，都是构建直接在原始数据上学习的神经网络。过去，在计算机视觉中，你要做特征工程——SIFT 特征、Gabor 滤波器——来描述图像，从而预测其中是否有汽车。自然语言处理也类似；IBM 赢得 Jeopardy 的系统是高度手工打造的，但需要 300 人构建，而且很脆弱。Transformer 只是对 token 进行学习——没有语法，没有句法——就是直接在原始数据上训练神经网络。结构化表格数据却没有发生同样的事情。在那里，我们不在原始数据上学习；我们运行 SQL 查询、ETL 和特征工程来提取信号用于预测。当我们提出关系深度学习的想法时，我们的目标就是从根本上颠覆这一点。我们问：为什么不能直接在原始关系数据上学习？为什么总是要在单张表上学习？考虑一个多表的例子：一组客户和一组产品——两张表。每个客户有一个 ID，每个产品有一个 ID。可能还有一张交易表，将客户 ID 和产品 ID 关联起来，包含时间和价格。这是一个三表模式。组织有 50 或 60 张表的模式。我们的问题是：如何直接用神经网络在多表数据上学习？答案出奇地简单：把数据库表看作实体之间关系的图。作为一个图研究者，我喜欢用图来思考。图有顶点——节点——可以是我的用户、产品、交易等。连接表明某个用户 ID 属于某笔交易，该交易又属于某个产品，从而形成从用户到交易再到产品的路径。一旦有了图，我们就可以应用图深度学习，比如图神经网络，它将深度学习推广到图结构数据，并通过训练获得准确的预测。这会产生两个效果。

Let me explain. First, if we think about machine learning, it hasn't really changed over the last 30 years. We used to have decision trees, then support vector machines, logistic regression, then neural networks, then gradient boosted trees, and so on. But fundamentally, it has always been: you have your data, you feature engineer a single table of features, you add a label, and you train a supervised model that predicts the label from the features. We've been doing that over and over again. The predictive model might be a deep neural network, but I would argue that AI has not transformed the structured data space in the same way as computer vision or natural language understanding have been fundamentally transformed. Let me quantify what I mean. The big breakthrough in both computer vision and natural language understanding was building neural networks that learn directly on the raw data. In the old days, in computer vision, you would do feature engineering—SIFT features, Gabor filters—to describe an image so you could predict whether there is a car in it. In NLP, it was similar; IBM won Jeopardy with a system that was heavily hand-engineered, but it took 300 people to build and was brittle. Transformers just learn over tokens—no grammar, no syntax—just a neural network directly on raw data. The same thing is not happening on structured tabular data. There, we don't learn on raw data; we run SQL queries, ETL, and feature engineering to derive signals for prediction. When we came up with the idea of relational deep learning, our goal was to fundamentally disrupt this. We asked: why can't I just learn directly over raw relational data? Why do we always have to learn over a single table? Consider a multi-table example: a set of customers and a set of products—two tables. Each customer has an ID, each product has an ID. There might be a transactions table linking customer ID to product ID with time and price. That's a three-table schema. Organizations have schemas of 50 or 60 tables. Our question was: how could we learn directly with a neural network over this multi-table data? The answer is surprisingly simple: think of the database tables as a graph of relationships between entities. As a graph person, I like to think in terms of graphs. Graphs have vertices—nodes—which would be my users, products, transactions, etc. The connections show that a user ID was part of a transaction, which was part of a product, forming a path from user to transaction to product. Once we have a graph, we can apply graph deep learning, like graph neural networks, which generalize deep learning to graph-structured data, and train to get accurate predictions. Two things happen.

首先，你不再需要手动做特征工程了。所以速度更快，训练这些模型所需的精力也少得多。其次，你的模型会更准确。然后你会问，为什么我的模型能更准确？答案和计算机视觉中发生的情况非常相似。对吧？如果你说，我是人类，我知道汽车是什么。所以我会构建完美的特征来检测图像中是否有汽车。我了解汽车，我开过它们，我是汽车专家，我能构建检测汽车的最佳特征。没有哪个头脑正常的人会这么说，对吧？但在机器学习、数据科学预测中，人们仍然在说，我是领域专家，我来设计特征。你的特征只是你对数据的一些任意、带有偏见的汇总统计量，你凭空想出来的。把它作为特征放进训练表，重新训练模型，然后看看是否提高了准确率，对吧？而通过梯度下降训练的神经网络能够进行更细微的、几乎像是特征发现的过程，通过在图结构上进行注意力机制来提取更多的信号。所以我们看到模型准确率有两位数的提升，因为神经网络能够从原始数据中提取更多信号。而且我完全透明地说，如果你处理的是一个超级简单、落在一条直线上的问题，那么没有任何神经网络能比线性模型更好。所以我想说的是，我不能保证你总能获得更好的性能，因为有时数据是线性的，如果你恰好训练了线性模型，你已经有了很好的性能，没什么可做的了。但大多数数据不是线性的，要复杂得多，这就是好处出现的地方。所以这就是关系深度学习背后的关键思想：现在我们可以让神经网络直接在原始数据库数据上学习。不需要构建那些手动特征管道和特征存储，它们非常痛苦，导致许多不同类型的错误、不一致、信息泄露和时间旅行问题，使得将模型投入生产变得极其困难。相反，只需引入原始数据，使用神经网络，就能获得更好的结果。这就是我们这样做的理念和原因。

The first thing that happens is you don't have to do manual feature engineering anymore. So it's much faster. It requires much less effort to train these models. And the second thing that happens is your models are more accurate. And then you say why can my models be more accurate? And the answer is very similar to what happens in computer vision. Right? If you are saying I am a human, I know what a car is. So I will build perfect features that detect whether there is a car on the image or not. I know cars. I drive them. I'm such a car expert. I can build the best features for detecting cars. Nobody in the right mind claims that, right? But you know in machine learning, data science prediction, people are still saying, you know, I'm the domain expert. I'll engineer the features. Your features are just some arbitrary human biased summary statistic of your data that you kind of dreamt up with. Put it as a feature in your training table. Retrain the model and then you saw whether that increased the accuracy or not, right? And a neural network that trains with gradient descent is able to do so much more nuanced almost like feature discovery by basically attending over this graph to extract much more signal. So we see this double-digit increases in model accuracy because the neural network is able to extract more signal out of the raw data. And I will just, full transparency, if you are working on a super simple problem that falls on a line, then no neural network is ever going to be better than a linear model. So what I'm basically trying to say, I cannot guarantee that always you will get better performance because sometimes the data is linear and if you happen to train the linear model to it, you already have good performance, there's nothing more you can do. But majority of the data is not linear, is much more complex, and that's where the benefit happens. So that's kind of the key idea behind relational deep learning: now we can have neural networks just learn directly on the raw database data. Don't need to build this manual feature pipelines and feature stores that are super painful and lead to so many different kinds of bugs and inconsistencies and information leakage and time travel, making putting models in production super hard. Rather, just bring the raw data, have a neural network, and get better results that way. So that's the philosophy and the reasons why we are doing this.

你能给我们举一些例子，说明你们用这些模型预测什么类型的事情吗？你们是主要预测关于结构的事情，还是预测单个值？你们如何看待模型的能力？

Can you give us some examples of the types of things you're trying to predict with these models? Are you trying to predict things that are primarily about structure or are you trying to predict individual values? How do you think about what the models are capable of?

这是个好问题。我现在描述的这个框架非常通用，你可以引入任意一组表、它们之间的任意连接、任意列。底层的数学表示基本保持不变，当然底层的图会变化，但图神经网络或图 Transformer 可以应用上去。所以你想预测什么取决于数据。如果你有一个交易图，例如，我们看到在欺诈检测方面效果很好，各种欺诈检测、反洗钱、账户级欺诈、交易级欺诈。效果非常好。你只需将这些异构的多表数据整合在一起，然后学习什么是欺诈。欺诈很有趣，因为它非常非平稳。欺诈者一直在试图钻系统的空子，所以作为机器学习工程师，你总是落后，你的模型总是在退化，你会想，好吧，我如何设计下一个特征，如何设计下一个特征？如果你有一个神经网络，直接从原始数据中提取信号，那就好了。所以欺诈是一个例子。你可以把欺诈看作一个分类任务。然后你可以考虑很多回归类型的任务，例如客户行为方面的客户流失、下一步最佳行动等。然后你还可以考虑图方面的链接预测。所以预测两种实体之间的链接。典型的任务是推荐系统，因为它预测客户（用户）和产品之间的链接。所以我们在推荐系统中看到了很好的应用，用于广告、产品推荐等。

That's a great question. The way I describe the framework right now, it's very generic in a sense that you can bring any set of tables, any set of connections between them, any set of columns. The underlying mathematical representation kind of remains the same, and of course the underlying graph changes, but the graph neural network or the graph transformer can be applied to that. So what would you want to predict depends on the data. If you have a transaction graph, for example, where we see great results is on fraud, all kinds of fraud detection, anti-money laundering, account level fraud, transaction level fraud. It works beautifully. You just bring this heterogeneous multi-tabular data together and just learn over it what fraud is. And fraud is interesting because it's so non-stationary. Fraudsters are trying to game the system all the time, so as a machine learning engineer, you are always behind, your model is always deteriorating, and you're like, okay, how do I design the next feature, how do I design the next feature? You have a neural network if you just pick the signal directly out of the raw data. So fraud is an example. Fraud you can think of as a classification task. Then you can think a lot around regression type tasks, for example for customer behavior in terms of customer churn, next best action, things like that. And then you can also think about in graph terms about link prediction. So predicting links between two types of entities. The canonical task there is a recommender system because it's predicting a link between the customer, the user, and the product. So we've seen great uses of this in recommender systems for ads, product recommendations, and things like that.

从历史上看，当我与人们谈论深度学习、机器学习用于表格数据时，结果……我不知道怎么形容最好。我总觉得我们还没完全到位。你认为你正在做的事情也是如此吗？还是说，问题在于缺少了一个环节，而这个缺失的环节就是图结构，现在我们有了它，就能做得更多。我试图将你的工作和说法与这些技术在文本和图像上已被证明极其有效、现在应用于表格数据的更广泛结果联系起来。

And historically, when I've talked to folks about deep learning, machine learning for tabular data, the results were, I don't know the best way to characterize this. Like I always get the impression that we're not quite there yet. And would you say the same is true for what you're doing or is it an issue of like there was this missing link and that missing link is the graphical structure and now we have it and we're able to do much more. I'm trying to kind of ground what you're working on and saying with this broader results of applying these techniques that have shown to be extremely effective with text and images to tabular data.

这是个很好的观点。当我们说表格数据和表格机器学习时，这是研究单表问题的社区。数据已经被扁平化、预格式化、汇总以适应单个表。我们试图获得比 XGBoost 之类更好的结果。我们看到的是，这些模型和架构，深度学习，并没有真正取代 XGBoost。XGBoost 仍然是主力。也许在个别例子中，你可以做得更好，但它仍然是主力。我对这个单表问题不太感兴趣的原因在于，这不是要解决的正确问题。我不知道有哪个组织把所有数据都放在一个表里。所以困难的部分和信息丢失的地方，是在从丰富的关联结构转到单表的过程中，而一旦你进入单表，我们几乎就是在谈论二阶效应了。

That's a great point. Like when we say tabular data and tabular machine learning, this is the community that works on single table problems. The data has already been flattened, pre-formatted, summarized to fit in a single table. We're trying to get better results than we might get with XGBoost or something like that. And what we see there is that these models and architectures, deep learning, didn't really displace XGBoost. XGBoost is still the workhorse. Maybe on individual examples, yeah, you can do better, but it's still the workhorse. And the reason I'm kind of less interested in this single table problem is because that's not the right problem to solve. I don't know any organization that has all their data in a single table. So the hard part and where the information gets lost is when you go from this rich relational structure into the single table, and once you are in a single table, then we are kind of talking almost like second order effects.

你用了这个架构吗？用了那个架构吗？用了这个表格模型或表格基础模型吗？对吧？所有信息都在那个单表里，所有方法提取信息的能力都差不多。我认为真正的区别在于，如果你退一步说，单表模型不是难点，它不够通用或现实。你需要的是多表场景，因为那才是你拥有的原始数据。它不是某种汇总、特征工程后的数据，而是原始数据，其中包含大量在数据被压平到单表时丢失的信号。所以对我来说，单表问题已经解决了。差异只是二阶效应。未解决的是多表问题，那里才隐藏着真正的收益。

Did you use this architecture? Did you use that architecture? Did you use this tabular model or this table foundation model or not? Right? Like all the information is there in that single table and all the methods are about equally good at extracting it. Right? I think where the difference happens is if you actually make a step back and say, hey, single table model is not the hard part. It's not general or realistic enough. Where you need to go is the multi-table setting, because that's truly the raw data you have. It's not some summarized, featurized data. It's the raw data, and there is much more signal there that got dropped when the data got flattened into a single table. So to me, single-table problems are solved. I think the differences are kind of second-order effects. What is unsolved is the multi-table problem. That's where the wins are being hidden.

那么你如何看待这些类型问题的基准测试性能？有没有针对多表预测问题的既定基准？

So how do you think about benchmarking performance for these types of problems? Are there established benchmarks for multi-table prediction problems?

实际上，由于机器学习的整个历史，单表数据非常多。我认为即使人们从原始数据开发新基准，他们也只发布那个单表，因为每个人都在单表上学习，对吧？所以我们在斯坦福的做法是，好吧，多表基准在哪里？没有多表基准。即使你看 Kaggle，在数千场比赛中，只有四场是多表的。其他所有比赛的特征都已经为你工程化好了。有一个单表，然后你开始 bagging、boosting 和创造技巧，直到你赢，对吧？所以我们通过收集和整理我们在网上能找到的开放多表格数据集，在斯坦福创建了一个基准。我们称之为 RelBench。我们现在有两个版本的 RelBench。它包含大约 40 个不同的预测任务，涉及大约 10-15 个不同的数据库。还有趣的是，SAP，这家德国大型 IT 公司，发布了一个企业数据的多表格基准，叫做 SALT。所以我认为这两个是主要的表格或多表格（即关系型）基准：SAP 的 SALT 和我们通过斯坦福一直在做和推广的 RelBench 系列工作。

Actually, there is quite a lot of single-table data out there because of all the history of machine learning. And I think even when people develop new benchmarks from raw data, they just release that single table because everyone learns on the single table, right? So what we did at Stanford was like, okay, where is a multi-table benchmark? And there's no multi-table benchmark. Even if you look at Kaggle, out of thousands of competitions, you know, there are four that are multi-table. All the others have features already engineered for you. There is a single table, and you start bagging and boosting and creating tricks until you win, right? So we created a benchmark at Stanford by collecting and curating open multi-tabular datasets that we were able to find on the web. We call it RelBench. We have now two versions of RelBench. It's about 40 different predictive tasks over, I think, about 10-15 different databases. And then what's also interesting is that SAP, the big German IT company, they released a benchmark, a multi-tabular benchmark of enterprise data called SALT. So those are, I would say, the two big tabular or multi-tabular, so relational, benchmarks: SALT from SAP and the RelBench line of work that we've been doing and promoting here through Stanford.

这也让我想知道，是否有一种方法可以通过反规范化（比如宽单表）来重用现有基准？你研究过这个吗？

It also makes me wonder if there's a way to reuse existing benchmarks by like denormalizing, you know, wide single tables or something like that. Is that something you've looked into?

这是个好问题。你可以尝试反规范化，但仔细想想，你只能反规范化一对一关系。一旦你有一对多关系，你就必须聚合。这就是关键。一旦聚合，你就丢失了信息。

That's a great point. Like you can try to denormalize, but if you think about it, you can only denormalize one-to-one relations. As soon as you have many-to-one, you have to aggregate. And that's the key. Once you aggregate, you lose information.

你丢失了信息。

You've lost information.

完全正确。完全正确。我知道我可以多讲一点，对吧？想象你在做一个流失模型。一个客户流失模型可能是：我有一个客户，这里有客户的历史交易。我需要聚合它们。所以首先我说，我会统计上个月你买了多少次，然后我可能会取这些购买价格的中位数。然后你知道，另一个数据科学家说，不不，让我们取你买的所有东西的最便宜价格。然后有人说，不不，你应该取最贵的。然后有人说，不，是平均值。有人说，哦，但分布是偏斜的，你应该取中位数。然后另一个人醒来，说，嘿，是关于早上购物。那才是预测流失的关键。让我们再加一个特征，对吧？然后有人说，哦，但我们得考虑假期。把数据给我。你知道，这就是我的意思，对吧？然后你就像，哦，假期。人们在假期睡得更久。让我们创建一个新特征来考虑假期。哦，但还有夏令时变化。让我们考虑那个。你看这变得多荒谬？只需关注交易，让注意力机制找出什么能预测。

Exactly. Exactly. And I know I can dwell on this point a bit, right? Imagine you are doing a churn model, right? So a customer churn model could be: I have a customer and here are historic transactions of the customer. I need to aggregate them. So first I say, I'll count how many purchases you made last month, and then I'll maybe take the median price of those purchases. And then you know some other data scientist says, no no, let's take the cheapest price of everything you bought, right? And then somebody says, no no, you should take the most expensive one. And then somebody says, no, it's the average. Person says, oh but distributions are skewed, you should take the median. Then another person wakes up and says, hey, it's about shopping in the morning. That's what's predictive of churn. Let's add another feature, right? And then somebody says, oh, but we have to account for holidays. Give me the data. You know, like that's what I mean, right? And then you're like, oh, holidays. People sleep longer on holidays. Let's now create a new feature that accounts for holidays. Oh, but then there is summer daylight change. Let's account for that. You see how kind of ridiculous this gets? Just attend over the transactions and let the attention figure out what predicts.

在我介绍你时，我提到你是 Kumo 的联合创始人。除了研究，谈谈研究和你目前在 Kumo 所做工作之间的关系。

When I introduced you, I mentioned that you were co-founder at Kumo. In addition to the research, talk about the relationship between the research and what you're doing at Kumo.

我们在 Kumo 构建的是一个商业级企业平台，允许我们进行大规模的关系型深度学习模型。我们使用这个平台有两个目的：一是允许合作伙伴、客户在多表格关系数据上训练、微调单任务模型。我可以谈谈那部分。但我们最近的突破，我们刚刚发布了第二个版本，是我们所谓的关系基础模型。那是一个预训练的基础模型，能够推理结构化关系数据。这个模型能做的事情很疯狂。它能做什么：它可以在任何数据库和任何预测任务上做出准确预测，无需任何模型训练。

What we built at Kumo is a commercial enterprise-grade platform that allows us to do large-scale relational deep learning models. And we are using this platform to two effects: one is to allow partners, customers to train, tune single-task models over the multi-tabular relational data. And I can talk about that part. But the recent breakthrough that we had, and we just released the second version, is our what we call a relational foundation model. And that's a pre-trained foundation model that can reason over structured relational data. And it's crazy what this model can do. So what this model can do: it can make accurate predictions on any database and any predictive task without any model training.

是的。我觉得这个说法几乎难以置信。就像它们只是带有一些未知关系的数字，而你要说，你只训练一个模型在随机业务数字之间的关系上，它就能在某个未知用例中工作。怎么让我理解这一点？

Yeah. And I find that proposition to be almost outlandish. Like they're just numbers with some unknown relationship, and you're going to say that you're going to train a model on just the relationship between random business numbers and it's going to work in some unknown use case. How make that make sense to me?

谢谢。谢谢。我觉得这很好。我想当我说这个时，听众应该会想，这家伙在说什么？所以谢谢。对吧？所以我同意，对吧？因为很容易说，哦，这是一个基础模型。耶，对吧？太好了。但然后，好吧，它到底能做什么？所以这里也许可以这样想。关键在于上下文学习，对吧？就像语言模型做上下文学习一样，我给它一个提示，给它信息，给它一个任务，然后它给我答案。所以我们这里做的是，系统有几个组件。有数据库，然后需要一种方式让我指示预训练的基础模型我想要什么样的预测。对吧？我想说，预测这个客户未来一个月的购买价格总和。那可能就是我预测客户会花多少钱。或者我说，预测交易 ID 为多少的交易是否欺诈。对吧？所以我有一种指定预测任务的方式。然后系统做什么，系统现在进入数据库。

Thank you. Thank you. I think it's great. I think as I say this, people who listen should be like, what is this guy talking? So thank you. Right? So I agree, right? Because it's easy to say, oh, it's a foundation model. Yahoo, right? Great. But then, okay, what does it really do? So here's maybe how to think about this. So the key here is to do in-context learning, right? The same way as a language model does in-context learning, where I give it a prompt, I give it the information, I give it a task, and then it gives me the answer. So what we do here is the system has several components. So there is the database, and then there needs to be a way for me to instruct the pre-trained foundation model what kind of prediction I want. Right? I want to say, predict me the sum of purchase prices over the next one month for this particular customer. And that maybe is like how much I'm predicting how much the customer is going to spend. Or I'm saying, predict me transaction is fraud equals true for transaction ID this much. Okay, so this would be like predict me whether the transaction is fraudulent for this particular transaction ID. Right? So I have a way to specify my predictive task. And now what the system does, the system now goes into the database.

它提取一组带标签的上下文示例，然后通过预训练的神经网络进行预测。

It extracts a set of labeled in-context examples that then get passed through a pre-trained neural network to make a prediction.

好的。所以当我说一组带标签的上下文示例时，这意味着你可以拿欺诈检测这个任务为例。我有已经标记好的历史欺诈数据，还有一些新进来的交易没有标签，我需要预测它们是否欺诈。

Okay. So now when I say a set of labeled in-context examples, this means that you can take the task for the example of fraud. I've got historical fraud that's already been labeled and I've got some new transactions coming in that don't have that label attached that I'm trying to predict for example.

好的。那我们就用欺诈检测吧，这个可能更容易理解。是的。所以它的工作方式是：如果我说预测欺诈概率，系统会进入你的数据库，提取出之前我们知道是否欺诈的交易记录。对于每一笔交易，我们再提取它周围的实体子图。那么关系基础模型的输入就是一组历史交易的子图及其欺诈标签，加上那笔没有标签的新交易。我们不知道它是否欺诈。然后通过关系基础模型架构的前向传播，给未标记的图打上标签，对吧？就是未标记的交易及其周围的图。

Okay. So let's do fraud. Fraud might be easier. Yes. So the way this would work, right? If I say predicting the probability of fraud, the system would go into your database and extract previous transactions for which we know whether they are fraudulent or not. For each of those transactions, we would then extract the subgraph of entities around it. Okay. So now what the relational foundation model gets on the input is a set of historical subgraphs of previous transactions and their fraud labels plus the new transaction that is unlabeled. We don't know its fraud. And then this is passed through the relational foundation model architecture forward to label the unlabeled graph, right? Like the unlabeled transaction and the graph around it.

我现在不确定欺诈是不是个好例子，因为我能看出它怎么运作。你收集了这些已知点周围的图，然后让模型推断可能导向单个标签的关系。所以也许——这可能就是你的意思——我觉得回归类问题比分类问题更有挑战性。

I'm not sure now that fraud is a good example because it's kind of I can see how that could work. Like you've collected the graph around these known points and you're asking a model to infer relationships that might lead to this one individual label. And so maybe I think maybe and this may be where you're going like I think a regression type of a problem would strike me as more challenging than a classification problem.

是的。我认为关键点在于：首先，你有一种语言来指定任务。我们可以生成几乎像一个小型带标签的训练数据集，也就是上下文示例。然后你有一个预训练模型，能够接收这些上下文示例——这些具有特定列和表的领域子图——并以领域无关的方式编码它们。然后神经网络能够在其内部通过一次前向传播构建一个预测模型，从而给出准确的预测。

Yeah. I think the key here is that first, you have a language where you specify the task. We can generate almost like a mini labeled training dataset of in-context examples. Then you have a pre-trained model that is able to take these in-context examples, these domain subgraphs that have certain columns and tables, and encode them in a domain-agnostic way. Then the neural network is able to essentially build a predictive model in its brain in a forward pass to give you accurate prediction.

对。所以这并不一定是对数字的某种通用理解，而是能够识别它从未见过的数字之间的正确关系，查询正确的示例，创建正确的空间，然后将其表述为正确的推理请求之类的东西。

Right. So it's not necessarily about some universal understanding of numbers or what have you. It's about being able to identify the right relationships between numbers that it hasn't seen before, query the right examples, create the right universe, and then formulate that as the right inference request or something.

完全正确。所以这里有两个方面。一是如何将数据以领域无关的方式编码。因为我们可以处理任何数据库、任何列集合。模型需要能够以通用方式编码。编码完成后，第二步是执行上下文学习。这意味着模型在其内部需要能够构建模型。没有训练、没有反向传播、没有梯度。仅仅是一次前向传播，神经网络在某种意义上构建模型，从而给出准确的预测。所以不需要训练，不需要超参数优化，不需要特征工程。你只需要一个原始数据库和一种指定任务的方式。

Exactly. So there are two aspects to this. One is how can you take data and encode it in a domain-agnostic way. Because we can take any database, any set of columns. The model needs to be able to encode that in a universal way. Now that it's been encoded, the second step is to perform in-context learning. So it means that the model in its brain needs to be able to build the model. There's no training, no backpropagation, no gradients. It's just a single forward pass in which the neural network kind of builds the model in a sense that gives us the accurate prediction. So no training is necessary, no hyperparameter optimization, no feature engineering. All you need is a raw data database and a way to specify the task.

模型是否需要某种记忆结构、黑板之类的东西来做草稿工作以得出表示，还是说这完全是像思维痕迹之类的东西？

Does the model require some type of memory structure, blackboard or something in order to do scratch work to come up with a representation, or is this all like thought traces or something like that?

不不不。这不是一个智能体。这是 Transformer 类神经网络的一次前向传播。所以这纯粹在神经网络内部。没有智能体，没有记忆，没有草稿板，没有“让我做这个，让我做那个”。答案就是神经网络的一次前向传播。没有循环，没有类似的东西。所以你得到答案，大概 2 秒、半秒，不管多长时间。这真的只是一个预训练冻结神经网络的一次前向传播。这里没有语言模型。这是与语言模型并行或互补的技术。你不能把数据库文本化，然后去问 ChatGPT：“嘿，你觉得这笔交易欺诈的可能性有多大？”你会得到糟糕的结果。所以这是一个冻结的预训练架构，让你能做到这一点。我感觉我经历了一个完整的循环：从“这是个离谱的说法”到“哦，是的，我能看出它怎么运作”再到“我不知道，它居然能工作，这还是挺疯狂的。”

No, no, no. This is not an agent. This is a single forward pass of a transformer-like neural network. Right? So this is purely inside the neural network. There is no agent. There is no memory. There is no scratch pad. There is no "let me do this, let me do that." The answer is truly a single forward pass of a neural network. There is no loop. Nothing like that. So you get the answer in, I don't know, 2 seconds, half a second, whatever the time is. It's really a single forward pass of a pre-trained frozen neural network. There is no language model here. This is technology that's parallel or complementary to language models. You cannot textify a database and then go to ChatGPT and say, "Hey, what do you think? How likely is this transaction to be fraudulent?" You get horrible results. So this is a frozen pre-trained architecture that allows you to do that. I feel like I've gone the full cycle from "that's an outlandish claim" to "oh yeah, I can see how that would work" to "I don't know, it's still kind of crazy that it works."

是的。不，这很有趣，对吧？当我们把模型用在那些被锁起来、隐藏的、模型从未训练过的数据集上，以及我们甚至没想过的任务上时，我们看到它比最好的监督模型表现更好。如果你说，我雇一个数据科学家，他们花几周时间构建模型、调优模型，用最新的神经网络等等，这个模型仍然差几个百分点。然后如果你针对特定任务用更多数据微调基础模型，就能达到超人的准确率，这是当前手动、半手动或智能体式解决方案无法达到的。

Yeah. No, it's interesting, right? And when we test this on datasets that are locked away and hidden and the model has never been trained on, and on tasks that we haven't even thought about, we see a gain over best supervised models out there. If you would go and say, I'll hire a data scientist. They'll spend several weeks building the model, tuning the model, the latest neural networks, whatever. This is still a couple of percentage points worse. And then if you fine-tune the foundation model on more data for the specific task, then you get to this superhuman accuracy performance that present manual, semi-manual, or agentic solutions are just not able to attain.

那就是 RFM2，Kumo 的 RFM2，关系基础模型。你最近还在 ICLR 发表了关系图 Transformer。这两者是一个基于另一个，还是独立的研究方向？

That is the RFM2, Kumo's RFM2, the relational foundation model. You also recently published at ICLR your relational graph transformer. Is the one based on the other or are they independent lines of research?

我想说的是，在斯坦福，我们公开推进新的架构改进和理解。作为学者，我们尽可能开源一切，讨论一切。当然，在公司内部，我们推出的一些创新也会内部扩散。我认为内部使用的架构有点不同。它由两个不同的部分组成。第一部分基本上是对这组表的编码或注意力机制。第二部分是这种上下文学习类型的机器。这里有两篇相关论文。一篇是关系图 Transformer，我们主要用它来做监督微调类型的任务。但另一篇我们也在 ICLR 发表的论文叫关系 Transformer，它实际上支持上下文学习。

What I would say is that at Stanford we are pushing forward in the open new architectural improvements and understandings. As much as we can as academics, we release everything open source. We talk about everything. And then of course what happens inside the company is that some of these innovations that we put out also diffuse inside. I would say that internally the architecture we are using is a bit different. It's composed of two different parts. The first part is basically the encoding or the attention mechanism over this set of tables. And then the second part is this in-context learning type machine. There are two papers that are relevant here. One is the relational graph transformer that we mainly use for supervised fine-tuning type tasks. But then another paper we also published at ICLR called the relational transformer, and that one actually allows for in-context learning.

所以，这个机制在数据库的单个单元格层面做注意力，本质上你有三种注意力。你有对给定列的注意力。所以，如果你对一个单元格感兴趣，通过关注同一列的其他单元格，它就能给你一种分布的感觉，对吧？然后我们关注一行中的单元格。这能让你了解那一行的信息。然后我们还有一种基于图的注意力机制，让你可以说，哦，这是一个用户，这些是他所有的历史交易，对吧？然后每笔交易是一行，每行有列。这意味着我们可以关注数百万、数千万个单元格，而美妙之处在于，我们的注意力机制由于图结构而具有更多的结构性。所以注意力机制永远不会是二次的，这意味着我们可以更高效地计算。要做好推理，你真的需要巨大的上下文窗口，对吧？即使是今天最大的 LLM，我不知道，能达到一百万个 token，对我们来说一百万个 token 算小的。

So that one does attention all the way at the individual cell levels of a database and essentially you have three types of attention. You have attention over a given column. So if you are interested in a cell by attending over other cells in that same column it kind of gives you a sense of a distribution right then we attend over the cells in a row. And that kind of then gives you a sense of what's the information in that row and then we also have a graph based attention mechanism that allows you to say oh this is a user and these are all their older transactions right and then each transaction is a row and each row has columns this way. So this means that you know we can be attending over millions tens of millions of cells and the beautiful thing is that our attention mechanism because of the graph has much more structure. So the attention mechanism is never quadratic and this means we can compute much more effectively and to do good reasoning you really need humongous context sizes right even the largest LLMs today I don't know go to a million tokens for us million tokens is small.

所以我想问，是否有数据要求、形状或用例适合这个，或者反过来不适合？听起来部分原因是规模。你需要大量数据才能让它工作，对吗？

So I was going to ask are there data requirements or shapes or use cases that this works well for or conversely doesn't work well for. It sounds like part of that is size. Like you need a lot of data in order for this to work. Is that fair?

我其实可能稍微反驳一下，因为模型是预训练的。它在数据很少的情况下也能做很棒的事情，因为你知道，从头训练模型确实需要大量数据，但一旦模型预训练完成，它基本上知道自然界中会出现什么样的函数。所以这意味着你可以给它少量样本，它就会给出非常准确的预测，比你需要训练的监督模型更准确。

I would actually maybe push back on that a bit actually because the model is pre-trained. It can do amazing things where you have very little data because it's you know like training models from scratch yeah requires a lot of data but once the model is pre-trained it kind of knows what functions kind of appear in nature. So it means that you can give it few examples and it's going to give you very accurate predictions more accurate predictions than something you know supervised model that you have to train.

我想我理解这个想法是基于你说你处理的上下文通常很大。这是不是说当你有大量数据可用时你可以用它，但你不一定需要它？

I think I picked that idea up based on you saying that the context that you work with is typically large. Is that saying that when you have a lot of data available you can use it but you don't necessarily need it.

完全正确。然后，如果你有大量数据，你可以增加上下文窗口，通过增加上下文窗口你得到更准确的预测；或者如果你说，哦，我在做欺诈检测，你可以直接微调你的模型用于欺诈，从某种意义上说，你甚至不需要做上下文学习，因为你知道你的数据，你知道任务，你只需为那个单一任务调整模型，然后模型可以更小，运行更高效，也更准确，因为它不必每次都重新学习任务，因为你给了它上下文示例。

Exactly. And then once if you have a lot of data you can either increase the context size and by increasing the context size you get more accurate predictions or if you are saying oh I'm doing fraud you can just fine-tune your model for fraud in a sense that you don't even have to do in-context learning because you know your data you know the task you just tune the model for that single task and then the model can be smaller much more efficient to run and also more accurate because it doesn't have to almost re-learn the task every single time because you give it the in-context examples.

我们看到效果最好的是预训练和上下文示例的某种混合，因为你选择上下文示例的方式实际上可以取决于目标实体是什么。所以从某种意义上说，你会说，哦，如果我在预测欺诈，比如针对我，那么你可以说，哦，让我把其他斯坦福教授放在我的上下文示例中，让我把其他湾区的人放进来，因为你知道那就像是同行群体，或者是最有用的示例，你可以从中学习，对我是否是欺诈者做出准确预测。

What we see works best is some mixture of pre-training and in-context examples because the way you choose in-context examples can actually depend on what the target entity is right. So in a sense you'd say oh if I'm predicting fraud for I don't know for me then you could say oh let me put some other Stanford professors in my in-context examples let me put some other Bay Area folks in here because you know that's kind of the peer group or the most useful examples from which you can learn to make accurate predictions about me being a fraudster.

我在想模型和系统之间的界限。系统是构建上下文示例的，而模型只是那个前向传播。

I'm thinking about the line between the model and the system. The system is what is constructing the in-context examples and the model is just that forward pass.

你需要两者，对吧？我认为这很重要，因为必须有人生成这些上下文示例，你不会手动生成它们，对吧？

And you need both right I think it's important because somebody has to generate these in-context examples you won't generate them manually right.

那部分也是学习的，还是某种公式化的图遍历或其他什么？

And is that part also learned or is it a kind of formulaic graph traversal or something else?

介于两者之间。你可以把它作为一种图遍历和一点时间旅行来生成前瞻性标签。但当然，你如何做以及生成什么上下文示例至关重要。所以，要获得顶级性能，有很多因素需要考虑。

Somewhere in between. You can do it as a form of just a graph traversal and a bit of time travel to generate the forward-looking labels. But of course how you do that and what in-context examples you generate makes all the difference. So there's a lot that goes into that to get top performance.

说到性能，你提到了一些收集基准的挑战。但你觉得在这些基准上表现如何，更重要的是在现实世界中呢？

And so speaking of performance, you talked a little bit about some of the challenges with collecting benchmarks. But how do you find performance relative to those benchmarks and also more importantly in the real world?

是的。所以我可以直接说，我们有一份关于 Kumar RFM 2 的白皮书，人们可以阅读，里面有很多不同的基准。我们看到的是，基础模型本身就在这个基准上改进了所有已发表的监督模型的最优结果。对吧？所以基线非常高。就像构建你能做的最好的模型，看看能有多高。基础模型将准确率相对提高了大约 5%，然后如果你进一步调整模型，意思是如果你微调它，做一些梯度更新，那么性能比最优结果提高 12%，这些是相当可观的收益，特别是如果你考虑将其投入生产，在推荐系统或欺诈检测中，准确率每提高一个百分点，就可能意味着数百万、数千万的商业影响。

Yeah. So I can say right like we have a white paper on Kumar RFM 2 that people can read with a bunch of different benchmarks. What we see is that the foundation model by itself improves state-of-the-art over all supervised models ever published on this benchmark. Right? So the baseline is very high. It's like just build the best model you can and see how high you can get. The foundation model improves that I think for about 5% relative the accuracy and then if you further tune the model meaning if you would fine-tune it do some gradient base updates then the performance goes to 12% over the state-of-the-art and those are quite sizable gains especially if you think about putting this in production in recommender systems or fraud detection where every single digit performance increase in accuracy can mean millions tens of millions in business impact.

也许我想说的第二点是，我们看到这些方法在噪声和不完整数据、冷启动问题上也表现出色，因为关系结构，模型能够更好地聚焦，对数据缺失、数据损坏等更具鲁棒性。所以我们也做了很多分析，了解它在真实世界数据、稀疏数据、少量数据、噪声、不完整性、无关列等方面的表现。

Maybe the second thing I would say is where we see these methods also shine is with noisy and incomplete data cold start problems because of the relationships because of the relational structure the model is able to much better kind of hone in and be much more robust to the data missingness data corruption and things like that. So we've also done quite a lot of analysis around understanding how this performs on real world data sparse data small amounts of data noise incompleteness irrelevant columns and things like that.

当你提到冷启动时，那意味着，嘿，我想开始识别欺诈交易，但我没有标签，我只有一堆数据。你能告诉我应该从哪里开始看吗？它适用于那种问题吗？

And when you mention cold start like that suggests hey I want to start identifying fraudulent transactions but I have no labels I just have a bunch of data. Can you tell me where I should start looking? Like does it work for that kind of problem?

呃，是的，也许我应该量化一下冷启动的含义。通常冷启动意味着当新用户出现、新产品出现时，对吧？所以你仍然需要一些历史标签。我不是说不需要历史标签，但通常你知道，一旦你有大量关于某个用户或某个产品的数据，预测就很容易。但当产品是新的或用户是新的时，你的数据很少。这就是技术上所谓的冷启动问题。所以我仍然需要历史标签，但要做出可靠的预测，我不需要太多数据。

Uh yeah, maybe I should quantify what cold start means. Usually cold start would mean when a new user shows a new product shows up, right? So you still need to have some historical labels. I'm not you still need some historical labels, but usually you know prediction is easy once you have a lot of data about a given user or a lot of data about a given product. But when the product is fresh or when the user is fresh you are data poor. That's what technically it's called cold start problems. So I still need historical labels but to make reliable predictions I don't need much data.

我想我在某个地方看到这个系统部署在像 DoorDash 这样的地方。

I think I saw somewhere that the system is deployed at like places like Door Dash and others.

能谈谈部署流程吗？

Can you talk a little bit about the process for deploying it?

好问题。我们的系统、平台，可以用多种方式部署。可以当作 SaaS 运行，本质上是一个算力平台。也可以部署在客户的私有云或公有云里，也就是我们说的虚拟私有云，这样所有数据都留在客户那里。根据组织的偏好，有多种部署方式。在用例方面，DoorDash 用于餐厅推荐和通知系统——哪个用户在什么时间收到什么通知——我们看到了数亿美元的收入影响。另一个重要客户是 Reddit。Reddit 上的广告模型是用 Kumo 构建的，点击率提升了近两位数。通常，整个团队每年可能只能将准确率提高 1% 左右，因为点击率已经优化得很好了。

Yeah, great question. So the system, the platform, we can deploy it in many different ways. We can run it as a SaaS, basically as a compute platform. We can deploy it in people's private or public clouds, inside what we call a virtual private cloud, so all the data stays with the customer. There are a bunch of different deployments depending on what organizations prefer. In terms of use cases, at DoorDash, it's restaurant recommendations and the notification system—which user gets what notification at what time of day—and we've seen revenue impact of hundreds of millions of dollars. Another great client is Reddit. The advertising models on Reddit are built with Kumo, and it was nearly a double-digit increase in click-through rates. Usually, an entire team might increase accuracy by maybe 1% year over year, because click-through rate is already optimized.

这正印证了你之前的观点：领域知识和手工特征——你会觉得他们已经做了很久，把柠檬榨干了，但机器来了。

And this is your original point about domain expertise and manual features—you would imagine they've been working on this for a long time and squeezed a lot of the juice out of that lemon, but here comes the machine.

没错。这很有意思。我们与 Reddit 团队合作得很好，他们非常专业。他们自己构建了高度优化的特征工程流水线。我们的方法是：把数据表示为图，为用户、子版块、广告等创建嵌入。这些嵌入被附加到他们自己的特征上。即便如此，点击率仍有大幅提升，因为神经网络学到的信号与人工特征工程是互补的。生产中的模型结合了 Kumo 的图嵌入和手工特征工程。所以这对推荐点击率预测是加成作用。

Exactly. It's interesting. We have a great collaboration with the Reddit team, and they are amazingly sophisticated. They built their own super-optimized feature-engineered pipeline. Our approach was: let's take your data, represent it as a graph, and create embeddings for users, subreddits, ads, etc. These embeddings get appended to their own features. Even with that, there was a huge increase in click-through rate because the signal the neural network learned was complementary to what human feature engineering already had. The model in production combines neural network embeddings from Kumo with manual feature engineering. So it's additive for recommendation click-through rate prediction.

你有没有研究过他们的手工特征是否真的有效？是心理安慰，还是确实有可衡量的提升？

Have you looked at whether their manual features really make a difference? Is that a feel-good thing, or do they provide measured lift?

好问题。我们还没试过关掉它们。这是个很有意思的问题。有时候你仍然需要这些特征，不是为了模型准确率，而是因为有很多业务规则。广告系统不只是纯粹的优化；广告要展示给用户，需要触发很多业务规则。有时你确实需要这些信号来触发业务规则。

Good question. I don't think we've tried turning those off yet. It's a very interesting question. Sometimes you still want those features, not for model accuracy, but because you have many business rules. Advertising systems are not just pure optimization; there are many business rules that need to trigger for an ad to be shown. You sometimes need those signals to trigger business rules anyway.

我好奇这些手工特征以及更广泛的 RFM 的可解释性。这也是人们喜欢 XGBoost 的原因——那些树相当可解释，而基于 Transformer 的网络一直是个挑战。

I wonder about explainability with those hand-engineered features and more broadly with RFM. That's another reason people like XGBoost—those trees are fairly interpretable, and that's been a challenge with transformer-based networks.

说得好。我认为我们的可解释性做得很好，甚至比树模型更好。在树模型中，你只能得到特征排名列表，所以只能通过你设计的特征来解释预测。而我们能做到的是，因为模型是完全可微的，我们可以反向运行模型，看到模型关注哪些表、列和单元格。我们得到这种基于结构的解释，然后用大语言模型来说明：注意力在哪里，列是什么，它们的语义是什么。我们生成基于文本的解释——对数据显著性图的可读性极强的文本解释。LLM 用人类世界背景知识丰富它，所以变得非常可操作。

That's a great point. I would say we do explainability really well, even more than tree-based models. In a tree-based model, all you get is a ranked list of features, so you can only explain predictions by the features you engineered. What we can do is, because the model is fully differentiable, we run the model backwards and see what tables, columns, and cells the model is attending over. We get this structure-based explanation, then we use a large language model to say: here's where the attention is, what the columns are, and what their semantics are. We generate a text-based explanation—a super readable text explanation of a saliency map of the data. The LLM enriches it with human world background knowledge, so it becomes very actionable.

你刚才在谈用例——推荐。你最后提到了欺诈。

You were talking about use cases—recommendations. You mentioned one last one around fraud.

我们在欺诈方面看到了很好的结果。我们与 Coinbase 的优秀团队合作。这些模型在 Coinbase 的生产环境中运行，覆盖整个比特币区块链网络。这些方法确实能规模化。但对于 Databricks 或 Snowflake 这样的客户，他们用我们运行销售模型——预测客户下一步会买什么，哪个客户会转化为付费客户——从而优化销售团队。销售团队数据较小，因为销售团队只有几百或一千人，所以我们在小数据上也能做得很好。

We've seen great results with fraud. We've been partnering with an amazing team at Coinbase. We have these models running in production at Coinbase on the entire Bitcoin blockchain network. These methods really scale. But with some clients like Databricks or Snowflake, they use us to run their sales models—predicting what a customer will buy next, which customer will convert into a paying customer—allowing them to optimize their sales team. Sales team data is smaller because sales teams are hundreds or maybe a thousand people, so we do well on small data as well.

有些方面听起来像免费午餐——我怎么付钱？代价是什么？

There are aspects of it that sound free lunchy—how am I paying for my lunch? What's the catch?

最终，支付的是算力。我们真正做的是将机器学习工作负载从 CPU 转移到 GPU。传统 ML 中的大部分计算发生在 CPU 上，除了最后的神经网络训练。我们把那个工作负载转移到 GPU。

At the end, what is being paid for is compute. What we are really doing is taking machine learning workloads from the CPU to the GPU. Most of the compute in traditional ML happens on the CPU, except maybe the final neural network training. We move that workload to the GPU.

所以现在 GPU 算力更大，因为计算是在原始数据上进行的，而不是在 CPU 生成的摘要上。这就是成本所在。最终，这些模型当然不是万亿参数级别的；它们是十亿参数级别的模型。它们可以非常小，因此实际上非常高效且运行成本低廉。因为我们做预测是为了做决策，对吧？我们的指挥系统正在决定向每个用户展示什么。所以这些决策的速度是每秒数万、数十万、数百万次。因此性能成本真的非常重要。

So now the amount of GPU compute is larger because it's computed over the raw data, not over the summaries generated on the CPU. And that's what the cost is. In the end, of course, these models are not in trillions of parameters; they are billion-parameter type models. They can be quite small, so they're actually quite efficient and cheap to run. Because the reason we are making predictions is to make decisions, right? Our commander system is making decisions about what to show to every user. So the speed of those decisions is tens of thousands, hundreds of thousands, millions of times per second. So performance cost really, really matters.

我想我也在试图探讨局限性。如果有人带着一个实际上是多表关系型的问题来找你，你最终可能会告诉他们这可能不太合适，原因是什么？

I think I was also trying to get at limitations. If you had someone come to you with a problem that was in fact multi-tabular relational, what might be some reasons why you ultimately tell them that it's probably not a good fit?

我的说法是，我们知道如果使用我们的技术，至少会与现有技术持平或更好。没错。现在我们通常看到的瓶颈在于实际获取价值，比如将这些预测与下游业务决策流程连接起来，以便可靠地衡量价值。这可以说是最大的瓶颈，因为模型被构建、开发出来，运行得很好，但工程团队需要将它们接入，才能真正展示这些预测或基于这些预测做出决策。另一个用例是，我们或这项技术擅长的是预测性的、定义明确的预测类型问题，这些问题可以数学化地良好表述和优化。如果更多的是“我们想理解模式，我们想了解过去发生了什么”，那更像是传统的数据分析类事情或某种模式检测类事情，我们的平台和讨论的内容可能不是一个好的用例。所以这是一个例子。因此你需要知道你在预测什么。你需要能够表述它，并且你需要能够衡量准确性，然后我们才能优化。

The way I would say is we know if we use our technology, we'll be at least on par or better than what is already there. Right. Now where we see bottlenecks usually is in actually getting the value out, like connecting those predictions to some decision-making downstream business process so that the value can be reliably measured. That's been the biggest bottleneck, in a sense that models are built, developed, they work great, but then engineering teams need to hook them up to actually surface those predictions or make decisions based on those predictions. Another use case is sometimes where we shine, or where the technology shines, is in predictive, well-defined predictive type problems that can be mathematically well formulated and optimized. If it's more about 'hey we want to understand the patterns, we want to understand what is happening in the past,' that is much more traditional data analytics type things or some pattern detection type thing, that our platform and what we discussed is maybe not a good use case. So that's an example. So you need to know what you are predicting. You need to be able to formulate that, and you need to be able to measure accuracy, and then we can optimize.

而且你不会解决组织中传统的“你有了一个模型，如何使用它？”这样的数据科学问题。这个问题仍然存在。

And you're not going to solve the traditional data science problem of like in organizations of you've got a model, how do you use it? That's still going to exist.

完全正确。仍然存在一个问题：现在我们有了模型，如何将其推向生产？我不会说推向生产很容易。问题在于如何将其与下游应用或下游系统连接起来，以便有人真正根据这些预测采取行动。我们最近还看到很多进展是在智能体工作负载上，对吧？因为智能体需要做决策来采取行动，现在你可以基于这种基于 LLM 的常识来做决策。但对于更复杂的事情，做决策的最佳方式是估计或预测它们的下游效果。

Exactly. There is still the problem of how now that we have the model, how are we pushing that to production? I wouldn't say to production, that's easy. The question is how do we connect it with the downstream app or the downstream system so that actually somebody's acting on these predictions. And where we also see a lot of traction recently is in agentic workloads, right? Because agents need to make decisions to take actions, and now you can make decisions based on this LLM-based common sense. But for anything more, the best way to make decisions is to estimate or predict their downstream effect.

所以现在我设想这个模型位于一个工具接口后面，当智能体需要对数据做出预测时可以调用它。

So now I'm envisioning this model sitting behind a tool interface that an agent can call to when it needs to make a prediction about the data.

完全正确。完全正确。即使你想想，比如说，一个客户支持智能体：你打电话给我，我需要估计你的终身价值，你流失的可能性有多大？我会以不同的方式回应。我该给你什么最佳报价？我实际上需要问一个反事实问题。如果我给你这个报价，你会更开心吗？这些都是预测问题。我不能只是凭空想象或问 ChatGPT。它会做一些合理的事情，比如常识，但那远非最优。因此，这些预测，这种对结构化关系数据的推理，捕捉了组织中客户的行为模式，对于做出准确决策至关重要。随着我们部署这些智能体，我们不能为每个问题构建单独的模型并预先设想问题。基础模型的妙处在于你可以问任何问题。我想在这里说的一点是展示这个问题有多大。比如你想想像 SAP 这样的组织，对吧？SAP 有，我想，7 万到 10 万个客户。每个客户都有结构化数据，因为这是一个组织。每个客户都会稍微改变一下模式。所以每个人都有自己的数据、自己的模式，并且每个人都想做流失预测。但每个人对流失的定义都略有不同。那么你能雇佣 7 万名数据科学家，用客户的数据和客户对流失的定义来为每个客户构建流失模型吗？你不能。一个基础模型：智能体可以问“根据这个定义和这些数据预测我的流失概率”，半秒钟后你就得到了答案。

Exactly. Exactly right. And even if you think about, let's say, a customer support agent: you call me in, I need to estimate what's your lifetime value, how likely are you to churn? I will respond differently. What's the best offer for me to give you? I need to actually ask a counterfactual question. If I make you this offer, how will that make you happier? And these are all predictive problems. I cannot just hallucinate them or ask ChatGPT. It will do something reasonable, like common sense, but that's far away from optimal. So this predictions, this reasoning over this structured relational data that captures the patterns and behavior of, let's say, customers inside the organization, is crucial to make accurate decisions. And as we are deploying these agents, we cannot be building separate models for each of these and pre-anticipate the questions. The beauty of the foundation model is that you can ask any question. And one thing I want to say here is just to show how big the problem is. Like if you think about an organization like SAP, right? SAP has, I think, 70,000 to 100,000 customers. Each of their customers has structured data because it's an organization. Every one of them changes the schema a bit. So everyone has their own data, their own schema, and every one of those wants to do a churn prediction. But every one of them has a bit different definition of what churn is. So now can you hire 70,000 data scientists that are going to build per-client churn model with the client's data and the client specification of what churn means? You can't. A foundation model: an agent can just ask 'predict me probability of churn under this definition under this data' and you get the answer half a second later.

这引发了我关于模型后训练或微调的问题。中间微调有价值吗？我想象的是你与 SAP 合作，SAP 有数百个这样的模块。比如供应链模块、流失模块等等。针对用例进行微调（与单个客户数据分开）有意义吗？还是基础模型的广度已经捕捉了该用例抽象级别的所有信息，实际上只有当你查看特定客户的数据时才能改进它？

That raises a question for me around post-training the model or fine-tuning the model. Is there any value to intermediate fine-tuning? I think what I'm envisioning is like you partner with SAP, SAP has hundreds of these modules. There's like a supply chain module and a churn module and some other thing. Does it make sense to tune on the use case separate from the individual customer's data, or does the breadth of the foundation model already capture all of the information at that use case level of abstraction and really you're only improving it if you're looking at a specific customer's data?

这是个好问题。实际上，这是我们目前正在深入研究的事情。我认为有多个原因需要进行后训练。一个原因是，即使在上下文学习场景中，你也希望进行后训练，以便更好地学习底层数据的分布，这样数据就能被更好地编码，后续的预测也会更准确。所以这是你希望以任务无关的方式在底层数据上进行训练的原因之一，以更好地捕捉分布、更好地学习先验。另一个你想要微调的原因是成本，因为如果我针对特定任务进行微调，我就不需要做上下文学习了，对吧？因为现在我的上下文小得多。

That's a great question. Actually, that's something we are deeply looking into right now. I would say there are several reasons why to post-train. One reason you would want to post-train even in the in-context learning scenario is to better learn the distribution of the underlying data, so that prediction then the data gets better encoded and prediction later will be more accurate. So that's one reason you would want to train even in a task-agnostic way over the underlying data to better capture distributions, to better learn prior. Another reason why you would want to fine-tune is for cost reasons, because if I fine-tune for a specific task, I don't need to do in-context learning, right? Because now my context is much smaller.

我不需要引入标签数据，只需引入我想要预测的实体。现在注意力机制更小、更快、运行成本更低。如果我有大量数据，模型可以从中学到很多。所以我认为存在一个频谱、一个连续体，你可以做的事情以及其收益，取决于你在这个连续体上的位置。

I don't need to bring in the label data. I just bring in the entity I want to predict on. Now the attention is smaller. It's faster. It's cheaper to run. If I have large amounts of data, the model can learn a lot from. So I would say there is a spectrum, a continuum of what you can do, and the benefits are kind of different depending on where on this continuum you sit.

那么下一步是什么？

So what's next?

是的，下一步是什么？我们对智能体非常兴奋，基本上是将它们作为工具呈现出来。第二件事是，现在编码智能体已经存在，但我们发现编码智能体需要适当的抽象和适当的基础设施才能有效。例如，你可能会说：“为什么我不直接把建模任务交给 Claude Code，让 Claude Code 为我构建模型呢？”这有什么大不了的？我们在内部这样做时，发现这些模型会写数千行代码，但存在非常微妙的数据科学错误。例如，我们与 Expedia 合作时，在账户级欺诈检测中，智能体犯的一个错误是：当它为给定账户创建特征时，它将交易聚合到午夜，而不是当前时间。所以它说：“今天是 4 月 30 日，我们将使用截至 4 月 30 日午夜的数据”，而没有说：“嘿，现在是 4 月 30 日上午 10 点，我们只能使用截至此时的数据。”这是信息泄露，一个小错误。它犯的另一个错误是，我们在交易级别而不是账户级别进行操作。这些是微妙的错误，确实需要人类参与。但如果你给它一个更高级的、类似 Kumo 的 API，那么它就能用大约 50 行代码完成同样的工作，没有错误。

Yeah, what's next? We are very excited about agents, both basically surfacing these as tools. The second thing is right now, coding agents are out there, but what we see is that coding agents require a proper abstraction and a proper infrastructure to be effective. For example, you could say, 'Why don't I just give this modeling task to Claude Code and Claude Code will build the model for me?' So what's the big deal? When we do that internally, we see that these models write thousands of lines of code, but there are super subtle data science mistakes. For example, we did this together with Expedia, and when it was account-level fraud, a mistake the agent made was that when it created features for that given account, it aggregated the transactions till midnight, not till the current time. So it said, 'Today is April 30th, so we'll use the data up to midnight of April 30th,' not actually saying, 'Hey, it's 10:00 a.m. on April 30th, we can only use data up to here.' That's information leakage, a little mistake. Another mistake it made was that we did it at the transaction level instead of the account level. These are subtle mistakes that really need the human in the loop. But if you give it a higher-level Kumo-like API, then it's able to do the same work in about 50 lines of code, no mistakes.

这个案例中的任务是什么？我以为你描述的任务是编写类似 Kumo 试图做的事情。任务是“为我基于这些数据构建一个账户级欺诈检测模型”。

The task in this case is to do what? Like I thought the task that you were describing was to code up something like what Kumo was trying to do. The task is 'Build me an account-level fraud detection model over this data.'

所以你提出的建议是，与其让智能体从头开始编码，不如创建某种技能或东西，教它如何使用 Kumo 来获取相同的信息。

And so what you're proposing is, as opposed to trying to have the agent code it up from scratch, you create some kind of skill or something that teaches it how to use Kumo to get the same information.

是的。或者我的意思是，智能体可以自主地走两步，但不能走一百步。所以当我要求它完成一个任务时，我可以说：“这是 PyTorch，去给我构建模型”——用 PyTorch 构建模型需要一千行代码。我可以说：“这是 XGBoost，给我构建模型”——那需要大约 500 行代码来设计特征等。或者我可以说：“使用 Kumo API，去给我构建模型”——那只需要 50 行代码。现在如果你用步骤来类比，50 行代码可能相当于两步，500 行代码相当于 20 步，而你在 20 步中很容易迷失方向。

Yeah. Or what I'm saying is agents can autonomously maybe make two steps but not 100 steps. So now when I ask it for a task, I can say, 'Here's PyTorch, go build me the model' — that takes a thousand lines of code to build a model with PyTorch. I could say, 'Here's XGBoost, build me a model' — that takes about 500 lines of code to engineer features and so on. Or I can say, 'Using the Kumo API, go build me the model' — that only takes 50 lines of code. Now if you think of this analogy of steps, 50 lines of code is maybe like two steps, 500 lines of code is 20 steps, and you can get quite lost navigating 20 steps in this sense.

我认为这个观察更具普遍性。观察结果是，为了让智能体有效，它们需要对智能体友好的 API。

I think the observation is more general. The observation is that for agents to be effective, they need APIs that are agent or agentic-friendly.

好的，太棒了。非常感谢你参加节目，让我们了解你和 Kumo 的最新进展。非常酷的东西。

Okay, awesome. Well, thank you so much for jumping on and catching us up on what you and Kumo are up to. Super cool stuff.

是的。非常感谢你的对话和非常有见地的问题。

Yeah. Thank you so much for the conversation and very insightful questions.

太棒了。非常感谢。

Awesome. Thanks so much.