Virtual Cells to Lab Compass: AI That Predicts and Steers Organoid Experiments
打开互动全文版(中英对照 + 朗读 + 问答)→Fabian Theis 讲述虚拟细胞模型如何从图谱构建走向预测,并进一步引导实验,包括用于抵达目标细胞状态的逆向生成模型 Lab Compass。
Fabian Theis discusses how virtual cell models evolved from mapping to prediction and now to steering experiments, including Lab Compass, an inverse generative model for reaching desired cellular states.
欢迎来到 Latent Space AI for Science 播客。我是 RJ Honicky,Miro Omics 的首席技术官。和我一起的是联合主持人 Brandon Anderson,他在 Atomic AI 开发 RNA 疗法。我怎么强调 Fabian Theis 和他的实验室及公司对系统生物学发展以及虚拟细胞概念各种迭代的贡献都不为过。他实验室发表的论文和模型比我读得还快,而且实验室一直在推动我们对如何观察和建模细胞行为与相互作用的理解。同样重要的是,他的工作常常专注于创建研究人员和科学家日常实际使用的工具。大多数计算生物学家可能都用过 Scanpy 和 AnnData。这项工作促成了 SCverse 的创建,这是一个了不起的开放工具集合,吸引了许多许多贡献者。当然,Fabian 是那种荣誉多到列不完的学者之一,但 suffice it to say,他在 Helmholtz Munich 及相关机构领导着一批 AI 和计算生物学工作,也参与许多其他行业项目。Fabian,很高兴见到你。非常感谢你加入我们。
Welcome to the Latent Space AI for Science podcast. I'm RJ Honicky, CTO of Miro Omics. I'm joined by my co-host Brandon Anderson, who develops RNA therapeutics at Atomic AI. I can't emphasize enough how much Fabian Theis and his labs and companies have contributed to the development of systems biology and various iterations of the virtual cell concept. Papers and models come out of his lab faster than I can read them, and the lab is consistently moving the ball forward on our understanding of how to observe and model the behaviors and the interactions of cells. Just as importantly, his work often focuses on creating the tools that researchers and scientists actually use in their day-to-day work. Most computational biologists have probably used Scanpy and AnnData at some point. And this work has led to the creation of SCverse, which is an amazing collection of open tools that's attracted many many contributors. Of course, Fabian is one of those academics who has way more honors than you can possibly list, but suffice it to say that he leads a bunch of the AI and computational biology efforts at Helmholtz Munich and related institutions, and participates also in many other industry efforts. Fabian, it's really great to meet you. Thanks a lot for joining us.
感谢邀请。我非常期待这次讨论,伙计。
Thanks for having me. I'm really looking forward to the discussion, man.
是的,我关注你的工作很久了,所以很荣幸终于能请你上节目和你聊聊。
Yeah, I've been tracking your work for a long time, and so it's a great honor to finally get you on the show and talk to you.
让我们从当前的一些热点话题开始。现在结构生物学有大量关注,Anthropic 和 OpenAI 的生物制药计划目前重点放在设计分子上。另一方面,虚拟细胞和系统生物学方法最近确实没有获得那么多关注。也许 Zera 和 Noetik 是很好的反例,但 Biohub 刚刚宣布与 DOE、NIH、Meta 和 Google 合作,投入 18 亿美元收集并开源多尺度生物数据,用于开发新一代虚拟细胞。所以我的问题是,基于我们目前在学术界看到的活动,你认为制药行业即将对虚拟细胞重新产生兴趣吗?
Let's kick things off with some of the hype happening right now. There's a ton of interest right now in structural biology, Anthropic and OpenAI biopharma initiatives focus heavily on designing molecules right now. On the other hand, virtual cell and systems biology approaches really haven't gotten as much attention recently. Maybe Zera and Noetik are good counterexamples there, but Biohub just announced a $1.8 billion collaboration with the DOE, NIH, Meta, and Google to collect and open-source multiscale biological data that can be used to develop a new generation of virtual cells. So, my question, do you think that we're about to see renewed interest from the pharma industry in virtual cells based on the activity that we're seeing in academia right now?
在回答之前我应该说明,显然我的观点相当学术,因为每当我与那些真正每天在初创公司里的朋友和同事交谈时,他们对我所拥有的那种概览和对不同研究实验室所做工作的一点直觉,他们对初创公司也有同样的了解。他们知道这是热门大事,知道这些投资正在发生。我并没有那种规模的了解。但话虽如此,例如你提到了 Biohub 的倡议,顺便说一句,这也与人类细胞图谱紧密耦合,那是一个国际性的真正全球联盟,推动下一代数据集生成。我确实认为有很多原因让我们会看到这种规模扩大,所有这些模型在生成方面表现良好,而且现在在分子层面被导出并真正在细胞层面更上一层楼地工作。我认为有很多原因。第一个我认为可能是最重要的,就是你真正指出的数据生成。我认为我们拥有虚拟细胞的概念已有一段时间,但我的实验室在 2019 年就开始做扰动预测建模并使用机器学习,当时我们称之为 SEGAN 的论文,那是一个非线性模型,但本质上在潜在空间做了一些线性变换。它实际上相当不错,即使现在许多这类伪线性类型的东西仍然表现良好,但原因是我认为数据集和我们在此基础上提出的问题仍然偏简单。这就是为什么它还没有真正兴起。所以每当人们谈论虚拟细胞,也许你的问题,我认为你实际问的是:你能预测细胞在所有扰动下会如何行为吗?我认为这是一个相当深刻的问题。如果你真的想一想,这甚至可能是计算生物学系统生物学的一个圣杯。这个输入输出系统,一个细胞,有记忆,有所有这些规格,有所有不同层次的分子等等,不仅仅是 RNA。我认为能够绘制出这些,甚至理解意味着什么?我一直觉得这有点像一个元问题,但我认为当你能够预测系统在不同输入下的输出行为时,你至少很好地建模了它。所以我认为这是一个很好的问题,这就是为什么现在人们常常将虚拟细胞等同于扰动预测。我认为这有点过于简单化,但我认为这些扰动预测类型的模型——也许这就是你所说的虚拟细胞——在初创领域正在大量兴起。我认为那里有点爆发,因为我多考虑美国和欧洲,但在中国生态系统中现在也有许多这些额外的努力。我希望这些不仅仅是每个人都在给自己的东西加 AI 而点燃的火花,而是因为制药公司真的迫切需要在筛选中做到细胞类型上下文特异性,而目前他们还做不到。
I should disclose before I sort of go to the answer that obviously my opinion is quite academic in the sense that whenever I talk to my friends and colleagues that really sort of day-by-day sit in the startups, they have like what I sort of have as an overview and sort of a bit of intuition what the different research labs do, they have the same thing for the startups. And they know this is sort of the hot big thing that, you know, this and this investments are happening. I don't really have that on that scale. But having said that, for example, you may you mentioned a Biohub's initiative, which by the way is also heavily coupled to human cell atlas, which is sort of this international really worldwide consortium that sort of pushes this next level of data set generation. I do think that there's a whole bunch of reasons why we going to see that sort of scale up from all these models working well in terms of being generative, but also sort of lap in the loop on the molecule level now being exported and sort of really working one level up on the cell level. And I think there's a bunch of reasons for that. And the first one I think it maybe the most important one, which you really hit here, is the data generation. I think we had that concept of a virtual cell for a while, but my lab we started this perturbation prediction modeling and using machine learning in 2019 with a paper we called SEGAN at the time, which was like it was a non-linear model, but essentially sort of in the latent space did some linear transformation. And it was actually quite okay, and even nowadays many of these sort of pseudo-linear type of things are still doing a good job, but the reason for that being that I think the data sets and the questions that we asked on top were just sort of still on the simplistic side. That's why also sort of it didn't yet take up. So whenever people talk about virtual cells, maybe also your question, I think what you actually ask is how does Can you predict how a cell would behave under all kind of perturbations? And that's sort of I think a quite deep question. If you really think about it, it's maybe even a bit of a holy grail in computational biology systems biology. Sort of this input-output system that is a cell with memory and like with all these specifications and with all the different sort of levels of molecules and so on, not just RNA. I think being able to map this out, what does even understanding mean? And I've been this sort of a bit of a meta problem, but I think you understand or at least you model a system very well when you sort of predict its output behavior given different inputs. So I think it's a very good question, and this is why often people nowadays equate a virtual cell with perturbation prediction. I think it's a bit too simplistic, but I think these perturbation prediction type of models and that's sort of being a maybe what you mean by virtual cell is taking off a lot in the startup landscape. I think it's a bit exploding there because I think a lot about US Europe, but also in the Chinese ecosystem there's a lot of these additional efforts now happening. And I hope that these are not just sort of now a kindling of a flame because everyone sort of adds AI to their things, but because pharma is really desperately needing to be cell type context-specific in their screens, which at the moment they can't do yet.
也许你可以稍微阐述一下,这个——所以现在扰动,比如我们最近和 Zara 聊过,他们在做这些扰动筛选。
Maybe you could expound a little bit on what does this So perturbations right now that like we talked to Zara recently, they're doing these perturbation screens.
这解锁了什么?然后你认为下一代数据除了这些之外还会解锁什么?
What does that unlock? And then what do you think the next generation of data is going to unlock in addition to that?
对我来说,如果可以的话,当我太深入这些老故事时你就打断我。我长期以来一直是一名受过训练的机器学习科学家,做各种各样的事情,但我早期是从单细胞 qPCR 开始的。也许最初,说实话,并不是因为我对生物学感兴趣,而是因为我们有高样本数,纯粹是为了这个,对吧?因为更早的时候就已经清楚,规模化的 qPCR 数量,然后当然转录组,变得非常庞大。当我早期开始职业生涯时,我们只有几个 bug RNA,现在我们有数千个之类的,对吧?所以这是早期。但现在我们进入了一个新阶段,那是令人大开眼界的时刻之一,当 PacBio 有——我想他们称之为 giga lab 之类的。现在竞争公司也有类似的,但当他们在一次实验中测量了 1000 万个转录组时,对吧?我觉得那是下一个层次的东西,从仅仅获取更多 PBMC 作为对照工具,到实际上在规模上做些什么,这里有了细胞空间,但还有扰动空间,后者通常非常稀缺。就像对照 versus 疾病之类的。所以这里在扰动空间采样方面的点数通常比你在细胞方面的观察数量要稀缺得多。但现在这在爆炸式增长。现在你有,我不知道,100 种不同的细胞系,你做全 perturb-seq 像全 CRISPR,或者你有 1000 种药物之类的组合添加,突然这个空间变得非常丰富,对吧?然后你可以开始做所有这些花哨的扰动预测类型的事情。这目前正在发生。发生的规模令人疯狂。我们自己参与了一大堆事情,我们实际上超越了细胞系,做类器官扰动,也许如果你有时间我们可以多谈谈这个,因为我相当确信我们需要进入多细胞系统才能有真正有趣的读数,但我认为每个人都在扩大这些水平,在某个时候,我们甚至可能将人类遗传学作为扰动。
I mean for me, if I may, you stop me when I'm getting too much into these old stories. I've been a trained machine learning scientist for the longest time doing all kinds of things, but I started with single cell early on qPCR. Maybe initially, if I'm really honest, not so much because it was the biology that I was interested in, but because we had high sample numbers, purely for that, right? Because even earlier on it was clear that the number of qPCRs at scale, but then of course transcriptomes was just getting really big. And when I started my career earlier on, we had like a few bug RNA, now we sort of had thousands of them or whatever, right? So this is like early times. But now we're going to a regime, and that was one of these eye-opening moments when PacBio had like I think they called it the giga lab or something like that. And there's similar ones from competing companies now, but when they had like in a single experiment 10 million transcriptomes measured, right? I felt that was like next-level stuff, and this step from just getting more PBMCs as a control tool, like let's actually do something at scale to it sort of having here the cell space, but then having here the perturbation space, which typically was very scarce. It was like control versus disease or so. So this sort of number of points what you had here in terms of sampling on the perturbation space was typically much more scarce than the number of observations in terms of cells that you had. But now this is exploding. Now you sort of have, I don't know, 100 different cell lines where you do sort of full perturb-seq like full CRISPR, or you have like batteries of 1,000 drugs or so added, and suddenly this space gets really rich, right? And then you can start doing all these fancy perturbation prediction type of things. And this is currently happening. It's crazy what scale it's happening. We are ourselves involved in a whole bunch of things where we actually go beyond cell lines and do organoid perturbations, and maybe if you have time we can talk a bit more about that because I'm quite convinced that we need to go to multi-cellular systems to have actually really interesting readouts, but I think everyone is scaling these levels up, and at some point, we're going to go maybe even to human genetics as a perturbation.
你能解释一下什么是扰动吗?你指的是什么?只是为了不熟悉的人。
Can you just explain what is a perturbation? What do you mean by that? Just for the people who are not familiar.
是的,是的。你实际上——你介意再退一步吗?比如我们说的测量转录组学到底是什么意思?你谈到了基于 PCR 的,然后还有 perturb-seq。
Yeah, yeah. You actually Would you mind maybe backing up even one step more than that? Like what do we even mean by like measuring transcriptomics? And you were talking about like PCR-based and then also perturb-seq.
当我做公开演讲时,我通常说,你知道,人类基因组计划,我们有身体的基因组,但它在各处几乎相同,对吧?那么为什么我们有所有这些不同的细胞类型呢?具体发生了什么?第一个层面是,每个细胞从基因组中读取不同的东西,对吧?这些信使 RNA,然后编码各种下游蛋白质,嗯,这些在不同细胞中表达不同,正如人们所说。在人类中,可能有大约 20,000 种不同的类型,加上剪接可能更多,但这是粗略的水平。所以事实上你每个细胞有一个 20,000 维的向量,有很多零和不存在的东西,但然后有很多数字,你有多少这些信使 RNA 的计数。因为这些东西被称为转录本,对吧?这被称为转录组。是的,事实上你有一个 20,000 维的空间,你经常将其编码在一个更好的潜在空间中,有一个子流形,就像这个子空间,实际的细胞云生活在那里,对吧?如果细胞云然后像这些 20,000 维向量,每个点是一个细胞,对吧?如果它们聚类,那么我们称这些为细胞类型之类的。如果它们进入一个香肠之类的,对吧?我们称它们为轨迹,像细胞分化,像受精卵细胞然后制造身体中的所有细胞类型之类的。
When I give public talks I typically say, you know, human genome project, we sort of have the genome for the body, but it's pretty much the same everywhere, right? So why do we have all these different cell types, right? What's happening specifically? And one of the first levels is that per cell different stuff is being read off of the genome, right? And these messenger RNAs, which then encode the various downstream proteins, well, these are differently expressed, as one says, in different cells. And in a human maybe there's something like 20,000 different types of these, and with splicing many many more, but that's the rough level. So what you de facto have is per cell a 20,000 dimensional vector that has a whole bunch of zeros and stuff is not there, but then a whole bunch of numbers, how many of these counts of these messenger RNAs you have. Because these things are called transcripts, right? This is called then the transcriptome. Yeah, and de facto then you have this 20,000 dimensional space that you very often encode in a nicer latent space that has like a sub-manifold, like this subspace where this actual cell cloud lives, right? And if the cell cloud then like these 20,000 dimensional vectors, each point then being one cell, right? If they cluster up, then we call these things cell types or something like that. And if they go into a sausage or whatever, right? We call them into trajectories, like a cell differentiation, like the fertilized egg cell then making all the cell types in the body or something like that.
所以就像 DNA 有点像一本食谱,或者也许是一堆基因是一堆食材。然后每个细胞是它自己的一种 meal 或菜肴之类的,是不同食材的组合。现在转录组学有点像弄清楚——拿不同的 meal 然后分解它们,像“哦,这个 meal 用了什么食材?”所以你觉得这是一个愚蠢的类比吗?
So like DNA is sort of like a recipe book, or maybe it's a bunch of genes are a bunch of ingredients. And then as you each cell is its own sort of meal or dish or something, which is a combination of different ingredients. And now the transcriptomics is sort of figuring out uh taking different meals and then sort of decomposing them like, "Oh, what ingredients went into make this meal?" So as you is that a dumb analogy?
我有几个孩子是美食家。我会更经常使用这个比喻。我喜欢它。
I have a bunch of foodies as kids. I will use this metaphor a bit more often. I like it.
谢谢,Brandon。
Thanks, Brandon.
我喜欢它。然后扰动是当你以某种方式扰动基因组或转录组,使其中有不同的转录本。然后通常通过 CRISPR 敲除来完成,你从基因组中移除一个基因,导致它不表达那个基因组。你看到然后从那里你看到由于扰动而发生的表型或物理生物学的实际变化。这或多或少
I love it. And then perturbations is when you somehow perturb the genome or the transcriptome so that it has different transcripts in it. And then typically that's done with a CRISPR knockout where you remove a gene from the genome and that causes it to not express that genome. You see and then from there then you see what are the phenotypic or the actual changes in the physical biology that occur because of a perturbation. Is that more or less
是的。
Yeah.
也许我可以退一步,因为我想告诉你你完全正确,但这是硬核类型的扰动之一,对吧?像完全敲除一个基因。很快在进化中会发生一点,但大多数这些东西更微妙一些。但当然,为了理解系统,取出一些部分是个好主意。我认为在正常发育或正常我们处于稳态时发生的是,当然这些细胞一直相互交流,你知道,它们生活在这些复杂的多细胞状态中,对吧?所以细胞之间发生各种通信。所以它们总是有东西从外面来然后对它们做些什么。所以对我来说,扰动空间实际上只是细胞发生的事情的输入空间,也许只是因为它坐在另一个细胞旁边,有一些信号发生,对吧?或者一些梯度,一些激素进来,等等,对吧?当然所有这些你也可以调高或调低。
Maybe I can take a step back there because I think I'll tell you you're fully right but this is one of the hardcore type of perturbations, right? Like fully knocking out a gene. Soon enough in evolution happens a bit but mostly these things are a tad more subtle. But of course in order to understand the system, it's a good idea to take out between pieces. I think what happens on a normal development or normal us being in homeostasis is of course that these cells talk to each other all the time, you know, they're living these complex multicellular states, right? So there's all kind of communications happening between cells. So they always have stuff coming from the outside and then doing something to them. So for me a perturbation space in fact is just an input space to stuff that happens to a cell just maybe because it just sits to another one and there's like some signaling happening, right? Or some gradient, some hormones coming in, whatever, right? And all this stuff of course you can also tune up or tune down.
也许甚至像这样基础的事情:当你使用这些体外系统时,制药公司做药物筛选,当你把它们放在培养皿中,你可以做不同的所谓培养基条件,它们也有很大的影响,对吧?对我来说,这仍然有点生物驱动,甚至像炼金术,人们想出什么样的混合物,然后取皮肤细胞,把它们变成或多或少像胚胎状态的细胞,对吧?然后你可以制造新细胞来治愈身体的不同部位,或者他们所说的山中因子。这些是诺贝尔奖级别的成果,对吧?所以我认为有一个扰动空间,本质上是对细胞可以做的事情,这可能是一个非常丰富的东西。它可以深入到基因的改变,比如完全敲除,或者更微妙的东西,比如人类遗传学。我认为所有这些我们都需要放入一个大的空间,我们可以做的对细胞状态有影响的事情。而一个虚拟细胞模型应该接受所有这些,对吧?当然还有它们的组合。也许还有疾病发生、时间等等,然后影响那个。
Maybe even something as basic as when you go to these in vitro systems where pharma does drug screens, when you put them in a dish, you can do different what's called media conditions and they also have so much impact, right? And it's still a bit bio-driven for me, maybe even like alchemy, what type of cocktails people come up with to then take skin cells and turn them into more or less embryo-like state cells, right? Where you can then make new cells to heal different stuff in the body, or these what they call the Yamanaka factors, for example. These were Nobel laureate type of stuff, right? So I think having a space of perturbations, essentially of things that you can do to a cell, this can be a very rich thing. It can go as far as changes in the gene, which would be like a full knockout, or something more subtle, which is sort of human genetics. And I think all of that we need to put in a big space of stuff that we can do that has an influence on cell state. And a virtual cell model should take all of it, right? And also combinations thereof, of course. And maybe disease happening, time, whatever, that then impacts that.
这与类器官的想法有很好的联系。所以你能谈谈——好吧,那么类器官是什么?它如何与扰动空间这个概念联系起来?除了这个背景之外,你还对它做了什么?
That's a great connection to the organoids idea. So can you talk a little bit about—okay, so an organoid is what? And how does that connect to this concept of perturbation space? And what do you do to it in addition to it just this context?
老兄,当我第一次听说——了解到这个概念时,我应该说,你知道,我是一个过度训练的生物学家,所以我通常通过阅读一点或与同事和朋友交谈来了解事物。当我第一次看到人们能够取一个细胞——实际上是从身体中取出细胞,比如癌细胞之类的,对吧?——然后让它们生长成小的多细胞系统,细胞实际上开始相互连接,我超级超级印象深刻。所以,你知道,人们现在可以,例如,取一些细胞,放在培养皿中,然后生长出小迷你大脑,本质上是一组神经元,它们真正开始连接、相互交流。人们已经证明他们可以让培养皿中的那些迷你大脑进行计算。同样,人们可以取心脏细胞,将它们生长成小心脏类器官。我们与出色的合作伙伴有一些本地合作,它们实际上开始跳动。就像你在培养皿中有跳动的小心脏细胞团,对吧?然后当然,一旦你有了这些——这和我说的脑一样——还有更简单的系统,比如结肠,这非常有效。有一大堆癌症系统。但也可以说,如果你想研究难以获取材料的疾病。比如你想研究某种神经退行性疾病的遗传学。通常很难获得组织,对吧?所以你从皮肤取干细胞——你从皮肤取细胞,分化并变成这些迷你小大脑,然后你可以在那里潜在地研究一些东西。所以我认为这种复制器官小部分的概念,就是他们所说的类器官,在培养皿中非常迷人。显然有各种各样的限制。通常免疫细胞——免疫系统缺失。可能没有——也许喂养它们之类的。所以有各种各样的问题,对吧?但它远比只是一个细胞系——那是一种细胞类型——要丰富得多。
Dude, when I first heard about—learned about this concept, I should say, you know, I'm an overtrained biologist, so I typically learn about things when I read a bit or when I talk to colleagues and friends. And when I first saw that people are able to take a cell—actually cells out of a body of a cancer cell or whatever, right?—and let them then grow into small multicellular systems that actually cells start connecting to things, I was super super impressed. So, you know, people can now, for example, take some cells, put them in a dish, and then grow little mini brains, essentially a set of neurons that essentially really start connecting, talking to each other. And people have shown that they can get those sort of mini brains in a dish to compute. Similarly, people can take heart cells, grow them into little heart organoids. We have a bunch of local collaborations with fantastic partners that actually start beating. Like you have beating little heart cell clumps in a dish, right? And then of course, once you have these—and it's the same thing as I spoke about brain—there's even simpler systems like the colon where this works very well. There's a whole bunch of cancer systems. But also for, let's say, if you want to study diseases with hard to acquire material. Let's say you want to look into the genetics of some type of neurodegenerative diseases or so. Typically it's very hard to get access to tissue, right? So you take stem cells from the skin—you take cells from the skin, differentiate and turn them into these mini little brains, and then you can potentially study some things there. So I think this concept of replicating little bits of organs, it's what they call organoids, in a dish is very fascinating. There's all kinds of limitations, obviously. Often immune cells—the immune system is missing. There's like no—maybe feeding them or something like that. So there's all kinds of problems, right? But it's much much more than just a cell line which is like one cell type.
转到建模方面,你如何建模那个?
Moving to the modeling side, how do you model that?
所以我认为这处于研究前沿。所以我不认为人们已经就最佳建模方式达成一致,特别是也因为数据没有人们想要的那么多。Jay,对吧?如果我理解正确,你参与了一大堆空间组学以及 H&E 病理类型的工作。所以像病理学,有那么多样本可用,对吧?现在越来越多地我们做一点空间组学。所以我认为我们甚至如何表示空间连接的数据集。然后你还必须在类器官层面分解等等。人们不像对解离细胞那样清楚。所以再次,把它简单一点,我认为如果我们看解离细胞的空间,看这些群体,我们有这个美丽的比喻,我们人类称之为细胞类型,而自然并不关心。但这只是我们在东西上面放类别。但它帮助我们理解这种复杂性。我不认为我们对细胞的局部微环境有类似的比喻。也许有一点,但在 H&E 中也是如此,对吧?每当我交谈并看到病理学家做什么时,它通常非常定性,也许还没有达到那个水平。所以我认为我们需要为我们开发一点分类语言。也许计算上,也许我们不需要这么多,但计算上我们也需要思考一下如何处理这个。目前我们基本上将其视为细胞的分布。所以这是目前的解决方案,但我认为它可以更加微妙,比如图之类的。
So that's I think at the research frontier. So I don't think people have agreed on what the optimum way to model is, particularly also because there's not as much data as one wants. Jay, right? You're involved, if I understand correctly, in a whole bunch of spatial omics slash also H&E sort of pathology type of effort. So like pathology, there's so much samples available, right? And now increasingly we sort of do a bit more spatial omics. So I think how we even represent spatially connected data sets. And then you also have to break it down on an organoid level and so on. People are not as clear as it is with dissociated cells. So again, taking this a bit simpler, I think if we look at the space of dissociated cells and we look at these groups, we have this beautiful metaphor of what we humans call a cell type, which nature doesn't care about. But this is just us putting categories on top of stuff. But it helps us understand this complexity. I don't think we have a similar metaphor for local niches of cells. Maybe a little bit, but also in H&E, right? Whenever I talk and see what pathologists do, it's often very qualitative and it's maybe not at that level yet. So I think we need to develop a bit of a categorization language for us. Maybe computationally, maybe we don't need this so much, but computationally we also need to think a bit how we deal with this. At the moment we essentially deal with this as distributions of cells. So this is sort of the solution at the moment, but I think it can be much more subtle, like graphs or something like that.
所以,一方面你有苦涩的教训,你知道,你移除归纳偏置,你最终能够拥有一个适用于你问题的更大的数据集,然后这帮助你获得更准确的模型,因为分布更广等等。所以那是其中之一,但然后有这种抵消的趋势,也许你可以称之为反苦涩的教训,或者像苦涩教训的另一面,当你在数据贫乏的情况下,对于一件事,你必须引入大量的归纳偏置来获得足够的统计功效以做出好的推断。但还有第二个方面,我认为 John Platt 在我们不久前与他的讨论中提出,你有这些预测模型和描述模型。而作为科学家,你真正想要的是达到简化,一个描述模型,你可以说:“哦,我测试了所有的苹果,重力对苹果的作用是这样的。”然后但我将其建模为一个普遍概念,简单地以便我们知道它也适用于植物。对吧?所以我想从你那里了解,有哪些简化——或者你知道,也许是简化,也许是见解,也许是实际的数学或其他科学模型——正在变得明显,将有助于减少所有的复杂性,以便我们能够从这些真正大的数据集中实际获得有意义的收获、有意义的理解。
So, on the one hand you have the bitter lesson and you know, sort of you remove inductive bias and you end up being able to have a larger data set that's applicable to your problem and then that helps you to get more accurate models because the distribution is wider and whatever. So that's like the one and then but then there's this like countervailing tendency which is like maybe you could call it the anti-bitter lesson or like the other the flip side of the bitter lesson where when you're in data poor regimes for one thing then you have to introduce a lot of inductive bias to get enough statistical power to make good inferences. But there's also a second side of that which I think John Platt in our discussion with him not too long ago brought up was that you have these like predictive and then descriptive models. And what you really want as a scientist is to get to a simplification a descriptive model where you can say, "Oh well, I've tested all the apples and the gravity works in such a way on apples." And then but I'm modeling this as a universal concept with simply so that we know that it works on plants as well. Right? And so that I want to understand from you, what are the simplifications or you know, sort of maybe simplifications, maybe insights, maybe actual mathematical or otherwise scientific models that are becoming apparent that are going to help to reduce all the complexity so that we're able to actually make meaningful gain meaningful understanding from these really large data sets.
是的,这是一个非常重要、我觉得有点深刻的观点,它把领域分成了两类人,对吧?我们需要在模型里加入多少归纳偏置、先验,来让它更可解释,同时可能对分布外数据更鲁棒?还是我们只依赖更多数据,让模型更有表现力等等。在深入哲学之前,我需要说清楚:文献已经非常清楚地表明,几十年来,至少对我们所有人、很多数据集来说,如果我们把先验归纳偏置建得太强,可能会损失预测能力,对吧?所以是有代价的。如果你是一名医生,或者甚至你是一名患者,可能得了癌症也可能没有,也许你想要最好的预测模型。也许在接受治疗时,你想要一个稍微差一点但你能理解的模型。但至少对于预测,我想你会想要最好的。这一点我觉得并不完全清楚。在我们领域,扰动空间的规模,细胞规模我觉得很大,但扰动空间的规模,比如我们对它做的事情没那么大,我不认为我们已经处于超级数据丰富的状态。所以,在我们这种情况下,这些混合模型可能仍然是首选,但当然会越来越多,这是我从大规模机器学习中学到的诚实教训,不知道能不能叫悲伤,但就是:在某个时刻,数据会战胜人类额外添加的概念。我经常举的一个简单例子是 AlphaFold,第一个模型做出了非凡的努力,让分子具有旋转不变性,把它建到学习过程中,而只有在后来的版本中,旋转对称性才被模型自动发现,只需更多数据,而且更灵活,因为无论你加入什么物理约束,它并不总是那么刚性,如果你让模型从中学习,你会有所收获。所以,我认为未来,至少更多被使用的模型将是那些更灵活的、非常依赖数据的模型,然后你再事后把解释加在上面。
Yeah, that's a very important, I think kind of deep point, which I think sort of breaks the field into maybe two different groups of people with answers, right? So how much do we need to put inductive biases, maybe priors into a model to maybe make it better explainable, right? But potentially also more robust in going out of distribution. Or how much are we going to just rely on bigger data to sort of have the model be more expressive and so on. And I think there I need to be very clear before we may go into the full philosophy. I think literature has really clearly shown over, I think now decades, at least for all of us, very many data sets that if we build an a priori inductive biases too strongly, that we might lose predictive power, right? So it comes at a cost. So, and if you know, let's say you're a physician and you just want to, or maybe even if you're a patient and you maybe you might have cancer or not, maybe you want to have the best model as a predictor. Maybe she getting treated, you want to have a model that is maybe a bit less good, but you understand it a bit more. But for the prediction, at least I think you want to have the best. And this is I think not entirely clear. And I think in our field with this perturbation space scale, the cell scale I think is big, but the perturbation space scale, like the stuff that we do to it not being as big, I don't think we're yet in the super data rich regime. So, I think in our case these hybrid models might be still the thing to go for, but more and more of course, and this is I think this I don't know if you can call it sad, but the honest learning from big scale machine learning is that at some point data beats human-made additional concepts in these models. And I know at least one simple example I often put is AlphaFold, where you know, the first model sort of made this extraordinary effort to make the molecules rotation invariant, they sort of built it into the learning process, and only in later versions, you know, sort of rotation symmetry was automatically found by the model just with more data and was a bit more flexible because whatever physical constraint you put in, it's not always as rigid, and then you sort of you gain something if you let the model learn from this. So, I think in the future the bigger part at least of the sort of more used models will be those more flexible, very much database ones, where you then a posteriori put the interpretation on top.
这也让我想到一个问题。你最近写了一篇或合著了一篇论文,关于当前一代虚拟细胞在数据方面的统计功效,即从额外数据中获得的提升。我想你基本上说过,目标函数或训练方法可能是问题所在,而不一定是信息不在数据中。所以,你能解释一下吗?为什么你认为信息中可能有大量未开发的信息?然后这能带我们到哪里?我们需要做什么才能更进一步?
This also brings up the question for me. So, you actually I think recently wrote or co-authored a paper on the statistical power that the current generation of virtual cells have in terms of the data, the lift that you get from additional data in those current generation of virtual cell models. And I think you had said basically that the objective function or the training methodology is potentially the problem there and not necessarily that the information is not in the data. So, like what can you explain that a little bit and why you think that maybe there's plenty of untapped information in the information and then like what where does that get us and what do we need to do to get further?
是的,可以公平地说,早期我们有一些非常小、更有限的预测模型,先学习潜在空间,然后再加上扰动。然后,当这些真正大规模的基础模型表示出现时,每个人都相信了炒作,可能有点过度相信,然后,显然之后来了所谓的清醒期之类的。然后有一大堆论文出来,基本上说,嘿,也许甚至更线性的模型也做得很好。实际上现在我看到 gen 论文,早期我们已经有一些基准测试表明,这不是那么超级简单容易的任务,只是改进它。我认为有一点道理,在当前数据状态下,对于某些事情,比如你想预测细胞中 CRISPR 的效果,你可能不需要最花哨的模型。而如果你问一点,或者你想预测细胞类型之类的,你知道,对于那个,逻辑回归通常就够了。但如果你想预测更难的东西,例如在我们的 niche former 论文中,这是一个空间组学的基础模型,我们展示了一个我们引入的新任务,你说你有基因表达,你有单个细胞在潜在空间中的状态,你能预测其邻近细胞的基因表达状态吗?因为你知道,它们总是在交流,或者反过来。所以,它们总是在交流,因为你知道,它们互相交谈,你会在一个细胞中看到另一个细胞状态的残留。然后实际上这更难做,如果你有这些更难的任务,你需要更强的、更线性的东西,你尤其需要寻找更微妙的数据。所以,你的问题,你能发现更多信息吗?哪些类型的东西,我认为很大程度上取决于你最终问什么。当然,在自监督学习阶段,我认为我们绝对还没有找到最好的表示和学习方式。有很多复制 NLP 类型 LLM 中发生的事情,对吧?掩码自编码器总是典型的方法,但损失的类型、表示的类型,我认为人们还在争论。也许因为你知道,这通常是表格数据,所以你可以直接尝试表格基础模型和各种东西。现在有很多 JEPA 类型的方法,因为就像我们有一件事用对比学习。所以人们只是在玩弄损失,但也玩弄表示。我认为现在审稿人,至少我长期以来一直倡导这一点,真的更仔细地看,你真的正确展示了缩放定律。这是早期基础模型论文中缺失的,所以人们还没有真正展示你需要这种复杂性才能真正扩展。但如果你真的能在规模上展示,比如跨例子和样本,你有这种缩放行为,那么我认为人们就会相信这些模型。对于这些,当然,至少使用的归纳偏置不多。所以我确实认为我们实际上会很快达到这个状态,我们也在几篇论文中看到了这一点。
Yeah, I think it's fair to say that early on we had some very small, more limited prediction models of learning, let's say latent spaces first and then also perturbations on top. And then, you know, when these really large-scale foundation model representations came out everyone was sort of buying into the hype and maybe potentially sort of bit over buying then, you know, obviously after this you know, comes what's it called the sobering up value or something like that. And then there were a whole bunch of papers coming up where they were basically saying, you know, hey, maybe even more or less linear model were doing good job. And actually now I see gen paper early on we had already some of these benchmarks showing that it's not so super easy and simple task is just to improve on that. I think there's a truth to it that the particular in the current data state for some things let's say you want to predict the effect of a crisper in a cell and you might not need the most fancy model. Whereas if you ask a bit or if you want to predict a cell type or something like that, you know, for that mostly logistic regression is fine. But if you want to predict something a bit more difficult, for example, in our niche former paper which is a foundation model for spatial omics, we showed we had a new task which we introduced where you say you have a you have the gene expression you have sort of the state in latent space of a single cell, can you predict the gene expression state of its neighboring cells? Because you know, they're always in communication or maybe the other way around. So, like they're always in communication, so because you know, they talk to each other you sort of see a remnant of that sort of cell state in this one. And then actually this is harder to do and if you sort of have these harder tasks you need the stronger more linear things and you particularly need to look for more subtle data. So, your question, can you sort of find out more sort of information which type of things I think very much depends on what you sort of ask at the end of it. Of course, like in the self-supervised learning phase, I think we're definitely not there yet with the best way how we represent how we learn these things. There's a lot of copying of what happens in NLP type of LLMs, right? And like masked autoencoders are always the typical way to go, but the type of losses, the type of representations, I think people are still debating. Maybe because you know, this is sort of often kind of tabular data, so you can just like tabular foundation models and all sorts of being tried out. There's a lot of JEPA type of approaches now because it's like we have one thing where we use contrastive learning. So it's just people playing around with the losses, but also with the representations. And I think nowadays reviewers, and at least I've been advocating this for a long time, are really much more closer looking that you really properly show scaling laws. This was what has missed from the early papers of the foundation models, and so people haven't really shown that you need this complexity to actually sort of really scale up. But if you go really if you can really demonstrate at scale like across examples and samples that you have this sort of scaling behavior, then I think you have this buying into these models. And for these, of course, there's not so much at least inductive biases used. And so I do think we're going to hit this regime actually quite soon, and we do see this also in a few papers.
例如,在我们的对比学习论文中,我们称之为 SE concept,我们已经展示了相当强的 Scaling(规模扩张)行为。在论文中,我们有一些进展,因为我们与 Gene Former 相比,达到了大约 3000 万个细胞,但现在在 Hugging Face 上,我们有一个 3.5 亿个细胞的模型,参数可能高达 1 亿,所以对我们来说真正走向了大规模。我们确实看到良好的 Scaling(规模扩张)行为已经开始显现。
So for example, in our contrastive learning paper, we call it SE concept, we do show already quite strong scaling behavior. In the paper, we had something for going because we compared to Gene Former going to maybe 30 million cells, but on Hugging Face now, we have a 350 million cell model with I think going up to maybe 100 million parameters, so really going to the big scale for us. And we do see nice scaling behavior already starting to emerge.
那么明确一下,在 1 亿参数和你说的大约 3 亿个细胞的情况下,你在哪些应用上获得了提升?是在这些 niche former 类型的问题上,比如你根据我的转录组问“我的环境是什么?”
Just to be clear, then at 100 million parameters and three you said I think 300 million cells or something like that, that you get a lift in what kinds of applications? In these niche former kinds of questions where you're saying, "What is my environment?" based on my transcriptome?
是的,我的意思是,当然你最终想要做的是更好地进行扰动预测,因为这就是人们现在至少理解的虚拟细胞,对吧?为此,我不会声称纯表示学习是正确的方向。抱歉再次强调这个微妙之处,但我认为通常你用这些基础模型学习状态空间,但然后它们在动作、扰动下的行为方式,这就是你需要更多世界模型类型方法的地方。世界模型是一个非常时髦的术语,我相信你在播客中已经听过。所以,去年所有机器学习会议上都有它,但事实上,像虚拟细胞这样的扰动预测模型就是一种转录组或其他组学空间的世界模型,对吧?所以,它不仅仅是一个 f(x) 的模型,而是 f(x, a),对吧?状态 x 加上动作,然后预测下一个状态 x' 之类的。我认为为此你可能需要其他组合。所以我们通常做的是,例如用 SE concept 学习表示,但然后我们放一个生成模型来沿动作推进。为此我们使用类似流匹配的方法。在我们的 self flow 论文中,这不仅被 Nature Methods 接受,我们实际上真的展示了这些方法在生成扰动效应方面要好得多,但它们非常依赖于一个好的潜在空间。而你在这个 SE concept 潜在空间上。在这种情况下,对于这 3.5 亿个细胞,我们实际上进入了多物种。所以你现在有大约 16 个不同物种,你可以潜在寻找相似性。突然之间,你也许可以问一些以前无法问的进化问题。并不是说我们能做得很好,这不是我们的核心专长,但你知道,你可以开始做这些花哨的跨物种、跨组织类型的潜在空间。
Yeah, I mean you know, the ultimate thing of course you want to do is you want to be doing this perturbation prediction better cuz that's sort of what people now at least understand by virtual cell, right? And for this, I would not claim that pure representation learning is the right way to go. Sorry for making the subtlety again, but I think typically you learn the state space with these foundation models, but then sort of the way how they behave under an action under perturbation, this is where you need this more world model like type of approaches. World model being the very hip term, I'm sure you had this already at some point in your podcast. So, it's been on all of them machine learning conferences in last year, but de facto a perturbation prediction model like a virtual cell is kind of a world model in terms of transcriptome or other type of omic space, right? So, it's not just a model that is sort of f of x, but f of x comma a, right? So, the state x plus the action, right? And then sort of predicting the next state x prime or something like that. And I think for this you might need other combinations. So, what we typically do we learn the representations with for example as a concept, but then we put a generative model for the push forward along the action. So, for this we use something like flow matching or something like that. And in our self flow paper, this is not just uh got accepted for nature methods. We actually really show that these approaches are much better generating the effect of a perturbation, but they're very much dependent on a good latent space. And you're on this SE concept latent space. In this case, for these 350 million cells, we actually went into multi-species. So, you have now like 16 different species where you can potentially look for similarities. Suddenly, here you can maybe ask some evolutionary questions that you couldn't before. Not that we can do this so well, it's not our core expertise, but you know, sort of you can start doing these fancy cross species cross tissue type of latent spaces.
我对这些不同模型的历史有点好奇。我想你刚刚给了我们一个快速版本。我认为在社区中 somewhat 著名的一件事是关于线性模型是一个大点。你能具体解释一下这里的 key point 是什么吗?因为我认为这是经典机器学习中的事情之一,确保你总是很好地理解你的基线,你知道,否则你会花很多时间 spinning your wheels 并欺骗自己。我认为 specifically 有一个著名的公开竞赛,通过 Arkin 的 two,那是一种 like 冷水泼在每个人脸上的 class。你能 maybe 给一点关于那个的历史,以及为什么我们认为这很令人兴奋,然后为什么这让我们有点 pivot。
I'm kind of curious at the history of some of these different models. I think you just gave us a sort of a rapid version of them. One of the things that I think was somewhat famous in the community was this thing about linear models being kind of a big point. Can you explain specifically what was the key point here? Cuz I think this is like one of those things in classic machine learning, make sure you always understand your baselines well, you know, otherwise you spend a lot of time spinning your wheels and fool yourself. I think there was specifically a famous open competition on with the through the Arkin's the two that was a sort of a class of like cold water on everyone's faces. Can you maybe give a little bit of history about that and like why we thought this was exciting and then where why this had us pivoting a bit.
是的。这听起来像是一个如此明显的教训,对吧?正确地进行基线比较。但在一个急于奔向虚拟细胞金罐的领域,人们有时可能对基线不够小心,并且关于 holdout 等有一大堆微妙之处,你知道,并不总是被完全观察到,以及为什么有些这些事情发生。所以,第一个观察是在像细胞类型预测这样基本的事情上,好吗?所以,没什么超级花哨的,好吗?基本上,你给我一个新的转录组,你告诉我它来自哪种细胞类型,对吧?甚至可能来自哪个器官之类的。事实证明,即使在今天,我认为一些接近最先进的模型实际上只是逻辑回归,对吧?所以,对于这样一个简单的分类,你知道,这实际上并不那么糟糕。例如有一个叫做 cell type 的东西,它做得很好,我想我们大约五六年前有一篇论文,标题很直白,“细胞类型预测不需要深度学习。”如果是在一个器官内,这是真的。如果你跨器官,突然变得有点微妙,因为你知道,细胞类型在不同器官中重复出现,免疫细胞根据它们所在的组织有细微变化。突然之间,你知道,在那里你需要做更多,而我们 like,我们发表了一篇叫做 SC tab 的论文,like 一个表格基础模型,预测模型类型的东西。我们实际上展示了非线性确实在基线之外增加了价值。所以,那是 for that class。对于扰动模型,然后你问 Brennan 类似地 like 早期,许多分析特别是扰动可以用 Suda pack 中的经典线性模型运行。然后我们有我们的 SCGen,大约在 2019 年,我们开始引入线性偏移但在潜在空间中,所以是非线性编码,我们总是与微妙的更简单的线性偏移比较。但然后,领域变得复杂,你知道,数据集涌现,人们开始问越来越多的扰动类型问题。早期有一大堆这些竞赛,like 例如 MIT 定期举办的一个。我们一直在运行定期的 NeurIPS 竞赛,关于单细胞数据中的预测问题,有时是整合数据,也是扰动预测。那非常受欢迎。锦上添花 most definitely 是那个 arc 竞赛,它把东西称为不仅仅是扰动预测挑战,而是虚拟细胞挑战。听起来更酷一点,对吧?我们 picked up 很好。但 with 指标是稳健的 were maybe 但也许不那么容易。到目前为止,实际上我们在一个由许多实验室贡献的 consortium 中 called open problems 的解决方案实际上是人们需要提交他们自己的指标和 additions so 为了真正 maybe 微妙的事情,你可能以前没有想到。Dark one 也修复了一堆甚至不是单细胞特定的指标,它们可能 just 被利用。
Yeah. It sounds such an like such an obvious learning, right? Do your baselines properly. But in a field that is rushing towards the gold pot of the virtual cell, people sometimes maybe were not as careful with the baselines and there's a whole bunch of subtleties with respect to holdout and so on that, you know, wasn't always fully observed and why some of these things happen. So, one of the first observations there was in a thing as basic as cell type prediction, okay? So, nothing super fancy, okay? Basically, you give me a new transcriptome, you tell me from which cell type it is, right? Maybe even from which organ or something like that. And it turns out that even nowadays, I think some close to state-of-the-art models are really just logistic regression, right? So, for such a simple categorization, you know, this is actually not so bad. There's one thing called cell type is for example, which is doing a good job and I think we had a paper maybe 5 or 6 years ago where we had the title very plain, "Deep learning is not needed for cell type prediction." This is true if it's within an organ. If you sort of go cross organ, suddenly gets a bit more subtle because, you know, sort of cell types reoccur in different organs and immune cells are a little subtle change depending which tissue they are. Suddenly, you know, there you need to do a bit more and we like for us, we published on a paper called SC tab, like a tabular foundation model, predictive model type of thing. We actually showed that this nonlinearity was really adding beyond the baseline. So, that was sort of for that class. For the perturbation models, which then you were asking Brennan similarly like early on, many analyses particular perturbation could run with classical linear models in Suda pack. We had then our SCGen that was like something around 2019 where we started introducing linear shifts but in the latent space, so sort of non-linear encoding and we always compared with subtle sort of simpler linear shifts. But then, the field getting complex and you know, sort of data sets popping up, people started asking more and more perturbation type of questions. There have been a whole bunch of these early on competition for like a regular one run for example by MIT. We've been running a regular NeurIPS competition on prediction problems in single cell data, sometimes for integration data, also for perturbation prediction. That was very well attended. The cherry on the cake was most definitely that arc competition which called the thing not just perturbation prediction challenge, but virtual cell challenge. To sound a bit cooler, right? We picked up very well. And but with metrics that are robust were maybe but maybe not that easy. So far actually our solution in this consortium where a lot of labs contributed called open problems was actually that people need to submit their own metrics and additions so in order to really maybe subtle things that you might not have thought before. Dark one is also fixed a bunch of metrics that weren't even so single cell specific and they could be just exploited.
所以实际上我认为最终被加冕为真正获胜模型的,本质上是一种流匹配的最佳方法。基于人们争论的这种流匹配、细胞流类型的想法,我认为这表明在这种情况下需要一些非线性。所以真正的赢家,或者如果你排除利用的话,实际上是采用了一些非线性。我认为如今我们确实看到,仅仅线性或加性类型的模型对于更复杂的事情是不够的,但即使在现在,如果你做一个简单的细胞系实验,有基因敲除,特别是当效应比较微妙时,所有这些更复杂的模型有时会有点过拟合,这就是为什么线性基线总是很好。所以我总是告诉我的学生,伙计们,我知道我们有伙计们,我是说伙计们和姑娘们。我们总是需要做,我知道你们想做最花哨的深度学习东西,但请务必有一个基线,比如对于药物,一个简单的基线就是预测均值并在此基础上改进。你知道,那就像是最热的基线。那甚至不是线性模型,比均值更好有时并不那么难,如果数据中有很多噪声,而且这些东西超级嘈杂。
So actually I think the winning model that was then in the end actually sort of crowned as the actual winning model, but essentially is a flow matching best thing. So based on this idea that people fought with this flow with this cell flow type of idea. I think it shows you that some non-linear things are in this case needed. So that the actual real winners or if you sort of remove the exploitation was actually something that is taking some non-linearities and I think nowadays we do see that just linear or additive type of models are not cutting the cake for more complex things, but for I think even nowadays if you go in a simple if you cell lines and you have sort of gene knockouts then particularly when the effects are a bit subtle all these sort of more complex models are sometimes a bit overfitting and that's why a linear baseline is always very good. So I always tell my students, guys I know we have guys I mean guys and girls. We always need to do I know you want to do the most fancy deep learning thing, but please please you know have the baseline and like say for drugs you know a baseline as simple as just predicting getting better at the mean. You know, like that's the like the hottest base. That's not even a linear model like better than the mean is sometimes not as hard if there are so much noise in the data and this stuff is super noisy.
我只想补充一点,Bo Wang 和他的一些合作者实际上刚刚在《自然》上发表了一篇论文,指出在某些情况下,线性基线击败一些模型的问题实际上在于度量指标。所以很明显,这不仅仅是度量指标被操纵,还在于问错误的问题会导致错误的答案。
I just wanted to add one point that Bo Wang and some of his collaborators actually just had a Nature paper on how the metric was actually the problem in some of those linear baselines beating some of the models as well. So it's clearly it's not only metric hacking, but also just that asking the wrong question leads to the wrong answer.
我们选择度量指标,我认为他们可以,你可以讨论它。在某个时候,我们将会有智能体尝试各种你能想到的算法组合。事实上,我们确实与一些自动机器学习类型的实验室有合作,尝试这个。所以,也许剩下的听起来有点太苛刻了,对吧?但也许我们最大的想法实际上在于这个度量指标的制定,RJ,对吧?
Our choice of the metric and I think that they could you can discuss it. At some point we're going to have agents sort of trying out all kind of combinations of whatever algorithmic thing you can come up with. In fact, we actually do have some collaborations with some of those sort of auto ML type of labs try this out. So, maybe what is left sounds a bit too harsh, right? But like what maybe our biggest ideas go into is actually this metric formulation, RJ, right?
所以,我一直非常感兴趣的是学术研究中不同的资助模式。我认为德国的结构从根本上与美国不同。你的机构是如何设置的?这如何带来与美国研究模式不同的机会?
So, one thing I've been really interested in is the different models of funding in research academia. I think Germany has a different structure fundamentally from the US. How is your institution set up and like how does this give like distinct opportunities versus like a US-based research model?
我非常同意这些资助模式对灵活性非常重要,但对你能够建立的团队以及你能够拥有的长期愿景也很重要。所以,嗯,Helmholtz,这不仅仅是德国,而是我认为可能是欧洲最大的所谓研究组织在资助方面。它有不同的重点领域。健康是其中一个大的领域,然后还有材料、信息科学、物理学等等。它基本上获得联邦资金来投入深层研究问题。我一直在建立所谓的计算健康中心,在 Helmholtz Munich,我认为我们现在大概有 45 个课题组组长和教授,可能现在有 500 到 600 人。所以,我认为我们是欧洲最大的健康 AI 类型研究的学术场所,我们有人在各个尺度上工作。比如,你知道,我有点专注于细胞尺度,但你知道,有人在研究 DNA 基础模型。有人在蛋白质设计方面做一些有趣的事情。有人在研究患者队列类型的表示、遗传学等等。现在你可以开始把它们放在一起,我们的一个大愿景是交叉并走向虚拟人类等等。但话虽如此,关键是与大学相比,尽管我们都隶属于大学,主要是慕尼黑工业大学或慕尼黑大学,这两个是欧盟排名最高的大学,有趣的是。所以我们有这种地方,我们在大学有隶属关系,对吧?但是,我们没有本科教学。我们基本上只有博士生和博士后,但我们与大学互动很多,特别是与诊所,因为我们想要有这种转化性的东西。而且越来越多地,这可能是一个有趣的联系,所以这也许有点类似于美国 NIH 类型的模式,但分布在全国各地,而不仅仅是在一个地方。也许一个有趣的联系是,政府以及德国这种认识逐渐明朗,需要有更强大的转移组件,不仅仅是通过转化到大学,还要在学术领域之外创造价值,这显然越来越多地进入初创企业等等,我认为波士顿环境一直是一个关键例子,但现在当然也更是在湾区,你知道,最近我在访问,我相信可能是 Genentech 或同事,我们去了一家餐厅,然后其中一个服务员问我们:“嘿,你们在谈论 AI 和生物吗?你知道,我也一直感兴趣。”所以,就像每个人都在思考这些初创想法等等。我有点觉得这是一个好环境,这是我认为我们的博士生现在越来越多地做的事情,我们在那里学习,我认为在价值创造方面,特别是慕尼黑实际上至少在欧洲大陆,或者可能是现在风险投资资本的主要地方,但也是深科技,特别是生物和国防领域,真的在爆炸式增长。所以,这些东西开始起作用了。
I very much agree that these funding models very much matter for the flexibility, but also the set of teams that you can set up and like the long-term vision that you can have. So, um Helmholtz, this is not just Germany, but I would say maybe Europe's largest what's called research organization in terms of funding. It has different set of areas that it focuses on. Health being one of the big ones, but then it has sort of material, information sciences, physics, and so on. And it's essentially getting fed federal money to put into a deep research question. I've been building up what's called the computational health center at Helmholtz Munich, which I think we has now something like maybe 45 group leaders and professors and maybe 500 to 600 people now. So, we are I think the largest academic place for health AI type of research in Europe where we have people really working on all scales. Like, you know, I'm sort of focusing on the cellular scale, but you know, there's people doing DNA foundation models. There's people sort of doing interesting things on protein design. There's people doing like, you know, patient cohort type of representation, genetics, and so on. And now you sort of can start putting them together, and one of our big visions is sort of to intersect and sort of go virtual human, and so on. But having said that, the key thing is in contrast to universities that even though we all have affiliation to universities, mostly to the Technical University or the University of Munich, the two top-ranked universities interestingly in the European Union. So, we have this sort of place where we have these affiliations at universities, right? But, we don't have undergraduate teaching. We essentially just have PhD students and postdocs, but we interact very much with the universities and in particular also the clinics because we want to have sort of this translational thing in there. And increasingly, that's maybe an interesting link to So, this is maybe a bit similar to NIH type of models like in US, but sort of distributed all over the country, not just sort of in one place. And maybe the one interesting um um um link is that increasingly the government and this sort of realization also in in in Germany is is sort of dawning that there needs to be much stronger transfer component, not just by going into translation to universities, but also in into sort of value creation outside of the academic realm, which is obviously going so more more into startups and so on, which I think the Boston environment has always been one of the key examples, but now of course more so now also the the the Bay Area when, you know, recently I was was visiting uh I believe maybe Genentech or colleagues, and we went to a restaurant, and then one of the waiters was asking us, "Hey, are you guys talking about AI and bio? You know, I have been also interested." So, like everyone sort of thinking about this startup ideas and so on. And I I kind of felt that's a it's a good environment, and this is something I think our PhDs are now more and more doing, and and we we are sort of learning there, and I think also in terms of sort of value creation, Munich in particular is actually at least in continental or maybe the prime place now for VC capital, but also sort of deep tech and particularly the bio and and defense area and sort of really exploding. So, this stuff starts to work.
我们谈论初创企业的一件事,我认为非常有趣。我听说德国的初创企业生态系统存在一些问题。与欧洲其他地方相比,在那里创建初创企业很困难。那么,德国目前的初创企业竞争格局如何,具体来说,德国如何变得更广泛地具有竞争力?嗯,我对此有既得利益,因为我可能很快就要获得德国公民身份了。
One thing we're talking about startup, I think is really fascinating. I've heard that Germany has had some issues with their startup ecosystem. It's try It's difficult to create a startup there compared to in other places in Europe. How like what is the competitive landscape for startups right now in Germany, and specific and how does Germany become more competitive broadly? Um I I have a vested interest in this because I'm um probably going to get my German citizenship soon.
欢迎。如果你来慕尼黑,告诉我。所以,嗯,我认为传统上欧洲,但我会说德国肯定是相当官僚主义的,流程远没有你在其他地方期望的那么数字化。我并不是说美国可能是完全数字化流程的最佳例子,但事情肯定更快。
Welcome. Let me know if you come come by Munich. So, um I think traditionally Europe but I would say definitely Germany is is quite bureaucratic and processes are far from as digital as you would expect them in different places. Not that I'm saying that the US is maybe the best example for fully digitized processes, but stuff is definitely faster.
问题是人们现在已经意识到了这一点,我认为在德国,创业公司的创建和实施比欧洲其他地方更有竞争力。实际上,这几乎是首要的,尤其是一些热点地区在燃烧,但特别是慕尼黑,我认为是创业公司数量最多的首都,甚至如今如果你看欧盟,创业公司的创建数量和美国一样多。问题在于,欧洲尤其是德国在 Scaling(规模扩张)阶段落后了。所以,你知道,早期风投以及那种非稀释性资金,这些是有的。但当你进入 Scaling(规模扩张)阶段,那些承担更大风险的大型风投,我认为并没有那么密切地存在。所以我一次又一次地看到,如果你想真正获得更大的投资,比如 A 轮、B 轮,人们开始更多地转向美国。但也许我在这里过于简化了,但这至少是我目前的学习心得。
The thing is people have realized that now and I think the particular startup creation and implementation is more than competitive in Germany versus Europe. It's actually sort of the prime number and in particular like some hotspots burning, but in particular Munich I think is the capital for the highest number of startups and I think even nowadays if you look into the European Union, there's equally as many startups being created as in the US. The thing is where Europe and particularly also Germany drops off is at the scaling part. So, you know, early VC and also sort of non-diluted type of money, this is there. But then when you go into scaling, these big VCs that take these bigger risks are I think not as closely as present. So, I saw this again and again that, you know, if you want to really go into the bigger tickets sort of Series A and so on Series B, that people start going more into the US also for that. But maybe I'm oversimplifying here, but this is at least my current learning.
好的,你是董事会成员,或者我猜是几家本地初创公司的一部分,对吧?在德国,从想法到创建一家新初创公司需要什么?
Okay, you are on the board or I guess part of several local startups, right? What does it take to go from, you know, idea to creating a new startup in Germany?
我不知道你是否听说过欧盟试图监管一大堆东西。这有时可能是一个优势。你知道,这很酷,比如现在在整个欧洲我们都可以用手机。你可能听说过现在每个人的手机都需要有 USB-C 接口。这是很酷的东西。所以有一些好东西即将到来。但当然,也有像《人工智能法案》等东西让事情稍微复杂一些。幸运的是,研究还没有受到影响,但你知道,对于医院之间的数据访问等,有一些系统我认为在欧洲非常棒。我想到的是芬兰的 FinnGen、丹麦等,它们对人群有非常详细的描述,在采样、连接医疗记录方面比……我的意思是,美国在让所有系统相互通信方面也像德国一样混乱。就像患者在美国去不同的州,和在德国去不同的地方类似。建立患者目录非常困难,对吧?但有些地方知道如何做到这一点,欧盟有一个叫做欧洲健康数据空间的东西,可能在接下来的两年内,这些事情需要跨越边界。你需要开始构建这些健康记录类型的组合。所以我认为愿景开始实现,但规模不够快。所以我认为对于科技初创公司来说,我注意到的最重要的事情是你需要拥有最好的团队。你知道,我们有教授和 PI 真正在思考这个问题。例如,对于我协调的部门,我说:“伙计们,每个 PI,每个组长,我期望你们至少在未来 3 年内有一个衍生公司。”看看效果如何,对吧?但你看到这会让人们走上这条轨道。但我认为现在这很大程度上是自下而上发生的,学生们实际上在推动这一点。他们确实意识到目前有大量机会,特别是随着计算角度的推动,我们现在可以开始实际指导实验。我们还没有真正谈过这个,对吧?但一旦你有了这些扰动世界模型,你实际上可以回去驱动实验,对吧?所以我认为计算型初创公司也可以非常擅长数据生成。我认为这不仅是变成漂亮的论文——这主要是我们的业务——而且我认为这是下一代初创公司。你确实看到一些这样的公司冒出来,你实际上把实验室在环真正实现了。
I don't know if you heard that the European Union tries to regulate a whole bunch of stuff. And that can be sometimes an advantage. You know, it's kind of cool like, you know, all across Europe we can now go with a mobile phone. And you might have heard that everyone needs to have USB-C things now for their mobile phones. This is kind of cool stuff. So, there's some good stuff coming. But then of course there's something like the AI Act and so on that make things a tad more complicated. Luckily not yet for research, but you know, for data access across hospitals and so on, there's some and you know, and maybe we're talking about health deep tech type of stuff, right? So, there's some systems that I think fantastic in Europe. And I'm thinking about Finland, you know, FinnGen, Denmark and so on with this sort of very detailed description of the population in terms of sampling, getting medical records connected far better than I mean US is also similar to a mess like Germany in terms of getting all the systems to talk to each other. Like when patients, you know, go to different states in US is similar as in Germany when they go to different places. It's like very hard to build patient directories, right? But there are some places that know how to do this and there's a thing like the European Union tells you it's called the European Health Data Space where in like maybe next 2 years these things need to be going across boundaries. You need to start building these health record type of combination. So I do think the visions are starting to happen, but the scale is not as fast. So I think what you need for a tech startup what I noticed is always the biggest thing is you need to have the best team for that. And you know, we have professors and PIs that really think about this. So for example, for the department that I coordinated, I said, "Guys, every PI, every group leader, I would expect you to at least have one spin-off in the next 3 years." And see how that works, right? But you sort of see this sort of get people a bit on that track. But I think this is happening very much from the bottom up nowadays anyway where the students are actually pushing for that. They do realize that there's a whole bunch of opportunities at the moment, particularly with this computational angle getting pushed even more in the forefront that we can now start actually guiding experiments. We haven't really spoken so much about, right? But once you have these perturbation world models, you can actually go back and drive the experiments, right? So I do think computational startups can be very good also data generation. And I think this is what will be also turned not just into nice papers, which is sort of mostly our business, but also I think it is next generation of startups. And you do see a few of these popping up where you actually get the lab in the loop really implemented.
你认为这些虚拟细胞相关想法中最有前景的商业化领域是什么?
What do you see as some of the most promising areas for commercialization of sort of these virtual cell adjacent ideas?
这很有趣,对吧?如果你看看早期,当然人们总是会想到 OpenAI 等想要做的事情。他们并不真正知道如何赚钱,有些人可能会说他们现在甚至都不知道,对吧?有一些初创公司获得了非常强劲的投资,它们也说我们构建生物学的基础模型。我们解码生物学等等,却没有真正说明你想用它做什么。我一次又一次地看到,这些平台类型的想法非常有吸引力。你知道,它们符合我们所知道的。我的意思是,你谈到了我们构建的一些工具和平台,比如 Scanpy 和 scverse 等,对吧?但将其真正转化为金钱并不是那么明显。我的意思是,可能会有一些咨询业务,但如果你说你构建了一个虚拟细胞平台,通常投资者理解的是——在欧洲和美国更保守的风投生态系统中可能更是如此——你在某个时候开发了一种药物,然后你就有明确的东西。所以我看到许多这样的平台公司在某个时候转向“我们制造药物”。是的,当然,另一方面我们有我们的虚拟细胞业务之类的,但你知道,我们找到了两种药物,现在这个。所以我们投入了很多钱。也许你有一些其他的反例。有一些公司仍然坚持扩大 AI 业务,但如何直接将其转化为可行的、真正可扩展的商业模式并不那么清晰。当然,然后还有这种焦虑,即某个时候像 Astra 2.0 可能已经在做其中的一部分了,是的。
It's interesting, right? Like if you look at earlier and of course this is what people always reflect to what OpenAI and so forth wanted to do. Like they didn't really know how to make money and some people might argue they don't even know nowadays, right? And there's a few startups that got really quite strong investment that also said we build a foundation model of biology. We decode biology and so on, without actually saying what you want to do with it. And I've been seeing this again and again that these platform type of ideas are super attractive. You know, they sort of fit into what we know. I mean, you spoke about some of the tools and platforms that we built like Scanpy and scverse and so on, right? But turning this actually into a monetary thing is not as obvious. I mean, there might be some sort of consulting business, but if you say you build a virtual cell platform, typically what investors understand, and this is maybe even more so in the more conservative VC ecosystem in Europe and US is that at some point you develop a drug and then you have this clear thing. So, I saw so many of these platform companies pivoting at some point to we make a drug. And yeah, at the side, of course, we have our virtual cell whatever business, but you know, we have two drugs that we found and now this. So, we put a lot of money in. Maybe you have some other counter examples. And there's a few companies that are sort of still stick with the scaling up of the AI business, but it's not as clear how you would directly turn this into a viable really scalable business model. And there's even room for many of these. Of course, then there's also this angst that at some point like Astra 2.0 might be doing parts of this already anyway, yeah.
是的,我的意思是,我认为这实际上一直是我的一个论点:直到最近,每一家构建模型和构建东西的公司最终都只是制造药物,因为这是随着时间的推移实现货币化的路径,对吧?然后,你必须与制药公司达成销售,这非常困难,或者其他一些大金主。如果你是一家工具公司,如果你说我要构建一个资产,然后我可以把它卖掉,这对投资者来说非常有吸引力。
Yeah, I mean, I think that it's actually this has been a thesis of mine that up until recently every single company that builds a model and builds something ends up just building drugs because that's the path to monetization over time, right? And then like whereas you have to like land sales with pharma that's very difficult or some other big pocket spender. If you're a tools company, if you say I'm going to build a port like an asset and then I can sell that off, that's really attractive to an investor.
但今年我们实际上看到了相反的情况,对吧?比如在结构生物学领域,有几家公司,像 Bolts,Brandon 会更了解它们,但有几家这样的公司,然后 Ty 是我们聊过的一家。我们还采访了几家在做强化学习环境模拟的公司。我很想看到结构生物学领域的活跃程度,也能像 No it 那样做类似的事情。
But we've seen actually the opposite happening this year, right? So, you have like in structural, you have a couple, you have bolts, you have Brandon will know them better, but there's a couple of those and then Ty is one that we talked to. And then we've interviewed a few that are doing reinforcement learning environment plays as well. I would love to see the activity level in structure that we see in structural also doing things like what No it is doing.
我认为 No it 是一个很好的例子,因为他们也有相当不错的数据生成规模,其中很大一部分是数据获取。我们之前谈过这些类器官扰动之类的事情,我们实际上一直在与一个农民联盟合作建立一个我们称之为癌症类器官扰动图谱的项目,然后农民实际上会买账,因为他们想尽早获得这些扰动类型的数据集,对吧?我认为其中一些交易就是朝着这些方向发展的。所以我认为如果你将数据生成与建模部分结合起来,我认为这就是最佳结合点,然后农民实际上也会买账。我看到越来越多的公司不仅围绕虚拟细胞模型构建他们的模式,还结合独特的特定数据。这当然也创造了更多价值。
I mean I think No it is a good example in the sense that they also have I think a rather nice scale of data generation and a big part of that is also access to data. So you know we talked before about these organoid perturbation type of things and we've been setting up actually in collaboration with a whole bunch of farmer a consortium which we call the cancer organoid perturbation atlas with then farmer actually buys in because they want to get early access to some of these sort of perturbation type of data sets right and I think some of these deals sort of go in these directions. So I think if you couple it with data generation not just with the modeling part, I think this is where sweet spot hits where then also actually farmer gets buying. I think more and more of the companies I see they sort of build their mode not just around a virtual cell model but in combination with sort of unique sort of specific data. This is then of course also more value creation.
我不确定这是否完全正确,但我读到你在某处说过,你让你的学生遵循你所说的 Y 模型,其中你必须有一个方法和一个湿实验室导师。这听起来超级有趣,与我们现在的对话非常相关,关于数据如此重要,以及所有对实验室工作的关注,这对许多公司来说是一种模式。你能谈谈这个吗?
I don't know if this is exactly correct but I read that you said somewhere that you have your students follow what you I think called a Y model in which you have to have a methods and a wet lab advisor. That sounds super interesting, super relevant to the conversation we're having right now about data being so important and all the attention on lab work and that being mode for a lot of companies. Can you talk a little bit about that?
是的,感谢你提出这个问题。我一直在建立,从那时起,这已经成为一个相当大的事情,叫做慕尼黑数据科学学院,对吧?所以这不仅仅是生物领域,我们还有许多其他学科,在那里我们有这种导师模式,总是需要两位 PI 一起提出提案,然后向学生提出,在研究生类型的模式中,有这种 Y 形,所以有一个方法一个更领域相关的,对吧?然后有导师学生。有时我们在 Y 模型下有两个学生,所以 Y 加 Y 等于 H。你明白吗?但我认为这个想法在如今甚至有点微不足道,对吧?但我认为把它如此硬性地融入是好的,这样学生理想情况下真的去两个实验室会议。一个实验室会议,他们需要真正学习和理解一些更深层次的生物学问题,理解医学设置的语言之类的。然后他们去计算,通常是所有计算数据科学类型的学生,对吧?但然后他们去计算的那个,他们会被推方法方面,从而吸收一堆问题,这些问题在生物实验室中并没有精确地表述。他们不能直接把它放入一个函数中,但因为他们有点方法性,对吧?他们是定量的,他们实际上可以把它放入一个度量标准或其他什么,然后变成一个模型,然后论文,我认为这就是方式,我的意思是我们已经这样做了大约 8 年,这种类型的学校。真的有很多论文出来了,我了解了以前不知道的系统,对吧?你知道,人们接近我,我们能不能做这样的事情,然后你知道,我学到了一些关于肺或大脑的东西,我完全不知道,但然后我说,“哦嘿,很酷的数据。我认为我们的模型类型可以适应。让我们围绕它建立一个项目。”然后那些学生成为我实验室中那种详细生物学事物的专家,与合作伙伴一起。然后也许我们推动这个前进,这实际上现在也扩展到数据类型。早些时候很多事物只是围绕 RNA,但你知道,从那时起,你知道,我们去年在《科学》上有一篇关于单细胞蛋白质组学的非常好的论文。我们一直在扩展到空间观察。你知道,有所有这些附加性和你可以测量的东西,我认为然后在那个特定领域出现了有趣的问题,你知道,这些 essay 工作得很好。
Yeah I think thanks for raising this. So I've been setting up and since then this has been sort of become quite a big thing what's called the Munich School for Data Science right? So this is beyond just bio so we had a whole bunch of other disciplines and there we had this supervisor model that is always two PIs needed come together to make a proposal and sort of proposes to the students in a graduate student type of model which has this Y so there's like a method e more domain one right and then there's supervisor student. And sometimes we had two students under the Y model so Y plus Y equals H. You get it? But I think this idea is it's somewhat even nowadays a bit trivial right? But I thought it was good to bake this in so hard so that students really ideally go to two lab meetings. One lab meeting where they need to really learn and understand some deeper biological problem, understand the language maybe of a medical setup or something like that. Then they go to a computation typically all computation data science type of students, right? But then they go to the computation one they sort of get pushed the method aspect of it and thereby sucking up a whole bunch of problems that are sort of not exactly precisely as formulated in that bio lab. They sort of can't directly put that in a function, but then because they are sort of methody, right? And they're quantitative, they can actually put this into a metric or whatever that then turns into a model and then paper and I think this is the way how uh I mean we've been doing this now for maybe 8 years or so this type of school. I really a whole bunch of papers came out where I learned about systems that I had no idea about this before, right? You know, people approaching me and could we do something like this and then you know, I learned something about the lung or the brain that you know, I had no idea about, but then uh I said, "Oh hey, cool data. I think our model types could fit. Let's build a project around it." And then those students became the experts in my lab for that sort of detailed biological thing together with a partner. And then maybe we drive this forward and this actually nowadays extends also across the data types. Earlier on a lot of things have been around just RNA, but you know, since then you know, we had a very nice paper in science last year on single cell proteomics. We've been extending things to spatial observations. You know, there's sort of all these additionalities and things that you can measure and I think then interesting questions come up in that particular domain where you know, these essays work very well.
是的,我的意思是这是我天生的倾向,喜欢收集跨领域的东西。我认为这通常很容易产生真正好和有趣的想法。作为一个后续问题,你这样做已经八年了,我想。那么你看到了什么样的长期成果效益?
Yeah, I mean this is my natural inclination is to like gathering cross-domain stuff. It I think it's all often ripe for good generation of really good and interesting ideas. Have you just as a follow-up these you've been doing this eight years now, I guess. So what kinds of you know, sort of longer term outcome benefits have you seen from this?
我认为一件事是,这也是我们如何让学生甚至加入这个项目的原因,也许最初听起来像是更多的工作,对吧?你需要学习两件事,你去两个实验室会议等等。但我认为一旦你的学生在德国完成学业,或者我认为一般来说,但特别是在欧洲,人们通常先读硕士再读博士,对吧?所以不仅仅是美国的学士学位,对吧?所以你多读一两年硕士。然后他们有点了解科学,但他们不完全知道自己对什么感兴趣。所以在这种组合类型的事情中,你知道,他们可以稍微尝试一下,因为在这个研究生院,他们只是收到不同的提案,所以他们可以稍微尝试一下他们对什么领域感兴趣,但然后他们也可以稍微玩弄方法方面。所以我认为这有点吸引力。是的,我认为通过这种尝试不同的事情,我的印象是这让学生成为非常好的数据分析师,但也是建模的人。然后下游,你知道,他们进入的领域也同样多样化,你知道,所以有些人去了,现在他们在生物医学领域进入制药,但你知道,有些人显然留在研究领域,有些人更多地进入创业方向,有些人进入了非常纯 AI 类型的工作。我认为 Nvidia 就在几个月前,在 1 个月内雇佣了我的两个人,非常进入计算方面,也有一点生物部分,Nvidia 现在正在建立,还有其他大公司类似。
I think one thing that this is also I think how we got students to even join in this maybe initially it sounds like more work, right? You kind of need to learn two things and you go to two lab meetings and so on. But I think you once your students are done in in in in in Germany or I think in general but particularly I think in Europe in general people sort of typically do a master's first before they go into a PhD, right? So not just a bachelor's in the US, right? So, you do like one or two year more of master. And and then they sort of know science a little bit, but they don't exactly know what sort of interests them. So, in this combination type of thing, you know, they can sort of sample a bit because in this grad school they just get different proposals in, so they can sample a bit what they're interested in in terms of domain, but then they can also play around a bit with the method side. So, I think that was kind of attractive. Yeah, I I think with this sort of trying out uh different things, I had the impression that this sort of turned the students into very good data analysts, but then also modeling people. And then a downstream, you know, the sort of areas where they they went into was also equally diverse, you know, so some people went like and now they're in the biomedical domain into pharma, but you know, some obviously stayed in research, some went more into some sort of startup direction, some went into the very pure AI type type of work. I think Nvidia just a few months ago were hired in 1 month two of my people going very much into sort of the computation aspect and also a bit of a bio part that Nvidia's now building up and there's similar other of the big companies.
所以,我认为这给了他们一个有趣的履历,但我认为当你在外面接受面试时,最好说,我不只是做了这个在 NeurIPS 论文中可能提升了 2% 的炫酷模型,而且你还必须帮助找到一个用例,进行真正更复杂、更深入的应用。
So, I think it gives them an interesting profile, but I think when you're out there and you get interviewed, it's a good idea to say, I didn't only do this fancy model that improved maybe 2% in a NeurIPS paper, but you also had the real, more complex, deeper application that you had to help find as a use case.
在 AI 领域出现了一场运动,我们请过的几位嘉宾或我们的一篇博客文章提到,湿实验室科学家基本上开始变得计算化,这要归功于 AI。我很好奇:随着这一发展,基本上现在 AI 使任何科学家都能成为自己的计算专家,你认为这种共同指导策略的演变会怎样,以及你认为湿实验室生物学家和计算团队之间的整合演变会怎样?我的意思是,我也能想到我听过几位著名 CEO 说过,我们的优势在于计算和湿实验室团队之间的整合。所以,我很好奇你认为这会如何演变,以及你如何建立文化来利用这些新技术。
There's been a movement amongst AI and several guests we've had or we've had a blog post about how the wet lab scientists have started basically becoming computational thanks to AI. I'm kind of curious: as this evolves, as basically now AI enables any scientist to become their own computational, what do you think the evolution of this co-advising strategy is and also what do you think the evolution of integration between wet lab biologists and computational teams are? I mean, I also can think of I've heard several famous CEOs say things like our strength is integration between computational and wet lab teams. So, I'm curious to see how you think that evolves and how you also set up culture to exploit these new technologies.
让我先把文化部分放一放,因为我觉得这是一个很难回答但非常好的问题。那么,我先谈谈我认为你问到的非常酷的一点,即这些事情的发生与深度整合有关。我认为我们目前面临的最激动人心的前沿之一,也许是一个机会,就是我们现在真的可以开始指导实验了。所以,以前我们构建这些人类细胞图谱时,我们有这种映射的方面。然后我们谈到了世界模型预测问题,对吧?所以,从映射我们走向了预测。但现在我们可以真正地去引导下一个实验,对吧?这一直是科学方法,你知道,把我们的假设放入循环中,然后对它做些什么,但坦率地说,很多时候论文以某个模型结束,对吧?也许是一些概念图,也许有一点验证。但然后你开始下一个项目,对吧?当然你学到了一点,但没有像持续学习那样的东西。但现在,我认为我们可以真正地越来越多地循环回来。这在分子设计或材料设计等领域非常常见。你有点像是在实验室在循环中被解决。尽管一些大型制药公司已经把这作为一个普遍的大主题,但没有很多例子真正人们做多个循环的这种实验、计算模型、然后回去的整合,无论这是现在两个人一个,对吧?但这种整合很难建立。而且很难建立一次,因为建模,因为实际上一旦你有了一个模型,决定什么是最优的下一个问题并不简单。我们实际上推出了一我认为今天或明天应该在 bioRxiv 上,我们称之为 Lab Compass 的新东西,它是一个逆向生成模型,实际上允许你一旦你有一个前向世界模型或其他什么,告诉你需要做什么才能达到某个状态。所以,这对我们来说非常令人兴奋,对吧?我认为工作的很大一部分不仅仅是模型,而是与实验人员的工作,在这个案例中是一位来自剑桥大学的好朋友和同事,他做造血作用。此外,有一个快速循环系统,我们实际上设置了三个四个实验室在循环中,我们迭代地改进模型。我认为这是一件非常困难的事情。这涉及到你的第二个问题,Brandon,对吧?比如你如何带来一种文化,一旦你的计算人员或模型或其他什么预测了某件事,你实际上要说服实验人员投入他们的时间,因为这些实验通常需要一些时间,对吧?真正接受并实际测量一些后续工作。有时你知道如果涉及分子,这效果会好一些,因为那样你知道风险也不那么高,但你知道如果你说现在你需要做这个新细胞,你需要添加这些非常奇怪的东西。你真的想这样做吗?我们是不是不应该凭直觉,然后你知道这些去中介化的解决方案就来了。所以我认为真正建立这个远非易事。这需要文化,可能是同一个人在做。很多人然后开始抛出机器人技术等等,但我认为这有点太早了,因为我们的实验的周期时间仍然有点长,但我确实认为这会到来,你知道显然最终的愿景是我们想要对一切进行强化学习。我们也想在这个循环中进行强化学习。这有点遥远,但你知道我认为我们正在慢慢接近。目前它更多可能是主动学习。
Let me maybe sort of keep the culture part a bit out because I think it's a hard one to answer, but a very good one. So, let me first sort of talk about the I think very cool point that I think you asked about where these things happen sort of goes with deep integration. I think one of the most exciting frontiers we currently face and I think maybe have as an opportunity is we can really now start guiding the experimentation. So, where before we build these human cell atlases, we had sort of this mapping aspect. Then we talked about the world model prediction problem, right? So, from mapping we went to prediction. But now we can go to actually steering the next experiment, right? And this has always been the scientific method, you know, sort of putting our hypothesis in the loop and then doing something with it, but quite frankly, very often the papers ended with some model, right? maybe some sort of concept figure, maybe had a little bit validation. But then to be you start the next project, right? And of course you learn a bit, but there's not like a continual learning type of thing. But now, you know, I think we can actually really loop back more and more. This is happening very much so in molecule design or material design and so on. You sort of the lab in the loop is being worked on out. And even though some big pharma have been putting this out as sort of a general big theme, there's not many examples where really people do multiple loops of this integration of experiment, a computation model, then going back, whether this is now two person one, right? But like this integration is hard to set up. And it's hard to set up once because of modeling because actually sort of once you have a model, actually deciding what's an optimal next question is not trivial. And we actually brought out I think it should be on bio archive today or tomorrow a new thing we call Lab Compass, which is an inverse generative model that actually allows you once you have a forward world model or whatever to tell you what you need to do in order to reach a certain state. So, it's super exciting for us, right? And I think a big part of the work was not just the model, but actually the work with experimental people, in this case a good friend and colleague from University of Cambridge who was doing hematopoiesis. In addition, had a sort of a rapid cycle system where we actually set up three four lab in the loops where we iteratively improved on a model. And this is I think a really hard thing. And this sort of goes into your second question, Brandon, right? Like how do you bring a culture where once your computational person or model or whatever predicts something that you actually get to convince experimental people to put their life time in because these experiments often take some time right to actually buy into it and actually measure some of these follow ups. And sometimes you know this works a bit better if it's sort of molecules because then you know also the stakes are not as high but you know if you like say now you need to do this new cell and you need to add these very strange things. Do you really want to do this? Like shouldn't we maybe not take a bit of our gut feeling and then you know these sort of disintermediate solutions come. So I think really setting this up is far from trivial. That needs culture and it might be the same person doing it. A lot of people then start throwing around robotics and so on as well but I think this is a tad too early because the cycle times in terms of our essays are still a bit on the long side but I do think this will come and you know obviously the ultimate vision being that we want to RL everything. We want to do RL also in this loop. It's a bit out there but you know I think we're slowly getting there. At the moment it's more maybe active learning.
我认为这很像我们与 Lila 的谈话,他们的核心论点是基本上实验室是强化学习环境的一部分。这就是我的意思。是的,那很好。Fabian,现在对你来说已经很晚了。我真的很感谢你抽出晚上的时间来和我们交谈。期待关注你的研究,当我说你时,我指的是你的实验室和同事,以及你。我真的很期待看到这一进展,并密切关注我们今天谈论的事情。
I think that very much like we talked to Lila there that is there they're kind of their core thesis is that basically the lab is part of the RL environment. Kind of is what I mean. Yeah, no that's great. Fabian, it is late at night for you now. I really appreciate you taking the time out of your evening to talk to us. Look forward to watching your research and when I say you I mean your lab and colleagues and you as well. I really look forward to watching this progress and keeping careful track of the things that we talked about today.
是的,非常感谢 RJ Brendan 邀请我。我认为这些联系很重要。这种超越你可能在自己较小的学科中所做的事情,你知道这些讨论对我帮助很大。我真的很享受这次讨论。玩得很开心。谢谢。
Yeah, thanks so much RJ Brendan for having me. I think these links are important. This sort of looking beyond what maybe you do like in your own sort of smaller disciplines and you know these discussions helped me a lot. I really enjoyed the discussion. Had a lot of fun. Thanks.
太好了。谢谢。保重。谢谢。
Great. Thanks. Take care. Thank you.