AlphaFold 为何未真正解决蛋白质折叠：数据、规模化与生物科学的未来

所以，当人们说蛋白质折叠问题已经解决时，在概念层面上，是的，确实有一些进展。但我们不知道蛋白质真正的基态是什么，也不知道蛋白质实际采取的结构分布是什么。我们试图做的基本上是：有人获得了一个结构，将其存入 PDB，而我们试图复制并得到相同的结构。这就是我们所做的，对吧？而它恰好有用。但这并不意味着我们已经理解了所有的蛋白质动力学。

So, when people say the protein folding problem has been solved, at the conceptual level, yes, there have been some advances. But we don't know what the actual true ground state of a protein is, or what the actual distribution of structures that proteins take is. What we are trying to do is basically: someone got a structure, deposited it in the PDB, and we are trying to replicate and get the same structure. That's what we did, right? And it just so happens that it's useful. But that does not mean that we have understood all of protein dynamics.

是的，很高兴来到这里。多么激动人心的早晨，这么多酷炫的发布。我认为生物科学的未来正在这里被宣布。所以，这是建模会议。我们都是建模者，所以当然谈论数据是很自然的。首先，我想思考的一件事是：是否存在 Scaling（规模扩张）？如何正确地扩展数据？这让我想到了关于“苦涩的教训”的问题，但重新以数据为框架。所以，对于那些不是 AI 领域的人，“苦涩的教训”是说可扩展的方法最终会获胜。如果你能扩展得足够多，它就会赢。所以，我的问题是，我先从 Sal 开始：数据是否存在“苦涩的教训”？

Yeah, so great to be here. What an exciting morning, so many cool announcements. I think the future of bioscience is being announced right here. So, this is the modeling session. We are all modelers, so of course it's natural for us to talk about data. To start out with, I think one of the things that I like to think about when it comes to data is: is there a scaling? How do you scale data properly? And this has brought me to this question about the bitter lesson, but recast in the frame of data. So, the bitter lesson for those of you who are not AI people is the statement that methods that scale win eventually. If you can just scale enough, it wins. And so, my question, I'll start with Sal here, is: is there a bitter lesson for data?

是的，当然。我认为你显然需要正确的数据，对吧？为了让它起作用。所以，我认为对缩放定律的一个误解是，缩放定律无处不在，而且总是存在。我认为很多工作实际上是找到那个缩放定律，对吧？所以我们真正做的很多是试图弄清楚在什么情况下，如果你投入更多算力，如果你投入更多数据，你实际上会得到更好的结果。而这是一个很棒的情况，因为一旦发生这种情况，你就可以直接转动曲柄，对吧？就像它变成了一个工程问题，这是我喜欢的。但这与架构有关，也与数据有很大关系。所以如果你没有具有正确信息统计的数据来解决你想要的问题，我认为这很明显，如果你稍微想一想，你就不会得到一个具有你想要的能力和理解的模型。所以你只能从你拥有的数据中提取信息，然后利用它来泛化到更远，对吧？

Yeah, I mean, for sure. I think that you obviously need the right data, right? Like in order for it to work. So, I think like one misconception of scaling laws is that scaling laws are everywhere and they always exist. I think a lot of the work is actually finding that scaling law, right? Like so a lot of what we really do is trying to figure out what's the situation where if you put more compute into it, if you put more data into it, you actually get a better result out. And the reason that's a great situation is then, once that happens, you can kind of just turn the crank, right? Like it becomes an engineering problem, which is something I like. But it has to do with architecture, but it also very much has to do with data. So if you don't have the data with the right information statistics to solve the problem that you want, which is, I think probably pretty obvious, like if you think about it for a second, you're not going to get a model that has the capabilities and the understanding you want. So you can only really pull the information from the data that you have and then use that to generalize beyond, right?

是的。那么关于数据收集，你如何考虑哪些类型的数据模态是最好的？也许一个相关的问题是：你认为建模者倾向于研究数据可用的问题，而不是可能最能解决你的目标的问题？比如可能对转化医学有最大影响的问题。

Yeah. So when it comes to data collection, how do you think about what types of data modalities are best? And maybe a kind of related question is: do you think that modelers have a tendency to work on problems which are the data's available rather than maybe the data that or the problem that solves your goal most? Like maybe has the most impact for translational medicine.

哦，当然。所以我认为，我不会代表所有建模者，但我很懒，所以我会用现有的东西。我认为这实际上有好处也有坏处。好处是，我认为当你做机器学习或传统机器学习时，你真正在寻找的是你能找到的最原始、高质量的数据示例。但如果你像我一样，你实际上是那种去后屋翻找别人的垃圾看看有什么的人。一个具体的例子是，当我们训练蛋白质语言模型时，我们在宏基因组序列上训练，这些不是最高质量的数据。事实上，我可以保证其中很多数据甚至不是真正的完整蛋白质。然而这却提高了模型在设计真正有效的蛋白质、理解我们知道是东西的蛋白质方面的性能。所以，这是积极的一面，但这可能导致你走向消极的一面，你会说：“好吧，让我们只是扩展我们能轻松生成的数据。”我认为这不是必要的方式。我认为这正是今天我们在 BBI 谈论的事情让我真正兴奋的地方之一。它真正去说：“我们需要什么数据来解决问题？”这是正确的资源，也是正确的社区。就像我认为，我们在 Biohub 做或试图做的一件事是公开工作，与社区合作，推动整个社区前进。我认为这是非常关键的一环，因为如果你只有构建模型的人，我们会倾向于现有的数据。如果你只有生成数据的人，他们会倾向于可以生成的东西。但如果你能作为一个社区以开放的方式每一步都合作，那么你实际上可以弄清楚我们需要什么数据，然后你可以找到那个缩放定律。

Oh, for sure. So I think that like I won't speak for all modelers, but like I'm lazy, so I'm going to work with what's available. And I think there is actually a good side and a bad side to this. And the good side is that I think like when you're doing like machine learning or like conventional machine learning, you're really looking for the most pristine, high-quality data examples that you can find. But when you actually if you're like me, you're actually someone who goes and you're kind of like rooting around in the back room looking through people's junk like what's here. So, a concrete example of this is when we train a protein language model, we train on metagenomic sequences, which are not the highest quality data. In fact, much of that data I can guarantee you isn't even a real whole protein. And yet that makes the performance of the model go up for designing real proteins that work, for understanding proteins that we know are things. And so, that's like the positive side of it, but that can lead you to a negative place where you say like, "Well, let's just scale up the data that we can generate easily." And I think that's not the necessary way to do it. I think that's one of the things that's really exciting to me about what we're talking about here today with the BBI. It's really going out and saying like, "What is the data that we need to solve the problem?" It's the right resources, but also the right community. Like I think that, you know, one thing that we do at or we're trying to do at Biohub anyway, is to work in the open, work with the community, and move the whole community forward. And I think that's a really critical piece for this because if you just have people who are building models, we're going to lean towards the data that exists. If you just have people who generate the data, they're going to lean towards the things that can be generated. But if you can work together as a community every step of the way in an open fashion, then you can actually figure out what is the data that we need and then you can find that scaling law.

酷。你怎么看，Sal？数据方面是更好还是更少？

Cool. What do you think, Sal? Better or less for data?

是的。所以我认为——当 Rich 还在我们身边工作时，我在 DeepMind，思考着“苦涩的教训”这个想法。当 Rich——我从 Rich 在 DeepMind 做的原始讲座中得到的东西，不是关于数据是否有用或我们应该如何思考机器学习的实际概念。我认为我的理解是他在谈论更概念化的东西。他谈论的概念性东西是：有时当我们看问题时，我们以非常宗教的方式思考解决方案。我是建模者，或者我是数据生成者。我认为这就是苦涩的教训：如果你以这种心态处理问题，你可能不会成功。问题优先。你应该在解决方案空间上保持灵活。你应该尝试两者。你应该试着理解你要解决的问题是什么。如果它需要建模工作，就做建模工作。如果它需要收集更多数据，就收集数据。但在那些时候机器学习中发生的事情是，人们说我们是机器学习研究者。数据集在那里。这里有一些训练数据。这里有一些测试数据。我们只需优化模型。而这是有问题的。

Yeah. So I think the— So, I was at DeepMind when Rich was with us and working and thinking about this idea of the bitter lesson. When Rich— the thing that I took from Rich's original lecture that he actually gave at DeepMind was not about the actual notion of whether data is useful or how should we think about machine learning. I think my take was that he was talking about something more conceptual. And the conceptual thing that he was talking about is: sometimes when we are looking at problems, we think about solutions in a very religious way. I'm a modeler or I'm a data generation person. And I think that is the bitter lesson: that if you approach the problem with that mindset, you might not succeed. The problem comes first. And you should be flexible in terms of your solution space. You should try both things. You should try to understand what is the problem that you're trying to solve. And if it requires modeling effort, do the modeling effort. And if it requires collecting more data, then collect data. But the thing that happened in machine learning in those times is people said we are machine learning researchers. The dataset is there. Here is some training data. Here is some test data. And we will just optimize the model. And that is broken.

对吧？从某种意义上说，如果你的最终目标是解决这个问题，那你就必须同时审视这个过程中的两个方面。而进入这个过程的不只是数据或建模，还关乎专业知识。我举的例子就是 AlphaFold。我们审视了用现有数据集能做到什么，因为我们当时并没有那种核心专长——顺带说，也没有那种资源——去说“我们把 PDB 扩充一个数量级吧”。那根本做不到——我的意思是，全世界各机构和科学家为构建那个数据集所投入的心血是无价的。所以在那里，你必须专注于把资源投在建模上，让每一分钱发挥最大效用。但在其他领域，比如如果你要处理细胞基因组学，我们就采取同样的思路，问“用 cell by gene 你能做什么？”如果你看细胞基因组学，情况很清楚：做了一堆工作之后，数据还不足以支撑构建虚拟细胞这一宏大愿景。所以，这能把人们聚在一起，聚焦于推进科学时真正的挑战是什么，而不是教条地追随数据生成或建模上的进展。

Right? In the sense that if your eventual goal is to solve the problem, then you have to be able to look at both aspects of what goes into the process. And what goes into the process is not just data or modeling. It's also about expertise. So, the example that I have is like with AlphaFold. We looked at what was possible with current existing data sets because we did not have the core expertise of now sort of going to — or even the resources, by the way — to sort of say, "Let's augment the PDB by a significant order of magnitude." It was just not — I mean, the amount of investment that organizations and scientists across the world had put in to construct that data was invaluable. So, there you have to sort of focus on getting the biggest bang out of your buck by investing it in modeling. But in other areas, if you're approaching, say, cell genomics, then we took the same approach and said, "Well, what can you do with cell by gene?" If you're looking at cell genomics, and there it was very clear that after a bunch of work, the data was not there yet to be able to go after that grand ambition of building the virtual cell. And so, it just brings people together to focus on what are the actual challenges if you want to advance science, rather than being religiously following advances on data generation or in modeling.

太好了，我很喜欢这个回答。那你觉得这里面可操作的要点是什么？我的意思是，永远先定义问题，然后再弄清楚要搜索哪个解空间。但更宽泛地说，在面向下一代、基本上是要解决转化医学的时候，整个社区应该怎么思考这件事？

Cool. I really like that answer. So, what do you think the actionable takeaway is from this? I mean, always define the problem first and then figure out what solution space to search over. But more broadly, how should the community be thinking about this as we go forward for the next generation of basically trying to solve translational medicine?

是的，我给任何在这个领域工作、刚起步的人的建议是：把自己看成一个多学科的人。先理解问题。你为什么做这个问题？我们想实现什么？然后再想需要什么。是建模，是算力 Scaling（规模扩张），还是数据生成。所以，对问题的理解极其重要。当然，然后你必须在内部建立自己的专长。但——而且也不是说你就没有约束。也许有约束。比如你能生成的数据只有特定数量，或者你负担得起的训练规模只有特定大小。理解这些约束，停下来，尽量快速失败。然后看看哪些方法从长期看是可行的，能把你带到你想追求的那个影响力水平。

Yeah, I think the advice that I give to anyone who is working, who is starting in the area is that think of yourself as a multi-disciplinary person. Understand the problem first. Why are you working on this problem? What are we trying to achieve? And then think about what is it that will be needed. Whether it's modeling, whether it will be compute scaling, whether it will be data generation. So, understanding of that problem is extremely important. And of course, then you have to build your expertise in the house. But — and there are also — it's not as if you don't have constraints. Maybe there are constraints. Like there is only a specific amount of data that you can generate. Or there is a specific bottle size that you can afford to train. Understand those constraints and stop and try to fail fast. And look at what are the approaches that are going to be feasible in the long term in getting you to that impact level that you're going after.

好，太好了。这正好引到我的下一个问题：当我看着建模的演进，比如说 AlphaFold 2 本质上是一件艺术品。一堆非常精心手工打造的特征。解决方案的每一部分都经过大量思考、非常刻意的思考。然后我觉得 Google 或 Alphabet 的一些工作，你知道，某种程度上还留在那个空间里，但似乎普遍共识是转向更可扩展、更通用的策略。你觉得是否仍然有很多需要手工技艺式解决方案的问题？还是说我们的资源最好花在——一般来说，固定的资源、固定的钱可以投到算力、人才、数据上。我们应该优先关注规模吗？

Okay, cool. So, this actually leads me right into my next question, which is: when I look at the evolution of modeling, let's say AlphaFold 2 was essentially a work of art. Like a bunch of very carefully hand-crafted features. There was a lot of thought, very intentional thought done to every part of the solution. And then I think some of Google or Alphabet's work has been, you know, kind of still staying in that space, but it seems like the general consensus is to move to more scalable, more general strategies. Do you think that there is still a lot of artisanal craft solutions or problems that need that? Or are our resources better spent on — generally speaking, fixed resources, fixed money can go to compute, it can go to talent, it can go to data. Should we be focusing on scale first?

那么，我的——还是让我们从第一性原理出发。想一想，当你思考如何构建一个更好的模型这门手艺时，我觉得那不是偶然的，对吧？我们做了大量实验，但背后有一个愿景——所有这些来自生物物理和生物化学的科学直觉——即那些相互作用、那些氨基酸残基不是各自为政，它们受到其他残基的影响。所以，让我们把这一点嵌进去。为什么——如果你从科学界学到了东西，就用它。利用那些信息，试着引导模型，给它那种不公平的优势。这基本上让模型的数据效率高得多，因为它不必复制一切。我还必须提到，整理好数据本身就是一门艺术，对吧？不是说你只要说“多放点数据”。如果你只是复制同样数量的数据，你哪儿也去不了。所以，不只是大数据，而是好数据。而真正理解在问题上取得进展需要什么样的数据覆盖，事实上，我会说，本身就是一个更有趣、也更具挑战性的问题。

So, my — again, let's approach it from first principles. And think about when you think about the arts and crafts of how do you construct a model to be better, I think it was not an accidental thing, right? We did a lot of experimentation, but there was a vision behind it — all this scientific intuition that came from biophysics and biochemistry — that those interactions, that amino acid residues are not just doing their own thing, they are being influenced by other residues. So, let's bake that in. Like why — if you have learned something from the scientific community, use it. Use that information and try to encourage the model and give it that unfair advantage that it has. And it basically makes the model much more data efficient because it doesn't have to replicate everything. I also have to mention that curating good data is an art in itself, right? It's not as if you would say, "Well, put more data." If you replicate the same amount of data, you're not going anywhere. So, it's not just about big data, it's about good data. And actually understanding the coverage of what data is necessary for making progress in the problem is, in fact, I would say, a much more interesting and much more challenging problem in itself.

是啊。看起来你有想法，请讲。

Yeah. Looks like you have a thought, so go ahead.

是的，我想——我非常同意。我觉得这真的取决于，正如你所说，要解决的问题。所以，如果你的数据量较小，那在模型里加入更多归纳偏置肯定会有帮助，对吧？而且在较小的数据规模上，你确实需要它来得到结果。随着数据越来越多，你会看到有时模型能发现你未必知道的东西，而有时那种归纳偏置，如果并不完全正确，反而会拖你后腿，对吧？所以随着情况变好，肯定存在那个临界点。但我想我也要稍微反驳一下你的问题，因为我觉得在 Scaling（规模扩张）这件事上其实也有很多手艺。所以，你需要非常——有大量算法工作实际上投入到拿这些模型、用更多数据训练它们、给它们更多算力、让它们更大，对吧？而且我们看到，不仅是从基础设施角度、让模型推理更快，我们实际上看到事情正在超越 Transformer，走向更定制的架构，能够——嗯，我的意思是，我不认为我们在任何意义上进入了后 Transformer 时代，但我们在修改那些架构，对吧？为了让它们更贴合用途，甚至在互联网规模的数据上也能表现得更好。

Yeah, I think — I very much agree. I think it really depends on, as you're saying, the problem to be solved. So, if you have a smaller amount of data, then certainly having more inductive bias in the model is going to help you, right? And you actually need that to get the results on smaller scales of data. As you get more and more data, you see that sometimes the model can find things that you didn't necessarily know about, and sometimes that inductive bias, if it wasn't exactly correct, can hold you back, right? So there is definitely that tipping point as things get better. But I guess I also kind of object to your question too, because I think there actually is a lot of craft to the scaling part of things as well. So, you need very — there's a lot of algorithmic work that actually goes into taking these models and training them on more data, putting more compute in them, making them bigger, right? And we see that not only from an infrastructure perspective and making the models run inference faster, but we're actually seeing that things are going beyond transformers to more bespoke architectures that can — well, I mean, I don't think we're in a post-transformer world in any way, shape, or form, but we're modifying those architectures, right? In order to make them more fit for purpose and actually work better even at internet scale data.

所以，我觉得这就是那个项目，对吧？随着越来越多的数据进来，它不仅在帮助整理这些数据、挑选我们真正需要的下一批信息。这更多是信息而非数据。你如何获得模型所需的下一批信息？而且在每一步、每一个规模上，什么才是能最大限度利用这些数据的正确架构？

So, I think that's kind of the project, right? As more and more data comes in, it's not only helping to curate that data and select what is the next batch of information that we actually need. It's more information than data. How do you get that next batch of information that the models need? But then also at every step of the way and every scale, what is the right architecture to get the most out of that data?

好。那么如果你想想蛋白质结构预测现在的状况，新闻会说蛋白质结构预测已经被解决了。但如果你和我的朋友们聊，他们都会说：“嗯，这里还有太多事情要做。”功能、动力学，还有设计，这些都还是完全开放的问题。AlphaFold 2 发布已经大约 5 年了。进展是有的，但我很好奇，Pushmeet，你怎么看——这里还会有另一次大跃迁吗？解决这些问题有阻碍吗？是我们还没有合适的数据？还是还没有合适的算法或想法？还是说这是我们在未来能看到逐步演化的东西，只是时间问题？

Cool. So if you think about the state of protein structure prediction right now, the news will say protein structure prediction has been solved. But then if you talk to my friends, they will all say, "Well, you have so much left to do here." So many things like function, dynamics, and then design are still just wide-open problems. It's been 5-ish years since AlphaFold 2 was announced. Progress has been made, but I'm curious, Pushmeet, what do you think — is there another big leap being made here? Are there blockers in solving these problems? Do we not have the right data yet? Do we not have the right algorithms or ideas? Or is this something that we can see evolving in the future, and it's just a matter of time?

是的，我觉得科学的运作方式，基本上就是先把某个东西隔离出来，然后一步步取得进展。所以当人们说蛋白质折叠问题已经被解决了——在概念层面上，是的，确实有一些进展。但我觉得这太——而且你想想“蛋白质是构建块”这个说法。蛋白质不是积木，它们也不像积木那样运作，对吧？我总说蛋白质是构建块，但其实我并不相信这个说法，对吧？蛋白质极其复杂。它们是无序的。它们的形状可能随环境而改变。而 AlphaFold 做的是——事实上，John Jumper 和我以前就经常讨论这个。我们在试图解决什么？我们并不知道蛋白质真正的基态是什么，也不知道蛋白质实际采取的结构分布是什么。我们做的其实是：有人得到了一个结构，把它存进了 PDB，我们试图复现并得到同样的结构。这就是我们做的事，对吧？而它恰好很有用。但这并不意味着我们已经理解了蛋白质的全部动力学。所以在最高层面上，说我们取得了进展更容易传达，但我们当中的科学家、这个人群里的科学家，我们知道实情。

Yeah, I think science, how it operates, is basically by isolating something and then making progress step by step. So when people say the protein folding problem has been solved — at a conceptual level, yes, there have been some advances. But I think it's too — it's also like, when you think about that narrative of proteins being the building blocks. Proteins are not blocks, and they don't act as blocks, right? I say proteins are the building blocks all the time, but actually I don't believe in it, right? Proteins are extremely complex. They are disordered. Their shape might change depending on the context. And what AlphaFold did — in fact, John Jumper and I basically used to discuss this. Like, what are we trying to solve? We don't know what is the actual true ground state of a protein, and what is the actual distribution of structures that proteins take. What we are trying to do is basically: someone got a structure, deposited it in the PDB, and we are trying to replicate and get the same structure. That's what we did, right? And it just so happens that it's useful. But that does not mean that we have understood all of protein dynamics. So at the top level, it's easier to communicate that we have made progress, but the scientists among us and in this crowd, we know.

对吧？我觉得这对我们来说非常重要，要理解并达成这个共识——还有很多工作要做。我们确实取得了进展，有很多值得庆祝的，但别停止对蛋白质结构预测和蛋白质动力学的资助，因为我们才刚刚开始。

Right? And I think that's the really important thing for us to understand and have that common ground — that a lot of work needs to be done. And we have made progress, and there's a lot to celebrate, but let's not stop the funding of protein structure prediction and protein dynamics because we are just getting started.

那你觉得最大的阻碍是什么——如果有一件事你可以挥一挥魔杖说“我们有更多某某东西”，就能加速功能、动力学或设计中的某一项？那根魔杖会是什么？你会挥出什么来？

So what do you think the biggest blocker is — if there's one thing you could wave a magic wand and say, "We have more of blank," that would accelerate one of those things: function, dynamics, or design? What would the magic wand be? What would you wave into existence?

哦，我——我的背景相当杂。我一开始是安全研究员，然后进入计算机视觉、贝叶斯理论，最后进入判别式机器学习和深度学习等等，然后是用于编程的 AI，最后是科学。所以你问我这个问题时，我内心那个计算机视觉研究者对冷冻电镜显微图像超级兴奋。我当时想：“这 PDB 数据是什么？我应该从源头做起，对吧？我应该看冷冻电镜显微图像。我不想要那些结构。它们肯定漏掉了所有数据。我应该直接在冷冻电镜显微图像上操作。”现在，能在这个层面上扩展、拥有合适的数据量、并能提取其中捕捉到的所有动力学和分布信息的模型，会是非常了不起的东西。我试过，但这需要更多工作，对吧？

Oh, I — my background is quite eclectic. I started as a security researcher, then went into computer vision, Bayesian theory, and finally got into discriminative machine learning and deep learning and so on, and then AI for coding, and then finally science. So when you ask me that question, the computer vision researcher in me was super excited about cryo-EM micrographs. I was like, "What is this PDB data? I should be working at the source, right? I should be looking at the cryo-EM micrographs. I don't want those structures. They must be missing out all the data. I should just directly operate at cryo-EM micrographs." Now, getting models that can scale at that level with the right amount of data and can extract all the dynamics and distributional information that is captured there would be an amazing thing. I tried it, but it requires more work, right?

那就去做吧，但你相信这是一条路。这能给你——

So work on it, but you believe that this is a route. This can give you —

是的。我觉得在某个时候，也许有比我更厉害的人会去尝试，我们会有所进展。

Yeah. I think at some point in time, maybe some people better than me would take a stab at it and we'll get somewhere.

你怎么看，Sal？

What do you think, Sal?

是的，我觉得这非常有意思，因为我认为这些模型——我是说，它们相当有用，但它们并不完全是大多数人想解决的问题。它们解决的是一个非常特定的目的，然后你也可以用它们做别的事——你可以用它们设计新的蛋白质，对吧？比如，这并不是你一开始会做的事。所以它们非常有用，但我有点把我们目前构建的模型想成——你知道，有人想理解自行车是怎么运作的，而你在给它的一根辐条建模，对吧？这些模型可以随着时间越来越好，但你真正需要做的很多事，是从辐条的模型走向车轮，也许再走向整辆自行车，因为那才是人们真正想理解的。其实，顺着这个类比说下去，似乎人们想用自行车的模型去设计皮卡的某个零件之类的，对吧？但我认为，当我们把这些模型更多地放进它们所处的具体情境中，我们才能真正在更广的意义上了解这些相互作用，对吧？我觉得那才是事情要去的方向。但我同意你。我觉得我们应该继续努力做折叠模型，因为我认为它们会越来越好。

Yeah, I think it's really interesting because I think these models are — I mean, they're quite useful, but they're not exactly the problem that most people want to solve. They solve a very specific purpose, and then you can also use them to do other things — you can use them to design new proteins, right? For example, which is not the thing that you would kind of start with there. So they're very useful, but I kind of think of the models that we build right now as — you know, you've got someone who's trying to understand how a bicycle works, and you're modeling a spoke on it, right? And those models can get better and better over time, but a lot of what you really need to do is you need to move from models of spokes to wheels to maybe whole bicycles, because that's actually what people want to understand. Actually, I think to continue the analogy, it seems like people want to use the model of the bicycle to design the part for a pickup truck or something like that, right? But I think that as we put these models more into the particular contexts which they're in, then we'll actually be able to learn more about these interactions in a broader sense, right? And that's really where I think things are going to be going. But I agree with you. I think we should keep trying to work on folding models because I think they're going to keep getting better and better.

那么关于设计，有一句著名的费曼名言：我无法创造之物，我便无法理解。现在我们身处一个很容易在完全不理解的情况下创造东西的世界。所以我很好奇你的感受和想法：拥有能帮助人类理解事物的模型，相对于能一口气给你一个皮摩尔级结合物之类的神奇黑箱，这到底有多重要？

So with regards to design, there's this famous Feynman quote: that which I cannot create, I do not understand. Now we're in a world where it's really easy to create things without understanding them at all. So I'm curious what your feelings, your thoughts are about how important it is to actually have models which can help humans understand things versus magical black boxes which can effectively one-shot you a picomolar binder or something like that.

我的意思是，我认为在 AI 出现之前很久，我们就在用神奇的黑箱来设计东西了，对吧？所以从某种意义上说，这仍然有用。但我认为理解真的很重要，这实际上是我在 AI 方面经常思考的一大问题，因为这些模型有太多东西要学。随着智能变得越来越便宜、越来越容易获得，你可以部署它去学习世界上越来越多的事情。但然后你如何真正把那些知识从机器里提取出来，这么说吧？所以这是我能理解的事情——也许那只是我深奥的好奇心。我认为有太多东西要学。

I mean, I think that we were designing things with magical black boxes long before AI came around, right? So in some sense, that's still useful. But I think that the understanding is really important, and that's actually one of the big things I think about a lot with AI, because there's so much for these models to learn. As intelligence is getting cheaper and cheaper and more available, you can deploy it to learn more and more things about what's going on in the world. But then how do you actually pull that knowledge out of the machine, so to speak? So it's something that I can understand—maybe that's just my esoteric curiosity. I think there's so many things to learn.

我认为很多科学家真的很想理解事物，终点也许没那么重要。但然后，你知道，我们在这里是为了解决转化医学这个问题的，对吧？

I think lots of scientists really want to understand things, and the end points are maybe not as important. But then, you know, we are here to solve translational medicine as a problem, right?

是的。

Yeah.

或者，我的意思是，我认为你越深入挖掘，就越能找到正确的方法来推动它们前进。所以，我认为关于这一点，一个非常突出的事情是，我认为模型实际上包含的信息比我们知道的多得多。所以，例如，我们在模型的可解释性方面做了很多工作。你会在那里发现很多信息，我认为人们知道蛋白质语言模型在其表示中学习到了某种结构概念。但我们发现了关于一些功能的信息。我们发现了关于运动的信息，对吧？所以，我认为即使是我们现在的模型，也还有很多东西有待解锁。而且我认为，随着我们提高能力，理解这些模型如何工作对我们来说很重要。归根结底，你期望通过试图将所有信息压缩到模型中来产生一个世界模型，对吧？模型如何完成它的工作？它如何设计蛋白质？嗯，它把进化中的所有信息压缩到了这个模型中。所以，除了它能够输出对我们有用的东西之外，仅仅通过观察内部就肯定有东西要学。

Or, I mean, I think that the more you dig into things, the more that you can find the right way to keep pushing them forward. So, I think one thing that's really salient to me about this is I think the models actually have a lot more information in them than we know. So, we've worked a lot on interpretability, for example, for our models. And you find a lot of information in there, which is—I think people know that protein language models learn some notion of structure within their representations. But we find information about some functions. We find information about motions, right? And so, I think there's a lot in there still to be unlocked even from the models that we have now. And I think that's important for us to understand how these models work as we raise the capability. At the end of the day, you expect a world model to come out of trying to compress all this information into the model, right? How does the model do its job? How does it design a protein? Well, it's compressed all the information from evolution into this model. So, in addition to it being able to spit out something which is useful to us, there's certainly to learn just by looking at what's inside.

是的。

Yeah.

你怎么看，Pushmeet？设计还是理解？

What do you think, Pushmeet? Design versus understand?

所以，我对此有不同的看法，我认为当我们思考模型时，某种程度的理解是必要的。那么，我会说什么程度的理解是必要的。AlphaFold 实际上并不完美。对吧？它并不完美。它是 90 GDT。当时 AlphaFold 2 在那个集合上是 90 GDT。但即使它是 95 GDT，如果 PLDDT 分数完全没有校准，谁会信任它？它只会神奇地给出好答案，但突然告诉你这是答案，非常非常自信，然后你会在接下来的一年里研究它，发现它完全错了。不确定性度量的校准极其重要。所以，从这个意义上说，我们确实理解，而且我们在理解 AlphaFold 2 的行为方面确实取得了很大进展。现在，这与它如何工作并找到解决方案是不同的。在这一点上，我同意 Al 的观点，即存在可解释性的概念。它为什么工作？它为什么给出这个答案？我认为如果你想一想，AlphaFold 2 在最高层次上是可解释的，因为如果你看看它如何——这是泛化的稳定性——我们在发布之前并没有发现它。当我们发布 AlphaFold 2 并提供权重时，人们发现它是一个很好的无序蛋白质预测器。它可以弄清楚蛋白质的哪些元素是无序的。所以，这只是表明它具有泛化能力。现在，可解释性提出了一个问题：对谁可解释？如果你说的是对人类理性系统可解释，而人类大脑有认知和计算限制，那么不，AlphaFold 2 是不可解释的。但如果你问的是，AlphaFold 2 对一个更大、更复杂的模型来说，关于它如何工作是否可解释？也许它是可解释的。我们只是不理解。但我认为作为 AlphaFold 2 的用户，我们确实需要理解它能做什么和不能做什么。所以，理解行为特征以及模型的优势和局限性——我认为这极其重要，我们不能在没有行为特征的情况下使用这些模型，否则它不但不会有所帮助，反而会伤害我们。

So I have a different take on this in the sense that I think when we think about models, some level of understanding is necessary. So, I'll say what level of understanding is necessary. AlphaFold actually is not perfect. Right? It's not perfect. It's 90 GDT. AlphaFold 2 was 90 GDT at that time on that set. But even if it was 95 GDT, but the PLDDT score was completely uncalibrated, who would trust it? It would just magically give good answers, but suddenly tell you here's the answer, very very confident, and you'll be working on it for the next one year and finding out it was completely wrong. The calibration of the uncertainty measure was extremely important. So, in that sense, we do understand, and we actually made a lot of progress in understanding how AlphaFold 2 behaves. Now, that's different from how did it work and find the solution. And there I agree with Al that there's a notion of interpretability. Why did it work? Why did it give this answer? And I think if you think about it, AlphaFold 2 was at the highest level interpretable because if you look at how—and this is the stability to generalize—and we did not discover it before launching. When we launched AlphaFold 2 and made the weights available, people found out that it was a great disordered protein predictor. It could figure out which elements of the protein are disordered. So, it just shows that it generalizes. Now, interpretability asks the question, interpretable by whom? If you're saying interpretable by a human rational system with the cognitive and computational limitations of the human brain, then no, AlphaFold 2 is not interpretable. But if you are asking, is AlphaFold 2 interpretable to a much larger and more sophisticated model as to how it works? Maybe it is. We just don't get it. But I think as users of AlphaFold 2, we do need to understand what it is able to do and what it is not able to do. So, understanding the behavioral characterization and the strengths and limitations of the model—I think that is extremely important, and we can't just be using these models without having that behavioral characterization, because otherwise it will, rather than being helpful, harm us.

所以你的观点是，严格来说，可解释性并不是必要的，但确保模型值得信赖、以便人类能够做出可操作决策的方法，才是人们应该关注的。

So your take is that interpretability, strictly speaking, isn't necessary, but a way of ensuring the model is trustworthy so humans can make actionable decisions is the thing that people should be focusing on.

正是如此。我还认为，可解释性也取决于观察者。比如谁在解释它？如果是一个人类科学家试图解释模型是如何进行的以及它将如何做出这个预测，那与另一个大得多的 LLM 是不同的——如果你让未来的一些大型前沿模型访问激活层，并说‘好吧，告诉我，你能预测 AlphaFold 会做什么吗？’也许它们能够预测，并且它们会提出一个关于 AlphaFold 2 实际上如何解释和产生这些结果的理论。

Exactly. And I also think that interpretability is also in the eyes of the beholder. Like who is interpreting it? If it is a human scientist who is trying to interpret how the model is going about and how will it make this prediction, that's a different question from another much larger LLM—if you give some of these large frontier models of the future access to the activation layers and say, 'Okay, tell me, can you predict what AlphaFold will do?' Maybe they will be able to predict, and they will come up with a theory of how actually AlphaFold 2 was interpreting and producing these results.

好的，这是两分钟警告。所以，我最后一个问题问你们两位，很快。所以，AI 在不久的将来极有可能对人类健康产生巨大改善。我喜欢定量预测。最好的猜测，多久我们开始看到 AI 结果出现在诊所？

Okay, that's the 2-minute warning. So, I have one last question for both of you, quick one. So, there's a real chance that AI will make dramatic improvements in human health in the immediate future. I like quantitative predictions. Best guess, how long until we start seeing AI results in the clinic?

Prithvi，你想先开始吗？

Prithvi, you want to start?

我的意思是，关键在于‘AI 结果出现在诊所’这个说法本身就有问题。并不是——AI 今天已经被用于药物发现过程的每一个环节。所以，从这个角度来看，它已经在那里了，对吧？但如果你问我，我们什么时候会看到时间线加速 10 倍或 100 倍，那么另一个问题是，我们在加速什么？是在做先导化合物优化吗？是靶点发现吗？是临床前或毒理学工作吗？所以，我认为在未来几年——这就是为什么我们今天宣布的这项努力极其重要——我们需要解决生物学中的一些难题。

I mean, the point is AI results in the clinic—it's ill-posed. It's not—AI is being used today already in every part of the drug discovery process. So, from that perspective, it's already there, right? But if you are asking me the question, when will we see that 10x acceleration or 100x acceleration in the timelines, then the other question is, what are we accelerating? Is it doing lead optimization? Is it the target discovery? Is it the preclinical or the toxicology work? So, I think over the next few years—and which is why this effort that we announced today is extremely important—we need to tackle some of these hard challenges of biology.

只有到那时，我们才能真正解锁那种加速，从根本上改变药物发现的进程。所以 AI 会一直在那里，比如在临床中，也会一直影响进入临床的各个环节，但真正更大规模的加速，只有在对生物学和生物模型有更好理解之后才能解锁，而这正是这项努力试图创造的东西。

And only then will we be able to get these true unlocks of acceleration that dramatically transform how drug discovery will happen. So AI will be there, like in the clinic, and will be impacting the things that go into the clinic all the time, but the actual larger acceleration will only be unlocked with a better understanding of the biology and the biological models that this effort is trying to create.

好，谢谢。那么快速问一个问题，如果我们——我们时间快到了，但你——

All right, thanks. So, real quick question, if we — we're almost out of time, but you—

是的，我有点想直接戴上我的生物学帽子来回答这个问题。但我觉得认真思考这件事非常重要。就像你说的，推动这个领域向前到底意味着什么？我们真正想看到的是那些结果。那么，什么时候会出现一种完全由 AI 制造的药物？我不知道，这很难说。但我确实认为我们会非常非常快地看到快速进展，因为所有这些工具都在被使用，等等。回到你关于基础的观点——我们需要基础研究，我们需要基础理解。我在职业生涯中很久以前学到的一件事，其实是在 Google 的时候，就是有时候通过思考“要做到 10 倍提升需要什么”来接近一个问题，会比思考 10% 的提升更容易。这并不是因为那条路一定更容易，而是因为它让你能采取更广阔的视角，看到一些你之前没有接近过的解决方案。真正的路径是什么——你会回到第一性原理去看。我们要怎么做？我们要如何真正推动这件事？所以我认为两种方法，10% 和 10 倍，都是有价值的，我们应该两者都做。我只是很高兴在 Biohub，我们有一个非常美好而崇高的使命宣言：治愈所有疾病。如果你想做到这一点，你真的需要去弄清楚 10 倍的方法是什么。所以我认为能够经历这个过程是一个非常好的机会。

Yeah, I mean, I'm tempted to literally put on my bio hat to answer this question. But I think it's really important to think about this. As you were saying, what does it mean to push the field forward? The thing that we really want to see is those outcomes. So when will it be that a drug was made entirely by AI, by an AI or something? I don't know. It's hard to know that. But I do think we're going to see rapid progress here very, very quickly, because all these tools are being used and stuff like that. And going back to your point about the basics — we need basic research, we need basic understanding. One thing I learned a long time ago in my career, actually back at Google, is that sometimes it's easier to approach a problem by looking at what it's going to take to make a 10x improvement rather than a 10% improvement, right? And that's not because that's necessarily an easier path. It's because it allows you to take a broader view and see some solutions that you haven't been approaching, right? And what is actually the path to do it — you would go back and look from first principles. How are we going to do this? How are we going to really push this? So I think both approaches, the 10% and the 10x, are valuable and we should do both. I'm just very happy that at Biohub we have a very beautiful and lofty mission statement to cure all disease, right? And if you want to do that, you really need to go and figure out what is the 10x approach. So I think it's a really great opportunity to be able to go through that.

太棒了。谢谢你们两位。谢谢你们坐在真正的“热座”上。

Awesome. Thank you both. Thank you for being in the literal hot seat.

非常感谢——

Thank you very—