AlphaFold：AI 在蛋白质折叠领域的诺贝尔奖级突破

我不太喜欢人们试图应用的那条苦涩教训。实际上，AlphaFold 2 恰恰相反。

I don't really love the bitter lesson as people try and apply it. In fact, AlphaFold 2 is the opposite of that.

蛋白质折叠是生物学中这类圣杯式的问题之一。

Protein folding is one of these holy grail type problems in biology.

我们只需按下一个按钮，就能在非常狭窄的自然科学类别中预测特定蛋白质的结构。

We predict nature level science with the press of a button in a very narrow category of nature level science of the structure of a specific protein.

John Jumper 领导了 AlphaFold 团队，该系统预测了 2 亿个蛋白质结构。2024 年，他获得了诺贝尔化学奖。现在，Jumper 即将离开 DeepMind。但 AlphaFold 解决了什么？还有什么未解决？AlphaFold 能否成为 AI 用于科学的模板？

John Jumper led the team behind AlphaFold, the system that predicted 200 million protein structures. In 2024, he won the Nobel Prize for chemistry. And now, Jumper is leaving DeepMind. But what did AlphaFold solve? What remains unsolved? And could AlphaFold be the template for AI for science?

我们并不试图告诉你一切。我们不是整个细胞的模型。你尝试，你测量。十有八九你会发现自己错了，对吧？如果你十次错九次，你是一个非常成功的机器学习者。你非常有生产力。

We are not trying to tell you everything. We are not a model of the entire cell. You try it, you measure. Nine times out of 10, you find out you're wrong, right? If you're wrong nine times out of 10, you're a very successful machine learner. You're incredibly productive.

半个世纪以来，结构生物学一直存在巨大瓶颈。DNA 容易读取，但蛋白质结构并非如此。蛋白质结构始于一条氨基酸链，然后通常在细胞的帮助下，它们形成三维形状。这种形状决定了它结合什么、催化什么化学反应、在细胞中的位置，甚至是否起作用。但从机器学习的角度来看，如果你只有序列，你能预测折叠吗？你能预测结构吗？

So, for half a century, structural biology had a massive bottleneck. DNA was easy to read, but protein structures were not. A protein structure begins as a chain of amino acids. And then, often with help from the cell, they settle into a three-dimensional shape. And that shape determines what it binds, what chemistry it catalyzes, where it sits in the cell, and whether it even works at all. But from a machine learning perspective, if you only have the sequence, can you predict the fold? Can you predict the structure?

我们比之前的任何文明都更多地发现了世界。但我们一直卡在这个问题上：蛋白质是如何折叠的？

We've discovered more about the world than any other civilization before us. But we have been stuck on this one problem. How do proteins fold up?

每两年有一个大型科学实验叫 CASP。基本上，来自世界各地的团队聚集在一起，看看他们能否根据最近完成但尚未公开的实验来预测蛋白质结构。几十年来进展缓慢，直到 2020 年 John Jumper 的团队 AlphaFold 取得了显著优于竞争对手的结果。对于许多单链目标，AlphaFold 的预测与目标非常接近，以至于活动组织者说这个问题基本上已经解决了。因此，一个可能需要专家一年工作的蛋白质结构现在可以在几分钟内预测并投入应用。

So, every 2 years, there's a big scientific experiment called CASP. Essentially, teams from around the world gather to see if they can predict protein structure from sequences based on recently done but not yet publicly available experiments. So, for many decades the progress was incremental until 2020 when John Jumper's team, AlphaFold, they produced a result which was significantly better than the competition. For many single chain targets, the predictions from AlphaFold were so close to the targets that the organizers of the event said that the problem had been essentially solved. So, a protein structure that might have taken a year of specialist work can now be predicted and operationalized in minutes.

大家干得好，整个团队。这是令人难以置信的努力。

Good job everyone, the whole team. It's been incredible effort.

祝贺这项工作。它确实非常出色。

Congratulations on this work. It is really outstanding.

AlphaFold 代表了巨大的飞跃，我希望它能真正加速药物发现，帮助我们更好地理解疾病。这太令人兴奋了。

AlphaFold represents a huge leap forward that I hope will really accelerate drug discovery and help us to better understand disease. It's so mind-blowing.

你知道，这些结果对我来说，在这个问题上工作了这么久，经历了无数次停顿和启动，怀疑是否会有结果，突然之间这就是一个解决方案。我们解决了这个问题。

You know, these results were for me, having worked on this problem so long, after many, many stops and starts and will this ever get there, suddenly this is a solution. We solved the problem.

而且 DeepMind 做得很好，他们本可以保密，但他们决定发布。他们发布了一个包含超过 2 亿个预测蛋白质结构的数据库。现在，重要的是强调“预测”这个词。这些不是实验，但它们是生物学界现在可以进行的许多有趣新搜索工作的基础。如今，AlphaFold 已被 190 多个国家的超过 300 万人使用。因此，在 2024 年，诺贝尔奖委员会做出了正式裁决。化学奖的一半授予 David Baker，以表彰计算蛋白质设计，另一半授予 Demis Hassabis 和 John Jumper，以表彰蛋白质结构预测。因此，AI 已成为化学家的新工具，一种以结构生物学家以前无法想象的分辨率观察分子结构的方式。

And fair play to DeepMind, so they could have kept this close to their chest, but they decided to release it. They released a database with over 200 million predicted protein structures. Now, it's important to emphasize the word predicted. These are not experiments, but these are the basis for a lot of interesting new search work that the world of biology can now perform. And today, AlphaFold is now used by more than 3 million people in over 190 countries. So, in 2024, the Nobel Prize Committee made a formal verdict. Half of the chemistry prize went to David Baker for computational protein design, and the other half went to Demis Hassabis and John Jumper for protein structure prediction. So, AI had become a new tool for chemists, a way of seeing molecular structure at a level of resolution that structural biologists couldn't even have dreamed of before.

所以，能来到这里真是太好了。能向你们介绍这项工作，介绍我们团队在蛋白质结构预测方面的工作，真是莫大的荣幸。大约 10:30 时，我说：‘哦，好吧，我想今年不是了。’我告诉了我的妻子，她说：‘不，不，等等。’就在她让我等的时候，我的手机亮了，接到一个来自瑞典的电话。谢天谢地，那不是世界上最恶毒的恶作剧电话。

So, it's absolutely wonderful to be here. It's truly an extraordinary honor to tell you about this work, to tell you about the work of our team in protein structure prediction. So, by about 10:30, I said, 'Oh, well, I guess not this year.' And I told my wife, and she goes, 'No, no, wait.' And as she's telling me to wait, my phone lights up with a phone call from Sweden. And thankfully, it was not the world's meanest prank call.

所有这些都让 John 的下一步行动非常有趣。就在几天前，他宣布离开谷歌，前往 Anthropic。值得注意的是，John Jumper 并不是在构建像 Claude 或 Gemini 那样的通用预测架构。它是高度结构化、设计和工程化的，目的是做一件特定的事情。为什么这对 Anthropic 有吸引力？我们只能猜测。所以，这不仅仅是关于 Google DeepMind 和 CASP 竞赛等等。现在世界各地的结构生物学家都可以创新、开发新产品并可能拯救生命，因为他们可以访问这个蛋白质数据库。我与 Emmanuel Nee 进行了交谈。他是一位驻非洲的结构生物学家。

And all of this makes John's next move very interesting. So, just a few days ago, he announced his departure at Google, and he's going to Anthropic. It's important to note that John Jumper was not building generic prediction architectures like Claude or like Gemini. It was extremely structured, designed, and engineered for the purpose of doing a specific thing. Why might that be interesting to Anthropic? We can only guess. So, this isn't just about Google DeepMind and the CASP competition and whatnot. There are now structural biologists around the world that can innovate and build new products and save lives potentially because they have access to this protein database. I spoke with Emmanuel Nee. He is a structural biologist based in Africa.

我回去只做了一次蛋白质纯化，收集了数据，然后我们用了 AlphaFold。结合起来，我在不到两三个月的时间里得到了结构。

I went back and did just one protein purification, collected the data, and we AlphaFold. In combination, I got the structure in less than two, three months.

所以，是的，我们说的是可能数年的工作压缩到几个月。这相当不错。智能体每天都在变得更聪明，但即使是最好的智能体，如果没有良好的上下文也会卡住，这就是 Notion 的用武之地。随着最近自定义智能体的推出，Notion 成为了智能体与人类并肩工作的协作 AI 平台。现在他们新的开发者平台正在将其转变为开发者可以使用的基础设施。我认为它是我所有工作的精心策划的具体化记忆平面。Notion 的优点是它很容易以编程方式使用。它有 CLI，有 MCP 服务器，内置了智能体，对吧？这意味着用我的手机，我可以让智能体去做一些研究或把一些信息放进去。我可以将事情同步到日历。老实说，没有 Notion，我认为我无法做任何我在 MLST 上做的事情，所以它真的非常非常好。所以我强烈建议你试试 Notion 的平台。你可以在 notion.com/mlst 注册，如果你这样做，你就是在支持这个节目。现在，让我们回到 John Jumper。我们是在 Anthropic 宣布之前拍摄的，所以能和 John 交谈真的很酷。他是一个非常鼓舞人心的人。我很幸运前一天晚上和他共进晚餐，所以我们进行了一些热身对话，然后深入探讨了我们想讨论的各种话题。让我印象深刻的一件事是，John 对于 AlphaFold 能解决什么和不能解决什么异常谨慎。所以，他并不把它推销为生命的模型或治愈疾病的模型。他把它推销为更狭窄的东西，实际上可能更激进。

So, yeah, we're talking about potentially years of work compressed into several months. That's pretty good. Agents are getting smarter every day, but even the best agents get stuck without good context, and this is where Notion comes in. With the recent launch of custom agents, Notion becomes the collaborative AI platform where agents and humans work side by side. And now their new developer platform is turning that into infrastructure that developers can work on. The way I think about it is it is the kind of curated materialized memory plane for all of my work. And the great thing about Notion is it's so easy to work with programmatically. It has a CLI, it has an MCP server, it has agents built into it, right? So, that means using my phone, I can ask an agent to go and do some research or to put some information in there. I can sync things to my calendar. Without Notion, I honestly don't think I'd be able to do anything that I do on MLST, so it's really, really good. So, I highly recommend you give Notion's platform a go. You can sign up at notion.com/mlst, and you'll be supporting the show if you do. And now, let's get back to John Jumper. Now, we filmed this before the Anthropic announcement, so it was really cool to speak with John. He's such an inspirational guy. I was lucky enough to have dinner with him the night before, so we had a bit of a warm-up conversation, and we drilled into the various topics we wanted to discuss. One thing that struck me is John is unusually careful about what AlphaFold does and does not solve. So, he doesn't sell it as a model of life or as a model of curing disease. He sells it as something a little bit more narrow, and possibly more radical, actually.

一台机器，能够预测某类结构生物学测量，其准确度足以改变科学家下一步能做的事情。那么，现在有请约翰·詹珀。

A machine that predicts one class of structural biology measurement well enough to change what scientists can do next. So, now I give you John Jumper.

我的意思是，AlphaFold 本身可以说是 AI 和科学领域的一个里程碑，但它真正的意义在于我们如何用 AI 解决人类无法解决的问题——那些极其困难、需要多年实验的问题。就 AlphaFold 而言，就是蛋白质结构预测这个难题。这更像是机器学习领域的行话，而不是生物学行话。你知道，DNA 是生命的说明书，但它到底告诉你要构建什么？它告诉你的众多事情之一就是如何构建蛋白质。这些是微小的纳米机器，由细胞中几千个原子组成，实际上执行着细胞的工作。DNA 的三个字母告诉你如何向这个蛋白质添加一个额外的片段。蛋白质就像一根长绳，有一个由蛋白质和 RNA 组成的微小机器一次构建一根这样的绳子。你制造这根由 20 种化学基团组成的绳子，就像 20 种字母，人们当然用字母表来称呼它们。每一种都相当不同，对吧？我的博士导师可以满怀爱意地告诉你每一种的特殊之处。但当你构建好这根蛋白质绳子后，它会自行组装。它扭曲、卷曲、折叠成一个非常紧凑而有趣的形状。它有螺旋、片层等等。这才是真正起作用的。我总喜欢用的一个类比是：你有一个宜家书架，打开盒子它自己就组装好了。所以，这个在生物学中可能持续了 70 多年的核心问题是：好吧，我能读取 DNA？事实上，我现在能很好地读取 DNA。你知道，你可能——你的许多听众可能已经测序过自己的 DNA。但理解哪怕一个蛋白质的结构都极其困难，对吧？那是一个值得的博士项目。我估计典型的时间框架是一年。如果要用金钱衡量，大概要花 10 万美元才能得到一个答案。这对生物学非常重要，因为我们想了解这些蛋白质如何工作。当它们错误折叠时，有时会导致疾病。即使它们正常工作，它们也是细胞的组成部分。它们完成细胞的所有功能，你知道，它们是美丽的蛋白质。细胞之所以能运动，是因为这个巨大的蛋白质机器在旋转，驱动细胞运动的力。细胞的所有功能基本上都体现在这些蛋白质中。人类基因组中大约有 20,000 种不同类型的蛋白质，分布在不同的位置。科学家们所做的是使用巨大的机器——通常是同步加速器，大小相当于小镇——来产生极其明亮的 X 射线。即便如此，他们也要先进行极其困难的实验，试图弄清楚如何让蛋白质结晶。这需要很多年。一旦他们做到了，再解决另一个数学问题（我们可能会讨论，也可能不会），他们就能得到一张蛋白质的图片。然后他们会有这种进展，通常会有丰富的理解：“哦，好吧，我能理解在人群中发现的这个 DNA 变化如何影响帕金森病，因为看，它就在这个蛋白质上，这就说得通了。”所以，人们研究这个问题已经很长时间了。几乎有数不清的诺贝尔奖颁发给了单个蛋白质，比如核糖体等等。社会投入了巨大的资源，收集了大约 20 万个这样的结构，在我们做 AlphaFold 的时候大约是 14 万个。每一个仍然极其困难，对吧？每一个仍然——我记得看到人们在博士答辩时谈论他们接近博士毕业时的进展，关于结晶某个蛋白质的进展。对吧？所以，我做了我的博士，我要成为博士了，但我可能无法结晶这个蛋白质。我想我一直在讲蛋白质，没讲我们做了什么，但我们所做的是利用公开的实验数据开发了一个新的深度学习系统。所有数据都是公开的，它在预测蛋白质结构方面要准确得多。它的预测精度达到了原子半径级别，在典型精度下。而且它的精度开始与至少某些实验方法相媲美，但更重要的是，它非常快。只需 5-10 分钟就能得到一个蛋白质的结构，而不是一年。我应该在某个时候算算这个时间比。当然，它还具有极强的可扩展性。我们预测了 2 亿个蛋白质的结构，基本上涵盖了所有基因组已被测序的生物体的每一个蛋白质。我们广泛公开了这些数据，科学家们正在疯狂地使用它们。

I mean, AlphaFold itself is this kind of, I guess I now say landmark in AI and science, but it's really about how do we use AI to solve problems that humans can't, that are really hard, that we go and we do years-long experiments. And in the case of AlphaFold, it's this problem of protein structure prediction. This um I guess it's machine learning street talk, not biology street talk. So, you know, DNA is the instruction manual for life, but what does it actually tell you what to build? And it tells you one of the many things it tells you how to build are proteins. And these are little nanomachines, couple thousand atoms in the cell, that actually do the work of the cell. And so, three letters of your DNA uh tell you how to add one extra piece to this protein. This protein is kind of a long string, and there's a tiny machine itself made of proteins and RNA that's built one kind of string at a time. And you you make this rope of 20 types of chemical groups. So, it's kind of 20 types of letters, and people of course use the alphabet for these things. Um and each of them are quite different, right? My my PhD supervisor could tell you lovingly about what's special of each one. But you build this kind of rope of the protein, and then it assembles itself. It twists, it curls, it folds up into a really kind of compact and interesting shape. It has these helices, sheets, all these things. And that's actually what works. And the the analogy I always kind of like to say is it's like you have an IKEA bookshelf, and you open the box and it builds itself. And so, this really really central problem for maybe 70 plus years in in biology is okay, how do I I can read DNA? In fact, I can read DNA really well now. You know, you can you probably many people in your in your uh listeners have had their DNA sequenced. But understanding the structure of even one protein is extraordinarily difficult, right? That's a worthy PhD project. I would say maybe a typical kind of time frame is a year. If you want to put money on it, maybe $100,000 to get one answer. And this is really important to biology because we want to understand how these proteins work. When they misfold, sometimes it's disease. Even when they work, they are the parts of the cell. They do all the parts, the things of the cell, you know, they're beautiful proteins. The reason that, you know, cells can move, right? Or is this giant protein machine whirling around, driving uh the force to move cells. All of the functions of the cell basically are in these proteins. Humans have about 20,000 different types uh in different locations in in your genome. And so, what scientists have done is they've gone to these enormous enormous machines, synchrotrons normally, you know, the size of small towns in order to produce extraordinarily bright x-rays. And even that only after they've done really, really hard experiments trying to figure out how to what's called crystallize a protein. You know, and this takes years and years. And once they do that, and then they solve another mathematical problem that maybe we'll talk about, maybe we won't. They get one picture of a protein. Um and they get kind of this progress and often this whole wealth of understanding of, "Oh, okay, I can understand how this DNA change that was found in the population might affect Parkinson's because look, it's right here on this protein and that makes so much more sense." And so, people have studied this problem for a long time. There've been almost innumerable Nobel Prizes given for individual proteins, right? Ribosomes and many others. People have an incredible societal investment collected around 200,000 of these structures, about 140,000 at the time uh we did AlphaFold. Each one still extraordinarily difficult, right? Each one still, I remember seeing people talk about their PhD and give their one of their talks near the end of their PhD, progress toward crystallizing whatever protein. Right? So, I did my I'm going to be doctor and I probably I'm not going to crystallize this protein. I guess I I'm telling you all about proteins and nothing about what we did, but what we did was develop a new deep learning system from the publicly available experimental data. So, all very public data that was vastly more accurate at predicting protein structure. So, predicts it to something like within the radius of an atom, right, in in typical accuracy. And an accuracy that starts to rival at least some experimental methods, but more importantly than that is you know, extraordinarily fast. So, it takes 5-10 minutes to get the structure of a protein instead of a year. I should at some point figure out what that ratio is in terms of time. But then also, of course, it's incredibly scalable. So, we've predicted the structure of 200 million proteins, basically every protein from an organism whose genome has been sequenced. Right? We've made this widely available and scientists are using it like crazy.

这太神奇了。你们发布了一个包含所有这些蛋白质的数据库，地图被点亮了。现在，来自世界各地的科学家可以访问这些蛋白质结构，用于许多下游任务。但要让这变得生动起来，你知道，蛋白质在体内发挥作用，我们可以利用这些结构做药物发现之类的事情。但差距在哪里？也就是说，现在人们有了这些结构，他们能做什么？

It's absolutely amazing. You have released a database of all of these proteins and the map lit up. So, now scientists from all around the world, they can access these protein structures for many downstream tasks. But to bring this to life, you know, we have proteins doing things in the body and we can use these structures and we could do things like drug discovery and and and whatnot. But what what's the gap? So, what can people do now that they have these structures?

我认为正确的思考方式是，这是生物学研究的起点。如果你想想人们做了什么，有哪些漂亮的研究，我们一路都能看到。最近刚发表的一项研究是科学家试图理解胆固醇如何在体内运输。到底是什么东西把胆固醇从一个地方运到另一个地方？该蛋白质的突变如何影响高胆固醇、心脏病等？有一种美丽而奇特的蛋白质，它以某种形状包裹胆固醇，直到几个月前这篇论文发表，我们才知道这个形状。他们所做的实际上是科学家们我认为非常普遍地使用 AlphaFold 的方式之一：他们同时使用实验技术和 AlphaFold。他们使用了一种实验技术——冷冻电镜——拍摄了一张非常模糊的图片。你知道，他们过去称冷冻电镜为“模糊学”。它已经进步了很多，但仍然是一张非常粗糙的图片，他们并不真正知道原子细节。然后他们也运行 AlphaFold，他们发现，实际上 AlphaFold 预测的形状几乎完美地契合在这个模糊的轮廓内。这样你既得到了确认，又得到了更多细节。

I think the right way to think about this is it's a starting point for biological research. If you think about what what people do, what are some beautiful studies that people have done, you know, we we see it all the way. One that just came out was scientist trying to understand how cholesterol is moved about in the body, right? What what actually is the thing that takes cholesterol and moves it from one place to another? How might mutations in that affect high cholesterol, heart disease, etc.? There's this beautiful weird protein that kind of wraps around it in a shape that we really didn't know until a few months ago when this paper came out. And what they were able to do actually is a it's one of the ways in which scientists I think really commonly use AlphaFold is they use both experimental techniques and AlphaFold. So they used an experimental technique um cryo-electron microscopy to take an incredibly blobby picture. You know, they used to call cryo-EM blobbology. It's gotten much better, but it's incredibly kind of rough picture. And they don't really know the atomic details. And then they also run AlphaFold and they see, well, actually AlphaFold has this shape that almost exactly fits within this kind of blob. And so you get both confirmation and more detail.

突然之间，你就有了这个漂亮的原子模型，可以开始去说，现在这个蛋白质的变化在哪里？它可能影响什么？它如何影响它把胆固醇从一个地方运到另一个地方？然后你必须去弄清楚。现在我要如何为此制造药物？我是否要结合这个蛋白质？我认为当你看到它在药物开发中的应用时，实际上有两三种使用方式。首先我要说的是，药物开发中最难的部分是我们对生物学的运作了解得不够好。对吧？阻止我们治愈比如自闭症的原因，并不是我们确切知道某个蛋白质，只要有了它的结构，自闭症就能被治愈。那是一种涉及全身的巨大疾病。所以我们试图解开和阐明生物学，足够好地弄清楚哪些蛋白质，这些蛋白质如何相互作用？最终如何导致这些表型？所以人们在这些不同的尺度上研究生物学。而 AlphaFold 的贡献在于，它指出这个蛋白质，比如，你甚至不知道它很重要。就像几年前有一项关于某个蛋白质的研究。身体里有各种各样的回收机制，把不再需要的蛋白质处理掉或不想要的。事实上，人们发现有数百个基因在细胞发育的某个阶段被关闭，他们并不确切知道是哪个蛋白质参与其中。他们做了一些遗传学实验，发现了一个以前从未被研究过的蛋白质，一种叫做 Midnolin 的人类蛋白质。如果你把它敲低，那么这些蛋白质就不会被回收，他们大致就知道这些，而且他们知道它不以标准方式工作。他们运行了 AlphaFold，观察它，看到了一些有暗示性的片段，然后他们用 AlphaFold 结合了几乎所有 500 个对这个蛋白质敲低有反应的蛋白质，对吧？所以，变化。这就是生物学家如何建立证据。他们发现，在大约 40%的这些蛋白质中，当运行 AlphaFold 时，出现了一种非常非常特定的模式，其中那个蛋白质的一部分被夹在 Midnolin 的两个部分之间，像钳子一样抓住它。然后他们可以找到，然后他们会去做实验，对吧？因为他们会说，“好吧，如果我取这个蛋白质的这部分，去掉 AlphaFold 说被 Midnolin 夹住的地方，会发生什么？”然后突然那个蛋白质在细胞中不再下降，对吧？所以，他们在大概 10 个例子中发现了这一点，其中 9 个完全以这种方式工作。其中一个只是部分减少了敲低程度，但随后他们查看了 AlphaFold 的预测，发现 AlphaFold 实际上把它放在了两个地方。所以，如果他们同时去掉第二个地方，那么降解就完全被消除了。所以，现在他们有了对这个从未想过的新蛋白质的机制性理解，现在他们确切知道它如何识别在细胞分裂这个非常重要的阶段发育的东西。

And suddenly you have this beautiful atomic model where you can start to go and say, now where are the changes in this protein? What might it affect? How might it affect how it takes cholesterol from one place to another? Then you have to go figure out. Now how do I make drugs for this? Do I bind to this protein? I think when you see it in drug development, there's actually kind of two or three ways in which it's used. I mean, the first I will say is that the hardest part of drug development is that we do not know how biology works very well. Right? It's not that the thing preventing us from curing, say, autism, is that we know exactly one protein and if we just had its structure, then autism would be cured. It's a huge disease that involves the whole body. And so we kind of are trying to unwrap and unravel biology well enough to figure out which proteins, how do these proteins interact? How does that ultimately contribute out to these phenotypes? And so people do biology across all these length scales. And the contribution of AlphaFold is to say, this protein, for example, that you didn't even know was important. Like there was a study from a few years ago on a protein. There are all sorts of recycling mechanisms in the body that take proteins it doesn't need anymore and gets rid of them or doesn't want anymore. There were hundreds of genes in fact that people found were turned off at a certain phase in cell development and they didn't exactly know what protein was involved. They did some genetics and they found this protein that had been essentially never been studied before, a human protein called Midnolin. If you knocked it down, then these proteins didn't get recycled and that's kind of more or less what they knew and they knew it didn't work in the standard way. And they ran AlphaFold and they looked at it and they saw some pieces that were suggestive and then they ran AlphaFold together with, I think it was almost all 500 proteins that were responsive to knocking this protein down, right? So, change. So, this is kind of how biologists develop evidence. And they found in about 40% of these, when they ran AlphaFold, this very, very specific pattern where one part of that protein was trapped between two parts of Midnolin kind of grabbing it like clamps. And they could find and then they would go and they would do experiments, right? Because then they would say, "Well, what happens if I take this bit of protein and I remove the place where AlphaFold says it's clamped by Midnolin." And suddenly that protein doesn't drop down in the cell, right? So, and they found this on maybe 10 examples, nine of them worked exactly this way. One of them only partially was reduced in how much it's knocked down, but then they looked at the AlphaFold predictions and found that AlphaFold actually put it two places. And so, if they take out that second place as well, then the degradation is completely abolished. So, now they have this mechanistic understanding of this new protein they had never thought about before and now they know exactly how it recognizes what's developed in this really important stage of cell division.

是的。

Yes.

所以，现在问题变成了，好吧，现在你如何利用这些知识进行药物开发？这就是 AlphaFold 2 在 5 年前问世的原因。我们最近大约一年前所做的是 AlphaFold 3，它说，好吧，我们不只是做蛋白质，我们来做蛋白质的宇宙。所以，你知道，我说过，蛋白质结合胆固醇，对吧？所以这是一种非蛋白质的脂肪分子。嗯，不是更重要，但非常重要，它们也结合药物。药物是小分子，可能 20、50 个原子，它们附着在蛋白质上并改变它们的行为。你甚至不能向 AlphaFold 2 问这个问题。你不能说，这个药物如何附着？它会说，好吧，你最好给我一个蛋白质。只有当你的药物是蛋白质时才行，有些药物确实是。但 AlphaFold 3，我们把它扩展到了 PDB 中出现的所有东西，整个细胞的宇宙。现在我们可以说，嗯，这就是药物附着的确切位置。然后世界各地的人们都在使用这些想法，构建其他东西。例如，Alphabet 内部的 Isomorphic Labs，受到 AlphaFold 突破的启发，试图说，好吧，让我们真正利用这个开始做药物设计。让我们开始采用这些最终有效、最终具有预测性的技术。现在让我们看看我是否可以用它设计一个小分子，或者我可以设计一种药物，它结合并改变这个机器的工作方式。然后以一种希望让人健康的方式。我认为现在你应该如何看待药物开发的最佳类比，或者也许思考它有多难的方式，是这个老笑话。你知道，有个笑话，有一个巨大的工厂，其中最重要的机器之一停止了工作。他们叫来一个技术人员，他来了，看了看，走到某个螺丝或螺母前，把它转了四分之一圈。工厂恢复了运转。他们说，太棒了。非常感谢。能给我们账单吗？他说，“一万美元。”

So, now the question becomes, okay, now how do you take that knowledge and do drug development? And that's where AlphaFold 2 was what came out now 5 years ago. What we've done more recently about a year ago is AlphaFold 3, which says, well, let's not just do proteins, let's do the protein cinematic universe. And so, you know, I said, proteins bind cholesterol, right? So, this is a non-protein kind of fatty molecule. Well, more not more importantly, but very importantly, they also bind drugs. Drugs are small molecules, maybe 20, 50 atoms, that stick to proteins and change how they behave. And you couldn't even ask this question to AlphaFold 2. You couldn't say, how does this drug stick? It'd say, well, you better give me a protein. Only if your drug is a protein, which some are. But AlphaFold 3, we expanded it to kind of do the whole universe of things that appear in the PDB, the whole universe of cells. And now we can say, well, this is exactly where that drug sticks. And then people around the world are using these ideas, building others. For example, Isomorphic Labs inside Alphabet, kind of developed inspired from the AlphaFold breakthrough, are trying to say, okay, let's really use this to start to do drug design. Let's start to take these technologies that finally work, that are finally predictive about this. And now let's see if I can design a small molecule with it, or I can design a drug that binds, that changes how this machine works. And then in a way that hopefully makes someone healthy. And I think the best kind of analogy for how you should think about drug development really now, or maybe the way to think about how hard it is, is this old joke. You know, do you know this joke that there's this giant factory and one of the most important machines in this factory has stopped working. And they call in a technician who comes, he looks at it, he goes to some screw or like some nut and turns it a quarter turn. Factory back to life. And they said, that's wonderful. Thank you so much. Can we have a bill? And he says, "$10,000."

嗯。

Yeah.

他们说，“什么？”

And they say, "What?"

知道该转哪里。

Knowing what to turn.

是的。是的。你知道，知道该转哪里，或者转这个 50 美分，知道该转哪里，其余的都是。我认为这是一个恰当的类比，我们既在学习如何转动这个，对吧？如何制造药物，也在学习在这个庞大复杂的细胞工厂中，我们需要做什么来治愈疾病？

Yeah. Yeah. There's, you know, knowing what to turn or turning this 50 cents, knowing what to turn all the rest. And I think this is the right analogy that we are both learning how to turn this, right? How to build drugs, and also learning in this big complex factory of the cell, what do we need to do to cure a disease?

是的。但这太复杂了，不是吗？因为人体是活的，有一整套复杂的适应性补偿机制。我想这里的想法是，我们提出了一个机制性的理解，这意味着我们可以设计非常有效的干预措施。但在机器学习中，我们学到了相反的教训，那就是我们关于事物如何运作的所有直觉并不真正有效，我们需要大量数据，我们需要测试很多东西。这里会不会有类似的情况，你知道，就像打地鼠？你做了一件事，然后别的东西就会补偿。

Yes. But it's so incredibly complex, isn't it? Because the human body is alive, and there is a symphony of complex adaptive compensatory mechanisms. I guess the idea here is that we're proposing a mechanistic understanding of how this works, which means we can design interventions that are very effective. But in machine learning, we've kind of learned the opposite lesson, which is that all of our intuitions about how things work don't really work, and we need lots of data, and we need to test lots of things. Could it be a similar thing here that, you know, it's like whack-a-mole? You kind of you do one thing, and then something else compensates.

我认为在某种意义上非常重要的一点是 AlphaFold 的谦逊，你知道，人们说，我们试图预测这个实验会给你什么结果。我们不是要告诉你一切。我们不是整个细胞的模型。我们是这个你一直在做、花了一年时间的实验的预测器。

I think really important in a certain sense is almost the humility of AlphaFold in that, you know, people say, we are trying to predict what this experiment will give you. We are not trying to tell you everything. We are not a model of the entire cell. We are a predictor of this experiment that you did all the time and took you a year.

所以，从某种意义上说，我们有有效性，因为我可以很好地描述我们将在多大程度上重现那个实验。然后人们想办法用这个机器做其他我们没预料到的事，发现新机制，尝试数千次 AlphaFold 预测来找到两个粘在一起的蛋白质，并发现这个复杂系统中的未知成分。所以，人们正在想办法把它推得更远。但另一方面，我们是狭窄的：我们预测一篇科学论文的结果，这些论文经常出现在《自然》、《科学》和《细胞》这些大期刊上，对吧？我们一键预测自然级别的科学，但只是在一个非常狭窄的类别里：特定蛋白质的结构。但生物学这个广阔宇宙最终我们得去搞清楚，理解我们将把自己绑定到什么数据上，预测什么实验，并且预测得非常好。我的意思是，机器学习的另一个故事可能是，预测得还行是可以的，但预测得异常好就能产生惊人的机器。我们当然在语言模型、图像生成中看到了这一点，在蛋白质领域也是如此。所以，我认为我们不是在建造一台通用的生物学机器，或者至少如果我们这样做，它必须更像一个语言模型，而不是一个狭窄的预测器，但我们正在做一些真正有用的事情。

And so, in a certain sense, we have validity in that I can characterize very well how well we will reproduce that experiment. And then people figure out how to take this machine and use it in other ways that we didn't expect, to discover new mechanisms, to try thousands of AlphaFold predictions to find two proteins that stick together and find this unknown component of this complex system. So, people are finding ways to push this further. But, in a certain sense, we are narrow: we predict the result of a scientific paper that often appears in Nature, Science, and Cell in these big journals, right? We predict nature-level science with the press of a button in a very narrow category: the structure of a specific protein. But there's this enormous wide universe of biology that ultimately we're going to have to figure out and understand what data we will pin ourselves to, what experiments we will predict, and predict really, really well. Such that, I mean, maybe the other story of machine learning is that predicting things okay is all right. Predicting things extraordinarily well starts to produce amazing machines. We see this, of course, in language models, in image generation, but also in protein. So, I think this kind of thing—we aren't building just one universal biology machine, or at least if we do, it will have to look a lot more like a language model than a narrow predictor, but we are doing something truly useful.

我们能聊聊不同版本 AlphaFold 的预测架构吗？第一个版本是 CNN，最新版本是扩散模型。第二个版本，我们昨晚聊过，它有一个结构组件，显然几何深度学习被谈论很多，我认为人们错误地归因了这些对称性的好处。它用了 SE3 对称性。跟我讲讲这个过程，因为你一开始说你真的试图注入你对这个的人类理解，然后经验告诉了你不同。

Can we talk through the predictive architectures of the different versions of AlphaFold? So, the first version was a CNN, the last version is a diffusion model. The second version, we spoke about this last night, it had a structure component, and obviously, geometric deep learning is spoken about a lot, and I think people misattributed the benefit of having these symmetries. It did these SE3 symmetries. Just talk me through that process because you were kind of saying at the beginning you were really trying to imbue your human understanding of this, and then experience told you differently.

我认为有两三件事。首先，我几乎要反对，不是技术上，而是主题上，说 AlphaFold 3 是扩散模型。我们喜欢把东西放进盒子里。我们喜欢用最高层的东西作为这些工作原因的解释。对吧？哦，他们从 CNN 切换到了……我认为答案其实是：AlphaFold 1 作为一个网络，它预测了问题的一个子部分。它从生物数据、进化相关性开始，最终得到几何类数据，原子间的距离。中间是一个 CNN，对吧？它实际上是计算机视觉中别人做的一个现成 CNN。那是一个 CNN。但在 AlphaFold 1 之后，所有蛋白质特定的部分都包裹在机器学习周围。所以我会说 AlphaFold 2 是：让我们构建科学，而不是构建图像识别的科学然后应用到蛋白质上，因为人类视觉系统正是我们折叠蛋白质所需的东西，这并不正确，对吧？人类不擅长预测蛋白质结构。我们到底要怎么构建所有部分？现在，有一个 SE(3)部分。实际上，AlphaFold 2 是迭代构建的。有很多阶段。实际上，SE(3)部分是 AlphaFold 构建的第一部分。但 AlphaFold 2 最终是一个巨大的架构主干，我们称之为 EvoFormer，它是轴向注意力加上一堆其他东西。那占了 90%以上的计算量和准确性。然后它产生这种中间结果。所以你从两片数据开始：蛋白质序列，然后你找到每个进化相关蛋白质的序列。蛋白质结构变化缓慢。我的蛋白质的结构在大多数情况下与酵母中的蛋白质结构相似，有时甚至与大肠杆菌中的相似。所以你抓取许多相关结构。你经常有数百或数千个。你提供这些信息，我们有这个专门的架构叫 EvoFormer，它有两种形式的轴向注意力，在我们相信的几何和进化之间进行对话。我们有这两种表示。最后我们取几何部分，N 乘 N，我们实际上将其作为中间损失，说这些原子之间的距离是什么，并做出分类预测。然后我们把它交给所谓的结构模块，它最好被看作一个几何化引擎。对吧？如果你有关于 N 个位置的 N 平方预测，总得有人来协调这个东西。这使用了 SE3——我想它在每一层上都是不变的——SE3 不变注意力。这实际上是我的一个想法，甚至在 DeepMind 刚开始时就有。我想，哦，我们应该把点放进去，或者我在想蛋白质残基，对吧？所以你有这个主链，有三个原子。你可以把框架对齐到它上面。它非常刚性，我知道业务端，这些残基不同的地方，就在那附近。所以如果你把它们对齐到参考框架，然后你在那些参考框架中的点上操作，那么很自然，你可以直接使用注意力，让它在其局部框架中投影点。你可以变换它，然后你可以用这些点的距离来偏置你的注意力，这就是不变点注意力。我们最后就是这么叫的。挺有趣的。比这更重要的是定义框架——几乎可以肯定。定义框架让我们写下一个非常有趣的损失函数。所以我们称之为框架对齐点误差，FAPE。这有点像说：在第 i 个残基的参考框架中，其他每个人在哪里？它是局部注册的，然后你有 N 平方误差，然后你把它们平均在一起。我认为这非常重要。我认为这是早期的突破之一，就是这个损失。但当然，真正有趣的部分是 SE3 不变性。但记住，我们不是从任何几何数据开始的。我们只从非几何数据开始。所以我们的几何在中间出现。我们从所谓的黑洞初始化开始，我们把所有残基堆叠在一起，形成世界上最不物理的结构。同样重要的是，我们不尊重已知的蛋白质对称性。

I think there's two or three things. First, I would almost object, not technically, but thematically to AlphaFold 3 being a diffusion model. We love to stick things in boxes. We love to have the highest level bit be the answer for why these things work. Right? Oh, they switched from CNN to... I think the answer is really: AlphaFold 1 really was as a network it predicted a subpart of the problem. It started from biological data, evolutionary correlations. It ended in geometric-ish data, distance between atoms. In between was a CNN, right? It was actually an off-the-shelf CNN from computer vision that someone else had done. That was a CNN. But after AlphaFold 1, all the protein-specific bits were kind of wrapped around the machine learning. So I would say AlphaFold 2 was: let's build the science instead of building the science of image recognition and then applying it to proteins because, you know, the human visual system is exactly what we needed to fold proteins is not something true, right? Humans are bad at predicting protein structures. How are we going to actually build all the pieces? Now, there was an SE(3) piece. In fact, AlphaFold 2 was built iteratively. There were many stages. Actually, the SE(3) piece was the first part of AlphaFold built. But AlphaFold 2 at the end was really this giant trunk of an architecture we called EvoFormer, which is axial attention plus a bunch of other stuff. And that is 90 plus percent of the compute and the accuracy. And then it produces this kind of in-between end. So you start off with two pieces of data: the protein sequence and then you find the sequence of every protein evolutionarily related. And protein structure changes slowly. The structures of my proteins are in most cases similar to the structure of proteins in yeast, sometimes even in E. coli. So you grab many related structures. You often have hundreds or thousands. You provide this information and we have this specialty architecture called EvoFormer which had two forms of axial attention that were having a conversation between what we believed about geometry and what we believed about evolution. We had these two representations. And we end up taking the geometric bit, the N by N, which we have actually as an intermediate loss, said what are the distances between these atoms and made categorical predictions. And then we hand it to what we call the structure module, and it's best thought of as a geometrization engine. Right? If you have N squared predictions about N positions, somebody's going to have to harmonize this thing. And this used an SE3—I guess it was invariant in the sense we collapsed it on every layer—SE3 invariant attention. This was actually one of mine that was kind of even starting at DeepMind. I'm like, oh, we should probably put points in, or I was thinking about protein residues, right? So you have this backbone which has three atoms. You can align the frame to it. It's extraordinarily rigid and I know the business end, the places where all these residues differ, is kind of just off that. So if you align them to reference frames and then you operate in points in those reference frames, then it's natural and you can just take attention and you can let it project points in its local frame. You can transform it, then you can use the distance of those points as a way to bias your attention, and this is invariant point attention. This is what we called it in the end. It's kind of fun. More important than that probably was this defining the frames—almost certainly. Defining of frames let us write down a really interesting loss function. So we call it frame aligned point error, FAPE. And this was kind of saying: in the reference frame of the Ith residue, where is everyone else? And it's locally registered and then you have N squared errors and then you average them together. That I think was really important. I think that was one of the breakthroughs early on was this loss. But of course, the really fun part is SE3 invariants. And remember, we didn't start with any geometric data. We started only with non-geometric data. So our geometry emerged in the middle. We started with what we would call black hole initialization, where we just stick all the residues on top of each other in the world's least physical structure. It was important also that we disrespected known symmetries of a protein.

例如，蛋白质的已知对称性是这样的：残基中的原子之间距离为 1.3 埃，误差±0.015。实际上，即使在 AlphaFold 1 中，我们进行优化时，也会像操作关节机器人手臂一样，对典型的 300 个残基进行扭转。但这样会导致几何结构非常丑陋——一个 300 关节（实际上不是 300，是 900 关节）的机器人手臂非常糟糕，优化器需要很多步才能收敛。因此，一个重要的想法是将其分解，我们称之为“残基气体”，这样只需四步或八步，而不是那种扭曲几何所需的步数。然后我们使用了等变性，这确实有帮助。但令人惊讶的是，也许是因为早期讨论，或者人们正在研究等变几何深度学习，它变得非常流行，人们会说：“啊，他们提到了我的关键词，那一定是它成功的原因。”我记得当时有点困惑，心想：“好吧，我们会非常仔细地验证这一点。我们对 AlphaFold 2 做了不少消融实验，等论文发表后大家就会明白。”论文发表后，我记得第五行叫做“no IPA”。AlphaFold 2 在 GDT 尺度上比 AlphaFold 1 好了大约 30 分，对吧？我们做的这些消融实验，结果都很小，几乎都很小。移除等变性大约损失了 2 分，可能 2.5 分。所以它确实有贡献，但只贡献了 30 分中的 2.5 分。我以为这能平息争论，但完全没有。人们仍然把 AlphaFold 2 说成是等变性的伟大胜利，他们从不谈论 FAPE，却谈论他们认为来自其他人的等变 Transformer。实际上，我们在 2018 年就做了这个等变 Transformer，我记得是 2018 年 10 月。我们尝试改进它，但并没有让 AlphaFold 变得更好，所以就继续做下一件事了。我们对此非常务实。但这是个很酷的东西，所以人们确实被它吸引了。我认为真正的情况是，昨晚在 AlphaFold 晚宴上我们也在讨论，等变性（比如全局 SE3 对称性）并不是一个非常强大的对称性，远不如“所有残基是置换不变的”这种对称性强大。我们仍然有置换不变性作为 AlphaFold 的主要对称性，我们的 Transformer 只使用相对位置编码，并截断了相对位置编码。但我觉得这种对称性不像物理学那样，写下对称群就能推导出物理定律和标准模型。这是一个混乱现实问题的对称性，可能并不能完全确定它。所以我认为它很好，但我们不应该痴迷于一件好事，也不应该过分推崇。我最喜欢 AlphaFold 2 的评审意见是，提交论文时有一位评审说：“这相当于六七篇论文的想法。”我认为这很正确。很多想法加起来才构成了一个变革性的系统。用棒球类比，不是一两个本垒打，而是 18 个二垒安打。这些中等大小的胜利叠加在一起，构成了一个变革性的系统。在我们的消融实验中，有时我们会做双重消融，比如去掉循环和 IPA，性能就会暴跌。我们解决大多数问题都用了两种方法，因为比一种方法更好。如果同时去掉两者，建筑可能会倒塌。这可能是最大的消融，损失了 12 或 15 分，但仍然只有与 AlphaFold 1 差距的一半。我记得做消融时说过：“伙计们，我们从未超过 AlphaFold 1 的性能。”但很多消融实验实际上用到了 AlphaFold 3 中。所以我们说：“好吧，等变性不是超级重要。”另一个消融是去掉原始遗传信息而只给成对相关性，性能差了一两分。所以也许一直处理这些信息并不那么好。我们做了可解释性投影，将每一层投影到结构并制作了视频，可以看到 AlphaFold 的大部分能力都花在了几何优化结构上，它更像一个几何引擎，而不是进化引擎（除了前几层）。所以我们说：“为什么不把 EvoFormer 缩减到只运行几层，然后做一个更简单的版本叫 PairFormer？”这提高了性能。我们基本上用这些消融实验来说明“机器学习告诉我们的东西”。作为机器学习研究者，你思考数据、观察数据、提出假设，比如等变性可能重要，然后尝试、测量，十有八九会发现自己是错的。如果十次有九次是错的，那你就是非常成功的机器学习研究者，非常有生产力。你建立局部直觉，理解问题需要什么以及如何运作，在我看来，这建立了一种局部科学，一个在蛋白质结构预测领域靠近这些架构的思想局部流形。我们会发展出直觉，比如“不要在 EvoFormer 附近使用一维表示而不是二维表示，否则性能会下降”。在 AlphaFold 2 开发一年或六个月时，我们在成对处理中混合了轴向注意力和卷积，架构有些不同。我记得有人做了一个实验，只是删除了卷积层，没有增加注意力层，删除了卷积，没有增加参数，参数更少了，但模型更准确了，验证损失改善了。这在机器学习中通常不会发生——减少参数反而提高泛化能力。

For example, the known symmetries of a protein are these residues are separated the atoms this atom and that atom in a residue are separated by 1.3 angstroms plus or minus 0.015. And in fact, even in AlphaFold 1, when we would do the optimization, we would actually use turning kind of like a jointed robot arm to optimize these say typically 300 residues. And so you would do all this twisting, and you would actually have a very ugly geometry. The geometry of a 300 joint, or actually, no, sorry. Wouldn't be 300. It would be uh 900 joint robot arm is really bad. And so that means that your optimizer has to take many steps. So one of the important things is let's just break it up. Let's just treat them as a residue gas, we called it, so that this can proceed and say four steps, eight steps instead of the number of steps of this twisty geometry. And then we used equivariance, and it helped. But one of the things that was really surprising, I think it's maybe because the early talk, or maybe people were working on equivariance geometric deep learning has been very popular, and people said, "Ah, they mentioned my keyword. That must be the reason it worked." And I remember being a little bit confused, and I thought, "Okay, but we'll we'll very carefully blate this. We did quite a few ablations for AlphaFold 2, and we'll publish the paper, and everyone will realize." And we published a paper, and I remember the fifth row was called no IPA. Um so AlphaFold 2 was about 30 points on the GDT scale better than AlphaFold 1, right? So, so that's that's the kind of 30 points is is your thing. And we did these ablations, they were all small. Um almost all small. Uh removing the invariant the equivariance cost about two points. Right? And you could measure it, maybe two and a half. Right? So, it it contributed, but it contributed two and a half out of 30. And I thought that would put it to bed. And it put it it didn't even put it to bed at all. People still talked about AlphaFold 2 as the great victory of equivariance. They They never talk about FAPE. And they talk about, you know, an equivariant transformer that they think came from others. And actually, we did this equivariant transformer in like 2018. I actually remember it was October 2018. So, it was like we did one, we tried a little bit to improve it, it didn't make AlphaFold better to try and improve that part. So, we went on to the next thing, right? We're kind of ruthlessly empirical about it. But, it's a very cool thing. And so, I think people really hooked on to it. And I think what it really happens is I think we we were talking about it kind of at dinner last night at this AlphaFold dinner, but the equivariance is one like global SE3 symmetry is not a very powerful symmetry. It's not nearly as kind of big and powerful as a symmetry like, "Oh, all the residues are permutation invariant." Right? So, we do still have permutation uh invariant as probably the big symmetry of AlphaFold, right? We have a transformer that is position is relative position coded only. We clipped the relative position codings. But, I think this particular symmetry it's not like physics where you write down the symmetry group and then you derive the laws of physics from your big symmetry group and you get the standard model. This is This is a symmetry of a messy real-world problem that probably doesn't pin it down so much. So, I think it's good, but we shouldn't obsess about one good thing. Or you can't You don't want to valorize things. My favorite review of of AlphaFold 2, we got the reviews back when we submit the paper. And one of them said, "This is six or seven papers worth of ideas." Right? And I think I think that was that was right. There are many many ideas that added up to be a transformative system. And many, you know, to use a baseball analogy, it's not one or two home runs. It's, you know, 18 doubles. Right? That it it's really, you know, these mid-size wins stack together and together make a transformative system. Now, we would sometimes find in our ablations, we ran a double ablation, I think it was no recycling and no IPA. We turned off two things and performance cratered. Right? And I think it was kind of there are many problems we need to dissolve. We solved most of them two ways because it was better than solving one. And if you knock out both things, then your building maybe collapses. Or this was maybe a 12 or 15 point, which was our biggest ablation, which was still only half the gap to AlphaFold 1, right? We I remember doing the ablations and saying, "Guys, we've never crossed AlphaFold 1 performance." But a lot of those ablations actually went into AlphaFold 3. So, we said, "Okay, well, equivariance isn't super important." Um we had another ablation uh that if we take out, you know, giving the raw genetic information and give the pairwise correlations, that's one or two worse. So, maybe this fact that we're processing these all the time is not so good. We looked at we made this kind of interpretability kind of projection of each layer into a structure and made movies and could see that most of AlphaFold's capacity was spent optimizing the structure geometrically. It's much more a geometry engine than an evolution engine outside the first few layers. So, we said, "Okay, why don't we just cut back this EvoFormer to just operate a few layers and then we'll do a much simpler version called a PairFormer." And that improved performance. And we basically used these ablations to say, "This is what the machine learning is telling us." Whatever we may, you know, feel like being a machine learner is all about, you know, you think about the data, you look at it, you come up with hypotheses, maybe equivariance is important, you try it, you measure, nine times out of 10 you find out you're wrong, right? If you're wrong nine times out of 10 you're a very successful machine learner, you're incredibly productive. And and you build this local intuition, you build this notion of what the problem needs and how it works and you build, in my view, a kind of science local to your area, kind of a local manifold of ideas in protein structure prediction near these architectures. We would develop intuitions like, thou shalt not put a 1D representation rather than a 2D representation anywhere near the EvoFormer or your performance will go down. Maybe a year, six months in AlphaFold 2, at some point we had a mix of axial attention and convolutions in our pairwise processing. It was somewhat different architecture. And I remember someone did an experiment where they just deleted the convolutional layers, not like added attention layers, deleted the convolutions, added no parameters, just strictly fewer parameters, and the model got more accurate. The validation loss improved. And that doesn't normally happen in machine learning. They don't say remove parameters and your generalization will improve.

但卷积可能对学习我们想要的东西实际上是有害的。我对此有一个假设，但所有这些教训和探索都是关于深度学习如何在蛋白质中泛化的。你必须建立那种知识和专长。AlphaFold，尤其是 AlphaFold 2 中真正特别的是生物学假设、物理几何假设和经验的结合——那种多年在这个特定问题和数据集上碰壁的触感。AlphaFold 1 和 AlphaFold 2 使用了完全相同的数据。我们决定增加评估数据，但完全不增加训练数据。效果非常显著。AlQuraishi 实验室有一项精彩的研究，他们用 1%的 PDB 数据重新训练了 AlphaFold 2——大约 15,000 个结构，而不是 150,000 个。他们发现，用 1%的 PDB 数据训练的 AlphaFold 2 比 AlphaFold 1 更准确。所以，我们投入 AlphaFold 2 的架构和训练想法相当于数据量的 100 倍提升。

But convolutions were probably actively harmful to learning what we wanted to learn. I have a hypothesis about that, but all these lessons and explorations were about how deep learning generalizes in proteins. You have to build that knowledge and expertise. What's really special in AlphaFold, especially AlphaFold 2, and I'll talk about AlphaFold 3 in a minute, is the mix of biological hypothesis, physical geometric hypothesis, and experience—the tactile feel of years of banging our head against this particular problem and dataset. AlphaFold 1 and AlphaFold 2 used the exact same data. We decided to have more eval data but not increase our training data at all. The effect was huge. There was a wonderful study by the AlQuraishi lab that retrained AlphaFold 2 on 1% of the PDB—about 15,000 structures instead of 150,000. They found AlphaFold 2 on 1% of the PDB was more accurate than AlphaFold 1. So the architecture and training ideas we put into AlphaFold 2 were worth a clean 100x in data.

我们昨晚谈到了你们获得的所有隐性知识。但先放一放。机器学习的事业是为我们不了解的事物构建理解模型。我们正在建造这些外星人工制品。你举了一个很好的例子：想象生成文本。我们可能天真地认为边缘的渲染方式在承担主要工作，但实际上承重的东西可能完全不同。我们的许多直觉都不管用。对我来说——我知道你对‘理解’这个词过敏——理解就是拥有一个能做这件事的生成模型。如果你能创建一个现象的物理模拟器，假设它没有损失，我会说你理解了那个东西。我们认为机器学习是对现有示例建模，是对最终结果建模，而不是对构建方式建模。但你昨晚描述了一个迷人的东西：AlphaFold 中的循环机制。你可以多次将东西通过结构传递。一开始它解决最复杂的问题，然后进行细化。这有点像生命游戏。它不是模拟创造；它是在任何时刻学习如何细化和优化结构。

We were speaking last night about all the tacit knowledge you guys acquired. But let's park that for a second. The enterprise of machine learning is about building models of understanding for things we don't understand. We're building these alien artifacts. You gave a wonderful example: imagine generating text. We might naively think the way it's rendered around the edges is doing the heavy lifting, but actually the load-bearing thing might be completely different. Many of our intuitions don't work. For me—and I know you're allergic to the word 'understanding'—understanding is possession of a generative model that can do the thing. If you can create a physics simulator of some phenomenon, assuming it's not lossy, I'd say you understood that thing. We think of machine learning as modeling extant examples, modeling the thing at the end rather than how it's constructed. But you were describing something fascinating last night: the recycling mechanism in AlphaFold. You can place things through the structure many times. At the beginning it solves the most complex problem, then it's refining. This is a bit like the Game of Life. It's not simulating creation; it's learning how to refine and optimize the structure at any point.

好的，我认为我们应该区分三件事：预测、控制、理解。预测意味着你说‘我要做一件事，未来我的电脑屏幕上会出现什么？’这就是预测。控制是‘我未来要测量这个东西，我希望它结果是 17。’这就是控制。理解很像预测，但有人类参与。理解意味着我有一小撮事实，你可以用我能传达给另一个人的事实进行预测——紧凑，能写在索引卡上。这差不多就是理解。所以这些机器让我们预测和控制。我们目前必须自己推导理解。我们可以对人工制品进行实验，查看 2 亿个预测结构，而不仅仅是 20 万个实验结构，来帮助我们理解，但它不会替我们做理解这件事。它做预测，也许还有控制。还有另一件事：算法。这是机器学习中一个非常重要的概念。有我们编程的算法，也有我们得到的算法。机器学习是代码遇到数据产生权重。一个持久的争论是代码做了多少工作，而最终进入权重的数据做了多少。在 AlphaFold 中，我们看到一个非常直观的算法——连续的几何细化。我用几句话就传达给了你；你可能几乎在脑海中看到了它，尽管你没看过那些视频。那是人类已经想出的算法。也许我们应该在某个经验模型上做梯度下降，让每个东西更正确。那不是我们编程的，但我们确实思考过。我们思考过循环：‘AlphaFold 在层中必须给出答案，无论问题有多难，这难道不奇怪吗？也许我们应该给它更多层。但我的 GPU 内存不足，所以也许我应该让它再跑一遍以避免更多内存。’即使没有循环，AlphaFold 也在学习这种迭代。然后我们加入了一个代码想法，一个架构想法，来帮助这个将从数据中学习的过程。回到残基之间的距离：我们没有告诉 AlphaFold。我们知道数据会尖叫着说 I 和 I+1 相距 1.3 埃。当我们思考人类理解时，我不太喜欢人们试图应用的苦涩教训。AlphaFold 2 恰恰相反。我们做了很多专门的事情，因为我们的数据不是无限的。现在我们已经转向语言模型，我们发现数据仍然是有限的。互联网是有限的。所以‘不要做架构研究’是从中得出的错误结论。要谦虚地看待哪些东西进入你的代码，哪些将从数据中推导出来。看看缺少什么，理解深度学习试图学习的算法。你如何加速它？你如何添加假设，特别是添加沟通？

Okay, I think we should distinguish three things: predict, control, understand. Predict means you say 'I'm going to do a thing, what will appear on my computer screen in the future?' That's predict. Control is 'I want to measure this thing in the future and I want it to come out 17.' That's control. Understand is a lot like predict except there's a human in the loop. Understand means I have such a small collection of facts that you will predict and you will do it with facts that I can communicate to another human—compact, fits on an index card. That's almost understand. So these machines let us predict and control. We have to derive our own understanding at this moment. We can experiment on the artifact, look at the 200 million predicted structures, not just the 200,000 experimental structures, to help us understand, but it doesn't do the act of understanding for us. It does predict and maybe control. Then there's one other thing: the algorithm. It's a really important concept in machine learning. There's the algorithm you program and the algorithm you get. Machine learning is code meets data produces weights. One of the lasting debates is how much work is done by the code versus the data that ends up in the weights. In AlphaFold, we see a beautifully intuitive algorithm—successive geometric refinement. I communicated that to you in a few words; you probably almost saw it in your head even though you haven't seen the videos. That's an algorithm humans already came up with. Maybe we should do gradient descent in some empirical model that makes each thing more correct. That wasn't what we programmed, but we still thought about it. We thought about recycling: 'Isn't it weird that AlphaFold in layers has to give an answer no matter how hard the problem is? Maybe we should give it more layers. But my GPU's out of memory, so maybe I should run it back through to avoid more memory.' Even without recycling, AlphaFold was learning this kind of iteration. Then we put in a code idea, an architectural idea, to help this process that it was going to learn from the data. Going back to how far residues are apart: we didn't tell AlphaFold that. We knew the data would scream that I and I+1 are 1.3 angstroms apart. When we think about human understanding, I don't really love the bitter lesson as people try to apply it. AlphaFold 2 is the opposite. We did a bunch of specialty stuff because our data is not finite. Now that we've gone to language models, we found our data is still finite. The internet is finite. So 'don't do architectural research' is the wrong thing to draw from it. Have humility about which things go into your code and which will be derived from your data. Look at what's missing, understand the algorithm deep learning is trying to learn. How can you accelerate it? How can you add hypotheses, especially communication?

在架构中，我们做的最重要的事情是修改哪些单元进行通信以及如何通信。我认为所有这些都是一种我们如何推导理解，最终形成一个迭代过程的方式。如果你试图制造一个复杂的几何物体，你会进行迭代，这应该不会让任何人感到惊讶。或者类似地，如果你考虑生成文本，对吧？人们会做出一种天真的假设，认为这些只是下一个词生成器，所以它们完全不知道两个词或三个词之后会发生什么。但当然，你不能那样想；你不可能在不知道句子如何结尾的情况下写下下一个词——我大多数时候不会在不知道句子如何结尾的情况下开始一个句子，对吧？有时我会改变，但我会提前思考一点以完成我的任务。因此，我们在这些模型中看到的理解，是我们想要的结构有时会涌现，有时不会。我认为我们过分推崇那些强加的高级概念，例如在 AlphaFold 3 中，回到 AlphaFold 3，对吧？你说，“它是一个扩散模型。”但我会说它是一个与图像模型不同的扩散模型。也许有些不同。首先，有一个巨大的主干部分根本不是扩散模型，它只运行一次。那个主干可能才是结构真正被确定的地方，而扩散就像结构模块是一个几何化引擎，它接受一组非常好的约束，这些约束对结构有非常清晰的概念，然后解决细节。

The most important thing we would do within the architecture is modify which units communicated and how. I think all of these have been kind of how we derive understanding to ultimately make an iterative process. And it should shock no one that if you're trying to make an intricate geometric object that you are going to iterate. Or similarly, if you think about generating text, right? And one kind of naive assumption that people will make is that these are next word generators, so they have no idea what's going to happen in two words ahead or three words ahead. But of course, you can't think of that; you can't write down the next word without—I don't start a sentence not knowing how it's going to end most of the time, right? At some points I change, but I think ahead a little bit in order to accomplish my task. And so, the understanding that we see built into these models are kind of the structures that we want sometimes emerge and sometimes don't. And I think we valorize the high-level ideas that impose, for example, in AlphaFold 3, coming back to AlphaFold 3, right? You said, "It is a diffusion model." But I would argue it's a different diffusion model than an image model. Maybe well, there's some different. For one thing, there's a huge trunk that is not in any way a diffusion model that's only run once. That trunk is probably where the structure is actually determined, and the diffusion is just like the structure module was a geometrization engine that took a set of really quite good constraints that had very clear notion of the structure within those constraints and solved up the details.

我认为 AlphaFold 3 的扩散是相似的，而且特别相似，因为在图像中，你开始生成图像，你会看到这些早期训练的扩散模型生成彩色斑点，然后它们开始决定这些彩色斑点意味着什么，并且它们相当清楚地决定这些彩色斑点以后会意味着什么，因为你可以在过程中停止它们并重新运行，得到对这些彩色斑点略有不同的解释。在 AlphaFold 3 中，实际上有一个有趣的事情：如果你看 AlphaFold 2，我们可以通过投影中间层的过程看到它首先解决什么，它基本上解决局部细节、局部片段。它开始将局部片段组合在一起。它在解决结构时是聚集式的，这很自然。最容易预测的是局部结构。最难预测的是最大尺度的结构。这就是 AlphaFold 2 的工作方式。如果你看 AlphaFold 3，并取那些添加了大量噪声的坐标，那么你必须解决的第一件事是，比如如果有两个蛋白质，它们如何关联？它们的两个斑点相对位置在哪里？它们的高斯分布是什么？所以，AlphaFold 2 最后解决的问题正是 AlphaFold 3 的扩散必须首先意识到的问题。它是如何做到的？答案不是它想出一个方向然后围绕它构建蛋白质，因为当然它要追求一个正确答案或至少一个非常窄的分布。答案实际上是它之前的大型网络加上通过扩散网络的第一次传递解决了整体结构，然后扩散实现它之前无法解决的任何细节。它基本上是在采样。所以，从技术上讲它是扩散，但更接近 AlphaFold 2。我认为没有理由说它是出于懒惰和几何学等非常具体的技术原因，使得扩散成为 AlphaFold 3 的绝佳选择。它使得处理配体更容易，处理了一些键距和局部问题。但它不像那种“哦，它画出斑点并在最后决定它们意味着什么”的扩散。

I think AlphaFold 3 diffusion is similar and it's especially similar because in fact in images, okay, you start generating an image and you see especially these early trained diffusion models generate kind of colored blobs and they start to decide what those colored blobs mean and they pretty clearly kind of decide what those colored blobs will mean later because you could stop them in the middle of the process and run them again and get a somewhat different interpretation of those colored blobs. In AlphaFold 3, you actually have an interesting thing that if you look at AlphaFold 2, we can kind of through this process of projecting out intermediate layers see what it solves first and it basically solves local details, local pieces. It starts to put local pieces together. It's agglomerative in how it solves the structure as is kind of natural. The easiest thing to predict is your local structure. The hardest thing to predict is your largest scale structure. That's how AlphaFold 2 works. If you look at AlphaFold 3 and you take coordinates which you've added a very large amount of noise to. Well, the very first thing you have to solve is how say if you have two proteins, how do they associate? Where are their two blobs relative to each other? What are their Gaussians? So, the problem that AlphaFold 2 is solving last is the problem that AlphaFold 3's diffusion has to realize first. And how does it do it? The answer is not that it comes up with an orientation and builds the protein around it because of course it's going for one correct answer or at least a very narrow distribution. The answer is really the big network before it plus the first pass through the diffusion network is solving the overall structure and then the diffusion is realizing any details it couldn't solve before. It's basically sampling among. So, it is diffusion technically but it's much closer to AlphaFold 2. I think there's no reason that it was kind of very specific technical reasons around kind of laziness and geometry that made diffusion a really good choice for AlphaFold 3. It made it easier to handle ligands and handled some bond distances and local things. But, it's not like diffusion in the same way as oh, it's drawing up blobs and deciding what they mean at the end.

所以，我认为所有这些，你知道，人们喜欢说这有效是因为它是 Transformer。而且，这有效是因为它是 Transformer 并不能解释为什么聊天模型在过去三四年里变得好得多。它不能解释所有的研究。它不能解释研究人员每天在做什么。所有这些细节远比我们想讨论的“它是 Transformer 还是扩散模型”这种高级标签重要得多。而且，即使这些扩散机制也不是以那种对图像有意义的渐进式精炼方式工作的，对吧？也许你会制作彩色斑点，然后决定这些斑点意味着什么。即使这样，我认为你可以说可能不完全是这样，但对于蛋白质来说肯定不是这样，因为最难的问题是大型结构。

So, I think all of these are you know, people like to think of these like to say this works because it's a transformer. And, you know, this works because it's a transformer doesn't explain why, you know, chat models have gotten vastly better in the last 3-4 years. It doesn't explain all the research. It doesn't explain, you know, what researchers do every day. All of these details are far more important than this high-level bit of is it a transformer, is it a diffusion model that we want to talk about. And then also even these diffusion mechanisms don't work in the way of kind of progressive refinement that makes sense for images, right? Maybe you'll make colored blobs and you'll decide what those colored blobs mean. Even that, I think you can argue maybe not entirely the story, but it's definitely not the story for proteins cuz that's the hardest problem is the large-scale structure.

我的意思是，从某种意义上说，这倾向于我们之前讨论的构造复杂性的想法。我很想听听你对此对通用人工智能意味着什么的总体看法。因为以语言模型为例，我们基本上通过行为克隆来训练它们。所以，我们拥有这个丰富的自适应生成过程，我们生成所有这些语言，并在其上训练语言模型。对我来说，智能是粗粒度表示的自适应获取。文化和语言一直在变化。所以，我们发明了“unalive”这个词来绕过社交媒体平台的过滤器。这是一个语言能动性的例子。语言模型，我们注意到，当我们通过主动微调和适应进行这种迭代自适应精炼时，它们变得有点智能。它们学习新的表示并适应。在某种程度上，它们所做的，即使它们与路径脱节，它们可以像 AlphaFold 那样采用代码解决方案，并对其进行精炼，再精炼。而且效果似乎非常好。但是，你认为我们是否处于这样一种状态，即我们不一定在构建具有相同类型通用性的制品？我的意思是，你总体上如何看待智能？

I mean, in a sense, this is leaning towards this idea of constructive complexity that we were talking about before. And I'd love to get your general take on what this means for artificial general intelligence. Because with language models, for example, we train them basically with behavior cloning. So, you know, we have this rich adaptive generative process, and we generate all of this language and we train language models on them. And for me, intelligence is the adaptive acquisition of coarse-grained representations. Culture and language is changing all of the time. So, we invent the word unalive to get around the filters on social media platforms. And that's an example of linguistic agency. Language models, we notice that when we do this iterative adaptive refining with active fine-tuning and adaptation, they become a bit intelligent. They learn new representations and they adapt. And in a way, what they're doing is even though they're ungrounded from the path, they can take a code solution like AlphaFold, and they can refine it and they can refine it. And it seems to work really, really well. But, are we in this regime, do you think, that we're not necessarily building artifacts that have the same type of generality? I mean, what do you think about intelligence in general?

所以，这个表示问题非常重要。而且远没有五年前人们以显式方式认为的那么重要。就像我们刚才讨论的，AlphaFold 做的事情以及它被代码强制做的事情，显然它做了所有被强制的事情，但很多事情它是在没有被强制的情况下做的。因为它必须学习这些才能对数据做出好的预测模型。它必须找到好的中间表示。所以，从某种意义上说，我认为机器学习中最诱人的想法总是：有一样东西我知道最终必须存在。

So, this question of representations is very, very important. And far less important than people believed 5 years ago in the explicit way. So, just like we were talking about the things that AlphaFold does and the things that AlphaFold is forced to do by its code are, you know, obviously everything that's forced to do by its code it does, but many things it does it does without being forced. Because it had to learn it to make a good predictive model of the data. It had to find good intermediate representations. So, in a certain sense, I think the most seductive idea in machine learning is always there's this thing I know will have to be in the end there in the end.

所以，我必须在代码里放一个叫这个名字的东西，然后强制机制变成一个高级概念构建器，对吧？这是一种非常流行的做法：我会创建概念单元。我会通过这个损失函数强制解耦表示，有时在中间层。这就是它存储这些概念的地方。这样去测试是合理的，但我们看到很多情况是，你认为智能所需的东西，其实是通过拼命地、非常好地预测下一个词而发展出来的。它们不是因为预测下一个词而出现，而是因为你把这件事做得非常好才出现的。所以，这些广义的空间、表示、对概念的理解，都是随着数据非常缓慢地被强制出来的，对吧？几乎所有的对数线性关系，或者我最不喜欢的那个函数，对吧？事情随着努力指数的增加而线性增长。我们在缩放定律中一直看到这一点。但我们确实得到了这些概念和表示，而我们真正没有答案的是，如何更便宜地得到它们？有时我们可以通过编程得到，比如得到类似记忆的东西。现在我们有语言模型为自己写笔记，然后检索这些笔记，或者我们发现最好不断提醒智能体它们在做什么，这样它们就不会在长轨迹中遗忘。所以我们构建权重，构建工件，找到效率。我们经常可以用某种软件框架来掩盖这些缺陷，但我们还不知道如何将这些反馈回机器学习本身。你不能放一个带有外部记忆的框架，然后将其蒸馏回网络，得到不再需要这个框架的惊人记忆能力。我们还没搞清楚怎么做。

So, I'm going to have to have a place in my code that is named that and then forces the mechanism to a high-level concept builder, right? And that was a very popular kind of approach: I'll make the concepts units. I shall force disentangled representations via this loss, sometimes on the intermediate layer. This is where it will store those. And that's reasonable to go test, but what we've seen a lot is that many things you would imagine are needed for intelligence are developed by desperately trying to predict the next token really, really well. And they're not developed because you predict next tokens at all. They develop because you do a really, really good job at it. And so, these generalized spaces, representations, understanding of concepts are forced very slowly with data, right? Pretty much all the kind of log-linears, or my, everyone's least favorite functional, right? The things go up linearly with the exponent of effort. That we see all the time in our scaling laws. But we do get these concepts and representations, and what we don't really have an answer for is how do we get them cheaper? Now, sometimes we can get them via programming, right? We get memory-like things. Now we have language models writing notes for themselves and then retrieving those notes, or we find out it's better to keep reminding agents what they're doing so they don't forget over long trajectories. So we build weights, we build artifacts, we find efficiencies. We can often paper over those deficiencies in some kind of software harnesses, but then we don't yet know how to drive that back into exactly our machine learning. You don't put a harness with external memory and then distill it back into the network and have amazing memory things that no longer need this harness. That we haven't figured out how to do.

不幸的是，约翰，我们得结束了。但约翰·詹珀博士，非常荣幸能邀请您上节目。非常感谢您今天参加我们的节目。

Unfortunately, John, we have to wrap it. But Dr. John Jumper, it's been an honor to have you on the show. Thank you so much for joining us today.

非常开心。谢谢。

Been tremendous fun. Thank you.

正如我之前所说，伊曼纽尔，他常驻非洲，他实际上在培训科学家，不仅让他们使用 AlphaFold，还教他们如何使用、解读结果，以及如何帮助科学家利用数据库设计实验。

So as I said earlier, Emmanuel, he's based in Africa and he's actually training scientists, not just giving them access to AlphaFold, but he's training them how to use it, how to interpret the results, and how to help scientists build experiments using the database.

是的，我的研究重点是疟疾和肠道细菌的药物发现。我还参与为非洲研究人员进行能力建设，使用 AlphaFold 等工具。最初，非洲科学家无法获得昂贵的结构生物学工具。有了 AlphaFold，这些研究人员现在可以进行以前不可能完成的复杂实验，应对疟疾、艾滋病毒和其他抗生素耐药感染等疾病。在我自己的药物发现研究中，我使用 AlphaFold 解析冷冻电镜数据的结构，并利用它来绘制蛋白质的机制。

Yes, so my research focuses on drug discovery for malaria and enteric bacteria. And then I'm also involved in capacity building for Africa-based researchers using tools like AlphaFold. Initially, African scientists didn't have access to expensive structural biology tools. With AlphaFold, these researchers can now do complex experiments that were not possible before and tackle diseases such as malaria, HIV, and other antibiotic-resistant infections. In my own research on drug discovery, I use AlphaFold in terms of solving structures of cryo-EM data. And I also utilize that to map out the mechanisms of the proteins.

所以，对他来说，AlphaFold 影响巨大。想想之前和之后，我们现在生活在不同的世界里。

So, for him, AlphaFold was so impactful. Like, if you think about the before and after, we're living in a different world now.

那时，解析蛋白质结构仍然非常非常困难。所以我尝试了好几年，将近四五年，都没有成功。而有了 AlphaFold，想象一下，那是十多年前，我回去只做了一次蛋白质纯化，收集数据，结合 AlphaFold，我在不到两三个月内就得到了结构。

At that time, to phase a protein was still really, really difficult. So I tried for several years, close to four or five years, and it wasn't successful. And with AlphaFold, imagine this is more than 10 years ago, with AlphaFold, I went back and did just one protein purification, collected the data, and with AlphaFold in combination, I got the structure in less than two or three months.

现在他的目标是尽可能多地培训科学家如何使用这项技术，造福人类。

And now it's his goal to train as many scientists as he can how to use this technology for the betterment of humankind.

今年，在 Google DeepMind 和瑞典研究委员会的资助下，我们已扩大到 100 人，培训质量没有下降。事实上，还有所提高。因此，基于此，我们希望在接下来的 10 年里每年培训 100 名科学家。所以，我们的目标是在未来十年内让近 1000 名非洲科学家能够有效使用这个工具。然后，我们希望形成一个新兴的结构生物学实践者社区，致力于研究非洲的流行疾病。

This year, with funding from Google DeepMind and Swedish Research Council, we have scaled up to 100, and there's no drop in the quality of the training. In fact, there was an improvement. So, based on this, we want to train 100 scientists every year for the next 10 years. So, we're targeting close to 1,000 African scientists in the next decade to be able to utilize this tool effectively. And then, we want to form an emerging community of structural biology practitioners working on prevalent diseases in Africa.

以上就是 AlphaFold 特辑。非常感谢约翰和伊曼纽尔。与约翰的对话非常有趣。他非常鼓舞人心，因为我认为他证明了这样一个事实：尽管我们谈论所有这些通用基础模型，但要真正推进前沿并在科学领域构建尖端应用，我们需要做大量的工程。我们需要领域知识。我们需要深厚的专业知识。而且我们的许多模型实际上会相当混合，相当定制化。我认为 AlphaFold 是我们可以部署以推动科学领域发展的混合模型类型的一种存在证明。约翰，祝你在 Anthropic 的新职位上一切顺利。感谢收看本期节目。

So, that was the AlphaFold show. Thank you very much to John and Emmanuel. The conversation with John was very interesting. He's so inspiring because I think he is testament to the fact that even though we talk about all of these general-purpose foundation models, to really advance the frontier and to build cutting-edge applications in science, we need to do a lot of engineering. We need domain knowledge. We need serious expertise. And a lot of our models will actually look quite hybrid. They'll look quite customized. And I think AlphaFold is a kind of proof of existence for the types of hybrid models that we can deploy to further the field of science. John, I wish you the very best of luck in your new position at Anthropic. And thanks for watching the show.