Max Welling：物理、对称性与 AI 驱动的材料发现

今天非常高兴邀请到 Max Welling 作为嘉宾。Max 在他的职业生涯中做了很多让我非常兴奋的工作。如果你在深度学习社区，你可能因为他在变分自编码器上的工作而认识他，这项工作 literally 经受住了时间的考验。如果你是一名科学家，你可能因为他在图神经网络和等变性上的开创性工作而认识他。如果你在材料科学领域，你可能知道他的新初创公司 CUSAI。Max 有很长的历史处理各种很酷的问题。你从量子引力开始，我认为这和你做的其他事情非常不同。对于 AI 工程师和科学家来说，第一个问题是：你思考问题的线索是什么？让你兴奋的那类事情的线索是什么？你如何决定下一个你想做的大事？

It's a pleasure to have Max Welling as a guest today. Max has done so much over his career that I've been so excited about. If you're in the deep learning community, you probably know Max for his work on variational autoencoders, which has literally stood the test of time. If you are a scientist, you probably know him for his pioneer work on graph neural networks and equivariance. And if you are in material science, you probably know about his new startup, CUSAI. Max has a long history doing lots of cool problems. You started in quantum gravity, which is I think very different than all of these other things you worked on. The first question for AI engineers and for scientists: what is the thread in how you think about problems? What is the thread in the type of things which excite you and how do you decide what is the next big thing you want to work on?

实际上这已经演变了很多。在我年轻的时候，我会追随我觉得超级有趣的东西。我有一种传感器——我想很多人都有，但可能没有真正充分利用——就是你对某个问题感到非常兴奋的感觉，对吧？比如可能是黑洞里面是什么，或者宇宙的边界是什么，或者量子力学到底是什么。所以我基本上在整个职业生涯中都追随这一点。但我必须说，随着年龄增长，这会有一些变化，因为有一个新的维度加入进来，那就是影响力。从事二维量子引力研究，你几乎可以保证你的工作不会有什么影响力——也许有几篇论文，但在这个能量尺度上不会对现实世界产生影响。随着我接近退休——幸运的是还有大约 10 年——我确实想对世界产生积极的影响。而且我对气候变化相当担忧。我认为我们应该——而政治似乎很难解决这个问题，尤其是现在——所以我想最好从技术方面着手，这就是我们创办 CUSAI 的原因。但材料科学中也有很多非常有趣的科学问题。所以这有点结合了你可以产生的影响以及有趣的科学。所以就是这两个维度：一方面研究那些你觉得“哇，这里有一些非常深刻的东西”的问题，另一方面尝试构建能够真正对世界产生影响的工具。

So it has actually evolved a lot. In my young days, let's prove I would just follow what I would find super interesting. I have kind of this sensor—I think many people have but maybe not really use very much—which is like you get this feeling about getting very excited about some problem, right? Like it could be what's inside of a black hole or what's at the boundary of the universe or what is quantum mechanics actually all about. And so I followed that basically throughout my career. But I have to say that as you get older this changes a little bit in the sense that there's a new dimension coming to it, and this is impact. Working in two-dimensional quantum gravity, you're pretty much guaranteed there's going to be no impact on what you do relative—maybe a few papers, but not in this world at this energy scale. As I get closer to retirement, which is fortunately still 10 years away or so, I do want to kind of make a positive impact in the world. And I got pretty worried about climate change. I think we should—and politics seems to have a hard time solving it especially these days—and so I thought better work on it from the technology side and that's why we started CUSAI. But there's also a lot of really interesting science problems in material science. And so it's kind of combining both the impact you can make with it as well as the interesting science. So it's sort of these two dimensions: like working on things which you feel is like, wow, there's something very deep going on here, and on the other hand trying to build tools that can actually make a real impact in the world.

所以当我回顾你做的不同事情时，其中一些似乎有预先的联系，比如物理学到等变性，也许还有图神经网络，这似乎和 CUSAI 有些关系。你有一条贯穿其中的线索吗？

So the thread that when I look back at the different things you worked on, some of them seem preconnected like the physics to equivariance and graph neural networks maybe, and that seems to be somewhat related to CUSAI. Do you have a thread through there?

是的，我认为物理学是线索。所以，在理论物理学中花了很长时间后，我认为首先有非常基本和令人兴奋的问题，比如量子引力中实际上尚未解决的问题。所以，这确实是前沿，但也有很多数学工具可以使用，对吧？例如，在粒子物理学中，但在广义相对论中，对称性扮演着极其重要的角色。这一直延伸到规范对称性。所以将这类对称性应用于机器学习实际上——我认为这是一个非常深刻和有趣的数学问题。我和 Taco Cohen 一起做了这个，Taco 是背后的主要推动者，从简单的旋转对称性一直到球面上的规范对称性等等。还有 Maurice Weiler，他也在场，当他是我博士生时，他写了一整本书，我真的推荐，关于对称性在 AI 和机器学习中的作用，我发现这是一个非常深刻和有趣的问题。所以最近，我走了一条不同的路，即扩散模型和一个叫做随机热力学的领域之间的关系。这基本上是热力学，是平衡态的理论。但然后为非平衡系统表述。事实证明，我们用于扩散模型的数学，甚至用于强化学习、薛定谔桥、MCMC 采样的数学，与非平衡系统的物理理论具有相同的数学。这让我非常兴奋。实际上，当我在南非开普敦附近的非洲数学科学研究所教授一门课程时，我把它变成了一本书。所以两年后书完成了。我已经把它寄给了出版商，这是关于自由能、扩散模型——基本上是生成式 AI——和随机热力学之间的深层关系。所以总是某种——我不知道——我觉得物理学非常深刻。我也经常思考量子力学，这是一个完全奇怪的理论，实际上没有人真正理解。有一个非常有趣的故事，也许可以讲一讲，把我博士阶段和现在联系起来。所以我和一位诺贝尔奖得主一起做了博士，他是我见过的最聪明的人。只要我见过他，他从未在任何事情上犯错。现在他说量子力学是错的，他有一个新的量子力学理论。没有人理解他在说什么，尽管他写下的东西在数学上并不复杂。但他试图正面解决量子力学的可理解性问题。我觉得这非常勇敢，我完全被迷住了。所以我也在思考，好吧，我能否以一种更平凡的方式真正理解量子力学，没有那些奇怪的多重宇宙和坍缩之类的东西。所以物理学一直是线索，我试图将物理学应用于机器学习，以构建更好的算法。

Yeah, I think physics is the thread. So, having spent a lot of time in theoretical physics, I think there's first very fundamental and exciting questions like things that haven't actually been figured out in quantum gravity. So, there's really the frontier, but there's also a lot of mathematical tools that you can use, right? In for instance, in particle physics, but also in general relativity, symmetry plays an enormously important role. And this goes all the way to gauge symmetries as well. And so applying these kinds of symmetries to machine learning was actually—I thought of it as a very deep and interesting mathematical problem. I did this with Taco Cohen and Taco was the main driver behind this, went all the way from just simple rotational symmetries all the way to gauge symmetries on spheres and stuff like that. And Maurice Weiler, who's also here, when he was a PhD student with me, he wrote an entire book which I can really recommend about the role of symmetries in AI and machine learning that I find is a very deep and interesting problem. So more recently, I've taken a sort of different path which is the relationship between diffusion models and a field called stochastic thermodynamics. This is basically the thermodynamics which is a theory of equilibrium. So but then formulated for out-of-equilibrium systems. And it turns out that the mathematics that we use for diffusion models but even for reinforcement learning, for Schrödinger bridges, for MCMC sampling has the same mathematics as this physical theory of non-equilibrium systems. And that got me very excited. And actually when I taught a course in South Africa close to Cape Town at the African Institute for Mathematical Sciences, I turned that into a book. So two years later the book is finished. I've sent it to the publisher and this is about the deep relationship between free energy, diffusion models—basically generative AI—and stochastic thermodynamics. So it's always some kind of—I don't know—I find physics very deep. I also think a lot about quantum mechanics and it's a completely weird theory that actually nobody really understands. And there's a very interesting story which is maybe good to tell to connect sort of my PhD back to where I am now. So I did my PhD with a Nobel laureate, the most brilliant man I've ever met. He was never wrong about anything as long as I've seen him. And now he says quantum mechanics is wrong and he has a new theory of quantum mechanics. Nobody understands what he's saying, even though what he's writing down is not mathematically very complex. But he's trying to address this understandability, let's say, of quantum mechanics head-on. I find it very courageous and I'm completely fascinated by it. So I'm also trying to think about, okay, can I actually understand quantum mechanics in a more mundane way, sort of without all the weird multiverses and collapses and stuff like that. So physics has always been the thread and I'm trying to apply the physics to machine learning to build better algorithms.

我想把它看作我称之为物理处理单元的东西，就像 PPU，对吧？就是你有数字处理单元，然后你有物理处理单元。

I want to think of it as what I would call a sort of a physics processing unit like a PPU, right? Which is you have digital processing units and then you have physics processing units.

所以基本上是自然为你做计算。它甚至是已知可能的最快计算机。

So it's basically nature doing computations for you. It's the fastest computer known possible even.

编程有点难，因为你必须做所有这些实验，也相当笨重。就像你必须做的一件非常大的事情。但在某种程度上，它是一种计算，这就是我想看待它的方式。你可以在数据中心做计算，然后你可以让自然做一些计算，对吧？你与自然的接口有点复杂，但这些事情必须无缝地协同工作，才能得到你感兴趣的新材料。

It's a bit hard to program because you have to do all these experiments, also quite bulky. It's like a very large thing you have to do. But in a way it is a computation and that's the way I want to see it. You can do computations in a data center and then you can ask nature to do some computations, right? Your interface with nature is a bit more complicated, but then these things will have to seamlessly work together to get to a new material that you're interested in.

你仍然非常投入于理解物理和世界，甚至超越了机器学习的应用或引入新的形式体系。这真的很酷。

You are still very involved in understanding physics and the world, even beyond just applications to machine learning or introducing new formalisms. That's really cool.

是的。所以我会说我对物理本身贡献不大，但我是在为物理与科学之间的接口做贡献，这被称为 AI for science 或 science for AI。这实际上是一门正在兴起的新学科。

Yes. So I would say I'm not contributing much to physics, but I'm contributing to the interface between physics and science, and that's called AI for science or science for AI. It's actually a new discipline that's emerging.

是的。

Yeah.

而且它不只是兴起，而是爆发式增长，我觉得这个词更贴切。

And it's not just emerging, it's exploding, I would say that's the better term.

因为我知道投资从数亿美元现在到了数十亿美元。所以现在实际上有一个 Jeff Bezos 的初创公司，你知道那个 62 亿美元的 C 轮融资，对吧？这简直疯狂，我想这是有史以来最大的初创公司，对吧？而且那是在 AI for science 领域。这告诉你我们正在这里制造一个新的泡沫。

Because I know you go from investments into like in the hundreds of millions now in the billions. So there's now actually a startup by Jeff Bezos that you know that 6.2 billion C round, right? It's like insane, like this is the largest startup ever I think, right? And that's in this field AI for science. Like it tells you something that we are creating a new bubble here.

对。

Right.

那么你认为原因是什么？是什么变化促使人们开始研究 AI for science 类型的问题？

So why do you think it is? What has changed that has motivated people to start working on AI for science type problems?

实际上有两个原因。一是人们一直在将 AI 的新工具应用于科学，这很自然。当然我认为有两个大例子：蛋白质折叠是一个，另一个是机器学习力场，有时称为机器学习原子间势。这两者实际上都非常成功，也都与对称性有关，这也很酷。

So there's two reasons actually. The one is that people have been applying sort of the new tools from AI to the sciences, which is quite natural. And there's of course I think there's two big examples: protein folding is a big one, and the other one is machine learning force fields or sometimes called machine learning interatomic potentials. Both of these have been actually very successful, both also had something to do with symmetries which is also cool.

AI 科学领域的人们看到了一个机会，将他们开发的工具从广告投放或多媒体应用扩展到能够对社会产生非常积极影响的事情上，比如健康、药物开发、能源转型材料、碳捕获。这些都是非常酷、有影响力的应用。

And sort of people in the AI sciences saw an opportunity to apply the tools that they had developed beyond advertising placement or multimedia applications into something that could actually make a very positive impact in society like health, drug development, materials for the energy transition, carbon capture. These are all really cool, impactful applications.

除此之外，科学本身也非常有趣。而且我认为这两个领域正在融合，我们现在处于能够有效建模这些东西并推动一些科学方法论发展的阶段，这也是一个非常独特的时刻。人们认识到，好吧，现在我们正处于新事物的边缘，这也是公司名字的由来。我们正处于新事物的边缘，当然这总是会创造很多能量。就像，好吧，有东西了。就像一片处女地，对吧？就像没有人去过的绿地。没人去过那里，你知道吗？我可以冲进去开始收获，对吧？我认为这也是引起该领域大量热情的原因。

Besides that, the science itself is also very interesting. And I would say the fact that these two fields are coming together and that we're now at the point that we can actually model these things effectively and move the needle on some of these science methodologies is also a very unique moment, I would say. And people recognize that, okay, now we're at the cusp of something new, which is also where the company is named after. We're at the cusp of something new, and of course that always creates a lot of energy. It's like, okay, there's something. It's like a virgin field, right? It's like nobody's green field. Nobody's been there, you know? I can rush in and I can sort of start harvesting there, right? And I think that's also what's causing a lot of enthusiasm in the fields.

如果你是一名 AI 工程师，很多听这个播客的人，你可能没有很强的科学背景，但很兴奋。大多数 AI 从业者，无论是工程师还是科学家，都会认为自己算是科学家，他们有一些背景，一点物理，一点大学课程，甚至研究生院，一直在工作或刚刚开始。那么一个不是日常科学家的人，如何参与进来？

If you're an AI engineer, many of the people that listen to this podcast, and you maybe don't have a strong science background, but are excited. Most AI practitioners, be engineers or scientists, would consider themselves scientists and they have some background, a little bit of physics, a little bit of college, maybe even graduate school, that have been working or are starting out. How does somebody who is not a scientist on a day-to-day basis, how do they get involved?

嗯，他们可以等我的书出版后读一读。但这基本上是说还有更多。我们应该在这个接口上创建课程。所以我不确定是否已经有大学开设了实际课程，也许你可以上在线课程。我们现在所在的这些研讨会实际上也很好。我们可能应该在研讨会开始前提供更多教程。实际上我在某个时候提出过这个建议。就像也许先有一个小时的教程，这样人们就可以进入这个领域。

Well, they can read my book once it's out. But this is basically saying that there is more. We should create curricula that are on this interface. So I'm not sure there is possibly already some universities actual courses you can take, maybe online courses you can take. These workshops where we are now are actually very good as well. And we should probably have more tutorials before the workshop starts. Actually I've kind of proposed this at some point. And it's like maybe first have an hour of a tutorial so that people can get new into the field.

但是的，外面有很多资源。当然大部分是难以获取的，但我想说我们会创作更多书籍和其他更易于获取的内容，包括这个播客。

But yeah, there's a lot out there. Most of it is of course inaccessible, but I would say we will create much more books and other content that is more accessible, including this podcast, I would say.

我想你知道它会来的，而且现在你可以看视频之类的，有大量的内容你可以去看。

I think you know it will come and you know these days you can watch videos and things there's a huge amount of content you can go and see.

那么也许对此有一个后续问题：人们如何学习和参与？但为什么他们应该参与？我的意思是，我们有很多观众会对 AI 工程感兴趣，但他们可能正在寻找对世界更大的影响。AI for science 为他们提供了什么机会来产生影响、改变世界，而在纯比特世界中工作所不能提供的？

So maybe a follow-up to that: how do people learn and get involved? But why should they get involved? I mean, we have a lot of people who are of our audience will be interested in AI engineering, but they may be looking for bigger impacts in the world. What opportunities does AI for science provide them to make an impact, to change the world, that working in the world of pure bits would not?

所以我的观点是，几乎一切的基础都是材料。我们现在非常关注 LLM，这算是软件层。但我想说，如果你深入思考，一切的基础都是材料。所以 LLM 的基础是 GPU，而 GPU 的基础是晶圆，我们必须在上面沉积材料。

So my view is that underlying almost everything is a material. So we're focusing a lot on LLMs now, which is kind of the software layer. But I would say if you think very hard, underlying everything is a material. So underlying an LLM is a GPU, and underlying a GPU is a wafer on which we will have to deposit materials.

我们要不要稍等一下？

Do we want to wait a little bit?

一切的基础都是材料。所以我说有 LLM，LLM 的基础是运行它的 GPU，然后为了制造那个 GPU，你必须将材料沉积在晶圆上，并用 EUV 光照射以蚀刻出结构。但现在这实际上是一个材料问题，因为我们或多或少已经达到了缩小尺寸的极限，现在我们正试图通过新材料进一步改进。所以这是一个基本的材料问题。

Underlying everything is a material. So I was saying there's the LLM, underlying the LLM is a GPU on which it runs, and then in order to make that GPU you have to put materials down on a wafer and shine on it with EUV light in order to etch the structures in. But that's now an actual material problem because more or less we've reached the limits of scaling things down, and now we are trying to improve further by new materials. So that's a fundamental materials problem.

如果我们不想搞乱这个世界，我们需要快速完成能源转型。所以例如有电池，那完全是一个材料问题，对吧？有燃料电池，有太阳能板。所以现在他们可以制造在硅层之上带有新钙钛矿层的太阳能板，理论上可以捕获高达 50% 的光，而现在我们大概只有 22% 左右，对吧？所以这些都是材料创新带来的巨大变化。而且是的，我认为无论你走到哪里，我可能都能深入挖掘并告诉你，实际上你所做事情的基础是一个材料问题。所以我认为在这个非常非常基础的问题上工作非常好。而且我认为现在可能正在发生的事情是我们可以开始搜索这个材料空间。这从来不是这样的情况，对吧？就像科学家，正常的工作方式是读论文，然后提出假设，做实验并学习等等。所以这是一个非常缓慢的过程。

We need to get through the energy transition fast if we don't want to mess up this world. And so there is for instance batteries. That's a complete materials problem, right? There's fuel cells. There is solar panels. So they can now make solar panels with new perovskite layers on top of the silicon layers that can capture, you know, theoretically up to 50% of the light where now we're at I don't know maybe 22 or something, right? So these are huge changes all by material innovation. And yeah, I think wherever you go, I can probably dig deep enough and then tell you well actually the very foundation of what you're doing is a material problem. And so I think it's just very nice to work on this very very foundation. And also because I think this is maybe also something that's happening now is we can start to search this material space. This has never been the case, right? It's like scientists, the normal way of working is you read papers and then you come up with a hypothesis, you do an experiment and you learn etc. So there's a very slow process.

现在我们可以把这当作一个搜索引擎。就像我们搜索互联网一样，我们现在搜索所有可能分子的空间——不仅是人们已经制造出来的或宇宙中存在的分子，而是所有分子。

Now we can treat this as a search engine. Like we search the internet, we now search the space of all possible molecules—not just the ones that people have made or that are in the universe, but all of them.

对。我们可以让这个过程完全自动化。这就是希望所在，对吧？你只需输入——它变成一个工具，你输入你想要的东西，然后某个东西开始运转，一些实验就开始进行。

Right. And we can make this fully automated. That's the hope, right? You can just type—it becomes a tool where you type what you want and something starts spinning and some experiments get going.

对。然后结果就是一份材料清单。你看着它说，也许不行，然后稍微改进你的查询。你就像用这个搜索引擎做研究，大量的计算和实验在远处的某个实验室或数据中心之类的地方进行。我觉得这是一个非常非常有前景的视角，让我们能够在几乎一切事物之下构建一个更好的材料层——以及更可持续的材料。或者塑料正在污染地球。如果你能发明一种塑料，在几周后自我降解，实际上变成肥料——这些根本不是不可能的事情。这些事情是可以做到的，对吧，我们应该去做。

Right. And then you know, outcome: a list of materials. And then you look at it and say, maybe not, and you refine your query a little bit. And you kind of do research with this search engine where a huge amount of computation and experimentation is happening somewhere far away in some lab or some data center or something like this. I find this a very, very promising view of how we can build a much better materials layer underneath almost everything—and also more sustainable materials. Or plastics are polluting the planet. If you can come up with a plastic that kind of destroys itself after, I don't know, a few weeks, right, and actually becomes a fertilizer—these are things that are not impossible at all. These things can be done, right, and we should do it.

你能大致给我们讲讲 CuspAI 吗？我们——我有很多问题。

Can you tell us a little bit just generally about CuspAI? And we—I have tons of questions.

是的。CuspAI 大约成立于 20 个月前，起因是我担心——现在仍然担心——气候变化。我意识到，为了将升温控制在两度以内，我们不仅要在 2050 年前将排放降至零，还需要再花半个世纪甚至一个世纪从大气中去除二氧化碳——不是通过减少排放，而是以我们目前排放速度的大约一半来实际去除。这是一个未解决的问题。如果我们不解决它，2°C 的目标就无法实现——升温会高得多。我认为人们并不完全理解那会有多糟糕。比如四度——非常糟糕。所以这项技术需要被开发出来。这就是我和我的联合创始人 Chad Edwards 创办这家初创公司的动机。同时我们也看到技术已经成熟，这也很好。所以时机正好。现在我们已经发展到大约 40 人，公司获得了 1.3 亿美元投资，对一家欧洲公司来说这相当多。有趣的是，在那之后其他初创公司获得了更多投资，这说明了这个领域发展有多快。但我们现在——当然我们已经构建了平台，但它是针对一系列材料类别的，需要不断扩展到新的材料类别。而且它可以更加自动化，因为我们知道引入 LLM 等，整个流程会越来越自动化。现在我们正转向高通量实验——将计算平台与实际实验连接起来，这样你也能从实验中获得快速反馈。我有点把实验看作是在最后才做的事情。尽管到目前为止我们一直是这么做的。我想把它看作我称之为物理处理单元的东西，就像 PPU，对吧？你有数字处理单元，然后你有物理处理单元。所以基本上是自然在为你做计算。它甚至是已知最快的计算机。它有点难编程，因为你必须做所有这些实验。它也相当笨重。就像你必须做的一件非常大的事情。但在某种程度上它就是一种计算，这就是我想看待它的方式。所以你可以在数据中心做计算，然后你可以让自然做一些计算。你与自然的接口有点复杂，但这些事情必须无缝协作才能得到你感兴趣的新材料。这就是我们的愿景。你不说超级智能，因为我不太清楚它是什么意思，我也不想过度推销，但我确实想自动化这个过程，并把一个非常强大的工具交到化学家和材料科学家手中。

Yeah. So CuspAI started about 20 months ago, and it was because I was worried about—I'm still worried about—climate change. And so I realized that in order to stay within two degrees, let's say, we would not only have to reduce our emissions to zero by 2050, but then another half century or even a century of removing carbon dioxide from the atmosphere—not by reducing your emissions but actually removing it at a rate that's about half the rate that we now emit it. And that is an unsolved problem. And if we don't solve it, 2°C is not going to happen—it's going to be much more. And I don't think people quite understand how bad that can be. Like four degrees—like very bad. So this technology needs to be developed. And so this was my and my co-founder Chad Edwards' motivation to start this startup. And also because we saw the technology was ready, which is also very good. So the time is right to do it. And yeah, so we now—in the meanwhile we've grown to about 40 people, we've collected 130 million investment into the company, which for a European company is quite a lot, I would say. It's interesting that right after that, other startups got even more, so that tells you how fast this is growing. But yeah, we are now at the—so we've built the platform of course, but it's for a series of material classes and it needs to be constantly expanded to new material classes. And it can be more automated because we know putting LLMs in and so the whole thing gets more and more automated. And now we're moving to high-throughput experimentation—so connecting the actual platform, which is computational, to the experiments so that you can also get fast feedback from experiments. And I kind of think of experiments as something you do at the end. Although that's what we've been doing so far. I want to think of it as what I would call a sort of physics processing unit, like a PPU, right? Which is you have digital processing units and then you have physics processing units. So it's basically nature doing computations for you. It's the fastest computer known possible even. It's a bit hard to program because you have to do all these experiments. It's also quite bulky. It's like a very large thing you have to do. But in a way it is a computation, and that's the way I want to see it. So you can do computations in a data center and then you can ask nature to do some computations. Your interface with nature is a bit more complicated, but then these things will have to seamlessly work together to get to a new material that you're interested in. And that's the vision we have. You don't say superintelligence because I don't quite know what it means and I don't want to oversell it, but I do want to automate this process and give a very powerful tool in the hands of the chemist and the material scientist.

这正好引出了我想问你的一个问题。首先，你能谈谈你的平台吗，不管到什么程度——比如解释一下它是如何工作的，以及你在开发它时的思路是什么？

That actually brings up a question I wanted to ask you. First of all, can you talk about your platform to whatever degree—like explain kind of how it works and what your thought process was in developing it?

是的，实际上它出乎意料地——这不是火箭科学。是的，我可以说在设计意义上这不是火箭科学。基本上我一开始写下的设计——现在或多或少还是那个设计。尽管你添加了一些东西——我一开始没怎么考虑多尺度模型，后来我们意识到多尺度实际上非常重要。一开始我没怎么考虑自驱动实验室，但现在我认为我们已经到了应该添加这个的阶段。所以有一些细节我们在添加，但或多或少就是你在幻灯片中看到的：有一个生成组件，你需要训练它来生成候选材料，然后有一个数字孪生——多尺度、多保真度的数字孪生——你沿着阶梯一步步走。你知道，先做便宜的事情，剔除所有明显无用的东西，然后再做越来越昂贵的事情。这样你把范围缩小到少量候选，它们进入实验，做实验，获得反馈，等等。最近还添加了更多智能体式的部分。我们有智能体搜索文献并提出——实际上是化学文献——并提出化学建议或实验建议。我们有智能体自主协调所有需要进行的计算和实验。它们处于不同的成熟阶段，可以不断改进。所以基本上——我不认为那部分是火箭科学，设计并不令人惊讶，但实际构建它却出奇地难。所以这就是——护城河在于你能获得的数据以及实际构建平台。我想特别提到两个人：Felix Hunker，他正在构建平台的科学部分；以及 Alessandro de Maria，他正在构建 SCA——平台的 MLOps 部分。

Yeah, actually it's been surprisingly—it's not rocket science. Yeah, I would say it's not rocket science in the sense of the design. And basically the design that I wrote down at the very beginning—it's still more or less the design. Although you add things like—I wasn't thinking very much about multiscale models and it came on our radar that actually multiscale is very important. In the beginning I wasn't thinking very much about self-driving labs, but now I think we are now at the stage we should be adding that. So there are bits and details that we're adding, but more or less it's what you see in the slide decks here as well, which is there's a generative component that you have to train to generate candidates, and then there is a digital twin—multiscale, multifidelity digital twin—which you walk through the steps of the ladder. You know, they do the cheap things first, you weed out everything that's obviously unuseful, and then you go to more and more expensive things later. And so you narrow things down to a small number; those go into an experiment, do the experiment, get feedback, etc. Now, things that also have been more recently added is sort of more agentic parts. We have agents that search the literature and come up with—actually the chemical literature—and come up with chemical suggestions or for doing experiments. We have agents which sort of autonomously orchestrate all of the computations and the experiments that need to be done. They're in various stages of maturity and they can be continuously improved, I would say. And so that's basically—I don't think that part is rocket science, you know, the design of that thing is not like surprising, but it is surprisingly hard to actually build it right. So that's the thing—where the moat is in the data that you can get your hands on and actually building the platform. And I would say there's two people in particular I want to call out, which is Felix Hunker, who is actually building the scientific part of the platform, and Alessandro de Maria, who is building the sort of the SCA—the MLOps part of the platform.

最近我们还把 Aaron Walsh 加入了团队，他是来自帝国理工学院的非常杰出的科学家。我们对此非常高兴。他将担任我们的首席科学官。我们还有一个合作团队，负责寻找所有客户，因为我认为在原则上非常重要的一点是：将材料真正带入现实世界非常复杂，你必须与领域专家合作，而这些专家通常是公司。因此，只有当我们找到优秀的工业合作伙伴与我们同行时，我们才会开始投资某个方向。

And recently we also added Aaron Walsh to our team, who is a very accomplished scientist from Imperial College. We're very happy about that. He's going to be our chief science officer. And we also have a partnerships team that seeks out all the customers, because I think this is one thing I find very important in principle: it's so complex to actually bring a material to the real world that you must do this in collaboration with the domain experts, which are typically the companies. So we only start to invest in a direction if we find a good industrial partner to go on that journey with us.

在平台演进过程中这非常合理。你是否发现人类干预——我想你可以从纯……你可以想象两个方向。一是，你一开始让一切纯自动、自动化、智能体式，然后后来发现需要在不同步骤有更多人类输入和反馈。或者也许你一开始在很多步骤有人类反馈，然后想办法去除……

Makes a lot of sense over the evolution of the platform. Did you find that human intervention—I guess you could start out with a pure... you could imagine two directions. One, you start out making everything purely automatic, automated, agentic, and then later on you find that you need to have more human input and feedback at different steps. Or maybe did you start out with having human feedback at lots of steps and then figure out ways to remove...

是第二种。所以你构建工具。它比你想象的模块化得多。但就像我们需要这些工具用于这个应用。我们需要这些工具。所以你构建所有这些工具，然后你走一个工作流，其实一开始只是手动。所以你把它们——先这个工具，然后运行这个工具，然后运行那个。所以你把它们放进一个工作流。然后你发现，哦实际上，你知道，我们试图制造的这种多孔材料，如果你摇一摇它实际上会坍塌。好吧。然后你添加一个新工具，说测试稳定性。对。所以工具越来越多，然后你构建智能体，它可以是贝叶斯优化器，或者可以是实际的 LLM，也许训练成优秀的化学家，然后开始以正确的方式和正确的顺序使用所有这些工具。

That's the second one. So you build tools. So it's much more modular than you think. But it's like we need these tools for this application. We need these tools. So you build all these tools and then you go through a workflow, actually in the beginning just manually. So you put them—first this tool, then run this tool, then run this. So you put them in a workflow. And then you figure out, oh actually, you know, this porous material that we're trying to make actually collapses if you shake it a bit. Okay. Then you add a new tool that says test for stability. Right. And so there's more and more tools, and then you build the agent, which could be a Bayesian optimizer or it could be an actual LLM, maybe trained to be a good chemist, that will then start to use all these tools in the right way and the right order.

是的。

Yeah.

对。但一开始，你作为化学家把工作流组合起来，然后你想，好，我如何自动化这个？对。一个非常简单的问题你可以问自己：每次有人不是 DFT 超级专家，想做计算，必须去找懂 DFT 的人。所以你能不能开始自动化掉这个？也就是，让它变得用户友好，这样你实际上为正确的问题、正确的时间长度做正确的 DFT，并且你能评估结果是否良好等等。所以你开始自动化越来越小的和更大的部分，等等。最终整个事情都自动化了。

Right. But in the beginning it's like you as a chemist are putting the workflow together, and then you think about, okay, how am I going to automate this? Right. One very easy question you can ask yourself is, you know, every time somebody who is not a super expert in DFT wants to do a calculation, has to go to somebody who knows DFT. And so could you start to automate that away? Which is like, okay, make it so user-friendly so that you actually do the right DFT for the right problem and for the right length of time, and you can actually assess whether it's a good outcome, etc. So you start to automate smaller and bigger pieces, etc. And in the end the whole thing is automated.

所以你的理念是提供一套特定工具，让做决策的科学家更知情，而不是试图创建一个自动化流程。

So your philosophy is you want to provide a set of specific tools that make it so that the scientists making decisions are better informed, and less so trying to create an automated process.

我认为这和你说的是一回事，因为是的我们想要自动化。

I think it's the same as what you're saying, because yes we want to automate.

是的。

Yeah.

但我们不认为很快会出现化学家和领域专家脱离循环的情况。

But we don't see something very soon where the chemists and the domain expert is out of the loop.

是的，但这是一个退却，对吧？就像，好，首先你需要专家告诉你精确如何设置 DFT 计算的参数。好，也许我们可以去掉那个。我们也许可以自动化它，对吧？所以越来越多这些东西会被移除。

Yeah, but it's a retreat, right? It's like, okay, so first you need an expert to tell you precisely how to set the parameters of the DFT calculation. Okay, maybe we can take that out. We can maybe automate it, right? And so increasingly more of these things are going to be removed.

是的。最终愿景是它会成为一个搜索引擎，有人，一个化学家，输入东西，我们会得到候选列表，但化学家仍然会决定列表中哪个是好材料，哪个不是好材料，对吧。

Yeah. In the end the vision is it will be a search engine where somebody, a chemist, will type things and we'll get a list of candidates, but the chemist will still decide what is a good material and what is not a good material out of that list, right.

所以完全黑暗实验室的愿景，你可以关上门，你只说，你知道找点有趣的东西，然后它就会找出什么有趣，我们会发现，你知道，就像哦我发现了这个新材料 blah blah blah blah 对，那不是我的愿景，至少很长时间内不是。

And so the vision of a completely dark lab where you can close the door and you just say, just you know find something interesting and then it will just figure out what's interesting and we'll figure out, you know, it's like oh I found this new material to blah blah blah blah right that's not the vision I have at least not for a long time.

所以对我来说，这真的是赋能公司和大学里的领域专家，让他们更快地开发材料。我应该说，有时保持一点谦逊也是好的。

So for me it's really empowering the domain experts that are sitting in the companies and in the universities to be much faster in developing their materials. And I should say it's also good to be a little humble at times.

是的。

Yeah.

对。就像我怀疑他们会不会挠头说，嗯，你知道，你怎么可能在接下来 5 年完全自动化掉那个？我完全不认为那会发生。嗯，是的。所以对我来说，它是化学家手中越来越强大的工具。

Right. And it's like I wonder if they scratch their head and say, well, you know, how are you going to completely automate that away like in the next 5 years? I don't think that's going to happen at all. Um, yeah. So to me, it's an increasingly powerful tool in the hands of the chemists.

我有个问题。你之前谈过通过材料的大突破还是渐进变化来引起人们兴趣。我好奇你对你现在拥有并正在迈向的平台怎么看，以及你如何追求大变化？或者这是渐进的？或者它们显然不是互斥的，但你怎么看？

I have a question. You've talked before about getting people interested based on having a big breakthrough in materials versus just incremental change. I'm curious what you think about the platform you have now and are stepping towards, and how are you chasing the big change? Or is this incremental? Or are they not mutually exclusive, obviously, but what do you think about that?

我们遵循混合策略。所以我们肯定在追求一种大材料。再次，我们与合作伙伴一起做。我不会透露具体是什么，但我们有自己的长期目标。你可以称之为灯塔或登月计划或其他，但它将是一种真正有影响力的材料，我们想开发它作为证明点，证明它可以做到，它将进入现实世界，并且 AI 在使其发生中至关重要。同时，我们也非常乐意与目标更 modest 的公司合作。比如我会说一个是深度合作，你与一家公司一起踏上旅程，那是长期的承诺；另一个是有人说我需要一个力场。你能帮我训练这个力场，然后也许为我分析这个特定问题吗？我会为此付你一大笔钱，然后也许之后我们再看。那也可以，对吧？但我们更喜欢深度合作，我们可以真正做出好的改变。

We follow a mixed strategy. So we are definitely going after a big material. Again, we do this with a partner. I'm not going to disclose precisely what it is, but we have our own long-term goal. You could call it a lighthouse or a moonshot or whatever, but it is going to be a really impactful material that we want to develop as a proof point that it can be done and that it will make it into the real world and that AI was essential in actually making it happen. At the same time, we also are quite happy to work with companies that have more modest goals. Like I would say one is a very deep partnership where you go on a journey with a company and that's a long-term commitment together, and the other one is like somebody says I need a force field. Can you help me train this force field and then maybe analyze this particular problem for me? And I'll pay you a bunch of money for that and then maybe after that we will see. And that's fine too, right? But we prefer the deep partnerships where we can really change something for the good.

是的。

Yeah.

从平台的角度看，你觉得你们准备好了吗？或者说，有哪些事情——再说一次，我不是要你透露专有的独门秘方——但总的来说，从现在到实现那种重大突破，还需要发生哪些事情？

And do you feel like from a platform standpoint, you're ready for that? Or what are the things that — and again, not asking you to disclose proprietary secret sauce — but what are the things, generally speaking, that need to happen from where we are to get those big breakthroughs like that?

我觉得这个领域有意思的地方在于，你每做出一样东西，它其实立刻就有用，对吧？不像量子计算或者核聚变，你得干上不知道 20、30、40 年，什么都没有，什么都没有，然后它必须得发生。

What I find interesting about this field is that every time you build something, it's actually immediately useful, right? And so unlike quantum computing or nuclear fusion, where you work for I don't know 20, 30, 40 years and nothing, nothing, nothing, and then it has to happen.

是啊。

Yeah.

对。而它一旦发生，就是巨大的。所以这里很不一样，因为每次你引入——你去找客户，问他们你需要什么？对吧。好。比如说我们做一个像水过滤这样的问题，我们想从水里去除 PFAS，对吧？我们和一家叫 Chimera 的公司合作，他们是我们的深度合作伙伴，对吧？所以我们一起走一段旅程。我认为突破会在大量人类参与的情况下发生，因为化学家对他们领域的了解要多得多，而我们是帮他们做 AI 训练和新方法的人。在这种接口、这种互动中，会诞生美好的东西。而且我认为，这个领域真正腾飞之前，必须先发生这件事。所以从这个意义上说，它不是泡沫，这么说吧。人们看到的是真实的、正在发生的事情。所以一开始会非常依赖人类参与，我会这么说。而且我希望在一切完全自动化之前，我们就能得到这种新的突破性材料，因为完全自动化还需要一段时间。另外它非常垂直特定。也就是说，把问题 A 完全自动化——你也许能做到，但到了问题 B 你就得从头再来，因为你的实验装置看起来非常不同，你用来表征材料的机器看起来非常不同。甚至你平台里的模型也得重新训练、微调，去适应新的类别。所以每次你都有很多经验要迁移，但问题本身其实也不一样。所以是的，我希望在完全自动化之前先有那种突破性材料，我认为那是一种长期愿景。而且我会说，每次你转向新东西，都得重新训练，人类又得介入，想一想，好，这个问题长什么样？然后再把机器指向新的方向，然后使用它。

Right. And when it happens, it's huge. So it's quite different here, because every time you introduce — so you go to a customer and you say, so what do you need? Right. Okay. So we work, let's say, on a problem like water filtration. We want to remove PFAS from water, right? So we do this with a company, Chimera. So they are a deep partner for us, right? So we're on a journey together. I think that the breakthrough will happen with a lot of human in the loop, because there is the chemists who have a whole lot more knowledge of their field, and it's us who will help them with AI training and new methods. And in that kind of interface, these interactions, something beautiful will happen. And that will have to happen first before this field will really take off, I think. And so in that sense, it's not a bubble, let's put it that way. So people see that's actual, real, what's happening. So in the beginning it will be very, you know, with a lot of humans in the loop, I would say. And I would hope we will have this new sort of breakthrough material before everything is completely automated, because that will take a while. And also it is very vertical-specific. So it's like completely automating something for problem A — you know, you can probably achieve it, but then you'll have to start over again for problem B, because your experimental setup looks very different. The machines that you characterize your materials with look very different. Even the models in your platform will have to be retrained and fine-tuned to the new class. So every time, you know, you have a lot of learnings to transfer, but also the problems are actually different. And so yes, I would want that breakthrough material before it's completely automated, which I think is kind of a long-term vision. And I would say every time you move to something new, you'll have to start retraining, and humans will have to come in again and sort of go, okay, so what does this problem look like? And now point the machine again in the new direction, and then use it.

对于我们这些非科学家——包括我自己，勉强算个科学家——有太多术语了。你提到了 DFT，我们聊过超材料。你能用工程术语，或者说以工程师能理解的深度，解释一下——到底什么是等变性？

And for the non-scientists among us — me included, a bit of a scientist — there's a lot of terminology. You mentioned DFT. We've talked about metamaterials. Can you sort of explain in engineering terms, or at the level of sophistication of an engineer — well, what is equivariance?

等变性就是把对称性注入神经网络。比如我建一个神经网络，假设它要识别这个瓶子，对吧？然后我把瓶子旋转一下，它实际上就得完全重新开始，因为它不知道旋转后的瓶子——其实代表旋转后瓶子的输入，就是一个旋转后的瓶子。它就是不明白这一点。而如果你把等变性建进去，基本上你在一个方向上训练好之后，它就能理解任何其他方向。这意味着训练这些模型需要的数据少得多。这些是对模型权重的约束。所以基本上你得约束权重，让它理解这一点，你可以把它建进去，可以硬编码进去。而且，对称群可以是平移、旋转，也可以是置换，比如在图神经网络里就有置换，而在物理里当然还有更多这类群。

So equivariance is the infusion of symmetry in neural networks. So if I build a neural network, let's say that needs to recognize this bottle, right? And then I rotate the bottle, it will then actually have to completely start again because it has no idea that the rotated bottle — well, actually the input that represents a rotated bottle is actually a rotated bottle. It just doesn't understand that. Whereas if you build equivariance in, basically once you've trained it in one orientation, it will understand it in any other orientation. So that means you need a lot less data to train these models. And these are constraints on the weights of the model. So basically you have to constrain the weights such that it understands it, and you can build it in, you can hardcode it in. And yeah, the symmetry groups can be translations, rotations, but also permutations, like in graph neural networks there are permutations, and in physics of course there's many more of these groups.

唱个反调，为什么不直接用数据增强——让你的模型过一遍所有不同的方向？

To play devil's advocate, why not just use data augmentation — put your model through all the different orientations?

作为一种选项，它只是不精确。你为什么要费那个劲去做这些，你其实需要无限多的增强才能完全正确，而你也可以直接硬编码进去？不过我得说，有时候数据增强实际上比硬编码等变性效果更好。这和以下事实有关：如果你在优化开始前就约束权重，优化曲面或者说目标会变得更复杂，因此更难找到好的极小值。所以我认为优化过程和你在网络里加入的约束之间也有复杂的相互作用。所以是的，在这个领域你会听到互相矛盾的说法。比如有些人说，对某些应用，这样做就是比不做好。而有时你又会听到另一些人说：如果你有大量数据，能做数据增强，那其实优化起来更容易，效果也比把等变性加进去更好。

As an option, it's just not exact. Why would you go through the work of doing all that, where you would really need an infinite number of augmentations to get it completely right, where you can also hardcode it in? Now I have to say, sometimes actually data augmentation works even better than hardcoding the equivariance in. And this is something to do with the fact that if you constrain the weights before the optimization starts, the optimization surface or objective becomes more complicated, and so it's harder to find good minima. So there is also a complicated interplay, I think, between the optimization process and these constraints you put in your network. And so yeah, you'll hear kind of contradicting claims in this field. Like some people — for certain applications it works just better than not doing it. And sometimes you hear other people: if you have a lot of data and you can do data augmentation, then actually it's easier to optimize them, and it actually works better than putting the equivariance in.

你觉得对于有数学基础的模型和深度学习的策略来说，存在某种“苦涩的教训”吗？

Do you think there's kind of a bitter lesson for mathematically founded models and strategies for doing deep learning?

是的，归根结底这是数据和归纳偏置之间的权衡。

Yeah, ultimately it's a trade-off between data and inductive bias.

是的。

Yes.

所以如果你的归纳偏置不是完全正确，你就得小心，因为你会给自己设一个天花板。但如果你知道对称性就在那里，很难想象没有办法真正利用它。但是的，所以确实有苦涩的教训。其中一个苦涩的教训是，你永远应该确保把规模架构好，除非你的数据集很小，那种情况下就无所谓。但如果你——你知道，在 LLM 领域能得出的同样的苦涩教训，我认为最终在这个领域也会成立。

So if your inductive bias is not perfectly correct, you have to be careful, because you put a ceiling to what you can do. But if you know the symmetry is there, it's hard to imagine there isn't a way to actually leverage it. But yeah, so there is a bitter lesson. One of the bitter lessons is you should always make sure you architect your scale, unless you have a tiny data set, in which case it doesn't matter. But if you — you know, the same bitter lessons, or lessons that you can draw in LLM space, are eventually going to be true in this space as well, I think.

你能聊聊你即将出版的书，告诉听众它有什么令人兴奋的地方吗？

Can you talk a little bit about your upcoming book and tell the listeners what's exciting about it?

是的，他们应该读一读。这本书讲的是——书名叫《生成式 AI 与随机热力学》。它基本上揭示了一个事实：生成式 AI——也就是生成图像和视频的技术——所用的数学，和非平衡统计力学这个领域——研究分子系统四处运动、弛豫到基态，或者你可以控制它处于某个状态——这两者的数学其实是完全相同的。这很迷人。而事实上，有意思的是，Jeff Hinton 和 Radford Neal 很久以前就已经为机器学习写下了变分自由能。

Yeah, they should read it. So this book is about — it's called Generative AI and Stochastic Thermodynamics. It basically lays bare the fact that the mathematics that goes into both generative AI, which is the technology to generate images and videos, and this field of non-equilibrium statistical mechanics, which is systems of molecules that are just moving around and relaxing to their ground state, or that you can control to be in a certain state — the mathematics of these two is actually identical. And so that's fascinating. And in fact, what's interesting is that Jeff Hinton and Radford Neal already wrote down the variational free energy for machine learning a long time ago.

还有 Karl Friston 关于自由能原理和主动推理的工作。但现在我们把它和物理学中一个非常新的领域联系了起来，叫做随机热力学或非平衡态热力学，它有自己的非常有趣的定理，比如涨落定理，我们通常不怎么谈这些，但可以从中获益良多。我认为现在可以开始交叉融合了：当我们看到这些东西其实是同一回事，就可以像我们对待对称性那样，去看这些非常聪明的物理学家发展出的新理论，然后说，好，我们能从中借鉴什么，让我们的算法更好？同时，我们也可以用自己的模型帮助科学家做更好的科学，对吧？于是这就成了两个领域之间美妙的交叉融合。这本书相当技术性，我会这么说，它把随机热力学里做过的各种东西和机器学习文献里做过的各种模型都拿来，基本上把它们一一对应起来。我希望这种统一感对人们会有启发。

And there's also Karl Friston's work on the free energy principle and active inference. But now we've related it to this very new field in physics, which is called stochastic thermodynamics or non-equilibrium thermodynamics, which has its own very interesting theorems, like fluctuation theorems, which we don't typically talk about but we can learn a lot from. And I think it can now start to cross-fertilize: when we see that these things are actually the same, we can, like we did for symmetries, now look at this new theory that's out there, developed by these very smart physicists, and say, okay, what can we take from here that will make our algorithms better? At the same time, we can use our models to help the scientists do better science, right? And so it becomes a beautiful cross-fertilization between these two fields. The book is rather technical, I would say, and it takes all sorts of things that have been done in stochastic thermodynamics and all sorts of models that have been done in the machine learning literature, and it basically equates them to each other. And I hope that sense of unification will be revealing to people.

等等，那它什么时候出版？

Wait, and when is it out?

嗯，现在取决于出版社了，但我希望是四月。我要在 ICLR 做主题演讲，如果那时手里能拿着这本书就太好了，不过这类时间线很难控制。

Well, it depends on the publisher now, but I hope in April. I'm going to give a keynote at ICLR, and it would be very nice if I have this book in my hand, but it's hard to control these kinds of timelines.

是啊，我很期待。太好了。

Yeah, I'm looking forward to it. Great.

我也一样。

Likewise.

非常感谢。

Thank you very much.