AI 用于科学：神经算子与天气建模

欢迎收听 Latent Space。这里是 Latent Space 的 AI for Science 板块。我是 Brandon，从事基于 AI 和原子 AI 的 RNA 疗法研究。和我一起的是我的联合主持人 R.J. Honiki，他开发空间转录组学，也是 Mirror Omix 的 CTO 和创始人。今天，我们很高兴邀请到加州理工学院的 Bren 数学与计算机科学教授 Anima Anandkumar。Anima 在将 AI 与物理世界模型结合方面做了各种非常酷的工作，背景非常多元，我觉得我根本无法完全概括。不过，还是让 Anima 自己介绍吧。感谢你来到节目。

Welcome to Latent Space. This is the AI for science section of Latent Space. I'm Brandon. I work on RNA therapeutics using AI and atomic AI. I'm joined by my co-host R.J. Honiki who develops spatial transcriptomics and is the CTO and founder of Mirror Omix. Today we're excited to be joined by Anima Anandkumar, the Bren Professor of Mathematics and Computer Science at Caltech. Anima has done all sorts of really cool work combining AI with basically models of the physical world, and has a really diverse background. I don't think I could even remotely cover it. But anyway, I'll let Anima introduce herself. Thank you for coming on the show.

是的，谢谢你，Brandon 和 R.J.。我很高兴来到这里，我真的很喜欢“latent space”这个词，因为它在我很多工作中都占有重要地位，因为确实，世界是潜在的。不过，简单介绍一下，我已经在 AI 领域工作了二十多年，某种程度上，甚至在深度学习之前，当时很多概率模型的理论基础都需要建立起来，我参与了这些工作。随着深度学习开始起飞，我也一只脚踩在工业界，直到最近。所以我曾在 NVIDIA 领导 AI 研究，之前还在 Amazon Web Services 帮助创立了云 AI 团队，并构建了第一批云 AI 产品，那几乎是十年前的事了。所以，一只脚在工业界、一只脚在学术界，让我对如何将理论与实践结合、如何思考大规模 AI 以及有原则的 AI 有了很多有趣的视角。

Yeah, thank you Brandon and R.J. It's a pleasure to be here, and I really like the term latent space because that very much figures in a lot of my work, because it's really, you know, the world is latent. But yeah, just as a brief introduction, I've been working in AI for more than two decades, in a way, even before deep learning, when a lot of the theoretical foundations had to be built for probabilistic models. I worked on them, and then as deep learning started taking off, I also had a foot in industry until recently. So I was at NVIDIA, I led AI research there, and before that at Amazon Web Services, helped found the cloud AI team and build the first cloud AI products, almost a decade ago. So having this one foot in industry and academia has given me a lot of interesting perspective on how to bring theory and practice together and think of AI at large scale, but also AI that is principled.

你的很多工作都与使用某些类型的物理系统进行物理系统建模有关，你用微分方程对这些系统建模，并用机器学习帮助建模。所以，也许我们先从高层次谈谈这个，但稍后我们会深入讨论神经算子以及像天气这样的应用。不过首先，我真的很想听听关于 TorchLean 以及你最近的工作是如何与那个更大的研究计划联系起来的。

A lot of your work has been related to the modeling of physical systems using certain types of physical systems which you model with differential equations, and you help model them with machine learning. So maybe first let's go ahead and talk a little bit about that at a high level, but we'll get to the details about neural operators and some of the applications like weather later. But first, I'm actually really curious to hear about TorchLean and how this recent work you've been doing connects with that larger research program.

对我来说，广义上讲，我的主题是 AI 与科学，如何将它们结合起来。所以，大约十年前我在加州理工学院起步时，我的热情一直是科学、物理学，但我做的是 AI，所以如何将两者结合是第一批基础奠定之处。对我来说，这有几个方面。一是人们一直在思考如何将语言模型用于科学。是的，你可以做很多假设生成，你可以有想法，但想法是不够的。你可以有很多想法。瓶颈在于测试和验证它们在现实世界中是否有效。因此，我最近的很多关注点都在于如何确保我们能够构建出有保证能在物理世界或科学领域的任何方面工作的 AI。一种思考方式是，我们能否对物理世界建模并保持物理正确性？这就是神经算子的用武之地。另一个方面是，我们能否以符号方式验证某些方面？例如，如果我们声称一个定理是正确的，我们必须去验证它。这就是 Lean 作为形式语言在验证中的用处。那么如何将其与语言模型结合，正是许多数学推理的前沿所在。所以 TorchLean 就属于那个领域，我们说，你不仅想验证数学陈述，你可能还想验证神经网络本身声称要交付的东西。例如，如果你现在使用一个神经网络，你想问它是否稳健。假设你想在控制回路中使用神经网络，你想控制无人机或核反应堆。所以，最终，当我们构建将深度学习融入控制回路的 AI 系统时，我们需要稳健性。因此，现在 TorchLean 可以帮助我们无缝地进行这些验证。所以我们现在可以让神经网络成为验证回路的一部分，并确信我们可以适当地使用它们。

To me, broadly, my thesis is AI and science, how we bring that together. So where I started at Caltech almost a decade ago, my passion was always science, physics, but I was doing AI, so how to bring that together was where the first kind of foundations got laid. And to me, there are several aspects to that. One is people have been thinking how to use language models for science. Yes, you can do a lot of hypothesis generation, you can have ideas, but ideas are not enough. You can have a lot of ideas. The bottleneck is going testing and verifying that they work in the real world. And so this aspect is where a lot of my recent focus has been on how do we ensure that we can build AI that has guarantees that it will work in the physical world or any aspects in scientific domains. And one way to think about it is, can we model the physical world and keep the physics correct? And that's where neural operators come in. The other aspect is, can we verify symbolically certain aspects? For instance, if we claim that a theorem is correct, we have to go verify it. That's where Lean as a formal language can be useful for verification. So how do we bring that together with language models is where a lot of mathematical reasoning has been at the forefront. And so TorchLean kind of is in that realm where we say, not only that you want to verify mathematical statements, you may want to verify what neural networks themselves claim to deliver. For instance, if you're now using a neural network and you want to ask whether it's going to be robust. Say you want to use a neural network in a control loop, you want to control whether it's a drone or a nuclear reactor. So all of this ultimately, when we build AI systems with deep learning into control loops, we want robustness. And so now TorchLean can help us do those verification seamlessly. So we can now have neural networks be part of the verification loop and have confidence that we can use them appropriately.

我们已经在播客中讨论过 Lean，大家可能对神经网络也很熟悉。但神经网络似乎非常不受约束。你所说的证明是什么类型的？你是在说输出、输入的界限吗？你能用 TorchLean 证明什么？

We have already discussed on the podcast Lean, and everyone should probably be familiar with neural networks. But neural networks seem very unconstrained. What kinds of proofs are you talking about? Are you talking about bounds on the outputs, inputs? What can you prove with TorchLean?

是的，所以 TorchLean 是一个整体框架。它真正实现的是，你现在基本上可以用 Lean 编写神经网络。所以，不是用 PyTorch 写，而是像 PyTorch 那样的抽象，但你可以用 Lean 写，因此它可以在 Lean 中完全形式化。然后有几种实现。我们有用于认证稳健性的算法，比如 CROWN，这些算法都在这个框架下实现。

Yeah, so TorchLean is an overall framework. What it really enables is that you can now write neural networks essentially in Lean. So instead of writing in like PyTorch, it's like a PyTorch-like abstraction, but you can write it in Lean, and so it can be fully formalized in Lean. And then there are several implementations. We have algorithms for certified robustness like CROWN, and those are implemented under this framework.

抱歉，是像什么，像 CROWN 吗？

Sorry, was it like what, like CROWN?

CROWN，CROWN 是其中之一……所以有不同的方法来界定，对于认证稳健性，这些界限可以有多紧。

CROWN, CROWN is one of the... so there are different ways to bound, for certified robustness, how tight those bounds can be.

所以我们开始寻找有趣的例子，其中之一就是天气建模，因为天气数据是开源的。既然数据在那里，我们就想，好吧，让我们去试试。这就是它的美妙之处，只要有数据，就是好消息。但当时很多气象科学家确实警告过我们，那是 2021 年，他们说，不，不，不，这太难了。传统天气预报已经发展了几十年，那是非常谨慎的、完全自下而上的基于物理的建模，对吧？所以假设，哦，这是流体动力学，你能预测第二天的天气等等。所以很多想法是，AI 就是无法击败天气建模几十年的工作。但令我们惊讶的是，我们直接去做了，我们训练了模型，我们使用神经算子来有效地捕捉现象，然后我们发现它不仅准确，几乎接近传统天气模型能准确做到的程度，而且快了几万倍。所以需要大型超级计算机运行的东西现在可以运行了，我们只需要一个消费级 GPU。这是一个小模型，它非常适合。它非常快，而且准确。我认为这改变了每个人的想法。

So we set out looking for interesting examples, and one of them was weather modeling, because the weather data is open source. And so given that the data was there, we were like, okay, let's just go try it. And that's the beauty of it, whenever data is available, it's really good news. But a lot of weather scientists did caution us back then, this is back in '21, and they said, no, no, no, this is so difficult. There have been decades of development in traditional weather forecasting, and that's very careful, full bottom-up physics-based modeling, right? So assuming, oh, this is the fluid dynamics, can you go predict the weather the next day and so on. And so that's how a lot of the thinking was, that AI is just not going to be able to beat the decades of work in weather modeling. But to our surprise, we just went ahead, we trained them, we used neural operators to be able to effectively capture the phenomena, and then we found that it's not only accurate, it's almost as close to what the traditional weather models can do accurately, but also tens of thousands of times faster. So what would take a big supercomputer to run can now be run, and we only needed a consumer-grade GPU. It was a small model, it fit very well. It's very fast and it's accurate. And I think that just changed everybody's thinking.

这取决于松弛技术，不深入细节的话，有很多这样的算法。我们正在 Lean 中实现它们并使其可用，这样我们就可以无缝地运行两者。我们可以先用类似 PyTorch 的框架非常简单地写出神经网络，然后我们也可以对它们进行形式化陈述并验证它们。所以所有这些都可以整合到一个框架中。

It depends on the relaxation techniques, and without going into those, there are many such algorithms. But we are implementing them and enabling them in Lean so we can seamlessly run both. We can first write down PyTorch-like framework neural networks very simply, and then we can also make statements about them formally and verify them. So all of that can be brought together in one framework.

那么你能给出的一个界限的例子是什么？比如我们在运行核反应堆，我们不希望它熔毁。你能在输入和输出上提供什么样的保证来帮助避免熔毁？

So what's an example of a bound that you could claim? Like we're operating a nuclear reactor, we don't want it to melt down. What are the sort of guarantees that you could provide on the inputs and outputs that would help that not melt down?

最自然的是我提到的认证鲁棒性。也就是说，如果我们的输入被扰动了一定程度，输出会被扰动多少？这也可以称为敏感性分析。对不同神经架构拥有这些界限，你可以自动获得这些界限，这不仅能帮助我们在控制回路中训练神经网络表现良好，还能关注安全性、鲁棒性和稳定性。这些都是人们关心的控制系统的一部分。所以这是一个应用例子。更广泛地说，这个想法是在许多涉及神经网络的场景中都需要验证。控制回路是一个。另一个例子是我们使用物理信息神经网络来求解偏微分方程或提出保证满足特定物理定律的系统。但我们也想验证，例如，我们的神经网络仅在有限精度下训练。我们能克服这些要求吗？当我们使用有限精度时会发生什么？有什么缺点？我们也能对这些进行界限约束吗？所以这些是 TorchLean 中有效的其他类型的界限。因此，关于精度的影响、扰动的影响的所有方面，我们现在都可以在 Lean 中实现算法，无缝地成为验证循环的一部分。

The natural one is the certified robustness that I mentioned. So saying that if our inputs are perturbed by a certain amount, how much is the output going to be perturbed? This sensitivity analysis is another term. Having those kinds of bounds for different neural architectures, so you automatically get those bounds, can then help us not only train neural networks to do well in a control loop but also worry about safety, robustness, and stability. These are all part of control systems that people worry about. So that's one example of an application. More broadly, the idea is you need verification in lots of scenarios that involve neural networks. Control loops are one. Another example is we used physics-informed neural networks to solve partial differential equations or come up with systems that are guaranteed to satisfy certain physical laws. But we also want to verify, for instance, that our neural network is only trained in finite precision. Can we overcome those requirements? What happens when we use finite precision? What are the shortcomings? Can we also bound those? So those are other kinds of bounds that work in TorchLean. So all aspects of the effect of precision, the effect of perturbation—all of these we can now have algorithms implemented in Lean that can be seamlessly part of the verification loop.

那么 TorchLean 的描述能力是否足以描述基本上任何神经网络，还是存在限制？

So is the descriptive power of TorchLean sufficient to describe basically any neural network, or are there constraints on that?

是的。它本质上是一个类似 PyTorch 的框架，所以你可以用同样的方式很好地定义神经网络层，但后端使用 Lean 帮助我们形式化并证明它。

Yeah. So it's essentially a PyTorch-like framework, so you can just nicely define neural net layers in the same way, but the backend having Lean helps us formalize and prove it.

那么对于 Transformer 架构，例如，在非常大的神经网络上证明这些类型的界限是否合理？

And for transformer architecture, for example, is it reasonable to prove these kinds of bounds on a very large neural network?

一方面是拥有框架，另一方面是可扩展性。Lean 在这方面仍有很多缺点；它基于 CPU，将其放到 GPU 上有很多细微差别。所以还有很多工作要做。我们开始时是一个框架。使其更高效，尤其是在非常大的规模上，仍需要做很多工作。

There is the aspect of having the framework, and the other is scalability. Lean still has a lot of shortcomings there; it's CPU-based, and getting it onto the GPU has a lot of nuances. So a lot of work needs to be done. What we've started with is a framework. Making that more efficient, especially at a very large scale, requires still a lot of work to be done.

是的。

Yeah.

但这对 Lean 来说也是普遍成立的。

But that's true broadly for Lean as well.

我只是想理解我如何想象这一点。如果我上一门研究生级别的数值分析课程，你有一个微分方程，有一些离散化误差，然后你根据这些性质对解进行界限约束。解决一些基于物理或基于 AI 的微分方程解——从历史上看，这有点像狂野西部。你提到了物理信息神经网络；非常酷的想法，讨论一下会很有趣。但我知道有时它们很特殊，人们并不总是知道它们何时会起作用或不会起作用。我不是专家，但我只是想知道这是否是你的经验。我想知道的是：这是否帮助你理解了 PINN 的适用领域？目标是要严格地说这个解会收敛，还是对于神经网络来说，在受控方式下不一定有相同的收敛概念？

I'm just trying to understand how I imagine this. If I were to take a graduate-level numerical analysis class, you have a differential equation, you have some discretization error, and you bound the solution given these properties. Solving some of these physics-based or AI-based solutions to differential equations—historically it's been kind of the wild west. You mentioned physics-informed neural networks; really cool idea, it'd be fun to talk about that a little. But I know that sometimes they are particular and people don't always know when they will or won't work. I'm not an expert, but I'm just wondering if that's your experience. What I'm wondering is: has this helped you understand the domain of applicability for PINNs? Is the goal to rigorously say this solution will converge, or is there not necessarily the same concept of convergence in the controlled way for neural networks?

是的。物理信息神经网络是说，你写下偏微分方程，希望优化成功并得到正确答案。当然，如果优化不是问题，这将是通用的——你解决一切，我们都高兴。但事实并非如此。优化通常非常困难，尤其是对于时间依赖的问题，意味着它不仅仅是静态的；你还有时间，而时间分量在许多情况下可能是湍流的。在流体动力学的情况下，如果你运行足够长的时间，它可能变得混沌。所以你有非常小的细尺度效应很重要。在这些情况下，试图在所有时间求解偏微分方程是无望的；这不是我们能掌控的优化景观。这就是为什么从头开始用神经网络解决这些方程是不可能的。所以 PINN 并非在所有地方都有效，我们提出的神经算子概念就是为了克服这一点。我们不能仅仅依靠物理约束来得出答案。我们有大量可用的数据。我会提到天气的例子，我们甚至收集数据；我们不仅解方程并拥有合成数据，我们还通过观察天气获得真实数据。那么为什么不利用所有可用的数据呢？我们不仅仅依赖从头开始求解偏微分方程和其他物理问题，因为真正使快速答案成为可能的是数据驱动的方法。通过神经算子，我们可以将两者结合起来。我们可以拥有所有可用的数据，利用它，添加物理约束，这克服了 PINN 面临的限制。

Yeah. Physics-informed neural nets are about saying that you write down a PDE, and hopefully the optimization succeeds and you get the answer right. Of course, if optimization were not an issue, this would be universal—you solve everything, we're all happy. But that's not the case. Optimization ends up being usually very difficult, especially for problems that are time-dependent, meaning it's not just stationary; you also have time, and the time component in many cases could be turbulent. In the case of fluid dynamics, if you run it long enough, it can become chaotic. So you have very small fine-scale effects that matter. In those cases, trying to solve a partial differential equation at all times is hopeless; this is not an optimization landscape that we can have any handle on. This is where the idea that from scratch we would be able to solve these equations using a neural network is not possible. So PINNs don't work everywhere, and our idea of neural operators came as a way to overcome this. We can't rely just on physics constraints alone to come up with answers. We have lots of data available. I'll talk about the weather example where we even collect data; we don't just solve equations and have synthetic data, but we also have real data by observing the weather. So why not make use of all the data available? We don't just rely on trying to solve partial differential equations and other physical problems from scratch because it's really the data-driven approach that makes it possible to get quick answers. With neural operators, we can bring both of them together. We can have all the data that's available, utilize it, add physical constraints, and that overcomes the limitations that PINNs face.

你能就那里的区别以及为什么这是可能的，再多给一点直觉吗？我听到你提到，你知道，用 PINN 时基本上就是把物理约束直接烘焙进神经网络，但这样会随着时间或其他变量变得不稳定。而如果加一点数据，我大概能直觉理解那为什么会有所帮助。但你能给一点直觉，讲讲到底发生了什么，区别在哪里吗？

Can you give a little bit more intuition on the difference there and why that is possible? So I heard you mention, you know, with PINNs you're basically just baking the physics constraints into the neural network, but that this becomes unstable over time or other variables. Whereas if you add a little bit of data, I can kind of intuitively understand why that might help. But can you give a little intuition for what's going on, what's the difference here?

所以用 PINN 时，你知道，每一个方程实例你都是从零开始求解，至少在经典意义上是这样。你从指定要解的方程开始，然后希望优化景观能成功，但很多情况下它不会。而用神经算子，我们做的是拥有大量优质数据。所以我们有一个训练阶段，教它如何为不同的方程实例生成解。就像其他监督学习一样，在测试时你可以问它：你能给出答案吗？而且你仍然可以用物理约束来引导。所以它可以同时是数据驱动和物理信息驱动的。但好处是，因为我们有数据，你知道，就像你不会被困在优化景观里。你在训练时就知道答案是什么，所以即使在测试时，你也有更好的机会得出正确答案。

So with PINNs, like, you know, every instance of an equation you solve from scratch, right, at least in the classical sense. So you start, you take the specification of what equation you want to solve, and you hope that the optimization landscape succeeds, which in many cases it doesn't. Whereas with neural operators, what we do is we have lots of great data. So we have a training phase, we teach it how to come up with solutions for different instances of equations. And so just as in other supervised learning, at test time you can now ask, can you come up with an answer? And you can still have physics constraints as a way to guide that. So it can be both data-driven and physics-informed together. But the benefit is, because we have data, you know, it's like you're not stuck in an optimization landscape. You know what the answers are during training, so you now have a better chance to come up with the right answers even at test time.

我的理解是，神经算子是一个拟合数据的函数，或者说神经网络学习将函数拟合到数据。这是一个好的直觉吗？

My understanding is that a neural operator is a function fit to data, or a neural network learns to fit functions to data. Is that a good intuition here?

是的。所以神经算子在这个意义上类似于，你知道，和神经网络一样，你是在数据上学习。但区别是，神经算子你可以把它看作神经网络的一种泛化。对于标准神经网络，输入和输出是固定大小的。在语言中，我们有固定的词汇表，我们固定输入和输出是什么。图像和计算机视觉、视频也一样，我们假设一个固定分辨率，输入和输出总是那个固定分辨率。我们不能事后改变它。而对于很多这类物理数据，想法是我们的世界本质上是多尺度的。所以你不应该事先决定分辨率是什么。也许你只有粗分辨率的天气数据，但实际现象发生在更细的尺度上。也许你之后想加入更细分辨率的数据或物理约束。所以我们应该有这种灵活性，我们应该把世界看作不是固定分辨率的，而是无限发生的。真实世界以无限分辨率发生，这正是神经算子所实现的，因为它们把输入和输出建模为连续函数，可以无限解析，可以无限离散化。现在在推理时，你可以给它输入，并请求任何分辨率的输出。所以你不只限于标准神经网络中看到的训练分辨率，这就是神经算子所实现的。所以神经算子让我们可以随心所欲地放大和缩小。

Yeah. So neural operators are in that sense similar to, you know, it's the same as neural networks, right, you're learning on data. But the difference is, neural operators, you can think of it as a generalization of neural networks. So with standard neural networks, the inputs and outputs are a fixed size. So in language, we have a fixed vocabulary, we fix what the input and output are. And same with images in computer vision, in videos, we assume a fixed resolution, and our inputs and outputs are always at that fixed resolution. We can't change it post hoc. Whereas with a lot of this physical data, the idea is our world is inherently multiscale. So you should not be deciding beforehand what the resolution is. Maybe you have weather data available only at coarse resolution, but really the actual phenomena are happening at a finer scale. And maybe you want to after that incorporate either additional data at finer resolution or add in physical constraints at finer resolution. So we should be having that flexibility, and we should really think of the world not at these fixed resolutions but one that's happening infinitely. The real world happens at that infinite resolution, and that's what neural operators enable, because they model inputs and outputs as continuous functions that can be infinitely resolved, that can have infinite discretization. And now at inference time, you can give it inputs and ask for outputs at any resolution. So you're not just limited to the resolution of training that we see in standard neural networks, and that's what neural operators enable. So neural operators enable us to zoom in and out as we like.

所以显然，正如所说，任何欠约束的函数，对吧，你可能有很多函数都能拟合数据。很容易过拟合。那你如何正则化呢？

So obviously, as stated, any function that's under-constrained, right, you could have many functions that fit the data. It would be easy to overfit. So how do you regularize that?

是的，当然。如果你问的是在比所见更高分辨率下做预测，比如我们所说的零样本超分辨率，你是在做一些猜测，对吧，这些模型正是在做这个。它们试图正则化并平滑地扩展到更高分辨率。但当然，如果你现在给模型额外的信息，比如物理定律，你可以给它偏微分方程约束、守恒定律，你可以在比现有数据更细的分辨率上强制执行它们。那么从某种意义上说，就有更多的引导。这样它就能在更高分辨率下得出正确答案，因为你在更高分辨率上给它约束。这就是我们如何确保这些物理信息神经算子能在比可用训练数据更高的保真度和分辨率下工作。

Yeah, certainly. If you're asking about making predictions at a higher resolution than what's seen, like what we call zero-shot super-resolution, you're kind of making some guesses, right, and that's what these models are doing. They're trying to regularize and kind of smoothly extend to higher resolution. But of course, if you now give the model additional information in terms of, let's say, physical laws, so you could give it partial differential equation constraints, conservation laws, and you can now enforce them at a finer resolution than the data you have. Then there's more guidance in a way. So that way it can now come up with the right answers even at higher resolution, because you're giving it constraints at higher resolution. And so that's how we can ensure that these physics-informed neural operators can work at higher fidelity and higher resolution than even the training data that was available.

我的理解是，你的很多工作使用一种特定的神经算子，即傅里叶神经算子。傅里叶是一个对偶域。它扩展到了输入的整个域。这有很多术语。也许你能给一些直觉，为什么这很重要？它有什么帮助？

My understanding is a lot of your work uses a particular kind of neural operator, a Fourier neural operator. So Fourier is a dual domain. It is extended across the entire domain of the inputs. That's a lot of jargon. Maybe can you give some intuition for why that is important? How does that help?

我提到的，神经算子作为一类模型，允许我们有任何分辨率的输入和输出，并学习它们之间的映射。这实际上被称为算子，即函数空间之间的映射，所以这就是神经算子这个名字背后的原因。而傅里叶神经算子是我们提出的早期设置或架构之一。它之所以如此成功，是因为它在效率和表达力之间取得了很好的平衡。那么为什么傅里叶空间是好的？傅里叶空间允许我们，你知道，它是一个对偶空间，就像你提到的，但它确实允许我们捕捉非局部现象，对吧？也就是说，在傅里叶域中非局部的东西可以被高效捕捉。我们在自然界中看到的很多现象，无论是流体动力学、材料变形、量子化学，很多都是非局部的。你知道，像导数这样的微分方程是局部的，但它的逆，你基本上是在做积分，它是非局部的。所以解是非局部的，这些模型能够捕捉到这一点。但同时，做傅里叶变换是高效的，它很好地捕捉了我们在许多自然现象中看到的归纳偏置。但这并不意味着我们完全在傅里叶基中捕捉世界。它不是傅里叶基中的线性表示，那是经典数值方法所做的。我们在傅里叶层之间加入非线性，就像 Transformer 和其他神经网络一样，我们还加入残差连接。所以所有这些受其他神经网络启发且在它们中表现良好的架构方面，把它们结合起来确实帮助我们获得两全其美。你可以想象，如果我们使用 Transformer 并且需要非常高的分辨率，那将变得不可行，因为二次复杂度和所有连接。

What I mentioned, neural operators as a class of models that allow us to have any resolution input and any resolution output, and learn the mapping between them. That's really called an operator, so the mapping between function spaces, so that's the reasoning behind the name neural operator. And the Fourier neural operator was one of the early setups or architectures we came up with. And the reason why that's been so successful is because it kind of strikes a nice tradeoff between efficiency and expressivity. So why is the Fourier space a good one? The Fourier space allows us to, you know, it's a dual space like you mentioned, but it really allows us to capture non-local phenomena, right? So meaning something that's non-local in the Fourier domain could be efficiently captured. And a lot of phenomena we see in nature, whether it's fluid dynamics, material deformation, quantum chemistry, a lot of them are non-local. You know, the differential equation like the derivative is local, but the inverse of it, you're kind of doing essentially integration, it's non-local. So the solutions are non-local, and these models are able to capture that. But at the same time, doing Fourier transform is efficient, and it kind of nicely captures a lot of inductive bias we see in many of these natural phenomena. But this doesn't mean that we are capturing the world entirely in the Fourier basis. It's not a linear representation in the Fourier basis, which is what classical numerical methods do. We add nonlinearity just as in transformers and other neural nets in between Fourier layers, and we also add residual connections. So all of these architectural aspects that are inspired by other neural nets that work well in other neural nets, bringing that together really kind of helps us get the best of both worlds. So you can think of, if we were to use transformers and we require a very high resolution, it would become untenable because of the quadratic complexity and all connections.

另一方面，如果你用傅里叶变换来做，我们就有准线性复杂度，而且我们仍然有全局连接，可以建模这些非局部现象，所以这是一个很好的折中方案。这样你就能从另一边发生的事情中学习，比如天气，芝加哥发生的事情可能会对旧金山产生影响，也许不会，但这就是这个思路。

On the other hand, if you did that with Fourier transforms, we have like quasi-linear complexity and still we have global connections in a way we can model these non-local phenomena, and so that's why it's a nice middle ground. So that allows you to learn from what is happening on the other side, like, talking weather, like what's happening in Chicago may have some impact on what's happening in San Francisco, maybe not, but that's the idea.

嗯。

Yeah.

所以这就是这个思路，时间上，是的，在某个时刻可能是局部的，但最终它们会对其他地点产生影响，所以我们想在空间和时间上都捕捉这种依赖关系。

So that's the idea, and time, like, kind of yes, at this point may be local, but eventually they have an impact in other locations, and yeah, so both in space and time we want to capture that dependence.

是的。所以今天芝加哥发生的事情，一个月后会对旧金山产生影响，或者类似这样。

Yeah. So what happens today in Chicago will have an impact in a month in San Francisco or something like that.

是的，所以你知道，既有短期影响也有长期影响。短期来看，我们考虑的是可预测的天气，但长期来看，我们说的是气候，对吧？所以具体会发生什么，你可能无法精确地说出芝加哥会发生什么、旧金山会发生什么，这就是蝴蝶效应。另一方面，我们可以给出平均值。比如，如果整个地区有热浪，我们大致知道气温会高于平均水平。所以这些方面我们可以一起捕捉。

Yeah, so you know, there is both the short-term and the long-term effects. So in the short term, we think about predictable weather, but longer term, we're talking about climate, right? So what happens, you may not be able to say precisely, you know, what happens in Chicago will happen in San Francisco, that's like the butterfly effect. On the other hand, we can kind of give averages. You know, if there's a heat wave in this kind of overall region, we kind of have an idea that it's going to be higher than average temperatures. So those are the aspects we can capture together.

从架构的角度来看，对于这里所有的 AI 工程师，我们是不是只是在傅里叶域里做所有的工作，但基本上还是同一个神经网络，只是我在傅里叶域里操作，还是说需要其他方面才能正确做到这一点？

From an architectural standpoint, for all the AI engineers here, are we just talking about doing all the work in the Fourier domain, but it's basically the same neural network, but I'm just operating in the Fourier domain, or is there other aspects that are required in order to do this properly?

所以想一想，我觉得也许最简单的思考方式是，如果你考虑 Transformer 架构，不再用注意力机制图，而是用傅里叶变换，但你仍然有其他非线性部分，比如残差连接，架构中还有很多其他部分仍然存在，赋予它表达能力，我们还提升到更高维度，比如在通道空间中，以增加表达能力。所以所有这些最佳原则仍然可用。但傅里叶变换帮助我们捕捉那种全对全的依赖关系，而不需要像 Transformer 那样巨大的复杂度。这很有道理。

So think of it, I guess the maybe the easiest way to think about it is, you know, if you think of a transformer architecture, instead of like the attention map, you now have the Fourier, but you still have other nonlinearities, you have like the residual, you have many other parts of the architecture still there that give it expressivity, and we have lifting to higher dimension, like in a channel space, to give it more expressivity. So all of those kind of best principles are still available. But the Fourier helps us capture that all-to-all dependence without requiring very huge complexity that I see in transformers. That makes sense.

另一个优势是它给你天然的多尺度。一个隐含的截止点是什么，你知道，如果你有信号处理或物理背景，你可能会问，如果你完全线性地做，有一个最大频率，超过那个你就无法表示任何东西。但在非线性域中添加这些其他架构变化，实际上如何影响你对频率范围的选择？

The other advantage is that it gives you the natural multiscale. What's sort of an implicit cutoff is, you know, if you have a signals background or physics background, you might ask, you know, if you're doing everything linearly, there's a maximum frequency, and above that you can't represent anything. But how does adding these other architectural changes in a nonlinear domain actually affect your choice of frequency bounds?

是的，不，这是个很好的问题，这就是表达能力发挥作用的地方，对吧？否则，如果你只是对信号进行傅里叶变换并试图表示它，你知道，数值方法也尝试过这样做，这需要非常精细的离散化，这就是为什么经典方式做模拟非常昂贵。相反，如果你想摆脱这一点，说我们要学习特征，这正是深度学习的核心，那么我们不能强迫它只在傅里叶域中。我们必须给它非线性，让它找出正确的基，你知道，表示信号的最佳基。所以我们拥有的这种很好的组合，就是所有这些非线性会帮助它找到正确的潜在空间。无意双关。这是广播。所以你知道，如果你在那个潜在空间里做傅里叶变换，那可能是一种更有效的表示方式。所以这是一种思考方式，因为首先，我们把信号提升到更多维度。即使信号是二维或三维的，我们现在也把它提升到高得多的维度。所以在这个空间里，想法是更容易学习，而且我们做的是非线性提升，对吧？所以那里已经有一个潜在空间，然后我们在傅里叶变换之间做进一步的非线性变换。所以这意味着我们说，是的，也许用这些有限数量的频率模式，它不够有表达力，但当我添加非线性时，我可以更好地捕捉它们。

Yeah, no, that's a great question, and that's where the expressivity comes in, right? Otherwise, if you're just taking a Fourier transform of a signal and trying to represent it, you know, that's what numerical methods have also attempted to do, and that requires very fine discretization, and that's why it's very expensive to do the simulations in a classical way. Instead, if you want to move away from that and say we want to learn the features, which is what deep learning is all about, then we cannot force it to be only in the Fourier domain. We have to give it nonlinearity to figure out what the right basis, you know, the best basis to represent the signals are. And so that's the kind of nice combination we have, that it's like all these nonlinearities will help it kind of find the right latent space. No pun intended. This is the radio. And so you know, if you do Fourier in that latent space, that may be a more efficient way to represent. So that's one way of thinking, because first of all, we're lifting the signal to more dimensions. Even if the signal is two or three dimensions, we are now lifting it to much higher dimension. So in that space, the idea is it's easier to learn, and we're doing it as a nonlinear lifting, right? So there's already a latent space there, and then we are doing further nonlinear transformations in between our Fourier transforms. So that means we are saying, yes, maybe with these limited number of frequency modes, it's not expressive enough, but when I add nonlinearities, I can more nicely capture them.

是的。是的。所以你的职业生涯始于神经网络，我想，还没有起飞之前，对吧？所以我想那时候人们确实思考了很多关于合适的基组、函数展开、正交多项式之类的。从你的研究角度来看，随着社区从那个阶段进化到‘哦，管他呢，全都扔进去’，这种演变是怎样的？看起来你仍然相信至少其中一些概念作为指导原则。你认为实际上有没有可以从经典、严谨的数学技术中汲取的教训，你可以用这些技术来帮助改进对现实世界的建模，即使你仍然只是把所有东西都扔进去？

Yeah. Yeah. So you started your career back before neural networks were, I guess, taken off, right? So I think back then people really did think a lot about appropriate basis sets and function expansions and orthogonal polynomials or whatever. How does that evolution from your research standpoint, as the community has evolved from that to 'oh just screw it, throw it all in'? It seems like you still believe in at least some of those concepts as being guiding principles. Do you think that there are actually lessons to be taken from classical, rigorous mathematical techniques, that you can use those techniques to actually help improve modeling of the real world, even if you still are just throwing the kitchen sink at things?

不，我认为这涉及一个很好的权衡。我的意思是，有趣的是，我二十多年前的本科论文就是关于分数阶傅里叶变换的，对吧？所以，是的，我的意思是，单独来看，你知道，那不足以做计算机视觉，但我很好奇，好吧，这些技术是什么，它们效果如何？所以，你知道，我完全同意你的观点，我们不能强迫自己使用石器时代的技术或经典技术，对吧？我的意思是，我们必须有特征学习，我们必须有灵活性、表达能力，你知道，它们必须容易优化。所以所有这些方面对深度学习都非常重要。但当涉及到物理世界和物理数据时，它永远不会像我们在语言模型中看到的那样丰富，因为我们，你知道，我们的天气模型大约有 5 万个样本，对吧？5 万个相当高分辨率的全球天气图样本，但这和语言相比根本不算什么。在其他领域，甚至更少，因为模拟太昂贵了，真实数据可能根本不可用。所以在这里，我们必须更多地考虑归纳偏置，我们必须加入物理约束，不能仅仅依赖数据。这就是我认为需要更多思考架构设计的地方。另一个方面是计算复杂度。所以想想语言。它只是一维，即使在那里，上下文长度，你知道，我们正在达到数百万，而且我们已经在挣扎了，对吧？

No, I think it's a nice trade-off involved. I mean, it's funny, my undergraduate thesis more than two decades ago now was on fractional Fourier transform, right? And so, yes, I mean, by themselves, like, you know, that wasn't enough to do computer vision, but I was curious, okay, what are these techniques and how well do they work? And so, you know, I'm completely with you that we cannot just force ourselves to use stone age techniques or classical techniques, right? I mean, so we have to have feature learning, we have to have flexibility, expressivity, you know, they have to be easily optimized. So all of these aspects are very important with deep learning. But when it comes to the physical world and physical data, it's never going to be as plentiful as we see with language models, because we are, you know, our weather model like had about like 50,000 samples, right? 50,000 samples of fairly high resolution like global weather maps, but it's nothing like what we see with language. And in other domains, it's even less, because it's so expensive to simulate, and the real data may just not be available. And so here, we have to think about the inductive biases more, we have to add in the physics constraints, cannot be just reliant on data. And that's where I think a little bit more thinking of the architectural design comes up. The other aspect is computational complexity. So think about language. It's just one dimension, and even there, the context length, you know, we are getting to millions, and we are struggling, right?

我的意思是，另一方面，我们现在考虑的不只是 2D、3D，甚至 4D，你知道，3D 加时间。如果每个维度只有几百个网格点，而工业规模是从每个维度一千个网格点开始的，那我们谈论的就是数千亿甚至上万亿的上下文长度。所以，别指望 Transformer 能处理这种规模。全世界的算力加起来都不够。而且，首先，它们必须放在一起才能做到这一点。所以这就是为什么我们需要其他架构。

I mean on the other hand, now we're thinking about not just 2D, 3D, even 4D, you know, 3D and time. And if each dimension is even a few hundred grid points, which is where industrial scale starts, like a thousand grid points in each dimension, we're talking about hundreds of billions to even a trillion context length. So forget ever having a transformer for anything of this scale. All of the world's compute will not be enough. And first of all, they all have to be collocated to be able to ever do this. So that's why we need other architectures.

但我要稍微反驳一下。对，我们有视觉和视频语言模型，它们基本上是在学习一种映射。

But I would push back a little bit. Right, we have the vision and video language models, and they basically learn a mapping.

但分辨率非常低。这是关键。对于物理世界，我们需要的分辨率，我提到过一千乘一千乘一千乘一千。所以如果你算一下，那已经是数千亿了。

But the resolution is very low. That's the key. For the physical world, the resolution we require, I mentioned like thousand by thousand by thousand by thousand. So if you count that, that's already in the hundreds of billions.

是啊。

Yeah.

所以当我们考虑图像和视频时，我们并没有达到那么高的分辨率，对吧。

So we are not doing that high resolution when we think about images and videos, correctly.

我有一个朋友，然后……

I have a friend and...

而且视频也是自回归的。所以基本上你只需要做下一步。

And the video is also autoregressive. So it's essentially only like you only need to do the next step.

对，是的，但你在学习，我的意思是，通常你在学习一个码本，对吧？所以你是在学习潜在空间的偏差，或者从现实世界到潜在空间的映射，所以如果存在从物理世界到潜在空间的压缩，那么这些自回归技术就成功了。

Right, yeah, but you're learning, I mean, generally you're learning a codebook, right? So you're kind of learning the bias of the latent space, or the real world to the latent space, and so if there is a compression that you can do from the physical world into the latent space, then these autoregressive techniques have been successful.

是的。但问题是，很多用于视觉和视频模型的自回归技术主要是为了好看，对吧？它们不是为了非常精确的模拟。而在那里，拥有更高的分辨率和细节非常重要。所以我们至少需要摄入那么高分辨率的数据，对吧？我们需要能够处理这些数据并对其进行推理。这就是很多瓶颈所在，因为我们不能把一切都扔掉，说‘哦，我们就在每个维度放 100 个网格点或 50 个网格点吧’，因为没有足够的细节来正确建模流体动力学、等离子体、材料如何变形等现象。所以所有这些都需要高保真度，为此我们需要高分辨率。

Yeah. But the idea is, you know, a lot of these autoregressive techniques for vision and video models are mostly for looking good, right? So they are not for very precise simulations. And there, having that higher resolution and details is really important. So we need to at least take in the data of that high resolution, right? So we need to be able to process that and reason over them. And so this is where a lot of the bottleneck is, because we cannot afford to just throw away everything and say, 'Oh, let's just have 100 grid points in each dimension or 50 grid points,' because there just isn't enough detail to correctly model phenomena like fluid dynamics, plasma, how materials deform. So all of this requires high fidelity, and for that we need high resolution.

我的理解是，你有一个论点，即 AI 需要将物理世界融入其中，才能在未来扩展并保持准确。很多人都有这个论点。你的独特之处在于，你有几个将神经算子应用于物理世界的例子，然后你似乎在围绕你的经验构建一个论点。你能和我们分享一些你用神经算子和其他技术做的非常有趣和令人兴奋的事情吗？

My understanding is you have a thesis that AI needs to incorporate the physical world into it in order to scale and be accurate going forward. Many people have this thesis. You are somewhat unique in that you have several examples of applying neural operators to the physical world in this way, and then it seems like you're constructing a thesis around your experience here. Can you share with us some of the really interesting and exciting looking things that you've done using neural operators and other techniques?

是的，我的意思是，对我们来说，当我们开始用神经算子处理偏微分方程时，但更广泛地说，你甚至不需要假设它们是偏微分方程，对吧？可以是任何时空或多尺度的数据。所以我们开始寻找有趣的例子，其中之一就是天气建模，因为天气数据是开源的。它来自 ECMWF（欧洲中期天气预报中心），即欧洲全球天气建模机构，称为 ERA5。既然数据在那里，我们就想，好吧，让我们试试。这就是它的美妙之处：只要有数据，就是好消息。但当时很多气象科学家确实警告过我们。那是在 2021 年，他们说：‘不，不，不，这太难了。传统天气预报已经发展了几十年，那是非常谨慎的自下而上的基于物理的建模。’所以他们认为，哦，这是流体动力学，你能预测第二天的天气吗？等等。所以很多想法是：AI 不可能击败几十年的天气建模工作。但令我们惊讶的是，我们只是继续前进，我们训练了它们，我们用神经算子有效地捕捉了这些现象，然后我们惊讶地发现，它不仅准确，而且几乎接近传统天气模型所能达到的准确度，而且速度快了数万倍。所以以前需要大型超级计算机才能运行的东西，现在只需要一台消费级 GPU 就能运行。这是一个小模型，它非常适合，非常快，而且准确。我认为这改变了所有人的想法。在那之后，DeepMind、华为等许多公司在一年后也发布了他们自己的模型。我们是第一个真正开源我们的天气模型 FourCastNet 并采用宽松许可的。所以这让公司、气象机构、所有人都能基于我们的工作继续发展。所以，你知道，看到天气模型现在已经问世，这真是一场有趣的革命。气象机构正在采用它们。它让我们现在可以让全球南方的小型气象机构，例如，拥有过去只有非常大的机构才能拥有的保真度，对吧？所以它正在使天气建模民主化。所以这只是一个例子，说明那里取得了非常快速的进展和范式转变，即我们现在可以将 AI 作为天气建模的可靠方法。

Yeah, I mean, for us, when we started with neural operators for partial differential equations, but also more broadly, you don't even need to assume they're partial differential equations, right? Could be any spatial temporal or data at multiple scales. So we set out looking for interesting examples, and one of them was weather modeling, because the weather data is open source. It's available, called the ERA5 from the ECMWF, the European agency for global weather modeling. And given that the data was there, we were like, okay, let's just go try it. And that's the beauty of it: whenever data is available, it's really good news. But a lot of weather scientists did caution us back then. This was back in '21, and they said, 'No, no, no, no, this is so difficult. There have been decades of development in traditional weather forecasting, and that's very careful bottom-up physics-based modeling.' So assuming, oh, this is the fluid dynamics, can you go predict the weather the next day and so on. And so that's how a lot of the thinking was: that AI is just not going to be able to beat the decades of work in weather modeling. But to our surprise, we just went ahead, we trained them, we used neural operators to be able to effectively capture the phenomena, and then we found, to our surprise, that it's not only accurate, it's almost as close to what the traditional weather models can do accurately, but also tens of thousands of times faster. So what would take a big supercomputer to run can now be run, and we only needed a consumer-grade GPU. It was a small model, it fit very well, it's very fast, and it's accurate. And I think that just changed everybody's thinking. After that, DeepMind, Huawei, many others followed us a year later and released their own models. We were the first to actually open-source our weather model, FourCastNet, and do it permissively. So that's what allowed companies, weather agencies, everybody to build on us. And so, you know, it's been a really interesting revolution to see that the weather models are now out there. Weather agencies are adopting them. And it allows us to now have small weather agencies in the global south, for instance, have the same kind of fidelity that very big agencies were in the past only able to do, right? So it's democratizing weather modeling. And so that's just one example of where there's been very rapid progress and a paradigm shift in terms of saying that, oh, now we can have AI as a reliable way to do weather modeling.

我看到你提到的那些模型，你提出了一个别人没有的见解，而且人们能够设计其他机制来跟随你，但这里需要某种思维转变。是不是仅仅因为我们相信这些数据中有足够的结构可以学习，而人们只是做错了，人们找到了其他学习结构的方法，但你的方法非常……

I saw that the models you mentioned, you made an insight that nobody else had, and that people were able to devise other mechanisms to kind of follow behind you, but there was some sort of shift in thinking that was required here. And was it simply that we believe that there's enough structure in this data to learn and that people are just doing it wrong, and people found other ways to learn the structure, but that your method was very...

所以让我澄清一下。对，首先，最早的工作就是说，看，传统上这是通过每次尝试求解偏微分方程来完成的，一遍又一遍地做，而 AI 从数据中学习，学习模式，并且可以同样准确但更快。然后接下来的迭代是说，我们如何让它更准确？还有一个方面，就是有短期天气，比如未来两周可预测的，然后还有长期，从季节内到最终的气候建模。传统上，人们做的是为这些不同场景建立不同的模型。所以有一套系统适用于短期，另一套系统适用于长期。但对我来说，只有一个地球。

So let me clarify. Right, so there is, first of all, the very first work was to just say that, look, traditionally this has been done by trying to solve partial differential equations each time, doing it again and again, whereas AI learns from data, learns patterns, and can be just as accurate but fast. And then the next iterations was to say, how do we make it even more accurate? And there's the aspect that there is the short-term weather, like what is predictable for the next two weeks, and then there's the long-term, going to subseasonal to ultimately climate modeling. And traditionally, what people did was to have different models for these different scenarios. So there's a different kind of system that works for short-term, another system works for long-term. But to me, there's only one earth.

你知道，如果你想要一个基础模型，要求是它应该能同时处理非常短期和非常长期的任务。这正是 ForecastNet 3（最新一代模型）能做到的。这还因为我们融入了地球的球面几何。很多其他架构在短期天气预测上已经取得了不错的精度，但当你把它们运行更长时间，比如几个月甚至一年，甚至在那之前，它们很快就会崩溃，对吧？因为它们假设世界是矩形的，而事实并非如此。所以，把所有这些几何信息和神经网络算子结合起来，意味着我们可以忠实地运行同一个模型进行更长期的预测，从而把它变成气候模型。艾伦 AI 研究所现在已经基于我们的神经网络算子架构构建了气候模型，这是唯一一个作为 AI 模拟器有效的架构。对吧？其他架构都不适用于气候，因为气候要求我们假设世界是一个球体，并且如果你反复滚动预测，你需要保留这些信息，而如果它是一个狭窄的替代模型——我认为那就是天气模型——你只是狭隘地看几个指标，很多不同的架构都能胜任，对吧？但如果你要求一个架构像基础模型一样处理一系列不同的任务，那么融入地球的几何形状（即它是一个球体）并使用神经网络算子作为高效实现方式，就能让我们做到这一点。

You know, if you want a foundation model, the claim is that it should be able to do both very short-term as well as very long-term together. And that's where in ForecastNet 3, the latest iteration of the model, we're able to do both. And that's because you also incorporate the spherical geometry of the earth. And so with a lot of the other architectures that have been getting fairly good accuracies for the short-term weather, when you run them for longer term, when you run them for like several months to even a year, even before that, it just very quickly blows up, right? Because it assumes the world is a rectangle, which it isn't. And so incorporating all of the geometry and that information into neural operators means that we can faithfully run the same model also longer term, and you make this into a climate model. This is where the Allen AI Institute has now built climate models based on our neural operator architecture, and that's the only one that works as an AI emulator. Right? None of the other architectures work for climate, because climate requires us to assume the world is a globe, and that if you are repeatedly rolling out, you kind of keep that information, whereas if it's like a narrow surrogate—that's what I consider a weather model—you just narrowly look at a few metrics, many different architectures should do the job, right? But if you are asking one architecture to do a range of different tasks like a foundation model, that's where incorporating the geometry of the earth, which is that it's a sphere, and using neural operator as an efficient way to do that, enables us to accomplish that.

是的。所以 ForecastNet，你说是在 50,000 个数据点上训练的。我们能聊聊这具体是什么样吗？数据输入是什么样的？你实际上想从这里预测什么？然后，你说的从天气到气候的大规模泛化是什么样子的？因为我可以想象，如果你有 50,000 个数据点，这些可能是北美的高分辨率数据，那么它们可能甚至依赖于当地地理，比如如果你总是建模堪萨斯州，这能迁移到比如说瑞士阿尔卑斯山之类的地方吗？然后这能迁移到喜马拉雅山吗？

Yeah. So ForecastNet, you said trained on 50,000 data points. Can we just talk about what does this look like? What does the data input look like? What are you actually trying to predict from here? And then what is sort of the large-scale—you said you're going from weather to climate—what does it look like to do that generalization? Because I could imagine if you have 50,000 data points, these are you know some high resolution in North America, then you know they might even depend on the local geography, like if you are always modeling Kansas, does this going to transfer to, let's say, the Swiss Alps or something, and then does that transfer to the Himalayas?

首先，澄清一下，我们是在全球天气模型上训练的，对吧？所以我们有地球周围的所有信息，然后我们让它根据当前天气（比如风况、湿度等）以自回归方式预测接下来会发生什么，每 6 小时一次。所以预测接下来 6 小时会发生什么，以此类推，你滚动预测，并训练模型进行预测。所以实际上同一个模型有可能永远预测下去，对吧？但可预测窗口就像天气一样，如果你想超越这个，你就做我们所说的集成，意思是你对几个月到几年后会发生什么有一个概率估计，这样你就得到了一个气候模型。

So first of all, to clarify, we are training it on the global weather model, right? So we have all the information around the earth, and then we are asking it to predict, given the current weather, like wind conditions, humidity, and so on, what happens in an autoregressive way, and it's every 6 hours. So what happens in the next 6 hours, and so on, and you roll out, and you train the model to predict. And so you can have the potential actually for the same model to predict forever, right? But the predictability window is like the weather, and if you want to go beyond, you do what we call ensembles, meaning you have a probabilistic estimate of what happens in several months to years, and that's how you get a climate model.

好的。所以你实际上有这些局部预测器的集成，然后你对集成使用某种统计来得到气候预测。

Okay. So you actually have an ensemble of these local predictors, and then you use some sort of statistics on the ensemble to get a climate prediction.

是的。所以你基本上有几次滚动预测。你有几条滚动预测的轨迹，然后你得到……

Yeah. So you kind of have several rollouts essentially. You have several trajectories of rollouts, and then you get...

所以这实际上是这里的关键。

And so is actually pretty key here.

是的。这就是为什么这是传统气候建模的最大瓶颈，即使只运行一次也非常昂贵。你必须以非常高的分辨率进行长轨迹预测，这就是为什么我们没有很多高分辨率能力来做气候变化预测，例如。

Yes. And that's why that's the biggest bottleneck with traditional climate modeling, that it's so expensive to do even one single run. You have to do long trajectories at very high resolution, and that's why we don't have a lot of very high resolution ability to do climate change predictions, for instance.

那么你如何验证气候模型呢？你做很多次滚动预测，然后看事情总体上如何演变。我的意思是，你提到了局部蝴蝶效应。我认为我们可以相信，即使是一个完美的气候模型或完美的天气模型，也只能给你大约 2 周的时间，之后它就变得不可计算了。所以你做了很多次滚动预测，有一些混沌，你平均所有这些。你实际上如何验证这在足够长的时间尺度上有效呢？

So how do you validate climate? So you do lots of rollouts and you look at sort of how things evolve in aggregate. I mean, you said butterfly effect locally. I think we can believe even a perfect climate model or perfect weather model would only give you maybe 2 weeks before it becomes non-computable. So you do lots of rollouts, there's some chaos, you average all these things. How do you actually validate that this works over long enough time scales?

是的，这是一个棘手的问题，对吧？例如，你必须确保满足所有的物理约束，如果你只是做一个标准的滚动预测，那很可能不会满足。所以我们正在进行的一些研究是，如何在滚动预测时强制执行正确的物理约束？比如你不想完全抹去细节，因为那样就不准确了，但另一方面，如果你保留它们，也许它们在物理上无效。所以这仍然是一个开放的问题，这就是为什么困难：你希望 AI 快速，你希望能够进行这些长期气候模拟，同时能够对它们有充分的信心。但这些都是我们正在努力解决的问题。

Yeah, and it's a tricky question, right? So for instance, you have to ensure that you satisfy all of the physical constraints, and if you just do a standard rollout, that is likely not going to happen. And so some of the ongoing research we're doing is how do you enforce the right physical constraints as you do the rollouts? Like you don't want to completely wash out the fine details because then it's not accurate, but on the other hand, if you keep them, maybe they're physically invalid. So this is still an open problem, and that's what makes it difficult: you want AI to be fast, and you want to be able to do these long climate simulations, and at the same time be able to have full confidence in them. But these are things we are working on, though.

我不知道这属于天气还是气候，但我们最近刚刚经历了现代气候数据史上最极端的极端热浪之一，对吧？就在美国西南部。我想知道，你定期参与这个吗？你知道你或其他人是否真的正确建模或预测了那个吗？

I don't know if this falls into weather or climate, but we recently just had one of the most extreme heat waves in the history of modern climate data, right? In the southwest United States. I'm wondering, were you involved in this regularly? Do you know if you or anyone actually modeled that or predicted that correctly?

是的。所以我们有——我没有关于这个具体事件的信息——但我们在最新的 ForecastNet 3 模型中测试了各种极端天气事件，对吧？这是关键：你需要概率性的答案。所以当我们面对极端事件时，仅仅给出一个确定性的输出并说这就是天气是不够的。所以我们需要仔细的概率校准，我们表明我们能够很好地捕捉这些。我认为即使在我们的第一次尝试中，这也是一个惊喜，我们可视化了某些飓风和风暴，它表现得很好，这非常令人惊讶，因为你会认为罕见事件不是 AI 擅长的；它擅长典型事件。但我认为更广泛的教训是：物理世界可能更宽容，因为像飓风这样的极端事件有非常具体的物理特征，对吧？所以它虽然极端，但方式非常特定。所以也许你不需要那么多样本，因为物理世界有很多结构，这是我们一次又一次看到的——在很多其他例子中也有很多结构。你知道，我谈到聚变反应堆中的等离子体。

Yeah. So we have—I don't have information on this specific one—but we've tested in our latest ForecastNet 3 model extreme weather events of all kinds, right? And that's the key: you need probabilistic answers. So having just one deterministic output and saying that this is the weather is not enough when we are looking at extreme events. So we need careful probabilistic calibration, and we show that we are able to capture those well. And I think that was the surprise even in our very first attempt, that we visualized certain hurricanes and storms, and it was able to do well, which is very surprising because you would think that rare events are not something AI would do well; it would do well on typical events. But I think this is where more broadly the lesson is: the physical world may be more forgiving because where there are extreme events like hurricanes, they have very specific physical signatures, right? So it's like extreme but in a very specific way. So maybe you don't need as many samples because the physical world has a lot of structure, and that's something we see again and again—that there is a lot of structure in so many other examples. You know, I talk about plasma in fusion reactors.

我们只有几千个样本，却能非常准确地预测像扰动这样的事件，而且比传统模拟快一百万倍。对我来说，这一切听起来很令人惊讶，但这是因为大自然帮了我们很多忙。它有很多潜在空间结构，传统数值方法无法揭示这些结构，因为它们更注重正确性，确保你在任何情况下都能解这些方程。另一方面，AI 从数据中学习，揭示结构，揭示这些问题有多简单或易于处理。这就是我们在许多案例中看到的。

We barely have a few thousand samples, but we can accurately predict events like disruption very well, and we can do that a million times faster than traditional simulations. To me, all this sounds surprising, but it's because nature helps us a lot. It has a lot of latent space structure that traditional numerical methods aren't able to uncover, because they focus more on correctness, ensuring you can solve these equations in any scenario. On the other hand, AI learns from data, uncovering the structure and how easy or tractable these problems are. That's what we're seeing in many cases.

是的，我也有类似的类比。我自己的领域更接近计算生物学，AlphaFold 是社区里显而易见令人兴奋的进展，解决了蛋白质结构预测，当然还有关于实际解决了什么的种种注意事项。我想我们在之前一期节目里和 Boltz 团队讨论过这个，鼓励听众如果想了解更多可以去听。但关于蛋白质结构的一个关键点是，它确实受物理约束。这就是为什么它在某种程度上是生物学领域少数几个重大胜利之一，而生物学其他方面非常复杂，我们总体上很难取得很多成功。看起来，那些能用微分方程很好建模的问题，在整合这些技术作为通用形式方面有更多空间。

Yeah, I've had a similar analogy. My own domain is closer to computational biology, and AlphaFold is the obvious exciting development in the community, solving protein structure prediction, with all the caveats of what was actually solved. I think we discussed this in a previous episode with the Boltz team; I encourage listeners to listen to that if they want more. But one of the key points about protein structure is that it's really constrained by physics. That's why it was one of the few big wins in biology, which is otherwise very complex, and we've had trouble generally making a lot of success. It seems like problems modeled well by differential equations have a lot more room for integrating these techniques as a general form.

那么，回到我的问题：我们有气候、天气和气候、等离子体、生物学，我知道你为我们准备了一些可视化。你能和我们分享一下那是什么样的吗？这些可视化是什么，贯穿其中的主线是什么？我想听众可能已经有提示了，但我真的很期待看到。

So, getting back to my question: we have climate, weather and climate, plasma, biology, and I know you prepared a few visualizations for us. Can you share with us what that looks like? What are the visualizations, and what is the thread that runs through here? I think the listeners might already have a hint, but I'd be really excited to see that.

是的，我当然可以分享一些。这个只是展示我们有不同尺度的世界，对吧？这些是从原子到蛋白质，甚至到行星尺度（比如我们谈到的天气）的现象例子。我们需要捕捉所有这些，这正是神经算子设计的目的。你可以输入这些不同尺度的数据，而这正是让许多物理世界问题变得困难的方面——对精细尺度的需求。我们谈到许多传统计算机视觉视频模型只是为了视觉效果好看，这需要足够低的分辨率才能处理，自回归，视频也要足够短。但这不是许多工业级高保真物理模拟的工作方式。你确实需要高分辨率。大气就是一个例子：根据分辨率，你可以捕捉到不同的现象，所以如果没有高分辨率，你就会错过。现在的问题是，有了 AI，我们能否比传统模拟快得多？这个神经算子的例子显示，如果你用标准神经网络，像素数固定，就像你在这边看到的，放大后会变模糊——就这样，固定分辨率之外什么都没有。但神经算子的想法，因为它是函数空间表示，你可以不断放大，通过给它更高分辨率的数据或更高分辨率的物理约束来添加相关细节，把多尺度现象结合起来。

Yeah, I can certainly share some. This one just shows that we have the world at different scales, right? These are examples of phenomena at different scales, from atomic to protein to even planetary scales like the weather we talked about. We need to capture all of that; that's what neural operators are designed to do. You can feed in data at these different scales, and that's really the aspect that makes many physical world problems hard—the need for fine scale. We talked about how many traditional computer vision video models are just designed to make things look visually good, which requires low enough resolution to be tractable and autoregressive, and short enough videos. But that's not how a lot of physical simulation for industrial-scale high-fidelity work operates. You really need high resolution. The atmosphere is one example: depending on the resolution, you can capture different phenomena, so you miss out if you don't have that high resolution. Now the question is, with AI, can we do this much faster than traditional simulation? This one with neural operators shows that if you use a standard neural network with a fixed number of pixels, as you see on this side, when you zoom in it gets blurry—that's the end of it; there's nothing beyond that fixed resolution. But the idea with neural operators, because it's a function space representation, is you can keep zooming in, adding relevant details either by giving it data at higher resolution or physical constraints at higher resolution, bringing that multiscale phenomena together.

那么物理约束在哪里？我猜这里的物理约束就像局部模拟流体方程或某种动力学。

So where are the physical constraints? I assume physical constraints here are like locally simulating fluid equations or some sort of dynamics.

可能是对的。可以是任何性质的。想法是你可以添加守恒定律，比如在不可压缩流体中。你可以添加材料变形，比如物体如何拉伸。或者可以是完整的偏微分方程。这也是一个有趣的问题。我们一直在研究如何对不同物理进行课程设计——有些物理可能很难作为损失函数施加或添加，其他可能更容易，所以你也需要考虑施加什么。

Could be right. It could be of any nature. The idea is you can add conservation laws, for instance, in an incompressible fluid. You can add material deformation, like how things stretch. Or it can be a full partial differential equation. That's also an interesting question. We've been researching how to curriculum different physics—some physics may be very hard to impose or add as a loss function, others may be easier, so you also need to think about what to impose.

所以这里的直觉是，当我添加物理约束时，我是把它加到损失函数里——这大致就是正在发生的事情吗？

So the intuition here is that when I'm adding a physical constraint, I'm adding it to the loss function—is that more or less what's happening?

是的，因为那是可处理的。把它作为硬约束是不可处理的，而作为损失函数添加则是可行的，当然还有那个损失和我们拥有的数据之间的平衡，所以我们必须以适当的方式来做。

Yeah, because that's what's tractable. Making it a hard constraint isn't tractable, whereas adding it as a loss function is, and of course there's still the balancing of that loss with the data we have, so we have to do that in the appropriate way.

对。

Right.

是的，正如我提到的，这是天气模型的例子，我们展示了如何捕捉大气河流，这是我们在加利福尼亚看到的导致大风暴的现象。我想这周晚些时候会有一个。

Yeah, so as I mentioned, this is the example of the weather model where we show how we can capture atmospheric rivers, which is the phenomenon we see here in California, resulting in large storms. I think we have one expected later this week.

我们会记住你的话。

We'll hold you to that.

所以我展示这个的原因是这种全球现象，对吧？这些有数千英里宽。所以你确实需要非局部模型来捕捉这些跨度非常大的现象并准确做到，这正是我们的神经算子能够做到的。

So the idea of why I showed this is this kind of global phenomenon, right? These are thousands of miles wide. So you really need non-local models that capture these very large-span phenomena and do that accurately, and that's what our neural operators are able to do.

那么训练数据——你之前谈过一点，但我还是想知道这是什么？是气象卫星，还是有地面数据？是两者的混合吗？

So the training data for this—you were talking about this a bit before, but I'm still wondering what is this? Is it weather satellites, or is there ground-based data? Is it some hybrid of the two?

它是不同来源的组合。这就是我们所说的再分析数据。这是历史天气数据，以某种方式重新分析，意味着原始卫星数据与物理求解器告诉你的内容结合，一起同化，这些数据由气象机构提供，我们可以用来训练。

It's a combination of different sources. It's what we call reanalysis data. This is historical weather data that is in a way reanalyzed, meaning the raw satellite data is combined with essentially what the physics solvers tell you, together assimilated, and this is made available by the weather agencies, and we can train on them.

所以你是说他们拿一个低分辨率数据集，它汇集了我们在世界各地拥有的所有气象数据。

So you're saying they take a low-resolution dataset which is compiling all of the world's meteorological data we have across the world.

嗯，然后他们用基于物理的经典技术做短时间模拟来填补细节。是的。

Uh, and then they do short-time simulations using physics-based classical techniques to fill in the details. Yes.

嗯，你可以在短时间内做到，但时间一长，它很快就会崩溃。

Uh, and you can do that over short time spans, but as you go farther, it breaks down very quickly.

所以你把成本分摊到所有人身上，每个人本来都得做这件事，有人做了之后你就能利用它。

So you're amortizing that across all the everyone would have to do that and so somebody does it and then you are able to take advantage.

是的。我的意思是，这些数据是已经准备好的。但关键在于，这种带物理的数据模拟已经让我们的模型隐式地具备了物理知识。所以它能够保留这些信息，这也许就是它在极端天气事件上表现良好的原因之一。

Yeah. I mean this is data right that's already prepared. But the idea is already this data simulation with physics kind of makes our model physics informed implicitly. So it's able to kind of you know keep that information and that's why maybe that's one reason maybe it does well on even extreme weather events.

是的。这正好说明我们的模型已经在 ECMWF（欧洲中期天气预报中心）上线了，就是欧洲的天气机构。这个模型大概在两年多前发布，我记得是 2023 年秋天。对我来说，ECMWF 向公众开放这些基于 AI 的天气模型是一个非常大的进步，因为这样每个人都能看到实际效果。

Yeah. So this is just showing that our model is available in ECMWF which is the weather agency like you know the European weather agency uh and so this was launched like more than 2 years ago but you know I think fall 2023. So you know I think ECMWF making these AI based weather models available to the public to me was a very big step because that's where you know everybody could see what's happening.

当时有几个飓风，比如飓风李，公众就能看到这些天气模型的表现。例如，我们的 ForecastNet 能够比标准天气预报模型提前几天准确预测飓风的登陆。

There were several hurricanes like for instance there was hurricane Lee and that's where the public could see what are these weather models doing. For instance, our forecast net was able to correctly predict that the hurricane making the landfall several days earlier compared to the standard uh weather forecasting models.

所以，这些模型在极端天气事件上表现出色，并能提前预测，这对保护生命和减少经济损失都非常重要。正因为在这些事件中的出色表现，公众和气象科学家对这类模型的认可度大大提高。

And so the idea that these models could be very good for extreme weather events uh and do early prediction, you know, both for human lives, for economic costs is a very big deal. And so that's when the public kind of got a lot more I think buy in uh and from weather scientists because of how well it was doing in these events.

这就是我之前提到的集合预报，无论是针对极端天气还是气候预测。我们不能只看单一轨迹，对吧？除非你是那种拿着记号笔随便画飓风路径的人（开个玩笑）。你真正需要的是概率预测。

And this is what I was talking about in a ensemble prediction both for extreme weather or if you're thinking about climate. It's not just about looking at one trajectory, right? Because you know, unless you're somebody with a Sharpie somehow saying where hurricane is going to go, no, pun intended. But you know what you really want is a probistic prediction.

也就是说，我会在初始条件中加入不同水平的噪声。比如当飓风在加勒比海形成时，我会加入一些噪声，因为那里的观测本身就有不确定性，我无法确切知道测量值。然后我会观察可能的飓风路径，从而得出在不同地区登陆的概率，这样就能进行风险评估。

meaning, you know, I'm going to try different adding noise levels to my initial condition. What the when the weather when the hurricane is forming in the Caribbean, I'm going to add some noise because anyway it's noisy. I don't know truly what the measurement there is and then I'm going to look at what happens to the possible hurricane tracks and then I can come up with a probability of landfall in different regions and that's how I can do risk assessment.

对于传统天气模型来说，这更加昂贵，因为你需要运行所有这些集合。而现在 AI 天气模型速度极快，比传统模型快数万倍，这意味着我们可以进行非常大的集合预报。这大大提升了我们风险评估的能力。

And so this is where it gets even more expensive for traditional weather models because you have to run all of these ensembles. And now AI weather models being so fast, tens of thousands of times faster means we can now do very large ensembles. And this is a very big improvement in terms of what we can do for risk assessment.

你有没有做过类似的事情，比如查看历史飓风地图，然后尝试进行集合预测，并校准你的预测准确率？

Have you gone through and done let's say looked over the historical uh hurricane maps and then tried to do ensemble predictions and calibrated how often your predictions are like a

是的。在 ForecastNet 3 的论文中，我们有极端天气事件和集合预测的指标，实际上我们训练模型就是为了做好集合预测，所以校准对这些事件非常重要。

Yeah. So in forecast net 3 paper there are you know uh we have metrics of like extreme weather events and ensemble prediction and in fact we've trained the model to do good ensemble prediction and so this is where the calibration matters uh for for these kind of events.

ForecastNet 3 相比第二版和第一版，有哪些关键的见解或进展？

What was the sort of key insights or developments in forecast net 3 in versus two versus the first version.

是的。第一版使用了 FIA 神经算子，但没有考虑球面几何。在第二版中，我们认为世界是球体这一点很重要，因为如果不考虑的话，预测会出现扭曲。

Yeah. So the first version was kind of the you know the using like the FIA neural operators but we didn't incorporate the sperical geometry right in this next version we said I think you know it's important that the world is a sphere because first of all otherwise it's distorted so you kind of are not predicting uh

问个傻问题，如果不是球面，你们是不是用了墨卡托投影之类的？

dumb question but if it wasn't spherical what did you do like a mercurator projection or something?

是的，标准做法和其他天气模型一样，使用标准投影来预测天气，这对短期预测没问题。但我们的目标是让同一个模型也能做长期预测，这时加入球面几何就增加了额外的稳定性，我们可以进行更长时间的滚动预测。在 ForecastNet 3 中，我们不仅关注确定性预测，还希望做好集合预测。因此，我们训练模型时也考虑了概率预测的准确性。

Yeah, the standard like kind of the you know like as you all and all the other weather models do the same right so they just kind of have the standard projection and then uh you know predict the weather and which is okay for short-term prediction but when we you know our goal was to have the same model also do longer term and that's when incorporating the sperical geometry added this additional stability we could do longer rollouts and then in forecastet 3 the idea was it's not just about deterministic prediction. We want to get ensemble predictions right. So we have to train them based on this objective that we get the probabistic predictions correct as well.

你们预测的时间范围是多长？你们做了多少次滚动预测？

How long are you predicting out? Um and how many roll outs are you doing?

是的。滚动预测就是预测的时间长度，对吧？每一步是 6 小时，然后你可以预测任意时长，只需要不断滚动下去。

Yeah. So the I know roll out is how long you predict right? So each step is 6 hours and then you predict for however long you want. You know you just have to roll out.

抱歉，你们的集合里有多少个样本？

Sorry. How many how many uh examples in the ensemble do you have?

这取决于我们的选择。我们可以有不同规模的集合。目前我们展示的是几十个，但也可以做得更大。

So, and again that's our choice. We can have like ensembles of different levels. So, we I think it's like a few tens or something like is what we are currently you know shown but you can do much larger too

那这足以预测多远呢？我的直觉是，你想预测的时间越长，需要的样本就越多。

and that's adequate to get out how far my intuition is. the longer you want to predict the more

不一定，关键在于校准集合，确保它们有正确的离散度，而不是样本数量本身。

so not necessarily it's really like about again calibrating the ensembles and ensuring that they have the right spread rather than you know so

好的，所以你的集合里有几十个模型或样本，即使是非常长的滚动预测也足够了。

okay so so you have tens of these um models or examples in your ensemble and that even with a very very long roll out that's adequate.

正如我所说，对于非常长的滚动预测，还有很多悬而未决的问题，因为你需要以某种方式加入物理约束来确保这一点，这是我们目前正在积极研究的。但我们的模型已经能够进行比其他完全忽略球面假设的天气模型更长的滚动预测。

So again like you know there's as I said a lot of still outstanding questions to do very very long rollouts right because you you do need to incorporate like the physical constraints in a way to ensure that that's something that we are actively researching now uh but uh these models that we have are able to do the longest rollouts compared to any of the other weather models that completely ignore spherical assumption and a range of other things.

那么你说在长期气候预测中融入物理定律，具体是怎样的？因为有很多局部守恒量，如果你对集合取平均，这些守恒量可能会被破坏，即使每个单独的快照都满足守恒。

So when you say inc incorporate um physical laws for climate over long times I mean what does that look like? Because there's a lot of local conservation which may be just broken if you take an ensemble of average even though any given snapshot is respects that.

不，我们的想法是确保每个集合成员都遵守物理规律。

No the idea is to make sure you look at each ensemble member and it respecting the physics.

好的，好的。你不是在假设集合平均，也不是在推导一个粗粒化的概率等价物之类的。

Okay. Okay. You're not assuming the ensemble had you're not deriving a like coarse grained like equivalent of a probability or something

因为那样你会失去分辨率。

because then you would lose that you know resolution and the f

好的。

okay

这让我有点困惑，等等，所以不是平均，那你们是如何组合集合的？

that's a little confusing to me wait so you're so it's not an average how are you combining the ensemble

不，你们是在做平均，但你们是分别预测每一个。

no you are doing the average but you're predicting each one

哦，你们是分别预测每一个，好的。

oh you're predicting each one separately okay yeah

所以每一个都独立满足这些约束，但集合本身并不满足。

so each one is independently satisfies these constraints but the ensemble

对，集合本身不满足。

it does not yeah which

所以这就是你们确保物理一致性的方式。

and and so that's how you ensure physical biology.

是的。

Yeah.

那么当你在球面上操作时，你基本上是用球谐函数还是某种……

So when you go on a sphere you operate in, do you use basically spherical harmonics or some sort of...

对，是的，我们有一个球面基，这实际上非常自然。如果是 48 的话，可能更严格，如果你在做……

Okay, yeah, you have a spherical basis, which is actually very natural. With 48 is probably much harsher if you're doing...

嗯，正是如此。所以这就是傅里叶能帮我们的地方，可以很好地融入这些几何。所以我认为这对地球来说非常忠实。

Um, other exactly. So that's where the Fourier saves us, can incorporate these geometries well. And so I think it's very faithful to what the globe is.

是的，可以说这比墨卡托投影或其他什么更自然。

Yeah, arguably it's much more natural than a Mercator projection or whatever other...

是的，哪个……就像格陵兰变得巨大，那是另一回事。但我觉得这就是更广泛地融入几何和领域信息变得对物理世界更加重要的地方，对吧？所以我再次强调，我们需要融入更多结构，因为一方面数据有限，另一方面我们要求的很多是外推，要超越它训练过的内容。我们只是训练它预测接下来的六个小时，也许做一点多步微调用于自回归展开，对吧？所以我们不是训练它做很长的气候模拟，因为那太贵了。但我们希望它能神奇地工作，而如果你只是放一个标准 Transformer 或其他什么，它不会成功。所以我们加入更多领域约束，比如球面几何。我们可能以某些方式加入更多物理，这也是算法上变得更有趣的地方。这里有更多涉及的设计。

Yeah, which... It's like Greenland becomes huge, so that's a different story. But yeah, I think this is where the aspect of more broadly incorporating more geometry and information about the domain becomes a lot more important for the physical world, right? So this is me again emphasizing that we need to incorporate more of the structures because one is the data is limited, and the other is a lot of what we are asking is extrapolation, to go beyond what it's trained on. We're just training it to predict the next six hours and maybe do a little bit of multi-step fine-tuning for autoregressive rollouts, right? So we're not training it to do very long climate simulation because that's just too expensive. But we hope magically it works well, and it cannot if you just say I'm going to put a standard transformer or whatever else there, and it won't work out. So we add in more of the domain constraints like spherical geometry. We add in maybe more of the physics in certain ways, and that's where it becomes more interesting algorithmically as well. There's more involved design here.

那么你训练的时间尺度是多长？

So the time scale that you train on, how long is that?

预测接下来的六个小时？

To predict for the next six hours?

哦，所以只有六个小时。

Oh, so only six hours.

是的。还有一点多步微调，有点像……

Yeah. And a little bit of multi-step fine-tuning, like kind of like...

是的，我说过。好的，明白了。

Yeah, I said that. Okay, understood.

所以这非常令人惊讶，对吧？

So which is very surprising, right?

是的，这非常令人惊讶。我本以为会是几周或几个月。

Yeah, that is very surprising. I would have expected it was weeks or months.

不，然后它就这样工作得很好，甚至现在我们展示了几个月它都能做到。

No, and then it kind of just works well even for now we are showing for several months that it's able to do that.

步数是在数百级别还是……你的傅里叶基是基谐波的很多倍？

The number of steps is on the hundreds or... and your Fourier basis, it's many times the base harmonic?

我的意思是，这是空间上的，对吧？所以我们说的是在时间上自回归的展开。

I mean, this is like that's in space, right? So we're talking rollout is autoregressive in time.

不，但在时间上，也许我误解了，因为这是在傅里叶域，对吧？

No, but like in time, maybe I'm misunderstanding here because it's in a Fourier domain, right?

不，不，在时间上不是。这就是我说的，它是自回归的。

No, no, in time it's not. That's what I'm saying, it's autoregressive.

哦，哦，我明白了，好的。所以是空间上有趣。好的，角分辨率是多少？

Oh, oh, I understand, okay. So it's space it is interesting. Okay, what's the angular resolution?

至少在这个场景中，在其他情况下，我们也在时间上用傅里叶域表示，这也是一个问题，我们能做到吗？但在这个例子中，它是自回归的。

At least in this scenario, in other cases we also have in time is also represented in the Fourier domain, and that's a question as well, can we do that? But in this example, it's autoregressive.

明白了。嗯，你在这个球面版本中使用的角分辨率是多少？

Got it. Um, what's the angular resolution that you use for this, in the spherical version?

是的，所以所有可用的数据都是，我想是四分之一，像 0.25 度。所以……

Yeah, so all of the data that's available is like, I think a quarter, like 0.25 degrees. So...

也许用 L 表示，或者用球谐频率？

In terms of L maybe, or in terms of like a spherical harmonic frequency?

所以这就像，哦，你是说我们用了多少模式？我想我们，对于那个分辨率，我们基本上只用其中少数几个，我们会省略，我忘了细节。

So this is like, oh you mean like how many modes we utilize? I think we, for that resolution, we essentially utilize most of only a few of them, we'll leave out, I forget the details.

我只是好奇，你实际解析到地球上的角分辨率是多少，或者也许只是物理分辨率？

I'm just curious, like what is the actual angular resolution on the globe that you are resolving to, or maybe just the physical resolution?

我的意思是，就是这样，因为那是我们有的数据，对吧？所以现在我们得到的数据是四分之一，像 0.25°。

I mean, that's what that is, because that's the data we have, right? So right now the data we get is like a quarter, like 0.25°.

哦，好的，像 0.25 立体角？

Oh okay, like 0.25 solid angle?

我想它大概相当于，我不知道，700 乘几千的分辨率，但这已经是标准流程了。

I think it kind of comes out to like, I don't know, 700 by few thousand like resolution, but this is already standard kind of process.

是的，只是想理解你需要多大的基来表示它？

Yeah, and just trying to understand like how large of a basis do you need to represent that?

是的，我的意思是这真的取决于分辨率，想法是，你知道，现在我们的天气数据只受这个分辨率限制，但如果你能，你知道，你可以做更高分辨率的合成气候模拟，对吧？这就是下一步如何结合这些。

Yeah, I mean that really depends on the resolution, and the idea is, you know, right now our weather data is just limited by this resolution, but if you could, you know, you could do synthetic climate simulations of even higher resolution, right? And that's kind of the next thing on how to combine these together.

你认为你能用超分辨率预测，基本上比提供的数据更低的分辨率吗？

Do you think you can predict with super resolution, basically resolution lower than the data provided?

再说一次，是的，我们总是可以用神经算子预测它们。但你知道，你确实想融入更多物理约束以确保它们有效。

Again, like yes, we can always predict them with the neural operators. But you know, you do want to incorporate more of the physical constraints to ensure that they are valid.

好的。那么嗯，我们能谈谈其他一些……

Okay. So at um, can we talk about some of the other...

是的。是的。我知道这很多。

Yes. Yes. So I know it's a lot.

所以这只是展示，你知道，我描述的，左边假设世界是矩形，它很快爆炸，右边因为我们假设世界是球体，它持续展开并保持稳定。

So this is just showing how, you know, what I described, that on the left where the world is being assumed it's a rectangle, it blows up very quickly, and on the right, because we assumed the world was a sphere, it kept rolling it out and it kept being stable.

我在右边还是看到一点奇点。

I still see a little bit of a singularity there at the right.

而且它仍然像，你知道，所以想法是是的，因为这是一个非常长的展开，我们没有物理护栏，我们没有把它投影到正确的物理，对吧？这是完全外推，但想法是球面假设使它稳定得多。

And it's still like, you know, so the idea is yes, because it's a very long rollout and we have no guardrails of physics, we are not kind of projecting it to the right physics, right? This is full extrapolation, but the idea is the sphere assumption stabilizes it to a much greater extent.

与左边相比，它好多了，但如果你在南极，你仍然不会得到……

Compared to the left, it's much better, but if you're at the south pole, you're still not going to get...

正是如此，所以极点是难点。所以这是聚变反应堆的例子。这是一个托卡马克，我们能够模拟复杂的等离子体演化，并且比传统模拟快一百万倍。这在某种程度上是创建等离子体的数字孪生，对吧？然后我们可以做进一步的事情，比如现在我们下一步是研究控制，但要有完全有效的物理，比如理想情况下能够防止破裂并使聚变可持续。

Exactly, so the poles are the hard part. So this is the example of the fusion reactor. So this is a tokamak, and we are able to model the complex plasma evolution and do this a million times faster than what we could do with traditional simulations. And this was in a way creating a digital twin of the plasma, right? And then we can do further things, like right now we are as a next step looking at control, but with a full valid physics, like being able to prevent disruptions ideally and make fusion sustainable.

那么你在这里模拟 MHD 方程，还是抱歉，磁流体动力学方程？

So are you simulating MHD equations here, or sorry, magnetohydrodynamics equations?

是的。好的。对。

Yes. Okay. Yeah.

然后为了上下文，这种情况下的破裂是困扰等离子体物理学家的现象，在某个时刻你的整个等离子体聚集在一个小束中，然后射向你的容器，它……

And then so for context, disruption in this case is this phenomenon which plagues plasma physicists, where at some point your entire plasma collects in a little tiny beam and then shoots a strong, you know, right to your containment vessel, and it...

它可能损坏反应堆，这是一个大瓶颈，因为那样你必须在发生之前关闭它，而且不再可能实现可持续聚变。所以这里有很多未解决的挑战，但想法是，你知道，做物理实验非常昂贵。你越能在数字孪生中捕捉这一点，同时确保物理有效性，你就越能在数字领域进行设计和其他考虑。

And it can damage the reactor, and that's a big bottleneck because then you have to kind of shut it down before that happens, and then plus it's no longer possible to have a sustainable fusion. So there's a lot of open challenges here, but the idea is, you know, it's very expensive to go do physical experiments. The more you can capture that in the digital twin but ensure physical validity, the more you can even do design and other considerations in the digital realm.

你知道，我们希望能取得进展，这些是迈向那个目标的第一步。目标是，如果发生这类事件，你可以以某种方式调整磁场，使其包含并稳定它。

You know, we can hopefully make advances, and these are the first steps towards that. The goal is that if you have one of these events, you can somehow adjust the magnetic field so that it contains that and stabilizes it.

是的。那是我们现在正在做的下一步。我们正在考虑同时设计控制和模拟。

Yes. And that's the next step we're doing now. We are looking at designing both the control and the simulation together.

你们是在和这个特定的实验室合作吗？我只是好奇。

Are you working with this specific lab? I'm just curious.

所以这个项目是和英国原子能管理局合作的，现在我们也在和美国这里的其他几个机构合作。所以我们正在从聚变本身的许多不同方法中获取信息。这是托卡马克。我们也在研究仿星器。我们正在与不同的……

So this one was with the UK Atomic Energy Agency, and now we're also working with a few others here in the US as well. So we are, you know, kind of getting the information from many different approaches of fusion itself. So this is the tokamak. We're also working with stellarators. We are working with different...

仿星器很棘手。

Stellarators are tricky.

是的。

Yeah.

但理想情况下，你知道，我们的目标是在数字孪生中设计它们。所以我们能否提出好的设计，使其更实用？

But the idea is ideally, you know, like our goal is to be able to design them in the digital twin. So can we come up with good designs that would make it maybe more practical?

所以我认为，作为一个人工智能从业者，这也是一件好事，而且更像是一种不可知论，不预先选择赢家，对吧？我喜欢尝试不同的方法，看看人工智能是否能加速所有这些方法，然后我们可以，你知道，不 prematurely 排除一种方法而不是另一种。这就是人工智能让我们能够更多地承担风险，探索不同的方法，而不是在物理世界中尝试构建这些方法。你不得不砍掉很多有风险的方法，然后说：“我只做这个，因为它最可能成功。”

And so that's, I think, also a good thing as an AI person, and much more like, you know, agnostic and not picking a winner beforehand, right? Like, I like to work with different approaches, you know, and see whether AI can accelerate all of them, and then we can kind of, you know, not prematurely rule out one approach over the other. So that's what AI enables us to be more kind of taking risks and exploring different approaches, as opposed to in the physical world, trying to build any of these. You kind of have to cut a lot of the risky ones and say, "I'm only going to do this because this is the most likely to work."

我注意到在你的职业生涯中，你开始时花了很多时间在真正的理论基础、机器学习数学上，也许，我不知道，大约六到八年前，你开始大量从事应用，并扩展到各种不同的问题。是什么促使了你在方法和关注点上的转变？我的意思是，你仍然在研究非常困难的数学问题，比如 torch lean 的工作，但应用确实增长了，我想知道是什么促使了这一点，以及从那以后你学到了哪些教训？

I noticed over your career, you started out spending a lot of time on, you know, really theoretical foundations, mathematics of machine learning, and maybe, I don't know, something like six or eight years ago, you started working really working a lot on applications and branching out in a diverse set of problems. What sort of prompted that shift in your approach and what you're looking at? I mean, so you're still working on very hard math problems as well, like for example, the torch lean work, but the applications have really grown, and I'm wondering what prompted that, and like, what were some of the lessons you've learned since then?

当然。我的意思是，对我来说，就像，你知道，我觉得我是和人工智能一起成长的，对吧？所以当人工智能处于神经网络因为数据不足和各种其他原因而无法工作的时代，你知道，那时你不得不建立理论基础，并希望这能让你达到算法能够工作的地步，对吧？那时候，张量方法就是基于这样的想法：在深度学习之前，我们仍然想要结构。我们有概率模型，比如用于主题建模的潜在狄利克雷分配，而解决这些问题很困难。但现在张量方法给了我们一种非常实用的方式；它是并行的，可以大规模进行，但仍然有很好的理论基础。所以那些是，你知道，起步。然后随着深度学习开始起飞，我们看到它在实践中效果很好，是的，也许有一点理论理解，但不多，因为它太复杂了。对我来说，理论不应该是约束，对吧？它应该是推动者。所以很多探索都是：哦，让它在实践中运作良好，然后进入亚马逊网络服务，然后是英伟达，所以真正让事情大规模运作，真正动手实践，对吧？那是很多发展所在。现在我看到一个完整的循环，因为很多纯粹数据驱动的方法在某种程度上正在饱和，对吧？所以现在我们要问，好吧，要么让它们更硬件高效。现在有很多空间可以说，我们能否让它们更节能或硬件高效？所以这是一个方面。但另一个是像这样的领域，在物理世界中我们没有足够的数据。我们要求的是困难的推断。你知道，我们想要考虑做发现；本质上，它是关于推断的。所以我们永远不会有关于新发现的数据，对吧？这是我的定义。所以在那里，我们需要再次回到原则性的思考，无论是架构设计、算法设计、正确的损失函数。所以我们需要更加注意。所以我看到这正在形成一个完整的循环，因为所有与深度学习有效的方法，让我们采用它们，但让它们更有原则性。

Sure. I mean, to me, it's like, you know, I feel like I've grown along with AI, right? So when AI was in this where neural nets were not working because there wasn't enough data and all kinds of other reasons, you know, then you kind of have to build the theoretical foundations and try to hope that that leads you to a place where you know you get algorithms to work, right? And back then, like tensor methods was with that idea that, you know, pre-deep learning, we still wanted structure. We have probabilistic models like latent Dirichlet allocation for topic modeling, and solving those were hard. But now tensor methods gave us a way to be very practical; it's parallel and can be done at large scale, but still has nice theoretical basis. So those were, you know, starting off. And then as deep learning started taking off, and we could see that it works well in practice, and yes, there is a little bit of maybe theoretical understanding, but not a whole lot because of the way how complex it is. To me, theory should not be a constraint, right? It should be an enabler. And so that's where a lot of the exploration was: oh, to make this work well in practice, and into Amazon Web Services, then Nvidia, so really like making things work at scale and really kind of getting hands dirty, right? That was kind of like where a lot of the development is. And now I see a full circle, because a lot of purely data-driven approaches in a way are seeing saturation, right? So now we want to ask, okay, either make them more hardware efficient. There's a lot of now room to kind of say, can we now, you know, make them much more energy efficient or hardware efficient? So that's one aspect. But the other is areas like this where in the physical world we don't have enough data. We are asking for hard extrapolation. You know, we want to think of doing discovery; by nature, it's about extrapolation. So we will never have data about a new discovery, right? That's my definition. And so there, we need to again go back to thinking in principled ways, whether it's architecture design, algorithm design, the right loss functions. So we need to be much more mindful. So I see that coming up full circle, because all of the things that work with deep learning, let's take them but make them a bit more principled.

还有其他几个应用看起来非常自然。我想知道你是否在这些方面工作过，还是我错过了一些论文？如果我错过了，我很抱歉。嗯，所以一些例子是电磁电路的设计，我认为这是一个大领域。或者也许不是一个大领域，但我认为在不久的将来会出现。比如材料的设计，比如散热器中的耗散，或者任何类型的流体流动。当我思考我知道哪些微分方程时，我知道电磁学，我知道扩散方程，你知道，磁流体动力学。是的。我想知道其他一些领域……

There's several other applications which seem very natural. I'm wondering if you've worked on these, or did I just miss some papers? If I did, I'm sorry. Um, so some examples are design of like electromagnetic circuits, I think is a big one. Or maybe not a big one, but I think will be coming up in the near future. Design of, let's say, materials, design of, let's say, dissipation in heat sinks, or any sort of like fluid flow. As I'm going through what differential equations do I know? I know electromagnetism, I know diffusion equations, you know, MHD. Yeah. I'm wondering some of the other domains that are...

是的，我的意思是，对我来说，可能性是无穷无尽的，对吧？所以，你知道，你可以让这个工作在任何数据上，我们还有其他几个例子。所以这就像，你知道，能够问，我们能否将二氧化碳封存在地下，并模拟二氧化碳如何膨胀，或者你知道，这些储层中的压力积累是什么，以及你知道，我们能否模拟它们如何迁移几十年？所以这个，我们能够比传统模拟快得多地完成。我的意思是，另一个方面是能够处理各种几何形状，比如你知道，能够模拟汽车、飞机等的空气动力学。所以这又是一个潜在空间的很好的例子，因为你可以将汽车或任何其他形状转换为甜甜圈，然后在甜甜圈上建模，然后再将甜甜圈转换回汽车。

Yeah, I mean, to me, there is just endless possibility, right? So there, you know, as you can just have this work on any data, and we have several other examples. So this was like, you know, being able to ask, can we sequester carbon dioxide underground and model how carbon dioxide expands, or you know, what is the pressure buildup in these reservoirs, and you know, can we kind of model how they migrate over several decades? And so this one, we were able to do much faster than what traditional simulations could do. I mean, the other aspect is being able to do all kinds of geometric shapes, like you know, being able to model aerodynamics in cars, planes, and so on. And so again, this is a nice example of a latent space, because you can transform a car or any other shape to a donut, and then model on the donut, and then transform the donut back to the car as well.

你没有把它变成咖啡杯。那不是经典笑话吗？你的甜甜圈和咖啡杯。

You didn't turn it into a coffee cup. Isn't that the classic joke? Your donut and your coffee cup.

正是如此。所以，一个潜在空间来处理各种不同的几何形状，并能够在潜在空间中很好地捕捉物理特性，这意味着我们现在可以有一个模型，能够泛化到许多不同的几何形状。

Exactly. So the idea of like a latent space to handle all kinds of different geometries and be able to capture the physics there in the latent space well means we can now have a model that generalizes across a lot of different geometries.

我的理解是，也许这里更大的愿景是，你可以训练一个基础模型，能够用同一个模型模拟许多不同的物理现象。所以你可以微调，或者可能有某种提示，让它理解特定的几何形状。但你在所有这些不同的物理问题上训练，然后你有你的特定问题，你能够非常有效地模拟它。

And my understanding is that maybe the larger vision here is that you can train a foundation model in the sense of being able to model many different physical phenomena with the same model. And so you may fine-tune or there may be some kind of prompt that you give it to have it understand the particular geometry. But that you train on all these different physical problems, and then you have your particular one, and you're able to model that very effectively.

是的。

Yeah.

我的意思是，那才是真正的未来，对吧？因为我们有语言、甚至视觉的基础模型，但还没有物理的基础模型。所以想法是，与其像现在这样，我们看到的都是狭窄的替代模型，然后试图不断扩大它们的范围，理想情况下我们应该有更广泛的模型，能够处理一系列现象，还要有多物理场。所以不只是单一的物理，而是耦合的物理。现实世界中的所有物理现象都是以耦合的方式结合在一起的。那么我们能把这些都整合起来吗？这是一个方面：拥有能做设计和模拟的基础模型。但另一个真正有趣的方面是逆问题。所以我现在不仅能模拟，还能问什么是最优设计？这类模型可以做模拟，但你甚至可以隐式地做到这一点，并得出最优设计。在早期时代，是人类试图提出设计，然后你去模拟，或者去风洞或任何物理测试来验证。但现在你让 AI 提出优化设计，但你有物理的护栏。所以你有在物理上准确的模型，你有信心它们能很好地工作，所以你也能在同一个模型中做到这一点。

I mean that's really the future, right? Because we have foundation models for language, maybe vision, but not for physics. So the idea is, instead of what we've seen so far—narrow surrogates that we try to broaden more and more—ideally we have much broader models that can work on a range of phenomena, but also multiphysics. So not just one single physics, but coupled physics. The real world has all of the physics coming together in coupled ways. So can we bring all that together? That's one aspect: having foundation models that can do design and simulation. But the other aspect that's really interesting is the inverse problem. So can I now not just simulate, but ask what is the best design? These kinds of models can do simulation, but you can even do that implicitly and come up with the best design. In the earlier era, it was humans trying to come up with a design, then you go and try to simulate or go to the wind tunnel or whatever physical testing and validate that. But now you have AI come up with optimized designs, but you have the guardrails of physics. So you have models that are accurate in physics, you have the confidence they work well, so you're able to do that as well in the same model.

你谈到这些多物理场能够迁移，或者你可能能够泛化到未见过的物理现象，你见过任何证据吗？

Is there any evidence that you've seen, you talked about these sort of multiphysics being able to transfer, or that you may be able to generalize to unseen physics?

所以我的意思是，物理本质上，如果是完全未见过的，就不可能迁移，对吧？我的意思是，如果你说我们要超越标准模型，那绝对没有数据，那是不可能的。但如果你问的是，比如，我给它看过热如何传播的例子，还有材料如何拉伸的例子，现在有耦合，因为热也有拉伸，或者说是联合现象，你现在可以希望用更少的样本进行微调，因为它单独知道这些现象，然后把它们结合起来。也许它不能从零开始做，因为那要求太高了。它是高度非线性和耦合的，但它可以用更少的例子做到。我们在很多论文中都看到了证据，你基本上可以建立一个课程。这就是我们在许多例子中反复看到的：现实世界，我们可以在很大程度上控制课程，说，让我们构建模块并把它们放在一起，这就是它现在允许我们以系统的方式做到的事情。

So I mean, physics by nature, if it's completely unseen, it's not possible to transfer, right? I mean, if you're saying that we're going beyond the standard model, there's absolutely no data, that's not possible. But if you're asking about, for instance, like I've shown it examples of just how heat propagates, and there are other examples of how the material stretches, and now there is coupling because of heat there's also stretching, or kind of the joint phenomena, you could now hope to fine-tune with much fewer samples because it kind of individually knows these phenomena, then combining them together. Maybe it can't do it from scratch because that's still too much to ask. It's highly nonlinear and coupled, but it can do it with fewer examples. And we've seen evidence of that in a lot of our papers, that you're able to essentially build up a curriculum. And that's what we see again and again in many of these examples: the real world, we can kind of control a lot of curriculum and say, let's build in modules and put them together, and that's what it now allows us to do in a systematic way here.

我想设计方面，我不知道你是否想快速展示一下。所以这个是关于为逆光刻设计掩模的，这意味着这是一个逆设计问题。我们也能为量子点中的门设计做到这一点。这就像非线性光子学。所有这些的共同点是，有一个前向模型在模拟物理，但现在我们想要的是逆设计，即我们可以优化最优设计的问题。而人类通常不擅长这个，对吧？我们不擅长看着高度非线性的现象说，哦，不知怎么的，所有这些门组合在一起可能有助于在量子门中把电子拉在一起。所以我们的合作者手动做这件事很吃力，而有了 AI，我们现在能够提出非常高效的设计，而且我们知道这些设计实际上有效，因为我们已经把模拟作为循环的一部分，说明它们工作得很好。所以我认为这些例子表明，这不仅仅是模拟。这是真正新颖的设计和新发现，使我们能够推动创新本身的发展。

I guess the design aspect, I don't know if you wanted to show very quickly. So this one was looking at designing the mask for inverse lithography, meaning this is an inverse design problem. And we are also able to do that for designing gates in quantum dots. This is like nonlinear photonics. And all of this, what is common is the idea that there's a forward model that is simulating the physics, but now what we want is the inverse design, the problem that we can optimize the best design. And humans are usually not good at this, right? We are not good at looking at highly nonlinear phenomena and saying, oh somehow maybe this combination of all these gates coming together helps pull the electrons together in a quantum gate. And so our collaborators were struggling to do that manually, and with AI we're now able to come up with very efficient designs, but also those we know actually work because we have already the simulation as part of the loop saying that they work well. So I think these are examples where we see that it's not just about simulation. It's about really novel designs and novel discoveries that enable us to move the needle of innovation itself.

这些例子中的每一个都需要大量的领域知识。如果是一个领域专家，他们如何利用你的基础研究，快速开始将神经算子和你开发的其他框架应用到他们的问题中？

Each one of these examples takes a lot of domain knowledge. How could somebody take your basic research, if a domain expert, and quickly get started applying neural operators and the other frameworks that you've developed to their problem?

是的，你知道，神经算子是一个开源库。它已经被广泛采用。它是 PyTorch 生态系统的一部分。它不仅被许多研究人员使用，也被公司使用。我们有很多文档。所以我鼓励人们去那里。我们有很多不同的架构、示例和配方。所以我认为那是一个很好的起点。

Yeah, you know, neural operators are an open-source library. It's extensively already adopted. It's part of the PyTorch ecosystem. It's used by a number of not only researchers but also in companies. We have a lot of documentation there. So I encourage people to go there. We have many different architectures, examples, recipes. So I think that's a great place to get started.

你最近加入了联合国科学顾问委员会。我知道我们时间不多了，但也许你能快速讲一下这背后的故事，以及你希望完成什么？

You recently joined the UN scientific advisory board. I know we're running out of time, but maybe just can you quickly give a bit of the story behind this and what you hope to accomplish?

是的，我非常荣幸能成为联合国顾问委员会的一员。在这个充满地缘政治挑战的时代，我并不是这方面的专家。但当涉及到特别是与 AI 相关的方面时，有科学家在场是我认为非常重要的。我希望我能有一个公正的视角，并努力为任何方面提供科学证据。对吧？我们想要思考 AI 如何在全球产生影响，比如我们如何确保 AI 的好处惠及每个人，我们如何使 AI 的获取民主化，我们如何确保意外后果和有害影响得到控制。我认为这些只是开始。当然，在天气模型方面，我已经很兴奋了，因为有一股推动力，看看我们如何能有更好的天气气候建模。然后是粮食，你知道，比如利用天气来改善农业。所以所有这些方面也是联合国在世界各地有很多专门机构和人员的地方。所以我期待做出贡献并成为其中的一部分。

Yeah, I'm really honored to be part of that advisory board for the UN. And in these tricky times with a lot of geopolitics there, which again I'm not the expert on that. But when it comes to aspects especially related to AI, having scientists in the room is something that I think is very important. I hope I can have an unbiased view and try to provide scientific evidence for any aspect. Right? We want to think about how AI impacts globally, like how do we ensure the benefits of AI reach everybody, how do we democratize access to AI, how do we ensure the unintended consequences and harmful impacts can be controlled. I think these are just the beginning aspects. Of course, the other side when it comes to weather models, I'm already excited like there is a push to seeing how we can have better weather climate modeling. So then or food, you know, like using weather for better agriculture. So all these aspects are also where UN has a lot of dedicated agencies and people on the ground across the world. So I'm looking forward to contributing and being part of this.

你知道，看看你的职业生涯，以及你说话的方式和你所从事的工作，你似乎是一个非常喜欢解决具体问题的人。你不喜欢对事情进行哲学思考。而且我认为你可能比 AI 领域的很多人更乐观。你对世界有非常充满希望的看法。我认为不总是这样。你能以哪些独特的方式将这种观点带到委员会，而不是其他人？

You know, looking at your career and how you talk and what you work on, it seems like you very much are a person who likes to solve concrete problems. You don't like to philosophize about things. And you're also, I think, maybe more optimistic than a lot of people in the AI space. You have a very hopeful view of the world. I think not honestly always true. What are the ways that you can uniquely bring that viewpoint to the board versus maybe some others?

是的。

Yeah.

谢谢。

Thank you.

我认为，正如我所说，作为科学家，我尽量保持客观。当我们只考虑有害影响时，往往会忽略 AI 的许多有益方面。这在 AI 用于科学领域时尤其如此，因为许多监管框架将 AI 等同于语言模型。语言模型可以操纵人们，产生各种有害影响，我们应该加以控制，但 AI 用于科学则不同。所以我认为，这种一刀切的做法正是许多问题的根源。我们必须意识到，AI 可以通过新发现改变世界，我们应当让世界各地的人们不仅从中受益，还能进行研究、获得 AI 资源，并以有趣的方式进行创新。

I think, as I said, I tried to be unbiased as a scientist. There are many beneficial aspects of AI that are sometimes missed when we think only of the harmful impacts. This is especially true for AI for science, because many regulatory frameworks equate AI with language models. Language models can manipulate people and have harmful impacts that we should control, but AI for science is different. So I think this one-size-fits-all approach is where many problems arise. We must be mindful that AI can change the world with new discoveries, and we should enable people around the world not only to benefit from them but also to do research, have access to AI, and innovate in interesting ways.

我们一直在向每位嘉宾问一个问题：如果你能神奇地消除你领域中的一个瓶颈，那会是什么，为什么？

One question we've been asking every guest: if you could pick a bottleneck in your domain that you could magically remove, what would that be and why?

更多的算力，你知道，那就是……

More compute, you know, that's...

我知道这是个简单的答案，也许有点偷懒，对吧？

I know that's an easy one, maybe a lazy one, right?

因为，你知道，我们的算力比几年前增长了很多，多亏了英伟达和其他公司……

Because, you know, our compute is growing so much more than even a few years ago, thanks to Nvidia, thanks to others...

又来了？

Again?

所以对此我不予置评。但我的意思是，对于研究来说，提供越来越多的算力非常重要。我知道国家实验室正在建造更多超级计算机，希望我们能有更多算力用于研究。但没有算力，我们就无法实验，无法创新。我认为这是我一直大力推动的一点，我怎么强调它的关键性都不为过。

So no comments on that. But what I mean is, for research, enabling more and more compute is very important. I know national labs are building more supercomputers, hoping we can have more compute for research. But without that, we cannot experiment, we cannot innovate. I think this is a part that I push a lot, and I cannot emphasize enough how critical it is.

如果你有一个行动号召，或者你希望人们去做、去思考、去了解的事情，那会是什么？

If you had a call to action, or something you'd like people to do, think about, or learn about, what would that be?

是的，你可以去神经算子库，亲身体验不同的架构、配方和用例。但也要思考 AI 用于科学——它不仅仅是语言模型和智能体。是的，那是一个方面，但最终，在某种程度上，那些仍然像是外部包装，直到我们拥有完全理解物理世界的 AI，不仅仅是作为符号，而是能够基于此进行模拟、设计和控制。有一个很大的缺失部分。所以这是人们真正应该思考的另一个方面：以这种方式将 AI 用于物理世界。

Yeah, so you can go to neural operator libraries and get hands-on with different architectures, recipes, and use cases. But also think about AI for science—it's not just language models and agents. Yes, that's one aspect, but ultimately those are still like external wrappers in a way, until we have AI that fully understands the physical world, not just as symbols but as one that can simulate, design, and control based on that. There's a big piece missing. So that's the other aspect that people should really think about: AI for the physical world in this way.

Anima，这太迷人了。我很期待自己去看看神经算子。我脑子里已经有一些想法了。非常感谢你抽出时间与我们坐下来交流。

Anima, this has been so fascinating. I'm excited to check out neural operators myself. I have some ideas in my head already. I really appreciate you taking the time to sit down with us.

谢谢，Ar。谢谢，Brandon。我真的很享受这次交流，我们确实深入探讨了许多事情。所以，感谢你们这么做。谢谢。

Thank you, Ar. Thank you, Brandon. I really enjoyed it, and we really dug deep into a number of things. So, I appreciate you doing that. Thank you.

谢谢。

Thank you.