OpenAI 数学突破：发现背后的故事

大家好，我是 Andrew Mayne，欢迎收听 OpenAI 播客。今天这期节目，我们邀请到了推理研究团队的 Alexander Wang、Hongxun Wu 和 Lijie Chen，他们最近在数学领域取得了突破。他们将讲述发现背后的故事，以及他们对反响的感触。

Hello, I'm Andrew Mayne, and welcome to the OpenAI podcast. On today's episode, we're speaking with Alexander Wang, Hongxun Wu, and Lijie Chen from the reasoning research team behind a recent math breakthrough from an OpenAI model. They'll tell us the story behind the discovery and what stood out to them about the reaction.

我激动得都睡不好觉了，太令人兴奋了。

I would have kind of a hard time sleeping because it's so exciting, yeah.

这个模型真的很了不起。

Okay, this model is something that's really amazing.

这成果可以发表在最好的数学期刊上。

I mean, this is something that can be published in the best journal of math.

也许这就是那百分之一的好得令人难以置信，但却是真的。

Maybe this is the one in a hundred times where it's too good to be true, but it's actually true.

Lijie，说说你的工作吧。

Lijie, tell me what you work on.

我和 Alex 一起研究推理。

Oh, I work on reasoning with Alex.

你是怎么进入推理这个领域的？

Okay. How did you find your way into reasoning?

去年夏天，Alex 在 IOI 和 IMO 上取得了突破。我以前参加过 IOI。

Last summer, Alex had this breakthrough in like IOI and IMO. You know, I used to be a participant in IOI.

嗯。

Okay.

我当时想，‘天哪，模型都能拿金牌了。’那时我是加州大学伯克利分校的助理教授。但我在想，‘也许我该重新考虑我的职业规划。’让模型变得更聪明，可能会对世界产生更大的影响。去年十月我和 Alex 聊了一次，然后我就特别兴奋，最后就加入了 OpenAI。

And then I was like, 'Oh, that's crazy, you know, model can already win medals, and gold medals.' At that time, I was an assistant professor at UC Berkeley. But then I was thinking like, 'Maybe I should try to rethink my career.' And it seems like making a model smarter will have maybe some bigger impact on the world. And then I just kind of had a conversation with Alex back in last October. And then I got super excited about this thing, and eventually I just joined OpenAI.

我们经常听到 IOI 和 IMO。Alex，你能给大家解释一下吗？

We hear IOI and IMO come up a lot. Alex, you want to unpack those for everybody?

IMO 和 IOI 是两项面向高中生的竞赛，分别代表国际数学奥林匹克和国际信息学奥林匹克。题目非常难，每场考试分两场，每场四个半小时到五个小时，只做三道题。长期以来，这算是 AI 领域一个隐含的巨大挑战：什么时候模型能像最优秀的人类一样在这些考试中表现出色？

So, IMO and IOI are these two competitions for high schoolers. They stand for the International Math Olympiad and International Olympiad of Informatics, respectively. And these are just devilishly hard math problems. You get two sessions for each of these exams that are like four and a half to five hours, and you just have to do three problems. And so, for a long time these were sort of an implicit grand challenge in AI. Like, when would we be able to get models that could perform as well as the best humans on these exams?

这确实是衡量模型成功的一个很有意思的起点。我们来聊聊从那以后取得了多大的进展，真是不可思议。不过，你是怎么进入推理这个领域的？

That was a pretty interesting starting point, I think, for measuring the success of the model, and we're here to talk about how far things have gone since then, which is pretty incredible. But, how did you find your way into reasoning?

我读的是机器学习博士。博士快结束时，我对在推理时投入更多算力来解决越来越难的推理问题这个想法很感兴趣。当时我在用 API 玩 GPT-3.5 Turbo。

So, I did my PhD in ML. Towards the end of my PhD, I got excited about this idea of spending more compute at inference time to solve harder and harder reasoning problems. At the time, I was playing with GPT-3.5 Turbo in the API.

嗯。

Mhm.

我没得到什么有趣的结果，但 OpenAI 有个团队似乎在研究类似的东西。我特别兴奋，也很幸运能加入。

And I didn't really get any interesting results, but there was this team at OpenAI that seemed to be doing something pretty similar. And so, I got super excited about it, and was lucky enough to be able to join.

最简单的描述可能就是，推理时让模型多思考一会儿。

So, probably the simplest way to describe that is like letting it inference time is basically letting the model think longer about it.

没错。在测试时算力时代之前，模型基本不假思索就回答了。而推理时算力或测试时算力让它们有机会思考、改进答案、尝试不同方法，最后再输出。这显然让它们更聪明，能做那些原本无法瞬间完成的事情。

Yes, that's right. So, basically before this era of test time compute, models sort of answered immediately without thinking. And what inference time compute or test time compute does is you now let them have a chance to think and improve its answer and try different things before having to finally output something. That obviously just helps make them smarter, like lets them do things that they wouldn't otherwise be able to do instantly.

你开始研究推理时，有没有想过希望它发展到什么程度？你的期望是什么？是纯粹从学术角度觉得这很酷，还是有其他愿景？

When you started to work on reasoning, did you have an idea of where you wanted to see this go? Like, what your expectations were? Were you looking at it purely from hey, this is very cool from an academic point of view, or did you have some sort of other vision?

我最初对推理感兴趣，是因为模型显然还做不到这一点。

I think for me the draw of reasoning when I first got excited about it was that this was something that models just obviously can't do right now.

嗯。

Mhm.

那是 2023 年底、2024 年初，模型连小学数学题都做不好。当时我们只想着，能不能让模型在数学上表现合理，更别说比我还强了。我记得入职 OpenAI 第一天，Noam Brown 问我模型什么时候能拿 IMO 金牌。那只是个我们讨论的基准。当时很多研究人员都觉得今年拿金牌没戏，可能要到 2026 年。但我有个想法，如果我们努力推进，也许四月就能做到。结果到六月才做出一个真正好的模型。然后 IMO 来了，我们拿到了金牌。回过头看，这比我预想的快得多。更疯狂的是，从那以后进展一直保持这种惊人的速度。才过了十个月，但 IMO 级别的问题在今天的 AI 看来已经像是遥远的过去了。

So this was like end of 2023, start of 2024, models were struggling with grade school math problems. And so at that time, it was just like, can we just get these models to do something reasonable on math at all, let alone have them be much better than I am at it. I remember when my first day at OpenAI, Noam Brown asked me what I thought models would get IMO gold. That was just a benchmark we talked about. I think at the time, a lot of people even within research thought that IMO gold was out of reach this year, but maybe like 2026. I felt like I had an idea that if we just pushed for it, maybe I thought we could do it by April. It took until June to get a really good model. But then IMO rolled around and we were able to get gold. And I think zooming out, this happened a lot faster than I expected. And it's crazy to me that progress since then has kept up at this same sort of blistering pace. It was just 10 months ago, but it feels like the IMO level of problem feels like far in the rearview mirror of AI today.

Noam Brown 也问过我同样的问题，不过不是关于 IMO 金牌，而是我的模型能不能解决 P 与 NP 问题。

Noam Brown asked me the same question. I mean, not about IMO gold, but about whether my model can solve P versus NP.

嗯。

Mhm.

我觉得 P 与 NP 问题可能非常难。因为要解决它，需要建立一套新理论，可能需要写很多本新思想的书。目前看来我们还差得远。但谁知道未来会怎样呢。

I think P versus NP might be something quite hard. Because I think the reason is that for solving P versus NP, you would need to build a new theory. You need maybe to write many books of new ideas to get there. So currently, it seems we are still far from that. But who knows what will happen in the future. So, yeah.

Hongxun，你做什么工作？

Hongxun, what do you work on?

我之前研究理论计算机科学。读博时和 Lijie 合作很多。我在伯克利，记得这些成果出来时，我跟导师说，模型解决数学问题已经没有障碍了。他只是笑了笑，可能他知道自己快要失去一个学生了。

Oh, I was working on theoretical computer science. I was collaborating a lot with Lijie in my PhD. I was at Berkeley and I remember when all this came out I was talking to my advisor saying oh, there's no barrier in like model solving math problems anymore. I think he just smiled and maybe he knew that he was going to lose a student.

哇。我们来聊聊这个，因为很有意思。就像你说的，模型从只有片刻时间找答案，突然变成了能花更长时间思考推理。结果来得很快，让很多人惊讶。你们的模型基本上推翻了一个 Erdos 猜想。能解释一下吗？

Oh, wow. So, let's talk a bit about that because it's an interesting point because as you said it went from the model just having a moment to try to figure out the answer then all of a sudden you've given it the ability to spend longer and to think about it, you know, reasoning. And the results have come pretty quickly and I think surprising a lot of people. You had a model that was able to basically disprove one of the Erdos conjectures. Could you explain that just a little bit?

是的，上周我们的模型给出了一个证明，确切地说是反证，推翻了 Erdos 的单位距离猜想。

Yeah, so our models last week they were able to produce a proof of the a disprove rather of the unit distance conjecture due to Erdos.

这是一个组合几何领域存在了 80 年的未解问题：如果你在一张纸上有 n 个点，其中有多少对点可以恰好相距 1 英寸？这个数量如何随着点的数量渐近增长？

And this was an 80-year-old open problem in the field of combinatorial geometry where basically the question concerns if you have n points on a piece of paper, how many of them can be 1 inch apart exactly? And how many pairs can be 1 inch apart exactly? And how does this number grow asymptotically with the number of points?

Erdos 提出这个问题时，它并不简单。想法是可以在平面上解决，但可能没有更好的方法。这个问题一直存在，因为它非常有趣。一个模型解决了这个问题，意义深远。而且这个模型是通用模型，对吗？

This wasn't a trivial problem when Erdos put this together. The idea was to say that it could be done on a plane, but there was the idea that maybe there was no better way. And this has been out there because it's a very interesting problem. And the fact that a model solved this is pretty profound. And also this model was a general purpose model, correct?

是的，没错。Erdos 最初的猜想是，在平面上让尽可能多的点相距 1 单位的最优解是将它们排列成正方形网格。

Yes, that's right. So, Erdos' original conjecture was essentially that the optimal solution to having as many distance one points on the plane was to arrange them in a square grid.

而模型证明，正方形网格实际上远非最优。通过使用大量高深数论的不同构造，可以做得更好。

And what the model proved was that the square grid was not actually close to optimal at all. And that you can do much better with a different construction using a lot of high-powered number theory.

Jean，你们是如何选择这些问题的？

And Jean, how did you choose these problems?

我想我们并没有真正选择问题。事情是这样的，我们想测试模型能力的上限。所以我们只用了 Erdos 问题的一个子集来测试模型的能力。

I guess we didn't really choose the problem. What happened was we wanted to test the upper bound of our model's capability. So, we just used a selected subset of Erdos' problems to test the capability of the model.

我想知道，是谁按下回车键向模型提问的？

I would love to know one, who is the one that hit enter and asked the model the question?

我想是我们俩，我和黄轩。

I guess both of us, like I and Huang Xuan.

你们同时按下的？

You guys at the same time, like pressed?

是的，也许吧。实际上我们在测试两个略有不同的内部模型。我们都看到了一些正确的解决方案。这对我们来说非常非常激动。

Yeah, maybe. I think what happened was actually we were testing like two slightly different internal models. And we both saw like some correct solutions. It was really really exciting for us.

你们怎么知道它有效？

How did you know that it worked?

当然，首先让模型自己检查。

Of course, you first ask the model to check it.

好的。

Okay.

但当然，模型有时不可靠。所以，我们用模型检查后，看起来合理，就问了公司里其他一些数学朋友，比如 Matab 和 Max Salkin。起初他们觉得这不可能是真的，这是一个重大未解问题。但思考一天后，他们找不出任何错误，就变得更确信了。最终，他们说这可能是正确的。然后大家都激动得难以入睡。

But of course, models sometimes they are not reliable. So, yeah, then we just after we checked it with the model, it seems plausible, then we just ask a bunch of other mathematics friends in the company, like Matab and Max Salkin. And at first, they were like, 'Oh, there's no way this can be true. This is a major open problem.' But after they think about it for a day, they couldn't figure out any mistake. Then they become more convinced. Then eventually, they are like, 'Actually, this may be correct.' Yeah, then like everyone had a hard time sleeping because it's so exciting.

当人们开始说这是准确的时候，你们的对话是怎样的？

What was the conversation like when you started to get people saying that this was accurate?

对我来说，我并不那么惊讶，因为起初我的上司说这肯定是错的。但我知道他可能只花了 5 到 10 分钟检查。所以我心里并不相信。后来他告诉我 50%可能。我想，如果按这个趋势，也许第二天晚上就 100%了。所以，这有点梦幻，但也觉得模型做出惊人事情很自然。后来越来越真实，这可能是正确的，可能是个大事件。第一次 AI 能发表能进顶级数学期刊的东西。我们知道这一天会来，但没想到这么快成真。就像活在梦里。

For me, I was not that surprised because I guess when my top first say, okay, first what happened was first my top say this is definitely wrong. But I actually knew that he probably just spent like 5 minutes, 10 minutes counting it. So, like in my heart I don't really believe that. Um, but later he told me it's 50%. I was thinking, okay, if we extrapolate the trend, then maybe next night if we 100%. So, yeah, I guess it's a little bit dream-like, but also was like it feels a bit natural that this model would do something amazing. Later it just become more and more real that this might actually be correct. This might actually be a big deal. The first time Auto can publish something that would get it into top math journals. It's like we knew this day is going to come but never knew that it's going to become reality so fast. It's like living your dream.

我的意思是，这可以发表在最好的数学期刊上，远超 IMO 水平。所以我预料到会有这么一天，但没想到是今年五月。

I mean, this is something that can be published in the best journal of math. It is way beyond IMO level. So, I would expect something to happen at some point, but maybe not this May.

我认为 OpenAI 强调的一点是，他们不针对特定基准训练，而是构建真正优秀的通用模型。有时人们说，我们只是构建一个普遍智能的模型，然后在这个过程中发现这些能力。推理也是如此。一个整体推理能力很强的模型，你会发现这些能力。你觉得是这样吗？

One of the things I think that we've seen emphasized at OpenAI is that OpenAI doesn't try to train to specific benchmarks. OpenAI tries to build really good general overall models. And I think sometimes people say, well, we just try to build a generally smart model and we find these things along the way. And when it comes to reasoning it's the same thing. Something that's really good at reasoning overall, you find these capabilities. Does that ring true for you?

是的，特别是这个模型，我们所有人都用它来代替 Codex 中的当前模型。它作为通用模型工作得很好。具备解决 Erdos 单位距离问题的能力，我认为在不久的将来人们在家也能做到。

Yeah, I think for this model in particular, I think it's one that all of us have also just used in lieu of the current model in Codex. And it works quite well as just a general purpose model. Having the capabilities to do this Erdos unit distance result, I think people will be able to do this at home in the near future.

看到人们的反应和关注很令人兴奋。不久前人们还说模型不擅长数学，现在模型做到了这个。你在网上看到哪些有趣的事情或人们的反应？

It's been exciting to see people react to this and pay attention to this. We went from just a very short period of time ago where people said models weren't good at math, and now models are doing this. What have been some of the more fun things you've seen online or reactions from people?

自从我们宣布结果后，我在 TC 的朋友们开始让我在他们的未解问题上尝试，包括我的导师给了我两三个未解问题。我认为反应非常积极。

Ever since we announced our results, my friends in TC have started to ask me to try it on their open problems, including my advisor gave me like two three open problems to try on. I think the reaction was very positive.

我认为人们真正感受到，当今 AI 的前沿能够产生许多人类数学家会引以为傲的研究成果。我们能向世界传达这一点，真是太棒了。我看到人们绘制这些设计，试图勾勒出几乎完成的构造，如果画在网格上，它实际上是一个非常漂亮的对称几何设计。

I think people really get a sense that the frontier of AI today can really come up with research output that I think many human mathematicians would be proud to achieve. I think it's really great that we're able to communicate this to the rest of the world. I've seen people make these designs of trying to sketch out the almost construction, if you plot it on a grid it's actually like this very pretty symmetric geometric design.

是的，我们想也许可以做一个设计，装裱起来放在桌上之类的。可以庆祝这一刻。

Yeah, I guess we are thinking maybe try to make one of the design, put them in a frame and put them on desk or something. Can do celebrate this moment.

是的，当我们开始看到像铺砌问题之类的东西时，会很有趣，我们可以直接查看需要的产物。最近我们听到更多关于 Erdos 问题，有些似乎并不像人们想象的那么难解决，它们只是需要一些关注。

Yeah, I think it's going to be fun when we start seeing things like tiling problems and other stuff where we can actually just look at the artifacts when we need. So, we've been hearing more about Erdos' problems lately, and some seem like they weren't as challenging to solve as perhaps people thought. They just needed some attention.

不过，这个问题似乎更复杂一些。你会怎么给它排名？

Yet, this one seems to be a little bit more complicated. Where would you rank this?

Erdős，我想他提出了大概一千个或更多的问题，对吧？所以，他的问题集就是他问过的所有问题的集合。有些问题他悬赏了奖金，有些他只是记下来。而这个问题，他悬赏了 500 美元。

Erdős, I think he proposed like a thousand questions or more, right? So, like he, you know, Erdős' problem is just a collection of all the problems he has asked. Some problems he has offered some money for solution. Some problems he just noted. And this problem he offered, I think, $500.

嗯。

Mhm.

而且这是上世纪的问题。所以，有点……嗯。这也是离散几何领域的核心问题之一，在许多离散几何论文中被大量讨论。所以，这是人们思考了很多、非常想看到答案的问题之一。我会说这更像是一个具体数学领域的主要开放问题，而不是其他一些 Erdős 问题，那些可能只是 Erdős 午饭后随口问的。

And which is from last century. So, it was a little bit. Yeah. And also this is one of the central questions in this field of discrete geometry. And it has been heavily discussed by mathematicians in many discrete geometry papers. So, it's kind of one of the questions people have thought about a lot and really want to see the answer. So, I would say this is more like a major open problem in a concrete field of mathematics instead of some other Erdős questions which may be just something like Erdős asked after lunch or something.

那么，怎么领取那 500 美元？他去世后奖金就消失了吗？

So, how do you collect that $500? Did it disappear when he passed away?

我想有个专门的机构负责，但通常人们只是把支票裱起来。

I think there's a special agency for that, but usually people just frame the check.

嗯。

Yeah.

是啊。所以，也许我们就把支票裱在 Sam 的办公室里。我也不知道。

Yeah. So, maybe we'll just frame the check in Sam's office. I don't know.

你觉得这如何证明了推理是有效的？

How do you feel this proves that reasoning is effective?

嗯，我认为最大的证据是，如果你看官方博客的图表，给模型更多思考时间，它在这个问题上的准确率提升得更快。如果给很多时间，几乎能达到 50%的正确率。所以，思考越多，正确率越高。我认为这确实证明了推理是有效的。

Well, I think the biggest proof is that if you look at the plot in the official blog, if you give the model more time to think, the accuracy on this problem goes faster. Like if you give it a lot of time, you can get almost 50% correct. So, more thinking, more correctness. I think that's really a proof of reasoning being effective.

但我也想回到这一点。这不是一个数学模型。这是一个能做很多不同事情的模型。你看到它在解决数学等问题上变得更好，与其他通用问题的能力之间存在关联吗？

But I'll also go back to this. This isn't a math model. This is a model that can do a lot of many different things. Do you see a correlation between as these get better at solving things like mathematics that it works with other general problems?

至少假设是这样的：这个模型并没有专门针对数学进行训练。所以我们只是想，我们有了这个新模型，本质上想试驾一下。于是我们在一些非常具有挑战性的数学问题上评估它，看看它能做什么。

That's the hypothesis at least is that, you know, this model was not trained specifically for math. And so we just wanted to, like, we had this new model. How we came about this we wanted to take it on a test drive essentially. And so we evaluated it on some very challenging math problems just to see what it can do.

当你仔细看这个证明，看它得出的结果，有没有让你惊讶的地方，有没有你觉得可以称之为有创造力的地方？

When you go through the proof and you look at what it came up with, were there things that surprised you, things that you would describe as creative?

先说背景，这个证明远超出我自己的数学水平。但粗略来说，我的理解是，将类域论应用于组合几何问题的想法以前从未真正实现过。虽然有些人知道这两个领域之间可能存在桥梁。能够做到这一点并执行它，首先需要建立联系，这需要相当惊人的创造力，然后执行证明也是一件非常精细谨慎的事情，很少有人能做到。

So for some context, the proof is well above my own mathematical pay grade. But just at a high level, my understanding was that this idea of taking class field theory and applying it to problems in combinatorial geometry hadn't really been done before. Though some people knew there could be this bridge between these two fields. Being able to do that and execute it requires first of all to make the connection, which requires quite a bit of insane creative creativity, and then to execute the proof is also a very delicate careful affair that very few people would be able to do.

对我来说最惊讶的是，你让模型做某事，然后去吃午饭，回来时发现它做得比你想的好得多。那一刻你觉得，好吧，这个模型真的很了不起。

I think the most surprising thing for me is you tell the model to do something and you go to have lunch and when you come back you see that it actually does much better than you thought. And at that moment you feel like okay, this model is something that's really amazing.

回到 GPT 3.5 Turbo，当时那个模型在做自动即时推理和预判，现在这个模型能完成惊人的数学证明。它使用工具吗？使用 Lean 吗？还是完全在模型内部完成？

So going back to GPT 3.5 Turbo and working with that and looking at a model that was doing automatically instant sort of inference and pre-figuring these things out to now a model that's able to do incredible mathematical proofs. Is it using tools? Is it using Lean? Is it using some other things like that? Or is it doing purely inside the model?

在这个案例中，模型基本上像 Codex。它能编程，能查看网站并找到信息。所以它基本上是一个通用的 ChatGPT 设置。ChatGPT 也能写 Python 并执行。但我不认为模型会写任何东西。

For the case, the model basically is like Codex. It can code. It can look at the website and find information. Yeah, so it's basically a general ChatGPT setup. ChatGPT can also write Python and execute them. But I don't think that the model would write anything.

我想他们有个关于剑桥词典的故事。

I think they just have a story about the Cambridge dictionary.

哦，好吧。所以模型访问网站时做的第一件事就是查剑桥词典里“unit”的意思。有点荒谬，是吧。

Oh, okay. So, the first thing the model does when it gets to the website is to check what 'unit' means in the Cambridge dictionary. It's a little bit ridiculous, yeah.

所以它查了“unit”这个词？

So, it looked up the word 'unit'?

是的，它想确保自己对“unit”的理解绝对正确。

Yeah, it wants to make sure it has the absolute correct understanding of what is unit.

你见过它做其他类似的事情吗？比如看到它试图让自己扎根，确保理解问题。

Have you seen it do other things like that where you're seeing like, 'Oh, it's trying to ground itself to make sure it understands the question.'

确实，很多时候模型在回答时，会再次解释定义，以表明它确实扎根于定义。

Definitely, a lot of time in the model answering, it will actually explain the definition again to show that it actually grounded definitions.

作为非常了解计算机科学和数学的人，看到这一切发生，我们会不会感到畏惧？

As people who are very knowledgeable about computer science, people who know a lot about mathematics, is it intimidating to all of us and see this happen?

我认为不应该畏惧。实际上，我认为这应该是赋能。证明出来后，数学家们首先改进了边界证明。其次，他们利用构造的直觉和动机也攻克了其他开放问题。所以，我认为这个趋势会继续。模型能在一些我们不知道如何解决的难题上取得突破。但如何消化那个想法，如何将那个方法用于其他好事，我认为人类仍然有角色。

I think it should not be intimidating. I actually think it should be empowering. After the proof actually came out, mathematicians first improved the boundary proof. And secondly, they used the intuition and motivation of the construction to knock down other open problems as well. So, I think the trend is going to continue. Models can make good breakthroughs on some very hard questions we don't know how to solve. But then, how to digest that idea, how to use that method for other good things, I think humans still have a role in this.

那么，你认为五年后从事数学工作的人的角色会是什么样的？

So, what do you think the role of somebody working in mathematics is going to be like 5 years from now?

我认为会有很多 AI 和人类的合作。是的，因为现在的 AI 知道很多，对吧？它们能连接遥远的想法，但人类也能思考更长时间。比如目前看来 AI 还不能为数学建立新理论。但我想人类一旦有了 AI 的帮助，就能从数学的不同领域获取所有想法，我认为这能极大地赋能人类。

I think there will be a lot of AI and human collaboration. Yeah, because AI now, they know a lot, right? They can connect distant ideas, but humans can also think for longer. Like currently it seems AI cannot build a new theory for math, for example. But I guess humans, once they have the help of AI, they can just grab all the ideas from different fields of math, I think they can empower humans way more.

你看到这延伸到其他领域了吗？我们会看到物理学上的发现吗？

Do you see this working into other fields? Are we going to see discoveries in physics?

所以，我无法代表物理学发言，但我想我们都是 AI 研究者，对我来说，我的日常工作与我刚开始研究这个领域时完全不同。现在我的很多工作都由编码智能体完成。我能做更多的事情。我认为这是一种神奇的感觉，有了 AI，你真的开始觉得可以用 AI 更快地构建 AI。

So, I mean, I can't speak for physics, but I guess we're all researchers in AI, and I think definitely for me, my day-to-day looks completely different than when I first started doing research in this field. I think so much of my work is now done by coding agents. I can just do so much more. And I think that's been a sort of magical feeling that with AI, you're really starting to feel like you can use AI to build AI faster.

AI 在多大程度上改变了你们做这些事情的方式？

How much has AI changed the way you do these sorts of things?

我觉得完全改变了。甚至就在半年前我刚加入时，我还在手写代码，在 Slack 频道里找指导，但现在默认就是直接问 Codex。我让 Codex 做很多事情，然后我就去吃午饭，去和人聊天。工作完全变了。

I think totally completely. Even when I just joined half a year ago, I was hand coding the codes, looking up the Slack channels for directions, but now the default is just ask Codex. And I ask Codex to do a lot of things, and then I just go to lunch, I just go to talk to people. The work completely changes.

而且我在手机上用 Codex，你可以随时查看。

And I use Codex on your phone when you can check on it.

是的。有趣的是，有了这种可以一直工作的工具，我现在想做更多事情了。

Yeah. It's interesting how much more I want to do things now that you have this sort of tool that can work all the time and do stuff.

Lejay，你怎么向朋友们解释这件事，让他们理解这意味着什么以及它将如何影响其他领域？

Lejay, how do you explain this to your friends who are trying to understand what this means and how it's going to impact other fields?

我有些数学家朋友，还有其他领域的朋友。我想我会告诉他们，有些人可能害怕 AI 会取代他们，但我认为这实际上是赋能每一位理论研究者。因为 AI 确实有优势，它知道很多东西并能建立联系。目前看来，对人类困难的问题对 AI 可能并不难。这是一件非常好的事情。我们可以用 AI 解决这些问题，获得新想法，然后消化它们并做出新发现。所以我觉得有些人对此非常兴奋。有一点是，目前这还只限于数学，但我相信因为这是一个通用推理模型，其他理论研究者也能从中受益。我认为理想的世界是每个人都能接触到顶级的推理能力。这样所有研究者都可以用它来发现他们想发现的东西。基本上，OpenAI 将大大加速科学，因为你正在赋能每一位科学家来加速全球科学。我认为这是我们的使命。

So, I have some mathematician friends and some friends in other fields. I think the way I would tell them is that I feel like some may be afraid that AI will replace them, but I think it's really about empowering every theoretical researcher. Because AI really has this advantage of knowing so many things and connecting things. Currently, it seems like the problem hard for human may not be hard for AI. And that's a really great thing. We can use AI to solve those problems, get new ideas, and then we can digest them and make new discovery. So I think some of them get very excited about this. One thing is that currently it's only on math, but I believe that because it's a general reasoning model, other theoretical researchers can benefit a lot from that. I think the dream world will be like everyone have some access to the top-level reasoning ability. So all the researchers can use them to discover whatever they want to discover. And then basically, OpenAI will accelerate science a lot, because you are empowering every scientist to accelerate science worldwide. I think that's our mission.

那么，如果我是一名研究者，我该如何开始？你有什么建议？先从你开始，Hongqing。

So, if I was a researcher, how would I get started? What advice would you have? We'll start with you, Hongqing.

订阅 GPT Pro。这比不用 Pro 思考要好得多，因为它思考时间更长。试着问你能问的最大胆的问题。我有一次经历，我试着把问题分解成小问题去问模型，结果发现不如直接问问题好，因为我的分解不是最佳方式。

Get GPT Pro subscription. It's really much better than thinking without Pro. Because it thinks longer. And try to ask the boldest question you can ask. I had an experience that sometimes I try to decompose a problem into smaller problems and ask the model. And turns out that it was not as good as just directly ask a question because my decomposition was not the best way.

你觉得这是为什么？

Why do you think that was?

我认为是因为作为人类，我们对问题应该如何解决有各种先验，这些先验在减少思考时间上很有帮助。但很多时候这些先验是错误的，存在盲点，而 AI 模型有时能通过发现这些隐藏的东西给我们惊喜。

I think because as humans we have all kinds of priors on how problems should be solved and they are very helpful in reducing thinking time. But very often the priors are wrong and there are blind spots, and AI models sometimes can surprise us by discovering these hidden things.

我和 Alex Muchasta 聊过，他谈到要像对待研究生一样对待它。不要说得太低也不要太高，在合适的水平上，这样它就能理解术语并为你工作。Alex，你呢？你会给想更有效使用它的人什么建议？

When I spoke to Alex Muchasta, he talked about treating it like a graduate student. Not talking down too low but not too high, at the right level so it could understand the terms and work for you. Alex, how about for you? What advice would you give somebody who wants to be more effective with this?

是的，我认为很大程度上是学会信任模型，弄清楚你能在多大程度上信任模型，同时也要了解模型不能做什么。因为如果你没有这种感觉，你就不能最大化地利用模型的全部能力。我觉得 Lije 教了我很多如何更好地使用这些工具。在采用方面，我觉得自己在某些方面像恐龙，因为我在这些工具出现之前很久就在 OpenAI 工作了，所以有很多旧坏习惯，不够信任模型。我还以为模型是六个月前的样子。

Yeah, I think a lot of it is actually learning to trust the model and figuring out how far you can go in trusting the model and also learning what's beyond what the model can do. Because if you don't have a sense of that, you don't maximally use the full capabilities of the model. I think Lije has taught me a lot about how to use these tools better. I feel like I'm sort of a dinosaur in some respects in terms of adoption because I started working at OpenAI well before these tools existed, so I have a lot of old bad habits where I don't trust the models enough. I still think it's like the models of 6 months ago.

这是一个有趣的范式。好的。那么，Lije，你有什么建议？

That's an interesting paradigm. Okay. So, Lije, what advice would you give?

哦，我有这个：搞砸它。每次你对模型的信任加倍，然后看它为什么失败。如果失败了，你就退回去。然后你每个月都这样做。这样你就能很快达到既能最大程度信任模型又不破坏东西的程度。显然在过去五个月里，这呈指数级增长。

Oh, I have this: mess it up. Every time you double your trust on a model and see why it fails. And if it fails, you just go back. And then you do this every month. Then you can quickly get to the point where you can maximally trust the model but also not breaking your stuff. And apparently for the last 5 months it's going really exponentially.

在 GPT-3 时代，我有一系列测试要做。我看着它们逐渐变好，然后是 GPT-4，等到 01 出来时，我不得不扔掉它们，因为那只是玩具问题了。我觉得我必须不断调整，不断尝试更复杂的事情。你认为对于数学或相关领域感到担忧的人，他们应该采取更乐观的态度吗？

Back in the GPT-3 days, I had a list of tests and things I would do. And I'd watch them incrementally get better, and then GPT-4, and then by the time 01 came out, I had to throw it out because that was just toy problems at that point. I feel like I have to continuously adjust and keep trying figuring more complicated things to do. Do you think that for somebody in mathematics or a related field who's feeling concerned, they should take a more optimistic approach?

我认为感到担忧是合理的。尤其是当领域很大程度上以解决问题为导向时。因为模型会非常擅长解决问题。但数学远不止解决问题，它更多的是理解结构和构建新理论，就像 Li Jie 说的。我认为我们应该想办法更好地利用模型来帮助我们解决遇到的问题，然后加速我们构建新理论和获得新理解的速度。我认为这是更乐观的观点。

I think it is legit to feel concerned. Especially when the field is a lot like problem-solving oriented. Because models are going to be really good at problem-solving. But mathematics is really much more than problem-solving. It's more about understanding the structure and building new theories, like Li Jie said. I think we should try to figure out how to better use the model to help us in solving the problems we meet, and then try to accelerate the speed that we build new theory and come up with new understandings. I think that's the more optimistic view.

当 Codex 变得更好时，它能做更多事情。你会期望因为 Codex 好而工作更少。但不知何故你实际上工作更多，因为你能做的事情更多了。所以我希望数学也能这样。模型变得如此之好，那么我想象你有 10 个想法，你可以让 10 个模型去尝试，看看哪个成功。而且你不需要自己做繁琐的计算。所以我想象编程领域发生的事情也能发生在数学家身上。

When Codex becomes much better, they can do so much more. You would expect you are working less because Codex is good. But somehow you actually work more because there are way more things you can do. So I hope this can happen for math as well. The model becomes so good, then I imagine you have 10 ideas, you can ask 10 models to try them and see if one succeeds. And you don't have to do tedious calculation by yourself. So I imagine what happened with coding can happen to mathematicians.

确实如此，因为当我们谈到 Erdős 问题时，Paul Erdős 是一个非常有趣的人，他对很多事情感到好奇，会说'哦，这很妙'。他整理了一类问题，但这些问题并没有太多逻辑或理由，只是他觉得好奇或者和别人一起研究的东西。我认为这正是科学的美妙之处：我们常常认为存在特定的层级结构，但实际上可能只是我们好奇的东西。话虽如此，距离 Erdős 问题全部解决还有多久？

It does actually because, you know, when we talk about the Erdős problems, Paul Erdős was a very interesting person who found a lot of things curious and said, 'Oh, this is neat.' And we have this category of problems he put together, but there's not a lot of rhyme or reason to them. They were just things that he found curious or worked with other people on. And I think that's kind of a big thing that's neat about science in general: often we think that there are these real specific hierarchies, but literally it can just be things we're curious about. That being said, how long before there are no more unsolved Erdős problems?

其中一些问题非常非常难。

Some of them are very, very hard.

是的。

Yeah.

嗯，所以我不知道。

Yeah, so I don't know.

你预见到我们可能需要提出一类新的问题吗？

Do you foresee us maybe needing to come up with a new category of problems?

我认为那个列表上最难的问题，比如 Collatz 猜想，这些问题感觉远远超出了当今数学技术的能力范围，尽管其中许多表述非常简单。

I think that probably the hardest problems on that list, that list includes the Collatz conjecture. These are problems that feel very, very far out of reach of the mathematical technology of today, even though many of them are quite simple to state.

所以我们仍然有更多事情要做，并不断推进。这让人欣慰。但令人兴奋的是，想想当开始将这种方法应用到物理和天文学等其他领域，开始查看数据集等，会发现什么样的发现。你有特别希望看到的领域吗？

So we'll still have some more things to work on and continuously move things through. That's good to know. It's exciting though, too, just to think about what happens when you start applying this to other areas in physics and astronomy and start looking at data sets and stuff, and what kind of discoveries are going to be in store. Do you have any particular area that you're hoping to see?

哦，我希望解决 P 与 NP 问题。

Oh, I'm hoping to solve P versus NP.

好的。你呢，Alex？

Okay. How about you, Alex?

我认为我心目中的下一个里程碑是能够进行 AI 研究的 AI。我认为这里有很多未解决的问题，在某种意义上受限于我们自身智能的局限性。我对 AI 作为一种广泛可用的技术持乐观态度，因为世界上对智能的需求远远超过人类所能提供的。

I think the next milestone in my head is really like AI that can do AI research. I think there are so many unsolved problems here where it's in a sense limited by all the limitations of just our own intelligences. I'm optimistic about just having AI broadly available as a technology because there's just so much more demand for intelligence in the world than humans can supply.

是吗？

Did you?

哦，我本来想说 P 与 NP，但不幸说出来了。是的。所以，除此之外，我特别感兴趣的一个具体问题是，目前 AI 似乎试图以非常新颖和复杂的方式结合不同领域的想法，但 AI 能否真正从零开始产生全新的想法？这是我们在 AI 中还没有具体看到的。我希望接下来能看到这一点，那将非常酷。

Oh, I wanted to say P versus NP, but unfortunately said it. Yeah. So, I guess beyond that, one concrete thing I'm very interested in is that currently it seems AI is trying to combine ideas from different fields in a very novel and sophisticated way, but can AI actually generate completely new ideas from scratch? That's something we haven't really seen concretely in AI. And that's something I maybe want to see next happening, and that can be very cool.

你看到过这方面的迹象吗？

Have you seen traces of that yet?

我认为有。即使在这个最新问题中，如果你看那个思维链，大约有 125 页，我认为有些想法相当有创意。

I think so. Even in this latest problem, if you look at the chain of thought, which is like 125 pages, I think some of the thoughts are pretty creative.

嗯。

Mhm.

虽然它们没有成功。最终的想法更像是把所有东西结合起来，但其中有一些有创意的想法。

Although they didn't work out. The final idea is more like combining all the stuff, but some of it has some creative thoughts.

但有趣的是，早期的论点认为这些模型没有创造力，但你可以给它两个从未联系过的想法，问'它们之间有什么关系？'那就会产生非常新颖的东西，感觉与众不同。我觉得我们可能会看到更多这样的情况。你认为我们会提出新的数学形式吗？

But it's interesting, you know, early on arguments were like these models weren't creative, but you could give it two ideas that never been connected before and say, 'What's the relationship?' And that would be something very, very new and felt like something different. And I feel like we'll probably see more of that. Do you see us coming up with new forms of mathematics?

我认为那实际上会更晚一些。

I think that actually will be further down the line.

嗯。

Mhm.

因为模型非常擅长提出解决某个问题的想法，但不擅长提出全新的数学类型或新理论。如何让模型做到这一点仍然非常开放。

Because models are very, very good at coming up with some idea to solve a problem, but it's not good at proposing a completely new different kind of math or proposing new theory. How to get model to do that is still very, very open.

我的看法是，我们看到这些模型有效的时间跨度类似于摩尔定律。在数学中，你几乎能感觉到，每隔几个月，这些模型能独立工作的时间就会翻倍，至少是人类等效时间。所以对于解决问题，如果你非常擅长，有些问题的解决路径可能很短，不需要那么长时间。但我认为，对于发明新的数学方法，那更像是一个数年或数十年的过程。所以我认为指数增长还需要一些时间才能达到那个水平。

How I would think about it is we see this like Moore's law for the time horizon at which these models are effective. And I think you sort of feel that in math where every few months the amount of time these models can sort of work independently for doubles, at least the amount of human equivalent time. So for solving problems, if you're really good at it, maybe some problems actually have pretty short paths for the solution. You don't need to take that long. But I think for inventing new ways of doing mathematics, that's much more like a years or decades long process. So I think it'll still take a bit of time for that exponential to get there.

这是你们内部模型完成的。从那以后，5.5 也能做到同样的事情。我们看到其他实验室也表示他们能做到。但那是几周前的事了，现在已经是老黄历了。从那以后我们看到了什么？

This was done by an internal model that you guys worked on. And since then 5.5 has been able to do the same thing. We've seen other labs have said that they've been able to do this as well. But this was several weeks ago, which is now ancient history. What have we seen since then?

我认为原始结果和后续发现的一个区别是，对于原始模型，不需要任何脚手架。你只需让它解决问题，它就会给出答案。你可以在我们上传到博客的笔记中看到原始提示和回答。而后续的努力则对模型有更多的结构或引导。但我认为一个有趣的数据点是，这完全关乎推理时算力扩展。在我们最初解决问题后，只要有足够的推理时算力预算，模型大约有 50% 的时间能解决问题。所以用其他方法也能达到这一点并不奇怪。但真正重要的是，随着你投入更多的推理时算力，你会得到更好的结果。

I think one difference between the original result and the follow-up findings is that for the original model, there was no scaffolding needed. You just asked it to do the problem and then it gave you the answer. So you can read the original prompt and response in the note we uploaded on the blog. Whereas the follow-up efforts have had a little bit more structure or steering of the models. But I think one interesting data point here is that it's really all about test-time compute scaling. After we initially solved the problem, with enough test-time compute budget, the model is able to solve the problem around 50% of the time. So it's not surprising that you can get there with other methods as well. But what was really important here is that as you pour in more test-time compute, you get better results.

这似乎是一个良性循环：你拿今天的模型，给它更多算力，让它解决这个问题，然后你了解这些发现如何被解决。下一代模型可以从中学习，变得越来越高效，基本上似乎可以永远扩展下去，对吧？你认为到今年年底我们会看到什么？

It seems like it's kind of a virtuous cycle where you take today's model, give it more compute, let it solve for that, and you understand how these findings can be solved. Next generation models can learn from that and just get more and more efficient, and you just have this basically, it seems like it just scales forever, right? What do you think we're going to see by the end of the year?

我希望看到人们用我们的模型发现大量新东西，不仅在数学领域，也在所有科学领域。当然，OpenAI 想做些酷炫的数学工作，但我认为如果每个人都能用这个模型去发现自己的科学，那会更好。

What I want to see is that people use our model to discover lots of new stuff, and not only in math, but also in all of science. Of course, OpenAI wants to do some cool math stuff, but I think it would be better if everyone can use the model to discover their own science.

嗯。

Mhm.

而且我预计许多数学家会使用这个模型。

And I would expect many mathematicians will use the model.

我的意思是，可能不完全依赖模型，但我与模型合作来发现更多数学成果。我觉得那会很酷。

I mean, maybe not completely on the model, but I collaborate with the model to discover a lot more math results. I think that'll be really cool.

我和一些数学家聊过，他们认识其他人，或者他们自己非常不愿意尝试在数学中使用 AI。你能给出的最好理由是什么？

I've talked to some mathematicians who know others or who are very reticent to even try using AI in mathematics. What is the best argument you can give?

我想我会直接给他们看距离猜想的证明。这个猜想的证明。我认为这关乎生产力，对吧？我们做数学不仅是为了享受解题的乐趣，也是为了推动领域发展，理解我们寻找的真理。而使用 AI 会大大加速这个过程。它会告诉我们真正难以找到的东西，总有一天会让人难以抗拒使用 AI。

I think I'll just show them the proof of the distance conjecture. This proof of the conjecture. I think just about productivity, right? Like we do math not just to enjoy the pleasure of problem solving, but also to advance the field and to understand the truth that's what we're looking for. And using AI is going to speed that up by a lot. And it's going to tell us what we are really struggling to find, and it will be hard to resist using AI at some point.

你可以当一个不用望远镜的天文学家，但你得问为什么。

You could be an astronomer not use a telescope, but you kind of has to ask why.

是的。

Yeah.

确实。

Exactly.

我知道这里有一位研究员喜欢看电脑互相对弈，他觉得自己有时能从中学习。你认为我们通过观察模型得出的解决方案，能成为更好的数学家、研究者、科学家，或者仅仅是更好的思考者吗？

I know one of the researchers here likes to watch computers play chess against each other and he feels like he sometimes learns things from that. Do you think that we'll learn to be better mathematicians or researchers or scientists or just thinkers in general by watching the solutions the models come up with?

看那 125 页的思考过程，对数学家来说可能没什么帮助。但只看答案，你确实能学到一些以前没有的想法，一些我之前不知道的东西。

Looking at the 125 pages of thinking, it's probably not very helpful for a mathematician. But just by looking at the answer, you actually do learn some idea that was not there before, that I didn't know before.

嗯。

Mhm.

而这启发了后续的数学工作，解决了其他问题。所以我确实认为人们，比如数学家，能从 AI 解决方案中学到东西。是的，我们请来和合作者一起审阅证明的一些数学家，实际上利用这个想法否证了和积猜想，不过是在实数域上。我认为这是一个很好的例子：AI 能攻克重要问题，并给我们提供可以应用到其他领域的想法。

And that inspires the later mathematical works that knocked out other problems. So I definitely think people learn, like mathematicians learn something from AI solutions. Yeah, so some of the mathematicians that we asked to review the proof together with collaborators, they actually used the idea to disprove the sum-product conjecture, but for real numbers. I think that's like one very good example: AI can crack down important questions and give us ideas that we can apply elsewhere.

嗯。是的，我觉得很了不起的是，这群数学家已经在一周内用它否证了这个结果，我认为其重要性可能和单位距离猜想相当。所以我认为这是一个绝佳的例子，数学家们看到了这一点，将其作为灵感，并将这些想法应用到另一个问题上。

Mhm. Yeah, I think it's remarkable that like you know like this group of mathematicians has already like just in the span of a week already used it to, you know, disprove this result that I think is maybe of like similar importance to the unit distance conjecture. So I think this is a wonderful example of mathematicians like seeing this and using it as inspiration and bringing the ideas to bear on a different problem.

这对数学界意味着什么？

What does this mean for the mathematical community?

我认为对我们来说，做这些实验时，我们想确保赋能我们互动的学术社区。我们不是去某个社区，从外部试图解决一堆问题，然后给他们一堆 AI 垃圾，而是真正想做的是让研究人员能使用这些工具，让他们把 AI 测试时算力引导到他们认为重要的问题上。我认为这不应被视为一场尽可能多解决埃尔德什问题的竞赛，而更像是我们想让人们意识到这项技术已经存在。这就是它能做到的。

I think for us, when we do these experiments, we want to make sure we empower the academic communities we interact with. We don't just go to some community and from the outside try to solve a bunch of their problems and give them a bunch of AI slop, but what we really want to do is make these tools available to researchers and let them direct all this AI test-time compute at the problems they think are important. I think it shouldn't be viewed as a race to solve as many Erdős problems as we can, but more like we want to make people aware that the technology is out there. This is what it can do.

你不是在试图解决每一个埃尔德什问题。

You're not trying to solve every Erdős problem.

是的，我不会说那是我们的目标。我认为这恰好是一个特别重要的成果，我们认为有必要与世界分享，让大家了解当今模型的能力水平。但这真的不是像竞赛一样去遍历列表的目标。

Yeah, I would not say that is our goal. I think this has just happened to be a particularly significant result that we thought would be important to share with the world that this is the capability level of models today. But it's really not the goal to just go through the list as if it were a race.

你预见这些会应用到密码学之类吗？还有一些争论，这些模型会不会变得如此强大，以至于我们甚至超越了量子计算的水平，这听起来有点疯狂？

Do you foresee things applying to like cryptography? And there's also some debate too about do these models get so good that we kind of surpass even where quantum computing goes, which sounds kind of crazy?

是的，我认为密码学如今是一个重要话题，因为密码学的基础实际上是一些问题，比如因式分解对计算机来说很难解决，对吧？但基本上我们只有猜想，没有数学证明。假设一个模型在算法上变得非常擅长，也许它会证明一些密码学猜想，说‘好吧，这些协议实际上是安全的，我们不需要猜想它们是安全的。’或者它可能会找到一些漏洞。那也非常重要。我认为我们需要确保我们安全的基础是好的。所以模型可以对密码学的基础进行压力测试，以确保我们有更好的安全性。

Yeah, I think cryptography is really an important topic these days because the foundation of cryptography is really about some problems like factoring is hard to solve by computers, right? But basically we only have conjecture. There's no mathematical proof of this fact. And let's suppose a model gets really good at algorithms, maybe they will prove some of the cryptography conjecture and say, 'Okay, those protocols are actually secure. We don't have to conjecture them to be secure.' Or maybe they'll find some loophole. And that's also very important. I think we need to make sure the foundation of our security is good. So the model can stress test the foundation of cryptography to make sure we have better security.

那量子计算呢？

What about quantum computing?

我认为那是一个非常不同的领域。比如量子计算，实际上我以前研究过量子计算。我的第一篇论文是关于量子优势的，它表明在某些任务上，量子计算机可以比经典计算机做得更好。但到目前为止，我认为模型只是经典计算机。它们做人类能做的事，可能稍微好一点。量子计算机有时能做更花哨的事情，比如模拟化学中的量子效应，这我们可能不……我不是专家，但可能不……还不清楚，这只是两种不同的范式。所以我不太确定它们如何相互比较。

I think that's a very different territory. Like quantum computing, actually I used to study quantum computing. My first paper is on quantum advantage, which shows for some tasks, quantum computer can do better than classical computers. But so far, I think the models, they are just classical computers. They do what human can do, maybe a bit better. Quantum computers can sometimes do more fancy stuff, like simulating some quantum effect in chemistry, which we probably not... I'm not an expert on that, but it might not... It's not clear, it is just two different paradigms. So I'm not super sure how they compare to each other.

但我认为 AI 将大大加速我们开发量子计算机的步伐。近年来，纠错方面有了改进。有量子纠错码只使用更简单的操作类型，这大大加快了物理实现。所以我期待更多这样的成果来自与 AI 的合作。AI 可以提出新的量子纠错算法，然后我们可以更快地开发量子计算机。

But I think AI is going to greatly accelerate the pace that we develop quantum computers. In recent years, there's improvement in error correcting. You have quantum error correcting codes that only use simpler types of operations, and that really speeds up the physical implementation. So I expect more of these to come from collaboration with AI. AI can propose new quantum error correction algorithms, and then we can develop quantum computers much faster.

一旦你让模型解决一个问题，你可以接着问，你是怎么解决的？你能给我解释一下证明的这一部分吗？然后模型会耐心地尝试教你每一步是如何进行的。所以这实际上不是一次性的问题解决。你可以让它教你证明是如何运作的。我真的很喜欢这一点。

Once you ask the model to solve a question, you can ask follow up with, you know, how did you solve it? And can you explain this part of the proof to me? And then the model will patiently try to teach you how everything goes line by line. So it's actually not just one-shot problem solving. You can ask it to learn how the proof works. And I really like that.

作为研究者，你很快会学到一件事：如果你的结果好得令人难以置信，那很可能某个地方有 bug。

One thing you learn very quickly as a researcher is that if your results are too good to be true, you probably have a bug somewhere.

我觉得每个研究人员都有过这样的经历：看到实验数据非常惊人，结果却发现实验其实有问题，数据是错的。当我第一次从 Hongxun Wu 那里听说这件事时，我的先验就是如此。我想，‘哦，我会等他们找到那个 bug。’

I think every researcher has had an experience where they see amazing numbers from their experiments and it turns out the experiment was actually wrong. The numbers were wrong. When I first heard about this from Lujian Hongshuang, that was my prior. I was like, 'Oh, I'll wait for them to find the bug.'

但随着时间的推移，我越来越乐观，也许这就是那百分之一的情况——好得令人难以置信，但却是真的。

But then as the days went on, I had this growing optimism that maybe this is the one in a hundred times where it's too good to be true, but it's actually true.

好了，两位，非常感谢。

Well, gentlemen, thank you very much.

是的，非常感谢。

Yeah, thanks so much.

非常感谢。

Thank you so much.