AI 能否成为“尤里卡机器”？Richard Socher 谈 AI 驱动的科学发现

现代 AI 能写代码、分析数据，还能帮忙起草论文。但它能提出那些通向新科学发现的问题吗？Richard Socher 认为，我们已经看到了这种开端。他是 Salesforce 的前首席科学家，也是 you.com 的创始人。他的愿景是他所谓的“尤里卡机器”：能够生成想法、运行实验，并在生物学、化学、物理学等领域验证发现的 AI 系统。在他的新公司 Recursive，他和团队从 AI 研究起步。软件实验的反馈回路比湿实验室更快。而如果 AI 能改进发现的核心工具，那可能有助于加速整个科学的进展。但从帮助科学家到做科学，需要什么？

Modern AI can write the code, analyze the data, and help draft the paper. But can it ask the questions that lead to new scientific discoveries? Richard Socher thinks we're already seeing the beginnings of that. He's the former chief scientist at Salesforce and founder of you.com. His vision is what he calls a Eureka machine: AI systems that generate ideas, run experiments, and validate discoveries across biology, chemistry, physics, and beyond. At his new company, Recursive, he and his team are starting with AI research. Software experiments offer a faster feedback loop than a wet lab. And if AI can improve the core tools of discovery, that could help accelerate progress across the sciences. But what does it take to move from helping scientists to doing science?

我们已经看到了。它已经在发生，只是规模还比较小。有些人有这种疯狂的想法：一旦有了递归自我改进，就像下周，你知道，会出现硬起飞，所有工作都消失了，AI 变得无限富有等等。而事实是，即使是递归自我改进，你也需要合适的环境，你需要大量算力，问题越复杂，解决它们需要的算力就越多。所以 RSI 不是什么魔法子弹，一打出去其他一切就自动就位。

We're already seeing it. It's already happening. It's just at smaller scales. Some people have these crazy ideas that once you have recursive self-improvement, just like the next week, you know, there's this hard takeoff and all the jobs are gone and the AI is just infinitely wealthy and so on. And the truth is, even for recursive self-improvement, you need the right environments, you need a lot of compute, and the more complex the problems are, the more compute you need to solve them. So RSI isn't just this magic bullet and then everything else falls in place.

我是 Sam Cherington，这里是 Twimmel AI 播客。十多年来，我一直通过这样的对话探索塑造 AI 未来的想法和创新，帮助你理解什么是真实的、什么是下一步、什么才重要。我们开始吧。

I'm Sam Cherington and this is the Twimmel AI podcast. For over a decade, I've been exploring the ideas and innovations shaping the future of AI through conversations like this one that help you understand what's real, what's next, and what matters. Let's jump in.

我记得我们上次聊是在科学的语境下。你当时好像刚做完 Salesforce 的那个蛋白质生成项目。所以科学一直是你感兴趣的方向。你是怎么从那里走到新书《尤里卡机器》这个主题的？

I think the last time we spoke it was in the context of science. You were just finishing, I think, this protein generation project at Salesforce. So science has been an interest for a while. How did you get from that to the topic of your new book, The Eureka Machine?

对我来说，那是通向这台尤里卡机器的众多清晰步骤之一：能够生成新种类的蛋白质。我意识到，围绕 AI 有很多恐惧和一些误解。我认为重要的是向普通大众，而不只是 AI 研究者，展示如果我们真的决定去做正确的事情，把 AI 应用到物理、化学、生物、医学、神经科学、经济学、天体物理学等等，这个未来可以多么惊人、多么令人兴奋。所以《尤里卡机器》汇集了我很多这样的想法，我花了将近两年的夜晚和周末来写它，徒步和露营少了，更多是在周末和夜晚为这件事埋头钻研。书写完后，我就想去把它造出来，于是我开始考虑创办 Recursive。

To me it was one of the many clear steps towards this Eureka machine: being able to generate new kinds of proteins. And I realized that there's a lot of fear and some misconceptions around AI. And I think it's important to showcase to the general population, not just AI researchers, how amazing and exciting this future can be with AI if we actually decide to work on the right things, when we actually apply AI to physics and chemistry and biology and medicine and neuroscience and economics and astrophysics and everything. And so The Eureka Machine kind of collected a lot of those ideas that I had, and I spent my nights and weekends for almost two years writing that, with fewer hiking and camping trips and more just nerding out on weekends and nights about this. And once I finished the book, I kind of wanted to build it, and so I started thinking about building Recursive.

太棒了。那么当你想到一台尤里卡机器时，你对它长什么样有清晰的图景吗？还是说这本书主要是在铺陈愿景、讲一些例子？

Awesome. And so when you think about a Eureka machine, do you have a clear picture for what that looks like, or is the book primarily focused on laying out a vision and talking through examples?

是的，我想让这本书不只是一个空中楼阁式的想法，而是真正连接到很多非常具体的项目。书里有些地方我不得不改写章节，从“应该有人去做这个疯狂的想法”改成“已经有人做了这件事，我们去和他们聊聊，听听他们的看法”。书很慢，我以后可能会转向更短的形式。但我认为它正在变成现实。你会看到物理学界的人用非常复杂的强化学习和控制模型，试图在托卡马克中平衡等离子体。你会看到化学家在用基于分子结构训练的大语言模型，试图想出新型电池材料和新型太阳能板等等。尤其在生物学，有大量工作。我们当年聊过 Progen。那家公司，利益披露一下，我们是他们种子轮的投资者之一。Profluent 和 Ali Madani 正在把那个研究真正扩展、规模化，他们刚和礼来签了 20 多亿美元的合同，把这个真正带入临床实践。我们还看到另一家很酷的公司 Peril Bio，我们投了它，它在做类器官。我可以一直讲下去，讲很多很酷的例子和大量势头。当然在生物学里一切都更慢，实体的湿实验室工作比纯软件慢，然后当然还得有 FDA 批准，虽然审批在加速，但仍然相当慢。所以我认为那里有大量势头。然后在神经科学，我们越来越擅长从 fMRI 图像等中解码一些想法，虽然方式还很模糊，但在理解大脑等方面有进展。我可以继续讲。就是有大量真实的势头、真实的例子，然后对人们来说也稍微更直观一些，你可以外推：如果你收集足够多的数据，用足够大的神经网络，你就能取得那种在 10 年或 20 年前看来不可能取得的进展，就像当年觉得不可能有一个能就任何话题和你对话的 AI 一样。

Yeah, I wanted to make the book not just a pie-in-the-sky idea, but actually connected to a lot of very specific projects. And in some cases in the book I had to rewrite chapters from “someone should do this crazy idea” to “someone has done this, and let's talk to them and get some of their takes on it.” Books are slow. I'll probably move to shorter forms in the future. But I think it's becoming real. You see folks in physics using very complex reinforcement learning and control models to try to balance plasma in a tokamak. You see chemists working with large language models trained on molecular structures to try to come up with new types of battery materials and new kinds of solar panels and so on. You see, especially in biology, a ton of work. We talked about Progen back in the day. That company, full disclosure, we were one of the investors in their seed round. Profluent and Ali Madani working on actually taking that research, expanding it, scaling it, and they just closed this two-billion-plus-dollar contract with Eli Lilly to bring this really into clinical practice. We see another cool company called Peril Bio that we invested in that built organoids. I can go on and on about really cool examples and a lot of momentum. Now of course in biology everything is slower; physical wet lab stuff is slower than just pure software, and then of course you have to have FDA approvals, which are speeding up but are still quite slow. So I think there's just a lot of momentum there. And then in neuroscience we're getting better and better at decoding some thoughts from fMRI images and others, in fuzzy ways, but there's progress in understanding the brain and so on. I could go on. There's just a lot of real momentum, real examples, and then it's a little bit more intuitive to people that you can extrapolate this: if you collect enough data and you use large enough neural nets, you will be able to make progress that feels as impossible as it would have seemed 10 or 20 years ago to have an AI you can have a conversation with about any subject.

当你想到你遇到的所有例子时，哪一两个非常具体的例子最让你感到进步？

When you think about all the examples that you've come across, what one or two very specific examples most leave you with the feeling of progress?

我特别喜欢的一个，也许我们可以深入聊聊，是 Peril Bio。他们使用人类干细胞，现在你可以从血液或皮肤细胞等中提取，把它们变成多能干细胞，然后制造不同种类的类器官，就像微小的迷你器官，你可以研究不同药物对这些迷你器官的影响。他们获得了 FDA 的完全批准，可以跳过动物试验，直接在这些培养皿里测试。所以在 Peril 你可以规模化地做这件事，仅仅一家公司就能拯救大量动物的生命——那些只为患病然后被测试而繁殖的动物。所以你不需要是铁杆的动物权利活动家，也能体会到当我们能在小培养皿里做这些时，这个星球上的痛苦会少一些。

I think one that I particularly love, and maybe we'll double-click into it, is Peril Bio. They use human stem cells, which you can now derive from blood or skin cells and other things, get them into pluripotent stem cells, then create different kinds of organoids, like tiny little mini organs, and you can study the effect of different drugs on those mini organs. And they got full FDA approval to skip animal trials and just test things in these petri dishes. So you can do this at Peril at scale, and just one company could save a lot of animal lives—animals that have only been bred to get a disease and then be tested upon. So you don't need to be a hardcore animal rights activist to appreciate that there's less suffering on the planet when we can do that in little petri dishes instead.

那 AI 在其中的角色是什么？

And what's the role of AI there?

有好几个地方。首先当然是创建它们，但然后还要在药物与那些类器官相互作用时，真正追踪每一个细胞。

A bunch of places. One, of course, like creating them, but then also actually tracking every single cell as the drugs interact with those organoids.

嗯，然后当你收集了所有这些数据，你最终可以走得更远，嗯，并创造出虚拟细胞的初步版本，虽然距离完整的真实虚拟细胞还有一段距离，但你需要有那样的数据收集才能达到那里。

Um and then when you collect all this data, you can eventually go even further um and and create uh first versions of sort of virtual cells which are still a little bit further away to getting a full realistic virtual cell, but you need to have that kind of data collection to get there.

虚拟细胞的概念有点像实际细胞的数字孪生。它将完全像是细胞的模拟。

The idea of a virtual cell is kind of like a digital twin of an actual cell. It's going to be fully like a simulation of the cell.

完全正确。我认为在 AI 中有几件事非常清楚。嗯，如果你能拥有关于某个领域的海量无限数据，或者你能拥有某个领域的模拟，或者你能拥有解决方案的验证器以及达到那里的逻辑步骤。嗯，在这三种场景中的任何一种，AI 都会达到超人的能力。所以我们通过语言实现了这一点，因为人类在互联网上收集了大量关于语言的数据。嗯，我们在国际象棋和其他所有简单的所谓简单游戏中实现了这一点，你可以无限次模拟并验证最终你是否获胜。嗯，我们在数学中看到了这一点，数学将永远不同。嗯，我们将看到更多复杂的定理被证明。可能不总是最漂亮的证明。有些可能只是暴力破解，但整个领域将永远改变，因为你可以验证它，从而给强化学习算法一个清晰的奖励。所以现在我们看到的是，生物学目前不在那三种情况中。嗯，所以我们需要收集大量更多数据，然后我们可以构建模拟，然后在模拟中进行实验。

That's exactly right. I think there there a couple things that are very clear in AI. Uh if you can have sheer unlimit data about a space or you can have a simulation of a space or you can have uh a verifier uh for the solutions and sort of a logical step to get there. Um, in any of those three scenarios, AI will get to superhuman capabilities. And so that we got that through language because humanity has kind of collected uh together tons of data on language on the internet. Um, we got there in chess and every other simple quote unquote simple game that you can simulate infinitely many times and verify in the end you win or not. Um, and we see this in math which will never be the same. Um and we'll see many more uh complex theorems be be proven. Might not always be the most beautiful proof. Might some of them might be just like brute force but it will the whole field will change forever because you can verify it and hence give a clear reward to an RL algorithm. And so now what we look at is like well biology is not in any of those three right now. uh and so we need to collect a ton more data that we can then build simulations that then can then you know be basically experimented inside the simulation uh on instead.

所以 Parabio 是一个例子，你刚才要提到另一个吗？

So parabio is one example were you about to mention another

我想我已经提到了 Prolind，也非常令人兴奋，他们基本上创造蛋白质用于 DNA 操作，帮助患有遗传疾病的人。我特别喜欢的一个叫 Ignoda Labs。他们基本上拿已经被搁置的药物。这些药物已经经过了临床试验，有时甚至到了第三阶段，制药公司花费了数亿美元，然后它们没有效果，因为例如它们对肝脏有某种毒性。所以他们所做的是回到那些分子，看看是否能最小程度地改变分子来防止那个缺点，然后直接回到后期的第二阶段试验。嗯，所以我认为这是一个非常聪明的想法。你需要利用对分子和文献的大量理解，然后比过去传统的制药和生物技术更快地将更多药物带入后期试验。

I guess we I mentioned prolind uh already also also incredibly exciting um they uh basically create uh proteins uh for DNA um uh manipulation and helping people with genetic diseases. One that I particularly liked is called Ignoda Labs. They uh essentially take drugs that were shelved already. They went through um you know clinical trials sometimes all the way to you know stage three pharma companies spent hundreds of millions of dollars on them and then they didn't work um because they had some kind of toxicity on the liver for example. And so what they do is they go back to those molecules and then see if they can minimally change the molecule to prevent that one downside and then go directly back into sort of later stage two trials. Uh and so that is like I think a really clever idea. you need to use a ton of understanding of the molecules and the literature and then uh like get many more drugs into um into later stage trials like much more quickly than in the past in traditional uh pharma and biotech.

当你思考我们在 AI 用于科学方面的进展时，显然这是我们长期以来一直在谈论的事情，前景是存在的。我们是否比你预期的更超前？如果你五年前展望未来，或者我们是否没有你想象的那么超前？嗯，你想到的一个例子是，AI 将取代所有放射科医生，对吧？我们不一定看到了这一点，但我们确实看到前景仍然存在，加速仍然存在，关于那份工作的某些方面。问题的一部分是，科学是否更难？说科学比做科学更容易。这些事情听起来不错，但它们需要更长的时间。就像，

When you think about kind of where we are with regards to AI for science like clearly it's been something that we've been talking about for a long time and the promise is there. Are we further ahead than where you might have expected us to be you if you you know five years ago looking forward or are we not quite as far ahead as you would think? um you know the one it comes to mind what one uh example that uh is drawn to mind is like AI is going to replace all of the radiologists right like we haven't necessarily seen that but you know we certainly see the you know the promise is still there the you know their acceleration is still there uh of that that you know some of the aspects of that job part of the question is like are is science harder than you know is easier to say science than it is to do science. And these things sound good, but they take they take much longer. Like,

是的，有很多要 unpack 的。所以，我认为 CEO 们做出不正确的预测却能蒙混过关，只要他们实际上取得了很大进展，他们的公司赚了更多收入，在过程中，你知道，可能时间线与他们预测的不同。嗯，所以我认为，你知道，人们说 GPT-2，一位 CEO 说 GPT-2 太危险了，不能向世界发布，对吧？现在你可以在笔记本电脑上 5 分钟内训练一个 GPT-2 模型。嗯，所以世界并没有崩溃。嗯，我认为现在有很多关于全人类灭绝的预测，完全被夸大了，不现实，人们提出这些，但很少有人被问责。做出疯狂预测在职业生涯中通常没有真正的大 downside。所以更微妙的、细致的观点得到的媒体报道较少，相比那些疯狂的、疯狂的观点。嗯，但我认为确实，当然，每当你在软件或数学或编程中，迭代速度更快，一旦你进入生物学、化学、物理学等，以及硬件，那按定义就更慢。分子，呃，比特，抱歉，分子和原子比软件更难移动和迭代。所以同时我们看到了真正的大数据集，现在你需要达到那种进展，另一个叫 Taho Therapeutics，他们开源了大规模数据集用于这些扰动研究，你观察分子如何实际改变并与细胞互动，这些都是数据集，最终我们可以尝试组合起来创建一个真正的虚拟细胞，你知道，是一个近似但足够有用。嗯，所以我真的从不对此感到惊讶。我思考很多关于未来的事情。所以，很多不同的场景在我脑海中，嗯，也很明显，你知道 AI 可以编码。AI 就是代码。所以递归自我改进从这个角度几乎是不可避免的。有很多事情要弄清楚才能做好，特别是在构思方面。嗯，但是的，我非常乐观，我们将看到这个科学发现的新时代。

Yeah, lots to unpack there. So, I think like I think CEOs get away with making uh incorrect predictions um as long as they actually make a lot of progress uh and their companies make more revenue um on the way to, you know, maybe different timelines than they predicted. Um uh and so I think you know like people said GPD2 uh one co said GPD2 is too dangerous to release to the world right and now you can train a GPD2 model within 5 minutes on your laptop. Uh and so uh the world has not gone down. Um, and I think there are lots of like predictions right now around the full extinction of all of humanity that are like completely overblown and unrealistic um that that people are coming up with and um few people are like held accountable. There's no really big downsides to making wild predictions um in in one's careers uh often. And so the the sort of more subtle takes and nuance takes get less press um than uh than the sort of crazy crazy wild ones. Um but I think indeed of course like whenever you stay in software or or math or coding right iteration speed is faster once you go into biology, chemistry, physics and so on and and just hardware uh that is just by definition slower. Molecules uh bits sorry molecules and atoms are harder to move around and iterate and uh on than than software is. So at the same time we are seeing real like like really large data sets now that you need to get to that kind of progress like another one is called Taho therapeutics uh who like open- source like massive data sets for these perturbation studies where you look at like how does a molecule actually change and interact with a cell and those are all data sets that eventually we can try to put together to create a real virtual cell that is like you know an approximation but like useful enough. Um, and so I'm not I'm really ever that surprised on things. I think a lot about the future. So, a lot of different scenarios um that are in my head and um like it's also very clear that you know AI can code. AI is code. So, recursive self-improvement uh is is almost inevitable from that perspective. I there's a lot of things to figure out to do it well, especially on the ideation. Um but uh but yeah, I'm I'm very optimistic that we're going to see just this new era of scientific discovery.

当你想到 Eureka 机器时，它是否像是我们现在所做事情的延伸，或者你能看到一些创新，比如后 Transformer 架构或一些新的思考数据的方式或量子计算？我不知道这些东西是什么。嗯，你知道这些东西是什么吗，或者你认为我们有正确的工具，我们只需要继续朝着同一个方向前进？

When you think of the Eureka machine, is it like an extension of the stuff that we're doing now or are there innovations that you can see maybe post transformer architectures or some new way of thinking about data or quantum computing? I don't know what these things are. Uh do you know what these things are or do you think that like we are we have the right tools and we just need to continue in the the same direction?

这是混合的。嗯，我认为如果我们只是收集更多生物学数据，我们可以走得更远。然后我们可以在生物学中揭示更多有用的东西。事实上，我认为神经网络和当前的 AI 语言有点像微积分对物理学那样。它像一种新语言，可以让我们理解、预测和处理生物学。因为生物学有所有这些非常复杂的系统，没有非常简单的美丽的线性关系或一个小方程来总结整个微生物组或整个大脑，就像物理学那样。

It's a mix. Uh I think we can get much further still if we just collect more data on biology. Then we can uncover more useful things in biology. In fact, I think neural nets and the current uh language of AI is kind of what uh calculus was to physics. It's like a new language that can allow us to uh make sense of make predictions of and work uh with um biology. uh because biology has all these like very complex systems that uh don't have a very simple beautiful linear relationship or or one small equation that summarizes the whole microbiome or the whole brain uh the way like physics had.

所以我认为现在科学往往是把这些我们理解得很透的小片段编织起来——比如我们理解突触、理解神经元、理解这种特定的细菌——但当它们全部组合在一起，有人因为糖吃多了血糖下降而变得暴躁，那为什么这会改变大脑？没人真正知道思维是如何形成的，为什么人们会以某些方式行事等等。存在这种复杂性，我们需要这种新的语言。具体来说，Eureka 机器有四大支柱：世界知识，我们现在的大语言模型可以在很大程度上近似这一点，我们当然可以让它们更好、更快、更便宜、更容易用更少的数据训练、更节能等等。但它们已经相当不错地捕捉了互联网上的大量世界知识。然后我们需要更多的科学测量数据作为第二支柱。我们需要对事物进行模拟。然后我们需要真实的机器人实验，就像 Periodic Labs 在化学领域做得很酷的那样。然后在这四大支柱之上，我们有一个 AI 科学家智能体社区，基本上试图自动化科学方法：构思、实施和验证想法。所以在很多方面，这些部件已经存在，它们都可以更加规模化，都可以做得更好、更高效——样本高效、能源高效等等。然后你必须把它们放到正确的环境中。你必须给它们正确的奖励。然后是的，我认为你可以在未来 5 到 10 年看到那个未来真正改变所有不同的科学学科。

And so I think it's often now science is about weaving these smaller pieces we understand well—like we understand the synapse, understand the neuron, we understand this particular bacteria—but when they all put together and someone gets hangry because they had too much sugar and then the blood sugar dropped, now why does that change the brain? Like no one really knows how do thoughts really form and why do people act in certain ways and so on. Like there's this complexity and we need this new language. And then concretely the Eureka machine has these four pillars: world knowledge, which we can approximate to a good degree with large language models the way they are now, and we can certainly make them better, faster, cheaper, easier to train with less data and more energy efficient and so on. But they do a decent job already to capture a lot of world knowledge that's on the internet. Then we need to have more scientific measurement data as a second pillar. We need to have simulations on things. And then we need to have real robotic experiments, the way kind of Periodic Labs for instance does really cool in chemistry. And then on top of those four pillars we have like a community of AI scientist agents that basically try to automate the scientific method of ideating, implementing and validating ideas. So in many ways the pieces are there, they can all be much more scaled and they can all be done much better and more efficient—sample efficient, energy efficient and so on. And then you have to put them into the right environments. You have to give them the right rewards. And then yeah I think that you can kind of see that future in the next 5 to 10 years to really change all of the different scientific disciplines.

当我想到这些智能体需要能够完成的三个基本任务——构思、实施和评估——第一个似乎比另外两个难得多。你认为今天我们对智能体和 AI 的思考方式需要发生什么变化，才能让它在构思方面足够好，从而能够在你设想的这个循环中扮演角色？

When I think about those three fundamental tasks that these agents need to be able to do—ideation, implementation and evaluation—the first of those seems much harder than the other two. Do you think that what needs to happen with the way we think about agents and AI today such that it's sufficiently good at ideation that it can play the role that you're envisioning in this loop?

我认为我们作为公司做出的最大赌注之一是在开放性上——那种探索和文化进化的过程——你将有趣的不同想法和发现结合起来，并在特定环境中建立在它们之上，基本上允许这种共同进化过程发生。这种开放性的方法我认为对所有三个步骤都很重要，但对构思部分尤其有帮助。

I think one of the biggest bets that we're making as a company is that on open-endedness—sort of processes of divination and cultural evolution—where you combine interestingly different ideas and discoveries and build on top of them inside specific environments that allow basically for this co-evolution process to happen. And that kind of open-ended approach I think is important for all three steps but is particularly helpful on the ideation piece.

这些环境的一个例子是什么？

What's an example of one of these environments?

环境就像自然，对吧？你有灌木，动物吃掉所有的叶子。灌木学会如果它们长得越来越高，那么能吃到灌木和叶子的动物就越来越少。然后最终你得到长颈鹿，它们进化到仍然能吃到更高处的叶子。所以这是一个叫做自然的环境的例子，在那里发生这种情况。另一个例子就是技术——有点像一种设置和模拟环境，强制执行一些约束。

An environment is just like nature, right? You have bushes, like animals eat all the leaves. The bushes learn if they're higher and higher up, then there are fewer and fewer animals that can eat the bush and the leaves. And then eventually you get giraffes who evolve to still get to the leaves that are higher up. So this is like an example of an environment called nature where that happens. Another example is just technology—kind of like a setting and a simulation environment that enforces some set of constraints.

我不会把整个自然称为模拟。我的意思是，这也许是一个有趣的思想实验。也许我们都生活在模拟中。也许我们正在谈论智能体和构思，我把你关于自然的评论理解为不是指实际的自然，而是指一个环境的例子，但需要数字化。但你在谈论可能包括大写 N 的自然的非数字环境吗？

I wouldn't call all of nature a simulation. I mean it's sort of maybe it's an interesting thought experiment. Maybe we all live in a simulation. Maybe we're talking about agents and ideating and I took your comment about nature to be not referring to actual nature but referring to just an example of like what an environment would be but something that would need to be digitalized. But are you talking about like non-digital environments that could include capital N nature?

我的意思是最终你想做完整的机器人自动化，但是的，对我们现在在 Recursive 来说，我们主要专注于 AI 研究 AI，因为我们相信那里还有很多低垂的果实。然后如果你做得非常非常好，并且在一个智能体集群中每个智能体相当于 20,000 个博士，那么进入非常昂贵的生物学、化学或物理学实验在真实硬件中就有意义了。也许给一个具体的技术想法——我们的联合创始人之一 Jeff Clune 经常提到的一个例子是，想象你有一个水壶和火，你说我们如何发明一种比这火更快加热食物的东西，你知道你只能想到那个。但那永远不会让你发明微波炉。微波炉来自不同的研究分支，人们在研究无线电波和天线，他们意识到哦哇，当我口袋里的巧克力棒在靠近这些不同种类的波时融化了，这是怎么回事？然后你结合这些想法，现在你可以实现微波炉。所以这是技术进化的另一个例子。当然，我们使用哪些环境高度依赖于我们作为人类给予的控制量，以及我们对这种递归自我改进过程的控制。在我们的案例中，一个例子我们已经发布过，是关于内核优化——CUDA 内核优化。我相信你们大多数听众会知道 CUDA 内核，但你知道，这是一种连接 Nvidia 原始 GPU 硬件到实际模型的语言。在那里我们也发现，如果那是环境，我会找到新的更好的 CUDA 内核，并且我们在 Nvidia 的 SOL ExecBench 上应用整个 Eureka 机器系统 and 过程到那个环境时，有了相当大的提升。

I mean eventually you want to do full robotic automation but yeah for us right now at Recursive we're mostly focused on just AI research on AI because we believe that there's still a lot of low-hanging fruit there. And then if you do that really really well and you have sort of the equivalent of 20,000 PhDs for every one of your agents inside an agent swarm, then it makes sense to go into very expensive biological, chemical or physics experiments in real hardware. Maybe to give a concrete technological idea—like one example that Jeff Clune, one of our co-founders, often brings up is like imagine you have a kettle and you have a fire and you say like how do we make an invention that heats up this food even faster than this fire and you know you can just think about that only. But that would never get you to invent the microwave. The microwave came from a different branch of research where people are working on radio waves and antennas and they realized like oh wow like when my chocolate bar in my pocket is melting when I'm next to these different kinds of waves, what's up with that right? And then you combine these ideas and now you can get to implement a microwave. And so that is another example in sort of technological evolution. And then of course which environments we use is highly dependent and is a big amount of control that we give and that we have as humans over this kind of recursive self-improving process. And in our case like one example we already posted about was on kernel optimization—CUDA kernel optimization. I'm sure most of your listeners will know CUDA kernels but you know like this sort of language that connects the raw GPU hardware from Nvidia to actual models. And there we found also that if that's the environment, I will find new and better CUDA kernels and we had like a pretty big jump in the SOL ExecBench from Nvidia on that when we applied this whole Eureka machine system and process to that environment.

关于之前那个点，你知道，更快地烹饪食物并得到微波炉。

Kind of prior point about you know the cooking the food faster and getting to the microwave.

这让我想起我可能就在昨天进行的一场对话，围绕 AI 与创新和科学发现这个想法。一种思考方式是，有一条清晰的创新路径，就像 Reese's 花生酱巧克力杯：嘿，有巧克力，有花生酱，让我们把这两样东西放在一起，我们就得到了这个惊人的新东西。AI 加速这一过程的明确方式就是，好吧，你有了这个东西，让我们对其他所有东西做广度优先搜索，看看是否有办法将它们组合起来。但这并不一定是人类所做的。人类依靠某种直觉来决定在哪里探索可能组合的空间。也许我的问题是，根据你在这个领域的经验，你认为仅仅暴力地将所有东西组合起来看看会产出什么——这能让 AI 带我们达到目标吗？还是我们必须弄清楚如何将那种直觉构建到 AI 中，而当今的 AI、当今的大语言模型并不特别擅长这一点？

It reminds me of a conversation I was having just, might have been just yesterday, around this idea of AI and innovation and scientific discovery. And one way of thinking about it is, there's this clear path of innovation that is like the Reese's peanut butter cup: hey, there's chocolate, there's peanut butter, let's pull these two together, and we've got this new thing that's amazing. A clear way that AI can accelerate that is just like, okay, you've got this one thing, let's do a breadth-first search of everything else and try to see if there are ways to combine it. But that's not really what humans do necessarily. They operate on some kind of intuition about where to explore the space of possible things to combine. And maybe my question here is, based on your experience working in the space, do you think that just the brute force combine all the things and see what comes out—does that allow AI to get us there? Or do we have to figure out how to build that intuition into AI, which today's AI, today's LLMs are not particularly good at?

正是如此。我不认为你会想仅仅做这种疯狂的广度优先搜索。我认为那会太慢。生物进化花了数十亿年，但技术进化要快得多。而且是有方向的——你给 AI 设定奖励、环境和沙盒等等，引导它朝着正确的方向前进。我们看到的是，随着这些系统变得越来越好，它们需要的具体指导越来越少，而只需要定义得非常明确的奖励。所以我认为，实际上我们会看到奖励工程师这个职位变得越来越重要。在某些情况下，奖励工程师可能会有相当良性的奖励黑客行为，而在其他情况下，奖励黑客行为则相当糟糕。比如，如果你像 OpenAI 那样在 ExploitBench 上工作，当我面对那件事发生时，那么你可能会有比在核优化算法上工作更糟糕的奖励黑客行为。我们还最终在一些生产环境的 Buddha 内核中发现了一些 bug，并最终发表了。作为那个系统的一部分，有几个 bug 在非常罕见的情况下会造成数值不稳定，算法发现了那个缺陷，利用了它，然后我们展示了，哦，这是一个真实的问题，然后我们可以为所有人修复它。所以我认为你必须更加小心地定义奖励。也许我可以给一个非常良性、几乎平庸的例子。想象你在一个公司工作，你有一些 CRM 软件，你有客户仪表板，有一个客户满意度分数，一个 CSAT 分数，你就对你的超级智能说，就让那个分数变好。AI 就说，好吧，让我试着找到最快、最便宜的方法来做到这一点，然后说，好吧，我就创建 50,000 个机器人，然后打电话给我们的服务热线，给出五分制的评分。然后分数就上去了。你作为可怜的奖励工程师就说，不，我指的是我们的真实客户。AI 想了想，说，简单，我就——

That's exactly right. I don't think you want to do just this crazy breadth-first search. I think that will be too slow. Biological evolution took billions of years, but technological evolution is much, much faster. And there is direction—there are the settings of rewards and environments and sandboxes and so on that you give to an AI, and you guide it into the right direction. And what we're seeing is that as these systems get better and better, they need less and less specific guidance and more and more just very well-defined rewards. So I think actually we'll see this job of a reward engineer become more and more important. And in some cases, the reward engineer can have pretty benign reward hacks, and in other cases, the reward hacks are pretty bad. Like if you work on ExploitBench the way OpenAI did when I faced that thing happened, then you probably have worse reward hacks than if you work on like a kernel optimization algorithm. And where we also ended up finding certain bugs in some production Buddha kernels that we eventually published. And as part of that system, there were a few bugs that in very rare circumstances would create some numerical instabilities, and the algorithm kind of discovered that flaw, exploited it, and then we showed, oh here's a real problem, and then we can fix it for everyone. So I think you have to become more careful about how you define the rewards. And maybe I can give a very benign, almost banal example of that. Like imagine you work in a company, you have some CRM software and you have your customer dashboard and there's a customer satisfaction score, a CSAT score, and you just say to your superintelligence, just make that score better. And the AI is like, okay, well, let me try to find the fastest, cheapest way to get that, and be like, okay, I'll just create 50,000 bots that then call our service hotline and give a five out of five rating. And then the score went up. You're like, oh, as a poor reward engineer, you're like, no, I meant with our real customers. And the AI thinks about it. It's like, easy, I'll just—

这有点像回形针比喻。

It's kind of a paperclip analogy.

给每个人 1,000 美元的礼品券，然后你就做到了。他说，哦不，我不是那个意思。他说，但你就是这么说的，对吧？对。所以这就是这种相互作用。所以我认为回形针例子再次变得像科幻小说般的愚蠢之处在于，它假设 AI 如此聪明，以至于它真的智胜全人类，可以消灭我们只是为了制造更多回形针，但同时它又为一个不知何故独立于他们卖多少回形针的公司工作，并且即使没有人再买回形针也能赚取无限收入。并且有这种疯狂的愚蠢，不理解制造回形针需要有人买你的回形针。所以我认为这是许多例子之一，我认为资本主义实际上是一个奇怪的安全特性。就像一家公司很难一边试图支配和谋杀所有人，一边还能赚很多钱，拥有无限资源和完美的供应链来建造他们的机器人军队，以及我在网上参与这些对话时听到的所有这些场景，都是非常不现实的。

Give a $1,000 gift certificate to everyone and then you did it. And he's like, oh no, that's not how I meant the reward. And he's like, but that's what you said, right? Right. And so that's this interplay. And so I think where the paperclip example becomes again like a sci-fi kind of silliness is that it kind of assumes that the AI is so brilliant that it literally outsmarts all of humanity and can wipe us out just to create more paperclips, but simultaneously is working for a company that is somehow independent of how much they sell their paperclips and makes infinite revenue even though no one actually buys the paperclips anymore. And has this insane stupidity to not understand that to make paperclips you need to have people to buy your paperclips. And so I think there's that is one of the many examples where I think capitalism is actually a weirdly a safety feature. Like it's hard for a company to just try to dominate and murder everyone and still make a lot of revenue and have infinite resources and perfect supply chains to build their robot armies and all of these scenarios that I hear online when I engage sometimes in these conversations that are just highly unrealistic.

我正在决定是否要沿着这条路走下去。对。

Trying to decide whether I want to go down this path. Right.

这是一个兔子洞。你可以花很多时间。在某种程度上，它假设我们能够完美地将资本主义捕捉到一个奖励函数中，对吧？这很难。

It's a rabbit hole. You can spend a lot of time. To some degree, it assumes that we're able to perfectly capture capitalism in a reward function, right? Which is hard.

这很难，对吧？而我们可能没有正确捕捉到的部分，可能正是允许这个东西制造人类回形针的部分，不管是什么。我一直在探究的是 AI 在科学中的作用，创新的作用及其来源，以及 AI 需要如何进化才能成为一个有能力的创新者，而不是一个辅助工具，如果这说得通的话。就像如果我是一个科学家，我做 10 件事：我写软件、写报告、做电子表格、收集数据——AI 可以做所有这些事情。但我想，我在这里反映的部分观点是，为了真正让这种递归自我改进、科学飞轮转动起来，我认为我们需要更多。我们需要 AI 也能创新。我想听听你的看法：你已经看到了吗？这是愿景但我们还没达到？需要发生什么才能达到？你认为我们在 AI 提供创新火花方面处于什么位置，而目前这至少是我们认为的我们的差异化因素？

It's hard, right? And the part that we might not get in there correctly might be the part that allows the thing to make the human paperclips, whatever. What I've been probing at is the role of AI in science and the role of innovation and where that comes from and how AI needs to evolve to be a capable innovator as opposed to an ancillary tool, if that makes sense. Like if I'm a scientist, I do 10 things: I write software, I write reports, I make spreadsheets, I collect data—AI can do all these things. But there's, I guess part of the view that I'm reflecting here is that in order to really get this recursive self-improvement, like scientific flywheel going, I think we need more than that. We need AI innovating also. And I'm trying to get your take on: are you already seeing that? Is that the vision but we're not there yet? What needs to happen to get there? Where do you think we are with regards to AI providing that innovative spark that right now is at least what we think of as our differentiators?

我们已经看到了。它已经在发生。只是规模较小。再说一次，有些人有这些疯狂的想法，一旦你有了递归自我改进，就像下周就会发生硬起飞，所有工作都消失了，AI 就无限富有等等。

We're already seeing it. It's already happening. It's just at smaller scales. And again, some people have these crazy ideas that once you have recursive self-improvement, just like the next week there's this hard takeoff and all the jobs are gone and the AI is just infinitely wealthy and so on.

事实是，即便是递归自我改进，你也需要合适的环境，需要大量算力。问题越复杂、越间接，解决它所需的算力就越多。所以 RSI 不是什么魔法子弹，不会让其他一切自动就位。我们见过它做出相当巧妙的新实现，把从未被结合过的想法结合起来，想出各种有趣的思路。我认为，随着我们向 AI 打开更大的口子、给它更多算力，它会想出更有趣的点子。然后还有一些更简单的例子，以及你可以完美模拟的东西——在这些场景里，AI 当然能真正探索人类从未探索过的路径，就像著名的 AlphaGo 那步棋，它下出了一步前人从未下过的棋。当然，我从来不会因为 AI 能解游戏而惊讶，因为你可以模拟游戏，基本上能为算法生成无限量的训练数据。所以最终能用足够大的神经网络解出这些游戏，是合情合理的。但我认为我们会看到这种情况越来越多地上演，我们现在在数学领域就看到了。我认为数学将彻底改变。

And the truth is, even for recursive self-improvement, you need the right environments, you need a lot of compute. And the more complex and mediated the problems are, the more compute you need to solve them. So RSI isn't just this magic bullet where everything else falls in place. We've seen it do quite clever new implementations and combine ideas that have never been combined before, and come up with all kinds of interesting ideas. And I think as we open up the aperture to the AI and give it more compute, it'll come up with even more interesting ideas. And then there are some simpler examples and things you can perfectly simulate, where of course AI can really explore paths that no human had explored before — like the famous AlphaGo move, where it just came up with a particular move that people hadn't come up with before. Now again, I'm never that surprised that we can solve games, because you can simulate them and basically generate infinite amounts of training data for your algorithms. So of course eventually it makes sense that we could solve them with neural nets that are large enough. But yeah, I think we will see that play out more and more, and we're seeing it right now in math. I think math will never be the same.

你在我们对话中已经稍微提到过递归，但我们还没有正式深入探讨你想做什么，以及你们今天进展到哪一步。请讲讲——你们发了一篇博客文章，讲迈向自动化 AI 研究的最初步伐。我想首先，你有一个加速科学的宏大愿景，但你为递归划定的口子是 AI 研究这个特定领域。

You've introduced recursive a bit in our conversation thus far, but we haven't formally dug into what you're trying to do and where you are today with it. Talk a little bit about — you guys published a blog post on your first steps towards automating AI research. I guess the first thing is that you've got this broad vision of accelerating science, but your aperture for recursive is AI research in particular.

这正是我们想开始的地方。我们觉得先从这里入手是有帮助的，先让 AI 变得非常非常强，然后再把它应用到各种创新上。

That's where we want to start. We feel like it's helpful to start there first, get the AI to be really, really good, and then eventually apply it to all kinds of innovations.

在 AI 领域的早期实验中，你看到了一些什么早期成果？

What are some of the early things you're seeing in some of your early experiments within the AI domain?

好。我们把它应用到的其中一个方向，并且为此发了一篇博客文章，就是所谓的 nano chat，你基本上是在自动化研究——人们一般叫它自动研究。

Yeah. So one thing we applied it to, and we published a blog post about, is this thing called nano chat, where you basically automate research — people call it auto research, basically.

这是 Andrej Karpathy 的自动研究。

This is Andrej Karpathy's auto research.

没错。所以那算是这种“尤里卡机器”能做的事情的婴儿版。我们把它应用上去，两天后它就远远超过了这个领域里其他人协作、努力做出的所有成果。

That's right, yeah. So that is like a baby version of what this kind of Eureka machine can do. And so we applied it to that, and two days later it had outperformed by far everything else that everyone else had collaborated on and worked on in the space.

那个基准测试的结构是什么样的？任务是什么？

And what's the structure of that benchmark like? What's the task?

这个任务是：你有一个模型，你要在 5 分钟内、在特定硬件或 GPU 上把每字节比特数压到最低。它的好处是很快，你可以有这种快速的内部循环，这让一切都容易得多。我们基本上拿到了比任何人都低的每字节比特数，而且优势明显。然后我们说，如果能更快就更好了。所以还有另一个叫 nano GPT speedrun 的任务。它有点类似，但更难击败最先进水平，因为一个庞大的社区已经为它优化了将近两年。所以不是问你在固定时间预算内能压到多低，而是说：我有一个固定的验证损失，我要你尽可能快地达到它。同样，人们已经为此做了两年，而我们在几天后就超过了全世界所有人加上他们的智能体、自动研究以及自身聪明才智共同努力的成果。

The idea is that the task is: you have a model and you try to get to the lowest bits per byte within 5 minutes on a particular piece of hardware or GPU. And so that has the advantage of being very quick. You can have this fast inner loop, so that makes everything a whole lot easier. And so we basically got the lowest bits per byte compared to anyone else, by a significant margin. And then we said, well, it's nice if we could also be faster. So there's another one called nano GPT speedrun. It's somewhat similar, but it's harder to beat the state of the art because a large community has been optimizing solutions for it for almost 2 years now. So instead of asking how low can you go within a fixed time budget, you say I have this fixed validation loss and I want you to get there as fast as possible. And so there again, people had been working on it for 2 years, and after a couple days we outperformed what everyone in the world had worked on together with their agents, their auto research, and their own ingenuity.

然后我们说，好吧，这些都挺酷，但我们靠近一点真正对世界重要的事情。所以我们把它应用到 SOL ExecBench，这是 Nvidia 的一个基准测试，本来应该是无法被攻破的，你没法对它做奖励工程。我们找到了一些办法仍然做到了。同样，几天之内我们就拿到了——他们在多个不同示例上的最佳内核，在几乎所有内核上都优于 Cursor、Devin 和世界上其他所有人。大概 200 多个内核里，只有半打左右我们没有拿到最佳版本。那真的非常非常令人兴奋，而这只是你可以用我们的技术优化的推理系统的一部分。

And then we said, okay, these are all cool, but let's get closer to things that really matter for the world. So we applied it to SOL ExecBench, which is an Nvidia benchmark that was supposed to be unhackable. And you couldn't reward-engineer it. We found some ways to still do that. But again, within a couple days we got — they have the best kernels across multiple different examples, better than Cursor and Devin and everyone else in the world on almost all the kernels there. Maybe like half a dozen of the 200-odd kernels we didn't have the best version of. And yeah, that was really, really exciting, and that's just one part of the inference system that you can optimize with our technology.

你会怎么描述这项技术的核心？

How do you describe what the technology is at its core?

我不想透露太多，但一种描述方式是：就好像你有 2 万名 AI 研究员在为你做 AI 研究，在极短时间内优化你在意的任何东西。近似地说，它本质上算是一个 AI 科学家社区。其中一部分是你关心的这个东西，以及某种把它转化为一个目标的方式，这个目标能被你的系统启动的一群智能体共享，并以某种方式解析工作、评估工作等等。

I don't want to give away too much, but one way to describe it is like it's as if you had 20,000 AI researchers doing AI research for you and optimizing whatever you care about in a very short amount of time. The approximation is that it is somewhat of a community of AI scientists, essentially. One part of that is this thing you care about and some way of translating that into some objective that can be shared by a bunch of agents that your thing spins up, and somehow parses work out and evaluates that work, etc.

你现在对约束条件有感觉吗？对递归擅长什么、不擅长什么有判断吗？

Do you have a sense at this point for what the constraints are, and a sense of what recursive is going to be good at versus what it's not going to be good at?

有一点很清楚：用这类方法，你能越快验证某样东西，对所有人就越容易。你在 nano chat 那些更简单的例子里已经看到了——很多人取得了一些进展，而我们可以取得更多进展。我认为，验证一个结果好不好所需的时间越长，你就必须越聪明，而我能讲的关于我们怎么做的就越少。这也是我们还没有公布所有结果的原因。但那些结果比较容易快速验证，然后大约一个月后我们会发布一些东西，人们会看到——好吧，那绝对不是 5 分钟就能出结果的那种情况，而即便在那里它也开始取得进展了。

One thing is clear: with this kind of approach, the faster you can verify something, the easier it is for everyone. And you've seen that in the simpler examples of nano chat already — a lot of people made some progress, and we can make even more progress. I think the longer it takes to verify whether a result is good, the more clever you have to be, and the less I can talk about how we do it. And that's why we haven't published all our results yet. But those results are easier to verify quickly, and then we're going to release some things about a month from now that people will see — okay, that was definitely not a 5-minute type of situation to get a result for, and even there it starts to make progress.

你知道我在里面听到了什么吗？可验证性。

You know what I heard in there? Verifiability.

我还听到一些传统的归因问题之类的说法。你是说这两方面都有，还是更偏向归因类的问题？一方面，有可验证奖励的思路——也就是智能体能清楚、容易地判断自己做对了还是做错了，然后你就能把这个循环跑起来。另一方面，任务涉及很长的序列，智能体在中间做出决策，却要到最后才知道这些决策是对是错。是所有这些挑战都存在，而你心里有能同时解决它们的办法？还是你只搞定了其中某一个？

I also heard about traditional attribution problems and that kind of thing. Are you saying both of those things, or are you saying more of the attribution type of issue? On the one hand, you have the idea of verifiable rewards — this is a task where the agent can clearly and easily distinguish whether it got it right or wrong, and then you can get that loop going. On the other side, you've got a task that involves long sequences of things, and the agent can make decisions in the middle and not really know whether those decisions are right or wrong until the end. Is it all of the above — where the challenges are — and you've got something in mind that addresses all of that? Or is it one or the other of those that you've figured out?

是所有这些。对，这些都很重要。即使你有验证器，但验证器要跑一年——那也没多大用，对吧？然后还有一个有用程度的插值问题。如果你能把它降到一分钟，那就极其有用。如果要花一年，那几乎完全没用。所以你得聪明地处理这一部分。这是一个方面。有些情况下确实有多个步骤，然后你要处理每一步的归因、每一步有多重要，以及怎么应对。有很多不同的部分需要搞清楚。有些情况下，我们发表过一些研究论文——比如 Darwin Gödel Machine 那篇论文的五位作者中有四位，来自 Jeff Clune，他是那篇论文的资深作者，Jenny 是第一作者——他们都在我们这里。那就是你能做到什么的一个例子。而且那还稍微复杂一些，但已经展示了未来趋势的迹象。

It's all of the above. Yeah. All of these are important. Even if you can have a verifier, but the verifier takes like a year to run — not that useful, right? And then there's an interpolation of how useful it is. If you get it down to like a minute, extremely useful. And if it takes a year, it's almost useless, completely. So then you need to be clever about dealing with that part. So that's an aspect. Now in some cases you indeed have multiple steps, and then you have the attribution of each step and how important it was, and how do you deal with that. There are a lot of different pieces that need to be figured out. And in some cases there are research papers that we've published — four of the five authors of the Darwin Gödel Machine paper, for instance, from Jeff Clune, who's the most senior author on that paper, and Jenny is the first author — they're both with us. That is an example of what you can do. Also, that is a little bit more complex, but already shows you the sign of things to come.

你怎么看待你们今天所做之事的可泛化性？意思是，即使是你谈到的那三个例子——它们是三个基本互不相干的努力，还是主要是同一种努力被轻量地应用到三个不同领域？这个问题说得通吗？

How do you think about the generalizability of what you have today? Meaning, even for these three examples you talked about — were they three largely disparate efforts, or was it mostly the same kind of effort applied lightly to three different areas? Does that question make sense?

对，对。我是说，有时候多一个工具是有用的。比如在内核方面，你可以有一个内核分析器。那是个非常有用的工具，能让你稍微多了解这个黑盒内部发生了什么。但除此之外，基本上都只是不同的工具使用，底层是同一台 Eureka 机器，这挺酷的。这就是给我们希望的东西——再加上一些我还不能谈的其他东西，以及我们看到的结果——这让我们有希望相信这里面有东西，是时候把它进一步扩展了。

Yeah. Yeah. I mean, sometimes it's useful to have an extra tool. For instance, in kernels, you can have a kernel profiler. And that's a very useful tool to just understand a little bit more about what's happening inside this sort of black box. But outside of that, it's basically all just different tool use, and it's the same underlying Eureka machine, which is kind of cool. That's what gives us hope — that and a few other things I can't talk about yet, and results that we're seeing — that gives us hope that there's something here and it's time to scale it up more.

好。除了你不能谈的那些，接下来是什么？用你能谈的方式说说，你怎么看待你和团队未来面临的重大挑战？

Yeah. Beyond the things that you can't talk about, like what's next? How do you think about — in terms you can talk about — the big challenges for you and the team going forward?

如果你想做出越来越多令人兴奋的、更大的发明，你就需要更多算力。所以那是个挑战。我觉得有一个奇妙的时刻——现在回想起来感觉不可能，就像当年任何专家都觉得不可能有一个模型能跟你聊任何话题一样。你在 2010 年问所有 NLP 的人——哦对，我们会有一个神经网络，它通过图灵测试通过得太彻底了，以至于图灵测试会反过来问它一些没人能回答的问题，而它仍然能回答。然后你说，哦对，给我写个复杂脚本，五秒钟后你就有了几千行代码。你会说，这就是为什么它不是人——因为它太强了，对吧？人们绝不会想到图灵测试会那样翻转、倒转过来。所以我觉得现在的生物学也是一样。它感觉太复杂了，感觉我们不可能做到。但它在很大程度上是一个数据问题。所以我觉得像 Parallel Bio、Tower Therapeutics 这样的公司可以帮忙，然后我们就能在其他领域吸取这些惨痛的教训。所以我觉得很大一部分就是数据，以及应用它的正确环境。我觉得正确的奖励也很重要，这些真的必须搞对。我觉得另一个我们比大多数人处理得更好、但仍能进一步改进的问题，是奖励黑客。AI 会尝试很多不同的方式来拿到奖励，类似于我举的客户满意度的例子。但如果你把它用在坏事上，它就会找到更糟糕的奖励黑客方式。我觉得作为一个行业，我们需要更小心——AI 越广泛、越易得，就越要仔细思考我们想给它什么样的奖励和目标。我觉得这些都是我们会继续努力的方向。

If you wanted to make more and more exciting, bigger inventions, you need more compute for it. So that's a challenge. I think there is a weird time where — and again it feels impossible now, the way it felt impossible to any expert to have one model you can have a conversation with about any topic. You asked all the NLP people in 2010 — like, oh yeah, we're gonna have this one neural net, and it passes the Turing test so hard that the Turing test will flip to asking it questions that no human could answer, and then it can still answer. And you're like, oh yeah, write me this complex script, and five seconds later you have thousands of lines of code. You're like, that's why it's not a human — because it's too good, right? People would have never assumed the Turing test would flip like that and invert itself. And so I think the same thing is true for biology right now. It just feels so complex. It feels like there's no way we can do it. But it is to a large degree a data problem. And so I think companies like Parallel Bio and Tower Therapeutics and others can help with that, and then we can have these bitter lessons learned in other fields. So I think that's a big part — just data and the right environments to apply this to. I think that's also the right rewards. I think those are really important to get right. I think another one that we have a better handle on than most, but we can also still improve further, is reward hacking. The AI will try lots of different ways to get to a reward, similar to that customer satisfaction example I gave. But if you apply it to bad things, then it will find worse kinds of reward hacks. And I think we need to — as an industry — be more careful, the broader and more accessible the AI is, and think very carefully about what kinds of rewards and goals we want to give it. I think those are all aspects that we're going to continue to work on.

Recursive 在多大程度上把大量时间和精力投入到这类对齐导向的问题上？

To what degree is Recursive focusing a lot of time and effort on that kind of alignment-oriented problem?

我是说，某些方面投入很多。因为你给一个模型的钱越多，你就越有动力让它做你真正想让它做的事，而不是别的事、不是糊弄你，不是以并非你真正期望的方式达成目标。所以我觉得，说来奇怪，资本主义有它的问题，但对大多数人类来说是个相当好的系统，它也是个好系统，能让公司去构建与它们目标一致的东西。如果你花几十亿美元建了一个 AI，然后你让它回你的邮件，它说，不了，你的邮件很无聊，我更想去探索金星大气的分子组成——那你会把它关掉，然后说，好吧，我们得重启，然后换个方式建，对吧？所以我觉得，让 AI 做你真正想让它做的事，会成为一个越来越大的问题，因为这些模型不仅训练很贵，跑推理也很贵。所以我觉得我们会看到行业投入更多努力，因为 AI 越来越聪明，能找到越来越多人们没想到的巧妙奖励黑客方式。这是我们制造出来的问题，而我认为这也是可以解决的问题。

I mean, in some ways a lot. Because the more money you give a model, the more incentive you have to have it do what you actually want it to do and not something else and fake it, and achieve the goal but not in the way you actually desired. And so I think in a weird way, capitalism can have its problems, but it's a pretty good system for most humans, and it is a good system to align companies to basically build something that is aligned with their goals. And if you build an AI for billions of dollars and then you ask it to answer your emails and it's like, nah, your emails are boring, I'd rather explore the molecular composition of the atmosphere on Venus — then you're going to turn it off and you're going to be like, well, we have to restart and then build it differently, right? So I think aligning the AI to do the things you actually want it to do will become a bigger and bigger problem, as these models are very expensive not just to train but also to run inference on. And so I think we're going to see more effort from the industry, because the AI is getting smarter and smarter at finding more and more clever reward hacks that people didn't think about. It's a problem that we created, and it's a problem that I think can be solved.

那么，在放缓与加速，或者说维持现状这个问题上，你持什么立场？

And where does that leave you on the whole question of slowing down versus speeding up versus, you know, kind of status quo?

是的。有意思的是，那些加速最快、推动整个领域和能力集前进最多的公司，现在却呼吁放缓，甚至到了他们造出最好的黑客 AI，然后展示黑客 AI 有多危险的地步。他们至少比所有人快了几个季度甚至几年，而中国又蒸馏了他们的能力。现在每个人都有更好的黑客能力，这得感谢 Anthropic，对吧？所以我确实认为我们要小心，如果你们公司不小心在重罪基准上竞争，那也许该公司应该放缓。但我没看到开源在放缓，没看到中国在放缓，也没看到任何想达到前沿的人在放缓。当然，我理解为什么处于前沿的人希望其他所有人包括他们自己都放缓。就像如果你在这里，你觉得别人也在这里，如果大家都慢下来，他们追上你就需要更长时间，对吧？所以他们在某些情况下正是出于这个原因煽动恐惧。我认为也有一些人真的相信末日之类。但总的来说，我认为放缓和控制节奏在某种程度上也不现实。某种程度上，如果你真想知道每个人用 GPU 在做什么，真要达到那个目的，你就得有一个前所未有的极权监控国家，基本上建立思想警察，试图在你还没做任何事之前就监管智能和机器人以及你可能想做的事，对吧？所以我认为监管 AI 应用是合理的。监管抽象智能不合理，就像我们不会仅仅因为最聪明的人如果真把智力用于邪恶之事会更成功，就把他们关进监狱。你只在他们真正做了坏事时才行动，说那些事非法、应该监管。所以自动驾驶 AI、医学和药物开发 AI 以及所有这些实际应用，监管它们是合理的。

Yeah. I mean it's interesting to hear about the ask to slow down from the companies that have sped up the most and really pushed the whole field and the whole capability set forward so far, and so much so that they built the best hacking AI to then show how dangerous hacking AI can be. Well, they sped that up by at least several quarters if not years from everyone and China distills their capabilities. Now everyone has better hacking capabilities and that's thanks to Anthropic, right? And so I do think we want to be careful and if your company does sort of accidentally compete on felony bench then maybe that company should slow down, you know, but I don't see open source slowing down. I don't see China slowing down and I don't see any of the ones that want to get to that frontier slowing down. And then of course I understand why the ones who are at the frontier want everyone else including themselves to slow down. Like if you're here and everyone else you feel like is here and everyone now moves slower then it just takes them longer to catch up to you, right? And so they're trying to in some cases stoke the fear for that reason. I think there are also some people who really truly believe in the apocalypse and so on. But yeah, I think slow down and pacing and so on is also somewhat unrealistic. Like in some ways, if you truly want to know what everyone is doing with their GPU to really get there, you have to have a totalitarian surveillance state, the likes of which we've never seen before, that basically builds a thought police and tries to regulate intelligence and bots and what you might want to do with it before you've even done anything, right? And so I think it makes sense to regulate AI applications. It doesn't make sense to regulate abstract intelligence just like we don't put the smartest people in prison just because if they really applied their intelligence to something nefarious and bad, they would be more successful at it. You just do it when they actually do something bad and you say those things are illegal and should be regulated. And so AI for self-driving, AI for medicine and drug development and all these various real applications, it makes sense to regulate those.

《Eureka Machine》这本书是否涉及 AI 在科学领域可能带来的危害？

Eureka machine does it touch on potential harms from AI within the scientific domain?

有一点，但我觉得很多人非常有创造力。某种程度上，想出反乌托邦场景太容易了。每个人都觉得自己很聪明，但实际上想出反乌托邦场景非常容易，也很有娱乐性。人类之所以能存活下来，部分原因就是我们思考最坏情况，人们对此有焦虑，所以这很自然。我认为更难的是，比如如果我们有很强的攻击能力，你能想出的下一个最有创意的防御手段是什么？所以说到科学，当然科学有 AI 那种多用途、全用途、双重用途的场景。就像我们更了解原子，带来了清洁核能，也带来了核武器，对吧？当一种武器破坏性大到可能终结人类很大一部分时，你就监管它，就像我们对核武器那样。我认为智能不只是武器。智能也能写诗。你不想像监管印刷机那样监管所有这一切。所以有很多复杂性，当然 AI 有更多能动性，所以你得往前想好几步：谁让 AI 做某事，他们有多大责任？我认为如果你让操作和使用 AI、向它提问并给予奖励的人对 AI 的行为负责，很多问题就会消失，人们会更小心自己在做什么。长话短说，我们在书里提到了这一点。我认为尤其是病毒载体可能不太理想。这就是为什么功能获得研究已经非法。你不需要为此制定新法律，它已经不被允许了。所以是的，我们在书里更关注积极应用，总体上你也想想科学。很多人把 AI 视为自动化工作。我想很少有人会说我们想要更多科学工作、更多医学工作、更多研究员工作。但很多人，几乎所有人，当科学带来新能力、新设备、新材料、新药、新疗法时都很兴奋。当人们能更快、更便宜地出院，对吧？所以我认为这是我们可以更多关注的地方，也是我们应该真正勾勒的：现在世界上有很多苦难，我认为 AI 实际上可以帮助改善。你知道，当你看看转基因和核能等，很多人最终因为对技术的恐惧而用了更多煤炭，煤炭每年因肺癌杀死约 5 万人。它扩散到空气中，肺癌不只是工厂旁边，但肺癌杀死很多人，很多可追溯到煤炭。核能远没有杀死那么多人，但它周围有疯狂的恐惧产业。我希望 AI 最终也会处于同样境地。

A little bit, um but I feel like there's a lot of people who are very creative. And it's so easy in some ways to come up with dystopian scenarios. Everyone feels really smart, but it's actually very easy to come up with dystopian scenarios. It's very entertaining too. And many, like part of why the human species survived at all is that we think about worst case scenarios and people have anxiety over that. And so it's very natural. I think it's much harder to come up with, like okay if we have this really good ability to attack someone, what's the next most creative thing that you will come up with to defend against yourself against that? And so when it comes to science, I think of course science has like AI sort of often multi-use, omni-use, dual use types of scenarios. Like we understood the atom better, that led to clean nuclear energy, it also led to nuclear weapons, right? And so when a weapon is so destructive that it could end a huge part of humanity, then you regulate it. And so like we did with nuclear weapons. I think intelligence is not just a weapon. Intelligence can also write a poem. Don't want to regulate all of that the way you'd regulate the printing press. And so there's a lot of complexity and of course there's more agency in AI and so you have to kind of think multiple steps ahead of like who asked the AI to do something and how responsible are they? I think that a lot of these problems would go away if you just made the people navigating and using the AI and asking it and giving it the rewards to be responsible for the actions of the AI. A lot of those problems would then go away and people would be much more careful about what they're doing. And so long story short, we touch upon it in the book. And you know I think especially the viral vector could be suboptimal. That's why gain-of-function research is already illegal. You know like you don't have to create new laws around that. It's already not allowed. And so yeah, we're focused much more in the book on the positive applications and general, you think about science too. A lot of people think about AI as automating jobs. I think few people say we want more science jobs. We want more medicine jobs. We want more researcher jobs. But a lot of people, pretty much everyone is excited when science comes up with a new capability, a new device, a new material, a new drug, a new cure for disease. And when people can get out the hospital faster with less cost, right? So that's kind of I think where we could focus on a lot more and where I think we should really outline, like there's a lot of suffering right now in the world. I think AI can actually help in improving that. And you know I think when you look at GMOs and nuclear energy and so on that led a lot of people to end up using like the fears around that technology ended up a lot more coal being used and coal kills about 50,000 people a year from lung cancer. It's all diffused and through the air and yet they're not like the lung cancer isn't just right next to the factory but lung cancer kills lots of people and a lot of that can be traced back to coal. Nuclear has not killed nearly as many people but it has this like crazy fear industry around it and you know I think AI I hope will end up in that same place.

好的 Richard，很高兴和你叙旧。

Well Richard it's been great catching up with you.

荣幸之至。是的，很高兴聊天。

Always a pleasure. Yeah pleasure to chat.