AI 的数据饥渴：从互联网到合成数据，再到真正知识的探索

好的，Yajin Choy，斯坦福大学教授、麦克阿瑟天才奖得主，刚刚在 NeurIPS 上做了主旨演讲。感谢你今天和我聊天。很高兴能深入探讨 AI 的一切。你能简单介绍一下你在斯坦福领导的实验室、你专注的研究领域，以及你的主旨演讲是关于什么的吗？

All right, Yajin Choy, professor at Stanford, MacArthur Genius Award winner, just gave a keynote here at NeurIPS. Thank you for chatting with me today. Excited to dive into all things AI. Do you want to maybe say a little bit about the lab you run at Stanford, the area of research you're doing that you're focused on, and what your keynote was about?

当然。我研究很多不同的事情。最近我对推理、强化学习以及事情如何不像我们预期的那样运作很感兴趣，想知道到底发生了什么。

Sure. So, I work on a number of different things. These days I'm excited about reasoning, reinforcement learning, and how things don't work in the way that we expected them to be, and what's going on.

今天的主旨演讲我谈到了几件事。一件事是生活中没有什么是容易的，我的意思是即使是强化学习，除非你非常努力且正确地做好所有事情，否则它可能效果不佳。所以我演讲的很大一部分是关于这个的。事实上，有些人想说 RL 比 SFT 顺序微调更优越，但即使如此，如果在顺序微调上投入大量精力，比如 OpenThought，你实际上也可以胜过基于强化学习的方法。所以有很多工程细节、设计选择被低估了，而这些对于看到这些方法的真正威力至关重要。这是我今天主旨演讲的一个角度。

So today at the keynote there were multiple things that I was talking about. One thing is that nothing is easy in life, and I mean it even with reinforcement learning you can arrive at the conclusion that it doesn't work very well unless you do all the things really effortfully and really right. So a lot of my talk was about that. In fact, some people want to say that RL is more superior than SFT sequential fine-tuning, but even that with a lot of effort into sequential fine-tuning, for example OpenThought, you can actually win over reinforcement learning based approaches as well. So there are a lot of these engineering details, the design choices that are somewhat underappreciated, that actually matter so much in order to see the true power of these methods. So that was one angle of my keynote today.

然后其他角度涉及当前方法，无论你做什么，最终都归结为数据。因为你知道 LLM 香肠是如何制作的：从对整个互联网数据的预训练开始，越多越好。不幸的是，AGI 并不会因此就出现。所以人们开始编写大量考试题目，越多越好。这就是顺序微调，基本上是在大量考试题目上进行监督训练。当这耗尽时，人们开始做强化学习，这听起来更令人兴奋，因为模型可能在探索而不是模仿，听起来更神奇。但在幕后，这一切都归结为合成更多数据，因为那么多数据仍然不够好，我们现在需要让 AI 生成更多数据。当 AI 生成数据时，如果 AI 生成的内容恰好根据奖励模型是好的，那么模型就间接地在其成功的探索上进行了训练。从这个意义上说，这是另一种形式的合成数据。

And then other angles had to do with how the current method, no matter what you do, it all boils down to data. Because you know how the LLM sausages are made: starting with pre-training on the entirety of internet data, the more the better. Unfortunately, AGI doesn't just arrive based on that. So then people start writing a lot of exam problems, the more the better. This is sequential fine-tuning, basically supervised training on lots of exam problems. And then when that exhausts, people now start doing reinforcement learning, and this sounds more exciting because maybe the model is exploring instead of imitating, and this sounds more magical. But under the hood, this all boils down to synthesizing even more data, because that much data is still not good enough, and we somehow need to now ask AI to generate more data. And when AI generates data, by chance it's really good if what AI generated happens to be good based on the reward model. In the eyes of a reward model, then the model is sort of indirectly trained on its own successful explorations. So in that sense, it's another form of synthetic data.

所以一切都回到数据上。这算是你的一个关键点。

So it all comes back to data. That's kind of one of your key points here.

是的，是的，一切都关于数据。这对于让事情在短期内见效是好的，但从长远来看令人失望。因为人类不像我们头脑中的当前智能那样依赖数据。

Yeah, yeah, it's all about data. And this is both good to know for making stuff work for more immediate benefit, but also in the longer term disappointing. Because humans are not as data dependent compared to how data hungry the current intelligence we have in our heads is.

是的。而且我们非常样本高效。就像我父母教我一些东西，他们不必花十亿美元生成一个庞大的数据集让我预训练。

Yeah. And then we're very sample efficient. So like my parents teach me something and they don't have to generate a billion dollars to generate a massive dataset for me to pre-train on.

但根本上，我真正担心的是，当我们考虑互联网数据与合成数据或策划数据时，比如人类专家坐下来编写的困难数学问题，这些都接近我们的互联网数据，是人类知识的产物。但人类知识并不等同于实际知识的宇宙。存在一些真理，比如如何治愈癌症。有很多事情不是真理，不在互联网上，也不在任何当前的数据策划方式中，甚至雇佣最好的科学家让他们编写策划数据，也不会真正教会模型如何提出可以预防或治愈癌症的个性化药物。所以有一个根本性的缺失。我试图提出这个问题：我们如何真正达到那个高度？

But fundamentally, what I'm really worried about is that when we think about internet data versus synthetic data or curated data, like hard math problems that human experts sat down and wrote, these are all in the neighborhood of our internet data, which is the artifact of human knowledge. Now human knowledge is not equivalent to the universe of the actual knowledge out there. There are truths, like how to cure cancer for example. There's a non-truth about a lot of things that's not there on the internet, nor any of these current ways of doing curation of data, or even hiring the best scientists and asking them to write curated data, is not going to really teach the model how to come up with personalized drugs that could either prevent or cure cancer. So there's something about the fundamental piece that's really missing. I was trying to raise that question: how do we really get all the way up there?

我想总结一下我从你主旨演讲中听到的要点，至少第二部分是：一切都经常回到数据，或者总是回到数据。我们目前处理数据的方式常常基于一个假设：如果我们从人类专业知识中生成足够的数据，我们就会得到一个强大的模型，帮助解决重大人类问题。但你的观察是，几乎所有我们想要 AI 智能去解决的最有趣的问题，都是我们尚未解决的问题。所以让科学家教它他们所知道的，或者工程师、律师、医生，并不能让我们达到目标。那么也许这又回到了你关于推理的观点：也许人类的特殊之处在于代表我们如何推理的信息分布。如果我们能捕捉到这一点，让模型学习推理过程，那么我们就让它们能够利用我们已经教给它们记忆或从人类知识中学到的东西，填补我们没有教给它们的空白，然后继续前进，为我们治愈癌症。这样说对吗？

I guess if I was to summarize what I'm hearing about the points in your keynote, one on the second part anyway, is that it all often comes back to data or always comes back to data. And the way that we're doing data right now is often with this assumption that if we generate enough data from human expertise, we'll get to a powerful model that will help solve big human problems. But yet what your observation is that nearly all the most interesting things we want AI intelligence that we would engineer to do would be solve the problems we haven't yet. And so having scientists kind of teach it what they know or engineers or lawyers or doctors doesn't get us there. And so then maybe that brings you back to this point you have about reasoning: if maybe the special thing about humanity is the distribution of information that represents how we reason. And if we can capture that and have the models learn the reasoning process, then we've set them up to take even what we've already taught them to memorize or learn from human knowledge and fill in the gaps that we haven't taught them and then continue past that to cure cancer for us. Does that sound right?

是的，差不多。但现在，我觉得我们所做的归结为插值，也许还有互联网数据或人类知识的归纳封闭和演绎封闭。但我们如何真正超越知识的边界，探索这个未知，伟大的未知，尤其是我们不知道如何触及的知识暗物质？

Yeah, something like that. But right now, I feel like what we do boils down to interpolation, maybe a little bit of inductive closure and deductive closure of the internet data or human knowledge. But how do we really transcend that boundary of knowledge and then explore this unknown, the great unknown, especially the dark matter of knowledge that we don't know how to get to?

太棒了。你听说过人们谈论推理类型的分类吗？我会想到逻辑学家或哲学家，他们花时间思考归纳推理、演绎推理的类型，当我们进行推理、概括，或者将更一般的知识投射到具体假设时。

Amazing. Have you heard people talking about sort of a taxonomy of the types of reasoning? I would think of logicians or philosophers who spend time thinking what types of inductive reasoning, deductive reasoning, when we make inferences, when we generalize, or project more general knowledge down to specific hypotheses.

好问题。是的。当人们谈论推理类型时，往往只关注归纳和演绎，我也强调了这一点。但我真正兴奋的是溯因推理。溯因推理。是的。溯因推理不太为人所知。它源于查尔斯·皮尔士，他观察到大多数所谓的推理，比如归纳和演绎，只是对你已有信息的重复。你已经有了信息，然后得出结论，这在某种意义上只是对你已有知识的改写。

Excellent question. Yeah. So like when people talk about types of reasoning, people tend to just focus on induction and deduction, which I did highlight. But what's really exciting in my mind is abduction. Abduction. Yeah. Abduction is less known. It's due to Charles Peirce, who had this observation that most reasoning, quote unquote reasoning like induction and deduction, is regurgitation of the same information that you already had. You already had information and then you draw the conclusion that's just in some sense like a paraphrase of what was already in your knowledge.

而溯因推理是一种心理活动，即对你观察到的不完整信息提出最佳解释，从而形成一个假设。事实上，当你看到像福尔摩斯这样的侦探故事时，作者错误地认为福尔摩斯是在做演绎推理，但不对，这不是演绎推理，因为你必须带着一点信念的跳跃得出结论，这就是溯因推理，因为你必须提出一个假设。这正是科学家所擅长的，至少有些科学家是这样。对人类科学家来说，难点在于我们还需要证伪，这也是我今天主题演讲的一部分。当你审视科学文献时，会发现两个不可能同时成立的事情却被认为同时成立，因为你必须对科学家分享的任何信息进行证伪，而真正重要的是正确的解读和语境化。所以证伪对于科学的真正创新进步至关重要，也就是检验假设真伪的能力。有些科学家，尤其是那些想获得大量引用的人，很容易提出一个论证有力但没有明显方法检验真伪的假设，即使检验成本很高。

Whereas abduction is this mental act of coming up with the best possible explanation of your partial observation, so you're coming up with a hypothesis. In fact, when you look at detectives like Sherlock Holmes stories, the author incorrectly thinks that Sherlock Holmes is doing deductive reasoning, but no, it's not deductive reasoning because you have to jump to the conclusion with a bit of a leap of faith, which is abductive reasoning, because you have to come up with a hypothesis. This is very much what scientists are good at, or at least some scientists are. The hardness for human scientists is that we also have to falsify, which was part of my keynote today. When you look at scientific literature, two things that cannot be true simultaneously are supposed to be true because you have to falsify whatever information scientists share, and it's up for interpretation and correct contextualization that really matters. So falsification really matters to the true innovation advancements of science, being the ability to test whether a hypothesis is true or false. It's tempting for some scientists, especially if they want to get a lot of citations, to put forward a hypothesis that has compelling argumentation but doesn't present an obvious way to test whether it's true or false, even if it's expensive.

明白了。是的，很有意思。你在主题演讲一开始就提出的第一个观点是‘没有免费的午餐’，我想可以这样总结，比如强化学习与监督微调。我一直在问别人的一个问题是：有没有哪件事让你很困扰，是一个关于人工智能的迷思，你想揭穿或反驳，让你很抓狂？听起来对你来说，研究人员过度简化，说‘我的技术比那个技术好’，形成阵营，比如‘强化学习现在是圣杯’，而根据你的经验，细节才是魔鬼，真正重要的是参数选择、实验设置以及你用来获得结果的小巧思，而不是核心技术。

Got it. Yeah. Fascinating. And the first point you made at the very beginning of your keynote was that there's no free lunch, I think is how I might summarize it, that RL versus SFT. Maybe a question I've been asking others is: what is one thing that bothers you, a myth about AI that you would want to debunk or refute that kind of drives you crazy? Sounds like for you, this oversimplification of researchers to say 'well, my technique versus that technique,' the building of camps of 'oh no, reinforcement learning is the holy grail now,' versus in your opinion, what you've often seen is the devil's in the details, that it's like the parameter choices and the experimentation setup and the small ingenuity that you use to get a result, not the core technique.

是的。让我解释一下科学文献中关于强化学习的一些矛盾发现。NeurIPS 的一篇荣誉提名最佳论文是关于 RLVR 的基本局限性。他们发现——我很喜欢这篇来自清华大学和中国其他机构的论文，抱歉我记不清所有合著者的机构了。他们研究了 RLVR（带可验证奖励的强化学习）前后的 pass@k 性能。结果令人震惊：pass@1，即模型的最佳样本，在 RLVR 后确实如预期般变好了，但 pass@k 性能——在抽取 k 个样本后，检查其中是否有任何一个正确，而不知道哪个是更好的答案——pass@k 性能在 RLVR 后下降了。这意味着基础模型已经在其前 k 个样本中知道了大致正确的答案，而 RLVR 只是微调了一下，让它学会排序相对顺序。所以能力已经存在于基础模型中了。另一篇来自华盛顿大学的关于超级研究的很有趣的论文报告说，如果你在 Qwen 上做强化学习，奇怪的事情就会发生。

Yeah. So let me explain a little bit about the contradictory findings in the scientific literature about reinforcement learning. One of the honorable mention best paper honorable mentions at NeurIPS was about the fundamental limitations of RLVR. What they found is—I really like this paper from Tsinghua University and other institutes in China, sorry I cannot remember all the co-authors' institutions. They looked at pass@k performance before and after RLVR (reinforcement learning with verifiable rewards). What they found is striking: pass@1, the top performing sample from the model, does become better after RLVR as expected, but pass@k performance—after drawing k samples, checking whether any of them happens to be correct and not knowing which one is the better answer—pass@k performance decreases after RLVR. This implies that the base model already knew the roughly good answers within its top k samples, and during RLVR you're just fine-tuning a little bit so that it knows how to sort the relative order. So the capacity was already in the base model. Another really fun paper from UW about superior research reports that if you do RL on top of Qwen, strange things happen.

我好像听说过这个。

I think I heard of this one.

是的。你知道，不用担心奖励模型。随便给奖励。随机奖励，甚至故意给错误的奖励。不管你做什么，性能都会提升。这到底是怎么回事，对吧？

Yeah. You know, don't worry about the reward model. Just give any rewards. Random rewards or even deliberately incorrect rewards. No matter what you do, performance goes up. What on earth, right?

这太疯狂了。

That's crazy.

更奇怪的是，如果你对其他模型如 Llama 或 Almo 这样做，效果就没那么明显。所以基础模型之间存在某种奇怪的化学作用。无论基础模型之前见过什么，很可能是 SFT 风格的中期训练，那是预训练的最后阶段。我认为人们猜测，那种数据确实会改变强化学习结果的行为。但无论如何，这些论文基本上给这种热潮泼了冷水。所以我会这样描述：如果你以一种非常不费力的方式做强化学习，你会得出那个结论。但如果你付出更多努力，通过非常小心地控制熵来克服这个问题——最近有很多论文试图做得更好——那么如果你以更努力的方式去做，你可以在一定程度上克服这个问题。这并不意味着我们完全克服了这个问题；这仍然是一个开放的研究问题，但有办法克服。所以我仍然乐观，更积极地认为我们可以把事情做得更好，但了解极限在哪里也是很好的。

What's even weirder is that if you do that for other models like Llama or Almo, not as much. So there's something strange going on regarding the chemistry between the base model. Whatever that base model saw, most likely SFT-style mid-training, that's the last phase of pre-training. What happens, I think people speculate, is that kind of data really changes the behavior of the RL result. But anyway, those papers basically pour cold water over the vibe. So that's a version that I would characterize as: if you do RL in a very effortless way, you can arrive at that conclusion. But if you put even more effort to overcome that problem by controlling entropy really carefully—there are a lot of recent papers that try to do that better—so if you try to do it in a more effortful way, you can to some degree overcome this. It doesn't mean we entirely overcome this problem; it's still an open research question, but there are ways to overcome. So I'm still optimistic and more positive that we can make things work better, but it's also really good to know where the limits are.

是的。我们必须努力工作。所以这里有一种精神：没有什么是容易的。

Yeah. And that we have to work hard. So there's an ethos here of nothing's easy.

是的。就像再次埋头苦干。

Yeah. Like get your nose to the grindstone again.

是的。好的。太棒了。我们来谈谈开放研究的现状，尤其是在美国、北美，但也包括全球。在你的职业生涯中，过去五年、一年、六个月里，你有没有注意到开放研究数量的变化？你对此有什么担忧吗？你认为一切都还好吗？

Yeah. Okay. Great. That sounds awesome. Let's talk about the state of open research, especially in the United States, North America, but also globally. Have you noticed some shifts in your career in the last five years, one year, six months in terms of the amount of open research happening, and do you have any worries about that? Do you think everything's copacetic?

是的，好问题。我认为在某种意义上，开放研究既变少了也变多了。前沿实验室越来越倾向于封闭。我们确实注意到 Meta 过去通过 Llama 贡献了很多，但似乎他们现在也想更封闭了，例如。所以从这个意义上说，有点令人担忧。但另一方面，仅仅是范围，开源模型的种类增加了不少，尤其是现在 Nvidia 也加入了，还有 Neotron，他们非常致力于支持开源社区。在我的主题演讲中，我还展示了一篇来自 AI World 的非常有趣的博客文章，关于过去一年不同公司贡献的仓库数量趋势。阿里巴巴在最后阶段与 Nvidia 一起冲到了顶部。

Yeah, great question. I think there are in some sense less and more simultaneously. The frontier labs prefer to close more and more. We did notice that Meta contributed so much through Llama in the past, but it seems they want to close more, for example. So in that sense, it's a bit concerning. But on the other hand, just the sheer scope, the variety of open source models increased a fair amount, especially now Nvidia is also jumping on it, but also with Neotron, and they're very dedicated to supporting the open source community. In my keynote, I also showed a very interesting blog post from AI World about the trend over the last one year about the number of repositories contributed by different companies. Alibaba really shows up to the top toward the end together with Nvidia.

这说明中国公司对开源社区的贡献要大得多，我认为这确实有助于中国的人才引进。因为很多在美国读博的学生以前肯定更愿意留在美国，但现在我看到越来越多的人拒绝美国顶级公司的 offer，选择回国。你觉得他们为什么这么做？是因为中国现在对开放研究的态度比我们这里更开放，还是资金更充足、资源更容易获取？

So that shows how Chinese companies are contributing to the open source community so much more and I think that really helps China in terms of talent acquisition because a lot of students who did a PhD in US used to prefer to stay in US for sure. But now I see more and more cases where they turn down one of the top companies' offers to return to China. So why do you think they're doing that? Is it that there's a more open posture in China to do open research right now than we see here, or better funding, more access to resources?

我认为开放科学对人才引进非常有吸引力，而且中国现在确实更开放。相比其他地方的情况，确实非常开放。更不用说现在中国公民在美国的移民难度大多了，签证更难保住。我们不是在欢迎，而是在关门。我听过一个笑话，好笑又不好笑：中国人为 AI 贡献这么大，以至于实际上有两组中国人在互相竞争——中国的中国人和美国的中国人。但这种情况很快就不复存在了，因为他们正在回国。

I think open science is very attractive for talent acquisition, and it's more open in China right now. Yeah, it's currently very more, I mean relative to how things are elsewhere. Yeah, it's very open. And not to mention that immigration is much harder for Chinese nationals in the United States right now. It's harder to keep your visas. Instead of welcoming, we're sort of closing the doors. I heard this joke that's funny but not funny: Chinese people contribute so well to AI that there are practically two groups of Chinese people competing with each other—Chinese group in China, Chinese group in US. This will no longer be true soon because they're returning to China.

确实很引人入胜。你或我能做的最有杠杆作用的事情是什么，来改变美国这里开放程度的趋势？

Yeah, it's pretty fascinating. What's the best highest leverage thing you or I can do to change the tide of the amount of openness we're seeing here in the United States?

我的看法是，我们需要理解民主化生成式 AI 的重要性。我的意思是，AI 应该真正是“属于人类、由人类创造、为了人类”的。所谓“属于人类”，是指 AI 应该反映人类价值观，源自整个人类社会，而不是一种与我们截然不同的、拥有不同价值体系的智能。这就是所有权。在创造方面，“由人类创造”意味着许多不同的国家和不同的社会部门都应该能够创造 AI，而不仅仅是两三个国家的少数公司。所以是“全人类”而非“少数人”。它太强大了，不能只交给少数人。你理想中希望有多少人掌握按钮？每个人，每个人，字面意义上的每个人。我们又回到了民主的话题。最后，“为了人类”是指受益者方面，AI 应该真正造福人类，而且是所有人类，而不仅仅是某些掌权者。更重要的是，如果我们不认真对待这一点，我对此深感担忧。如果我们不努力，我们可能会面临一个未来：AI 不是服务人类，而是服务 AI，甚至更糟，人类服务 AI。我的意思是，某种程度上这也许已经发生了，我们必须做点什么。仅仅在数学基准上做得更好并不能解决这个问题。

We need to understand the importance of democratizing generative AI, is my take. What I mean by that is that AI should really be of humans, by humans, and for humans. So what I mean by AI of humans is that AI should reflect human values and originate from human society at large, versus being a quite different type of intelligence with a different value system than ours. So that's ownership. In terms of creation, AI by humans: a lot of different countries and different social sectors should be able to create that AI, not just a few companies in two or three countries. So by all of humanity versus by a few. It's too powerful to just leave it to a few. How many would you ideally prefer have their finger on the button? Everyone. Everyone, literally everyone. We're talking democracy again. And then finally, AI should be for humans in terms of beneficiary, in that it should really benefit humans and all humans, not just some humans in power. More importantly, if we don't really worry about this, I'm deeply worried about this beneficiary part. So if we don't really work on this hard, we could be facing a future in which instead of AI serving humans, AI might be serving AI, or even worse, humans might be serving AI. I mean, to some degree maybe this is already happening, and we got to do something about it. Just doing better on math benchmark is not going to solve this problem, is my take.

我完全同意你的看法。你认为开源、开放权重和开放讨论的研究人员，那些致力于开放研究的人，现在能否更好地合作，共同筹款，建立协作环境，分享路线图，构建共同成果，共享 GPU，让我们的项目更好地相互促进？这个问题有点复杂，因为我确实相信这一点，而且我认为在学术史上我们一直很合作。在我博士早期，有人这样描述：当我想和卡内基梅隆大学的人合作时，我那位卡内基梅隆学生的导师说：“哦，我们在做非常相似的事情，我们应该合作。”我当时想：“什么意思？我们不能直接协作吗？”我认为学术界历来有点保守，非常独立，甚至对我们的项目也 fiercely independent。你认为在一个新时代，在所有大实验室都关门、情况更严峻的情况下，我们是否有可能跨研究实验室、大学以及开放和非大学非营利组织，以更高效、更投入的方式进行合作？

I wholeheartedly agree with you. Do you think that open source and open weight and open discourse researchers, people committed to the open side of research right now, could be doing a better job teaming up to put forward fundraising efforts and to put forward collaborative environments where we're sharing our road maps and building common artifacts, sharing GPUs in a way that our projects are building off of each other better? It's a little bit of a loaded question because I do believe that and I think that in the history of academia we've been very cooperative. As somebody very early in my PhD described, when I wanted to collaborate with someone at CMU, my CMU student's advisor said, 'Ooh, we are working on pretty similar things, we should be cooperative.' I was like, 'What does that mean? Can't we just be collaborative?' I think academia has historically been a little bit close to the chest and very independent, fiercely independent even with our projects. Do you think that there's perhaps an opportunity in a new era where, in slightly more dire circumstances with all of the big labs having closed their doors, we need to collaborate across research labs and universities and open and non-university nonprofits in a fundamentally more efficient, more committed way?

是的，百分之百。事实上，我们在这一点上非常一致。我几乎有点惊讶听到这个，因为我今天早些时候主题演讲的另一个角度是大卫与歌利亚，以及开源社区如何更好地联合力量。为此，我强调我们需要一些非常规的东西。必须是非常规的。我们不能没有计算机、什么都没有，却指望奇迹发生。非常规的数据、算法，然后是协作。我所说的协作基本上就是合作，一种超越机构界限的非常规方式。我想强调的一个例子是 OpenThought，因为它是一种非常不同的论文写作方式，不同的大学——我是那个团队中一个非常微不足道但非常自豪的合著者——因为有很多不同的大学、个人甚至公司，尽其所能地贡献。我真的认为我们必须以非常规的方式做事，而要做到这一点，我们必须对抗的一件事是自我。当每个人都想成为第一和最后时，我们就别想了。

Yeah, 100%. In fact, we are so aligned in this thought. I'm almost surprised to hear this because another angle of my keynote earlier today was David and Goliath, and what open source community can do better in order to join forces together. For that, I was highlighting we need something unconventional. It's got to be unconventional. We cannot have no computer, no nothing, somehow wish for the log. No dog is not going to come. Unconventional data, algorithms, and then collaboration. What I meant by collaboration is basically cooperative, something unconventional that goes beyond the boundaries of institutions. One example that I wanted to highlight was OpenThought, because it's a really different genre of paper writing where different universities—I'm a very insignificant but very proud co-author of that team—because it's just so many different universities and people and even companies contribute however way they can. I really think that we have to do things in a way that's really unconventional, and in order to do that, one thing we have to fight against is ego. When everybody wants to be the first and last, we can forget about it.

确实如此。这在学术传统中根深蒂固。是的，我们学术界自我选择了 fiercely independent，h 指数是我们的主要指标。所以我认为你和 Ludwig 以及斯坦福的其他人在做的，甚至 Percy 在 Menlo 做的，是一种新型研究。我认为伯克利的 Dave Patterson，他创办的实验室如 RISC 和 AMP（我也在其中），还有 Yon Stoka 和 Mate，我们也有一种高度互补的协作方法。Ludwig 体现了这一点。我们做的 Terminal Bench 实际上是因为与 Ludwig 以及 DataComp 的那群人合作。作者名单非常庞大。我是那篇论文的作者之一，我贡献了一个任务实例，但我也为项目提供了建议，并带来了资源。我认为这是我第一次看到一个研究项目——而且我知道 Ludwig 以前也这样做过。

For sure. And that is pretty deeply seated in the academic tradition. Oh yeah. And the sort of we self-selected in academia for fierce independence and h-index being our primary metric. So I think what you and Ludwig and others at Stanford have been doing, and even Percy with Menlo, is a new type of research. I think Dave Patterson at Berkeley with his labs that he started like RISC and AMP that I was in, but then Yon Stoka and Mate, we've got a collaborative approach as well that is highly complementary. And Ludwig embodies that. We've been Terminal Bench was actually because it was a team-up with Ludwig and the same group that was the DataComp people. The authorship is very large. I'm an author in the paper, I made one task instance, but I also advised the project and we brought resources to the project. I think it's the first time I've seen a research project—and I know Ludwig has done this before.

这是我参与的第一个项目，我们基本上众包了作者身份，并找到了一种激励模式：如果你提交三个任务，就可以成为合著者。博士生们愿意参加这些大约二十多人的 Zoom 会议。参与者来自 CMU、MIT、伯克利、斯坦福和华盛顿大学。这非常鼓舞人心，对项目中的每个学生来说都更有趣。现在我们在旧金山举办这些会议，还把人飞到 Nurups 这里。Open Thoughts 智能体团队就在这里到处活动，与 Moren 和 DSPI 等栈中的其他人合作。所以我认为我们实际上处于边缘，准备推出一些东西，也许会因为一些更大、非常规的研究人员合作而获得更多关注。是的，推动使命，做一些非常规的事情。我真的很感激你。

That's the first one I participated in where we basically crowdsourced the authorship and found a model of incentivizing, which was you get to be a co-author if you submit three tasks. PhDs were willing to show up to these two dozen, you know, people like in size Zoom meetings. It was people from CMU, MIT, Berkeley, Stanford, and UW. It was super inspiring and way more fun for every student on the project. And now we've got these meetings that we have in San Francisco and like flying people here to Nurups. The Open Thoughts agent team is just hanging out here all over and collaborating with other people in the stack like Moren and DSPI. So I think that we are at the edge actually kind of poised to roll out and maybe get a lot more attention for something bigger, very unconventional team up across researchers. Yeah, driving the mission, doing something unconventional. I really appreciate you.

是的，这是一次很棒的聊天。感谢你抽出时间。

Yeah, this was a wonderful chat. Thank you for taking the time.

嗯。

Yeah.