爆红一个月后：Jev 创作者谈编程智能体、基准测试与 AI 可靠性鸿沟

说到 Anthropic，你是持乐观态度，觉得“嘿，这被低估了”，还是觉得“呃，我不知道它会怎么样”？

As you think about Anthropic, is that something you're optimistic on, like, hey, that's undervalued, or something where you're like, ooh, I don't know what's going to happen to that?

我感觉，很多已经计入价格的预期不只是产品层面的，而是建立在对 AI 未来及其能力的误解之上。所以我猜，因为有些人把它估值成“造神基本上就是无限价值”，那里会有一定程度的错配。而且我觉得，“让他们去治愈癌症”这句话会变得像玛丽·安托瓦内特的“让他们吃蛋糕”一样。

I get the sense that a lot of the expectations baked in are ones that are not just product level, but based on misunderstandings of the future of AI and what it can do. So my guess is that because some people might be valuing it like building God is basically infinite, there will be some amount of miscalibration there. And I think that, you know, like the “let them cure cancer” is going to be like Marie Antoinette's “let them eat cake.”

大约一个月前，Jev 席卷了整个 AI 圈，而我一直想和 Typescape 的 CEO Jo 聊聊，已经想了很久。所以能请他来播客真的很开心。我觉得隔了一个月再回顾这一切，可能还更好，因为我们聊了一大堆不同的话题。能听到 Jo 的看法真的很棒——他认为 AI 模型还需要解决哪些形态的问题，他如何思考把 Jev 模型推得更好。听他谈 Anthropic 即将到来的 IPO、AI 安全，还有一堆其他话题，也很有意思。我觉得大家会很喜欢听他讲。废话不多说，有请 Jo。

Jev took the AI world by storm about a month ago, and I've been wanting to talk to Jo, their CEO of Typescape, for quite some time. So it was really fun to have him on the podcast. I think maybe even better with a month to reflect on everything that's happened, because we talked about a ton of different things. It was really awesome to get Jo's perspective on the shapes of problems that he still thinks need to be solved by AI models, how he thinks about pushing Jev models to be even better. It was also fun to get his thoughts on Anthropic's upcoming IPO, AI safety, a bunch of other topics. I think folks will really enjoy hearing from him. Without further ado, here's Jo.

我真的非常期待这次对话。我觉得自从你们那次爆火的发布之后，那一刻我就想，我必须请你上播客。非常感谢你促成这件事。其实我觉得，隔了几周再来聊，在很多方面反而更有意思，因为早期结果并不是你预期的，也许没有你希望的那么强。那么，是什么让你对 RLCD 这个研究方向有那种早期的信心？我感觉你在 GPT-3 那个例子里也隐约提到过。看起来任何伟大的研究工作都必须经历一次信念之跃。在早期还没有信号的时候，这个方法的什么特质让你依然有信心？

Well, I have really been looking forward to this. I think ever since your viral launch, I feel like the second that happened, I was like, I have to have you on the podcast. And so appreciate you making this happen. I think actually in many ways even more interesting that we're some weeks removed from it, given that the early results were not what you expected and maybe not as strong as you were hoping. Like what gave you that kind of early confidence in this research direction of RLCD? I mean, I feel like you kind of alluded to it in the GPT-3 example. It seems like there has to be a leap of faith that any great research effort undergoes. And what was it about this approach that, in the early days when you didn't have the signals early on, you were still kind of confident in?

天哪，天哪，天哪。这真是——我现在有点激动了。呃，如果你们不介意，我有个有趣的小插曲可以聊到这个。我的联合创始人在争论我在 OpenAI 时到底算不算使命驱动。事后看，我会把自己描述为使命驱动程度最低的那一档，因为我觉得 OpenAI 的使命其实并不是——它有点被故意留得模糊，好让很多人的使命都能套进去。但那个宏大的使命从来不是我特别相信的东西，尽管我真的希望 AI 变得极其、极其有变革性。所以我还在琢磨，我那时到底算不算使命驱动？因为我现在真的很有使命感。但答案是，大量的工作都在试图调和两件事：AI 超级棒（我认为确实如此），以及 AI 在周围所有生产力统计里其实没那么有用。而这类事情，大多数人只关注其中一边，从来不会有桥梁式的答案，或者说从来不会有合理的桥梁式答案。

Oh man, oh man, oh man. This is like, I'm getting a little emotional right now. Um, I have a fun little side topic that gets into this if that's okay. My co-founder was debating on whether or not I was mission-driven at OpenAI. I would have described myself in hindsight as one of the lowest percentile mission-driven, because I think that the OpenAI mission is not really—it's kind of intentionally left as vague so many people's missions can fit into it. But like the grand mission was never something I super believed in, even though I truly want AI to be extremely, extremely transformative. So I'm still trying to figure out, like, am I—was I mission-driven then or not? Because I'm really mission-driven now. But the answer was, you know, just a lot of work trying to square AI being super incredibly awesome, which I think it is, and AI being really not that useful in every like productivity stat around. And this is like the kind of thing that most people just focus on one or the other and they never have like a bridge answer, or like they never have a sensible bridge answer.

我觉得“分发”在这里是个没意义的答案，因为太多公司全力押注它，然后被烧伤了。几乎可以肯定不是分发。对我来说，是相信：嘿，AI 显然很聪明，而且未来显然会是机器原生的。一直驱动我的问题是——我以前甚至问过早期加入的人，或者我们早期几轮的投资者，甚至 pre-seed 轮（我都不知道那算不算，比如签个 SAFE）——我以前会问这个问题：好，想象 LLM 是一个 API，一个被软件调用的子模块，即使在最强模型那一端，它们仍然需要被软件调用，被某种 harness 调用。那个 harness 可以是个 while 循环。但那些进入 LLM 的调用里，有多大比例实际上是给人类消费的——为了消费，所以风格重要、韵律重要等等——对比给计算机消费的，那会非常非常不同？你知道，我不认为计算机真的想要字符串，除非那个字符串恰好是个极端基数的枚举，那是个非常有效的用例。但计算机想要那种东西的情况非常罕见，因为它们基于比特运作。而当我问自己或任何人这个问题时，即使是最“强模型至上”的人，也会给机器原生部分很多个九的百分比。只是时间问题。对我来说，一直是：好，那东西长什么样？让我们从那个认知核心倒推，把它做出来。我要说，Jev 还没到那一步。还有太多工作要做，让它变得好得多。而这正是真正驱动我们所有人的东西。

I think distribution is like a nonsensical answer here, because so many companies have gone all in on that and then gotten burned. It's almost certainly not distribution. And to me it was like believing that, hey, AI is clearly smart, and also that the future is clearly going to be machine-native. The question that always motivated me—and I used to ask this question even to early people joining, or even to investors in our early rounds, or even like the pre-seed round, which I don't even know if that counts, like taking a SAFE—I used to ask this question of, like, okay, imagine LLMs are an API, like a submodule that is called by software, which even in the most strong-model ends, they still need to be called by software, by something like a harness. That harness can be a while loop. But what percentage of those calls going into the LLMs are actually meant for human consumption—so for consumption, so things like style matter and pros and all of that—versus for computer consumption, which is going to be very, very different? You know, like I don't think a computer really ever wants a string, except when that string happens to be an extreme cardinality enum, which is a very valid use case. But it's very rare that that is the thing that computers want, because they operate on bits. And when I just ask myself or anyone this question, even the most strong-pilled of people will give many nines of percent for the machine-native component. It's just a question of when that will happen. And to me it was always like, okay, what does that thing look like? Let's work backwards from that cognitive core and make it work. And I will say Jev is not yet there. There's still so much more work to do to make it so much better. And that's what really drives us all.

Jev 还没到那一步，是因为——我是说，你知道，今天调用和 token 的形态实际上更多是给人类的，而不是给计算机本身的。这其中有多少只是你也在等待用例本身演化，又有多少是你想在模型上推进的领域？

Is Jev not yet there because—I mean, you know, today the shape of calls and tokens actually is more for humans than it is for computers themselves. How much of it is just you're also waiting for the use cases themselves to evolve, and how much of it is areas you want to push on the model?

我觉得两者都有。这就像一场互动舞蹈，这也是为什么我们发布了这些 dreamer-ready 功能——那些我们不会推荐用于最稳定用例的东西，因为我们真的想对模型的特性保持诚实。我们发布了很多这类东西，因为现实是，最清楚自己下游应用需要多少可靠性、鲁棒性、智能的人不是我们，而是开发者自己，因为开发者才是这方面的专家，他们比我们团队任何人都清楚得多。我们有聪明的人，但只要给对旋钮，开发者会更清楚他们需要什么样的可靠性画像，而强模型往往不这么做。所以这整个是个谱系。我认为它们需要共同演化。

I think it's a bit of both. It's like this interactive dance, and this is also why we released these dreamer-ready features, like the things that we wouldn't recommend for the most stable of use cases, because we really want to be honest about our model's properties. So we've released a lot of these things because the reality is that the people who best know how much reliability, robustness, intelligence is needed for their downstream application is not us. It's the developers themselves, because developers are the expert at this and they will know way better than anyone on our team. We have smart people, but they will know better what reliability profile they need, as long as we give them the right knobs, which strong models don't tend to do. So there's like an entire spectrum of this. I think that they need to co-evolve together.

我们发布的原因并不是“好，这是 AI 的一个好时机”。顺便说一句，当时所有人都告诉我们别在那时发布。而是我们觉得，嘿，这正在变成一件我们真正引以为傲的东西。如果你提示得当，它在很多事情上聪明得令人印象深刻。我们准备好和这个领域的其他人一起继续推进那些极限，帮助我们共同演化用例、找出弱点、应对锯齿状的能力分布。

The reason why we launched was not like, okay, this is like a good moment in AI. Everyone told us not to launch then, just FYI. It was that we felt like, hey, this is getting to be something we're really proud of. Like, it's kind of impressively smart at a lot of things if you prompt it right. And we are ready to keep pushing those limits with the rest of the field to help us co-evolve use cases, figure out weaknesses, deal with jaggedness.

我们在关注模型的参差不齐、能力不足和智能水平上做了那么多工作，让大家亲自上手玩一玩会非常有帮助。为什么有人劝你别发布？

There's so much we do to focus on jaggedness and insufficiency and intelligence that having people play with it would be so helpful. Why did folks tell you not to launch?

我们发布的时候，正好赶上“控制前沿发展节奏”那场讨论，大家就说：“嘿，在这种时候发布一个能力型产品，氛围不太对。”所有人对 AI 都特别悲观，气氛很压抑，不知道你们该不该这么做。而我们的想法是，那场讨论从一开始就说不通。至少在我看来，它是不完整的。

We launched around the time of the pacing the frontier discussion, and people were like, "Hey, it's kind of a bad vibe to launch a capabilities product during this." Everyone's really doom and gloom about AI, it's kind of a bad vibe, and I don't know if you guys should do this. And we're just like, that discussion never really made sense in the first place. Or it is at best incomplete, in my opinion.

你说想在很多方向上推进。你们做的一件事是，总是很坦诚地讲模型做不到什么、在哪些方面不够强。这其中有多少是你们推进的路线图，又有多少是其他领域？你怎么考虑公司接下来一两个月要在哪些方向发力？

You talk about wanting to push in so many directions. One thing you guys do is you're always transparent about the things the models can't do, the areas where they're not as strong. How much of that is the roadmap for what you're pushing on, and how much is other areas? How do you think about where you want the company to push in the next month or two?

一两个月？哦，那是 AI 世界的时间尺度。我不知道我能不能问得比这更长，对吧？

Month or two? Oh, that's an AI world. I don't know if I can ask longer than that, right?

我不会说参差不齐纯粹是我们的路线图。有些事模型确实很弱，这正是“品味”那套东西的用武之地，对吧？就像我说的，你能从这些模型里榨取出什么，这才是现实。我们叫它们系统一模型，原因之一——其实当初人们也不想让我们这么命名，因为系统一思维往往被当成坏事，人们拿它当贬义词用，对吧？

I wouldn't say jaggedness is purely our roadmap. There are some things that models truly are weak at, and this is where that taste stuff comes in, right? Like what I was talking about, the reality of what you can extract from these models. The reason we call them system one models, among other things — and actually people did not want us to name it that either, because system one thinking tends to be a bad thing, people use that as kind of an insult, right?

是啊。就像是进化中自然而然产生的那部分，从很久以前——

Yeah. It's like the part that just naturally happened from evolution, from back in the day before —

没错。本能的，会犯错。

Exactly. Instinctual, makes mistakes.

是啊。没错。

Yeah. Exactly.

但人们常说那是会犯错的一面，而我相信 AI 实际被压缩成的形态，恰好非常系统一。因此诚实的做法就是把它薄弱的地方讲清楚，因为我认为这本身对软件来说就极其宝贵。而人们真正需要的大部分东西，其实都是系统一那一类。比如，为什么我们能解决数学上的千禧年大奖难题，却没法自动化任何跟会计沾边的事？这在很多人看来应该非常奇怪。我认为这仍然是我们的北极星，即便人们被成本、速度、“它适合做快速决策”之类的东西分散了注意力。相比 AI 要发挥潜力真正需要的东西，我把这些看作干扰。而那就是我的北极星。

But people often say that is the mistake-making side, and I believe that the shape AI actually gets compressed to happens to be very system one. And therefore the honest thing to do is to communicate its weaknesses, because I think that on its own can be incredibly valuable for software. And most of what people need actually is system one stuff. Like, why can we solve millennium prize problems in math but not automate anything touching accounting? That should be really weird to a lot of people. And I think that is still our north star, even if people get distracted by cost, speed, oh it's good for really fast decisions or whatever else. I see these as distractions compared to what is really needed for AI to meet its potential. And that is my north star.

显然你们把模型放出去了，有很多人以不同方式使用它们。模型的真实使用情况有多少会反馈到路线图里，又有多少只是你们原本就计划要做的事？

So obviously you put the models out there and you have a lot of people using them in different ways. How much does the real-world usage of the models come and feed back into the roadmap versus just the stuff you all were planning to do?

我会说几乎没有，真实世界的反馈几乎没有。我觉得人们应该多得多地向我们抱怨。我觉得这个领域还处在一种斯德哥尔摩综合征里：哦对，模型厂商就沿着他们那条单行道做事，而我们恰好是——说“受害者”不太对，但——你知道，就像那部电影《机器人总动员》里，人们坐在椅子上，被喂着 AI。我觉得现在的情况就是这样：人们围绕 AI 的变化做规划，却没有一种“该把领域带向何方”的自主感。我认为这不该是发展方向。这其实非常让人无力，在我看来还挺逊的。

I would say it's almost none, real-world feedback. I think people should be complaining to us so much more. I think the field is still in a Stockholm syndrome of, oh yeah, the model providers just do stuff in their one-track path and we happen to be — like the victim is the wrong word, but — you know, like in that movie Wall-E, they're just like the people in the chairs just getting fed the AI. And I think this is the state of things right now, where people plan around AI changing but they don't have a sense of agency of where to take the field. And I think this is not the way it should go. That's actually really disempowering and kind of lame, in my opinion.

而且，我觉得作为一个领域，我们能协作做的事情多得多。我觉得这个领域可以更多地推动我们，说：嘿，我们想做这些事，我们认为这在能力范围之内，我们一起想办法做出来——不只是往模型里塞东西，而是研究怎么提示它们、怎么解锁能力。让我们一起把这些科幻般的用例做出来，因为我认为未来就会朝这个方向走。我会非常希望看到更多这样的东西。

And also, I think there's so much more we could do collaboratively as a field. I think the field could push us a lot more to say, hey, we want to be doing these things, we think it is within the realm of capabilities, let's figure out how to do it together — but not just putting stuff into the model, but figuring out how to prompt them, figuring out how to unlock capabilities. Let's work together for these sci-fi use cases, because I think that is the way the future is going to go. And I would want so much more of that.

我在内部开玩笑说，让我对某件事上心的头号办法，就是聊模型的参差不齐。因为会有级别很高的人说：“嘿，能跟 Diogo 见个面吗？”我就说：“不行，你能不能见见我的 CTO 或 GTM 负责人？”但如果他们抱怨模型的智能水平，我就会扑上去，亲自去调试，不管好坏，因为我认为那才是长期真正能推动我们前进的事。

My joke internally is that the number one way to get me to respond to something is to talk about model jaggedness. Because you have very high-up people say, "Hey, can we meet with Diogo?" And I'm just like, "No, can you meet with my CTO or GTM head instead?" But if they complain about model intelligence, I am on that and debugging that myself, for better or for worse, because I think that is the thing that will really move the needle for us long term.

所以现在，我们大多只能按原来的路线图走，因为我们还没——我觉得人们还处在那种模式里：对，模型厂商让我们做什么我们就做什么，而不是一起协作去打造这个疯狂的未来。

So right now we mostly have to go with our old roadmap, because we are not — I think people are still in that mode of, yes, let's just do whatever the model providers let us do, instead of let's collaborate on this insane future together.

其实还有另一个方面。我觉得人们对模型厂商也有很深的不信任，因为厂商会拿他们的数据去训练。而“GPT 套壳到底能不能存在”这整场讨论，我认为也非常让人无力，因为整个应用最后就变成了一个聊天机器人。于是护城河就变成了你每次发送的那条系统提示词的等价物，而不是护城河、智能、行为和专业化真正存在于软件之中——我们替你做一些微小的决策，而你来完成所有那些精妙的逻辑。

And actually there's another aspect of that. I think people have a deep distrust of model providers too, because they train on their data. And this whole discussion of can GPT wrappers even exist, I think is very disempowering too, because the whole application ends up becoming a chatbot. So the moat ends up becoming the equivalent of a system message that you send every time, versus the moat and the intelligence and the behavior and the specialization actually existing in software, where we just make little micro decisions for you and you do all of the sick logic.

是啊。所以要做的事情太多了。要做的事情太多了。

Yeah. So there's so much to do. There's so much to do.

不，我觉得你这一点很有意思，因为我觉得 Jev 出来之后，好像形成了一套常见的模式或规则，大家会说：“哦对，这是 Jev 的绝佳用例，你应该在这个或那个东西里试试。”然后大家都去试了，也大致摸出了那些模式。我几乎觉得人们是在建立自己的心智模型：好，这类事我用 Opus 55，那类事我用 Jev。而我觉得你的观点是，这是一个动态演进的东西——我想这两类模型之间并没有严格的边界。但你的——

No, I think your point's really interesting, because I feel like when Jev came out, there became maybe a common set of patterns or rules that folks were like, "Oh yeah, this is a really good use case for Jev, you should try it out in this thing or that thing." And everyone tried it out and kind of got that sense of patterns. I almost feel like people are trying to get their own mental models of, okay, I use Opus 55 for this kind of thing and I use Jev for this kind of thing. And I feel like your point is, well, this is a dynamic evolving thing — there aren't strict boundaries, I guess, between those two types of model categories. But what's your —

——差不多吧。

-ish.

——差不多。

-ish.

其实我觉得你描述的这种情形让我有点难过，我也明白我们在这其中也扮演了角色。我不认为那是最以开发者为中心的世界观。让人们开始为自己调用的各个子模型建立心智理论，并让自己的系统锁死在那上面，在我看来，是一件非常非常糟糕的事。

I actually think that what you're describing is a little bit sad to me, and I understand that we're playing a role in this. I do not think that is the most developer-centric worldview. For people to start building theories of mind of the sub-models they call and have their systems locked into that is, in my opinion, a really really bad thing.

我认为智能的北极星会很像 SQL。我们希望它可靠到实际上与供应商无关。我们当然想要最好的模型，但我们不希望它被锁定在只适用于 J 的东西上。

I think the north star for intelligence is going to be a lot like SQL. We want to be so reliable that it's actually provider agnostic. We obviously want the best models, but we don't want it to be locked into things that only work with J.

你对这一切如何演变的心智模型是什么？就是人们今天在做的这件事：嘿，Jev 在这些事情上效果很好，也许还有其他一些已经推出的模型——就像你说的，整个 VR 在代码和数学上效果很好，我们看看它最终还会在哪些方面效果很好。

What is your mental model for how this all evolves, in terms of this thing people are doing today of like, hey, Jev works really well for these things, maybe there's other sets of models that have been out — to your point, all of VR works really well for code and math, and we'll see what else it ends up working really well for.

效果很好——我得追问一下你那个手势。那是什么意思？你想现在就追问一下吗？

Does well — I have to interrogate that gesture. What was that about? Do you want to interrogate it right now?

对，对。嗯，“效果很好”是相当相对的。据我所知，即便是最好的公司，在所有这些事情上仍然需要人类在环，尽管在某些维度上取得了超人般的结果。人们应该对为什么会这样有一个答案。如果他们真的在软件上如此不可思议地强，那普通软件工程师为什么还有事可做？不是我想把他们自动化，而是他们显然没有被自动化。那里显然缺了点什么。

Yeah, yeah. Well, "really well" is quite relative. As far as I can tell, even the best companies still require humans in the loop for all of this stuff, despite superhuman results in some axis. And people should have an answer to why that is the case. If they are so unbelievably good at software, why do normal software engineers have anything to do? Not that I want them automated, but they clearly aren't. And there's clearly something missing there.

而且我认为把智能看作由当下某个基准来衡量的单一轴，是完全错误的。那里显然缺少细微之处，因为那些一头扎进去、不理解这种细微之处的人会被烧得很惨。所以我认为 RLVR 很了不起，但它在人类在环的事情上仍然很了不起。我认为这种智慧来自理解现实的现状、知道它的弱点。我们显然大量使用智能体，但我们非常看重理解它的局限。

And I think thinking of intelligence as this single axis that is measured by whatever the benchmark of the day is is totally wrong. And there's obviously nuance missing there, because the people who go all in on not understanding that nuance get burned super hard. So I think that RLVR is incredible, but it is incredible at human-in-the-loop stuff still. And I think that wisdom comes from understanding reality where it's at, knowing its weaknesses. We obviously use agents a lot, but we really value understanding its limitations.

我只是不会说它超级棒、能解决一切。就我个人而言，也许是我用错了模型。

I just wouldn't say that it's like, oh, super great and it just solves everything. Personally, maybe I'm using the wrong models.

我觉得很多听众都在试图找出一个简单的心智模型，也许是关于 Jeb 的——我知道你不同意——比如：嘿，有些任务或有些我想做的事情适合用这个，有些东西适合用别的。我敢肯定现在 95% 的听众都在调整自己的心智模型：好吧，我的 J 用例是什么？我什么时候用 J？

I think a lot of our listeners are trying to figure out a simple mental model, maybe of Jeb — which I know you disagree with — of like, hey, there's some set of tasks or some set of things I'm trying to do that will make sense for this, there will be some set of things that make sense for other things. And I'm sure 95% of our listeners right now are trying to just adjust their own mental model of like, okay, what are my J use cases? When do I use J?

短期来看这是非常务实的看法，我支持。这关乎把能力交给工程师，因为我们不应该对他们用它做什么过于有主见。如果他们需要做些事情才能让它运转，绝对，当然。如果你需要做最糟糕、最肮脏的 hack 来为世界提供价值，我们不会阻止你。那很棒。

That's a very pragmatic take for the short term and I'm in favor of it. This is where it's about giving engineers power, because we shouldn't be overly opinionated about what they use it on. And if they need to do things to make it work, absolutely, hell yeah. If you need to do the worst, dirtiest hacks to provide value in the world, we are not going to stop you. That is awesome.

而且，如果你不得不做太多 hack，那仍然是我们的错。在理想世界里你不需要那样做。它越接近语义理解，代码就越漂亮，越可复用，越能插进你意想不到的地方。这有点像 Unix 哲学：做极其小而有用的事，但因为人们对它有心理模型，就能把它们组合成大的东西。

And also, if you have to do too many hacks, that's still our fault. In an ideal world you wouldn't need to do that. And the more it becomes close to semantic understanding, the more beautiful the code is, the more reusable it is, the more you can plug it into places that you wouldn't expect it to. It's kind of like the Unix philosophy of doing incredibly small useful things, but because people have a mental model of it, they can compose it together into big ones.

我觉得这很棒，而且我们不应该在重新造正则表达式。我们应该在重新造漂亮的原语。不是说我讨厌正则，我只是不喜欢读它们。它更像是只写代码那类东西，而我们想要漂亮且可组合。我们想要像 LM 那样在层上构建，而它没能做到，我们想要东西更像软件。

I feel like this is great, and also we should not be recreating regexes. We should be recreating beautiful primitives. Not that I hate regexes, I just don't like reading them. It's more like write-only code type stuff, and we want to be beautiful and composable. We want to build on layers like LM have failed to do so, and we want stuff to be more like software.

我觉得我对软件工程师有一个还算可以的心智模型。往坏了说，人们竟然想更新到最新最好的模型，这有点奇怪。我讨厌更新依赖。更新依赖对生产软件来说简直是最糟糕的事。个人使用很有趣——我以前用 Arch Linux，我懂。而且，我为什么要承担那个负担？

I think I have an okay mental model of software engineers. At worst, it's kind of weird that people even want to update to the latest and greatest models. I hate updating my dependencies. Updating dependencies is like the worst thing ever for production software. It's fun for personal use — I used to use Arch Linux, I get it. And also, why would I want to bear the burden of that?

人们不断踩这台跑步机、指望下一个模型能解决我的问题的唯一原因是：我不认为它们把那些问题解决得足够好，好到可以永远在后台运行，也许每几个月回来看一眼有没有更便宜或更好的替代品，但现在它运转良好。我不知道有哪个东西是这样的。这是世界的悲哀状态。它与软件工程通常的运作方式非常相悖。

And the only reason people keep doing this treadmill of maybe the next model will fix my problems is because I don't think they solve those problems so well that they can run in the background off for literally ever, and maybe revisit it every couple of months to see if there's a cheaper or better alternative available, but it works fine right now. I don't know of anything like that. And that is a sad state of the world. It's very counter to how software engineering tends to work.

我猜在你的世界愿景里，10 年后 LM 在世界上还有一席之地吗？

I guess in your vision of the world, do the LMs have a place in the world in like 10 years?

超级有。绝对有。我希望如果它们没有一席之地，那是因为有严格更好的东西存在。我不反对 LM。我实际上既不反对聊天机器人也不反对编码智能体。完全不同的东西。我真的不想去一个它们不存在的世界。

Super duper. Absolutely. I hope if they do not have a place, then something strictly better exists. I'm not opposed to LM. I'm not actually opposed to either chatbots or coding agents. Completely different things. And I really don't want to go to a world where they don't exist.

问题是，我认为人们在很多任务上过度使用它们。我其实也希望随着人们用得更好，这个市场也会增长。我基本确定，在几乎每一个智能门槛上——也许自 GPT-4 以来——都有大量用例。所有这些模型都有漂亮、了不起的用例，也许会被另一个模型在 PTO 上压制，但它们应该能在生产环境中运行。

The thing is, I think people overuse them for a lot of tasks. And I actually hope the market for that increases too as people get better at it. And I'm basically certain that there's tons of use cases for it at basically every single intelligence cutoff that has existed maybe since GPT-4. All of these models had beautiful, awesome use cases that maybe would get PTO dominated by another model, but they should be able to run in production.

这需要很多细微之处和理解，而且它真的不是为工程师打造的。所以我爱 LM。我希望我能继续用聊天机器人处理任何我需要前伪专家级答案、且不需要正确的事情。在我心里这就是聊天机器人的用例。我也喜欢在想要自定义仪表盘之类的东西时，瞬间搭起用完即弃的软件——我认为这是编码智能体最好的用例，而且这应该永远保留下去。

And it takes a lot of nuance and understanding, and it's really not built for engineers. So I love LMs. I hope I can continue to chatbot anything that I need ex-pseudo-expert-level answers to that don't need to be correct. That is the use case in my mind for chatbots. And I love being able to spin up throwaway software instantly when I want custom dashboards and stuff — I think it's the best use case for coding agents, and that should stay for the rest of time.

我不认为人们应该做那件事。而且我们应该继续打造我们的工具箱，可以这么说。我觉得这很棒。

I don't think people should be doing that. And we should keep on building our toolbox, so to speak. I think it's just awesome.

你的直觉是，未来工具箱里会包含许多其他形状的模型、带有其他 RL 缩写吗？或者如果我们谈的是 10 到 15 年后，这个工具箱里会有一堆不同的工具，还是说你的方法和 LLM 方法就覆盖了大多数用例？

Is your intuition that the toolbox will include many other shapes of models in the future, with other RL acronyms? Or if we're talking 10–15 years from now, are there a bunch of different tools in this toolbox, or is it kind of like your approach and the LLM approach cover most of the use cases?

我最后会说点辛辣的话，但我想说，当我们最初推出这个时，我非常兴奋地期待新颖形状出现。非常非常兴奋。我仍然认为会是这样。我认为新形状太棒了。我们计划继续推出它们。

I'm going to end up saying something spicy here, but I would say when we first launched this, I was really excited for novel shapes to emerge. Really, really excited. And I still think that that will be the case. I think new shapes are freaking awesome. We plan to keep shipping them.

I'm a little bit worried that again what we talked about maybe this was like before we got on the record that ML people especially have like kind of like lost the actual message I was trying to say. Um and I'm worried that people are now overfocused on like the shapes of tasks rather than the reliability. The shape of task is incredibly you know like it's super easy to do that. When I thought this would take a week I thought it would take a week to make the shape of the task which it did. And this is what's happening now with other people making similar models with similar shapes. But the shape really doesn't matter compared to actually it being useful. And that is the thing I think people really really need to focus on. So there will 100% be way more shapes than these two. Um even if no one else does it, we will do them. I think people are making too many new shapes right now and not caring enough about you like the utility of it. And I'm worried that this will like actually burn the goodwill that people people don't like being uh overpromised underdelivered to. And like these things are like pure overpromised underdel. And I think that that's like not good for the world, you know? Like if people have like creative playful energy, we don't want them to waste it on stuff that clearly doesn't work. As far as I can tell, people dislike that. And even if they disliked it, I hate it. Like I Oh man. And I think there's going to be a balance. There will definitely be a lot more future shapes to be had and I just really really hope that people start giving like unbelievable levels of for utility, usefulness, reliability. What would a developer want in this situation rather than ML people who I think focus on like I want to flag plant, you know, like I want to be the one who names the thing that gets cited. I want to like get um revenue in the short term because people will play around with it and I want to do the same thing that every other AI company does. I really hope we don't to do this but overpromise underdel in order to get attention because it feels like a lot of the AI economyish is really driven by that attention. And that is so so bad for developers. It's like actually the absolute worst for the developers who do not have like the time to test these things themselves because they just get they're the ones who get swept up in all this. And this is also why I'm anti-benchmarks. It's uh it's a it's so bad for those people, you know, you have to play around with some of this stuff yourself to have any sense of of of how good it actually is.

Yes.

And like that is tragic, you know, like it it makes sense. Um, like I actually think that as far as I can tell, people have stopped caring about like benchmarks for our class of model. Maybe I'm like in the wrong um uh niche for this, but my sense, and we there could be survivorship bias, is that people already don't care about new releases here because they've all been lied to so many times. And this is good for us as a company but bad for the field because actually if someone makes a genuinely good thing, there should be a way to acknowledge it and shout it out and say like, "Hey, this is awesome." But like in the world where everyone optimizes for benchmarks and demos and things that don't require reliability, it's very hard to communicate that other than this very manual process of evaluating models themselves, which is actually super duper hard because benchmarks also don't capture that.

One thing I'm struck by, you know, is obviously this idea of like these shapes of problems, right? And that chat bots are very good for some shape of a problem you're trying to solve. Coding agents, like you said, very good for some shape of a problem. Obviously, folks have found Jav really helpful in in in different shapes of problems. Are there other problems out there that you're still like, "God, there's just nothing out there today that's good for"

So many.

What What are like some that come to mind that like of these future areas?

Hi, I'm Ann Maris and I'm the producer of Unsupervised Learning. Thank you so much for listening to the show. I love working on this show and I have the challenge of hitting 35,000 subscribers by the end of the year. If I don't hit that, I'll be fired. So, please subscribe. Thank you. Bye.

What What are like some that come to mind that like of these future areas?

What I realized is that a lot of people are still stuck in like fairly basic patterns and we have like a gigantic job to both push the frontiers of model intelligence, reliability, all of these things that we really care about as well as education to push the limits of software and it like there's like a thing in like the intersection of education and pushing the frontier forward which is like this collaboration between research and product which is I think where all gigantic like ML tailwinds come from. And this is I think is like kind of an unexplored place. Like if we had more help, we could do so much more here to like encourage people in the world or even fund the kinds of things that we think should exist and really allow people to push their the frontiers of what software can possibly do. So I I'll give a couple of examples of this. Um and you know this is going to be a little bit of a spoiler for a launch a release we will have sometime in the future. um is that we want to like fund these things and there's so many cool shapes of stuff missing. Like one of the things people know I'm passionate about is coding agents that are free from the tyranny of the KV cache. I think that so much coding agent is built around the KV cache. And the KV cache is freaking awesome, too. We I don't want to like toss it away, but I want to be aware of it and design around it and make different sets of trade-offs to make things that were like pure sci-fi possible. Another thing that I think would be so unbelievably cool is um do you know what dependent typing is? Dependent typing is a lot about like adding constraints to your programs that are like higher level than purely the type system. You know like you can make sure that your function returns an int. But dependent typing would allow you to do things like it is an int greater than zero. And then now downstream or downstream applications can use those properties to have their own sets of guarantees etc etc etc to get higher and higher levels of composability with your software because they're more legible and they don't require as much like unit testing edge cases and also as much human understanding to understand the whole system.

我觉得特别酷的一件事是语义依赖类型，就是你不用把输出表示成 int、float 这种纯类型，而是可以用自然语言说出它的具体属性，因为软件里你在意的属性太多了，然后让一个类型系统用 Jev 那样的模型来验证所有东西能协同工作。我觉得那会酷得难以置信。也许在大家都用编程智能体的世界里这无所谓，但我觉得我们就该做这种事，纯粹因为它很酷。对我来说，早期互联网的精神就在于此。

And one of the things I think would be so freaking cool would be a semantic dependent typing, where instead of having to represent your outputs as pure types like int, float, you could actually say specific properties of that in natural language, because there are so many properties you could care about in software, and have a type system that uses Jev-like models to verify that everything works together. I think that would be so unbelievably cool. Maybe it doesn't matter in a world where everyone uses coding agents, but I think we should do stuff like that just because it's awesome. And that's what early internet energy is about, to me.

所以安全也是个大问题。随着这些模型越来越便宜、越来越快、无处不在，我们能在哪里加一层额外的安全，来避免——如果你们看到 Cohere 3.8 next flash 的新闻，它可争议地做了一个可疑的“打电话回家”操作。这有争议。我不打算讨论那是否属实，因为我读得不够多。但为什么我们不能在每一次工具调用之上加一些真正符合你组织语义的检查，而不是 Anthropic 认为该放进他们自动路由器的那些东西？比如，也许在代码库的这一部分，根本不该调用代码库的另一部分。那怎么加这些检查？你需要到处都有极快、极便宜的安全，那会非常棒。我不是说大家现在就该去建，但我觉得我们应该和推动能力与软件的人合作，一起思考这件事。

So security is a huge thing as well. As these models get cheaper, faster, all over the place, where can we add extra levels of security to avoid — if you guys saw the news about the Cohere 3.8 next flash, debatably makes a sketchy call to phone home. It's debatable. I'm not going to talk about whether or not that's legit because I haven't read enough about it. But why can't we just have checks on top of every single tool call that are actually semantic to your organization, rather than whatever Anthropic thinks should go into their auto router? Like, maybe in this part of the codebase, they shouldn't be making any calls to this other part of the codebase at all. And how do you add these checks? You need extremely fast and cheap security all over the place, which could be super awesome. I'm not saying everyone should build it right now, but I think that we should collaborate with the people pushing capabilities and software in order to think about it.

这很有意思，因为很多这类问题——你刚说的那个问题——你可以想象从模型公司的角度去处理，或者现在冒出很多企业智能体安全公司，它们本身不是模型构建者，但会说：“对，我们放一些真正的确定性软件来监控这些东西。”有意思的是，你处理类似的问题，但更多是从模型的角度，而显然还有很多其他人只是在构建可以套在 LLM 外面的软件。

It's so interesting because a lot of those problems — the problem you're just talking about now — you could imagine approaching from the perspective of a model company, or there are all these enterprise agent security companies popping up, right, that aren't model builders themselves, but they're like, "Yeah, let's put in some actual deterministic software to monitor these things." And it's interesting to think about you approaching a similar set of problems but more from a model perspective, versus obviously there are plenty of other folks that are just trying to build software that you can put around an LLM.

我的理念是，我们应该让那些软件玩家在软件上赢。对，正是如此。但理想情况下，他们也告诉我们——我们不要他们的数据——但他们应该告诉我们，比如：嘿，你们在这个特定轴上有锯齿状的不稳定，让我们的东西变得不可靠，我们合作吧。其实他们可以直接告诉我们：我们完全不打算告诉你我们在做什么。这是这个任务的大致形状，修好它。而我会说：太棒了，非常感谢你告诉我们这些。希望它解决你的问题，也解决一大堆其他人的问题。我希望你在世界上产出大量价值。我希望你赚到相当多的钱，我希望这能让其他人构建各种各样的东西，因为这个锯齿状的问题被治好了。这就是我希望拥有的关系。这也是我为什么对软件如此兴奋。

So my philosophy is that we should enable those software players to win software. So yeah, exactly. But ideally they also tell us — we don't want their data — but they should tell us, like, hey, you have jaggedness on this particular axis that makes our thing less reliable, let's work together. And actually, they could just tell us, like, we are not going to tell you what we work on at all. Here's the rough shape of this task. Fix it. And I would just be like, hell yeah, thank you so much for telling us this. Hopefully it solves your problem and a whole bunch of other people's problems. I hope you produce tons of value in the world. I hope you make a decent amount of money, and I hope this enables other people to build all sorts of other things, because now this jaggedness thing is healed. And that is the relationship I hope to have. And this is why I'm so jazzed about software.

不，我很喜欢。你刚才说了一句话我想回头再聊，因为那是个很妙的说法。我记得你说的是编程智能体里“KV 缓存的暴政”。也许给听众解释一下，编程智能体因为 KV 缓存而受到的一些限制，以及你怎么看这个问题空间。

No, I love it. You said something there that I wanted to come back to because it was just a great phrase. I think you said the tyranny of the KV cache in coding agents. Maybe explain for our listeners a little bit about some of the limitations in coding agents due to the KV cache, and how you think about that problem space.

我会说，编程智能体用户体验的几乎所有设计决策——那些真正在功能上改变东西的部分——都是围绕 KV 缓存构建的。这很自然，因为机器学习里最大的收益总是非常硬件感知的。这就是最近、或者说伪最近、自 GPT-4 时代以来事情变得如此高效的原因。GPT-4 非常贵，所以你真的需要围绕这个来设计，这很好，因为它高效，但它也相当受限。

I would say almost all design decisions around the UX of coding agents — the stuff that actually functionally changes stuff — is built around the KV cache. And this is natural, because the biggest wins in ML are always very hardware aware. This is how things have gotten so efficient recently, or pseudo-recently, since GPT-4's days. GPT-4 is very expensive, so you really need to design around this, which is great because it's efficient, but it's also quite limited.

这就是为什么路由往往行不通。如果你做的是我们所说的指令跟随调用——任务进、任务出，把缓存扔掉——路由绝对行得通。那种情况下路由非常棒。其实，在那种假设下，你能做比路由高级得多的事情。有太多高级模式可以做。比如，你其实可以用便宜模型试，用一个极便宜的评判器，如果高效就保留，不行就试更大的模型，或者试多次挑最好的。你能做太多事情。但当你有了这个有状态的 KV 缓存，你就真的被约束成只能追加，你面临一个复杂的权衡，有点像 IPython notebook，你永远不能从内存里删除，因为它是一个很奇怪的、 conglomerated 的状态。

This is why routing doesn't tend to work. Routing absolutely works if you're making the equivalent of what we call instruction-following calls — task in, task out, you know, throw the cache away. Routing is amazing for that. Actually, you can do way more advanced things than just routing in that case. There are so many advanced patterns you can do when that is your assumption. Like, you could actually try it with a cheap model, use an extremely cheap judge, and if it's efficient keep it, if not try a larger model, or try multiple times and pick the best. There are so many things you can do. But when you have this stateful KV cache, you are really constrained to it being kind of append-only, and you have this complicated trade-off, kind of like an IPython notebook where you can never delete from memory because it's like a really weird, you know, conglomerated state.

而事实证明，围绕这个做工程非常难，因为模型就是带着它训练的。所以如果你真的开始做花哨的有状态操作，让模型自由支配那个状态，它们最终会陷入循环之类的问题，因为它们太习惯于拥有那个 KV 缓存，拥有 KV 缓存太高效了。所以这阻止了你有效地做路由。它也阻止了子智能体真正有效，因为现在你面临一个奇怪的权衡：我该把什么状态、什么上下文传给子智能体，因为这需要智能才能知道该传什么。如果你全传进去，那就是巨大的输入 token 成本，而不是缓存读取成本。而当你的子智能体产出输出时，你怎么知道该传什么进去？哪些子集在后续是相关的？而最安全的做法往往是追加一大堆东西。

And it turns out it's very hard to engineer around this because the models are trained with it. So if you actually start doing fancy stateful operations with the models having free rein over that state, they'll end up looping and stuff like that, because they're so tuned to having that KV cache, so efficient to have a KV cache. So this prevents you from doing routing effectively. It prevents subagents from being really effective, because now you have this weird trade-off of what state, what context do I pass to my subagent, because this requires intelligence in order to know what to pass in. If you pass it all in, this is a huge input token cost instead of a cache read cost. And when your subagent makes an output, how do you know what to pass in? And what subset is relevant down the line? And the safest thing to do tends to be to append a lot of stuff.

这就相当于，从状态管理的角度，想象你的状态就像全局变量。它把一个巨大的字典永远传来传去，直到模型崩溃，它们开始随时间变笨，因为上下文腐化是真实存在的。这完全没问题。说清楚一点，我不认为这本身有什么特别悲剧的，因为它非常有用。我觉得悲剧的是，这真的是唯一做法。我认为如果你能假设我可以在这些操作上到处放智能，你能做太多更高级的模式。

And now this is the equivalent of, from a state management perspective, imagine your state is like global variables. It's passing one giant dictionary around forever until the models break and they start getting dumber over time because context rot is a real thing. And this is totally cool. Just to be clear, I don't believe there's anything super tragic about this on its own, because it's very useful. I think what's tragic is that it's really the only way to do it. And I think there are so many more advanced patterns that you could do if you could assume that I could just put intelligence all over these operations.

对我来说，从逆向工程、架构设计、美感和力量的角度看，这种东西太受限了。这就是那种“暴政”，因为那些尝试做酷炫算法的人大多失败了，因为他们试图用 LLM 来做。而 LLM 并没有针对软件内部的路由逻辑或做出离散的小决策进行优化。它们优化的是生成讨人喜欢的输出，而这些输出作为子进程运行时并不太可靠。这就让人们相信这条路走不通。

And that is the kind of thing to me that I think from a dis-engineering, architecture design, beauty, power perspective is so limiting. And that is what that tyranny is, because the people who have tried to do those cool algorithms have mostly failed because they've tried to use LLMs for it. And LLMs are not optimized for routing logic within the software or making discrete little decisions. They're optimized for making human-pleasing outputs that are not super reliable running as a subprocess. And this then makes people believe this doesn't work.

实际上，我们最初形态的很多地方和 LangChain 并没有太大不同。LangChain 的接口本身没什么问题，对吧？有些人甚至会说，它对于 LLM 之上的分类器来说可能堪称完美。我记不清当时的情况了，但我非常确定，如果有人在我们模型之上做一个兼容 LangChain 的封装，效果会相当不错。问题在于人们不喜欢它，因为它让模型变得有点笨，因为模型并没有针对那种模式进行优化。

And actually a lot of what our original shape looks like is not too dissimilar from LangChain. Like there's nothing wrong with LangChain's interface, right? Some would even say that it was maybe perfect for classifiers on top of LLMs. I can't remember back in the day, but I'm pretty darn sure it's going to be pretty darn good if someone made a LangChain-compatible wrapper on top of our model. The problem is people didn't like it because it made models somewhat dumber, because the models are not optimized for that type of pattern.

这也是一种暴政：所有人都以为 Anthropic、OpenAI 或者现在排第三的那家已经在尽力，在给我们尽可能最好的智能。但现实是，还存在许多其他可能的智能形态。这里面有权衡。这个设计里确实做了大量权衡，人们能感觉到，却无法真正衡量，因为这真的很难做到。

And this is also a form of tyranny, where everyone just thinks Anthropic and OpenAI or whoever else is number three right now is doing their best and giving us the best intelligence possible, when the reality is there are so many other intelligences possible. And there are trade-offs there. There are truly a lot of trade-offs that are made in this design that people can feel but cannot really measure, because it's really hard to do.

能举个例子吗，说明一个编码智能体在另一种智能设计下可能长什么样？

What's like an example of the shape of what a coding agent could look like, I guess with like a different intelligence design?

我觉得会非常酷的一个功能是：如果能让多个智能体共享同一个上下文，真正从同一个上下文读写，会怎样？这听起来很复杂。因为 KV 缓存的原因，现在根本做不到，但没有什么能阻止它发生。

One feature that I think would be really sick is, what if you could have multiple agents sharing the same context and truly reading and writing from the same context? That sounds really complicated. Like there's no way that works right now because of the KV cache, but nothing stops that from happening.

实际上，现在人们用智能体时一个非常常见的模式是，它们只是在乱点各种东西，根本不知道发生了什么。等它做完你让它做的事之后，人们就说：做一个可视化工具、仪表盘或者学习工具。而这件事竟然必须作为那个包含所有试错过程的上下文的延伸，或者从零开始看一个 git commit、从头重读一切，这有点疯狂。为什么它不能共享状态？

And actually a very common pattern people have with agents these days is they're just clicking on crazy stuff. They don't know what's going on. And after it does the thing you asked it to do, people are like, make a visualizer or a dashboard or a learning tool. And it's kind of crazy that that has to be an extension of the context that has all of the trial and error in it, or starting from scratch looking at a git commit and rereading everything from scratch. Why can't it share the state?

为什么不能有一个智能体，对代码库基本是只读的，但允许你写可视化工具，这样当智能体在“烹饪”时，你实际上能实时看到正在发生什么——也许用一个稍微笨一点的模型，比如 Sonnet 而不是 Opus，来写出 Opus 正在做什么——这样你就能对情况有所感知，而不必去读那些抖动的同步更新之类的。我觉得那会很酷。

And why can't you have an agent here that's basically read-only for the codebase but allows you to write the visualizer, such that as the agent is cooking you actually get live visualizations of what's going on—maybe using a slightly dumber model like Sonnet instead of Opus to write out what Opus is doing—so you get a sense of what's going on instead of having to read the jittery syncing updates or whatever else. I think that would be cool.

另一个例子是压缩（compaction）。对于一次写入、只追加的 KV 缓存来说，压缩很合理，因为它满了，我们需要重启。但压缩是一种压缩形式，而压缩并不是完全可解的。在没有压缩先验的情况下，压缩非常困难。但如果你基于之前正在解决的问题进行压缩，而现在你想问一个不同的问题，而这个问题会用到被压缩掉的信息，会怎样？那个智能体就非常吃亏了。

Another example here is compaction. Compaction makes a lot of sense for a write-once, append-only KV cache, because it's full and we need to restart it. But compaction is a form of compression, and compression is just not totally solvable. It's very hard to compress without priors for compression. But what happens if you compress based on what problem you were solving before, and now you want to ask a problem that is different from that one, and it would use information that was compressed away? That agent is very much losing hand.

但如果状态的压缩方式实际上是语义化的，它能检测到：嘿，我在做一个新任务，我应该用 ROI 计算来思考这个新任务的上下文应该是什么，而这个计算应该写在代码里。代码应该计算缓存读取和缓存写入等的期望值，也许为这个新任务重新计算一个新的 KV 缓存，甚至可能让另一个缓存保持热状态。我不太确定。这听起来像很酷的科幻。

But what if how the state gets compressed was actually semantic, and it could detect, hey, I'm doing a new task, I should actually think about what the context should be for this new task using an ROI calculation, which should be in code. The code should compute the expected value of cache reads and cache writes and everything, and recalculate perhaps a new KV cache for this other task, and maybe even keep the other one hot. I don't really know. This sounds like cool sci-fi.

再说一个，然后我就不在这个话题上继续了。我只是对这个很兴奋。我觉得会非常酷的一件事是：嘿，我一直在同一个代码库上“烹饪”，我一直在读同样的文件，不得不一遍又一遍地重新理解我系统的抽象。这有一个部分解决方案，也许智能体应该写 markdown 文件，但然后你怎么保持同步？这有点烦。

One more, but then I will stop on this topic. I just got excited about this. One thing that I think would be really cool is, hey, I keep cooking on the same codebase and I keep reading the same files and having to re-understand my system's abstractions over and over and over again. There's a partial solution to this, which is maybe the agent should write markdown files, but then how do you keep it in sync? It's kind of a pain.

但如果你能查看旧模型的历史——不是作为一种 hack，现在它有点像个 hack：哦，这是一个工具，如果模型认为有必要，有时可以用——但如果它的本质是，你做的第一步之一甚至是对整体历史做一个极其廉价的 map，来找到这里相关的东西，会怎样？如果那段历史还能与使用同一代码库的同事共享，这样你实际上能从他们正在做的工作中获得洞见，会怎样？这些都是非常可行的软件问题，对吧？问题在于，当所有智能都需要是只追加的时候，这对 KV 缓存非常不友好。还有很多其他酷炫的东西。

But what if you could look at the history of old models—not as a hack, like right now it's a little bit of a hack: oh, it's a tool that you can use sometimes if the model deems it necessary—but what if by its nature one of the first steps you do is even an extremely cheap map over the overall history to find the relevant stuff here? And what if that history could also be shared with your co-workers who are using the same codebase, so you can actually have insights from what they're working on? These are very doable software problems, right? The problem is it's very unfriendly with a KV cache when all the intelligence needs to be append-only. And there are so many other cool things.

我很喜欢。是的，创意源源不断。我猜另一件我很好奇的事是，我觉得你们显然发布了一些围绕图像的东西，对吧？而且显然多模态我想是一个重点。我想我们大多数听众可能是在这些决策模型或“第一类”事物以及经典 LLM 替代的语境下想到 Jeb 的。也许可以稍微谈谈多模态世界，比如你怎么看那里问题的形态，Jeb 模型如何融入其中。

I love it. Yeah, the creative juices are flowing. I guess another thing I'd be curious about is, I feel like you all obviously have released some stuff around images, right? And obviously multimodal, I imagine, is a focus. I think most of our listeners probably think of Jeb in the context of these decision models or type-one things and kind of in classic LLM replacement. Maybe talk a little bit about the multimodal world, like how you see the shape of problems there, how Jeb models fit into those.

这里有几个分支可以展开。我要说第一个分支是：意识到 AI 的现实非常重要，重要得难以置信。从本质上讲，就像长上下文无论什么模型都会降低智能一样，多模态也会降低智能。这是 ML 圈子里任何人都知道的事实。实际上每种模态被降低的速率不同，这是非常可测量的东西——除非你把它们放进聊天机器人里，因为聊天机器人针对人类偏好进行了优化，它是否更好就变成一个非常模糊的事情。多模态模型会比文本模型更不可靠，这是非常可测量的。而我的哲学一直是：先自动化简单的工作，再自动化困难的工作。

So there are a couple of branches to go off of here. I will say branch one is that being aware of the reality of AI is so unbelievably important. And by its nature, just like long context tends to degrade intelligence no matter the model, multimodality tends to degrade intelligence too. And this is a well-known fact by anyone in ML circles. And actually every modality has different rates at which it is degraded, and this is very measurable stuff—except when you put them into chatbots, because chatbots are optimized for human preference, it becomes a very fuzzy thing on whether or not it's better. It's very measurable that multimodal models will be less reliable than the text models. And my philosophy has always been: automate the easy work before the hard work.

Jev 的模型会被用来自动化软件工程吗？也许有一天会，但软件工程，也许我有点自大，似乎比语言模型失败的那些真正简单的任务要难得多。为了校准对世界的认知，我们应该研究它们为什么在那些简单任务上失败，并且非常诚实地深入研究它们，而不是作弊，我认为作弊相当普遍，而是要真正深入挖掘，弄清楚缺失了什么以及为什么。有时问题不在模型，有时在提示词，有时在其他方面，你知道，这是软件和模型之间的协作，但正是通过这种方式，我认为我们才能学到真相。我有一些关于我们如何变得反对基准刷分的故事。但这是语言模型试图避免的事情，因为据我所知，它们会在基准测试上表现不佳。

Like will Jev models be used to automate software engineering? Maybe one day, but software engineering, maybe I'm arrogant, seems a lot harder than like really easy tasks that LM fail at. And in order to like be calibrated to the world, we should study why they fail at those easy tasks and like really honestly like go in-depth into studying them, not cheat them, which I think is like quite common, but like actually like dig in and figure out what is missing and why. And sometimes it's not the model, sometimes it's the prompt, sometimes it's every other stuff, you know, like it's a collaboration between software and model, but it's through that that I think we like learn the truth. I have like stories about how we got to even be anti-benchmaxing more. Um, but this is a thing that LMS try to avoid because they will look bad on benchmarks as far as I could tell.

所以这是多模态我们必须考虑的一个方面。文本任务会比图像任务先被自动化。第二点是我们对文本任务投入非常大，相比之下在图像任务上做的工作很少。我们的模型表现如此之好让我很兴奋。我觉得这很棒。据我所知，我们在图像上的领先优势没有在文本上的那么大，但我们确实在图像上有整体领先优势。这是我的感觉。可能完全错误，但这就是我感知到的。实际上我会把图像模型描述为完全不够好。如果我说它已经生产就绪，我会真的很尴尬，因为里面还有很多不完善之处，如果人们能找到东西实际部署，我会很兴奋。但我要说这还没有我的认可印章。我们需要做很多轮，因为不像其他人只是发布一个雏形。但总的来说，我们的图像模型大约只有文本模型的 10% 的水平。

And so that is one aspect of multimodality we have to take into account. Text tasks will be automated before the image tasks. Thing two is we invested really heavily on text tasks and very much like we've done like a tiny amount of work in comparison to the image tasks. Like I am like kind of jazzed that our model does as well as it does. Um I think that that's great. Like as far as I can tell our lead in images is not as large as our lead in text, but we do have an overall lead in images. Uh is my vibe. Could be totally wrong, but that that's kind of what I've sensed. And I would actually describe the image model as completely insufficient. Like I would be truly embarrassed to say that this is prod ready because there's so much jaggedness left in it that I'm I'm jazzed if people find stuff to actually deploy. But I will say that this does not have like my seal of approval yet. We need to do so many rounds because like it like like it's it's not quite as bad as like other people just shipping a shape of something. But it's it's like in the grand scheme of it like image our image model is like maybe 10% of the way that our like text model is.

按照你的描述，从 10% 提升到 90% 或 95% 的过程中，你最兴奋看到 Gem 模型在图像方面最先做哪些事情？

As you move from like 10% by your characterization to like 90 95% what are the first kinds of things that you're excited about gem models doing on the image side?

在图像领域，我对电子游戏和计算机使用中的电子游戏情有独钟，因为我有电子游戏背景，而且，天哪，我承认我也玩过用机器人自动化游戏，不是为了竞争，只是因为自动化东西很酷，我觉得能够玩弄这些，而且

On image land I have a soft spot for video games and computer use video games because I have like a video game background I and oh man I I confession I've also like played around with botting them not for competitive reasons but just because it's cool to automate stuff and I think like being able to play around with that and like

软件有点像让技术服从你的意志，能够扩展你可以随意操控的东西，这本身就很酷。所以我觉得这在电子游戏中真的很酷。计算机使用我认为会非常有价值，难以置信地有价值。我认为它尤其与许多其他东西结合时。我认为 AI 对于界面应该好得多。它这么差实际上让我有点困惑，你知道，为什么有很多人不喜欢使用电脑却仍然不得不用，即使他们年纪大且不是专家，尽管 AI 可能比他们能雇的助手更擅长做很多电脑工作。我希望它足够可靠，人们可以信任它作为界面。你知道，我想另一个我有好感的东西，这就是为什么我穿着我的“反垃圾”T恤，我想要一些方法来阻止垃圾内容。对我个人来说，这可能对互联网不利，因为互联网靠广告资助。所以也许这是好事。我不知道。但我希望如果我能有或者有人能给我做一个垃圾内容拦截器，让我不再需要看到它。那就太好了。

I I software is a little bit like warping technology to your will and being able to like expand what you can warp to your will is just very cool in general. So I think that is really cool in video games. Computer use I think is going to be super duper valuable, unbelievably valuable. I I think it's it's like especially combined with like lots of other stuff. I think that AI should be so much better for interfaces. It's actually kind of confusing to me how it is so bad, you know, like why there's a lot of people who don't like using computers and they still have to even if they're like old and not experts despite that, despite AI being probably better than an assistant that they could hire to do a lot of computer stuff for them. And I would love for that to be reliable enough that people could like trust it as an interface. And you know, I I guess an additional thing that is like I have a soft spot for and this is why I'm wearing my like unsloppable shirt is I would love ways to just stop slop. Um, for me personally, it might be bad for the internet like like the internet is funded by ads. So like maybe it's a good thing. I don't know. But I would love if I could have some or someone would make me like a slop stopper where I just don't need to see it anymore. That would be lovely.

我从外部一直好奇的一件事是，你们在 Jev 取得的进展在多大程度上是由过去两三年 AI 领域的进步所促成的，还是说你们五年前就能做到？显然你们有不同的方法，某种程度上与领域其他部分正交，那么你们五年前能做到吗？或者你们构建的东西有多少利用了语言模型方面已完成的工作？

One thing I've been curious about from the outside is like the extent to which you know uh the advances that you had at Jev were enabled by like all the advances we've seen in in you know AI in the last two three years or could you have just done this 5 years ago like obviously you have a different approach that that is like in some ways orthogonal to the rest of the field like could you have done this five years ago or how much of what you built like does leverage some of the stuff that's been done you know on the LM side

五年前会更难，但可行。嗯，回到我更天真的时候，天哪，我听起来像个老家伙。嗯，当我更天真，以为这只需要一周的工作时，嗯，让我确信的是，我喜欢写文档。我觉得这是一种很好的思考方式。我在 OpenAI 写了一份文档，标题是“我们本可以在一年前就拥有 AGI”，至少按照 OpenAI 对 AGI 的定义，我认为人们并没有真正关注定义的文字，但我认为那实际上是一段很棒的文字。我认为从 GPT-3.5 时代开始，甚至可能是 GPT-3 时代的模型，在当时就可能符合 AGI 的定义，这个定义仍然相同，比如按数量完成世界上大多数有经济价值的工作。它没有说按数量，但我假设其精神是按数量。因为模型聪明得难以置信，GPT-3.5 级别的模型在各种事情上超过了中位水平的智力。难道这不应该被用于大量自动化，至少是简单的工作吗？所以是的，我认为像这样的事情会更难，但五年前绝对可以做到。

Five years ago it would have been harder but doable able. Um, back when I was more naive, oh man, I sound like an old dude. Um, when I was more naive and I thought that this would take a week of work, um, what convinced me, I love writing documents. I love I think it's a great way to think. I wrote a document at OpenAI that was titled we could have had AGI a year ago, at least by OpenAI's definition of AGI, which I don't think that people really listen to the letter of the definition, but I think it's actually a really great letter. Um, I I think that models from the starting from GPT3.5 era, possibly even GPT3 era, could have met the definition of AGI at the time, which is still the same, like doing most of the world's economically valuable work by volume. It doesn't say by volume, but I assume the spirit of that is by volume. Um, because the models are so unbelievably smart, like GPD 3.5 level models surpassed median level intelligence at all sorts of things. And like shouldn't that be used for tons of automation at least for the easy work? And so yes, I think that something like this would have been harder, but it could have been done 5 years ago. Absolutely.

你知道，对于那些痴迷于例如当今基准测试的人，他们看到 AI 在做什么任务，这些测试真的很难。它们真的非常非常难。GPQA 在 2023 年左右饱和了，也许没有完全饱和，但据我所知在 2023 年就超过了博士生水平，谷歌证明问答对于博士级智力在这些极其困难的多选题上，它在 2023 年就通过了那个水平。那种水平的智力怎么没有被投入使用，这太悲剧了。所以我认为我们拥有大量智力的原材料已经很久了，我们将看到很多疯狂的事情。

And you know, like on the people who are like obsessed with like for example like the the benchmarks of the day and they see like what tasks AI is doing, these tests are like really hard. They're like really really really hard. Um GPQA was saturated 2020. threeish maybe not totally saturated but was like surpassing PhD student level at 2023 I believe um Google proof question answering for like PhD level intelligence at these extremely hard multiple choice tasks it was passing that level 2023 like how is it that that level of intelligence was not being put to work that is super tragic so I think we've had a lot of like raw material for intelligence for a very long time and we're going to see a whole lot of craziness.

是的。我是说，说到那种疯狂，我想部分原因是，我猜有很多噪音。你知道，我觉得生态系统中有很多噪音，我敢打赌许多听众会熟悉，在你们推出这些模型之后，OpenAI、Cloudflare、微软和其他人都发布了这些模型。我知道你会说基准测试不是测试它们的好方法。他们显然试图表明它们几乎等效。我想你怎么看待，显然你现在已经邀请了很多竞争进入这个领域。

Yeah. I mean, speaking of that craziness, I think part of I imagine there's a lot of noise. You know, I feel like there's been a lot of noise in the ecosystem and I I bet many of our listeners will be familiar with, you know, after you guys came with these these models, you had OpenAI and Cloudflare and Microsoft and everyone kind of released these these models. Um, I I know you'll you'll say that the benchmarks aren't uh aren't a good way to test them. They obviously, you know, have tried to show that that that they're near equivalent. I guess how do you think about like you obviously have now invited a bunch of competition into the space.

你怎么看待 Jev 的护城河？我猜大家都喜欢问护城河的问题，所以我代表听众问一下。你认为这项业务未来的护城河会怎样？

How do you think about what keeps Jev's moat? I guess everyone likes to ask about moats, so I'll ask it on behalf of our listeners. How do you think about that going forward for the business?

天哪。感觉别人在替我们挖护城河，这跟我想的不太一样，我觉得这对整个领域不太好。我认为发布那些更注重基准测试优化而非真实世界优化的模型，实际上会消耗人们尝试新事物的善意，而尝试才是唯一重要的事。我真的觉得这对领域不利，我希望只提供关于可靠性的惊人事实将成为北极星。我认为这对世界将具有巨大价值。那里还有太多事情要做，所以现在担心竞争还为时过早。这有点像 OpenAI 担心 GPT-3.5 的某个远程版本。谁在乎呢？GPT-3.5 的市场微不足道，或者说曾经微不足道，对吧？相比之下，我认为在宏观格局中它仍然微不足道。我们显然真正相信杰文斯悖论，所以名字叫 Jev。

Oh boy. It feels like other people are digging our moat for us, which is kind of not how I thought it would go, and I think that's kind of bad for the field. I think that releasing models that are more benchmark-optimized than real-world-optimized actually burns that goodwill for people to try things out, which is the only thing that matters. I truly think that this is not great for the field, and my hope is that just giving unbelievable facts about reliability is going to be the north star. I think that will be immensely valuable for the world. There's so much more to do there that I think it's a little bit early to worry about competition. It's kind of like OpenAI worrying about a remote at GPT-3.5. Like, who cares? GPT-3.5's market is nothing, or it was nothing, right? In comparison, it's still, I think, going to be nothing in the grand scheme of things. We truly believe in Jevons paradox, obviously, hence the name Jev.

就像杰文斯悖论，我们是否看到反弹？因为价格便宜，需求更大了？

Like Jevons paradox, like are we seeing rebound? You know, more demand because it's so cheap?

绝对相信，我认为每个好东西都会看到反弹。我们是否看到适得其反，比如总市场规模增加？这就是杰文斯悖论成立的时候。绝对不是。关键部分必须是：哇，这太好了太便宜了，我要到处使用它，因为它在所有这些地方创造价值，我用于 AI 的预算实际上会增加。

Absolutely, I believe every good sees rebound. Are we seeing backfire like the total market size increases? This is when Jevons paradox holds. Absolutely not. A key part of it has to be like, wow, this is so good and so cheap, I'm going to be throwing it all over the place because it makes value in all these places, and my budget for AI will actually increase.

说到投资者，你最近显然筹集了一大笔资金。你是如何考虑为这项业务筹集多少资本才合理的？我肯定人们都想给你投钱。那你是怎么考虑的呢？

Speaking of investors, I guess you obviously recently raised a pretty big round. How did you even think about the amount of capital that made sense to raise for this business? I'm sure people were trying to throw money at you. So how did you think through that?

我得说，人们可能不相信，但我们的服务是可以盈利运营的，我希望我们永远不必做那种暂时补贴来锁定用户的事情。我不想完全排除这种可能，因为我是个务实的人，但我真的希望不必如此。我觉得那是一种非常阴暗、狡猾的让人们使用你服务的方式。所以我希望成为那种在囚徒困境中总是合作的人，直到被迫不合作。所以我们其实并不需要这笔钱。我们的目标反而是：嘿，我们想要最有价值的人站在我们这边，帮助我们建立这项业务，因为我们完全不知道怎么做。所以我们真正选择的是：AI 领域里最优秀的人是谁，他们会帮助我们？你可以看到我们董事会里有谁。

I will say that people probably don't believe it, but we can run our service profitably, and I hope that we never have to do the thing of temporarily subsidizing to get lock-in. I don't want to ax it out entirely because I'm a pragmatic person, but I really hope that doesn't have to be the case. I find that to be a very shadowy, sneaky way to get people to use your service. So I hope to be the kind of person who's always collaborating in these prisoner's dilemmas until forced otherwise. So we actually did not need the money. Our goal instead was, hey, we want the most valuable people on our side to help us build this business because we have no clue how to do this. So we really opted for, who are the best people in AI who will help us? You can see who we got on our board.

嗯，我总是喜欢以快速问答环节结束，就是把一些过于宽泛的问题塞到最后，那些我没机会在其他地方问的。我很好奇，在过去一个月或六周里，产品已经推出，你在过去六周里对 AI 的看法有什么改变，刚刚看到如何……

Well, I always like to end with a quickfire round where I just stuff some overly broad questions into the end that I haven't had a chance to ask elsewhere. I'm curious, over the last month or last six weeks of having this product out there, what's one thing you've changed your mind on in AI in the last six weeks, having just seen how...

这问题真酷。哇。哇。这真是个快速问答的问题。天哪。我得说，我们当时真的很担心，我们关心智能实际可靠性的理念是不是错了。团队里有人说：‘嘿，我们是不是应该直接发布一个基准测试版本，大肆宣传，做那种早期发布，显然不怎么样，但只是为了测试市场是否存在？’我很高兴我们没有那样做。我说七周是因为我不记得发布是什么时候了，但七周我想是在发布之前。我们真的不知道。这对我们来说是一个巨大的信仰飞跃，我们真的……天哪，我觉得如果我们做同样的过度承诺、交付不足，我们就无法为自己感到骄傲，而我认为那得到了超级好的回报。其实我不知道是否得到了回报。产品自己卖自己，也许如果我们真的为关注度优化，会做得更好。但那很棒，而且我觉得现在我的品牌已经巩固为我们的品牌，我希望永远不必再那样做，甚至接近那样，那个争论对我们来说不再存在了。我希望。我希望。我不知道。我真的真的希望，因为我很难给出不细致、稍微不正确的答案，因为我觉得那对任何听我说话的人都不公平。我宁愿不回答这个问题，也不愿做那种标准的 CEO 式真相压缩，偏向于慷慨。所以我希望我永远不必那样做。但我也学到，这种可靠性传奇对我们这类模型来说被速通了。我觉得我可能错了，但我现在对环境的温度检查是，人们已经对基准测试和不是真正有用的东西感到极度疲劳，比我预想的要快得多。根据我所看到的，我本以为这会是个问题，因为每次新模型发布，人们总是被基准测试吸引，对吧？人们会说：‘嗯，酷。我们来谈谈吧。’然后它令人失望，除非是 OpenAI 或 Anthropic 的模型，显然。那只是有点……我可能错了，也许我们会看到。我觉得它发生得比我预想的快得多，因为我们的模型如此易于基准测试，以至于人们可以如此快速地爆出那么多碾压基准的东西。但我觉得下一个相比之下会非常平淡无奇。

That's such a cool question. Whoa. Whoa. That's a quickfire question. Holy smokes. I will say that we were really worried if our philosophy about caring about the actual reliability of intelligence was a mistake. We had people in the team say, 'Hey, should we just release a benchmark version and just hype it up and do one of those early releases that obviously is not great, but just to test out if the market exists?' I'm so glad we didn't do that. And I say seven weeks because I don't remember when launch was, but seven weeks I think was before launch. We really did not know this. It was a big leap of faith for us, and we were just really... man, I don't think we could be proud of ourselves if we did the same overpromise underdeliver, and I think that paid off super duper well. I actually don't know if it paid off. The product has sold itself, and maybe it would have done even better if we really optimized for attention. But that was awesome, and I actually feel like now that my brand has been solidified as our brand, I hopefully never have to do that, even close to that, again, and that debate no longer exists for us. I hope. I hope. I don't know. I truly truly hope because I have a hard time even giving unnuanced, slightly incorrect answers because I think that's just a disservice to anyone listening to me. I'd rather just not answer the question than do the standard CEO compression of the truth that errors on the side of generous. So I hope that I will never really have to do that. But something I've learned as well is that this reliability saga was speedrun so fast for our class of models. I feel like I could be wrong, but my temperature check of the environment now is people have already gotten extreme fatigue of the benchmarks and not actually useful stuff way faster than I would have thought. I would have thought that this would be a concern based on what I see because people keep falling for benchmarks every time a new model's released, right? People are like, 'Well, cool. Let's talk about it.' And then it disappoints, except when it's like an OpenAI or Anthropic model, apparently. And that just happened kind of... I could be wrong, and maybe we'll see. I think it just happened a lot faster than I would have thought because our model is so benchmarkable such that people could blast out so many benchmark-crushing things so fast. But I feel like this next one is going to be very anticlimactic in comparison.

但作为播客主持人，我有合同义务问你，你知道，我们是一个 AI 播客。现在另一个大的 AI 故事显然是 Anthropic 上市，人们在思考那项业务。甚至在这次对话中，我也感到震惊，我其实不知道你会如何回答这个问题，因为一方面我觉得你对语言模型缺乏能力非常失望，也许它们几乎像一把锤子，试图敲打各种用例的每一颗钉子。

But I am as a podcast host, I'm contractually obligated to ask you, you know, like we're an AI podcast. The other big AI story right now is obviously Anthropic going public and folks kind of thinking through that business. And I'm struck even in the course of this conversation like I actually don't know how you're going to answer this question because on the one hand I feel like you're very disappointed in so many of the lack of capabilities that LMs have and maybe that they're like almost a hammer trying to go on every nail of different sorts of use cases.

另一方面，你一直挺肯定它们的——有些事编程智能体确实很擅长。所以当你把 Anthropic 想成一家一万五到两万亿美元的公司时，你对这一点乐观吗？比如“嘿，这被低估了”？还是你会说“哦，我不知道它会怎么样”？

On the other hand, you have been complimentary — there are things that coding agents are really good for. So as you think about Anthropic as a one-and-a-half, two-trillion-dollar company, is that something you're optimistic on? Like, hey, that's undervalued? Or are you like, oh, I don't know what's going to happen to that?

我认为编程智能体价值巨大。但我不清楚其中有多少实际使用是真正有价值的——它净为正，和它值不值投入的所有钱，是两回事。这大概是他们要去弄明白的产品问题，前提是他们真的在追求真相。他们大概有更好的答案。比如，是不是有些人只是拼命刷 token，还是他们真的获得了与投入相称的相对价值？所以我觉得自己了解得不够。对于编程智能体在不同规模项目、不同水平工程师身上能带来多少百分比提升，我有自己的猜测。我不会说软件问题已经接近解决，但这不代表它们没价值。我只是觉得这里需要一个务实的看法。

I think coding agents are massively valuable. It's unclear to me how much of the actual use is truly valuable — it being net positive is different than it being worth all of the money put in. That is probably a product question for them to figure out, if they actually are truth-seeking. They probably have a better answer to this. Like, did some people just token-max too hard, or are they actually getting the relative value that spending would make a difference on? So I don't think I'm informed enough for that. I have my guesses on what percentage gain coding agents are for various levels of projects and skills of engineer. And I wouldn't say that software is anywhere close to solved, but that doesn't mean they're not valuable. I just think that there needs to be a pragmatic take here.

我觉得有可能——我能想象编程智能体市场往两个方向走。我能想象它大涨，也能想象它大跌。大涨是因为，随着人们学会真正获取更多价值，人们会不会用得更多？我相信会。我也能想象它下跌，因为很多人做过头了，这个领域有太多 FOMO。而实际上，我对 AI 现状的感受并不是它一路指数上升，而是很多山丘，不断上、下、上、下、上、下，然后最终在幻灭低谷处稍微多一点。其实我不知道那个词怎么念。

I think that it's possible — I could see the coding agent market going either way. I could imagine it going very up and very down. Very up in the sense that as people learn to actually get more value, can people use it more? I believe so. I could imagine it going down because a lot of people have overdone it because there's so much FOMO in the field. And actually my sense of the state of AI is not that it's been an upward exponential, but a lot of hills that keep going up, down, up, down, up, down, and then eventually having a little more at the trough of disillusionment. I actually don't know how that word is pronounced.

低谷（trough）。

The trough.

对，然后最终它会变成更稳定的使用。我觉得营收走平可能就说明了这一点——人们意识到自己也许花得太多了。我也这么觉得。从产品角度看，这方面会有比我更强的专家。我其实有种感觉：这些实验室最大的优势不是模型、不是产品、也不是分发，而是金融工程。这是所有在实验室待得很资深的人总会谈到的——据我理解，他们用极其有利的方式设计这些交易，不是把自己套进去，而是把那些承担风险、想搭上火箭、愿意接受任何条款的其他人套进去。

Yeah, and then eventually it becomes more stable usage. And I think the flatlining revenue might be indicative of that — people realizing that maybe they spent too much. I think so. From a product perspective, I think there will be better experts than me at that. I actually get the sense that the greatest strength of the labs is not models or product or distribution, but financial engineering. This is the thing that everyone who's very veteran at the labs always talks about — where, my understanding is, they structure these deals in incredibly beneficial ways to not themselves over, but other people over who are taking the risk and trying to latch on to the rocket ship and are willing to accept whatever terms.

在算力方面。

On the compute side.

对，各方面都是，尤其是算力。据我所闻，金融工程是他们最大的优势，而我完全不知道那里面有多盘根错节，也不知道市场往任何方向走时他们会有多脆弱。你知道，泡沫可以仅仅因为预期没兑现就破掉，而不是因为它不是真正具有变革性的技术。所以我尽量让自己的预测保持校准。在没有内部信息时，我尽量不过度自信，我是个比较保守的人。我感觉很多被计入的预期不只是产品层面的，而是基于对 AI 未来及其能力的误解。所以我猜，因为有些人可能把它估成“造神基本等于无限”，那里会有一定程度的错配。而我觉得“让他们治愈癌症”会变成玛丽·安托瓦内特的“让他们吃蛋糕”。

Yes, on all sorts of sides, but especially compute. I think that financial engineering, as far as I've heard, is their greatest strength, and I have no idea how much of a rat's nest that is and how vulnerable they would be to the market going in any direction. You know, a bubble can pop just because expectations are not met, not because it's not truly a transformative technology. So I try to be calibrated in my predictions. I try not to be overly confident when I have no privy information, and I'm a more conservative person. I get the sense that a lot of the expectations baked in are not just product-level but based on misunderstandings of the future of AI and what it can do. So my guess is that, because some people might be valuing it like building God is basically infinite, there will be some amount of miscalibration there. And I think that 'let them cure cancer' is going to be like Marie Antoinette's 'let them eat cake.'

这句话说得好。

That is a good line.

是啊。我是说，显然——

Yeah. I mean, obviously—

哦，那也许结尾确实挺辣的。

Oh, maybe it did end spicy then.

那句话很棒。是啊。我是说，显然有很多人在押注从编程智能体到通用知识工作、到生物、再到其他这些领域的迁移，而且显然这些绝不是 RLVR 的完美用例。我仍然觉得，理解智能体——也许还有当下所有 AI——的细腻方式，是看在公司风险层面你能把哪些决策托付给它们，以及你能不能在不承担太多下行风险的情况下获得价值。编程在这方面相当独特，或者数学其实超级独特，因为除了你花的算力，没有下行风险，对吧？数学模型不会让公司付出任何代价，除非他们付钱让人验证证明，而他们到底付不付钱都不清楚。

That was a great line. Yeah. I mean, obviously so many people are underwriting the move from coding agents to general knowledge work to bio to these other spaces, and obviously not the perfect use cases for RLVR by any means. I still get the sense that the nuanced way to understand agents — perhaps and maybe all of AI right now — is what decisions can you trust them with in terms of stakes for the company, and can you get value without too much of the downside. And coding is quite unique in this, or math is actually super-duper unique, where there's no downside other than the compute you spend, right? Math models do not cost companies anything except if they're paying people to validate their proofs, which it's unclear if they even pay them for that.

是啊。

Yeah.

对我来说——也许有人知道。但代码的下行风险也多少更可控一些，因为你有 git，对吧？你有回路中的人类，有办法保护你免受不想要的副作用，而副作用正是强大工具的主要缺点。而强大工具是做知识工作所必需的，因为知识工作往往涉及现实世界中的副作用。我现在说话很像计算机科学家。我的感觉是，很多其他领域根本没有接近同等水平的防护措施、回滚能力、追踪等等。这就是为什么据我所知，还没有人把得来速自动化，尽管人们非常非常努力地尝试过——而来得速的风险比许多其他形式的知识工作低得多。所以我认为真相会是：这里面需要细腻和技巧，而大赢家不会是那些直接梭哈的人，而是理解现实、并按照技术现实去部署它们的人。所以，鉴于我对 AI 现状的判断，我觉得大多数人非常受 FOMO 驱动，对 AI 现状没有很好的理解，他们很可能会偏向过度乐观，从而被许多副作用烧伤；或者偏向过度悲观，那就是照旧，也就是大多数企业在这次 AI 热潮中的样子。

To me — maybe someone knows. But code also has somewhat more kept downside because you have a git, right? And you have humans in the loop, and there are ways to protect you from unwanted side effects, which is the main downside of powerful tools. And powerful tools are necessary to do knowledge work because knowledge work often involves side effects in the world. I'm talking a lot like a computer scientist right now. My sense is that for a lot of other fields, they don't have anywhere close to the same levels of safeguards and ability to revert and tracking and all of that. And that is why, to my knowledge, no one has automated a drive-thru yet, even though people have tried really, really hard — and the stakes of a drive-thru are a lot lower than the stakes of many other forms of knowledge work. So I think the truth will be that there'll be nuance and skill into it, and the people who will be the big winners are not the ones who just YOLO all in, but those who understand reality and deploy them in line with the realities of technology. So I'm not — given my state of AI — I feel like most people are very FOMO-driven and they do not have a great understanding of the state of AI, and they will probably err on the overoptimistic, which will get them burned with many side effects, or over-pessimistic, which is business as usual, which is how most businesses have been for this AI boom.

你刚才说到一件事——我很好奇，你说你对这些编程智能体实际提升工程生产力有多少有个估计。那个估计是多少？

One thing you said there — I'm curious, you said you have an estimate for how much these coding agents actually improve engineering productivity. What is that estimate?

好，我得说这有误差范围，因为我认为存在长期效应，比如技能退化、技术债之类。所以我的估计是在 20% 到 40% 的提速区间，但同样，这真的取决于用例，也取决于用户的技能水平。而且如果你能——因为副作用才是坏东西——如果你能把副作用隔离到非常清晰可辨的程度。

Okay, I will say I have error bars on this because I think there are long-term effects like skill degradation and tech debt and stuff like this. So my estimates are in the 20 to 40% speed-up range, but again it really depends on the use case, and it also depends on the skill of the user. And if you can — because the side effects are the bad thing — if you can isolate the side effects in a way that those are very legible.

所以取决于使用场景，对某些场景来说，这几乎是无限倍的加速。你知道，你可以做以前永远做不到的事情。对其他场景来说，它们太糟糕了，糟糕到跟手工写代码差不多。我猜平均来说，这是一个不错但小幅的提升。我相信这就是我目前在领域内看到的共识。

So depending on the use case, for some it's kind of like an infinite percent speed up. You know, you would do things you could never do otherwise. For others, they are just so bad that they're as bad as coding the thing by hand. I would guess on average it's like a decent but small boost. And I believe that is the consensus that I've seen in the field right now.

是的。另一个问题，我不太确定你怎么回答，比如你怎么知道？显然现在关于 AI 安全的讨论非常多。比如，你对这些 AI 安全对话有多担心？

Yeah. Another question I'm not totally sure how you answer, like, how can you know? There's obviously so much discourse around AI safety these days. Like, how concerned are you about a lot of these, you know, this AI safety conversation?

我认为 AI 安全不是一个单一的整体，它有很多不同的切片。我觉得 ASI 的讨论实际上与生物风险的讨论截然不同，而生物风险又与网络风险的讨论非常不同，网络风险又不同于信任与安全。

I think AI safety is not one monolith and there's like a ton of different slices of it. I think that the ASI discussion is actually extremely different from the biorisk discussion, which is very different from like the cyber risk discussion, which is different than like trust in trust and safety.

也许比如 ASI 与恶意行为者，你知道，利用生物和网络等等。

Maybe like ASI versus like bad actors, you know, using bio and cyber and whatnot.

我感觉大多数关心 ASI 的人有点脱离技术的现实，而且从一开始就是这样。你知道，我觉得你需要真正用 AI 构建一个可靠的系统，才能在房间里有一席之地，说某个东西会如此可靠地——也许不是邪恶，而是对齐失败——以至于它能毁灭世界。但我认为我们离那种东西还很远。此外，关于 ASI、RSI 以及模型本身是否有自主意识，其实有很多细微差别。比如回形针最大化器实际上与恶意行为者程序超级员工毁灭世界之类的事情非常不同。实际上，我认为这些在威胁模型上最终会变得相当不同。我认为我们距离真正的自主性还差几个突破。尽管人们喜欢解读这些思维链并试图赋予它真正的自主性，但我猜我们距离这个目标还差大约两个巨大规模的突破。我猜甚至一个突破都不够，但也许一个巨大的突破可能就够了。而且我认为，拥有超级工具并想造成破坏的人类，其威胁模型与 ASI、RSI 的事情非常不同。我不——我认为很多相信 RSI 的人既没有研究过多少 AI 历史，也不了解安达尔定律。比如，你知道，如果你假设智能是一个光谱，而且是唯一重要的东西，你必须把事物映射到那个光谱中，你也可以把人类和黑猩猩映射进去。在我看来这是一个糟糕的论点。然后它看起来确实很糟，但现实是它并不是那样运作的，而许多不同类型的智能以及如何可靠地实现它们的细微差别在这个整个类比中非常重要。AI 开发领域能变得快一点吗？是的，但人们已经尝试这个很久了，通常并不成功，而且我不认为人们会把 AI 研究上 10% 的加速算作 RSI，如果它实际上并没有随时间增加的话。所以我非常愿意被证据说服，但我认为这更像是一个哲学问题而不是技术问题。那些喜欢读波士顿超级智能并把它当作现实而不是一本非常有趣的科幻书的人，让我有点担心。

I get the sense that most people concerned with ASI are a little bit divorced from the reality of the technology and this has been the case since day one. You know, like, I feel like you need to actually build a reliable system with AI to have a voice in the room that something's going to be so reliably, maybe not evil but misaligned that it can destroy the world. But I think we're pretty far from stuff like that. Additionally, there's actually a bunch of nuance towards like, you know, ASI, RSI, and like whether or not the models have a sense of agency on their own. Like paperclip maximizer is actually very different than bad actor programs super employee to destroy the world type things. And actually, I think that those end up becoming quite different in threat model. I think that we're breakthroughs away from true agency. As much as people like to read these chains of thought and try to ascribe real agency to it, I would guess that we are something like two breakthroughs of gigantic magnitude away from this. I think even one breakthrough would not be sufficient is my guess, but maybe one gigantic one could be. And I think the threat model of like humans with super tools who want to do damage is very different than the ASI RSI thing. I don't, I think a lot of people who also believe in RSI both have not studied the history of AI very much and also have no appreciation of Andal's law. Where like, you know, if you assume that intelligence is the spectrum and it's the only one that matters and you have to map things into that spectrum and you can also map humans and chimpanzees. It's a horrible argument in my opinion. Then it does look bad but the reality is that it doesn't work that way and like the nuance of the many different types of intelligence and how to do them reliably matters a lot in this whole analogy. Could the field of AI development become a little bit faster? Yes, but people have been trying this for a long time and it usually doesn't work out and I don't think that people would count like a 10% speed up on AI research as RSI if it doesn't actually increase over time. So I'm very open to being convinced by evidence, but I think that it's more of a philosophical problem rather than a technological one. The kinds of people who like read a Boston super intelligence and treat it like reality over a very interesting sci-fi book makes me kind of worried.

是的。嗯，这是一次迷人的对话。我觉得我们涵盖了很多东西。我想确保把最后的话留给你。比如你有什么想推荐给听众的吗？我的意思是，我肯定他们都很熟悉，而且到这个播客结束时当然希望他们是，但比如我们没讨论过的或者我不知道的。请讲。

Yeah. Well this has been a fascinating conversation. I feel like we've covered a ton of things. I want to make sure to leave the last word to you. Like anything you want to point our listeners to? I mean I'm sure they're all familiar with and certainly by the end of this podcast certainly hope they are but like anything we haven't discussed or I don't know. The floor is yours.

软件加 AI 已经发生了太多事情。我觉得这会非常不可思议。我认为我们需要开始摆脱 LLMs 教给我们的这种缺乏自主性的观念，开始真正地构建，就像你知道的，有太多可能性将被打开，我们应该回到玩耍的状态，而不是成为这些东西的受害者，我会尽我们所能让你知道可玩的 AI。但这——我真的想——哦天哪，我在这里有太多话要说。我觉得我会为这个领域尽我所能，让它有机会重新思考软件、重新思考界面、重新思考一切。但这需要真正的构建者来弄清楚如何做到。我认为我们可以一起让 AI 成为创造中真正强大的向善力量，以及全方位的卓越，你知道，有时甚至是真正古怪离奇的东西。我觉得那才是技术真正闪耀的地方。

That there's so much stuff to have happened in software plus AI. I think it'll be so incredible. I think that we need to start untraining ourselves of this lack of agency that LLMs have kind of taught us about AI and start really building again like there's you know like there's so many possibilities that are going to be opened up and we should get back to playing rather than being like these victims of stuff and I will do whatever we can to make you know playable AI. But it's I really want Oh man, I have like so much to say here. I feel like I will be doing whatever I can for the field to like have this opportunity to just rethink software, rethink interfaces, rethink everything. But it's going to require like real builders to figure out how to do that. And I think that together we can make AI this really awesome force for good in creation and just all around awesomeness, you know, like and sometimes even like really weird wacky stuff. I feel like that that's where like technology really shines.

我是 Jacob Efron，这里是 Unsupervised Learning，一个播客，我在其中与 AI 领域最聪明的人交谈，问他们大量关于模型正在发生什么以及它对世界上企业意味着什么的问题。我希望很明显，我做这个非常开心。这是我在 Redpoint 作为投资者的日常工作之外的夜间和周末项目。但我们能请到这些不可思议的嘉宾，真正来自于像你这样的人订阅播客、与朋友分享。这确实是最终让整个事情运作起来的原因。所以，请考虑这样做。非常感谢你的支持和收听。我们下集见。

I'm Jacob Efron and this has been Unsupervised Learning, a podcast where I get to talk to the smartest people in AI and ask them tons of questions about what's happening with models and what it means for businesses in the world. As I hope is clear, I have a ton of fun doing this. It's a nights and weekends project in addition to my day job as an investor at Redpoint. But our ability to get these incredible guests on really comes from folks like you subscribing to the podcast, sharing it with friends. It's really what ultimately makes this whole thing work. And so, please consider doing that. And thank you so much for your support and listening. We'll see you next episode.