Cursor 团队谈 AI 辅助编程的未来

好了，太棒了。我们请到了来自 Cursor 团队的 Michael、Aman、Sual 和 Arvid。第一个大问题：代码编辑器的意义是什么？

All right, this is awesome. We have Michael, Aman, Sual, and Arvid here from the Cursor team. First up, big ridiculous question: what's the point of a code editor?

代码编辑器主要是你构建软件的地方。长期以来，它意味着你通过文本编辑来编写形式化编程语言的地方。对于非程序员来说，可以把代码编辑器想象成一个为程序员打造的超级增强版文字处理器。它之所以增强，是因为代码有很多结构，所以代码编辑器能为你做很多文字处理器在文本编辑中做不到的事情。这包括从代码中实际标记的视觉区分，让你快速浏览，到让你像浏览互联网超链接一样在代码库中导航——跳转到你使用的定义——再到错误检查以捕捉基本错误。传统上，这就是代码编辑器的含义。我认为未来 10 年，随着构建软件的含义开始发生变化，代码编辑器的定义将发生很大变化。

So the code editor is largely the place where you build software. Today, for a long time, that's meant the place where you text edit a formal programming language. For people who aren't programmers, think of a code editor as a really souped-up word processor for programmers. The reason it's souped up is that code has a lot of structure, so the code editor can actually do a lot for you that word processors haven't been able to do for people editing text. That includes everything from visual differentiation of the actual tokens in the code so you can scan it quickly, to letting you navigate around the codebase like you're navigating the internet with hyperlinks—going to definitions of things you're using—to error checking to catch rudimentary bugs. Traditionally, that's what a code editor has meant. I think what a code editor is will change a lot over the next 10 years as what it means to build software starts to look a bit different.

我还认为代码编辑器应该有趣。是的，这非常重要。这其实是我们决定构建什么时被低估的一个方面。我们构建的很多东西，我们会尝试、做实验，然后因为不好玩而扔掉。有趣的一个重要方面是快。很多时候，快就是有趣。快就是有趣——这应该印在 T 恤上。从根本上说，我认为吸引很多人用电脑构建东西的原因之一就是这种惊人的集成速度。在其他学科中，你可能受限于资源或召集大团队的能力。而编程是一件神奇的事情，只有你和电脑，仅凭这一点，你就能非常快速地构建出很酷的东西。

I think also code editor should just be fun. Yes, that is very important. It's actually an underrated aspect of how we decide what to build. A lot of the things we build, we try them out, do an experiment, and then throw them out because they're not fun. A big part of being fun is being fast. A lot of the time, fast is fun. Fast is fun—that should be a t-shirt. Fundamentally, I think one of the things that draws a lot of people to building stuff on computers is this insane integration speed. In other disciplines, you might be capped by resources or the ability to get a large group together. Coding is just an amazing thing where it's you and the computer, and that alone you can build really cool stuff really quickly.

对于不了解的人来说，Cursor 是一个很酷的新编辑器，它是 VS Code 的一个分支。我很想听听你们自己对编辑器之旅的解释。你们是怎么接触到 VS Code 的，这又如何引导你们走上了 Cursor 的道路？

For people who don't know, Cursor is this super cool new editor that's a fork of VS Code. It would be interesting to get your explanation of your own journey of editors. How did you arrive at VS Code, and how did that lead to your journey with Cursor?

是的，我想我们很多人——实际上我们所有人——最初都是 Vim 用户。纯 Vim，纯 VI，在终端里。至少对我自己来说，大约在 Copilot 发布的时候，也就是 2021 年，我非常想尝试它。我进入了 VS Code，这是唯一可用的平台。尽管我非常喜欢使用 Vim，但 Copilot 与 VS Code 的体验足以说服我切换。这成了默认选择，直到我们开始开发 Cursor。也许我们应该解释一下 Copilot 的功能。它是一个非常好的自动补全工具。当你开始写东西时，它会建议一两三行来完成它。有一种有趣的体验：当你和好朋友关系密切，朋友帮你完成句子时，如果做得好，会有一种亲密感。有一种很酷的感觉，就像‘天哪，它懂我了’。然后当它不懂你时，会有一种不愉快的感觉。但我想对很多人来说，它懂你的感觉压过了它不懂你的感觉。实际上，GitHub Copilot 被低估的一个方面是，即使它错了，也只是有点烦人，但没那么糟糕，因为你只需再输入一个字符，也许它就懂你了。所以即使错了，也没那么糟。你可以迭代修复它。对我来说，Copilot 另一个被低估的地方是，它是第一个真正的人工智能产品——第一个面向消费者的语言模型产品。Copilot 有点像 LLM 的第一个杀手级应用。测试版是在 2021 年发布的，对吧？

Yeah, so I think a lot of us—well, all of us—were originally Vim users. Pure Vim, pure VI, in a terminal. At least for myself, it was around the time that Copilot came out, so 2021, that I really wanted to try it. I went into VS Code, the only platform in which it was available. Even though I really enjoyed using Vim, the experience of Copilot with VS Code was more than good enough to convince me to switch. That kind of became the default until we started working on Cursor. Maybe we should explain what Copilot does. It's a really nice autocomplete. As you start writing a thing, it suggests one or two or three lines to complete it. There's a fun experience: when you have a close friendship and your friend completes your sentences, when it's done well, there's an intimate feeling. There's a cool feeling of 'holy, it gets me now.' And then there's an unpleasant feeling when it doesn't get you. But I would say for a lot of people, the feeling that it gets you overpowers that it doesn't. Actually, one of the underrated aspects of GitHub Copilot is that even when it's wrong, it's a little bit annoying but not that bad, because you just type another character and then maybe it gets you. So even when it's wrong, it's not that bad. You can sort of iterate and fix it. The other underrated part of Copilot for me was just the first real AI product—the first language model consumer product. Copilot was kind of like the first killer app for LLMs. The beta was out in 2021, right?

那么 Cursor 的起源故事是什么？

So what's the origin story of Cursor?

大约在 2020 年，OpenAI 发表了缩放定律论文。那是一个时刻，看起来这个领域有了清晰可预测的进展。即使我们没有任何新想法，似乎只要有更多的算力和数据，就能让这些模型变得更好。顺便说一句，我们可能会花三四个小时讨论缩放定律这个话题，但简单总结一下：这是一篇论文和一系列论文及观点，它们认为在机器学习领域，更大的模型规模和更多的数据可能更好。更大更好，而且是可预测地更好。总之，大约在那时，我们中的一些人进行了很多概念性的讨论，探讨这将会是什么样子，所有不同的知识工作者领域将如何因这项技术的进步而变得更好。然后，有几个时刻，论文中预测的理论收益开始变得非常具体。感觉就像是一个时刻，你实际上可以去做有用的人工智能工作，而不需要博士学位。有一整套系统可以构建，非常有用。第一个时刻我们已经稍微谈过了：早期使用 Copilot 的体验非常棒和神奇。下一个重大时刻是当一切开始融合在一起时，实际上是我们早期获得了 GPT-4 的访问权限，大概是 2022 年底。我们在摆弄那个模型，能力的提升感觉非常巨大。

So around 2020, the scaling laws papers came out from OpenAI. That was a moment where this looked like clear predictable progress for the field. Even if we didn't have any more ideas, it looked like you could make these models a lot better if you had more compute and more data. By the way, we'll probably talk for three to four hours on the topic of scaling laws, but just to summarize: it's a paper and a set of papers and ideas that say bigger might be better for model size and data size in the realm of machine learning. It's bigger and better, but predictively better. Anyway, around that time for some of us, there were a lot of conceptual conversations about what this is going to look like, what the story is going to be for all these different knowledge worker fields about how they're going to be made better by this technology getting better. Then I think there were a couple of moments where the theoretical gains predicted in that paper started to feel really concrete. It started to feel like a moment where you could actually go and do useful work in AI without needing a PhD. There was this whole set of systems one could build that were really useful. The first moment we already talked about a little bit: playing with the early bit of Copilot was awesome and magical. The next big moment where everything kind of clicked together was actually getting early access to GPT-4, so sort of end of 2022. We were tinkering with that model, and the step up in capabilities felt enormous.

在那之前，我们一直在做几个不同的项目。因为 CoPilot、缩放定律以及我们之前对这项技术的兴趣，我们一直在为程序员捣鼓一些工具，但都是非常具体的东西。比如我们为需要在 Jupyter notebook 中工作的金融专业人士构建工具，或者尝试用这些模型做静态分析。然后 GPT-4 的进步感觉真正实现了我们之前预测的理论收益。感觉当时你立刻就能构建更多东西。而且，如果我们保持一致的话，真的感觉这不会只是一个点解决方案；整个编程都将通过这些模型进行。这感觉需要一种不同类型的编程环境，一种不同类型的编程。所以我们开始构建那个更大的愿景。

Previous to that we had been working on a couple of different projects. Because of CoPilot, because of scaling laws, because of our prior interest in the technology, we had been tinkering around with tools for programmers, but things that are very specific. So we were building tools for financial professionals who have to work in a Jupyter notebook, or playing around with can you do static analysis with these models. And then the step up in GPT-4 felt like that really made concrete the theoretical gains that we had predicted before. It felt like you could build a lot more immediately at that point in time. And also, if we were being consistent, it really felt like this wasn't just going to be a point solution thing; this was going to be all of programming flowing through these models. It felt like that demanded a different type of programming environment, a different type of programming. And so we set off to build that larger vision.

有一个我特别记得。我的室友是 IMO 金牌得主，美国有个比赛叫 Putnam，相当于大学生版的 IMO。这是个数学竞赛，他非常厉害。所以尚彤和阿曼，我记得大概是 2022 年 6 月，打了个赌，赌到 2024 年 6 月或 7 月，模型能否在 IMO 中赢得金牌。IMO 是国际数学奥林匹克。是的，IMO 是国际数学奥林匹克。所以阿维德和我都参加过，所以有点个人感情。我记得当时想，老兄，这根本不可能。尽管我有点相信进步，但我觉得，你知道，我只是个女孩，阿曼简直是妄想。那就是那个赌局。老实说，我大错特错，但这可能是团队里最有先见之明的赌局。所以 DeepMind 的新结果证明你是对的？嗯，严格来说不算错，只差一分。

There's one that I distinctly remember. My roommate is an IMO gold winner, and there's a competition in the US called Putnam, which is sort of the IMO for college people. It's this math competition. He's exceptionally good. So Shang Tong and Aman, I remember it sort of June of 2022, had this bet on whether by 2024 June or July, you were going to win a gold medal in the IMO with models. IMO is International Math Olympiad. Yeah, IMO is International Math Olympiad. So Arvid and I both also competed in it, so it was sort of personal. And I remember thinking, man, this is just not going to happen. This was like, even though I sort of believed in progress, I thought, you know, I'm a girl, just like Aman is just delusional. That was the bet. And to be honest, I was very wrong, but that was maybe the most prescient bet in the group. So the new results from DeepMind turned out that you were correct? Well, technically not incorrect, but one point away.

阿曼当时对这些东西非常热衷。之前，阿曼有件缩放定律的 T 恤，他经常穿着走来走去，上面有图表和公式。

Aman was very enthusiastic about this stuff back then. And before, Aman had this scaling laws t-shirt that he would walk around with, where it had charts and the formulas on it.

所以你感受到了 AI，或者说你感受到了 Scaling？

So you felt the AI, or you felt the scaling?

是的，我记得有一次我和 Michael 的对话，在那之前我没有深入批判性地思考过缩放定律，他提出了一个问题：为什么 Scaling 不是全部，或者为什么 Scaling 不会带来巨大的进步？我想我经历了悲伤的各个阶段：愤怒、否认，最后思考之后，接受。从那以后，我对进步一直相当乐观和充满希望。我想有一点要说明，我认为这还取决于你在哪些领域看到进步。数学是一个很好的领域，尤其是形式定理证明，你能得到验证正确性的绝佳信号。所以这意味着强化学习可以工作得非常好。而且我认为你可以拥有在数学上可能非常超人的系统，但仍然没有技术上的 AGI。

Yeah, I remember there was this conversation I had with Michael where before I hadn't thought super deeply and critically about scaling laws, and he kind of posed the question: why isn't scaling all you need, or why isn't scaling going to result in massive gains in progress? And I think I went through the stages of grief: anger, denial, and then finally at the end, just thinking about it, acceptance. And I think I've been quite hopeful and optimistic about progress since. I think one thing I'll caveat is I think it also depends on which domains you're going to see progress. Math is a great domain because, especially like formal theorem proving, you get this fantastic signal of actually verifying if the thing was correct. And so this means something like reinforcement learning can work really, really well. And I think you could have systems that are perhaps very superhuman in math and still not technically have AGI.

好的，那么我们能一路聊到 Cursor 吗？Cursor 是什么？它是 VS Code 的一个分支。VS Code 长期以来是最流行的编辑器之一。每个人都爱上了它，每个人都离开了 Vim，我离开了 Emacs 投奔它。抱歉。所以它在某种程度上统一了开发者社区。然后你看看这个领域，看看缩放定律，AI 变得惊人，你决定，好吧，仅仅为 VS Code 写一个扩展是不够的，因为那有很多限制。如果 AI 要变得越来越好，我们需要真正重新思考 AI 如何成为编辑过程的一部分。所以你决定分支 VS Code，并开始构建许多我们将会谈到的惊人功能。但那个决定是怎样的？因为 VS Code 上有很多扩展，包括 CoPilot，都在做 AI 相关的事情。分支 VS Code 的决定是怎样的？

Okay, so can we take it all the way to Cursor? And what is Cursor? It's a fork of VS Code. And VS Code is one of the most popular editors for a long time. Everybody fell in love with it, everybody left Vim, I left Emacs for it. Sorry. So it unified in some fundamental way the developer community. And then you look at the space of things, you look at the scaling laws, AI is becoming amazing, and you decide, okay, it's not enough to just write an extension for VS Code because there's a lot of limitations to that. We need, if AI is going to keep getting better and better and better, we need to really rethink how the AI is going to be part of the editing process. And so you decided to fork VS Code and start to build a lot of the amazing features we'll be able to talk about. But what was that decision like? Because there's a lot of extensions, including CoPilot, on VS Code that are doing AI type stuff. What was the decision like to just fork VS Code?

对我们来说，做编辑器的决定似乎是不言而喻的，至少对于我们想要做的事情和实现的目标来说是这样。因为当我们开始做编辑器时，想法是这些模型会变得更好，它们的能力会提升，这将完全改变你构建软件的方式，既带来巨大的生产力提升，也从根本上改变构建软件的活动。如果你只是现有编码环境的插件，你对代码编辑器的控制非常有限。我们不想被这些限制束缚；我们希望能够构建最有用的东西。

The decision to do an editor seemed kind of self-evident to us for at least what we wanted to do and achieve. Because when we started working on the editor, the idea was these models are going to get much better, their capabilities are going to improve, and it's going to entirely change how you build software, both in terms of big productivity gains but also radical changes in how the activity of building software is going to change a lot. And so you're very limited in the control you have over a code editor if you're a plugin to an existing coding environment. And we didn't want to get locked in by those limitations; we wanted to be able to just build the most useful stuff.

好的，那么自然的问题是，VS Code 加上 CoPilot 算是竞争对手。那么你怎么赢？基本上就是靠速度和功能质量吗？

Okay, well then the natural question is, VS Code is kind of with CoPilot a competitor. So how do you win? Is it basically just the speed and the quality of the features?

是的，我的意思是，我认为这是一个非常有趣、也许非常独特的领域。如果你看之前的技术浪潮，可能只有一件大事发生，开启了一波新公司。但每一年，每一次模型能力的跃升，都会解锁一波新功能，尤其是编程领域的新可能。所以我认为在 AI 编程中，哪怕只是领先几个月，更不用说领先一年，都会让你的产品有用得多得多。我认为一年后的 Cursor 需要让今天的 Cursor 看起来过时。而且我认为微软做了很多了不起的事情，但我不认为他们处于一个很好的位置，能够像初创公司那样真正持续创新和推进，快速实现功能并推动，进行必要的研究实验来突破天花板。我不确定我是否从功能的角度来思考，而是从程序员的能力角度。就像，你知道，新模型出来了，而且肯定会有更多不同类型的模型，比如更长的上下文窗口和更快的速度，有所有这些疯狂的想法你可以尝试，希望 10% 的疯狂想法能变成酷而有用的东西。我们希望人们能更快地拥有这些。换句话说，一个被低估的事实是，我们是为自己做的。当我们开始做 Cursor 时，你真的感受到这种挫败感：模型在变得更好，但工具体验没有改变。就像，老兄，这些家伙，赌注越来越高，他们为什么不做新东西？他们应该做新东西。

Yeah, I mean, I think this is a space that is quite interesting, perhaps quite unique. Where if you look at previous tech waves, maybe there's kind of one major thing that happened and unlocked a new wave of companies. But every single year, every single model capability jump you get, you unlock this new wave of features, things that are possible, especially in programming. And so I think in AI programming, being even just a few months ahead, let alone a year ahead, makes your product much, much, much more useful. I think the Cursor a year from now will need to make the Cursor of today look obsolete. And I think Microsoft has done a number of fantastic things, but I don't think they're in a great place to really keep innovating and pushing on this in the way that a startup can, just rapidly implementing features and pushing, and doing the research experimentation necessary to really push the ceiling. I don't know if I think of it in terms of features as I think of it in terms of capabilities for programmers. It's that, you know, as the new model came out, and I'm sure there are going to be more models of different types, like longer context and maybe faster, there's all these crazy ideas that you can try, and hopefully 10% of the crazy ideas will make it into something kind of cool and useful. And we want people to have that sooner. To rephrase, it's an underrated fact that we're making it for ourselves. When we started Cursor, you really felt this frustration that models were getting better, but the tool experience had not changed. It was like, man, these guys, the stakes are getting higher, why are they not making new things? They should be making new things.

应该像这样，你知道，所有那些前沿功能在哪？根本没有前沿功能。我敢肯定它卖得很好，我也肯定它是个好生意，但感觉上……我是那种真的很想尝试和使用新事物的人，但很长一段时间里就是没有新东西。

Should be like, you know, where are all the alpha features? There were no alpha features. I'm sure it was selling well, I'm sure it was a great business, but it didn't feel... I'm one of these people that really want to try and use new things, and there was just no new thing for a very long while.

是啊，这很有意思。我不知道该怎么形容，但当你把 Cursor 和 Copilot 比较时，Copilot 很快就让人觉得有点过时了，不知为什么。

Yeah, it's interesting. I don't know how you put that into words, but when you compare Cursor with Copilot, Copilot pretty quickly started to feel stale for some reason.

是的，我认为有一点对我们有帮助，就是我们几乎是全栈式地做这件事，我们同时开发用户体验和与模型交互的方式，以及如何让模型给出更好的答案。比如如何构建提示词，如何找到上下文，还有对于 Cursor Tab，如何训练模型。所以我认为这有助于我们让同一批人从头到尾负责整个体验。

Yeah, I think one thing that helps us is that we're sort of doing it all in one, where we're developing the UX and the way you interact with the model, and at the same time as we're developing how we actually make the model give better answers. So like how you build up the prompt, or how do you find the context, and for a Cursor Tab, like how do you train the model. So I think that helps us to have all of it sort of like the same people working on the entire experience end to end.

是啊，就像做 UI 的人和训练模型的人只隔 18 英尺，甚至经常是同一个人。

Yeah, it's like the person making the UI and the person training the model sit like 18 feet away, so often the same person even.

是的，经常甚至就是同一个人。所以你能创造出一些东西，如果你不交流、不实验，而且像你说的，用 Cursor 来写 Cursor，那是不可能的。

Yeah, often even the same person. So you can create things that are sort of not possible if you're not talking, you're not experimenting, and you're using, like you said, Cursor to write Cursor.

当然，没错。好吧，我们来谈谈其中一些功能。我们来谈谈那无所不知、无所不能的 Tab 吧。基本上就是增强版的自动补全。那么 Tab 是如何工作的？Tab 是什么？请从高层次概括一下。

Of course, yeah, yeah. Well, let's talk about some of these features. Let's talk about the all-knowing, all-powerful, praise be to the Tab. So the autocomplete on steroids basically. So how does Tab work? What is Tab? To highlight and summarize it at a high level.

我想说目前 Cursor 有两件事做得很好。它还有其他功能，但有两件事对程序员很有帮助。一个是这种在你背后看着你、像一个非常快速的同事一样，能跳到你前面打字并猜出你下一步要做什么的想法。这就是最初的想法，内核，一个好的自动补全背后的理念就是预测你下一步要做什么。你可以把这个概念做得更宏大，不只是预测光标后的字符，而是实际预测你将要做的下一个完整修改，下一个差异，下一个你要跳转的地方。第二件 Cursor 目前也做得不错的事是，它有时能帮你跳到 AI 前面，告诉它要做什么，从指令到代码。在这两方面，我们都做了很多工作，让这些操作的编辑体验更符合人体工程学，同时也让它们更智能、更快速。

I'd say that there are two things that Cursor is pretty good at right now. There are other things that it does, but two things it helps programmers with. One is this idea of looking over your shoulder and being like a really fast colleague who can kind of jump ahead of you and type and figure out what you're going to do next. And that was the original idea behind that, the kernel, the idea behind a good autocomplete was predicting what you're going to do next. You can make that concept even more ambitious by not just predicting the characters after cursor, but actually predicting the next entire change you're going to make, the next diff, the next place you're going to jump to. And the second thing Cursor is pretty good at right now too is helping you sometimes jump ahead of the AI and tell it what to do, and go from instructions to code. And on both of those, we've done a lot of work on making the editing experience for those things ergonomic, and also making those things smart and fast.

我们非常想要的一件事是让模型能够为我们编辑代码。这算是一个愿望，在拥有一个能为你编辑代码的好模型之前，我们尝试了很多次。之后我们有了一个好模型，我认为为了获得良好体验，在推理速度上做了很多努力。我们开始整合，迈克尔也提到过，跳转到不同位置的能力。这种跳转我认为源于一种感觉：一旦你接受了一个编辑，下一步该去哪应该非常明显。就像，我做了这个改动，模型应该知道下一步要去的地方是往下 18 行。如果你是 Vim 用户，你可以按 18j 之类的，但为什么我要做这个？模型应该直接知道。所以想法就是，你只需按 Tab，它就会往下 18 行，然后显示下一个编辑，你再按 Tab。就这样，只要你一直按 Tab 就行。所以内部的竞争就是，我们能让他们按多少次 Tab。

One of the things we really wanted was we wanted the model to be able to edit code for us. That was kind of a wish, and we had multiple attempts at it before we had a sort of a good model that could edit code for you. Then after we had a good model, I think there have been a lot of effort to make the inference fast for having a good experience. And we've been starting to incorporate, I mean Michael sort of mentioned this, the ability to jump to different places. And that jump to different places I think came from a feeling of, once you accept an edit, it should be just really obvious where to go next. It's like, I made this change, the model should just know that the next place to go to is like 18 lines down. If you're a Vim user, you could press 18j or whatever, but why even am I doing this? The model should just know it. And so the idea was you just press Tab, it would go 18 lines down and then show you the next edit, and you would press Tab. So it's just you as long as you could keep pressing Tab. And so the internal competition was how many tabs can we make them press.

一旦你有了这个想法，更抽象地说，要思考的是，一旦编辑是某种零熵状态。也就是说，一旦你表达了你的意图，编辑中已经没有新的信息位来完成你的想法，但你仍然需要输入一些字符来让计算机理解你实际在想什么，那么也许模型就应该直接读懂你的心思，所有零熵的部分都应该通过按 Tab 消除掉。是的，这大概就是抽象概念。

Once you have the idea, more abstractly, the thing to think about is sort of like, once the edit is sort of zero entropy. So once you've sort of expressed your intent and the edit is there's no new bits of information to finish your thought, but you still have to type some characters to make the computer understand what you're actually thinking, then maybe the model should just sort of read your mind and all the zero entropy bits should just be tabbed away. Yeah, that was sort of the abstract.

有一件有趣的事，如果你看语言模型在不同领域的损失，我相信代码的每字节比特数（一种字符归一化损失）比自然语言要低，这意味着通常代码中有很多 token 是超级可预测的，很多字符是超级可预测的。我认为当你不仅仅是自动补全代码，而是预测用户在编辑现有代码时下一步要做什么时，这一点甚至被放大了。所以 Cursor Tab 的目标是：当意图被有效确定时，让我们消除你在编辑器中执行的所有低熵操作。让我们直接把你向前跳转，跳过时间。

There's this interesting thing where if you look at language model loss on different domains, I believe the bits per byte, which is kind of character-normalized loss for code, is lower than language, which means in general there are a lot of tokens in code that are super predictable, a lot of characters that are super predictable. And this is I think even magnified when you're not just trying to autocomplete code but predicting what the user is going to do next in their editing of existing code. And so you know the gold for Cursor Tab is: let's eliminate all the low-entropy actions you take inside of the editor when the intent is effectively determined. Let's just jump you forward in time, skip you forward.

那么，做下一个光标预测的直觉和技术细节是什么？这个跳转对人们来说并不那么直观，我想。

Well, what's the intuition and what's the technical details of how to do next cursor prediction? That jump is not so intuitive, I think, to people.

是的，我想我可以谈谈让这些东西工作的一些细节。它们需要极低的延迟，所以你需要在这个任务上训练小模型。特别是，它们非常消耗预填充 token。这意味着它们有非常非常长的提示词，能看到你很多代码，但实际上并不生成那么多 token。所以完美的方案是使用稀疏模型，也就是混合专家模型。这是我们取得的一个突破，大大提高了它在长上下文下的性能。另一个是我们构建的一种推测解码变体，叫做推测编辑。我认为这两个是让它高质量且非常快速的重要因素。

Yeah, I think I can speak to a few of the details on how to make these things work. They're incredibly low latency, so you need to train small models on this task. In particular, they're incredibly prefill-token hungry. What that means is they have these really really long prompts where they see a lot of your code and they're not actually generating that many tokens. And so the perfect fit for that is using a sparse model, meaning a Mixture of Experts model. So that was kind of one breakthrough we made that substantially improved its performance at longer context. The other being a variant of speculative decoding that we kind of built out called speculative edits. These are two I think important pieces of what make it quite high quality and very fast.

好的，所以是混合专家模型，输入巨大，输出很小。好的，那么关于如何实现它，你还能说些什么？比如缓存起了作用吗？

Okay, so Mixture of Experts, the input is huge, the output is small. Okay, so what else can you say about how to make it? Like caching plays a role in this?

缓存起了巨大作用，因为你处理的是这么多输入 token。如果你在给定行输入的每一次击键都要重新对所有这些传入的 token 运行模型，那么第一，你会显著降低延迟，第二，你会让 GPU 负载过重。所以你需要设计模型实际使用的提示词，使其对缓存友好，然后，是的，你需要在请求之间重用 KV 缓存，以减少计算量。

Caching plays a huge role, because you're dealing with this many input tokens. If every single keystroke that you're typing in a given line you had to rerun the model on all those tokens passed in, you're going to one, significantly degrade latency, two, you're going to kill your GPUs with load. So you need to design the actual prompts used for the model such that they're caching-aware, and then yeah, you need to reuse the KV cache across requests just so that you're spending less compute.

再说一次，Tab 在短期内应该能够做哪些事情，只是为了……

Again, what are the things that Tab is supposed to be able to do kind of in the near term, just to sort of...

继续深入那个生成代码就像填充空白空间的想法。还要跨多行编辑代码，然后跳转到同一文件内的不同位置，再然后希望能跳转到不同文件。所以如果你在一个文件中做了编辑，也许你需要去另一个文件完成你的思路，它也应该能去第二个文件。然后完整的泛化就是下一个动作预测。有时你需要在终端运行一个命令，它应该能根据你写的代码来建议命令。或者有时你需要它建议一些东西，但你很难知道是否正确，因为你需要更多信息来学习。你需要知道类型才能验证它是否正确，所以也许它应该带你去某个定义的地方，然后再把你带回来，这样你就有了所有必要的知识来接受下一个补全。同时给人类提供知识，是的，没错。

Linger on that generate code like fill empty space. Also edit code across multiple lines, yeah, and then jump to different locations inside the same file, yeah, and then hopefully jump to different files also. So if you make an edit in one file and maybe you have to go to another file to finish your thought, it should go to the second file also. Yeah, and then the full generalization is like next action prediction. Sometimes you need to run a command in the terminal, and it should be able to suggest the command based on the code that you wrote too. Or sometimes you actually need to suggest something, but it's hard for you to know if it's correct because you need some more information to learn. You need to know the type to be able to verify that it's correct, so maybe it should actually take you to a place that's like the definition of something and then take you back so that you have all the requisite knowledge to be able to accept the next completion. Also providing the human the knowledge, yes, right, yeah.

你能整合一下吗？比如我刚认识一个叫 Prime Jen 的人，他好像有个 SSH 可以点咖啡。哦对，我们做过那个。所以模型也能做那个吗，比如喂你东西，给你提供咖啡因？好的，所以这就是整体框架。神奇的时刻在于，编程是一个奇怪的学科，有时接下来五分钟你要做的事情实际上可以从你最近做的事情预测出来。所以你能达到这样一个世界吗？那五分钟要么你放手让它带你完成，要么你只是看到它下一步要做什么，然后你说好的好的好的，然后你就能轻松地点击通过这些大改动。

Can you integrate like I just got to know a guy named Prime Jen who I believe has an SSH you can order coffee via SSH. Oh yeah, we did that. So can that also the model do that like feed you and provide you with caffeine? Okay, so that's the general framework. And the magic moment would be if programming is this weird discipline where sometimes the next five minutes of what you're going to do is actually predictable from the stuff you've done recently. So can you get to a world where that next five minutes either happens by you disengaging and it taking you through, or maybe a little bit more of just you seeing the next step what it's going to do and you're like okay that's good, that's good, that's good, and you can just sort of tap tap tap through these big changes.

我们聊这个的时候，我应该提一下 Cursor 一个非常酷且引人注目的地方，就是整个 diff 界面。模型用红色和绿色来建议如何修改代码，在聊天窗口中你可以应用，它会显示 diff，你可以接受 diff。所以你能谈谈那个方向吗？我们可能会有四五种不同的 diff。我们已经为自动补全优化了 diff，所以它有一个不同的 diff 界面，不同于审查大段代码的时候。然后我们正在尝试为多文件操作优化另一种 diff。从高层次看，区别在于自动补全时，它应该非常非常快就能阅读。实际上在所有情况下都应该很快阅读，但在自动补全时，你的眼睛聚焦在一个区域，你不能看太多地方，人类不能同时看太多不同的地方。

As we're talking about this, I should mention one of the really cool and noticeable things about Cursor is that there's this whole diff interface situation going on. So the model suggests with the red and the green of here's how we're going to modify the code, and in the chat window you can apply and it shows you the diff and you can accept the diff. So maybe can you speak to whatever direction of that? We'll probably have four or five different kinds of diffs. So we have optimized the diff for the autocomplete, so that has a different diff interface than when you're reviewing larger blocks of code. And then we're trying to optimize another diff thing for when you're doing multiple different files. At a high level, the difference is for when you're doing autocomplete, it should be really really fast to read. Actually it should be really fast to read in all situations, but in autocomplete you're really like your eyes focused in one area, you can't be in too many places, the humans can't look in too many different places.

所以你在说界面方面。它目前有一个侧边框。我们有当前的框，如果它试图删除某处的代码并添加其他代码，它会尝试显示一个框。如果我们打开 cursor.com 也许能展示出来。这就是我们在说的。所以大概有三四次不同的尝试来让这个东西工作。第一次尝试是这些蓝色的删除线。所以在侧边框之前，它用类似 Google 文档的风格显示要删除的代码，你会看到一条线划掉它，然后你会看到新代码。那非常分散注意力。然后我们尝试了很多不同的东西。有删除，有尝试红色高亮。然后下一个迭代，有点搞笑，你在 Mac 上按住 option 键，它会高亮一块代码区域，告诉你可能有东西要来。所以在这个例子中，input 和 value 会变成蓝色，蓝色表示 AI 有一个建议给你。所以不是直接显示内容，它只是提示 AI 有建议。如果你真的想看，你就按住 option 键，然后你会看到新建议。如果你松开 option 键，你就会看到你原来的代码。

So you're talking about on the interface side. It currently has this box on the side. So we have the current box, and if it tries to delete code in some place and tries to add other code, it tries to show you a box. You can maybe show it if we pull it up on cursor.com. This is what we're talking about. So it was like three or four different attempts at trying to make this thing work. First the attempt was like these blue crossed out lines. So before it was a box on the side, it used to show you the code to delete by showing you like Google Doc style, you would see a line through it, then you would see the new code. That was super distracting. And then we tried many different things. There was sort of deletions, there was trying to red highlight. Then the next iteration of it, which is sort of funny, you would hold on Mac the option button, so it would sort of highlight a region of code to show you that there might be something coming. So maybe in this example, the input and the value would get blue, and the blue would highlight that the AI had a suggestion for you. So instead of directly showing you the thing, it would just hint that the AI had a suggestion. And if you really wanted to see it, you would hold the option button and then you would see the new suggestion. Then if you release the option button, you would then see your original code.

顺便说一句，那很不错，但你必须知道要按住 option 键。顺便说一句，我不是 Mac 用户，但我明白了。那是一个按钮，我猜。你们这些人，这又是，它就是不直观。我觉得这是关键，而且这很可能也不是最终版本。我个人很期待在这方面做很多改进。我们经常把它称为验证问题，这些 diff 对于小编辑很好，但对于大编辑或者多文件之类的情况，审查这些 diff 实际上有点困难。所以有几个不同的想法。我们有一个想法是，diff 的部分是重要的，包含很多信息，而部分 diff 熵很低，就像重复同样的东西。所以也许你可以高亮重要的部分，然后把不重要的部分变灰。或者你可以让一个模型查看 diff，发现这里可能有个 bug，就用红色波浪线标记，并说你应该审查这部分 diff。这类想法很令人兴奋。

That's by the way pretty nice, but you have to know to hold the option button. Yeah, by the way I'm not a Mac user, but I got it. It was a button I guess. You people, it's again, it's just nonintuitive. I think that's the key thing, and there's a chance this is also not the final version of it. I am personally very excited for making a lot of improvements in this area. We often talk about it as the verification problem, where these diffs are great for small edits, for large edits or when it's multiple files or something, it's actually a little bit prohibitive to review these diffs. So there are a couple of different ideas here. One idea that we have is okay, parts of the diffs are important, they have a lot of information, and parts of the diff are just very low entropy, they're like the same thing over and over again. So maybe you can highlight the important pieces and then gray out the not so important pieces. Or maybe you can have a model that looks at the diff and sees oh there's a likely bug here, I will mark this with a little red squiggly and say you should probably review this part of the diff. Ideas in that vein are exciting.

是的，这真的是一个非常迷人的 UX 设计工程领域。所以你基本上是在试图引导人类程序员阅读他们需要阅读的所有东西，仅此而已。是的，理想情况下是这样。而且你希望一个智能模型来做这件事。比如 diff，AI diff 算法，它们就像普通算法，没有智能。设计算法时是有智能的，但之后你并不关心它是关于这个还是那个。所以你希望一个模型来做这个。我认为一般的问题是这些模型会变得更聪明。随着模型变得更聪明，它们能提出的变化也会更大。所以随着变化越来越大，人类需要做越来越多的验证工作，变得越来越困难。你需要帮助他们。我不想把所有时间都花在审查代码上。

Yeah, that's a really fascinating space of UX design engineering. So you're basically trying to guide the human programmer through all the things they need to read and nothing more. Yeah, like optimally. And you want an intelligent model to do it. Like diffs, AI diff algorithms, they're like normal algorithms, there's no intelligence. There's intelligence that went into designing the algorithm, but then you don't care if it's about this thing or that thing. So you want a model to do this. I think the general question is these models are going to get much smarter. As the models get much smarter, the changes they will be able to propose are much bigger. So as the changes get bigger and bigger, the humans have to do more and more verification work, it gets more and more hard. You need to help them out. I don't want to spend all my time reviewing code.

你能再多说一点关于多文件 diff 吗？

Can you say a little more across multiple files diff?

GitHub 试图通过代码审查来解决这个问题，但正如 Arvid 所说，你可以做得更好。代码审查有点糟糕——你花大量时间试图理解不熟悉的代码，而且往往抓不到多少 bug。你可以利用语言模型显著改善这种体验，比如将你引导到关键区域。另外，如果代码是由语言模型生成的，你就不必关心作者的体验；你可以完全围绕审查者来设计，让他们的工作尽可能有趣和高效。天真地让这些东西看起来像代码审查的问题在于，你可以更有创意并突破边界。一个想法：顺序很重要。审查 PR 时，你有一个文件列表，从上到下审查，但实际上你希望先理解某个部分，因为它逻辑上在先，然后再看下一部分。你不想自己琢磨这些，而是希望模型引导你。

GitHub tries to solve this with code review, but as Arvid said, you can do much better. Code review kind of sucks—you spend a lot of time trying to grok unfamiliar code, and it often doesn't catch many bugs. You can significantly improve that experience using language models, for example, by pointing you towards the regions that matter. Also, if the code is produced by a language model, you don't have to care about the author's experience; you can design the entire thing around the reviewer to make their job as fun and productive as possible. The issue with naively making these things look like code review is that you can be more creative and push boundaries. One idea: ordering matters. When reviewing a PR, you have a list of files and review them top to bottom, but you actually want to understand one part first because it's logically first, then the next. You don't want to figure that out yourself; you want a model to guide you.

创作过程会越来越依赖自然语言吗？这是目标吗？

Is the step of creation going to be more and more natural language? Is that the goal?

我不认为所有编程都会变成自然语言。例如，如果我和某人结对编程，他们在键盘前，有时我想说“实现这个函数”就行，但有时解释起来很烦，我就直接接管键盘写一部分示例——这是最简单的沟通方式。类似地，对于 AI，有时最简单的方式是展示一个例子，或者拖拽东西来设计网站。也许最终我们会拥有脑机接口。自然语言会有一席之地，但不会成为大多数人大多数时候的编程方式。

I don't think all programming will be natural language. For example, if I'm pair programming with someone and they're at the keyboard, sometimes I want to say 'implement this function' and that works, but other times it's annoying to explain what I want, so I take over the keyboard and write part of an example—that's the easiest way to communicate. Similarly, with AI, sometimes the easiest way is to show an example, or drag things around for a website. Maybe eventually we'll get to brain-machine interfaces. Natural language will have a place, but it won't be how most people program most of the time.

用这个编辑器我真的感受到了 AGI。底层有很多机器学习。说说让这一切运转起来的 ML 技术吧。

I'm really feeling the AGI with this editor. There's a lot of machine learning going on underneath. Tell me about some of the ML stuff that makes it all work.

Cursor 实际上是通过一组我们训练的自定义模型与前沿模型协同工作的。例如，Cursor Tab 是一个专门的模型，在我们的评估中甚至比前沿模型更好。另一个需要自定义模型的领域是“应用”——前沿模型擅长草拟代码计划，但实际生成 diff 很困难。它们会在数行号等简单事情上出错，尤其是在大文件中。所以我们让模型草拟一个粗略的代码块来指示更改，然后训练一个模型将该更改应用到文件中。“应用”不是一个确定性算法；其他地方浅拷贝的实现大多数时候会失败，因为确定性匹配至少 40% 的时间会出错，导致糟糕的产品体验。

Cursor really works via an ensemble of custom models that we've trained alongside frontier models. For example, Cursor Tab is a specialized model that can be even better than frontier models on our evals. Another domain that requires custom models is 'apply'—frontier models are good at sketching plans for code, but actually creating diffs is hard. They mess up things like counting line numbers, especially in large files. So we let the model sketch a rough code block indicating the change, and train a model to apply that change to the file. Apply is not a deterministic algorithm; shallow copies elsewhere break most of the time because deterministic matching fails at least 40% of the time, resulting in a terrible product experience.

“应用”还能让你用最智能的模型消耗更少的 token，这在延迟和成本上都很昂贵。你可以给出非常粗略的草图，让较小的模型去实现，因为实现草拟好的代码更容易。这种模式会持续：更智能的模型做规划，不那么智能的模型处理实现细节。为了速度，我们使用推测性编辑，这是推测性解码的一种变体。当语言模型生成受限于内存时，一次处理多个 token 比逐个生成更快。通常，推测性解码使用一个小模型预测草稿 token，然后由大模型验证。对于代码编辑，我们有一个很强的先验——现有代码本身。所以我们把原始代码的片段反馈给模型。

Apply also lets you use fewer tokens with the most intelligent models, which is expensive in latency and cost. You can give a very rough sketch and have smaller models implement it, because implementing a sketched-out code is easier. This regime will continue: smarter models do planning, less intelligent ones handle implementation details. For speed, we use speculative edits, a variant of speculative decoding. When memory-bound in language model generation, processing multiple tokens at once is faster than one at a time. Normally, speculative decoding uses a small model to predict draft tokens that a larger model verifies. With code edits, we have a strong prior—the existing code itself. So we feed chunks of the original code back into the model.

然后模型大部分时候都会同意，好吧，我就直接把这些代码原样输出，这样你就可以并行处理所有这些行，只要用足够多的块就行。最终你会遇到一个分歧点，模型会预测出与原始代码不同的文本，它会生成那些 token，然后我们会在足够多的 token 匹配原始代码后决定重新开始推测代码块。这实际上看起来就像是正常编辑代码的一个更快版本，就像是模型重写所有代码的更快版本。所以我们可以使用与 diff 完全相同的界面，但它会更快地流式输出。优势在于，通过无线流式传输，你可以在代码完成之前就开始审查，没有大的加载屏幕。所以这可能就是优势的一部分：人类可以在事情完成之前就开始阅读。

Then the model will just pretty much agree most of the time that okay I'm just going to spit this code back out and so you can process all of those lines in parallel and you just do this with sufficiently many chunks and then eventually you'll reach a point of disagreement where the model will now predict text that is different from the ground truth original code it'll generate those tokens and then we kind of will decide after enough tokens match the original code to restart speculating in chunks of code. What this actually ends up looking like is just a much faster version of normal editing code, so it's just like it looks like a much faster version of the model rewriting all the code. So we can use the same exact interface that we use for diffs but it will just stream down a lot faster. And then the advantage is that with wireless streaming you can also be reviewing, start reviewing the code exactly before it's done, so there's no big loading screen. So maybe that is part of the advantage: the human can start reading before the thing is done.

我觉得这里有趣的一点是，推测（speculation）现在是一个相当常见的概念。不仅限于语言模型，CPU 里有推测，数据库里也有推测，到处都是推测。让我问一个有点荒谬的问题：哪个 LLM 更擅长编程，GPT 还是 Claude？在编程方面谁赢了？我肯定答案要微妙得多，因为听起来这里的每个部分都涉及不同的模型。

I think the interesting riff here is something like speculation is a fairly common idea nowadays. It's not only in language models, I mean there's obviously speculation in CPUs and there's speculation for databases and like speculation all over the place. Let me ask the sort of the ridiculous question of which LLM is better at coding: GPT or Claude? Who wins in the context of programming? And I'm sure the answer is much more nuance because it sounds like every single part of this involves a different model.

是的，我认为没有哪个模型能完全主导其他模型，也就是说，在所有我们认为重要的类别上都更好。这些类别包括速度、编辑代码的能力、处理大量代码的能力、长上下文，以及其他一些东西，还有编程能力。我现在觉得总体上最好的是 Sonnet。我认为这是共识。O1 非常有趣，它非常擅长推理，所以如果你给它非常难的编程面试题或 LeetCode 题，它能做得很好。但它似乎不像 Sonnet 那样理解你的大致意图。如果你看很多其他前沿模型，我有一个疑虑是，它们感觉不一定——我不是说它们在基准上训练——但它们在基准测试中表现非常好，相对于所有中间的东西。所以如果你尝试所有这些基准测试以及它们被评估的分布内的东西，它们会做得很好。但当你把它们稍微推离那个范围时，Sonnet 是那种能最好地保持相同能力的模型。就像你在基准测试中的能力与当你试图指示它做任何编程相关事情时的能力是一样的。

Yeah, I think there's no model that dominates others, meaning it is better in all categories that we think matter. The categories being speed, ability to edit code, ability to process lots of code, long context, you know, a couple of other things, and kind of coding capabilities. The one that I'd say right now is just kind of net best is Sonnet. I think this is a consensus opinion. O1 is really interesting and it's really good at reasoning, so if you give it really hard programming interview style problems or LeetCode problems it can do quite well on them. But it doesn't feel like it kind of understands your rough intent as well as Sonnet does. If you look at a lot of the other frontier models, one qualm I have is it feels like they're not necessarily over—I'm not saying they train on benchmarks—but they perform really well in benchmarks relative to kind of everything that's kind of in the middle. So if you try on all these benchmarks and things that are in the distribution of the benchmarks they're evaluated on, you know they'll do really well. But when you push them a little bit outside of that, Sonnet is the one that kind of does best at maintaining that same capability. Like you kind of have the same capability in the benchmark as when you try to instruct it to do anything with coding.

另一个荒谬的问题是正常编程体验与基准测试所代表的内容之间的差异。你认为在评估这些模型时，基准测试在哪些方面存在不足？顺便说一句，这真的是一个非常非常难的问题，这是一个至关重要的细节，比如基准测试与真实编程之间的差异有多大。真实编程不是面试式的编程。你是在做这些，你知道，人类有时会说半生不熟的英语，有时你会说‘哦，做我之前做的’，有时你会说‘去添加这个东西，然后为我做那个，然后制作这个 UI 元素’，然后你知道，很多事情都是上下文相关的。你真的需要理解人类，然后做人类想要的，而不是——也许抽象地说，面试问题定义得非常明确，它们非常依赖规范，而人类的东西则不那么明确。

Another ridiculous question is the difference between the normal programming experience versus what benchmarks represent. Like where do benchmarks fall short do you think when we're evaluating these models? By the way, that's like a really really hard, it's like critically important detail, like how different benchmarks are versus where is like real coding. Real coding it's not interview style coding. It's you're doing these, you know, humans are saying like half broken English sometimes and sometimes you're saying like 'oh do what I did before', sometimes you're saying 'go add this thing and then do this other thing for me and then make this UI element and then you know it's just like a lot of things are sort of context dependent. You really want to like understand the human and then do what the human wants, as opposed to sort of this maybe the way to put it is sort of abstractly is the interview problems are very well specified, they lean a lot on specification, while the human stuff is less specified.

是的，我认为这个具体问题既因为 Sol 刚才提到的而复杂，也因为 Aman 之前提到的。即使你喜欢，你知道，存在一个偏差问题，即你实际上能在基准测试中建模的内容与真实编程之间的差异，这有时很难概括，因为真实编程非常混乱，有时事情并不非常明确什么是对的什么是错的。但这也因为公共基准测试问题而加倍困难。这既是因为公共基准测试有时会被爬坡优化，而且也很难从模型中去除公共基准测试的数据。例如，最流行的智能体基准测试之一 SWE-bench，在这些基础模型的训练数据中受到了非常严重的污染。所以如果你让这些基础模型做一个 SWE-bench 问题，你实际上没有给它们代码库的上下文，它们可以幻觉出正确的文件路径，它们可以幻觉出正确的函数名。所以这些事情的公共方面也很棘手。

Yeah, I think that this specific question is both complicated by what Sol just mentioned and then also to what Aman was getting into. Even if you like, you know, there's this problem of the skew between what can you actually model in a benchmark versus real programming, and that can be sometimes hard to encapsulate because it's like real programming is very messy and sometimes things aren't super well specified what's correct or what isn't. But then it's also doubly hard because of this public benchmark problem. And that's both because public benchmarks are sometimes kind of hill climbed on, then it's like really really hard to also get the data from the public benchmarks out of the models. And so for instance, one of the most popular agent benchmarks, SWE-bench, is really really contaminated in the training data of these foundation models. And so if you ask these foundation models to do a SWE-bench problem, you actually don't give them the context of a codebase, they can like hallucinate the right file path, they can hallucinate the right function names. So it's also just the public aspect of these things is tricky.

是的，比如在这种情况下，它可能是在实际的问题或拉取请求本身上训练的，也许生活会开始做得更好。或者他们已经很好地去除了这些污染，但他们不会输出仓库本身的实际训练数据。比如这些都是最流行的 Python 仓库，SimPy 就是一个例子。我不认为他们会为了在这些基准测试中获得真实的评估分数而削弱他们的模型在 SimPy 和所有这些流行 Python 仓库上的表现。

Yeah, like in that case it could be trained on the literal issues or pull requests themselves and maybe the lives will start to do a better job. Or they've already done a good job at decontaminating those things but they're not going to emit the actual training data of the repository itself. Like these are all like some of the most popular Python repositories, like SimPy is one example. I don't think they're going to handicap their models on SimPy and all these popular Python repositories in order to get true evaluation scores in these benchmarks.

是的，我认为鉴于基准测试的问题，有一些有趣的辅助手段，那些用这些模型构建系统或构建这些模型的地方实际上会用来判断他们是否在正确的方向上。在很多地方，人们实际上会让人类去玩这些东西，并给出定性反馈。比如一两家基础模型公司，他们有专门的人，这是他们角色的重要部分。而且你知道，在内部，我们也会定性地评估这些模型，并且实际上非常依赖这一点，除了我们拥有的私人评估之外。这就像是现场，氛围。

Yeah, I think that given the issues with benchmarks, there have been a few interesting crutches that places that build systems with these models or build these models actually use to get a sense of are they going in the right direction or not. And in a lot of places, people will actually just have humans play with the things and give qualitative feedback on these. Like one or two of the foundation model companies, they have people who that's a big part of their role. And you know internally we also qualitatively assess these models and actually lean on that a lot in addition to like private evals that we have. It's like the live, the vibe.

氛围基准测试，人类基准测试。你拉人来做个氛围检查。是的，好吧。我的意思是，这基本上就是我在做的，比如阅读在线论坛、Reddit 和 X。就像，我不知道如何正确加载人们的意见，因为他们会说‘我觉得 Claude 或 GPT 变笨了’之类的话。他们会说‘我觉得’，然后我有时也这么觉得，但我想知道这是模型的问题还是我的问题。

The vibe benchmark, human benchmark. You pull in the humans to do a vibe check. Yeah, okay. I mean that's kind of what I do, like just reading online forums and Reddit and X. Just like, well I don't know how to properly load in people's opinions because they'll say things like 'I feel like Claude or GPT's gotten dumber' or something. They'll say 'I feel like' and then I sometimes feel like that too, but I wonder if it's the model's problem or mine.

是的，关于 Claude 有一个有趣的观点。

Yeah, with Claude there's an interesting take.

我听说 AWS 有不同芯片，我怀疑它们的数值精度与 Nvidia GPU 略有不同。有人猜测 Claude 性能下降可能是因为使用了 AWS Bedrock 上的量化版本，而不是 Anthropic GPU 上运行的版本。我采访过很多有阴谋论的人，所以很高兴你谈到了这个阴谋论。

I heard where I think AWS has different chips and I suspect they've slightly different numerics than Nvidia GPUs. Someone speculated that Claude's degraded performance had to do with maybe using the quantized version that existed on AWS Bedrock versus whatever was running on Anthropic's GPUs. I interview a bunch of people that have conspiracy theories, so I'm glad you spoke to this conspiracy.

嗯，与其说是阴谋论，不如说只是细节问题。人类就是人类，这些细节是存在的。你要进行海量计算，芯片也很复杂。你可能会遇到 bug。很难夸大避免 bug 的难度。

Well, it's not a conspiracy theory as much as it's just details. Humans are humans, and there are these details. You're doing these crazy amounts of flops, and chips are messy. You can just have bugs. It's hard to overstate how hard bugs are to avoid.

好的提示词在其中扮演什么角色？你提到基准测试有结构良好、精心设计的提示词。人类应该怎么做才能最大化成功，人类行为的重要性是什么？你写了一篇关于提示词设计的博客文章。

What's the role of a good prompt in all this? You mention that benchmarks have really structured, well-formulated prompts. What should a human be doing to maximize success, and what's the importance of what the human does? You wrote a blog post on prompt design.

我认为这取决于你使用的模型，所有模型可能都不同，对不同的提示词反应也不同。但我认为最初的 GPT-4 和去年那批模型对提示词相当敏感。它们的上下文窗口也很小。所以代码库周围有所有这些可能相关的信息，比如文档、你添加的文件、对话历史。然后有一个问题：在空间有限的情况下，你如何决定实际放入提示词的内容？即使是今天具有长上下文的模型，填满整个上下文窗口意味着速度变慢，有时模型会感到困惑。有些模型比其他模型更容易困惑。我们内部有一个系统叫 Preum，在这方面帮了一点忙。它是在我们拥有 8000 token 上下文窗口之前的时代构建的。这有点像制作网站：你想让它在移动端和桌面端都能工作，并且你有动态信息。当你有一个提示词时，你有输入，需要格式化它们使其始终有效。如果输入非常大，你可能需要裁剪一些内容。这个想法是从设计网站的最佳方式中汲取灵感。我们非常喜欢 React 和声明式方法，你在 JavaScript 中使用 JSX，声明你想要什么，并赋予优先级。然后你有一个渲染引擎——在网页设计中是 Chrome，在我们的例子中是预渲染器——它将所有内容适配到页面上。你声明式地决定你想要什么，它就会解决。我们发现这非常有帮助。随着时间的推移，它的角色发生了变化：最初是为了适应小的上下文窗口，现在它有助于拆分进入提示词的数据和实际渲染。调试更容易，因为你可以更改提示词的渲染，并在旧提示词上尝试，因为你拥有原始数据。你可以看到你的更改是否改善了整个评估集。

I think it depends on which model you're using, and all of them are likely different and respond differently to different prompts. But I think the original GPT-4 and the original breed of models last year were quite sensitive to the prompts. They also had a very small context window. So we have all these pieces of information around the codebase that might be relevant in the prompt, like the docs, the files you add, the conversation history. Then there's a problem: how do you decide what to actually put in the prompt when you have limited space? Even for today's models with long context, filling out the entire context window means it's slower and sometimes the model gets confused. Some models get more confused than others. We have this one system internally that we call Preum, which helps us with that a little bit. It was built for the era before we had 8,000 token context windows. It's a bit similar to making a website: you want it to work on mobile and desktop, and you have dynamic information. When you have a prompt, you have inputs and need to format them so it always works. If the input is really big, you might have to cut something down. The idea was to take inspiration from the best way to design websites. We really like React and the declarative approach where you use JSX in JavaScript, declare what you want, and give priorities. Then you have a rendering engine—in web design it's Chrome, in our case it's a pre-renderer—which fits everything onto the page. You declaratively decide what you want, and it figures it out. We found that quite helpful. The role has shifted over time: initially it was to fit into small context windows, now it's useful for splitting up the data that goes into the prompt and the actual rendering. It's easier to debug because you can change the rendering of the prompt and try it on old prompts, since you have the raw data. You can see if your change improved it for the entire eval set.

所以你真的是用 JSX 来写提示词吗？

So do you literally prompt with JSX?

是的，是的。所以它看起来有点像 React。有组件，比如我们有一个文件组件，它接收光标。通常你的文件中有一行是光标所在的位置，那可能是最重要的一行，因为那是你正在看的那一行。所以你可以赋予优先级：那行有最高优先级，每远离一行就减一。然后当它被渲染时，它会计算出实际能容纳多少行，并以那行为中心。这太棒了。你还可以做其他花哨的事情：如果你有来自整个代码库的大量代码块，你可以使用检索、嵌入和重排序分数来为每个组件添加优先级。

Yes, yes. So it kind of looks like React. There are components, like we have one component that's a file component, and it takes in the cursor. Usually there's one line where the cursor is in your file, and that's probably the most important line because that's the one you're looking at. So you can give priorities: that line has the highest priority, and you subtract one for every line that is farther away. Then when it's rendered, it figures out how many lines it can actually fit, and it centers around that thing. That's amazing. And you can do other fancy things: if you have lots of code blocks from the entire codebase, you could use retrieval, embeddings, and reranking scores to add priorities for each of these components.

人类在提问时也应该尝试使用类似的东西吗？在提示词中写 JSX 会有好处吗？整个想法应该是松散和混乱的。

Should humans, when they ask questions, also try to use something like that? Would it be beneficial to write JSX in the prompt? The whole idea is it should be loose and messy.

我认为我们的目标是，你应该做对你来说最自然的事情，然后我们的工作是弄清楚如何检索相关的东西，使你的东西真正有意义。

I think our goal is that you should just do whatever is the most natural thing for you, and then our job is to figure out how to retrieve the relevant things so that your thing actually makes sense.

这有点像我和 Perplexity 的 Aravind 讨论过的问题。他的整个想法是，你应该让人尽可能懒惰。这是一件好事，但我觉得你可以对程序员要求更多，对吧？如果你说“做你想做的”，人类是懒惰的。在懒惰和提供更多之间存在一种张力。被提示是否就像系统在迫使或激励你表达清晰，不是语法上的，而是你在问题中传达的思想深度？

This is sort of the discussion I had with Aravind of Perplexity. His whole idea is you should let the person be as lazy as they want. That's a beautiful thing, but I feel like you're allowed to ask more of programmers, right? If you say just do what you want, humans are lazy. There's a kind of tension between just being lazy versus providing more. Is being prompted almost like the system pressuring you or inspiring you to be articulate, not in terms of grammar but in terms of the depth of thoughts you convey inside the problems?

我认为即使系统接近某种完美水平，当你向模型提问时，你往往没有传达足够的意图让它知道该做什么。有几种方法可以解决这个意图问题。一种是简单的让模型直接问你：“根据你的查询，我不确定如何做这些部分，你能澄清一下吗？”另一种可能是，如果根据你查询中的不确定性有五六种可能的生成结果，为什么不直接全部展示给你，让你选择呢？

I think even as a system gets closer to some level of perfection, often when you ask the model for something, you just don't convey enough intent for it to know what to do. There are a few ways to resolve that intent. One is the simple thing of having the model just ask you: 'I'm not sure how to do these parts based on your query, could you clarify that?' I think the other could be maybe if there are five or six possible generations given the uncertainty in your query, why don't we just show you all of those and let you pick?

模型选择回问而不是生成有多难？这很难，有点像如何处理不确定性：我是否选择询问更多信息来减少歧义？

How hard is it for the model to choose to speak back versus generate? That's hard, sort of like how to deal with the uncertainty: do I choose to ask for more information to reduce the ambiguity?

我们做的一件事是最近新增的：尝试建议你可以添加的文件。当你输入时，可以猜测不确定性是什么，并可能提出建议。例如，也许你在写 API，我们可以利用你之前在同一文件中的提交来猜测客户端和服务器非常有用。有一个困难的技术问题是如何在所有提交中解决：在当前上下文中哪些文件最重要？

One of the things we do is a recent addition: try to suggest files that you can add. While you're typing, one can guess what the uncertainty is and maybe suggest that. For example, maybe you're writing your API, and we can guess using the commits you've made previously in the same file that the client and the server are super useful. There's a hard technical problem of how to resolve it across all commits: which files are the most important given the current context?

你在多大程度上使用智能体式方法？智能体有多有用？

To what degree do you use agentic approaches? How useful are agents?

我们认为智能体非常酷。它们像人类一样；你会感觉离 AGI 更近了，因为你看到一个演示，它像人类一样行动。但智能体在很多事情上还不是特别有用。我们正在接近它们真正有用的阶段。有些特定类型的任务有智能体会很好。例如，我们有一个 bug，有时你无法在聊天输入框内使用 Command C 和 Command V。这是一个非常明确的任务：我只想说两句话，‘这个不行，请修复它’，然后我希望有一个智能体去找到正确的文件，尝试重现 bug，修复它，并验证正确性。这可能是一个需要很长时间的过程。我认为很多编程工作——人们常认为智能体会接管所有编程。我不这么认为，因为很多价值在于迭代。你其实不想一开始就指定所有东西，因为你直到看到初始版本才知道你想要什么，然后你想在此基础上迭代。所以对于很多编程工作，你实际上想要一个即时系统，立即给你一个初始版本，然后你可以非常快速地迭代。

We think agents are really cool. They resemble a human; you can feel like you're getting closer to AGI because you see a demo where it acts as a human would. But agents are not yet super useful for many things. We're getting close to where they will actually be useful. There are certain types of tasks where having an agent would be nice. For example, we have a bug where you sometimes can't Command C and Command V inside our chat input box. That's a task that's super well specified: I just want to say in two sentences, 'This does not work, please fix it,' and then I would love to have an agent that goes off, finds the right files, tries to reproduce the bug, fixes it, and verifies it's correct. This could be a process that takes a long time. I think a lot of programming — there is often this belief that agents will take over all of programming. I don't think that's the case because a lot of the value is in iterating. You don't actually want to specify something upfront because you don't really know what you want until you've seen an initial version, and then you want to iterate on that. So for a lot of programming, you actually want a system that's instant, gives you an initial version instantly, and then you can iterate super quickly.

那最近出现的 Rep Agent 呢？它做设置开发环境、安装软件包、配置一切、配置数据库，并实际部署应用。这也是你梦想中的事情吗？

What about something like recently came out, Rep Agent, that does setting up the development environment, installing software packages, configuring everything, configuring the databases, and actually deploying the app? Is that also in the set of things you dream about?

我想是的。对于某些类型的编程来说，那会非常酷。这在 Cursor 的范围内吗？我们现在没有积极开发，但确实我们希望让程序员的生活更轻松、更有趣。有些事情非常繁琐，需要经历一系列步骤，你想把这些委托给智能体。还有一些事情，你可以在工作时让一个智能体在后台运行。例如，假设你有一个同时涉及后端和前端的 PR，你正在做前端部分。然后你可以让一个后台智能体不工作，而是弄清楚你在做什么，然后当你处理 PR 的后端部分时，你有一些初始代码可以迭代。那也会非常酷。

I think so. That would be really cool for certain types of programming. Is that within scope of Cursor? We aren't actively working on it right now, but it's definitely like we want to make the programmer's life easier and more fun. Some things are just really tedious and you need to go through a bunch of steps, and you want to delegate that to an agent. And then some things you can actually have an agent in the background while you're working. For example, let's say you have a PR that's both backend and frontend, and you're working on the frontend. Then you can have a background agent that doesn't work and figure out kind of what you're doing, and then when you get to the backend part of your PR, you have some initial piece of code that you can iterate on. That would also be really cool.

我们之前谈过的一件事是速度。我想知道我们是否可以再多谈一些，涉及使这个东西非常快的各种技术细节。Cursor 的每个方面都感觉非常快。apply 可能是最慢的。这是一个我们正在感受的痛苦，我们正在努力修复它。这说明一件事，感觉像 1 或 2 秒的东西感觉慢——这实际上表明其他一切都非常快。那么有没有一些技术细节关于如何让聊天快，如何让 diff 快？

One of the things we already talked about is speed. I wonder if we can linger on that some more, the various technical details involved in making this thing really fast. Every single aspect of Cursor feels really fast. The apply is probably the slowest thing. It's a pain that we're feeling and we're working on fixing it. It says something that something that feels like 1 or 2 seconds feels slow — that actually shows that everything else is just really fast. So is there some technical details about how to make the chat fast, how to make the diffs fast?

是的，我们可以介绍我们使用的很多策略。一个有趣的事情是缓存预热。当用户正在输入时，你可以知道他们可能会使用某些上下文，所以你可以在用户输入完成之前就知道。正如我们之前讨论的，重用 KV 缓存可以降低延迟和跨请求的成本。所以当用户开始输入时，你可以立即用当前文件内容预热缓存。然后当他们按下回车时，实际需要预填充和计算的 token 很少，然后才开始生成。这将显著降低首 token 时间。

Yeah, we can go over a lot of the strategies we use. One interesting thing is cache warming. As the user is typing, you can know that they're probably going to use some piece of context, so you can know that before the user's done typing. As we discussed before, reusing the KV cache results in lower latency and lower cost across requests. So as a user starts typing, you can immediately warm the cache with, let's say, the current file contents. Then when they've pressed enter, there are very few tokens it actually has to prefill and compute before starting the generation. This will significantly lower time to first token.

你能解释一下 KV 缓存是如何工作的吗？

Can you explain how KV cache works?

Transformer 的工作方式中，允许 Transformer 不仅独立地查看每个 token，还能看到之前 token 的机制之一是注意力中的键和值。通常，注意力的工作方式是，在当前 token 处你有一个查询，然后你有所有之前 token 的键和值，这些是模型内部存储的提示中所有之前 token 的某种表示。默认情况下，当你在进行聊天时，模型必须为每个 token 执行整个模型的前向传播——这涉及大量矩阵乘法，非常慢。相反，如果你已经做了这些，并存储了键和值，并保存在 GPU 中，那么假设我已经为最后 N 个 token 存储了它们，如果我现在想计算第 N+1 个 token 的输出 token，我不需要将前 N 个 token 通过整个模型，因为我已经有了所有这些键和值。所以你只需要对最后一个 token 进行前向传播，然后当你在做注意力时，你重用了这些已经计算过的键和值，这是 Transformer 中唯一的顺序部分或顺序依赖部分。

The way Transformers work, one of the mechanisms that allow Transformers to not just independently look at each token but see previous tokens are the keys and values in attention. Generally, the way attention works is you have at your current token some query, and then you have all the keys and values of all your previous tokens, which are some kind of representation that the model stores internally of all the previous tokens in the prompt. By default, when you're doing a chat, the model has to for every single token do this forward pass through the entire model — that's a lot of matrix multiplies that happen, and that is really slow. Instead, if you have already done that and you stored the keys and values and you keep that in the GPU, then when I'm, let's say, I have stored it for the last N tokens, if I now want to compute the output token for the N+1 token, I don't need to pass those first N tokens through the entire model because I already have all those keys and values. So you just need to do the forward pass through that last token, and then when you're doing attention, you're reusing those keys and values that have been computed, which is the only kind of sequential part or sequentially dependent part of the Transformer.

有没有更高级的缓存，比如缓存提示之类的东西，可能有帮助？

Is there higher-level caching, like caching of the prompts or that kind of stuff, that could help?

是的，还有其他类型的缓存可以做。对于 Cursor Tab，一个有趣的事情是你可以基本上提前预测，就好像用户接受了建议，然后触发另一个请求，然后你就缓存了，你做了推测。

Yes, there are other types of caching you can do. One interesting thing that you can do for Cursor Tab is you can basically predict ahead as if the user would have accepted the suggestion and then trigger another request, and then you've cached, you've done the speculative.

这是推测和缓存的结合。你在推测如果用户接受了建议会发生什么，然后你缓存了这个值。这个建议，当用户按下 Tab 时，下一个建议会立即等待他们。这是一个巧妙的启发式方法或技巧，利用了更高级的缓存，让体验感觉很快，尽管模型本身没有任何变化。如果你能缩小 KV 缓存，一个优势是你可以进行更多推测。也许你可以预测出 10 个可能有用的东西。预测接下来的 10 个，然后用户有可能命中其中一个，这比命中你展示的精确一个的概率高得多。也许他们输入另一个字符，然后命中了缓存中的其他内容。所以有各种各样的技巧。这里的普遍现象对强化学习也非常有用。也许模型的一个样本不太好，但如果你预测 10 个不同的东西，结果发现其中一个正确的概率要高得多。这里有 pass@k 曲线。强化学习的一部分就是你可以利用这种 pass@k 现象来做出许多不同的预测。一种思考方式是：模型内部对哪个关键东西是正确的，或者人类想要哪个关键东西，存在一些不确定性。当我们对 Cursor Tab 模型进行强化学习时，我们做的一件事是预测模型生成的 100 个不同建议中，哪些更受人类欢迎，哪些是人类更喜欢的。也许模型可以预测得很远，或者只预测一点点，或者介于两者之间。然后你可以给人类更喜欢的东西奖励，惩罚不喜欢的，从而训练模型输出人类更喜欢的建议。这些强化学习循环非常有用，并且利用了 pass@k 曲线。

It's a mix of speculation and caching. You're speculating what would happen if they accepted it, and then you have this value that is cached. This suggestion, when they press Tab, the next one would be waiting for them immediately. It's a clever heuristic or trick that uses a higher-level caching and can make it feel fast, despite there not actually being any changes in the model. If you can make the KV cache smaller, one advantage is you can speculate even more. Maybe you can get seriously 10 things that could be useful. Predict the next 10, and then it's possible the user hits one of the 10, which is much higher chance than hitting the exact one you show them. Maybe they type another character and hit something else in the cache. So there are all these tricks. The general phenomenon here is also super useful for RL. Maybe a single sample from the model isn't very good, but if you predict 10 different things, it turns out that one of the 10 is right with much higher probability. There are these pass@k curves. Part of RL is you can exploit this pass@k phenomenon to make many different predictions. One way to think about this: the model internally has some uncertainty over which of the key things is correct, or which of the key things the human wants. When we do RL on our Cursor Tab model, one thing we're doing is predicting which of the hundred different suggestions the model produces is more amenable for humans, which of them humans like more. Maybe the model can predict very far ahead versus a little bit, and somewhere in the middle. Then you can give a reward to the things humans would like more and punish the things they wouldn't, and train the model to output the suggestions humans like more. You have these RL loops that are very useful and exploit these pass@k curves.

也许你可以更详细地讲讲。

Maybe you can go into even more detail.

这和速度有点不同，但技术上你可以联系起来，因为如果你的小模型能达到和大模型相同的性能，你就可以使用小模型。这就像我提到的缩小 KV 缓存大小。还有其他技术对速度非常有帮助。大约两年前，人们主要使用多头注意力。我认为已经向更高效的注意力机制迁移，比如分组查询或多查询注意力。这对于在更大批量下更快地生成 token 非常有帮助。有趣的是，这对首 token 预填充速度没有影响。它影响的是生成 token 的速度。为什么？因为生成 token 时，瓶颈不再是跨所有 token 进行超可实现的矩阵乘法，而是对于长上下文和大批量，你读取缓存键和值的速度。这是内存带宽。我们如何让它更快？我们可以尝试压缩这些键和值的大小。多查询注意力是最激进的。通常，多头注意力有一定数量的注意力头和查询头。多查询保留了查询头，但去掉了所有键值头，所以只有一个键值头和所有剩余的查询头。分组查询保留了所有查询头，键和值的头数较少，但不止一个。关键是缩小 KV 缓存的大小。然后是 MLA，即多潜在注意力，这更复杂一些。它将所有头的键和值整体转化为一个潜在向量，然后在推理时展开。MLA 来自一家叫 DeepSeek 的公司。这是一个非常有趣的算法。关键思想是，在 MQA 和其他地方，你减少了 KV 头的数量。好处是头数少了，但也许理论是你实际上需要很多不同的键和值。所以一种缩小尺寸的方法是保留一个大的共享向量用于所有键和值，然后为每个 token 保留较小的向量。这样你只存储较小的东西作为某种低秩压缩。最后，当你想计算最终结果时，记住你是内存受限的，这意味着你还有一些计算资源可用。所以如果你能把潜在向量展开回来，这会更高效，因为你减少了，比如从 32 减少到你保留的向量大小。也许拥有独立的键、值和查询对，它们成对匹配，与将它们全部压缩成一个相比，会有一些丰富性。这种交互至少是可以接受的。

It's a little different than speed, but technically you tie it back in because you can get away with a smaller model if your smaller model gets the same performance as the bigger one. That's like what I was mentioning about reducing the size of your KV cache. There are other techniques as well that are really helpful for speed. Back in the day, about two years ago, people mainly used multi-head attention. I think there's been a migration towards more efficient attention schemes like group query or multi-query attention. This is really helpful for generating tokens much faster with larger batch sizes. The interesting thing is this has no effect on the time to first token pre-fill speed. The thing this matters for is generating tokens. Why? Because when generating tokens, instead of being bottlenecked by doing the super-realizable matrix multiplies across all your tokens, you're bottlenecked by how quickly you can read those cache keys and values for long context with large batch sizes. That's memory bandwidth. How can we make this faster? We can try to compress the size of these keys and values. Multi-query attention is the most aggressive. Normally with multi-head attention, you have some number of attention heads and some number of query heads. Multi-query preserves the query heads but gets rid of all the key-value heads, so there's only one key-value head and all the remaining query heads. With group query, you preserve all the query heads, and your keys and values have fewer heads, but not just one. The whole point is reducing the size of your KV cache. Then there is MLA, multi-latent attention, which is a bit more complicated. It turns the entirety of your keys and values across all your heads into a kind of one latent vector that is then expanded at inference time. MLA is from a company called DeepSeek. It's quite an interesting algorithm. The key idea is that in MQA and other places, you reduce the number of KV heads. The advantage is there are fewer of them, but maybe the theory is you actually want a lot of different keys and values. So one way to reduce size is to keep one big shared vector for all keys and values, and then have smaller vectors for every single token. So you store only the smaller thing as some sort of low-rank reduction. At the end, when you want to compute the final thing, remember you are memory-bound, meaning you still have some compute left. So if you can expand the latent vector back out, this is far more efficient because you reduce, for example, from 32 to something like the size of the vector you're keeping. There's perhaps some richness in having a separate set of keys, values, and queries that pairwise match up, versus compressing that all into one. That interaction is at least okay.

所有这些都涉及内存受限。最终，这如何映射到用户体验？

And all of that is dealing with being memory bound. Ultimately, how does that map to the user experience?

它映射到两件事：第一，你现在可以让缓存大得多，因为分配给 KV 缓存的空间更少了。你可以更积极地缓存更多东西，从而获得更多缓存命中，这有助于减少首 token 时间，原因如前所述。第二，当你开始处理越来越多的请求和更大的批量时，生成 token 的速度不会明显变慢。它还可以让你在某些情况下使用更大的提示词。KV 缓存的大小是所有提示词的大小乘以并行处理的提示词数量。

The two things it maps to: first, you can now make your cache a lot larger because you've less space allocated for the KV cache. You can cache a lot more aggressively and more things, so you get more cache hits, which are helpful for reducing the time to first token for the reasons described earlier. Second, when you start doing inference with more and more requests and larger batch sizes, you don't see much of a slowdown in the speed of generating tokens. It also allows you to make your prompt bigger for certain things. The size of your KV cache is both the size of all your prompts multiplied by the number of prompts being processed in parallel.

增加这些维度，对吧？批次大小或提示的大小，而不降低生成令牌的延迟。Arvid，你写了一篇博客文章《影子工作区：在后台迭代代码》。是的，所以是怎么回事？

Increase either those dimensions, right? The batch size or the size of your prompts without degrading the latency of generating tokens. Arvid, you wrote a blog post, 'Shadow workspace: iterating on code in the background.' Yeah, so what's going on?

明确一下，我们希望后台发生很多事情，我们现在正在试验很多东西。除了像缓存预热或找出进入命令提示的合适上下文之外，我们还没有太多这样的功能。但想法是，如果你能在后台实际花费算力，那么你可以在比仅仅预测你接下来几行代码更长的时间范围内帮助用户，实际上是接下来的 10 分钟你要做什么。通过在后台进行，你可以花费更多算力来做这件事。所以我们实现并在内部用于实验的影子工作区的想法是，要真正利用后台做事的优势，你需要某种反馈信号回馈给模型，因为否则你可以通过让模型思考更长时间来获得更高的性能。O1 就是一个很好的例子，但另一种提高性能的方法是让模型迭代并获得反馈。对于程序员来说，一个非常重要的反馈是语言服务器，这个东西存在于大多数不同的语言中，每种语言都有一个单独的语言服务器。它可以告诉你那里使用了错误的类型并给出错误，或者它可以让你转到定义并理解代码的结构。语言服务器是由不同团队开发的扩展，比如 TypeScript 团队开发的 TypeScript 语言服务器，Rust 团队开发的 Rust 语言服务器，然后它们都通过语言服务器协议与 VS Code 接口，这样 VS Code 就不需要内置所有不同的语言，而是可以使用现有的编译器基础设施进行代码检查。它的用途是代码检查、转到定义以及查看你使用的正确类型。所以它也做类型检查。是的，类型检查和转到引用。当你在一个大项目中工作时，你基本上需要这个。如果没有它，在大项目中编码会非常困难。

So to be clear, we want there to be a lot of stuff happening in the background, and we're experimenting with a lot of things right now. We don't have much of that happening other than like cache warming or figuring out the right context that goes into your command prompts, for example. But the idea is, if you can actually spend computation in the background, then you can help the user at a slightly longer time horizon than just predicting the next few lines you're going to make, but actually in the next 10 minutes, what are you going to make? And by doing it in the background, you can spend more computation doing that. So the idea of the Shadow workspace that we implemented and we use it internally for experiments is that to actually get advantage of doing stuff in the background, you want some kind of feedback signal to give back to the model, because otherwise you can get higher performance by just letting the model think for longer. O1 is a good example of that, but another way you can improve performance is by letting the model iterate and get feedback. And one very important piece of feedback when you're a programmer is the language server, which is this thing that exists for most different languages, and there's a separate language server per language. It can tell you you're using the wrong type there and gives you an error, or it can allow you to go to definition and sort of understand the structure of your code. So language servers are extensions developed by, like, there's a TypeScript language server developed by the TypeScript people, a Rust language server developed by the Rust people, and then they all interface over the Language Server Protocol to VS Code, so that VS Code doesn't need to have all of the different languages built into VS Code, but rather you can use the existing compiler infrastructure for linting purposes. What it's for is linting, going to definition, and for seeing the right types that you're using. So it's doing type checking also. Yes, type checking and going to references. And that's like when you're working in a big project, you kind of need that. If you don't have that, it's really hard to code in a big project.

你能再说一遍它是如何在 Cursor 内部使用的吗？语言服务器协议通信这件事。

Can you say again how that's being used inside Cursor? The Language Server Protocol communication thing.

所以它在 Cursor 中被用来向程序员展示信息，就像在 VS Code 中一样，但想法是你想向 AI 模型展示相同的信息，并且你想以一种不影响用户的方式做到这一点，因为你希望它在后台进行。所以影子工作区背后的想法是：好的，我们可以这样做的一种方法是生成一个隐藏的 Cursor 单独窗口。你可以设置这个标志，Electron 是隐藏的；有一个窗口但你实际上看不到它。在这个窗口中，AI 智能体可以随意修改代码，只要它们不保存，因为仍然是同一个文件夹，然后可以从代码检查器获得反馈，转到定义并迭代代码。所以实际上是在后台运行一切，对吧？是的，甚至可能运行代码。所以那是最终版本。好的，那就是你想要的。博客文章的大部分内容实际上是关于如何实现这一点，因为这有点棘手。你希望它在用户的机器上，以便完全镜像用户的环境。然后在 Linux 上，你可以做一件很酷的事情，实际上可以镜像文件系统，让 AI 对文件进行更改，它认为它在文件级别操作，但实际上存储在内存中，你可以创建这个内核扩展来使其工作。而在 Mac 和 Windows 上，这有点困难。这是一个有趣的技术问题。一个可能有点 hacky 但有趣的想法是持有保存锁。所以基本上你可以让语言模型持有保存到磁盘的锁，而不是你在操作保存到磁盘的文件的地面真实版本，你实际上是在操作之前的影子工作区，这些未保存的东西只存在于内存中，你仍然可以得到代码检查错误，并且可以在其中编码。然后当你尝试运行代码时，会有一个小警告说有一个锁，然后如果你试图并发执行，你会从语言服务器或影子工作区那里拿回锁。

So it's being used in Cursor to show to the programmer just like in VS Code, but then the idea is you want to show that same information to the AI models, and you want to do that in a way that doesn't affect the user because you want to do it in the background. So the idea behind the Shadow workspace was: okay, one way we can do this is we spawn a separate window of Cursor that's hidden. You can set this flag, and Electron is hidden; there is a window but you don't actually see it. Inside this window, the AI agents can modify code however they want, as long as they don't save it because it's still the same folder, and then can get feedback from the linters and go to definition and iterate on their code. So literally run everything in the background, as if right? Yeah, maybe even run the code. So that's the eventual version. Okay, that's what you want. And a lot of the blog post is actually about how do you make that happen, because it's a little bit tricky. You want it to be on the user's machine so that it exactly mirrors the user's environment. And then on Linux, you can do this cool thing where you can actually mirror the file system and have the AI make changes to the files, and it thinks that it's operating on the file level, but actually that's stored in memory, and you can create this kernel extension to make it work. Whereas on Mac and Windows, it's a little bit more difficult. It's a fun technical problem. One maybe hacky but interesting idea that I like is holding a lock on saving. So basically you can then have the language model kind of hold the lock on saving to disk, and instead of you operating on the ground truth version of the files that are saved to disk, you actually are operating on what was the Shadow workspace before, and these unsaved things that only exist in memory that you still get lint errors for, and you can code in. And then when you try to maybe run code, it's just like there's a small warning that there's a lock, and then you kind of will take back the lock from the language server if you're trying to do things concurrently, or from the Shadow workspace if you're trying to do things concurrently.

顺便说一句，这是一个非常令人兴奋的功能。这有点跑题，但允许模型更改文件对人们来说很可怕，但能够让智能体完成一组任务，然后你第二天回来观察，就像它是一个同事一样，这真的很酷。

That's such an exciting feature, by the way. It's a bit of a tangent, but like to allow a model to change files, it's scary for people, but it's really cool to be able to just let the agent do a set of tasks and you come back the next day and kind of observe like it's a colleague or something like that.

是的，我认为可能有不同版本的可运行性。对于简单的事情，当用户编程时你在几分钟内代表他们做事，让某些东西在本地机器上工作是有意义的。我认为对于更激进的事情，当你进行更大、更耗时的更改时，你可能希望在某些沙盒远程环境中进行。这是另一个非常棘手的问题：如何精确地复制，或者大部分复制到有效等同于运行代码的程度，用户的远程环境。

Yeah, and I think there may be different versions of runnability. For the simple things where you're doing things in the span of a few minutes on behalf of the user as they're programming, it makes sense to make something work locally on their machine. I think for the more aggressive things where you're making larger changes that take longer periods of time, you'll probably want to do this in some sandbox remote environment. And that's another incredibly tricky problem of how do you exactly reproduce, or mostly reproduce to the point of it being effectively equivalent for running code, the user's environment which is remote.

我很好奇你想要什么样的编码智能体。你想要它们找 bug 吗？你想要它们实现新功能吗？你想要什么样的智能体？

I'm curious what kind of agents you want for coding. Do you want them to find bugs? Do you want them to implement new features? What agents do you want?

顺便说一句，当我想到智能体时，我不仅仅考虑编码。对于这个特定的播客，有视频编辑，如果你看 Adobe，背后有很多代码，代码文档很差，但你可以用代码与 Premiere 交互。基本上所有上传，我在 YouTube 上做的一切，你可能想象得到，我都是通过代码完成的，包括翻译和配音等等。所以我设想所有这些任务：自动化许多与编辑不直接相关的任务。所以这就是我在想的。但就编码而言，我基本上会考虑找 bug，比如多个层次的找 bug，还有找逻辑 bug，不是那种精神上的 bug 之类的。这算是大的方向。

By the way, when I think about agents, I don't think just about coding. I think for this particular podcast, there's video editing, and a lot of if you look in Adobe, there's code behind it, it's very poorly documented code, but you can interact with Premiere for example using code. And basically all the uploading, everything I do on YouTube, as you could probably imagine, I do all of that through code, and including translation and overdubbing, all this. So I envision all those kinds of tasks: automating many of the tasks that don't have to do directly with the editing. So that's what I was thinking about. But in terms of coding, I would be fundamentally thinking about bug finding, like many levels of kind of bug finding, and also bug finding like logical bugs, not logical like spiritual bugs or something. One's like sort of big directions.

实现，那种东西，找 bug。是啊，我觉得很有意思，这些模型在直接提示找 bug 时表现这么差。它们校准得极差，即使是最聪明的模型。没错，连 o1 也是。你怎么解释？有什么好的直觉吗？

Implementation, that kind of stuff, bug finding. Yeah, I mean, it's really interesting that these models are so bad at bug finding when just naively prompted to find a bug. They're incredibly poorly calibrated, even the smartest models. Exactly, even o1. How do you explain that? Is there a good intuition?

我认为这些模型强烈反映了预训练分布。我确实认为随着损失越来越低，它们会泛化，但我不认为损失和规模已经低到足以让它们在代码上完全泛化。我们使用这些前沿模型做的事情，它们很擅长代码生成和问答。这些在预训练中大量存在：GitHub 上所有代码，规模达数万亿 token，以及 Stack Overflow 和 GitHub issues 上的问答。所以当你尝试推动一些网上不太存在的东西，比如 Cursor 的 tab 目标——根据已有编辑预测下一个编辑——脆弱性就显现了。Bug 检测是另一个很好的例子，实际检测真实 bug 并提出修复的例子并不多。模型在这方面确实很挣扎。但我认为这是一个迁移模型的问题。就像从预训练模型（仅针对一般代码）到 Cursor tab 目标有很好的迁移一样，你会看到非常类似的情况：从擅长代码的通用模型到 bug 检测。只需要一点点的引导。明确地说，我认为它们在预训练时确实很理解代码。构建的表示几乎肯定让模型知道可能有些可疑的地方。它有一些可疑性。但实际引出这种可疑性……部分原因是人类对哪些 bug 真正重要有很好的校准。不仅仅是说“有点可疑”——是可疑但微不足道，还是可疑到“你会让服务器宕机”。部分原因可能是文化知识：为什么高级工程师是高级工程师？高级工程师优秀是因为他们知道三年前有人写了一段非常可疑的代码导致服务器宕机，而相比之下，也许这只是一个实验，所以一些 bug 没问题。如果模型在你写实验时变得非常烦人，那就很糟糕。但如果你在写生产级代码，比如数据库、postscripts 或 Linux 代码，即使是一个边缘情况也是不可接受的。校准用户的警惕程度……但即使你设置最大警惕，它还是不太行。

I think these models are a really strong reflection of the pre-training distribution. I do think they generalize as the loss gets lower and lower, but I don't think the loss and the scale are quite low enough such that they're really fully generalizing in code. The things we use these frontier models for, they're quite good at code generation and question answering. These things exist in massive quantities in pre-training: all the code on GitHub on the scale of many trillions of tokens, and questions and answers on things like Stack Overflow and GitHub issues. So when you try to push some of these things that really don't exist much online, like the Cursor tab objective of predicting the next edit given the edits done so far, the brittleness shows. Bug detection is another great example where there aren't really that many examples of actually detecting real bugs and then proposing fixes. The models just really struggle at it. But I think it's a question of transferring the model. In the same way that you get fantastic transfer from pre-trained models just on code in general to the Cursor tab objective, you'll see a very similar thing with generalized models that are really good at code to bug detection. It just takes a little bit of nudging in that direction. To be clear, I think they sort of understand code really well while they're being pre-trained. The representation that's being built up almost certainly has the model knowing that maybe there's something sketchy going on. It sort of has some sketchiness. But actually eliciting that sketchiness... Part of it is that humans are really calibrated on which bugs are really important. It's not just saying 'there's something sketchy' — it's sketchy trivial versus sketchy like 'you're going to take the server down.' Part of it is maybe the cultural knowledge of why a staff engineer is a staff engineer. A staff engineer is good because they know that three years ago someone wrote a really sketchy piece of code that took the server down, as opposed to maybe it's just an experiment, so a few bugs are fine. If the model gets really annoying when you're writing an experiment, that's really bad. But if you're writing something for production, like a database, code in postscripts or Linux, it's unacceptable to have even an edge case. Having the calibration of how paranoid the user is... But even then, if you put in maximum paranoia, it still just doesn't quite get it.

是啊，是啊。我的意思是，这对人类来说也很难，要理解哪行代码重要哪行不重要。我记得你网站上的一条原则：如果一段代码可能造成很大损害，就应该加注释说“这行代码很危险”，全部大写写十遍。不，你说的是函数内的每一行代码都得这样。这很有深意。它说明了人类的一些问题，因为工程师会离开，即使是同一个人也可能忘记它怎么让泰坦尼克号沉没。一个函数——你光看单段代码可能看不清楚。

Yeah, yeah. I mean, but this is hard for humans too, to understand which line of code is important and which is not. I think one of your principles on a website says: if a code can do a lot of damage, one should add a comment that says 'this line of code is dangerous' in all caps 10 times. No, you say for every single line of code inside the function you have to. And that's quite profound. It says something about human beings because engineers move on, even the same person might just forget how it can sink the Titanic. A single function — you might not see it quite clearly by looking at the single piece of code.

是的，我认为这也部分适用于今天的 AI 模型，如果你真的在每一行都写上“危险危险危险”，模型会更关注那里，更可能在那片区域找到 bug。这实际上是一种很好的实践，用代码可能造成的损害程度来标注代码。

Yeah, and I think that one is also partially for today's AI models, where if you actually write 'dangerous dangerous dangerous' in every single line, the models will pay more attention to that and will be more likely to find bugs in that region. That's actually just straight up a really good practice of labeling code with how much damage this can do.

是啊，这有争议。有些人觉得丑。嗯，我其实觉得……事实上，这是我从 AR 学到的东西之一。你知道，我审美上不喜欢它，但我认为它对模型确实有用。而且人类很容易忘记，犯个小错误就可能导致服务器宕机。当然我们做了很多测试，但总有些事情需要非常小心。

Yeah, I mean it's controversial. Some people think it's ugly. Well, I actually think it's... In fact, I think this is one of the things I learned from AR. You know, I sort of aesthetically don't like it, but I think there's certainly something where it's useful for the models. And humans just forget a lot, and it's really easy to make a small mistake and cause bring down the server. Of course we test a lot and whatever, but there are always these things that you have to be very careful about.

是的，就像普通的文档字符串，我觉得人们在修改时经常只是扫一眼，心想“哦，我知道怎么做”，而你真的需要指出来，这样才不会漏掉。你需要被提醒你可能造成很大损害。我们通常不会想这个。你想的是“我怎么弄清楚这个怎么工作以便改进它”，你不会想另一个方向可能出错。直到我们有了对所有事物的形式化验证，那时你可以为所欲为，并且如果证明通过，你就能确定没有引入 bug。

Yeah, like with just normal doc strings, I think people will often just skim it when making a change and think 'oh, I know how to do this,' and you kind of really need to point it out to them so that doesn't slip through. You have to be reminded that you could do a lot of damage. We don't really think about that. You think about 'how do I figure out how this works so I can improve it,' you don't think about the other direction that could go wrong. Until we have formal verification for everything, then you can do whatever you want and you know for certain that you have not introduced a bug if the proof passes.

但具体来说，你认为那个未来会是什么样子？

But concretely, what do you think that future would look like?

我认为人们将不再写测试，模型会建议——你写一个函数，模型会建议一个规范，你审查规范。同时，一个智能推理模型会计算一个证明，证明实现遵循规范。我认为这对大多数函数都会发生。

I think people will just write tests anymore, and the model will suggest — you write a function, the model will suggest a spec, and you review the spec. In the meantime, a smart reasoning model computes a proof that the implementation follows the spec. I think that happens for most functions.

你不觉得这触及了你之前谈到的，为软件指定意图的困难吗？有时因为意图很难指定，那么也很难证明它实际匹配你的意图。你觉得规范很难生成吗？

Don't you think this gets at a little bit of the stuff you were talking about earlier with the difficulty of specifying intent for what you want with software? Where sometimes it might be because the intent is really hard to specify, it's also then going to be really hard to prove that it's actually matching whatever your intent is. You think that spec is hard to generate?

是的，或者对于给定的规范，也许你可以……我认为有一个问题：你真的能做形式化验证吗？这可能吗？我觉得还有更多需要深挖。但即使你有这个规范，你怎么……规范是用自然语言写的吗？规范会是形式化的，但那有多容易？所以我认为你关心的是那些不容易在规范中很好指定的东西。

Yeah, or just for a given spec, maybe you can... I think there is a question of can you actually do the formal verification? Is that possible? I think there's more to dig into there. But then also, even if you have this spec, how do you... The spec is written in natural language? The spec would be formal, but how easy would that be? So then I think that you care about things that are not going to be easily well-specified in the spec.

我明白了。所以也许反对形式化验证的一个论点是，你只需要它。但担心的是，会有一份庞大的文档取代像单元测试这样的东西。

I see. So maybe an argument against formal verification is all you need. The worry is there's this massive document replacing something like unit tests.

是的。我认为我们可能也能改进规范语言，以捕捉它们目前未能捕捉的一些东西。但我不确定，我觉得这非常令人兴奋。而且你谈的不只是单个函数，而是整个代码库。我认为整个代码库更难，但那正是我想要的，而且我认为应该是可能的。因为最近有很多工作，你可以一直形式化验证到硬件：你形式化验证 C 代码，然后通过 GCC 编译器形式化验证，再通过 Verilog 一直验证到硬件。那是一个极其庞大的系统，但它确实有效。我认为大型代码库在某种程度上类似，它们是多层系统。如果你能分解它并形式化验证每个部分，那么我认为应该是可能的。我认为规范问题是一个真正的问题。

Yeah. I think you can probably also evolve the spec languages to capture some of the things they don't really capture right now. But I don't know, I think it's very exciting. And you're speaking not just about single functions, you're speaking about entire codebases. I think entire codebases is harder, but that is what I would love to have, and I think it should be possible. Because there's a lot of recent work where you can formally verify down to the hardware: you formally verify the C code, then you formally verify through the GCC compiler, and then through the Verilog down to the hardware. That's an incredibly big system, but it actually works. And I think big codebases are sort of similar in that they're multi-layered systems. If you can decompose it and formally verify each part, then I think it should be possible. I think the specification problem is a real problem.

你如何处理副作用或外部依赖，比如调用 Stripe API？也许 Stripe 会为他们的 API 写一个规范。但你能为你使用的所有东西都这样做吗？还有，如何处理程序中作为原语使用的语言模型？你如何将其纳入？

How do you handle side effects or external dependencies like calling the Stripe API? Maybe Stripe would write a spec for their API. But can you do this for everything you use? And how do you handle language models used as primitives in programs? How do you include that?

我认为你可能仍然能够证明这一点。至于语言模型？我觉得有可能你实际上可以证明一个语言模型是对齐的，例如，或者它确实给出了正确的答案。那是梦想。如果可能的话，那肯定有助于确保你的代码没有错误，并确保 AI 不会毁灭整个人类文明。所以从 AI 安全到仅仅找 bug 的整个范围。

I think you might be able to prove that still. What about language models? I think it feels possible that you could actually prove that a language model is aligned, for example, or that it actually gives the right answer. That's the dream. If it's possible, that will certainly help with making sure your code doesn't have bugs and making sure AI doesn't destroy all of human civilization. So the full spectrum from AI safety to just bug finding.

你说模型在找 bug 方面有困难。希望是什么？

You said the models struggle with bug finding. What's the hope?

我最初的希望是，它应该首先帮助解决愚蠢的错误，比如差一错误。有时你在注释中写了一些东西，但代码中却做了相反的事，比如我在注释中写“小于”，但代码中可能写了“大于”。模型会说：“嗯，看起来不太对，你确定要这么做吗？”但最终它也应该能捕捉 100%的错误。而且同样重要的是，拥有好的找 bug 模型对于让 AI 为你做越来越多的编程工作来说是必要的。如果 AI 为你构建了更多的系统，你不仅需要生成代码，还需要验证它。没有这一点，我们之前讨论过的用这些模型编程的一些问题将变得不可持续。所以这不仅是为了人类——你写了一个 bug，我写了一个 bug，帮我找到它——而且能够验证 AI 代码并检查它也非常重要。

My hope initially is that it should first help with the stupid bugs, like off-by-one errors. Sometimes you write something in a comment and do the opposite, like I write 'less than' in a comment but maybe write 'greater than' in code. The model says, 'Yeah, that looks sketchy, you sure you want to do that?' But eventually it should be able to catch 100% of bugs too. And it's also important to note that having good bug-finding models feels necessary to get to the highest reaches of having AI do more and more programming for you. If the AI is building more of the system for you, you need to not just generate but also verify. Without that, some of the problems we've talked about before with programming with these models will become untenable. So it's not just for humans—you write a bug, I write a bug, find the bug for me—but also being able to verify the AI code and check it is really important.

你实际上如何训练一个找 bug 的模型？我们为此有过激烈的晚餐讨论。

How do you actually train a bug-finding model? We've had contentious dinner discussions about this.

一个非常流行的想法是，引入一个 bug 可能比找到一个 bug 更容易。所以你可以训练一个模型在现有代码中引入 bug，然后训练一个反向的 bug 模型，利用这些合成数据来发现 bug。这是一个例子。但还有很多想法。你也可以在模型层面之外做很多工作：拿最大的模型，给它们访问不仅仅是代码的大量信息，比如跟踪和逐步调试。盯着一个文件说“bug 在哪里？”是一个难题，对人类来说也常常很难。所以还有另一个方向倾向于那样。也可能有两种不同的产品形态：一个非常专业、快速的模型在后台运行，试图发现 bug；而有时，就像 Arvid 之前关于一个恶意输入框 bug 的例子，你可能想投入大量算力来解决一个特定问题，并愿意花 50 美元来解决那个 bug。

One very popular idea is that it's potentially easier to introduce a bug than to find one. So you can train a model to introduce bugs in existing code, and then train a reverse bug model that can find bugs using this synthetic data. That's one example. But there are lots of ideas. You can also do a bunch of work not even at the model level: take the biggest models and give them access to a lot of information beyond just the code, like traces and stepping through a debugger. It's a hard problem to stare at a file and say 'where's the bug?' That's hard for humans often. So there's another direction that tends toward that. It could also be that there are two different product form factors: a really specialized, fast model running in the background trying to spot bugs, and sometimes, to Arvid's earlier example about a nefarious input box bug, you might want to throw tons of compute at a specific problem and be willing to pay $50 to solve that bug.

你有没有考虑过将金钱整合进来？如果你找到了一个 bug 或生成了我非常欣赏的代码，我愿意付一大笔钱。几天前，我开始使用 Cursor，它生成了完美的函数来与 YouTube API 交互，以更新字幕进行本地化。API 文档不好，我谷歌了好一会儿也没找到确切的信息。Cursor 完美地生成了代码。我读了代码，测试了，它是正确的。我想要一个打赏按钮，上面写着“给你 5 美元，太棒了”，以支持公司并发出强烈的信号。对于找 bug，有漏洞奖励模式。你们考虑过吗？

Have you thought about integrating money into this? I would pay a large amount if you found a bug or generated code I really appreciated. A few days ago, I started using Cursor and it generated perfect functions for interacting with the YouTube API to update captions for localization. The API documentation is not good, and I couldn't find exactly what I needed after Googling for a while. Cursor generated it perfectly. I read the code, tested it, it was correct. I wanted a tip button that says 'here's $5, that's really good' to support the company and send a strong signal. For bug finding, there's the bug bounty model. Have you thought about that?

这在公司内部是一个有争议的想法。这有点取决于你对人性的信任程度。我认为如果你不花钱去找 bug，如果没找到，你花零元；如果找到了一个 bug，你点击接受，它会在括号里显示“$1”，所以你花 1 美元接受一个 bug，那会很酷。但担心的是：我们花了大量算力，也许人们会直接复制粘贴。这是一个担忧。而且将金钱引入产品中……

It's a controversial idea inside the company. It sort of depends on how much you believe in humanity. I think it would be really cool if you spend nothing to try to find a bug, and if it doesn't find one, you spend zero. If it does find a bug and you click accept, it shows in parentheses '$1', so you spend $1 to accept a bug. But there's worry: we spent a lot of computation, maybe people will just copy-paste. That's a worry. And also introducing money into the product...

这样一来，感觉就没那么有趣了。你不得不考虑钱，而你只想专注于代码。所以也许把它分开更合理：每月付一笔费用，然后免费获得所有这些功能。但也可以有小费机制，虽然还是有金钱符号。我觉得没问题，但我也理解为什么可能不想引入它。

Makes it like, you know, it doesn't feel as fun anymore. You have to think about money, and all you want to think about is the code. So maybe it actually makes more sense to separate it out: you pay some fee every month and then you get all of these things for free. But there could be a tipping component, which is not like it still has that dollar symbol. I think it's fine, but I also see the point where maybe you don't want to introduce it.

是的，我想说的是，人们真正这样做的时候，是他们分享的时候。当他们有一个很棒的示例，就会分享给朋友。对于这种信用体系问题，也可能存在技术解决方案。如果我们能更好地理解系统的输出——比如我们讨论过的用 LSP 进行错误检查以及运行代码——但如果你能真正验证'哦，我修复了这个 bug'，那么赏金系统可能就不需要依赖信用体系了。

Yeah, I was going to say the moment that feels like people do this is when they share it. When they have this fantastic example, they just kind of share it with their friends. There is also a potential world where there's a technical solution to this honor system problem too, where if we can get to a place where we understand the output of the system more—I mean the stuff we were talking about with error checking with the LSP and also running the code—but if you could get to a place where you could actually somehow verify, 'Oh, I have fixed the bug,' maybe then the bounty system doesn't need to rely on the honor system too.

终端和代码之间有多少交互？比如，在终端中运行代码能获得多少信息？能不能做一个循环，运行代码，如果代码和运行时出错，就建议如何修改代码？

How much interaction is there between the terminal and the code? Like, how much information is gained from running the code in the terminal? Can you do a loop where it runs the code and suggests how to change the code if the code and runtime gives an error?

目前，它们完全是两个独立的世界。我知道你可以在终端里用 Control+K 来帮助写代码。你也可以在 Cursor 中使用终端上下文。基本上都可以。但我们还没有循环部分。我们觉得类似的东西可能很有意义。问题是它是在前台还是后台进行，就像我们讨论的那样。当然，后台很酷。我们以不同的方式运行代码。此外，还有数据库方面的问题：如何保护它不被修改数据库？但好吧，肯定有很酷的解决方案。有一个新的 API 正在开发中——不在 AWS 上，但我想是在 PlanetScale 上。我不确定 PlanetScale 是不是第一个添加这个功能的——就是给数据库添加分支的能力。如果你在开发一个功能，想针对生产数据库测试，但又不想真的针对生产数据库测试，你可以给数据库添加一个分支。实现方法是给预写日志添加一个分支。正确实现显然有很多技术复杂性。我猜数据库公司需要找新事情做，因为他们现在已经有很好的数据库了。我认为我们使用的数据库之一 Turbo Buffer 将会——希望如此——给预写日志添加分支功能。所以也许 AI 智能体会使用分支；它们会针对某个分支进行测试，这将成为数据库支持分支之类功能的一个需求。

Right now, they're separate worlds completely. I know you can do Control+K inside the terminal to help you write the code. You can use terminal context as well inside of Cursor. Kind of everything. We don't have the looping part yet, though. We suspect something like this could make a lot of sense. There's a question of whether it happens in the foreground or in the background, like what we've been discussing. Sure, the background is pretty cool. We do run the code in different ways. Plus, there's a database side to this: how do you protect it from not modifying the database? But okay, I mean, there's certainly cool solutions there. There's this new API that is being developed—it's not in AWS, but I think it's in PlanetScale. I don't know if PlanetScale was the first one to add it—the ability to sort of add branches to a database. If you're working on a feature and you want to test against the production database but you don't actually want to test against the production database, you could sort of add a branch to the database. The way to do that is to add a branch to the write-ahead log. There's obviously a lot of technical complexity in doing it correctly. I guess database companies need new things to do because they have good databases now. I think Turbo Buffer, which is one of the databases we use, is going to add—hopefully—branching to the write-ahead log. So maybe AI agents will use branching; they'll test against some branch, and it's sort of going to be a requirement for the database to support branching or something.

如果你能给文件系统做分支，那会非常有趣，对吧？

It would be really interesting if you could branch a file system, right?

是的，我觉得所有东西都需要分支。就是这样。

Yeah, I feel like everything needs branching. It's like that.

是啊，是啊。就像多元宇宙的问题一样，对吧？如果你对每件事都做分支，那太多了。

Yeah, yeah. Like that's the problem with the multiverse, right? If you branch on everything, that's a lot.

我的意思是，显然有一些非常巧妙的算法来确保你不会占用太多空间或 CPU 之类的。

I mean, there's obviously these super clever algorithms to make sure that you don't actually use a lot of space or CPU or whatever.

好了，这是个问基础设施的好时机。你们主要用 AWS。有什么有趣的细节？有什么有趣的挑战？为什么选择 AWS？为什么 AWS 还在赢？标签 AWS。

Okay, this is a good place to ask about infrastructure. So you guys mostly use AWS. What are some interesting details? What are some interesting challenges? Why did you choose AWS? Why is AWS still winning? Hashtag AWS.

AWS 真的非常非常好。真的很好。每当你使用 AWS 产品，你就知道它会正常工作。设置步骤可能非常痛苦。为什么界面这么糟糕？因为它太好了，不需要改进。这就是赢家的本质。

AWS is just really, really good. It's really good. Whenever you use an AWS product, you just know that it's going to work. It might be absolute hell to go through the steps to set it up. Why is the interface so horrible? Because it's just so good it doesn't need to. The nature of winning.

我觉得就是这样。这就是赢家的本质。是啊，是啊。但 AWS 你永远可以信任。它总是能工作，如果有问题，那很可能是你的问题。

I think it's exactly it. It's just nature of winning. Yeah, yeah. But AWS you can always trust. It will always work, and if there is a problem, it's probably your problem.

好的，有什么有趣的挑战吗？你们是一家相当新的初创公司，要扩展到这么多用户。

Okay, is there some interesting challenges? You guys are a pretty new startup, getting to scale to so many people.

是的，我认为这是一段有趣的旅程。每增加一个零的每秒请求数，你都会遇到用于缓存和数据库的通用组件的问题。随着规模越来越大，它们会出现问题。现在我们到了表出现整数溢出之类的规模。此外，我们还构建了一些自定义系统，比如用于计算代码库语义索引并回答代码库问题的检索系统，这些系统一直是扩展中最棘手的事情之一。我有几个超级资深工程师的朋友，他们常说的一句话是：很难预测系统在扩展时会在哪里崩溃。你可以尝试提前预测，但当你增加那个零时，总会有一些奇怪的事情发生。你以为你考虑周全了，但实际上并没有。对于那个特定系统，我们做的是：将你的所有代码分块，上传进行嵌入，嵌入代码，然后将嵌入存储在数据库中。但我们实际上不存储任何代码。这样做是为了确保不引入客户端错误，因为我们对客户端错误非常谨慎。我们将大部分细节存储在服务器上。所有内容都是加密的。技术挑战之一始终是确保本地索引——本地代码库状态——与服务器上的状态一致。我们最终的技术实现是：对于每个文件，你可以保留一个哈希值，对于每个文件夹，你可以保留一个哈希值，该哈希值是所有子文件的哈希值，然后递归直到顶层。为什么要做这么复杂的事情？一种做法是：为每个文件保留一个哈希值，然后每分钟尝试下载服务器上的哈希值，找出哪些文件在服务器上不存在，也许你刚创建了一个新文件，也许你刚删除了一个文件，也许你切换了一个新分支，然后尝试协调客户端和服务器之间的状态。但这会引入极其庞大的……

Yeah, I think it has been an interesting journey. Adding each extra zero to the requests per second, you run into all these issues with the general components you're using for caching and databases. They run into issues as you make things bigger and bigger. Now we're at the scale where we get integer overflows on our tables and things like that. Also, there have been some custom systems that we've built, like our retrieval system for computing a semantic index of your codebase and answering questions about a codebase, that have continually been one of the trickier things to scale. I have a few friends who are super senior engineers, and one of their sort of lines is: it's very hard to predict where systems will break when you scale them. You can sort of try to predict in advance, but there's always something weird that's going to happen when you add that extra zero. You thought you thought through everything, but you didn't actually think through everything. For that particular system, what we do is: we chunk up all of your code, send it up for embedding, embed the code, and store the embeddings in a database. But we don't actually store any of the code. There are reasons around making sure that we don't introduce client bugs because we're very paranoid about client bugs. We store much of the details on the server. Everything is sort of encrypted. One of the technical challenges is always making sure that the local index—the local codebase state—is the same as the state that is on the server. The way we ended up doing that technically is: for every single file, you can keep a hash, and for every folder, you can keep a hash which is the hash of all its children, and you can recursively do that until the top. Why do something complicated? One thing you could do is keep a hash for every file, then every minute try to download the hashes that are on the server, figure out what files don't exist on the server, maybe you just created a new file, maybe you just deleted a file, maybe you checked out a new branch, and try to reconcile the state between the client and the server. But that introduces absolutely ginormous...

网络开销——无论是客户端还是服务端——没人希望我们一直占用他们的 Wi-Fi，如果你在用 Cursor 的话。而且它还会给数据库带来巨大的开销，相当于每秒读取一个几十 TB 的数据库，接近 20TB 左右。这太疯狂了，你绝对不想这么做。所以我们的做法是，只尝试协调项目根目录的单个哈希值。如果出现不匹配，再去查找所有不一致的地方。比如检查子节点，看哈希值是否匹配；如果不匹配，再继续检查它们的子节点，以此类推。但只有在不匹配的情况下才这样做。对大多数人来说，大多数时候哈希值都是匹配的。所以这是一种分层协调机制。

Network overhead both on the client side—nobody really wants us to hammer their Wi-Fi all the time if you're using Cursor—but also it would introduce ginormous overhead in the database. It would sort of be reading this tens of terabyte database, approaching like 20 terabytes or something, every second. That's just crazy. You definitely don't want to do that. So what you do is you try to reconcile the single hash which is at the root of the project. If something mismatches, then you go find where all the things disagree. Maybe you look at the children and see if the hashes match. If the hashes don't match, go look at their children, and so on. But you only do that in the scenario where things don't match. For most people, most of the time, the hashes match. So it's a kind of hierarchical reconciliation.

对，差不多是这样。这叫 Merkle 树。

Yeah, something like that. It's called the Merkle tree.

对，Merkle。看到你们需要思考所有这些问题是挺酷的。之所以变得困难，只是因为使用人数太多了。有些客户的代码库非常庞大，以至于我们之前重新整理了自己的代码库——它已经很大了，但跟那些存在了 20 年、需要训练海量文件的公司相比还是小巫见大巫。你想让它在程序员之间扩展。有很多细节：做简单的东西很容易，但扩展到很多人、很多公司显然是个难题，这跟 AI 本身无关。所以这是扩展的一部分：我们当前的解决方案也在提出新想法，我们正在努力，然后在过去几周里把所有这些都扩展了。

Yeah, Merkle. It's cool to see that you have to think through all these problems. The reason it's gotten hard is just because of the number of people using it. Some of your customers have really, really large codebases, to the point where we originally reordered our codebase, which is big, but just not the size of some company that's been there for 20 years and has to train an enormous number of files. You want to scale that across programmers. There are all these details where building the simple thing is easy, but scaling it to a lot of people—a lot of companies—is obviously a difficult problem, which is sort of independent of the AI. So there's part of this scaling: our current solution is also coming up with new ideas that we're working on, but then scaling all of that in the last few weeks.

这个索引系统里还有很多巧妙的设计。例如，成本瓶颈不在于向量数据库或普通数据库的存储，而在于对代码进行嵌入。你不想为公司的每个员工都重新嵌入完全相同的代码库，除非他们处于不同的分支，有少量文件不同，或者做了一些本地修改。由于嵌入是瓶颈，你可以用一个巧妙的技巧：根据给定块的哈希值缓存实际计算出的向量。这意味着当公司里的第 n 个人进入他们的代码库时，速度会非常快。而且这一切都不需要在我们的服务器上存储任何代码——不存储代码数据。我们只把向量存储在向量数据库和向量缓存中。

There are a lot of clever additional things that go into this indexing system. For example, the bottleneck in terms of costs is not storing things in the vector database or the database; it's actually embedding the code. You don't want to re-embed the codebase for every single person in a company that is using the same exact code, except maybe they're in a different branch with a few different files or they've made a few local changes. Because embeddings are the bottleneck, you can do this one clever trick: you have some cache on the actual vectors computed from the hash of a given chunk. This means that when the nth person at a company goes into their codebase, it's really, really fast. And you do all this without actually storing any code on our servers at all—no code data stored. We just store the vectors in the vector database and the vector cache.

索引代码库带来的最大好处是什么？纯粹好奇，用户能得到什么好处？长期来看似乎好处会越来越多，但短期内，只是对代码库提问——这有什么用呢？

What's the biggest gain you get from indexing the codebase? Just out of curiosity, what benefit do users have? It seems like longer term there'll be more and more benefit, but in the short term, just asking questions of the codebase—what's the usefulness?

我认为最明显的一点是，你想在大代码库里找到某个东西在哪里。你模糊记得“我想找到我们做 X 的地方”，但你不确定在普通文本搜索里该搜什么。你通过聊天提问——按 Command+Enter 用代码库聊天功能——它经常能准确找到你想到的那个位置。我认为未来这只会越来越强大。我们正在大力改进检索质量，而且我认为它的天花板比人们想象的要高得多。

I think the most obvious one is you want to find out where something is happening in your large codebase. You have a fuzzy memory of 'I want to find the place where we do X,' but you don't exactly know what to search for in a normal text search. You ask a chat—you hit Command+Enter to ask with the codebase chat—and very often it finds the right place that you were thinking of. I think in the future, this is only going to get more and more powerful. We're working a lot on improving the quality of our retrieval, and I think the ceiling for that is really, really much higher than people give it credit for.

有个问题：你们考虑过吗，为什么没有更多地做本地化处理，比如在本地做嵌入？似乎我们刚才讨论的一切在云端做都异常困难——你需要考虑缓存、大型代码库、多个程序员使用同一代码库等等。大多数软件都是在本地做这种重计算任务的。

One question: have you considered and why haven't you done much local stuff, where you can do the embeddings locally? It seems like everything we just discussed is exceptionally difficult to do in the cloud—you have to think about all these things with caching and large codebases with many programmers using the same codebase. Most software does heavy computational stuff locally.

我们考虑过，而且我认为在本地做会很酷。但只是太难了。需要记住的是，我们有些用户用的是最新的 MacBook Pro，但大多数用户——超过 80%——用的是 Windows 机器，而且很多配置并不高。本地模型真的只适用于最新的电脑，而且构建这样的功能开销也很大。所以即使我们想这么做，目前也无法集中精力去做。有些人确实在这么做，我觉得那很好。但随着模型越来越大，你想用更大的模型做更花哨的事情，在本地做就变得更难了。

We thought about it, and I think it would be cool to do it locally. I think it's just really hard. One thing to keep in mind is that some of our users use the latest MacBook Pro, but most of our users—more than 80%—are on Windows machines, and many of them are not very powerful. Local models really only work on the latest computers, and it's also a big overhead to build that in. So even if we would like to do that, it's currently not something we are able to focus on. There are some people that do that, and I think that's great. But especially as models get bigger and bigger and you want to do fancier things with bigger models, it becomes even harder to do it locally.

而且这不只是电脑性能弱的问题。例如，如果你是一家大公司，拥有大型代码库，即使在最顶配的 MacBook Pro 上处理起来也非常困难。即使你是大公司里最优秀的程序员，如果所有事情都在本地做，体验也会很糟糕。你也许能勉强应付，但那就不好玩了。

And it's not a problem of weaker computers. For example, if you're some big company with a big company codebase, it's just really hard to process that even on the beefiest MacBook Pros. Even if you're the best programmer at a big company, you're still going to have a horrible experience if you do everything locally. You could scrape by, but it wouldn't be fun anymore.

没错，在这个庞大的代码库里做近似最近邻搜索会直接吃掉你的内存和 CPU。就是这样。

Yeah, approximate nearest neighbors in this massive codebase is going to just eat up your memory and your CPU. That's just that.

我们来谈谈模型方面。本地模型面临着巨大的阻力。第一：事情似乎在向 MoE（混合专家模型）发展，它有一个好处——可能更受内存带宽限制，这对本地运行有利，而不是使用 GPU 或 Nvidia GPU。但缺点是这些模型总体上更大，而且通常需要放在多个节点上，甚至一个节点都放不下。它们不可能装进即使是很好的 MacBook 里。我认为特别是对于编程来说，问题不在于“模型是否达到了某个足够好的标准”，然后我们就满意了——其他问题可能如此，本地模型在那里表现出色。但人们总是想要最好、最智能、最强大的东西。

Let's talk about the modeling side. There are these massive headwinds against local models. One: things seem to move towards MoEs, which have one benefit—maybe they're more memory bandwidth bound, which plays in favor of local versus using GPUs or Nvidia GPUs. But the downside is these models are just bigger in total, and they're going to need to fit often not even on a single node but multiple nodes. There's no way that's going to fit inside even really good MacBooks. I think especially for coding, it's not a question of 'does it clear some bar of the model being good enough?' and then we're satisfied—which may be the case for other problems where local models shine. But people are always going to want the best, the most intelligent, the most capable things.

而且这对几乎所有人来说都很难在本地运行。你难道不想要最强大的模型吗？比如你想要 Sonnet，还有 o1。我喜欢你推销的方式。你会满足于一个较差的模型吗？

And that's going to be really, really hard to run for almost all people locally. Don't you want the most capable model? Like you want Sonnet, and also with o1. I like how you're pitching me. Would you be satisfied with an inferior model?

听着，我就是其中之一，但有些人喜欢在本地做事情。有一个完整的开源运动在抵制，他们的存在是好的，因为你需要抵制正在增长的权利中心。实际上，有一种我特别喜欢的本地模型替代方案。我认为它仍处于研究阶段，但你可以想象对语言模型推理进行同态加密。你在本地机器上加密输入，然后发送上去，服务器可以用大量算力在你无法本地运行的模型上处理这些加密数据，但他们看不到数据是什么。然后他们返回答案，你解密，只有你能看到答案。我认为这仍处于研究阶段，关键在于降低开销，因为目前开销非常大。但如果能实现，那将非常酷且影响深远。因为令人担忧的是，随着这些模型越来越好，它们会变得越来越经济实用，世界上越来越多的信息和数据将流经一两个中心化参与者。然后会有传统黑客攻击的担忧，但更可怕的是，如果世界上所有的信息都以明文形式流经一个节点，就可能出现非常糟糕的监控。有时这会出于好的原因，比如人们想防止恶意行为者以坏的方式使用 AI 模型，然后你加入一些监控代码，接着别人会介入，你就滑向了滑坡，然后开始用大量世界数据做坏事。所以我非常希望我们能解决同态加密，实现隐私保护的机器学习。但这是当今所有软件面临的挑战：云能提供太多功能，我们越来越依赖它，生活变得更美好，但也有缺点。这就是为什么你需要非常好的安全性来抵御基本攻击，但也只有少数公司控制着这些数据。他们有影响力，可能以各种方式被渗透。这就是我们生活的世界。

Listen, I'm one of those, but there are some people that like to do stuff locally. There's a whole open source movement that resists, and it's good that they exist because you want to resist the power centers that are growing. There's actually an alternative to local models that I'm particularly fond of. I think it's still very much in the research stage, but you could imagine doing homomorphic encryption for language model inference. So you encrypt your input on your local machine, then you send that up, and the server can use lots of computation to run models that you cannot run locally on this encrypted data, but they cannot see what the data is. Then they send back the answer, and you decrypt it, and only you can see the answer. I think that's still very much research, and it's all about trying to make the overhead lower because right now the overhead is really big. But if you can make that happen, I think that would be really cool and impactful. Because one thing that's actually kind of worrisome is that as these models get better and better, they're going to become more and more economically useful, and so more and more of the world's information and data will flow through one or two centralized actors. Then there are worries about traditional hacker attempts, but it also creates this scary part where if all of the world's information is flowing through one node in plain text, you can have surveillance in very bad ways. Sometimes that will happen for good reasons, like people will want to protect against bad actors using AI models in bad ways, and then you add in some surveillance code, and then someone else will come in, and you're on a slippery slope, and then you start doing bad things with a lot of the world's data. So I'm very hopeful that we can solve homomorphic encryption for doing privacy-preserving machine learning. But that's the challenge we have with all software these days: there are so many features that can be provided from the cloud, and all of us increasingly rely on it and make our life awesome, but there are downsides. That's why you rely on really good security to protect from basic attacks, but there's also only a small set of companies controlling that data. They have leverage and could be infiltrated in all kinds of ways. That's the world we live in.

是的，我真正担心的是这样一个世界：Anthropic 有负责任的 Scaling 政策，我们处于低 ASL 级别，即 Anthropic 安全级别，但随着我们达到代码 ASL 3、ASL 4 等非常强大的模型，出于合理的安全原因，你会想监控所有提示。但我认为这可以理解，每个人都有自己的出发点，但如果世界上所有信息都受到如此严密的监控，那将非常可怕。这太集中了。就像走在一条细线上，一方面你不想让模型失控，另一方面，人类，我不知道我是否信任所有世界信息都经过三个模型提供商。

Yeah, I mean, the thing I'm actually quite worried about is the world where Anthropic has this responsible scaling policy, and we're on the low ASLs, which is the Anthropic security level, but as we get to code ASL 3, ASL 4, whatever models, which are very powerful, for mostly reasonable security reasons you would want to monitor all the prompts. But I think that's reasonable and understandable where everyone is coming from, but man, it'd be really horrible if all the world's information is monitored that heavily. It's way too centralized. It's like a really fine line you're walking where on one side you don't want the models to go rogue, on the other side, humans, I don't know if I trust all the world's information to pass through three model providers.

是的，为什么你认为这与云提供商不同？因为我认为很多这些数据一开始就不会交给云提供商。而在这里，你经常想给 AI 模型更多数据，那些你本来绝不会放到网上的个人数据，给这些公司或模型。这也集中了控制权，而对于云，你通常可以使用自己的加密密钥，它做不了什么。但在这里，中心化参与者能看到所有东西的明文。

Yeah, why do you think it's different than cloud providers? Because I think a lot of this data would never have gone to the cloud providers in the first place. Here, you often want to give more data to the AI models, personal data that you would never have put online in the first place, to these companies or to these models. It also centralizes control, whereas for cloud you can often use your own encryption keys and it can't really do much. But here, it's just centralized actors that see the exact plain text of everything.

关于上下文的话题，这实际上是我写代码时的一个摩擦点。在 Python 中，有很多导入的东西。可能有一些我想包含在上下文中的内容。自动找出上下文有多难？

On the topic of context, that's actually been a friction for me when I'm writing code. In Python, there's a bunch of stuff imported. There's probably stuff I would like to include in the context. How hard is it to auto-figure out the context?

这很棘手。我认为未来我们可以在自动计算上下文方面做得更好。需要注意的一点是，包含自动上下文是有权衡的。你为这些模型包含的上下文越多，首先它们会变慢，请求也更昂贵，这意味着你可以在后台做更少的模型调用和更少的花哨操作。此外，对于很多模型，如果提示中有大量信息，它们会感到困惑，因此你包含的上下文的准确性和相关性门槛应该很高。但这已经在产品中的一些地方实现了。这绝对是我们想改进的地方，我认为有很多酷想法可以尝试，包括学习更好的检索系统，比如更好的嵌入模型、更好的排序器。我认为还有一些很酷的学术想法，我们内部尝试过，但整个领域也在纠结：你能让语言模型达到一个地步，让模型本身理解一个新的信息语料库吗？最流行的讨论版本是：你能让上下文窗口无限吗？然后如果你让上下文窗口无限，你能让模型真正关注无限上下文吗？在你能让它关注无限上下文之后，为了使其可行，你能为那个无限上下文做缓存，这样你就不必一直重新计算吗？但还有其他正在尝试的酷想法，更类似于微调，实际上是在模型权重中学习这些信息。如果你在权重层面做，而不是在上下文学习层面做，你可能会得到一种性质不同的理解。我认为关于这一切最终如何运作，还没有定论。但在此期间，作为一家公司，我们对更好的检索系统和挑选代码库的相关部分感到非常兴奋。

It's tricky. I think we can do a lot better at computing the context automatically in the future. One thing that's important to note is there are trade-offs with including automatic context. The more context you include for these models, first of all, the slower they are, and the more expensive those requests are, which means you can then do less model calls and do less fancy stuff in the background. Also, for a lot of these models, they get confused if you have a lot of information in the prompt, so the bar for accuracy and relevance of the context you include should be quite high. But this is already something we do in some places within the product. It's definitely something we want to get a lot better at, and I think there are a lot of cool ideas to try there, both on learning better retrieval systems like better embedding models, better rankers. I think there are also cool academic ideas, stuff we've tried out internally, but also the field is grappling with: can you get language models to a place where you can actually have the model itself understand a new corpus of information? The most popular talked about version of this is: can you make the context window infinite? Then if you make the context window infinite, can you make the model actually pay attention to the infinite context? And after you can make it pay attention to the infinite context, to make it somewhat feasible, can you then do caching for that infinite context so you don't have to recompute that all the time? But there are other cool ideas being tried that are a bit more analogous to fine-tuning, of actually learning this information in the weights of the model. It might be that you actually get a qualitatively different type of understanding if you do it more at the weight level than if you do it at the in-context learning level. I think the jury is still a little bit out on how this is all going to work in the end. But in the interim, as a company, we are really excited about better retrieval systems and picking the parts of the code base that are relevant.

与你所做的最相关的是，我们可以做得更好。一个有趣的将知识直接学习到权重中的概念验证是 VS Code。我们正在使用 VS Code 的一个分支，VS Code 的代码都是公开的，所以这些模型在预训练中已经看到了所有代码。它们可能也看到了关于代码的问题和答案，然后它们经过了微调和基于人类反馈的强化学习（RLHF），以便能够回答关于代码的一般问题。所以当你问它一个关于 VS Code 的问题时，有时它会胡编乱造，但有时它确实能很好地回答问题。我认为这只是因为它碰巧在这方面还行。但如果你能专门训练或后训练一个模型，使其真正理解这个代码库呢？这是一个开放的研究问题，我们对此很感兴趣。还有一个不确定性是，你是希望模型端到端地做所有事情，即它在内部进行检索，然后回答你的问题、生成代码？还是你想将检索与前沿模型分开，也许几个月后你会得到一些比最好的开源模型强大得多的模型，然后你想单独训练一个非常好的开源模型作为检索器，为这些更大的模型提供上下文？你能多谈谈后训练模型以理解代码库吗？你指的是什么？这是合成数据的方向吗？

Most relevant to what you're doing, we could do that a lot better. Like one interesting proof of concept for learning this knowledge directly in the weights is with VS Code. So we're in a VS Code fork, and VS Code's code is all public, so these models in pre-training have seen all the code. They probably also seen questions and answers about it, and then they've been fine-tuned and RLHF'd to be able to answer questions about code in general. So when you ask it a question about VS Code, you know sometimes it'll hallucinate, but sometimes it actually does a pretty good job at answering the question. And I think this is just by it happens to be okay at it. But what if you could actually specifically train or post-train a model such that it really was built to understand this codebase? It's an open research question, one that we're quite interested in. And then there's also uncertainty of like, do you want the model to be the thing that end-to-end is doing everything, i.e., it's doing the retrieval in its internals and then kind of answering your question, creating the code? Or do you want to separate the retrieval from the frontier model, where maybe you know you'll get some really capable models that are much better than the best open source ones in a handful of months, and then you'll want to separately train a really good open source model to be the retriever, to be the thing that feeds in the context to these larger models? Can you speak a little more to the post-training a model to understand the codebase? Like what do you mean by that? Is this synthetic data direction?

是的，我的意思是，有很多可能的方法可以尝试。想法肯定不少。问题在于去尝试所有方法，并凭经验找出哪种效果最好。你知道，一个非常天真的做法是尝试复制在 VS Code 和这些前沿模型上所做的。所以让我们进行某种持续预训练，包括通用代码数据，但也加入大量你关心的特定仓库的数据。然后在后训练中，即在指令微调中，你有一个关于代码的正常指令微调数据集，然后你加入大量关于该仓库代码的问题。所以你可以获取真实答案，这可能很困难，或者你可以做你暗示或建议的，使用合成数据。即让模型对代码的各个部分提出问题。所以你取出代码片段，然后提示模型或让模型为那段代码提出一个问题，然后将这些作为指令微调数据点添加进去。理论上，这可能会解锁模型回答关于该代码库问题的能力。

Yeah, I mean there are many possible ways you could try doing it. There's certainly no shortage of ideas. It's just a question of going in and trying all of them and being empirical about which one works best. You know, one very naive thing is to try to replicate what's done with VS Code and these frontier models. So let's do some kind of continued pre-training that includes general code data but also throws in a lot of the data of some particular repository that you care about. And then in post-training, meaning in instruction fine-tuning, you have a normal instruction fine-tuning dataset about code, then you throw in a lot of questions about code in that repository. So you could either get ground truth ones, which might be difficult, or you could do what you kind of hinted at or suggested, using synthetic data. I.e., kind of having the model ask questions about various pieces of the code. So you take the pieces of the code, then prompt the model or have a model propose a question for that piece of code, and then add those as instruction fine-tuning data points. And then in theory, this might unlock the model's ability to answer questions about that codebase.

让我问问你关于 OpenAI 的看法。你认为那种测试时计算系统在编程中扮演什么角色？

Let me ask you about OpenAI. What do you think is the role of that kind of test-time compute system in programming?

我认为测试时计算非常非常有趣。预训练机制随着数据量和模型规模的扩大，会在损失和下游基准测试上带来更好的性能，在编码或其他任务上也是如此。我们开始遇到数据墙，这意味着继续扩大这一机制将变得困难。因此，扩大测试时计算是一种有趣的方式，可以增加推理时使用的 FLOPs，同时仍然获得模型性能的相应提升。传统上，我们只能训练一个更大的模型，它总是使用更多的 FLOPs。但现在，我们也许可以使用相同规模的模型，运行更长时间，从而获得与更大模型质量相当的答案。我真正喜欢的一点是：有些问题可能需要一个在 100 万亿 token 上训练的 100 万亿参数模型的智能，但这可能只占所有查询的 1%，甚至 0.1%。那么，你会投入所有精力和算力去训练一个如此昂贵的模型，却很少运行它吗？这感觉完全浪费。相反，你得到一个能处理 99.9% 查询的模型，然后在推理时，为那些真正想要最大智能的少数人运行更长时间。

I think test-time compute is really, really interesting. So there's been the pre-training regime, which will, as you scale up the amount of data and the size of your model, get you better and better performance both on loss and on downstream benchmarks, and just general performance when we use it for coding or other tasks. We're starting to hit a bit of a data wall, meaning it's going to be hard to continue scaling up this regime. And so scaling up test-time compute is an interesting way of now increasing the number of inference-time FLOPs that we use, but still getting corresponding improvements in the performance of these models. Traditionally, we just had to literally train a bigger model that always used that many more FLOPs. But now we could perhaps use the same size model and run it for longer to be able to get an answer at the quality of a much larger model. And so the really interesting thing I like about this is: there are some problems that perhaps require a 100-trillion parameter model intelligence trained on 100 trillion tokens, but that's like maybe 1%, maybe 0.1% of all queries. So are you going to spend all this effort, all this compute training a model that costs that much and then run it so infrequently? It feels completely wasteful. Instead, you get the model that is capable of doing the 99.9% of queries, then you have a way at inference time of running it longer for those few people that really, really want max intelligence.

你如何确定哪个问题需要什么级别的智能？是否可能动态地判断何时使用 GPT-4，何时使用小模型，何时需要 o1？

How do you figure out which problem requires what level of intelligence? Is that possible to dynamically figure out when to use GPT-4, when to use a small model, and when you need the o1?

我的意思是，这当然是一个开放的研究问题。我认为没有人真正很好地解决了这个模型路由问题。我们想解决，我们在 Cursor Tab 之类的东西上有初步实现，但在 4 Sonnet 和 o1 之间切换的层面上，这有点棘手。还有一个问题是，你需要什么级别的智能来判断某个问题对于 4 级模型来说是否太难？也许你需要 o1 级别的模型。这真的很不清楚。

I mean, that's an open research problem, certainly. I don't think anyone's actually cracked this model routing problem quite well. We'd like to, we have kind of initial implementations of this for things like Cursor Tab, but at the level of going between 4 Sonnet to o1, it's a bit trickier. There's also a question of what level of intelligence do you need to determine if the thing is too hard for the 4-level model? Maybe you need the o1-level model. It's really unclear.

你提到了预训练过程，然后是后训练，然后是测试时计算。这确实有所区分。最大的收益在哪里？

You mentioned there's a pre-training process, then there's post-training, and then there's test-time compute. That does sort of separate. Where's the biggest gains?

嗯，这很奇怪，因为测试时计算需要一整套训练策略才能工作。另一个奇怪的地方是，除了大型实验室，也许只有 OpenAI，没有人真正知道它是如何工作的。有一些非常有趣的论文暗示了他们可能在做的事情，也许他们正在使用过程奖励模型。但我觉得问题是我们并不确切知道它是什么样子，所以很难评论它属于哪个部分。我会把它放在后训练中，但也许让测试时计算为一个模型工作所花费的算力最终会超过预训练。所以我们甚至不知道 o1 是否只是使用了思维链强化学习。我们不知道他们如何使用这些。我们什么都不知道。推测起来很有趣。如果你要构建一个竞争模型，你会怎么做？

Well, it's weird because test-time compute requires a whole training strategy to get it to work. And the really other weird thing about this is no one outside of the big labs, and maybe even just OpenAI, really knows how it works. There have been some really interesting papers that show hints of what they might be doing, and so perhaps they're doing something with process reward models. But yeah, I just think the issue is we don't quite know exactly what it looks like, so it would be hard to kind of comment on where it fits in. I would put it in post-training, but maybe the compute spent for getting test-time compute to work for a model is going to dwarf pre-training eventually. So we don't even know if o1 is using just chain-of-thought RL. We don't know how they're using any of these. We don't know anything. It's fun to speculate. If you were to build a competing model, what would you do?

是的，所以要做的一件事是，我认为你可能需要训练一个过程奖励模型。所以也许我们可以深入探讨奖励模型、结果奖励模型与过程奖励模型。结果奖励模型是人们为语言建模训练的传统奖励模型，它只看最终结果。

Yeah, so one thing to do would be, I think you probably need to train a process reward model. So maybe we can get into reward models and outcome reward models versus process reward models. Outcome reward models are the kind of traditional reward models that people train for language modeling, and it's just looking at the final thing.

你在做一道数学题。我们看看你做的最后一步，给它打个分。我们觉得这个结果有多好？奖励是多少？过程奖励模型则是尝试给思维链打分。OpenAI 去年夏天有一篇初步论文，他们用人类标注员创建了一个几十万条思维链的数据集。最终，我觉得除了用它来影响我们在多个样本之间的选择之外，我没看到人们用过程奖励模型做出什么有趣的事。所以这些论文里大家做的都是：从语言模型中采样一堆输出，然后用过程奖励模型给这些生成打分，可能还结合一些其他启发式方法，然后用它来选出最佳答案。真正有趣且人们希望实现的是用这些过程奖励模型做树搜索，因为如果你真的能给思维链的每一步打分，你就可以分支出去，探索思维链的多条路径，然后用这些过程奖励模型来评估你走的这条分支有多好。

You're doing some math problem. Let's look at that final thing you've done everything and let's assign a grade to it. How likely we think, like, what's the reward for this outcome? Process reward models instead try to grade the chain of thought. OpenAI had some preliminary paper on this last summer where they use human labelers to get a pretty large several hundred thousand data set of creating chains of thought. Ultimately, it feels like I haven't seen anything interesting in the ways that people use process reward models outside of just using it as a means of affecting how we choose between a bunch of samples. So what people do in all these papers is they sample a bunch of outputs from the language model and then use the process reward models to grade all those generations alongside maybe some other heuristics, and then use that to choose the best answer. The really interesting thing that people think might work and want to work is tree search with these process reward models, because if you really can grade every single step of the chain of thought, then you can kind of branch out and explore multiple paths of this chain of thought, and then use these process reward models to evaluate how good is this branch that you're taking.

是的，当分支的质量与最终结果的质量强相关时，你就能很好地知道该走哪条分支，不仅是短期，长期也是如此。我认为已经完成的有趣工作是弄清楚如何正确训练过程奖励模型。那些开源且人们讨论的有趣工作是如何以更自动化的方式训练过程奖励模型。我可能说错了，没提到一些论文。我还没看到什么特别有效的方法，能创造性地用过程奖励模型做树搜索和代码。

Yeah, when the quality of the branch is somehow strongly correlated with the quality of the outcome at the very end, so you have a good model of knowing which branch to take, not just in the short term but in the long term. The interesting work that I think has been done is figuring out how to properly train the process reward models. The interesting work that has been open-sourced and people talk about is how to train the process reward models in a more automated way. I could be wrong here, not mentioning some papers. I haven't seen anything super that seems to work really well for using the process reward models creatively to do tree search and code.

这有点像 AI 安全，也许有点哲学问题。OpenAI 说他们向用户隐藏了思维链，而且他们说这是一个艰难的决定。他们没有展示思维链，而是让模型总结思维链。他们还在后台说会监控思维链，确保模型不会试图操纵用户，这是一个迷人的可能性。但不管怎样，你对隐藏思维链怎么看？

This is kind of an AI safety, maybe a bit of a philosophy question. So OpenAI says that they're hiding the chain of thought from the user, and they've said that that was a difficult decision to make. Instead of showing the chain of thought, they're asking the model to summarize the chain of thought. They're also in the background saying they're going to monitor the chain of thought to make sure the model is not trying to manipulate the user, which is a fascinating possibility. But anyway, what do you think about hiding the chain of thought?

OpenAI 的一个考虑——这完全是猜测——可能是他们想让人们难以从他们的模型中蒸馏出这些能力。如果你能访问那个隐藏的思维链，复制技术可能更容易，因为那是非常重要的数据——看到模型得出最终结果所采取的步骤，所以你也可以用它来训练。这与一些大型语言模型提供商的情况类似。这也是猜测，但有些 API 曾经提供对生成 token 的对数概率的轻松访问，还有提示 token 的对数概率。然后一些 API 取消了这些。再次，完全是猜测，但一个想法是取消这些的原因是，如果你能访问对数概率，类似于这个隐藏的思维链，那会给你更多信息来尝试从这些 API、从这些最大的模型中蒸馏出能力，到你控制的模型中。

One consideration for OpenAI, and this is completely speculative, could be that they want to make it hard for people to distill these capabilities out of their model. It might actually be easier if you had access to that hidden chain of thought to replicate the technology, because that's pretty important data—seeing the steps the model took to get to the final result, so you can probably train on that also. There was a mirror situation with some of the large language model providers. Also this is speculation, but some of these APIs used to offer easy access to log probabilities for the tokens they're generating, and also log probabilities over the prompt tokens. Then some of these APIs took those away. Again, complete speculation, but one thought is that the reason those were taken away is if you have access to log probabilities, similar to this hidden chain of thought, that can give you even more information to try and distill these capabilities out of the APIs, out of these biggest models, into models you control.

作为之前关于我们集成 o1 的讨论的一个补充，我认为我们还在学习如何使用这个模型。所以我们在 Cursor 中提供了 o1，因为我们拿到模型时真的很想试试。我想很多程序员也会有兴趣试试。但 o1 还不是默认 Cursor 体验的一部分，我们还没找到一种方式把它集成到编辑器中，让我们每小时甚至每天都会用到。所以我认为如何使用这个模型还没有定论。我们还没看到有人发布的东西能明确说‘哦，这就是用例’。一个明显的方向是，也许这能让你更容易地运行后台任务，让模型处于循环中，让模型具有智能体特性。但我们还在探索。明确地说，我们有想法，只是需要在发布之前尝试做出一些非常有用的东西。但它有这些显著的局限性。即使不谈能力，它也不支持流式输出，这意味着在你想监督输出的场景下使用起来非常痛苦，你只能等着整段文字出现。而且，这感觉就像是测试时算力搜索的早期阶段，非常像 v0，有很多地方感觉不太对劲。我猜想，在人们增加预训练数据量、模型规模和预训练技巧的同时，你现在会有另一条线索，就是让搜索越来越好。

As an asterisk on the previous discussion about us integrating o1, I think that we're still learning how to use this model. So we made o1 available in Cursor because when we got the model, we were really interested in trying it out. I think a lot of programmers are going to be interested in trying it out. But o1 is not part of the default Cursor experience in any way yet, and we still haven't found a way to integrate it into the editor in a way that we reach for sort of every hour, maybe even every day. So I think that the jury's still out on how to use the model. We haven't seen examples yet of people releasing things where it seems really clear, like 'oh, that's the use case.' The obvious one to turn to is maybe this can make it easier for you to have these background things running, to have these models in loops, to have these models be agentic. But we're still discovering. To be clear, we have ideas, we just need to try and get something incredibly useful before we put it out there. But it has these significant limitations. Even barring capabilities, it does not stream, and that means it's really painful to use for things where you want to supervise the output, and instead you're just waiting for the wall of text to show up. Also, it does feel like the early innings of test-time compute search, where it's very much a v0, and there are so many things that don't feel quite right. I suspect, in parallel to people increasing the amount of pre-training data and the size of the models and pre-training and finding tricks there, you'll now have this other thread of getting search to work better and better.

那么让我问问你关于 Strawberry 的事。看起来 GitHub Copilot 可能会以某种方式集成 o1，我想有些评论说这意味着 Cursor 完了。我记得看到一条评论说‘是时候关闭 Cursor 了’。那么，是时候关闭 Cursor 了吗？

So let me ask you about Strawberry tomorrow eyes. It looks like GitHub Copilot might be integrating o1 in some kind of way, and I think some of the comments are saying this means Cursor is done. I think I saw one comment saying 'time to shut down Cursor.' So is it time to shut down Cursor?

我认为这个领域与 2010 年代的软件领域有些不同，这里的上限真的非常高。所以我认为 3 到 4 年后最好的产品会比今天最好的产品有用得多。你可以大谈护城河和品牌，这是我们的优势，但我认为最终，如果你不在产品上创新，你就会输。这对初创公司也很好，对试图进入这个市场的人也很好，因为这意味着你有机会通过构建更好的东西来战胜已经拥有大量用户的人。所以我认为未来几年，关键就是构建最好的产品，构建最好的系统，这既取决于模型引擎方面，也取决于编辑体验。

I think this space is a little bit different from past software spaces over the 2010s, where I think that the ceiling here is really, really, really incredibly high. So I think that the best product in 3 to 4 years will just be so much more useful than the best product today. You can wax poetic about moats and brand and this is our advantage, but I think in the end, if you don't innovate on the product, you will lose. And that's also great for startups, great for people trying to enter this market, because it means you have an opportunity to win against people who have lots of users already by just building something better. So I think over the next few years, it's just about building the best product, building the best system, and that both comes down to the modeling engine side of things and also the editing experience.

是的，我认为 Cursor 相比其他产品的额外价值不仅仅在于快速集成像 o1 这样的新模型；它来自于这些定制模型的深度。

Yeah, I think most of the additional value from Cursor versus everything else out there is not just integrating the new model fast like o1; it comes from all of the kind of depth that goes into these custom models.

你提到你对合成数据有一个分类法。能解释一下吗？

You mentioned you have a taxonomy of synthetic data. Can you please explain?

是的，我认为合成数据主要有三种。首先，什么是合成数据？普通数据是自然产生的，通常来自人类活动。合成数据：第一种是蒸馏。让语言模型输出词元或词元上的概率分布，然后用这些数据训练一个能力较弱的模型。这种方法不会让你得到比原始模型更强的模型，但有助于将昂贵、高延迟模型的能力蒸馏到更小、特定任务的模型中。第二种是当问题的一个方向比反向更容易时。一个很好的例子是错误检测：引入看起来合理的错误比检测它们容易得多。你可以用一个不太聪明的模型在代码中引入错误，然后用这些合成数据训练一个擅长检测错误的模型。最后一种是用语言模型生成易于验证的文本。例如，如果你有一个能检测语言是否达到莎士比亚水平的验证系统，再加上一群猴子打字，你最终可以得到足够多的训练数据来训练一个莎士比亚水平的语言模型。这在数学领域尤其如此，因为形式语言的验证非常容易。你可以用一个还不错的模型生成大量输出，然后选择那些证明了基本定理的输出，并进一步训练。对于 LeetCode 问题之类的代码任务也可以这样做：如果你有测试用例可以验证解决方案，就可以在通过测试的输出上进行训练。要让这在所有领域都有效会很棘手，因为对于开放式任务来说，拥有一个完美的验证器非常困难。

Yeah, I think there are three main kinds of synthetic data. First, what is synthetic data? Normal data is naturally created, usually from humans doing things. Synthetic data: the first kind is distillation. Having a language model output tokens or probability distributions over tokens, and then you train a less capable model on this. This approach won't get you a net more capable model than the original one, but it's useful for distilling capabilities from an expensive, high-latency model into a smaller, task-specific model. The second kind is when one direction of the problem is easier than the reverse. A great example is bug detection: it's a lot easier to introduce reasonable-looking bugs than to detect them. You can get a not-so-smart model to introduce bugs in code, then use that synthetic data to train a model good at detecting bugs. The last category is producing text with language models that can be easily verified. For example, if you have a verification system that can detect if language is Shakespeare-level, and you have monkeys typing, you can eventually get enough training data to train a Shakespeare-level language model. This is very much the case for math, where verification is really easy for formal language. You can have an okay model generate a ton of rollouts and choose the ones that prove ground truth theorems and train further. Similar things can be done for code with LeetCode problems: if you have tests that verify a solution, you can train on outputs that pass the tests. It's going to be tricky getting this to work in all domains, because having a perfect verifier for open-ended tasks is hard.

那带反馈的强化学习呢，RLHF 与 RL 相比？它们在提升模型性能方面扮演什么角色？

What about RL with feedback, RLHF versus RL? What is the role of that in getting better performance on the models?

RLHF 是指奖励模型是根据人类反馈收集的标签训练的。如果你能获得大量针对该任务的人类反馈，这种方法就有效。RLHF 很有趣，因为它依赖于验证比生成更容易这一约束。你是在用语言模型查看输出并改进模型。如果语言模型验证解决方案比生成它容易得多，那么这种方法可能有效。我们做的另一件事是 RL 和 RLHF 的混合：在 Cursor Tab 中，当在两个可能的生成结果之间进行选择时，模型通常已经很正确，只需要少量（大约 50-100 个）人类示例来微调，使其先验与你的期望对齐。这与常规的 RL 不同，常规 RL 通常需要大量示例来训练奖励模型。

RLHF is when the reward model is trained from labels collected from human feedback. This works if you can get a ton of human feedback for the task. RLHF is interesting because it depends on the constraint that verification is easier than generation. You're using a language model to look at outputs and improve the model. It may work if the language model has a much easier time verifying a solution than generating it. Another thing we do is a mix of RL and RLHF: in Cursor Tab, when picking between two possible generations, the model is often quite correct and just needs a little human nudging with only 50-100 examples to align the prior with what you want. That looks different from normal RL, where you usually train reward models on tons of examples.

当你比较生成和验证，或者生成和排序时，你的直觉是什么？排序比生成容易得多吗？

What's your intuition when you compare generation and verification, or generation and ranking? Is ranking way easier than generation?

我的直觉是肯定的。这又回到了如果你相信 P 不等于 NP，那么有一大类问题在给定证明的情况下验证起来比实际证明要容易得多。我想知道同样的事情是否会证明 P 不等于 NP 或 P 等于 NP。那将非常酷。无论谁因此获得菲尔兹奖……我其实非常好奇，对于 AI 是否会先获得菲尔兹奖，有什么好的赌注。

My intuition would say yes. This goes back to if you believe P does not equal NP, then there's a massive class of problems that are much easier to verify given a proof than actually proving it. I wonder if the same thing will prove P not equal to NP or P equal to NP. That would be really cool. Whoever gets the Fields Medal for that... I'm actually surprisingly curious about what a good bet is for whether AI will get the Fields Medal first.

另一个开放的哲学问题。我其实非常好奇，对于 AI 是否会先获得菲尔兹奖，有什么好的赌注？我不知道好的赌注是什么。诺贝尔奖还是菲尔兹奖？我认为菲尔兹奖会先出现。

Another open philosophical question. I'm actually surprisingly curious what a good bet is for one: will AI get the Fields Medal first? I don't know what a good bet is. Nobel Prize or Fields Medal? I think Fields Medal comes first.

你当然会这么说。但这也是一个可以验证的孤立系统。我觉得达到 IMO 的路径更清晰一些，因为它已经能解决一些 IMO 问题，而且根据当时的文献，有很多容易摘的果实。对于解决真正困难的开放问题，我们离得有多近，我的直觉要少得多。所以你认为菲尔兹奖会先出现？

You would say that, of course. But it's also an isolated system you can verify. I felt like the path to get to IMO was a little more clear because it could already get a few IMO problems and there were a bunch of low-hanging fruit given the literature at the time. I have much less intuition about how close we are to solving really hard open problems. So you think Fields Medal first?

不会像物理学那样……哦，100%？我觉得，我觉得那可能更有可能。很可能它会被解决。嗯，是的。我认为它会像 BSD 猜想（伯奇和斯温纳顿-戴尔猜想）、黎曼假设，或者任何一个真正困难的数学问题。我们甚至不清楚通往解决方案的路径是什么。我们连路径长什么样都不知道，更不用说……

It won't be like in physics or in... oh, 100%? I think, I think that's probably more likely. It's probably much more likely that it'll get in. Yeah, yeah, yeah. Well, I think it goes to something like BSD, which is the Birch and Swinnerton-Dyer conjecture, or the Riemann hypothesis, or any one of these hard math problems which are just really hard. It's sort of unclear what the path to even a solution looks like. We don't even know what a path looks like, let alone...

你不认同这是一个孤立系统，并且实际上可以有一个好的奖励系统这种想法吗？感觉训练起来更容易。

And you don't buy the idea that this is like an isolated system and you can actually have a good reward system? It feels like it's easier to train for that.

我认为我们可能在 AGI 之前就拿到菲尔兹奖。我会非常高兴，非常高兴，但我不确定……我觉得 2027 年或 2030 年感觉像菲尔兹奖？考虑到事情发展得这么快，感觉像是很久以后。

I think we might get a Fields Medal before AGI. I think I'd be very happy, very happy, but I don't know if I... I think 2027 or 2030 feels like a Fields Medal? It feels like forever from now, given how fast things have been going.

说到事情发展得有多快，我们来谈谈缩放定律。对于不了解的人来说，也许最好谈谈缩放定律这个概念：它们是什么，现状如何，你认为未来会怎样？

Speaking of how fast things have been going, let's talk about scaling laws. For people who don't know, maybe it's good to talk about this whole idea of scaling laws: what are they, where do things stand, and where do you think things are going?

我觉得很有意思：OpenAI 最初的缩放定律论文因为学习率调度的一些问题而略有错误。然后 Chinchilla 展示了一个更正确的版本，从那以后人们又有点偏离了算力最优的做法，因为现在人们开始更多地优化，以便在给定的推理预算下让模型工作得非常好。我认为这些曲线有更多维度，比我们最初使用的算力、参数数量和数据更多。推理算力是显而易见的。我认为上下文窗口是另一个明显的维度。所以如果你关心，比如说，推理算力和上下文窗口这两件事，也许你想训练的是一种 SSM，因为它们在超长上下文下更便宜、更快。即使在训练期间缩放特性差 10 倍，意味着你需要花费 10 倍多的算力来训练模型以达到相同的能力水平，这也是值得的，因为你最关心的是超长上下文窗口的推理预算。所以看看人们如何利用所有这些维度会很有趣。

I think it was interesting: the original scaling laws paper by OpenAI was slightly wrong because of some issues they did with learning rate schedules. Then Chinchilla showed a more correct version, and from then people have again kind of deviated from doing the compute-optimal thing because people now start optimizing more for making the thing work really well given an inference budget. I think there are a lot more dimensions to these curves than what we originally used: just compute, number of parameters, and data. Inference compute is the obvious one. I think context length is another obvious one. So if you care about, let's say, the two things of inference compute and context window, maybe the thing you want to train is some kind of SSM because they're much cheaper and faster at super long context. Even if it is 10x worse scaling properties during training, meaning you have to spend 10x more compute to train the thing to get the same level of capabilities, it's worth it because you care most about that inference budget for really long context windows. So it'll be interesting to see how people kind of play with all these dimensions.

所以你谈到了多个维度。显然最初的概念只是看模型大小（以参数衡量）和数据大小（以 token 数量衡量）这两个变量，以及它们的比例。有一个数字，或者至少一个最小值，这似乎是一个引人注目的概念，而且似乎有一个正在出现。你仍然相信某种“越大越好”吗？

So you speak to the multiple dimensions. Obviously the original conception was just looking at the variables of the size of the model as measured by parameters and the size of the data as measured by the number of tokens, and looking at the ratio of the two. It's kind of a compelling notion that there is a number, or at least a minimum, and it seems like one was emerging. Do you still believe that there is a kind of bigger is better?

我的意思是，更大的模型在原始性能和原始智能上肯定更好。我认为人们可能采取的路径是……我特别看好蒸馏。你可以调节多少旋钮？如果我们花大笔钱训练，得到最有能力的廉价模型，尽可能多地关心。关于推理时间算力，人们已经用 Llama 模型做了天真的版本：只是在远多于最优数量的 token 上过度训练 7B 模型。但如果你真的关心，也许该做的是 Gemma 所做的：不只是训练 token，而是真正训练最小化与 Gemma 27B 分布的 KL 散度。所以是知识蒸馏。你花费算力真正在这个 270 亿参数模型上训练所有这些 token，只是为了得到这个更小的模型。蒸馏提供了更快的模型；更小意味着更快。蒸馏在理论上是从训练数据中获取更多信号。这也许是部分帮助解决数据墙的另一种方式，因为你只有这么多数据可以训练。让我们在这个非常大的模型上训练所有这些 token，然后将其蒸馏到这个小模型中，也许我们可以为这个小得多的模型每个 token 获得比原本训练它时更多的信号。

I mean, bigger is certainly better for just raw performance and raw intelligence. I think the path that people might take is... I'm particularly bullish on distillation. How many knobs can you turn? If we spend a ton of money on training to get the most capable cheap model, really caring as much as you can. The naive version of caring as much as you can about inference time compute is what people have already done with the Llama models: just overtraining the 7B models on way more tokens than is optimal. But if you really care about it, maybe the thing to do is what Gemma did: let's not just train on tokens, let's literally train on minimizing the KL divergence with the distribution of Gemma 27B. So knowledge distillation there. You're spending the compute of literally training this 27 billion parameter model on all these tokens just to get out this smaller model. Distillation gives a faster model; smaller means faster. Distillation, in theory, is getting more signal from the data that you're training on. It's perhaps another way of partially helping with the data wall, where you only have so much data to train on. Let's train this really big model on all these tokens and distill it into this smaller one, and maybe we can get more signal per token for this much smaller model than we would have originally if we trained it.

那么如果我给你 1 万亿美元，你会怎么花？我的意思是，你不能买岛之类的。你会如何分配，用于改进大模型，还是可能用于 RLHF？

So if I gave you $1 trillion, how would you spend it? I mean, you can't buy an island or whatever. How would you allocate it in terms of improving the big model versus maybe paying for RLHF?

我认为关于训练这些大模型有很多秘密和细节，我不知道，只有大型实验室才知道。问题是，如果我尝试，我会浪费很多钱，因为我不知道那些东西。暂且搁置怀疑，假设你有专业知识，或者如果你说你必须用现在有限的信息来操作……不，不，不，实际上我会说，你冲进去，获取所有信息，所有的小特性，所有定义模型如何训练的参量。

I think there's a lot of these secrets and details about training these large models that I just don't know and are only privy to the large labs. The issue is I would waste a lot of that money if I even attempted this because I wouldn't know those things. Suspending a lot of disbelief and assuming like you had the knowhow, or if you're saying you have to operate with the limited information you have now... No, no, no, actually I would say you swoop in and you get all the information, all the little characteristics, all the parameters that define how the thing is trained.

如果我们看未来 5 年如何投资以最大化你所说的原始智能，答案不是很简单吗？你只是尽量获取尽可能多的算力。归根结底，你只需要买 GPU，然后研究人员可以找到所有……他们可以调整你想要大模型还是小模型。

If we look at how to invest money for the next 5 years in terms of maximizing what you called raw intelligence, isn't the answer really simple? You just try to get as much compute as possible. At the end of the day, all you need to buy is the GPUs, and then the researchers can find all the... they can sort of tune whether you want between a big model or a small model.

嗯，这就引出了一个问题：你真的受限于算力和金钱，还是受限于其他东西？我更倾向于 Arvid 的信念，即我们有点受限于想法。但总是有……如果你有大量算力，你可以运行很多实验。所以你会运行很多实验，而不是用那些算力训练一个巨大的模型。我会的，但我确实相信我们在想法上受到限制。我认为即使有所有这些算力和你在世界上能收集的所有数据，你最终真正受限于的甚至不是想法，而是真正优秀的工程。即使有世界上所有的资本，你真的能聚集……世界上没有多少人能真正在这里产生影响。而且研究中有很多工作只是纯粹的、非常困难的工程工作。作为一个非常粗略的例子，如果你看原始的 Transformer 论文，你知道有多少工作是连接这些有趣的概念。

Well, this gets into the question of: are you really limited by compute and money, or are you limited by these other things? I'm more prone to Arvid's belief that we're sort of idea-limited. But there's always that... if you have a lot of compute, you can run a lot of experiments. So you would run a lot of experiments versus using that compute to train a gigantic model. I would, but I do believe that we are limited in terms of ideas that we have. I think even with all this compute and all the data you could collect in the world, you really are ultimately limited by not even ideas but just really good engineering. Even with all the capital in the world, would you really be able to assemble... there aren't that many people in the world who really can make the difference here. And there's so much work that goes into research that is just pure, really hard engineering work. As a very hand-wavy example, if you look at the original Transformer paper, you know how much work was kind of joining together a lot of these interesting concepts.

从文献中的想法到实际编写所有代码，比如 CUDA 内核之类的。我不知道它最初是在 GPU 还是 TPU 上运行的，但要让代码充分利用 GPU 性能。让 Gomes 去写所有这些代码，而 Gomes 可能是世界上最优秀的工程师之一。或者更进一步，下一代模型需要实现模型并行，并在数千甚至数万个 V100 上扩展——我认为 GPT-3 可能就是这样。要让这一切运转起来，需要投入大量的工程努力。如果你真的能把成本降到不是零，而是降低 10 倍，让有绝妙想法的人能非常容易地直接实现他们构想的新架构，并且能在 GPU 上达到 40-50%的利用率，我认为这将极大地加速研究。

Embedded in the literature versus then going in and writing all the code, like maybe the CUDA kernels, maybe whatever else. I don't know if it ran on GPUs or TPUs originally, such that it actually saturated the GPU performance. Getting Gomes to go in and do all this code, and Gomes is probably one of the best engineers in the world. Or maybe going a step further, the next generation of models having these things like getting model parallelism to work and scaling it on thousands or maybe tens of thousands of V100s, which I think GPT-3 may have been. There's just so much engineering effort that has to go into all of these things to make it work. If you really brought that cost down to maybe not zero but just made it 10x easier, made it super easy for someone with really fantastic ideas to immediately get to the version of the new architecture they dreamed up that is getting 40-50% utilization on the GPUs, I think that would just speed up research by a ton.

如果你看到一条清晰的改进路径，你应该总是先摘低垂的果实，对吧？我认为 OpenAI 和其他实验室做了正确的事，先摘了低垂的果实，也就是扩展到 GPT-4 或 GPT-5 的规模，然后不断扩展，效果越来越好。只要一切顺利，就没有必要尝试新想法。你应该继续努力，尽可能榨取更多价值。然后，当你真正需要新想法时——如果你要花 10 万亿美元，你可能需要重新评估你的想法，因为那时你可能已经想法枯竭了。我认为我们都相信，要实现 AGI 可能需要新想法，而且我们也相信存在在较小规模上测试这些想法的方法，并且有相当的信心它们会成功。只是对于实验室来说，在目前的位置上，很难将非常有限的研究和工程人才投入到探索所有这些其他想法上，因为有一个核心的东西可能会在相当长一段时间内持续提升性能。

If you see a clear path to improvement, you should always take the low-hanging fruit first, right? I think OpenAI and all the other labs did the right thing to pick off the low-hanging fruit, where the low-hanging fruit is scaling up to GPT-4 or GPT-5 scale, and you just keep scaling and things keep getting better. As long as everything is working, there's no point experimenting with new ideas. You should bang on and try to get as much juice out as possible. Then maybe when you really need new ideas—if you're spending $10 trillion, you probably want to reevaluate your ideas because you're probably idea-limited at that point. I think all of us believe new ideas are probably needed to get all the way to AGI, and all of us also believe there exist ways of testing out those ideas at smaller scales and being fairly confident that they'll play out. It's just quite difficult for the labs in their current position to dedicate their very limited research and engineering talent to exploring all these other ideas when there's this core thing that will probably improve performance for some decent amount of time.

是的，但这些大实验室正在获胜，所以他们只管放手去做。

Yeah, but also these big labs are winning, so they're just going wild.

好的，那么展望未来的大问题。你现在处于编程世界的中心。你认为编程的本质在未来几个月、一年、两年、五年、十年会如何变化？

Okay, so big question looking out into the future. You're now at the center of the programming world. How do you think the nature of programming changes in the next few months, in the next year, in the next two years, the next 5 years, 10 years?

我们对一个程序员长期处于主导地位的未来感到非常兴奋。你们听过我们谈过一点，但重点是强调程序员的速度和自主性，以及控制力——能够修改任何你想修改的东西，能够非常快速地迭代你正在构建的东西。我认为这与一些人在这个领域的想法有些不同。一个吸引人的想法是：你能和你的电脑对话，让它为你构建软件，就像你在 Slack 上和一个工程部门或工程师交谈一样，它只是一个孤立的文本框。我们对此不兴奋的部分原因是之前谈到的延迟问题，但很大一部分原因是这会让你放弃很多控制权。在文本框中说话时很难做到非常具体，如果你只是像和工程部门沟通那样与一个东西交流，你实际上是把大量重要决策交给了那个东西。这从根本上触及了工程是什么。我认为一些离工程较远的人可能会认为：规格完全写好了，然后工程师来执行，只是用代码实现而已。但我认为很多最好的工程——我们享受的工程——涉及大量关于你到底在构建什么的微观决策，以及关于速度、成本和系统中所有其他因素的艰难权衡。只要人类实际上是设计软件并指定他们想要构建什么的人——而不是一个完全由 AI 运营的公司——我们认为你会希望人类坐在驾驶座上做出这些决策。所以具体是什么样子还没有定论。一个奇怪的想法是：你可以控制查看代码库的抽象层次，你可以指向代码库的特定部分。也许你通过伪代码的形式来消化一个代码库，你实际上也可以编辑那个伪代码，然后让更改在正式的编程层面实现。你保留了编程的文本编辑组件，保留了控制权，你甚至可以深入到代码中，也可以上升到更高的抽象层次，同时获得巨大的生产力提升。如果你能在抽象栈上下移动，那就太好了。还有很多细节需要解决；这只是一个模糊的想法。时间会证明它是否真的有效，但我们认为这些原则——人类在驾驶座上的控制力和速度——非常重要。我们认为对于某些事情，比如 Arvid 之前提到的，对于某些编程风格，你可以像聊天一样交给它——如果你有一个定义得很明确的 bug——但那不是编程的大部分，也不是我们认为很多人重视的编程。

I think we're really excited about a future where the programmer is in the driver's seat for a long time. You've heard us talk about this a little bit, but one that emphasizes speed and agency for the programmer, and control—the ability to modify anything you want to modify, the ability to iterate really fast on what you're building. This is a little different, I think, than where some people are jumping to in the space. One idea that's captivated people is: can you talk to your computer, can you have it build software for you as if you're talking to an engineering department or an engineer over Slack, and can it just be this sort of isolated text box? Part of the reason we're not excited about that is some of the stuff we've talked about with latency, but a big piece is that it comes with giving up a lot of control. It's much harder to be really specific when you're talking in a text box, and if you're necessarily just going to communicate with a thing like you would be communicating with an engineering department, you're actually abdicating tons of really important decisions to the spot. This kind of gets at fundamentally what engineering is. I think some people who are a little bit more removed from engineering might think of it as: the spec is completely written out, and then the engineers just come and implement, and it's just about making the thing happen in code. But I think a lot of the best engineering—the engineering we enjoy—involves tons of tiny micro-decisions about what exactly you're building, and about really hard trade-offs between speed, cost, and all the other things involved in a system. We want, as long as humans are actually the ones designing the software and specifying what they want to be built—and it's not just a company run by all AIs—we think you'll really want the human in the driver's seat dictating these decisions. So there's the jury still out on what that looks like. One weird idea for what that could look like is you can control the level of abstraction you view a codebase at, and you can point at specific parts of a codebase. Maybe you digest a codebase by looking at it in the form of pseudocode, and you can actually edit that pseudocode too, and then have changes get made down at the formal programming level. You keep the inflow text editing component of programming, you keep the control, you can even go down into the code, you can go at higher levels of abstraction while also giving you these big productivity gains. It would be nice if you can go up and down the abstraction stack. There are a lot of details to figure out there; that's sort of a fuzzy idea. Time will tell if it actually works, but these principles of control and speed with the human in the driver's seat we think are really important. We think for some things, like Arvid mentioned before, for some styles of programming you can kind of hand it off chat-style—if you have a bug that's really well specified—but that's not most of programming, and that's also not most of the programming we think a lot of people value.

那么编程的基本技能呢？现在有很多人，比如年轻人，有点害怕，因为他们热爱编程，但担心如果选择这条职业道路，是否还能有未来。你认为编程这项技能本身会发生根本性的变化吗？

What about the fundamental skill of programming? There are a lot of people, like young people, right now kind of scared, thinking because they love programming but they're scared about whether they will be able to have a future if they pursue this career path. Do you think the very skill of programming will change fundamentally?

我实际上认为现在是构建软件非常非常激动人心的时刻。是的，我们记得 2013 年、2012 年编程是什么样子，那时有更多的冗余和样板代码，还要查一些很棘手的东西。这些东西仍然存在，肯定没有消失，但今天的编程已经……

I actually think this is a really, really exciting time to be building software. Yeah, like we remember what programming was like in 2013, 2012, whatever it was, and there was just so much more cruft and boilerplate, and looking up something really gnarly. That stuff still exists, it's definitely not at zero, but programming today is...

比过去有趣多了。我们真的在深入挖掘那些吸引人们编程的核心要素，比如快速构建东西的能力、速度，还有个人掌控感。所有这些都被大大增强了。所以我认为对于构建软件的人来说，这将是一个非常非常有趣的时期。我认为技能可能也会改变。人们的品味和创意想法会被放大，而关于样板代码编辑的部分会减少，甚至可能对谨慎性的要求也会降低——这在今天对程序员来说非常重要。我认为这会更有趣。你们觉得呢？

Way more fun than back then. It's like we're really getting down to the delight concentration and all the things that really draw people to programming, like this element of being able to build things really fast and speed and also individual control. All those are just being turned up a ton. So I think it's just going to be a really, really fun time for people who build software. I think that the skills will probably change too. I think that people's taste and creative ideas will be magnified, and it will be less about boilerplate text editing, maybe even a little bit less about carefulness, which I think is really important today if you're a programmer. I think it'll be a lot more fun. What do you guys think?

我同意。我很兴奋能够改变。最近发生的一件事是，我们想对代码库进行一次较大的迁移。我们在 Node.js 中使用了异步本地存储，这已知性能不佳，我们想迁移到我们的上下文对象。这是一个大迁移，影响整个代码库。我花了大概五天时间，即使有今天的 AI 工具。我真的很期待这样一个未来：我只需展示几个例子，然后 AI 将其应用到所有位置，然后它高亮显示‘哦，这是一个新例子，我该怎么办？’然后我展示具体该怎么做，然后这可以在 10 分钟内完成。然后你可以迭代得快得多。你不需要事先想太多，站在黑板前仔细思考我们到底要怎么做，因为成本太高了。但你可以先尝试一下，然后意识到‘哦，这其实不是我想要的’，然后立即再次更改。所以是的，我认为未来当程序员会非常有趣。

I agree. I'm very excited to be able to change. Just one thing that happened recently was we wanted to do a relatively big migration to our codebase. We were using async local storage in Node.js, which is known to be not very performant, and we wanted to migrate to our context object. This is a big migration; it affects the entire codebase. And I spent, I don't know, five days working through this even with today's AI tools. And I am really excited for a future where I can just show a couple of examples and then the AI applies that to all of the locations, and then it highlights, 'Oh, this is a new example, what should I do?' and then I show exactly what to do there, and then that can be done in like 10 minutes. And then you can iterate much, much faster. Then you don't have to think as much up front and stand at the blackboard and think exactly how are we going to do this because the cost is so high, but you can just try something first and you realize, 'Oh, this is not actually exactly what I want,' and then you can change it instantly again after. So yeah, I think being a programmer in the future is going to be a lot of fun.

是的，我真的很喜欢你提到的这一点：很多时候编程有两种方式。一种是事先非常仔细地思考最佳方案，然后用有限的工程时间去实现。但我更喜欢直接进入代码，尝试看看它如何布局，然后快速迭代。这感觉更有趣。是的，生成样板代码很好，这样你就可以专注于困难的设计决策。迁移，我觉得这是一个很酷的例子。看起来大型语言模型基本上能够从一种编程语言翻译到另一种，或者像迁移那样的一般意义上的翻译。但那是当前的情况。所以恐惧在于，随着这些模型越来越好，你做的创造性决策越来越少。它会不会发展到你是在自然语言的设计空间中操作，自然语言成为主要编程语言？我想通过建议来问：如果现在有人对编程感兴趣，你认为他们应该学什么？比如你们开始学了一些 Java 和 PHP、Objective-C。我的意思是，最终我们都知道 JavaScript 会赢，而不是 TypeScript，它就像原生的 JavaScript，它会吞噬世界，也许还有一点 PHP。这也引出了一个问题，我认为高德纳有一个想法，即人口中有一部分是极客，编程需要一种特定的心理。感觉越来越多地，能够做好编程的人的类型可能会扩大。我认为不同的人编程的原因不同，但我认为真正最好的程序员是那些真正热爱编程的人。例如，我们团队中有些人下班回家后，他们会启动 Cursor，然后整晚编写他们的副项目，一直待到凌晨三点。

Yeah, I really like that point about it feels like a lot of the time with programming there are two ways you can go about it. One is you think really hard carefully upfront about the best possible way to do it, and then you spend your limited time of engineering to actually implement it. But I much prefer just getting in the code and taking a crack at seeing how it kind of lays out and then iterating really quickly on that. That feels more fun. Yeah, like just speaking to generating the boilerplate is great so you just focus on the difficult design decisions. Migration, I feel like this is a cool one. It seems like large language models able to basically translate from one programming language to another or translate like migrate in the general sense of what migrate is. But that's in the current moment. So I mean the fear has to do with like okay as these models get better and better, then you're doing less and less creative decisions. And is it going to kind of move to a place where you're operating in the design space of natural language where natural language is the main programming language? And I guess I could ask that by way of advice: if somebody's interested in programming now, what do you think they should learn? Like you guys started in some Java and PHP, Objective-C. I mean in the end we all know JavaScript is going to win, and not TypeScript, it's just going to be like vanilla JavaScript, it's just going to eat the world, and maybe a little bit of PHP. And it also brings up the question of, I think Don Knuth has this idea that some portion of the population is geeks and there's a particular kind of psychology in mind required for programming. And it feels like more and more that expands the kind of person that should be able to do great programming might expand. I think different people do programming for different reasons, but I think the true best programmers are the ones that really love just absolutely love programming. For example, there are folks in our team who literally when they get back from work, they go and then they boot up Cursor and then they start coding on their side projects for the entire night and they stay till 3:00 a.m.

当程序员感到沮丧时，他们会说‘我真的需要写代码’。我认为这种对编程的痴迷和热爱造就了最优秀的程序员。这类人会深入钻研事物运作的细节。我想问的是：正是这样的程序员，当超级 Tab 键大获成功时，他们会不停地按 Tab 键。团队里那个人比任何人都更喜欢 Cursor Tab，对吧？

Doing that, and when they're sad, they said, 'I just really need to code.' And I think there's that level of programmer where this obsession and love of programming makes the best programmers. These types of people will really get into the details of how things work. I guess the question I'm asking: that exact programmer, I think about that person when the super tab, the super awesome praise be the tab, succeeds. You keep pressing tab. That person in the team loves Cursor Tab more than anybody else, right?

是的，而且这不仅仅是按 Tab 键那么简单。这只是个口号式的说法。但实际上，当你按 Tab 键时，你一直在注入意图。有时你会拒绝它，有时你会多打几个字符。这就是你塑造所创造之物的方式。我认为编程将发生很大变化，变成‘你想做什么？’这种更高带宽的与计算机的沟通。打字比传达意图的带宽要低得多。

Yeah, and it's not just about pressing tab. That's the easy way to say it, the catchphrase. But what you're actually doing when you're pressing tab is injecting intent all the time. Sometimes you're rejecting it, sometimes you're typing a few more characters. That's the way you shape what's being created. I think programming will change a lot to just 'what is it that you want to make?' It's higher bandwidth communication to the computer. Typing is much lower bandwidth than communicating intent.

这引出了你的宣言《工程天才》。我们是一个应用研究实验室，致力于构建非凡高效的人机系统。谈到这种混合元素，我们正在构建未来的工程师：一个比任何单一工程师都高效一个数量级的人机程序员。这个混合工程师将能毫不费力地掌控代码库，没有低熵的击键。即使在最复杂的系统中，他们也能以判断的速度迭代，结合人工智能和人类的智慧。他们将智胜并超越最优秀的纯 AI 系统。我们是一群研究人员和工程师。我们构建软件和模型，在有用与可能的边界上进行发明。我们的工作已经改善了数十万程序员的生活，并且在此过程中，至少会让编程变得更有趣。所以今天感谢你的分享。

This goes to your manifesto titled 'Engineering Genius.' We are an applied research lab building extraordinary productive human-AI systems. Speaking to this hybrid element, we're building the engineer of the future: a human-AI programmer that's an order of magnitude more effective than any one engineer. This hybrid engineer will have effortless control over their code base and no low-entropy keystrokes. They will iterate at the speed of their judgment even in the most complex systems, using a combination of AI and human ingenuity. They will outsmart and out-engineer the best pure AI systems. We are a group of researchers and engineers. We build software and models to invent at the edge of what's useful and what's possible. Our work has already improved the lives of hundreds of thousands of programmers, and on the way to that, will at least make programming more fun. So thank you for talking today.

谢谢。感谢邀请我们。

Thank you. Thanks for having us.

谢谢。感谢收听本期与 Michael、Swall、Arvid 和 Aman 的对话。要支持本播客，请查看描述中的赞助商。最后，我想用我在 Reddit 上看到的一段随机、有趣且可能深刻的编程代码来结束：‘没有什么比一个有效的临时解决方案更持久了。’感谢收听，期待下次再见。

Thank you. Thank you for listening to this conversation with Michael, Swall, Arvid, and Aman. To support this podcast, please check out our sponsors in the description. And now let me leave you with a random, funny, and perhaps profound programming code I saw on Reddit: 'Nothing is as permanent as a temporary solution that works.' Thank you for listening, and hope to see you next time.