打造 21+智能体 Grok Bot 团队：信任、验证与工程师即管理者

而对于 Grok Bot，我们真的很想从一个全新的角度来看待：如果你的智能体更像一个同事，会是什么样子？于是我开始思考，如何让我的智能体在我睡觉时自动帮我修复 bug。这就是我们早期实验的灵感来源，构建一个更持久的机器人。

And with Grok Bot, we really wanted to take a fresh perspective on what it would look like if your agent was really more like a colleague. And so I started thinking about how I would make it so my agents could just automatically fix bugs for me while I sleep. And so that was the inspiration for some of our earlier experiments with building a more persistent bot.

如果你早期投入很多，我认为在智能体世界里工程师角色的演变是，正如我提到的，你有点像你智能体的工程经理。但我认为现在作为工程师，我们责任的很大一部分也是保持代码库处于一种状态，让产品经理和设计师能够贡献高质量的代码。

If you invest a lot early on, the way I see engineers' roles evolve in the agent world is, as I mentioned, you're sort of like an engineering manager for your agents. But I think a big part of our responsibility now as engineers is also to keep the code base in a condition where product managers and designers can contribute high-quality code.

好的，那么我假设你能看到我的屏幕。

Okay, so I assume you can see my screen.

是的。嗯，没问题。

Yes. Yeah, we're good.

所以是的，今天我认为我使用智能体写代码时的一大主题，我相信很多人也有同样的经历，就是如何信任它？尤其是如果你是一个写了很久代码的工程师，你对做好工程有很多意见和学到的教训。当你看到智能体只是随意发挥、猜测、产生幻觉，自信地声称他们第一百次找到了确凿证据，但实际上并不是真正的问题时，你会失去很多信任。而当你对智能体没有太多信任时，我觉得你真的无法充分利用它们。

So yeah, today I think the big theme for me as I've been using agents to write code, and I'm sure a lot of you have had the same experience as well, is how do you trust it? Especially if you are an engineer that's been writing code for a very long time, you have a lot of opinions and lessons that you've learned about doing good engineering. And when you see agents just winging it and guessing, hallucinating, confidently stating that they found the smoking gun for the hundredth time, but it's actually not the real problem, you lose a lot of trust. And when you don't have much trust in your agents, I feel like you really can't get the most out of them.

对我来说，这和管理有相似之处。所以如果我是一个团队的工程经理，我团队里有一群工程师，而我不信任他们，那么我的操作模式就会像微观管理，对吧？我将不得不花大量时间监督我的下属，检查他们工作做得好不好，是否没有将 bug 发布到生产环境。

And for me, the parallel is like with management. So if I'm an engineering manager for a team, and I have a team of engineers on my team, and I don't trust them, then the mode of operation I'm going to be in is going to be like micro-management, right? I'll have to spend a lot of time looking over my reports' shoulders and checking that they're doing their work well, that they're not shipping bugs to production.

所以我画了这张图，因为它不是一张很科学的图表，但这就是我想象中自己和我在使用智能体过程中的旅程。所以快退到大约一年前，当没有人或没有多少人使用智能体写代码时，我想你会进入这样一种模式：你与一个或几个智能体紧密地处于循环中，你发现自己不断试图理解你的智能体在做什么，你非常非常地处于循环中。你观察每一个输出。你坐在那里提示，你真的无法并行化 beyond that，因为 again，你没有那种信任，对吧？当你甚至不信任一个智能体的输出时，你不可能去 100 个智能体，像生成 100 个智能体。

And so, I drew this chart because it's not a very scientific chart, but this is how I imagine myself and my journey through using agents. So, fast backwards like a year or so, when nobody or not many people were using agents to code, I think you get into this mode where you are very heavily in the loop with one or several, like a handful of agents, and you find yourself just constantly trying to understand what your agents are doing, and you're very, very in the loop. You're watching every single output. You are sitting there prompting, and you really can't parallelize beyond that because again, you don't have that trust, right? You can't go to 100 agents, like spawn 100 agents when you don't even trust the output of one agent.

所以，在过去 5 个月里，我觉得我真的能够攀升这条信任曲线，现在我已经到了这样的地步：我实际上有——这么说听起来有点吓人，让我听起来像个吹牛大王，但我保证我不是——但我现在实际上让我的智能体自动为我合并 PR，这说起来很疯狂，但我今天醒来，有大约 20 个 PR 已经落地，我只是在 main 上审查了它们，它们已经落地了。而且它们很好。

So, over the past 5 months, I feel like I've really been able to ascend this trust curve, and now I'm at the point where I actually have—this sounds kind of scary to say this, and it makes me sound like a slot artist, but I promise I'm not—but I actually have my agents now auto-merging PRs for me, which is like a wild thing to say, but I woke up today, and there were like 20 PRs landed, and I just reviewed them on main, like they were already landed. And they were good.

那么，我是如何达到这一点的？这基本上就是我今天想谈论的原因。再次，如果你有问题，Colin，随时插话。哦，是的，当然我得展示这张图表。

So, how did I get to that point? It's basically why I wanted to talk about today. And again, feel free to jump in if you have questions, Colin. Oh, yeah, of course I got to show this chart.

但是是的，所以我分享这张图表，我认为不是为了炫耀，而是为了展示旅程。所以你可以看到曲线与我能够在 Cursor 落地的贡献大致成反比。所以，我 5 个月前加入。5 个月前，我的第一个月效率不高，因为我在学习代码库。不知道到底发生了什么。随着我对我的智能体越来越有信心，我真的能够提高我的生产力。再次，上个月我发布了 1,000 个 PR，这很荒谬。然后这个月，才 12 号，我已经有将近 800 个 PR 落地。所以速度绝对很高。

But yeah, so this chart I think I'm sharing this chart not to kind of flex, but to kind of show the journey. So you can see the curve sort of inversely matches the contributions I've been able to land at Cursor. So, I joined 5 months ago. And 5 months ago, my first month I was not very productive because I was learning the code base. Didn't know what the heck was going on. And as I got more confident in my agents, I've really been able to ramp up my productivity. And again, last month I shipped 1,000 PRs, which is ridiculous. And then this month, we're only on the 12th, I'm already at almost 800 PRs landed. So the velocity is definitely high.

而你们——我分享很多人肯定会质疑这代码有多少实际上是好的。我认为是的，这绝对值得质疑。但是是的，我认为如果你设置好你的智能体，你绝对可以达到非常相似的水平。所以我要谈谈我们如何做到这一点。

And you—I share a lot of you will definitely be questioning how much of this code is actually good. And I think yeah, that's definitely fair to question. But yeah, I think if you set up your agents well, you can definitely get to a very similar level. And so I'm going to talk about how we do that.

所以对我来说，我想我很好奇，我猜 Colin，你的经验也是，但对我来说，我认为当你与智能体合作时，你应该在工具箱中拥有的最重要的技能是验证。所谓验证，我指的是智能体能够实际运行代码，或获取 CPU 跟踪或堆快照，或打开 iOS 模拟器，无论你的应用程序如何暴露给用户，它都能做同样的事情，真正运行并实际测试和验证它能工作。因为那才是真正闭环的事情。它不保证你的智能体写出好的代码，但它允许它们至少写出正确的代码，这对于能够信任你的智能体来说是一个非常大的进步。

So for me, I think I'm curious like I guess Colin, your experience as well, but for me I think the most important skill that you should have in your toolbox when you work with agents is verification. And by verification I mean the ability for an agent to actually run the code or take CPU traces or heap snapshots or open an iOS simulator, whatever, however your application is exposed to users it can do the same thing and run it for real and actually test and verify it'll work. Because that's the thing that really closes the loop. It doesn't guarantee your agent writes good code but it allows them to at least write correct code which is a really big step forward for being able to trust your agent.

我可以分享一个我们在 Cursor 内部的例子。让我打开这个。让我把这个放大。让我全屏。好了。所以对于 Cursor 的智能体窗口，这实际上是一个有趣的故事。但当我五个月前加入 Cursor 时，他们实际上——嗯，我本来应该加入一个不同的团队。我本来应该加入云智能体团队。但后来因为我在 React 方面有很多经验，而智能体窗口是一个 React 应用程序，我被要求基本上帮助智能体窗口的工作。但并没有很多技能可以帮助我。所以我发现自己，好吧，Agent Swarm 将在一周内发布。好吧，我们有一个非常紧迫的截止日期。然后——我只是坐在那里，好吧，我要打开性能 Chrome DevTools，然后拿一个跟踪，自己看看，试着理解这个火焰图。记住，我才第一周。所以我不知道我在看什么，不知道什么——我的意思是，我有一些想法，但代码库对我来说完全是新的。

I can share one example that we have within Cursor. Let me open this. Let me make this bigger. Let me go full screen. There you go. So for Cursor's agent window, this is actually an interesting story. But when I joined Cursor five months ago, they were actually—well, I was supposed to join a different team. I was supposed to join the cloud agents team. But then since I have a lot of experience working on React and the agents window is a React application, I was asked to basically help out with the agent window work. But there wasn't really a lot of skills to help me. So, I just found myself that okay, Agent Swarm was going to launch in like a week. All right, we have a really tight deadline. And there was—I was just sitting there like okay, I'm going to open up the performance the Chrome DevTools and just take a trace, look at it myself, and try to make sense of this flame graph. And keep in mind, I was just in my first week. So, I had no idea what I was looking at, no idea what—I mean, I had some idea, but the code base was completely fresh to me.

我意识到我的智能体也完全不知道。我会截个图，试着下载 trace 或者发给它，说“嗯，看起来有点像这个”。然后它会自信地声称就是那个东西。接着我试着修复，结果发现根本不是那个东西。所以这个过程非常非常慢。如果你自己做过任何性能优化工作，或者甚至只是用智能体开发却没有验证技能，那么你就是验证者。你是瓶颈。你告诉智能体去做某事，然后它去写一些代码。接着你打开本地的开发构建，然后你开始说“哦，这不行”。然后你得复制粘贴截图或控制台错误之类的。然后你的智能体慢慢地处理这些，试着理解并修复问题。但你一直处在循环中，成为瓶颈。所以真的没有办法并行化。

And I realized my agent had no idea either. I would take a screenshot to try to download trace or send it to it and be like, 'Yeah, it kind of looks like this.' And it would confidently state it's this thing. And then I try to fix that, and it turns out that's not the actual thing. So this was a very, very slow process. And if you've ever done any performance work yourself, or even development with an agent where you don't have a verification skill, you are the verifier. You're the bottleneck. You tell your agent to do something, and then it goes off and writes some code. Then you open up your local dev build, and then you start to say, 'Oh, it doesn't work.' Then you've got to copy-paste screenshots or console errors or whatever. And then your agent slowly works with that and tries to understand it and fix the thing. But then you're constantly in the loop and being a bottleneck. So there's really no way to parallelize.

所以 control glass 算是我为 Cursor 构建的最早的技能之一。顺便说一句，glass 是我们内部使用的 Agent Window 的代号，但其实就是 Cursor 吧。所以这个技能，我猜代码本身非常有趣。你的智能体可以很容易地为你做一个。如果你在构建 Electron 应用或 Web 应用，甚至 iOS 应用，你可以教你的智能体如何使用 Chrome DevTools 协议，或者通过苹果的一些工具来运行模拟器、获取 trace 以及进行程序化控制。所以这真的很有用。

So the control glass goes like one of the first skills I built for Cursor. And glass, by the way, is the code name for Agent Window that we use internally, but it's just Cursor, I guess. And so this skill, I guess the code itself is super interesting. Your agent can very easily make one for you. If you're building an Electron app or a web app or even iOS applications, you can teach your agent how to use the Chrome DevTools protocol or through Apple's utilities for running the simulator and taking traces and controlling programmatically as well. So that's really useful.

但有一件事我其实想谈谈，就是这个东西。readme 在哪里？所以这个技能附带了一个非常独特的特性——或者还不是特性——独特的文件，叫做 feature map。故事是这样的：我构建了这个技能，现在智能体能够实际运行 agent window 并获取 trace 之类的。但它完全不知道 agent window 是什么。所以有人会说“哦，左侧边栏很卡”之类的，或者“右边的 PR 标签页不工作”。然后智能体就会手忙脚乱。它会花很多时间试图查找代码，弄清楚这个功能在哪里？我如何在 UI 上实际到达那里？这使得它基本上完全没用。我会在本地运行这个技能，它会启动一个开发构建，但然后它只是空转——我试着点击这里。它不知道如何到达各个地方。这体验很糟糕。那么是谁在我的屏幕上放箭头？所以是的，这个 feature map 真的很有用，因为它教智能体如何到达你拥有的所有功能。

But one thing I actually want to talk about is this thing. Where is the readme? So this skill comes with this very unique feature—or not feature yet—unique file called a feature map. And so the story then is like I built this skill and so now the agent was able to actually run the agent window and take traces and whatnot. But it had no idea what the agent's window was. So someone would say like, 'Oh, the left sidebar is laggy' or something like that. Or 'The right side the PR tab is not working.' And the agent would just be flailing around. It would spend a lot of time trying to look up the code and figure out where is this feature? How do I actually get to it on the UI? Which made it basically completely useless. I would run this skill locally and it would spawn a dev build, but then it just be churning—I just try to click here. It wouldn't know how to get to things. And it was just an awful experience. So who was putting arrows on my screen? So yeah, this feature map has been really useful because it teaches the agent how to get to all of the features that you have.

在我做的插件 P Sack 里，如果你在谷歌上搜索 P Sack cursor，你会找到它。那个插件里有一个 create verification 技能，它实际上帮助你为自己设置类似的东西，包括 feature map。所以它会探索代码并构建这个初始的 feature map，告诉你的智能体如何到达你拥有的所有不同功能。这非常强大，因为现在有了这些用户报告，你甚至可以映射一个模糊的报告，甚至一张截图。我们在 Cursor 内部有一个 Slack 频道，很多人给我们反馈 agent window 和 walk bar 之类的。通常报告很差，质量很低。我们经常收到一张截图，然后有人只说“？？？”，这是什么？

And in P Sack, the plugin that I've made, if you search for P Sack cursor on Google, you'll find it. But there is a create verification skill in that plugin where it actually helps you set up something like this for yourself, including the feature map. So it will actually explore the code and build up this initial feature map that tells your agent how to get to all the different features that you have. And this is extremely powerful because now that you have these user reports that come in, you can actually map even a vague report or even a screenshot. So we have this internally at Cursor where we have a Slack channel with lots of people giving us feedback on the agent's window and walk bar and whatnot. And often times the report is very bad, like very low quality. Very often we get a screenshot, and then someone just says {question mark} {question mark} {question mark}, like what is this?

没有这个，你的智能体会说“我完全不知道”。但有了这样的 feature map，它就有更多的上下文和理解，知道如何实际导航，如何到达所有不同的功能。所以例如，侧边栏——什么是侧边栏？其中存在的所有不同子功能。从用户的角度来看，这里是到达它的方法，所有不同的键盘快捷键。甚至 DOM 元素或你通过 CDP 选择事物时使用的属性都在那里。所以再次，这对智能体来说非常非常强大。

And without this, your agent says, 'I have no clue.' But with a feature map like this, it has a lot more context and understanding of how to actually navigate, how to get to all the different features. So for example, the sidebar—what is the sidebar? All the different sub-features that are present in it. From the user point of view, here's how to get to it, all the different keyboard shortcuts. Even the DOM elements or the attributes that you use for selecting things through the CDP are all there. So again, this is really, really powerful for agents.

它还附带了一个 create verification 技能，还有一个 maintain verification 技能，这样你可以保持更新。

And it also ships a create verification skill, but also a maintain verification skill, so you can keep this up-to-date.

酷。是的，我正想问你如何创建那个。所以你介意多分享一些在 P Sack 背景下的过程吗？也许只是为不熟悉的人解释一下 P Sack 是什么。

Cool. Yeah, I was just going to ask how you created that. So do you mind sharing a little bit more about that process in the context of P Sack? And maybe just what P Sack is for the folks who aren't familiar.

是的，P Sack 挺有意思的，因为首先，这个名字有点傻。P Sack 里的 P 就像 potato sack（土豆袋）。因为有一个很有名的人，Garry Tan，他是 Y Combinator 的 CEO。他想出了一个叫 G Stack 的插件——Garry Stack。有趣的是，我们同姓。我们没有亲戚关系。但我觉得开 Garry 的玩笑会很有趣，把 P Sack 做成我版本的他的插件。但只是根据我自己的一套工程实践来定制。但老实说，我从未打算构建 P Sack。它只是从一堆技能开始的。比如我从那个 control glass 技能开始。然后我开始另一个叫 How 的技能，这也是我通过观察智能体注意到的。

Yeah, so P Sack is pretty interesting because, well, first of all, the name is kind of goofy. The P in P Sack is like potato sack. Because there is a pretty famous person, Garry Tan, who is the CEO of Y Combinator. And he's come up with this plugin called G Stack—Garry Stack. And funnily enough, we share the last name. We have no relation. But I thought it'd be funny to poke fun at Garry and make P Sack my version of his plugin. But kind of just tailor it to my own set of engineering practices. But I honestly never set out to build P Sack. It just started with a bunch of skills. Like I started with that control glass skill. And then I started with another skill called How, which I also noticed through observing agents.

在我早期试图爬这个梯子的时候，我非常亲力亲为。我基本上对智能体极端挑剔。我会告诉它“这个功能停止工作了。这是 bug 报告。为什么它不工作？”而通常智能体会自信地声称“哦，一定是这个，对吧？一定是这个东西。”当我查看实际的工具调用时，我注意到它实际上并没有读取应该受影响的代码。这让我非常怀疑。那时我就想“我不能再信任这个智能体了，因为它完全在幻觉。”我认为很容易积累这种不信任，并感到无助，就像你不知道如何帮助你的智能体成功。

So in the early days of me trying to climb this ladder, I was super in the loop. And I was basically nitpicking my agents to an extreme degree. I would tell it, 'This feature has stopped working. Here's a bug report. Why isn't it working?' And very often the agent would just confidently state, 'Oh, it has to be this, right? It has to be this thing.' And I noticed when I looked at the actual tool calls, it wasn't actually reading the code that should be affected. And that made me extremely suspicious. And at that point I was just like, 'I can't trust any of this agent anymore because it's just completely hallucinating.' And I think it's very easy to build up that distrust and kind of feel helpless, like you don't know how to help your agents succeed.

但再说一次，我觉得管理的类比非常有帮助。想象你是工程团队的管理者，团队里有个工程师编码能力很强，但完全没有人脉。你刚招了他，他五秒前才入职。那你怎么教这个人变得高效？通过技能。技能就是 markdown，对吧？但它编码了大量信息和指令。你可以从一个智能体身上激发出大量智能——嗯，推特上有些人称之为把智能体拉到不同的潜空间。这其实是一种花哨的说法：因为大语言模型是预测下一个词的，当你一开始就给它高质量的 token，它就能在更聪明的更高维空间里进行模式匹配。所以那是一个非常有趣的模型。

But again, I think the management analogy is super helpful. Imagine if you were a manager of an engineering team and you had an engineer who was a really good coder but had no business contacts whatsoever. You just hired them and they onboarded five seconds ago. How do you actually teach that person to be effective? You do that through skill. Skill is just markdown, right? But it codes a lot of information, instructions. You can really draw out a lot of intelligence from an agent by—well, some people on Twitter call it pulling the agent to a different latent space. Which is a fancy way of saying that since LLMs predict the next token, when you give it high-quality tokens to begin with, it can pattern match on a higher space that's smarter. So that's a very interesting model there.

但没错，我是非常非常渐进地构建 P Stack 的。我开始时就是认真观察智能体如何失败——它们出现的各种失败模式。每次看到，我就想，好，我要把那个做成一个技能。比如停止幻觉，真的去搜索、查找代码。多用子智能体。对，停止猜测。

But yeah, I built P Stack very, very incrementally. I started by really observing how agents failed—all the different failure modes the agents were having. Every time I saw that, I just thought, okay, I'm going to make that a skill. All right, like stop hallucinating, actually go and search up, look up the code. Use a lot of sub-agents. And yeah, stop guessing.

是的，这说得通。这里有一个后续问题，既来自我自己，也来自聊天中的很多人。我想是两部分。第一部分是，你如何维护这些技能？产品随时间变化。显然有很多人在代码库上提交代码。那么这些技能如何维护？其次，你怎么知道你的验证足够好了？比如，你可以信任你构建的验证循环会——我想你信任它们完成后会给出结果。

Yeah, that makes sense. One follow-up question here, both from myself and from a bunch of people in the chat. I guess it's two parts. So one is, how do you maintain these skills? The product changes over time. Obviously there's a lot of people shipping against the code base. So how do these skills get maintained? And then second to that is, how do you know when your verification is good enough? Like, you can trust that the verification loops that you've built are going to—I guess you trust that that would put when they're done.

好的，也许我谈谈——我想我有相关的内容。也许我先从这个开始。那么，我如何维护这些技能？如果你不熟悉这个概念，eval 本质上是一种方式——我心中的模型是，它就像智能体的单元测试。你其实可以自己做 eval。你不需要特殊的框架。你可以根据你想有多科学、多严谨来构建一个。我的屏幕红了。

Yeah, maybe I'll talk about—I think I have something related. Maybe I'll start with this one first. So, how do I maintain these skills? If you're not familiar with this concept, an eval is essentially a way to—I think the mental model I have is like it's a unit test for an agent. And you can actually make your own evals. You don't need a special framework for them. You can build one depending on how scientific and how rigorous you want to be. My screen is red.

是的，有一个小按钮。抱歉。

Yeah, there's a little button. Sorry.

你是在禁用绘图还是什么？我看不到我的屏幕。

Are you disabling the drawing or something? I can't see my screen.

是的，抱歉。如果大家不要在屏幕上画就好了，但有一个小按钮在——对，那个小下拉菜单。

Yeah, sorry. If you guys could not draw on the screen that would be great, but there's a little button in the—yeah, the little drop-down.

我喜欢清除。

I like clear.

是的。

Yeah.

好的，是的。

Okay, yeah.

你搞定了。完美。

You got it. Perfect.

嗯，是的。所以 eval 基本上是对技能进行单元测试的一种方式。实际上在 P Stack 中，我们在 potato 模式下发布了一个 playbook，如果你搜索的话叫 eval playbook。它的做法其实相当严谨。但基本上我所做的是生成大量不同的子智能体。我让主协调智能体为我想让技能做的事情制定一个评分标准。然后它生成所有这些子智能体，并为它们创建单独的目录，这些目录命名巧妙，不让子智能体知道自己正在被评估，因为智能体其实能看出来，一旦它们知道就会改变行为。但它做了一堆类似的事情，本质上就是测试我正在制作或修改的技能是否真的做到了我认为它该做的。Cursor 的一个非常好的地方是，我们支持很多不同的模型。所以你可以跨各种不同的模型评估你的技能，了解它在不同矩阵上的表现，尤其是你使用的模型。所以我经常这样做。每次我修改一个技能，我都会运行其中一个——比如 eval playbook——确保它确实导向我想要的结果。

Um, yeah. So evals are a way to unit test your skills basically. And actually in P Stack we ship under potato mode there's a playbook if you search for it called eval playbook. And it's actually pretty rigorous the way it's done. But essentially what I do is I spawn a lot of different sub-agents. I have my main coordinator agent come up with a rubric for what I want the skill to do. And then it spawns all these sub-agents and it creates individual directories for them, which are cleverly named to not let the sub-agent know that it's being evaluated because agents can actually tell and when they do they change their behavior. But it does a bunch of stuff like that to essentially test whether or not the skill I'm making or changing is actually doing what I think it does. And one of the really nice things about Cursor is that we support so many different models. So you can actually eval your skill across all sorts of different models and get a sense of how well it performs across that different matrix, especially for the models that you use. So I do this a lot. Every time I modify a skill, I will run one of these—like the eval playbook—and make sure that it's actually leading to a result I want.

但我要说，维护技能其实相当难。它需要很多品味和观察力。所以你需要非常擅长当后座司机。你懂我的意思吗？比如如果你做过结对编程，你看着同事写代码，你就想，我可能能做得更好。你问同事很多问题。这里也类似。你不想只是智能体的被动观察者。在初始阶段构建你自己的技能集时，你要非常坐在驾驶座上。显然你可以用 P Stack 之类的东西，但如果你在构建自己的技能集，我认为打开所有工具调用、阅读代码、阅读所有智能体行为和它们的思考块，是发现它们在哪里失败的好方法，对吧？比如它们哪里出错了？然后你可以为此构建一个技能。

But I will say maintaining skills is actually pretty hard. It requires a lot of taste and observation. So you need to be very good at being a backseat driver. You know what I mean? Like if you've ever done pair programming, for example, and you watch a coworker code and you just think, I could probably do this better. You ask a lot of questions to your coworker. And it's kind of a similar thing here. You don't want to just be a passive observer of the agent. You want to be very in the driver seat in the initial stages when you're building up your own set of skills. Obviously you can use something like P Stack, but if you're building your own set of skills, I think opening up all the tool calls and reading the code and reading all the agent behavior and their thinking blocks is a really great way to see where they fail, right? Like where are they going wrong? And then you can go and build a skill for that.

然后关于验证，你如何信任它——我认为这也是一个非常相似的迭代循环。我实际上对验证技能本身也做了同样的过程。关于 eval 有趣的一点是，你可以对它们进行爬山优化，意思是你的 eval 可以产生一个分数，你可以让协调器产生一个分数，但你也可以让不同模型的评判智能体来交叉参考，确保第一个模型没有偏见——那个评判所有运行子智能体的模型。但你也可以爬山。意思是你可以用 Cursor 中的 /loop，然后说‘好，继续在这个 eval 上循环，直到所有东西都是 10 分（满分 10 分）’作为例子。我对控制技能也做了基本相同的做法。所以它其实非常放手。我就是那样构建那个技能的，比如那个技能中的 CLI。随着时间的推移它变得非常好。但没错，一开始绝对不是特别顺利。它需要大量迭代。

And then with verification, how you trust it is—I think it's also a very similar iteration loop. I actually did the same process for verifying the verification skill. One thing that's interesting about evals is that you can sort of hill climb them, meaning that your eval can produce a score, a score that you can get your coordinator to produce, but also you can have a judge agent of a different model to cross-reference and make sure that the first model is not being biased—the model that's judging all of the sub-agents that are running the thing. But you can also hill climb. So meaning that you can use /loop in Cursor and you can say, 'Okay, keep looping on this eval until everything is 10 out of 10,' as an example. And I did basically the same approach with the control skill. And so it was very hands-off, actually. I built that skill that way, like the CLI in that skill. And over time it's gotten really good. But yeah, it was definitely not super smooth at the beginning. It required a lot of iteration.

我觉得这里对我来说有一个类比，那就是——嗯，我稍后在我画的另一张幻灯片里会做这个类比。但我认为，作为现在的工程师，你更像一个管理者，或者我喜欢的类比是，你像餐厅里的厨师。

And I think there's an analogy here for me, which is—well, I make this analogy later in a different slide on my drawing here. But I think of it like, as an engineer now, you're sort of more like a manager, or the analogy I like is like you're a chef in a restaurant.

你知道，你就是主厨。你不再自己烹饪所有食物了。你有一个厨师团队，对吧？有流水线厨师，有副主厨，有各个不同的岗位。而你的职责是真正营造环境。你负责布置厨房。你负责给不同的人分配任务。所以是的，这是一种非常有趣的工作方式。但没错，我基本上就是这样构建这些验证技能的。

You know, you're the head chef. You're not cooking all the food yourself anymore. You have a team of cooks, right? You have line cooks, you have a sous chef, you have all these different stations. And it's your job to really set the environment. You're in charge of setting up the kitchen. You're in charge of giving tasks to different people. So yeah, it's a very interesting way of working. But yeah, that's how I basically built these verification skills.

是的，就这一点我想再深入一层。那么，假设我们想为某件事构建一个评估或技能，并希望它能自行改进，我想这就是你的建议。你是在像工作树那样隔离的环境中，用子智能体和审查智能体等来做吗？还是发生在某种云托管环境中？更实际的步骤是什么？如果我想去做这件事，为某件事建立一个验证系统，我该怎么做或从哪里开始？

Yeah, just one thought there and I try to go one layer deeper. So, let's say we wanted to build an eval or a skill for something and we wanted to kind of get better on its own, which is what I think you're suggesting. Are you doing that in like a work tree kind of isolated with like the sub agents and then the reviewer agents and all that? Is it happening like in some type of cloud-hosted environment? Like what are the more practical steps? If I wanted to go do this and set up a verification system for something, what would I do or where would I start?

我认为最好的开始方式是本地，因为你可以观察。你绝对可以观察你的智能体在做什么。所以，如果你在为自己构建验证技能，我绝对会从本地开始，让你的智能体启动应用程序，无论是 CLI 还是桌面应用或其他什么。这样你就可以实际观察，对吧？你可以看到智能体如何与应用程序交互。你可以看到它如何调用不同的 API 来与应用程序交互。但对我个人而言，我基本上已经全面、大部分全面转向云智能体，因为它们极其强大。而 Cursor 真正强大的地方其实是云智能体，如果你花一点时间设置环境，这些控制技能、这些验证技能会带来巨大的红利，因为它不仅仅是让你作为单个工程师变得更好，它实际上提升了整个团队甚至整个公司的水平，因为你可以开始思考云智能体，开始思考自动化，自动做一些事情，比如——我稍后会详细讲，但我先提一下——例如，我经常谈到我们有一个叫 Benny 的智能体，对吧？它接收我们收到的所有错误报告，自动在云中运行，打开一个云——它是桌面版，在自己的计算机上运行 Cursor，并使用相同的控制技能与应用程序交互，尝试复现错误或用户报告，对吧？这非常强大，因为我可以立即从这个自动化中获得大量信息。就像这个例子中，你可以看到 Benny 实际上复现了错误，但它在主分支上已经修复了。所以它实际上确认我们已经修复了这个问题，我只需要发布另一个 Cursor 构建。所以这是巨大的信息，我不必去和智能体坐在一起花一个小时弄清楚这个问题是否已修复。所以你可以最大化节省时间，但我团队中的每个人都能从中受益。公司里的每个人都能从中受益。所以绝对认为使用云智能体非常强大。但没错，这是一个旅程。你必须先信任它，对吧？在你达到这个阶段之前，这又回到我在这里说的，这非常困难，几乎不可能，我绝对鼓励你不要试图跳跃——如果你还在这个区域，你不想跳到“我现在要生成一百或一千个云智能体”，因为你只会浪费大量 token，而且会非常昂贵。

I think that the best way to start is local because you can observe. You can definitely observe what your agents are doing. So, if you're building a verification skill for yourself, I would definitely start local and just have your agent bring up the application, whether it's a CLI or a desktop app or whatever. And so, you can actually observe, right? You can see how the agent is interacting with the application. You can see how it calls the different APIs that allow it to interact with the application. But for me personally, I have basically been kind of all in, mostly all in on cloud agents because they're extremely powerful. And the really powerful thing about Cursor is the cloud agents actually, where if you spend a little bit of time setting up your environment, these control skills, these verification skills pay a huge amount of dividend because it's not just something that makes you as a single engineer better, it actually levels up your whole team and even your whole company because you can actually start thinking about cloud agents and start thinking about automations that automatically do things like—I get, I kind of talk about this a bit later, but I'll just kind of get into it—where, for example, I talk a lot about this agent we have called Benny, right? Who takes all of the bug reports that we get and it automatically goes off in the cloud, opens up a cloud—it's desktop, it runs Cursor in its own computer, and it uses the same control skills to interact with the application and try to reproduce the bug or the user report, right? And this is so powerful because at once I can immediately get so much information from this automatically. Like here in this example, you can see that Benny actually reproduced the bug, but it's already fixed on main. So, it actually confirms that we fixed this problem already, and all I need to do is just release another build of Cursor. So, that's like huge information there that I didn't have to go off and sit with an agent and spend an hour trying to figure out is this fixed or is this not fixed? So, you can max so much time, but everybody on my team benefits from this. Everybody at the company benefits from this. So, definitely think that keeping these using cloud agents is super powerful. But yeah, it's like a journey. You have to trust it first, right? Before you get to this point, and that goes back to what I was saying here where it's very hard, it's almost impossible, and I would definitely encourage you not to try to jump from—you know, like if you're still in this zone, you don't want to jump to like I'm going to spawn a hundred or a thousand of cloud agents right now because you're just going to waste a lot of tokens and it's going to be extremely expensive.

是的，那么到目前为止总结一下，基本上如果我们想走你走过的旅程，那就是从验证开始，构建一些技能和一些方法来确定智能体至少产生正确的代码，或者像你说的，代码好不好可能是另一个问题，但从技术上讲，它通过查看堆栈跟踪、查看应用中的实际行为等来解决问题。一旦我们在本地信任它，然后我们可以开始考虑扩展到云中，运行更多智能体，它们自己捕捉信号，对吧？所以，无论是收到的错误报告还是什么，它们可以去接取并解决问题，然后给我们返回一个 PR。然后也许最后一步是自动合并 PR，这就是你现在的阶段。

Yeah, so just to kind of recap so far, basically if we wanted to go on the journey that you've kind of gone on, it would be just start with verification, building some skills and some ways of determining that the agents are producing at least like correct code or there like you said, whether it's good code or not is maybe a separate question, but like it's technically solving the problem by looking at stack traces, looking at the actual behavior in the app and so on. And once we trust it locally, then we can start to think about scaling into the cloud and running more agents that are picking up signals I guess on their own, right? So, whether that's like a bug report that comes in or something and they can go and pick it up and solve the problem and give us back a PR. And then maybe the last step is like auto merging the PRs, which is where you're at.

也许不是每个人都在这个阶段。

Maybe not where everyone is.

是的，是的。然后在主分支上审查它们，但这样对吗？

Yeah, yeah. And then reviewing them on main, but is that about right?

是的，没错。我想是的，这就是我画这条曲线的原因，对吧？因为这基本上描述了我的旅程，当我开始时几乎只能使用几个智能体，我只是观察每一件事。我认为从这里到那里真的没有捷径，因为这真的关乎你对智能体的个人信任程度。对吧？显然，作为工程师，你不想把垃圾代码直接投入生产。那么，你如何真正建立这种信任？这需要很多品味和判断力。但你知道，像 Pstack 这样的插件绝对能帮助你更快上手。所以，我想如果你信任我并信任 Pstack，那么推而广之，你也许可以信任你的智能体。但如果你不信任我，我绝对不鼓励人们盲目信任我。如果你建立自己的技能集，你显然可以看看 Pstack 并分叉它，使其成为你自己的，改进技能，绝对鼓励这样做。但对我来说，这真的都是关于信任。你知道，我们在这个聊天中的每个人都有不同的工程标准。在我们的代码库中，有不同的事情对我们很重要。当你能够将所有这些编码到技能中，并且你可以验证你的智能体确实在执行它们时，这让你真正能够攀升这条曲线。并开始自动化事情。还有另一件事我想谈谈，如果——

Yeah, exactly. I think yeah, that's why I drew this curve, right? Because this basically describes my journey of when I started barely could use a couple agents and I was just observing every single thing. I think there's really no shortcut for going from here to there because this is really about your personal level of trust in the agents. Right? Obviously, as an engineer, you don't want to just slop code into production. So, how do you actually build up that trust? Takes a lot of, I guess, taste and judgment. But, you know, like I think plugins like Pstack definitely kind of help you get up to speed much quicker. And so, I guess it's like if you trust me and you trust Pstack, then by extension, you can maybe trust your agents. But if you don't trust me and I definitely would not encourage people to blindly trust me. If you build up your own set of skills that you can obviously, you know, take a look at Pstack and kind of fork it, make it your own, improve the skills, definitely encourage that. But for me, it's really all about it just keeps coming back to trust. You know, every one of us here in this chat have a different standard for engineering. And there are different things that are important for us in our code base. And when you are able to encode all of that into skills and you can verify that your agents actually doing them, that allows you to really kind of ascend this curve. And start automating things. There's another piece I wanted to talk about if there's—

好的，请讲。

Yeah, go for it.

我会边讲边回答更多问题。不过是的，我觉得还有第三部分我还没谈到，挺有意思的，就是重构和重写。我认为业界最有争议的话题之一就是：应不应该重写你的应用？因为工程师很容易这样，尤其是你加入一家公司，进来看到代码库，你会想：“天哪，这太糟糕了。谁写的这代码？”你知道，太糟了，你想把整个东西重写。这是一种非常常见的倾向，我觉得很多人在智能体出现之前，甚至现在，肯定会劝你不要重写。但我其实要在这里论证一下为什么你可能想考虑重写。因为我觉得这真的要看情况。棕地应用（brownfield applications）其实处境不错，尤其是如果它们已经搭建得很好。最近我跟一些人聊到——我只是观察。我注意到一个平行现象：很多大科技公司的问题现在成了所有人的问题。大科技公司的问题，比如我在 Meta 工作时，我们有一个巨大的 monorepo。我们有，我不知道，成千上万的工程师在键盘上敲打、发布代码。Meta 有很多非常优秀的工程师，但我要说，你会惊讶于代码质量其实并不那么好。所以我常开玩笑说，在 AI 懒惰之前，我们有人的懒惰。所以我认为很多大科技的基础设施，比如 Meta 或 Google 的，真正的大科技公司，其实是为那种情况设计的：你在迎合——这么说听起来很糟——但你团队里能力最差的工程师，对吧？你构建框架、构建约定、构建护栏，你限制凭证，这样你的实习生不会清空生产数据库。如果你已经有那种级别的基础设施，我认为你的智能体其实已经可以做得非常扎实。因为护栏已经到位，智能体不会造成混乱，或者不会在你的代码库里造成太多混乱。而且你总是可以添加更多护栏。但我认为绿地应用（greenfield applications），尤其是全新的应用，在我看来是最大的风险。同时也是最大的机会。因为如果你 vibe code 一个项目、一个原型，就像我们为 Grok Bot 做的那样——Grok Bot 是非常非常非常快速地搭建起来的。如果你还没听说过 Grok Bot，那是我们昨天刚发布的新应用。真的很酷。它让你可以编排——创建有自己身份的个体智能体，你可以编排它们。超级酷，一定要去看看。但是的，那是一个非常绿地的应用，就像大多数原型一样。所以它是非常快速地 vibe code 出来的。人类根本没读代码。我最近发了一条推文，说了些关于有机架构（organic architecture）的东西。也许我会找到它。但想法是，当你有一个完全 vibe code 的应用时，你基本上没有任何护栏。所以你的智能体，当你给它们一个任务时，它们会用最方便的方法解决。随着时间的推移，你会陷入这种情况：代码库失控，因为你不理解它。你的智能体大概在某种方式上理解它，但它们构建的东西是为捷径优化的。你会受苦——那个应用会有很多问题。所以我认为用非常强的约束来启动你的代码库是非常必要的。因为当你有一个你可以信任的代码库，当你有真正帮助你的智能体的护栏，好的代码，你可以进入曲线的那一部分，就像我说的，我今天醒来有 20 个 PR 被我的智能体合并了。那是因为我投入了大量时间——我昨天算了一下，超过 600 个 PR——当我将整个 Grok Bot 重构为我一直在构建的新架构时。是的，我已经到了不再真正看代码的地步。我说这话不只是为了卖你 token，而是因为达到那个点需要很多工作。我花了很多 token 才让代码库达到这个我不再需要看它的点。但我非常兴奋，因为潜力——这不仅对我有益。它对所有为 Grok Bot 做贡献的人都有益。它还赋能设计师、产品经理，甚至 GTM 人员为 Grok Bot 添加功能。而且我不必担心——我不必半夜醒来担心，哦有人刚合并了一个性能回归。我有很多约束，在 Grok Bot 里写代码其实很烦人，但智能体吸收了所有这些烦人之处。但是的，我很乐意谈谈那到底是什么。

I'll pick up more questions as I go. But yeah, I think there's a third part to this which I haven't talked about yet, which is kind of an interesting one, which is like refactoring and rewriting. Like one of the most controversial topics in the industry, I think, is like should you rewrite your app or not? Because I think engineers are very prone to this where especially when you join a company, you come in and you see the code base and you're like, "Man, this is terrible. Who wrote this code?" You know, it's terrible. You want to rewrite the whole thing. There is a very common inclination and I think a lot of people, before agents, and arguably even now, will definitely discourage you from rewriting stuff. But I'm actually here to make a case for why you might want to consider it. Because I think it really depends. Brownfield applications I think are actually in a pretty good spot, especially if they're set up well already. And like recently I've been talking to some people about—I was just observing. I just noticed this parallel, which is that a lot of big tech company problems are now everybody's problems. And the big tech company problem, like when I was working at Meta, we had this giant monorepo. We had like, I don't know, tens of thousands of engineers just banging on their keyboards and shipping code. And a lot of really great engineers at Meta, but I'll say like, you'll be surprised that the code quality is actually not that good. And so I often joke that before AI sloth, we had human sloth. And so I think a lot of big tech infra, like what Meta has or Google, really big tech companies, are actually designed for that where you're catering to the—this sounds so bad to say—but like the least capable engineer on your team, right? You build frameworks, you build conventions, you build guardrails, you restrict credentials so that your intern doesn't wipe your production database. If you have that level of infra already, I think your agents can actually already do a very solid job. Because the guardrails are already in place for agents to not cause havoc or not cause too much havoc in your code base. And you can always add more guardrails. But I think greenfield applications, especially brand new applications, are the biggest risk in my opinion. And also the greatest opportunity. Because if you vibe code a project, a prototype, like we did for Grok Bot—Grok Bot was spun up very, very, very quickly. And if you haven't heard of Grok Bot, it's our new application we just launched yesterday. It's really cool. It lets you orchestrate—create individual agents that have their own identity and you can orchestrate them. It's super cool, definitely check it out. But yeah, that was a very greenfield application like most prototypes are. So it was vibe coded very quickly. Humans were not reading the code at all. And I had this tweet recently where I said something about organic architecture. Maybe I'll find it. But the idea is that when you have a completely vibe coded application, you essentially have no guardrails whatsoever. So your agents, when you give them a task, they will just solve it in whatever method is the most convenient. And over time you get into this situation where you have a code base that is spiraling out of control because you don't understand it. Your agents understand it, I guess, in a way, but they've built something that is optimized for shortcuts. And you will suffer—you have a lot of issues with that application. So I think starting your code base with very strong strains is very much needed. Because when you have a code base that you can trust, when you have guardrails that actually help your agents, good code, you can get into this part of the curve where I said, you know, I woke up today and I had like 20 PRs merged by my agents. And that's because I invested a lot of time—over 600 PRs I calculated yesterday—when I refactored all of Grok Bot to this new architecture that I've been building. And yeah, I've gotten to a point where I really don't look at the code anymore. And I say that not just to sell you tokens, but because it took a lot of work to get to that point. I spent a lot of tokens to get the code base to this point where I no longer have to look at it. But I'm very excited because of the potential—it doesn't just benefit me. It benefits everyone contributing to Grok Bot. And it also empowers designers and product managers and even GTM people to add features to Grok Bot. And I don't have to worry—I don't have to wake up in the middle of the night and worry like, oh someone's just merged a perf regression. I have a ton of constraints and CI is actually very annoying to write code in Grok Bot, but agents absorb all of that annoyance. But yeah, I'm happy to talk about what exactly that is.

是的，我觉得在我们进入这部分之前，有一个问题就是关于你的 CI 看起来什么样，或者一些约束，然后还有平均 PR 大小。我之前看到一个关于这个的问题。只是给大家一个概览。不一定要数学上的平均值，但就是一般 PR 的大小，如果只有几行代码，或者你知道的，是的。

Yeah, I think one question before we get into this part here is just around that element of like what your CI looks like or maybe some of the constraints and then also like the average PR size. I saw a question about that earlier. Just to give people a glance. It doesn't have to be like mathematically average but just like what generally the size of a PR is if it's only a couple lines of code or you know yeah.

我觉得看情况。也许我试图以一种我不会——

I think it depends. Maybe I'm trying to do this in a way where I'm not going to like—

是的，你不必分享实际数字，如果你喜欢实际平均值就——

Yeah you don't have to share the actual number if you like the actual average just—

这没关系。

This is fine.

基准。

Benchmark.

但就像我们有——好吧。这没那么有趣，但有趣的是，Grok Bot 和 Cursor 中的虚拟化实际上是由 Pretext 驱动的，这是一个某人构建的新库。真的很有趣。你应该去看看。但那不是很重要。我认为平均 PR 大小我其实不知道。我可能——我不知道我是否想点击这些。

But like we have—So okay. This is not that interesting but fun fact is that virtualization in Grok Bot and in Cursor is actually powered by Pretext, which is a sort of new library that someone's built. That's really interesting. You should check it out. But that's not really that important. I think the average PR size I actually don't know. I probably—I don't know if I want to click on these.

我大概可以，但我想说它们的范围可以从几百行或 50 行到一千行左右，取决于任务是什么。比如这里我实际上在删除一堆文件，所以我预计这主要是删除操作。但没错，情况各不相同。没有像……

I probably can, but I would say they can range anywhere from a few hundred lines or 50 lines to like a thousand depending on what the thing is doing. So like here I'm actually like deleting a bunch of files so I expect that it's just this like mostly deletion. But yeah, it kind of varies. There's no like...

比如有多少个是 50 行的 PR，有硬性上限吗？

Like a hard cap for how many of them are like 50 line PRs?

上限。绝对没有硬性上限，但我确实鼓励我的智能体把工作拆分成多个 PR。我这样做主要是因为我喜欢 Git 历史是上下文非常丰富的来源这个想法。我希望每个 PR 能原子性地描述那一小块东西在做什么。这也让我更容易回滚更改，并找出，哦，我发布了一个 bug，它在这里，对吧？而不是在这个 4 万行的 PR 里，谁知道里面塞了什么。但没错，我对 PR 大小没有硬性上限。

Cap. There's definitely no hard cap but I do encourage my agents to split up their work into multiple PRs. I do that mostly because I like the idea that the Git history is a very rich source of context. And I like each PR to sort of atomically describe what that small piece of thing is doing. Which also makes it easier for me to revert changes and like figure out, oh, I shipped a bug and it's here, right? It's not in this 40,000 line PR where who knows what landed in there. But yeah, I don't have a hard cap on PR size.

酷。然后，是的，还有一个关于 CI 的快速问题。所以，再次，你不需要像屏幕共享那样深入你的 CI 做什么，但一般来说，你能描述一下 CI 大概是什么样子或者有多严格吗？

Cool. And then, yeah, also quick question on like CI. So, again, you don't have to go into like the screen share of like your CI does, but just generally, would you describe what the CI kind of looks like or how strict it is?

是的，那么具体来说，对于 Grok Bot，Dune 是我们为 Grok Bot 构建的架构的戏称代号。CI 看起来相当烦人，因为所有东西都有检查。所以 literally，我有……嗯，如果你读过任何 React，你知道 React 中最大的陷阱之一是 useEffect。所以在 Dune 和 Grok Bot 中，我们禁用了 useEffect。所以 Dune 就是……你可以把 Dune 的心智模型想象成 Electron 应用的 Next.js，它是为智能体编写而设计的。它是为我们的智能体驱动的应用定制的。所以 CI 检查非常具体。比如，不要用 useEffect。它被禁用了。CI 会失败并冲你吼。我们有一些更有趣的、可能让人挑眉的检查，比如我实际上也禁用了代码注释。这很有趣。但我注意到 99% 的情况下，智能体写的代码注释只是描述一些历史事情，实际上与代码完全无关。比如它经常说，哦，Lauren 说你永远不应该这样做。现在这出现在代码注释里。我就想，什么？为什么……我不是作为持久的全局规则说的。我只是说你这个 PR 很烂，你应该改那部分。智能体真的不太理解这个，令人惊讶。或者它们假设太多，以非常愚蠢的方式做事。所以，是的，我们禁止一切你能想象的智能体不擅长的事情，我们都禁止。所以，一个我们在智能体窗口中深受其害的例子是，我们有……你知道，如果你用过智能体窗口，你肯定见过性能问题，我们一直在努力修复它们。但这就像一场无休止的斗争，因为有太多拉取请求被合并。其中任何一个都可能使性能、稳定性或可靠性回退。你知道，智能体窗口还没有这个架构。我计划把这一学习成果带回那里，并重构那里的一切。但它经常回退，因为一个例子是我们进程间隔离很差。所以，在 Electron 上，你有一个渲染 UI 的渲染线程。但你也有一个主线程，可以运行不需要阻塞渲染器的其他代码。但我们分离这些做得不好。所以，很多时候你只是不小心让代码被拉入渲染线程运行。然后突然之间，你在与渲染器竞争，渲染器有非常……如果你想要 60 FPS，每一帧绘制必须在 16 毫秒内完成。所以每帧的截止时间非常非常小。如果你想要一个非常流畅的产品。当你开始构建，意外引入计算量很大或大量 IO 的东西时，你就会遇到很多卡顿。你的 FPS 真的下降，开始丢帧，出现超过 16 毫秒的长任务，你就得到非常卡顿的体验。所以，我们基本上在构建 Electron 应用中学到的所有模式，我们都编码到这个框架中。它变成了硬性失败。所以，我在 Grok Bot 中 literally 有一个名为 electron main、electron renderer 的目录，我们有导入 CI，我猜，我们实际上检查依赖图，确保你不会意外从一个目录导入代码到另一个目录。所以这由 CI 强制执行，还有 Bobbocks，这是我们的……这是 Cursor 的代码审查工具，在 CI 上运行。你知道，在我们的 agents MD 中，到处都是。所以我有这个东西，我谈到构建良好代码库有多个层次。显然代码库是其中之一，如果你有这样的架构，它极其严格，构建功能的方式非常传统。这是最强的强制执行级别，因为智能体就是喜欢复制现有模式。所以，Grok Bot 中的一个例子是我们有这些叫做 feature 的概念，我们有入口点和转录卡片。就像你在聊天中看到的所有卡片。这些在框架中都是名词，我猜。所以有一种非常传统的创建方式。比如一个 feature 全部在一个目录中，举个例子，所以所有贡献该功能的代码都活在一个目录中。所以全部共置在一处。非常容易，智能体不必 grep 到处找所有东西在哪里。它只需看 feature，就像，哦，好的，我在做 Grok Bot 的 onboarding 功能。我就只在这个目录工作。对于 80% 的工作，它主要就非常封装在那里。但就像它再次为最笨的智能体设计。你不必思考。我对这个框架的关键原则之一是，最短路径就是最佳路径。

Yeah, so well, specifically for Grok Bot, so Dune is the sort of cheeky code name for the architecture that we built for Grok Bot. The CI looks pretty annoying because there's checks for everything. So like literally, I have... well, if you've read any React for example, you know that one of the biggest footguns in React is useEffect. So in Dune and in Grok Bot, we've banned useEffect. So Dune is just... you can think of the mental model of what Dune is as like Next.js for Electron apps and it's designed for agents to write. And it's like custom for our agent powered applications. So the CI checks are very specific to that. Like, don't use useEffect. It's banned. CI will fail and yell at you. We have some of the more interesting ones that people might raise eyebrows is like I actually banned code comments as well. Which is very interesting. But I've noticed that 99% of the time agents just write code comments that kind of describe some historical thing that is actually totally irrelevant to the code. Like it will often say like, oh, Lauren said you should never do this. And it's now in the code comment. I'm like, what? Why... that was I didn't say that as like a durable, global rule. I just meant like your this PR sucks and you should change that part. Like, agents don't really understand this that well, surprisingly. And or they kind of assume too much and kind of do things in very stupid ways. So like, yeah, we just ban everything you can imagine like the agents are bad at, we ban. So, one example that we actually suffer a lot in the agents window is we have... you know, if you've used agents window, you've definitely seen performance issues and we're constantly trying to fix them. But it's like a never-ending struggle because there's so many pull requests that get merged. Every anyone of them could just regress performance or stability or reliability. You know, the agents window doesn't have this architecture yet. I plan to do bring this learning back there and kind of refactor everything there. But it just regresses super often because there's one example is like we have very poor isolation between processes. So like, on Electron, you have a renderer thread that renders your UI. But you also have like a main thread that you can run other code that doesn't need to block the renderer. But we do a poor job of separating those things. And so, often times you just accidentally have code that gets pulled into running on the renderer thread. And then all of a sudden you're competing with the renderer that has a very... if you want like 60 FPS, you have to every frame that gets drawn has to be done in 16 ms. So, very very small deadline per frame. If you want a very smooth product. And when you start building, bringing in, accidentally bringing in, things that are like very computationally heavy or they have a lot of IO, then you just get into like a lot of jank. And your FPS really drops, you start losing frames, you get long tasks that take more than 16 milliseconds and you just get this really choppy experience. So, all of those patterns that we've learned basically building Electron apps, we've encoded into this framework. And it becomes like a hard failure. So, I literally in Grok Bot, we literally have a directory called electron main, electron renderer, and we have import CI, I guess, where we actually check the dependency graph to make sure you're not accidentally importing code from one directory to another. So that's enforced by the CI, as well as Bobbocks, which is our... which cursors, like code review tool that runs on CI. You know, in our agents MD, it's everywhere. Like, so I have this thing here where I talk about like there are multiple layers, I think, for building a good code base. Obviously the code base is one where, if you have an architecture like this, where it's extremely strict, the way to build features is very conventional. That's like the strongest level of enforcement because agents just love to copy existing patterns. So, one example of this in Grok Bot is like we have these concepts called like a feature and we have entry points and transcript cards. Like, all the cards that you see in the chat. These are all like nouns, I guess, in the framework. And so there's a very conventional way of creating them. And so like a feature is all in a single directory as an example, and so all of the code that contributes to that feature lives in one directory. So, it's all co-located in one place. Makes it super easy, agents don't have to like grep around and try to figure out like where all the things are. It just looks at the feature and like, oh, okay, I'm working on the onboarding feature in Grok Bot. I'm just going to work in this directory. And for 80% of the work, it's mostly just very encapsulated there. But like it's designed again for the dumbest agent. Like, you don't have to think. One of the key principles I have for this framework is like the shortest path is the best path.

因为这正好契合智能体写代码的方式。它们喜欢走捷径，会找到解决问题的最快方法。那为什么不把最快的方法变成最好的方法呢？我可能不会深入所有具体细节。这个框架更像是一组想法和原则的集合，而不是会开源的东西。你可以截图，如果你愿意的话，然后告诉你的智能体也为你构建类似的东西。但关键就在于分层。代码库是一部分，包含功能、目录和导入阻止——不应该被导入的依赖。所有这些都强制执行，还有静态分析。所以有 CI 检查。我们有很多针对观察到的坏模式的 lint 规则。编译器诊断。bug bot 里也有规则，我觉得三、四、五比较软。这两个实际上会让 CI 变红。所以有一个硬约束，智能体不能随便写烂代码。对于规则、技能和 bug bot，你的智能体仍然可能忘记，或者不一定总是一致地应用它们。所以我喜欢分层，但我不喜欢把它们当作唯一的执行来源，因为它们非常软。如果你只有规则、bug bot、技能和代码风格指南，你的代码库迟早会变得一团糟。我很抱歉这么说，但我绝对推荐投资于可以硬性执行的东西。这就是为什么你选择的技术栈也很重要。比如，我认为 Rust 又变得超级流行，因为编译器非常严格。编译器强制执行许多不同的东西。有一个借用检查器你必须满足。只要确保你的智能体不写不安全的代码块，你或多或少可以有点信心：如果代码编译通过，它可能就能工作，而且质量不错。但它给了你那种信任和信心，让你作为人类工程师不再需要自己去检查。你依靠代码和静态分析来让这一切更顺畅。我想最糟糕的情况是，你被困在代码审查阶段，你实际上通过人工阅读代码来强制执行所有约束和不变量，然后说“你不应该这样做”。每次你不得不这样做时，你应该把它视为一种代码异味，一种反模式。你应该说：“好吧，与其在 PR 上评论，我如何把它变成硬规则？如何变成 lint 规则？如何变成 CI 失败？或者我如何彻底消除这个问题？”我可以谈谈我做的另一个迁移，但我可能先停在这里。

So, because that plays exactly to how agents love to write code. They like to take shortcuts. They'll find the quickest way to solve the problem. So, why not make that the best way to solve the problem? So, I probably won't get into all the specific details. And this framework is really more of a collection of ideas and principles rather than something that will open source. You can screenshot this, I guess, if you want and tell your agent to build something like this for you, too. But it's really all about the layers. The code base is one part with features and directories and import blocking—import dependencies that shouldn't be imported. And it all enforces that and static analysis. So, there's CI checks. We have a lot of lints for bad patterns that we observe. Compiler diagnostics. There's also rules in bug bot which are, I think like three, four, five are more soft, right? These two actually make CI red. So, there's a hard constraint where the agent can just write crappy code. For rules and skills and bug bot, your agents can still forget, or it may not always consistently apply them. So, I like to layer them, but I don't like to rely on them as the only source of enforcement because it's very soft. And if you only have rules and bug bot and skills and a style guide for your code, it's only a matter of time before your code base looks like complete trash. I'm sorry to say that, but I definitely recommend investing in things that can be hard enforced. And that's why maybe the choice of tech stack that you use is also very important. For example, I think Rust is getting super popular again because the compiler is so strict. The compiler enforces so many different things. There's a borrow checker that you have to appease. And as long as you make sure your agents don't write unsafe code blocks, you can more or less feel somewhat confident that if the code compiles, it probably works and it's good. But it gives you that level of trust and confidence that you as a human engineer no longer need to go and check it yourself. You rely on code and static analysis to actually make that a lot smoother. And I guess the worst place to be in is if you are stuck in code review land where you actually enforce all of the constraints, the invariants in your code base by literally the human person reading the code and being like, "Okay, you should not do this." Every time you have to do that, you should consider that as a code smell, like an anti-pattern. And you should say, "Okay, instead of me commenting on the PR, how do I turn this into a hard rule? How do I turn this into a lint rule? How do I turn this into a CI failure? Or how do I even categorically eliminate this problem entirely?" I can talk about another migration I've done, but I'll probably pause here.

当然。说实话，我觉得我现在的状态就是你所描述的那样，我还没有所有这些规则。所以这次会议之后我有一些事情要做，以便能够扩展我的智能体。我肯定是在本地阶段并行运行几个，大概两到三个。我相信这里大多数人也是一样，所以，我知道我们只剩几分钟了。Lauren，你还有什么想强调的吗？显然有很多问题，我可以再收集一些，但我想给你几分钟，如果你还有什么想谈的。

Sure. Yeah, I feel like that's where I am, to be honest, is what you're describing right now, which is that I don't have all of these rules. So I have some things to go do after this session in terms of being able to scale my agents. I'm definitely on maybe a couple of parallel ones locally stage, so like two to three locally. And I'm sure most people here are on the same, so yeah, I know we only have a couple minutes left. Lauren, was there anything else that you wanted to highlight? I obviously there's lots of questions, so I can grab more, but I want to give you a few minutes if there's anything else you want to talk about.

嗯，我觉得我已经唠叨了很多，所以也许我们直接回答问题吧。

Well, I think I've been yapping for quite a lot, so maybe let's just do questions.

好的，酷。我有个问题，出现了几次，是关于 token 使用量的。问题是，你所描述的对那些使用正常 token 量的人来说是现实的吗？他们基本上没有无限的 token 可用。

Okay, cool. One question that I had a couple of came up a couple times was just around token usage. So the question is like is what you're describing a realistic thing for people who are on a normal set of token usage, they don't have basically unlimited tokens to work with.

嗯，我觉得这是一个很好的观点。显然，我在一个 AI 实验室工作，我们有无限的 token，所以我绝对不能说我做的这件事每个人都应该以完全相同的方式去做。我认为有可能在不破产的情况下达到这个程度。但是，如果你是一个工程领导者，或者你领导一个初创公司，我认为主要问题是投资回报率。是的，你在前期阶段花了很多钱在 token 上。重构代码库会消耗大量 token。添加所有这些内容会消耗大量 token。但是，如果我们正走向一个智能体编写所有代码的世界，你想保持非常精简，你不想不得不雇佣——你不想变成像 Meta 那样，你不想变成一个一万人的工程组织，因为那是一个很酷的问题，但你也需要很多开销。有规划，个人来说我不会——那不是特别有趣。但我认为你想保持非常灵活。你想让智能体让你做以前做不到的事情。这对我来说才是智能体的价值。不仅仅是把 token 扔到每一件小事上，但对我来说，我以前做不到的事情是自己在一个代码库中强制执行这种级别的约束。我只是一个人，我自己构建这个框架、做所有重构、测试一切并验证，会花我好几年的时间。想象一下，如果只有我，在前智能体时代，只有我自己运行——那会花我很长时间。而且我的薪水相当高。所以，我认为工程领导者可能会问的问题是，权衡是：你雇一个人来做这个，还是你花 token 来设置一个代码库，以便即使是最天真的、最笨的智能体也能做好工作。当你真正达到这一点时，即使不是最顶尖的智能体，也能出色地编写代码。这对我来说也带来了很多好处，因为我不仅赋予了自己能力，还赋予了产品经理、设计师、不熟悉 Grok 的工程师以可持续的方式做出贡献。

Well, I think that's a really good point. I mean, obviously, I work at an AI lab where we have unlimited tokens, so I definitely cannot say that this is something everyone should do in the exact same way that I did it. I think it's possible to get to this point without breaking the bank. But, if you are an engineering leader or you have a startup that you lead, I think the main question is ROI. And it's like yes, you spend a lot of money on tokens in the upfront stage. Refactoring a code base is going to take a lot of tokens. Adding all these things is going to take a bunch of tokens. But, if we're heading to a world where agents are writing all the code, and you want to be very lean, you don't want to have to hire—you don't want to become like Meta, in terms of you don't want to become a 10,000-person engineering org because that's a cool problem to have, but also you have so much overhead. There's planning, it's personally I wouldn't—it's not super fun. But I think you want to stay very nimble. And you want to be like agents are all about allowing you to do things that you couldn't do before. That's really to me the value of agents. It's not just throwing tokens on every single little thing, but to me the thing I couldn't do before is enforce this level of constraints in a code base by myself. I'm just a single person, it would have taken me years to build this framework and do all the refactoring and test everything myself and verify. Imagine if there was just me, in the pre-agent era, just running by myself—it would take me so long. And my salary is pretty high. So, the question I think an engineering leader might have is just then, what is the trade-off of do you hire someone to do this, or do you spend the tokens to set up a code base so that even the most naive, the dumbest agents can do a good job. And when you actually get to this point, even agents that are not, you know, fable size, do an excellent job of writing code. And this pays a lot of dividends as well for me personally where I have empowered not just myself, but again, PMs, designers, engineers who are not familiar with Grok to just contribute in a way that is sustainable.

所以，我觉得，是的，这肯定是一种权衡，你知道，没有什么是免费的。呃，而且 token 相当昂贵。呃，但是哦，实际上，呃，我我我不知道你们有多少人看到了，但我们今天实际上宣布了 Grok 4.6。所以，非常令人兴奋，终于发布了。嗯，所以，是的，Grok 4.6 会是一个很棒的……它非常非常聪明。呃，它在基准测试上真的很好。呃，而且 token 是一样的……呃，我希望我没有说错，但我相信每个 token 的成本与 4.5 相同。所以，你实际上以相同的成本获得了更多的智能。

So, I think, yeah, it's definitely like a trade-off for sure, you know, like nothing is like free for sure. Uh and tokens are pretty expensive. Uh but oh, actually, uh I I I don't know how many of you have seen this, but we actually announced Grok 4.6 today. So, very exciting, finally out. Um so, yeah, Grok 4.6 would be like a great It is very, very smart. Uh it's really good on the on the benchmarks. Uh and it's the same the tokens uh I hopefully I'm not saying this incorrectly, but uh I believe the cost per token is the same as 4.5. So, you're actually getting more intelligence for the same cost.

呃，我认为这是 Cursor 试图……Cursor 和 SpaceX AI 试图真正优化的领域，就像成本与智能的帕累托前沿。呃，你知道，我们不一定想要构建有史以来最大的模型，因为运行起来极其昂贵。这真的是关于如何找到那个最佳平衡点，对吧？你不需要一个巨大的模型，但它只是超级聪明，对吧，而且推理成本不是很高。

Uh I think this is an area that Cursor tries to Cursor and SpaceX AI try to really optimize for, like that Pareto frontier of, you know, cost versus intelligence. Uh you know, we don't necessarily want to build the biggest model ever because that is extremely expensive to run. It's really about like how do you find that sweet spot, right? You don't You don't need a giant model, but it's just super smart, right, and it's not very expensive for inference.

呃，但是嗯，是的，我想总结一下，嗯，我觉得这就像……如果你自己做分析，我觉得这相当积极。你从投资这类东西中获得的 ROI 会相当积极，它不仅能赋能你自己，还能让你的整个团队生产力大大提高。对吧？就像想象一下，你有一支像我这样的工程师大军，每天都能交付大量的改进和错误修复。好吧，这相当令人兴奋。

Uh but um yeah, I think to kind of round it up, um I think it's like a it's it's there's a if you do your own analysis, I feel like it's pretty positive. It It'll be pretty positive that the ROI you get from investing in stuff like this just empowers not just yourself but your whole team to be so much more productive. Right? Like imagine if you have an army of engineers like me who are shipping so much improvements and and bug fixes uh you know, every day. All right, like that is pretty exciting.

酷。呃，在我们结束之前最后一个问题。呃，这个问题是给电话会议中产品部门的人的。所以，假设我们确实有一支像 Lauren 这样交付的工程师大军。我只是好奇，产品团队或你们公司的其他职能如何跟上，鉴于如果你们交付得这么快，他们是否更多地使用 AI 来完成工作？就像你能说的那样。显然你没有……你不是那个角色，但只是好奇那是如何运作的。

Cool. Uh one last question before we wrap up. Uh this one is for the people in product on the on the call. So, let's say we do have an army of engineers who are shipping like Lauren. I'm just curious like how is the product team or other functions of your company keeping up given that like if you're shipping so quickly, have have are they using AI more to do their jobs? Like as much as you can speak to that. I obviously you don't have like you're not in that role but just curious about how that works.

嗯，我认为这就是 Grok Bot 实际上非常强大的地方。呃，所以在 Grok Bot 之前，你知道，显然 Cursor 只有 Cursor。就像我们只有智能体窗口，我们有 CLI，我们有 IDE。这些真的是高级用户工具，对吧？它们是为开发者设计的，所以非常非常以开发者为中心。你可以在其中做知识工作，但它的 UI 并没有真正为此优化。所以，我们实际上并没有……嗯，我觉得像很多 GTM 产品的人，他们可能用过 Cursor 来做他们的工作，但对他们来说绝对不是一个愉快的体验。嗯，我认为现在有了 Grok Bot，呃，它变成了……在我看来，Grok Bot 基本上就像非技术人士的 Cursor 时刻。就像它是一种非常非常容易使用智能体的方式，在一个非常舒适、非常熟悉的界面中。它看起来像 iMessage。嗯，它也很有趣。你知道，你可以给你的智能体起名字。呃，你可以以一种非常自然的方式进行编排，你可以说每个智能体就像一个人，现在你有一个智能体团队在工作，例如，你每个账户有一个智能体来管理。

Um I think this is where Grok Bot has been actually exceedingly powerful. Uh where so before Grok Bot like you know, obviously Cursor only had Cursor. Like we only had agents window, we had a CLI, we had an IDE. And these are really like power user tools, right? Like the they're designed for developers so it's very very developer-centric. You can do knowledge work in them but it like the UI is not really optimized for that. So, we actually didn't really have Well, I think like a lot of people in like you know, GTM product like they might have used Cursor to do their work but it definitely wasn't like a delightful experience for them. Um I think now with Grok Bot uh it's become Grok Bot is basically like the Cursor moment for people who are not in tech in my opinion. Like it's like it's like a very very accessible way to use agents in a very comfortable, very familiar interface. It looks like iMessage. Um and it's very fun too. You know, you can give your agent to have name. Uh you can have you can kind of do orchestration with in a very like natural way where you can sort of you know, each agent is like a person and now you have a team of agents like working on you have one one agent per account that you manage as an example.

或者如果你是 PM，你知道，你可以有一个智能体来总结 Lauren 昨晚做的所有工作，然后现在你知道我做了什么，对吧？所以我认为我们的 PM 们正在大量利用这一点。他们也在交付代码。所以你知道，通常他们会说哦，这里有个 bug，我修了。你能看一下吗？然后我会去审查，实际上它完美无缺。我就说好的，盖章。所以我认为这表明，你知道，Dune 架构是站得住脚的，对吧？所有那些严格的约束允许非工程专家的人在高水平上做出贡献。所以我我觉得我已经看到这带来了很多回报，你知道，设计师和 PM 们能够直接交付功能。这让 Grok Bot 团队超级快，对吧？我们可以如此快速地交付。嗯，我们有很多计划，所以我非常兴奋，你知道，交付更多东西。

Or if you're a PM, you have you know, you can have an agent that summarizes all the work that Lauren did last night and then now you know what I did, right? So I think our PMs are leveraging that a lot. And they're shipping code too. So you know, like often times they will just say oh here's a bug, I fix it. Can you look at it? And then I'll go review it and actually it's just perfect. I'm like okay, stamp. So I think that shows that you know, the the Dune architecture is holding up, right? The all the the really strict constraints allow people who are not experts in engineering to contribute at a high level. So I'm I feel like I'm already seeing that pay off a lot where you know, designers and PMs are just able to to to ship features directly. And that just makes the Grok Bot team super fast, right? Where we can ship so quickly. Um and we have a lot planned so I'm very excited to you know, to to ship more ship more stuff.