代码仓库会变成软件工厂吗?——对话 Monaco AI 负责人 Mihail Eric

Will Repos Become Software Factories? — Mihail Eric, Monaco

米哈伊尔·埃里克 Mihail Eric · Latent Space · 2026-11-08 · 约 121 分钟 · 原视频 ↗

打开互动全文版(中英对照 + 朗读 + 问答)→

本期速览 · Overview

斯坦福教授、Monaco AI 负责人 Mihail Eric 探讨编程智能体如何重塑软件开发生命周期,以及对新一代工程师意味着什么。

Mihail Eric, Stanford professor and Head of AI at Monaco, explores how coding agents are reshaping the software development lifecycle and what it means for the next generation of engineers.

要点 · TL;DR

核心观点 · Key points

反共识 · Contrarian takes

本期章节 · Chapters(共 53)

全文 · Full transcript(中英对照)

节目介绍与嘉宾背景 Episode intro and guest background

Host

好,这期节目相当特别,因为我们在庆祝 AI Engineer YouTube 频道突破 100 万订阅。如果你还没听说过,就去那边订阅吧。我们的嘉宾是 Mihail Eric,他不仅在斯坦福讲授“现代软件开发者”这门课——这门课现在正在走红,基本上覆盖了所有 AI 原生开发实践——他还是 Monaco 的 AI 负责人。所以他不仅是老师,也是实践者。下周来参加 AI Code Summit 的朋友们,那将是对过去一年你在编程领域所见的所有极端进展的一次庆祝。我很期待这些主题:对抗 slop、系统一模型,以及提升你的雄心去对代码里你见过的一切做大规模重写。做 AI 工程师,从来没有比现在更激动人心的时刻。而有 100 万 AI 工程师,我觉得这才刚刚开始。那我们进入访谈吧。

Okay, this is a pretty special episode because we're celebrating 1 million subscribers on the AI Engineer YouTube channel. So go over there and subscribe if you haven't heard about it. Our guest is Mihail Eric, who is not only teaching the Modern Software Developer at Stanford, which is going viral right now for basically covering all the AI-native development practices, but he is also Head of AI at Monaco. So he's not only a teacher but a practitioner. And those of you coming to the AI Code Summit next week, it's going to be the celebration of all the extreme progress that you've seen in coding in the last year. I'm excited about all the themes of fighting slop, of system one models, and of increasing your ambition for major rewrites of basically everything that you've seen in code. There's never been a more exciting time to be an AI engineer. And with 1 million AI engineers, I think it's just getting started. So let's get into the interview.

Host

好,我们在演播室,和斯坦福兼任教授、Monaco AI 负责人 Mihail Eric 在一起。欢迎。

Okay, we're in the studio with Mihail Eric, adjunct professor at Stanford and Head of AI at Monaco. Welcome.

Mihail

谢谢。很高兴来到这里。

Thank you. I'm glad to be here.

Host

你引起我们注意,是因为你这门课正在第三次更新——第三次,还是第二次?

You caught our attention because of your course that you are refreshing for the third time—third time, second time?

Mihail

CS146,斯坦福的“现代软件开发者”。

CS146, the Modern Software Developer at Stanford.

Host

这一代初级开发者、初级工程师、现在正进入职场的人,我觉得会是这次新转变的第一代。我看了课程大纲,心想:这就是我们覆盖的一切。所以我时不时喜欢邀请像你这样覆盖广泛主题的人,然后我们可以一个主题一个主题地聊,给大家讲讲每件事的最前沿,我猜你平时对所有嘉宾也是这么做的。

This particular generation of junior developers, of junior engineers, of people that are now entering the workforce, will I think be the first kind of generation of that new shift. And I looked at the syllabus and I was like, this is everything we cover. So every now and then I like to invite people like yourself who cover a broad range of topics, and then we can sort of go topic by topic and give people the state of the art of each of these things, which I assume you're doing with all your guests anyway.

Mihail

当然。

For sure.

CS146的起源故事 Origin story of CS146

Host

讲讲这门课的历史吧。是什么让你想开设 CS146?

Tell us the history of the course. What made you want to spin out CS146?

Mihail

是的,这是第二次开课,我第一次教是去年秋天,2025 年秋。那是在 Opus 4.5、5.4、6 之前,但甚至在 2025 年秋之前,这门课已经酝酿了大约一年。这门课的起点是 2024 年 11 月,我当时有个想法:我们看到编码智能体领域发生了这么多事,是的,那还是编码智能体的早期版本,但已经感觉有些东西在剧烈变化——开发者做事的方式、我们完成工作的速度,诸如此类。当时这些东西都只是零散的博客文章、零散的网络资料,如果你想真正深入这些方法论的话。所以我就有了这个想法:嘿,有没有办法围绕这个构建一套课程?让它更正式一点。

Yes, this is the second time it's being taught and I taught it for the first time last fall. So fall 2025. This was pre like Opus 4.5, 5.4, 6 days, but even before fall 2025 the course had been about a year in the making. The genesis of the class was November of 2024 and I had sort of had this idea that we're seeing all this stuff happening in coding agents and yes they were an earlier version of the coding agents but already it felt like there was something drastically changing in terms of how developers were doing their work, how quickly we were getting work done, all that fun stuff. And so at the time it was all just scattered blog posts, scattered things you could find online if you were trying to really get into the methodology of how to do these things. And so I had this idea like, hey, well, is there a way to kind of structure a curriculum around this? It's a little bit more formal.

Mihail

有了这个想法,而且你知道,我的母校就是斯坦福,我一直梦想回去教点东西,把这份恩情传递下去。我一直对教育非常投入。我跟 Chip 聊过,她也教过一门课——是我的好朋友。

Had the idea and, you know, my own alma mater is Stanford and I always had this dream to go back and teach something and kind of pay it forward. I've always been very deep into education. I talked to Chip, who also taught a class—one of my good friends.

Host

对,Chip。没错。

Yeah. Chip. Exactly.

Mihail

我跟她聊了——她教过《设计机器学习系统》——我问她那个流程是什么样的,从她设计课程的方式里获得了很多灵感,也思考了怎么让课程通过学校的审批。把它做出来,花了大约 9 到 10 个月——那是从 2024 年秋到 2025 年中后期——让它通过了委员会,为这门课的存在做了论证。我得准备一份提案之类的,然后到了 2025 年秋我第一次教这门课,反响非常好。学生们真的很感兴趣。去年甚至还有一篇 Business Insider 的文章写了这门课。然后今年,很多事情变了——我会说真正的编码智能体、自主编码智能体,以我们前所未见的方式发展起来了。所以感觉是时候再做一次、更新一下,然后就有了现在。

Spoke to her—she had taught, I guess, Designing Machine Learning Systems—and so asked her what that process was like and took a lot of inspiration from how she designed the curriculum and thought about getting something through the school. Put it together, spent about 9 to 10 months—this was now fall of 2024 through to mid to late 2025—got it through the committee, made a case for the class to exist. I had to put together a proposal and everything, and that took me to fall 2025 when I taught it for the first time, which was very well received. Students were really interested in it. There was even, I guess, a Business Insider article that was written about the class last year. And then yeah, this year a lot of things changed—I would say really coding agents, autonomous coding agents, really picked up in a way that we had not seen before. So it felt like it was time to do this again, refresh things, and then here we are.

从去年调查到今年原则 From last year's survey to this year's principles

Host

同时你还是 Monaco 的 AI 负责人,这也是一家非常热门的创业公司。

Meanwhile you're also Head of AI at Monaco, which is also a very buzzy startup.

Mihail

是的,是的。我们现在正处在风口浪尖,而且做得很开心。

Yes. Yes, we are. We're in the airwaves right now and having a fun time doing it.

Host

对。我挺喜欢你同时也是个实践者。你们显然在 Monaco 落地了所有这些。从高层面上讲,关于你从去年到今年改变了什么,有什么原则或观察吗?

Yeah. So I do like that you're also, I guess, a practitioner. You guys are obviously implementing all this stuff at Monaco. Any principles or observations about just what you have changed from last year to this year at a high level?

Mihail

是的,是的,太多了。我想,当我回顾去年那版课程时,编码智能体已经存在了,但我们还处在那个阶段:公司纷纷涌现,去攻克 SDLC 的每一个环节。有做代码审查的编码智能体,有做规格设计的编码智能体,有做 SRE 的编码智能体,当然其中一些公司现在还在,但去年感觉更多是一次广泛的巡览——这里是你可以用编码智能体的所有地方。某种程度上,去年很多课程内容都偏向于曝光和这种概览,像是一场奥德赛之旅:嘿,你对 SDLC 所知的一切现在都因为 AI 而改变了。

Yeah. Yeah. I mean a ton. I think that when I reflect on what last year's version of the class was, coding agents had been around, but we were still in kind of that phase where companies were emerging that were tackling every single part of the SDLC. There was a coding agent for code review, there was a coding agent for spec design, there was a coding agent for SRE, and some of those companies still exist, of course, but it felt like last year it was a lot more about a broad tour of here's everywhere you can use coding agents. And in a way, a lot of the curriculum last year was geared toward exposure and this kind of survey, this kind of odyssey of sorts through, hey, everything that you knew about the SDLC is now changing because of AI.

Mihail

今年,我觉得有些原则正在固化,对吧?比如非常细致的上下文工程,比如真正把设计和架构想透,作为处理非常复杂任务的方式。所以我的课的性质是:它不是给初学者的。它不是传统计算机科学在算法意义上的替代品,事实上很多这类课程是我这门课的先修课。但我确实知道,如果你有好的基础,现在朝着应用它、在真实世界的生产环境里运用这些原则的方向走——这件事我觉得已经到了一个阶段:你知道,在 Monaco 我们要处理庞大的代码库。我也帮很多公司梳理他们庞大代码库的设计之类的事情。所以它更像是一门实践者的课,面向学生毕业之后或刚进入职场时,如何成为职场中最高产的人。

This year, I think there's some principles that are solidifying, right? The idea of very careful context engineering, the idea of really thinking through design and architecture as a way of handling really complex tasks. And so the nature of my class is it's not intended for the beginner programmer. It is not a replacement for traditional computer science in the sense of algorithms, and in fact a lot of those classes are prerequisites to what I'm doing here. But I do know that if you have a good foundation, going now toward applying that and using those principles in production environments, in the real world—that's something that I think is now at a place where, you know, at Monaco we deal with massive codebases. I help a lot of companies also with thinking through their massive codebase designs and things like that. And so really it's more of a practitioner's course for once the students have graduated or are now entering the workforce, how to be the most productive members of the workforce.

现在该学CS吗? Should people study CS now?

Host

你说你对教育很有热情。我觉得处在大学阶段的学生处在一个挺有意思的位置。有什么建议和看法吗?现在人们还应该学计算机科学吗?Claude Code 和 Codex 就是我们需要的全部了吗?我想听听你的想法。

You said you're passionate about education. I feel like students in that phase of being in college are in a pretty interesting spot. Any advice and any take? Should people study computer science right now? Is Claude Code and Codex all we need? I want to hear your thoughts.

学生焦虑与AI原生工程师 Student Anxiety and AI-Native Engineers

Host

是啊,你能想象到学生们有很多焦虑。你想向他们传达什么?

Yeah, there's a lot of anxiety from students, as you can probably imagine. What are you trying to communicate to them?

Mihail

这次我想传达的是,焦虑依然存在,去年也有焦虑。但我想论证的是,一个训练有素的 AI 原生工程师——我知道这是个流行词,但我们来深入探讨其中哪些方面最有趣——一个训练有素的 AI 原生工程师可以成为任何公司最有用的资产。这是开课第一周,在第一堂课上我说,我真的相信每家公司都需要你,但门槛高得多。所以你刚毕业进入职场,但如果用谷歌那种职级体系来类比,仅仅达到 L3 是不够的。你通常必须比高级别的人还要优秀。那么,我们如何更快地达到那个水平?我想说的是,仍然有一部分核心的基础软件工程很重要——关于软件系统如何构建以及这些系统的权衡的一些核心价值观。但除此之外,作为人类,你们仍然应该是有价值的,而其余的实现细节你可以更多地交给 AI。

I think a lot of what I'm trying to communicate this time around is that the anxiety is still here, and there was anxiety last year as well. But I want to make the case that a well-trained AI-native engineer—and I know that's a buzzy word, but let's dive into what aspects of that are most interesting—a well-trained AI-native engineer can be the most useful asset to any company around. This is the first week of class, and in the first lecture I say, hey, I really believe that every company needs you, but the floor is much higher. So you're entering as your first job out of school, but it's not sufficient to just be an L3 at Google, if you use that kind of leveling system. You have to generally be better than almost at the senior level. So how do we get there more quickly? I'm trying to say that there is still a core bit of foundational software engineering that is important—some of these core values of how software systems are built and the trade-offs of those systems. But beyond that, you as the humans should still be valuable, and the rest of the implementation details you can defer more often.

大多数学生已完蛋 Most Students Are Cooked

Mihail

是的,这是我之前说过的话,几乎像是争议性的——那种为了制造话题而说的话,但我几乎真的相信——那就是大多数学生完蛋了。他们中的一些人会非常有价值。我不认为这是一个很受欢迎的观点。斯坦福的孩子会没事的,对吧?他们来自斯坦福。但美国其他地方的人会怎样?

Yeah, this is something I've said before, which is almost like the controversial—it's the kind of thing you say to be spicy, but I almost genuinely believe it—which is most students are cooked. Some of them are going to be super valuable. I don't think it's a very popular thing. Stanford kids are going to be fine, right? They're from Stanford. But like what happens to the rest of America?

Host

是啊,这是一个开放的问题。

Right, that is an open question.

课程动态性与客座讲座 Curriculum Dynamism and Guest Lectures

Host

奇怪的情况,因为我觉得很多教育机构行动缓慢。我甚至好奇你如何确定课程内容的截止日期——你在斯坦福教一门现代软件课程,而一切都在变化。

Weird situation, because I feel like a lot of education is slow to move. I'm even curious how you determine where to put the end date to curriculum in a course—you're teaching a Stanford modern software course that's changing everything.

Mihail

你作弊——你直接请编码智能体公司的人来演讲。

You cheat—you just get the people from the coding agents to come and speak.

Host

哦对,让他们当嘉宾。那是个有趣的嘉宾。

Oh yeah, have them be the guest. That's a funny guest.

Mihail

有趣。关于嘉宾讲座这一点,去年也是这样,所以这是有意设计的。我一开始就说我是从业者,正如你指出的——我每天都在做这件事,我设计这些系统,我们在 Monaco 构建这些东西,以前见过很多次。话虽如此,我不想声称我今天拥有所有答案——那会非常傲慢。所以课程的结构一直是:这里有一个从业者从实地工作中、从与许多领域内人士交谈中汇总的见解,但还有那些真正处于前沿的人,看到更远的东西,邀请他们提出他们的方案、他们对世界的看法。这就是为什么有嘉宾讲座。

It's funny. Well, on the guest lectures point, that was also last year, so that was by design. I came in saying I'm a practitioner, as you point out—I do this thing day-to-day, I design these systems, we build these things at Monaco and have seen that many times before. That being said, I don't want to claim that I have all the answers today—that would be a lot of hubris. So the structure of the class was always: here's one practitioner's aggregated set of insights from working in the field, from talking to a lot of people in the field, but then there are people who are literally on the frontier, seeing things even beyond that, and invite them to put their proposals forward, their view of the world forward. That's why the guest lectures.

Host

我认为对学生来说,看到大牌人物、真正在做这件事的人,直接听他们讲,能够问他们问题,是非常有影响力的——当然他们想实习、想找工作。有嘉宾讲座是好事。

I think it's really impactful for students to see big names, people actually doing it, hear from them directly, be able to ask them—of course they want to intern, they want to get jobs. It's good to have guest lectures.

Mihail

不,我想他说的——好吧,我换个说法。我们也采访过教 Coachella 课程的人,对吧?那就像是这里全是名字——只有名字——而他实际上也在产出,比如用 200 行代码实现 Claude Code。没人告诉你要那样做;没人会来跟你讲那个。但当你想在一个你可以使用的框架中教授基础知识,并建立工作理解时,这就是你要做的事情。

No, I think what he's saying—okay, I'll rephrase. We've also interviewed someone who teaches the Coachella course, right? Which is like here's all the names—it's only the names—whereas he's actually also producing, here's Claude Code in 200 lines of code. No one's telling you to do that; no one's going to come speak to you about that. But when you want to teach fundamentals in a framework that you can use and a working understanding, this is the kind of thing you do.

Host

没错。没错。是的。是的。正是如此。

That's right. That's right. Yeah. Yeah. Exactly.

Mihail

我确实认为这类材料需要平衡,正如你所说——它不是 100% 固化的。从设计上讲,这是一个特性,也许某种程度也是个缺陷,但课程的大部分内容确实从去年发生了变化。所以整个课程开发具有动态性。但我至少试图提出一些我认为会持续存在的东西,是的,提示编码智能体——它们变得更聪明了。这在处理它们的新方式方面意味着什么?但 Boris 也会知道一些我不知道的东西,对吧?所以让他也来分享他的学习成果。

And I do think there's a balance for this kind of material, to your point—it's not 100% solidified. By design, it's a feature and maybe a bug of some kind, but a big portion of the curriculum did change from last year. So there's a dynamism to the whole development of the curriculum. But I'm trying to at least put forth that there are some things that I believe are sticking around, and yes, prompting coding agents—they've gotten smarter. What does that mean in terms of new ways of dealing with them? But then still Boris is also going to know something that I don't know, right? So let's have him also come to the table and share his learnings.

课程中的开源贡献 Open Source Contributions in the Course

Host

这次我注意到的一个有趣的事情是,很大一部分内容是关于开源贡献的。这背后的思考是什么?

One of the fun things I noticed this time around is a good bit of the portion is on open source contributions. What's the thinking behind that?

Mihail

是的,这是我老实说最自豪、最兴奋的事情之一。这将是课程的第二周。这些是非常慷慨的开源合作伙伴,我花了这个夏天的大部分时间与他们交谈,让他们对这个项目感到兴奋和感兴趣。基本上,想法是:去年我教这门课时,我们有九个作业,然后是一个开放式的顶点项目。那九个作业基本上是我设计的一些教学练习——比如,这里有一个代码库,使用 AI 代码审查工具发布代码审查,然后看看为什么好,为什么不好。相对教学化,老实说相当简单。我认为很多人可以一次性完成。我想,好吧,在智能体已经存在的现在,这没那么有帮助。我周二调查的每个学生——没有一个人真正手动打字了;每个人都在使用编码智能体。这些都是斯坦福的大二、大三学生。

Yeah, this is one of the things that I'm honestly most proud of and most excited about. This is about to be week two of the class. So this is the very gracious open source partners that I spent the greater part of this summer just talking to and getting them excited about and interested in this offering. Basically, the idea was: last year when I taught the class we had nine assignments and then a capstone open-ended project. The nine assignments were basically me coming up with these pedagogical exercises—like, here's a repo, use an AI code review tool to post a code review and then see why is it good, why is it not good. Relatively pedagogical, honestly pretty easy. I think for a lot of folks they could have just one-shot it. And I thought, okay, this is not that helpful at this point—agents are around. Every student that I surveyed on Tuesday—not a single person really types by hand anymore; everyone's using coding agents. And this is, you know, all sophomores, juniors at Stanford.

Host

你调查他们吗?市场份额是多少?

Do you survey them? What's the market share?

Mihail

你是说他们使用什么工具吗?好的。我还没有,但这是个好主意。是的。我会的。

Was that like what tools they use? Okay. I haven't yet, but that's a good call. Yeah. I will.

Host

来点劲爆的。斯坦福偏爱。

Be spicy. Stanford prefers.

Mihail

不。不。

No. No.

课程安排与学生参与 Course Logistics and Student Access

Host

嗯,我觉得通过大学,他们有一些教育项目,对吧?大多数提供商都有教育项目。

Well, I think through the university, they have some educational program, right? Most of the providers have an educational program.

Mihail

是的,他们对学生计划非常慷慨,对吧?

Yeah, they're very generous on plans for students, right?

Host

对。所以我认为默认情况下,甚至在课程开始之前,就有 Claude 许可证,我猜还有 Cursor 许可证,可能还有 Cognition 许可证。我不太确定,你可能比我更了解。但除此之外,我还争取到了:嘿,如果我不想让人们因为每月 20 美元的订阅而受限,我希望你们如果想把 token 用到极致,看看能做什么,那就去做。所以,我稍后会回到合作伙伴的话题,但我通过与人交谈争取到的额外东西是给学生的额外额度。所以 Factory 也参与进来了。Cognition 的人参与进来了,OpenAI 也是。

Right. So I think by default even before the class there's like a Claude license, I imagine a Cursor license, there's probably a Cognition license as well. I don't know, you might know that better than I do. But what I was able to also accrue on top of that was, hey, if I don't want people to be bottlenecked by the $20 a month subscription, I want you guys to go if you want to token max something just to see what's possible, then you should. And so, and I'll come back to the partners, but the additional thing I was able to accrue through instead of talking to folks was additional credits for students. So Factory is also getting involved. The Cognition folks are getting involved and OpenAI as well.

导师制与开源项目 Mentorship and Open Source Projects

Mihail

但不管怎样,所以对于开源合作伙伴,我想,好吧,我在教育下一代学生时最担心的一件事是,如果每个人都说,就像 Six 说的,他们完蛋了,你知道他们会怎样。几乎有一种他们被落下的感觉。我认为导师制是提升每个人水平、确保不会出现巨大差距的重要一环,这个差距出现在那些已经在行业里、在编码智能体之前就了解情况并采用编码智能体变得很厉害的资深人士,和那些刚说“好吧,现在我们该怎么办?”的初级人士之间。所以某种意义上,一个目标是:嘿,编码智能体真的很棒。我想把你们置于处理复杂代码库、大规模大型代码库的情境中。我们将与一些我认为是最好的 AI 原生仓库合作,你知道,它们都做 AI 相关的事情,要么是编码智能体本身,要么是为智能体构建工具。我希望你们参与这些项目。我希望你们的作业一部分是实际为这些项目做贡献。在参与这些项目的过程中,你们将实际与维护者交谈。他们会审查你的代码。他们会帮助你建立良好软件的心智模型。这样,实际上就传递了真正优秀软件设计的火炬,我认为大多数学生没有机会这样做。当然还有好处,我确信有些人会因为在这些作业中表现出色而被挖走。

But anyway, so for the open source partners, I was like, okay, one of the things I'm most concerned about when it comes to educating the next generation of students is that if everyone is saying, to Six's point, like they're cooked, you know what's going to happen to them there. There's almost this sense of them being left behind. And I think that mentorship is a really important piece of just upleveling everyone and making sure that there isn't this massive gap that occurs between the senior people that have already been in the industry before coding agents and know what's up, and they adopt coding agents and they become amazing, and then the juniors that are just like, okay, now what do we do? So in a sense, one of the goals was, hey, coding agents are really good. I want to put you guys in situations where you're dealing with complex code bases, large code bases at scale. We're going to work with some of what I think are among the best AI native repos, you know, that all do AI focused things or either coding agents themselves or building tools in the service of agents in the modern world. I want you guys to get involved with these projects. I want part of your assignments to be actually contributing to these projects. In the course of being involved with those projects, you will actually be talking to the maintainers. They will be reviewing your code. They will be helping you build mental models of good software. In that way, there's effectively this passing on of the guard of true good software design in a way that I don't think most students get the opportunity to do. And then there's the benefit, of course, that I'm sure some of them will get poached and picked up as part of doing well in those assignments.

术语澄清 Clarifying Terminology

Host

是的。有趣的是,顺便说一句,如果我不说这个会被很多人恨。我想你说“完蛋”和“ cracked ”是为了戏剧效果,但你知道,陈述极端情况总是好的。

Yeah. One of the interesting things, for what it's worth, I'll get a bunch of hate if I don't say this. I think you say 'cooked' versus 'cracked' for dramatic effect, but you know, it's just nice to state extremes.

Mihail

当然。当然。

Sure. Sure.

课程概览 Curriculum Overview

Host

所以也许我们应该深入课程细节,或者你还有什么想在顶层探讨的吗?

So maybe we should go into the detail of the curriculum, or is there anything else you want to explore at a top level?

Mihail

哦,不。我是说,那很有帮助。是的。都在那里了。

Oh, no. I mean, that's very helpful. Yeah. All that's there.

Host

好的。我可以问一些关于学生使用工具、最佳实践的高层问题吗?你支持像个人导师那样吗?我假设学生也会看到这个。所以在我们直接深入课程之前,你对此有什么指导吗?

Okay. Can I have high-level questions about students using tools, best practices? Are you pro it like individual tutors? I'm assuming this will also be picked up by students. So before we dive straight into the course, just anything you have guidance on there?

Mihail

比如在哪些方面?

Like in terms of what?

Host

是的,你说你对教育充满热情,对吧?也有一种污名。一方面是老师和教授如何使用它,然后学生如何最好地使用这些工具,而不仅仅是把你的作业扔进去,它就能一次性完成。你如何仍然学习?如果你有什么要说的?

Yeah, you said you're passionate about education, right? There's also a stigma. One side is how teachers and professors use it, and then how can students best use these tools that's not just throw your assignments in and it can kind of one-shot it. How do you still learn? If you have anything there?

Mihail

是的。是的。我认为,我仍然有一些作业,我认为它们不是世界上最难的,而是更多关于概念,更概念性的东西,比如你知道技能意味着什么,也许更多是一些工具的策略性内容,技能比如智能体就绪的代码库、agents MD 之类的。所以有一定量的,嘿,如果你还没听说过这些术语,就赶紧跟上。不过我认为大部分学习,在测试工具方面,将发生在开源项目和最终项目中,最终项目非常开放,他们必须探索一些有趣的东西。

Yeah. Yeah. And I think that there's, I still have some assignments thrown in there which I think are not intended to be the hardest things in the world but are meant to be more about concepts, like more conceptual things, like you know what does a skill mean, sort of like the maybe more of the tactical of some of these tools, skills like you know agent ready code bases like agents MDs things like this. So there's a certain amount of just, hey, if you haven't heard these terms, get up to speed on them. I do think most of the learning though in terms of putting the tools to the test will happen both in the OSS and then also in the final project where it's very open-ended and they'll have to just explore something interesting.

逐周分解 Week-by-Week Breakdown

Host

酷。好的。所以也许,是的,我只想谈谈,让我们过一遍 1、2、3、4、10,然后我们再深入。所以基本上是广度优先。

Cool. Okay. So maybe, yeah, I just want to talk about, let's go through 1, 2, 3, 4, 10, and then we'll dive in deeper. So like breadth first basically.

Mihail

当然。当然。是的。所以我们从内部机制和编码智能体开始。你教人们如何编写自己的 Claude Code 克隆。它现在是开源的,这非常有帮助。然后,是的,是的,是的,所以第一周全是 harness 的内部机制,你知道,从头构建一个编码智能体。第二周更多是,好吧,现在假设有,我们如何开始思考大规模的上下文工程,超越普通的提示。然后我认为与此相结合的是,在做这件事时,思考工具设计、MCP。你知道,人们,我认为有一段时间 MCP 受到很多批评。我仍然支持 MCP。我经常看到它。所以我认为,在这一点上它是一个已经存在的标准,我认为它将继续存在。所以谈论有效的工具设计,那是第二周。第三周我会说更多是关于技能,以及超越简单的单一用途工具到更多程序化意味着什么,这就是我们将讨论技能的地方,这开始引导到智能体的自动化。

Sure. Sure. Yeah. So we start with internals and coding agents. You teach people how to write their own Claude Code clone. It's very helpful that it's now open source. And then, yeah, yeah, yeah, so week one it's all internals of harnesses, you know, kind of build a coding agent from scratch. Week two is more, okay, now assume that there's, how do we start thinking about context engineering at scale, beyond just vanilla prompting. And then I think coupled with that is just in doing that, thinking about tool design, MCP. You know, people, I think for a while MCP was getting a lot of hate. I'm still pro MCP. I see it all the time. And so I think that there's, at this point it's a standard that's existed and I think it will continue to exist. So talking about that effective tool design, that's week two. Week three I would say is more about just like skills and what does it mean to extend beyond simple single purpose tools to more like procedures, and that's where we'll talk about skills and that kind of starts leading into the automation of agents.

课程概览:定制智能体 Course Overview: Customizing Agents

Mihail

第四周主要是更多关于如何定制智能体的基础内容。这包括智能体配置文件、AGENTS.md、CLAUDE.md、钩子、子智能体等。现代框架中有很多不同的机制,你可以用于不同目的。所以我想让学生接触这些。

Week four is a lot of just more of the foundational pieces of how can you customize an agent? This is things like agent config files, AGENTS.md, CLAUDE.md, hooks, subagents. There are a lot of different mechanisms that exist in modern harnesses which you can use for different purposes. So I want to expose students to that.

智能体就绪代码库 Agent-Ready Codebases

Mihail

智能体就绪的代码库。这可能是我最喜欢的话题之一。这对那些进入工业界并思考大规模代码的人尤其相关。本质上,什么才算是一个准备好被智能体有效摄取和使用的代码库?所以要大量思考设计服务级边界的权衡,还有文档、可验证性之类的东西。这就是第五周。

Agent-ready codebases. This is probably one of my favorite topics. This is especially relevant for folks that are going into industry and thinking about code at scale. Essentially, what does it mean to have a repo that's ready to be ingested and consumed effectively by an agent? So thinking a lot about the trade-offs of designing service-level boundaries, but also docs and verifiability and things of that nature. So that's week five.

智能体代码审查 Agentic Code Review

Mihail

第六周是智能体式代码审查。这里我觉得有趣的是,从第五周延伸——第五周围绕智能体就绪的代码库——到第六周更多是:好的,为了——代码审查现在正成为瓶颈。我不知道学生们——除非他们做过很多——有些是硕士生,所以他们在工业界待过,但很多人没见过这个。但对于今天大多数使用代码智能体的人来说,我们都在经历代码审查的新瓶颈。那么我们如何解除这个瓶颈,或者稍微缓解瓶颈?有哪些机制?还有可验证性之类的东西,以及我们如何将其扩展到代码审查流程中。

Week six is sort of agentic code review. And here I think what is interesting is thinking also kind of extending from week five—week five around agent-ready codebases—going to week six is more about: okay, in order to—code review is now becoming a bottleneck. I don't know that students—unless they have done a lot—some of them are master's students so they've been in the industry, but a lot of them have not seen this. But for most people that are using code agents today, we're all experiencing something of a new bottleneck in code review. So how do we unblock that or sort of unbottleneck the bottleneck a little bit? What mechanisms are there there? And so verifiability and things of that nature, and how we can extend that into the code review process.

安全 Security

Mihail

第七周:安全。我的意思是,显然我认为有一个点——所以这也在去年的课上出现过,我想强调这一点。我的意思是,对于任何采取大量行动的自主系统,我们今年看到很多——尤其是很多东西被黑,不仅仅是 Vercel 的一些核心软件,还有 LM 网关完全被黑。所以我想花些时间谈谈安全中存在哪些新的攻击向量,以及我们如何缓解其中一些。

Week seven: security. I mean, obviously I think there was a point—so this was in last year's class as well and I wanted to reinforce this. I mean, with any autonomous system taking a lot of actions, we're seeing a lot of this year—especially a lot of things getting hacked, not least of which are just some of the core pieces of software from Vercel to LM gateways entirely. So I want to spend some time just talking about what new attack vectors exist in security and how we can mitigate some of them.

后台智能体 Background Agents

Mihail

第八周是后台智能体。所以这是我想开始涉及的:你基本上如何在许多不同的产品服务领域运行智能体?不仅仅是在传统的 IDE 中,还有基于代码库、仓库中的触发器,在 CI/CD 期间,甚至基于事件触发的东西,比如 SRE 风格的机器人。你甚至如何构建这样一个系统——我想过去一年是很多公司开始发布他们的文章的时候,比如 Ramp、Inspect——Spotify 有他们自己的版本,我想还有 Stripe Minions——所以所有这些人都写了非常酷的关于他们内部构建的信息。我想谈谈这个,让人们了解那是什么。

Week eight is background agents. So this is what I want to start touching on here is: how do you basically run agents in many different product service areas? Not just in what was traditionally an IDE, but also having things that are based on triggers in your codebase, in your repo, during CI/CD, things that get triggered even based on events, things like SRE-style bots. How do you even build a system like—I think this past year was when a lot of companies started releasing their posts like Ramp, Inspect—there was Spotify has their own version of this, I think Stripe Minions—so all these folks were doing really cool write-ups of their internal info they were building to do this. I wanted to touch on that, get people exposed to what that is.

团队级基础设施与角色演变 Team-Level Infrastructure and Evolving Roles

Mihail

是的,这一周更多是关于在现代世界中你在团队中会发生什么,以及团队级基础设施的核心部分。所以从 LM 网关到 MCP 门户,也就是大规模 MCP,然后还有如何思考不断演变的角色,我想说。所以这也许有点像软性的东西。就像现在是什么——而 vanilla 的、也许不是超级细致的说法是每个人都是构建者,但我想更深入一点,超越这一点,比如什么东西实际上被吸收了,哪些角色实际上要么在演变,要么在出现以前不存在的东西。学生应该思考什么是可辩护的技能组合,也许以前不是这样?所以更多是团队动态类型的考虑。

Yeah, this one is more on what happens when you're in a team in the modern world and sort of the core pieces of team-level infrastructure. So everything from LM gateways to MCP portals, which is like MCP at scale, and then also how to think about the evolving roles, I would say. So this is maybe a little bit like a soft thing. It's like what is now—and the vanilla, maybe not super nuanced thing to say is like everyone's a builder, but I want to go a little bit deeper beyond that and sort of like what thing is actually getting subsumed, what roles are actually either evolving or emerging that didn't once exist. What should students think about in terms of defensible skill set that's maybe wasn't true before? And so a lot more like team dynamic type considerations.

软件工厂与人类参与 The Software Factory and Human Involvement

Mihail

然后是的,这一切都朝着工厂发展,我认为这是今天很多人最关心的事情。一旦我们建造工厂,人类在其中还存在于哪里?什么是——新的瓶颈点在哪里,这些瓶颈点有意围绕人类参与,比如意图,比如建立可验证系统,以及在极限情况下会发生什么,当也许 80-90% 的代码完全由智能体驱动时?我们有点填补了所有这些。所以是的,不错的扎实的 10 周。

And then yeah, this all kind of builds up toward the factory, which I think is top of mind for a lot of folks today. Once we're building factories, where do humans still exist in all that? What is—where are the new choke points that are intentionally there around human involvement, things like intent, things like setting up verifiable systems, and what happens in the limit when maybe 80-90% of code is fully driven by just agents? Where we kind of filled into all that. So yeah, nice solid 10 weeks.

开放问题与实验 Open Problems and Experimentation

Host

我想说,其中一些是更成熟的模式,我认为人们需要知道并非所有这些都是定论的科学。

I was going to say, some of these are more established patterns than others and I think people need to know that not all of this is settled science.

Mihail

当然。但这一切——我会认可所有这些都是人们谈论的事情,你应该知道。我们只是——我们知道开放问题,我们可能不知道答案。

Sure. But it is everything—I'll endorse everything as things that people talk about and you should know. We just—we know the open problems, we may not know the answers.

Host

完全正确。是的。我的意思是,甚至想想——我的意思是,你可能也有自己的版本。我很好奇你怎么看哪些仍然不那么可辩护,但——或者不是不可辩护,而是不那么板上钉钉。但甚至像软件工厂——我们是在谈论黑暗工厂,还是在谈论这些半亮灯的工厂?我认为甚至那里也有一个光谱。

That's absolutely right. Yeah. And I mean, think even—I mean, you probably have your own version of this. I'd be curious to hear what you think is still not as defensible but—or not defensible but not as set in stone. But even like the software factory—are we talking about dark factories, are we talking about these sort of half-lights-on factories? I think even there's a spectrum there.

Mihail

是的。在那里,你知道,有些人在最前沿,比如 warps 和那些建造工厂并作为服务出售的工厂,甚至他们只声称大约 30% 多,甚至那也只是完全由基因驱动的贡献。那么在极限情况下这意味着什么?很难说。

Yeah. And there, you know, there's the people that are on the very frontier, like the warps and the factories that are building factories and selling them as services, and even they only claim like 30-some% of even that of fully, you know, genetically driven contributions. So what does that mean in the limit? It is hard to say.

Host

还有那些走得非常非常远进入黑暗工厂的人,然后出来说“我错了,伙计们”,对吧?所以就像——我们在——上覆盖过这个。

And there's people who went very very far dark factory and then came out saying I was wrong guys, right? So like—we covered that on the—

Mihail

只要意识到不是每个人——他们当时可能相信他们说的话,但他们有权改变主意,而且他们确实改变了。

Just be aware that not everyone—they probably believe the things they're saying at the time, but they have the right to change their mind and they do.

Host

甚至 Boris 最近发推说“是的,我经常犯错”。就像每个人都因为他是 Claude Code 的创造者而把他的的话当作福音,而我说不,有时他自己也不同意自己。

And even Boris recently tweeted that like yeah I am often wrong. Like everyone is like because he created Claude Code is like treating his word as gospel and I'm like no well sometimes he disagrees himself.

Mihail

是的。这没关系。我的意思是,这不是政治,对吧?这是——我们仍然是科学家。我们是工程师。我们受制于——我不声称自己是教条主义的,但你知道,我只是想让人们去实验。我认为这也是一条原则。你必须有原则,就像实验一样。我的意思是,你只需要准备好——很多时候当我做这些讲座或与人们谈论这些事情时,他们会说,我如何——有这么多事情在发生。我如何跟上?就像,你知道,有很简单的事情,比如每天实验 10 分钟。

Yeah. And that's okay. I mean, like this isn't politics, right? This is—we're still scientists. We're engineers. We're subject to—I don't claim to be dogmatic, but you know, I just want people to experiment. I think that's like a principle as well. You have to have principles earlier. Just like experimentation. I mean, like you just have to be ready to—and often times when I sort of give these lectures or talk to people about this stuff, they're like, how do I—there's so much going on. How do I stay on top of this? And it's like, you know, there's something as simple as just like experimenting for 10 minutes a day.

在实践中学习 Learning by Doing

Mihail

比如花 10 分钟做一件你以前没做过的事。如果那是你过去解决问题的方式,现在有了新工具或新框架,哪怕只是用新东西做完全相同的问题,单是这一点就能弥补大部分差距。

Like just spending some 10 minutes doing something that you didn't do before. If that's a traditional way that you were solving a problem, there's now a new tool you want to try or a new framework or something and just spending maybe even doing the exact same problem just with that new kind of thing is alone will cover most of the gap.

Host

是的。我要提一下,比如你提到的 Cognition,就在你谈论代码审查的那一周。

Yeah. I will call out so some of your, for example you have Cognition on the week that you're talking about code review.

Mihail

我实际上和 Silus 一起做过 Devon 审查。

I actually did work with Silus on Devon review.

Host

是的,是的。

Yeah. Yeah.

Mihail

我会说相当不错,你知道这些事很难基准测试。有一个审查基准,但没人用。不过从质量上说,Devon 审查相当好。

Pretty good I would say like you know it's hard to benchmark these things. There is a review bench. No one uses it. Um, but like qualitatively Devon review is pretty good.

Host

嗯,我们用你们。那也是,是的。所以 Monaco 确实用 Devon。哦,那是,我是说 Sus 去年也讲过,所以 Sus 很棒。

Well, we use you guys. That's also Yeah. So, so Monaco does use Devon. Oh, and that's that's I mean and Sus actually spoke last year and so I mean Sus is great.

Mihail

他非常非常聪明。但嗯,是的,我们用你们,所以我想好吧,在代码审查工具提供商中,我们抽样了很多。嗯,我们实际上在很多用例上确实收敛到了 Devon。

It's just super super smart. But um yeah, we we use you guys and so I was like okay among among the providers of of kind of code review tools that we and we sampled a lot of them. Um we actually did kind of converge on Devon for a lot of use cases.

Host

除了实际上现在更多是模型侧,然后实际上你想要的是 Walden。

Besides actually more model side these days and then actually it's Walden that you want.

Mihail

是的,是的,是的。我,是的,我也在和 Walden 聊。我还要提一下 factory,因为谈到智能体就绪度,他们一直专注于此,他们构建了信号工具,你们用吗?有什么看法?

Yeah. Yeah. Yeah. I Yeah, I was talking to Walden as well. I will also shout out you know from factory because uh you know talking about agent readiness they've been all about that they build signals tooling on that do you guys use that opinions?

Host

是的。所以我们确实用自己的智能体就绪度。我是说,Eno 也很棒。我觉得他们在这趟列车上非常早,当他在演讲中将其编码化,我猜是在 AIE,那是什么时候,我忘了,但他的一个,是的,Europer,非常棒的演讲,我觉得在某种方式上超前于时代,所以我想只有一个人我能看到是最适合谈论这个的,我真的很想给他们聚光灯,他们非常兴奋。

Yeah. So so we do use agent readiness of our own of our own kind. I mean, Eno is is awesome as well. Like I I think they were super early on this on this train and I think when he codified that in his talk, I guess at AIE, what was that like last I forget when it was, but one of his Yeah. Europer um just fantastic talk and I think it was kind of in a way ahead of its time and so I was like there's only one person that I could see being like the most most appropriate person to talk about this where I like really want to give them the spotlight and they're super excited.

Mihail

是的,很有趣。我当时在 COG,我们在讨论这些东西,我们说好吧,我们需要形成一些观点,然后 factory 就在我们讨论的那周发布了一个完整产品,我想好吧你赢了。

Yeah, it was funny. I was at COG. we were talking about this stuff and like we're like okay we need to form some opinions about it and then like factory like shipped a full product on it that week that we were talking about it I was like okay you won

Host

就像我的意思是,大家差不多都同意同样的事情。

like I mean like everyone kind of agrees on the same things

Mihail

问题只是你想不想把它变成仪表盘。

and it's just a question of like do you want to turn it into a dashboard

Host

是的,是的,是的。

yeah yeah yeah

Mihail

因为它是仪表盘。

because it's a dashboard

Host

对,对,但你知道有价值,我觉得他们从营销角度做得很好,抓住了半径报告,我肯定和,你知道我会经常向别人推销这个,说你应该运行自己的报告,当然现在人们有开源版本,但我认为原则非常重要。

right right but you know there's there's value and like I think they did well from a marketing perspective jumping on like the the radius report and I've definitely spoken to you know I I will regularly kind of pitch that to other people and and say like yeah you should run reports of your own ind and um you know of course now people have open source versions of it but but I think the the principles are super important.

Host

是的。嗯,你想逐条讨论还是你有其他事情?我可以逐条讨论。

Yeah. Uh do you want to go piece by piece or do you have any other things? I I can go piece by piece.

Mihail

是的。好的。所以第一个,你关注 Claude Code,但你也让 pi 作为开源赞助商。怎么回事?好的。所以这个可能更多是营销,但好吧,我想强调一点,我去年也明确说过,我不代言任何工具。好的。我不,我认为那是错误的方式。工具来来去去,透明地说,去年教这门课时,Claude Code 没那么流行。它没那么被接受。Boris 只有 4000 个推特粉丝,现在他有几百万。我甚至不知道是不是,但你知道他有很多粉丝。所以 Claude Code 确实崭露头角。我认为

Yeah. Okay. So the first one right uh you focus on Claude Code but then you also have pi as open source sponsor. What's up? Okay. So so okay this one is maybe a little bit more marketing but but okay um the one thing I do want to highlight and I've made that I also made that clear last year but I'm not a I'm not endorsing any tool. Okay. like like I I don't I I I think that is uh the wrong way to approach something like this. Um there are tools that come and go and when you know transparently when when the class was taught last year Claude Code was not as popular. It wasn't as like rarely accepted. Boris was only at 4,000 Twitter followers and now he's at you know whatever million. I don't even know if he is but you know like he's at a lot of followers. Um so so Claude Code has really like emerged on the scene. Um I think

Host

我认为是的,我认为我们在播客上请 Boris 和 Cat 时也低估了 Claude Code。它当时才 2 个月大。我没真的,我用过一次。一次查询花了我 50 美分。我想谁会用它。完全。

I think yeah I think we we also underestimated Claude Code when we had Boris and Cat on the podcast. It was like 2 months old. I hadn't really like I used it once. A query cost me 50 cents. I was like, who's going to use this Totally.

Mihail

它还在测试阶段一段时间。但我认为他们好在在模型准备好之前就构建了它。就像你听到很多为模型未来方向而构建。

It was still in beta for a while. But I think it's good on them for how they built it before the model was ready. Like you hear so much build for where the model will go.

Host

完全。

Totally.

Mihail

超级雄心。模型当时没那么好。你感觉不到。然后我们看到它达到了那个位置。

Super ambitious. Like models were not that good. You couldn't feel it. And then ah we see we see that it's where it came

Host

是的。有趣的是,去年他演讲时,我记得那是一组非常高层级的幻灯片,但我记得其中一张幻灯片字面上就是为模型 6 个月后的位置而构建的。那是去年 10 月。我能理解他为什么提出这一点,如果他知道 6 个月后 Opus 会达到什么位置,因为那也是我认为的巨大解锁,我会说那是课程变化如此之大的原因之一,对吧?确实存在围绕这些智能体中真正自主性的巨大军备竞赛,比如长时间运行的智能体式编码任务。但是的,回到你的点,少谈我不代言 Cloud。我的意思是,你知道 Claude Code 我当然用,也用其他东西,但这里我更多是关于现代编码工具链是什么样子。我写了一篇博客文章,实际上是的,这篇,今年早些时候,我分解了一些类似的原则。太长不看版是,你可以通过几乎入门算法级别的理解,基本上定义一个智能体循环,完成你所谓的编码智能体的大部分工作。当然有花哨的东西。没有子智能体,有 MCP 等等。但如果你想看到你与之对话然后它字面上编辑你的文件,你可以用它做相当复杂的事情。它不需要超过 200 行。所以你知道,我在教育中尝试做的很多是剥离那些秘方,展示其核心。我的意思是,这些仍然是学生,你想教概念原则。这没什么比斯坦福有一门叫 106b 的课更复杂的。那是入门算法。你不需要超过 106b 就能构建这样的东西。所以就是这样。就在这里。

that it's Yeah. It's funny because when he gave his talk last year, I remember um you know it was like a very high level sort of set of slides but I remember one of the slides was literally like built for where the model is going to be 6 months from now. And this was like October of last year. And I can see why he would have made that point if if he knew in 6 months where like Opus was going to be cuz that was also what I think was a huge unlock where I would argue was one of the reasons why the curriculum changed as much as it did, right? Where there was certainly this huge arms race around true like autonomy in these agents like longunning agentic coding agent tasks. But yeah, so so you know back to your points like less on I'm not endorsing cloud. I mean you know Claude Code I of course I use it and and use other things as well but but here I was more about like the what does a modern coding harness look like. I wrote a blog post that actually uh yeah this one where this was yeah earlier this year where I sort of broke down some of some similarish principles. The TLDDR was that you know you could sort of through like pretty much introductory algorithms level understanding you could just define like essentially a gentic loop that does the bulk of what you would call a coding agent. Um, and of course there's like bells and whistles. There's no sub agents, there's an MCP and none of that stuff. But if you want to see something that you talk to and then it literally will edit your file and you can do pretty sophisticated things with it. You, it doesn't require more than 200 lines. So I, you know, a lot of what I try to do in education is to try to like peel away the stuff that's like the secret sauce and show that like really at its core. I mean, these are still students and you want to teach conceptual principles. There's nothing more to this than in Stanford there's this class called 106b. It's like the introductory introduction algorithms. You don't need more than one 6B to build something like this. And so there you go. There it is.

从零教授harness Teaching the harness from scratch

Host

既然你已经教过这门课了,有没有遇到什么反对意见、问题,或者你从学生身上观察到的奇怪之处?他们哪里没搞懂?他们的问题似乎集中在哪些方面?

Given that you already taught this class, any pushback, any questions, any weirdness that you observe from the students? What do they not get? What do their questions seem to cluster around?

Mihail

你是说就现在这个特定的概念,还是整体上?

You mean like now that this particular concept, or just in general?

Host

就是这个概念——从零开始搭建,然后做你做过的事,因为这基本上就是你做 Pi 所需要的东西。

This concept of building it from scratch and then doing what you did, because this is basically what you needed for Pi.

Mihail

对,对,对,没错,对,是的,是的,对。所以,其实并不是——我想我完全没有收到对这个想法的任何反对意见。我觉得他们其实更多只是——他们大多数人都是消费者,对吧?大多数学生最终就只是使用这些工具。我不认为他们有很多机会——如果有的话——尤其是在课程作业里——去把这层洋葱剥开,对吧?所以这周是讲 harness 的一周。周二第一节课,你知道,讲些后勤安排之类的。但前半部分——课程真正开始的第一块内容——就是那 200 行的 harness。然后昨天我做了后续那节课,我们实际上拿了 Claude Code,然后你知道有这些代理工具,你可以拦截请求——

Right. Right. Right. Exactly. Right. Yeah. Yeah. Right. So, yeah, it's not really—I think I haven't received any pushback on this idea at all. I think they're actually more just—most of them are consumers, right? Most of the students do just end up using the tools. I don't think that they get a lot of exposure, if any, certainly in the coursework, around peeling that onion back, right? And I think—so this was the week of the harness. So Tuesday first lecture, you know, logistics, blah blah blah, all that stuff. But then first half is—you know, the first true bit of curriculum is the 200 lines of a harness. And then yesterday when I had sort of the follow-up to that, is like we actually took Claude Code and then you know you have these proxy things, these proxy tools where you can intercept the request—

Host

代理。

Proxy.

Mihail

对,没错,没错,对。用那个工具拦截请求和响应,看看底层,看看那里发生了什么。然后——然后就像,好吧,我确实说过你可以在 200 行里实现 Claude Code,但说真的,好吧,这才是真正的 Claude Code。所以我 literally 做的就是,我拿了 Opus 5.5——就是那个,你知道,才出来两天的模型——然后说,这是——我拦截了那些请求,这就是确切发生的事,这是提示词,这是系统提示词,这是规划提示词,这是压缩提示词。然后我们就过了一遍所有这些,展示给学生看。所以现在他们真的真的有了一个他们能理解的最先进的 harness。

Yes. Exactly. Exactly. Right. To use that tool, intercept requests and responses, look under the hood, see what's happening there. And then—then like, okay, I did say you have Claude Code in 200 lines, but really, okay, here's like actual Claude Code. And so what I literally did is I took Opus 5.5—so this like, you know, model 2 days old—and was like, this is—I intercepted those requests, here's exactly what's going on, here's the prompt, here's the system prompt, here's the plan prompt, here's the compaction prompt. And then we just like went through all of that and sort of showed it to students. So now they really, really have like a state-of-the-art harness that they can understand.

平衡学生技能水平 Balancing student skill levels

Host

是啊,能呈现这三个层次挺好的——学生大多只是消费者,这是循环里的基线智能体,然后这是真正的那个。我也喜欢你给博客文章配了视频讲解。你看到学生是不是处于两个极端——有些人是重度用户,他们消费很多。对有些人来说,你可以假设如果你懂智能体怎么运作,这就很直截了当,对吧?你在课堂环境里怎么平衡这一点?比如你会针对平均水平来设计吗?

Yeah, it's nice to get those three levels—students are very much consumers, here's the baseline agent in a loop, and then the real one. I like how you have a video walkthrough of your blog post as well. Do you see students are on extremes—some people are power users, they consume a lot. For some people, you could assume that this is kind of very straightforward if you know how agents work, right? How do you kind of balance that in the classroom setting? Like do you tailor towards the average?

Mihail

是啊,我觉得这有点自我筛选的性质,因为课程内容都在这里,对吧?所以课程内容已经可用——我是说,你知道,至少有几周了,我猜我几周前宣布的,但它大概至少酝酿了一两个月。所以如果你去找,你大概能找到。我觉得现实是,如果学生对这个不感兴趣,他们就不会——他们就不会来,他们会选择不来。我觉得有意思的是,仍然肯定有学生会来,但也有很多来自更广泛的斯坦福的人——其他院系的博士后,有很多跨学科的兴趣,比如在病理学或医学领域的人,甚至其他非纯计算机科学的工程领域的人,也来参加,也对接触这个感兴趣。还有很多商学院的学生、MBA,他们也想,我猜,学习如何不需要技术联合创始人,或者不管动机是什么。

Yeah, I think it's a bit self-selecting because the curriculum is all here, right? So the curriculum has been available since—I mean, you know, at least it's like a few weeks, and I guess I announced it a few weeks ago, but it had kind of been in the works for like probably at least one to two months. So if you looked for it, you could probably find it. And I feel like the reality is if students are not interested in this, then they won't—they'll just won't show up, they will choose not to show up. What I'm finding that is interesting is there are still definitely students that show up, but then there's also just a lot of folks from, you know, broader Stanford—postdocs at other departments, there's a lot of interdisciplinary sort of interests where folks that are in, you know, somewhere in pathology or like medicinal sort of fields, or even like random other engineering fields that aren't pure CS, that are coming in and also interested in kind of getting exposed to this. Also a lot of GSB students, MBAs, that also want to, I guess, learn how to not have a technical co-founder, or whatever the motivation there is.

MITM代理练习 The MITM proxy exercise

Mihail

但是啊,所以我觉得——顺便说一句,如果谁还没做过中间人代理那个练习,你应该去做。

But yeah, so I think—by the way, if anyone hasn't done the MITM proxy exercise, you should do it.

Host

是的,你会震惊的。Claude Code 的系统提示词是所有编程智能体里最长的。大概 2 万个 token。

Yes, you'll be shocked. Claude Code has the longest system prompt of all the coding agents. Something like 20,000 tokens.

Mihail

是啊,是啊。不过——是的,那是真的,那是真的。是啊,是啊,是啊。不过他们也把体积缩减了 80%。

Yeah. Yeah. Although—yeah, that is true. That is true. Yeah. Yeah. Yeah. Although they've also reduced the size by 80%.

Host

是啊,我知道。但然后它们又长回来,然后他们又砍,然后又长。

Yeah, I know. But then they grow again and then they cut and they grow again.

Mihail

这是个跷跷板。

It's a seesaw.

Host

是啊,是啊。不,我觉得这很有意思。你会惊讶里面放了什么,其实有时候你根本用不到。

Yeah. Yeah. No, I think it's interesting. You'll be surprised like what goes in there that actually sometimes you don't use.

Mihail

有时候它其实暗示了某些工具的内部运作方式。

Sometimes it actually hints at the internal workings of some of the tools.

Host

你在那里发现了什么值得一聊的东西吗?

Anything that you found there that could be cool to talk about?

模型与harness的权衡 Model versus harness trade-off

Mihail

是啊。不,我是说,其实昨天我讲这个的时候,就有点这个意思——我觉得当年 Claude Code 最有趣的一点,也是它为什么在人气上真正爆发的原因之一,就是他们对简洁的专注,对吧?所以在早期,他们是最早真正倾向于“让更少的工具做更多事”的那批人,对吧?倾向于 SQL,倾向于 Bash,倾向于那些强大的东西,像瑞士军刀一样的工具,让这些成为能力的主要驱动力。我觉得他们在这方面做得非常出色。我觉得这就是他们为什么那么受欢迎。我觉得当我展示系统提示词之类的东西时,是这样的——大约两个月前有一个不同的系统提示词。现在他们降低了复杂度,结果现在曾经属于系统提示词的一部分内容,比如甚至如何使用 git,对吧?或者如何设置任务之类的东西,不再是系统提示词的一部分了。实际上现在更多期望这些被模型本身捕获,比如模型权重本身。所以他们在模型层面把很多这些学习成果烘焙进去了。这就引出了有趣的问题,你知道,模型与 harness 的对比,对吧?你怎么思考这种权衡?公司该投资什么?你应该投资什么才能真正推动你自己能力的前沿?所以我觉得我们,甚至作为一个社区,也在这两者之间来回摇摆很多。

Yeah. No, I mean, in fact yesterday when I was talking about this, it was a little bit of this—one of the things that I thought was most interesting about Claude Code back in the day, and I think part of why it really blew up in terms of popularity, was their focus on simplicity, right? So in the early days they're among the first to really lean into this 'let's fewer tools do more,' right? Lean into SQL, lean into Bash, lean into the things that are powerful, like Swiss Army knife tools, and have that be a lot of the driver of capability. I think they did a fantastic job leaning into that. I think that's why they were as popular as they did. I think that when I presented something like the system prompt, it was like, well, there was a different system prompt about 2 months ago. It is now—they have reduced in complexity, and it turns out that now what was once part of the system prompt, things like how to use even git, right? Or how to set up tasks or things of that nature, are no longer part of the system prompt. It's actually expected to be more kind of captured in the models, like the model weights itself. So they've kind of baked in a lot of those learnings more at the model level. And that just raises interesting questions around, you know, the model versus the harness, right? And how do you kind of think through that trade-off? What do companies invest in? What should you invest in to really push the frontier of your own capabilities? And so I think we still, even as a community, we seesaw between the two a lot.

系统提示为何增减 Why system prompts grow and shrink

Host

我觉得你不该因为他们先扩大系统提示词然后又砍掉而责怪他们。在某种程度上,这也取决于模型本身是什么样,对吧?更弱的旧模型,你需要给它们例子,这是好的,这是坏的。然后显然你在某个时候把这些砍掉,对吧?这只是学习已经做过的事情的一部分,对吧?你可以把同样的道理应用到——如果你用一个不是最强模型的本地编程智能体,你可能需要加上这些例子,对吧?

I think you don't want to fault them for growing a system prompt and then cutting it. At some level, it's also how the model was, right? Weaker older models, you need to give them examples of this is good, this is bad. And then obviously you cut that away at some point, right? And that's just part of learning what has been done, right? You can apply the same towards if you're using a local coding agent with not the strongest model, you might need to add these examples, right?

Mihail

没错。对的。是啊,是啊。

That's right. That's correct. Yeah. Yeah.

课程预告与一小时版 Course Trailer and One-Hour Version

Host

当然,从中也能得到一些有趣的 AI 经验,这就是为什么随着模型变强,你会思考:什么时候加入那些示例反而是在过度约束它,对吧?然后你又如何撤掉这种过度约束,我觉得这也是学习的一部分。好,第二周你要讲上下文工程。RPI 我没看过,但规格驱动开发有很多版本。你想向更广泛的受众强调这里面的哪些内容,是大家应该知道的?

And then, of course, there are interesting AI learnings to fall out of that, which is why, as the models get better, when are you over-constraining it by adding those examples, right? And then how you pull away the over-constraining, and that's, I think, part of the learning as well. Okay, week two, you are going to talk about context engineering. RPI—I haven't seen it, but there are many versions of spec-driven development. What would you highlight to a broader audience about what you're putting in here that people should know?

Mihail

对,这就像是这门课的一小时版本。

Yeah, this is like the one-hour version of the course.

Host

对,就是这样。请把所有内容浓缩一下。这就是它的定位。

Yeah, exactly. Like please condense everything. That's where this is going.

Mihail

有道理。对。

That's fair. Yeah.

Host

不,它其实也是个预告片,对吧?就是说,如果你想要更多,明年或明年秋天报名课程。你知道,这就是你的一小时。

Well, no, it's also a trailer, right? It's like, well, if you want more, sign up next year or next fall for courses. You know, this is your one hour.

Mihail

这几乎像是课程界的 Quibi,你懂我意思吧?当预告片太长,当课程又太短。

This is almost like the quibi of the class, you know what I mean? It's too long to be a trailer. It's too short to be the class.

Host

我猜人们会——如果它是预告片,等你下次再教的时候,它又会变了。

I guess people will part—if it's a trailer, by the next time you teach it, it'll change again.

Mihail

没错。没错。没错。没错。对。好吧。好,就这么叫吧。挺有意思的。

That's true. That's true. That's true. That's true. Yeah. Well, yeah. So, okay. So, good, call it. It's funny.

超越普通提示的上下文工程 Context Engineering Beyond Vanilla Prompting

Mihail

我在这里最想强调的一点是——甚至在我面试别人的时候也能看到——业内很少有人能超越最基础的那种提示词,对吧?就是那种很朴素的写法,比如“嘿,帮我做那件事”,对吧?就只是“做那件事”。当然,随着模型变强,这能带你走一段距离。但不可避免地——我跟任何从 Google 之类有庞大代码库的地方出来的人聊,现实就是你必须更用心地思考如何把上下文喂进去,对吧?

The main thing that I just want to reinforce here is—and I see this even when I'm interviewing folks—the very few people in industry even progress beyond just vanilla prompting, right? There's like a vanilla form, like, hey, do me the thing, right? Like, do the thing. And that certainly, as the models get good, can carry you a certain distance. But inevitably—and any folks I speak to that come out of like a Google or some place where there's a massive code base—the reality is you have to be a lot more thoughtful about how you're ingesting context, right?

Mihail

所以要思考围绕如何指向——我描述的 RAPID 框架在功能上类似于 Dex 的 RPI,对吧?或者类似的东西,就是你先做研究,然后有个提案,再有个规格,然后做实现,最后用一些测试之类的把它串起来。

And so thinking through what are some of those principles and those techniques around how to point—and I sort of like—the RAPID framework that I'm describing here is functionally similar to Dex's RPI, right? Or something of that nature, where you have this research, you kind of have a proposal, you sort of a spec, and then you do your implementation, and then you kind of tie it together with some testing and whatnot.

Mihail

所以我在这里想强调的是,你应该有这一步,尤其是当你进入不熟悉的代码库时,当你试图把一个智能体需要理解的、关于一个超大代码库的东西注入上下文时。研究阶段是有价值的,然后在我提出的这个特定框架里,研究完之后还要给出一些提案,列出解决问题的几种备选方案。

So what I just wanted to reinforce here is that you should have this step, especially when you're entering unfamiliar code bases, when you're trying to infuse into the context something that an agent needs to understand about a very large codebase. There's value in something that is a research phase, and then in the case of this particular framework that I'm putting forward, it's like also then suggest some proposal where there's a few alternative ways of solving a problem once you've done the research.

Mihail

所以这也是我邀请人类参与进来、在一种方案和另一种方案之间做决策的地方。然后你进入计划阶段,也就是规格。总而言之,一句话:确实有一种做法能真正扩展到数百万行长的代码库。它不只是最基础的提示词。我想强调的就是这一点。如果你要在这些大代码库里工作,你应该用稍微更复杂一点的方法。

And so this is where I sort of also invite the human in the loop to sort of be the decision maker on one approach versus the other. And then you enter the plan, which is kind of the spec. All to say, the TL;DR is like there is a way to do this that actually does scale out to code bases that are, you know, millions of lines long. And it's not just the vanilla prompt. And I think that's what I want to highlight. If you're going to go work in these big code bases, you should use something a little bit more involved.

子智能体与多智能体上下文管理 Sub-Agents and Multi-Agent Context Management

Host

关于上下文工程还有什么要补充的吗?

Anything to add on context engineering?

Mihail

我觉得我现在对上下文最奇怪的一点,是当你拆分出一个子智能体的时候,因为子智能体可能受益于继承主线程的全部上下文,也可能不受益。有时候大家对此经常有分歧。然后有时候你想把子智能体做的事压缩一下,研究者再把它放回主线程,这非常取巧。我做的任何事我都不满意。所以我没有答案。我只知道——

I think the probably the weirdest thing about how I think about context right now is when you split out to a sub-agent, because sub-agents can benefit from inheriting your entire context of the main thread or not. Sometimes there's a regular disagreement on that. And then sometimes you want to condense what the sub-agent does and researchers put it back onto the main thread, and it's very hacky. I don't like anything I'm doing. So I don't have any answers. All I know is like—

Mihail

当你想把工作铺开时,它不只是你有一个大规格、你做 RPI、你做 RAPID 之类的。它不再是一个智能体了。它关乎把工作铺开,并确保它们都有正确的上下文、都有正确的指引。如果你有一个审查智能体来审查其他所有工作,你得让那家伙拥有一切。而你总会经历这个不断扩充契约的阶段,比如“天哪,这些家伙上下文不够,我要给他们更多上下文”。好,很好。然后他们就会过度迎合你之前随口说的一句评论,然后犯一些非常愚蠢的错误,因为你并不是那个意思,但他们把它当成了法律,对吧?

When you want to fan out work, it is not just about you have a big spec and you do RPI, you do RAPID, whatever. It is not one agent anymore. It is about fanning out work and making sure they all have the right context and they all have the right guidance. If you have a review agent reviewing all the other work, you have to make that guy have everything. And you always go through this phase of expanding contracts like, “Oh my god, these guys don't have enough context. I'm going to give them more context.” Okay, great. And then they're going to overtune to some random one-off comment that you made a while ago and then make some really stupid mistakes because you didn't mean it that way, but they took it as the law, right?

Mihail

差不多就是这样。所以,我也不知道我在说什么词,对吧?这就像是针对上下文的多智能体管理。

Something like that. So, I don't know what the word is I'm saying, right? This is like multi-agent management for context.

Host

对。对。对。对,完全正确。这里面有些方面——这可能是最偏艺术而非科学的部分,对吧?我觉得是某种组合,正如你所说。是的,为探索而铺开,为子任务而铺开,然后是整合、压缩,随你怎么叫。我觉得要用心思考上下文的刷新,理解什么时候东西开始变得有点臃肿。我觉得 Dex 有个“愚蠢区”的概念,要对这个愚蠢区更用心一些。

Yeah. Yeah. Yeah. Yeah, that's absolutely right. There's like some aspect of—this is maybe the part that's more among the most art than science, right? It's like some combination, I think, to your points. Yes, the fan-out for the exploration, the fan-out for the subtasks, and then the consolidation, compaction, whatever you might want to call it. I think being thoughtful about refreshing of the context, understanding when things are getting a little bit too bloated. I think Dex had this concept of like the dumb zone and being a little bit more thoughtful about the dumb zone.

Mihail

他很擅长这种打比方的东西。

He's good at these equating things.

Host

哦对,我觉得那些小图。

Oh yeah, I think little diagrams.

Mihail

那些小——对。哦对。你为什么请他来做客?

The little—yeah. Oh yeah. Why did you have him on?

Host

呃对,我觉得他——我是说,他确实对这里的很多课程内容有启发,所以他是个很棒的思考者,诸如此类。不,我觉得那些图很重要。

Uh yeah, well I think he's—I mean, he definitely informed a lot of the curriculum here, and so he's just a fantastic thinker and all this. No, I think the diagrams are huge.

Mihail

那很有用。

That's useful.

面包屑与上下文窗口历史 Breadcrumbs and Context Window History

Mihail

我会给想研究这个的人加一些线索。看看上下文工程的最佳实践偏好是好的,比如关于子智能体继承——Claude 托管智能体有一个版本是会话分离、不继承的。在推理模型出现之前,当智能体运行时间没那么长的时候,这曾是个热门话题。我们当时的上下文窗口更短。每个人都非常非常小心地管理那宝贵的上下文窗口,对吧?LangChain 以前写过很多关于这个和良好上下文工程的博客。这些东西对大家来说都值得了解。

I would add breadcrumbs for people that are trying to look into this. It's good to see best context engineering preferences, like on your sub-agent inheriting—Claude managed agents has a version where sessions are separate and it's not inherited. This used to be a big talk topic before reasoning models, when agents didn't run as long. We had shorter context windows. Everyone was very, very managing that precious context window, right? LangChain used to write a lot of blogs about this and good context engineering. All this stuff is good for people to know.

Mihail

我觉得在学生这个语境下也一样,你可以预期普通学生来上你的课,因为这是门好课,他们会知道所有这些。所以你也应该更了解它们。这也是个有趣的研究领域。在 AIE 上展示的一篇研究论文就研究了 Claude Code 系统提示词的变化。

I think in the student context as well, you can expect the average student to take your class because it's a great class, and they'll know all these things. So you should know them better as well. This is a fun space for research as well. One of the research papers presented at AIE looked at the delta in the Claude Code system prompt.

研究与上下文窗口管理 Research and Context Window Management

Mihail

他们改变了什么,并绘制了一大堆性能变化。他们试图逆向工程它。这只是你可以做的很好的研究。还有另一篇也研究了上下文窗口管理及其如何影响下游性能。它们不是最好的论文,但相当容易理解,而且发布了很好的博客文章,所以对于那些试图研究它的人,我只是推荐这是一个你可以轻松涉足的好领域。

What got changed and they plotted a whole bunch of changes in performance. They tried to reverse engineer it. It's just good research that you can do. There was another one that also looked at context window management and how it affects performance downstream. They weren't the best papers, but it's quite accessible and it puts out good blog posts, so people that are trying to work on it, I just recommend it's a good area you can easily poke into.

Host

是的,这是一个很好的推荐。

Yeah, it's a good shout out.

Mihail

智能体技能 CLA。

Agent skills CLA.

Host

是的,是的。我认为所有这些都是对单一功能工具调用(如 MCP)的一点扩展,对吧?使用与第三方外部服务交互,然后扩展到所谓的过程,对吧?比如你如何一遍又一遍地自动化某些动作,然后以技能 MD 或类似的形式将其形式化。所以我认为只是让这一点更……给人们练习编码一些过程。我想我还会在这里包括一些我觉得好用的有趣技能,比如我认为 Matt Pox 的 grill me 就是一个好技能。然后还有一些我从教育角度更喜欢的,比如 Jeffrey Litz,他有 explained diff,我认为一旦你开始谈论代码审查,这就变得特别有趣,因为你知道整个

Yeah. Yeah. I think all this is a little bit of an extension from single function tool calling like MCP, right? Using interacting with third party external services and then extending into what you would call procedures, right? Like how do you automate some set of actions over and over again that you then formalize in the form of a skill MD or something like that. So I think just making that a little bit more, giving people exercise in codifying some of those procedures. I think what I'll also sort of include here is a few fun skills that I think are good to use like I think Matt Pox's grill me is a good one. And then there's ones that I also like from more of the educational perspective like Jeffrey Litz, he has explained diff which I think becomes especially interesting once you start talking about code review because you know the whole

Mihail

理解代码。

Understand the code.

Host

理解代码。没错。是的,你必须理解代码。我不知道你们是否用过那个,但有效的是你运行,你有你放入代码的差异,然后你运行技能,最后你不仅有发生的事情的可视化表示,这样你可以浏览并理解其核心内容,而且我真正喜欢的是最后几乎像测验一样,你通过它真正强化你是否理解正在发生的事情。我认为这种交互式的来回相当不错。

Understand the code. Exactly. Yeah, you got to understand the code. And I don't know if you guys have used that, but effectively you run, you have your diff you've put in your code and then you run the skill and at the end of it you have this not only a visual representation of what's happened so you can skim it and understand what's actually at the core of it, but then what I really liked was this almost like quiz at the end of it where you go through and really reinforce whether you understand what's happening. And I think that level of interactive back and forth is quite nice.

Mihail

是的,在这个领域有几篇博客文章我非常想提一下。我认为最近有一篇,说的是在任何情况下关闭大脑都不是好主意。我认为这些技能实际上迫使你开启大脑。而当每个人都想使用编码智能体时,却关闭大脑。他们就像‘给我做个应用,读我的心,猜对一切,不用我告诉你’。

Yeah, there's a couple blog posts in this domain that I really like to bring up. There's one I think recently which is like there's no situation in which turning your brain off is a good idea. And I think these skills actually force you to turn brain on. And when everyone wants to use coding agents turn your brain off. They're just like make me an app and read my mind and guess everything correct without me telling you about it.

Host

没错。但很多这些技能大多属于‘不不,请多问我一些问题’的类型。

That's right. But like a lot of these skills are mostly in the genre of like no no like ask me more questions please.

Mihail

我认为输出比人们习惯的要多得多。比如我可以输入一行提示,得到数万行代码,而且都是结构化的。

I think there's a lot more output than people are used to. Like I can type in a one-line prompt and get tens of thousands of lines of code and all structured.

Host

就像‘看看我的生产力,伙计们’。

Like look at my productivity guys.

Mihail

但你知道更有效的是你应该知道你在做什么。你真的可能会脱节。你应该花更多时间思考你实际在做什么。做工作和写代码比以前容易多了。

But you know more effectively you should know what you're working on. You can really be put out of touch out of the loop. You should spend more time thinking about what you're actually doing. Doing the work and getting the code written is much easier than it was.

Host

是的,是的。理解,这肯定没有借口。

Yeah. Yeah. The understanding there's no excuse for sure.

Mihail

抱歉,什么是网络技能?

Sorry. What is a web skill?

Host

是的。所以我的意思是有效地与网络本身交互的技能。计算机使用

Yeah. So I mean effectively like skills that interact with the web itself. Computer use

Mihail

计算机使用。是的,类似的东西。

Computer use. Yeah. Things like this.

Host

还有别的吗?

Anything else?

Mihail

呃,我主要是指涉及从网络拉取信息的事情。我认为这是

Uh I mean mainly things that involve pulling information from the web. I think it's

Host

有没有关于与网络交互的技能的最佳实践,相对于

Are there best practices on skills that will interact with the web versus

Mihail

我认为这更像是一个调查或演示,展示了我发现的一些有趣的东西。我想可能是来自 Vercel?我现在试图回忆。我得回头看看完整的阅读材料,但我发现有一个网站,我认为他们可能实际上是基于浏览器的。他们可能做了

I think this is more of a survey or a demo of stuff that I found that was kind of interesting. I think it might have been out of was it like a Vercel? I'm trying to remember now. I'd have to look back at the full reading materials, but there was like one site that I found where I think they maybe it was browser based actually. They might have done

Host

你记得这个吗?

Do you remember this?

Mihail

是的。嗯,应该是他们这个东西。我只是想我是不是错过了什么?我认为这是技能社区的重要部分。很好。他正在接受教育。

Yeah. Well, it would be them that this thing. It's just I was like did I miss something? I think this is like a big part of the skills community. Good. He's being educated.

Host

是的,我认为他们做了某种……因为我猜他们是……是的,我的意思是浏览器基础显然像

Yeah, I think they like made some sort of because I guess they were Yeah, I mean browser base obviously like

Mihail

他们喜欢浏览器。

They like browsers.

Host

他们喜欢浏览器,但是是的,他们有一个很好的网站,我认为他们展示了一些有趣的能力。

They like browsers and but yeah, they had like a good I think website where they showed some of these interesting capabilities.

Mihail

看起来合理。CLI 是一个有趣的。你有很多学生,或者这是他们第一次在终端中使用 CLI,还是现在这已经成为常态?

Seems legit. CLI is a fun one. Do you have a lot of students or this is their first time in a terminal in a CLI or is it kind of the norm now?

Host

呃,不,我不这么认为。我认为考虑到他们大多数人去年,我想说有些人可能还记得手指在键盘上打字的日子,这是……嗯,我认为大多数学生此时已经接触过 Claude Code。

Uh no, I don't think so. I think that given that most of them again last year I would say that some of them had probably still remember the days of finger on keyboard typing the this is um most I think students have like interacted with Claude Code at this point.

Mihail

我的意思是我很好奇现在只使用应用的人的比例,对吧?Claude Code 也有桌面应用,Codex 也是。Codex 现在主要是一个应用。你有点不在 CLI 中,不在终端中。这是让很多普通人望而却步的事情。但是

I mean I'm curious the split of people that are now just using the apps right Claude Code also has a desktop app so does Codex. Codex is primarily an app now. You're kind of not in the CLI not in the terminal. It's a thing that puts off a lot of average people. But

Host

我只是随便说说。

I'm just random note.

Mihail

是的。我想说,你知道,我认为还有另一件事关于技能,就是技能是技能的捆绑,它们相互交叉引用。

Yeah. I would say, you know, I think there's also another thing about skill that are skills that are bundles of skills where they sort of cross reference each other.

Host

是的。就像插件,对吧?

Yeah. Like the plugins, right?

Mihail

像插件。然后还有我想是编写技能的艺术,这也是我在 AIE 中关注的事情。

Like plugins. And then there's also I guess the art of writing a skill, which is a thing that I've also focused on for AIE.

Host

我认为很多这些都在关注技能消费。你有所有这些影响者,比如 Matt Pog 等等,他们说,‘是的,这个技能会拯救你,会解决你所有的问题。’但不,你应该自己编写你的技能。

I think a lot of these are focusing on skill consuming. And you have all these influencers like Matt Pog and what have you, they're like, "Yeah, this skill will save it will solve all your problems." But no, you should write your skills yourself.

Mihail

而且实际上,当你天真地尝试编写它们时,它们通常很糟糕。实际上,编写技能需要一点技巧。

And actually, when you try to write them naively, they're usually bad. And actually, it takes a little bit of skill to write a skill.

Host

完全正确。完全正确。

That's absolutely right. That's absolutely right.

Mihail

Swix 关于编写技能的最佳建议是什么?

What are Swix's best tips on writing skills?

Host

我有一个技能。我有一个公共技能仓库。我肯定经历过一个精神病阶段,就像给一切加技能。

I have a skill. I have a public skills repo. I went through definitely a phase of psychosis of like skill everything.

Mihail

不,不,不,不要给我们技能。你如何编写好的技能?

No, no, no, don't give us skills. How do you write good skills?

Host

是的。没错。没错。所以,我有一个用于切割技能。所以我喜欢那里,每个像 Claude Code 和 Codex 它们都有技能创建技能,对吧。

Yeah. Exactly. Exactly. So, I have a skill for cutting skills. So I like there so every like Claude Code and Codex they all have skill creation skills right.

技能切割器与垃圾内容 Skill Cutter and Slop

Mihail

所以有一个技能切割器技能,它会把它削减,因为它们只是记录了那些创建的默认技能不好的所有方式。

So then there's a skill cutter skill that cuts it down because they just document every way in which those default skills that are created are bad.

Host

所以你总是在创建任何技能后运行这个来削减它。

So you always run this after any skills created to cut it down.

Mihail

对。

Right.

Host

嗯,就像这样。

Um so like this.

Mihail

是的。这真的很烦人,因为有时会有垃圾,但垃圾中也有好的部分,然后你必须像这样削减它们是什么

Yeah. It's really annoying because sometimes there is slop and like but there's good in the slop and then you have to like sort of whittle down what they're what they're

Host

对。对。哦,有趣。正如你所说,即使是垃圾,它仍然是用智能体的语言。所以垃圾是有意义的。这就像是一个是的。好的。好的。明白了。明白了。

Right. Right. Oh, interesting. As you're saying like even the slop is it's still in the language of the agent. So that the slop means something. It's like it's like a it's Yeah. Okay. Okay. Got it. Got it.

Mihail

没关系。每个人都有自己的版本。我一开始用,很容易对技能进行氛围检查,对吧?你只需调用它运行几次,然后我把这些偏好提炼成一个技能,就像这是我喜欢的这样。不喜欢,它会随时间调整,然后你知道它变得自动化。

It's fine. Everyone has their own version. I used first there's very easy to like vibe check a skill, right? You just call it run it a few times and then I distill these preferences into a skill that's like this is what I like this way. doesn't like and it it adjusts over time and then you know it get automated.

Mihail

问题在于,是的,你在调用它,但实际最糟糕的部分是当一个技能在你没有预料到的时候触发,比如你写了一次来修复一个问题,然后现在你在做一个完全不同的事情,它触发了。为什么?因为它写得不好,因为触发器说哦只要发生 blah blah blah 就调用我,你说好的读起来不错,然后你做一个不同的项目,它触发了,你会说这到底是怎么回事?你知道,有少数人就像,好吧,我有一堆技能。我有时会不断添加,我知道有些人像 Eugene Yen,他们对自己有多少技能非常讲究。他们会修剪它们。如果不使用,他们每几周清理一次。

The problem is that yeah you're calling it but the actual the worst part is when a skill triggers when you're not expecting it to like you wrote it one time to fix a thing and then like now you're working on a completely different thing and it triggers. Why? Cuz it's badly written because the trigger says like oh call me whenever blah blah blah happens and you're like okay read's good and then you do a different project and it and it triggers and you're like what the hell's happening? You know, there's a there's like a few people that are just like, "Okay, I have a bunch of skills. I keep adding on sometimes when I know some people like Eugene Yen, they are pretty particular about how many skills they have. They prune them. If not used, they clean them every few weeks."

Host

是的。

Yeah.

Mihail

是的。所以,呃,所以是的,如果你在那个技能世界里,这是一个巨大的问题。我想这实际上是我怀疑的原因,呃,他们很好。Anthropic 有非常聪明的人希望技能消亡。

Yes. So, uh, so yeah, this is like a a huge problem if you're if you're sort of in that skills world. I think this is actually why I suspect, uh, they're good. There are very smart people at over in anthropic that want skills to die.

Host

呃,这不是一个受欢迎的说法。不,不,他们不会公开说这个,

Uh, which is not a popular statement. No, no, they they won't say this publicly,

Mihail

但这感觉很草率。这感觉很草率。我知道,我能看到我自己的技能人员很草率。这感觉不

but it just feels sloppy. This feels sloppy. And I know it's I I can see my own skills people being sloppy. It doesn't feel

Host

呃 AIA 药丸,

uh AIA pill,

Mihail

对吧?你懂我的意思吗?

right? You know what I mean?

Host

是的。

Yeah.

Mihail

这不是 AGI 应该有的样子。就像每个人都有一大堆 markdown 文件。

It's it's not it's not what AGI was supposed to look like. It's like everyone having a giant bunch of markdown files.

Host

我的意思是,我认为部分原因是有一个园艺问题。是的。某种意义上。

I mean, I think part of it is there's a gardening problem. Yeah. some sense.

Mihail

很多人实际上不看他们的 markdown 文件,对吧?就像用户体验只是聊天你想要什么,在聊天中创建技能,使用它,看到输出。

A lot of people actually don't look at their markdown files, right? Like the the UX of it of just chatting what you want, creating a skill and chat, using it, seeing output.

Mihail

有很多你可以获得很多价值而无需查看你的 markdown。不要在某种程度上为自己过度复杂化。呃,如果你重复做事情,你可以获得很多价值。就像,看看我们上次做的五次。给我做一个技能。你可以用聊天迭代它。也有好的一面,对吧?

There's a lot you can there's a lot of value you can get without ever looking at your markdown. Don't don't over complicate it at some level for yourself. Uh you can get a lot of value if you do stuff repeatedly. Just be like, look at the last five times we did this. Make me a skill. And you can iterate it with chat. There's a good side of it as well, right?

Mihail

还有另一件事。嗯,我想还有另一种看待它的方式,因为有一种是使用技能作为智能体在需要时运行的程序,比如做代码审查或其他什么。但在像 Devon 这样的情况下,对吧,我们把技能编纂为一种代表我们最佳实践的方式,对吧,然后将其暴露给智能体、代码审查系统,然后你知道,按照我们喜欢的方式定义 API。按照我们定义的方式建立服务级别边界。这是一种替代方式,代表组织级别的知识,也许不是这种你希望智能体做的任意事情,这样你就不必每次都写提示。所以我喜欢那个。我觉得那个定位可能那个目的可能比这些随机程序存在得更久。

There's another thing. Um there's I guess another way to look at it as well cuz there's a there's the use of the skill as just this procedure that the agent kind of runs whenever to you know do like a code review or whatever. But then in the case of something like Devon, right, where we've codified skills as a way of of sort of representing our best practices, right, and then making and exposing that to the agent, the code review system to then, you know, institute the APIs the way that we like to define them. Institute kind of the service level boundaries the way we define them. That's like an alternative way of sort of representing like org level knowledge and maybe not so much this like arbitrary thing you want the agent to do so you don't have to write the prompt each time. And so I like I like that. I feel like that positioning might that that purpose might exist longer than just u you know these kind of random procedures.

Host

我刚谷歌了 Monaco AI 技能,它给我的是那个国家。

I just Googled Monaco AI skills and it's giving me the the the country.

Mihail

嘿,那个国家

Hey the country

Host

有一天。有一天。好吧,我们会在 AEO 上打败他们。这是肯定的。

one day. One day. Well, we're going to beat them on AEO. That's for sure.

Mihail

你能直接买下那个国家吗?

Can you just buy the country?

Host

好吧,事情是这样的。好的。关于这个,我们我们呃我们即将买不,我们不是要买那个国家。但是但是但是嗯你知道我们我们确实呃我们确实在买 monaco.com 域名时遇到了一些麻烦。我会说呃我们确实被他们起诉了。所以

Well, here's the thing. Okay. On this we we uh we're about to buy No, we're not about to buy the country. But but but um you know we we did uh we did get into a little bit of flack when we uh bought the monaco.com domain. I will say that uh we did get uh sued by them. So

Mihail

什么

what

Host

是的,有一个整个公开的有一个整个公开的事情关于这个,是的,所以他们不喜欢那样,但是我们在这里,呃我们赢了。我们赢了。所以

yeah there's there's like a whole public there's a whole public thing about this where yeah so they didn't like that but but here we are and uh we won. We won. So

Mihail

好吧,那我们保留它。我正要说这很辣,但你赢了。

okay then we're keeping it in. I was about to say this is spicy but you won.

Host

是的。是的。不,不,这是

Yeah. Yeah. No no it's

Mihail

这就是为什么我们看到旗帜飘扬吗?

Is that why we is that why we see the flags flying around?

Host

因为这就像一场胜利。这是一场胜利。这是我们的胜利。这很有趣。

Because it's like a victory. It's It's a victory. It's our victory. That's funny.

Mihail

对于在旧金山的人,抬头看,你知道,所以对于听音频的人,我们在 YouTube 上展示这个。呃,是你开飞机绕圈。我想你仍然是少数经常这样做的人之一。Sam Blonde 想出来的。每天 6000 美元。呃,我不一定认为它直接导致我谷歌 Monaco 是什么,但 Monaco 的所有广告都只是 Monaco 美元符号,

For for those that are in SF, just look up and, you know, so for for those who are on audio, we're showing this on YouTube. Uh it's you fly a plane around. I think you're still one of the few that actually do this on a regular basis. Sam Blonde came up with it. $6,000 a day. Uh, I don't necessarily think it directly causes me to Google what Monaco is, but that all of Monaco's ads are just Monaco dollar sign,

Host

对吧?

right?

Mihail

就像美元符号然后 X 数字。

Like dollar sign and then X number.

Host

它开启了对话,对吧?广告的全部意义就是 Monaco 到底是什么鬼?

It starts the conversation, right? All the the whole point of the ad is what the hell is Monaco?

Mihail

是的。而且你知道,你可以你可以喜欢那是一个有待讨论的问题,但在某种意义上,营销的目的是什么,如果不是为了吸引注意力,对吧?而且是的,也许还有一个层次,就是如果有人所以,你知道,我会去活动,人们说,哦,你们是那些有飞机的人。你们是那些广告牌上有大数字的人,blah blah blah,1.93 亿,不管是什么。嗯,然后他们说,但你们做什么?然后但那但那但也许这就是全部意义,对吧?就是我只需要有人来把它当作破冰话题。

Yeah. And and you know, you could you could like that's a debate to be had, but but in some sense like what is the purpose of marketing if not to like get attention, right? And and yes, maybe there's one level beyond this, which is if somebody So, you know, I'll go to events and people like, "Oh, you guys are the ones with the planes. You guys are the ones with like the big number on the billboard, blah blah blah, 193, you know, million, whatever it is." Um, and and that but they're like, "But what do you do?" And then but that's that's but that's but maybe that's the whole point, right? Is that I just needed someone to come and use it as an icebreaker.

Host

我会说这也是你走进来时我说的。我当时想,啊,那些飞机。

I will say it's what I said as you walked in as well. I was like, "Ah, the planes."

Mihail

这就对了。现在,你们做什么?好吧,好吧,让我告诉你我们做什么。所以,你知道,作为一个参加过那些决定我们在 Devon 广告牌上做什么的会议的人,我们花了太多时间在文字上,也许我们应该就像 Devon,这太蠢了。但是像,

There you go. And now, what do you guys do? Well, well, let me tell you about what we do. So, so you know, as someone who's been in those meetings deciding what we do on the the Devon billboards, we spend so much time on the words and maybe we should just be like Devon, which is so dumb. But like,

Host

好吧,你知道,看,看,我想 Sam 上周又发了一个帖子,他只是在说,你知道,我的意思是,Sam 实际上就像我们的,你知道,营销方面的策划者。

okay, you know, look, look, I I think Sam had another post uh this past week where he was just talking about like, you know, I mean, Sam is effectively like our our, you know, marketing sort of mastermind here.

飞机故事 The Plane Story

Host

嗯,我觉得这些飞机背后也有一点历史,但简单来说,他的整套理念就是:我并不是什么都懂,你只能去尝试。我觉得很多事就是这样,我们去尝试,如果行得通,那很好。我们当时知道这会像后来那样火吗?其实并不真的知道。但既然它火了,那就成了一个点子。

And yeah, I think there's a little bit of history behind the planes as well, but the TL;DR is that his whole shtick is like, I don't have all the answers. You just got to try things. And I think a lot of this is we just try things and if it sticks, great. Did we know this was going to blow up as much as it did? I mean, not really. But now that it did, it's an idea.

Mihail

是的,没错。

Yeah. Exactly.

Host

还有极简主义,从小处着手,往大处发展,而不是从大处着手,往小处发展。

And minimalism, start small, go big instead of start big, go small.

Mihail

从小处着手、往大处发展要容易得多。

Way easier to start small, go big.

Host

是的,没错。而且你知道,这每天要花 6000 美元。这非常符合“就去尝试”的理念。我觉得人们

Yeah. Exactly. This also, you know, it's 6K a day. Like it's very much just try things. I think people

Mihail

比广告牌便宜。肯定比广告牌便宜。

Cheaper than a billboard. Cheaper than a billboard for sure.

Host

是的,而且很有趣。

Yeah. And it's fun.

Mihail

是的,没什么坏处。

Yeah. Not much harm done.

Host

但你知道,它确实,嗯,但这里值得提一下的那种简单逻辑是,当我们想比一架飞机更胜一筹时,我们会想,你知道什么比一架飞机更好吗?两架飞机。

But you know it does, it's um, but you know the kind of simplicity here that's worth mentioning is then when we're trying to one-up the plane, we're like, you know what's better than one plane? Two planes.

Mihail

两架飞机。三架。

Two planes. Three.

Host

我的方式就是火箭。

My way to rocket.

Mihail

好吧,我也说,我见过

Well okay I also say I've seen

Host

我有过那个想法。我也希望那能在某个时候实现,但我们会看看。这是下一次 SpaceX 发射。我们只是看到 Monaco

I had that idea. I would like for that to happen as well at some point but we'll see. This is the next SpaceX launch. We just see Monaco

Mihail

为了让飞船开始。

For the ship to start.

Host

这就对了。这就对了。得和我们的 SpaceX 朋友们热络起来。

There you go. There you go. Got to warm up to our SpaceX friends.

Mihail

好的。我很高兴。

Okay. I'm happy.

Host

你不想

You don't want to

Mihail

嗯,我是说,我们正在聊 Monaco。值得稍微谈谈 Monaco 吗?你把它留到最后?

Well, I mean, we're on Monaco. Is it worth talking a little bit about Monaco? Do you leave it to the end?

Host

是的。是的。嗯,因为这就像是话题之间的中场休息。

Yeah. Yeah. Um because this is like an intermission between the topics.

Mihail

你决定。

You decide.

Host

好的。我们提到了 Monaco,对吧?让我们给……我觉得只提 Monaco 到底是什么然后不回答有点烦人。当然。你能回答吗?当然。嗯,我希望如此。是的。所以,嗯,这是一个国家。所以,它是一个小小的,嗯

Okay. We're We mentioned Monaco, right? Like let's give I think it's I think it's a little bit annoying to like mention what the hell is Monaco and then not answer it. Sure. Can you answer it? Sure. Well, I hope so. Yeah. So, um this is the country. So, it's a small little uh

Mihail

这就对了。这就对了。好的。这就是 Monaco。这就是我所知道和热爱的 Monaco。嗯,是的。所以,我们是,嗯,你知道,我们是 Basin SF 的一家 B 轮公司。我们实际上在构建一个一体化的销售平台。嗯,完全原生的智能体来做所有有趣的事情,或者至少是人类不喜欢做但智能体喜欢做的事情,因为它们没有真正的意见,但能以更少出错的方式去做,这样销售人员 and 销售团队就能专注于做人类擅长的事情,也就是,你知道,完成交易

There you go. There you go. Okay. That this is Monaco. This is the Monaco that I know and love. Uh yeah. So, we are uh you know, we're a series B company uh here at Basin SF. We are building effectively like an all-in-one sales platform. Uh fully native agents doing all the fun stuff. uh or at least the stuff that humans don't like to do but that agents do like to do because they don't really have opinions but do in a less error-prone fashion so the sales people and sales teams can focus on doing what humans are good at which is you know closing deals

Host

所以想必马克·贝尼奥夫不喜欢你们

So presumably Mark Benioff doesn't like you guys

Mihail

大概不喜欢

Probably not

Host

但他也,老实说,是一个非常 AI 原生的人,非常激进,还有 AgentForce 等等

But he's like also honestly a very AI native person very aggressive and agent force and everything

Mihail

你们在做什么是他不做或他不能做的

What are you doing that he doesn't do or he cannot do

Host

好吧,那么现在,我们,或者我们创办公司时,我们瞄准的市场并不是一开始就面向企业,对吧?我们大概说过,如果我们的目标是构建一体化平台,那么我们需要以迭代的方式与一群客户一起构建,并和他们一起成长。从 Salesforce 那里抢人要比说“嘿,如果我们在你还只是用 Google 表格跟踪 pipeline 的时候遇到你,并向你展示一个更 AI 原生的解决方案,那么我们就能和你一起成长”难得多。在某种程度上,这遵循了同样的模式,就像你知道的 YC 公司那样,你向你的同批同学销售,你和他们一起成长,然后你向上、向上、向上进入高端市场。所以我会说,我们是,你知道,我会声称我们是,如果不是最好的,也是最好的之一,那种专注于初创公司的销售平台。所以现在我们所有的客户都是早期初创公司,可能一直到 A 轮,这是我们主要关注的,从种子轮到 A 轮,然后现在我们正更多地走向高端市场,在过去一年半这样做的时间里,我们对一体化平台学到了很多。

Well okay so so right now we or uh we started the the company in a sort of like the the market that we went after was not enterprise from the get-go, right? We sort of said like we're going to start by like if our goal is to build the all-in-one, then we need to kind of build this in an iterative fashion with with a group of customers and we need to like grow with them into this. It's much harder to pull someone away from from Salesforce than to say like hey if we meet you at the point when you are still just a Google spreadsheet tracking pipeline and we sort of show you a you know more AI pill sort of answer to the problem then we can grow with you and in a way it kind of follows the same the same uh model of like you know BS or all the like YC companies where you kind of sell to your batch mates you kind of grow with them and and you then you go up up up market and so we I would argue are the you know the the I would I would claim the among the best if not the best sort of like startup focused uh sales platform and so all of our customers are like early stage startups at this point probably through to like series A is is kind of where where we've primarily been focused kind of seed to series A and then now we're going more up market and we've kind of learned a lot about the all-in-one in the last year and a half of doing that

Host

而你是 AI 负责人。

And you are the head of AI.

Mihail

是的,没错。

Yeah, that's right.

Host

你们构建了哪些酷炫的 AI 东西?

What's some cool AI stuff you guys have built?

Mihail

是的,我是说,嗯,我们为销售用例构建了自己的智能体框架。所以所有这些,我是说,呼应一下编码框架,嗯,从像 Codex 和 Claude 这样的系统中汲取了很多灵感,并构建了非常专注于销售的定制工具、定制工作流,所有这些都面向 GTM 用例。嗯,所以我们构建了自己的版本,它几乎为 Monaco 平台的所有功能提供支持。所以从你知道的,嗯,TAM 评分,嗯,ICP 评分以及 ICP 的定义,嗯,到 pipeline 演进,再到所有那些也许销售人员不想做的平凡事情,比如哦更新我的会议,你知道,会前笔记,更新我的摘要,所有这些事情。嗯,你知道,我们有智能体为所有这些工作流提供支持。所以我们做到了。它几乎是开箱即用的。嗯,还有,是的,所以长智能体,长视野智能体,对于长视野智能体,有时也用于短视野智能体,用于 GTM 用例。

Yeah, I mean uh we have built our own agent harness for for sales use cases. So all this I mean to call back to the to the coding harnesses uh taking a lot of inspiration from from the systems like the codexes in the clouds of the world and sort of building bespoke tools that are very sales focused bespoke workflows that are all kind of geared toward GTM use cases. Um so we built our own version of that that powers pretty much every single thing that that powers the Monaco platform. So from you know uh scoring of scoring of TAMs uh scoring of ICPS and and develop defining of ICPS uh through to pipeline evolution through to like all the mundane kind of things that maybe a salesperson doesn't want to do which is like oh update my meeting you know premeeting notes update my summaries like all these things. Um you know we have agents powered powering all those workflows. So we we've done that. It it's it's kind of comes comes out of the box. Um and uh yeah, so long long agents long horizon agents for long horizon agents and short horizon agents some cases for for for GTM use cases

Host

而且它相当领域特定,构建在基础模型之上。你们会为此做自己的模型吗?

And it's pretty domain specific built on foundation models. Would you do any of your own models for this?

Mihail

是的,这是个好问题。嗯,因为到目前为止,我是说,我认为这对很多垂直 AI 公司来说普遍如此,就是你开始时使用前沿模型作为第三方服务来看看什么是可能的,因为它们就是最前沿的,然后我想我们看到了这一点,我猜是在过去一周,关于整个 Harvey 的事情,他们喜欢从 +50% 到 -50% 的利润率问题。

Yeah, it's a good call out. Uh because so far I mean I think this is generally true of a lot of like vertical AI companies is like you kind of start where you sort of use the frontier models like as third party services to see what's possible cuz they are like the very frontier and then I think we've seen this I guess in the past week with the whole like Harvey sort of where they like the plus 50% to minus 50% margin kind of question.

成本与开放权重模型 Cost and Open-Weight Models

Mihail

我的意思是,我认为不管数字如何,有趣的是在某个节点上,从成本角度看事情确实变得有趣了。你会开始想:能不能在开放权重模型上做得更快?然后我认为 AI 部分的目的很大程度上就是对这些模型进行后训练,构建用于后训练的环境,使用强化学习之类的方式。所以我们现在大概处于那个阶段。

I mean, I think irrespective of the numbers, what is interesting is that at a certain point things do become interesting from a cost perspective. You start to wonder: can you do it faster on open-weight models? And then a lot of what I think the purpose of the AI pieces are is just to post-train those models, build environments for post-training using RL and things of that nature. So we're kind of in that phase right now.

Host

有意思。是的,我一直把这个打法称为“智能体实验室打法”,不过这算是我的叫法。就像智能体先从用户入手,然后使用你需要的任何模型来构建产品,但一旦有了用户基础,你就可以回填模型侧。而模型实验室则相反:他们从大规模预训练开始,然后试图寻找用户。

Fun. Yeah, I've been calling this playbook the agent lab playbook, but that is kind of my coining of it. Like, agents start from the user first, and then they use whatever models you need to build products, but then you can backfill the model side once you have the user base. And then the model labs go the other way: they start with the big pre-trains and then they try to find users.

Mihail

而且这非常,你知道,希望它能作为一种分工运作,而 Monaco 会站在营收和 GTM 那一侧。

And it's like very, you know, it's hopefully working as a split, and Monaco would be on the sort of revenue GTM side.

Host

没错,是的。

That's right, yeah.

Mihail

我觉得这很有趣。他们既想服务你,又想和你竞争。

And I think it's interesting. They would want to serve you while also trying to compete with you.

Host

作为模型实验室。

As model labs.

Mihail

没错。

That is correct.

数据增强与供应商 Data Enrichment and Vendors

Host

我一直在做的一件事——我涉猎过一些——我认为真正的难题之一其实是数据丰富。

One of the things that I've been—I dabble in some of this—I think one of the hard problems is actually enrichment.

Mihail

嗯。

Mhm.

Host

我想说,我花了很多时间做这个,基本上是为了 SEO 和研究之类的。我猜对你来说,这实际上是核心部分。

I would say I spent a lot of my time doing that for basically SEO and research and what have you. And I imagine for you it's actually a core part of that.

Mihail

没错。

That's right.

Host

你觉得模型擅长这个吗?有没有你喜欢的供应商?想不想给他们打个招呼?随便说说。

Do you find that models are good at it? Do you have any vendors that you like working with? You want to give them a shout out? Whatever.

Mihail

是的,是的,当然。我们用过——我们是 Parallel 的早期采用者,我觉得他们非常强。

Yeah. Yeah. Absolutely. So we have used—we were early adopters of Parallel, and I think they're very strong.

Host

好的。所以就是通用搜索。

Okay. So just general search.

Mihail

是的,我是说——

Yeah. I mean—

Host

不是像 ZoomInfo 或那类公司。

Not like ZoomInfo or one of those guys.

Mihail

不,不,不。但是的,我们用 Parallel,也用 Exa。所以给这两家都打个招呼。

No. No. No. But yeah, we use Parallel. We also use Exa. So we'll give a shout to both of those folks.

Host

你能比较一下它们吗?

Can you compare them?

Mihail

他们都是朋友。都是朋友。都是朋友。都是我们喜欢的人。

They're both friends. They're both friends. They're both friends. They're both folks that we like.

Host

如果你喜欢其中一个,他们会非常感兴趣的。

They'll be very interested if you like one of them.

Mihail

是的,我会保持中立。今天我会像瑞士一样中立。但没错,我们也做自己版本的数据丰富。我们大多数数据系统——我是说,我们构建的最有价值的软件或最有价值的系统之一就是我们的世界数据库,即公司数据库、丰富后的公司、丰富后的账户、丰富后的联系人。所以每当我们为客户构建 ICP 时,我们拥有关于此时此地在哪家公司应该联系谁的最新信息。我想说,我们的一大优势——我认为我们毫无疑问是世界级的,就是需求。我们是一家非常擅长需求的公司。我们帮助企业以真正——的方式增加预约会议。

Yeah. I'll keep it neutral. I'll stay Switzerland in this today. But yeah, so we do our own version of enrichment as well. Most of our data systems—I mean, one of the most valuable pieces of software that we build or most valuable systems is our world database, like the database of companies, enriched companies, enriched accounts, enriched contacts. So whenever we are constructing ICPs for folks, we have the most up-to-date information about who you should reach out to at which company at this point in time. I would say that one of our—I think where we are world-class hands down is just demand. We are a very good demand company. We help companies increase their meetings booked in a way that actually—

Host

在正确的时间找到买家。

Find buyers at the right time.

Mihail

在正确的时间找到买家,对吧?这有点像在正确的时间进行组合。有时候——举个例子:如果你瞄准的是你要销售的对象,比如销售副总裁,联系他们的正确时机不是他们在同一个地方待了五年的时候。如果他们现在正处于工作变动中——一个很好的信号是,一旦他们从一份工作转到另一份新工作,他们现在进入了一个更有可能购买或愿意购买新销售平台的新地方,因为他们正处于工作过渡期。

Find buyers at the right time, right? And it's kind of like a combination at the right time. There are times when—here's an example: if you are targeting someone that you are selling to, VPs of sales, let's say, the right time to reach out to them is not when they've been at the same place for the last 5 years. If they are now in the middle of a job—a good signal is once they've gone from one job to a new job, they're now entering a new place where they're more likely to purchase or be open to purchasing a new sales platform because they're in the middle of a job transition.

Host

从 A 公司跳到 B 公司做销售——一个判断此时是否应该联系他们的好信号是,你是否检测到他们的 LinkedIn 发生了变化之类的?这其中有多少是你的专业知识被放入模型和框架中, versus 模型本身很好——有通用的最佳实践——显然两者都有价值?是不是只是粘合、让它可靠、在一个平台上完成、做好?似乎还有一些我们不知道的、你们也是领域专家并融入其中的东西。

Going from company A to company B for sales—a good signal of whether you should reach out to them at this point in time is did you detect that their LinkedIn changed or something like that? How much of this is your expertise put into the model and harness versus models are good—there's generic best practices—is there obviously some value in both? Is it just the glue, getting it reliable, doing it all in one platform, doing it well? It seems like there's also stuff that we wouldn't know that you guys are also domain experts in that's baked in.

Mihail

是的,我认为如果没有前者,它就会完全商品化——我是说不是完全,因为一体化仍然有很多价值。我认为这绝对是我们开创的。销售领域的大多数公司一开始只是单点解决方案,我认为他们更多尝试了 YC 式的“落地并扩展”方法。你选择一个细分市场,然后从那里扩展。而我们从一开始——我是说,这就是为什么我们在一年多的大部分时间里保持隐身——就是在构建,因为我们知道我们想从一开始就做一体化。但仅此一点就很强大,很有意义。但我认为真正的力量在于,今天 Monaco 拥有一支世界级的 GTM 团队,一支真正世界级的 GTM 团队来销售 Monaco,当然。但我们也让销售 Monaco 的同一批 GTM 人员担任我们所说的“前置部署客户主管”。他们是 FDA。他们实际上是——我说是人类销售人员,有点像顾问,帮助入职我们所有客户,让他们构建智能体工作流,并有效地指导他们完成自己销售 GTM 生命周期的每一个部分。所以回答你的问题,是一体化,是的,但也是因为我们有这些——你知道,SAM 和——我们引进了很多前 Sam 团队成员之类的人,他们是 Brex 和其他地方 GTM 方面最优秀的人。所以我们把所有这些都编码到平台中。

Yeah, I think without the former it would be like a fully commoditized—I mean not fully, because there's still a lot of value in the all-in-one. I think that is something that we definitely pioneered. Most companies in the sales space started as just point solutions, and I think they tried more of the YC approach of land and expand. You pick a niche and then you expand from there. And we, from the get-go—I mean, that's why we stayed in stealth for the greater part of a year and some months—was just building because we knew we wanted to be the all-in-one from the start. But that alone is powerful. It's meaningful. But I think the real power is the fact that today Monaco has a world-class go-to-market team, a truly world-class go-to-market team selling Monaco, of course. But we also have the same go-to-market people that sell Monaco are the ones that are what we call forward-deployed account executives. They're FDA. They're effectively—well, I say human sales people that are like consultants of sorts that help onboard all of our customers, get them building agentic workflows, and are effectively there to help guide them through every part of their own sales go-to-market life cycle. So to your question, it's the all-in-one, yes, but it's also the fact that we have these—you know, SAM and—we have brought on a lot of former Sam's team members and stuff like that who were the best people in GTM at Brex and other places. So we're codifying all of that in the platform.

Host

是的,有趣的是,你的谈吐与普通的 AI 负责人可能非常不同,因为你也是前创始人,对吧?你也习惯于以这种方式思考这些事情,在你的角色中拥有这一点很重要。

Yeah, interesting to have you talk very differently than a normal head of AI might also talk to because you're also a former founder, right? And you're also used to thinking about these things in this way, and it's important to have that in your role.

Mihail

是的。

Yeah.

Host

酷。我想你已经足够介绍 Monaco 了,显然随着 Monaco 故事的发展我们可以再聊。

Cool. I think you've introduced Monaco enough there, and obviously as the Monaco story develops we can revisit.

Mihail

当然。没问题。也许回到正题,也可以谈谈 Monaco 内部的编码——或者说智能体原生编码或现代软件开发——是什么样的。

Sure. For sure. Maybe the bridge back is also what is coding—or agent-native coding or modern software development—like inside of Monaco.

内部AI开发 In-house AI development

Host

嗯,你知道我觉得你在教学时可能试图呈现一个全景,但也许在 Monaco 内部,你们对某件事的看法比其他事更有主见。

Um you know I think when you're teaching you're maybe trying to be here's a lay of the land but maybe inside of Monaco you guys are more opinionated on one thing versus the other.

Mihail

是的,是的。不,我的意思是我们从第一天起就在编码智能体上进行了投资,我们学到了很多。我们有自己的伤疤,算是积累了一些经验和教训,我觉得我们现在越来越深入地分享一些我们的学习成果。所以比如在我们的博客上,可能值得在这里分享,我们越来越多地发布一些……也许不是那个,那个更像是一个产品,最上面的那个,但这些东西我会说,是的,当 AI 在……之前传播开来,就像 Scaling 贡献构建者购买 S。所以你知道我们花了很多时间研究不同的提供商,研究供应商,最终很多东西都是内部构建的。所以对我们来说,像后台智能体平台,我们有自己的版本,就像 ramp inspect,我们称之为 monocoder,它构建在 coder 平台上,但它是我们用来扩展工程产品的东西。

Yeah. Yeah. No I mean we have so we made investments in coding agents from basically like day one and we've like learned a lot. We have our own like scars that we've sort of like picked up and learnings and I think we've now been getting much deeper into sort of sharing some of our learnings. So like in our blog which might be worth sharing here we've been increasingly posting some of what uh maybe not that so that one is more of like a product the one at the very top but this stuff I would say like yeah when AI gets that spread in before like scaling contributions builder buy S. So you know we spent a lot of time looking at different providers looking at vendors and ended up building a lot of things in house. So for us something like a background agent platform we have our own version of like a ramp inspect um that we call monocoder um which is is built on the coder platform um but it is what we use to essentially scale up our engineing product

Host

像 coder.com

Like coder.com

Mihail

Coder.com 是的

Coder.com yeah

Host

哦

Oh

Mihail

是的,我们实际上已经把它适配到我们的一些用例中,是的,这就是我们很多智能体开发发生的地方,它真的是一款设计精良的软件,本质上用于隔离开发环境,扩展到多智能体,你知道能够运行这些长视野的智能体式编码任务。所以 Monicoder,感谢 John Tracy 和公司构建了它,然后是的,从那里我们还构建了自己的版本,比如一个 Slack 机器人,叫做……但你知道有效地它就像是部分 AI,部分客户成功,部分像是 QA 工程师,因为它住在我们的 Slack 里,但它服务于所有目的,帮助我们调试东西,在东西坏了的时候提交 PR,也被我们自己的销售人员用来了解产品如何工作,了解如何深入某个问题。所以我们不得不回答一些有趣的问题,关于 HAL 有不同的个性,取决于它是在和销售人员还是开发者说话。像这样的系统是我们随着时间的推移构建的,很多是在过去一年左右。感谢 Yuna 为 Hal 做的贡献,也感谢 John Tracy。

Yeah we actually yeah have sort of adapted it to some of our use cases and yeah it's where a lot of our agent development happens it's just this really well-designed piece of software for essentially isolating of of sort of like um development environments scaling out to multi-agents um you know being able to run these long long horizon agentic coding agent tasks. So Monicoder, shout out to John Tracy and and and Co for for building that out and and then yeah like from there we've also built our own versions of like a you know like like a slack bot which is call but um you know effectively how is like this part AIS part like customer success part like uh kind of QA engineer because it it it lives in our Slack but it serves all the purposes of helping us debug things and put up PRs when things are broken but also used by our own sales people to find out how the product works, find out like how to get to the bottom of something. And so there's like interesting questions we've had to answer around the different personalities that HAL has u depending on whether it's talking to like a salesperson versus like a developer. And so systems like this that we built out over time a lot in the last like year or so. Uh shout out to Yuna for for Hal and and John Tracy as well.

Host

所以你内部构建了很多东西。

So uh you're building a lot in house.

Mihail

是的。

Yeah.

Host

你也使用其他工具。

Uh you also use other tools.

Mihail

我们确实使用其他工具。是的。

We do use other tools. Yeah.

Host

那么界限是什么?

So what is the line?

Mihail

是的。所以

Yeah. So

Host

通过

by

Mihail

是的,构建者购买它。这是个好问题。我认为有一个普遍原则,我一般遵循的,就是“构建以学习,购买以保留”,或者说“购买以学习,构建以保留”。是的,反过来。所以你是购买以学习,构建以保留。我认为这在这里普遍适用,在早期我们还在摸索,站稳脚跟的时候,代码审查就是一个例子,我们花了很多时间研究不同的代码审查平台。结果今天我们仍然使用 Devon,因为我们实际上仍然在系统的构建方式中发现了很多价值。所以感谢 cognition 团队。但后来我们也尝试了一些其他工具,我们尝试了一些现成的,也许更像后台智能体平台,决定鉴于我们的基础设施,鉴于我们设置自己云平台的方式,内部构建更有意义。当然我们不是教条的,我们愿意重新审视,如果有新东西出现,但在这里我觉得通常当我们觉得没有成熟的 offerings 时,我们最终会内部构建,所以代码审查可能是个有趣的例子,因为我们确实做了一次全面的代码审查工具调查,大概一年多以前,当时

was that Yeah, the builders by it. It's a good question. Uh and we I think there's this um uh like a thing a general principle that that I generally which is like you build to learn and you buy to keep or sorry you buy to learn and you build to keep. Yeah. The other way around there. Yeah. So so you you buy to learn and you build to keep. And I think that's generally true here where in the early days when we were still trying to figure out like you know get our feet under us a little bit we were you know code review is an example one of them where we spent a lot of time looking at different code review platforms. It turns out today we still use Devon because we've actually still find a lot of value in how the systems been built. So you know shout out to the cognition team. Um but then along the way we did try out some like S sur tools and you know we tried some sort of like offtheshelf maybe more like background agent platforms decided that it was just given our infrastructure given how we had set up our own cloud platform it just made more sense to kind of build this out internally of course we're not dogmatic we we are open to revisiting them if if something comes out but here I felt I think usually we ended up building in house when we felt that there was no like mature offering out there so code review is maybe an interesting example because we did do a whole survey of code review tools like maybe over a year ago and at the time

Host

你可以提一下 Code Rabbit 对吧

you can shout out like code rabbit right

Mihail

是的,Code Rabbit 和他们,你知道还有像 Code Graphite,Graphite 一段时间,在他们被收购之前,像 Ellipsus,之前是 Y 公司,所以但有很多,对吧,我的意思是,有很多人我们尝试过,他们还可以,但我不认为他们跨过了立即有用的门槛,所以我们就把它搁置了一下,这是远超过一年前的事,然后我们有点回到这个,有点有机地,我的意思是,再次感谢 Devon 团队和 cognition 的人,我们又试了一次。我想我有点像是有人告诉我,“嘿,你应该试试这个,似乎好多了。”可能是在去年课程中 Silus 的演讲期间,它只是很好,它是有机的,就像

yeah code rabbit and and them and you know that there was also like code graphite graphite graphite for a while before they got acquired um like ellipsus and which like a y company before the fact so but there's a bunch right I mean so so there were a lot of folks that we did try and they were okay but I don't think they had like crossed the threshold of like they're immediately useful yet and so then we kind of like we sort of like put it aside a little bit this was you know I'm saying like well over a year ago then we kind of went back to this and sort of organically I mean, again, shout out to to the to the Devon team and and cognition folks where we tried it again. I think um I had sort of like gotten that somebody had told me like, "Hey, you should try this and it seems like it's gotten a lot better." It might have been actually during Silus's talk during the class last year and it was just good and it it was organically like like

Host

纯粹是感觉,对吧?就像这不可能你

purely vibes, right? Like this is there's no way that you

Mihail

有,是的。这就像是,我的意思是,好吧,好吧,我要说的是,当我们第一次评估时,我们没有真正选择任何供应商,我们说没有供应商让我们非常满意。我确实做了一个分析,我看了我们的 PR,基本上做了一些饼图,比如这解决了什么。我们对冗长程度和发布频率与实用性的权衡满意吗?所以这更多是统计驱动的,而不仅仅是感觉。第二次我不打算大力推动。第一次我有点像说嘿我们应该采用一个 AI 代码工具。第二次我说不,我不会再那样做了。但后来人们有机地说,“嘿,Devon 很好。你能把我加进去吗?”所以今天人们开始有机地被添加。所以这是一个供应商,你知道我们仍然倾向于供应商的情况之一。你知道长期来看会怎样,但除此之外的一切,我们只是觉得成熟度不够。

there was Yeah. This this was like I mean Okay. Okay. Here here's what I'll say. When we did that first evaluation the first time around where we didn't really pick any vendor um and we were like there's no vendor that we're super happy with. I did do like an analysis where I looked at our PRs basically did sort of like a you know set of pie diagrams like here's what this is solving. Are we are we happy with the kind of the trade-off of verbosity and how often it's posting with utility. Um and so there was it was more statistically driven than just vibes. The second time around I wasn't going to push hard on it. The first time I was kind of like hey we should adopt an AI code tool. Second time I was like no I'm not going to do that again. Um but then organically people were just like, "Hey, Devon is like good. Like can can you like add me to this?" And so today people started just like getting organically added. So this is a vendor where you know one of the cases where we're still leaning to the vendor. You know we'll see in the long term what that looks like but um everything outside of that we just didn't feel like maturity was there.

Host

是的。而且它还没有痛苦到让你想撕掉 Devon 自己写一个。

Yeah. And it's not painful enough that you would like rip out Devon and write your own.

Mihail

没错。

That's right.

Host

这很有趣。是的。

That's interesting. Yeah.

Mihail

所以对 Devon 来说是个好广告,我甚至都不知道。呃,你知道,声明一下,我只是顾问,我不再那么参与了。但像从外部听到这些很有趣,因为内部有一定的,我们称之为泡沫,在 Slack 上你只听到积极的事情,因为人们只想传播积极的事情。

So good good talk for Devon who I didn't even know. Uh you know for the record I'm only an adviser. I'm not that involved anymore. But like it's interesting to to hear about it externally because internally uh there is a certain amount of let's call it um the the bubble where like on Slack you're you're like you only hear positive things cuz people only want to circulate positive things.

Monaco课程结构与第四五周 Monaco course structure and weeks four and five

Host

但你在这件事上没有利益牵扯——你只是选最好的工具。

But you have no dog in the fight — you just choose the best tool.

Mihail

我是说,如果你想拿到 Slack 的免费额度,那也挺好。我觉得没人会拒绝这个。

I mean, if you want to get a Slack free credit, that'd be cool too. I would say no one would say no to that.

Host

我觉得 Silus 会帮你搞定。

I think Silus will hook you up.

Mihail

好。

Okay.

Host

所以就是这么回事,对吧?然后这也贯穿了你所有的教学工作,我觉得这非常好。你愿意的话我们可以多聊一些,因为我觉得这种亲身经历非常宝贵。

So there's that, right? And then that informs all your teaching work, and I think that's really good. We can bring in more of this as you like, because I think this lived experience is very good.

Mihail

对。

Yeah.

Host

我很乐意回到“智能体就绪的代码库”,或者第四、五周那种衔接的部分。

I'm happy to go back to agent-ready codebases, or week four and five kind of matching.

Mihail

对,它们有点融合在一起。

Yeah, they blend a little bit.

Host

对,它们有点融合。没错。你在 Monaco 里引入的那些强烈信念,还有你推荐人们从哪儿开始。

Yeah, they blend a little bit. That's right. Whatever strong beliefs you have introduced in Monaco, and recommend people to start with.

Mihail

对,当然。第四、五周,我会说它们有点——第四周可能更偏概念基础和实现方式,因为定制智能体的方法有很多。然后第五周是我真正深入的地方:好,我们来想一想,并把它应用到一个真实的代码库上。所以这里我一定要特别感谢 Factory 团队,并强调一下“就绪报告”这类东西的价值。关于“智能体就绪的代码库”,我想强调的核心点是:在我看来,它是后续一切的前置条件和一块基石。我说后续一切,是指人们一直——有一阵子了,我觉得现在还是——对多智能体特别狂热,比如并行化智能体,几十个智能体做各种事情。但实际上你需要先打好基础才能做好这件事。而这个基础就是:你的代码库有多可验证?结构有多好?智能体有多容易理解它?所以在我看来,智能体就绪几乎是接下来四五周的第一块拼图,也许到第八、九周我会花更多时间讲广义的 AI 软件团队和整体结构,但肯定是在为后台智能体、软件工厂搭建基石。我觉得从那个角度看,这才是真正有意思的地方。

Yeah. No, for sure. Week four and five, I would say they are a little bit — four is maybe a little bit more like conceptual foundations and ways to do this, because there are a lot of ways to customize agents. And then I would say week five is where I really lean into, all right, let's think about this and apply this in practice on a true codebase. And so here I will definitely give a shout out to the Factory team and just highlight the value of something like a readiness report. The main piece that I want to reinforce with something like agent-ready codebases is that to me it's a precursor and a bit of a building block toward everything to come afterwards. When I say everything to come afterwards, I mean people always got — for a while, and I think people still are — super hyped on multi-agents, like parallelizing agents, like tens of agents doing all kinds of things. It's like, you actually need to set a foundation in order to do that well. And the foundation is: how verifiable is your codebase? How well structured is your codebase? How understandable is it by an agent? And so I think agent readiness to me is almost like piece one of the next four to five weeks, where maybe until week eight or nine I spend more time talking about broadly AI software teams and global structure, but definitely building up the building blocks for background agents, the software factory. I think this is where it all gets interesting from that perspective.

Host

对。我注意到你在 Monaco 里讲过、但这里没提的一点,是 SRE(站点可靠性工程)那一侧。当你闭环时,你其实希望智能体去读构建日志,然后理想情况下读线上生产环境的分析数据。

Yeah. One thing I observed you talking about for Monaco, but it's not here, is the SRE side. When you close the loop, you actually want to have the agent read your logs for builds and then ideally read your analytics for live production.

Mihail

没错,没错。对。没有——没有明确讲,但这是个很好的提醒。所以我想我会把它织进两个地方。一个是后台智能体那一周,然后还有软件工厂那一周。所以完全正确。我确实——这是个很好的提醒,就是这种基于触发/事件的闭环,现在大概是人们谈论智能体在软件工厂里启动任务时的两三个入口之一。完全正确。那会加进去。

That's right. That's right. Yeah. Not as — not explicitly, but it's a really good shout out. So I think I'm going to weave that in in like two places. One is in the background agents week and then also in the software factory week. So absolutely right. I do — it's a really good call-out, this idea of trigger/event-based closing of the loop, which now is one of the probably two or three entry points when people talk about agents kicking off tasks in a software factory. That's absolutely right. That'll come in there.

Host

对,有意思的是人们叫它触发或事件驱动,我猜那是它实际的技术实现。只是听起来几乎太底层了。

Yeah, it's interesting that people call it trigger or event-based, which I guess is the actual technical implementation of it. It just seems too low-level almost.

Mihail

嗯。

Mhm.

Host

但触发器通常就是个定时任务,对吧?说真的。

But the trigger is usually a cron job, right? Like let's be real.

Mihail

当然,当然,当然。

Sure. Sure. Sure.

Host

事件驱动通常就是个定时任务,它抛出一个事件,然后就“事件驱动”了。但全都是定时任务。

Event-based is usually a cron job that surfaces an event and then it's event-based. But it's all crons.

Mihail

对,对。没错,没错。对,对,对。或者像 webhook 之类的。但没错,完全正确。你当然知道那个技术术语,因为说到底——我算是科班出身的计算机人——对,我们当时就那么叫。所以我猜——我会回到那套术语上。

Yeah. Yeah. That's right. That's right. Yeah. Yeah. Yeah. Or like a webhook or something like that. But yeah, that's absolutely right. And you know the technical term, of course, because at the end of the day — being sort of a CS person by trade — and yeah, we called it those things. And so I guess that's — I fall back on that terminology.

Host

那有没有哪家 AI SRE 公司给你留下过印象?

And have any AI SRE companies made an impression on you?

Mihail

好。我是说,我觉得 Resolve 那帮人做得不错。Resolve 做得挺好。我会说 Night Shift 在广义的后台智能体方面也做得相当不错。我很欣赏他们——他们是 Greylock 投资的。我觉得他们在这个领域也做得很好。我会说,SRE 是个有意思的用例,这可能又回到“自建还是购买”的问题。我觉得有意思的地方在于:SRE 真正价值最大的地方在哪?通常是在你有大量服务依赖、这些庞大系统、成堆的 Kubernetes 和 Helm 文件的时候——就是那种超大规模的 Kubernetes 基础设施,Helm 文件到处散落。我觉得这就是为什么 Resolve 在企业市场做得很好,对吧?你去卖给福特,去卖给沃尔玛,你说,嘿,你们的服务级依赖太疯狂了。你看到那些架构图,然后他们进去说,好吧,我们试着把这些理清楚。我觉得在较小规模下,通常还是可控的,你可以自己搞。但我要说,在企业级,我觉得他们做得很好。

So, okay. I mean, I think the Resolve folks are doing well there. Resolve is doing well. I would say Night Shift is also doing quite a good job there in terms of broadly background agents. I'm a big fan of — they're Greylock-backed. I think they're doing well in this area as well. I would say that where I think SRE is an interesting use case, and this might go back to the build-versus-buy question. I think what's interesting about it is: where does an SRE really see the most value? It's typically when you have a tremendous number of service dependencies and these massive systems and tons of Kubernetes and Helm files — just tremendously large Kubernetes infrastructure, Helm files kind of strewn everywhere around. I think that's why Resolve has done really well in the enterprise, right? Where you go and you sell to Ford, you go and sell to a Walmart, and you kind of say, hey, your service-level dependencies are crazy. And you see these diagrams of how things are set up, and then they kind of go in and they're like, well, we'll try and make sense of all this. I think at smaller scales, it's usually still manageable that you can kind of roll your own. But I will say that for the enterprise, I think they've done a good job.

Host

对。对于从没在这个规模上工作过的人——因为我也是花了一阵子才看明白——这个量级大概是,我们这么说吧,几万个代码仓库,仓库比开发者还多。这种情形下,你根本不知道公司里任何地方跑的是什么代码,因为写它的人很可能已经不在了。

Yeah. For people who have never worked at this scale — because it took me a while to see this — it is on the order of, let's call it, tens of thousands of repos, more repos than developers. These are the situations where you don't know what code is running anywhere in the company, because probably the guy who wrote it is no longer here.

Mihail

对。太荒唐了。而且顺便说一句,它还在承重。

Right. Absurd. And by the way, it's still load-bearing.

Host

太荒唐了。那——对,那是最吓人的地方。

That's absurd. That's — yeah, that's the scariest thing.

Mihail

完全正确。就像——然后你就是那个人,你知道,你就是那个值班的人,你在跟一个挂掉的系统打交道,而那个人还是不在,还有过时的旧运维手册,你就想,这到底怎么回事?这真是很吓人、很吓人的事。没错,在企业级,那完全是另一头猛兽。

That's absolutely right. It's like — and then you're the person, you know, you're the person who's on call, and you're interacting with a system that goes down where that person still isn't there, and there are old out-of-date runbooks, and you're like, what the hell is going on here? It's a scary, scary thing. And yes, at the enterprise level, it's a whole different beast.

Host

对。所以像那种大学黑客——一个人管一个仓库——他们从来——完全体会不到。

Yeah. So like the college hacker — one person per repo — they've never — totally no appreciation.

智能体就绪的沙盒作业 Sandbox assignment for agent readiness

Host

但我觉得如果你只是给出那个想法,就像你把他们丢进一个模拟里,好吧,就像你上班第一天发生了这个,现在怎么办?

But I think if you just give that idea, like you drop them in a simulation of, okay, like day one on your job this happens, what now?

Mihail

是的。是的。所以这里我要再提一下 Resolve 的人,他们去年实际上为课程做了一个作业,他们有效地创建了一个沙盒供学生进入,然后沙盒里散落着——我的意思是,沙盒里有很多很多服务在做各自的事情,甚至沙盒里的服务还有 bug。然后作为作业的一部分,学生必须去实际使用 Resolve 智能体,并像人类值班人员一样探测和扮演,探测系统,发现发生了什么。

Yeah. Yeah. So another shout out I'll give here. So the Resolve folks, they actually last year they made an assignment for the class where they effectively created a sandbox for students to go and then the sandbox was like strewn—I mean, within the sandbox there was like many many services that were doing their various things and there was even like bugs in the services within that sandbox. And then as part of the assignment the students had to like go and actually use the Resolve agent and kind of like probe and play the human on-call person probing the system, discovering what's happening.

Host

但他们有一个有趣的网站。

But they've got an interesting website.

Mihail

我不——我已经有一段时间没看到这个了。

I do not—it's been a while since I've seen this.

Host

是的,这非常互动。

Yeah, this is very interactive.

Mihail

好吧。他们现在成了 PagerDuty。

Okay. They're being a PagerDuty now.

Host

是的。是的。看看这个。是的。但还有这个,就像这个视频,我当时——

Yeah. Yeah. Check this out. Yeah. But also this like this video I was—

Mihail

你知道,你在喊有趣,有趣,但好吧。

You know, you were shouting out interesting, interesting, but okay.

AI SRE范围与企业挑战 AI SRE scope and enterprise challenges

Host

所以好吧,你知道我们除了这样的对话之外,从来没有机会谈论这个领域。还有其他部分吗?因为我觉得 AI SRE 可能太宽泛,甚至太狭窄,取决于你怎么看,对吧?太宽泛意味着好吧,AI 有几个细分领域——抱歉,这很有趣——或者太狭窄,就像其实那甚至不是我们最大的问题。所以随你怎么理解。

So okay, you know we never get a chance to talk about this side of the world except for conversations like this. Any other segments because I think AI SRE is maybe too broad or even too narrow, depending on how you look at it, right? Like too broad meaning well there's several subdivisions of AI—sorry, that's interesting—or too narrow being like well actually that's not even the biggest problem that we have. So take it however you like.

Mihail

是的,呃,你是指专门针对像 Monaco 这样的,还是泛指像组织在思考这个问题?

Yeah, uh, you mean like specifically for like Monaco or just like broadly for like organizations thinking about this?

Host

泛指,因为那是我们的受众。请讲。

Broadly, because that is our audience. Take it.

Mihail

是的,我认为这又取决于你的规模。对吧。而且我认为对于企业来说,有很多问题并不总是——我几乎觉得答案并不总是只是更多的智能体。呃,而且既然我们实际上还在第五周智能体就绪代码库,就像当我和一定规模的公司交谈时,阻碍他们的事情真的更多只是低层次的 DevOps CI/CD 之类的东西,对吧?比如他们的合并队列开始积压。他们启动一个应用需要太长时间。因为实际上有更基础的事情,几乎与此无关,比如再扔进一个 S bot 或者——你知道,一个机器人不会是问题的答案,因为他们需要弄清楚更系统性的东西。所以——

Yeah, I think it again it's a function of how your scale. Right. And I think that for the enterprise, there are a number of issues that aren't always—I almost feel like the answer is not even always just more agents. Uh and since we're still actually on the week five agent-ready code bases, like when I talk to companies of like a certain scale, the things that are holding them back are really more just like low-level like DevOps CI/CD stuff, right? They're like their merge queues are getting backed up. They it takes too long to like stand up an app. Because there's like actually more foundational things that have almost nothing to do with that like throwing in one more S bot or—you know, a bot wouldn't be the answer to the problem because there's like more systemic things that they need to figure out. And so—

Host

所以这把我们带到我们——

So that brings us to we—

Mihail

是的,完全正确。所以这就是我们进入——是的,完全正确。是的,我们这就去合并队列及更远的地方。

Yeah, exactly right. So that's where we get into—yeah, exactly right. Yeah, there we go to the merge queue and beyond.

智能体代码审查与合并队列 Agentic code review and merge queues

Mihail

嗯,是的,这是一封——是的,某种程度上是写给现代智能体式代码审查的情书。嗯,是的,我认为去年到今年可能发生的最大事情是,突然变得非常清楚,代码审查是一个瓶颈,尤其是当人们真的真的被 AI 渗透,生成的速率变得如此之高,你知道理论上你可以一天提交 30 多个 PR。呃,然后呢——在什么点上你决定要么必须把自己插入中间,以某种方式传统地把关和审查,但那样你就是字面上的约束,你是速度的瓶颈,那不好,对吧?那大规模生成的意义何在?或者你说,好吧,我们如何可能两全其美,我们把自己插入足够多,以至于我们对合并的内容有信任,但我们也能够大规模合并,但我们也并不是在这里放任自流、什么都行。所以我认为我在这里想触及的很多就是那个中间地带,对吧?也就是有一类问题,如果你在智能体就绪方面做了深思熟虑的设计,你在可验证性方面做了深思熟虑的设计,有一类问题你可以感到舒服,你知道,在某个百分比的时间基本上就像自动批准。从设计角度、从架构角度,必须满足什么才能直接自动批准?以及什么时候你必须插入自己并说,实际上这里我仍然想仔细审查?即使你仔细审查,你还是在逐行审查还是做别的什么,对吧?

Um yeah, this is a—yeah, a love letter of sorts to modern agentic code review. Um yeah, I think that the biggest thing that maybe happened between last year and this year was it suddenly became very clear that code review is a bottleneck, especially once people got really really AI-pilled and the velocity on generation became so high that you know you could in theory like put up 30 plus PRs a day. Uh and then what—like at what point do you decide that you either have to institute yourself and put yourself in the middle as some way to gate and review things in the traditional way, but then you're literally the constraint, you're the choke point of the velocity, and that's not good, right? And then what was the point of generating at scale? Or you say, well, how do we maybe get the best of both worlds where we're instituting ourselves enough that we have trust in what's getting merged, but we're also able to kind of like merge at scale, but we're also not kind of like freewheeling it here and anything goes. So a lot of I think what I want to touch on here is just that middle ground, right? Which is like there is a class of problems that if you've put in thoughtful design into your agent readiness, you put in thoughtful design around your verifiability, there's a class of problems that you can feel comfortable, you know, some percentage of the time essentially like auto-approving. And what has to be true from a kind of a design perspective, from like an architecture perspective, to just auto-approve? And when do you have to insert yourself and say actually here I still want to review this carefully? And even if you do review it carefully, are you still doing line-by-line review or you doing something else, right?

Host

所以是的,其中一些——我们今年最热的话题之一。呃,我们有一个 Latent Space 的客座作者计划。呃,这是来自 Aviator 的 Ankit,他有一整套关于干掉 AI 审查的内容,因为我——你知道那可能是最终 boss——呃,那是一个瓶颈。他有几个呃关卡来实现这个。我不知道你是否和 Ankit 谈过或者——

So yeah, some of this—we one of our hottest topics of this year. Uh we have a guest writer program for Latent Space. Uh this is from Ankit from Aviator and he has a whole thing on killing the AI review because I—you know that that is like maybe the final boss—uh of—that's that's a bottleneck. He has a few uh gates by which he does this. I don't know if you've talked to Ankit or—

Mihail

是的。不,我读过这篇文章。是的。是的。是的。我没有亲自和他谈过,但我肯定读过——

Yeah. No, I've read this post. Yeah. Yeah. Yeah. I haven't spoken to him personally, but I've definitely read—

Host

大体正确?有什么批评吗?

Mostly right? Any criticisms?

Mihail

不,我认为这是对的。我是说大体上正确。嗯,我猜你提到了一件事,比如最终 boss。嗯,也许不是——我认为有两个瓶颈。有审查,可能还有意图或顶层的那个东西,对吧?就像这是——

No, I think this is right. I mean like broadly right. Um I guess you mentioned one thing like the last boss. Um maybe not—I think there's two choke points. There's the review and there's probably like the intent or kind of the thing at the top, right? Like this is—

Host

还有一些我喜欢引用的统计数据。你知道,每个人的统计数据肯定不同。肯定不同。

And there's some stats as well that I like to pull up. You know, everyone's stats is different for sure. For sure.

Mihail

但显然更多人在合并。显然审查的数量并没有与——呃与合并同步增长。

But like clearly more people are merging. Clearly the number of reviews are not increasing in line with—uh with merge.

Host

没错。没错。如果你看看任何现代呃那种真正投资于此并仔细思考过的公司——有些人已经写了关于这个的文章,你,我相信我们也会写关于这个的文章。嗯,但我想最近像 Amplitude 和嗯我想 Cloudflare 写了一篇好的。我想 Augment 也写了一篇好的,他们谈论了他们的 AI 代码审查系统的设计。嗯这些是一堆非常好的——如果你想分享一些链接。我认为它们是写得极好的文章。嗯是的,所以向——我实际上嗯向 Cloudflare 致敬,因为我觉得像嗯非常像那种你知道在某种意义上是传统公司或者更大的公司,但我认为他们在 AI——AI SDLC 的内容方面做得非常出色。嗯所以我认为他们那里有一篇好文章,但 TL;DR 嗯是的我认为这是真的。我认为有——我们仍在寻找那个最佳平衡点。

That's right. That's right. And if you look at any modern uh sort of company that's really invested in this and thought about this carefully—and some people have already done posts about this and you I'm sure we'll do posts about this as well. Um but I think recently like the Amplitude and um I think Cloudflare has done a good one. I think Augment has also done a good one where they talk about the design of their AI code review system. Um these are a bunch of really good—if you want to share some of those links. I think they're fantastically written posts. Um yeah, so shout out to—I actually um shout out to Cloudflare because I think like um very like kind of like you know in a sense like legacy company or like you know bigger company but I think they've done a fantastic job in terms of just their content around the AI—the AI SDLC. Um so I think they have a good post there but TL;DR um yes I think this is true. I think that there's—we're still finding that sweet spot.

规格与PR描述 Specs and PR Descriptions

Host

什么是可自动批准的?什么不是真正可自动批准的?如果不是可自动批准的,即使在那个需要一定程度人工审核的世界里,我们审核什么?我们还在逐行审核,还是做某种规范与……的结合?我觉得这又是一个完全独立的问题。但人们会一直保持规范更新吗?规范根本不可能,对吧?

What is auto-approvable? What is not really auto-approvable? And if it is not auto-approvable, even in that world of something that has to be somewhat human-reviewed, what do we review? Are we still doing the line-by-line, or are we doing some combination of the spec and...? I think this is a whole separate question as well. But are people keeping the spec up to date at all times? The spec is no way, right?

Mihail

是的,不。嗯,我的意思是,好吧,在几个不同的地方,对吧?比如,有一个规范存在于 Linear 问题中,还有一个规范存在于 PR 描述的上下文中,对吧?所以,你可以让 PR 描述成为你以更简洁的格式描述整个实现的地方。只要那是最新的东西,代表代码库的这个时间点,我认为那实际上就足够了。所以,我们对 PR 描述很用心。我们让它们不仅仅是一段话。它更像是 PR 中所发生事情的有效规范,甚至还有行/文件级别的指针指向发生变化的地方。所以你不仅得到发生了什么的高层信息,甚至还有文件的构造以及你触及的服务级边界。所以我认为这就是我们投入相当多的地方。

Yeah. No. Well, I mean, okay, in a few different places, right? So, like there's a spec that as it exists in like a linear issue and there's a spec as it exists in like the context of a PR description, right? So, you can have your PR description be the place where you describe more or less the entirety of the implementation in a more condensed format. And as long as that's the most up-to-date thing representing this point in time of the codebase, I think that's sufficient actually. So, we're thoughtful about our PR descriptions. We make them be not just a one paragraph. It's more like it is effectively a spec of what has happened in the PR and there's even kind of line/file-level pointers to things where things have changed. So you not only get the high level of what's happened but even the construction of the files and sort of the service-level boundaries that you're touching. So I think that's where we've invested quite a bit.

Host

所以我对此提出异议,因为对我来说,当然最低限度是它应该与 PR 规范内部一致。

So I push back on that because to me of course it's bare minimum it should be internally consistent with the PR spec.

Mihail

当然,不,我的意思是功能的规范,代码库的规范,因为那是所有 PR 的总和。就像如果你的 PR 规范弄错了或遗漏了什么,那就会是整个代码库应该做什么与你的 PR 试图做什么之间的差异。

Sure, no, what I meant is the spec for the feature, the codebase, because that is the summation of all PRs. That is like if your PR spec got it wrong or missed something, it's going to be the delta between what the whole codebase is supposed to do versus what your PR is trying to do.

Host

嗯。所以我的意思是我不指望你有答案,因为我自己真的为此感到压力。我试着指导模型去做。它们不擅长。

Mhm. So I mean I don't expect you to have an answer because I really stress about this myself. I try to instruct models to do it. They're bad at it.

Mihail

就是这样。

That's it.

Host

所以人类是规范的上帝,就像“好吧,五周前我做了另一件事,现在你又在推翻它”。

So the humans are the spec gods of like well five weeks ago I did this other thing and now you're reversing it.

Mihail

你看到 Basis 的那篇文章了吗?他们有一个推文,一个小推文文章。我觉得他们很好地讨论了不同类型的文件工件。他们通过 git 跟踪,其中一些是规范,即意图是什么,另一个是他们所谓的代码的规范状态,这也许更像你描述的,即隐式知识、隐式意图,也必须在某处被编纂,对吧?所以那部分,都是版本控制的,我认为他们发现这对他们有效,所以那也可能是另一件可以分享的事情。

Have you seen the post from Basis? Like they have a tweet, a little tweet article. I thought they did a great job talking about different sort of file artifacts. They track via git and some of them are like the specs, the intent of what is to be, and the other is like what they call the canonical state of the code, which is like a little bit more maybe what you're describing, which is implicit knowledge, implicit intent that also has to be codified somewhere, right? So that part, that's all version controlled and I think they found that to work for them, so that might be another thing to share as well.

Host

我正试着把它调出来。这是 Basis,那家会计公司?

Is trying to pull it up. This is Basis, the accounting company?

Mihail

那家会计公司,没错。是的,我们很快会在我的纽约会议上请 Mitchell 来谈谈这个。我喜欢他们推出这个。

The accounting company, exactly right. Yeah, we're having Mitchell at my New York conference soon, talk to him about that. I like that they are coming out with this.

Host

我的解决方案是——我不知道我有多准备好谈论它,因为我可能永远不会发布它——是某种取代 git 以捕获所有这些的东西。所以我称之为程序数据库,它捕获所有痕迹,并将规范具体化为我们可以忽略并保留的东西,它还控制附加到它的技能。因为我认为基本上 git 只关注代码,然后你只需要一个真相来源,它不仅包括代码之外的所有其他东西,包括隐式假设或约束、意图、其他东西,而且还要多——高度并发,因为 git 在设计上到处引入互斥锁,这很糟糕。所以你需要超级并发。

My solution for this is, I don't know how ready I am to talk about it because I may never ship it, is something that is replacing git in order to capture all of this. So I call it a program database, where it captures all traces and reifies specs to something that we can ignore and keep, and it also controls the skills that are attached to it as well. Because I think basically what git is only focused on code and then you just need a source of truth that is not only everything else outside the code, including the implicit assumptions or the constraints, the intent, the other stuff, but also multi-, like highly concurrent, because git by design introduces mutexes everywhere, which sucks. So you need like super concurrency.

Host

你见过,嗯,GitButler 是吗?有像……是的。那个曾经是 CTO 的人,我想他叫 Ian。

Have you seen, um, GitButler is it? There's like... Yeah. The guy who used to be CTO, Ian I think is his name.

Mihail

我忘了他的名字,但是是的。

I forget his name but yeah.

Host

Scott。Scott Scott Scott。嗯,所以是的,我认为有很多人在追求这个。嗯,我认为有一个问题,即任务的理想表示到底是什么?

Scott. Scott Scott Scott. Um so yeah so I think there's a lot of people going after this. Um and I think there's the question of like really what is the ideal representation of tasks?

Mihail

是的。

Yeah.

Host

我认为赢家可能不是我,是 Linear。

And I think the winner probably not me, Linear.

Mihail

真的吗?

Really?

Host

因为那才是任务所在的地方。

Cuz like that's where the tasks are.

Mihail

是的。嗯,显然他们决定成为一家编码智能体公司,对吧?而且,你知道,只要你接受 Linear,他们那里就有所有的真相来源。

Yeah. Uh and like clearly they have decided to become a coding agent company, right? And well, you know that they have all the source of truth there as far as you as long as you buy into Linear.

Host

是的。是的。那是真的。那是真的。我想知道,我有一种有点坚定的信念,我认为很多公司会演变成某种版本的编码智能体公司,即任何与 B2B 软件有交互的公司,你在那里收集关于用户的一些东西或一些产品洞察。我认为你也可以闭环,因为最终编码智能体是什么,编码智能体的目的是对正在发生的涉及用户或客户的事情采取行动,然后利用这一点来理想地修复产品的某个方面,对吧?我认为 Linear 正在做的事情,我想象 Amplitude 也可以做同样的事情,对吧?因为他们也在获取关于用户分析、用户问题的洞察。我想象甚至,如果你想真正延伸,你甚至可以让像 Decagon 或 Intercom 做类似的事情,你知道这里坏了,去闭环。

Yeah. Yeah. That that is true. That is true. I wonder, I have like this somewhat held belief that I think a lot of companies are going to evolve into some version of a coding agent company, which is any company that has an interaction with B2B software where you're collecting something about the user or some product insight. I think you can also close the loop because ultimately what a coding agent is, the purpose of a coding agent is to act on some adventure thing that's happening where users are involved or customers are involved and then use that to fix ideally some aspect of the product, right? I think what Linear is doing, I imagine the same thing like an Amplitude could do as well, right? Because they're also getting insights about user analytics, user problems. I imagine even like, you want to really stretch it, you could even have like a Decagon or Intercom do something similar where you know here's what's broken, go close the loop.

安全与会议 Security and Conference

Host

好的,所以为智能体就绪的代码库设置所有这些,并设置代码审查。安全是第七周。不是第一周。

All right, so set all those things up for agent-ready codebase and set up code review. Security is week seven. It's not week one.

Mihail

学生们,你说安全不重要是什么意思?

Students, what do you mean security is not important?

Host

我开玩笑是因为我总是说,你知道,在亚马逊,零号工作是安全,他们总是这么说。

I make fun because I always say like, you know, at Amazon job zero is security and they always say this.

Mihail

是的。是的。

Yes. Yes.

Host

总之,但是是的,最后进行安全审查。好的。

Anyway, but yeah, security review at the end. Okay.

Mihail

我当时在亚马逊,我不记得那个,但我相信。

I was at Amazon and I don't remember that but I believe.

Host

这是 Andy Jassy 说的。

This is the thing Andy Jassy says.

Mihail

好的。好的。好的。我明白了。我明白了。那是,我猜他,我在那里部分是在 Jeff 时期,然后部分是在过渡到 Andy 的时候。所以,但是是的,我认为这是去年也是一个话题,今年也是一个,我认为在那里,嗯,Isaac 去年也发言了。我发现他是一个非常深思熟虑的思想家,你知道,这个领域的一个深思熟虑的领导者。我认为 Sam Grup 是一家有趣的公司,因为他们在智能体之前就已经存在了,对吧?我的意思是,OSS 仓库早在传统 SaaS 防御机制之前就存在了。

Okay. Okay. Okay. I see. I see. And that was I guess he, I was there partly during Jeff's time and then partly when was transitioning to Andy. So, but yes, I think this is a thing, a topic last year as well and it's also one this year and I think that there I, um, Isaac also spoke last year. I find that he's an incredibly thoughtful thinker, you know, a thoughtful leader in sort of the space. I think Sam Grup is an interesting company because they've been around even before sort of agents, right? I mean, the OSS repo was exist well before kind of traditional SaaS kind of defense mechanisms.

依赖安全与供应链 Dependency Security and Supply Chain

Mihail

但我觉得他们做得很好,转向或者不是转向,而是将他们的产品增强到这个领域。我觉得这是一个有趣的时期。现在有很多事情在发生,这些攻击向量正浮出水面。有更多的垃圾和 AI 生成的代码。我想在这里推动的一个核心见解是依赖跟踪和供应管理,基本上就像供应链。你必须小心你的依赖关系,适当地固定版本,不要随意运行 npm install 然后指望整个链条没问题。

But I think they've done a good job of pivoting, or not pivoting, augmenting their offering into this space. I think it's an interesting time. There's just a lot going on where these attack vectors are now coming to the surface. There's a lot more slop and AI-generated code. And I think maybe one of the core insights that I want to drive here is just the idea of dependency tracking and supply management, basically like supply chain. You have to be careful about your dependencies and pin things appropriately, and don't just arbitrarily hit npm install and expect the chain to go.

Host

是的,所以潜在的最安全的代码库是没有依赖的代码库。

Yeah, so potentially the most secure codebase is one with no dependencies.

Mihail

是的,对,没错。你看着依赖,觉得,酷依赖,酷实现,专业人士,但我还是自己做一个吧。

Yes, right, that's correct. You look at the dependency, you're like, cool dep, cool implementation, pro, but I'm just going to make my own.

Host

完全正确。完全正确。这是一个有趣的见解,因为它有点极端。我觉得这说起来容易做起来难,但如果你相信代码现在商品化了,代码容易、便宜,为什么不全部内部构建呢?当然那会是最安全的事情,因为你看到了一切,你控制了一切,而不是依赖 Axios 之类的。但我们会看看有多少人真正采取行动。但那确实是最安全的。我同意你。

That's absolutely right. That's absolutely right. And that's an interesting insight because it's sort of to the limit. And I think it's probably easier said than done, but if you believe that with code now being commoditized, code being easy, cheap, why not just build everything in house? Like, of course that would be the safest thing because you've seen everything, you control everything instead of depending on Axios or whatever. But we'll see how many people actually act on that. But that is the safest thing. I agree with you.

Mihail

不,所以那个说法是危险的,因为那样你只会犯自己的错误,而不是依靠别人发现错误。开源的全部意义在于,你知道,阳光是最好的消毒剂。

No, so that statement is dangerous because then you would just make your mistakes instead of relying on other people to discover mistakes. The whole point of open source is that, you know, what is it, sunlight is the best disinfectant.

Host

就像 Linus 的说法,嗯,是的,你会发现自己的自定义软件依赖非常慢,因为你只有你的用户,他们不会向你报告。但如果这个东西在全球有 10 万用户,总会有人发现问题的。

It's like Linus's thing, and it's like, well yeah, you will be very slow to discover your own custom software dependencies because you only have your users and they won't report it to you. But if this thing has 100,000 users worldwide, someone will find something.

Mihail

然后你可以更依赖它,对吧?

And then you can depend on it more, right?

Host

是的,这基本上没有,我试图做一个理性的声音,你不能在两方面都绝对化,我有点把你套进去了。我说服你进入了“全部自研”的谈话轨道。

Yeah, this is, there's no basically, I'm trying to be a voice of reason and you cannot be absolutist in both, and I kind of trapped you there. I talked you into the talk track of vendor everything.

Mihail

对。对。

Right. Right.

Host

那里有个 bug。

And there is a bug there.

Mihail

是的。是的。是的。这是一个有趣的问题。我想知道有多少,我买账某些方面,但有多少受到这个事实的影响:一切,我有一个信念,大多数开源仓库,如果不是全部,将来会变成实际上由智能体运行的软件工厂。那会发生什么,比如

Yeah. Yeah. Yeah. It's an interesting one. I wonder how much that, how much is that, I buy some aspect of that, but how much of that is impacted by the fact that everything, I have this belief that like most OSS repos if not all of them will just become software factories that are effectively agent-run in the future. What happens then where like

Host

我是说,即使你看看 Warp,对吧,Warp 是一个编码智能体,当然他们也卖软件工厂产品,但他们把他们的开源 Warp 终端作为一个软件工厂来运行,这几乎表明有一部分,是的,人类仍然参与,但越来越少,当然不是每个人都在审查进来的每一件事,达到我们过去可能相同的水平。这使这个代码库更容易还是更不容易受到安全威胁?有点难说。我认为这正是有点微妙的地方,我想我们仍然,甚至我认识的从事开源的人也在试图弄清楚那个稳定点到底是什么。

I mean even like even if you look at Warp, right, like Warp which is a monocoding agent, of course they're also selling a software factory product, but they are running their Warp open source Warp terminal as a software factory, and that almost suggests that there's a part where yes humans are still involved but increasingly less so, certainly not every single person is reviewing every single thing that comes in to the maybe the same exact level that we did in the past. Does that make this code base more or less vulnerable to security? It's a bit hard to say. I think that's just where there's a little bit of nuance that I think we're still, and even the folks that I know that work in OSS are trying to figure out exactly what that stable point is.

Mihail

是的,有几个相关的论点,然后我很乐意继续。安全是一个不停歇的巨大表面积的事情。有一个来自 Veracell 的 Malta 的论点,他说,好吧伙计们,让我们把每一个高度依赖的用 C C++ 编写的软件移植到 Rust,我们会更快更安全,可能不会花 600 万美元,但你会完成。然后还有另一个论点,这也与模型实验室的开源前沿节奏有关,他们说,好吧让我们慢下来,以便我们或者我们会做 Daybreak,我们会做 glass wing,我们会确保每个人都打了补丁,然后我们可以放任疯狂加速。所以我认为有一个固有的信念,即模型将解决所有安全问题。

Yeah, there's a couple of related arguments and then I'm happy to keep going. Security is like a non-stop huge surface area thing. There's one argument from Malta of Veracell who's like, okay guys, let's take every single highly dependent on software that's written in C C++ port it to Rust and it's like we'll be faster more secure, and it won't probably take like $6 million and but you'll get it done. And then there's another argument for this, this also links to pacing the frontier from the open source from the model labs who are like, okay let's slow down so that we or like we'll do Daybreak, we'll do what's glass wing, we'll make sure everyone's patched and then we can let loose on rampant acceleration. So I think there's an inherent belief that the models will fix all security.

Host

我觉得这对我来说只是未经证实,因为如果它们引入了新的 bug,而且只是不同的 bug 呢?

And I think it's just unproven to me because what if they introduce new bugs and they're just different bugs?

Mihail

没错。没错。所以我是说,我在寻找反驳。我在寻找像“不,你是个白痴。显然你的模型在这方面好得多”这样的回应。

That's right. That's right. So I mean well I'm looking for push back. I'm looking for like well no you're an idiot. Like obviously your models are way better at this.

Host

是的。我猜这几乎就像在极限情况下,问题是,模型,我们相信模型会创造完美的软件吗?如果模型创造了完美的软件,那你就完成了。有这样的东西吗,那就完成了,对吧?那我们在这里就完全完成了。就像然后我们只需回家,在沙滩上放松余生。

Yeah. I guess like that's almost like in the limit there's the question is like is a model, do we believe a model will create perfect software? If a model is creating perfect software then you're done. Is there such a thing then that's done right? Then we have then we're done here at all. Like then we're just going to go home and chill on a beach for the rest of our lives.

Mihail

我认为还有一个问题,是否足够好先给它用于安全?让人们打补丁然后只是延迟发布之后再这样做,或者有一个问题

I think there's also the question of is it good enough to first give it for security? To let people patch then just delay the release and do it after or is there a problem of

Host

哦不,它在发现漏洞方面会好得多,你可以全力测试更长时间,即使你已经打了补丁或者像

Oh no, it'll be much better at finding vulnerabilities, you can full send test it for longer even if you've patched or like

Mihail

多久,对吧?你可以给它 3 个月,有人可以尝试利用它 6 个月,问题没有解决,对吧?

How long, right? You can give it for 3 months, someone can try to exploit it for 6 months, the problem's not solved, right?

Host

对,对,是的。

Right, right, yeah.

Mihail

是的,我整个问题在于,再次这是一个我们不需要深入的长篇大论,但你知道,你命名了 100 个受祝福的组织,好吧这些是优先的,他们可以访问一个网络模型,你刚刚为我创造了 100 个攻击向量去攻破,而且顺便说一句它们很容易被攻破。

Yeah, my whole problem with like, and again this is a whole rant that we don't need to go into, but you know, you name 100 blessed organizations that like, okay these are like priority, they have access to a cyber model that you've just created 100 attack vectors for me to go compromise, and like they're by the way they're pretty easy to compromise.

Host

是的。是的。所以我不知道,无论如何,那超出了我们的薪资等级。

Yeah. Yeah. So I don't know, anyway, that's above our pay grade.

Mihail

在安全方面变得 juicy 很有趣,就像

It's fun to get juicy in security, like

Host

我是说安全现在非常 juicy,对吧,我觉得实际上安全人员喜欢它,因为他们通常得不到那么多关注,现在他们有了大量的关注。

I mean security is right now very juicy, right, like and I think actually the security folks love it because they usually don't get that much attention and now they have like a ton of attention.

Mihail

那是真的吗?我是说他们得不到关注。我发现,嗯,我发现至少从风投的角度来看,人们喜欢风投喜欢投资安全公司,并且有相当好的

Is that true? Like I mean and that they don't get attention. I find that, um, I find that at least from like a VC perspective, people love VCs love investing in security companies and have like pretty good

Host

像 Wiz 之类的做得非常非常好。

Like a Wiz or something like has done very very well.

Mihail

Wiz 可以说是 Wiz 和 Socket。我两天前真的和一位风投有过这样的对话。显然 Wiz 和 Socket 是过去 5 年里唯一成功的安全公司。

Wiz is arguably Wiz and Socket. I literally had this conversation with a VC two days ago. Apparently Wiz and Socket are the only security companies in the last 5 years that have worked.

安全与AI生成代码 Security and AI-generated code

Host

那异常情况呢?我不知道从风投的角度看这行不行得通,我是从经济学角度来看的。

What about like abnormal? I don't know what it worked for a VC point of view of I see from like from like an economics perspective.

Mihail

嗯,有意思。不,这不该由我来说,对吧?显然那个领域有很多优秀的玩家,但我觉得这是个艰难的行业。基本上,对任何安全公司来说,最好的时机——我觉得这没什么争议——就是刚发生了一起高调的安全事件的时候。

Yeah. Interesting. No, not for me to say, right? Obviously many great players in that field, but it's a tough industry, I think. And so basically like the best time but I think it's not controversial. Best time for any security company is when a high-profile incident just happened.

Host

是啊。

Yeah.

Mihail

对吧。显然那时候你肯定能卖出很多安全产品。完全没错。

Right. Obviously like then obviously you're going to sell a bunch of security. Totally.

Host

是啊。所以我觉得这是人们应该意识到的。人们应该知道基本的缓解措施。我确实认为代码扫描很重要,但我只是想让大家明白,模型不会引入人类不会犯的 bug,如果你不检查那些 bug,那么……

Yeah. So I think that's a fair thing like people should be aware. People should know the basic mitigations. I do think like code scanning is important, but like I just want to give people the idea that like no models introduce bugs that you wouldn't make as a human and like if you're not checking those bugs then

Host

甚至不是它们制造了 bug,而是你对正在写的东西了解得更少,对吧?就像当你……

it's not even that they make bugs, it's that you have less awareness of what's being written, right? Like when

Mihail

是啊,审查所有内容。

Yeah. reviewing everything.

Mihail

是啊。是啊。别管 bug 了。模型相当不错。平均来说,我会说它们的 bug 更少。问题还在于你甚至不知道你在发布什么,就像……

Yeah. Yeah. Forget bugs. Models are pretty good. They'll have on average, I would say, less bugs. The problem is also that you don't even know what you're putting out like

Host

是啊,

Yeah,

Mihail

这种情况很多。

there's a lot of that.

Host

是啊。我觉得我们必须考虑缓解措施,而且安全与代码审查之间有一种结合,对吧?我的意思是,做那些检查我认为很重要。

Yeah. And that's I think we have to be thought about mitigations and and like the and there's sort of a combination of security and and security as part of code review as well, right? I mean think this is also like doing those checks I think is important.

Mihail

是啊。所以我觉得,嗯,也许我在这里再说一点,或者你检验一下我的直觉:代码库的各个部分并不是生来平等的。代码库中安全非常重要的部分是数据存储,

Yeah. So I think um maybe I would say one more thing here uh or or check your check my intuition on on this thing is that uh not all parts of the codebase are created equal. the parts of the codebase where security is very important is data storage

Mihail

嗯,还有计费。所以我们姑且称之为数据、计费,嗯,off 和 off 显然与所有这些相关,那些基本上就是“别用 VIP 代码”,然后其他一切都可以放开搞。

um and uh billing. So let's call it that data billing uh off and off is obviously related to all those things and those are basically don't vip code and then everything else go nuts.

Host

是啊。

Yeah.

Mihail

我觉得这绝对正确。绝对正确。

I think that's absolutely right. That's absolutely right.

Host

而且这真的很有趣,就像,好吧,如果你不知道 off 和 off Z 的区别,请去学一下。然后你知道,如果你不知道,如果你从未看过 AWS IAM,请去学一下,这些是基本的概念性东西,一旦你掌握了语言,你就可以提示它。如果你没有语言,你就……完全没错。完全没错。是啊。是啊。在某种程度上,我甚至把这扩展到一般的某些云平台的事情上,对吧?我的意思是,有一个——我忘了是哪家公司——发生了一起巨大的事件,就像只是一个 DNS 问题导致宕机,是吗?你记得这个吗?是……

And and and it's really funny how like okay well if you don't know what the difference between offend and off Z is please learn it. And then you know if you if you don't know if you have never looked at AWS IM which you have please learn it like this like these are basic like conceptual things to understand that you can prompt for once you have the language you can prompt for it. You don't have the language you're Totally. Totally. Yeah. Yeah. To an extent I even extend that to like just general like certain certain cloud platform of things right. I mean there was the I forget which company got had that hu incident huge incident where like it was just like a DNS issue that brought down was it do you remember this? Was it

Mihail

DNS 问题?你得缩小范围。

DNS issues? You need to narrow it down.

Host

是啊,我知道。我知道。有一个特别突出的。是……

Yeah, I know. I know. There was a particularly prominent one. Was it

Mihail

也有很多突出的 DNS 问题。

There have also been many prominent DNS issues.

Host

好吧。

Okay.

Mihail

但把它们列出来。

But list them off.

Host

总是 DNS。总是,是啊。所以 DNS 更多是可靠性和正常运行时间的问题。

It's always DNS. It's always Yeah. So DNS is more of like a reliability uptime.

Mihail

当然。当然。

Sure. Sure.

Host

嗯,你知道,安全,我觉得人们会问:你损失钱了吗?你泄露客户信息了吗?

Uh you know security I think people like did you lose money? Did you leak customer information?

Mihail

是啊。是啊。是啊。是啊。是啊。那个我会,是啊,肯定把那个放在上面。只是因为正常运行时间才提 DNS。是啊。一般的正常运行时间。

Yeah. Yeah. Yeah. Yeah. Yeah. That I would Yeah. Probably put that above that for sure. Only DNS because of uptime. Yeah. General up time.

Host

是啊。

Yeah.

Mihail

SLA 之类的,嗯,后台智能体。嗯,现在你请来了 Coffee 的朋友,但我的意思是你的嘉宾不一定和那一周相关,对吧?

SLAs and stuff like uh background agents. Uh now you got your buddy from Coffee here, but I mean your guests aren't necessarily correlated with that week, right?

Host

是啊。不,我的意思是,第九周有点像是日程安排的问题,我插入了,你知道,Elad,而且我猜我肯定不是第九周。Elad 我本来想把他放在最后,但我觉得只是时间上没法安排。嗯,所以是啊,这有点奇怪,我猜我第九周根本没讲课,但我们在第十周会回到那些话题。所以有点错开,方式有点奇怪。是啊,第八周。嗯,Rajesh,他一直在领导 Cloudflare 的云基础设施、DevX 以及其他很多基础设施方面的工作,正如我之前说的,我觉得他们在这里做得很好,而且对于他们这种规模的公司、这种规模的上市公司来说,他们的采用绝对是极好的,然后他们的内容我觉得也非常好。我的意思是,这从根本上讲是关于使用异步后台智能体的所有方式。嗯,我想我也会花相当多的时间谈论,你知道,Stripe 的 minions、Spotify 的——我忘了他们那个版本的名字了。嗯,Ramp 的 inspects,你知道,诸如此类的东西,我觉得它们都在展示当你扩大规模时它会是什么样子。

Yeah. No, I mean like the the nine one was a little bit of a scheduling thing where I've thrown in, you know, Elad and and I guess I'm isn't with definitely week nine. Elad I was trying to get him at the end, but I think it was just timing wise couldn't work it out. Um so yeah, it's it's a bit of a weird one where I don't I guess I'm not lecturing at all week nine, but but we're coming back to those topics in week 10. So it's a little bit staggered in a in a bit of a bizarre way there. Yeah, week eight. Um, Rajesh who has been leading a lot of like the infra stuff at uh Cloud Infra uh DevX and and beyond at Cloudflare which as I earlier said was I think they're doing a great job here and definitely their adoption for a company of their size and a public company of their size is is fantastic and then their content I think is also really good. I mean this is fundamentally about all the ways you can use like asynchronous background agents. Um, I think I'm also going to spend quite a bit of time talking about, you know, the Stripe minions, the Spotify, I forget the name of their version of this. Um, the ramp inspects, the, you know, things of that nature that I think are all kind of showing what does it look like when you scale this out.

Host

Tag Instinct。

Tag Instinct.

Mihail

是的。

Yes.

Host

Muse。Muse。

Muse. Muse.

Mihail

是啊。我的意思是,我猜较少涉及消费者,更多是像,你知道,软件团队大规模使用它们。

Yeah. I mean, I guess less on consumer, more on like, you know, software teams using them at scale.

Host

它们本质上都是编码智能体。

They're all coding agents under the hood.

Mihail

没错。

True.

Host

一切都是编码。

Everything is a coding.

Mihail

是啊。就像 Grok Bot 是那些有趣的东西之一,它非常非常地以消费者电子产品的形式呈现,嗯,它是 Grok 底下的,你知道,它是 cursor 在底层。

Yeah. Like Grok Bot is one of those interesting things where like very very much presented as consumer e uh it's it's Grok under it's you know it's it's cursor under the hood.

Host

是 cursor 团队。是……

It's the cursor team. It's

Mihail

不,不,当然。就像我见过 cursor 在……你知道,在点点点里工作,但但它们没有代码库集成,对吧。

no no for sure. Like so I I've seen the like you know cursor working in in the like the dot dot dots but but but they are not they don't have like a codebase integration right.

Host

它们有。

They do.

Mihail

好吧。我没用过那个。

Okay. I haven't used that.

Host

我只用它来订 OpenTable 的位子。

I've only been using for like open table reservations.

Mihail

没错。好吧。

Exactly. Okay.

Host

嗯,我觉得这就是他们想做的事情的要点。

Uh which I think that's the point of what they're trying to do.

Mihail

我明白了。是啊。就像……

I see. Yeah. as in like

Host

他们想成为,嗯,好吧,你可以用它做消费者的事情,但如果你知道如何用好它,这里还有额外的能力,嗯,因为它实际上是 cursor 在底层。

they're trying to be like uh well you can use it for consumer stuff but here's the extra power if you know how to use it well uh because because it's actually cursor under hood

Mihail

我觉得这适用于所有编码智能体,对吧,就像 Claude Code、Cloud Design,嗯,人们在使用 Codex 却不知道自己在用 Codex,对吧,一个随机的代码解释器窗口弹出来,你只是在问一个问题,在某种程度上底层有一个编码智能体,对吧。

I think this goes to all coding agents right like Claude Code cloud design uh people using Codex not knowing they're using Codex right a random code interpreter window pops up and you're just asking jets a question at at some level there's coding agent under the hood right

Host

是啊,所以我再给你一个实际的例子。嗯,我把所有的 SEO 和 AEO 都放在 Codex 里运行,它有一个代码库,尽管传统营销人员只会检查一个地址之类的,因为我写了所有这些脚本来做各种疯狂的事情。当然。

yeah so so I'll give you another sort of practical example Uh so I I run all my SEO and AEO inside of Codex and it has a codebase even though like a traditional marketer would just check a address or something because I write all these scripts for doing all the all sorts of crazy stuff. Sure

Mihail

而且我还需要存储数据,还需要可视化数据,很快你就会发现,只要你在持续做任何事情,可能有很多步骤或大量数据,你就需要一个代码库。嗯,这是个有趣的观点。

and and also I need to store data and also I need to vis visualize data and like pretty soon you go like as long as you're doing anything on a ongoing basis that has like maybe a lot of steps or a lot of data you need a code base. Hm. That is an interesting take.

Host

我觉得模型也喜欢构建代码库。

I think models also like building code bases.

Mihail

不,但那可能是个暂时的事情。

No, but but that could be a temporary thing.

接口与代码库访问 Interface and Codebase Access

Host

我想我的观点更多是关于界面,对吧?比如你会不会说到这个程度:Grok 机器人会有一个地方让高级用户能看到底层的代码?还是说他们可能直接跳到 Cursor?你是这个意思吗?

I guess maybe my point is more around the interface, right? Like will you go so far to say that a Grok bot will have a place where the power user will be able to see the code under the hood? Or is it maybe they just jump to Cursor? Is that what you're saying?

Mihail

嗯,没错。因为否则你基本上会蚕食 Cursor。如果在极限情况下你说的是 Grok 机器人就是 Cursor,那么你甚至不需要 Cursor,如果底层一切都只是变成——

Well, exactly. Because otherwise you basically cannibalize Cursor. If in the limit what you're saying is Grok bot is the Cursor, then you don't even need a Cursor if all under the hood is just becoming—

Host

是同一个团队。所以现在他们不在乎,但没错,都是 Grok 机器人。

It's the same team. So they right now they don't care but yeah it's all Grok bot.

Mihail

是的。是的。是的。

Yeah. Yeah. Yeah.

Host

我接受这一点。我的意思是,我确实接受即使表面上看起来只是非技术用例的东西,底层也是技术性的,可以用代码编写、设计和实现。这一点绝对正确。问题更多是:开发者——那是他们的单一入口点吗?

I buy that. I mean I do accept that even things that seem outwardly just like non-technical use cases under the hood are technical can be written and designed and implemented in code. That much is absolutely true. The question is just more like: is a developer—is that their single point of entry?

Mihail

是的。

Yeah.

Host

我觉得甚至不是开发者,对吧?比如你想到一个做营销的普通人,现在使用 Claude Code 或应用中的 Claude,在一个他们永远不会知道、永远不会接触的代码库中工作,但是——

I think it's not even developer, right? Like you think of a regular person doing marketing that is using Claude Code or just Claude in the app now is working in a codebase that they would never know, never touch, but—

Mihail

有一个代码库管理器。

There's a codebase manager.

Host

嗯。

Mhm.

Mihail

我认为 Claude Tag 对于后台智能体来说是个好例子。这是 Anthropic 工作的重要方式,你知道他们做的。你应该看看他们如何工作。

I think Claude Tag is a good one for background agents. It's a big way Anthropic works and you know they make it. You should see how they work.

Host

是的。是的。做得好。呃,好的。AI 原生团队。呃,我们已经谈了很多关于网关的话题。呃,我们今天刚发布了一期关于路由的节目,还有 MCP 门户。我想说 MCP 门户可能是最——

Yeah. Yeah. Good job. Uh, okay. AI native team. Uh, we've talked a lot about these topics of like gateway. Uh, we just put a routing episode out today and MCP portal. I would say MCP portal is like the least—

Mihail

看好 MCP。

Bullish on MCPs.

Host

我看好 MCP。没有看涨或看跌。它只是——它现在只是格局的一部分,对吧?

I'm bullish on MCP. There's not bullish or bearish. It's just is—it just is a part of the landscape now, right?

Mihail

呃,那是个陈述。曾经有整个“MCP 回来了。MCP 死了。”人们喜欢极端——

Um, that's a statement. There was a whole MCP is so back. MCP is dead. People like extreme—

Host

MCP 门户是一个未经证实的业务。呃,我考虑过创办一个。呃,我实际上提出了一个指导主题,关于可能创办一个,然后我失去了兴趣。我对很多事情失去兴趣。但我呃,里面有什么?呃,所以好的,我在这里想的是,如果这是呃,如果这是本周的设计,呃,更倾向于将其扩展到比如当你采用一些原则时会发生什么,其中很多信息来自我与那些领导原生团队和各自公司团队的对话,所以在这里呃,我认为 MCP 门户只有在达到一定规模时才会真正有趣,那时你实际上必须有一个单一入口点来做治理、安全审计、成本优化等事情。我很想听听你对未来的看法,或者可能没有未来。但对我来说,如果你相信像 Open Router 这样的东西,或者你知道某个地方作为单一入口点,让许多事情发生和连接并有效地被治理。我觉得有一个世界,这也会继续存在。

MCP portals is an unproven business. Uh I thought about starting one. Uh I actually pitched a guiding topic on potentially starting one and then I lost interest. I lose interest in a lot of things. But I uh what's what's in there? Uh so okay so what I'm thinking here is if this is um if this is the week design uh more kind of geared toward scaling this out to like what happens when you take some principles and a lot of this was informed by like conversations I had with folks that were you know leading and native teams and teams at uh at respective companies and so here uh I think that something MCP portal really only becomes interesting once you're at a certain scale where you effectively have to have a single point of entry to do things like governance security auditing cost optimization I'd be curious to hear what you see as the future or maybe lack thereof. But it to me is if you believe in like an open router or you know some place like a single point of entry for for many things to happen and connect to and be effectively you know governed. I feel like there's a world where this also continues to exist.

Mihail

是的。我对此唯一的调整是,这归入 LM 网关。所以它们是完全相同的平台。

Yeah. My only tweak on this is that this folds into the LM gateway. So they are the exact same platform.

Host

这一点我同意你。这一点我同意你。我可以接受,实际上你可以同意。是的。呃,然后顺便说一下,它也归入另一件事,呃,我说的是什么,呃,权限和所有这些事情以及计费,所以就像所有基础设施在极限情况下都收敛到完全相同的问题,它们中间有一个盒子,然后两边各有一个盒子,呃,你知道上帝盒子会解决一切,对吧,只有当你接受呃,如果你理解上帝盒子的工作,这才是真的,所以将 MCP 门户和网关分开意味着你不完全接受它们是呃,你知道同一层问题,而且以非常相似的方式,我也发现人们不理解数据库查询呃,有很多类似的事情,他们会将过滤与排名分开,或者呃,或者只是像一般推荐,如果它们是同一个问题。

That I will give you. That I will give you. I can accept that actually you can agree with. Yeah. Uh and then by the way it also folds into the other thing of um what did I say was uh offs you know permissioning and all these things and billing so like all to the limit all infra converges to the same exact problems where they have the box in the middle and then you have the box in two sides uh you know god box will solve everything right and it's true only if you take uh if you understand the job of the god box and so separating MCP portals and gateways means you don't fully accept that they are uh you know the same layer of problem and in a very similar way I also find that people don't understand database querying uh to have a lot of the similar things where they will separate filtering from ranking from uh or or just like general recommendation if they're the same problem.

Mihail

呃,所以就像然后你就因为当你把所有这些事情放在同一层结合时,你就可以以有用的方式交叉依赖。如果它们是不同的层,那么它们就不能。

Uh and so like then you just com because like when you take all these things you combine them in the same layer then you can sort of cross depend on on things in useful ways. If there are different layers then they cannot.

Host

呃,是的,我实际上不会不同意这一点。我想呃,也许是的,所以稍微教学性地说,我认为这种将单独术语与一个术语分开。我认为绝对正确,底层架构是上帝盒子,一个东西进去,然后有一堆其他东西,有些是第三方的,有些是内部的,有些是第一方的,所有这些都存在于呃上帝盒子的另一边。所以是的,我实际上同意这一点。我有一个我有一个我有一个辛辣的呃辛辣的推文。我哦天哪,上帝盒子推文在哪里?哦是的。所以这是 M MX 到 M plus 架构。呃,所以任何从事基础设施工作的人需要理解呃恭喜你,你现在在基础设施领域有工作了,或者你想成为基础设施工程师,你需要理解你的整个工作就是制造上帝盒子然后拆解它们。所以要么你的问题是你没有足够的你没有一个好的上帝盒子,要么你的问题是别人的上帝盒子你不喜欢,所以现在你必须拆解它并制造你自己的。

Uh yeah I actually will not disagree with this. I think um maybe the yeah so slightly pedagogical I think that the this breaking up of like separate terms versus one. I think it is absolutely correct that the underlying architecture is the god box where one thing goes in and there's a bunch of other stuff that is some third party some internal some first party that all kind of exists uh on the other side of the god box. So yes I agree with this actually. I have a I have a I have a spicy uh spicy tweet. I Oh god, where where is the godbox tweet? Oh yeah. So this is the M MX to M plus architecture. Uh so anyone working in infra needs to understand uh congratulations, you now have a job in infrastructure or you want to be an infra engineer, you need to understand that your entire job is making God boxes and then taking them apart. So either your problem is either you don't have enough you don't have a good gut box or your problem is someone else's god box you don't like so now you have to take it apart and make your own.

Mihail

有趣。非常有趣。我喜欢那个。非常有趣。直到宇宙热寂。哇。有趣。

That's funny. That's very funny. I like that. That's very funny. Until the heat death of the universe. Wow. That's funny.

Host

这这这是对以下说法的重述:只有两种做软件的方式,就是你捆绑它们,你解绑它们。但比如你在捆绑什么?希望是那些一起的东西。然后也许随着时间的推移,人们的观点会改变,什么应该一起,什么不应该一起。

Which which is a this is a a restatement of like there's only two ways to make software which is you bundle them, you unbundle them. But like what are you bundling? It is the things that go together hopefully. And then like maybe over time people's opinions shift as what what goes together what should not go together.

Mihail

对。对。

Right. Right.

Host

所以就像你知道,有那个整个事情,你知道也许呃 Monaco 应该更像一个点解决方案。也许 Monaco 应该拥有一切。对吧。而且就像有个问题,在某些方面你为你服务的那种初创公司细分市场捆绑。在某些方面你从 Salesforce 解绑,因为 Salesforce 那个,然后那就是你从 Salesforce 赢得市场份额的方式。你从 Salesforce 解绑。嗯,那很有趣。是的,那是更广泛的陈述。我的意思是把它连接到所有垂直 AI 或呃,现在像捆绑解绑,我的意思是那就像整个 Craigslist 和 Airbnb,你知道 Tinder 和所有那些都是分开的——

So so like you know there there is that that the whole thing you know maybe uh Monaco should be a more of a point solution. Maybe Monaco should own everything. Right. And like there's the question of like in some ways you're bundling for uh the kind of startup segment that you serve. In some ways you're unbundling from Salesforce because Salesforce that and then that's how you win market share from Salesforce. You unbundle from Salesforce. M hm that's interesting. Yeah, that's that's an even broader statement. I mean connecting it to broadly all of vertical AI or uh and now like bundling unbundling I mean that's like the whole like Craigslist and Airbnb you know the Tinder and all those are separate—

Mihail

Tinder

Tinder

Host

但你知道就像个人广告基本上——

But you know like personals basically—

Mihail

从未那样想过,但是的呃,你知道其中之一,所以我们今天刚发布了 Open Router 播客,对吧

Never thought about it that way but yes uh you know one one of those things so we just shipped the open router pod today right

Host

而且呃 Open Router 巨大成功,很多人尝试构建网关,没有人像 Open Router 那样成功,Open Router 只构建网关。

And um open router huge success lots of people tried to build gateways none of them had success with open router open router only builds gateways.

单点方案与一站式方案 Point solutions vs. one-stop shop

Mihail

这里有个例子。他们不同意我的看法,认为路由层也应该做运维,因为 Arise 在做运维、可观测性之类的,而 MCP 网关——他们没有这些。他们只做模型路由。我特别喜欢这样被证明是错的,因为他们的态度是:我们是点解决方案,我们只把这一件事做到极致,这就是我们的工程重心,这很好。这也是营销重心,挺好的。

So there's an example of this. They disagree with me that the router layer should also do ops, because Arise is doing ops and observability and what have you, and the MCP gateway — they don't have that. They only handle model routing. And I love being wrong like that, because they're like, we are the point solution and we only do this one thing really, really well, and that's our engineering focus, which is good. It's also the marketing focus, which is nice.

Host

嗯。

Mhm.

Mihail

我觉得你描述的那种一站式方案,从理念上讲是合理的,但实际中会发生的情况——其中一部分大概也是营销——尤其是看企业市场,这些东西大体归在基础设施、归在平台下面,但不同团队各自拥有技术栈的不同部分。在企业规模上做这种整合,比如 MCP 门户和网关,得有人站出来说,现在你们访问这两样东西都走同一个端点,而这从根本上就很难让所有人达成一致。所以这往往更像是政治问题、甚至是人的问题,而不是——从纯软件设计的角度看,这非常合理。

I think what you've described — the one-stop shop for all the things — is ideologically what makes sense, but I think what ends up happening in practice, and part of it is probably also marketing, is that especially when you look at the enterprise, this is broadly under infra, broadly under platform, but then there are separate teams owning different parts of the stack. And I think part of that consolidation at the scale of an enterprise — if we're going to say MCP portal and gateway — someone has to come together and say, now there's a single endpoint that you're hitting for both of these things, and that fundamentally is a hard thing to get everyone on board with. So it's often more like political and even like a people problem rather than — from a pure software design perspective, this makes a ton of sense.

Host

是啊。

Yeah.

Mihail

但问题不在这,对吧?软件系统的瓶颈有时更多来自人的问题,而不是纯软件本身。

But that's not it, right? Software systems sometimes are bottlenecked by people problems more than pure software.

Host

我的说法是——你知道“你交付的是你的组织架构图”吧?我忘了这个原则叫什么了。你需要康威定律——你交付的是你的组织架构图,对吧?

The way I frame this is — you know how you ship your org chart? Well, I forget the name of this principle. You need Conway's law — you ship your org chart, right?

Mihail

你买的是你的组织架构图。就像你创业或者做一个工具时,公司里谁是你的最佳好友(目标用户)?

You buy your org chart. Like, when you start a startup or you make a tool, who in the company are you best friends for?

Host

如果它是给所有人的,那就是给没人的。

And if it's for everyone, it's for no one.

Mihail

有意思。这确实有意思。

That's interesting. That is interesting.

Host

所以 Latent Space 和 AIE 有意识的一部分就是:好吧,有这么一类软件工程师,他们非常非常非常关心 AI。这会不会成为他们的日常工作?那我们就只为他们做。

So a conscious part of Latent Space and AIE is like, okay, there's this category of software engineer that really, really, really cares a lot about AI. Is it going to make it their day job? Let's only make it for them.

Mihail

对,对,对,对,对。不是把工程打包。永远不可能成为所有人的开发工具。

Right, right, right, right, right. Not a bundling of engineering. Never going to be the dev for everyone.

Host

是啊。你看,我们谈数据库、谈安全,但这里面有个 AI 的角度。

Yeah. Look, we talk about databases, we talk about security, but there's an AI angle to it.

Mihail

嗯。

Mhm.

Host

是啊。走着瞧吧。有道理。

Yeah. We'll see. That makes sense.

平台工程师的崛起 The rise of the platform engineer

Mihail

我还想提一点,和这个话题相关:我认为未来越来越重要的一个角色是平台工程师。我觉得这个角色非常重要。

One thing I'll shout out as well, related to the point, is I think the role of the future that is increasingly important: platform engineer. I think this is a huge one.

Host

你知道吧。好,再多讲点背景——Temporal,在我这份工作之前,在这一切之前。对,我们当时想拿下平台工程师。如果我没创业,对,我会去做平台工程师,因为那是决定所有关键任务东西的角色,然后应用那边的孩子们可以在我们的平台之上想干嘛干嘛。但你知道,大佬们做平台。

You know. Okay. More lore — Temporal, pre my job, before all this. Yeah, so we wanted to own platform engineer. If I hadn't started, yeah, I would have done platform engineer, because that is the role that decides all of this mission-critical stuff, and then the kids over in applications can do whatever they want on top of our platform. But you know, the big boys do platform.

Mihail

是啊,我觉得在当下,他们拥有云、拥有开发者体验、拥有安全运维,所以我觉得是所有这些东西。

Yeah, and I think in the modern day they own cloud, they own DevX, they own security ops, so I think it's all the things.

Host

就像所有能让你真的把公司搞垮、或者决定成败的东西——对,平台的人。然后产品的人可以,你知道,调调 CSS。

Like everything where you can actually lose the company or make or break it — yeah, platform guys. And then product guys can, you know, adjust the CSS.

Mihail

这就是你开始在这儿争的东西。我是说,我不是调 CSS 的人,但我感谢大家。

That's what you're starting to fight here. I mean, I'm not the CSS adjuster, but I appreciate everyone.

Host

我是说,这是个有趣的时代。未来是什么?我感觉所有人都能调所有东西了。

I mean, it's a fun time. What's the future? I feel like all the guys can adjust all the things.

Mihail

是啊,是啊,是啊。

Yeah, yeah, yeah.

软件工厂与人在回路 The software factory and the human in the loop

Host

这里有几个方面。好,所以有一点上周内容的延续——编码智能体、大团队、一些概念——但接下来我们会谈到软件工厂。我想让大家真正开始思考这个“制造机器的机器”的世界,以及人在其中到底还在不在环里。你知道,今天我认为答案是意图和某种审查的组合——也许是意图,你在这里定义规格和要建什么;以及审查,你在这里自己对系统做一些检查。当然,这个——我觉得这个瓶颈会比意图那部分更早被松开。但就目前而言,我认为在一个完全自动化的系统里,这是两个卡点。所以对,我就是想让大家接触一下——其中一个开源合作伙伴,其实是 Warp,他们在为开源项目做贡献时做这件事:他们基本上会做一个运行工厂的 Warp 版本,学生们就负责建工厂、给工厂做贡献、让它持续运行。

So there are a few aspects here. Okay, so there's a little bit of spillover from the previous week — coding agents, big teams, some of the concepts — but then we'll touch on the software factory. I want people to really start thinking about this world of this machine that builds the machine, and where, if at all, people are still in the loop there. You know, today I think the answer is some combination of intent and kind of review — maybe the places like intent, where you're defining the specs and what needs to be built, and review, where you're doing some amount of your own checking of the system. And of course this — I think that bottleneck is going to be loosened up before even the intent piece does. But for now those are the two choke points, I think, in a fully automated system. So yeah, I just want to give people exposure — one of the OSS partners, actually Warp, they're doing this thing as part of their contribution to the OSS program, where they're going to essentially make a version of Warp that is running the factory, and students will just be like building the factory, contributing to the factory, and keeping it running.

Mihail

就像,在我们的工厂里干活。

Like, work in our factory.

Host

在我们的工厂里干活。没错。然后他们会发现那里有趣的东西,再把它带回来。

Work in our factory. Exactly. And then they're going to find things that are interesting there and pull it back.

Mihail

这就是那种我不太确定的事——显然工厂本身非常非常热衷于此。

This is one of those things where I'm not sure — obviously the factory itself is very, very keen on this.

Host

我感觉每个人都想建自己的。

I feel like everyone wants to build their own.

Mihail

是啊,我觉得这是个好问题,而且我觉得他们当然有权推销自己的产品。我认为要完整拥有这套系统很难。东西太多了。太多了。

Yeah, I think that is a good question, and I think they're, you know, of course entitled to try and sell the products they do. I think it's a hard system to own in its entirety. It's a lot. It's a lot.

Host

请替我把一切都决定了吧。

Decide everything for me, please.

Mihail

是啊,确实很多。我试着——你知道,我没有太深入研究他们在这里的市场进入策略,但我猜你得一块一块地做,对吧?你得先——先做他们的编码智能体,哦,然后也——基本上就是先落地再扩张,对吧?我是说,我们谈过这个打包和解包的问题。你不能一上来就说,这是你的整个工厂。你不可能让福特公司接受这个,你懂我意思吧?你不可能走进去就把整套东西往他们桌上一放。至少我会这么想:你先成为他们首选的编码智能体,你为他们搭一些后台智能体,你拿下他们的一些 SaaS 部分,然后你再说,好吧,现在只要你们把所有东西都用代码定义,或者类似配置的东西,我们就能把它全编织在一起。OpenAI 有一篇关于这个的有趣博客。我们的朋友 Ryan——我们请 Ryan 上过播客。基本上,他们花了五个月做一个内部工具,零行手写代码。他们大致讲了一遍——这叫 harness engineering(框架工程)。所以你可以学 harness engineering。

Yeah, it is a lot. And I'm trying to — you know, I haven't gone too deep in understanding how they approach their GTM here, but I suspect you would have to do it piecemeal, right? You would have to be like, hey, we first — your coding agent, oh, then also — you basically land and expand, right? I mean, we talk about this bundling and unbundling piece. You don't start by saying, here's your whole factory. There's no way you could get a Ford on board with that, you know what I mean? Where you're going to go in and just plop the whole thing on their desk. That's at least how I would think about it: you first be their coding agent of choice, you build up some background agent for them, you own some of their SaaS pieces, and then you're like, well, now we can just weave it all together if you define everything in code or something like that, like configs. There was a fun blog post on this from OpenAI. We had our buddy Ryan — we had Ryan on the podcast. Basically, they spent five months making an internal tool with zero lines of manually written code. They kind of walk through one — it's called harness engineering. So you can learn harness engineering.

软件工厂愿景 The software factory vision

Host

但学到的经验是,一开始会有点低迷,或者进展较慢,然后你把活儿卸载出去,还有同行评审。这是二月份的东西,但它描绘的是一个软件工厂的愿景,对吧?你全力投入,让智能体完全写出所有东西,以及如何把它做好。

But the learnings were that there was kind of a slump, or slower at first, and then you offload and you had peer review. This is from February, but it's a vision of a software factory, right? You're full sending, letting agents fully write everything, and how to do that well.

Mihail

是的,绝对会。我是说,某种版本的这个东西一定会存在。顺着 SP 的观点,我确实对这里的销售部分有疑问,也就是这件事实际的商业层面。那是个有意思的问题。但从一个负责人的角度看,我认为这是未来的一种版本。再往远一点,我觉得会有一定程度的探索,并且谈谈,好吧,未来做一名开发者意味着什么,对吧?人们谈品味,谈对系统和架构的理解,还有软件品味。我想在这里再深入一点,也许开始为这个问题给出一些答案。

Yeah, absolutely. I mean, some version of this is going to exist. To SP's point, I do wonder about the sales piece here, like the actual business of this. That's an interesting question. But from a principal perspective, I think this is one version of the future. And then beyond that, I think there's going to be some amount of dabbling into, and speaking about, like, okay, what does it mean to be a developer in the future, right? People talk about taste, people talk about understanding of systems and architecture, and software taste. I want to go a little bit deeper here on maybe starting to at least put some answers to the question around this.

Host

好,所以这就是你对 AI 软件工程下一步走向的回答。

Yeah, so this is your answer to where AI software engineering goes next.

Mihail

是的。我是说,工厂是战术层面的,但我觉得在那之外还有一个关于角色的问题,对吧?这个角色会演化成什么?如果工厂在运转,全速前进做它的事,传送带在源源不断地产出代码,那我们在这其中做什么?对吧?我记得你和我几周前有一次晚餐时和 Jerry、Dex 还有 Jerry 聊过工厂的事。但我觉得,至少那些讨论点里有一部分我是相当认同的,就是抽象层级不断提高。然后不知怎么的,仍然得有人,甚至一个公司的 CEO,也得有足够的监督、对不同组成部分的理解,才能对方向做出判断,对吧,才能纠偏,才能设定指标之类的事情。我觉得这有一个可以迁移的版本。这个类比有一个版本可以迁移到监督技术的人身上。我们叫它智能体管理者,我们用不同的说法,都是些流行词。但在某种意义上,是那种理解产品需求、约束、技术约束,把这些都串起来,并且知道如何裁决的人。

Yeah. So I mean, the factory is the tactical part, but I think beyond that there's a question around the role, right? What does the role evolve into? If the factory is running and, you know, full steam ahead doing its thing, the conveyor belt is turning out code, what do we do in all this? Right? And I think you and I had this discussion with Jerry at the dinner once upon a time a few weeks back, with Dex and Jerry, around the factory. But yeah, I think there was this part of at least the discussion points there which I'm pretty much on board with, which is increasing levels of abstraction. And then somehow there's someone still, even a CEO of a company has to have enough oversight, understanding of the different constituent parts, to be able to make a judgment on direction, right, to be able to course correct, to be able to set metrics and things of that nature. I think there's a version of that which translates. There's a version of that analogy that translates to someone who oversees the technical. We call it a manager of agents. We call it different terms. They're all buzzwordy. But in some respect, someone who understands product requirements, constraints, technical constraints, kind of loops through all of those and knows how to adjudicate.

Host

是的。我得说,在你之前坐在这把椅子上的,是 Anthropic 的平台产品负责人和工程负责人。我当时就问,你们的岗位什么时候会合并?他们说,还没呢,你知道吧?所以如果 Anthropic 都还没到那一步,你不该觉得这事迫在眉睫。

Yeah. I will say that in the chair before you, we just had Anthropic's head of product and head of engineering for the platform. And I was like, when are your jobs going to combine? And they're like, not yet, you know? And so if Anthropic is not there yet, you should not feel like it's imminent.

Mihail

好,好,好。他们说了要多久吗,还是只是说还没?

Good, good, good. Did they say how long, or are they just not yet?

Host

还没,但谁知道呢?而且我确实,确实有很多那种,好吧,对,我在干她的活儿,她在干我的活儿。就是这样。

Not yet, but who knows? And I definitely, there was definitely a lot of like, okay, yeah, I'm doing her job, she's doing my job. Like, yeah.

Host

我觉得这对学生来说也是个有意思的视角,对吧?你说有很多大二的本科生,你们还有两三年时间来规划自己想做什么。

I think it's also an interesting perspective for students, right? You said there's a lot of sophomore undergrads, like you've got two, three years to go to plan out what you want to be doing.

Mihail

从这个角度看有不同方式,对吧?你不是直接就进业界。

There's different ways to look at it from that, right? You're not straight into industry.

Mihail

我关心的、我思考很多的一件事是,这场讨论里更让我有共鸣的一部分是那个循环的东西,就是,好吧,你一开始写提示词,现在你不再写提示词了,你写的是能写提示词的提示词。没错。

The thing that I care about, I'm thinking a lot about, is, well, so one part of this discussion that resonates more with me is the loop stuff, where, okay, you started writing prompts, now you no longer write prompts, you write prompts that write prompts. That's right.

Host

对。然后你就一直往上走。

Right. And then you just keep going up.

Mihail

我觉得对我来说,那种递归的同心圆性质,既向外也向内,是构建工厂的东西。我认为你是往上构建到工厂,而不是往下构建。但我觉得很多人最终会开始往下构建,就像你从顶层开始,我的目标是什么,然后你再理清系统。那目前会产生大量垃圾,你不该那么做,但总有一天你会那么做。

And I think to me, the sort of recursive concentric circles nature, going outward but also going inward, is something that builds the factory. I think you build up to the factory, not build down. But I think a lot of people will eventually start building down, like you start from the top level, like what is my goal, and then you figure out the systems. That currently creates a ton of slop, and you should not do that, but someday you will.

Host

嗯。对。对。

Mhm. Yeah. Yeah.

Host

所以与其,你知道,那会是我要问你的问题,就是,人们存在的那个核心问题是什么?就像,那是你的东西,对吧?所以,比如说今天一个工程师,你是用 PR 合并率、PR 贡献率来衡量的。

So instead of, so you know, that would be my question to prompt you, is, what is the sort of prime question that people exist to solve? Like, that is your thing, right? So in, let's say, an engineer today, you are measured by your PR merge rate, PR contribution rate.

Mihail

嗯,两年后那会很荒谬。好吧,那你的主要指标是什么?

Well, two years from now that's going to be ridiculous. Okay, well, what's your main thing?

Host

对,我不知道。

Right, I don't know.

Mihail

是啊。

Yeah.

Host

钱。

Money.

Mihail

人们总是太快就想到钱。是介于钱和,

People always go to money too quickly. It's something in between money and,

Host

对,PR,

right, PRs,

Mihail

客户满意度,你做的功能的 CSAT 分数。

customer satisfaction, CSAT scores from your features.

Host

就连那个也像是产品决策,而不是真正的代码决策,对吧?就像,我不知道,就像——

That even that's like a product decision, not really a code decision, right? Like, I don't know, like—

Mihail

但如果工程正在演化成更像是 PM 工程师,也就是产品工程师,你知道,混合型的,那我觉得这就是对角色演化成什么这个问题的一个答案。我觉得不用说,我今天会建议学生花更多时间思考整体系统,而不只是工程拼图里你那一块,而是它如何串联起来。这在实践中最终就是这样:学生做毕业设计项目时,我鼓励他们走出去,事实上,我给他们作为期末项目点子种子的是 YC 的创业需求清单。我几乎就是说,去建一家公司吧,你懂我意思吧?去建点有意思的东西。

But if the engineering is evolving into more of a PM engineer, like product engineers, you know, hybrid, then I think that's one answer to the question of what does the role evolve into. I think it goes without saying that I would today recommend to students to spend more time thinking about holistic systems, not just your single part of the engineering puzzle, but how that ties in. And this is like in practice ends up happening: students for their capstone projects, I encourage them to go out and, in fact, what I give them as the seed of ideas for final projects is the YC request for startups. I'm almost like, go build a company, like, you know what I mean? Go build something interesting.

Host

去年人们建了什么?

What did people build last year?

Mihail

各种各样的东西。有个人进来,是电子系的,在电子学院学电子,他们实际上做了一个翻译器,用来解析带有 C 语言内存地址的手册,然后把它转换成他们可以解释甚至可视化的代码。所以是特定于他们用例的东西。还有个人本来是设计师,现在其实在 Vercel,她做的是,你知道,拍一张我设计笔记本的照片,然后根据我拍的照片给我生成一个应用。对。所以真的,她的演示就是,我说给你,她随手拍了一张她乱涂乱画的东西,然后她就从中得到了一个完整成形的 Vercel 应用。我觉得正因为这个,她大概因此在 Vercel 找到了工作,做某种版本的这个。

A variety of things. Someone came in, someone in the E department, was in the E school and was studying E, and they built effectively a translator for parsing manuals that have memory addresses in C, and then converting that into code that they could then interpret and even visualize. So stuff specific to their use case. Someone who was a designer and is now actually at Vercel, and she had, you know, take a picture of my design notebook and generate me an app based on what I take a picture of. Right. So literally, for her demo, I was like, here you go, snaps a picture of some random scribbles that she was doing, and she got a fully fleshed-out Vercel app from that. And I think because of that she probably got a job at Vercel doing some version of this.

Host

哦不错。那其实门槛很高。Vercel 以他们的设计工程师闻名。

Oh nice. That's a high bar actually. Vercel, very known for their design engineers.

Mihail

对。对。而且我记得她现在在做 Eve。

Yeah. Yeah. And I think she's now working on Eve, from what I remember.

元harness与学生接触 Meta Harnesses and Student Exposure

Host

啊,不,Eve 在你们的……Eve 是一个开源软件或类似的开源合作伙伴。我的意思是,合作方式之一就是他们让学生用 Eve 来构建。

Ah, no Eve in your... Eve is an OSS or like OSS partner. I mean, one of the ways is partnering with us is that they are having students build with Eve.

Mihail

是的。所以这就是元框架(meta harness)的概念,对吧?框架的框架。你总是可以再深入一层。但我确实认为这说得通,至少对我来说是这样,就像模型路由器封装模型 API 一样。如果你要在它们之间切换,你也需要封装框架。

Yeah. So this is the concept of meta harnesses, right? Harnesses of harnesses. You can always go one level deeper. But I do think it makes sense, at least for me, in the same way that model routers wrap model APIs. You need to wrap harnesses as well if you're going to switch between them.

Host

同意,同意。我认为在某种程度上,学生也应该保持脚踏实地,因为边缘有很多事情在发生。也有很多公司仍然在生产中只做单次调用,还在摸索,没有尝试过。这种情况很多。所以还有时间让它发挥作用。但我觉得你关于品味的观点很好。要有品味,多尝试。无论每家公司是否有自己版本的软件工厂,你都在构建工具来帮助公司做到这一点,比如紧跟潮流,实际使用工具,把玩它,你会在过程中弄明白的,你知道吗?

Agree. Agree. I think at some level students should also stay grounded in like there's a lot of stuff happening at the edge. There's also a lot of companies that are still like single call in production playing around haven't tried it. There is a lot of that. So there's still time for it to work. But I think your point on taste is good. Have taste, try stuff. Whether each company has their own version of a software factory, you're building tools to help companies do it, like staying on top of it, actually using the tools, playing with it, you'll figure it out along the way, you know?

Mihail

是的。我完全同意所有这些。首先,针对你说的很多公司仍然停留在提示 ChatGPT 或某个版本的 ChatGPT,也许他们刚拿到 Cursor 许可证,现在才让人上手。这比我们想象的要普遍得多,尤其是在硅谷。我可以百分之百肯定地说。但没错,我认为对学生来说,接触这些只是到了某个阶段,他们知道这些并且处于前沿,这最终会让他们在去往那些可能在前沿方面还不太够格的地方时,也能成为更有趣、更有价值的贡献者。

Yeah. And I think I 100% agree with all that. First, to your point that there are a lot of companies still in the prompt ChatGPT or some version of ChatGPT, maybe they just got a Cursor license and are just now getting people onboarded. That is way more true than we believe, certainly here in the valley. I can say that with 100% confidence. But yes, I think for the students being exposed to this is just at a certain point the fact that they know this and are on the edge is what will eventually allow them to also be some of the more interesting and more valuable contributors when they go to maybe places where that maybe aren't up to snuff in terms of being on the frontier.

Steve Yegge的精神病等级 Steve Yegge's Psychosis Levels

Host

是的。其中一个关于进展类型的演讲,仍然很有影响力,大约一年前,Steve Yegge 谈论精神病的级别。他现在已经到了第 10 级,他放弃了,搬到了农场。但是的,抱歉我在开他玩笑。他构建了 Gas Town。

Yeah. One of these progression-type talks, still very influential, about a year old, Steve Yegge talking about the levels of psychosis. He's all the way through level 10 now, where he's given up and he's moved to a farm. But yeah, like, sorry I'm making fun of him. He built Gas Town.

Mihail

是的,我记得。

Yeah, I remember.

Host

所有这些都搬到了 Gas City,现在他烧毁了城市,说我要构建我的游戏,不再有 Gas Town。我以为……我记得今年早些时候在工程师博览会上和他聊过。他的整个观点不就是 Gas Town 唯一有用的东西就是 Gas Town 吗?

All these things moved to Gas City, now he's burned down the city and said I'm building my game and no more Gas Town. I thought... I remember talking to him at the engineer fair earlier this year. Wasn't his whole thing like the only useful thing that was built at Gas Town was Gas Town?

Mihail

对吧?那是一个可怕的失败。可怕。

Right? That's a horrible failure. Horrible.

Host

完全同意。完全同意。

Totally. Totally.

Mihail

所以人们应该意识到精神病是真实存在的,如果你不要求自己对交付真实产品负责,你只是在瞎搞语言模型。

And so people should be aware that psychosis is real and if you're not holding yourself accountable to delivering real products, you're just screwing around with LMS.

Host

是的。这可能相关,但我想知道有多少……我玩过一点 Gas Town,非常复杂的系统,也许有点超前。但我也想知道他们是否被营销稍微瓶颈了,从术语角度来说很难理解。就像有很多事情在发生。我认为 Gas Town 的营销也是推动它的事情之一,对吧?

Yeah. This is maybe relevant, but I wonder how much of... I played a little with Gas Town, very involved system, maybe a little ahead of its time. But I'd also wonder if they were bottlenecked a little bit by the marketing and that it was a very difficult thing to grok from a terminology perspective. Like there's just a lot going on. I think the Gas Town marketing also is one of the things that propelled it, right?

Mihail

在 Gas Town 之前是什么?有像……

What was right before Gas Town? There were like...

Host

像他的……是的,他做的 Beats,除非你说的是别人的。不,不,有像……有整个 open claw,还有别的,还有 Gas Town。整个宣言也确实帮助推动了它的概念化。那些永远不会使用 Gas Town 的人,但了解了 Gas Town,因为你可以读宣言。有一个营销角度。

Like his... Yeah, Beats he did, unless you're talking about someone else's. No, no, there were like... there was the whole open claw, there was something else, and there was Gas Town. And the whole manifesto also really helped spur it into the conceptual. The people that would never use Gas Town but learned about Gas Town 'cause you can read the manifesto. There's a marketing angle to it.

Mihail

是的。我的意思是,所以不一定是要字面理解的人,但方向上是正确的,你知道吗?所以我认为他在这点上方向是正确的。所以如果你处于 ID 阶段,是时候进入高级用户阶段了。如果你是一个智能体,你可能该转向多智能体等等。而且你可能应该尝试构建自己的编排器,对吧?所有这些可以说……这甚至不是……这现在有点过时了。软件工厂还有更多内容 beyond this。但这篇文章早于软件工厂。我确实认为,你知道,其中一件事就像,我愿意……作为雇主,我今天在我的公司有这个问题,就是我不想雇佣任何没有经历过精神病并从中走出来的人。

Yeah. I mean, so not necessarily someone to take literally, but directionally correct, you know? And so I think he's directionally correct on this one. And so if you're at the ID stage, it's time to move into power user stage. If you're a single agent, you probably time to move to multi-agent and blah blah blah. And you probably should try to build your own orchestrator, right? And that's all arguably... this is not even... this is kind of out of date now. There is more to the software factory beyond this. But this article predates the software factory. I do think, you know, one of those things where like, I'm open to... as an employer, I have this problem in my company today, which is I do not want to hire anyone who hasn't been through psychosis and come out the other end.

Host

因为我目前正在付钱让某人经历精神病。

Because I am currently paying for somebody to go through psychosis.

Mihail

从薪水来说非常昂贵,从 token 来说非常昂贵,而我什么也没得到。他还不明白,就像我的反馈传达不到。所以就像,我几乎会鼓励你的学生经历精神病,然后走出来说,‘好吧,我怎么了?’对吧?因为就像 Steve 刚刚经历的那样,一个非常公开的两年旅程。

And that's very expensive from his salary, very expensive from tokens, and I'm getting nothing out of it. And he doesn't get it, like my feedback doesn't get through. And so like, it's almost like I would encourage your students to have psychosis and then come out and go like, 'Okay, what was wrong with me?' Right? 'Cause like that's what Steve just went through, like a very public two-year journey of this.

Host

是的。在学生方面很奇怪。有很多行业……你认为自己很高效。其实不是,这可能是学生版本,对吧?你在用 Notion 做图表,做所有这些优化,个人网站,然后就像,该死,我经历了精神病但什么都没变。

Yeah. It's weird on the student side. There's like a lot of industry... you think you're being productive. You're not, is probably the student version, right? You're notion charting and making all these optimizations, personal sites, and it's like dang I went through the psychosis but nothing has changed.

Mihail

对。对。

Right. Right.

Host

所以经历它。经历它。

So go through it. Go through it.

Mihail

对。是的。我没见过这个。我实际上见过 Boris 的一个版本,显然更针对 Claude 的用例,但我记得他做了一个 Claude 工件,展示的,我想更多是在团队层面,但我认为他有四个阶段,不是八个阶段,但它大致遵循了相同的进展,从没有采用到……你知道,我猜……

Right. Yeah. I haven't seen this. I've seen a version of this from Boris, actually, obviously more geared toward Claude use cases, but I remember he made like a Claude artifact where it showed, I think more at the team level, but I think he had four stages, not eight stages, but it kind of roughly followed the same progression of like no adoption to through to, you know, I guess...

Host

只有四个是好的。这个级别太多了。是的。好吧,看看 V 能不能把它调出来。但除此之外,拉远来看,我们讨论了所有事情吗?那是一个。哦,那是一个。好的,来了。它是一个架构师。好的。

Only four is good. This is too many levels. Yeah. Well, see, we'll see if V can pull it up. But other than that, zooming out, have we discussed all the things? That's one. Oh, that's one. Okay, there we go. It is an architect. Okay.

Mihail

所以是门控、辅助、并行。最后一个是什么?

So it's gated, assisted, parallel. What's the last one?

Host

监督自主和 AI 原生。好的。

Supervised autonomy and AI native. Okay.

Mihail

所以是零索引。所以是五个级别,对吧?

So it's zero index. So it's five levels, right?

Host

好吧。

All right.

Mihail

从零开始。

Start at zero.

Host

是的。哇。实际上,这相当不错。我需要更深入地读一遍。

Yeah. Wow. Actually, it's pretty good. I need to read through this in more depth.

Mihail

使用原生。这就对了。

Use a native. There you go.

Host

我要特别提一下动态工作流。这是子智能体模式之一,OpenAI 的人也在探索,这显然很合适。所以基本上就是你为正在尝试做的事情按需编写一个框架。

I would give a shout out to dynamic workflows. This is one of those things that is a sub-agent pattern that the OpenAI guys are also exploring, that is clearly fitting. And so it's basically you're writing a harness on demand for the thing that you're trying to do.

动态工作流与子智能体 Dynamic Workflows and Sub-Agents

Mihail

所以这也许就像很多子智能体的对话,你只是在复制父智能体。

And so this is maybe like a lot of sub-agent conversation is just you're copying the parent agent.

Mihail

但实际上你应该写更多的代码,然后把它作为子智能体来运行,这就是 Anthropic 的人选择称之为动态工作流的东西。

But you actually should be writing way more code and then running that as a sub-agent, and that is what the Anthropic people have chosen to call dynamic workflows.

Host

好的。

Okay.

Mihail

是的,我们也构建了一个版本。是的,这是一个好模式。

Yeah, we've built a version of this as well. Yeah, it's a good pattern.

Host

给 Monaco 用的?

For Monaco?

Mihail

嗯。是的。

Mhm. Yeah.

Host

它是做什么的?

What does it do?

Mihail

是的。我的意思是,动态地构建类似 harness 的用例,用于 GTM(市场推广)。

Yeah. I mean on the fly sort of like harness sort of use cases for GTM.

Host

哦,那很棒。

Oh, that's great.

Mihail

是的。

Yeah.

Host

嗯

Um

Mihail

我要指出,Boris 写这个是为了团队采用,给听众们。他还说,你知道,Anthropic 处于第三阶段。我们还在努力达到第四阶段。

I will note that when Boris wrote this, it's for team adoption for people that are listening. And also he says, you know, Anthropic is on stage three. We're still trying to get to stage four.

Host

哇。

Wow.

Mihail

他说,我处于第四阶段。我就是那个人。但你知道,这不像 Steve 的那么简单。Steve 是一个非常好的个人“做这个、做这个、做这个”。呃,这更像是如果你在管理一个团队,你想让你的团队 AI 化,而不仅仅是作为一个用户,你知道 Anthropic 仍然处于第三阶段。不要

He's like, I am at stage four. I'm the guy. But, you know, it's not as simple as Steve's. Like Steve is a very good personal do this do this do this. Uh this is a much more if you're running a team and you want to AI-pill your team and not just be a user of one, you know Anthropic is still at stage three. Don't

Host

不要像那样,是的

Don't be like yeah

Host

好的,我的意思是你应该教这个,这很棒,这包含了所有东西,这就是我们谈论的一切

Okay I mean you should teach this this is this is great this is this got all the stuff that this is everything that we talked about

Mihail

这就对了

There you go

Host

我不敢相信我还没遇到过这个

I can't believe I haven't come across this

Mihail

嗯

um

Host

好资源,是的

good resource yeah

Host

非常非常好,好的,那是一次疯狂的旅程,一个学期的斯坦福课程在 1 小时或 2 小时内。感谢你这样做。有什么临别想法吗?有什么行动号召吗?

very very good okay that was a wild tour once one semester of Stanford in 1 hour or two two hours. Thank you for doing this. Any parting thoughts? Any calls to action?

Mihail

不,这是,呃,谢谢,我的意思是很有趣。这是九个月的制作。我很兴奋。这是第一周结束。有很多事情要做,有很多要学。我很兴奋看到特别是 OSS 合作伙伴会发生什么,以及让学生深入一些复杂代码库的代码。嗯,让他们构建一些东西。所以不,会很有趣的。保持联系。

No, this was uh thanks for I mean ton of fun. It's been a you know nine months in the making. I'm excited. This is end of week one. A lot to do, a lot to learn. I'm excited to see what happens with especially the OSS partners and just getting students like deep in the code of some complex code bases. Um having them build some stuff. So no, it'll be fun. Keep you guys posted.

Host

你明年会做另一个版本。你需要给学生学分。如果 OSS 的人想参与,他们可以联系。

You're doing another version next year. You need credits for students. If OSS people want to get involved, they can reach out.

Mihail

当然。

Sure.

Host

因为学分很重要。他们想要吗?

Because credits matter. Do they want it?

Mihail

什么?

What?

Host

他们真的想要学分吗?

Do they actually want the credit? The

Mihail

学生还是他们想要你你有信用卡。我不认为很多学生想花钱。

students or they want you you have credit card. I don't think a lot of students want to spend.

Host

我想要,是的,信用卡。但是

I want Yeah. credit card. But

Mihail

是的。不,我认为,看,我记得,你知道,肯定有一定数量的学分,但除此之外,他说,看,如果他们想在这方面尽情发挥,告诉我,我们会做这些一次性实验,他们可以扩展到一千多个智能体。这是来自 Factory。

yeah. No, I think look I go like I remember you know it was like you know there's a certain number of credits for sure but then on top of that he's like look if they want to go hog wild with this like let me know and we'll do have these like one-off experiments where they can just like scale up to like a thousand plus agents. This is from factory.

Host

是的,你知道 Factory。

Yeah, you know factory.

Mihail

不,那种人很有价值。所以这是 Cursor 的 Wilson,对吧?他说,我要写一个浏览器,对吧?

No, those kind of people are valuable. So this is Wilson from Cursor, right? Who is like I'm going to write a browser, right?

Host

那些有雄心壮志并且能真正执行的人,那些非常有价值,这基本上是一个招聘号召。

Those people who are ambitious and like can actually execute, those are very valuable and that's basically a recruiting call.

Mihail

没错。说得好。是的,但我确实认为学生应该理解如何有雄心,因为拍张照片,我的意思是尊重地说,拍张照片并把它变成应用,在这个时代并不那么有雄心。呃,写一个浏览器才是,而且实际上还有更多类似的项目。

That's right. That's a good point. Yeah, I but I do think like students should should like understand how to be ambitious because snapping a photo I mean respectfully snapping a photo and turning into an app is not that ambitious in this day and age. Uh writing a browser is and there and like there actually is a lot more projects like that.

Host

是的。是的,我认为这是对的。是的,这都是关于提高下限。我的意思是,我认为,是的,去年是,呃,你知道,归功于去年的学生,但呃,是的,今年我显然我们会期望我们更多

Yeah. Yeah, I think that's right. Yeah, it's all about raising the floor. I mean I think yeah last year it was uh it would you know you know credit to the students of last year but uh yeah this year I obviously we're going to expect we're more

Mihail

那现在算是失败了。

that's a fail now.

Host

我们会期望更多。我的意思是,是的,但那是,但那是重点,对吧?就像我们在升级,对吧?去年就像是,嘿,你听说过提示吗?就像,这就是它。我花了一些时间谈论,你知道,K-shot 提示作为其中一课。而我觉得你们现在甚至不会想到那个。所以,

We're going to expect more. I mean, yeah, but that's but that's that's the whole point, right? Like we're leveling up, right? Last year it was like, yeah, hey, have you heard of prompting? Like, here's what it is. And I spent some time talking about like, you know, K-shot prompting as one of the lessons. And like, I don't even think you guys even think about that now. So,

Mihail

我认为有一种想法是你在同学中脱颖而出。然后还有一种是你也在早期工程角色中,很多行业里的人,人们还在努力找工作。和 snap snap,呃,你知道,不是要指出一个演示,但你可以做很多事情来脱颖而出。我,我会给,呃,学生一个第 11 周的免费提示。呃,制作基准,制作基准和评估。呃,这会让你,一个好的会带你进入每个前沿实验室,每个顶级智能体实验室的大门。

I think there's a there's a version of thinking you stick out in your cohort of classes. And then there's also a version of you're also in early engineering roles of a bunch of people in industry, people still trying to get jobs. and snap snap uh you know not to call out one demo but you can you can do a lot to stick out for sure. I I'll give uh students a freebie uh of a week 11. Uh make benchmarks, make benchmarks and evals. Uh this will get you uh a good one going to get you in the door of every frontier lab in every front every top agent lab.

Host

好的,好的工程博客文章,对任何有趣的事情都有一些严谨性。那里有很多基准或低垂的果实,但工程博客文章确实做得很好,对吧?你你深入钻研了一件事。

Good good engineering blog posts with some rigor on anything interesting. There's a lot of benchmarks or low hanging fruit there, but the engineering blog posts do really well, right? You you went deep on one thing.

Mihail

是的。嗯,可能也有很多钱可以赚。所以,不仅仅是,呃,进入大门,但如果你把它变成生意,你你你可以真的真的从中获利。

Yeah. Um and probably also there's like big a lot of money to be made in that as well. So, not just uh in the doors, but if you make that into businesses, you you can uh you can really really cash in on that.

Host

是的。人们不知道 SWE-bench 就像本科项目。

Yeah. People don't know like SWE-bench was like undergrad project.

Mihail

是的。是的。是的。没错。来自普林斯顿,对吧?是的。是的。是的。

Yes. Yes. Yes. That's right. From from Princeton, right? Yeah. Yeah. Yeah.

Host

太好了。好的。嗯,太棒了。他有。

Great. Okay. Well, fantastic. He has.

互动版:逐字朗读 + 针对本期提问 →