软件 3.0：AI 编程时代

欢迎前特斯拉 AI 总监安德烈·卡帕西。

Please welcome former director of AI at Tesla, Andrej Karpathy.

大家好。哇，人真多。大家好。我很兴奋今天能在这里和大家聊聊 AI 时代的软件。我听说你们很多人是学生——本科生、硕士生、博士生等等——即将进入这个行业。我认为现在进入行业是一个非常独特且有趣的时机。根本原因是软件正在再次发生变化。我说“再次”，是因为我之前就做过这个演讲，但问题是软件一直在变。所以我有很多素材可以创作新演讲，而且我认为变化非常根本。粗略地说，软件在 70 年里没有发生根本性变化，但在过去几年里却快速变化了两次。所以有大量的工作要做，大量的软件需要编写和重写。

Hello. Wow, a lot of people here. Hello. I'm excited to be here today to talk to you about software in the era of AI. I'm told that many of you are students—bachelors, masters, PhD, and so on—and you're about to enter the industry. I think it's an extremely unique and very interesting time to enter the industry right now. The fundamental reason is that software is changing again. I say 'again' because I actually gave this talk already, but the problem is that software keeps changing. So I have a lot of material to create new talks, and I think it's changing quite fundamentally. Roughly speaking, software has not changed much on a fundamental level for 70 years, and then it's changed about twice quite rapidly in the last few years. So there's a huge amount of work to do, a huge amount of software to write and rewrite.

我们来看看软件领域。如果我们把这看作软件的地图，这是一个很酷的工具叫 GitHub 地图。这是所有已编写的软件——给计算机的指令，用于在数字空间执行任务。如果你放大，这些都是各种仓库和所有已编写的代码。几年前，我观察到软件正在变化，出现了一种新型软件，我称之为软件 2.0。这个想法是：软件 1.0 是你为计算机编写的代码。软件 2.0 基本上是神经网络，特别是神经网络的权重。你不是直接编写这些代码；你更像是调整数据集，然后运行优化器来创建神经网络的参数。当时，神经网络被视为另一种分类器，比如决策树，但我认为这个框架更合适。现在我们在软件 2.0 领域有了一个相当于 GitHub 的东西，我认为 Hugging Face 基本上就是那个等价物。还有 Model Atlas，你可以可视化那里所有的代码。顺便说一句，中间那个大圆是 Flux（图像生成器）的参数。每当有人在 Flux 模型上进行微调，你就在这个空间里创建了一个 git 提交，并创建了一种不同的图像生成器。所以基本上，软件 1.0 是编程计算机的计算机代码。软件 2.0 是编程神经网络的权重。这是 AlexNet 的例子，一个图像识别神经网络。直到最近，我们熟悉的所有神经网络都是固定功能的计算机——图像到类别等。已经改变的是，我认为这是一个根本性的变化，神经网络通过大型语言模型变得可编程。我认为这是相当新颖和独特的——一种新型计算机。所以在我看来，值得给它一个新的名称：软件 3.0。基本上，你的提示词现在就是编程 LLM 的程序。值得注意的是，这些提示词是用英语写的，所以这是一种非常有趣的编程语言。

Let's take a look at the realm of software. If we think of this as the map of software, this is a cool tool called Map of GitHub. This is all the software that's written—instructions to the computer for carrying out tasks in the digital space. If you zoom in, these are all different kinds of repositories and all the code that has been written. A few years ago, I observed that software was changing and there was a new type of software around, which I called Software 2.0. The idea is that Software 1.0 is the code you write for the computer. Software 2.0 are basically neural networks, in particular the weights of a neural network. You're not writing this code directly; you're more like tuning the datasets and then running an optimizer to create the parameters of the neural net. At the time, neural nets were seen as just a different kind of classifier, like a decision tree, but I think this framing was much more appropriate. Now we have an equivalent of GitHub in the realm of Software 2.0, and I think Hugging Face is basically that equivalent. There's also Model Atlas, and you can visualize all the code written there. By the way, the giant circle in the middle is the parameters of Flux, the image generator. Anytime someone tunes on top of a Flux model, you create a git commit in this space and you create a different kind of image generator. So basically, Software 1.0 is computer code that programs a computer. Software 2.0 are the weights that program neural networks. Here's an example of AlexNet, an image recognizer neural network. Until recently, all neural networks we were familiar with were kind of fixed-function computers—image to categories, etc. What has changed, and I think is a fundamental change, is that neural networks became programmable with large language models. I see this as quite new and unique—a new kind of computer. So in my mind, it's worth giving it a new designation: Software 3.0. Basically, your prompts are now programs that program the LLM. Remarkably, these prompts are written in English, so it's a very interesting programming language.

总结一下区别：例如，如果你做情感分类，你可以想象写一些 Python 来做情感分类，或者你可以训练一个神经网络，或者你可以提示一个大型语言模型。这里有一个简短的提示词，你可以想象改变它，以稍微不同的方式编程计算机。所以我们有软件 1.0、软件 2.0，我认为我们看到很多 GitHub 代码不再只是代码；有很多英语夹杂在代码中。所以有一种不断增长的新型代码类别。这不仅是一种新的编程范式，而且值得注意的是，它是用我们的母语英语。几年前，这让我震惊，我发了这条推文，它引起了很多人的关注。这是我目前置顶的推文：‘最热门的编程语言是英语。’

To summarize the difference: if you're doing sentiment classification, for example, you can imagine writing some Python to do sentiment classification, or you can train a neural net, or you can prompt a large language model. Here's a short prompt, and you can imagine changing it and programming the computer in a slightly different way. So we have Software 1.0, Software 2.0, and I think we're seeing a lot of GitHub code that is not just code anymore; there's a bunch of English interspersed with code. So there's a growing category of new kind of code. Not only is it a new programming paradigm, it's also remarkable that it's in our native language of English. When this blew my mind a few years ago, I tweeted this, and it captured the attention of a lot of people. This is my currently pinned tweet: 'The hottest new programming language is English.'

当我在特斯拉时，我们正在研究自动驾驶，试图让汽车驾驶。我当时展示了这张幻灯片：汽车输入在底部，通过一个软件栈产生转向和加速。我观察到自动驾驶中有大量的 C++代码——软件 1.0 代码——和一些做图像识别的神经网络。随着时间的推移，随着我们改进自动驾驶，神经网络的能力和规模都在增长，而所有的 C++代码都被删除了。许多最初用 1.0 编写的功能被迁移到了 2.0。例如，跨不同摄像头和时间的图像信息拼接是由神经网络完成的，我们能够删除大量代码。所以软件 2.0 栈实际上吞噬了自动驾驶的软件栈。我当时认为这很了不起，我认为我们再次看到同样的事情：一种新型软件正在吞噬整个栈。我们有三种完全不同的编程范式，如果你要进入这个行业，最好能熟练掌握所有三种，因为它们各有优缺点。你可能想用 1.0、2.0 或 3.0 来编程某些功能。你是要训练神经网络？还是仅仅提示 LLM？这应该是一段显式代码吗？我们都必须做出这些决定，并可能在这些范式之间流畅地转换。

When I was at Tesla, we were working on the autopilot, trying to get the car to drive. I showed this slide at the time: the inputs to the car are on the bottom, going through a software stack to produce steering and acceleration. I observed that there was a ton of C++ code in the autopilot—Software 1.0 code—and some neural nets doing image recognition. Over time, as we made the autopilot better, the neural network grew in capability and size, and all the C++ code was being deleted. A lot of the functionality originally written in 1.0 was migrated to 2.0. For example, stitching up information across images from different cameras and across time was done by a neural network, and we were able to delete a lot of code. So the Software 2.0 stack literally ate through the software stack of the autopilot. I thought this was remarkable, and I think we're seeing the same thing again: a new kind of software eating through the stack. We have three completely different programming paradigms, and if you're entering the industry, it's a very good idea to be fluent in all of them because they all have pros and cons. You may want to program some functionality in 1.0, 2.0, or 3.0. Are you going to train a neural net? Are you going to just prompt an LLM? Should this be a piece of explicit code? We all have to make these decisions and potentially fluidly transition between these paradigms.

我现在想进入的是，首先在第一部分，谈谈 LLM 以及如何看待这个新范式和生态系统——这台新计算机是什么样子，生态系统是什么样子。实际上，很多年前，吴恩达的这句话让我印象深刻。我想安德鲁接下来会发言。

What I wanted to get into now is, first, in the first part, talk about LLMs and how to think of this new paradigm and the ecosystem—what this new computer looks like and what the ecosystem looks like. I was struck by this quote from Andrew Ng actually, many years ago. I think Andrew is going to be speaking right after me.

但他当时说 AI 是新的电力，我觉得这确实捕捉到了一些有趣的东西，因为 LLM 现在确实有公用事业的性质。像 OpenAI、Gemini、Anthropic 这些 LLM 实验室，他们投入资本支出训练 LLM，这相当于建设电网；然后通过 API 以运营成本的方式为我们提供智能服务，按量计费，比如每百万 token 付费。我们对 API 有很多类似公用事业的需求：低延迟、高可用、稳定的质量等等。在电力领域，有转换开关，可以在电网、太阳能、电池或发电机之间切换电源。在 LLM 领域，我们有 OpenRouter 这样的工具，可以轻松切换不同的 LLM。因为 LLM 是软件，不占用物理空间，所以同时存在六家电力供应商也没问题，你可以随意切换，因为它们不是直接竞争。还有一点很有趣，就在前几天，很多 LLM 宕机了，人们无法工作。我觉得当最先进的 LLM 宕机时，就像全球发生了智能停电。就像电网电压不稳一样，我们对这些模型依赖越深，世界就越笨，这已经很戏剧性了，而且我认为还会继续加剧。

Uh but he said at the time AI is the new electricity and I do think that it kind of captures something very interesting in that LLMs certainly feel like they have properties of utilities right now. So LLM labs like OpenAI, Gemini, Anthropic etc. They spend capex to train the LLMs and this is kind of equivalent to building out a grid and then there's opex to serve that intelligence over APIs to all of us and this is done through metered access where we pay per million tokens or something like that and we have a lot of demands that are very utility-like demands out of this API we demand low latency high uptime consistent quality etc. In electricity, you would have a transfer switch. So you can transfer your electricity source from like grid and solar or battery or generator. In LLM, we have maybe open router and easily switch between the different types of LLMs that exist. Because the LLM are software, they don't compete for physical space. So it's okay to have basically like six electricity providers and you can switch between them, right? Because they don't compete in such a direct way. And I think what's also a little fascinating and we saw this in the last few days actually a lot of the LLMs went down and people were kind of like stuck and unable to work. And I think it's kind of fascinating to me that when the state-of-the-art LLMs go down, it's actually kind of like an intelligence brownout in the world. It's kind of like when the voltage is unreliable in the grid and the planet just gets dumber the more reliance we have on these models, which already is like really dramatic and I think will continue to grow.

但 LLM 不仅具有公用事业的性质，我认为它们也有晶圆厂的一些特性。因为构建 LLM 所需的资本支出非常大，不只是建个发电站那么简单。你投入巨额资金，而且技术树发展非常快。所以我们处在一个技术树很深、研发秘密集中在 LLM 实验室的世界里。但这个类比有点模糊，因为 LLM 是软件，软件的可防御性较弱，因为它太灵活了。所以这只是一个有趣的思考方向。你可以做很多类比，比如 4 纳米工艺节点可能相当于一个具有特定最大浮点运算能力的集群。当你使用 Nvidia GPU，只做软件不做硬件时，这类似于无晶圆厂模式。但如果你像 Google 那样自己构建硬件并在 TPU 上训练，那就类似于 Intel 拥有自己晶圆厂的模式。所以这些类比有一定道理。但我觉得最贴切的类比是，LLM 与操作系统非常相似。这不仅仅是电或水，不是从水龙头流出的商品。它们现在是越来越复杂的软件生态系统，所以不像电力那样简单。有趣的是，生态系统的形成方式非常相似：有几个闭源提供商，比如 Windows 或 Mac OS，然后有一个开源替代品 Linux。我认为 LLM 也是如此，有几个竞争的闭源提供商，而 Llama 生态系统可能类似于 Linux 的雏形。当然，现在还为时过早，因为 LLM 还很简单，但我们开始看到它们会变得复杂得多。不仅仅是 LLM 本身，还有工具使用、多模态以及它们如何协同工作。所以当我意识到这一点时，我试着勾勒出来，感觉 LLM 就像一种新的操作系统。LLM 是一种新型计算机，相当于 CPU，上下文窗口相当于内存，LLM 协调内存和计算来解决问题。从这个角度看，它确实很像操作系统。

But LLMs don't only have properties of utilities. I think it's also fair to say that they have some properties of fabs. And the reason for this is that the capex required for building LLM is actually quite large. It's not just like building some power station or something like that, right? You're investing a huge amount of money and I think the tech tree and for the technology is growing quite rapidly. So we're in a world where we have sort of deep tech trees, research and development secrets that are centralizing inside the LLM labs. But I think the analogy muddies a little bit also because as I mentioned this is software and software is a bit less defensible because it is so malleable. And so I think it's just an interesting kind of thing to think about potentially. There's many analogies you can make like a 4 nanometer process node maybe is something like a cluster with certain max flops. You can think about when you're using Nvidia GPUs and you're only doing the software and you're not doing the hardware. That's kind of like the fabless model. But if you're actually also building your own hardware and you're training on TPUs if you're Google, that's kind of like the Intel model where you own your fab. So I think there's some analogies here that make sense. But actually I think the analogy that makes the most sense perhaps is that in my mind LLMs have very strong kind of analogies to operating systems. In that this is not just electricity or water. It's not something that comes out of the tap as a commodity. These are now increasingly complex software ecosystems right so they're not just like simple commodities like electricity and it's kind of interesting to me that the ecosystem is shaping in a very similar kind of way where you have a few closed source providers like Windows or Mac OS and then you have an open source alternative like Linux and I think for LLMs as well we have a kind of a few competing closed source providers and then maybe the llama ecosystem is currently like maybe a close approximation to something that may grow into something like Linux. Again, I think it's still very early because these are just simple LLMs, but we're starting to see that these are going to get a lot more complicated. It's not just about the LLM itself. It's about all the tool use and the multimodalities and how all of that works. And so when I sort of had this realization a while back, I tried to sketch it out and it kind of seemed to me like LLMs are kind of like a new operating system, right? So the LLM is a new kind of a computer. It's kind of like the CPU equivalent. The context windows are kind of like the memory and then the LLM is orchestrating memory and compute for problem solving using all of these capabilities here and so definitely if you look at it looks very much like operating system from that perspective.

再举几个类比。比如，你想下载一个应用，比如 VS Code，你可以下载并在 Windows、Linux 或 Mac 上运行。同样，你可以拿一个 LLM 应用比如 Cursor，在 GPT、Claude 或 Gemini 系列上运行，只需下拉选择。所以这方面也很相似。另一个让我印象深刻的类比是，我们正处于类似 20 世纪 60 年代的时代，LLM 计算对于这种新型计算机来说仍然非常昂贵，这迫使 LLM 集中在云端，我们都只是通过网络与之交互的瘦客户端，没有人能充分利用这些计算机，因此使用分时技术是合理的，我们只是云端运行计算机时批次的一个维度。这正是当时计算机的样子：操作系统在云端，一切通过流式传输和批处理。个人计算革命尚未发生，因为还不经济。但有些人正在尝试。例如，Mac mini 非常适合某些 LLM，因为如果你做单次推理，这完全是内存受限的，所以实际上可行。我认为这些可能是个人计算的早期迹象，但尚未真正实现，还不清楚会是什么样子。也许你们中的一些人会发明它。还有一个类比：每当我通过文本直接与 ChatGPT 或某个 LLM 对话时，我感觉就像通过终端与操作系统对话。就是文本，直接访问操作系统。我认为 GUI 还没有以通用方式被发明出来，比如 ChatGPT 是否应该有一个不同于文本气泡的 GUI？当然，一些应用有 GUI，但没有跨所有任务的 GUI。LLM 在某些方面与操作系统和早期计算有显著不同。

A few more analogies. For example, if you want to download an app, say I go to VS Code and I go to download, you can download VS Code and you can run it on Windows, Linux or Mac in the same way as you can take an LLM app like Cursor and you can run it on GPT or Claude or Gemini series, right? It's just a drop down. So, it's kind of like similar in that way as well. More analogies that I think strike me is that we're kind of like in this 1960s era where LLM compute is still very expensive for this new kind of a computer and that forces the LLMs to be centralized in the cloud and we're all just sort of thin clients that interact with it over the network and none of us have full utilization of these computers and therefore it makes sense to use time sharing where we're all just a dimension of the batch when they're running the computer in the cloud. And this is very much what computers used to look like during this time. The operating systems were in the cloud. Everything was streamed around and there was batching. And so the personal computing revolution hasn't happened yet because it's just not economical. It doesn't make sense. But I think some people are trying. And it turns out that Mac minis, for example, are a very good fit for some of the LLMs because it's all if you're doing batch one inference, this is all super memory bound. So this actually works. And I think these are some early indications maybe of personal computing. But this hasn't really happened yet. It's not clear what this looks like. Maybe some of you get to invent what this is or how it works or what this should be. Maybe one more analogy that I'll mention is whenever I talk to ChatGPT or some LLM directly in text, I feel like I'm talking to an operating system through the terminal. Like it's just text. It's direct access to the operating system. And I think a GUI hasn't yet really been invented in a general way like should ChatGPT have a GUI different than just a text bubbles. Certainly some of the apps that we're going to go into in a bit have GUI but there's no like GUI across all the tasks if that makes sense. There are some ways in which LLMs are different from kind of operating systems in some fairly unique way and from early computing.

我写过一个让我觉得这次非常不同的特性：LLM 逆转了技术扩散的通常方向。比如电力、密码学、计算、飞行、互联网、GPS 等许多新兴变革技术，通常是政府和企业先使用，因为新且昂贵，后来才扩散到消费者。但我觉得 LLM 是反过来的。早期计算机用于弹道和军事，而 LLM 却用来教人煮鸡蛋之类的。这确实是我很多的使用场景。所以我觉得很神奇：我们有了一台神奇的新电脑，它却在帮我煮鸡蛋，而不是帮政府做疯狂的事，比如军事弹道或特殊技术。实际上，企业和政府在采用这些技术方面落后于我们所有人。所以这是反过来的，我认为这或许能指导我们如何使用这项技术，以及哪些是首批应用。

And I wrote about this one particular property that strikes me as very different this time around. It's that LLMs flip the direction of technology diffusion that is usually present in technology. So for example with electricity, cryptography, computing, flight, internet, GPS, lots of new transformative technologies that have not been around. Typically it is the government and corporations that are the first users because it's new and expensive etc., and it only later diffuses to consumer. But I feel like LLMs are kind of flipped around. So maybe with early computers, it was all about ballistics and military use, but with LLMs, it's all about how do you boil an egg or something like that. This is certainly like a lot of my use. And so it's really fascinating to me that we have a new magical computer and it's like helping me boil an egg. It's not helping the government do something really crazy like some military ballistics or some special technology. Indeed, corporations and governments are lagging behind the adoption of all of us, of all of these technologies. So it's just backwards and I think it informs maybe some of the uses of how we want to use this technology or like where are some of the first apps and so on.

所以总结一下，LLM 实验室、LLM——我认为用这个词是准确的——但 LLM 是复杂的操作系统。它们相当于计算领域的 1960 年代，我们正在重做计算。它们目前通过分时共享和像公用事业一样分发。全新且前所未有的是，它们不在少数政府和企业手中，而是在我们所有人手中，因为我们都有电脑，而且它只是软件，ChatGPT 像光束一样瞬间传到数十亿人的电脑上，这太疯狂了。我觉得这很不可思议，现在是我们进入行业、编程这些计算机的时候了。这太疯狂了。所以我认为这非常了不起。

So, in summary so far, LLM labs LLMs. I think it's accurate language to use, but LLMs are complicated operating systems. They're circa 1960s in computing and we're redoing computing all over again. And they're currently available via time sharing and distributed like a utility. What is new and unprecedented is that they're not in the hands of a few governments and corporations. They're in the hands of all of us because we all have a computer and it's all just software and ChatGPT was beamed down to our computers like billions of people like instantly and overnight and this is insane. And it's kind of insane to me that this is the case and now it is our time to enter the industry and program these computers. This is crazy. So I think this is quite remarkable.

在编程 LLM 之前，我们需要花点时间思考它们是什么。我特别喜欢讨论它们的心理学。我喜欢把 LLM 看作人的精神体。它们是人的随机模拟。这里的模拟器恰好是自回归 Transformer。Transformer 是一种神经网络，它在 token 级别上运行，一块一块一块地处理。每个块的计算量几乎相等。这个模拟器基本上就是一些权重，我们把它拟合到互联网上的所有文本上。最终你得到这样一个模拟器，因为它是在人类数据上训练的，所以它产生了类似人类的涌现心理学。

Before we program LLMs, we have to kind of like spend some time to think about what these things are. And I especially like to kind of talk about their psychology. So the way I like to think about LLMs is that they're kind of like people spirits. They are stochastic simulations of people. And the simulator in this case happens to be an autoregressive transformer. So transformer is a neural net. It just kind of goes on the level of tokens. It goes chunk chunk chunk chunk chunk. And there's an almost equal amount of compute for every single chunk. And this simulator of course is just basically there's some weights involved and we fit it to all of text that we have on the internet and so on. And you end up with this kind of a simulator and because it is trained on humans, it's got this emergent psychology that is humanlike.

首先你会注意到，LLM 拥有百科全书式的知识和记忆。它们能记住很多东西，比任何单个人类都多，因为它们读了太多东西。这让我想起电影《雨人》，我强烈推荐大家看。这是一部很棒的电影，我很喜欢。达斯汀·霍夫曼饰演一个自闭症学者，拥有近乎完美的记忆。他能读电话簿并记住所有名字和号码。我觉得 LLM 很像。它们能轻松记住 SHA 哈希和各种东西。所以它们在某些方面确实有超能力。但它们也有很多认知缺陷。它们会频繁产生幻觉，编造东西，缺乏良好的自我认知模型，至少不够好。这有所改善但还不完美。它们表现出锯齿状智能：在某些问题解决领域超人类，然后会犯人类基本不会犯的错误。比如坚持 9.11 大于 9.9，或者草莓里有 2 个 R。这些是著名例子，但基本上有很多粗糙边缘会让你绊倒。我认为这也很独特。它们还患有顺行性遗忘症。我的意思是，如果你有一个同事加入你的组织，他会逐渐了解组织，获得大量上下文，回家睡觉巩固知识，随时间积累专长。LLM 天生不会这样，这在 LLM 研发中尚未真正解决。所以上下文窗口就像工作记忆，你必须直接编程工作记忆，因为它们不会默认变聪明。很多人被这种类比误导。流行文化中我推荐大家看《记忆碎片》和《初恋 50 次》。这两部电影的主角权重固定，上下文窗口每天早上被清空，这给工作和人际关系带来很大问题，而 LLM 一直如此。

So the first thing you'll notice is of course LLMs have encyclopedic knowledge and memory. And they can remember lots of things, a lot more than any single individual human can because they read so many things. It actually kind of reminds me of this movie Rainman, which I actually really recommend people watch. It's an amazing movie. I love this movie. And Dustin Hoffman here is an autistic savant who has almost perfect memory. So he can read like a phone book and remember all of the names and phone numbers. And I kind of feel like LLMs are kind of very similar. They can remember SHA hashes and lots of different kinds of things very very easily. So they certainly have superpowers in some respects. But they also have a bunch of I would say cognitive deficits. So they hallucinate quite a bit. And they kind of make up stuff and don't have a very good internal model of self-knowledge, not sufficient at least. And this has gotten better but not perfect. They display jagged intelligence. So they're going to be superhuman in some problem-solving domains. And then they're going to make mistakes that basically no human will make. Like they will insist that 9.11 is greater than 9.9 or that there are two Rs in strawberry. These are some famous examples but basically there are rough edges that you can trip on. So that's kind of I think also kind of unique. They also kind of suffer from anterograde amnesia. So I think I'm alluding to the fact that if you have a coworker who joins your organization, this coworker will over time learn your organization and they will understand and gain like a huge amount of context on the organization and they go home and they sleep and they consolidate knowledge and they develop expertise over time. LLMs don't natively do this and this is not something that has really been solved in the R&D of LLM. I think so context windows are really kind of like working memory and you have to sort of program the working memory quite directly because they don't just get smarter by default. And I think a lot of people get tripped up by the analogies in this way. In popular culture I recommend people watch these two movies: Memento and 50 First Dates. In both of these movies, the protagonists, their weights are fixed and their context windows get wiped every single morning and it's really problematic to go to work or have relationships when this happens and this happens to LLMs all the time.

我还想指出一点，LLM 使用中与安全相关的限制。例如，LLM 很容易受骗，容易受到提示注入攻击，可能泄露你的数据等。还有很多其他安全考虑。所以长话短说，你必须同时思考这个超人类的东西，它有一堆认知缺陷和问题。但它们又极其有用，那么我们如何编程它们，如何绕过它们的缺陷，享受它们的超能力呢？

I guess one more thing I would point to is security kind of related limitations of the use of LLM. So for example, LLMs are quite gullible. They are susceptible to prompt injection risks. They might leak your data etc. And there are many other considerations security related. So basically long story short, you have to simultaneously think through this superhuman thing that has a bunch of cognitive deficits and issues. How do we and yet they are extremely useful and so how do we program them and how do we work around their deficits and enjoy their superhuman powers.

现在我想转向讨论如何使用这些模型的机会，以及一些最大的机会。这不是一个全面的列表，只是我觉得这次演讲有趣的一些东西。我首先兴奋的是我称之为部分自主应用的东西。例如，以编程为例。你当然可以直接去 ChatGPT，开始复制粘贴代码、复制 bug 报告等，获取代码再复制粘贴。为什么要那样做？为什么要直接去操作系统？有一个专门的应用更有意义。所以我想你们很多人都在用 Cursor。我也用。Cursor 就是你想要的东西。你不想直接去 ChatGPT。

So what I want to switch to now is talk about the opportunities of how do we use these models and what are some of the biggest opportunities. This is not a comprehensive list just some of the things that I thought were interesting for this talk. The first thing I'm kind of excited about is what I would call partial autonomy apps. So for example, let's work with the example of coding. You can certainly go to ChatGPT directly and you can start copy pasting code around and copying bug reports and stuff around and getting code and copy pasting everything around. Why would you do that? Why would you go directly to the operating system? It makes a lot more sense to have an app dedicated for this. And so I think many of you use Cursor. I do as well. And Cursor is kind of like the thing you want instead. You don't want to just directly go to ChatGPT.

我认为 Cursor 是一个很好的早期 LLM 应用例子，它具备一系列我认为对所有 LLM 应用都有用的特性。具体来说，你会注意到我们有一个传统的界面，允许人类像以前一样手动完成所有工作。但除此之外，我们现在有了 LLM 集成，允许我们以更大的块进行操作。所以我认为 LLM 应用有一些共通的、值得指出的特性：第一，LLM 基本上做了大量的上下文管理。第二，它们编排了对 LLM 的多次调用，对吧？以 Cursor 为例，底层有用于所有文件的嵌入模型、实际的聊天模型、应用代码差异的模型，这一切都为你编排好了。一个非常重要但可能未被充分认识的点是应用特定的 GUI 及其重要性。因为你不想直接通过文本与操作系统对话。文本很难阅读、解释和理解，而且你也不想以文本形式原生执行某些操作。所以，最好只是看到差异以红色和绿色变化显示，你可以看到添加和删除了什么。按 Command Y 接受或 Command N 拒绝要容易得多。我不应该用文本输入，对吧？所以 GUI 允许人类审计这些易错系统的工作，并加快速度。我稍后会再回到这一点。我想指出的最后一个特性是我所谓的自主性滑块。例如，在 Cursor 中，你可以只做 Tab 补全，你主要负责。你可以选择一段代码并按 Command K 只更改那段代码。你可以按 Command L 更改整个文件。或者你可以按 Command I，让它自由发挥，在整个仓库中做任何事，这就是完全自主的智能体版本。所以你控制着自主性滑块，根据手头任务的复杂性，你可以调整你愿意为那个任务放弃的自主程度。也许再举一个相当成功的 LLM 应用的例子，Perplexity 也具有与我在 Cursor 中指出的非常相似的功能。它打包了大量信息。它编排了多个 LLM。它有一个 GUI，允许你审计它的一些工作。例如，它会引用来源，你可以想象检查它们。它还有一个自主性滑块。你可以只做快速搜索，或者做研究，或者做深度研究，10 分钟后回来。所以这只是你交给工具的不同程度的自主性。

I think Cursor is a very good example of an early LLM app that has a bunch of properties that are useful across all LLM apps. In particular, you will notice that we have a traditional interface that allows a human to go in and do all the work manually just as before. But in addition to that, we now have this LLM integration that allows us to go in bigger chunks. So some of the properties of LLM apps that I think are shared and useful to point out: number one, the LLMs basically do a ton of the context management. Number two, they orchestrate multiple calls to LLMs, right? So in the case of Cursor, there's under the hood embedding models for all your files, the actual chat models, models that apply diffs to the code, and this is all orchestrated for you. A really big one that I think maybe is not fully appreciated always is application-specific GUI and the importance of it. Because you don't just want to talk to the operating system directly in text. Text is very hard to read, interpret, and understand, and also you don't want to take some of these actions natively in text. So it's much better to just see a diff as red and green changes, and you can see what's being added and subtracted. It's much easier to just do Command Y to accept or Command N to reject. I shouldn't have to type it in text, right? So a GUI allows a human to audit the work of these fallible systems and to go faster. I'm going to come back to this point a little bit later as well. And the last kind of feature I want to point out is that there's what I call the autonomy slider. So, for example, in Cursor, you can just do tab completion. You're mostly in charge. You can select a chunk of code and Command K to change just that chunk of code. You can do Command L to change the entire file. Or you can do Command I which just lets it rip, do whatever you want in the entire repo, and that's the sort of full autonomy agentic version. So you are in charge of the autonomy slider, and depending on the complexity of the task at hand, you can tune the amount of autonomy that you are willing to give up for that task. Maybe to show one more example of a fairly successful LLM app, Perplexity also has very similar features to what I've just pointed out in Cursor. It packages up a lot of the information. It orchestrates multiple LLMs. It's got a GUI that allows you to audit some of its work. So, for example, it will cite sources and you can imagine inspecting them. And it's got an autonomy slider. You can either just do a quick search or you can do research or you can do deep research and come back 10 minutes later. So this is all just varying levels of autonomy that you give up to the tool.

所以我的问题是：我觉得很多软件会变得部分自主。我在想这看起来是什么样子。对于你们中许多维护产品和服务的人来说，你们将如何使你们的产品和服务部分自主？LLM 能看到人类能看到的一切吗？LLM 能以人类能做的所有方式行动吗？人类能否监督并保持在循环中？因为再次强调，这些是易错系统，还不完美。在 Photoshop 或类似的东西中，差异看起来是什么样子？你知道，现在很多传统软件有所有这些开关之类的东西，都是为人类设计的。所有这些都必须改变，变得对 LLM 可访问。

So I guess my question is: I feel like a lot of software will become partially autonomous. I'm trying to think through what that looks like. And for many of you who maintain products and services, how are you going to make your products and services partially autonomous? Can an LLM see everything that a human can see? Can an LLM act in all the ways that a human could act? And can humans supervise and stay in the loop of this activity? Because again, these are fallible systems that aren't yet perfect. And what does a diff look like in Photoshop or something like that? You know, and also a lot of the traditional software right now, it has all these switches and all this kind of stuff that's all designed for humans. All of this has to change and become accessible to LLMs.

所以我想强调一点，很多 LLM 应用可能没有得到应有的关注，那就是我们现在与 AI 合作，通常它们负责生成，而我们人类负责验证。让这个循环尽可能快符合我们的利益。这样我们就能完成大量工作。我认为有两种主要方式可以实现这一点。第一，你可以大大加快验证速度。我认为 GUI 对此极其重要，因为 GUI 利用了你大脑中的计算机视觉 GPU。阅读文本很费力且无趣，但看东西很有趣，而且是通往大脑的高速公路。所以我认为 GUI 对于审计系统和视觉表示非常有用。第二，我认为我们必须让 AI 保持牵制。我觉得很多人在 AI 智能体上过于兴奋了，对我来说，得到一个包含一万行代码差异的仓库没有用。我仍然是瓶颈，对吧？即使那一万行代码瞬间生成，我也必须确保这个东西没有引入错误，它在做正确的事情，并且没有安全问题等等。所以我认为，基本上，让这两者的流动非常非常快符合我们的利益，我们必须以某种方式让 AI 保持牵制，因为它反应过度。

So one thing I want to stress with a lot of these LLM apps that I'm not sure gets as much attention as it should is we're now kind of cooperating with AIs, and usually they are doing the generation and we as humans are doing the verification. It is in our interest to make this loop go as fast as possible. So we're getting a lot of work done. There are two major ways that I think this can be done. Number one, you can speed up verification a lot. And I think GUIs, for example, are extremely important to this because a GUI utilizes your computer vision GPU in all of our head. Reading text is effortful and it's not fun, but looking at stuff is fun and it's a kind of highway to your brain. So I think GUIs are very useful for auditing systems and visual representations in general. And number two, I would say is we have to keep the AI on the leash. I think a lot of people are getting way over excited with AI agents, and it's not useful to me to get a diff of 10,000 lines of code to my repo. Like I have to, I'm still the bottleneck, right? Even though that 10,000 lines come out instantly, I have to make sure that this thing is not introducing bugs, that it's doing the correct thing, and that there are no security issues and so on. So I think that basically, it's in our interest to make the flow of these two go very very fast, and we have to somehow keep the AI on the leash because it gets way too overreactive.

这张幻灯片不太好。抱歉，但我想我正在尝试，像你们许多人一样，找到一些在我的编码工作流程中利用这些智能体的方法，进行 AI 辅助编码。在我自己的工作中，我总是害怕得到太大的差异。我总是以小的增量块进行。我想确保一切都好。我想让这个循环非常快地旋转，我处理的是单个具体事物的小块。所以我认为你们许多人可能也在开发类似的与 LLM 合作的方式。我还看到了一些博客文章，试图开发这些与 LLM 合作的最佳实践。这里有一篇我最近读到的，我觉得相当不错。它讨论了一些技巧，其中一些与如何让 AI 保持牵制有关。例如，如果你在提示，如果你的提示模糊，那么 AI 可能不会完全按照你的意愿行事，在这种情况下，验证会失败。你会要求别的东西。如果验证失败，那么你就会开始旋转。所以花更多时间让你的提示更具体更有意义，这增加了成功验证的概率，你可以继续前进。所以我认为我们很多人最终会找到类似这样的技巧。我认为在我自己的工作中，我也对教育在 AI 和 LLM 下的样子感兴趣。教育会是什么样子？我认为我大量的思考在于如何让 AI 保持牵制。

This slide is not very good. I'm sorry, but I guess I'm trying to develop, like many of you, some ways of utilizing these agents in my coding workflow and to do AI-assisted coding. In my own work, I'm always scared to get way too big diffs. I always go in small incremental chunks. I want to make sure that everything is good. I want to spin this loop very very fast, and I sort of work on small chunks of a single concrete thing. So I think many of you probably are developing similar ways of working with LLMs. I also saw a number of blog posts that try to develop these best practices for working with LLMs. And here's one that I read recently and I thought was quite good. It discussed some techniques, and some of them have to do with how you keep the AI on the leash. So, as an example, if you are prompting, if your prompt is vague, then the AI might not do exactly what you wanted, and in that case, verification will fail. You're going to ask for something else. If a verification fails, then you're going to start spinning. So it makes a lot more sense to spend a bit more time to be more concrete in your prompts, which increases the probability of successful verification and you can move forward. So I think a lot of us are going to end up finding kind of techniques like this. I think in my own work as well, I'm currently interested in what education looks like together with AI and LLMs. What does education look like? And I think a large amount of thought for me goes into how we keep AI on the leash.

我不认为直接去聊天说‘嘿，教我物理’就能奏效。我认为这行不通，因为 AI 会迷失方向。所以对我来说，这实际上是两个独立的应用程序。例如，有一个给老师用的应用来创建课程，还有一个应用把课程分发给学生。在这两种情况下，我们现在有了一个中间产物——课程，它是可审计的，我们可以确保它质量好、一致。AI 被约束在特定的教学大纲、项目进度等范围内。所以这是约束 AI 的一种方式，我认为成功的可能性大得多，AI 也不会迷失方向。

I don't think it just works to go to chat and be like, 'Hey, teach me physics.' I don't think this works because the AI gets lost in the woods. So for me, this is actually two separate apps. For example, there's an app for a teacher that creates courses and then there's an app that takes courses and serves them to students. In both cases, we now have this intermediate artifact of a course that is auditable and we can make sure it's good. We can make sure it's consistent. And the AI is kept on the leash with respect to a certain syllabus, a certain progression of projects and so on. So this is one way of keeping the AI on leash and I think has a much higher likelihood of working and the AI is not getting lost in the woods.

我想提到的另一个类比是，我对部分自主并不陌生。我在特斯拉工作了五年，这也是一个部分自主的产品，共享很多特性。例如，仪表盘上有自动驾驶的图形界面，显示神经网络看到了什么。我们有自主滑块，在我任职期间，我们为用户做了越来越多的自主任务。我第一次驾驶自动驾驶汽车是在 2013 年。我有一个在 Waymo 工作的朋友，他邀请我开车兜风。我用谷歌眼镜拍了这张照片。你们很多人可能太年轻，不知道那是什么，但当时它风靡一时。我们上了车，在帕洛阿尔托的高速公路和街道上开了大约 30 分钟。这次驾驶完美无缺，零干预，那是 2013 年，12 年前了。这让我很震惊，因为我觉得自动驾驶即将到来，因为它就这么成功了。但 12 年后的今天，我们仍在研究自主性。我们仍在研究驾驶智能体，甚至现在还没有真正解决这个问题。你可能看到 Waymo 的车辆在行驶，看起来无人驾驶，但仍然有很多远程操作和人在回路中。所以我们还没有宣布成功，但我认为它最终会成功，只是花了很长时间。软件真的很棘手，就像驾驶一样棘手。所以当我看到‘2025 年是智能体之年’这样的说法时，我非常担心。我觉得这是智能体的十年，还需要相当长的时间。我们需要人在回路中。我们需要谨慎行事。这是软件。让我们认真对待。

One more analogy I wanted to allude to is I'm no stranger to partial autonomy. I worked on this for five years at Tesla, and this is also a partial autonomy product and shares a lot of features. For example, in the instrument panel there is the GUI of the autopilot showing what the neural network sees. We have the autonomy slider where over my tenure we did more and more autonomous tasks for the user. The first time I drove a self-driving vehicle was in 2013. I had a friend who worked at Waymo and he offered to give me a drive around Palo Alto. I took this picture using Google Glass at the time. Many of you are so young you might not even know what that is, but it was all the rage. We got into this car and went for about a 30-minute drive around Palo Alto highways and streets. This drive was perfect, zero interventions, and this was 2013, 12 years ago. It struck me because I felt like self-driving was imminent because it just worked. But here we are 12 years later and we are still working on autonomy. We are still working on driving agents and even now we haven't really solved the problem. You may see Waymos going around looking driverless, but there's still a lot of teleoperation and human in the loop. So we still haven't declared success, but I think it's going to succeed at this point, but it just took a long time. Software is really tricky, in the same way that driving is tricky. So when I see things like '2025 is the year of agents,' I get very concerned. I feel like this is the decade of agents and it's going to be quite some time. We need humans in the loop. We need to do this carefully. This is software. Let's be serious here.

我经常思考的另一个类比是钢铁侠战衣。我一直很喜欢钢铁侠。我认为它在很多方面都正确地反映了技术及其发展方式。我喜欢钢铁侠战衣的地方在于，它既是一种增强，托尼·斯塔克可以驾驶它，同时它也是一个智能体。在一些电影中，钢铁侠战衣相当自主，可以飞来飞去找到托尼。所以这就是自主滑块：我们可以构建增强或智能体，我们两者都想要一点。但在现阶段，处理容易出错的 LLM 时，我认为你要构建的更像钢铁侠战衣，而不是钢铁侠机器人。少一些花哨的自主智能体演示，多一些部分自主的产品。这些产品有定制的图形界面和用户体验。我们试图让人类的生成-验证循环非常快。但我们没有忘记，原则上自动化这项工作是有可能的。你的产品中应该有一个自主滑块。你应该思考如何滑动这个滑块，让产品随着时间的推移变得更加自主。我认为这类产品有很多机会。

One more analogy I always think through is the Iron Man suit. I always love Iron Man. I think it's so correct in a bunch of ways with respect to technology and how it will play out. What I love about the Iron Man suit is that it's both an augmentation and Tony Stark can drive it, and it's also an agent. In some movies, the Iron Man suit is quite autonomous and can fly around and find Tony. So this is the autonomy slider: we can build augmentations or we can build agents, and we kind of want to do a bit of both. But at this stage, working with fallible LLMs, I would say it's less Iron Man robots and more Iron Man suits that you want to build. It's less building flashy demos of autonomous agents and more building partial autonomy products. These products have custom GUIs and UI/UX. We are trying to make the generation-verification loop of the human very fast. But we are not losing sight of the fact that it is in principle possible to automate this work. There should be an autonomy slider in your product. You should be thinking about how you can slide that autonomy slider and make your product more autonomous over time. This is how I think there are lots of opportunities in these kinds of products.

我想换个话题，谈谈另一个非常独特的维度。不仅有一种新的编程语言允许软件中的自主性，而且正如我提到的，它是用英语编程的，这是一种自然界面，突然之间每个人都是程序员，因为每个人都会说英语这样的自然语言。这非常乐观，对我来说非常有趣，也完全前所未有。过去，你需要花五到十年学习某样东西才能在软件领域有所作为。现在不再是了。我不知道是否有人听说过‘氛围编程’。就是这条推文引入的，但我听说现在它成了一个热门梗。有趣的是：我在推特上已经 15 年了，但我仍然不知道哪条推文会走红。我以为这条推文会是后者，但它却成了一个梗。我想它引起了共鸣，给每个人都在感受的东西起了个名字。现在还有了维基百科页面。HuggingFace 的 Tom Wolf 分享了一个很棒的视频，孩子们在氛围编程。我觉得这太美好了。你怎么能看着这个视频对未来感到悲观呢？未来是美好的。我认为这最终会成为软件开发的门户。我对这一代的未来并不悲观。我喜欢这个视频。我自己也尝试了氛围编程，因为它太有趣了。当你想构建一个超级定制、不存在的东西，而且只想在周六即兴发挥时，氛围编程很棒。我构建了一个 iOS 应用。我实际上不会用 Swift 编程，但我很震惊自己能构建一个超级基础的应用。我不打算解释它，它很蠢，但我喜欢它只是一天的工作，当天晚些时候就在我的手机上运行了。我心想：‘哇，这太神奇了。’我不需要花五天时间阅读 Swift 就能开始。

I want to switch gears and talk about another dimension that is very unique. Not only is there a new type of programming language that allows for autonomy in software, but also as I mentioned it's programmed in English, which is this natural interface, and suddenly everyone is a programmer because everyone speaks natural language like English. This is extremely bullish and very interesting to me, and also completely unprecedented. It used to be the case that you needed to spend five to ten years studying something to be able to do something in software. This is not the case anymore. I don't know if anyone has heard of vibe coding. This is the tweet that introduced it, but I'm told it's now a major meme. Fun story: I've been on Twitter for like 15 years and I still have no clue which tweet will go viral. I thought this tweet would be the latter, but it became a total meme. I guess it struck a chord and gave a name to something everyone was feeling. Now there's a Wikipedia page. Tom Wolf from HuggingFace shared a beautiful video of kids vibe coding. I find it so wholesome. How can you look at this video and feel bad about the future? The future is great. I think this will end up being a gateway drug to software development. I'm not a doomer about the future of the generation. I love this video. I tried vibe coding myself because it's so fun. Vibe coding is great when you want to build something super custom that doesn't exist and you just want to wing it on a Saturday. I built this iOS app. I can't actually program in Swift, but I was shocked that I was able to build a super basic app. I'm not going to explain it, it's dumb, but I liked that it was just a day of work and it was running on my phone later that day. I was like, 'Wow, this is amazing.' I didn't have to read through Swift for five days to get started.

我还用 vibe-coding 做了一个叫 Menu Genen 的应用。这个已经上线了，你可以在 menu.app 上试试。我有个问题：去餐厅看菜单，完全不知道那些菜是什么，我需要图片。但之前没有这样的东西。所以我就想，‘嘿，我要用 vibe-coding 做一个。’这就是它的样子。你打开 menu.app，拍一张菜单照片，它就会生成图片。注册时每人免费获得 5 美元积分。所以这成了我生活中的一个主要成本中心。目前对我来说这是个负收入的应用，我在 Menu 上亏了很多钱。

I also vibe-coded this app called Menu Genen. And this is live. You can try it at menu.app. I had this problem where I show up at a restaurant, read through the menu, and have no idea what anything is. I need pictures. So this didn't exist. I was like, 'Hey, I'm going to vibe-code it.' So this is what it looks like. You go to menu.app, take a picture of a menu, and then it generates images. Everyone gets $5 in credits for free when you sign up. Therefore, this is a major cost center in my life. So this is a negative revenue app for me right now. I've lost a huge amount of money on Menu.

关于 Menu Genen，最有趣的是代码部分，也就是 vibe-coding 部分，其实是最简单的。大部分工作是在我试图让它真正可用的时候，比如添加身份验证、支付、域名和整体部署。这非常困难，而且所有这些都不是代码。这些 DevOps 的事情都是我在浏览器里点击各种东西，极其缓慢，又花了一周时间。所以有趣的是，我几个小时就在笔记本电脑上做出了 Menu Genen 的演示，但让它真正可用却花了一周，因为这部分实在太烦人了。

The fascinating thing about Menu Genen for me is that the code, the vibe-coding part, was actually the easy part. Most of it was when I tried to make it real, so that you can actually have authentication, payments, a domain name, and overall deployment. This was really hard, and all of this was not code. All this DevOps stuff was me in the browser clicking stuff, and it was extremely slow and took another week. So it was fascinating that I had the Menu Genen demo working on my laptop in a few hours, and then it took me a week because I was trying to make it real. The reason is this was just really annoying.

比如，如果你想给网页添加 Google 登录，我知道这很小，但 Clerk 库会给出大量指令告诉我如何集成。这太疯狂了。它告诉我去这个 URL，点击这个下拉菜单，选择这个，去那里，点击那个。它告诉我该做什么，就像电脑在告诉我应该采取什么行动。就好像在说‘你来操作’。为什么是我来做？搞什么鬼？我不得不遵循所有这些指令。这太疯狂了。

For example, if you try to add Google login to your web page, I know this is very small, but just a huge amount of instructions from this Clerk library telling me how to integrate this. It's crazy. It tells me to go to this URL, click on this dropdown, choose this, go to this, and click on that. It's telling me what to do, like a computer is telling me the actions I should be taking. Like, you do it. Why am I doing this? What the hell? I had to follow all these instructions. This was crazy.

所以我认为我演讲的最后一部分聚焦于：我们能不能直接为智能体构建？我不想做这些工作。智能体能做吗？谢谢。大致来说，我认为出现了一类新的数字信息消费者和操纵者。过去只有通过 GUI 的人类或通过 API 的计算机。现在我们有了全新的东西：智能体。它们是计算机，但像人类一样，对吧？它们是互联网上的人的灵魂，它们需要与我们的软件基础设施交互。我们能为他们构建吗？这是一个新事物。

So I think the last part of my talk focuses on: can we just build for agents? I don't want to do this work. Can agents do this? Thank you. Roughly speaking, I think there's a new category of consumer and manipulator of digital information. It used to be just humans through GUIs or computers through APIs. Now we have a completely new thing: agents. They are computers, but they are human-like, right? They are people spirits on the internet, and they need to interact with our software infrastructure. Can we build for them? It's a new thing.

举个例子，你可以在你的域名上放一个 robots.txt 来指导网络爬虫的行为。同样，你可以放一个 lm.txt 文件，它只是简单的 markdown，告诉 LLM 这个域名是关于什么的。这对 LLM 来说非常易读。如果它不得不获取你网页的 HTML 并尝试解析，那会非常容易出错且困难，会搞砸而且行不通。所以我们可以直接对 LLM 说话。这很值得。

As an example, you can have robots.txt on your domain to instruct web crawlers how to behave. In the same way, you can have an lm.txt file, which is just simple markdown telling LLMs what this domain is about. This is very readable to an LLM. If it had to instead get the HTML of your web page and try to parse it, that is very error-prone and difficult, and will screw it up and not work. So we can just directly speak to the LLM. It's worth it.

目前大量的文档是为人类编写的。你会看到列表、粗体和图片，这些 LLM 无法直接访问。所以我看到一些服务现在正在将大量文档转为专门面向 LLM 的格式。例如 Vercel 和 Stripe 是早期的先行者，但我还看到了一些其他公司。它们以 markdown 形式提供文档。Markdown 对 LLM 来说非常容易理解。这很棒。

A huge amount of documentation is currently written for people. You will see things like lists, bold, and pictures, which are not directly accessible by an LLM. So I see some services now transitioning a lot of their docs to be specifically for LLMs. Vercel and Stripe, for example, are early movers here, but there are a few more I've seen already. They offer their documentation in markdown. Markdown is super easy for LLMs to understand. This is great.

也许从我经验中举个简单的例子。有些人知道 3Blue1Brown，他在 YouTube 上制作精美的动画视频。他写了一个叫 Manim 的库。我想自己做动画，而且有大量关于如何使用 Manim 的文档。我不想真的通读一遍，所以我把整个文档复制粘贴给一个 LLM，描述了我想要的东西。它直接就能用。LLM 直接用 vibe-coding 给我做了一个完全符合我要求的动画。我当时想，‘哇，这太棒了。’所以如果我们能让文档对 LLM 易读，那将释放巨大的用途，我认为这很棒，应该更多发生。

Maybe one simple example from my experience. Some of you know 3Blue1Brown. He makes beautiful animation videos on YouTube. He wrote a library called Manim. I wanted to make my own, and there's extensive documentation on how to use Manim. I didn't want to actually read through it, so I copy-pasted the whole thing to an LLM and described what I wanted. It just worked out of the box. The LLM just vibe-coded me an animation exactly what I wanted. I was like, 'Wow, this is amazing.' So if we can make docs legible to LLMs, it's going to unlock a huge amount of use, and I think this is wonderful and should happen more.

我想指出的另一件事是，不幸的是，你必须改变文档。不仅仅是把你的文档变成 markdown 格式。那是容易的部分。我们实际上必须改变文档，因为只要你的文档说‘点击这个’，那就不行。LLM 目前无法原生执行这个操作。所以例如 Vercel 正在将每个‘点击’替换为等价的 curl 命令，你的 LLM 智能体可以代表你执行。我认为这非常有趣。

The other thing I wanted to point out is that you do unfortunately have to change the docs. It's not just about taking your docs and making them appear in markdown. That's the easy part. We actually have to change the docs because anytime your docs say 'click this', it's bad. An LLM will not be able to natively take this action right now. So Vercel, for example, is replacing every occurrence of 'click' with an equivalent curl command that your LLM agent could take on your behalf. I think this is very interesting.

当然，还有 Anthropic 的模型上下文协议。这是另一种方式，一种直接与智能体对话的协议，将智能体视为这种新的数字信息消费者和操纵者。所以我对这些想法非常看好。

And then, of course, there's the Model Context Protocol from Anthropic. This is another way, a protocol of speaking directly to agents as this new consumer and manipulator of digital information. So I'm very bullish on these ideas.

我真正喜欢的另一件事是许多小工具，它们帮助以非常 LLM 友好的格式摄取数据。例如，当我访问像我的 nanoGPT 这样的 GitHub 仓库时，我无法把它喂给 LLM 并提问，因为那是 GitHub 上的人类界面。所以当你把 URL 从 'github' 改成 'getingest'，它实际上会把所有文件连接成一个巨大的文本，并创建目录结构等。这就可以直接复制粘贴到你喜欢的 LLM 中，然后做各种事情。

The other thing I really like is a number of little tools here and there that are helping ingest data in very LLM-friendly formats. For example, when I go to a GitHub repo like my nanoGPT repo, I can't feed this to an LLM and ask questions about it because it's a human interface on GitHub. So when you just change the URL from 'github' to 'getingest', it will actually concatenate all the files into a single giant text and create a directory structure, etc. This is ready to be copy-pasted into your favorite LLM and you can do stuff.

也许更极端的例子是 DeepWiki，它不仅仅是文件的原始内容。这是来自 Devon 的，但他们让 Devon 对 GitHub 仓库进行分析，并为你的仓库构建一整套文档页面。你可以想象，这复制粘贴到你的 LLM 中更有帮助。所以我喜欢所有这些小工具，你只需改变 URL，就能让某些东西对 LLM 可访问。

Maybe even more dramatic example is DeepWiki, where it's not just the raw content of the files. This is from Devon, but they have Devon basically do analysis of the GitHub repo and build up a whole docs page just for your repo. You can imagine that this is even more helpful to copy-paste into your LLM. So I love all the little tools where you just change the URL and it makes something accessible to an LLM.

所以这一切都很好，我认为应该有更多这样的东西。我还想提一点，未来——甚至不是未来，就是今天——LLM 绝对能够四处点击东西等等。但我仍然认为非常值得与 LLM 各退一步，让它们更容易访问所有这些信息，因为使用起来仍然相当昂贵且困难得多。所以我确实认为很多软件会有一个长尾，它们不会适应，因为它们不像活跃的仓库或数字基础设施，我们将需要这些工具。但我认为对于其他所有人来说，非常值得在某个中间点相遇。所以我对两者都看好，如果这说得通的话。

So this is all well and great, and I think there should be a lot more of it. One more note I wanted to make is that it is absolutely possible that in the future—this is not even future, this is today—LLMs will be able to go around and click stuff and so on. But I still think it's very worth meeting LLMs halfway, making it easier for them to access all this information, because this is still fairly expensive to use and a lot more difficult. So I do think that lots of software, there will be a long tail where it won't adapt, because these are not like live player repositories or digital infrastructure, and we will need these tools. But I think for everyone else, it's very worth meeting in some middle point. So I'm bullish on both, if that makes sense.

所以总结一下，现在进入这个行业真是绝佳的时机。我们需要重写大量代码。大量代码将由专业人士和程序员编写。这些大语言模型有点像公用事业，有点像晶圆厂，但尤其像操作系统。不过现在还为时过早，就像操作系统的 1960 年代，我认为很多类比是相通的。这些大语言模型有点像这些会犯错的人的精神，我们必须学会与之共事。为了做到这一点，我们需要调整我们的基础设施。所以当你构建这些大语言模型应用时，我描述了一些有效使用这些大语言模型的方法，以及一些使之成为可能的工具，以及如何非常快速地旋转这个循环，基本上创建部分隧道产品。然后，是的，很多代码也需要更直接地为智能体编写。但无论如何，回到钢铁侠套装的类比，我认为在未来大约十年里，我们将把滑块从左移到右。我非常感兴趣。看看那会是什么样子将会非常有趣。我迫不及待地想和你们一起构建它。谢谢。

So in summary, what an amazing time to get into the industry. We need to rewrite a ton of code. A ton of code will be written by professionals and by coders. These LLMs are kind of like utilities, kind of like fabs, but they're kind of especially like operating systems. But it's so early. It's like the 1960s of operating systems, and I think a lot of the analogies cross over. And these LLMs are kind of like these fallible, you know, people spirits that we have to learn to work with. And in order to do that properly, we need to adjust our infrastructure towards it. So when you're building these LLM apps, I describe some of the ways of working effectively with these LLMs and some of the tools that make that kind of possible and how you can spin this loop very, very quickly and basically create partial tunneling products. And then, yeah, a lot of code has to also be written for the agents more directly. But in any case, going back to the Iron Man suit analogy, I think what we'll see over the next decade roughly is we're going to take the slider from left to right. And I'm very interested. It's going to be very interesting to see what that looks like. And I can't wait to build it with all of you. Thank you.