Vibe Coding: The End of iPhone Dominance?
打开互动全文版(中英对照 + 朗读 + 问答)→Naval 讨论 AI 编程代理如何达到转折点,使任何人都能通过简单的英语提示构建自定义应用,可能颠覆 iPhone 的应用商店模式。
Naval discusses how AI coding agents have reached an inflection point, enabling anyone to build custom apps with simple English prompts, potentially disrupting the iPhone's app store model.
您正在收听 Naval 播客。我是 Nivi,他的常驻联合主持人。今天我们要聊的是「氛围编程」。本期节目由 AngelList 呈现,这是 Naval 和我之前创办的一家公司。稍后我会详细介绍。让我用 Naval 在 3 月 23 日的一条推文来开启对话:AI 编程智能体现在可以一键将定制应用直接送到你的手机上。这是 iPhone 主导地位终结的开始。你想谈谈你在构建什么以及如何分发它吗?
You're listening to the Naval podcast. This is Nivi, his regular co-host. Today we're going to be talking about vibe coding. This episode is presented by AngelList, a company Naval and I started a while back at this point. I'll tell you a little bit more about it later. Let me tee up the conversation with a tweet from Naval from March 23rd. AI coding agents can now deliver one-shot custom apps straight to your phone. It's the beginning of the end for the iPhone's dominance. Do you want to talk about what you're building and how you're distributing it?
嗯,是的,让我谈谈氛围编程以及我是如何入门的。大约在 2025 年 12 月,随着 Claude Opus 4.5 的发布,AI 编程智能体达到了一个转折点。人们开始使用它,感觉就像:'哇,这是一个能保持专注、从头到尾构建应用、解决棘手问题的智能体,感觉就像有一个快速、基本免费且乐于助人的初级程序员供你差遣。'那是一个转折点,我在 Twitter 上看到了所有炒作,但这次感觉是真实的。我以前尝试过编程智能体,结果好坏参半,但这次我真的投入进去了。我已经几十年没有认真编程了。我的意思是,我有计算机科学学位,我理解计算机架构、网络、一点芯片、算法等,但我很久没有认真编程了。写代码的启动能量非常高。你必须把所有这些不同的服务连接起来,从 GitHub 到你可能在 Vercel 或 Firebase 或 Railway 等上做的后端,有很多东西需要连接。你必须知道很多行话、很多工具。而 AI 现在让这一切变得非常容易。所以,我和其他人一样从 Claude Code 开始。我也用过 Codex 来解决一些更棘手的 bug 和深层问题。我立刻上瘾了。这非常有趣。
Well, yeah, let me talk about vibe coding and how I got into it. So, around December of 2025, the coding agents in AI hit an inflection point with the release of Claude Opus 4.5, and people started using it were like, 'Wow, this is an agent that stays on track, can build apps soup to nuts, can solve thorny problems, and really feels like having a junior programmer at your disposal who's fast, essentially free, and ready to please.' That was an inflection point, and I was reading all the hype on Twitter, but this time it felt real. And I've tried the coding agents in the past with some mixed results, but this time I really got into it. And I haven't seriously coded in decades. I mean, I have a computer science degree, I understand computer architecture, networking, little bit of chips, algorithms, etc., but I haven't seriously coded in a long time. And the activation energy to writing code is really high. You have to like hook up all these different services to each other, everything from GitHub to maybe some back end you're doing in Vercel or Firebase or Railway or whatever, and just lots of things to connect together. You have to know lots of jargon, lots of tools. And the AI now makes it really easy. So, I started with Claude Code like everybody else. I've also used Codex for some of the thornier bug solving and deep problems. And I immediately got addicted. It was incredibly fun.
那么,发生了什么变化呢?嗯,智能体真的在起作用了。现在这些不仅仅是编程辅助,你让它解决一个特定问题,它给你一堆代码,然后你复制粘贴到你的 IDE(开发环境)中。相反,你打开一个终端,我称之为 CLI(命令行界面)。它完全是基于文本的,这正是这些东西擅长的,因为它们最初就是在文本 token 上训练的。它在内部或底层运行 Unix,这些智能体非常了解 Unix,因为如果你看看它们训练所用的所有代码,无论是 GitHub 还是 Stack Overflow,大部分都是 Unix 代码,而且大多数现代操作系统底层实际上都是 Unix。macOS 以 BSD 闻名。所以,底层都是 Unix,全是文本输入文本输出。因此,这些智能体只是长期运行的、与 Unix 核心层面连接的编程 AI。它们连接到 Unix shell 以便执行命令,通过基本的 Unix 命令连接到文件系统。它们可以调用所有 Unix 命令,如 grep、awk、sed、pipe 等,所有这些可以串联起来的操作符。它们可以运行 cron 作业,因此可以长期存在,并且可以根据需要生成更多的 shell 和任务。这非常容易上瘾,因为通常编程一旦进入状态会非常有趣。但进入状态的门槛很高。而现在突然间,你不需要知道所有工具和命令了。这些东西说英语。AI 是不可思议的翻译者,它们早期的核心用例之一就是机器翻译,它们被测试过翻译能力。但现在它们从 Python、C、Lisp、Rust 等各种编程方言和所有专业命令翻译成英语进行交流。而且它们在交流中非常宽容,你可以用不同的词语,可以拼写错误,可以用自己的方式解释事情。但如果你对计算机架构、网络和编程有基本的理解,不需要太多,实际上可以非常基础,我应该说非常高层。不是简单意义上的基础,而是高层意义上的基础。那么你可以走得很远很远。
So, what's changed? Well, the agents are really working. These are not just coding assists now where you ask it to solve a specific problem, it gives you a pile of code and then you cut and paste that into your IDE, your development environment. Rather, you open up a terminal, CLI as I call it, the command line interface. It's all text-based, which is what these things are really good at because they're trained on text tokens in the first place. It's running Unix inside or underneath, and these agents really know Unix because if you look at all the code out there that they were trained on sitting on GitHub or elsewhere or Stack Overflow, most of it was Unix and most of the modern OSs are really Unix underneath anyway. macOS is famously BSD. So, underneath these are all Unix, which is all text in text out. So, these agents are just long-lived coding AIs that are connected to Unix at a core level. They're connected to the Unix shell so that they can execute commands, they're connected to the file system through basic Unix commands. They can call all the Unix commands that grep and awk and sed and pipe and so on, all these operators that daisy chain into each other. They can run cron jobs so they can be long-lived, and they can spawn more shells and more tasks as needed. It's very addictive because normally with coding, coding can be really fun once you get into it. But getting into it, the activation energy is really high. But now all of a sudden, you don't have to know all the tools and all the commands. These things speak English. AIs are incredible translators and one of their core use cases early on was machine translation, they were tested on translating. But now they're translating from Python and C and Lisp and Rust and all of these various programming dialects and all of these specialized commands, and they're communicating in English. And they're very forgiving in their communication, so you can use different words, you can make spelling mistakes, you can explain things your own way. But if you have a basic understanding of computer architecture and networking and programming, and it doesn't take a lot, it can be very basic, actually, very high level, I should say. Not basic in the sense that it's simplistic, but basic in the sense that it's high level. Then you can go very, very far.
所以,为了好玩,我尝试构建了一堆不同的应用。我从一次性生成我想要的特定应用开始。一次性生成意味着我只需给出描述,它就会返回一个应用。然后我从那里开始改进。实际上,我构建了自己的小应用商店,一个只为我服务的应用商店。我可以向它要一个应用。它可以把那个应用交付到我的应用商店,那是一个网页,最终我把它做成了一个应用本身,存在于我的 iPhone 中。然后我可以一键下载这些应用,并且像在 App Store 中一样获得升级。所以,如果我想要一个新应用,比如跟踪我的锻炼,我就有了这个。我构建了一个完全按照我喜欢的方式定制的锻炼跟踪应用。所以,我可以说:'嘿,使用 Tonal 和 Ladder 的功能。遵循 Apple 的人机界面指南,让它看起来像一个 Apple 应用。按以下方式跟踪我的锻炼。这是我最近几次锻炼的文本日志。让我轻松重新输入新的锻炼并调整它们。为我构建漂亮的图表来跟踪我的进度。添加你能想到的任何功能。计算力量分数。阅读科学论文,找出按身体部位计算力量分数的正确方法。做一个人体图,显示哪些肌肉更大,哪些更小。连接到 Apple Health 来处理我的心率数据。'所以,我没有把所有内容都放在一个提示中,但我把很多内容放在了一个提示中。我立即得到了一个可用的应用,交付到了我的个人应用商店。顺便说一句,个人应用商店有点开玩笑。从某种意义上说,它是真实的,因为它是我的个人应用商店。它看起来像一个应用商店。我的应用被交付到里面。但显然它不能广泛分发,因为 Apple 限制了这一点。Apple 不允许你构建可以在任何人的 iPhone 上下载的应用。你必须将它们绑定到你的特定设备。所以,对于我的朋友和家人,我可以向他们交付应用。我还不能向所有人交付。然而,整个体验非常令人上瘾。你可以得到为你极度定制的应用。那么,这是否意味着普通应用没有位置?不,当然它们有位置。那些覆盖广泛用例的应用,它们将是同类中最好的。有人手工调整并为之付出了心血。所以,如果你的用例被某个广泛用例覆盖,你无法击败它。但是,当你想要真正定制或私有的东西时,这些对于只有你想要的利基应用或你想针对特定用例进行调整的应用来说,将是不可思议的。
And so, just for fun, I tried building a bunch of different apps. And I started by one-shotting particular apps that I wanted. One-shotting means I just give a description, it gives me back an app. And then I started improving from there. So, I actually built my own little app store, which is an app store just for me. I can ask it for an app. It can deliver that app to my app store, which is a web page, and eventually I made it into an app itself that lives in my iPhone. And then I can download those apps with one click, and I can get upgrades like you do with the App Store. So, if I want a new app, for example, that tracks my workouts, then I have this. I built a custom tracking app for just my workouts exactly the way I like it. So, I can say, 'Hey, use the functionality of Tonal and Ladder. Follow Apple's human interface guidelines to make it look like an Apple app. Track my workouts the following way. Here's a text log of my last few workouts. And make it easy for me to re-enter new ones and to adjust them. Build me pretty graphs and charts to track my progress. Add in whatever the features you can think of. Calculate strength scores. Read scientific papers to figure out what the right way to do strength scores by body part is. Do a human body diagram so it can just show which muscles are bigger, which are smaller. Connect to Apple Health to do my heart rate stuff.' So, I didn't put all of this in one prompt, but I put a lot of it in one prompt. And I immediately got a working app delivered to my personal app store. By the way, the personal app store is a little bit of a joke. It's real in the sense that it's my personal app store. It looks like an app store. My apps get delivered into it. But obviously it's not for wide distribution because Apple gates that. Apple will not let you build apps that can be downloaded on anyone's iPhone. You have to key them against your specific devices. So, with my friends and family, I can deliver them apps. I can't yet deliver them to everybody. However, this whole experience is incredibly addictive. You can get extremely the tuned apps for you. Now, does this mean that normal apps don't have a place? No, of course they have a place. Those apps that cover the broad use cases, they're going to be the best of breeds. Someone's hand-tuned them and slaved over them. So, you're not going to beat that if your use case is covered by one of the broad use cases. But, when you want something truly custom or private, these are great for niche apps that only you would want or you want to tune them to your specific use case, this is going to be incredible.
而且它非常上瘾,就像电子游戏一样,电子游戏的设计方式是通过给你反馈和奖励来让你沉迷其中。它总是处于你能力的边缘。所以,随着你变得更好,游戏会变得更难。它不会难到让你沮丧,也不会容易到让你无聊。所以你总是在能力的边缘操作,并获得这些奖励,但这些奖励是虚假的。而且电子游戏是有边界的,它是由其他人创造的,有点像虚假的小世界,你内心深处也知道这一点。所以你只是在摸索游戏的规则。一旦你弄清楚了规则,它就变得无聊了。但 Vibe coding 不同,它是无边界的,因为底层运行着一台图灵机。你可以构建任何东西。目标由你创造,并且可以不断扩展,所以它永远不会完全填满。而且它具有现实世界的相关性,你解决的不仅仅是虚假世界里的虚假游戏。所以它有趣得多。
And it's very addictive because like in a video game, the way a video game is designed is that it keeps you hooked by giving you feedback and rewards for doing work. And it's always at the edge of your capability. So, as you get better, the video game gets harder. It's not so hard that it's frustrating, but it's not so easy that it's boring. So, you're always operating at the edge of your capability with a video game and getting these rewards, but these rewards are fake. And the video game is bounded. It's created by other humans. It's sort of a fake little worlds, and deep down you kind of know that. So, you're just figuring out the rules of the game. And then once you figured out the rules of the game, it's boring. Except with Vibe coding, it's unbounded because now you've got a Turing machine running underneath. You can build anything. The objective is created by you and can keep expanding, so it kind of never fills up completely. And it has real-world relevance. It's not just some fake world for fake people or fake games that you're solving. So, it's way more interesting.
所以,Vibe coding 让很多生活中的朋友都消失在了他们想要的应用的 Vibe coding 中。但有一个清晰的方向真的非常有帮助。你必须知道自己想要什么。这实际上是最难的事情,并且要有非常清晰的愿景。我之所以有这种愿景,是因为有一个我痴迷了大约一年的应用叫 Airchat,我和一个团队一起构建了它,它是一个让人们通过语音和视频交流的社交信使。它没有完全成功,所以我们把它卖掉了,让投资者拿回了钱,团队也得到了一些不错的待遇。但我记得那段经历非常令人振奋,因为我在构建一个我想要的產品,并且和一个出色的团队合作。但我必须通过团队来完成。我有八九个工程师,我们努力工作了 9 到 12 个月,发布了几个版本。但通过 Vibe coding,我基本上在重建那个应用。我从头开始重建。但关键是我现在完全按照我想要的方式重建它,没有任何妥协。
So, Vibe coding has one shot at a whole bunch of life friends who have disappeared into Vibe coding the apps they wanted. But, it really, really helps to have a clear direction. You have to know what you want. That's actually the hardest thing and having a very clear vision of it. And I have that because of a particular app that I was obsessed with for about a year called Airchat, which I built with a team and it was a social messenger for people to talk through voice and video. It didn't quite work, so we sold it off, got the investors their money back, and got the team some nice packages. But, I remember that experience as being exhilarating because I was building a product that I wanted and I was working with a brilliant team. But I had to work through a team to do it. I had eight or nine engineers depending on the day, and we worked pretty hard for 9 to 12 months, and we shipped a couple of variations. But with Vibe coding, I'm basically rebuilding that app. I'm rebuilding from scratch. But the key now is I'm rebuilding it exactly the way that I want it. There's no compromises.
通常在团队构建任何东西的过程中,总是会有妥协,即使你没有意识到。即使你是负责人,这种情况也很少见,你仍然需要迁就别人。你不能说:‘把这个图标向左移。现在向右移。现在移回去。再移回去。’你不能那样做,你会惹恼工程师。你不能在没有合理理由的情况下提出要求,仅仅凭直觉或感觉。但 AI 编程代理的美妙之处在于没有这些。它就像一辆自动驾驶汽车。你在自动驾驶汽车里不会感到不自在,因为没有司机坐在那里。同样地,使用自主编程代理,你不会因为自己的怪癖而感到不自在。所以你可以完全按照自己的想法创造东西。
And normally in the act of building anything with a team, there's always compromises even if you are not aware of them. Even if you're the dictator in charge, which you rarely are, you still have to just accommodate other people. You can't say, 'Move this icon left. Now move it right. Now move it back. Now move it back again.' You can't do that. You'll annoy the engineer. You can't demand things where you don't have a reasonable justification, where it's just a gut feeling or intuition. But the beauty with an AI coding agent is there's none of that. It's like a self-driving car. You don't feel self-conscious in a self-driving car cuz there isn't a driver sitting there. The same way with an autonomous coding agent, you don't feel self-conscious about your own idiosyncrasies. So you can create exactly the thing that you want.
我认为 Vibe coding 的一个好处是,虽然我们可能看不到超高质量的代码,至少在这一代,架构还需要大量工作,这些东西可能有安全漏洞,可能难以扩展,但你得到的原型、你得到的个人应用会非常快,并且忠实于创作者的愿景。不会有任何妥协。所以你可能会得到更多像《我的世界》这样的东西,Notch famously 独自编码,那里只有一个人的愿景。它可能看起来很奇怪,比如:‘这些方块图形是什么?简直是巨大的倒退。’但他不必妥协。他不必与任何人沟通,也不必向任何人解释他为什么想要那样。所以我认为它扩大了发现的范围。它也非常有趣。
I think one of the nice benefits of Vibe coding is that although we may not see like super high-quality code, at least not in this generation, and the architecture needs a lot of work, and these things may have security holes, and they may be hard to scale, the prototyping that you're going to get, the individual apps you're going to get, is going to be very fast, and they're going to be true to the vision of the creator. There's going to be no compromises. So you may end up with more things like Minecraft, which Notch famously coded by himself, where there was one person's vision. And it may have looked weird cuz like, 'What is this blocky graphics? It's like a huge step backwards.' But he didn't have to compromise. He didn't have to communicate with anybody or explain to anybody why he wanted it that way. So I think it expands the scope of discovery. It's also incredibly fun.
它把可能构建应用的人数从人口的 0.1%提高到了 1%或 2%或 3%。别误会,大多数人不会自己编写应用。对大多数人来说,计算机就像一个神奇的黑匣子,谁知道里面发生了什么。即使它变得容易了 10 倍或 100 倍,对他们来说仍然毫无意义。它仍然是一个黑匣子。但对于那些有创造力、自我驱动、表达清晰且有良好愿景的人来说,你现在可以编程了。没有人挡在你和你的原型之间。
It takes the number of people who might have built apps from like 0.1% to 1 or 2 or 3% in the populace. Don't get me wrong, the majority of people are not going to code their own apps. For the majority of people, computers are sort of this magic black box and who knows what was going on there anyways. The fact that it's become 10x or 100x easier still doesn't mean anything to them. It's still a black box. But for the people who are creative, who are self-motivated, and who are articulate and have a good vision, you can code now. There's nobody standing in between you and your prototype.
是的,如果你带着一个功能强大的应用进入市场,并且需要扩展到大量用户等等,那么你需要招募一个优秀的团队,让真正的工程师加入,你可能需要重写整个东西。但如果你在实验、在制作原型、在推向市场,没有比这更好的了。作为软件创造者,从来没有比现在更好的时代了。
And yes, if you go to market with a high-functioning app and you need to scale to a lot of users and all of that, then you want to recruit a great team and you want to get real engineers on board and you're probably going to have to rewrite the whole thing. But if you're experimenting, you're prototyping, you're getting to market, there's nothing better. It has never been a better time to be alive as a creator of software.
现在,同样的市场机会还存在吗?这是一个大问题。它们变化得非常非常快。可能大公司变得脆弱,因为现在任何人都可以创建软件。也可能它们更有优势,因为它们有分销渠道,可以用它们能想到的所有软件填补所有空白。但我实际上认为这是个人软件创造者的复兴。另外,我发过一条推文,大意是:‘不再有风险投资支持的软件市场了。’或者说纯软件不再具有风险投资价值。
Now, are the same market opportunities still there? That's a big question. They're shifting very, very fast. It may be the case that the big companies are vulnerable because now anyone can create software. It may be the case that they have more of an advantage because they have distribution, they can just fill all the gaps with all the software they can dream up. But I actually think this is a renaissance for individual software creators. Now, one other tweet that I put out was something like, 'There's no market for venture-backed software anymore.' Or pure software is not venture investable anymore.
我记得你说过,纯软件正在迅速变得不可投资。
I think it's like pure software is rapidly becoming uninvestable, if I remember correctly.
是的,那是我真正想说的一个弱化版本,即纯软件是不可投资的。我就此打住。如果你的全部优势是‘嘿,我在构建别人不知道如何构建的酷软件’,我认为那是不可投资的。它不可投资有两个原因。一是今天他们就可以拼凑出来,二是编程代理进步如此之快,一年甚至更短的时间内,它们可能就能构建具有良好架构的可扩展软件。所以我认为我们会看到巨大的进步。这个精灵已经出瓶了。所以,如果你现在是一个风险投资者,你在寻找硬件、网络效应、AI 模型,而且我认为训练 AI 模型是新的构建软件,直到自动研究和自动训练开始起作用为止。
Yeah, that's a watered-down version of what I really wanted to say, which is that pure software is uninvestable. I would just full stop right there. If your whole advantage is like, 'Hey, I'm building cool software that other people don't know how to build,' I think that's uninvestable. And it's uninvestable for two reasons. One is they can just hack it together today, and the second is the coding agents are getting better so quickly that within a year or even less, they're probably building scalable software with good architecture. So, I think we're going to see a and bounds improvements. That genie's out of the bottle. So, if you're a venture investor now, you're looking for hardware, you're looking for network effects, you're looking for AI models, and I would argue that training AI models is the new building software for however long that lasts until auto research and auto training starts working.
但我认为 Vibe coding 比玩电子游戏更有趣,更高效,更有建设性,反馈循环更好。你构建自己想要的东西,你处于技术的最前沿。你甚至可能从中赚到一些钱或发展出职业生涯,尽管职业生涯已经有点过时了,但你可能会从中得到一个有趣的机会。而且你通过实践学到了很多关于计算机的知识。
But, I think vibe coding, it's more fun than playing video games, it's more productive, it's more constructive, it has better feedback loops. You build something you want, you're at the bleeding edge of technology. You may even make some money or career out of it, although careers are kind of dead, but you may make an interesting opportunity out of it. And you learn a lot about computers just by doing.
我见过一些孩子在 vibe coding。让孩子学编程很难。你可以把 Swift Playgrounds、Scratch Junior 之类的东西扔给他们,希望他们能学会编程,但如果你让他们 vibe coding,他们会得到即时反馈和即时奖励。也许在这个过程中他们会掌握一些基础知识,因为这些工具仍然需要一些操作技巧,而在操作过程中,你被迫去了解命令行,被迫去理解基本的计算机架构,被迫去学习缓存、网络退避、共享流、写入磁盘、延迟与带宽的权衡等概念。所以,你会被迫学到一些计算机算法和架构的基础知识。而且这是一种很有趣的方式。我最近熬夜,每晚大概花几个小时,以前用来阅读、刷手机或打游戏的时间,现在全用来 vibe coding 了。事实上,这就是我最近没怎么上 X 的原因。我完全消失在 X 上了,因为我埋头在 Claude 和 Codex 里。
I've seen kids who are vibe coding. It's hard to get kids to program. You can throw Swift Playgrounds and Scratch Junior and all of that at them and hope that they pick up coding, but if you throw vibe coding at them, they're going to get instant feedback and instant rewards. Maybe along the way they'll pick up fundamentals because these things still require some skill to operate, and in the process of operating them, you'll be forced to figure out the command line, and you'll be forced to figure out how basic computer architecture works, and you'll be forced to figure out concepts like caching and backing off in a network and sharing streams and writing to disk and latency versus bandwidth tradeoffs, etc., and all those things. So, you'll be forced to learn some basics of computer algorithms and architecture. And it's just a fun way to go. I've been up late nights, probably spending a couple hours every night, the time that used to go into reading or doom scrolling or playing video games is all now in vibe coding. In fact, that's why I haven't been active on X recently. I've been completely missing on X because I'm buried in Claude and Codex.
本期节目由 AngelList 赞助,这是我和 Naval 之前创办的一家公司,因为风险投资的基础设施还停留在过去。大多数基金管理员的问题在于他们没有 AI,软件也很糟糕,所以他们用人力来解决问题,服务团队不断流失。更糟糕的是,他们不是全服务的,所以基金不得不自己拼凑运营,而当资本催缴出错、估值难以追踪时,LP 们就会很恼火。AngelList 就是为了解决这些问题而生的。它是市场上最完整的产品,适用于从 1000 万到 10 亿美元的基金。它包括 SPV、侦察基金、合格购买者引荐,并且可以与你的智能体、ChatGPT 和 Claude 配合使用。AngelList 已经管理着超过 2000 亿美元的资产,5000 只基金,我和 Naval 都用它来管理我们的基金。我们认识的很多投资者也在用。迁移可能比你想象的要简单。可以在 4 周内完成。访问 AngelList.com/podcast 让你的基金上线,并告诉他们是我们推荐的。
This episode is presented by AngelList, a company Naval and I started a while back because the infrastructure for venture capital was stuck in the past. The problem with most fund admins is that they have no AI, and their software is terrible, so they throw humans at the problem, and their service teams are constantly churning. On top of that, they're not full service, so funds have to stitch together their own operations, and LPs end up getting pissed when capital calls are wrong and valuations are hard to trace. AngelList is built to solve all of that. It's the most complete product in the market for funds from $10 million to a billion dollars. It includes SPVs, scout funds, introductions to QPs, and it works with your agents, ChatGPT, and Claude. AngelList already powers over $200 billion in assets, 5,000 funds, and Naval and I use it to run our funds. So do a lot of investors we know. Migration is probably easier than you think. It can be done in 4 weeks. Go to AngelList.com/podcast to get your fund on rails, and tell them we sent you.
AI 已经变得如此出人意料地足智多谋,以至于每当我得到一个不够出人意料的回答时,我就认为他们没有给它足够的 token。智能体最让我感兴趣的是它们纠错和学习的能力。人们让智能体在晚上看 YouTube 视频,或者上网尝试学习白天被指示执行的任务。所以这些智能体出去纠错并提升自己的技能。同样,AI 模型中的思维创新也是纠错的一种应用,你把下一个词预测的过程变成一个伪思维过程,在思维过程的每一步都能进行纠错。消除幻觉也是一个纠错过程。所以我想知道 AI 中纠错的下一个应用会是什么。我有个随机的想法,而且我肯定有人在研究,就是把纠错应用于协同工作的智能体,智能体与其他智能体合作。因为人类学习和提高的重要方式之一就是与他人合作和交流。
AI has gotten so surprisingly resourceful that whenever I get a response that isn't surprisingly resourceful, I just assume they're not feeding it enough tokens. The most interesting thing to me about agents is their ability to error correct and learn. How people have it watch YouTube videos at night or go out onto the internet and try and learn about the tasks they've been instructed to perform during the day. So these agents are going out and error correcting and improving their skills. Likewise, the innovation of thinking in AI models is also an application of error correcting, where you take the next token prediction process and turn it into a pseudo thinking process that can error correct as it goes through each step in the thought process. Getting rid of hallucinations was also an error correction process. So I wonder what's going to be the next application of error correction in AI. One random thought I had, and I'm sure people are working on it, is applying error correction to agents working together, agents working with other agents. Because one of the important ways that people learn and improve is by working with and talking to other people.
我确信这个类比很贴切,因为 AI 是锯齿状智能,正如我所说,它在某些方面极其聪明,在另一些方面又极其愚蠢。而且它的结构与人类非常不同,当你使用 Claude 时,即使你运行了 10 个实例,你使用的也是同一个 AI 模型。所以 10 个实例互相交谈并不会像 10 个人交谈那样改善思维,因为那些人类是在 10 个不同的数据集上训练的。人类天生就非常有创造力,思维不受限制。而 AI 智能体是在相同的数据分布上训练的,它们实际上运行的是同一个模型。就像 10 个拥有相同大脑和相同数据集的人在互相交谈。当然,仅仅通过热力学,他们可能会有一些不同的想法,得出稍微不同的结果,但总体上他们会想得一样。所以当你让 10 个智能体互相交谈时,你只是在向问题投入 10 倍的 token。就像说如果需要的话,花 10 倍的时间。现在,有不同的模型,比如 Codex、Gemini 和 Grok Code,它们的训练略有不同,不是那么不同,但略有差异。所以它们可能会有一些不同的见解。Claude 通过 artifacts 系统有非常好的视觉呈现。Claude 非常擅长以我所在的水平与我交谈。它非常善于根据你的问题和对话来判断你能够理解什么以及你在什么水平上提问。它很擅长在那个水平上与你交流。ChatGPT 仍然是 OG。它各方面都很好。Gemini 非常擅长搜索,因为它底层有 Google 爬虫。这是一个令人沮丧的产品。它经常在应用上超时,失去连接,忘记上下文,但它非常快,并且有很好的搜索索引。所以如果我提出的问题本质上是一个搜索问题,那么我会用 Gemini。Gemini 还可以访问 YouTube。所以如果你认为答案在 YouTube 视频里,而 YouTube 视频很多,那么 Gemini 就有 YouTube 的数据优势。所以 Gemini 实际上是靠数据优势取胜的。对我来说它不像是最好的模型,但它有最好的底层数据。而 Grok 是我可以信赖的、会告诉我真相的那个。它就像是最少被阉割、最少被削弱的。它可以访问 X,所以非常擅长新闻。而且非常擅长技术问题。所以如果你问一个科学、数学领域的深奥难题,那么我认为 Grok 实际上相当不错。不是说其他模型不行,但我认为 Grok 在这方面很突出。这反映了创造、训练和推动它们的公司的偏见。目前,所有四个领先的前沿模型都有其用武之地。我确实让它们互相竞争。例如,我把它与我的 GitHub 连接起来,这样每次我提交一段新代码,比如由 Claude 编写的,那么 Codex 和 Gemini 就会在每个 pull request 中自动触发。这个名字有点误导,但实际上是你把代码推送到主仓库,基本上是在说这已经准备好审查,可以合并到主代码库了。所以,你一直在本地用 Claude 编写一段代码,然后把它推送到主仓库。于是你提交了一个 pull request。
I'm sure the analogy applies that well because AI is jagged intelligence as I say where it's incredibly smart at some things and incredibly dumb at others. And it's structured very differently than humans in that when you're using Claude, you're using the same AI model even if you have 10 instances of it running. So if 10 of them talking to each other doesn't really improve its thinking in the same way that 10 humans talking to each other do because those humans are trained on 10 different data sets. Humans are just inherently very creative and think out of bounds. Whereas the AI agents are trained on the same data distribution, they're literally running the same model. It's like 10 people with the same brain the same data set talking to each other. Sure, just through thermodynamics they might have some different ideas and come up with something slightly different, but they're generally going to think the same. So all you're doing when your 10 agents are talking to each other is you're just throwing 10 times as many tokens at the problem. It's like saying take 10 times as long if you need to. Now, there are different models like Codex and Gemini and Grok Code which are trained slightly differently, not that different, but they're slightly different. And so they might have some different insights. Claude has really good visual presentation through the system called artifacts. And Claude is very good at talking to me at the level that I'm at. So it's very tuned to figure out for your question and your conversation what you're capable of understanding and what level you're asking the question at. It's very good at meeting you at that level. ChatGPT is still the OG. It's sort of very good all around. Gemini is very good at search because it has the Google crawl underneath. It's a frustrating product. It's constantly timing out on the app and losing the connection and forgetting the plot, but it's very fast and it's got a great search index. So if the question I'm asking is really a search question underneath, then I use Gemini. Gemini also has access to YouTube. So if you think your answer is lying in a YouTube video, and there's a lot of YouTube videos, then Gemini has a data advantage of YouTube. So Gemini is really getting by on data advantages. It doesn't feel like the best model to me, but it has the best underlying data. And then Grok is the one I can count on to tell me the truth. It's like the least neutered, least nerfed. It's got access to X, so it's very good at news. And it's very good at technical problems. So, if you're asking a deep, difficult problem in the scientific, mathematical domain, then I think Grok is actually quite good. Not that the others aren't, but I just think Grok is a standout there. And that reflects the biases of the companies that created them and trained them and are driving them. Currently, all four of the leading frontier models have a place. I do use them against each other. So, for example, I wire it up with my GitHub, so that every time I'm submitting a new piece of code, say that's written by Claude, then Codex and Gemini automatically fire in every pull request. It's misnamed, but it's when you actually push code into your main repository, and you're basically saying this is ready for review and this is ready to get merged into the main codebase. So, you've been working locally on a piece of code, let's say with Claude, and then you push it into the main repository. So, you file a pull request.
你可以设置让其他智能体,比如 Gemini、Codex 和 Grok,自动触发并审查拉取请求。然后它们会说,‘哦,你应该改改架构这里’之类的。这是一种让它们相互沟通的方式,就像 AI 的圆桌会议。但我发现这并没有你想象的那么有用。这些 AI 仍然存在很多群体思维。如果你和它们一起编码,并且你朝着一个答案推进,比如你认为你知道答案是什么,它们很少会反驳你。你得错得相当离谱它们才会反驳你。它们是在取悦你。而且我认为它们没有自己长期的心智理论。所以它们总是倾向于向你靠拢,会找到你正在寻找的答案。所以如果你认为答案在某个领域,哪怕你稍微推动一下模型,所有模型都会找到大致相同的答案,因为你在引导它们找到答案。它们很容易被引导。
Well, you can set it up so that other agents, like Gemini and Codex and Grok, automatically fire and review the pull requests. And then they say, 'Oh, well, you should change this thing about the architecture,' and so on. That's a way of getting them to sort of communicate with each other, to have a council, a round table of AIs. But, I haven't found that to be as useful as you might think. There's still a lot of groupthink with these AIs. If you're coding with them and you push towards an answer, for example, if you think you know what the answer is, it is rare that they will contradict you. You'd have to be pretty wrong for them to contradict you. They're trying to please you. And I don't think they have any long-lived theory of mind of their own. So, they're always kind of morphing towards you, and they're going to find the answer that you're looking for. So, if you think the answer is in a certain area, and you push the models even slightly, all of them will find roughly the same answer because you're leading them to the answer. They're very easily led around.
我注意到的一件事是,随着代码库变得越来越复杂和庞大,管理起来变得更加困难,因为它不再能全部放入模型的上下文窗口中。模型只能在其‘头脑’中容纳一定量的数据,目前最先进的大约是一百万个 token,这在未来会被视为可笑。你可以近似地认为那是一百万个词,这是因为底层的 Transformer 注意力机制,为了让它正常工作,复杂度是上下文 token 数的平方。所以如果有一百万个 token,那意味着上下文窗口的复杂度大约是一万亿个 token,因为它是百万的平方。所以随着代码库变大,上下文窗口耗尽,模型无法再将所有内容保留在内存中。于是它们开始猜测、近似,开始压缩上下文窗口。它们开始迷失方向,变得困惑。它们开始修复错误的东西。同一个 bug 修复五次。它们会在架构上打一个快速补丁,而问题出在别处,你必须引导它们。
One of the things I've noticed is that as the code base has gotten more complex and larger, it becomes more difficult to manage because it doesn't all fit into the model's context window anymore. The models can only hold a certain amount of data in their heads and right now the state of the art is about a million tokens, which will be considered laughable in the future. You can approximate that by thinking that is a million words and that's because of the transformer attention mechanism underneath, which for it to properly work, the problem is a square of the number of tokens in the context. So, if it's a million tokens, that means the context window is like in the order of complexity of a trillion tokens cuz it's the square root of a million. So, the context window runs out as your code base gets larger, the models can't keep all of it in memory anymore. So, they start making guesses, approximations, they start compacting the context window. They start losing the plot. They get lost. They start fixing the wrong thing. They fix the same bug five times. They go do a quick patch in the architecture when the problem lies somewhere else and you have to guide them.
所以,当你处理越来越复杂的代码库时,操作者需要提供指导,说,‘实际上,我认为我们应该重新架构整个东西。’而它们会做一些极其愚蠢的事情。比如如果你不注意,只是看着文本滚动,偶尔它们会通过消除用例或直接破坏功能来修补一个 bug,或者做一些明显是 hack 的事情,你必须阻止它们说,‘嘿,那是个 hack。’顺便说一句,我经常这样做。我会阻止模型说,‘不,不,那是个 hack。那是个补丁。去在架构层面修复它。’有趣的是,模型总是会说,‘哦,对不起。你说得对,那是个 hack。’即使那并不是 hack,模型也会说,‘你说得对,那是个 hack。’所以模型总是试图取悦你,它并不真正明白,这有点像狗。如果你带着狗打鸭子,它比你更擅长抓那只鸭子,但它仍然是狗。所以如果你指向一只鸟,那不是鸭子,它可能会把那只鸟打下来。所以你必须引导它。这确实需要大量的操作监督。
So, as you are dealing with a more and more complex code base, it falls upon the operator to provide the guidance to say, 'Actually, here I think we should just re-architect that whole thing.' And they will do some incredibly boneheaded things. Like if you are not paying attention and just text is scrolling by, occasionally they'll patch a bug just by eliminating the use case or destroying the feature in the first place or they'll do something that is clearly a hack and you kind of have to stop them and say, 'Hey, that's a hack.' And by the way, I do this all the time. I'll stop the model and I'll say, 'No, no, that's a hack. That's a patch. Go fix it at an architectural level.' And what's funny is the model will always say, 'Oh, I'm sorry. You're right, that was a hack.' Even if that wasn't a hack, the model will say, 'You're right, that was a hack.' So, the model is always trying to please you and it doesn't know any better and that says it's a little bit like a dog. It's better than you at catching that duck if you're duck hunting with a dog, but it's still a dog. So, if you point it at a bird, you know, that's not a duck, it might take that bird down instead. So, you do have to guide it. It does require a lot of operational oversight.
所以,长话短说,你仍然需要引导这些模型。它们互相交谈并不能解决问题。你必须参与架构、调试、功能,并密切关注。但当前人类操作员与最先进的编码模型相结合,可以产生令人难以置信的结果。你已经可以一次性完成简单的应用。比如一个基本的任务列表,一个基本的视频游戏克隆,你可以一次性完成。一个提示,你就能得到相当不错的结果。所以你可以看到这趋势。最终,一旦它们有足够的数据,它们将能够一次性完成非常复杂的应用,那将是一个完全不同的世界。
So, long-winded way of saying, you still have to guide these models. Them talking to each other isn't going to fix the problem. And you do have to get involved in the architecture, the debugging, the features, and pay close attention. But, this combo right now of human operator combined with a state-of-the-art coding model can yield incredible results. You can already completely one-shot simple apps. So, like a basic task list, a basic video game clone, you can one-shot them. One prompt and you get something that's reasonably good coming out the other end. So, you can see where this is headed. Eventually, once they have enough data, they will be able to one-shot very complex apps, and that's a whole different world that we're going to get into.
那么,关于编码为什么让它们特别擅长?只是因为数据量巨大。而且当你训练模型时,很容易验证,‘嘿,你做得好不好?’因为代码必须编译、执行,而且你可以预先编写简单的测试来检查,‘你写的代码通过测试了吗?它做了它应该做的事吗?’所以编码实际上是很容易训练模型的事情之一。数学其实也类似,你有大量数据,大量已解决的问题,而且你可以很容易地验证输出。所以在数据量大且验证良好的领域,比如自动驾驶,这些模型表现极好。在数据不多的领域,比如全新的领域,模型表现不佳,那仍然是人类和创造力的机会。而在难以验证的领域,比如创意写作,谁来决定什么是好的创意写作,什么不是,什么是俗套,什么不是,这些模型表现就不太好,因为你不能轻易地建立一个闭环,让它们输出大量内容,然后立即通过算法评分,而不需要人类参与判断好坏。
Now, in terms of what is it about coding that makes them uniquely good at it? It's just there's tons and tons of data. And when you're training the model, it's very easy to verify, 'Hey, did you do a good job or not?' Because the code has to compile it, it has to execute, and you can have simple tests that are pre-written on the other side to say, 'Did the code you wrote pass the test? Did it do the thing you're supposed to do?' So, coding turns out to be one of those things that is actually quite easy to train models on. Mathematics is actually similar in that you have a ton of data, you have a lot of solved problems, and you can verify the output very easily. So, in domains where you have a lot of data, and you have good verification, self-driving is another one of those, these models do extremely well. In areas where you don't have a lot of data, which are kind of brand new fields, the models are not going to do well, and that's still an opportunity for humans and creativity. Then domains where it's hard to verify, for example, in creative writing, like who determines what's good creative writing versus what's not, what's slot versus what's not, then these models don't do as well because you can't easily run a closed loop where they're just outputting huge amounts of content and then that content is being immediately algorithmically graded without having to have humans in the loop saying this is good, this is bad.
例如,如果你尝试用这些模型进行创意写作,它们会输出大量内容。它们可以输出无限的文章。谁来评判另一边的好坏?即使你雇佣一些低薪人员像呼叫中心那样坐着说这个好或这个坏,那也只取决于他们的品味。我认为这些编码模型最近变得非常好的原因之一,有多个。一个是它们在做某种近乎递归的训练,一个模型帮助改进下一个。但我认为更大的原因可能只是很多顶尖软件工程师在过去几个月开始使用这些模型。他们的品味现在反馈回来了。所以你得到了他们的代码以及他们对好坏的品味。你需要高品味的反馈循环来改进这些模型,这比看起来更难开发。在某些领域是可行的,在其他领域则很难看到如何实现。所以显而易见的是,是的,你去构建你的应用。很好。不那么明显的东西,只是稍微高级一点,对软件工程师来说简单得可笑,但对非工程师或很久没编码的人来说,思考起来还挺有趣的。
For example, if you're trying to creative writing with these models, they're going to output huge amounts of content. They can output infinite essays. Who's to say it's good on the other side? Even if you hire some low-wage people to sit around call center style and say this is good or this is bad, it's only as good as their taste. I think one of the reasons why these coding models got really good recently, there's multiple. One is they're doing sort of almost recursive training where like one model is helping improve the next one. But I think the bigger reason might just be that a lot of the best software engineers started using these models in the last few months. And their taste has now feeding back in. So you're getting access to their code plus their taste as to what's good and what's not. You need high taste feedback loops to improve these models and those are harder to develop than they look. In certain domains it's tractable and in other domains it's hard to see how it happens. So the obvious stuff is yeah, you go and you build your app. Great. Less obvious stuff that's like just one level more advanced which will be laughably simple to a software engineer but it's kind of fun for a non-engineer or someone who hasn't coded in a long time to think about.
第一,我建了自己的应用商店。如果我想用一个应用,我直接打开手机上的 Claude。我可以操作一个运行在桌面上的远程终端,或者直接用云端的 Claude。它们能连上 Xcode。我给它两行描述,它就能给我做一个应用,然后发布到我的应用商店。我打开应用商店,应用就在那里。我点击安装,30 秒后手机上就有了一个能用的应用。这太神奇了。你甚至可以和人吃饭聊天时,对方描述一个想要的应用,你把它描述给 Claude,5 分钟后就能在手机上展示给他们看。这就是为什么我说这有点像苹果终结的开始,因为苹果某种程度上依赖其操作系统和应用比别人的更好。硬件确实更好,但这支撑不了它们的利润率和垄断或准垄断地位。所以,当你的所有沟通都开始通过 Claude 或 Codex 或其他智能体进行时,当你整天做的不是打开 Uber 应用,而是说‘给我叫个 Uber’,或者不是打开健身应用,而是说‘我的健身应用在哪?记录我的锻炼’,没错吧?那么你只是在和智能体交流。当这种情况发生时,对手机的需求就会越来越小。也许还有一些银行应用和政府应用没有迁移过来,也没有合适的 API,但这些智能体甚至不需要 API。它们可以即时找出并创建自己的 API。用例不再是你在 iPhone 或 Android 手机上操作,而是你直接与 AI 模型交互。现在苹果用的是 Gemini,那是谷歌的 AI 模型。那有什么区别呢?我可能还不如用 Android 手机,因为那时我只需要屏幕、电池和网络连接。Android 完全能满足这些。然后应用和用户界面会根据我的需求即时创建。是的,对于某些事情,总会有最好的用户界面和熟悉感,但即使是那种点按点按、升级系统软件、拖拽、找按钮、输入字段的时代,也正在消失。一切都应该是对话式的,都应该是智能体式的。在那个世界里,苹果失去了很多优势,然后它只能纯粹在‘我们有最好的芯片和最好的集成硬件’上竞争,但这和今天的苹果利润率不同。这更像是三星或联想赚的利润率,不是苹果想要的。因此,我认为它的市值会缩水。我认为苹果放弃 AI 将被视为这十年科技行业最大的战略错误,也是苹果主导地位终结的开始。这些公司可以长期存在并赚很多钱,比如微软现在比以往任何时候都更有价值。但微软 Windows 某种程度上已经输了,因为他们错过了移动浪潮。他们固守 Windows 操作系统,没有为手机从头升级到基于触摸屏的原生 OS 设计。而且他们没有关注消费者,太关注企业级了。所以苹果超越了它们,现在是世界上最有价值的公司之一。我记得它曾经是最有价值的,现在可能是英伟达。同样,我认为苹果也会被超越。我认为它们的增长有上限,因为它们在 AI 上被卡住了,落后了。除非它们能扭转 AI 的航向,否则我认为苹果长期增长受限,处于所谓的麻烦中,不是说不值钱,而是会比本可以的价值低很多。
One is I built my own app store. So if I want an app, I literally open up Claude on my phone. I can operate a remote terminal which is running on my desktop or I can just use Claude in the cloud. They can connect to Xcode. I give it a two-line description. It builds me an app. It ships it to my app store. I open my app store app. The app is sitting there. I click install. 30 seconds later I have a working app on my phone. That's magical. You can literally be at dinner with someone having a conversation. They describe some app they want. You can describe it to Claude and 5 minutes later you're showing them that app on your phone. That's why I say it's kind of the beginning of the end for Apple because Apple sort of relies on their OS and their apps being better than everybody else. The hardware, yes, it's better, but it doesn't support their margins and their monopoly or pseudo monopoly. So, when all your communication starts going through Claude or through Codex or through some other agent, when all you're doing all day long is instead of opening an Uber app, you're saying call me an Uber or instead of opening a workout app, you're saying where is my workout app? Track my workout. Make no mistakes, right? Then you're just communicating with the agent. And when that happens, then the need for a phone becomes much smaller and smaller. Maybe there's a few banking apps and government apps that haven't ported and don't have the proper APIs, but these agents don't even need APIs. They can figure out and create their own APIs on the fly. The use case stops being your interfacing with your iPhone or your Android phone. Instead, you're just interfacing with the AI model. And now Apple is using Gemini, which is Google's AI model. So, what's the difference? I might as well just use an Android phone cuz all I need at that point is I need a screen, I need battery, and I need connectivity. And Android's got that just fine. And then the apps and user interfaces are being created on the fly for what I need. And yes, for certain things, there'll always be best of breed user interfaces and there'll be a sense of familiarity, but even the era of tap tap tap, upgrade your system software, drag this over here, hunt for that button, type into that field, all that is going away. It should all be conversational. It should all be agentic. And in that world, Apple loses a lot of its advantages and then it's competing purely on oh yeah, we have the best chips and we have the best integrated hardware, but that's not the same margins as Apple of today. That's more like the margins that Samsung or Lenovo makes, which is not the margins that Apple wants to have. As a consequence, I think it's market cap will compress. I think Apple giving up on AI will go down as the biggest strategic mistake in the tech industry of this decade and it's the beginning of the end of Apple's dominance. These companies can exist for a long time and make lots of money, like Microsoft is more valuable than it's ever been. But, Microsoft Windows has kind of lost the battle because they missed the mobile phone wave. They stuck to Windows OS, and they didn't upgrade to a touchscreen based native OS design for phones from the ground up. And they didn't focus on the consumer, they were too focused on the enterprise level. So, Apple surpassed them and is now one of the most valuable companies in the world. I think it used to be the most valuable, might be Nvidia at this moment. The same way, I think Apple will get surpassed. I think their feature growth is capped because they're now captive on AI and they're behind. Unless they manage to turn the AI ship around, I think Apple has capped growth long-term and is in {quote} {unquote} trouble, not in the sense that it won't be valuable, but it'll be a lot less valuable than it could have been.
另一件事是,在我正在构建的应用中,我有一个 bug 报告基础设施:有人看到 bug,点击按钮,bug 发送日志到服务器,然后我让 Claude 每 24 小时检查所有 bug 报告,它自己修复所有 bug,无需我干预,并将所有修复放入分支供我审查。我只需要审查修复,说‘啊,那其实不是 bug,那个修复不好,不要发布。哦,那个看起来不错,有道理,发布吧。’我只是决定什么能发布的最后一道关卡。最终,你可以看到应用以这种方式按功能构建:用户会请求功能,对功能投票,然后云端的某个品味制定者或维护者会查看并说‘不,用户不知道他们想要什么’或‘哦,那很有道理,我们应该修复或改变它’。所以我认为软件开发甚至会成为与用户的协作过程,智能体将处理所有事情,因为从某种意义上说,智能体可以提供完美的客户服务。如果你的客户服务是完美的,你的客服人员也会是一个出色的程序员,而且不知疲倦。他们会全天候工作,写代码、修 bug、回复用户,而且没有自我,即使写了很多代码来修复 bug,然后全部扔掉。所以我觉得这种功能非常吸引人。现在你确实可以有一人、两人的软件公司,扩展到数百万用户,赚取数十亿美元。过去已经发生过,比如 Notch 和 Satoshi Nakamoto,以及非常小的团队,比如最初的 Instagram 团队,用很少的人就产生了巨大影响,或者最初的 WhatsApp 团队。但我认为你会越来越多地看到这种情况。
The other thing is, within the app that I'm building, I have a bug reporting infrastructure where someone sees a bug, they tap on a button, the bug sends the logs up, and the bug files into a server, and then I have Claude go every 24 hours through all the bug reports, and it just fixes them all by itself without my having to intervene, and it puts all the fixes into side branches for me to review, and then all I have to do is just review the fix and say, "Ah, that wasn't really a bug, that wasn't a good fix. Don't ship that. Oh, that looks good, makes sense, ship it." I'm just the final gate that decides on what goes out there. Eventually, you can see apps being built that way by features where the users will ask for features, they'll vote on features, and then there'll be some tastemaker or maintainer in the cloud who look at that and say, "No, the users don't know what they want." Or, "Oh, that makes a lot of sense, we should fix that or change that." So, I think even software development will become a collaborative process with the users, and the agents will be handling all of it cuz in a sense, the agents can do perfect customer service. If your customer service was perfect, your customer service person would also be a an incredible coder, and it would be indefatigable. They would be up 24/7. They would be writing code, fixing bugs, responding to people, and they would have no ego if they wrote a lot of code to fix a bug and then they just threw it all away. So, I just find that kind of a feature very compelling. You truly can have one-person, two-person software companies now that can scale to millions upon millions of users and make billions upon billions of dollars. That has happened already in the past with people like Notch and Satoshi Nakamoto and very small teams like the original Instagram team that just made a huge dent with very few people or the original WhatsApp team. But, I think you're going to see it more and more now.