AI 达到初级工程师水平：Jeff Dean 的大胆预测与智能体系统的未来

好的。我们开始吧？该开始了吗，杰夫？

All right. Should we go? Should we get started, Jeff?

当然，听起来不错。

Sure. Sounds great.

好的，杰夫，欢迎你。再次非常感谢你来到现场。我刚好感冒了，所以特别感谢你能来。

All right. Jeff, welcome. And again, thank you so much for being here. I just got a cold, so thank you for being here.

是啊，恐怕我嗓子都哑了。我平时声音不是这样的，但我们会尽力而为。

Yeah, I'm afraid I've lost my voice. I don't normally sound quite like this, but we'll do what we can.

你打造了 MapReduce、Bigtable、TensorFlow、TPU、Gemini。我们可以花整整一个小时聊你做过的事，但我更欣赏的是你仍然在公开场合做出大胆的预测。去年，也就是 2025 年 5 月在 AI Ascent 上，你说 AI 已经达到初级工程师的水平。那大约是一年前。我们现在离那个预测有多近？

So you built MapReduce, Bigtable, TensorFlow, the TPU, Gemini. We could spend a whole hour on all the things you've done, but what I love is that you're still making bold predictions in public. Last year, in May 2025 at AI Ascent, you said that AI is at the level of a junior engineer. That was about a year ago. How close are we to that prediction?

是的，我的意思是，模型在处理智能体式的长时间编码任务方面进步很大，而且很明显它们现在确实相当有能力。具体取决于你对初级工程师的定义，我觉得这个预测相当准确。

Yeah, I mean, the models have been getting a lot better at agent-based longer-running coding tasks, and it seems pretty clear that they are now actually pretty capable. Depending on exactly your definition of a junior engineer, it seems pretty spot-on, I would say.

你当时低估了什么？

What did you underestimate from that prediction?

我认为模型完成越来越复杂任务的能力增长得比我预想的要快。而且我也觉得，在编码之外，这些智能体式的系统在其他领域也开始大放异彩。我认为这将是未来的一个重要趋势。

I think the ability to do more and more complex tasks has been growing faster than I thought. And I also think outside of coding, these agent-based systems are really starting to shine in other domains. I think that's going to be an important trend in the future.

那就再给我们一个大胆的预测吧。你预测 2027 年会是什么情况？

So give us another bold prediction. What do you think is going to be the 2027 edition?

我认为你会看到机器学习系统本身的自动化程度大大提高。基本上，让机器学习系统通过运行大量实验来提升自身能力，把问题分解成子问题，在紧密的自动实验循环中运行这些子问题，把结果汇总起来，然后从这种完全自动化的任务分解和自动化实验中得出一个改进后的系统。我觉得这会非常令人兴奋。而且这不仅适用于机器学习，也适用于科学和工程的其他领域。基本上，任何有可衡量目标的事情，如今你都能取得很大的进展。

I think you will see a lot more automation of ML systems themselves. Basically, getting ML systems to improve their capabilities by running lots of experiments, breaking things down into subproblems, running those subproblems in a tight automatic experimentation loop, putting the results together, and then getting an improved system out of that sort of fully automated problem decomposition and automated experimentation. I think that's going to be really exciting. That also applies not just to ML but also to other fields of science and engineering. Basically, anything where you have a measurable objective, you can actually make a lot of progress these days.

现在让我们回顾一点历史。早在 2001 年，谷歌搜索还是运行在硬盘上的。

Now let's go back in history a bit. Back in 2001, Google search used to run on hard drives.

是的。

Yep.

你和桑杰做了计算，意识到在某个时候，整个搜索索引最终可以装进你们所有运行中的计算机的内存里。这个激进的认知促使你和桑杰在短短几天内就推出了一版全新的生产级搜索系统，它在内存中运行，而不是在硬盘上。这正是谷歌搜索速度如此之快的原因。历史总是会重演。那么，到了 2026 年，现在这个房间里每个人应该思考并为之设计的“能装进内存”的时刻是什么？

You and Sanjay did the math and realized that at some point the whole search index would finally fit in all of the RAM of the computers you had running. That radical realization led you to ship, in just a few days with Sanjay, a whole new production search version that worked in RAM rather than on hard drives. That was what made Google searches so fast. History tends to remix. So what is the 'fits in memory' moment right now in 2026 that everyone in this room should be thinking about and designing?

是的，情况有点不同，但我觉得你会看到越来越多高性能、低能耗的推理硬件系统。现在大家都意识到，推理是让智能体系统惠及更多人的关键，而延迟也非常重要。硬件专业化是让系统比 GPU 或 TPU 这类通用计算设备更节能、更低延迟的关键途径。

Yeah, it's a little different, but I think you're going to see more and more high-performance, low-energy inference hardware systems. Everyone is now realizing that inference is key to making agent-based systems available to more people, and that latency is really important. Specialization of the hardware is a key way to make things more energy-efficient and lower-latency than more general-purpose computational devices like GPUs or TPUs.

在座的各位都习惯了等待模型的响应。等待可不好玩。所以你的意思是，如果我们不用再等待了呢？

Everyone here is used to waiting for responses from models. Waiting is no fun. So you're saying, what if we don't have to wait anymore?

是的，想象一下，如果延迟能好上 50 倍，你能做些什么。

Yeah, I mean, imagine what you could do with something where the latency is 50x better.

有趣的想法。那么，现在在座的 6000 个人可能抱有的、关于 AI 的一个已经不再成立的假设是什么？

Interesting thought. Now, what's one assumption that perhaps 6,000 people in this room hold that's already false about AI?

这是个好问题。我认为有一点是，人们还没有充分意识到，智能体系统可以不仅仅在一个问题上运行一两个小时，而是对于某些问题领域，在底层有高能力模型的情况下，你可以让它们运行数天甚至数周，完成非常非常复杂的任务。有些人已经开始看到一些苗头，但我认为并非所有人都真正理解了这一点。这将是一件非常了不起的事。

That's a good question. I think one thing is that people don't quite realize how possible it is to have agent-based systems that can run not just for an hour or two on a problem, but for some problem domains, with highly capable models underlying them, you can get them to run for days or weeks and do really, really complicated tasks. Some people are starting to see inklings of this, but I don't think everyone has internalized it. That's going to be a pretty big deal.

你让系统跑了几个星期的具体任务是什么？是什么任务？你让智能体去解决什么问题？

What's a particular task that you've had run for weeks? What was it? What did you tell the agents to solve?

你可以让智能体去用不同的编程语言实现全新版本的软件，这些软件可能具有更好的安全性或更好的性能，然后它们真的会非常认真地去做这件事。

You can tell agents to go off and implement completely new versions of software in different programming languages that might have better safety properties or better performance properties, and they can go off and actually do that in a pretty serious way.

那太酷了。你有一项闻名遐迩的才能，就是非常擅长做餐巾纸计算。关于你的一个故事是，早在 2013 年，谷歌的语音识别开始奏效时，你做了个餐巾纸计算：如果每个谷歌用户都用手机对着它说话，每天只用语音识别系统三分钟，你发现谷歌就得把服务器数量翻倍，那将非常非常昂贵，而只是为了做语音识别。

That's pretty cool. Now, one thing you're well known for is being really good at napkin math. So, one of the stories about you is that back in 2013, when speech recognition started to work at Google, you did the napkin math: if every Google user used their phone and talked to it, using the speech recognition system for just three minutes a day, you found that you would have to double the fleet, which would be really, really expensive, just to do speech recognition.

是的。

Yeah.

而你没有这么做，而是打造了一款定制芯片，这就是 TPU 的起源故事。

And instead you basically built a custom chip and that was the origin story of the TPU.

是的。我们当时正在训练的基于深度学习的语音系统开始展现出非常好的效果。不过和旧的语音系统相比，它们的计算成本非常高，尽管错误率更低。这就像是把语音识别领域 20 年的进步浓缩到几个月里，我们只是调整了一下模型、稍微扩大了规模，并且用上了更好的数据。于是我们开始担心，如果语音识别的效果大幅提升，人们会更频繁地使用它。那次餐巾纸计算真正关心的是：如果人们开始更多地用语音识别来口述邮件、对着手机说话，或者其他类似用途，会怎么样？结果我们发现，当时需要比在 CPU 上运行更好的解决方案。于是我们想出了 TPU，它本质上非常擅长低精度的稠密线性代数，而这几乎是我们今天使用的所有现代机器学习算法的核心。

Yeah. We were starting to see really good quality results on the deep-learning-based speech systems we were training. But they were computationally expensive compared to the old speech systems, even though they had lower error rates. That was like the equivalent of 20 years of advances in speech recognition in just a few months of fiddling with the model, scaling it up a bit, and getting better data. So we started to get worried that if speech worked a lot better, people would use it more. That back-of-the-envelope calculation was really about, what if people start using speech recognition more to dictate emails or talk to their phone, or whatever. It turned out that we needed a better solution than running on CPUs at the time. So we came up with TPUs, which are essentially very specialized for low-precision dense linear algebra, which is at the heart of nearly all modern machine learning algorithms we use today.

如果你设计一款专用于低精度稠密线性代数、别的什么都不能做的专用芯片——这种事最终对机器学习推理非常有用，尽管它跑不了 Chrome 或 Word 之类的。那个系统几年后真的做出了一款芯片，能效是当时 CPU 和 GPU 的 30 到 80 倍，延迟也低得多，大概低 20 到 30 倍。

And if you build a specialized chip for low precision dense linear algebra—and can't do anything else—that turns out to be really useful for machine learning inference, even though it can't run Chrome or Word or whatever. So that system produced a chip a couple years later that was 30 to 80 times more energy efficient than CPUs and GPUs of the day, and also had much lower latency, like 20 to 30x lower latency.

这太不可思议了——TPU 如今已成为这么基础的设施。谁也想不到 TPU 会变得如此根基性，而 Transformer 架构是在 TPU 之后很久才发明的。

Which is incredible—what the TPU has become today. No way you would have predicted that TPU would be so foundational now, with the transformer architecture, which was invented much later than the TPU.

对，这正是我们构建通用线性代数系统的原因，TPU 本质上就是这样。因为我们知道机器学习算法仍在演进，不能过度专用化，但又要足够专用化，才能获得巨大的性能收益。我们可以有非常大的乘法单元、高速内存、高速互联——后来的 TPU 可以让非常多芯片高效地协同处理同一个问题。而且我们已经持续扩大规模并改进性能，迭代了很多代。

Yeah, I mean, that's sort of why we built a general purpose linear algebra system, which is what a TPU is really. Because we knew ML algorithms were still evolving, and you didn't want to overspecialize, but you wanted to specialize enough to get dramatic performance benefits. We could have very big multiplier units, high-speed memory, high-speed interconnect—later TPUs brought many, many chips to bear on the same problem efficiently. And we've continued to scale those up and improve their performance for many generations now.

凭一张餐巾纸就能算出来，太厉害了。

Incredible napkin math.

什么好？餐巾纸好。

What's good? Napkins are good.

那么，到底什么样的“餐巾纸估算”值得今晚在座想成为未来创始人的每个人都算一算，去做出像 TPU 这样有重大影响的东西？

So actually, what's a good napkin math that everyone here who wants to be a future founder should run tonight, to potentially build something as consequential as the TPU?

这总是很难说。我觉得：想想你在思考的事情中看到了什么问题——看到了哪些瓶颈——以及是否存在非常不同的思路来解决其中一些问题，从而带来一个数量级甚至两个数量级的性能或能力提升。因为有时候，如果你眯起眼睛看一个问题，不执着于这个问题今天是怎么解决的，而是从第一性原理出发去想该怎么做，你就能想出别人可能没想到的好点子。

Yeah, it's always hard to say. I think: think about what problems you see in whatever it is you're thinking about—what bottlenecks you see—and whether there are very different ways of thinking about solutions to some of those problems that would get you an order of magnitude or two orders of magnitude better performance or capability or whatever it is. Because sometimes if you just squint at a problem and think about not necessarily being anchored on exactly how that problem is solved today, but how you would solve it from first principles, you can come up with really good ideas that maybe other people aren't thinking about.

这个建议很棒。

That's a good tip.

在场各位可能不知道，多年前 Jeff 写了一份非常著名的清单，叫《每个工程师都应该知道的延迟数字》，里面都是像缓存未命中要多久、磁盘寻道要多久、数据包从加州传到荷兰要多久之类的数字，都是关于分布式系统和系统工程的。这份清单后来被贴起来，成了许多分布式系统工程师的圣经。

No, um, for everyone here who doesn't know, years ago, Jeff wrote a very famous list called 'Latency Numbers Every Engineer Should Know', and these are numbers around, for example, how long a cache miss takes, disk seek, a network packet traveling from California to Netherlands—lots of numbers like this about distributed systems and systems engineering. And it's been sort of taped up and become the bible for a lot of distributed systems engineers.

嗯，对。

Okay, yeah.

现在快进到今天，这份清单该更新了。给我们 2026 年的 AI 版吧。

Now fast forward, that list is up for an update. Give us the AI edition for now, 2026.

对，我觉得如果你看如今 AI 系统里什么最重要，你会想知道这些数字：加速器的主内存系统到片上内存、再到乘法单元之类的带宽是多少；做一次乘法运算要消耗多少能量；芯片之间的互联带宽是多少，以及用这种带宽能连接多少芯片。然后如果超出这个域，需要和 1 万块芯片而不是 500 块通信时，网络带宽的衰减是多少。我觉得这些都是真正需要了解的重要数字，它们会切实影响你思考如何解决特定类型的问题。

Yeah, I mean, I think if you looked at what is important in AI systems these days, you would want to know things like the bandwidth between your main memory system on your accelerator, to the on-chip memory, to the multiplier unit or whatever. You want to know how much energy it takes to do a single multiplier operation. What is the interconnect bandwidth between chips, and how many chips can you connect with that bandwidth? And then if you go beyond that domain, what is the falloff in network bandwidth when you need to talk to 10,000 chips instead of 500? I think these are all really important numbers to learn, and they really affect how you think about solving particular kinds of problems.

还有一件你聊过我觉得特别有意思的事：如今你衡量一切的单位是能量。

And one interesting thing that I've heard you talk about is that nowadays the unit you measure everything in is energy.

对。

Yeah.

你指出，做一次计算或数学运算大概消耗一皮焦耳，但搬移数据、做数据 I/O 要消耗那一千倍的能量。

You pointed out that doing a calculation or math costs about one picojoule. But moving the data and doing data I/O costs a thousand times that.

对。把数据从加速器的 HBM 搬到处理器里才能真正做计算。这个差距悄悄决定了什么样的产品能实现，以及 AI 算法是怎么被构建的。

Yeah. Just bringing it in from HBM on an accelerator into the processor so it can actually compute on it. That gap kind of quietly decides what products are possible and how these algorithms in AI are built.

那有哪些问题是创始人总以为属于模型问题，但实际上其实是能量或数据 I/O 问题？

So what are the kinds of problems that founders keep calling model problems, but are in fact actually energy or data I/O problems?

对，我的意思是，你刚才举的搬移数据和实际计算之间能量上相差一千倍，是一个非常重要的例子，它塑造了我们在机器学习里做的很多事。因为如果没有这一千倍的差距，你就不需要做批处理。但你必须一次批处理很多样本或很多 token，来摊薄数据搬移，这样就不是付出一千倍的延迟代价，而是付出“一千倍除以批大小”的能量代价。而且对于极低延迟场景，批处理其实不太好。所以我认为，这些事——以及我们所使用的计算机硬件中各种决策背后的能量开销——实实在在地影响了我们构建更高层系统时的大量决策。

Yeah, I mean, I think the example you raised of a thousandx difference in moving data versus actually computing on it, in terms of energy, is a pretty significant one, and it shapes a lot of aspects of what we do in machine learning. Because if you didn't have that thousandx difference, you wouldn't have to do batching. But you have to do batching of many examples—or many tokens at once—in order to amortize that data movement, so you don't pay a thousandx slowdown but pay a thousandx divided by batch size energy cost. And for really low latency, batching is not really very good. So I think these kinds of things—and the energy behind various decisions in the computer hardware we use—really affect a lot of decisions we make in building higher-level systems.

一个非常具体的例子就是训练模型的方式。有整个“把数据集分批并跑 epoch”的概念。那基本上——人们可能误以为这是模型问题，但实际上它是系统数据 I/O 问题，对吧？

A very concrete example is just how training models is done. There's this whole concept of batching the datasets and running epochs. That's basically—people perhaps may confuse that as a model problem, but it's really a systems data I/O problem, right?

对，对。你得组装批次来让硬件获得更好的效率。理想情况下你可能会做 batch size 1 的训练，但这样效率不太好。所以现在大家都用相当大的批次。

Yeah, yeah. I mean, you have to assemble batches to get better efficiency in your hardware. Ideally you might do batch size one training, but it's not as good in terms of efficiency. So people use pretty large batches these days.

你觉得有没有可能——我知道你以花一个长周或周末就拿出一个绝妙解决方案而闻名。会不会有那么一天，Jeff 花几周时间去研究，然后把 batch size 1 的训练做出来？

Do you think it's possible for—I know you're well known for taking off for a long week or weekend and coming up with a brilliant solution. Is there such a thing as Jeff going and working on it for a couple weeks and getting batch size equals one training done?

对，其实我最近想得更多的是推理。我觉得推理是个非常有意思的问题，因为推理确实需要极低延迟。训练倒不一定需要极低的延迟。而且我认为，在让硬件更专用于推理方面，还有很大空间，比我们现在做的更大。

Yeah, I've been thinking more about inference actually. I think inference is a pretty interesting problem because you do want very low latency. For training, you don't necessarily need incredibly low latency. And I think there's a lot of room for specializing hardware more for inference than we are today.

关于推理，有哪些你反复思考的有趣点？

What are some of those interesting things about inference that you're really thinking a lot about?

就是尽量最小化数据搬移，考虑极低精度的运算，可能不再支持很多很多种不同的精度。如果你对需要什么精度有明确答案，那就直接把它做进硬件里，别的都不用多管。

I mean, just trying to minimize data movement, thinking about incredibly low precision operations, and maybe not supporting lots and lots of different kinds of precisions. If you feel like you have a good answer for what kind of precision you need, maybe just build that into the hardware and not much else.

我觉得这归根结底是一个核心类比，是我从几位著名计算机科学家那里听来的：AI 的整个流程实际上是一个巨大的压缩问题，因为要对数据进行有损压缩再还原，你基本上必须理解这些数据。

Which I think brings it down to a core analogy I heard from famous computer scientists: the whole process of AI is really a big compression problem, because in order to compress data in a lossy way and then restore it, you basically need to understand it.

对，我是说，如果你真正理解了数据，就应该能把它压缩得非常好。

Yeah, I mean, if you truly understand the data, you should be able to compress it really well.

而现在，Transformer 架构基本上就是被证明效果非常好的一种方式。

And now the Transformer architecture is basically one of the ways that has turned out to work really well.

对，对，我会说——

Yeah, yeah, I would say—

到目前为止效果相当不错。

Working pretty well so far.

这要归功于我的同事们。

Good work by my colleagues.

是的。现在我们稍微把视野放大一点。AI 的进步过去就意味着更好的模型——你可以有更多数据，用更大的参数量来训练模型。但最近几年越来越明显的是，关键在于模型周边的整个生态。不只是模型规模、参数数量或更多数据，而是检索、工具、记忆、智能体工具这些周边的一切。这些东西可能会凝聚成人们所说的上下文工程，对吧？

Yes. Now let's zoom out a bit. AI progress used to mean just better models — you could have more data, train models with bigger parameters. But increasingly in the last few years, it's everything around the model. Not just the model size, number of parameters, or more data — it's everything around things like retrieval, tools, memory, agent tools. And it might kind of get consolidated into what people call context engineering, right?

是的，我认为模型其实只是你要做的事情中的一环，你要做的实际上是构建一个能解决真正有趣问题的完整系统。这涉及一个会使用各种工具的模型：它可能知道如何检索相关信息，可能有过去为旧问题检索过的信息历史，它能把信息放入模型上下文。这样做的好处是，这些信息对模型来说非常清晰——不像模型训练时所依赖的训练数据，那就像是把数万亿个 token 搅成一锅汤，混在数千亿或数万亿的参数里。那些信息远不如模型针对这个特定问题或用例直接看到的上下文清晰。然后，能够理解有哪些工具可用、哪些工具能帮助模型进入解决问题的下一个阶段、如何把一个任务分解成一系列工具调用、也许尝试多种方法看哪些有效，并能评估这些方案——这就是复杂智能体和多智能体系统的整体编排，我认为这会变得越来越重要。绝对是超级激动人心的时代。

Yeah, I think the model is really only one piece of what you're trying to do, which is to build an overall system that can solve really interesting problems. That involves a model that knows how to use various tools, that maybe knows how to retrieve relevant information, maybe has a history of other information it has retrieved for past problems, and it can put information into the context of the model. The nice thing about that is that information is really clear to the model — unlike the training data the model was trained on, where it's all kind of like trillions of tokens stirred together into a soup of hundreds of billions or trillions of parameters. That is all less clear than the actual context the model sees directly for this particular problem or use case. And then, being able to understand what tools are available, which ones are going to help the model solve this next phase of the problem, how to decompose a problem into a sequence of tool calls, maybe trying multiple approaches and seeing which ones work and being able to evaluate that — this is the whole orchestration of complex agent and multi-agent systems, which I think is going to become more and more important. Super exciting times, I would say.

我觉得这个特定问题领域的有趣之处在于，它其实是这个房间里每个人都能做到的事。以前要训练一个模型，你需要惊人的资源——大量的 GPU 和数据。但上下文工程的话，在座各位都能做。你只需要一个类似 Gemini 的 API，然后搭建你自己的方案，做自己的检索、自己的工具调用，等等。那么对在座各位有什么建议吗？大家怎样才能在上下文工程上做得更好，甚至出类拔萃？

And I think the fun thing about this particular problem domain is that it's actually something everyone in this room can do. Before, to train a model, you needed an incredible amount of resources — incredible access to GPUs and data. But for context engineering, everyone here can do it. You just need the API to something like Gemini, and then work on your own setup for your own retrieval, your own tool calls, and so on. So what are some tips for everyone here? How does everyone get better at, and become exceptional at, context engineering?

是的，我觉得一个非常好的方法就是去用这些模型，借助各种工具框架，实际去解决问题。然后有时候你真的能看到模型在哪里失败。你通常其实可以在外部让模型表现得更好，成功解决那类问题——不是通过调整模型参数，那从外部很难做到——而是通过为模型制定更好的指导规则，为模型编写技能，让它知道如何使用不同的工具，这些工具对解决这一类问题会非常有用。随着你不断这样做，你会进入一种不断改进、自我提升的循环，持续优化你用来解决问题的这套方案。这也是一个很好的途径，让你更好地理解模型还需要哪些额外信息才能变得更强大。

Yeah, I think a really good way to do it is to use these models, and harnesses and tools and so on, to try to solve problems. Then sometimes you can actually see where the models are failing. And often you can actually make the model work better and succeed at that kind of problem — not by adjusting the model parameters, which is hard to do from the outside — but by creating better guidelines for the model, writing skills for the model so it knows how to use different tools that would be incredibly useful for solving this particular class of problem. And as you do that, you end up in this kind of improving, self-improving loop of the setup you're using to solve things. And that's a really good way to get better at understanding what additional information the model would want in order to become more capable.

你能举一个你自己做过的上下文工程的例子吗？比如说你写的技能，或者那些真正改变你工作流程的工具？

Can you give an example of some context engineering you've personally done? I don't know — skills you wrote, tools that really made a huge difference in your workflow?

是的。几周前我和 Sanjay 在合作，我们经常做底层库的性能优化。我们在 Google 写了一个微基准测试库，你可以在里面写微基准测试，衡量不同类型操作耗时多久、填充某个数据结构需要多久之类的。有时候这些数据结构会被 Google 数以百万计的进程使用，所以确保它们的高性能确实非常重要。你可以写微基准测试。但在没有智能体系统的情况下，你通常的做法是：先在你关心的基准上测量当前性能；然后做修改，希望能提高性能；再重新运行基准，看看哪里提升了；也许再跑一组更广泛的基准，测量缓存占用等等。所以我们写了一个技能，基本教会模型如何按各种顺序做这些事情，这样它就能自主地进行基准测量、修改代码、测量性能提升，然后不断迭代。事实证明，这对某些类型的问题效果相当好。说到底，这就是把我们人类会用的方法，以一种模型能用的形式教给了它。

Yeah. Sanjay and I were working a few weeks ago, and we often do performance improvement for very low-level libraries. We have a microbenchmark library we've written at Google where you can write microbenchmarks of how long different kinds of operations take, or how long it takes to populate a particular data structure, and so on. And sometimes those data structures are used on millions of processes across Google, so it's actually pretty important to make sure they're high performance. So you can write microbenchmarks. But without an agent-based system, what you usually do is: you measure the current performance on some benchmarks you care about; you make some modifications to hopefully improve performance; then you rerun the benchmarks and see where things improved; you might run a broader set of benchmarks, measure the cache footprint of things. So we wrote a skill that basically taught the model how to do most of those things in various sequences, so that it could actually do self-improving benchmark measurement, make code changes, measure the performance improvement, and then iterate on that. That seemed to work pretty well for some kinds of problems. And it really just is us giving the approach we would use as people to the model in a form that it could use.

哇，这听起来太厉害了。你是说你有这么一个技能，如果别人能用上，就能像 Jeff Dean 一样做性能优化？感觉全世界都会为之疯狂——谁能用上这个，价值可是无限的。

Wow, that seems very impressive. So you're saying you have this skill that, if someone got access to it, could do performance optimizations like Jeff Dean? Seems like the world would love this — it's worth an infinite amount of money for someone to have access to.

其实我们几个月前发布了一份文档，叫《性能提示》，是 Sanjay 和我写的，大概 30 页，讲的是各种性能优化技巧。有些人把这份文档拿去，以摘要形式喂给各种模型，结果发现模型现在能更好地推理代码里的性能问题了。

Well, we actually published a document a few months ago called 'Performance Hints' that Sanjay and I wrote — about a 30-page document on various kinds of performance tricks. Some people have taken that, given it in summarized form to various models, and seen that the model can now get better at reasoning about performance issues in code.

所以你们都听到了。只要用他发表的这份《性能提示》论文，你就能像 Jeff Dean 一样优化自己的代码。

So you heard it all here. You could actually optimize your own code like Jeff Dean if you take this paper, the Performance Hints paper you published.

没错。

Yep.

它完全免费开放，所以大家都应该试试。太酷了。

It's all free and available, so you should all try it. Very cool.

是的。

Yeah.

既然你提到智能体——在座各位可能正在构建智能体，或者曾在某个时候构建过。我相信每个人都见过自己的智能体在大约第 30 或 40 步时跑偏。智能体大概在前 10 步左右表现很好，然后到第 50 步左右就开始变得不稳定了。

Now, you're talking about agents. Everyone here is probably building one or has built one at some point. And I'm sure everyone has seen their agent go off the rails at, perhaps, step 30 or 40. Agents are great up to, I don't know, step 10 or so, and then they get shaky around step 50.

你觉得现在的瓶颈是什么？是上下文评估器，还是因为本质上是一个开环系统，导致错误不断累积？

What do you think is the constraint today? Is it like context evaluators, or just errors that compound because it's basically an open-loop system?

是的，我的意思是，显然我们希望智能体能够运行非常长的时间，因为只有这样它们才能解决越来越复杂的问题。但就像你今天观察到的那样，它们有时在跟工具交互大约 10 次之后就会停止工作。有时是因为模型正在尝试做一些它没什么经验的事情。它是在一整套事情上训练出来的，一旦你稍微偏离它知道怎么做的事物的分布，那么和大多数机器学习模型一样，它的表现会突然开始下降。而且你离它擅长的事情的舒适区越远，它就越可能表现不佳。所以你可以做很多事情。一是给模型一些技能和提示，让它尽量走在自己更熟悉、更光亮的路径上。我也认为，构建多智能体系统是另一种方式：让多个智能体尝试不同方法，再由另一个模型或智能体评估哪些看起来有希望。这某种意义上是在可能的解决方案空间里搜索，保留最有希望的，丢弃那些看起来不行或者偏离轨道的。这是一个非常有用的通用技术：用推理时算力对解决问题的各种合理路径做搜索，这可以大大提高性能，或者让长时间运行的智能体流程更加可靠。

Yeah, I mean, obviously we want agents to be able to run for very long periods of time, because that's how they're going to solve more and more complicated problems. But as you observe today, they sometimes stop working after, you know, 10 interactions with the tools or something. And sometimes that's because the model is trying to do something it doesn't have a lot of experience doing. It's been trained on a whole set of things, and as soon as you get a little bit off the distribution of things it knows how to do, then like most machine learning models, its performance will suddenly start to degrade. And the farther you get off the comfort zone of what it knows how to do, the more likely it is to not work as well. So there are a bunch of things you can do. One is to give the model skills and hints that tend to keep it on the more brightly lit path of things it does know how to do. I also think having multi-agent systems, where you have multiple agents trying different approaches and have maybe another model or another agent evaluating which ones seem promising, is another way to, in some sense, search the space of possible solutions and stick to the ones that seem most promising and discard the ones that didn't seem to work or went off the rails. That's a very useful general technique: using inference-time compute to perform search over plausible ways of solving the problem, which can get much higher performance or much more reliability in long-running agent flows.

你在内部是怎么为智能体实现这种工作流的？有哪些具体方式？

How are some ways you implemented this particular workflow for your agents internally?

是的，我们有各种 harness（运行框架），还有一整套技能，尤其是在 Google 内部开发环境里。我们给智能体配置了技能，让它们知道如何使用我们内部大量的工具来做编码、代码审查、衡量性能或获取日志文件。这些技能就是你可以加在基础模型上的，让它更有能力——即使它未必专门训练过 Google 内部工程师如何从我们的专有系统里取日志。只要有正确的技能定义，你就能让它真正跑起来。

Yeah, I mean, we have harnesses, and then we have a whole set of skills, particularly in the internal Google development environment. We have skills so that the agents can know how to use lots of our internal tooling for coding, for code reviews, for measuring performance, or for fetching log files. And those are just skills you can add to make the base model more capable, even though it hasn't necessarily been trained on exactly how Google internal engineers would fetch log files from our proprietary system. With the right kind of skill definition, you can actually get it to work.

这确实提升了智能体的实用性。

And that improves the usefulness of the agents.

现在我们聊聊初创公司能在哪里胜出。这部分我个人非常关心，因为在座每个人都需要决定未来该做什么——如果你是未来的创始人。Google 的特点是，从处理器到产品，你会协同设计整个系统。那么哪些层次是 Google 这样的公司会持续投入、不断积累并变得更好的？哪些地方两三个人的团队仍然能赢？

Now let's talk about where startups can win. This section is one I personally care a lot about, because everyone here in this room needs to decide what to build in the future, if you're a future founder. The thing about Google is you co-design everything in the system, from the processors to the products. Which are the layers that someone like Google would keep building and compounding and getting better? And where can a two- or three-person team still win?

是的，我认为显然 Google、我们的 Gemini 模型和硬件基础设施真正在努力构建的是能做几乎所有事情的通用模型。但在很多情况下，这意味着我们对特定领域没有太多关注。一个设计得非常出色的产品界面，再加上一个模型和一组技能，或者一个不混在通用模型里的专用模型，实际上可以具备显著优势，因为你可以为一个你真正热爱的领域打造出令人愉悦、高精度、高质量的东西。我认为这正是两三个人在房间里做让他们特别兴奋的事情时能获得优势的地方。不过我也要提醒，通用模型确实在越来越广泛的领域不断变强。所以你必须想清楚：你现在做的事能维持多久？你觉得前沿模型在接下来 6 个月或 12 个月内会不会在这方面变得更好？还是它未来两三年都做不了？你在决定做什么的时候需要权衡这些。

Yeah, I think obviously Google, our Gemini models, and our hardware infrastructure are really trying to build very general models that can do almost anything. But in a lot of cases, that means we don't have a lot of attention on particular domains. Perhaps a really well-designed surface, plus maybe a model and set of skills, or a specialized model that isn't in the general mix of things our models do well, can actually have a significant advantage. Because you can build something delightful and with really high accuracy and really high quality for a domain you are passionate about. And I think that's where two or three people in a room building something they're really excited about can have an advantage. But I would also caution that the general models are definitely getting better at a broader and broader range of things. So you have to figure out: is the thing you're working on going to be durable? Do you think the frontier models are going to get better at it in the next six or 12 months? Or is it something they're not going to be able to do for a few years? You want to weigh that as you decide what to work on.

那我们深入聊一下。通用模型你们肯定会继续推进、不断改进。听众应该怎么判断哪些领域是机会？创始人应该怎么思考选什么来做？

So let's dive deeper into this. The general models, of course, you're going to keep working on and keep making them better. How should the audience reason about what those areas are? I mean, how should founders think about what to pick and work on?

是的，我觉得最重要的事情是选一个你超级兴奋、真正想去构建、并且认为对世界有用的东西，对吧？如果你能做到这一点，比起你醒来觉得‘哎，我真的不太想做这个’，或者你要做的这个东西实际上对世界、对很多人没什么用，你已经领先很多了。所以这是我选择下一个要解决的问题时，排在第一位的选择标准。第二，你要去观察当前更通用的模型在那个问题领域里能做到什么。你可以直接测试它们：它们能不能做得非常好？如果它们完全做不到，那可能是个好信号。如果它们已经能做一部分但做得不太好，那可能就不是好信号，因为这表示这种能力已经开始出现在那些模型里，而随着更多训练数据、更大规模的模型之类的东西，它很可能会变得更好。所以要找那种模型成功率只有 0% 或 1% 的任务，而不是 20% 的。

Yeah, I mean, the most important thing is to pick something you're super excited about, want to build, and think would be useful in the world, right? If you do that, you're already way ahead than if you wake up and think, 'Oh, I don't really want to do this,' or if you're going to build something that isn't actually useful to the world or to many people. So I think that's the number one selection criterion I try to apply for what problem I should work on next. Second, I think you want to look at what the current more general models can do in that problem domain. You can test them: are they able to do this thing very well? If they're completely failing, that's probably a good sign. If they're kind of able to do some of it but not very well, that's maybe not a great sign, because that's a sign that the capability is starting to be present in those models, and with more training data, larger-scale models, or whatever, it's likely to get better. So look for something where the model succeeds 0% or 1% of the time, not 20%.

你是怎么找到这些的？这些事本质上是不是训练集的分布外？具体什么样的任务形态符合这个条件？

How do you find those? I mean, are those things effectively out of distribution from the training set, and what exactly is the problem shape that fits that?

是的，我觉得有时候是因为你构建的产品能够接触到某种特定数据，而底层的通用模型接触不到。比如你在做一个帮助用户整理个人信息的工具，通用模型不一定能访问那些数据。这时候你就有很大优势，因为你的模型或产品突然能看到重要的数据。也可能它是个极难的问题，但如果你拿到正确的训练数据，训练一个比通用模型更专门的模型，你可以用非常划算的方式做到。也许训练一个针对特定问题的专精模型并不需要太多算力，但你却能得到一个高精度的东西。

Yeah, I think sometimes it's a product you build that might have access to a particular kind of data that the underlying general model doesn't. So maybe you're building something to help users organize all their own personal information, and the model won't necessarily have access to that. So there you can have a big advantage, because all of a sudden your model, or your product, has visibility into important data. It could also be some incredibly hard problem where, if you get the right training data and train a more specific model than a general-purpose one, you can do it in a very affordable way. Maybe it doesn't take that much compute to train a niche model for that particular problem, but you can get something that's highly accurate.

有时候，这可以成为解决一个重要问题的非常好的基石，而这个问题可能是一般模型处理不好的。

That can sometimes be a really good building block for solving an important problem that is maybe not handled very well by the general model.

我觉得这很有意思。我觉得基本上有两条路。第一条路有点意思：你们在做组织全球信息的事情。

I think that's interesting. I think there are basically two paths. The first path is a little bit funny: you guys are organizing the world's information.

是啊，这个可能已经被覆盖得很好了。

Yeah, that's probably kind of well covered.

是啊。但整理你的个人信息，这个领域还是开放的，挺有趣。

Yeah. But organizing your personal information, that's open, which is funny.

嗯。

Yeah.

然后第二条路，你谈到了某些领域的更专门化的模型。能再多讲讲这些领域有哪些吗？

And then the second path: you talked about more specialized models in certain domains. Can you tell us more about what are some of these domains?

是的。我是说，我觉得如果你看看我同事的工作，比如 AlphaFold，那是一个非常专门用于蛋白质折叠的模型，它非常成功，也确实能把那个领域处理得很好。于是突然间，你有了这样一个神奇的工具和模型，能非常高效地回答关于蛋白质及其结构的问题。但它不是通用模型，是一个高度专门的模型。还有其他领域也适合这种方法，比如材料科学或芯片设计等等，这些领域能让你利用一个非常精准但小众的模型，去做今天很难做到的事情。

Yeah. I mean, I think if you look at my colleagues' work on, say, AlphaFold, that was a very specific model for protein folding, and it was highly successful and was able to really handle that domain quite well. So all of a sudden, you now have this amazing tool and model that can give you answers to questions about proteins and their structure really effectively. But it's not a general model; it's a very specific one. And there are other domains where that kind of approach can work really well, maybe in materials science or chip design, or things like that, that will enable you to leverage the capabilities of a very accurate but niche model to do things that are hard today.

这是个好例子。所以如果你们有人发现一个与 AlphaFold 形状类似的问题，那可能是个值得研究的好问题。现在假设你们找到了一个问题去做。我们来聊一聊如何成为一名 AI 原生创始人？如何真正擅长这件事？你过去说过，管理一群智能体，比如 50 或 100 个智能体，关键在于写出非常清晰简洁的设计文档或规格说明。那人们怎么才能擅长这件事？这些文档长什么样？

That's a good example. So if some of you find a problem that's similar shape like AlphaFold, it could be a good problem to work on. Now let's assume you found a problem to work on. We're going to talk a bit about how do you become an AI-native founder? How do you really become good at it? You in the past said that managing a fleet of agents, like 50 or 100 agents, is all about writing really good crisp design docs or specs. And how do people get good at that? What do those look like?

是的。我是说，我认为在和你的虚拟智能体合作时，如果你能清楚地说明你想要什么，你会成功得多。你对想要什么描述得越清楚，智能体就越能拥有指导原则和规则，也就是它要完成的目标的概要。如果你不说明太多，智能体就得推断你的意思。很多时候，它推断出来的东西可能和你想的不一样。所以我们从一开始就一直告诉计算机科学家，在写软件之前，先明确你要写的软件要实现什么是非常重要的。现在我们确实有了能写代码的智能体系统，但明确你想要什么的重要性反而更高了，因为以前你是把它交给一个非常聪明的人，这个人可能有上下文，或者会问你后续问题。智能体有时也能做到这一点，但我认为清晰的规格说明确实是个好主意。举个例子，编码智能体有一个非常好用的场景：你可以让今天的模型把软件从一种计算机语言翻译成另一种语言，效果非常好，因为在这种情况下，你实际上有一份极其详细的规格说明。你拥有整个软件，它说明了系统应该做什么。所以如果你有一个 Python 实现，想要一个 Go 实现，这是当今模型看起来非常擅长的事情，因为它们可以提取 Python 中的所有测试，确保它们在 Go 版本中通过，把测试翻译成 Go，比较两个实现之间的行为差异，直到没有差异为止，并且非常高效，因为那份规格说明太清晰了。

Yeah. I mean, I think you'll have a lot more success when working with your virtual agents if you can clearly specify what it is you want. And the clearer you are on what it is you want, the more the agent will have guidelines and rules, an outline of what it is trying to accomplish. Whereas if you don't specify very much, the agent has to infer what you meant. And in many cases, it might infer things that are different than what you imagined. So we've always told computer scientists from the very beginning that it's really important to specify what the software that you're writing is trying to accomplish before going and writing it. And so now we actually have agent-based systems that can do the writing, but the importance of specifying what you want has actually gone up, because before you'd be handing it off to a very intelligent human who maybe has context or can ask you follow-up questions. Agents can sometimes do that, but I think clear specifications is a really good idea. And to give you an example of a use of a coding agent that works extremely well: you can ask today's models to translate software from one computer language to another very effectively, because in that case you actually have an incredibly detailed specification. You have the whole software that says what the system is supposed to do. So if you have a Python implementation of something and you want a Go implementation of it, that is something the models seem incredibly capable at doing these days, because they can take all the tests that are in Python, make sure they pass in the Go version, translate the tests to Go, compare behavioral differences between the implementations until there aren't any, and be highly effective, because that spec is so clear.

嗯。现在假设每个创始人都擅长同时运行数百个智能体，所有代码都由智能体替他们写完。那什么技能会变得稀缺？

Hm. Now let's assume every founder gets good at running hundreds of agents at the same time and all the code is written for them by the agents. What becomes the scarce skill?

是的，我是说，我觉得真正稀缺的是你要让你的智能体做什么，需要有非常好的品味，对吧？这是关键所在，从我研究背景来看，这是一个研究问题。研究者可以拥有所有工具和技术，但很多时候，关键是你把时间花在什么问题上面。如果你选对了问题，并且成功解决了它，那就比你愉悦地执行一个相当无聊问题的研究调查要好得多。所以选择做什么的宏观智慧，我认为非常重要。而且我认为模型不一定擅长这件事。所以你需要有人来引导大量 AI 辅助计算，以便更快地完成伟大的事情。但模型要做什么的本质，才是你该专注的关键。

Yeah, I mean, I think it's really having incredibly good taste in what you ask your agents to work on, right? That is the crux of, from my background, a research problem. A researcher can have all the tools and all the techniques, but often most of the battle is what problem are you going to spend your time on. And if you pick the problem well and you succeed in solving it, that's way better than if you delightfully execute a research investigation into a rather boring problem. So that high-level wisdom of what to work on, I think, is incredibly important. And I think models are not necessarily going to be that good at it. So you're going to have people steering a lot of AI-assisted computation in order to accomplish great things more quickly. But that essence of what it is you want your models to do is the key thing you should focus on.

那我们再聊聊品味，因为当下这个智能体编程时代，品味被频繁提及。你到底怎么培养品味并做到这一点？我是说，这听起来太玄了。你怎么让它变得具体？

So let's talk a bit more about taste, because it gets talked about a lot right now in this current era with agent coding. How do you exactly build taste and do that? I mean, yeah, that sounds so esoteric. How do you make it concrete?

是的，这确实很难。在很多时候，品味并不像有一个可衡量的客观标准。我觉得有一部分来自经验。过去处理过很多不同的问题，会教会你什么样的题目未来可能有趣，或者通过拼凑这些先前的方法，再加上一些必须解决的开放问题，才可能勉强触及某种神奇或高度有用的东西。另一个增加经验的方法是，写下你认为未来 12 个月里可能重要的一堆事情。也许你挑其中一件去做，但 12 个月后回过头评估：这些其他事情里，哪些实际上看起来重要，或者世界上有其他人去创造了哪些，还有哪些似乎还没人做。这能为你的品味培养能力提供更多样本。这是一项重要的技能。

Yeah, I mean, it is a difficult thing. It's not like there's a measurable objective of taste in a lot of cases. I think some of it is from experience. Working on a lot of different problems in the past kind of teaches you about what kinds of problems might be interesting in the future, or what kinds of things might be just barely possible by cobbling together these previous approaches and then some open problems you might have to work on in order to get to something kind of magical or highly useful. Another way you can get more experience for yourself is to just write down a bunch of things you think might be important in the next 12 months. And maybe you pick one of them to work on, but go back and evaluate in 12 months: of these other things, which ones actually seemed important, or which ones did other people in the world go out and create, and which ones did they not seem to do yet. That can give you a lot more samples for your own taste-creation capability. And that's an important skill to have.

我觉得我们之前谈到的第三种方式是做非常疯狂的思想实验。

I think a third way we were talking earlier was doing very crazy thought experiments.

哦对，这是另一个好办法。我是说，有时候确实不应该把大多数人视为理所当然的事情当成理所当然。前几天我和几个同事做了一个疯狂的思想实验：60 年来，整个硅芯片设计和制造行业做了巨大的努力，去制造越来越小尺寸、错误率极低的晶体管，对吧？因为我们想要的假设是，我们制造的同一种设计的每一颗芯片，都应该和其他每一颗芯片完全相同。

Oh yeah, that's another good way. I mean, I think sometimes it's good to not take as a given things that most people seem to take as a given. So I was doing a crazy thought experiment with some colleagues the other day about: for 60 years, the whole silicon chip design industry and fabrication industry have done tremendous work to make smaller and smaller scale transistors that are very low error rate, right? Because the assumption that we want is that every chip we manufacture of the same design should be identical to every other chip.

你并不希望任何比特位翻转。

You don't want any bits to flip.

一切——都不应该有比特翻转。内存里内置了各种错误余量，如今的内存都有 ECC 内存。在宏观尺度上，我们在构建大规模分布式系统时不会做这种假设。我们用不可靠的部件构建可靠的大规模分布式文件系统。单个磁盘可能会失效，但你的数据应该是安全的。所以我们在更高层有机制：把数据复制三份，放在三台不同的机器和三个不同的机架上，这样就算某个机架交换机、某台机器或某个磁盘失效，你仍然有数据。我们有里德-所罗门编码技术。但在我们所研究的晶体管级别的技术尺度上，我们似乎并没有把这种冗余做到极致。所以一个有趣的思维实验是：如果你试图用每天可能发生 20 个错误的晶体管来构建一个系统，会发生什么？

Everything — no bits should flip. There are all kinds of error margins built into memories; these days memories have ECC memory. At the macro scale, we don't make that assumption when we're building large-scale distributed systems. We build reliable large-scale distributed file systems out of unreliable parts. Individual disks can fail, but your data should be safe. So we have mechanisms at a higher level: three copies of the data on three different machines and three different racks, so that if any rack switch or individual machine or disk fails, you still have your data. We have Reed-Solomon encoding techniques. But we don't seem to do this at a really extreme level at the transistor-level scale of the technology we're working on. So an interesting thought experiment is: what would happen if you tried to build a system out of transistors that might have, say, 20 errors per day?

天哪。

Oh my god.

而不是每一百万年才出现一次错误，对吧？那会是一个非常不同的设计点，可能会在制造端带来非常有趣的可能性。你会有完全不同的设计方法论，因为如果你想把一个信号从这里传到那里，而这些晶体管又超级不可靠，你可能会采用非常不同的信号传输方式。你可能会沿着多条冗余路径发送信号，以确保至少有一条能到达。我认为这会是相当有意思的一组思维实验。我并不是说我们真该这么干，但这正是那种你偶尔该去质疑假设的情况。很多时候这些思维实验最终行不通，因为过去 50 年我们这样做而不是那样做，是有充分理由的。但每隔一段时间重新审视一下这些假设，是件好事。

Rather than one every million years, right? That would be a very different design point and might enable you to do really interesting things in the fabrication side. You'd have very different design methodologies, because if you want to get a signal from here to there and you have these super unreliable transistors, you might have very different ways of signaling. You might send it along multiple redundant paths to make sure it gets along one of them. I think that would be a pretty interesting set of thought experiments. I'm not saying we should go do this, but that's the kind of thing where you want to occasionally question assumptions. Oftentimes these thought experiments don't work out because there are very good reasons that for the last 50 years we've done this thing this way and not that way. But it's good to revisit those every so often.

这太疯狂了。我是说，这跟神经形态计算、人类大脑以及自然运作方式有很多相似之处。

That is so wild. Well, I mean, it's starting to rhyme a lot with neuromorphic computing or the human brain and how nature works.

没错。我们大脑中的信号在从一个地方传到另一个地方时，并不是特别可靠。所以在大脑中，当有非常重要的东西需要从一个地方传到另一个地方时，会有多条通路来确保完成传输。

Exactly. Signals in our brain are not especially reliable in getting from one place to another. So in brains, when there are really important things you need to get from one place to another, there are multiple pathways that enable you to do that.

你的职业生涯如此辉煌。有没有一个你曾抛之脑后的疯狂假设，最终真的构建出了一个举足轻重的系统？

You have such an impressive career. What is one of these crazy assumptions that you threw out of the window that actually built a consequential system in the past?

嗯，我想……

Yeah, I mean I guess...

结果真的成功了。

That worked out actually.

对，我想说的是，TPU 就是一个很好的例子。在那个问题领域看起来远没有今天这么重要的时候，就专门为这个非常垂直的领域定制硬件，这是一个思维实验。我认为 MapReduce 的起源是另一个好例子。当时桑杰和我，还有其他一些同事，在谷歌做了多轮爬取和索引系统的开发。我们写了很多手工并行化的代码，带大量检查点机制，以确保在 100 台或 1000 台计算机上运行时，即使部分机器宕机，系统依然健壮可靠。但这些代码往往会和你真正想做的简单事情混在一起，比如：我只想查看所有网页的内容，然后计算一个从 URL 到页面文本语言映射。而这些内容会被并行化和可靠性相关的代码淹没。于是我们想起自己在函数式语言方面的训练，意识到我们可以从宏观视角审视这些问题，从而开发出 MapReduce 抽象：在实现之上做抽象，在实现之下，把所有检查点和可靠性机制放进底层库，让所有上层应用都能构建在它之上。于是它成为谷歌处理超大规模计算的一种极成功的健壮可靠的方式。这个思维实验就是：如果我们换个角度审视，能不能找到大量适合这种抽象的问题？

Yeah, I mean I think, well, TPUs is a good example. Being able to specialize hardware for a very niche problem domain before that problem domain seemed as important as it is today is one thought experiment. I think the origin of MapReduce is another good example. So Sanjay and myself and a number of other colleagues had worked on various iterations of the crawling and indexing system at Google. We'd written lots of hand-parallelized code with lots of checkpointing to make sure it would be robust and reliable if it was running on a hundred computers or a thousand computers and some of those died. But that code tended to be intermixed with the actually relatively simple thing you often were trying to do, like: I just want to look at all the contents of all the web pages and then compute a mapping from URL to what language is this page in the text of this page. And it would get obscured by all this other code for parallelization and reliability. So we remembered our training in functional languages and realized we could squint at those problems, and developed this MapReduce abstraction that you could have above the implementation, and below the implementation you could put all the checkpointing and reliability mechanisms into that lower-level library that everything could then build on. And so that became a hugely successful way of dealing with very large-scale computations at Google in a robust and reliable way. From that thought experiment of: well, if we squint at it, could we find lots of problems that fit into this abstraction?

太了不起了。所以这个思维实验让你创造了 MapReduce。

That's impressive. So this thought experiment led you to create MapReduce.

是的。

Yeah.

太棒了。现在回到你刚刚谈到的话题：你现在对定制硬件很感兴趣。AlphaChip 在做芯片布局，而你又有了 AlphaEvolve，它提出解决方案、评估方案，并保留所有有效的方案。看起来你正在构建所有这些能够复合的系统，构建出能制造 AI 的 AI。

Awesome. Now let's go back to something you talked about: your interest right now in working on a lot of customized hardware. Right now AlphaChip lays out chips. Now you also got AlphaEvolve that proposes solutions, evaluates them, and keeps all the ones that work. Seems like you're starting to build all these systems that can compound and build AI that builds AI.

是的。我觉得更普遍地说，科学方法有这样一个基础：你提出一个实验，实现运行实验所需的东西，评估实验，然后从中得到结果。现在越来越多的领域可以实现这个完整循环，不只是跑几个实验，而是跑很多很多实验，因为你可以自动化这个循环，使它的延迟极低，这将会变得非常重要。它将使我们能够处理科学、工程、机器学习、模型设计本身，以及芯片设计等工程任务中的众多不同问题。如果你能以自动化的方式做这些事，并有一个编排框架，把高层目标分解成子问题，每个子问题都可以是一个自动化循环来探索解决该子问题的最佳方式，然后编排框架把子问题的解决方案组合成高层问题的整体方案——那真的会很有影响力。它将让我们加速机器学习的发展，加速科学进步，加速工程进步。那将会是惊人的。

Yeah. I think more generally, there's this foundation of the scientific method: you propose an experiment, you implement what you need to run the experiment, you evaluate the experiment, and then you get results from that. There are more and more problems that are now possible to address where that whole loop of running—not just a few experiments but many, many experiments—because you're able to automate that loop and make the latency of that loop extremely low, is going to be really really important. It's going to enable us to tackle lots of different problem domains in science and engineering and machine learning, model design itself, and also engineering tasks like designing chips. If you can do those things in an automated way and have an orchestration framework that can take very high-level objectives and break them down into subproblems, and each subproblem can be one of these automated loops that is exploring the best way to solve that subproblem, and then an orchestration framework that can put together the subproblem solutions into the overall solution for the higher-level problem—that's going to be really impactful. It'll enable us to accelerate machine learning progress, accelerate science, and accelerate engineering. That's going to be amazing.

听起来太棒了。基本上，很多领域只要你能有很好的评估机制，或者接近能形式化验证的东西，都很适合能自我改进的 AI 系统。

That sounds awesome. It sounds like a lot of fields basically where you can have very good evaluators, maybe adjacent to things that can be formally verified—those are ripe for AI systems that can self-improve.

是的，在很多情况下，有时候你需要让评估器变得更快。

Yeah, I think in a lot of cases sometimes your evaluators need to be made much faster.

举个例子，我的同事们大约十年前在量子化学的一些问题上做过一些工作。你要理解某个分子的性质，可以生成一些分子构型，然后想知道它有什么性质。你可以运行一个计算量非常大的密度泛函理论模拟器，这个东西可能需要一整晚的计算才能告诉你一个结果。但我的同事们做的是：把这些模拟运行的大量输出——输入分子构型和昂贵模拟器的输出——拿来训练一个该模拟器的神经近似。所以现在这是一个验证工具，但不需要一整晚，他们做出了快 300,000 倍的东西。

So, as an example, my colleagues did some work maybe a decade ago on some problems in quantum chemistry, where you're trying to understand the properties of a particular molecule. You can generate some molecule configuration and then you want to understand what properties it has. You can run a very computationally intensive density functional theory simulator, which is something that might take a night of computation to tell you the answer for one thing. But what my colleagues did was take a bunch of output from those simulation runs—the input molecule configurations and the outputs of the expensive simulator—and use it to train a neural approximation to the simulator. So this is now a validation device, but instead of taking a night, they made something that was 300,000 times faster.

哇。

Wow.

而且准确度几乎和运行完整规模的模拟器一样高。所以这彻底改变了你做科学的方式，对吧？因为现在你有一千万个东西要筛选。你可以去吃个午饭就做完，而不是一个六个月的工程，去拼命凑够算力来跑所有这些模拟。我认为在很多领域都有很大的空间，来建造更快的验证模型——可能是学出来的验证模型——能让你以快得多的速度获得真实答案的近似。那会改变人们对这些实验循环的思考方式，也会改变你围绕这些循环能有多快。

And nearly as accurate as running the full-scale simulator. So now that completely changes how you would do science, right? Because now you have 10 million things to screen. You could do that while you go to lunch, rather than it being a six-month endeavor where you could try to scrape together enough compute to run all these simulations. I think there's a lot of room in a lot of domains for much faster validation models—possibly learned validation models—that can get you an approximation to the true answer much, much more rapidly. And that changes how those experimental loops can be thought of and how quickly you can go around those loops.

对于这种超高速的科学方法，有哪些方向和问题是你特别期待它能解决或达成的？具体是哪些问题或方向？

What are some of the spaces and problems that you're super excited that this super sped-up scientific method is going to solve or achieve? What particular problems or spaces?

我觉得，显然机器学习本身就是其中一个。对吧？我们能不能有一个模型，通过运行大量实验来自我递归改进？你想想现在大型研究团队是如何改进模型的：通常人们想一些点子，跑一批小规模实验，看看这些实验是否效果好。如果有，他们挑选最有前景的，放到更大规模去试，然后评估，再把结果整合成新的模型配方。但我认为，没有什么真正的障碍能阻止这个循环变得更自动化——模型自己决定去探索，或者可能在高层次上由一些人点拨一下，比如‘你要不要在模型架构上试一些纳入这方面的新想法’，然后它就去跑大量实验，看哪些有效，然后以快得多的速度把这些整合进来。本质上，你想优化的是单位算力投入下的发现量。

Yeah, I think clearly machine learning itself is one, right? Can we have a model that is able to recursively self-improve itself by running lots of experiments? If you think about how models are improved today in large research teams, what usually happens is people think of some ideas, run a bunch of small-scale experiments, see if those work out well. If so, they take the most promising ones and try them at larger scale, which then gets evaluated, and then the results get integrated together into a new recipe for your model. But I think there's no real impediment to making that a much more automated loop, where the model itself decides to explore—or maybe with a nudge from some people at the various highest levels, like 'why don't you try some new ideas around model architectures that incorporate this'—and then it will go run lots of experiments, see which ones work, and those will get incorporated at a much more rapid rate. Effectively, you want to optimize your discoveries per unit of compute input.

非常棒。

Very cool.

是的。

Yeah.

现在回到现场：在座各位总有一天会成为创始人或开启职业生涯，你们可能会收到很多拒绝。那是必然的。杰夫，你也经历过。我的意思是，有一个故事：你和杰夫·辛顿、奥里奥尔·维尼亚尔斯（Oriol Vinyals）合写了一篇关于蒸馏的论文。

Now, going back to the room: as all of you will become at some point founders or start your careers, you will probably collect lots of rejections. That will happen. It has happened to you too, Jeff. I mean, there's a story that you with Jeff Hinton and Oriol Vinyals wrote a paper on distillation.

那篇论文是关于用一个大老师模型去训练一个更小、更高效的模型，计算成本低很多，模型参数也更少。这已经成为现在行业里大家都在用的技巧。是的。

Which has to do with taking a big teacher model to train a much smaller and more efficient model that's a lot cheaper to compute, with fewer model parameters. And it has become a trick that everyone is using right now in industry. Yeah.

但问题是，这篇论文在 ICLR 被拒了。

And the thing is, this paper got rejected at ICLR.

是的，我这么说吧，我不怪程序委员会，因为很多时候一篇论文会被三个人审，其中一个审稿人看了之后说，‘哦，这不太可能产生重大影响。’

Yeah, I mean, I think I don't fault the program committee, because a lot of times a paper gets three reviews, and one of the reviewers will look at it. In this case, they said, 'Oh, it's unlikely to have significant impact.'

不太可能产生重大影响。

Unlikely to have significant impact.

但你知道，我觉得我们写这篇论文的时候，实际上看到这是一个非常重要的问题，因为我们知道，从更大规模的模型出发，做出更便宜、能力很强的模型，是我们非常想做的事。我们想把模型服务给越来越多的人，覆盖许多不同领域，比如语音或视觉。但有时候，审稿人可能没有那样的体验，因为他们可能不是在考虑大规模 AI 服务，而是在想这是不是一个根本性的进展。所以它时不时被拒，没关系。我们把它放上 arXiv，人们读到它，人们使用它，都很好。我们也确实在用这个技术来打造我们的 Flash 模型，比如从我们的更大规模 Pro 模型出发。这也是为什么我们的 Flash 模型，比如 Gemini 里的，在相对其尺寸和速度来说会这么有能力。

But you know, I think when we wrote the paper, we actually saw this was a super important problem, because we knew making cheaper, highly capable models from larger-scale models was something we desperately wanted to do. We wanted to serve models to more and more people in many different domains, like speech or vision. But sometimes the reviewer maybe didn't have that experience, because maybe they're not thinking about large-scale AI services and are thinking about whether this is a fundamental advance. So it gets rejected every so often—that's fine. We put it on arXiv, people read it, people use it, it's all good. And we do use it in making our Flash models, for example, from our larger-scale Pro model. That's partly why our Flash models, for example in Gemini, are so capable relative to their size and speed.

它们是同类模型尺寸基准测试里最好的一批。是的，真的很厉害。我觉得其中一部分教训就是：即使被拒绝了，也要继续前进。

They're some of the best in the benchmark for their model size class. Yeah, just impressive. And I think part of the lesson is that even if you get rejected, keep going.

是的。这也是我会从那段经历中提炼出的教训。

Yeah. That's the lesson I would distill from that.

不，我觉得有趣的是，你是在 1999 年作为一个 20 人初创公司加入谷歌的。现在，如果要把当年年轻的杰夫·迪恩传送到今天，在这个时代，以你的技能，你会做什么？你会加入一个前沿实验室，还是创办一家公司？我不知道。你会怎么做？今天杰夫·迪恩的主题：25 岁的杰夫·迪恩。

No, I think the fun thing is that you basically joined Google as a 20-person startup back in 1999. Now, if you were to take the young Jeff Dean from back then and teleport him to today, in this era with your skills, what would you do? Do you join a frontier lab, start a company? I don't know. What would you do? The Jeff Dean theme today: 25-year-old Jeff Dean.

是的，我的意思是，这总是很难说，而且这是一个非常个人的选择，取决于你想把时间花在什么上。对我来说，最重要的一些问题是：你是否会去做一个你真正在乎的事情？你会不会在做这件事？如果你能和你喜欢共事的一群同事一起取得进展——如果你能一起解决它或取得进展——这件事会不会以一种积极的方式改变世界？比如，你会突然能够做一些事情，把那种服务提供给生物化学家，或者也许是一个更广泛的东西？它可以帮助程序员，或者帮助互联网上所有的消费者，或者别的什么。你应该努力去做的，是对世界产生积极影响，和你喜欢的人一起工作，并且努力工作、尽力而为。所以，关于你提出的那个特定取舍——加入一个前沿实验室，还是和你的一个、两个或三个亲密朋友一起创办公司——我觉得那是不同的体验。在一个大型成熟组织里，你有一些结构，你有很多很多了不起的同事，他们知道很多你不知道的东西。

Yeah. I mean, it's always hard to say, and it's a very personal choice of what it is you want to spend your time on. To me, some of the most important questions are: are you going to work on something you really care about? Will you be working on that? And if you're able to make progress on it with a bunch of colleagues you like working with—if you're able to collectively solve it or make progress on it—will that make a difference in the world in some positive way? Like, will you suddenly be able to do something and offer that service to, say, help biochemists, or maybe it's a broader thing? It'll help programmers, or it will help all consumers on the internet, or other things. What you should strive to do is to have impact in the world that is positive, to work with people you enjoy working with, and to work hard and do your best. So in terms of the particular trade-off you offered—joining a frontier lab versus starting a company with just one or two or three of you and your close friends—I think those are different experiences. In a large established organization, you have some structure, and you have lots and lots of amazing colleagues who know lots of things you don't.

你可以研究的有趣问题很多，而且你已经通过自己的工作拥有了一个发挥影响力的平台，已经影响了世界上很多很多人。而作为一家非常小的创业公司，你必须找到自己真正有热情的事情，并且承担很多风险，用能够成功、能够把事业做大的方式去攻克那个特定问题。但那也可能带来难以置信的回报。所以我觉得这真的取决于个人品味。但至少，无论你选择哪条路，都要问问自己：如果我研究这个问题，并且最好的结果出现了，世界会因此变得好很多吗？还是世界只会说，“嗯，挺酷的，但也就那样吧”？

You have lots of interesting problems that you can work on, and you already have a platform for impact through your work, influencing many people in the world already. And then as a very small startup, you have to have something you're passionate about, and there's a lot of risk in taking on that particular problem in a way that you're going to succeed and grow an endeavor. But that can also be incredibly rewarding. So I think it's really up to personal taste. But at the very least, regardless of what path you take, ask yourself: if I work on this problem and the best possible outcome happens, will the world be a lot better in some way, or will the world go, eh, that's kind of cool but whatever?

那就不值得你花时间。

That's not the kind of thing you should spend your time on.

现在我们再聊聊第二条路：和真正喜欢的人在小团队里一起工作。你曾是许多工程师的出色导师和管理者，也构建过庞大的系统。对在座的各位来说，关于如何从与聪明人合作中获得最大收获，或者如何去找到聪明人，你有什么经验可以分享吗？

Now let's talk a bit more about that second path: working with people you really like in a small team. You've been an incredible mentor and manager to many engineers and have built huge systems. What are some lessons for everyone here on how to get the most out of working with smart people, or how to find smart people?

是的，你总是想找到那些在你试图组建的团队中拥有真正优秀技能的人，无论那是在公司内部还是创业。但你也想找到你愿意与之相处的人，因为你会花很多时间和大家一起解决非常困难的问题。你希望那些人谦逊、有团队精神，而且技能上可能与你互补。我总是觉得，在一个小团队里，大家知道一些我不知道的东西，而我也可能拥有别人不太擅长的技能，这样的合作超级有趣。你们共同构建的某件东西，单靠任何一个人都做不出来，但在这个过程当中，你自己也会获得很多新知识和新技能，他们也一样。你应该把自己的工程或研究生涯看作一条神奇的工具带，上面有各种技术。你要不断往工具带上添加新工具，因为你永远不知道什么时候会遇到一个问题，需要这四个专用工具而不是那三个。添加更多工具会让你未来遇到的问题更有可能被你自己解决。

Yeah, you always want to find people who have really good skills in some area that's needed in a team you're trying to form, whether inside a company or starting a company. But you also want to find people you delight being around, because you're going to spend a lot of time around people working on really hard problems. You want people who are low ego, who are team players, and who perhaps have complementary skills to your own. I always find working in a small team where people know things I don't know, and where maybe I have some skills others don't have as much of, is super fun. You're collectively building something that none of you could do individually, but in the process you actually gain a lot of new knowledge and skills for yourself, and so do they. You want to view your engineering or research career as having an amazing tool belt of techniques. You always want to be adding new tools to that tool belt, because you never know when you might come across a problem where you need four specialized tools rather than three. Adding more tools makes it more likely that the problems you encounter in the future will be solvable by you.

最后一个问题。我很确定在座的人里，或多个人，最终会做出和你在 MapReduce、TPU、蒸馏等方面一样有影响力的成果。你希望他们去研究什么问题呢？

Now, one last thing. I'm pretty sure someone in this room—or multiple people—will eventually build something as consequential as you've done with MapReduce, TPU, distillation, etc. What problem do you hope they would be working on?

哦，没错。我觉得世界上有很多有趣的问题，我就随便说几个吧。这不是一个穷尽的列表，因为世界很大，到处都是问题。我特别兴奋的是硬件的新方法——之前的那个思想实验就是一种暗示——或者更高效的推理硬件。我认为可能存在一些根本不同的机器学习算法，比我们今天用的方法要数据高效得多。想想我们今天的大规模模型，它们看到的数据大概是一个 18 岁人类所看到数据的一千倍。然而一个 18 岁的人在很多事情上做得更好，并且和那些看了多得多的数据的前沿模型水平相当。那么，你能不能用数据高效得多的系统，让它能从自己的行动中持续学习？持续学习是一个很有意思的课题。我认为多智能体交互也很有意思。我觉得在世界上创造更好的对话方式也可能很有趣。有没有办法进行更文明的对话，并帮助人们认识全世界他们应该认识的人，基于兴趣？这些都是很有意思的事。我觉得世界上有很多很酷的东西，我们都应该去努力创造更酷的事情发生。

Oh yeah, I think there are a lot of interesting problems in the world, and I'll just rattle off a few. This is not exhaustive because the world is a very big place and full of problems. I'm particularly excited about new approaches to hardware—that thought experiment was a kind of indication of that—or much more efficient inference hardware. I think there are radically different kinds of algorithms for machine learning that might be much more data efficient than the approaches we're using today. If you think about our large-scale models today, they probably see a thousand times as much data as a human does by age 18. Yet the human by age 18 is better at a lot of things and on par with those frontier models that have seen way more data. So could you come up with much more data-efficient systems that can continuously learn from their own actions? Continual learning is a really interesting thing. I think multi-agent interactions is interesting. I think creating ways of having better discourse among people in the world could be interesting. Are there ways to have much more civil conversations, and helping people meet others all over the world that they should know based on their interests? These are kind of interesting things. I think there are lots of cool things in the world, and we should all go and strive to make even cooler things occur.

听起来太棒了。非常感谢你，Jeff Dean。今天我们就聊到这里。

That sounds wonderful. Thank you so much, Jeff Dean. That's all we have today.

感谢。

Appreciate it.

谢谢大家。

Thank you all.