Meta 的 AI 雄心：Llama 3、开源与智能未来

Mark，欢迎来到播客。

Mark, welcome to the podcast.

嘿，谢谢邀请。我是你播客的忠实粉丝。

Hey, thanks for having me. Big fan of your podcast.

哦，谢谢，你这么说真好。好，那我们开始聊聊这次采访播出时会发布的那些东西吧。给我讲讲这些模型，讲讲 Meta AI。有什么新内容？它们有什么令人兴奋的地方？

Oh, thank you. That's very nice of you to say. Okay, so let's start by talking about the releases that will go out when this interview goes out. Tell me about the models, tell me about Meta AI. What's new? What's exciting about them?

当然。我想世界上大多数人会看到的是新版本的 Meta AI。这是模型的升级。我们正在推出 Llama 3。我们既以开源形式提供给开发者社区，也将用它来驱动 Meta AI。关于 Llama 3 肯定有很多可以聊的，但我想核心是，有了 Llama 3，我们现在认为 Meta AI 是人们可以免费使用的最智能的 AI 助手。我们还整合了 Google 和 Bing 来提供实时知识。我们会在我们的应用中让它更加突出。基本上，在 WhatsApp、Instagram、Facebook 和 Messenger 的顶部，你就能直接用搜索框问任何问题。我们还添加了很多新的创作功能，我觉得很酷。动画是一个好例子。你可以把任何图片做成动画。但我觉得人们会觉得特别神奇的是，它现在能极快地生成高质量图像。我不知道你有没有试过，它会在你打字的同时实时生成并更新。你一边输入查询，它一边在调整。比如，“这里，给我看一张奶牛在田野里、背景有山、吃着夏威夷果、喝着啤酒的图片。”然后它就在实时更新图像。这真的很神奇，我想人们会喜欢的。所以这就是世界上大多数人会看到的。我们不是在所有地方都推出，而是从少数几个国家开始，未来几周和几个月会扩展到更多地区。这会是件大事，我真的很期待把它交到人们手中。这对 Meta AI 来说是一个巨大的进步。但如果你想深入了解一点，Llama 3 在技术上显然是最有趣的。第一个版本我们训练了三个尺寸：80 亿和 700 亿参数今天发布，还有 4050 亿参数的稠密模型仍在训练中，所以今天不发布。但 80 亿和 700 亿的版本，我对它们的效果非常兴奋。它们在各自规模上处于领先地位。我们会发布一篇博客文章，列出所有基准测试结果，人们可以自己查看，而且显然是开源的，所以大家有机会亲自尝试。我们还有后续发布的路线图，会带来多模态、更多语言、更大的上下文窗口。然后希望今年晚些时候，我们能推出 4050 亿参数的模型，它还在训练中，但以目前训练中的表现，MMLU 已经达到大约 85 分，我们预计它会在很多基准测试上取得领先成绩。所以我对此非常兴奋。700 亿参数的模型也很棒，我们今天发布，MMLU 大约 82 分，在数学和推理方面有领先分数。所以我认为把它交到人们手中会非常神奇。

Yeah, sure. So, I think the main thing that most people in the world are going to see is the new version of Meta AI, right? So it's the upgrade to the model. We're rolling out Llama 3. We're doing it both as open source for the dev community, and it is now going to be powering Meta AI. So there's a lot that I'm sure we'll go into around Llama 3, but I think the bottom line on this is that with Llama 3, we now think that Meta AI is the most intelligent AI assistant that people can use that's freely available. We're also integrating Google and Bing for real-time knowledge. We're going to make it a lot more prominent across our apps. So basically, at the top of WhatsApp, Instagram, Facebook, and Messenger, you'll just be able to use the search box right there to ask any question. And there's a bunch of new creation features that we added that I think are pretty cool. I think animations is a good one. You can basically just take any image and animate it. But I think one that people are going to find pretty wild is that it now generates high-quality images so quickly. I don't know if you've gotten a chance to play with this, but it actually generates it as you're typing and updates it in real time. So you're typing your query, and it's kind of honing in on it. You know, like, "Okay, here, show me a picture of a cow in a field with mountains in the background, eating macadamia nuts, drinking beer." And it's just updating the image in real time. It's pretty wild. I think people are going to enjoy that. So yeah, that's what most people are going to see in the world. We're rolling that out not everywhere, but we're starting in a handful of countries, and we'll do more over the coming weeks and months. So that's going to be a pretty big deal, and I'm really excited to get that in people's hands. It's a big step forward for Meta AI. But I think if you want to get under the hood a bit, the Llama 3 stuff is obviously the most technically interesting. So for the first version, we're training three versions: an 8 billion and a 70 billion, which we're releasing today, and a 405 billion dense model, which is still training, so we're not releasing that today. But the 8 and 70, I'm pretty excited about how they turned out. They're leading for their scale. We'll release a blog post with all the benchmarks so people can check it out themselves, and obviously it's open source, so people get a chance to play with it. We have a roadmap of new releases coming that are going to bring multimodality, more multilinguality, bigger context windows to those as well. And then hopefully sometime later in the year, we'll get to roll out the 405 billion, which is still training, but for where it is right now in training, it is already at around 85 MMLU, and we expect that it's going to have leading benchmarks on a bunch of the benchmarks. So I'm pretty excited about all that. The 70 billion is great too. We're releasing that today. It's around 82 MMLU and has leading scores on math and reasoning. So I think just getting this in people's hands is going to be pretty wild.

哦，有意思。这是我第一次听到那个基准测试。太令人印象深刻了。那 80 亿参数的版本是不是几乎和我们发布的最大的 Llama 2 版本一样强大？

Oh, interesting. Yeah, that's the first time hearing that benchmark. That's super impressive. Is the 8 billion nearly as powerful as the biggest version of Llama 2 that we released?

所以最小的 Llama 3 基本上和最大的 Llama 2 一样强大。

So it's like the smallest Llama 3 is basically as powerful as the biggest Llama 2.

好，在我们深入这些模型之前，我想回到过去。2022 年，我猜是你开始采购这些 H100 的时候，或者你可以告诉我具体时间。当时股价大跌，人们都在问，“这些资本支出是怎么回事？”人们不买元宇宙的账，而你大概是在花那些资本支出来买这些 H100。当时你怎么知道要去买 H100？你怎么知道我们会需要这些 GPU？

Okay, so before we dig into these models, I actually want to go back in time. 2022 is, I'm assuming, when you started acquiring these H100s, or you can tell me when. You're like, stock price is getting hammered, people are like, "What's happening with all this capex?" People aren't buying the metaverse, and presumably you're spending that capex to get these H100s. How back then did you know to get the H100s? How did you know we'll need the GPUs?

我想是因为我们在做 Reels。我们当时遇到的情况是，我们总是希望有足够的能力去构建一些我们暂时还看不到的东西。而我们在 Reels 上就遇到了这种情况，我们需要更多的 GPU 来训练模型。这对我们的服务来说是一个巨大的演变：不再只是对你关注的人、朋友或你关注的页面发布的内容进行排序，我们大力推动开始推荐我们所谓的“未连接内容”——也就是来自你没有关注的人或页面的内容。所以，我们可能展示给你的候选内容库从几千的数量级扩展到了几亿。这完全是不同的基础设施。我们开始做这件事，但受限于基础设施，无法像我们希望的那样快速赶上 TikTok 的做法。所以我当时看着这个情况想，“嘿，我们必须确保再也不会陷入这种境地。所以让我们订购足够的 GPU 来完成 Reels、内容排序和信息流所需的工作，但也要翻倍，对吧？”因为我们的常规原则是，地平线上总会有我们暂时看不到的东西。

I think it was because we were working on Reels. So we got into this situation where we always want to have enough capacity to build something that we can't quite see on the horizon yet. And we got into this position with Reels where we needed more GPUs to train the models, right? It was this big evolution for our services where instead of just ranking content from people you follow or your friends and whatever Pages you follow, we made this big push to basically start recommending what we call unconnected content—basically content from people or pages that you're not following. So now the corpus of content candidates that we could potentially show you expanded from on the order of thousands to on the order of hundreds of millions. So completely different infrastructure. And we started working on doing that, and we were constrained on basically the infrastructure that we had to catch up to what TikTok was doing as quickly as we would have wanted to. So I basically looked at that and I was like, "Hey, we have to make sure that we're never in this situation again. So let's order enough GPUs to do what we need to do on Reels and ranking content and feed, but let's also double that, right?" Because again, our normal principle is there's going to be something on the horizon that we can't see yet.

你当时知道会是 AI 吗？

Did you know it would be AI?

对我来说，要不要去尝试构建下一个东西根本就不是个问题。我就是忍不住要去做。有很多次我们想推出功能，然后苹果就说，“不，你们不能推出那个。”我当时想，“真糟糕。”我们为 AI 做好准备了吗？未来会有少数几家公司运行这些封闭模型，控制 API，从而能够决定你能构建什么。然后当你开始建造一个 300 兆瓦、500 兆瓦甚至 1 吉瓦的数据中心时——目前还没有人建成过单个 1 吉瓦的数据中心。无论你站在哪个位置，总会有某个你不信任的参与者拥有超级强大的 AI。我认为这潜在的风险要大得多。

That's not even a question for me whether we're going to go take a swing at building the next thing. I'm just incapable of not doing that. There's a bunch of times when we wanted to launch features and then Apple's just like, "Nope, you're not launching that." I was like, "That sucks." Are we set up for that with AI, where you're going to get a handful of companies that run these closed models that are going to be in control of the APIs and therefore are going to be able to tell you what you can build? Then when you start getting into building a data center that's like 300 megawatts or 500 megawatts or a gigawatt—just no one has built a single gigawatt data center yet. From wherever you sit, there's going to be some actor who you don't trust if they're the ones who have the super strong AI. I think that that's potentially a much bigger risk.

我们当时以为这会跟训练大模型有关，对吧？但那时我觉得可能更多是关于内容。不过这几乎就是模式匹配。经营公司总是有下一件事。我当时甚至不确定自己有没有做那种分析。我深陷于让推荐真正奏效，以及其他内容，因为这对 Instagram 和 Facebook 来说是巨大的突破——能够向用户展示他们未关注的人的有趣内容。事后看来，那是个非常好的决定。但那是源于落后。并不是我多么有远见。实际上，大多数时候我们做出后来看起来不错的决定，是因为之前搞砸了，只是不想重蹈覆辙。

We thought it was going to be something that had to do with training large models, right? I mean, at the time I thought it was probably going to be more something that had to do with content. But it's almost just pattern matching. Running the company, there's always another thing. I'm not even sure I had that analysis at the time. I was so deep in trying to get recommendations working for real, and other content, because that's such a big unlock for Instagram and Facebook—being able to show people content that's interesting to them from people they're not even following. That ended up being a very good decision in retrospect. But it came from being behind. It wasn't like I was so far ahead. Actually, most of the times we make a decision that ends up seeming good is because we messed something up before and just didn't want to repeat the mistake.

这完全是个题外话，但我确实想趁这个机会问一下。我们马上会回到 AI。所以你没有以 10 亿美元出售，但大概存在某个你愿意出售的价格，对吧？你心里有没有想过，‘我认为 Facebook 当时的实际估值是这样的，他们并没有给出正确的估值’？比如到了 5 万亿美元，你当然会卖。那么你是怎么考虑那个选择的？

This is a total detour, but I actually want to ask about this while we're on it. We'll get back to AI in a second. So you didn't sell for $1 billion, but presumably there's some amount you would have sold for, right? Did you write down in your head like, 'I think the actual valuation of Facebook at the time is this, and they're not getting the valuation right'? Like after $5 trillion, of course you would have sold. So how did you think about that choice?

我不知道。我觉得有些事情纯粹是个人因素。我当时并没有足够的金融素养来做那种分析。我身边有很多人都在论证 10 亿美元显然是多少年之后的事，远远领先于我们当时的位置。我并没有真正参与那种辩论的金融头脑。我只是内心深处相信我们在做的事情。我做了一些分析：‘好吧，如果我不做这个，我会去做什么？’我真的很喜欢构建东西，帮助人们沟通，理解人们的动态和人际关系。所以我想，如果我卖掉这家公司，我只会再去建一家类似的公司，而我喜欢我现在这家。所以为什么要卖呢？我认为人们做出的很多重大押注往往只是基于信念和价值观，而不是分析。试图向前连接点子的分析通常非常困难。

I don't know. I think some of these things are just personal. I wasn't sophisticated enough at the time to do that analysis. I had all these people around me making arguments for how a billion dollars was clearly so many years in the future, far ahead of where we were. I didn't really have the financial sophistication to engage with that debate. I just sort of deep down believed in what we were doing. I did some analysis: 'Okay, what would I go do if I wasn't doing this?' I really like building things, helping people communicate, and understanding what's going on with people and the dynamics between people. So I think if I sold this company, I'd just go build another company like this, and I kind of like the one I have. So why? I think a lot of the biggest bets people make are often just based on conviction and values, not analysis. It's usually very hard to do the analyses trying to connect the dots forward.

你拥有 Facebook AI Research 已经很长时间了。现在它似乎成了你们公司的核心。在什么时刻，打造 AGI——或者无论你怎么看待这个使命——在什么时刻这成了 Meta 的核心优先事项？

So you've had Facebook AI Research for a long time. Now it's become seemingly central to your company. At what point did making AGI—or however you consider that mission—at what point is that like a core priority of what Meta is doing?

这已经重要了一段时间。我们大约 10 年前启动了 FAIR，想法是，在通往通用智能或完整 AI 的道路上，会有各种不同的创新，这些创新将改善我们所做的一切。我们并没有把它当作一个产品来构思；它更像一个研究小组。在过去 10 年里，它创造了很多东西，改进了我们所有的产品，并推动了领域发展。但过去几年显然发生了巨大变化，ChatGPT 出现了，图像生成的扩散模型也出现了——这些非常疯狂的东西显然会影响人们与每个应用的交互方式。所以在那个时候，我们启动了第二个小组，即 Gen 小组，目标是将这些东西引入我们的产品——构建领先的基础模型来驱动所有这些不同的产品。最初的理论是，我们做的很多事情都是社交性的：帮助人们与创作者互动，帮助企业销售东西或做客户支持，为我们的应用、智能眼镜、VR 提供基本的助手功能。当时并不完全清楚你需要完整的 AGI 来支持这些用例。但通过实际工作，我们变得清楚：你需要。例如，对于 Llama 2，我们没有优先考虑编码，因为人们不会在 WhatsApp 上问 Meta AI 很多编码问题。但事实证明，编码对很多领域都很重要，不仅仅是编码本身。在编码上训练模型有助于它们更严谨，并在不同领域进行推理。所以对于 Llama 3，我们真的专注于用编码训练它，因为这让它在所有这些事情上变得更好，即使人们主要不问编码问题。推理是另一个例子。你可能想与创作者聊天或与客户互动。这种互动不是简单的回复；它是一个多步骤的互动，你需要思考如何实现对方的目标。通常客户并不确切知道他们在找什么，所以 AI 的工作不仅仅是回答问题——它需要更多思考。

It's been a big deal for a while. We started FAIR about 10 years ago, and the idea was that along the way to general intelligence or full AI, there are going to be all these different innovations that will improve everything we do. We didn't conceive it as a product; it was more of a research group. Over the last 10 years, it has created a lot of things that improved all our products and advanced the field. But there's obviously a big change in the last few years when ChatGPT came out, diffusion models on image creation came out—pretty wild stuff that is clearly going to affect how people interact with every app. So at that point, we started a second group, the Gen group, with the goal of bringing that stuff into our product—building leading foundation models to power all these different products. Initially, the theory was that a lot of what we're doing is pretty social: helping people interact with creators, businesses sell things or do customer support, basic assistant functionality for our apps, smart glasses, VR. It wasn't completely clear that you would need full AGI to support those use cases. But through working on them, it's become clear that you do. For example, with Llama 2, we didn't prioritize coding because people aren't going to ask Meta AI a lot of coding questions in WhatsApp. But it turns out that coding is important for a lot of domains, not just coding. Training models on coding helps them be more rigorous and reason across different domains. So for Llama 3, we really focused on training it with coding because that makes it better on all these things, even if people aren't primarily asking coding questions. Reasoning is another example. You might want to chat with a creator or interact with a customer. That interaction is not just a single reply; it's a multi-step interaction where you need to think through how to accomplish the person's goals. Often customers don't know exactly what they're looking for, so the AI's job isn't just to respond to the question—it needs to think about it more.

从整体来看，这实际上变成了一个推理问题。如果别人解决了推理问题，或者在推理上取得了重大进展，而我们还在做一个基本的聊天机器人，那我们的产品跟别人正在构建的东西相比就太逊色了。所以归根结底，我们意识到必须解决通用智能问题，于是我们提高了赌注和投资，确保能够做到这一点。

Holistically, it really becomes a reasoning problem. If someone else solves reasoning or makes good advances on reasoning, and we're sitting here with a basic chatbot, then our product is lame compared to what other people are building. So at the end of the day, we realized we've got to solve general intelligence, and we just kind of upped the ante and the investment to make sure we could do that.

能够为用户解决所有这些用例的 Llama 版本，会是强大到足以取代你这栋楼里可能有的程序员的那种版本吗？

Is the version of Llama that will solve all these use cases for users the version that will be powerful enough to replace a programmer you might have in this building?

我认为所有这些都会随着时间的推移逐步发展。就 Llama 而言，这个问题包含了很多层面。我不确定我们是在取代人，还是在给人提供工具去做更多事情。这栋楼里的程序员效率能提高 10 倍吗？我希望更多，但也不是。我不认为人类智能存在一个单一的阈值，因为人各有不同的技能。在某个时候，AI 可能会在大多数事情上超越人类，取决于模型有多强大，但这是渐进的。我不认为 AGI 是一件事；它本质上是不断增加不同的能力。

I think all this stuff is going to be progressive over time. In the case of Llama, there's a lot baked into that question. I'm not sure that we're replacing people as much as giving people tools to do more stuff. Is a programmer in this building 10x more productive? I would hope more, but no. I don't believe that there's a single threshold of intelligence for humanity, because people have different skills. At some point, AI is probably going to surpass people at most of those things, depending on how powerful the models are, but it's progressive. I don't think AGI is one thing; it's basically adding different capabilities.

多模态是我们现在关注的一个关键点，最初是照片、图像和文本，但最终会扩展到视频。而且因为我们非常专注于元宇宙，3D 类的东西也很重要。我特别关注的一个模态，但没看到行业里有多少其他人关注，就是情感理解。人脑有很大一部分专门用于理解他人、理解你的表情和情绪。我认为这本身就是一整个模态。你可以说它只是视频或图像，但它显然是这些模态中非常专门化的版本。所以有所有这些不同的能力，你希望训练模型去关注，同时还要在推理和记忆上做得更好——我认为记忆本身也是一整套东西。我不认为未来我们主要靠把东西塞进查询上下文窗口来问更复杂的问题。我认为会有不同的记忆存储或不同的定制模型，更个性化地服务于人。这些只是不同的能力。当然，还要让它们有大有小——我们两者都关心。如果你运行像 Meta AI 这样的东西，那主要是基于服务器的，但我们也希望它能在智能眼镜上运行，而智能眼镜空间有限，所以你需要高效的东西。

Multimodality is a key one we're focused on now, initially with photos, images, and text, but eventually with videos. And because we're so focused on the metaverse, 3D type stuff is important. One modality I'm pretty focused on that I haven't seen as many other people in the industry focus on is emotional understanding. So much of the human brain is dedicated to understanding people, your expressions and emotions. I think that's its own whole modality. You could say it's just video or image, but it's clearly a very specialized version of those too. So there are all these different capabilities you want to train the models to focus on, as well as getting a lot better at reasoning and memory, which I think is its own whole thing. I don't think we're going to be primarily shoving things into a query context window in the future to ask more complicated questions. I think there will be different stores of memory or different custom models that are more personalized to people. These are all just different capabilities. And obviously making them big and small—we care about both. If you're running something like Meta AI, that's pretty server-based, but we also want it running on smart glasses, and there's not a lot of space there, so you want something efficient for that.

如果你要做价值数百亿美元的推理，甚至最终是数千亿美元的推理，如果你在工业规模上使用智能，用例是什么？是模拟吗？是元宇宙中的 AI 吗？我们会用数据中心来做什么？

What is the use case if you're doing tens of billions of dollars worth of inference, or even eventually hundreds of billions of dollars worth of inference, if you're using intelligence at an industrial scale? Is it simulations? Is it the AI that will be in the metaverse? What will we be using the data centers for?

我们的赌注是，这将改变所有产品。会有一个 Meta AI 通用助手产品。它会从更像聊天机器人的东西——你问一个问题，它给出答案——逐渐演变成你给它越来越复杂的任务，然后它自己去完成。这会消耗大量的推理和算力。然后，我们做的很大一部分工作是为其他人——无论是企业还是创作者——与其他智能体进行交互。我的理论中很大一部分是，不会只有一个单一的 AI 与你互动。每个企业都会想要一个代表他们利益的 AI；他们不会想主要通过一个会推销竞争对手产品的 AI 来与你互动。创作者将是一个大领域。我们的平台上有大约 2 亿创作者，他们都有这样的模式：想要与社区互动，但受限于一天的时间，而他们的社区通常也想与他们互动，但也受限于时间。如果你能创造出一种东西，让创作者基本上拥有这个 AI，按照他们想要的方式训练它，并与他们的社区互动，我认为那也会非常强大。所以所有这些事情都会产生大量的互动。但这只是消费者用例。当你想到像 Chan Zuckerberg Initiative 这样的东西时，我们在科学上做了很多事情，显然有很多 AI 工作将推动科学、医疗保健等所有领域的发展。我认为这最终将影响产品和经济的几乎每个领域。

Our bet is that this is going to change all of the products. There's going to be a kind of Meta AI general assistant product. That will shift from something that feels more like a chatbot—you just ask a question and it formulates an answer—to things where you're increasingly giving it more complicated tasks and it goes away and does them. That's going to take a lot of inference and a lot of compute in other ways too. Then there's a big part of what we're going to do that is interacting with other agents for other people, whether it's businesses or creators. A big part of my theory is that there's not just going to be one singular AI that you interact with. Every business is going to want an AI that represents their interests; they're not going to want to primarily interact with you through an AI that will sell their competitors' products. Creators is going to be a big one. There are about 200 million creators on our platforms, all with the pattern where they want to engage their community but are limited by hours in the day, and their community generally wants to engage them but is also limited. If you could create something where a creator can basically own the AI, train it in the way they want, and engage their community, I think that's going to be super powerful too. So there's going to be a ton of engagement across all these things. But these are just the consumer use cases. When you think about stuff like the Chan Zuckerberg Initiative, we're doing a bunch of stuff on science, and there's obviously a lot of AI work that will advance science and healthcare and all these things too. I think this is going to end up affecting basically every area of the products and the economy.

你提到的那个能直接出去为你做多步骤事情的 AI——那是一个更大的模型吗？Llama 4 还会有一个 70B 的版本，但用正确的数据训练后就会非常强大吗？进展看起来是怎样的？是 Scaling（规模扩张）吗？还是像你之前说的，同样大小但不同的“银行”？

The thing you mentioned about an AI that can just go out and do something for you that's multi-step—is that a bigger model? Will Llama 4 still have a version that's 70B but just trained on the right data and be super powerful? How does the progression look? Is it scaling? Is it same size but different banks like you were talking about?

我不确定我们是否知道答案。一个似乎成为模式的事情是，你拥有 Llama 模型，然后围绕它构建一些特定于应用的代码。其中一些是针对用例的微调，但另一些只是关于模型应该如何与 Google 或 Bing 等工具集成以引入实时知识的逻辑。那不是基础 Llama 模型的一部分；那是应用的一部分。对于 Llama 2，我们有一些这样的东西，而且它更偏向手工工程。Llama 3 的部分目标是将更多这样的能力引入模型本身。但对于 Llama 3，随着我们开始涉足更多这类智能体式行为，我认为其中一些仍然会在应用层。

I don't know that we know the answer to that. One thing that seems to be a pattern is that you have the Llama model and then you build some other application-specific code around it. Some of it is fine-tuning for the use case, but some of it is just logic for how the model should integrate with tools like Google or Bing to bring in real-time knowledge. That's not part of the base Llama model; that's part of the application. For Llama 2, we had some of that and it was a little more hand-engineered. Part of our goal for Llama 3 was to bring more of that into the model itself. But for Llama 3, as we start getting into more of these agent-like behaviors, I think some of it will still be in the application layer.

其中一部分将更多依靠手工工程，而我认为 Llama 4 的目标是把更多能力内化到模型里。所以我觉得在每个阶段，你都会对地平线上可能出现的东西有所感知，然后开始摆弄它、破解它，这反过来又能帮你磨练直觉，知道该把什么能力训练进下一版模型。

of that is going to be more hand engineered and then I think our goal for Llama 4 will be to bring more of that into the model. So I think at each point, at each step along the way, you kind of have a sense of what's going to be possible on the horizon. You start messing with it and hacking around it, and then I think that that helps you hone your intuition for what you want to try to train into the next version of the model itself.

有意思，这让它更通用，因为显然任何手工编码的东西……你知道，可以解锁一些用例，但本质上很脆弱且不通用。

Interesting, which makes it more general, because obviously anything that you're hand coding is... you know, you can unlock some use cases, but it's just inherently brittle and non-general.

嘿，各位，我想快速介绍一个我希望更多应用能使用的工具。显然，你注意到每家公司都在往网站里塞 AI 聊天机器人，但作为用户，我通常觉得它们很烦人，因为给出的回答又长又泛，常常没用。Command Bar 是一个用户助手，你可以直接嵌入网站或应用，感觉就像在跟一个友好的人类客服聊天，他正陪着你、为你浏览。它比普通聊天机器人个性化得多：能查用户历史并据此给出不同回应，能用 API 执行操作，甚至能主动引导用户探索新功能。我觉得特别酷的一点是，它不只是输出文字，而是会说“来，我演示给你看”，然后开始和用户一起浏览。总之，它已经集成在很多优秀产品里了。你可以在 commandbar.com 了解更多。感谢他们赞助本期节目。现在回到 Mark。

Hey everybody, real quick I want to tell you about a tool that I wish more applications used. So obviously you've noticed every single company is trying to add an AI chatbot to their website, but as a user I usually find them really annoying 'cause they give these long generic, often useless answers. Command Bar is a user assistant that you can just embed into your website or application and it feels like you're talking to a friendly human support agent who is browsing with you and for you. And it's much more personalized than a regular chatbot. It can actually look up users' history and respond differently based on that. It can use APIs to perform actions. It can even proactively nudge users to explore new features. One thing that I think is really cool is that instead of just outputting text, Command Bar can kind of just say "here, let me show you" and start browsing alongside the user. Anyways, they're in a bunch of great products already. You can learn more about them at commandbar.com. Thanks to them for sponsoring this episode. And now back to Mark.

你说“内化到模型里”，意思是把想要的能力训练进模型本身。但“内化到模型里”具体指什么？

When you say "into the model itself", you train it on the thing that you want in the model itself. But what do you mean by "into the model itself"?

嗯，我是说，就像我举的例子：Llama 2 的工具使用非常特定，而 Llama 3 的工具使用能力就好得多。所以我们不需要手工编码所有东西让它用 Google 搜索，它自己就能做到。编码、运行代码之类的事情也一样。但一旦你有了这种能力，你就能瞥见“好了，接下来我们能做什么？”我不想等到 Llama 4 出来才开始构建那些能力，所以我们就开始破解它。于是你做了很多手工工程，这暂时让产品更好，但也指明了我们想尝试构建进下一版模型的方向。

Well, I mean, I think like the example that I gave: Llama 2, where you know, it's we really... I mean for Llama 2 the tool use was very, very specific, whereas Llama 3 has the ability to... has much better tool use, right. So we don't have to like hand code all the stuff to have it use Google to go do a search, it just kind of can do that. And similarly for coding and running code and just a bunch of stuff like that. But I think once you kind of get that capability, then you get a peek of "okay, well what can we start doing next?" Okay, well I don't necessarily want to wait until Llama 4 is around to start building those capabilities, so let's start hacking around it. And so you do a bunch of hand coding and that makes the products better for the interim, but then that also helps show the way of what we want to try to build into the next version of the model.

社区对 Llama 3 的微调中，你最兴奋的是哪一个？可能不是对你最有用那个，而是你最喜欢玩的那个。比如有人用古文献微调，然后你就像在跟维吉尔聊天一样。你对什么感到兴奋？

What is the community fine-tune of Llama 3 you're most excited by? Maybe not the one that will be most useful to you, but just... you'll just enjoy playing with it the most. They like fine-tuned it on antiquity and you'll just be like talking to Virgil or something. What are you excited about?

我不知道。我觉得这类东西的本质就是你会被惊喜到。所以任何我觉得有价值的具体东西，我们可能自己已经在做了。但我觉得你会看到蒸馏版本，更小的版本。我的意思是，80 亿参数对很多用例来说还不够小。我希望未来能有一个 10 亿参数模型或 20 亿参数模型，甚至 5 亿参数模型，看看能做什么。因为如果 80 亿参数已经几乎和最大的 Llama 2 模型一样强，那么 10 亿参数应该也能做出有趣的东西，而且更快。适合分类或很多基础任务，比如理解用户查询意图，然后喂给最强模型来优化提示词。所以我不知道，这可能是社区能帮忙填补的地方。但我们也在考虑自己来做一些蒸馏。不过现在 GPU 都在预训练 405B 模型。

I don't know. I think the nature of the stuff is it's like you get surprised, right. So I think any specific thing that I sort of thought would be valuable we'd probably be building ourselves. But I think you'll get distilled versions. I think you'll get kind of smaller versions. I mean, one thing that I think is 8 billion I don't think is quite small enough for a bunch of use cases, right. I think over time I'd love to get a 1 billion parameter model or a 2 billion parameter model, or even a 500 million parameter model and see what you can do with that. Because if with 8 billion parameters we're basically nearly as powerful as the largest Llama 2 model, then with a billion parameters you should be able to do something that's interesting, right? And faster. Good for classification or a lot of kind of basic things that people do before, kind of understanding the intent of a user query and feeding it to the most powerful model to hone what the prompt should be. So I don't know, I think that's one thing that maybe the community can help fill in. But we'll also... we're also thinking about getting around to distilling some of these ourselves. But right now the GPUs are... pre-training the 405.

所以你有这么多 GPU。我记得你说年底前有 35 万块？那是整个集群吗？

So you have all these GPUs. I think you said 350,000 by the end of the year? That's the whole fleet?

我们建了两个……我记得是 2.2 万或 2.4 万块 GPU 的集群，用于训练大模型。显然，我们做的很多事情中，很大一部分算力用于训练真实模型，比如 Facebook 信息流和 Instagram 信息流。推理对我们来说也是大事，因为我们服务大量用户。所以我们的推理算力与训练算力之比可能远高于其他做类似事情的公司，仅仅因为我们服务的社区规模巨大。

I mean, we built two... I think it's like 22,000 or 24,000 clusters that are kind of the single clusters that we have for training the big models. I mean, obviously across a lot of the stuff that we do, a lot of our stuff goes towards training like real models and like Facebook News Feed and Instagram Feed. And then inference is a huge thing for us because we serve a ton of people, right. So our ratio of inference compute required to training is probably much higher than most other companies that are doing this stuff, just because the sheer volume of the community that we're serving.

对，之前他们给我的资料里提到，你们训练用的数据量超过了单纯训练的计算最优值，因为推理对你们和社区来说太重要了，所以让模型包含数万亿 token 是合理的。

Yeah, that was really interesting in the material they shared with me before, that you trained it on more data than is compute optimal just for training, because the inference is such a big deal for you guys and also for the community, that it makes sense to just have this thing have trillions of tokens in there.

是的。不过有趣的是，即使在 70B 模型上，我们也发现它本应更早饱和。我们训练了大约 15 万亿 token，我们之前的预测是它会更快趋近渐近线，但直到最后它仍在学习。我们可能还能喂更多 token，它会变得更好。但作为公司，你需要做这些元推理问题：“好吧，我该怎么分配 GPU？”是继续训练这个 70B 模型，还是推进去做 Llama 4 的假设测试？所以我们得做这个决定，我认为对这个版本的 70B 模型来说，我们达到了合理的平衡。未来还会有其他版本，比如接下来会出的 70B 多模态模型。但确实，现在的架构能吸收这么多数据，这很迷人。

Yeah, yeah. Although one of the interesting things about it, we saw even with the 70 billion, is we thought it would get more saturated at... you know, we trained on around 15 trillion tokens. I guess our prediction going in was that it was going to asymptote more, but even by the end it was still learning, right. It's like we probably could have fed it more tokens and it would have gotten somewhat better. But I mean, at some point you're running a company, you need to do these meta reasoning questions of like "all right, how do I want to spend our GPUs?" On training this 70 billion model further, or do we want to get on with it so we can start testing hypotheses for Llama 4? So we kind of needed to make that call, and I think we got to a reasonable balance for this version of the 70 billion. There will be others in the future where you know, the 70 billion multimodal one that'll come over the next period. But yeah, I mean it was fascinating that you can just... that the architectures at this point can just take so much data.

对，这很有意思。那么这对未来模型意味着什么？你提到 Llama 3 8B 比 Llama 2 70B 更好？不，是几乎一样好。好吧，我不……但这是否意味着 Llama 的规模？是否意味着 Llama 4 70B 会像 Llama 3 405B 一样好？未来会怎样？这是个没人知道答案的大问题。

Yeah, that's really interesting. So what does this imply about future models? You mentioned that the Llama 3 8B is better than the Llama 2 70B? No, no, it's nearly as good. Okay, I don't... but does that mean like the Llama magnitude? Does that mean like the Llama 4 70B will be as good as the Llama 3 405B? Like what does the future look like? This is one of the great questions, right, that I think no one knows.

世界上最难规划的事情之一就是，当你有一条指数曲线时，它能持续多久？

One of the trickiest things in the world to plan around is when you have an exponential curve, how long does it keep going for?

是的，我认为它持续下去的可能性足够大，值得投入数百亿甚至上千亿美元来建设基础设施，假设如果这种趋势持续，你会得到一些真正了不起的东西，它们将催生出惊人的产品。但我不认为业内有人能确切告诉你 Scaling 会以那种速度继续下去。一般来说，历史上你会在某些点遇到瓶颈。现在这方面投入了如此多的精力，也许这些瓶颈会被很快突破，但我不确定。我认为这是一个有趣的问题：如果没有这些瓶颈，世界会是什么样子？假设进展就以这种速度继续下去，这似乎是有可能的。

Yeah, and I think it's likely enough that it will keep going that it is worth investing the tens or 100 billion plus in building the infrastructure to assume that if that kind of keeps going, you're going to get some really amazing things that are just going to make amazing products. But I don't think anyone in the industry can really tell you that it will continue scaling at that rate for sure. In general, in history, you hit bottlenecks at certain points. And now there's so much energy on this that maybe those bottlenecks get knocked over pretty quickly, but I don't know. I think that's an interesting question: what does a world look like where there aren't these bottlenecks? Suppose progress just continues at this pace, which seems plausible.

放远来看，会有不同的瓶颈，对吧？所以如果不是训练，那么……

Zooming out, there are going to be different bottlenecks, right? So if not training, then...

嗯，你说。

Yeah, go ahead.

嗯，我认为在过去几年里，有一个 GPU 生产的问题。即使有钱买 GPU 的公司也不一定能买到足够多的数量，因为存在各种供应限制。现在我觉得这种情况正在缓解。现在你看到很多公司在考虑真正投入大量资金来建设这些东西，我认为这会持续一段时间。我认为有一个资本问题：在什么节点上投入资本不再值得？但我实际上认为，在达到那个点之前，你会遇到能源限制。我认为还没有人建成一个千兆瓦级的单一训练集群。然后你会遇到这些在现实世界中进展缓慢的事情，比如获得能源许可是一个高度受监管的政府职能。所以从软件——它在一定程度上受监管，我认为它比很多科技界人士感觉到的更受监管，尽管如果你创办一家小公司，感受可能不同——到能源，如果你要建设大型新电厂或大型设施，然后建设穿越其他私有或公共土地的输电线路，那是一个高度受监管的事情。所以你需要很多年的准备时间。如果我们想建立一个大型设施来提供电力，我认为那是一个长期项目。我不认为这可以像魔法一样，只要有了 AI 和大量资本投入进去，模型就会突然……我认为你会在过程中遇到不同的瓶颈。

Well, I think at some point over the last few years, there's this issue of GPU production. Even companies that had the money to pay for the GPUs couldn't necessarily get as many as they wanted because there were all these supply constraints. Now I think that's sort of getting less so. Now you're seeing a bunch of companies think about really investing a lot of money in building out these things, and I think that will go for some period of time. I think there is a capital question of at what point does it stop being worth it to put the capital in. But I actually think before we hit that, you're going to run into energy constraints. I don't think anyone's built a gigawatt single training cluster yet. And then you run into these things that just end up being slower in the world, like getting energy permitted is a very heavily regulated government function. So you're going from software, which is somewhat regulated—I'd argue it's more regulated than a lot of people in the tech community feel, although it's obviously different if you're starting a small company—to energy. If you're talking about building large new power plants or large builds and then building transmission lines that cross other private or public land, that is just a heavily regulated thing. So you're talking about many years of lead time. If we wanted to stand up some massive facility to power that, I think that is a very long-term project. I don't think this is something that can be quite as magical as just getting a level of AI and a bunch of capital and putting it in, and then all of a sudden the models are just going to kind of... I think you do hit different bottlenecks along the way.

有没有一个项目，也许相关也许不相关，即使是像 Meta 这样的公司也没有资源去做？比如如果你的研发预算或资本支出预算是现在的 10 倍，那么你就可以去追求它——它在你脑海里——但今天的 Meta，也许你甚至无法通过发行股票或债券来筹集资金，它只是比你预算大 10 倍。

Is there a project, maybe related or not, that even a company like Meta doesn't have the resources for? Like if your R&D budget or your capex budget was 10x what it is now, then you could pursue it—it's in the back of your mind—but Meta today, maybe you can't even issue stock or bond for it, it's just 10x bigger than your budget.

嗯，我认为能源是一个方面。我想如果我们能获得能源，我们可能会建设比目前更大的集群。所以我认为这从根本上来说最终是资金瓶颈。比如如果你有一万亿美元，我认为是时间问题。这取决于指数曲线能走多远。很多公司正在建设 50 兆瓦或 100 兆瓦规模的数据中心，大的可能达到 150 兆瓦。你拿整个数据中心，装满训练所需的一切，建成你能建的最大集群。我认为很多公司都在做类似的事情。但当你开始建设 300 兆瓦、500 兆瓦甚至千兆瓦的数据中心时，还没有人建成一个千兆瓦级的数据中心。我认为这会发生，只是时间问题，但不会是明年。其中一些事情需要几年时间才能建成。打个比方，千兆瓦大约相当于一个像样的核电站的规模，而且只用于训练一个模型。

Well, I think energy is one piece. I think we would probably build out bigger clusters than we currently can if we could get the energy to do it. So I think that's fundamentally money-bottlenecked in the limit. Like if you had a trillion dollars, I think it's time. It depends on how far the exponential curves go. A number of companies are working on data centers on the order of 50 megawatts or 100 megawatts, or a big one might be 150 megawatts. You take a whole data center and fill it up with everything you need for training and build the biggest cluster you can. I think a bunch of companies are running stuff like that. But when you start getting into building a data center that's 300 megawatt or 500 megawatt or a gigawatt, no one has built a single gigawatt data center yet. I think it will happen, it's only a matter of time, but it's not going to be next year. Some of these things will take some number of years to build out. To put this in perspective, a gigawatt is around the size of a meaningful nuclear power plant, only going towards training a model.

亚马逊不是做了吗？他们有一个 950 兆瓦的……

Didn't Amazon do this? They have a 950 megawatt...

我不太确定他们做了什么。你得问他们。但不必在同一个地方，对吧？如果分布式训练可行，它可以分布进行。我认为这是一个大问题：这到底会如何运作。而且我确实认为，在未来，我们称之为这些大模型训练的东西，很可能更多地是沿着推理的方向——生成合成数据然后输入模型。我不知道这个比例会是多少，但我认为生成合成数据在今天更像是推理而不是训练。显然，如果你是为了训练模型而做，那它是更广泛训练过程的一部分。所以这是一个开放问题：其中的平衡点在哪里，以及它会如何发展。

I'm not exactly sure what they did. You'd have to ask them. But it doesn't have to be in the same place, right? If distributed training works, it can be distributed. I think that is a big question: how that's going to work. And I do think in the future, it seems quite possible that more of what we call training for these big models is actually more along the lines of inference—generating synthetic data to then feed into the model. I don't know what that ratio is going to be, but I consider the generation of synthetic data to be more inference than training today. Obviously, if you're doing it in order to train a model, it's part of the broader training process. So that's an open question: where the balance of that is and how that plays out.

如果是这样，Llama 3 以及可能从 Llama 4 开始是否也会出现这种情况？你发布模型后，如果有人拥有大量算力，他们就可以利用你发布的模型，不断让这些东西变得任意更智能？比如某个科威特、阿联酋或某个随机国家拥有大量算力，他们就可以直接用 Llama 4 做出更智能的东西。

If that's the case, would that potentially also be the case with Llama 3 and maybe Llama 4 onwards, where you put this out and if somebody has a ton of compute, then using the models that you've put out, you can just keep making these things arbitrarily smarter? Like some Kuwait or UAE or some random country has a ton of compute and they can just use Llama 4 to make something much smarter.

我确实认为会出现这样的动态。但我也认为在网络架构或模型架构上存在一个根本限制。所以我认为我们用 Llama 3 架构训练的 700 亿参数模型可以变得更好——它可以继续提升，就像我说的。它不是……

I do think that there are going to be dynamics like that. But I also think that there is a fundamental limitation on the network architecture, or the model architecture. So I think a 70 billion model that we trained with the Llama 3 architecture can get better—it can keep going, like I was saying. It's not...

我们觉得，如果持续喂入更多数据，或者把高价值 token 再轮转一遍，它就会继续变好。我们也看到全球很多其他公司，基本上拿 Llama 2 700 亿参数的基础模型，采用那个模型架构，然后构建一个新模型。但当你对 Llama 3 700 亿或 Llama 3 450 亿做出代际改进时，目前还没有任何开源的东西能与之相比。这是一个很大的阶跃函数，人们能在此基础上构建的东西，我不认为可以无限延伸。在到达下一个阶跃之前，可以做一些优化。

We felt like if we kept on feeding it more data or rotated the high value tokens through again, then it would continue getting better. And we've seen a bunch of other people around the world, different companies, basically take the Llama 2 70 billion base, take that model architecture and build a new model. It's still the case that when you make a generational improvement to the kind of Llama 3 70 billion or the Llama 3 45, there's nothing open source anything like that today. It's a big step function, and what people are going to be able to build on top of that, I don't think can go infinitely from there. I think there can be some optimization until you get to the next step function.

好，那我们稍微拉远一点，不谈具体模型，也不谈审批能源所需的多年提前量。宏观来看，未来这几十年，AI 正在发生什么？它感觉像是另一种技术，比如元宇宙或社交网络，还是感觉像是人类历史进程中一个根本不同的东西？

Okay, so let's zoom out a little bit from specific models and even the multi-year lead times you would need to get energy approvals and so on. Big picture, these next couple of decades, what's happening with AI? Does it feel like another technology like metaverse or social, or does it feel like a fundamentally different thing in the course of human history?

我认为它会非常根本。我觉得它更像当初计算技术的诞生。你会得到所有这些新应用，就像当初有了网络或手机时，人们基本上重新思考了所有体验，很多以前不可能的事现在变得可能。所以我认为这会发生，但我觉得它是一种更底层的创新。我的感觉是，它更像是从人们没有电脑到人们有电脑。但具体会如何发展也很难说。我倾向于认为，在宇宙尺度上，显然它会在几十年内快速发生，但我确实认为有一些人担心它会突然从有点智能变成极其智能。我只是觉得有各种物理约束使得这不太可能发生。我真的看不到那种情况。所以我认为我们会有一些时间来适应，但它会真正改变我们的工作方式，并给人们提供各种创意工具去做不同的事情。我认为它会让人们更能做自己想做的事，这是我的看法。

I think it's going to be pretty fundamental. I think it's going to be more like the creation of computing in the first place. You'll get all these new apps in the same way that when you got the web or you got mobile phones, people basically rethought all these experiences and a lot of things that weren't possible before now became possible. So I think that will happen, but I think it's a much lower level innovation. It's going to be more like going from people didn't have computers to people have computers, is my sense. But it's also very hard to reason about exactly how this goes. I tend to think that on the cosmic scale, obviously it'll happen quickly over a couple of decades or something, but I do think that there are some set of people who are afraid that it really just spins and goes from being somewhat intelligent to extremely intelligent overnight. I just think that there are all these physical constraints that make that unlikely to happen. I don't really see that playing out. So I think we'll have time to kind of acclimate a bit, but it will really change the way that we work and give people all these creative tools to do different things. I think it's going to really enable people to do the things that they want a lot more, is my view.

好，也许不是一夜之间。但你的观点是，在宇宙尺度上，如果你认为人类进化了，然后 AI 出现了，然后他们走向了银河系，也许需要几十年，也许需要一个世纪，但这就是现在历史正在发生的大图景吗？

Okay, so maybe not overnight. But is it your view that on a cosmic scale, if you think humans evolved and then AI happened and then they went out through the galaxy, maybe it takes many decades, maybe it takes a century, but is that the grand scheme of what's happening right now in history?

抱歉，从什么意义上说？

Sorry, in what sense?

我的意思是，有像计算机甚至火这样的其他技术，但 AI 的发生与人类最初进化一样重要。

I mean in the sense that there were other technologies like computers and even like fire, but the AI happening is as significant as humans evolving in the first place.

我觉得这很微妙。我认为人们倾向于认为人性的某些方面在各方面都很独特，然后接受事实并非如此，但人类实际上仍然非常特别。就像我们曾认为地球是宇宙的中心，但它不是，但人类仍然很棒、很独特。我认为人们还有另一个偏见，就是认为智能在某种程度上与生命根本相连，但这并不明确。我不确定我们对意识或生命有足够清晰的定义来充分探讨这个问题。但有很多科幻作品说，创造智能后它就开始表现出各种类人行为。我实际上认为，目前这些东西的形态至少感觉是朝着智能可以与意识和能动性相当分离的方向发展，这我认为只会让它成为一个非常有价值的工具。所以我不知道。显然，很难预测这些东西会随时间走向何方，这就是为什么我认为任何人都不应该对开发计划或行动计划教条。我认为你应该看每一次发布。我们显然非常支持开源，但我没有承诺我们会发布我们做的每一件事。但我总体上非常倾向于认为开源对社区有好处，对我们也有好处，因为我们会从创新中受益。但如果某个时候，这个东西的能力发生了质变，我们觉得开源它不负责任，那我们就不开源。所以一切都很难预测。

I think that's tricky. I think people like to think that certain aspects of humanity are really unique in different ways, and then come to grips with the fact that that's not true, but humanity is actually still super special. It's like we thought that the Earth was the center of the universe and it's not, but humans are still pretty awesome and pretty unique. I think that another bias that people tend to have is thinking that intelligence is somehow fundamentally connected to life, and it's not actually clear that it is. I don't know that we have a clear enough definition of consciousness or life to fully interrogate this. But there's all this science fiction about creating intelligence and it starts taking on all these human-like behaviors. I actually think that the current incarnation of all this stuff at least kind of feels like it's going in a direction where intelligence can be pretty separated from consciousness and agency, which I think just makes it a super valuable tool. So I don't know. Obviously it's very difficult to predict what direction the stuff goes in over time, which is why I don't think anyone should be dogmatic about how they plan to develop it or what they plan to do. I think you want to look at each release. We're obviously very pro open source, but I haven't committed that we're going to release every single thing that we do. But I'm generally very inclined to thinking that open sourcing it is going to be good for the community and also good for us, because we'll benefit from the innovations. But if at some point there's some qualitative change in what the thing is capable of and we feel like it's just not responsible to open source it, then we won't. So it's all very difficult to predict.

是啊。什么样的质变？比如你在训练 Llama 5、Llama 4 时看到了某个具体的东西，然后你想，‘我不确定是否要开源它。’

Yeah. What is a kind of qualitative change? Like a specific thing you're training Llama 5, Llama 4, and you've seen this and you're like, 'I'm not sure about open sourcing it.'

我认为抽象地回答这个问题有点难，因为任何产品都可能表现出负面行为，只要你能缓解它，就没问题。社交媒体有不好的地方，我们努力去缓解。Llama 2 也有不好的地方，我们花了很多时间确保它不会帮助人们实施暴力行为之类。这并不意味着它是一种自主或智能体；只是意味着它学到了很多关于世界的知识，可以回答一系列我们认为它回答无益的问题。所以我不知道。我认为问题不在于它会表现出什么行为，而在于它表现出这些行为后，我们无法缓解什么。我认为一件事物有好有坏的方式太多了，很难事先全部列举出来。即使看看我们在社交媒体上必须处理的问题，以及我们遇到的各种类型的伤害，大概有 18 或 19 类人们做的有害事情，我们基本上建立了 AI 系统来尝试识别人们正在做的那些事情，并尽量确保它们不会在我们的网络上发生。

I think that it's a little hard to answer that in the abstract, because there are negative behaviors that any product can exhibit that as long as you can mitigate it, it's okay. There's bad things about social media that we work to mitigate. There's bad things about Llama 2 that we spend a lot of time trying to make sure that it's not helping people commit violent acts or things like that. That doesn't mean that it's a kind of autonomous or intelligent agent; it just means that it's learned a lot about the world and it can answer a set of questions that we think it would be unhelpful for it to answer. So I don't know. I think the question isn't really what behaviors would it show, it's what things would we not be able to mitigate after it shows that. I think that there are so many ways in which something can be good or bad that it's hard to actually enumerate them all upfront. If you even look at what we've had to deal with in social media and the different types of harms we've basically gotten to, it's like there's 18 or 19 categories of harmful things that people do, and we've basically built AI systems to try to go identify what those things are that people are doing and try to make sure that that doesn't happen on our network as much as possible.

我认为随着时间的推移，你可以将其进一步细分为一个分类体系，这也是我们花时间研究的事情，因为我们想确保理解这一点。所以我问马克的一个问题是，LLM 的工业级规模使用会是什么样子。在之前的技术革命中，你看到人们起初以非常小的规模思考能实现什么，我认为对于 LLM 来说，聊天机器人可能就是那样。而大规模的使用案例可能类似于 V7 Go——顺便说一句，它由 V7 Labs 制作，他们是本集的赞助商。它就像一个电子表格：你输入原始信息，比如文档、图像等，它们变成行，列则由你选择的 LLM 填充。事实上，我用它来准备与马克的对话，我输入了 Meta AI 研究的一系列博客文章和论文，如果你在 YouTube 上观看，它会总结并提取出我想要的列信息。显然我的用例很小，但你可以想象，比如像联邦快递这样的公司每天要处理 50 万份文档。显然聊天机器人做不到，但电子表格可以，因为这就像一股智能的洪流，对吧？无论如何，你可以在 V7labs.com/slgo 或描述中的链接了解更多。回到马克。

I think you can over time break this down into more of a taxonomy too, and this is something we spend time researching because we want to make sure we understand that. So one of the things I asked Mark is what industrial-scale use of LLMs would look like. You see this in previous technological revolutions where at first they're thinking in a very small-scale way about what's enabled, and I think that's what chatbots might be for LLMs. And I think the large-scale use case might look something like what V7 Go is—by the way, made by V7 Labs, who is sponsoring this episode. So it's like a spreadsheet: you put in raw information like documents, images, whatever, and they become rows, and the columns are populated by an LLM of your choice. In fact, I used it to prepare for Mark, so I fed in a bunch of blog posts and papers from Meta's AI research, and as you can see if you're on YouTube, it summarizes and extracts exactly the information I want as columns. Obviously mine is a small use case, but you can imagine, for example, a company like FedEx has to process half a million documents a day. Obviously a chatbot can't do that; a spreadsheet can, because this is just like a fire hose of intelligence in there, right? Anyway, you can learn more about them at V7labs.com/slgo or the link in the description. Back to Mark.

是的，在我看来这是个好主意。如果未来 AI 系统没有被广泛部署，每个人都无法使用它们，我会感到失望。与此同时，我想更好地理解缓解措施。如果缓解措施是微调，那么开放权重的全部意义就在于你可以移除微调——这些微调往往只是能力之上的表面功夫。比如，在 Slack 上与一位生物学研究员交谈——再说一次，我认为模型离这个还很远，它们现在就像谷歌搜索——但就像我可以给他们看我的培养皿，他们会解释‘为什么你的天花样本没有生长，这里需要改什么’。你怎么缓解这个？因为有人可以直接把那个微调加进去，对吧？

Yeah, it seems to me it would be a good idea. I would be disappointed in a future where AI systems aren't broadly deployed and everybody doesn't have access to them. At the same time, I want to better understand the mitigations. If the mitigation is fine-tuning, well the whole thing about open weights is that you can then remove the fine-tuning, which is often superficial on top of these capabilities. Like if it's talking on Slack with a biology researcher—and again, I think models are very far from this, they're right now like Google Search—but it's like I can show them my Petri dish and they can explain, 'Here's why your smallpox sample didn't grow, here's what to change.' How do you mitigate that, because somebody can just fine-tune that in there, right?

是的，我是说这确实是真的。我认为很多人基本上会使用现成的模型，而一些心怀不轨的人会试图剥离所有不好的东西。所以我确实认为这是个问题。另一面是——这也是我在哲学上如此支持开源的原因之一——我认为未来 AI 的集中化可能与它的广泛传播一样危险。所以很多人会想，‘好吧，如果我们能做这些事，把它放到野外广泛可用是不是不好？’我认为另一个版本是，‘好吧，一个机构拥有比其他人强大得多的 AI 可能也很糟糕，’对吧？所以如果你看看——我想起的一个安全类比是，你知道，不需要 AI 就能——好吧，很多东西都有安全漏洞，如果你能回到一两年前，对吧，那不算 AI，就像你只是比现在多知道一两年的安全漏洞知识，你几乎可以入侵任何系统，对吧？所以相信一个非常智能的 AI 可能能够识别一些漏洞，并像一个能回到一两年前的人类一样破坏所有这些系统，这并不牵强。好吧，那么作为社会我们是如何应对的呢？一个重要的部分是开源软件，它使得当软件得到改进时，不会只停留在某一家公司的产品中，而是可以广泛部署到许多不同的系统，无论是银行、医院还是政府机构。而且，随着软件变得更强——因为更多人能看到它、更多人能测试它，并且有关于它如何工作的标准——世界可以一起快速升级。我有点认为，一个 AI 被广泛部署、随着时间的推移逐步加固、并且所有不同系统都相互制衡的世界，从根本上来说比一个更集中的世界更健康。所以风险无处不在，但我认为这是一个人们谈论得不够多的风险。我认为有一种风险是，‘好吧，如果 AI 系统做了坏事怎么办？’我更——你知道，我更担心的是，‘如果某个你不信任的行为者——无论你站在哪一边，总会有你不信任的行为者——如果他们拥有超级强大的 AI，无论是某个与我们国家敌对的政府，还是你不信任的公司，或者其他什么——我认为那可能是一个更大的风险，因为他们可能拥有别人没有的武器，推翻我们的政府，造成巨大混乱，对吧？我认为直觉是这些东西最终会变得非常重要，对经济、安全和其他方面都很有价值。我不知道，我只是觉得，如果某个你不信任或与你敌对的人得到了更强大的东西，那可能会是个问题。我认为缓解这个问题的最好方法可能是拥有良好的开源 AI，它基本上成为标准，并在很多方面成为领导者，这样就能确保一个更加公平和平衡的竞争环境。

Yeah, I mean that's true. I think a lot of people will basically use the off-the-shelf model, and some people who have basically bad faith are going to try to strip out all the bad stuff. So I do think that that's an issue. The flip side of this is—and this is one of the reasons why I'm kind of philosophically so pro open source—is I do think that a concentration of AI in the future has the potential to be as dangerous as it being widespread. So I think a lot of people, they think about the questions of, 'Okay, well if we can do this stuff, is it bad for it to be out in the wild, just kind of widely available?' I think another version of this is, 'Okay, well it's probably also pretty bad for one institution to have an AI that is way more powerful than everyone else's AI,' right? So if you look at—like I guess one security analogy that I think of is, you know, it doesn't take AI to basically—okay, there are security holes in so many different things, and if you could travel back in time a year or two years, right, it's like that's not AI, it's like you just have, let's say, one year or two years more knowledge of the security holes, you pretty much hack into any system, right? So it's not that far-fetched to believe that a very intelligent AI would probably be able to identify some holes and basically be like a human who could potentially go back in time a year or two and compromise all these systems. Okay, so how have we dealt with that as a society? Well, one big part is open source software that makes it so that when improvements are made to the software, it doesn't just get stuck in one company's products, but it can be broadly deployed to a lot of different systems, whether it's banks or hospitals or government stuff. And like, just everyone can—as the software gets hardened, which happens because more people can see it and more people can bang on it, and there are standards on how the stuff works—the world can get upgraded together pretty quickly. And I kind of think that a world where AI is very widely deployed in a way where it has gotten hardened progressively over time, and is one where all the different systems will be in check, seems fundamentally more healthy to me than one where this is more concentrated. So there are risks on all sides, but I think that that's one risk that I don't hear people talking about quite as much. I think there's sort of the risk of, 'Okay, well what if the AI system does something bad?' I am more—you know, I stay up at night more worrying, 'Well what if some actor—whatever it is, from wherever you sit, there's going to be some actor who you don't trust—if they're the ones who have the super strong AI, whether it's some other government that is sort of an opponent of our country, or some company that you don't trust, or whatever it is—I think that that's potentially a much bigger risk, as in they could overthrow our government because they have a weapon that nobody else has, cause a lot of mayhem, right? I think the intuition is that this stuff ends up being pretty important and valuable for both economic and security and other things. And I don't know, I just think yeah, if someone who you don't trust or is an adversary of you gets something that is more powerful, then I think that that could be an issue. And I think the probably best way to mitigate that is to have good open source AI that basically becomes the standard, and in a lot of ways can become the leader, and in that way it just ensures that it's a much more even and balanced playing field.

是的，这对我来说似乎合理，如果那样的话，那将是我更喜欢的未来。我想从机制上理解，如果有人要用 AI 系统制造混乱，世界上有其他开源系统如何能阻止这一点。比如有人带着生物武器来的具体例子——是不是只是我们在世界其他地方做大量研发来快速找到疫苗？会发生什么？如果你拿我刚才说的计算机安全例子，我认为一个较弱的 AI 试图入侵一个由较强 AI 保护的系统，成功率会更低，对吧？所以我认为那是——就如何而言——你知道，世界上的一切都是这样吗？比如如果生物武器不是这样呢？

Yeah, that seems plausible to me, and if that works out, that would be the future I prefer. I guess I want to understand mechanistically how, if somebody was going to cause mayhem with AI systems, the fact that there are other open source systems in the world prevents that. Like the specific example of somebody coming with a bioweapon—is it just that we'll do a bunch of R&D in the rest of the world to figure out vaccines really fast? What's happening? If you take the computer security one that I was talking about, I think someone with a weaker AI trying to hack into a system that is protected by a stronger AI will succeed less, right? So I think that that's—in terms of how—you know, everything in the world is like that? Like what if bioweapons aren't like that?

不，我是说我不知道世界上的一切都是这样。我认为——

No, I mean I don't know that everything in the world is like that. I think that—

我想生物武器是人们最担心的领域之一，关注这个问题的人主要聚焦于此。我认为思考这一点很有道理。而且有一些缓解措施可以尝试，比如不让模型学习某些知识。但某种程度上，如果你遇到一个足够恶劣的行为者，又没有其他 AI 能够制衡他们、了解事态和威胁，那就会构成风险。所以我认为这是我们需要警惕的事情之一。

That's I guess one of the bioweapons are one of the areas where I think the people who are most worried about this stuff are focused. And I think that makes a lot of sense to think about that. And I think that there are certain mitigations you can try, like not training certain knowledge into the model. But yeah, at some level, if you get a sufficiently bad actor and you don't have other AI that can sort of balance them and understand what's going on and what the threats are, then that could be a risk. So I think that's one of the things that we need to watch out for.

在这些系统的部署中，你是否能看到这样的情况：比如你在训练 Llama 4，它对你撒谎了，因为它以为你没注意到，然后你惊呼“哇，这是怎么回事？”——不是说 Llama 4 很可能这样，但你能想象类似的情况吗？你会非常担心欺骗性，尤其是当数十亿份副本散布在现实世界中时？

Is there something you could see in the deployment of these systems where you observe, like you're training Llama 4 and it lied to you because you thought you weren't noticing or something, and you're like, whoa, what's going on here? Not that this is likely with Llama 4, but is there something you can imagine like that where you'd be really concerned about deceptiveness, and if billions of copies of things are out in the wild?

嗯，我的意思是，这不一定……我是说，现在我们看到很多幻觉。所以我认为更多的是……如何区分幻觉和欺骗，这是一个有趣的问题。但确实，我认为有很多风险和事情需要考虑。另一方面，在经营公司时，我至少试图平衡这些长期的理论风险与我眼中今天真实存在的风险。所以当你谈到欺骗时，我最担心的形式是人们利用它生成虚假信息，然后通过我们的网络或其他渠道传播。我们打击大量有害内容的方式，就是构建比对抗方更聪明的 AI 系统。我想这在一定程度上构成了我的理论。如果你看看人们通过社交网络造成或试图造成的不同类型的伤害，有些并不那么具有对抗性。例如，仇恨言论我认为不是超级对抗性的，因为人们并没有变得更擅长种族歧视；他们只是……这是一个我认为 AI 通常比人类在这些问题上进步得更快、更复杂的领域。所以我们两方面都有问题：人们做坏事，无论是试图煽动暴力还是什么，但我们也有很多误报，即我们审查了本不该审查的内容，这无疑让很多人感到恼火。所以我认为 AI 在这方面越来越精准，长期来看是好事。但让我再举一个例子：国家试图干预选举。这是一个他们绝对使用尖端技术、每年都在进步的案例。我们封堵一种技术，他们学到我们的做法，然后用另一种技术来攻击我们。这不像一个人说些难听的话；他们是有目标的、老练的、拥有大量技术的。在这些情况下，我仍然认为让我们的 AI 系统以比他们更快的速度提升复杂程度是一场军备竞赛，但至少目前我们正在赢得这场竞赛。所以我不知道。我花了很多时间思考的是：是的，无论是 Llama 4、Llama 5 还是 Llama 6，我们都需要思考我们观察到的行为。而且不只是我们。我认为开源的部分原因是有很多其他人也在研究这个。所以是的，我们想看看别人观察到了什么，我们观察到了什么，我们能缓解什么，然后我们会评估是否可以开源。但我认为在可预见的未来，我乐观地认为我们可以做到。短期内，我不想忽视人们今天试图用模型做的实际坏事，即使它们不是生存威胁，但也是相当严重的、我们在运营服务中熟悉的日常伤害。这实际上也是我们需要花大量时间处理的事情。

Yeah, I mean, I think that's not necessarily... I mean, right now we see a lot of hallucinations. So I think it's more that... I think it's an interesting question how you would tell the difference between a hallucination and deception. But yeah, I look, I think there's a lot of risks and things to think about. The flip side of all this is that there are also a lot of... I try, in running our company at least, to balance what I think of as these longer-term theoretical risks with what I actually think are quite real risks that exist today. So like when you talk about deception, the form of that that I worry about most is people using this to generate misinformation and then pump that through whether it's our networks or others. So the way that we've basically combated a lot of the type of harmful content is by building AI systems that are smarter than the adversarial ones. And I guess this is part of what informs my theory on this. If you look at the different types of harm that people do or try to do through social networks, there are ones that are not very adversarial. For example, hate speech I would say is not super adversarial in the sense that people aren't getting better at being racist; they're just... it's like, okay, if you... that's one where I think the AIs are generally just getting way more sophisticated faster than people are at those issues. So we have issues both ways: people do bad things, whether they're trying to incite violence or something, but we also have a lot of false positives, where we basically censor stuff that we shouldn't, and I think understandably make a lot of people annoyed. So I think having an AI that just gets increasingly precise on that is going to be good over time. But let me give you another example: nation states trying to interfere in elections. That's an example where they are absolutely using cutting-edge technology and absolutely get better each year. So we block some technique, they learn what we did, they come at us with a different technique. It's not like a person trying to say mean things; it's that they have a goal, they're sophisticated, they have a lot of technology. In those cases, I still think the ability to have our AI systems grow in sophistication at a faster rate than theirs is an arms race, but I think we're at least currently winning that arms race. So I don't know. I think that's a lot of the stuff that I spend time thinking about: okay, yes, it is possible that whether it's Llama 4 or Llama 5 or Llama 6, we need to think about what behaviors we're observing. And it's not just us. I think part of the reason why you make this open source is that there are a lot of other people who study this too. So yeah, we want to see what other people are observing, what we're observing, what we can mitigate, and then we'll make our assessment on whether we can make it open source. But I think for the foreseeable future, I'm optimistic we will be able to. And in the near term, I don't want to take our eye off the ball of what are actual bad things that people are trying to use the models for today, even if they're not existential but they're pretty bad, kind of day-to-day harms that we're familiar with in running our services. That's actually a lot of what we have to spend our time on as well.

是的，是的。实际上，我对合成数据这件事非常好奇。我感兴趣的是，为什么你认为当前模型不会……如果它们变得更聪明，并且你使用了论文或即将发布的博文中提到的技术——即选择最正确的思维链——那么反复使用合成数据可能会有一个渐近线。为什么这不会导致一个循环？当然不会一夜之间，但经过数月或数年的训练，一个更聪明的模型可能会变得更聪明，产生更好的输出，然后变得更聪明，如此循环。

Yeah, yeah. Actually, I found the synthetic data thing really curious. I'm actually interested in why you don't think, like current models... it makes sense why there might be an asymptote with just doing the synthetic data again and again if they get smarter and you use the kind of techniques you talk about in the paper or the blog post that's coming out on the day this will be released, where it goes to the thought chain that is the most correct. Why this wouldn't lead to a loop that over, of course it wouldn't be overnight, but over many months or years of training potentially with a smarter model, it gets smarter, makes better output, gets smarter, and so forth.

嗯，我认为在模型架构的参数范围内是可能的。只是，在某种程度上，我不知道。我认为今天的十亿参数模型，我不认为你能达到与最先进的、融合了新架构研究的数千亿参数模型一样好。但那些也会开源，对吧？嗯，是的，但我想这取决于我们刚才讨论的所有问题。但是的，我们希望如此。但我认为在每一个节点上，我不知道。就像构建软件时，你可以用软件做很多事情，但在某种程度上你受到运行它的芯片的限制。所以总是会有不同的物理约束。模型的大小会受到你能获取并用于推理的能量的限制。所以我猜我同时非常乐观，认为这些东西会继续快速改进，同时也比一些人认为的更谨慎一些，关于失控的情况。我只是不认为失控是一个特别可能的情况。我认为保持开放的选择是有意义的。我们不知道的东西太多了。有一种情况是，保持权力平衡非常重要，这样就没有人会成为极权独裁者。还有一种情况是，你不想开源……

Well, I think it could within the parameter of whatever the model architecture is. It's just that, at some level, I don't know. I think today's billion-parameter models, I just don't think you're going to be able to get to be as good as the state-of-the-art multi-hundred-billion-parameter models that are incorporating new research into the architecture itself. But those will be open source as well, right? Well, yeah, but I think that's subject to all the questions we just talked about. But yes, I mean, we would hope that that'll be the case. But I think that at each point, I don't know. It's like when you're building software, there's a ton of stuff that you can do with software, but then at some level you're constrained by the chips that it's running on. So there are always going to be different physical constraints. And it's like how big the model is going to be constrained by how much energy you can get and use for inference. So I guess I'm simultaneously very optimistic that the stuff will continue to improve quickly, and also a little more measured than I think some people are about the runaway case. I just don't think the runaway case is a particularly likely one. I think it makes sense to keep your options open. There's so much we don't know. There's a case in which it's really important to keep the balance of power so nobody becomes like a totalitarian dictator. There's a case in which you don't want to open source...

人类历史上哪个时期你最想回到过去看看，从 10 万年前到现在？你只是想知道当时是什么样子。必须是过去，对吧？

What time period in human history would you be most interested in going into a 100,000 BCE to now? You just want to see what it was like. Has to be the past, huh?

必须是过去。我不知道。我是说，我有一些感兴趣的历史时期。我对美国历史和古典历史非常感兴趣，也对科学史很感兴趣。所以我实际上觉得，亲眼看看并更深入地理解一些重大进步是如何发生的——我们只有一些有限的文字记载。我不确定元宇宙能否让你做到这一点，因为很难回到我们没有记录的时代。但我其实不确定回到过去对他们来说会那么重要。我是说，我觉得这对历史课之类的会很酷，但那可能不是我对元宇宙最兴奋的用例。总的来说，我认为主要的能力是无论你在哪里都能感受到与人的真实连接。我觉得那会是——在我们讨论 AI 的对话中，很多都涉及物理约束，这些约束是这一切的基础，对吧？而你想要——我认为技术的一个教训是，尽可能把东西从物理约束领域转移到软件领域，因为软件更容易构建和演进，而且你可以让它更民主化。不是每个人都会拥有数据中心，但很多人可以写代码、拿开源代码并修改它。元宇宙的版本是，我认为实现逼真的数字存在将带来巨大的不同，让人们不再觉得必须为了很多事情而物理地聚在一起。当然，我认为有些事情物理上在一起会更好，所以这不是二元的——不是说，好了，你不再需要那样做了。但总的来说，我认为这对社交、与人建立联系、工作、工业的某些部分、医疗以及很多方面都会非常强大。

Has to be the past. I don't know. I mean, I have periods of time that I'm interested in. I'm really interested in American history and classical history, and I'm really interested in the history of science too. So I actually think seeing and trying to understand more about how some of the big advances came about—I mean, all we have are somewhat limited writings about some of that stuff. I'm not sure the metaverse is going to let you do that because, I mean, it's hard to kind of go back in time for things that we don't have records of. But I'm actually not sure that going back in time is going to be that important of a thing for them. I mean, I think it's going to be cool for history classes and stuff, but that's probably not the use case that I'm most excited about for the metaverse overall. I mean, the main thing is just the ability to feel present with people no matter where you are. I think that's going to be—I mean, in the AI conversation that we're having, so much of it is about physical constraints that underlie all of this, right? And you want to move—I think one lesson of technology is you want to move things from the physical constraint realm into software as much as possible, because software is so much easier to build and evolve, and you can democratize it more. Not everyone is going to have a data center, but a lot of people can write code and take open source code and modify it. The metaverse version of this is, I think, enabling realistic digital presence is going to be an absolutely huge difference for making it so that people don't feel like they have to physically be together for as many things. Now, I think there are going to be things that are better about being physically together, so it's not binary—it's not like, okay, now you don't need to do that anymore. But overall, I think it's just going to be really powerful for socializing, for feeling connected with people, for working, for parts of industry, for medicine, for so many things.

我想回到你一开始说的话，你没有以 10 亿美元卖掉公司，还有元宇宙，你明知道市场在猛烈抨击你，却仍然坚持要做。我很好奇：这种优势的来源是什么？你说“哦，价值观，我有这种直觉”，但每个人都这么说，对吧？如果你必须说一些对你来说很具体的东西，你会怎么表达？你为什么对元宇宙如此确信？

I want to go back to something you said at the beginning of the conversation, where you didn't sell the company for a billion dollars, and like the metaverse, you knew you were going to do this even though the market was hammering you for it. I'm actually curious: what is the source of that edge? You said, 'Oh, values, I have this intuition,' but everybody says that, right? What if you had to say something that's specific to you? How would you express what that is? Why were you so convinced about the metaverse?

嗯，我觉得这是不同的问题。是什么在驱动我？我想我们已经讨论过一些主题了。我是说，我真的很喜欢构建东西。我特别喜欢围绕人们如何沟通、理解人们如何表达自己以及如何工作来构建东西。对吧？我大学时学的是计算机科学和心理学。我觉得行业里很多人只学了计算机科学，对吧？所以对我来说，这始终是这两个领域的交叉点。但我觉得这也是一种非常深层的驱动力。我不知道怎么解释，但我就是觉得，如果我不在创造新东西，从本质上来说我就做错了什么。所以，即使我们在为投资千亿级别的 AI 或巨额资金到元宇宙做商业论证时，是的，我们有计划，我认为这些计划很清楚：如果我们的东西奏效，那会是笔好投资。但你不可能从一开始就确定。所以人们会有各种争论，无论是顾问还是其他人：‘你怎么能如此自信？’而我的回答是，如果哪天我不再尝试创造新东西，我就会去别的地方创造新东西。我根本没法经营一个东西或过自己的生活而不去尝试创造我觉得有趣的新东西。这对我来说根本不是问题。就像我们是否要去尝试打造下一个东西——我根本做不到不去做。而且，我在生活的各个方面都是这样。我们在考艾岛建了一个牧场，我就喜欢设计所有这些建筑。我们开始养牛，我就想，好吧，我要养出世界上最好的牛。所以，我们如何规划才能实现这个目标，并建立所有需要的东西来尝试做到这一点？我不知道，这就是我。

Well, I think those are different questions. What are the things that kind of power me? I think we've talked about a bunch of the themes. So, I mean, I just really like building things. I specifically like building things around how people communicate and sort of understanding how people express themselves and how people work. Right? When I was in college, I studied computer science and psychology. I think a lot of other people in the industry studied computer science, right? So it's always been sort of the intersection of those two things for me. But I think it's also sort of this really deep drive. I don't know how to explain it, but I just feel like constitutionally I'm doing something wrong if I'm not building something new. And so I think that even when we're putting together the business case for investing like a hundred billion in AI or some huge amount in the metaverse, it's like, yeah, we have plans that I think make it pretty clear that if our stuff works, it'll be a good investment. But you can't know for certain from the outset. And so there are all these arguments that people have, whether it's with advisers or different folks: 'How can you be confident enough to do this?' And it's like, well, the day I stop trying to build new things, I'm just going to go build new things somewhere else. It's like I am fundamentally incapable of running something or in my own life and not trying to build new things that I think are interesting. That's not even a question for me. It's like whether we're going to go take a swing at building the next thing—I'm just incapable of not doing that. And I don't know, I'm kind of like this in all the different aspects of my life. We built this ranch in Kauai, and I just like working on designing all these buildings. I'm kind of trying to—we started raising cattle, and I'm like, all right, I want to make the best cattle in the world. So it's like, how do we architect this so we can figure this out and build all the stuff up that we need to try to do that? So I don't know, that's me.

你看，Meta 真是一家非常了不起的科技公司，对吧？他们拥有这么多优秀的软件工程师，但就连他们也和 Stripe 合作处理支付。我觉得这是一个非常值得注意的事实：Stripe 在打造结账体验方面的工程能力如此出色，以至于像福特、Zoom、Meta，甚至 OpenAI 这样的大公司都和他们合作处理支付。因为想想看，你需要处理多少种不同的可能性：如果你在不同的国家，你会用不同的方式支付；如果你购买某种特定商品，那可能会影响你决定如何支付。而 Stripe 能够每天在数百亿笔交易中测试这些精细的优化，找出什么能促成转化。显然，转化意味着你更多的收入。而且，我不是 Meta 那样的大公司，但我在 Stripe 成为广告商之前很久就在使用它了。Stripe Atlas 是我成立有限责任公司最简单的方式，他们的支付和发票功能让我从广告商那里收款变得超级方便。显然，没有它，我通过播客赚钱会困难得多。所以它对我来说一直很棒。

Look, Meta is just a really amazing tech company, right? They have all these great software engineers, and even they work with Stripe to handle payments. I think that's just a really notable fact that Stripe's ability to engineer these checkout experiences is so good that big companies like Ford, Zoom, Meta, even OpenAI, they work with Stripe to handle payments. Because just think about how many different possibilities you have to handle: if you're in a different country, you'll pay a different way; if you're buying a certain kind of item, that might affect how you decide to pay. And Stripe is able to test these fine-grained optimizations across tens of billions of transactions a day to figure out what will convert people. And obviously conversion means more revenue for you. And look, I'm not a big company like Meta or anything, but I've been using Stripe since long before they were advertisers. Stripe Atlas was just the easiest way for me to set up an LLC, and they have these payments and invoicing features that make it super convenient for me to get money from advertisers. And obviously without that, it would have been much harder for me to earn money from the podcast. And so it's been great for me.

我不确定，但我其实对另一件事很好奇。19 岁的马克读了很多古代历史和经典著作，在高中和大学时期。你从中学到了什么重要的教训？不只是你发现的有趣的东西，而是到 19 岁时你消费的 token 并不多，其中很多都是关于经典的。显然那在某种程度上很重要。

I'm not sure but that I'm actually curious about something else. So 19-year-old Mark reads a bunch of Antiquity and Classics in high school and college. What important lesson did you learn from it? Not just interesting things you found, but there aren't that many tokens who consume by the time you're 19. A bunch of them were about the classics. Clearly that was important in some way.

我不知道，这是个好问题。我觉得非常有趣的一点是，当奥古斯都刚成为皇帝时，他试图建立和平。当时人们根本没有和平的概念。人们对和平的理解是敌人必然会再次攻击你之前的短暂间歇，所以你得到一点休息。他有一个观点：我们想把经济从唯利是图和军事化转变为积极的东西。这在当时是一个非常新颖的想法。我认为这其中有非常根本的东西，关乎当时人们能想象到的合理工作方式的边界。回到这个话题，它既适用于元宇宙也适用于 AI。很多投资者和不同的人无法理解我们为什么要开源。他们认为开源一定是你把东西做成专有之前的暂时状态。但我实际上认为这是科技中非常深刻的东西，它创造了很多赢家。我不想过度类比，但我确实认为很多时候，构建事物的模式让人们无法理解那怎么会是有价值的，或者怎么会是世界的合理状态。我认为比人们想象的更合理的事情要多得多。

I don't know, that's a good question. I mean, one of the things I thought was really fascinating is when Augustus first became emperor, he was trying to establish peace. There was no real conception of peace at the time. People's understanding of peace was the temporary time between when your enemies will inevitably attack you again, so you get a short rest. He had this view: we want to change the economy from being so mercenary and militaristic to something positive. It was a very novel idea at the time. I think there's something really fundamental about that, in terms of the bounds on what people can conceive at the time of what are rational ways to work. Going back to this, it applies to both the metaverse and the AI stuff. A lot of investors and just different people can't wrap their head around why we would open source this. They think open source must be the temporary time between which you're making things proprietary. But I actually think it's a very profound thing in tech that creates a lot of winners. I don't want to strain the analogy too much, but I do think there are many times where models for building things are such that people can't wrap their head around how that would be valuable or a reasonable state of the world. I think there are more reasonable things than people think.

这太迷人了。我能给你我的答案吗？我猜你可能从中得到的。这可能完全不对，但就是这些在帝国中担任非常重要角色的人有多年轻。凯撒·奥古斯都在 19 岁时就已经是罗马政治中最杰出的人物之一，领导战斗并组建了后三头同盟。我想知道 19 岁的你会不会想：'我真的能做到。'我认为这是一个有趣的例子，在很多历史和美國历史上都是如此。我最喜欢的一句名言是：所有孩子都是艺术家，挑战在于长大后如何保持艺术家的状态。因为当你年轻时，更容易有疯狂的想法，而且你没有承诺。在你的生活和公司中，都存在这些类似于创新者困境的类比。在你轨迹的早期，更容易转向并接受新想法，而不会受到其他承诺的干扰。我认为经营公司的一个有趣部分就是：如何保持活力？

That's super fascinating. Can I give you my answer? What I was thinking you might have gotten from it. This is probably totally off, but just how young some of these people are who have very important roles in the Empire. Caesar Augustus is by the time he's 19 actually incredibly one of the most prominent people in Roman politics, leading battles and forming the second triumvirate. I wonder if the 19-year-old is like, 'I can actually do this.' I think that's an interesting example both from a lot of history and American history too. One of my favorite quotes is that all children are artists, and the challenge is how do you remain an artist when you grow up. Because when you're younger, it's easier to have wild ideas, and you have no commitments. There are all these analogies to the innovator's dilemma that exist in your life as well as your company. Earlier on your trajectory, it's easier to pivot and take in new ideas without disruption from other commitments. I think that's an interesting part of running a company: how do you stay dynamic?

回到投资者和开源的话题。假设那个价值 100 亿美元的模型完全安全，你已经做了这些评估，而且不像现在这种情况，评估者也可以微调模型，希望未来模型也能如此。你会开源那个 100 亿美元的模型吗？

Going back to the investors and open source. The $10 billion model, suppose it's totally safe, you've done these evaluations, and unlike in this case the evaluators can also fine-tune the model, which hopefully will be the case in future models. Would you open source that $10 billion model?

嗯，只要它对我们有帮助，那就开源。

Well, as long as it's helping us, then yeah.

但它会有帮助吗？100 亿美元的研发投入，现在却对任何人开源。

But would it? $10 billion of R&D, and now it's open source for anyone.

嗯，我认为这是一个我们需要随着时间的推移来评估的问题。我们有很长的开源软件历史，对吧？我们通常不会开源我们的产品。我们不会把 Instagram 的代码开源，但我们会把很多底层基础设施开源。我们历史上最大的项目可能是开放计算项目，我们把所有服务器、网络交换机和数据中心的设计都开源了。这最终非常有帮助，因为很多人可以设计服务器，但现在行业标准化了我们的设计，这意味着供应链围绕我们的设计建立起来，产量上升，对所有人都更便宜，并为我们节省了数十亿美元。所以开源有多种方式可能对我们有帮助。一是如果人们找到更便宜地运行模型的方法。随着时间的推移，我们将在所有这些事情上花费数百亿甚至上千亿美元。如果我们能提高 10% 的效率，就能节省数十亿甚至上百亿美元。这本身可能就很有价值，尤其是如果还有其他竞争模型存在的话。我们的东西并不是在放弃某种疯狂的优势。

Well, I think here's a question we'll have to evaluate as time goes on too. We have a long history of open sourcing software, right? We don't tend to open source our product. It's not like we take the code for Instagram and make it open source, but we take a lot of the low-level infrastructure and make that open source. The probably biggest one in our history was the Open Compute Project, where we took the designs for all of our servers, network switches, and data centers and made it open source. That ended up being super helpful because a lot of people can design servers, but now the industry standardized on our design, which meant the supply chains got built out around our design, volumes went up, it got cheaper for everyone, and saved us billions of dollars. So there are multiple ways open source could be helpful for us. One is if people figure out how to run the models more cheaply. We're going to be spending tens or a hundred billion dollars or more over time on all this stuff. If we can do that 10% more effectively, we're saving billions or tens of billions of dollars. That's probably worth a lot by itself, especially if there are other competitive models out there. It's not like our thing is giving away some kind of crazy advantage.

所以你的观点是交易会被商品化？

So is your view that the trading will be commodified?

我认为有多种可能的发展方式。这是其中之一。另一种是商品化意味着它会变得非常便宜，因为有很多选择。另一个可能的方向是质的改进。你提到了微调。目前，对主流模型进行微调能做的事情非常有限。有一些选项，但通常不适用于最大的模型。所以我认为能够做到这一点，并且能够做不同应用特定或用例特定的事情，或者将它们构建到特定的工具链中，不仅会实现更高效的开发，还可能实现质的不同。这里有一个类比：我认为移动生态系统普遍糟糕的一点是，你有两个守门人公司，苹果和谷歌，它们可以告诉你允许构建什么。在我们的历史中有很多次。有经济版本：我们构建了一些东西，它们就拿走你一大笔钱。但还有质的版本，这更让我恼火：有很多次我们推出或想要推出功能。

I think there are a bunch of ways this could play out. That's one. The other is that commodity implies it's going to get very cheap because there are lots of options. The other direction this could go is qualitative improvements. You mentioned fine-tuning. Right now it's pretty limited what you can do with fine-tuning major other models out there. Some options, but generally not for the biggest models. So I think being able to do that and be able to do different app-specific things or use case-specific things or build them into specific tool chains will not only enable more efficient development, it could enable qualitatively different things. Here's one analogy: one thing that I think generally sucks about the mobile ecosystem is that you have these two gatekeeper companies, Apple and Google, that can tell you what you're allowed to build. There are lots of times in our history. There's the economic version: we build something, they just take a bunch of your money. But there's the qualitative version, which actually upsets me more: there are a bunch of times when we've launched or wanted to launch features.

然后苹果就说，‘不，你不能发布那个。’我觉得这很糟糕，对吧？所以问题是：我们是不是正在走向一个类似的 AI 世界——少数公司运行封闭模型，控制 API，从而能告诉你你能构建什么？

And then Apple's just like, 'Nope, you're not launching that.' I like that sucks, right? And so the question is: are we kind of set up for a world like that with AI, where you're going to get a handful of companies that run these closed models, that are going to be in control of the APIs, and therefore are going to be able to tell you what you can build?

首先，我可以为我们说，自己构建模型是值得的，以确保我们不会陷入那种境地，对吧？我不希望任何其他公司告诉我们能构建什么。但从开源的角度来看，我认为很多开发者也不希望那些公司告诉他们能构建什么。所以问题是：围绕这个会构建出什么样的生态系统？有哪些有趣的新事物？这能在多大程度上改进我们的产品？我认为在很多情况下，如果这最终像我们的数据库、缓存系统或架构一样，我们会从社区获得有价值的贡献，让我们的东西变得更好。而我们做的特定应用工作仍然会非常有差异化，所以这并不重要。就像我们能做我们的事，我们会受益，所有系统——我们的和社区的——都会因为开源而变得更好。还有一种可能，也许不是这样。我的意思是，也许模型本身最终就成了产品本身。在这种情况下，我认为关于是否开源的经济计算会更棘手，因为那样你就在很大程度上把自己商品化了。但就我目前所见，我们似乎还没有进入那个区域。

Well, for one, I can say for us, it is worth it to go build a model ourselves to make sure that we're not in that position, right? Like I don't want any of those other companies telling us what we can build. But from an open source perspective, I think a lot of developers don't want those companies telling them what they can build either. So then the question is: what is the ecosystem that gets built out around that? What are interesting new things? How much does that improve our products? I think there's a lot of cases where, if this ends up being like our databases or caching systems or architecture, we'll get valuable contributions from the community that will make our stuff better. And then our app-specific work that we do will still be so differentiated that it won't really matter. It's like we'll be able to do what we do, we'll benefit, and all the systems—ours and the community's—will be better because it's open source. There is one world where maybe it's not that. I mean, maybe the model just ends up being more of the product itself. In that case, then I think it's a trickier economic calculation about whether you open source that, because then you are kind of commoditizing yourself a lot. But from what I can see so far, it doesn't seem like we're in that zone.

你期望通过向云提供商授权你的模型来获得可观的收入吗——也就是他们需要付费才能实际提供该模型？

Do you expect to earn significant revenue from licensing your model to the cloud providers, so they have to pay you a fee to actually serve the model?

我们希望有这样的安排，但我不知道它会有多重要。我们有这个——这基本上就是我们的 Llama 许可证。在很多方面，它就像一个非常宽松的开源许可证，只是我们对最大的公司使用它设定了限制。我们设置这个限制的原因不是要阻止他们使用它。我们只是希望他们来和我们谈谈，因为如果他们基本上拿我们构建的东西转售并从中赚钱，那么，好吧，如果你是微软 Azure 或亚马逊，那么如果你要转售这个模型，我们应该从中获得一些收入分成。所以在你这么做之前，先来和我们谈谈。而实际情况就是这样。对于 Llama 2，我们基本上与所有主要的云公司都有协议，Llama 2 作为托管服务在所有那些云上可用。我假设随着我们发布越来越大的模型，这会变得更加重要。这不是我们正在做的主要事情，但我只是认为，如果那些公司要销售我们的模型，我们以某种方式分享其中的收益是合理的。

We want to have an arrangement like that, but I don't know how significant it'll be. And we have this—this is basically our license for Llama. In a lot of ways, it's like a very permissive open source license, except that we have a limit for the largest companies using it. And the reason we put that limit in is we're not trying to prevent them from using it. We just want them to come talk to us, because if they're going to basically take what we built and resell it and make money off of it, then it's like, okay, well, if you're like Microsoft Azure or Amazon, then yeah, if you're going to be reselling the model, then we should have some revenue share on that. So just come talk to us before you go do that. And that's how that's played out. So for Llama 2, I mean, we basically have deals with all these major cloud companies, and Llama 2 is available as a hosted service on all those clouds. And I assume that as we release bigger and bigger models, that'll become a bigger thing. It's not the main thing that we're doing, but I just think if those companies are going to be selling our models, it makes sense that we should share the upside of that somehow.

是的。关于其他开源风险，我认为你关于权力平衡的观点是合理且正当的，还有那些因为更好的对齐技术而可能消除的危害。我希望 Meta 能有某种框架，就像其他实验室那样，他们会说，‘如果我们看到这个具体的东西，那么开源就不行，’甚至可能部署也不行。就是把它写下来，这样公司就准备好了，人们对此有预期，等等。

Yeah. With regards to the other open source dangers, I think you have genuine legitimate points about the balance of power stuff, and potentially the harms you can get rid of because we have better alignment techniques or something. I wish there was some sort of framework that Meta had, like other labs have this, where they say, 'If we see this concrete thing, then that's a no-go on the open source,' or even potentially on deployment. Just like writing it down, so the company is ready for it, people have expectations around it, and so forth.

是的，不，我认为这是个公平的观点。在存在风险方面，目前我们更关注我们今天看到的那些风险，更多是内容风险。所以我们有底线：我们不希望模型基本上是在帮助人们实施暴力或欺诈，或以不同方式伤害他人。所以实际上，对于今天的模型——我猜下一代甚至下下一代——我认为虽然讨论存在风险在智力上更有趣，但我实际上认为需要更多精力去缓解的真正危害是那些有人利用模型、用今天的参数去伤害他人的事情。以及我们今天看到的更普通的危害，比如人们互相欺诈之类的事情。所以我不想轻视这一点。我认为我们有责任确保我们在这方面做好。

Yeah, no, I think that's a fair point. On the existential risk side, right now we focus more on the types of risks that we see today, which are more of these content risks. So we have lines on: we don't want the model to be basically doing things that are helping people commit violence or fraud, or just harming people in different ways. So in practice, for today's models—and I would guess the next generation and maybe even the generation after that—I think while it is somewhat more intellectually interesting to talk about the existential risks, I actually think the real harms that need more energy being mitigated are things that are going to have someone take a model and do something to hurt a person with today's parameters. And the types of more mundane harms that we see today, like people committing fraud against each other, things like that. So I just don't want to shortchange that. I think we have a responsibility to make sure we do a good job on that.

是的，Meta 是一家大公司，你可以两者都处理。

Yeah, Meta is a big company, you can handle both.

好的，关于开源，我其实很好奇，你是否认为 PyTorch、React、Open Compute 这些开源项目对世界的影响甚至比 Meta 的社交媒体方面还要大？因为我和使用这些服务的人聊过，我认为这是有可能的，因为互联网的很大一部分都运行在这些东西上。

Okay, so as far as the open source goes, I'm actually curious if you think the impact of the open source from PyTorch, React, Open Compute—these things—has been bigger for the world than even the social media aspects of Meta? Because I've talked to people who use these services, and I think it's plausible, because a big part of the internet runs on these things.

这是个有趣的问题。我的意思是，我认为世界上几乎一半的人使用我们的社交媒体服务，所以我认为很难超越那个。但不，我认为开源作为一种构建事物的新方式非常强大。是的，有可能。我的意思是，它可能是这样的事情之一——我不知道，就像贝尔实验室，对吧？他们研究晶体管是因为想实现长途通话，他们做到了，而实现长途通话最终让他们非常赚钱。如果你在之后 5 到 10 年问他们，他们发明的最有用的东西是什么，他们会说，‘好吧，我们实现了长途通话，现在所有这些人都用长途通话。’但如果你一百年后问，可能答案就不同了。所以我认为我们正在构建的很多东西——Reality Labs、一些 AI 的东西、一些开源的东西——都是如此。我认为具体的产品会演变，在某种程度上来了又去，但我认为对人类进步的贡献是持久的。这是我们都能够参与的很酷的一部分。

It's an interesting question. I mean, I think almost half the world uses our social media services, so I think it's hard to beat that. But no, I think open source is really powerful as a new way of building things. And yeah, it's possible. I mean, it may be one of these things where—I don't know, like Bell Labs, right? Where they were working on the transistor because they wanted to enable long-distance calling, and they did, and it ended up being really profitable for them that they were able to enable long-distance calling. And if you ask them 5 to 10 years out from that, what was the most useful thing that they invented, it's like, 'Okay, well we enable long-distance calling, and now all these people are long-distance calling.' But if you ask a hundred years later, maybe it's a different question. So I think that's true of a lot of the things that we're building right now—Reality Labs, some of the AI stuff, some of the open source stuff. I think it's like the specific products evolve and to some degree come and go, but I think the advances for humanity persist. And that's a cool part of what we all get to do.

Llama 模型什么时候会在你自己的定制芯片上训练？

By when will the Llama models be trained on your own custom silicon?

很快。不是 Llama 4。我们采取的方法是：首先，我们基本上构建了能够处理我们排名和推荐类推理任务的定制芯片——比如 Reels、信息流、广告。这消耗了大量 GPU，但当我们能够将其转移到自己的芯片上时，我们现在就能只将更昂贵的 Nvidia GPU 用于训练。所以在某个时候，我们希望能有自己的芯片，可以用于——可能先训练一些更简单的东西，最终训练这些非常大的模型。但与此同时，我会……

Soon. Not Llama 4. The approach that we took is: first, we basically built custom silicon that could handle inference for our ranking and recommendation type stuff—so Reels, news feed, ads. And that was consuming a lot of GPUs, but when we were able to move that to our own silicon, we now were able to use the more expensive Nvidia GPUs only for training. So at some point, we will hopefully have silicon ourselves that we can be using for—probably first training some of the simpler things, and eventually training these really large models. But in the meantime, I'd...

最后一个问题，完全出乎意料——如果你被任命为 Google Plus 的 CEO，你能让它成功吗？

Final question, this is totally out of left field, but if you were made CEO of Google Plus, could you have made it work?

哦，我不知道。这是一个非常困难的反事实。问题在于 Google Plus 并没有 CEO；它只是公司内部的一个部门。我觉得就像你之前问的什么是稀缺资源，但你问的是以美元计。实际上，我认为对于大多数这个规模的公司来说，至少是专注。当你是一家初创公司时，你可能更受资本限制，只专注于一个想法。我认为在某个阶段你会跨过一个门槛，你同时在做多件事，并在其中创造更多价值，但你能直接指导的事情反而更受限。总会有一些随机的好事在组织里发生，我甚至都不知道，那很棒。但总的来说，组织的能力很大程度上受限于 CEO 和管理团队能够监督和管理的范围。这一直是我们关注的重点：就像 Ben Horowitz 说的，把主要的事情作为主要的事情，并努力专注于你的关键优先事项。

Oh, I don't know. That's a very difficult counterfactual. The problem is there was no CEO of Google Plus; it was just a division within a company. I think it's like you asked before about what are the scarcest commodities, but you asked about it in terms of dollars. I actually think for most companies of this scale, at least, it's focus. When you're a startup, maybe you're more constrained on capital, you're just working on one idea. I think you cross some threshold at some point where you're building multiple things and creating more value across them, but you become more constrained on what you can direct. There are always cases where something random and awesome happens in the organization that I don't even know about, and that's great. But in general, the organization's capacity is largely limited by what the CEO and management team are able to oversee and manage. That's been a big focus for us: keep the main thing the main thing, as Ben Horowitz says, and try to stay focused on your key priorities.

好吧，那真正的最后一个问题：当 Gemini 发布时，办公室里有没有人欢呼？

Okay, then the real final question: when Gemini was launched, was there any chance that somebody in the office cheered?

没有，我觉得我们现在更冷静了。

No, I think we're calmer now.

酷，酷。太棒了。嗯，我不知道。这是个好问题。好了，太棒了。Mark，这期节目非常精彩，非常感谢。非常有趣。

Cool, cool. Awesome. Yeah, I don't know. It's a good question. All right, awesome. That was excellent, Mark. Thanks so much. That was a lot of fun.

是的，非常有趣。谢谢你邀请我。

Yeah, really fun. Thanks for having me.

当然。大家好，希望你们喜欢这期与 Mark 的节目。如你们所见，我现在开始做广告了。如果你有兴趣在播客上投放广告，请访问描述中的链接。否则，你知道最有帮助的事情就是把这个播客分享给你觉得会喜欢的人——你的朋友、群聊、Twitter，或者 Threads。希望你们喜欢，我们下期再见。

Absolutely. Hey everybody, I hope you enjoyed that episode with Mark. As you can see, I'm now doing ads. So if you're interested in advertising on the podcast, go to the link in the description. Otherwise, as you know, the most helpful thing you can do is just share the podcast with people who you think might enjoy it—your friends, group chats, Twitter, I guess Threads. Yeah, hope you enjoyed, and I'll see you on the next one.